Agreed, anything less than full speed is normally unacceptable because the music is one of the most important aspects of games for me. Dolphin is one exception though, because you can lower the emulated CPU clock speed to 40%, which will make the game skip frames naturally, instead of the emulator slowing it down in order to keep up, which keeps the audio perfect in many games when I run Dolphin on my Switch, but not all, as some games are still too heavy for the emulator even with the 40% CPU clock speed override. They might even be GPU bound which is why I don't see much improvement in the audio at all. And some games just have too low framerates to be considered playable, so even with perfect audio it's still no good. And depending on the game, game speed could be tied to the framerate so that it actually runs slower when the framerate is lower, but that's not a huge deal as long as the slowdowns aren't constant. But I digress.
Some people don't mind playing games without sound, I on the other hand can't stand it. So just turning the sound off isn't a good solution either. Maybe consider getting a good controller for your smartphone? Smartphones emulate most systems with ease, and you're always carrying it around anyway. GameSir makes some good ones that are designed for smartphones, where the phone is held in place with a clip so you can hold it like you would hold a 3DS, there's also iPega who has various models, including ones that more resemble the Switch with two halves where your phone fits inbetween them, or you could go with a DS3 or a DS4 with a clip addon to hold your phone, or really any wireless bluetooth controller that you can buy or 3D print a clip for. I know 8bitdo sells a clip addon for at least some of their controllers.