The psp gets my vote. I've used psp2000 and psp3000. Both emulate up to snes great. The fact that native psx emulation exists is fantastic. It's even got a decent battery. Look for one with a busted umd drive, no one uses that anyway. You can find them for $40-60.
I agree that the psp is a great emulation device for the price, but I must disagree on the snes emulation. Most games are laggy without frameskip, and with frameskip it's not really smooth...
I would say best emulation devices are either a Vita with an old firmware (even if retroarch is kind of garbage...), or a Android device like the GPD XD (but with a not so good build quality).
There is also the GPD Win, which is great to play small steam games too. But for this one also have flaws like overheating, not so good button layout, size, and also Windows 10 that's not optimised for portable gaming.
Here is a summary of the devices I use(d) for emulation:
- 3DS & New3DS: the unofficial emulator aren't very good (very laggy both for video and input), especially compared to the psp, but the Virtual Console injection works really well for all Nintendo old platform except Nintendo64 (and Snes for the old 3DS). Also the quality of the screen isn't great if you have a TN model.
- PSP 2000 with CFW: as I said, this one is great for emulation, with a good screen and controls, but lacks a bit of power for Snes and Nintendo64 games (it runs, but not very smooth)
- PS Vita 1000 with henkaku: this one is great, good build quality, control, and screen. But having to deal with the messy and not optimised retroarch is a pain in the a**. I really hope they will make more native emulators soon instead of going the easy but messy way. Also the memory card are just insanely expensive.
- GPD XD: this one is actually the best I tried for emulation (with the last legacy rom), thanks to the android os it emulates almost everything until the dreamcast, has great control, not a bad screen (still a LOT better than the 3DS TN screen...), and you can also play a lot of android & retro games on it. The build quality isn't the best for sure, but clearly not as bad as most people say, especially for the low price. Also even if the support isn't existent, you can buy parts to repair it yourself.
Overall if you can deal with the not so good build quality the GPD XD is the way to go. Or the GPD Win if you want a full keyboard and Windows 10.
Or wait for the next big thing in the emulation world