Just get a PSP (not PSP GO) for that, there are many kinds of emulators for that handheld console. They are so cheap used ones and can easily be set up for homebrew and emulators and has easy way to replace battery if you want to carry a spare battery while on travel for eg.