One of the reasons pcsx-rearmed stutters on 3DS is because reading from the SD card is incredibly slow. That's why chd / pbp run a lot faster than bin/cue. But even chd can be laggy when reading a bunch of stuff.
I borrowed an idea from Beetle PSX and added an option to do CD reads in the background. It also keeps around a limited amount of old data, so if the game keeps reading the same data, it'll come from memory instead of disk. So far this has helped in the games I've tried, especially RPGs, after playing for a while.
It makes Diablo playable with frameskip 1, at least the limited amount I've tried. And it'll make .bin/cue almost as fast as chd/pbp, in case you hate saving disk space.
I attached an unofficial core build with the patch in it, the source is here if you'd rather build it yourself:
https://github.com/justinweiss/pcsx_rearmed/tree/async-cd-reads
When you use the core, make sure you go to Quick Settings, set the "CD Access Method" to "Asynchronous", and restart RetroArch. Otherwise the new setting won't take effect.
Games do weird enough stuff that I'm sure this breaks some of them, so give this a try and let me know what you run into. If this is stable enough, I'll try to get the patch into the official builds.
This also includes the dynarec crash fix from yesterday, so also let me know if that keeps happening.