It certainly is. There are a few major RAM issues - the amount of RAM that Retroarch actually accesses, the amount that Ninjhax grants access to (64MB I believe) and the generally lacking amount of RAM available on the 3DS itself (192MB available to applications on the New 3DS, 128MB on the O3DS).
Some kind of proper cashing system should be able to overcome the issues, but the arbitrary 64MB limit granted by Ninjhax is a problem. That's why I suggested earlier that .cia users had the potential to run larger roms if Retroarch was coded for this.