Forgive my ignorance, but what is the normal amount of memory that the system has?
If the app doesn't specify a N3DS memory layout in the RSF/exheader, then 64MB, like the O3DS, but if it lets the system know it supports the New 3DS, then 124 (or 128, whichever). There's also a third mode on the New 3DS: 178 (https://3dbrew.org/wiki/Memory_layout#FCRAM_memory-regions_layout). If it's usable on retail New 3DSes as some claim, it'd be pretty useful.
And I assume this is why Smash and MH4U reboot the system on exit?
Correct. If I had to guess, the 3DS OS goes into a "low memory" mode when it launches Smash/MH on O3DS, in order to give them the RAM they need, which is why it does a hard reboot after you exit (however, if you run another app immediately from the limited home menu, like health & safety, my dummy app, or some other homebrew app, it won't boot you out of emuNAND, and I don't think anyone knows why even to this day). If we had access to the 178 MB mode on N3DS, it'd probably be a similar situation: when you exit the app it'd probably hard reboot, but you could probably stop it by running some other app instead of pressing Home -> X to close.









