I know a regular 3DS can't handle a Dreamcast emulator, not 100% sure about New 3DS. But my logic is if it were possible better minds than you probably would have already done it. Good luck though.
My first order of business as mayor, make EOF the most important part of GBATemp.
My second order of business as mayor, close down EOF forever.
Can I count on your votes?