There is definitely MP4 for PC-->3DS, but I don't know of any the other way around.
There is fastvideo3DS encoder, which can get up to the 100FPS range on a new3DS, but it's been unreleased for like 3-4yrs now.
Also, there is zero hardware-acceleration for *encoding* any sort of image or video. Decoding however is accelerated even on old3DS (YUV --> RGB), and new3DS has dedicated MPEG frame decoder as well. But sadly none of these can be reversed.
Also also, new3DS can do ~1MB/s, which results in ~10FPS uncompressed for the top screen if the pixels were 8bpp instead of 16/24/32bpp. Definitely still need compression.
It's also not much the fault of the WiFi chip. Both the SDIO driver is bad, and the syscall latency is also huge, so the speed deficiency just adds up. Not to mention the slow FCRAM...