Nice, you Should strip down VLC (Since there is a ARM based Source), you could use that. It has MPEG4 MPEG3 and Some other Mentions: 3GP, ASF, AVI, DVR-MS, FLV, Matroska (MKV), MIDI, QuickTime
File Format, MP4, Ogg, OGM, WAV, MPEG-2 (ES, PS, TS, PVA, MP3), AIFF, Raw audio, Raw DV, MXF, VOB, RM, DVD-Video, VCD, SVCD, CD Audio, DVB. ARM BASED VLC:
http://www.videolan.org/vlc/download-sources.html or you Could use the Source Libs.
Note: The only Arm Based builds are for Android, but you can still use them. There Armv6 and Armv7, as well as some BETA Versions of Arm8/9/10/11. The 3DS Use's: Dual-Core ARM11 MPCore, single-core ARM9.