Don't know about MPlayer CE but WiiMC can do this in three ways.
-Select the option in the video settings to automatically play the next video.
-Press the 'plus' button on the folder you want and it will create a temp playlist in memory, you can then set playback to continuous.
-Write a text file with all the file locations. Save the file with a supported playlist extension, my preferred extension is .m3u, then select it in WiiMC as you would a video.
Playlist example:
sd:/videos/animu/Steins Gate - 04.mkv
usb1:/docs/videos/TEST.avi
usb1:/docs/music/bloop.m4a
Then I can save it as "testlist.m3u" and profit.
I can't remember if it's usb1 or just usb though. Also I think the second option only applies to music files.
Your best bet is the first option but if you want to customize it or control it then use the third option.