Following what you mentioned above, I decided to run some tests myself and I'm a bit confused. I understand that H.264 is the standard and Xvid is old, that H.264 is more efficient with data, delivering better quality at a lower bitrate, and all that. Considering we are talking about the Wii's hardware and this homebrew in particular: on one hand, with H.264 you are completely limited to not exceeding 1000k (in fact, only 800k passes the test in scenes with particles or heavy action) and you have to factor in all sorts of parameters like fps and resolution for the video to play smoothly.
Meanwhile, Xvid, based on the tests I've run, seems to perform muuuuch better. Even with very high bitrates (over 5000k) at 480p, I didn't get any frame drops. The advantage of this codec is that, from what I've read, it is much less demanding to decode, whereas H.264 uses a more robust compression that inherently requires more processing power. I compared it using a movie clip—a storm scene with a violent sea and a monster. H.264 only ran smoothly when limited to 800k, while I could assign a much higher bitrate to Xvid without dropping a single frame. This gave Xvid a quality advantage; not by a huge margin, but noticeably better. That said, the Xvid file size was about double that of H.264
Xvid is obsolete, but so is the Wii's hardware, so it's not unreasonable to use it for this machine.
