Having spent a long time over the years working on the wii homebrew nintendont, a program that allows you to play gamecube games in wii mode, I've encountered some weird bugs over time. This blog entry is about the latest one I fixed up, mario party 4 freezing.
To give a quick rundown of what exactly was freezing, after booting the game and going into the extra room (last option in the main menu) , selected the characters you want to play it is then supposed to come up with a game list, instead it just seemingly froze up on this screen:
the game did not fully crash though, as the background music continues to play like normal and controller reset combinations still work, so it also updated the controller just fine, but it did not render any more frames and got stuck on this one frame.
A year ago, I already tried to look at this once and tried simple things such as compiling nintendont with different versions of the compiler which did sometimes seem to actually make a difference in the issue but it seemed to do so for no simple reason. Also, I noticed when you reset the game at any point after it was booted, the issue would also be completely gone at which point there now at least was a simple workaround so I left the issue there, waiting to be looked at in detail at a later point.
Now, I finally returned to this issue with the simple idea of maybe there being a memory difference somewhere, something in RAM that may overwrite parts of the game perhaps, so I went into the internal game loader and changed the memory clearing from being most of the RAM around it:
https://github.com/FIX94/Nintendont...8ae6cefc277ccfb8abe/multidol/global.c#L28-L31
to now instead clear all of the RAM around it:
https://github.com/FIX94/Nintendont/blob/master/multidol/global.c#L29-L33
which sadly did not make a difference, but I still left it in my latest commit simply because it probably is a good idea anyways to make sure the memory is as clear as possible in case some game does have issues with it.
Next up, I did also test to just move the memory position of nintendont up a bit which also made no difference at all, which I was expecting as it should be cleared by the game loader anyways. At this point I wanted to make sure my game loader was not at fault so I replaced it with the game loader of the game itself that gets executed when you reset the game, since we know that reset method worked I wanted to know if it was related to that loader, but nope, even when using the games own loader, the issue was still there on first boot. Though one positive thing that came from this test was that I noticed a lot of memory sets in my own game loader were not needed:
https://github.com/FIX94/Nintendont...8ae6cefc277ccfb8abe/multidol/global.c#L33-L58
So in my latest commit I now replaced them with ones that are much smaller and work just fine as well:
https://github.com/FIX94/Nintendont/blob/master/multidol/global.c#L34-L40
At this point, I tried to fully clear out all of the main memory and just tested that by moving nintendonts memory position, but I still was not fully convinced it actually cleared out everything so I ended up also writing the entire main memory into a file right after the game got loaded, before it jumped to the game start point. After looking at a hex editor it indeed was all cleared out and only the game and game loader were present in memory, exactly as it should be.
Now I did remember that there was that oddity about switching compiler versions which I now again tried out and while I did not expect any difference, switching back to a compiler version from 2014 suddenly gave me this after I selected characters in the extra room:
So there was no freeze this time, but normally there is supposed to be text instead of those arrows where you select a game to play from, this is getting weird... So I again dumped the main memory, compared it against my earlier dump that just straight up froze and... no differences. This means that somehow, just the way that the game selector is compiled has an effect deep in mario party 4 code and it has nothing to do with the main memory.
While I still had no idea how, I proceeded to now change more things about the compiler, namely the code optimization flag:
https://github.com/FIX94/Nintendont/blob/master/loader/Makefile#L31
You see how this reads -O3? Well that is basically telling the compiler hey, make this code run as fast as possible. So just to see if anything happened, I changed this out for -O2, which is just slightly slower, let it all compile with the old 2014 compiler, tried out mario party 4 again and:
The behavior changed yet again! Now it displayed as it should. I did also just to be 1000% sure dump the main memory yet again and there still was no visible difference at all.
So since the regular RAM seems to be unrelated for sure now, I had this absolutely crazy idea, what if some processor register got set and the game never initializes that register anywhere and then uses it in the extra room with whatever value it had when the game first booted up? To test this, I replaced my normal jump to the game loader:
https://github.com/FIX94/Nintendont...c277ccfb8abe/loader/source/main.c#L1594-L1598
With some code that first clears out most processor registers and then jumps to the game loader:
https://github.com/FIX94/Nintendont/blob/master/loader/source/jmp813.S#L10-L18
And then updated to the latest compiler again, set it to -O3 as it was originally, compiled everything just fine, fired up mario party again and guess what, IT WORKS FINE NOW. So after all my tests with memory, moving code around, verifying memory, it seems to really be related to processor registers. Now of course, I wanted to know which ones, so in the code above I started removing some of the init lines until I had issues again, the line that again broke everything was this one:
https://github.com/FIX94/Nintendont/blob/master/loader/source/jmp813.S#L11
which has its definition right below:
https://github.com/FIX94/Nintendont/blob/master/loader/source/jmp813.S#L20-L50
This did seem pretty crazy to me because these registers are basically what the processor uses for nearly every single processor instruction for fast variables, so somehow through everything thats still abead, game loading, going through the main menu and so on, one register variable stays uninitialized.
So now it was just a matter of me removing line per line of that to find out exactly what caused issues. After a while I finally found the magic line:
https://github.com/FIX94/Nintendont/blob/master/loader/source/jmp813.S#L37
I did also print out what the value of r19 was on crash, and it was -20, so just to verify I'm not going crazy I went back to the old 2014 compiler and -O2 which worked perfectly, and just added a line that loaded -20 into r19 and indeed, the game now crashed.
Of course I did the same test to see what caused the arrows with -O3 and the 2014 compiler by again removing line per line of that init register function and those arrows are related to this:
https://github.com/FIX94/Nintendont/blob/master/loader/source/jmp813.S#L36
Again printing out r18 revealed that it was also -20 when the arrows happened, so I went back to the latest compiler version with -O3 set and added a line to load r18 with -20 and sure enough, now the arrows were back.
So now in my latest commit, I have all the init code in place, resetting everything, as well as the code to clear out all the memory when the game loads to hopefully not run into a crazy issue like this again.
In conclusion, this bug was related to a simple processor register never being initialized by the game and then when used very late into its execution, if that register contains a unexpected value, the game suddenly gets very weird due to this being undefined behavior.
This bug was really difficult to catch because it involved something you would never expect to be at fault but now with all this extra protection in place I hope an issue like this does not happen again.
Thats it for now, thanks for reading about this crazy bug hunt.
To give a quick rundown of what exactly was freezing, after booting the game and going into the extra room (last option in the main menu) , selected the characters you want to play it is then supposed to come up with a game list, instead it just seemingly froze up on this screen:
A year ago, I already tried to look at this once and tried simple things such as compiling nintendont with different versions of the compiler which did sometimes seem to actually make a difference in the issue but it seemed to do so for no simple reason. Also, I noticed when you reset the game at any point after it was booted, the issue would also be completely gone at which point there now at least was a simple workaround so I left the issue there, waiting to be looked at in detail at a later point.
Now, I finally returned to this issue with the simple idea of maybe there being a memory difference somewhere, something in RAM that may overwrite parts of the game perhaps, so I went into the internal game loader and changed the memory clearing from being most of the RAM around it:
https://github.com/FIX94/Nintendont...8ae6cefc277ccfb8abe/multidol/global.c#L28-L31
to now instead clear all of the RAM around it:
https://github.com/FIX94/Nintendont/blob/master/multidol/global.c#L29-L33
which sadly did not make a difference, but I still left it in my latest commit simply because it probably is a good idea anyways to make sure the memory is as clear as possible in case some game does have issues with it.
Next up, I did also test to just move the memory position of nintendont up a bit which also made no difference at all, which I was expecting as it should be cleared by the game loader anyways. At this point I wanted to make sure my game loader was not at fault so I replaced it with the game loader of the game itself that gets executed when you reset the game, since we know that reset method worked I wanted to know if it was related to that loader, but nope, even when using the games own loader, the issue was still there on first boot. Though one positive thing that came from this test was that I noticed a lot of memory sets in my own game loader were not needed:
https://github.com/FIX94/Nintendont...8ae6cefc277ccfb8abe/multidol/global.c#L33-L58
So in my latest commit I now replaced them with ones that are much smaller and work just fine as well:
https://github.com/FIX94/Nintendont/blob/master/multidol/global.c#L34-L40
At this point, I tried to fully clear out all of the main memory and just tested that by moving nintendonts memory position, but I still was not fully convinced it actually cleared out everything so I ended up also writing the entire main memory into a file right after the game got loaded, before it jumped to the game start point. After looking at a hex editor it indeed was all cleared out and only the game and game loader were present in memory, exactly as it should be.
Now I did remember that there was that oddity about switching compiler versions which I now again tried out and while I did not expect any difference, switching back to a compiler version from 2014 suddenly gave me this after I selected characters in the extra room:
While I still had no idea how, I proceeded to now change more things about the compiler, namely the code optimization flag:
https://github.com/FIX94/Nintendont/blob/master/loader/Makefile#L31
You see how this reads -O3? Well that is basically telling the compiler hey, make this code run as fast as possible. So just to see if anything happened, I changed this out for -O2, which is just slightly slower, let it all compile with the old 2014 compiler, tried out mario party 4 again and:
So since the regular RAM seems to be unrelated for sure now, I had this absolutely crazy idea, what if some processor register got set and the game never initializes that register anywhere and then uses it in the extra room with whatever value it had when the game first booted up? To test this, I replaced my normal jump to the game loader:
https://github.com/FIX94/Nintendont...c277ccfb8abe/loader/source/main.c#L1594-L1598
With some code that first clears out most processor registers and then jumps to the game loader:
https://github.com/FIX94/Nintendont/blob/master/loader/source/jmp813.S#L10-L18
And then updated to the latest compiler again, set it to -O3 as it was originally, compiled everything just fine, fired up mario party again and guess what, IT WORKS FINE NOW. So after all my tests with memory, moving code around, verifying memory, it seems to really be related to processor registers. Now of course, I wanted to know which ones, so in the code above I started removing some of the init lines until I had issues again, the line that again broke everything was this one:
https://github.com/FIX94/Nintendont/blob/master/loader/source/jmp813.S#L11
which has its definition right below:
https://github.com/FIX94/Nintendont/blob/master/loader/source/jmp813.S#L20-L50
This did seem pretty crazy to me because these registers are basically what the processor uses for nearly every single processor instruction for fast variables, so somehow through everything thats still abead, game loading, going through the main menu and so on, one register variable stays uninitialized.
So now it was just a matter of me removing line per line of that to find out exactly what caused issues. After a while I finally found the magic line:
https://github.com/FIX94/Nintendont/blob/master/loader/source/jmp813.S#L37
I did also print out what the value of r19 was on crash, and it was -20, so just to verify I'm not going crazy I went back to the old 2014 compiler and -O2 which worked perfectly, and just added a line that loaded -20 into r19 and indeed, the game now crashed.
Of course I did the same test to see what caused the arrows with -O3 and the 2014 compiler by again removing line per line of that init register function and those arrows are related to this:
https://github.com/FIX94/Nintendont/blob/master/loader/source/jmp813.S#L36
Again printing out r18 revealed that it was also -20 when the arrows happened, so I went back to the latest compiler version with -O3 set and added a line to load r18 with -20 and sure enough, now the arrows were back.
So now in my latest commit, I have all the init code in place, resetting everything, as well as the code to clear out all the memory when the game loads to hopefully not run into a crazy issue like this again.
In conclusion, this bug was related to a simple processor register never being initialized by the game and then when used very late into its execution, if that register contains a unexpected value, the game suddenly gets very weird due to this being undefined behavior.
This bug was really difficult to catch because it involved something you would never expect to be at fault but now with all this extra protection in place I hope an issue like this does not happen again.
Thats it for now, thanks for reading about this crazy bug hunt.
