Homebrew [RELEASE] Wii-DOOM / Wii-HERETIC / Wii-HEXEN / Wii-STRIFE (ALL-IN-ONE) *PROPER*

[CaptainHIT]

Quick News:

MetroTRK Debugger + NDEV running under GCC.
I just made a test of a freshly compiled libOGC with some debugger additions and a simple main that causes a crash. Debugger sets in at the crash line, which is great. Stepping through the code then seems to work. Register addresses are accessible. Memory can be read. Setting breakpoints works. I also had to add some lines to the linker file to get it to work.

Next step: compiling WiiDOOM and test.
Expect to hear something new very soon.

Wow! Great news!

 

[nitr8]

Still problems over time after staying away from the code i once maintained.

So i've come across getting the latest devkitpro tools to compile my old stuff after fixing compiler errors. Now i have to face these situations:

Having the code compiled, it won't run on the NDEV as the apploader on the NDEV is complaining about "a section" exceeding it's boundaries. I found out what section that is: it's the ".bss"-section. There's lot's of space inside it. One big part is that one which handels the blood sprites. This variable can have a size of "0x10000". This is almost a lowered value because it SHOULD be 0x20000 for normal reasons. I already realized that 0x20000 would be a problem for the Wii in the past, when i was porting this stuff.

Anyway... If we narrow that down to 0x7500, the NDEV executes the binary. It is still throwing a warning but doesn't error out anymore.

So the next thing is that the game won't even get to the menu. It already crashes at initializing the graphics. The debugger doesn't step in which is strange. Though, i realized that the debugger will only start if we use the latest devkitpro tools (8.3.0). If i use 4.8.2 from the time when i did release that last binary here and try to run in NDEV TRK debug mode, then CodeWarrior crashes immediately upon loading the debug info. Running in NON-DEBUG mode, the game starts to the menu but with a MASSIVE performance drop that i haven't been able to figure out yet. This is the first thing i will work on NOW - the reason for that performance thing.

After releasing the last binary here (a few years back), i have been working on Wii-DOOM for a short while but i didn't upload anything new so i think i did something "wrong" to the code.

One more thing: devkitpro is no longer the same as it was at the time when i actively worked on these ports. Wintermute changed variable types which in the end also broke these ports. You are able to compile them but they most likely won't run. He ignored all the warnings from other developers. He doesn't also respond to any support requests. That's not kind of how to deal with people "using" your stuff.

Stay tuned while i'm trying to get this to work as expected...

 

[XDel]

Deja-Vu man.
There is a guy who was kind enough to port Odamex over to AROS for me so that I could bring my Alien Breed project to all modern Amiga/WorkBench variants as possible, but as fate would have it, the latest updates in AROS have rendered the old code somewhat broken, and all further attempts to re-compile have failed to resolve the issues. Also the old primary coder behind much of AROS has left or went on vacation or what have you, and the new guys are just kind of doing there own thing, so I believe all is lost for me on that front.

Now here we are with your project, and it, or rather you are going through your trials and tribulations on various levels. I do hope you get things sorted out as I had been very very hopeful that you would some day come back and expand upon where you had left off, and here you are; against all odds!

 

[nitr8]

I'll try to pull the code which i was using when i released r33 here. This one was working flawlessly. If that code compiles with devkitpro r35 (which is GCC v8.3.0) then i think there is something else very odd.

I have to try some different approaches. I'm currently working on that - using every free minute i have available at the time.

So Wii-DOOM r33 was somewhat back in 2015.
As i said: i've made some more changes and additions since r33. These changes did compile with devkitpro r27 (GCC v4.8.2) fine and they were working without any problems so far (on LINUX). Now with devkitpro r35 (GCC v8.3.0) it compiles but hangs at initialization. devkitpro r27 compiles but has a performance related problem (compiled under Windows using CodeWarrior which only links to the tools). Even using the exact same libraries from back the day won't help for the current state of the project. So we rather test old code and look at the changes... I hope that won't take too much time. I just wanna see these things running once again...

EDIT:

So it turns out: Freshly pulled source code of Wii-DOOM R33 from the day back at March, 29th 2015 compiles under devkitpro r27 with GCC v4.8.2 and the libraries of that time. Compiled under Windows using CodeWarrior - but with "some" errors - easy to fix anyway. It runs perfectly... no performance dropping. Runs on the NDEV. No warning or error regarding ".bss"-section.

Next step: compiling the exact same stuff using devkitpro r35 tools (GCC v8.3.0) and it's CRT binaries with freshly compiled libraries. Let's see what happens...

EDIT #2:

Compiled R33 under devkitpro r35 with GCC v8.3.0 and all libraries of that time compiled using the same tools. So everything is very fresh. Linked against MetroTRK. Runs on the NDEV. No performance related issues. Debugger SEEMS to be working (it actually IS currently executing while i'm running a game). Though, the game will hang - music playing right after 2 minutes. This is no debugger related thing as i then tried to compile libOGC against NON-MetroTRK-Integration. So... switch back to devkitpro r27 once again (incl. libraries) and try there...

Things i realized first compared between GCC v4.8.2 of devkitpro r27 and GCC v8.3.0 of devkitpro r35:

GCC v4.8.2 of devkitpro r27 throws ONLY 13 warnings of which 4 are of interest. The other ones are just empty text messages pointing to the previous warning.

GCC v8.3.0 of devkitpro r35 throws OVER !!!240!!! warnings. ---> SERIOUSLY?!

If this is a devkitpro related thing and caused by Wintermute's change to the variables then i will most likely NOT change my code because of his own change to the tools. If i would do that then it's only a matter of time until Wintermute changes one more thing and you have to change your own stuff then once again. That's not i wanna go through as i wouldn't only have to change MY code. I would also have to change the code of all the libraries required to compile these ports - which are like 18 libraries in a row btw..

EDIT #3:

So it turns out that the game doesn't hang when using the old stuff of the days when i compiled R33 of Wii-DOOM which means that it looks like we have to use devkitpro r27 with GCC v4.8.2. This one won't let the debugger run ATM. I don't know the reason for that so i think i have to compile 2 ELF's (one of devkitpro r27 and one of devkitpro r35) and compare them using IDA.

Anyway i forgot to say: THANKS WINTERMUTE!!

 

[Maeson]

I'm not versed with Wii homebrew or software making, but damn, I keep reading more and more people being annoyed and affected by WinterMute's choices with DevKitPro.

I understand wanting for people to use the most updated version possible, but I don't know, if you're going to make massive changes to the point of making previous projects made with your software not work anymore, maybe keeping old builds would be necessary.

 

[nitr8]

I have only 3 choices:

1.) Stay on devkitpro r27 - no NDEV Debugger (MetroTRK) support

2.) Update to devkitpro r35 - NDEV Debugger working but messed up ports

3.) Check every devkitpro release between r27 and r35 for compatibility with my ports (no hangs - which i currently have when using r35 and making the Debugger run upon start).

So i have 8 possible tests 'cause i have all other releases of devkitpro laying around here. I hope that at least 1 works as expected.

EDIT: r28 of devkitpro almost breaks everything. I came across some warnings regarding SDL for the Wii. When compiling the same library against r27 and then r28 it complains about redefining "int32_t" which is a "signed int" in r27 and a "signed long" in r28. He simply switched the check on expressions of "__INT_MAX__" and "__have_long32". The debugger runs on r28 and everything compiles but i get dozens of warnings and in the end when running, the fat library doesn't do it's job correctly. It recognizes the files and it seems that it checks for the MD5 but as soon as you hit "start" the game exits.

Maybe i have to go a long way on finding a solution to this...

EDIT (AFTER 2 DAYS):

Slow progress as said, i realized the following situation today upon getting the compiler to work under devkitpro r27:

Booting a very, veeerrryyy simple program that just prints text to the screen with libOGC now works upon the fact that we also have debug output and the debugger starts, BUT...:

1.) it doesn't read breakpoints (BAD)
2.) no memory access (BAD)
3.) no access to the registers (BAD)
4.) it DOES react on crash tests (GOOD)

These 4 points were tested using libOGC and all the other libraries from the time around the release of devkitpro r27.
After this i tried the exact same thing but with only slight differences because i had these suspicious thoughts about a "flag" possibly being set when the RVL-SDK boots to set something inside the memory that the debugger reads. This isn't the case for the RVL-SDK main function i tried to implement into my test code (OSInit).

Anyway i thought of "set a breakpoint right after initialization of the debugger but >>BEFORE<< it boots into OSInit".
And this is what happened:

It (the debugger) breaks correctly and does everything else correctly as well when using the standard RVL-SDK libraries. Though, i removed the __start routine from the RVL-SDK's extracted "OS" library for testing libOGC's startup assembly code (i "replaced" it). Both of them include the initialization code of the debugger.

So after this successful test, i deactivated the RVL-SDK libraries and just loaded libOGC into my project but kept that exact same __start assembly code including the initialization of the debugger.

The code starts and is being executed. THE BREAKPOINT IS STILL SET AND THE DEBUGGER STARTS BUT IT DOESN'T STEP IN!!

So this no longer is a problem on the code... It's a problem with GCC (the compiler itself or the linker).
My next thought was: If the same startup code works with stock libraries, why doesn't it work with libOGC....?
I will try compiling libOGC under the MetroWerks Compiler once again, like i did in the past as the stock libraries all have been compiled using CodeWarrior's tools.

Wish me luck...

 

[nitr8]

Big news for now:

I managed to compile everything using the RVL-SDK's MetroWerks C-Compiler (all libraries as well as the main game).

The interesting thing is: we don't need the RVL-SDK anymore (only for DEBUG stuff). The MetroWerks C-Compiler, it's binaries and everything else required for compiling, is also part of the FREELY AVAILABLE CodeWarrior for Power Architecture v8.8 release at NXP and this release ALSO INCLUDES THE REQUIRED RUNTIME SOURCE CODE FILES (LOL).

The 1 and ONLY thing we need from the RVL-SDK is for debugging purposes (EUART library, T. R. Kernel & NDEV Debugger / EXI module) but those files / binaries would NEVER be part of any official release i would provide in the future.

Anyway, here's what i came across for now:

Everything compiled. The game runs but hangs for unknown reasons (R33). Though, the exact same symptoms are there when compiling under the MetroWerks C-Compiler as well as devkitPro's GCC. So i think this is no longer a SDK (RVL-SDK / devkitPro) related thing but related to the game's source code.

The difference here: libOGC's assembly part of the GX / GU unit won't compile under MetroWerk's ASM-Compiler so i did use libOGC's C-sources for that.

Now it's time to get things sorted out. Maybe i have to take big steps backwards in order to figure out what's going on and fix things.

If i get this to work then we're ready for more stuff in the future.

Btw: It's a shame that the MetroWerks comiled ELF files WON'T run from within the HBC. They are simply NOT compatible.

 

[CaptainHIT]

Big news for now:

I managed to compile everything using the RVL-SDK's MetroWerks C-Compiler (all libraries as well as the main game).

The interesting thing is: we don't need the RVL-SDK anymore (only for DEBUG stuff). The MetroWerks C-Compiler, it's binaries and everything else required for compiling, is also part of the FREELY AVAILABLE CodeWarrior for Power Architecture v8.8 release at NXP and this release ALSO INCLUDES THE REQUIRED RUNTIME SOURCE CODE FILES (LOL).

The 1 and ONLY thing we need from the RVL-SDK is for debugging purposes (EUART library, T. R. Kernel & NDEV Debugger / EXI module) but those files / binaries would NEVER be part of any official release i would provide in the future.

Anyway, here's what i came across for now:

Everything compiled. The game runs but hangs for unknown reasons (R33). Though, the exact same symptoms are there when compiling under the MetroWerks C-Compiler as well as devkitPro's GCC. So i think this is no longer a SDK (RVL-SDK / devkitPro) related thing but related to the game's source code.

The difference here: libOGC's assembly part of the GX / GU unit won't compile under MetroWerk's ASM-Compiler so i did use libOGC's C-sources for that.

Now it's time to get things sorted out. Maybe i have to take big steps backwards in order to figure out what's going on and fix things.

If i get this to work then we're ready for more stuff in the future.

Btw: It's a shame that the MetroWerks comiled ELF files WON'T run from within the HBC. They are simply NOT compatible.

Man, that's awesome stuff you do out there! Keep up the great work!

 

[nitr8]

Current state:

 

[Maeson]

I can barely keep up with the software development jargon but what a roller coaster this is.

 

[nitr8]

OGG related game crash solved. Music is playing now.

Still have to fix the OPL music problem, which doesn't throw any error. Music isn't playing there...

For OGG it looks like we have to enable "_REENT_SMALL" for the C library. Otherwise the "FILE" structure's "stdin", "stdout", "stderror" at some point won't be allocated to memory.

"_REENT_SMALL" makes sure that memory is allocated BEFORE anything else get's written into memory.
If it's disabled, there might be no pointer to memory and therefore the game will crash when it tries to write into memory.
This info is located inside "reent.h" of the devkitPro headers:

/*
* If _REENT_SMALL is defined, we make struct _reent as small as possible,
* by having nearly everything possible allocated at first use.
*/

Usually this is NEVER being set when building devkitPro from ground up using it's buildscript and it's also NEVER being used when compiling stuff inside devkitPPC for the Wii.

There still is a flaw with OGG music playback: It works now but at some time suddenly starts lagging and slows down the game real bad. Still need to resolve this.

 

[nitr8]

I had time to test a little bit... These are my findings:

The game is behaving strange - sometimes randomly crashing.

1.) Starting a new game on E1M1 in skill level 4 (U.V.) makes the game crash immediately when spawning items / monsters onto the map.

2.) OPL (MUS / MID) music not working
3.) OGG music stuttering
4.) Some items aren't being displayed correctly (invisible)
5.) On E2M1 there's the chaingun sprite of the player's weapon seen on the map ground (WTF?!)
6.) Game crashes when precaching levels
7.) Game crashes when the player "sees" the health potion item
8.) Caching some lumps makes the game crash
9.) Debugger stepping into the game as if it would have been crashing while the game didn't actually crash
10.) Allocating memory to addresses BEYOND the buffer (buffer overflow)
11.) Demoscreen cycling crashes after several screen switches

It turned out that it seems as this was related to the MetroWerks linker variables "_SDA_BASE" and "_SDA2_BASE" not being correctly set as they are provided by the linker itself. Setting them manually now throws warnings upon linking the ELF but to my surprise it seems as if all of the errors above are suddenly gone... (except for the music thing)...

Let's see what happens next...

P.S.: The "_REENT_SMALL" workaround is also obsolete now. Turns out that libOGC v1.8.12 as of devkitPro r27 usually does "_REENT_INIT_PTR" for GCC only. When this was changed to work with the MWCC too, it no longer crashed upon disabling the previous fix.

 

[nitr8]

Get me some beer, because...

I got OPL to work this morning by only changing 1 thing.
This is the source code of the struct table of the MIDI chunk header:
-----------------------------------
#ifdef _MSC_VER
#pragma pack(push, 1)
#endif

typedef struct
{
byte chunk_id[4];
unsigned int chunk_size;
} PACKEDATTR chunk_header_t;

typedef struct
{
chunk_header_t chunk_header;
unsigned short format_type;
unsigned short num_tracks;
unsigned short time_division;
} PACKEDATTR midi_header_t;

#ifdef
#pragma pack(pop)
#endif
-----------------------------------
Changing the PACKEDATTR preprocessor #define for also using __MWERKS__ didn't work. It was only "enabled" for __GNUC__ until this time.

So it turned out that ONLY changing the kind of struct packing suddenly made OPL work.

The actual change was this one:

From: #ifdef _MSC_VER
To:
#if defined(_MSC_VER) || defined(__MWERKS__)

 

[Maeson]

Goddamn, that's a minuscule thing... Sounds infuriating.
Congratulations in finding it out and fixing it, though!

 

[nitr8]

This isn't easy at all. I'm trying to fix stuff for this special compiler which they once used to create real games for the Wii.

Will search for the OGG lag bug next. OGG music works but then (for some reason) it starts lagging the whole game. It kind of stutters. This situation is only caused when OGG music playback is enabled and running.

 

[Maeson]

With "minuscule" I didn't mean easy or simple, just that the actual change looks small for how big of a change it makes.

 

[nitr8]

I have been playing through the whole first episode of DOOM Registered v1.2 with OPL and Gamecube controls. No crash at all so far. I had a freeze at the intermission screen this morning when trying to get from E1M6 to E1M7. This didn't happen now. Good to know that it seems to be working at last. Hope to give some more news very soon.

 

[niuus]

I have been playing through the whole first episode of DOOM Registered v1.2 with OPL and Gamecube controls. No crash at all so far. I had a freeze at the intermission screen this morning when trying to get from E1M6 to E1M7. This didn't happen now. Good to know that it seems to be working at last. Hope to give some more news very soon.

Gamecube controller support? Awesome news. Thanks for your news flash!

 

[nitr8]

Gamecube controller support? Awesome news. Thanks for your news flash!

Yeah, i currently have no other choice than to implement this because there is no chance of getting the bluetooth module to work with the Wiimote / Classic controller. So there furthermore can't be any support to bluetooth devices like the Wii U Pro controller etc.

There must be some kind of bug inside the wiiuse or bte library that i were not able to find / fix.

One more freeze inside E2M3 right after starting the level... music was still playing

 

[niuus]

Yeah, i currently have no other choice than to implement this because there is no chance of getting the bluetooth module to work with the Wiimote / Classic controller. So there furthermore can't be any support to bluetooth devices like the Wii U Pro controller etc.

There must be some kind of bug inside the wiiuse or bte library that i were not able to find / fix.

So, if i understand right, the old source code (r33) will be the last version with bluetooth devices support?