Hacking Retroarch Wii U, devkitpro r38 and WUT

ploggy

WAKA! WAKA!
Member
Joined
Aug 29, 2007
Messages
4,800
Trophies
2
XP
7,773
Country
United Kingdom
Another new Core
REminiscene - Flashback - this Core needs:
The Flashback data files - Mandatory
Sound files from the Amiga version - Optional
Voice.vce from the Sega CD - Optional
- hot to setup ;)
Tested and working!
(Load the Core then Load Content, navigate to your Flashback Data files then load the LEVEL1.LEV file) :D
 

Attachments

  • reminiscence_libretro.rar
    2.8 MB · Views: 199

ShadowOne333

QVID PRO QVO
Editorial Team
Joined
Jan 17, 2013
Messages
12,106
Trophies
2
XP
32,408
Country
Mexico
Another new Core
REminiscene - Flashback - this Core needs:
The Flashback data files - Mandatory
Sound files from the Amiga version - Optional
Voice.vce from the Sega CD - Optional
- hot to setup ;)
Tested and working!
(Load the Core then Load Content, navigate to your Flashback Data files then load the LEVEL1.LEV file) :D

Nice to see you got a hold of it with gblues help!
Perhaps a tutorial would help for others that might want to develop :P
Also, don't forget to add cores to the Makefile if they work properly!

Oh one more thing, can you test if PrBoom is crashing after some quick saves and quick loads?
I think that issue is carried across all platforms.
 
  • Like
Reactions: ploggy

ploggy

WAKA! WAKA!
Member
Joined
Aug 29, 2007
Messages
4,800
Trophies
2
XP
7,773
Country
United Kingdom
Nice to see you got a hold of it with gblues help!
Perhaps a tutorial would help for others that might want to develop :P
Also, don't forget to add cores to the Makefile if they work properly!

Oh one more thing, can you test if PrBoom is crashing after some quick saves and quick loads?
I think that issue is carried across all platforms.
Yeah Gblues really saved my sanity :P
I'll attempt a tut tomorrow after some kip, its pretty easy once you get the steps down :)

Sure I can test PrBoom :)
I'm slowly going through the list of Cores to see what can be compiled (and work) for WiiU, fMSX and Prosystem used to be on WiiU before the build bot change but Flashback is brand new!
Once I get through them I'll see if Autechre will add them to the build bot ;)
I still have to post our issue list too :P I'll do that once I get through this :D PROGRESS!! lol
 
  • Like
Reactions: ShadowOne333

ShadowOne333

QVID PRO QVO
Editorial Team
Joined
Jan 17, 2013
Messages
12,106
Trophies
2
XP
32,408
Country
Mexico
Yeah Gblues really saved my sanity :P
I'll attempt a tut tomorrow after some kip, its pretty easy once you get the steps down :)

Sure I can test PrBoom :)
I'm slowly going through the list of Cores to see what can be compiled (and work) for WiiU, fMSX and Prosystem used to be on WiiU before the build bot change but Flashback is brand new!
Once I get through them I'll see if Autechre will add them to the build bot ;)
I still have to post our issue list too :P I'll do that once I get through this :D PROGRESS!! lol
You know what other core used to be there too?
Don't wanna say the name, but it ran at 6 fps lol

Thanks for testing!
And best of luck with your subsequent tests ;)
 
  • Like
Reactions: ploggy

ploggy

WAKA! WAKA!
Member
Joined
Aug 29, 2007
Messages
4,800
Trophies
2
XP
7,773
Country
United Kingdom
-: Docker WiiU Tutorial :-

(Assumes your using Windows)

1. Download and Install Docker from here
2. Download and extract this Dockefile to a Directory.
3. Open command prompt,
4. Change to directory where your Dockerfile is located.
5. Run: docker build . -t wiiu-build

1. Download/Git Clone Retroarch.
2. Download/Git Clone any Core your looking to Build.
3. Open Command Prompt.
4. CD into your chosen Core Directory.

then do the following:
1. Run docker run -it -v %CD%:/developer wiiu-build - which will get you a linux command prompt.
2. Run cd /developer
3. Run make platform=wiiu and it will run the build. shouldn't take too long.
4. Run exit to exit the container.

Goto you Core folder location.
The Core folder should now have a (Corename)_libretro_wiiu.a file, Rename file to libretro_wiiu.a and place in your Retroarch Folder. (overwrite if needed)

1. Now in Command Prompt CD to your Retroarch Folder.
2. Run docker run -it -v %cd%:/developer wiiu-build - which will get you a linux command prompt.
3. Run cd /developer
4. Run make -f Makefile.wiiu and it will run the build. (This will take longer than the Core.)
5. Run exit to exit the container.

Once finished it should generate a libretro_wiiu.rpx file in Retroarch/objs/wiiu
Rename to (Corename)_libretro.rpx and place in your SD card's Retroarch Core folder :)

Rinse and repeat for any other Core you want to Compile.
 

Attachments

  • Dockerfile.rar
    418 bytes · Views: 180
Joined
Apr 19, 2015
Messages
1,023
Trophies
1
Location
Stuck in the PowerPC
Website
heyquark.com
XP
3,900
Country
Australia
Had a go at running a random build off the git head with the latest devkitPPC tooling, was able to replicate the system memory error. Luckily the console also made a log, which led me to this check in decaf, and after significant tomfoolery and chasing dead ends I ended up at this rather spicy output:
Code:
$ readrpl -r retroarch_wiiu.rpx | grep REL24
  025EBCF4 0000030A REL24            00000000 $UNDEF + 0
  025EBD10 0000030A REL24            00000000 $UNDEF + 0
  025EBD2C 0000030A REL24            00000000 $UNDEF + 0
  025EBD60 0000030A REL24            00000000 $UNDEF + 0
  025EBDA4 0000030A REL24            00000000 $UNDEF + 0
  025EBDE8 0000030A REL24            00000000 $UNDEF + 0
  025EBE14 0000030A REL24            00000000 $UNDEF + 0
  025EBE28 0000030A REL24            00000000 $UNDEF + 0
  025EBE44 0000030A REL24            00000000 $UNDEF + 0
  025EBE60 0000030A REL24            00000000 $UNDEF + 0
  025EBE7C 0000030A REL24            00000000 $UNDEF + 0
  025EBE98 0000030A REL24            00000000 $UNDEF + 0
  025EBEB4 0000030A REL24            00000000 $UNDEF + 0
  025EBED0 0000030A REL24            00000000 $UNDEF + 0
  025EBEE4 0000030A REL24            00000000 $UNDEF + 0
  025EBF00 0000030A REL24            00000000 $UNDEF + 0
  025F08B4 0000030A REL24            00000000 $UNDEF + 0
  025F13B4 0000030A REL24            00000000 $UNDEF + 0
  025F13D0 0000030A REL24            00000000 $UNDEF + 0
  025F13EC 0000030A REL24            00000000 $UNDEF + 0
  025F1408 0000030A REL24            00000000 $UNDEF + 0

Those addresses all belong to things like _exit, __gthread_once, __libc_lock_init (?), basically devkitPPC-internal stuff that has been added or tweaked in the 10 or so intervening releases. Usually you'd have this covered by a BSP like wut (exit/libc lock, gthread) but since RetroArch doesn't use wut these have slipped in unnoticed.
I have zero idea why RA even compiles with the latest tooling, I would have really expected missing stuff like this to throw a build error instead of spitting out invalid binaries (specifically, they include .dynsym sections instead of erroring out, which is totally nonsensical for the Wii U - even on PC it's questionable). Presumably RetroArch periodically finds broken or missing functionality even in the old tools, which would explain why some versions inexplicably give system memory errors - the actual binaries are broken.

In theory RetroArch could copy in the code for ghtread and things, but honestly between apparently silent errors like this, fixes that never made it into RA's ancient build, and the lack of a clear answer to "what happens another few devkitPPC releases from now" I would think RetroArch would be better off with a port to wut proper and doing away with the in-tree toolchain.
 
Last edited by QuarkTheAwesome, , Reason: add decaf link

ploggy

WAKA! WAKA!
Member
Joined
Aug 29, 2007
Messages
4,800
Trophies
2
XP
7,773
Country
United Kingdom
Had a go at running a random build off the git head with the latest devkitPPC tooling, was able to replicate the system memory error. Luckily the console also made a log, which led me to this check in decaf, and after significant tomfoolery and chasing dead ends I ended up at this rather spicy output:
Code:
$ readrpl -r retroarch_wiiu.rpx | grep REL24
  025EBCF4 0000030A REL24            00000000 $UNDEF + 0
  025EBD10 0000030A REL24            00000000 $UNDEF + 0
  025EBD2C 0000030A REL24            00000000 $UNDEF + 0
  025EBD60 0000030A REL24            00000000 $UNDEF + 0
  025EBDA4 0000030A REL24            00000000 $UNDEF + 0
  025EBDE8 0000030A REL24            00000000 $UNDEF + 0
  025EBE14 0000030A REL24            00000000 $UNDEF + 0
  025EBE28 0000030A REL24            00000000 $UNDEF + 0
  025EBE44 0000030A REL24            00000000 $UNDEF + 0
  025EBE60 0000030A REL24            00000000 $UNDEF + 0
  025EBE7C 0000030A REL24            00000000 $UNDEF + 0
  025EBE98 0000030A REL24            00000000 $UNDEF + 0
  025EBEB4 0000030A REL24            00000000 $UNDEF + 0
  025EBED0 0000030A REL24            00000000 $UNDEF + 0
  025EBEE4 0000030A REL24            00000000 $UNDEF + 0
  025EBF00 0000030A REL24            00000000 $UNDEF + 0
  025F08B4 0000030A REL24            00000000 $UNDEF + 0
  025F13B4 0000030A REL24            00000000 $UNDEF + 0
  025F13D0 0000030A REL24            00000000 $UNDEF + 0
  025F13EC 0000030A REL24            00000000 $UNDEF + 0
  025F1408 0000030A REL24            00000000 $UNDEF + 0

Those addresses all belong to things like _exit, __gthread_once, __libc_lock_init (?), basically devkitPPC-internal stuff that has been added or tweaked in the 10 or so intervening releases. Usually you'd have this covered by a BSP like wut (exit/libc lock, gthread) but since RetroArch doesn't use wut these have slipped in unnoticed.
I have zero idea why RA even compiles with the latest tooling, I would have really expected missing stuff like this to throw a build error instead of spitting out invalid binaries (specifically, they include .dynsym sections instead of erroring out, which is totally nonsensical for the Wii U - even on PC it's questionable). Presumably RetroArch periodically finds broken or missing functionality even in the old tools, which would explain why some versions inexplicably give system memory errors - the actual binaries are broken.

In theory RetroArch could copy in the code for ghtread and things, but honestly between apparently silent errors like this, fixes that never made it into RA's ancient build, and the lack of a clear answer to "what happens another few devkitPPC releases from now" I would think RetroArch would be better off with a port to wut proper and doing away with the in-tree toolchain.

What would it take for Retroarch to be updated to used the latest WUT toolchain?
What would be the benefits of updating to the new WUT apart from stability?
Would the SD card speed be better/faster (I remember there was updates to the SD card that allowed faster read/write speeds than the WiiU port currently has?)
I know the new Toolchain easily allows RWX which helps with JIT which in itself would be amazing :)
I know Aroma allows direct booting of Homebrew from the WiiU Menu as Channels, which also is Amazing! but again it would require Retroarch to be updated to use the new WUT..
So really what I'm getting at is would it be worth starting a Bounty or something to get Retroarch Updated with the new WUT Toolchain?
 
Joined
Apr 19, 2015
Messages
1,023
Trophies
1
Location
Stuck in the PowerPC
Website
heyquark.com
XP
3,900
Country
Australia
What would it take for Retroarch to be updated to used the latest WUT toolchain?
What would be the benefits of updating to the new WUT apart from stability?
Would the SD card speed be better/faster (I remember there was updates to the SD card that allowed faster read/write speeds than the WiiU port currently has?)
I know the new Toolchain easily allows RWX which helps with JIT which in itself would be amazing :)
I know Aroma allows direct booting of Homebrew from the WiiU Menu as Channels, which also is Amazing! but again it would require Retroarch to be updated to use the new WUT..
So really what I'm getting at is would it be worth starting a Bounty or something to get Retroarch Updated with the new WUT Toolchain?
Benefits:
  • If I diagnosed the 160-2203s correctly then the updated linkscripts and elf2rpl in wut should reduce or remove that issue
  • Updated elf2rpl also supports compressed binaries, which should make rpxes a good deal smaller
  • First step towards Aroma compatibility and .wuhb builds (there are still a few other things to do, notably argc/argv handling and ioctl100 for rpx loading, and on a more difficult note handling ProcUI and the home menu overlay)
    • .wuhb can bundle files into the binary, so maybe we could include the menu assets and .info files? not 100% on how that'd interact with core loading but hey. I'm told accessing bundled files is much faster than reading SD even after optimisation (see the GTA3 port)
    • I'm sure someone could write tools to put a rom in the .wuhb too and do your own virtual-console-y things
    • ioctl100 (Aroma-exclusive) is a much, much nicer way to load RPXes and switch cores. it's not really a user-visible change but it makes me happy
    • Aroma mucks with the cos.xml permissions to allow "easy" JIT (it still has the W^X thing I've discussed previously, but it's better than the HBL's "half of installations can't do JIT at all") - though personally I still think this should be added to the libretro API or libretro-common rather than handling it in the core
    • (dreaming, this is very unlikely) wut can build custom .rpls and I've asked if Aroma can support loading them so maybe the wiiu port of RA could be the first non-static console platform - loading and unloading cores into one central RetroArch binary like on PC, rather than having each core be a standalone binary with menus and everything
  • wut's socket devoptab makes networking a bit more standard, which could (but may not) improve netplay, though we won't know until we see it
  • wut's filesystem devoptab is a bit more modern, which yes, should improve SD card speeds for large files and aligned buffers on non-iosuhax systems - it's also known to work properly with setvbuf which should guarantee ideal conditions for the speed boost - though it is possible to patch the same improvements into the existing RA code as I discussed previously, with a wut port you get it free
  • I'm not sure if retroarch ever added an rthreads wrapper for wiiu, but wut also supports threading via std::thread and c11 threads, so in case retroarch doesn't have the wiiu-specific code already we should get that free and potentially make use of the three CPU cores instead of just one (depends on how effectively RA utilises that, though)
  • Other minor things (compatibility with the latest devkitPPC release - newer compilers and better codegen, battle-tested Cafe headers should give more accurate structs, GX2 helper functions, wrappers for C++ libraries like nn::swkbd, etc.)
  • As long as wut is maintained, it should be possible to build RetroArch against it with the latest devkitPPC release - would only face the usual Odd devkitPro Changes (like the int/long int thing) rather than arcane bugs like what I found here
Downsides:
  • It could potentially be a lot of work, especially things like struct naming differences could wreak havoc in the GX2 driver
  • HBL ELF builds would be goneski, we'd be RPX only even for non-Aroma users
  • There's quite a few awkward points in the existing RetroArch code - things like the audio driver using AXMultiVoice, which is only mentioned on the internet inside Cemu changelogs and RetroArch itself. Things like that would either need to be added into wut (unlikely given the lack of independent documentation), workaround headers included in RetroArch (making wut updates a bit more difficult), or the audio driver changed to use a more standard API
  • The main motivation (beyond stability) would be Aroma compatibility, which has a lot of unique features and incompatible APIs, so at some point the we'd have to talk about whether Aroma should be a hard requirement and drop support for HBL users too. at least initially it should be possible to support both though
  • ProcUI support for Aroma would mean deciding between not using the HOME overlay, or mapping the RetroArch menu to something else (right stick? TV? not many options)
  • Existing playlists and the like would probably break since the paths are all different (different devoptab). Something could potentially be done about this in libretro-common but I would have to ask
I've been idly turning over doing something like this for a while now - Maschell wants me to so RetroArch can work under Aroma - so I'd be down to work on this, I've just been busy and other projects have taken up what time I do have at the mo (uni's in the final weeks). I dunno if bounties and me are a bit of a sensitive topic after I pushed so hard with the PSX one but, y'know, if y'all wanted to do one I'd take it, if nobody else did ^_^
Should probably get an OK from someone who's been working on RetroArch a bit more regularly though, I know they've had issues with external toolchains before (libogc) so maybe they like having one in-tree. There are ways around it - they've copy-pasted the libogc source into RA, or you could use a submodule. Need a game plan, I s'pose.
 
  • Like
Reactions: ploggy

ploggy

WAKA! WAKA!
Member
Joined
Aug 29, 2007
Messages
4,800
Trophies
2
XP
7,773
Country
United Kingdom
Benefits:
  • If I diagnosed the 160-2203s correctly then the updated linkscripts and elf2rpl in wut should reduce or remove that issue
  • Updated elf2rpl also supports compressed binaries, which should make rpxes a good deal smaller
  • First step towards Aroma compatibility and .wuhb builds (there are still a few other things to do, notably argc/argv handling and ioctl100 for rpx loading, and on a more difficult note handling ProcUI and the home menu overlay)
    • .wuhb can bundle files into the binary, so maybe we could include the menu assets and .info files? not 100% on how that'd interact with core loading but hey. I'm told accessing bundled files is much faster than reading SD even after optimisation (see the GTA3 port)
    • I'm sure someone could write tools to put a rom in the .wuhb too and do your own virtual-console-y things
    • ioctl100 (Aroma-exclusive) is a much, much nicer way to load RPXes and switch cores. it's not really a user-visible change but it makes me happy
    • Aroma mucks with the cos.xml permissions to allow "easy" JIT (it still has the W^X thing I've discussed previously, but it's better than the HBL's "half of installations can't do JIT at all") - though personally I still think this should be added to the libretro API or libretro-common rather than handling it in the core
    • (dreaming, this is very unlikely) wut can build custom .rpls and I've asked if Aroma can support loading them so maybe the wiiu port of RA could be the first non-static console platform - loading and unloading cores into one central RetroArch binary like on PC, rather than having each core be a standalone binary with menus and everything
  • wut's socket devoptab makes networking a bit more standard, which could (but may not) improve netplay, though we won't know until we see it
  • wut's filesystem devoptab is a bit more modern, which yes, should improve SD card speeds for large files and aligned buffers on non-iosuhax systems - it's also known to work properly with setvbuf which should guarantee ideal conditions for the speed boost - though it is possible to patch the same improvements into the existing RA code as I discussed previously, with a wut port you get it free
  • I'm not sure if retroarch ever added an rthreads wrapper for wiiu, but wut also supports threading via std::thread and c11 threads, so in case retroarch doesn't have the wiiu-specific code already we should get that free and potentially make use of the three CPU cores instead of just one (depends on how effectively RA utilises that, though)
  • Other minor things (compatibility with the latest devkitPPC release - newer compilers and better codegen, battle-tested Cafe headers should give more accurate structs, GX2 helper functions, wrappers for C++ libraries like nn::swkbd, etc.)
  • As long as wut is maintained, it should be possible to build RetroArch against it with the latest devkitPPC release - would only face the usual Odd devkitPro Changes (like the int/long int thing) rather than arcane bugs like what I found here
Downsides:
  • It could potentially be a lot of work, especially things like struct naming differences could wreak havoc in the GX2 driver
  • HBL ELF builds would be goneski, we'd be RPX only even for non-Aroma users
  • There's quite a few awkward points in the existing RetroArch code - things like the audio driver using AXMultiVoice, which is only mentioned on the internet inside Cemu changelogs and RetroArch itself. Things like that would either need to be added into wut (unlikely given the lack of independent documentation), workaround headers included in RetroArch (making wut updates a bit more difficult), or the audio driver changed to use a more standard API
  • The main motivation (beyond stability) would be Aroma compatibility, which has a lot of unique features and incompatible APIs, so at some point the we'd have to talk about whether Aroma should be a hard requirement and drop support for HBL users too. at least initially it should be possible to support both though
  • ProcUI support for Aroma would mean deciding between not using the HOME overlay, or mapping the RetroArch menu to something else (right stick? TV? not many options)
  • Existing playlists and the like would probably break since the paths are all different (different devoptab). Something could potentially be done about this in libretro-common but I would have to ask
I've been idly turning over doing something like this for a while now - Maschell wants me to so RetroArch can work under Aroma - so I'd be down to work on this, I've just been busy and other projects have taken up what time I do have at the mo (uni's in the final weeks). I dunno if bounties and me are a bit of a sensitive topic after I pushed so hard with the PSX one but, y'know, if y'all wanted to do one I'd take it, if nobody else did ^_^
Should probably get an OK from someone who's been working on RetroArch a bit more regularly though, I know they've had issues with external toolchains before (libogc) so maybe they like having one in-tree. There are ways around it - they've copy-pasted the libogc source into RA, or you could use a submodule. Need a game plan, I s'pose.

lol yeah the with the current elf2rpl the WiiU Cores can be 12MB plus (even the smaller Cores) I would guess just reducing the Core size would reduce load times? :P
Then there's the ioctl100 RPX loading thing you mention, that would provide the biggest reduction in load times? iirc the way the current RPX loader works on Retroarch means it reloads HBL to switch the Cores? (I may have misunderstood that tho) :P

Building roms into a .wuhb so we can launch Retroarch roms directly from the WiiU Menu Virtual-Console style is a massive plus if possible, every other Console/Handheld port of Retroarch can do this iirc and is one of those missing features I think a lot of users would like to have.

I know I keep harping on about this but the ability to display different things on the TV and WiiU gamepad is a big missed opportunity in Retroarch, to be able to display say a Keyboard that can be interacted with on the Gamepad and an Emulator on the TV at the same time is something that can only be done on the WiiU and would be a great WiiU specific selling point :P
(actually the DS/3DS can technically do this too... just pretend they don't exist) :P or perhaps looking into how they did it would help? I dunno.

Making Cores rpl's is very cool, I remember you mentioning on Discord before and would a nice feature to have.

I know who you feel, I'm still pushing for the PSX Bounty :tpi: I know Bounty can be a dirty word when it comes to WiiU/Retroarch but If a Bounty can grease the wheels then Imo it would be worth it, especially because of the benefits you pointed out. But I guess its up to the rest of the users how they feel about it?
WiiU Retroarch has been slowly been drifting further and further behind feature-wise to mainline Retroarch and I fear the same thing will happen when Aroma is released and becomes the standard.
It's great to hear your willing to work on updating the WiiU port :) It's times like this I wish I could code so I could at least help out :P
Gblues is the current WiiU dev with Retroarch but I don't know if he knows anything about Aroma?
 
  • Like
Reactions: ShadowOne333
Joined
Apr 19, 2015
Messages
1,023
Trophies
1
Location
Stuck in the PowerPC
Website
heyquark.com
XP
3,900
Country
Australia
lol yeah the with the current elf2rpl the WiiU Cores can be 12MB plus (even the smaller Cores) I would guess just reducing the Core size would reduce load times? :P
I'm not sure what impact it'd have, since they still need to be decompressed, but it would be very cool to find out! xP

Then there's the ioctl100 RPX loading thing you mention, that would provide the biggest reduction in load times? iirc the way the current RPX loader works on Retroarch means it reloads HBL to switch the Cores? (I may have misunderstood that tho) :P
ioctl100 still loads RPXes in much the same way RA does now, it's just that the code to achieve that is muuuch cleaner. It's not really a killer feature, I probably didn't need to include it.. but yes, all RPX loading we have right now gets an RPX into memory, then restarts the current application (Mii Maker for HBL systems, Health & Safety for Aroma). It's about as quick as you could expect that process to be.

Building roms into a .wuhb so we can launch Retroarch roms directly from the WiiU Menu Virtual-Console style is a massive plus if possible, every other Console/Handheld port of Retroarch can do this iirc and is one of those missing features I think a lot of users would like to have.
Assuming we get RetroArch working under Aroma, then it should also work as a .wuhb!

I know I keep harping on about this but the ability to display different things on the TV and WiiU gamepad is a big missed opportunity in Retroarch, to be able to display say a Keyboard that can be interacted with on the Gamepad and an Emulator on the TV at the same time is something that can only be done on the WiiU and would be a great WiiU specific selling point :P
(actually the DS/3DS can technically do this too... just pretend they don't exist) :P or perhaps looking into how they did it would help? I dunno.
It's not really related to any wut/toolchain-y things, and I'm not sure what the best way to handle it would be (on the DS, you can always show the game on one of the screens and dedicate the other to your keyboard, but on WiiU, people sometimes want to play on the Gamepad screen). I should have a peek at how they pull it off, though. my suspicion is that they just don't tell RetroArch about the secondary display and hide a separate keyboard somewhere in the console-specific code.

Gblues is the current WiiU dev with Retroarch but I don't know if he knows anything about Aroma?

I was just thinking I should hop on the Discord and say hi to him!
 
  • Like
Reactions: ploggy

ploggy

WAKA! WAKA!
Member
Joined
Aug 29, 2007
Messages
4,800
Trophies
2
XP
7,773
Country
United Kingdom
ioctl100 still loads RPXes in much the same way RA does now, it's just that the code to achieve that is muuuch cleaner. It's not really a killer feature, I probably didn't need to include it.. but yes, all RPX loading we have right now gets an RPX into memory, then restarts the current application (Mii Maker for HBL systems, Health & Safety for Aroma). It's about as quick as you could expect that process to be.
I see, still the wut devotab stuff will still help a lot for SD speeds, which really is one of the biggest gripes with the current port.. That and the fact we can't have the gamepad and a pro controller be player 1 at the same time.

It's not really related to any wut/toolchain-y things, and I'm not sure what the best way to handle it would be (on the DS, you can always show the game on one of the screens and dedicate the other to your keyboard, but on WiiU, people sometimes want to play on the Gamepad screen). I should have a peek at how they pull it off, though. my suspicion is that they just don't tell RetroArch about the secondary display and hide a separate keyboard somewhere in the console-specific code.
I know but I couldn't help throwing it in there. Using the TV button as a toggle for Keyboard/standard could be an option?

I was just thinking I should hop on the Discord and say hi to him!
He's a good guy, walked me through compiling wiiu with Docker so the guys got patience lol
 
Last edited by ploggy,

hito16

Member
OP
Newcomer
Joined
Jan 15, 2021
Messages
19
Trophies
0
Age
54
XP
90
Country
United States
Hi @ploggy and @QuarkTheAwesome. Catching up on recent comments, it sounds like the effort is worthwhile and doable. I'll have some time to revisit Wup & devkitpro modernization later this month (July 2021). I'd like to coordinate with the right folks. Is the best way to reach everyone to stay logged into discord?

Looking at recent comments, it sounds like the effort involves:

  1. Add dual devkitpro version support to 30+ makefiles: Need to support building cores with both devkitpro versions during migration. IIRC 20 or so cores use -mwup option which is required for older devkitpro versions, whereas -mwup will break builds using newer devkitpro version. Since each retroarch core has its own repo, retroarch makefiles need to support old and new devkitpro versions simultaneously until all core repos are fully upgraded to use devkipro >= r38. If we don't do this, the wiiu retroarch nightly builds will break.
  2. Modern WUP support : 30 or so cores build with WUP libs shipped with retroarch. Change these to use the WUP support shipped with modern devkitpro packages. Per @QuarkTheAwesome, even if the cores build with devkitpro >= r38 and package with older WUP toolchain, they won't run due to incompatibilities with old vs new WUP implementation.
  3. test, test, test - per re2, these cores will build with r38, but not launch until we use r38's built in WUP toolchain. Additionally, the base libraries and includes shipped with devkitpro have changed between r29 (current nightly environment) and r38 (latest build I tried). Simple things like paths and include flags will be easy, but if basics like networking api wrappers changed... TBD - check with @QuarkTheAwesome if his decaf patch was mainlined into retroarch for easy testing.
  4. fix unexpected breakages from 3.
  5. update official nightly build environment to use devkitpro >= r38
  6. cleanup (optional) - remove migration code from build scripts, remove wup libs from Retroarch repo, remove support for devkitpro < r38
 
Last edited by hito16,
  • Like
Reactions: ploggy

ploggy

WAKA! WAKA!
Member
Joined
Aug 29, 2007
Messages
4,800
Trophies
2
XP
7,773
Country
United Kingdom
Hi @ploggy and @QuarkTheAwesome. Catching up on recent comments, it sounds like the effort is worthwhile and doable. I'll have some time to revisit Wup & devkitpro modernization later this month (July 2021). I'd like to coordinate with the right folks. Is the best way to reach everyone to stay logged into discord?

Looking at recent comments, it sounds like the effort involves:

  1. Add dual devkitpro version support to 30+ makefiles: Need to support building cores with both devkitpro versions during migration. IIRC 20 or so cores use -mwup option which is required for older devkitpro versions, whereas -mwup will break builds using newer devkitpro version. Since each retroarch core has its own repo, retroarch makefiles need to support old and new devkitpro versions simultaneously until all core repos are fully upgraded to use devkipro >= r38. If we don't do this, the wiiu retroarch nightly builds will break.
  2. Modern WUP support : 30 or so cores build with WUP libs shipped with retroarch. Change these to use the WUP support shipped with modern devkitpro packages. Per @QuarkTheAwesome, even if the cores build with devkitpro >= r38 and package with older WUP toolchain, they won't run due to incompatibilities with old vs new WUP implementation.
  3. test, test, test - per re2, these cores will build with r38, but not launch until we use r38's built in WUP toolchain. Additionally, the base libraries and includes shipped with devkitpro have changed between r29 (current nightly environment) and r38 (latest build I tried). Simple things like paths and include flags will be easy, but if basics like networking api wrappers changed... TBD - check with @QuarkTheAwesome if his decaf patch was mainlined into retroarch for easy testing.
  4. fix unexpected breakages from 3.
  5. update official nightly build environment to use devkitpro >= r38
  6. cleanup (optional) - remove migration code from build scripts, remove wup libs from Retroarch repo, remove support for devkitpro < r38
Yes, the 4TU Discord is where most of the WiiU Dev folk hang out :) I'm looking forward to whatever you can do to update the WiiU RA port :)
 

hito16

Member
OP
Newcomer
Joined
Jan 15, 2021
Messages
19
Trophies
0
Age
54
XP
90
Country
United States
Quick Update: I revisited building on dkp r39 with WUT toolchain this weekend, with a goal of updating retroarch wiiu to use modern supported compilers and toolchains.

The Good: I made the modifications for the .o's to compile with dkp ppc r39 and wut dependencies. The Bad: I ran into issues with new linkscripts. The old retroarch/wiiu linkscripts provided externs that the retroarch libraries depend on; these externs are no longer provided by the WUT linker scripts provided with dpk.

More details at the end of post.

Next steps: I need to read up on elf link scripts. I'd like to a find a solution that doesn't require modifying the WUT toolchain linkscripts packaged and maintained by the Devkitpro teams.

The good details:
* All supported cores can compile to a ".a" using mwup hack (mentioned in early comment) removes -mwup options on cores. Doxbox now needs compile flag --std::c++14 to build against latest gcc in dkp ppc r39
* Makefile.wiiu does compile the ".o"'s against dkp ppc libraries. Diff patch in Pastebin dot com GLTWtZXJ (no urls for noobs like me)
* Linking doesn't fully work but I did get past some link errors by temporarily commenting out duplicate memory function declarations. (wiiu/system/memory.c)". TBD: if we need to support both old and new dpk builds, we'll need to refactor out duplicate [c,m]alloc methods into a separate util class gated by ifdef.

The Bad details:
* The built in link scripts (link_elf.ld & link_rpl.d) PROVIDE __code_start/__code_end and _end for hbl.c and exception_handler.c respectively. The hbl.c code looks like copy pasta from the homebrew launcher code, so no luck finding a modern example of a process launching a process using modern link scripts.
* The new link script (wut/share/wut.ld) doesn't seem to PROVIDE() equivalent symbols I can just add to hbl.c and exception_handler.c
* I have never touched link scripts before. Google time.
 
  • Like
Reactions: depaul

hito16

Member
OP
Newcomer
Joined
Jan 15, 2021
Messages
19
Trophies
0
Age
54
XP
90
Country
United States
Further update: I got the main retroarch_wii.elf and rpx linked using the DKP ppc r39 plus wut package tools chain. For classes using externs provided by the linker script, I had to comment out exception handling for now, but I think the section name used by hbl is right. I'll follow up with linking the individual cores later, since the main "stable" rpx build doesn't work with the older DKP. I'll follow up with discord if I have questions.

Changes were:
new file: wiiu/system/stubs_hbl.S
modified: Makefile.wiiu
modified: dist-scripts/wiiu-cores.sh
modified: wiiu/hbl.c
modified: wiiu/system/exception_handler.c
modified: wiiu/system/memory.c

git diff on pastbin dot com: nrHeK868

Gratitude to the authors on linker scripts and linker documentation. (noobs like me unable to post links, but search works)
* google "From Zero to main(): Bare metal C" by François Baldassari
* google "binutils/docs/ld/Source-Code-Reference"
* google "An Introduction to the GNU Compiler and Linker" by William Gatliff
 
  • Like
Reactions: willianholtz

hito16

Member
OP
Newcomer
Joined
Jan 15, 2021
Messages
19
Trophies
0
Age
54
XP
90
Country
United States
Another update: I tried the retroarch binaries I linked a few days ago. I see some obvious runtime errors and nonsensical function names per objdump. Looks like I need to take the time to update (or port) retroach, HBL libraries and deps to point to the wut includes/libs. This is because wiiu retroarch and the dependent homebrew launcher code includes an older version of wut libraries that are bundled into retroarch source. Notably we're still pulling in stuff like gx2.h,ios.h,sysapp.h, etc, from the old libraries, which are shuffled around and renamed slightly in modern includes. Tonight, I'll take a look updating these includes in the retroarch code, and see how much of the whole retroarch/wiiu/* code we can remove altogether.
 

hito16

Member
OP
Newcomer
Joined
Jan 15, 2021
Messages
19
Trophies
0
Age
54
XP
90
Country
United States
Update on WUT binding conversion: I modified the retroarch wiiu code to depend on devkitpro and wut headers rather than the older dependency headers shipped with retroarch.

File list of changes: pastebin dot com jgvsXF0v (excluding "deleted" files I temporarily moved out of the way)

The Good:
* Fully compiled all code against new bindings, except the shaders. To ensure dependencies don't sneak in, I moved retroarch/wiiu/includes/wiiu to retroarch/wiiu/includes/wiiu_BAK and only moved back a handful of files that aren't covered by devkitpro and wut.
* HBL dependency addressed by new linker script (previous post)

The Bad: Few things won't work yet.
* graphics driver and shaders aren't converted to use the newer GL2 bindings. TBD: I need to learn more about device access through GX2 (and GL) API or get help
* debugging / exception handling won't fully work - exception handler relies on symbols exposed from old linker script. TBD: test if bss_start(end) symbols are suitable replacements for "text" section start(end). If not, we may need to change the WUT linker scripts.

Unsure: The code mod was a mechanical replacement of old types & wrappers to new types and wrappers, making some guesses at what to replace at times.
* launching cores via hbl classes should work. I used _bss_start for code start which looks close to the _CODE_START, Again, the conversion to new bindings was based on guesswork, so I just have to try after the other kinks are worked out.
* new network bindings were different enough that I left old bindings in place... for now anyway. I even saw some compiler warning (conflicts) between WUT headers and DEVKITPRO network headers. TBD: followup.
 

Site & Scene News

Popular threads in this forum

Recent Content

General chit-chat
Help Users
  • No one is chatting at the moment.
    K3Nv2 @ K3Nv2: Sorry for accidentally bending over