Physical memory.

Discussion in 'Wii U - Hacking & Backup Loaders' started by andriy921, Mar 27, 2016.

  1. andriy921
    OP

    andriy921 GBAtemp Regular

    Member
    267
    97
    Dec 1, 2015
    I was looking into porting Loadiine GX to 4.0.0.

    Yes, I know that it's pointless and not so straightforward as for 410+, but the main goal of this project is learning.

    So, first big problem is physical memory layout. Seems like 410+ have similar memory layout that looks like this: http://wiiubrew.org/wiki/Physical_Memory. At least 4.1.0-5.4.0 have coreinit + loader at 0x32000000 to 0x327FFFFF. 4.0.0 has that stuff at 0x4D000000 - 0x4D7FFFFF.

    So, my questions:
    1. Is physical memory layout of old firmwares documented somewhere?
    2. How can we find physical location of module that does not export any functions (i.e root.rpx)?
    3. What is the best way to dump memory?
     
    Toscanelli likes this.
  2. NWPlayer123

    NWPlayer123 GBAtemp Addict

    Member
    2,629
    6,226
    Feb 17, 2012
    United States
    The Everfree Forest
    I've been up-porting to 5.5.1, your best way to dump memory is to use TCPGecko and a combination of the dotNet client and pyGecko. Basically what you should be doing is using pyGecko to acquire OSEffectiveToPhysical with get_symbol and doing some comparisons to see where the memory sections are mapped. 0 is never valid and will crash.

    On 551, All system libraries are loaded into 0x01000000, the loader is the only one that lines up since it's loaded exactly at 0x01000000, everything else comes after. coreinit on 5.3.2 and 5.5.1 is at 0x0101C400, you should be able to use the downloaded RPL hex to search for where it starts in the memdump. The loader is pretty much the only big one you'll need to change. 0x01 to 0x01800000 is for system, 0x018+ is the start of JIT for the browser (which you can't use in any other apps BTW), the application code starts somewhere around 0x0E to 0x10000000, for Mii Maker it's at 0x0F000000, and then all data is in 0x10000000 onwards, you get ~1GB for code+data, so it goes from 0x10 to somewhere around 0x50. I think I remember someone saying the JIT address/size and such was in an XML for the application so you could try downloading them from NUS too

    The main things you need to worry about, tho, are loader and kernel which should still be at 0xFFE00000 and decrypted in place (so the ancast header is still there but garbled and actual code starts at 0xFFE00100). If you need help with TCPGecko, it really just goes like this. Python 3, no other dependencies, just make sure firewall and antivirus aren't being stupid. Open with IDLE and run it in there so you can run commands thru it
    tcp = TCPGecko("192.168.0.10")
    >>Connecting...
    >>Connected!
    OSEffectiveToPhysical = tcp.get_symbol("coreinit.rpl", "OSEffectiveToPhysical")
    >>OSEffectiveToPhysical address: 0x00000000 (useful for copypasting hardcoded function offsets)
    OSEffectiveToPhysical(0x32000000)
    >>someaddress in decimal
    tcp.s.close() #Closes socket so you can use it for dotNet, just rerun first line to reconnect
     
  3. andriy921
    OP

    andriy921 GBAtemp Regular

    Member
    267
    97
    Dec 1, 2015
    I was more interested in physical memory dump. But it seems like that is not possible. From other hand dumping virtual ranges might also be useful. Does TCPGecko allow me to download dumps of address ranges?
    Are you talking about virtual addresses? loader and coreinit seem to be 0x01000000 - 0x18000000 on every fw version, but physical address is different (at least for <= 400).
     
  4. Marionumber1

    Marionumber1 GBAtemp Maniac

    Member
    1,234
    3,933
    Nov 7, 2010
    United States
  5. NWPlayer123

    NWPlayer123 GBAtemp Addict

    Member
    2,629
    6,226
    Feb 17, 2012
    United States
    The Everfree Forest
    Oh, yeah, physical address shouldn't matter aside from kernel exploit mapping 0xA0000000 to be able to RWX from somewhere. TCPGecko lets you dump virtual memory addresses, I tell you to use dotNet because I tried pyGecko and it is the slowest thing I have ever seen when doing stuff in bulk. It even has a dedicated Dump tab
     
  6. andriy921
    OP

    andriy921 GBAtemp Regular

    Member
    267
    97
    Dec 1, 2015
    From what I see it matters a lot.
    1. Installer writes jump hook into coreinit func by using 0xA0000000 -> 0x10000000 rw mapping. For firmwares 4.1-5.4 coreinit + laoder are at 0x32000000 physical, so virtual rw address is 0xC2000000, or if want to reuse address from readonly mapping - 0xC1000000 + readonly virtual address. For 400 fw phys address is 0x4D000000, so to hook coreinit we need to change those addresses (luckily they are still before the end of writable mapping).
    2. Installer loads sd_loader into some adress that it search starting from 0xC0000000 and then it loads rpx into 0xC0800000. From what I see this area is reserved for root.rpx. But I have no idea what it is for 400.
    3. I can not keep up with the offset by just changing 0xA0 mapping to 0x10000000 + (0x4D000000-0x32000000), because if I do this, browser fails on second start. It fails on loading (before it shows address bar / html page), so this just setting such mapping already breaks browser.

    Thanks for hint with TCPGecko. That should help a lot.
     
    NWPlayer123 likes this.
  7. FaTaL_ErRoR

    FaTaL_ErRoR AKA ŦƕƎ ƠṀƐƝ

    Member
    491
    346
    Mar 9, 2014
    United States
    You should go after memory using the Dmae.rpl
    Use an emulated terminal to do so.
    1d11 59ef 9666 eed1 f64f e82e eff9 7a1a
    1cbc 6583 ba29 1221 26d6 721b 1be9 9679
    a865 0d4d f237 01a1 afff 101f 5da0 ec83
    4686 b1c1 35ba 09f5 f5a7 020f 8ea9 aa3a
    3f5b 7011 3a9d 3a81 2268 457c 4551 c063
    99de 2830 788c 6266 1b9c 9aba d8a1 de7f
    1bcb 79d2 26f1 1089 8860 7fdc 5f66 71d1
    4dff ffff
    SDReadSector from addr 0x00001600 (sector 11) into buffer 0x03ff988c
    Something of this nature....
    Just read from the internal mem and dump it to sd card.
     
    andriy921 likes this.
  8. andriy921
    OP

    andriy921 GBAtemp Regular

    Member
    267
    97
    Dec 1, 2015
    I don't want to make more threads, so I'll add my new question here.

    I couldn't make tcpGecko work on 4.0.0. It load good, but after pressing any button it blackscreens and client app does not connect while in browser.
    Then I remembered that libwiiu has similar thing. They call it rpc and it does pretty much similar thing.
    So I tried to run it on top of kernel that maps 0xA... to 0x30000000 to be able to access memory ranges that are interesting to me (those are actually 0x30000000-0x50000000). I've tested several things just to check if this works. Then I decided to dump memory in 0x10000000 ranges. First range (0xA0000000 - 0xAFFFFFFF) dumped without problems and looks good. But the second range (0xB0000000-0xBFFFFFFF) is mostly zeroes with several very small intervals that are filled with random stuff that does not make any sense. The problem is I know for sure that 0xBD000000+ in such mapping is coreinit. It resolves correctly to physical in IDLE. The address of the function in coreinit was ‭4D01C76C‬. This area was filled in zeroes in prev dump. But when I tried to dump and read ‭(0xBD01C76C‬, 0x100) it showed me correct values (the same as coreinit lib from NUS). All of this one done in one IDLE/rpc session. I tried to dump that second area two time - one after dumping first range, second - after i checked if (0xBD01C76C‬, 0x100) is correct.

    So, today I'll test this with fresh connection/rpc session (maybe dumping first 256 MB block messed up python memory) and then I'll test with smaller ranges.

    But has someone experienced something like this? This is super annoying and I have no idea were the problem is.
     
    Last edited by andriy921, Mar 29, 2016
  9. Kafluke

    Kafluke GBAtemp Psycho!

    Member
    3,469
    1,719
    May 6, 2006
    United States
    Just so we know where to start, what firmware are you on?
     
  10. andriy921
    OP

    andriy921 GBAtemp Regular

    Member
    267
    97
    Dec 1, 2015
    I've specified it several times. I'm on 4.0.0.
     
  11. Kafluke

    Kafluke GBAtemp Psycho!

    Member
    3,469
    1,719
    May 6, 2006
    United States
    Sorry I missed that.
     
  12. andriy921
    OP

    andriy921 GBAtemp Regular

    Member
    267
    97
    Dec 1, 2015
    Ok, finally i managed to find those physical address changes and successfully managed to port HBL/Loadiine to 4.0.x. I'll provide a brief explanation of my investigation since it might help someone in porting HBL to even earlier firmwares.

    So first of all look at this page http://wiiubrew.org/wiki/Physical_Memory HBL uses physical addresses of root.rpx and loader+coreinit.rpl and relies on fact that they are always the same. But that's true only for 4.1.0+ firmwares.

    To experiment with these thing best instrument is rpc from libwiiu. It allows us to call OSEffectiveToPhysical form python shell, making experiments with addresses simple since we don't need to recompile app to test other addresses. Other great thing about this tool that it does not require kernel access, so it makes our work so much faster.

    So, first of all, we need to understand that physical layout was actually changed. Go here http://wiiubrew.org/wiki/Cafe_OS and look through the virtual mapping table. Virtual range 0x01000000 - 0x01800000 is always mapped to loader and coreinit. So we simply call OSEffectiveToPhysical(0x01000000) and for 4.0.x we receive 0x4D000000 (it's 0x32000000 for 4.1+, so that's a pretty big difference).

    But I still needed to find toot.rpx since hbl uses free space reserved for root.rpx to load elfs/rpls. My first idea was really horrible. I decided to map physical memory that is interesting for me to 0xA0000000 and then dump it using rpc. The problem was - dumps were inconsistent. Occasionally memory that was reserved for coreinit returned me dump full of zeroes or some garbage. Moreover the process of dumping is very slow. So I wasted several days with 0 result.

    So i decided to be smarter. I've found an interesting virtual range - 0x10000000 - 0x50000000. It seems like it is mapped to different parts of MEM2 region depending on the running process RAMPID (All of this is described on those two links that I specified previously). The problem here is that we run our rpc from browser, so we can find out only the address of "Background app" area (according to Cafe OS page). But that's at least something. So we do OSEffectiveToPhysical(0x10000000) and receive 0x30000000 (address of root.rpx for 4.1+ lol). Then I made a desperate move. I knew for sure that mapping for static regions like 0x01000000 to coreinit and that 0xA0000000 stuff is specified in the kernel in the table and we know addresses of those tables (since kernel exploit updates this table to change mapping of 0xA0000000). So I was pretty sure that there should be similar table for 0x10000000 that specifies where should we map that area depending on RAMPID. So I simply searched kernel for value 0x30000000 and to my surprise there were only two occurrences of that value and one of them was actual map table.

    So now let's look into this table and see what happened to those physical layouts. First of all - it seems like table contains of 3-property tuples (RAMPID, PhysicalAddressStart, RangeLength). And there are two very similar tables one after the other. The only difference between them is last entry - that's probably that difference in memory size for devkits. Next let's see into the actual tables for different firmwares. I provide (RAMPID, PhysicalAddressStart, RangeLength) and for the last entry alternative value from the second table in parentheses.

    Let's start with 5.3.2.
    5, 0x28000000, 0x8000000 - Home Menu
    1, 0x30000000, 0x2000000 - root.rpx
    6, 0x33000000, 0x1000000 - Error display
    4, 0x34000000, 0x1C000000 - Background app memory
    7, 0x50000000, 0x80000000 (7, 0x50000000, 0x40000000) - Foreground app memory
    Nothing special here. Everything is the same as in Physical memory page, so we can use names from there to have a better understanding of who is who. The only range that is not in the table is 0x32000000, 0x1000000 - loaer + coreinit.

    Next let's continue with 4.1.0
    4.1.0
    4, 0x28000000, 0x8000000 - Home Menu
    2, 0x30000000, 0x2000000 - root.rpx
    6, 0x32000000, 0x1000000 - loader + coretinit.rpl
    3, 0x33000000, 0x1000000 - Error display
    1, 0x34000000, 0x1C000000 - Background app memory
    7, 0x50000000, 0x80000000 (7, 0x50000000, 0x40000000) - Foreground app memory
    As you can see sizes, addresses and order of areas are the same. But there is a loader+coreinit section here. And it seems like later firmware changed RAMPIDs.

    And finally 4.0.0
    4, 0x28000000, 0x8000000 - Home Menu
    1, 0x30000000, 0x1C000000 - Background app memory
    3, 0x4C000000, 0x1000000 - Error display
    6, 0x4D000000, 0x1000000 - loader + coretinit.rpl
    2, 0x4E000000, 0x2000000 - root.rpx
    7, 0x50000000, 0x80000000 (7, 0x50000000, 0x40000000) - Foreground app memory
    As you can see that even though addresses and order (they seem to be actually ordered by address) were changed, RAMPIDs are the same as for 4.1.0, so we can easily identify the areas.

    Big thanks to everyone who replied to this thread. And special thanks to @dimok for helping me in IRC.
     
  13. fukseliten

    fukseliten GBAtemp Regular

    Member
    134
    33
    Apr 19, 2008
    Norway
    great piece of information.
    Thanks
     
  14. Onion_Knight

    Onion_Knight GBAtemp Advanced Fan

    Member
    878
    832
    Feb 6, 2014

    I ported TCPGecko to the 4.0 FW, look at the wiiudev pygecko github and look at the pull requests. I submitted a pull request with the correct addresses to make tcpgecko work with 4.0.0 firmware. if not, just hit mine up and it has the right installer and codehandler for that firmware.
     
  15. andriy921
    OP

    andriy921 GBAtemp Regular

    Member
    267
    97
    Dec 1, 2015
    By the time I saw your pr, I already ported it myself. It didn't help me though. The only difference between your and mine ports are addresses where codehandler is copied. It seems like that area contains some data. So I used the same as for 3.2.1 (it's full of zeroes on 4.0.x).
     
  16. Onion_Knight

    Onion_Knight GBAtemp Advanced Fan

    Member
    878
    832
    Feb 6, 2014
    Interesting...It runs with no issues on my 4.0.3. Thanks for the port though, I was like you and stumped by the addresses differences in firmware. The kernel mappings are very different below <4.1. Dimok and I could never seem to sync up our schedules to do the port. I'm glad you did it!
     
  17. NWPlayer123

    NWPlayer123 GBAtemp Addict

    Member
    2,629
    6,226
    Feb 17, 2012
    United States
    The Everfree Forest
    Merged it, I'll take your word for it that it works fine. I dug up the mapping table in the kernel binary and I've tried using kern_read and write in multiple apps, it seems like it only really works in the browser and you can't reopen it because TCPGecko likes doing dumb stuff with quick-launch titles like the browser and friends list. Here's the default mappings on 5.5.X, there's this weird 0xFFCE0000 mapping that isn't documented on WiiUBrew, the 0x24 just means 0x24 entries (which is how many lines there are there)
    Warning: Spoilers inside!
     
    paulloeduardo and rw-r-r_0644 like this.