ROM Hack Misc GBA ROM parser/memory map documentation

[Swillionaire]

Is there any documentation on the memory layout (i.e. data segments) of ROMs-- more importantly, does any consistent layout exist? I found an old StackOverflow article speaking on GBC games, and it doesn't look like there's any clean division of data there. I know gen 3 Pokemon specifically is pretty well documented, but I'm looking for anything more generally applicable. Thought a fun project would be to write a simple parser, something a little more sophisticated then searching hex for unique strings.

 

[DanTheManMS]

I imagine that outside of game engines shared between games released by the same company, there isn't going to be much consistency when it comes to how the compiled ROMs are laid out.

Closest thing I can think of would be the "Sappy" audio engine used by a larger-than-normal chunk of GBA games. But that's not much.

 

[FAST6191]

Yeah the GBA was the last of the incbin approaches where everything was bundled into a giant blob. Homebrew had a file system or two but nothing really like it has ever been found in a commercial GBA game that I am aware of.

You can find the binary easily enough (first byte is an instruction, usually to jump to the end of the header, then some IO and the next jump to somewhere in the 08?????? region is the binary).
Sappy audio has enough tells that even unknown to the tools games can usually find it easily.
GBA pointers are fairly basic for the most part (three regions/mirrors where the whole cart is visible but most only use the 0800000 through 09FFFFFF (and 09000000 through 09FFFFFF corresponds to the last 16 megs of the game, which any hack can use trivially unlike many older systems where that is a bigger ask, but the vast majority of games are less than). Consequently search for 08 in the game and if you find a sea of them a mostly fixed distance* apart you have some pointers.
BIOS compression. The BIOS features compression, which is used quite a bit (though sometimes it is code in the game instead of the BIOS). The calls to the BIOS which contain the location, type of compression and length (rather useful when hunting down things you care about) can be logged and subsequently read out, and possibly expanded upon (most games follow a pattern -- learn the position of the graphics for the first rusty dagger then iron then steel and your mithril end game affair is probably in a similar location with a similar construction). Compression itself also has enough tells that
Beyond this you have the standard ROM hacking tools of relative search, corruption, brute force (much has been found by opening tile editor, set to GBA4bpp mode and press down a lot, repeat for 8bpp), reverse searching (find graphics/palette/stats in VRAM or general RAM, snatch and search ROM for it). https://gbatemp.net/threads/gbatemp-rom-hacking-documentation-project-new-2016-edition-out.73394/ https://www.romhacking.net/start/

*so you would expect generally 4 byte pointers (or 3 bytes of random between 08) but some games might have pointer - formatting, pointer - formatting ... not to mention 08080808 is a valid pointer, indeed while I said random in the aside there then various things (like the compression noted above) want to be aligned to certain positions rather than completely arbitrary so actually more likely than some others.

 

[Swillionaire]

Gotcha, thanks for the pointers folks. Are there any emulators that have live memory monitoring, so you can see what's loaded into register and such? I'll take some time next weekend and see if I can backtrace some addresses and try to find where some sprites live, for instance.

 

[FAST6191]

GBA debuggers have been around for years and years now, though you will tend to find three in common rotation.

Those around for the longest might still be using VBA-SDL-H
https://www.romhacking.net/documents/361/ (that being a guide to tracing graphics) https://web.archive.org/web/20130215155212/http://labmaster.bios.net.nz/vba-sdl-h/
It sort of is command line based (or at least has its own internal command line equivalent) but has abilities that some others might not have/are normally only seen on PC and high end NES stuff like https://fceux.com/web/help/CodeDataLogger.html (fceux being the gold standard for console debugging, that which everything else is measured against, PC being the only thing that beats it, indeed PC with stuff like IDA blows it out of the water).

no$gba has long featured a top rated debugger and a few years back became free (was paid before, mainly as several devs of GBA and DS games paid for it).
http://problemkaputt.de/gba.htm
http://problemkaputt.de/gbatek.htm is probably also the main reference for GBA hardware, though https://www.cs.rit.edu/~tjh8300/CowBite/CowBiteSpec.htm#Graphics Hardware Overview http://www.coranac.com/tonc/text/asm.htm also have good stuff and there is much more (most of which I linked in my hacking guide).

mgba https://mgba.io/ is newer on the scene but the author of it does some really good stuff
https://mgba.io/tag/debugging/
Its debugger is like many modern emulators and farms it out to GDB (gnu debugger, people making gnu gcc, aka the main non Microsoft C compiler) which can then be spoken to by all manner of things
https://wrongbaud.github.io/posts/ghidra-debugger/

Ghidra https://ghidra-sre.org/ if you are not familiar is one of the three main open source answers these days to IDA https://hex-rays.com/ida-pro/ida-debugger/ which you might see some also use for this sort of thing (watch any hacking conference and if it needs a debugger chances are that will be the one used. I see it less for the GBA than I do DS, Wii and whatever other consoles people are looking at though). https://rada.re/n/ and https://x64dbg.com/ (which is primarily x86/x64 focused anyway, as will be the IDA free version).

There are a few more vintage emulators that can be used to do things but none of those have the feature set needed for this/comparable to the above (can be useful for certain types of hack though so some might still use them). VBAM https://vba-m.com/ (a largely successful project to merge all the VBA forks that existed before and after the end of vanilla VBA, which would have included at least elements of VBA-SDL and VBA-SDL-h ) does have some GDB stub options as well last I checked but the others are probably better choices. I don't know what the tool assisted speedrun peeps are using right now but they sometimes make their own debuggers (it is kind of necessary to get anywhere real in such fields), though I normally save searching that when I am not hacking a system with a nice debugger.
There are also static options, and things you can do with simpler emulators (my primary method of finding palettes is to dump them from where they are at in game, possibly with some cheats to force a scenario faster, and search the ROM for it -- they are usually effectively random and relatively speaking very small so rarely compressed and that then means palette animations* which mostly mean search for the non animated fragments and dynamic/selectable palettes like some things had to account for GBA, GBASP and GB player screen options being the main pitfalls of that approach). Live memory viewing and register info (as well as live disassembly) are available in fairly stock versions of VBA and thus most forks as well but breakpoints for tracing a rarer concept save for their GDB stubs.

*Mr Driller 2 rainbow blocks and summon night swordcraft story saving swords glow effect being some nice examples.

 

[metroid maniac]

How do retail GBA games usually manage RAM? Is EWRAM mainly allocated with malloc/free type functions, like under devkitARM?

 

[FAST6191]

I don't know that I have particularly ever read an analysis of such things, nor would most hackers particularly go there either (if the code is shoved in on chip RAM or run from the cartridge you tend to play to it as far as space available and timings, to say nothing of most attaching things to existing functions rather than allocating whole new things of their own).

http://pineight.com/gba/managing-sprite-vram.txt is for VRAM management in general, though a more considered approach than the naive ones.
https://adshomebrewersdiary.blogspot.com/2012/02/unequivocal-answer.html is not management as much as speed but have it anyway.
The GBA SDK is available, indeed it predated the GBA which is why emulation (and later debuggers) are so long standing -- they were a bit clunky but outside of saving issues you could probably do serious damage with emulators released before the NA release of the GBA. Unlike the xbox scene though most homebrew did not necessarily use the official SDK* though so not all that much there (though such things would be referred to as the AGB toolchains if you go looking on gbadev or similar such).

*it would be a while before devkitarm/devkitpro appeared on the scene and the GBA homebrew scene (and commercial from what we have seen) still benefited from extensive use of assembly (inline and straight up) for various purposes (which itself is split between a few things, but I think even mentioning the fun and games associated with the armSDT these days is going to get me nasty looks). visualHAM https://web.archive.org/web/20080310040402/http://www.ngine.de/ probably being the almost forgotten thing today in that aspect.

Various libraries were released or discussed, though this was usually homebrew adjacent stuff for audio and the like that got twisted to commercial purposes.
https://knzl.at/krawall/ https://gbatemp.net/threads/video-codec-for-gba.354591/#post-5038482 for the curious.

Don't think any of the unreleased games that saw devs release the efforts themselves ever had source go with it. https://www.unseen64.net/category/nin/game-boy-advance/ for a jumping off point, I have an older post around here somewhere with a tiny bit more https://gbatemp.net/threads/compreh...-leaked-unreleased-games.342235/#post-4539813

I don't know what would have been in the gigaleaks for GBA stuff but would probably be my first port of call unless unseen64 turns something up that I was unaware of.

Some of the pokemon disassembly projects have landed on the GBA https://archives.glitchcity.info/wiki/List_of_Pokémon_disassembly_projects.html and are reasonably advanced. That said at that point you are relating disassembled code back to any number of compilers back in the day for ARM (started off with some ARM in house stuff and I think eventually wound up for code warrior, or at least it was an established relationship by the time the DS SDK which did use that came on the scene, and various things in between along with devs doing their own thing).

Don't think I have seen any decompilation projects for the GBA be anything like mature yet, though I expect to see some things in the not massively distant future (ARM processors and their compilers not being so exotic as to not get attention).