Hacking Technical Info + Rom Manager

[Jack]

Hello!
Long time since my last visit here, many years have passed

First the long story: in the last weeks I recovered from the bag of goodies my old GBA and my Ez Flash just to rediscover the polish of so many titles.

Starting from this I recovered an old project that I had started time ago, a rom manager written in Java (I'm not a Windows user anymore so I needed cross-platform compatibility) that was meant to be an all around manager with multiple kind of dat support and such on.

The approach of the tool will be totally plugin based so that any one able to develop in Java could contribute by writing a specific plugin (for a single or multiple consoles) that does something, let it be organize a romset, or apply a path or download an asset or whatever.

The project is in good shape and shortly it could be used for basic things (as scanning, renaming and organizing by folders) but since I need to enrich the functionality to test the plugin framework (and I need to avoid using the EZ IV Client since I can't) I'd like to implement basic funcionality offered by GBATA tool as plugins inside the manager.

Now the short story comes: I'm practical with technical documents by being a hobby-project made in spare time I have no time to search for them so I'd like to know if you can promptly point me toward specific info about:

• save patching (I found this thread which could be useful: http://gbatemp.net/threads/reverse-engineering-gba-patching.60168) but broader info like if save patching applies in the same way to all carts or each one requires different things, special exceptions and such
• soft/hard reset
• sleep mode
• rom trimming (checks or algorithms needed for GBA and/or potentially other consoles)
• removing intro
• fixing overdumps

These are the basic operations supported by GBATA too which I'd like to implement as stand-alone plugins in my manager but if there is something obvious (excluding IPS patching which will be present for all the consoles) I'm missing please tell me.

The project is totally souce code, under MIT License and hosted on github here: http://github.com/Jakz/rom-manager, feel free to contribute in case!

Some screenshots just to give you a sneak peek:

 

[FAST6191]

For what it is worth some people have managed to slap EZ4client and Wine hard enough that they work together (mostly it involved setting a D drive or modding the program so it did not expect it) but if you are making another project to implement things then few would object.

Save patching should be the same for all flash carts that are not a supercard or supercard clone. The exceptions would be for things like real time clock, solar sensors, tilt sensors and such things, I cover most of them in http://gbatemp.net/threads/buying-a-gba-flash-cart-in-2013.341203/page-18#post-4756995

I have not really investigated sleep and soft reset. For the most part it involved finding a point to hook the ROM, preferably during a vblank, (the EZTeam using their own database, GBATA, BOMA, Kuwanger's stuff and the rest of it probably going more general) and using that to add a button combo check to then call the BIOS function that handles sleep, reset and whatever else.
http://problemkaputt.de/gbatek.htm#bioshaltfunctions has more on that. Kuwanger and dwedit have both done some work in this so look up what they have done, I believe it does call for a tiny bit of finesse -- don't quote me on this but it is similar to in hardware savestates in that if you do it at any arbitrary time it might work but if you give it a little help (by disabling certain registers and setting things into certain states) then you are less likely to have issues.

Soft reset and hard reset are usually similar to the above (indeed just scroll down from the previous link but for the sake of reference http://problemkaputt.de/gbatek.htm#biosresetfunctions ).

Hooking is annoying if you are trying to do a generic one. http://s462158067.onlinehome.us/index.php (author of GBAATM and DSATM) does not have a GBA one but it does have a DS one and if you look at the output of GBAATM for a few things you might also find something.

ROM trimming. Start from end of ROM, is this data something other than 00 or FF? Congratulations you have found the end of the ROM, delete everything afterwards. I have had a few games over the years be troubled by this (I think Teenage Mutant Ninja Turtles is one) as the ROM might legitimately have a section end with 00 or FF and that gets trimmed. To that end if you want to find that location and then add a bit as a safety margin (or even add a bit and change the last bytes to something else in case someone uses the ROM with another less safe trimming program) it will be good.
The DS has a file size indicator in it but the GBA has nothing of the sort.

ROM overdump fixing is even simpler than trimming. It is quite literally a function that deletes everything after the boundary you tell it to (overdumps tend not to trim), so down to 4 megabytes, 8 megabytes or 16 usually.

Intro removing... it is more for the simple intros than the more complex/removal resistant ones. I do not know how much ARM assembly you know but the general idea is the first byte in a GBA game is a jump (typically to the end of the header), if you look at the disassembly there then it will usually be a bunch of IO and then a jump to somewhere in the ROM region (typically 08??????) which is the start of the game. At some point the intro would subvert this, run its own (probably self contained) program and then jump back to the normal ROM. You get to figure out this jump, change it back to what it should be and then blank the intro part. Some earlier intros put the intro right at the end of the ROM which breaks trimming, later ones put it at the start of the trimmable section.
I would say don't bother, though if you are coding a general purpose function for the soft reset, sleep and whatever else then it should be a fairly basic extension of that knowledge. More generally it was useful back in the day but today everybody probably just uses the no-intro/pure/goodtools sets that are as the they are on the original cart rather than scene stuff.

Intro drawing (if it has that function, I might be remembering another tool) is basically the reverse where you delete everything that is not the intro.

Finally ew Java.

 

[Jack]

For what it is worth some people have managed to slap EZ4client and Wine hard enough that they work together (mostly it involved setting a D drive or modding the program so it did not expect it) but if you are making another project to implement things then few would object.

Yes, it sounds like a best effort solution but this is a separated project that was born years ago just because I liked the idea to have a cross-platform tool that works like I want. In addition to this I just used C++/C++11 in last years so I wanted to update my knowledge about Java 8 (I was stuck at 6) and this proved to be a nice occasion. It's not nice for such things like manipulating bytes (in C/C++ it would be rather simpler) but on the other side it's easy to extend the functionality through plugins and it's mostly totally cross-platform with no effort.

The aim of the project is not to limit to GBA rom management but to provide a good extensible framework for any kind of rom management (from consoles to arcade and whatever).

I do not know how much ARM assembly you know

I'm quite practical with ARM assembly and consoles hardware in general, I wrote a GBC emulator (which is not ARM but whatever) just to understand emulation (and no$cash pandocs was my main source indeed) so I'll try to get some deeper knowledge about GBA architecture to implement all the aforementioned functions.

I've read that the problem of sleep mode in EZ IV is not hooking to the bios sleep function but more the fact the EZ IV itself generates random interrupts which wake the CPU from its halt state.

According to gbatek there are multiple sleep bios possibilities, the most efficient one seems to be SWI 03h which is the stop, by possibly disabling flags before invoking it? Which kind of interrupts EZ IV generates? Game Pak? Does this problem applies to other kind of flash carts?

Finally ew Java.

I hope this is a positive feedback, unless you prefer Wine
There are not many choices when you need a crossplatform UI

 

[FAST6191]

If you want more on the GBA architecture and the no$gba specs, cowbite and the official arm manuals are not cutting it for you then I also highly suggest http://www.coranac.com/tonc/text/toc.htm (specifically http://www.coranac.com/tonc/text/asm.htm ), and also have a scan of http://blog.quirk.es/2008/12/things-you-never-wanted-to-know-about.html

I am not sure of the EZ4 specifics when it comes to sleep mode and basically never use it in anything I do.

The Java thing was more that with it being such a security nightmare I have removed it from all my machines and all machines I control that do not need it for a specific task (which is thankfully very few, though annoyingly most of those need a very specific version). I am not a particular fan of the language either but it does what needs to be done.

 

[Jack]

I was curious to see if I could make the tool support goodtools database but it looks like they are embedded in the executables, is that the case? Seems really a selfish move for their maintainers, I really wonder what is the need to embed such data in the executable and make it obscure to people.

 

[FAST6191]

Quite possibly. I tend not to use goodtools and instead http://www.no-intro.org/ or http://www.advanscene.com/ when I need a database of console games.

 

[Jack]

Yes, indeed I already wrote the parsers for Advanscene (which supplies OfflineList style DATs) and I'm working on nointro DATs. But I liked the idea to offer also GoodTools but seems like the author didn't like this possibility

I even tried to disasseble the executables just to see if there was a way to easily extract the database but they look like heavily packed/encrypted with a non-standard tool. Even IDA fails to debug it.

 

[Jack]

I managed to write all the patching routines described by @TrolleyDave in the manager. The only one that is missing is EEPROM v125 which wasn't reversed (maybe I'll do it by hand later but since it is used just almost nowhere I'll just skip it for now).

The EEPROM v111 applied by GBATA is slightly different from the one described by Dave, because the jump back to the instruction after the first patch block that goes in the added payload in the end has an offset of 32 + 1 compared to the start of the patch block. Or at least, by testing GBATA with Tony Hawk Pro Skater 2 this is what it does.

 

[Jack]

I managed to define a nice Workflow approach inside the rom manager, I still need to test everything but in code flavour it's something like:

List<Rom> favourites = set.list.filter(Searcher.buildSeachPredicate("is:fav"));
 
Fetcher<RomHandle> source = new MultipleRomSource(favourites);
Dumper<RomHandle> dumper = new EZFlashIVRomConsolidator();
Workflow<RomHandle> workflow = new Workflow<>(source, dumper);
 
IPSPatchOperation ipsOperation = new IPSPatchOperation();
ipsOperation.addPatch(romSet.list.find("yoshi universal loc:europe"), Paths.get("/Volumes/SSD/gba-ips/yoshi-universal-gravitation-tilt-fix.ips"));
ipsOperation.addPatch(romSet.list.find("wario twisted"), Paths.get("/Volumes/SSD/gba-ips/wario-ware-tilt-fix.ips"));
ipsOperation.addPatch(romSet.list.find("kuru paradise"), Paths.get("/Volumes/SSD/gba-ips/kururin-paradise-translation.ips"));
 
workflow.addStep(ipsOperation);

workflow.addStep(new GBASavePatchOperationGBATA());

RomTrimmerOperation trimOperation = new RomTrimmerOperation();
trimOperation.exclude(romSet.list.find("teenage mutant ninja turtles !2"));
workflow.addStep(trimOperation);

workflow.execute();

This is quite nice because it allows to skip completely EZ Client and produce a working directory with SAVER folder (and dummy save files) and patched games. Unfortunately I'm still studying the GBATA sleep patch algorithms so that part is missing, and save patching haven't been tested for now.

The main problem is how to present this thing in a nice way through the UI assuming that everything should stay enough generic to be used with any kind of romset or system (since everything else is totally generic and plugin based).

Do you think it is possible to find some volunteers for testing even considering the age of GBA?

 

[1__________]

Hey I think you replied to my thread earlier about Kuwanger's files, well I ended up just messaging him directly and he gave me links to the sleephack-v2k.zip and hexalter3.zip. Apparently whoever hosts his site isn't maintaining it anymore or something so he plans on figuring something out, but for now hopefully he won't mind me providing the links he gave me here if they could be of some use:

sleephack-v2k.zip
hexalter3.zip

I probably can't be much help otherwise but I'm down to test stuff out if you need I use my GBA and EZ Flash IV all the time.

 

[Jack]

well I ended up just messaging him directly and he gave me links to the sleephack-v2k.zip and hexalter3.zip.

Thanks! I downloaded it and I'm studying it. Meanwhile I created a standalone commented version and I uploaded it on gist for future reference, so this is not going to be lost again! Here it is, just in case: https://gist.github.com/Jakz/64066ab85f006072ff7e

 

[Jack]

I managed to implement it in my rom manager, I tested it by comparing the output produced when replacing both arm and thumb interrupt registration and the output is equal. I started testing it with Advance Wars but I just got white screen, maybe it doesn't work with that game?

The current status is

• all SRAM patching routines reversed from GBATA are implemented (but untested)
• sleephack v2 is implemented
• trimmer is implemented
• automatic dummy sav creation in SAVER folder is done (but this requires having the AdvanScene GBA dat to know the save size)
• IPS patching is implemented

Which is quite fine.

I'd like to reach a point in which you just set the favourites you like from the romset, press "execute workflow" and a nice folder is already generated with renamed and organized ROMs, already SRAM patched, trimmed, SRAM patched and fixed with IPS when needed. Sounds like an utopia