ROM Hack Identifying AP checks in commercial roms

[kinder_bueno]

I've downloaded the recommended set of compression tools, but none of them is able to decompress arm9.bin I had unpacked with DSLazy - am I doing something wrong? Using hex editor I can see clearly that I have compressed arm9.bin.

 

[FAST6191]

Afraid I don't know the markers of BLZ compression offhand (same sort of flag every fixed bytes type thing?) and while I can't rule out a proprietary thing I would not expect it from Nintendo themselves for this. Equally have you tried compressing it (double compression tends not to do a lot) and also compared it with a memory dump (it will be decompressed in RAM after all)?

 

[kinder_bueno]

I've compared it with memory dump and the memory dump differs. When I extract it using CT2 it seems to work but I've read that people have problems with compressing a decompressed arm9.bin, so I wouldn't like to use it. Double compression made some changes to the file but then decompression still didn't work :/

EDIT:
I've tried to decompress it with CT2 and compress with BLZ and both original and again compressed file are almost the same. The file compressed with BLZ differs only at the end of the file, has 3 additional bytes at the end (maybe some checksum?)

 

[kinder_bueno]

@FAST6191 Ok, I've know - when I export arm9.bin from CT2 or DSLazy (it's the same file in both cases) then I have to remove those additional bytes (I've no idea what they are or even if I should to add them again to my patched, compressed arm9.bin or if the yare some precalculated bytesum?) at the end and then I can use BLZ to decompress it. After my changes recompressed arm9.bin's size has changed a little - do I have to do any changes in ROM file in arm9's metadata or DSLazy's packer should do it for me?

Edit: I've checked with another game, those bytes are also there at the end of compressed, unpacked arm9.bin - so they have to stay there I suppose.

 

[kinder_bueno]

@FAST6191

I have moved forward with my AP project - now I am at the step of applying changes to arm9 and am able to test them. I've observed one strange case: I found a branch bne which always does a jump, so I have changed it to BAL offset. After that change rom crashes during the start, no$gba tells me that it jumped to empty WRAM - it happens even if I will make only that single change. On the other hand, if I apply the same change during runtime in no$gba then the problem never occurs. My theory is that somehow that instruction may be changed but only after specific point in time - the question is why? I have yet to read about thumb mode, but I have only 2 theories - some checksum gets triggered on the very beginning of running the rom or maybe the game runs at the beginning in thumb mode and that fragment of memory is both executed in ARM9 and thumb mode? I would really appreciate your insight on this, I am really lost on this one.

 

[FAST6191]

thumb mode is just a separate instruction set (basically a more limited 16 bit instruction set but one that can still speak to the 32 bit registers, in doing so then you can get some more speed). Trying to decode one's instructions in the other mode will make for some strange behaviours but it has no real bearing otherwise.

This jump. Is it to an overlay area? If there is no overlay loaded.initially then I can see that. I don't know if no$gba senses overlay locations for that kind of error message.

If not it could be that the game stuffs an instruction there during runtime. While I can't say I have heard of commercial DS games doing this it is common in other machines (the GBA even has this), and it would make a fairly nice type of obfuscation as well. If it is a below 8000h read you are trying to change then it might be worth instead redirecting the read instruction to the appropriate area given by the formula in gbatek and not having to worry about things.

I should probably also ponder whether BALL is a suitable replacement instruction - I would probably just use a straight jump with no conditions for most things.

Might have to return to this when I am not falling asleep in my chair.

 

[kinder_bueno]

It doesn't seem to be overlay area - I've checked in overlay table (y9.bin) where the earliest overlay starts - and it is far away than the faulty jump. I am worried that some checksum is triggered maybe on that particular part of memory? Not sure whether this is about below 8000h read, I would have to check it. Regarding B[All] - isn't it an alias to B without condition hence being the mentioned straight jump with no condition?

 

[kinder_bueno]

I noticed that in that memory area (around 0x‭2008724‬ - that's the exact address of that faulty jump) is something completely else in comparison to untouched rom... so changing 1 instruction which now is even never executed in arm9 made entire area of memory different? How can it be? I made nearly 200 other changes in arm9 and rom still works, but for some changes like this one everything breaks in such interesting way - I have no idea why, can it be some checksum which prevents loading rest of arm9.bin file properly?

UPDATE: I've noticed that those different values are in arm9.bin, but in COMPRESSED one. So changing that one instruction prevents arm9.bin to be uncompressed properly - why?

 

[habababa]

noticed that in that memory area (around 0x‭2008724‬ - that's the exact address of that faulty jump) is something completely else in comparison to untouched rom... so changing 1 instruction which now is even never executed in arm9 made entire area of memory different? How can it be? I made nearly 200 other changes in arm9 and rom still works, but for some changes like this one everything breaks in such interesting way - I have no idea why, can it be some checksum which prevents loading rest of arm9.bin file properly?

UPDATE: I've noticed that those different values are in arm9.bin, but in COMPRESSED one. So changing that one instruction prevents arm9.bin to be uncompressed properly - why?

BLZ decompression starts at the end of the arm9.bin and usually ends around 0x02007FFF. If you change the contents of the compressed arm9.bin at an address above 0x02007FFF, the decompressor will output garbage once that address is read due to the way compression wroks.

Use this to recompress the arm9.bin - https://www.romhacking.net/utilities/826/.
OR
If you want the game to load an uncompressed arm9.bin, search for this struct:

typedef struct {
  u32 autoload_list;
  u32 autoload_list_end;
  u32 autoload_start;
  u32 sbss_start;
  u32 sbss_end;
  u32 compressed_static_end;
	union {
	  u32 sdk_version_id;
  	struct {
	   u16 relstep;
  	 //relclass = relstep/10000; relnumber = relstep%10000; relnumber_major = relnumber/100; relnumber_minor = relnumber%100; 
	   u8 minor;
  	 u8 major;
	  };
	};
  u32 sdk_nitrocode_be; // 0xDEC00621 or 21 06 C0 DE
  u32 sdk_nitrocode_le; // 0x2106C0DE or DE C0 06 21
} _start_ModuleParams;

then replace the value in compressed_static_end to 0. this will always be located at an uncompressed area of the binary (below 0x02007FFF in your case).

 

[kinder_bueno]

@habababa Those are steps which I do for every change I make:

1. extract arm9.bin from rom
2. decompress arm9.bin
3. change particular instruction
4. compress arm9.bin
5. pack rom

I always make decompression and then compress it again and I always make changes only on decompressed arm9 (I have automated this process writing some chunk of code, and tested every step and made sure that arm9.bin is in fact decompressed). I can make changes around that faulty job and they still work, only that one isn't. I use ndstool for packing rom, should I modify any headers in case of recompressed arm9.bin changed its size?

I am using the compression tool you've provided.

UPDATE2: I did few more tries andin the end it's not about different arm9.bin size. I am really stuck at this one :/

 

[kinder_bueno]

@FAST6191 Small update - I've disabled redirection of below 8000h reads in desmume and discovered 3 such reads - after that one change the game in fact doesn't react in save menu. The thing is, that I've written some chunk of code which allowed me to identify which instructions prepare those reads, but couldn't find it directly in arm9.bin nor its overlays. It turned out, after small investigation, that one of overlays (one which I can exclude and game runs but it behaves like AP is triggered on untouched desmume) prepares instructions in memory which will be executed in order to insert cartridge address into particular part in memory (those additional registers for cart protocol)... so we have self-modifying code here :/ I don't have any clue how to proceed now. How would you approach this?