Reversing Compression on Pokemon Mystery Dungeon 2 files

Discussion in 'NDS - ROM Hacking and Translations' started by psy_commando, Jun 3, 2014.

  1. psy_commando

    psy_commando Newbie

    Jun 3, 2014
    Hi ! I've been working on reversing several things from pokemon mystery dungeon 2, explorers of sky/time/darkness.I'm planning to eventually make a sprite replacer for the game, and maybe more.

    I've had some decent success at it, mainly at figuring out a little better how the sprites are stored and etc, thanks to a lot of help from various people from Pokecommunity and Project Pokemon.

    However, while reversing a filetype is fairly trivial, reversing a compressed file is kind of headache inducing.. Mainly because it requires hardware knowledge that I'm lacking..

    And I was wondering if anyone here could give me a good lead as to how to figure out how a file is compressed, and how to decompress it, or even better, be able to decompress it without running it through the game code !

    I've made a good amount of research on the NDS's inner workings, but I can't seem to find anything about people figuring out how to identify and decompress file formats..Especially, when they have an unusual header..
    So far I know that, it could most likely be LZ10, LZ11, LZSS, LZ77, or another LZ variant. And I've read that the NDS can handle decompression via SWI, in the case the dev didn't opt for custom code. But that doesn't help me all that much.

    Here's what one of the compressed file looks like:

    They have this PKDPX magic number, and some have an oddly relatively unmangled SIR0 file magic number.

    I've been attempting to disassemble the compiled binaries from the game, but, I'm not a reverse engineering guru or anything of the sort, and so far, the most conclusive thing I managed to find was a string inside the binary that mentionned the filename of a container file containing many files like the one in the image above.

      94b00 %sfile = '%s'  line = %5d
      94b1c %sProgPos info NULL
      94b34  Print
      94b40 !!!!! Fatal !!!!!
      94b58 (NULL)
      94b64 EFFECT/effect.bin
      94b78 MONSTER/monster.bin
      94b8c BALANCE/m_level.bin
      94ba0 DUNGEON/dungeon.bin
      94bb4 MONSTER/m_attack.bin
      94bcc MONSTER/m_ground.bin
      94be4 file directory init %4d %4d %08x %s
      94d00 0123456789
    The most interesting of the bunch is "MONSTER/monster.bin". Because its that one I was working on reversing right now !

    But, I can't even begin to figure how I'd be able to somehow get a data breakpoint on that string and trace back to whatever decompress the sub-files inside that container...

    Feel free to ask for more details, I might be forgetting some things..

    Alright, so I was able to get a somewhat usable debugging toolchain going. I'm using the debugger from the iDeaS NDS emulator along with Cheat Engine for data watchpoints and memory searching.

    However, I didn't really get a satisfying result yet.

    It also seems the headers for the individual files are modified when they're loaded into memory. They change from SIR0 and offset relative to the beginning of the header, to SIRO and all offsets are converted to offsets relative to the NDS's work ram.