Help understanding save file checksums (and encryption?) on raw saves exported with savedatafiler

Discussion in '3DS - ROM Hacking, Translations and Utilities' started by havaya, Jul 15, 2015.

  1. havaya
    OP

    havaya Newbie

    Newcomer
    8
    2
    Jul 15, 2015
    United States
    It's possible to export raw saves with savedatafiler then find and edit values in a hex editor, like cheat codes. However, some games have checksum bytes within the save file that need to be correct or that save file gets read as corrupted by the game. A while ago I tried exporting two Lord of Magna saves (for this game, each save has three .sav files in the 8-digit ID folder) and searching for the money values. I changed that value in one of the files, then when I imported the whole save the edited file was corrupted and the game deleted it (the other two files were still there).

    I'm not sure what checksum methods are used in different games, so I'm posting some notes. It looks like each game uses its own checksums and possibly encryption in the exported saves, so if anyone has edited saves successfully it helps to share what kind of checksum is used and where it is. There are also several games with save editor programs, so reading about how those work could also help.

    I read about one case here:
    https://gbatemp.net/threads/ocarina-of-time-3d-save-editing-progress.380927/
    This is for The Legend of Zelda: Ocarina of Time 3D, which uses the last two bytes of the save with a CRC-16 checksum. It computes a CRC-16 checksum with those bytes set to zero (this is important, deleting them instead of setting to 0 gives a different checksum value). Then, it checks if those two bytes are equal to that checksum. This particular save is stored as Little Endian, which means every four bytes are reversed, so when you edit the checksum bytes (or any value in the save), you reverse the four-byte groups so it will read correctly.

    Lord of Magna appeared to be doing something similar, though I couldn't find the checksum method to use even when testing with several variations of byte flips. Maybe I made a silly error in my brute force code somewhere, since that looked simple to find if the method was the same (compare two saves, list changed byte groups that could be the checksum, set groups of bytes to 0 and see if the checksum matches the bytes or possibly the flipped zeroed byte groups. That's assuming a lot about the checksum method, though.).
    I think it helps to clarify another assumption, which is that only the saves in the 8-digit ID folder matter. When I export with savedatafiler, there's a datetime folder with the 8-digit ID folder with the saves, and also two .dat files and a .log file with the ID folder. The .dat files appear to be used for injecting the save back into the particular game ROM it was exported from. I don't think they matter for the save file since I could use the same save (files inside the 8-digit ID folder) on two different cartridges by exporting from the cartridge, replacing just those files, then importing.

    The exported save files for different saves appear to have their own formats. I tried looking at some saves from another game, Conception II, and the saves are very dissimilar (just a header in common) and sometimes even different sizes, so I guess that there's some kind of encryption or compression being used with them. I think that makes sense for Conception II since it's a port of a Vita game, and Vita saves are heavily encrypted. Maybe the encryption got carried over in the 3DS port.

    I tried another game, Etrian Odyssey Untold: The Millennium Girl, which has a raw save with no checksum or encryption, as I learned after the following test. I save in the Inn with 100 gold and export that, buy 2 potions (60 gold now) and export that, then compare the two saves in a hex editor. I find the address easily, so I back up a save and change the address around the gold from 00 3C 00 00 00 to 14 28 3C 00 00 (this makes it easy to tell if the value is flipped). I copy this edited file over the one in one of the exported saves, then import that back into the game. The save loads properly and shows 15400 gold, which is 3C28 in hex, so that means it's Little Endian here too and also I'd better go back and change that save because the 14 might have ruined something else. 999,999,999 base 10 = 3B9AC9FF base 16 is plenty, so I changed the 4-byte group with the gold byte to FF C9 9A 3B (flipped bytes for Little Endian format) and reimported the save, which showed 999,999,999 gold as desired. Pretty neat. If there was a checksum, the save would have said some corruption message instead of loading.
    [​IMG]

    So it looks like the individual games have their own exported save formats, including possible checksums and encryption/compression.
     

    Attached Files:

  2. SciresM

    SciresM GBAtemp Advanced Fan

    Member
    597
    1,867
    Mar 21, 2014
    United States
    Of course all games have their own save file formats and checksums...it's up to the game developer to implement those, if they choose to (and every game has to have a different save file format because games store different data).
     
  3. havaya
    OP

    havaya Newbie

    Newcomer
    8
    2
    Jul 15, 2015
    United States
    Last edited by havaya, Jul 16, 2015
  4. WulfyStylez

    WulfyStylez SALT/Bemani Princess

    Member
    1,149
    2,609
    Nov 3, 2013
    United States
    If brute-forcing checksums doesn't give you a result very quickly, you should generally move on to looking at a game's code. Find something unique to the save you're working with, and work backwards from there. For example, the size of a CRC'd region (between two checksums) is a static value which will return quite a few hits in an IDA search. Other searches could include save magic, or the save filename being opened.

    When dealing with saves from savedatafiler (i.e. not wading through IFVC tables and AES-MACs), anything could happen. It's not 3DS-specific at that point. I've seen both saves with no checks at all, and encrypted ECDSA signed saves. The best way to break save security is seeing how the game does it.
     
    RainThunder and havaya like this.
  5. evandixon

    evandixon PMD Researcher

    Member
    1,690
    812
    May 29, 2009
    United States
    Pokémon Mystery Dungeon: Gates to Infinity has an 8-bit sum of all the 1 bits in the file at the very end. Game_System and Gate_Header are pretty straightforward (besides finding out the bit shifting because the files are dynamically sized), but the Game_Data file is more complicated because there's something else added on top of it that's inconsistent between saves of differing sizes.

    I'm curious about whether or not other games use a similar checksum.
     
  6. havaya
    OP

    havaya Newbie

    Newcomer
    8
    2
    Jul 15, 2015
    United States
    WulfyStylez explained cracking the checksums well. I thought about it and there's so much to brute force (checksum algorithm, custom polynomial, which areas or blocks are used for the checksum, possible multiple checksums for partial blocks) that you would probably need to know something about the save format and game's code in order to brute force effectively. Like, on my first try I tried some code to zero each 2 byte chunk, calculate a checksum with several algorithms, and see if the calculated checksum (or its byte-flipped counterpart, in case of Little Endian format) matched the bytes that were zeroed. That's maybe how Ocarina of Time 3D does the checksum, but other games do it differently so I was wasting my time by doing that code without knowing more about how the game does its checks. If I knew everything but one or two things, it would be effective to brute force on one or two dimensions to learn those details, but on all of them? forget it, probably not going to work without knowing all the common algorithms and variables, and could take a really long time even then.

    I found a way to edit stuff without needing to know the save checksum/encryption: use the browser and scan a QR to load a code. Studying the save files is only for my curiosity =P
    http://gbatemp.net/threads/spider-arcode.383937/
    http://gbatemp.net/threads/spider-database-ar3ds-arcode-database-for-3ds.388252/

    I also found a (possibly dangerous, use at your own risk etc) shortcut while I was playing around with importing/exporting saves. If you're on Sysnand (ninjhax into pasta, or browser into RxTools devmode), you can power off the system and eject the SD card at the power off screen and it won't complain when you reinsert the SD card and hit Home. I couldn't find any way to edit stuff on the SD card in Emunand without turning off and rebooting, which takes about a minute, making this ejection trick the fastest way I know to hex edit the saves then get them back into the game. Emunand always complained no matter where I removed the SD card (even removing just the micro SD card made it crash after I hit Home).