What is: rom trimming?

Discussion in '3DS - Flashcards & Custom Firmwares' started by Tattorack, Apr 22, 2014.

  1. Tattorack
    OP

    Tattorack Interstellar Explorer

    Member
    217
    39
    Jun 15, 2011
    Denmark
    Hyperspace
    I've been googling this for a bit and I haven't found a satisfying or thurough explanation.
    What is trimming a rom?
    Sure, rom goes in one end, and gets out slightly smaller at the other, but what gets trimmed? How does it work? What is the process? Is it litteraly trimming or is it a special way of compression? Not that I don't trust it, I am just curious.
    Thanks in advance to anyone willing to answer! ^^
     


  2. Loke

    Loke GBAtemp Regular

    Member
    188
    32
    Jan 7, 2014
    Netherlands
    It's just removing the filler data with 3ds explorer.
     
  3. porkiewpyne

    porkiewpyne Report-er

    Global Moderator
    2,064
    1,222
    Jun 8, 2008
    Australia
    Long story short, roms are dumped such that the size is equivalent to the total size of the media, even if the actual game files do not take up all that space. Say, the cartridge is 4GB while the game itself only takes up 3.7GB. The remaining 0.3GB will be filled with rubbish data so to speak. When dumped, the file will contain every bit of the 4GB. What trimming does is take away that 0.3GB.

    That's essentially trimming in a nutshell.
     
    setsuya and Fishaman P like this.
  4. Celice

    Celice GBAtemp Advanced Maniac

    Member
    1,916
    354
    Jan 1, 2008
    United States
    So how does one figure out the padding space? Is it always the end-of-ROM that gets trimmed, or is there some generic header to the file that specifies dead ranges of padding? I've come across several games that have large swathes of unused space in between other valid data, for instance, on the GBA especially.
     
  5. bkifft

    bkifft avowed Cuthwaldian

    Member
    598
    266
    Jun 10, 2010
    Gambia, The
    Headers. See https://github.com/3DSGuy/rom_tool/tree/master/rom_tool.
     
  6. FAST6191

    FAST6191 Techromancer

    pip Reporter
    23,522
    9,360
    Nov 21, 2005
    For the 3ds I am not sure of the specifics used but as you mentioned other systems it varies somewhat.

    For the GBA there is no size value in the header. This makes GBA trimming run roughly "start at end of file, find the first byte that is no 00 or FF, delete from that point on". I have seen it cause issues before as some games might use a 00 or FF to end a segment which also ends a ROM. Likewise I have seen games go all the way to the boundary but have a gap at the end (some of the 32 meg titles, Riviera and Bleach if memory serves), I was later told deleting it caused issues but unlike deleting the end of most games it appeared to run. Earlier scene intros/trainers went right to the end of the file and stuck it there, this prevented trimming. The other exception would be overdumps which could occasionally go over size but not be 00/FF padded, I am not sure if there were ever any "the ROM chip is 64 mbit but the game actually uses less than 16" type arrangements but I would not be shocked to hear it.
    Edit. Theoretically you could shrink a ROM if there was some extra space before the last part of useful data in a game, it would be a nightmare though even under the best conditions and fairly pointless to boot.

    The DS started out using the GBA style methods. however the DS has a size value of the useful data in the header (no save though, unlike the GBA) which is usually accurate, indeed it is mostly SRL debug stuff that I see which lacks a proper value. All that said trimming to that value seems to sometimes break the download play stuff, no idea why so "safe trimming" appeared where DS roms were trimmed to the value plus a given amount.
    Several games have been seen to have masses of padding between files as well, nobody really ever trimmed that (by the time ROM ripping, as in asset removal to save size, got up to speed we all had gone external memory). I recall the US version of Tetris DS has a fair bit of useless padding at the end of its sound file, the EU version did not (or it might have been the other way around) as I found out when thinking about porting the "always play classic tetris song" patch I made years back.

    Iso stuff is where you might see padding shrunk between files. Most notable around here would probably be the Wii scrubber which had an extra scrubbing mode that did remove the space; there were several modes of scrubbing, some changed the iso in a manner that could not be "properly" fixed again, some changed it in a way that some theoretical AP might have been able to detect but was otherwise fine (this was the original one) and some changed it so said theoretical AP method would have to be improved considerably to test it.

    Other consoles vary. There are not always size and save values and beyond that with the likes of bankswitching, mappers, extra chips and beyond it can see the size vary a bit. This is why certain NES/SNES stuff could be something other than a power of 2 value for size (though it probably would be a basic addition of a few of them), also there are dumper added headers which are often times useful to explain what it to come. Sometimes these dumper headers are useless and sometimes they are vital unless you want to do a database of all games and fingerprint every one, always be wary of someone that is overly enthusiastic about header removal.
     
  7. Tattorack
    OP

    Tattorack Interstellar Explorer

    Member
    217
    39
    Jun 15, 2011
    Denmark
    Hyperspace
    Thanks for the info guys ^^
    That settled the trimming curiosity, but now I have another question; Whats the point of the extra useless data? If something is 6.4GB it size why should one add extra useless data in order to make it exactly 8GB?
    I've noticed (at least in NDS games) that everything follows a srict 2x size; 2, 4, 8, 16, 32, 64, 128, 256 and the largest I've seen 512.
    Games apparently work without having to be the exact size, so what is it then? Marketing reasons?
     
  8. Ris312

    Ris312 Advanced Member

    Newcomer
    59
    23
    Mar 5, 2011
    The game chips only come in certain sizes, 128mb, 256mb, 512mb, 1gb, 2gb and 4gb so far.
    (The chips are always sized to a power of two due to computers being binary 0/1,
    search google for something like "Why is memory capacity always a power of 2" for more info)

    Since you can't fit a 700mb game onto a 512mb chip, it gets put on a 1gb chip instead
    The other 300 odd megabytes don't ever get written to, so its just blank data (FF's)
    When the game gets dumped, the rom dumper just reads the entire 1gb chip from start to finish, which results in a 1gb rom file.
    The rom-trimmer program reads the rom header and finds the used data length and cuts out the blank data resulting in a smaller rom.

    (offset 0x300 in the Rom dump stores the 'Used Rom length' in bytes, the rom trimmer uses that info to determine the game size)
     
    bkifft and porkiewpyne like this.
  9. IronClouds

    IronClouds GBAtemp's Pokébro

    Member
    1,019
    460
    May 8, 2010
    United States

    So what you're saying is that the ROM dumper doesn't have the capability of reading chips partially?
     
  10. gamesquest1

    gamesquest1 Nabnut

    Member
    14,135
    9,478
    Sep 23, 2013
    Think it's more that they have no reason to start making partial dumps, people usually want a 100% clean/full dump and then it's up to the end user if they trim or not, especially given that trimming is not a guaranteed art and can/does break compatibility/features on certain games

    Basically it's a complication to the process that usually only saves 100mb or so
     
  11. zhdarkstar

    zhdarkstar GBAtemp Advanced Fan

    Member
    571
    225
    Jan 30, 2008
    United States
    If given the right parameters, it could read the chip partially. However, the cartridge dump scene is slightly different from the iso dump scene, as cartridge dumpers only view the dumps that still have the dummy data intact as being "clean." Dating all the way back to the beginning of the emulation days, cartridge rom dumps have been "whole chip" dumps. However, it's only in the last two generations (three, to a lesser extent, if you count GBA) of Nintendo handhelds that we have seen a significant enough size difference between the size of the dump and the size of the actual data to justify the development and safe use of trimmers. Shaving 1-2 MB off a GBA rom isn't as economical as shaving anywhere from tens to hundreds of MB. Sometimes you get even bigger savings, as in the case of Donkey Kong Country Returns: Tropical Breeze, which has almost 1.8 GB of dummy data on its 4 GB chip.
     
    Elliander likes this.
  12. IronClouds

    IronClouds GBAtemp's Pokébro

    Member
    1,019
    460
    May 8, 2010
    United States
    Well I already knew the purpose of clean dumps and all that jazz. My question was geared more toward retail carts during manufacturing, etc. The chips within retail carts are a set size, so when the game data is added to retail carts in production, does that machinery not have the capability of only partially writing to the chip so that "filler data" isn't necessary? Or does this whole process only apply to dumping games for pirating purposes?

    Thanks for the replies so far, though.
     
  13. zhdarkstar

    zhdarkstar GBAtemp Advanced Fan

    Member
    571
    225
    Jan 30, 2008
    United States
    I'm not 100% sure, but I think that the production machinery only writes the needed game data, as most storage medium have all of their empty bits set to 00 or FF by default when formatted.

    The only real difference between an occupied bit and an empty bit is that the memory allocation table says that one can be overwritten while the other cannot.
     
    Abcdfv likes this.
  14. gamesquest1

    gamesquest1 Nabnut

    Member
    14,135
    9,478
    Sep 23, 2013
    I'm not 100% on the details but I dont think they write anything beyond the game data it's just left blank, when dumping the chip the blank data is still dumpable but it's just blank data, assuming it's possible for Nintendo to program the specific size and just leave the rest blank the surely would, yeah 5 seconds extra programming first doesn't sound like much but over the millions of carts produced I'm sure they would rather save that time :)
     
    IronClouds likes this.
  15. FAST6191

    FAST6191 Techromancer

    pip Reporter
    23,522
    9,360
    Nov 21, 2005
    Come back and tell me that after you have used a 16 or 32 meg GBA flash cart that is annoying and slow to write as your primary cart (and probably paid about $100 USD for it). Over about 4 games it soon adds up to space enough for another game. or at least an emulator collection of a couple of games.

    On the "being clean" part sure pretrimmed stuff never really flew, however there was not necessarily a great push to have scene releases be exact copies -- nobody got a release nuked just because it had an intro. I know you did not say that but it risked being misinterpreted.
     
    cearp likes this.
  16. zhdarkstar

    zhdarkstar GBAtemp Advanced Fan

    Member
    571
    225
    Jan 30, 2008
    United States
    I didn't enter the flashcart scene until a generation later with the slot-2 M3 for DS, but I went through the same struggle back when SDHC wasn't a thing and a 2 GB SD card was $100+. So yes, it was very economical for its time. However, we are two handheld generations removed from the GBA and large external storage is extremely cheaper than it was 10 years ago. My statement was written from a 2014 perspective, not 2004. Sorry if it got you upset.
     
    cearp likes this.
  17. FAST6191

    FAST6191 Techromancer

    pip Reporter
    23,522
    9,360
    Nov 21, 2005
    I was not upset at all, I was just old manning it up.

    That said even in 2014 the standard GBA flash cart is probably the EZ4 and if you can squeeze a few more games onto your NOR section then trimming is still worth it.
     
  18. gamesquest1

    gamesquest1 Nabnut

    Member
    14,135
    9,478
    Sep 23, 2013
    Saving space is always good, but if trimming has adverse effects on the game I.e download play not working etc, surely it's better to leave the trimming down to the end user rather than having to make people untrim when things go wrong XD

    Not sure about the GBA stuff, but trimming 3ds roms causes issues and that causes HELP ME PL0X threads and needing to explain what and why trimming is good/bad useful/problematic, until trimming works 100% I think it's better that things are left as they where :P it stops the same thread popping up and users from having issues with certain games etc
     
  19. zhdarkstar

    zhdarkstar GBAtemp Advanced Fan

    Member
    571
    225
    Jan 30, 2008
    United States
    I hear ya, but I just wish that the replies to those threads weren't always the automatic "the problem is you trimmed the rom and nothing more" as 100% compatibility with flashcarts isn't guaranteed every time a new rom comes out. I posted a "HELP ME PL0X" thread myself just a few days ago for Conception II and was only met with the "blame the trimming" mentality. Frankly, it pisses me off when everyone wants to harp on one detail as the end-all-be-all source of all rom problems when another rom released around the same time had an issue that was found to be present regardless of the rom being trimmed. Sometimes the problems lie with the loader or the rom itself. We need to stop demonizing trimming as it's not the only offender in problems people have with loading/running games.
     
  20. Tattorack
    OP

    Tattorack Interstellar Explorer

    Member
    217
    39
    Jun 15, 2011
    Denmark
    Hyperspace
    Yeah, I understand why the sizes are 2x multiples ^^
    Thanks, now I also understand where that trimmable "extra" stuff comes from; apparently data on a game-card storage works a quite differantly than data stored on a flash drive (more differant than I had expected). So thanks for clearing that up ^^

    Kinda silly people blame everything on trimming. Granted, from what I read trimming in general needs a bit of refinement or just a "steady hand", but blaming everything on trimming when it comes to rom problems would be like blaming a volcano with an evil spirit inside for all life's misfortunes (ok, perhaps a little too overdone here XD ).