Hacking What is: rom trimming?

Tattorack

Interstellar Explorer
OP
Member
Joined
Jun 15, 2011
Messages
267
Trophies
1
Age
30
Location
Calisto Prime Orbital
Website
tattorack.deviantart.com
XP
428
Country
Denmark
I've been googling this for a bit and I haven't found a satisfying or thurough explanation.
What is trimming a rom?
Sure, rom goes in one end, and gets out slightly smaller at the other, but what gets trimmed? How does it work? What is the process? Is it litteraly trimming or is it a special way of compression? Not that I don't trust it, I am just curious.
Thanks in advance to anyone willing to answer! ^^
 

porkiewpyne

Report-er
Supervisor
Joined
Jun 8, 2008
Messages
2,496
Trophies
2
XP
4,546
Country
Australia
Long story short, roms are dumped such that the size is equivalent to the total size of the media, even if the actual game files do not take up all that space. Say, the cartridge is 4GB while the game itself only takes up 3.7GB. The remaining 0.3GB will be filled with rubbish data so to speak. When dumped, the file will contain every bit of the 4GB. What trimming does is take away that 0.3GB.

That's essentially trimming in a nutshell.
 

Celice

Well-Known Member
Member
Joined
Jan 1, 2008
Messages
1,920
Trophies
1
XP
628
Country
United States
Long story short, roms are dumped such that the size is equivalent to the total size of the media, even if the actual game files do not take up all that space. Say, the cartridge is 4GB while the game itself only takes up 3.7GB. The remaining 0.3GB will be filled with rubbish data so to speak. When dumped, the file will contain every bit of the 4GB. What trimming does is take away that 0.3GB.

That's essentially trimming in a nutshell.
So how does one figure out the padding space? Is it always the end-of-ROM that gets trimmed, or is there some generic header to the file that specifies dead ranges of padding? I've come across several games that have large swathes of unused space in between other valid data, for instance, on the GBA especially.
 

bkifft

avowed Cuthwaldian
Member
Joined
Jun 10, 2010
Messages
613
Trophies
0
XP
625
Country
Gambia, The

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,285
Country
United Kingdom
For the 3ds I am not sure of the specifics used but as you mentioned other systems it varies somewhat.

For the GBA there is no size value in the header. This makes GBA trimming run roughly "start at end of file, find the first byte that is no 00 or FF, delete from that point on". I have seen it cause issues before as some games might use a 00 or FF to end a segment which also ends a ROM. Likewise I have seen games go all the way to the boundary but have a gap at the end (some of the 32 meg titles, Riviera and Bleach if memory serves), I was later told deleting it caused issues but unlike deleting the end of most games it appeared to run. Earlier scene intros/trainers went right to the end of the file and stuck it there, this prevented trimming. The other exception would be overdumps which could occasionally go over size but not be 00/FF padded, I am not sure if there were ever any "the ROM chip is 64 mbit but the game actually uses less than 16" type arrangements but I would not be shocked to hear it.
Edit. Theoretically you could shrink a ROM if there was some extra space before the last part of useful data in a game, it would be a nightmare though even under the best conditions and fairly pointless to boot.

The DS started out using the GBA style methods. however the DS has a size value of the useful data in the header (no save though, unlike the GBA) which is usually accurate, indeed it is mostly SRL debug stuff that I see which lacks a proper value. All that said trimming to that value seems to sometimes break the download play stuff, no idea why so "safe trimming" appeared where DS roms were trimmed to the value plus a given amount.
Several games have been seen to have masses of padding between files as well, nobody really ever trimmed that (by the time ROM ripping, as in asset removal to save size, got up to speed we all had gone external memory). I recall the US version of Tetris DS has a fair bit of useless padding at the end of its sound file, the EU version did not (or it might have been the other way around) as I found out when thinking about porting the "always play classic tetris song" patch I made years back.

Iso stuff is where you might see padding shrunk between files. Most notable around here would probably be the Wii scrubber which had an extra scrubbing mode that did remove the space; there were several modes of scrubbing, some changed the iso in a manner that could not be "properly" fixed again, some changed it in a way that some theoretical AP might have been able to detect but was otherwise fine (this was the original one) and some changed it so said theoretical AP method would have to be improved considerably to test it.

Other consoles vary. There are not always size and save values and beyond that with the likes of bankswitching, mappers, extra chips and beyond it can see the size vary a bit. This is why certain NES/SNES stuff could be something other than a power of 2 value for size (though it probably would be a basic addition of a few of them), also there are dumper added headers which are often times useful to explain what it to come. Sometimes these dumper headers are useless and sometimes they are vital unless you want to do a database of all games and fingerprint every one, always be wary of someone that is overly enthusiastic about header removal.
 

Tattorack

Interstellar Explorer
OP
Member
Joined
Jun 15, 2011
Messages
267
Trophies
1
Age
30
Location
Calisto Prime Orbital
Website
tattorack.deviantart.com
XP
428
Country
Denmark
Thanks for the info guys ^^
That settled the trimming curiosity, but now I have another question; Whats the point of the extra useless data? If something is 6.4GB it size why should one add extra useless data in order to make it exactly 8GB?
I've noticed (at least in NDS games) that everything follows a srict 2x size; 2, 4, 8, 16, 32, 64, 128, 256 and the largest I've seen 512.
Games apparently work without having to be the exact size, so what is it then? Marketing reasons?
 

Ris312

Well-Known Member
Newcomer
Joined
Mar 5, 2011
Messages
59
Trophies
0
XP
92
Country
The game chips only come in certain sizes, 128mb, 256mb, 512mb, 1gb, 2gb and 4gb so far.
(The chips are always sized to a power of two due to computers being binary 0/1,
search google for something like "Why is memory capacity always a power of 2" for more info)

Since you can't fit a 700mb game onto a 512mb chip, it gets put on a 1gb chip instead
The other 300 odd megabytes don't ever get written to, so its just blank data (FF's)
When the game gets dumped, the rom dumper just reads the entire 1gb chip from start to finish, which results in a 1gb rom file.
The rom-trimmer program reads the rom header and finds the used data length and cuts out the blank data resulting in a smaller rom.

(offset 0x300 in the Rom dump stores the 'Used Rom length' in bytes, the rom trimmer uses that info to determine the game size)
 

IronClouds

GBAtemp's Pokébro
Member
Joined
May 8, 2010
Messages
1,019
Trophies
0
XP
625
Country
United States
When the game gets dumped, the rom dumper just reads the entire 1gb chip from start to finish, which results in a 1gb rom file.


So what you're saying is that the ROM dumper doesn't have the capability of reading chips partially?
 

gamesquest1

Nabnut
Former Staff
Joined
Sep 23, 2013
Messages
15,153
Trophies
2
XP
12,247
So what you're saying is that the ROM dumper doesn't have the capability of reading chips partially?
Think it's more that they have no reason to start making partial dumps, people usually want a 100% clean/full dump and then it's up to the end user if they trim or not, especially given that trimming is not a guaranteed art and can/does break compatibility/features on certain games

Basically it's a complication to the process that usually only saves 100mb or so
 

zhdarkstar

Well-Known Member
Member
Joined
Jan 30, 2008
Messages
573
Trophies
1
XP
566
Country
United States
So what you're saying is that the ROM dumper doesn't have the capability of reading chips partially?

If given the right parameters, it could read the chip partially. However, the cartridge dump scene is slightly different from the iso dump scene, as cartridge dumpers only view the dumps that still have the dummy data intact as being "clean." Dating all the way back to the beginning of the emulation days, cartridge rom dumps have been "whole chip" dumps. However, it's only in the last two generations (three, to a lesser extent, if you count GBA) of Nintendo handhelds that we have seen a significant enough size difference between the size of the dump and the size of the actual data to justify the development and safe use of trimmers. Shaving 1-2 MB off a GBA rom isn't as economical as shaving anywhere from tens to hundreds of MB. Sometimes you get even bigger savings, as in the case of Donkey Kong Country Returns: Tropical Breeze, which has almost 1.8 GB of dummy data on its 4 GB chip.
 
  • Like
Reactions: Elliander

IronClouds

GBAtemp's Pokébro
Member
Joined
May 8, 2010
Messages
1,019
Trophies
0
XP
625
Country
United States
Well I already knew the purpose of clean dumps and all that jazz. My question was geared more toward retail carts during manufacturing, etc. The chips within retail carts are a set size, so when the game data is added to retail carts in production, does that machinery not have the capability of only partially writing to the chip so that "filler data" isn't necessary? Or does this whole process only apply to dumping games for pirating purposes?

Thanks for the replies so far, though.
 

zhdarkstar

Well-Known Member
Member
Joined
Jan 30, 2008
Messages
573
Trophies
1
XP
566
Country
United States
I'm not 100% sure, but I think that the production machinery only writes the needed game data, as most storage medium have all of their empty bits set to 00 or FF by default when formatted.

The only real difference between an occupied bit and an empty bit is that the memory allocation table says that one can be overwritten while the other cannot.
 
  • Like
Reactions: Abcdfv

gamesquest1

Nabnut
Former Staff
Joined
Sep 23, 2013
Messages
15,153
Trophies
2
XP
12,247
Well I already knew the purpose of clean dumps and all that jazz. My question was geared more toward retail carts during manufacturing, etc. The chips within retail carts are a set size, so when the game data is added to retail carts in production, does that machinery not have the capability of only partially writing to the chip so that "filler data" isn't necessary? Or does this whole process only apply to dumping games for pirating purposes?

Thanks for the replies so far, though.
I'm not 100% on the details but I dont think they write anything beyond the game data it's just left blank, when dumping the chip the blank data is still dumpable but it's just blank data, assuming it's possible for Nintendo to program the specific size and just leave the rest blank the surely would, yeah 5 seconds extra programming first doesn't sound like much but over the millions of carts produced I'm sure they would rather save that time :)
 
  • Like
Reactions: IronClouds

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,285
Country
United Kingdom
Shaving 1-2 MB off a GBA rom isn't as economical as shaving anywhere from tens to hundreds of MB.

Come back and tell me that after you have used a 16 or 32 meg GBA flash cart that is annoying and slow to write as your primary cart (and probably paid about $100 USD for it). Over about 4 games it soon adds up to space enough for another game. or at least an emulator collection of a couple of games.

On the "being clean" part sure pretrimmed stuff never really flew, however there was not necessarily a great push to have scene releases be exact copies -- nobody got a release nuked just because it had an intro. I know you did not say that but it risked being misinterpreted.
 
  • Like
Reactions: cearp

zhdarkstar

Well-Known Member
Member
Joined
Jan 30, 2008
Messages
573
Trophies
1
XP
566
Country
United States
Come back and tell me that after you have used a 16 or 32 meg GBA flash cart that is annoying and slow to write as your primary cart (and probably paid about $100 USD for it). Over about 4 games it soon adds up to space enough for another game. or at least an emulator collection of a couple of games.

I didn't enter the flashcart scene until a generation later with the slot-2 M3 for DS, but I went through the same struggle back when SDHC wasn't a thing and a 2 GB SD card was $100+. So yes, it was very economical for its time. However, we are two handheld generations removed from the GBA and large external storage is extremely cheaper than it was 10 years ago. My statement was written from a 2014 perspective, not 2004. Sorry if it got you upset.
 
  • Like
Reactions: cearp

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,285
Country
United Kingdom
I was not upset at all, I was just old manning it up.

That said even in 2014 the standard GBA flash cart is probably the EZ4 and if you can squeeze a few more games onto your NOR section then trimming is still worth it.
 
  • Like
Reactions: Sonic3320

gamesquest1

Nabnut
Former Staff
Joined
Sep 23, 2013
Messages
15,153
Trophies
2
XP
12,247
Saving space is always good, but if trimming has adverse effects on the game I.e download play not working etc, surely it's better to leave the trimming down to the end user rather than having to make people untrim when things go wrong XD

Not sure about the GBA stuff, but trimming 3ds roms causes issues and that causes HELP ME PL0X threads and needing to explain what and why trimming is good/bad useful/problematic, until trimming works 100% I think it's better that things are left as they where :P it stops the same thread popping up and users from having issues with certain games etc
 

zhdarkstar

Well-Known Member
Member
Joined
Jan 30, 2008
Messages
573
Trophies
1
XP
566
Country
United States
Not sure about the GBA stuff, but trimming 3ds roms causes issues and that causes HELP ME PL0X threads and needing to explain what and why trimming is good/bad useful/problematic, until trimming works 100% I think it's better that things are left as they where :P it stops the same thread popping up and users from having issues with certain games etc
I hear ya, but I just wish that the replies to those threads weren't always the automatic "the problem is you trimmed the rom and nothing more" as 100% compatibility with flashcarts isn't guaranteed every time a new rom comes out. I posted a "HELP ME PL0X" thread myself just a few days ago for Conception II and was only met with the "blame the trimming" mentality. Frankly, it pisses me off when everyone wants to harp on one detail as the end-all-be-all source of all rom problems when another rom released around the same time had an issue that was found to be present regardless of the rom being trimmed. Sometimes the problems lie with the loader or the rom itself. We need to stop demonizing trimming as it's not the only offender in problems people have with loading/running games.
 

Tattorack

Interstellar Explorer
OP
Member
Joined
Jun 15, 2011
Messages
267
Trophies
1
Age
30
Location
Calisto Prime Orbital
Website
tattorack.deviantart.com
XP
428
Country
Denmark
Yeah, I understand why the sizes are 2x multiples ^^
Thanks, now I also understand where that trimmable "extra" stuff comes from; apparently data on a game-card storage works a quite differantly than data stored on a flash drive (more differant than I had expected). So thanks for clearing that up ^^

Kinda silly people blame everything on trimming. Granted, from what I read trimming in general needs a bit of refinement or just a "steady hand", but blaming everything on trimming when it comes to rom problems would be like blaming a volcano with an evil spirit inside for all life's misfortunes (ok, perhaps a little too overdone here XD ).
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
    AncientBoi @ AncientBoi: [louder cricket sounds]