Nitro Zip

Discussion in 'Supercard' started by BassAceGold, Apr 15, 2011.

Apr 15, 2011

Nitro Zip by BassAceGold at 11:02 PM (2,969 Views / 0 Likes) 22 replies

  1. BassAceGold
    OP

    Member BassAceGold Testicles

    Joined:
    Aug 14, 2006
    Messages:
    494
    Country:
    Canada
    So I've been playing around with zlib for a bit and finally managed to get it to output valid files with the DSTwo SDK. This got me thinking about creating a compressed rom type format. What I was thinking is that you'd have to just decompress the rom before playing. No on the fly decompression because that isn't really possible.

    It would be fairly easy to incorporate such format into a menu like BAGplug (auto decompression and deletion of uncompressed files), if enough people were really into the idea.

    Here are some (loosely accurate) tests of the format I have conducted on two opposite sides of the spectrum ( 16mb, 32mb, 64mb, 128mb, 256mb):
    *R4DS Rom Trimmer 2.0 was used*
    Tetris DS:
    normal size - 16 mb
    trimmed size - 13.1mb
    compressed size - 6.4mb
    compression time (on DS) - ~1 minute
    decompression time - < 1 minute

    Pokemon Soul Silver:
    normal size - 128 mb
    trimmed size - 120mb
    compressed size - 58mb
    compression time (on DS) - ~4minutes
    decompression time - ~1 minute

    Pokemon Black:
    normal size - 256 mb
    trimmed size - 256 mb
    compressed size - 105 mb
    compression time (on DS) - ~11 minutes
    decompression time - ~1 and a half minutes

    Dementium 2:
    normal size - 64 mb
    trimmed size - 54.2 mb
    compressed size - 39.2 mb
    compression time (on DS) - ~ 3 minutes
    decompression time - ~1 minute

    Brothers in Arms DS:
    normal size - 32 mb
    trimmed size - 30.9 mb
    compressed size - 17.5 mb
    compression time (on DS) - ~3 minutes
    decompression time - < 1 minute

    Club House Games:
    normal size - 16 mb
    trimmed size - 10 mb
    compressed size - 8.3 mb
    compression time (on DS) - < 1 minute
    decompression time - < 30 seconds

    Max compression level was used on the DSTwo, so faster compression times can be achieved using a lighter level, or if a pc application was made for compression.
    The format itself is just a gzip file, with bits of the nds rom header appended to the end, so menus such as BAGplug or iMenu can still access rom information without having to unzip the rom itself.

    So what my question is, is there really a need (or want) for this or are people just happy enough using trimmed roms? If there isn't much interest in this, then this idea won't go far.

    Source code for the compression / decompression routines can be found here:
    http://bag.nfnet.org/wp-content/uploads/2011/04/nzip-src.zip

    Source was built using zlib 1.5.2
     


  2. Cuelhu

    Member Cuelhu КУ!

    Joined:
    May 19, 2010
    Messages:
    536
    Location:
    Fantasy Land
    Country:
    this sounds brilliant even though I don't trim my ROMs.
     
  3. Rydian

    Member Rydian Resident Furvert™

    Joined:
    Feb 4, 2010
    Messages:
    27,883
    Location:
    Cave Entrance, Watching Cyan Write Letters
    Country:
    United States
    The people that would want to squeeze more games onto an SD are usually using 2GB ones, in which the space reserved for an uncompressed game could just fit more games in the first place. On something like a 4GB or 8GB card the savings add up a lot more, but people with those generally don't care about an extra game or two. I fall into the second category, I wouldn't even consider this.

    I like the idea, but I don't think people would really use it, I don't want you to waste your time.
     
  4. BassAceGold
    OP

    Member BassAceGold Testicles

    Joined:
    Aug 14, 2006
    Messages:
    494
    Country:
    Canada
    It just depends on the combination of games on your card. You may be able to fit one or two extra games on smaller cards if all your games compress small enough while leaving enough space for your biggest uncompressed rom (you can also trim and compress), while others with larger cards may notice the space gain a lot more with more games being compressed.
     
  5. geoflcl

    Member geoflcl Permanent GBAtemp Newbie

    Joined:
    Jan 18, 2009
    Messages:
    1,373
    Country:
    United States
    What a novel idea!

    Although something of this sort wouldn't be a concern for everybody (as Rydian said), I happen to be one of those lamers who use a 2GB card to cram all of their flashcart paraphernalia. Any saved space is quite beneficial, and I find the compression rates you've shown to be quite impressive!

    Heck, even if you don't decide to go far with this idea, you've made me ecstatic just by this proof of concept. Wonderful work!

    EDIT: I have just realized the problem of decompression free space for smaller cards. Oopse. Consider my above post exemplary of my stupidity.
    (I really need to stop browsing the 'Temp while struggling to stay awake. :S )
     
  6. gumbyx84

    Member gumbyx84 GBAtemp Advanced Fan

    Joined:
    Apr 23, 2010
    Messages:
    512
    Country:
    United States
    This is interesting. While I agree with Rydian that its useless on smaller cards, I can see this being useful to people like me with 8GB cards. Would love to help you with this (coding and testing) if you choose to continue, but sadly I don't have a DSTWO.

    PS Would it be possible to use this in DSi mode with the iEvo?
     
  7. Boriar

    Member Boriar GBAtemp Fan

    Joined:
    Sep 22, 2008
    Messages:
    313
    Location:
    Spain
    Country:
    Spain
    I want it so much because:

    1. I hate trim my roms backups. I can't explain it better or way but I dislike it [​IMG]
    2. I like carry on my mSD all the roms I want to play but only play one to three simultaneusly, so the other ones may be compressed to preserve space. [​IMG]

    Until now I use DSCompress v1.2 Author: Carpman http://voidptr.org but use dstwo processor will be great!!!

    Edit: I can test it or translate something if you need it
     
  8. soulx

    Member soulx GBAtemp Legend

    Joined:
    Apr 4, 2009
    Messages:
    10,130
    Country:
    Canada
    I would use this!

    I have a 2GB microSD card in my DSTWO and I can't fit every game I want to play even after I trim the rom. This would be great!
     
  9. BassAceGold
    OP

    Member BassAceGold Testicles

    Joined:
    Aug 14, 2006
    Messages:
    494
    Country:
    Canada
    Keep in mind that you do need to leave enough free space on your card to hold the largest uncompressed rom you have.
     
  10. gumbyx84

    Member gumbyx84 GBAtemp Advanced Fan

    Joined:
    Apr 23, 2010
    Messages:
    512
    Country:
    United States
    ... This really makes me want to get a DSTWO and iEvo. I would be all over helping you code or even test stuff for ya. Been wanting to get back in development for awhile and this sounds like fun. :-D
     
  11. FAST6191

    Reporter FAST6191 Techromancer

    pip
    Joined:
    Nov 21, 2005
    Messages:
    21,746
    Country:
    United Kingdom
    A word on Tetris DS- if you picked the Japanese or US version the SDAT file has a meg or two of padding on it which when coupled with the regular trimming leads to that massive compression you saw.

    Nice proof of concept though- if you are needing more code there is DSCompress as well.
     
  12. Schlupi

    Member Schlupi Gbatemp's Official Earthbound Maniac™

    Joined:
    Aug 31, 2007
    Messages:
    3,985
    Location:
    Rozen Queen Co, Chicago Branch
    Country:
    United States
    I fully support this idea! I have an 8 GB, 2 GB, and 4 GB (hahaha) and would love to help test this out.
     
  13. BassAceGold
    OP

    Member BassAceGold Testicles

    Joined:
    Aug 14, 2006
    Messages:
    494
    Country:
    Canada
    I compressed untrimmed roms only. I might just test a bunch more roms to get a more accurate average rate of compression.

    Updated first post with more results.
     
  14. BassAceGold
    OP

    Member BassAceGold Testicles

    Joined:
    Aug 14, 2006
    Messages:
    494
    Country:
    Canada
    Just posting a little bit based on the information in the first post. If you happened to have 512mb of free space, enough to hold all the games listed in the first post, and you compress them, you'd have 277.6 mb of free space left over. This space is enough for another compressed game (a small one) while still having enough room to decompress the largest rom (256mb), assuming you don't fill up the left over space with other things.
     
  15. gumbyx84

    Member gumbyx84 GBAtemp Advanced Fan

    Joined:
    Apr 23, 2010
    Messages:
    512
    Country:
    United States
    Those compression ratios are nice, as are the decompression times. I'm guessing you would make a PC app that would allow you to compress ROMs. Compression time would be much faster if done on PC.
     
  16. BassAceGold
    OP

    Member BassAceGold Testicles

    Joined:
    Aug 14, 2006
    Messages:
    494
    Country:
    Canada
    Added source for the compression / decompression routines if anyone wants to make something (sorry for the sparse comments, hopefully the code is self explanitory) of it in the first post. If anyone makes any modifications to the code just post it here so we don't get multiple different formats created.
     
  17. Another World

    Former Staff Another World Emulate the Planet!

    Joined:
    Jan 3, 2008
    Messages:
    10,476
    Location:
    From Where???
    Country:
    Colombia
    my only concern is the extra writing being done to the card each time a file is decompressed. if i play that file 10 times in a day its essentially written to the card 10 times. i'm thinking about the limited amount of read/writes a msd has.

    this is a really cool idea for nds roms, tho i'm not sure how practical it would be.

    -another world
     
  18. BassAceGold
    OP

    Member BassAceGold Testicles

    Joined:
    Aug 14, 2006
    Messages:
    494
    Country:
    Canada
    That's dependent on the menu using it. If the menu was smart, it wouldn't delete the uncompressed file unless a new compressed file was selected to unzip. So basically once you unzip a file, you wouldn't need to keep unzipping it until you select a new game to unzip.
     
  19. twiztidsinz

    Member twiztidsinz Taiju Yamada Fan

    Joined:
    Dec 23, 2008
    Messages:
    4,981
    Country:
    United States
    http://en.wikipedia.org/wiki/Flash_memory#Memory_wear
    They're rated to 100,000 writes, and using your 10 writes a day and doubling it for saves is 20 writes/day
    100,000 / 20 = 5,000 / 365 = 13.6 years.

    Add in something like 'wear leveling' that some cards use and you're going to get a longer life out of it.
     
  20. Boriar

    Member Boriar GBAtemp Fan

    Joined:
    Sep 22, 2008
    Messages:
    313
    Location:
    Spain
    Country:
    Spain
    Also, you can let user to set on/off a deleting option. In that way with on actived anyone can select a game and if it's already decompressed play it directly (not delete game at all) and with off activated only keep the decompressed one until a different game will be selected, saving more time/space like you choose. [​IMG]
     

Share This Page