Hacking Nitro Zip

BassAceGold

Testicles
OP
Member
Joined
Aug 14, 2006
Messages
495
Trophies
0
XP
420
Country
Canada
So I've been playing around with zlib for a bit and finally managed to get it to output valid files with the DSTwo SDK. This got me thinking about creating a compressed rom type format. What I was thinking is that you'd have to just decompress the rom before playing. No on the fly decompression because that isn't really possible.

It would be fairly easy to incorporate such format into a menu like BAGplug (auto decompression and deletion of uncompressed files), if enough people were really into the idea.

Here are some (loosely accurate) tests of the format I have conducted on two opposite sides of the spectrum ( 16mb, 32mb, 64mb, 128mb, 256mb):
*R4DS Rom Trimmer 2.0 was used*
Tetris DS:
normal size - 16 mb
trimmed size - 13.1mb
compressed size - 6.4mb
compression time (on DS) - ~1 minute
decompression time - < 1 minute

Pokemon Soul Silver:
normal size - 128 mb
trimmed size - 120mb
compressed size - 58mb
compression time (on DS) - ~4minutes
decompression time - ~1 minute

Pokemon Black:
normal size - 256 mb
trimmed size - 256 mb
compressed size - 105 mb
compression time (on DS) - ~11 minutes
decompression time - ~1 and a half minutes

Dementium 2:
normal size - 64 mb
trimmed size - 54.2 mb
compressed size - 39.2 mb
compression time (on DS) - ~ 3 minutes
decompression time - ~1 minute

Brothers in Arms DS:
normal size - 32 mb
trimmed size - 30.9 mb
compressed size - 17.5 mb
compression time (on DS) - ~3 minutes
decompression time - < 1 minute

Club House Games:
normal size - 16 mb
trimmed size - 10 mb
compressed size - 8.3 mb
compression time (on DS) - < 1 minute
decompression time - < 30 seconds

Max compression level was used on the DSTwo, so faster compression times can be achieved using a lighter level, or if a pc application was made for compression.
The format itself is just a gzip file, with bits of the nds rom header appended to the end, so menus such as BAGplug or iMenu can still access rom information without having to unzip the rom itself.

So what my question is, is there really a need (or want) for this or are people just happy enough using trimmed roms? If there isn't much interest in this, then this idea won't go far.

Source code for the compression / decompression routines can be found here:
http://bag.nfnet.org/wp-content/uploads/2011/04/nzip-src.zip

Source was built using zlib 1.5.2
 

Rydian

Resident Furvert™
Member
Joined
Feb 4, 2010
Messages
27,880
Trophies
0
Age
34
Location
Cave Entrance, Watching Cyan Write Letters
Website
rydian.net
XP
8,898
Country
United States
The people that would want to squeeze more games onto an SD are usually using 2GB ones, in which the space reserved for an uncompressed game could just fit more games in the first place. On something like a 4GB or 8GB card the savings add up a lot more, but people with those generally don't care about an extra game or two. I fall into the second category, I wouldn't even consider this.

I like the idea, but I don't think people would really use it, I don't want you to waste your time.
 

BassAceGold

Testicles
OP
Member
Joined
Aug 14, 2006
Messages
495
Trophies
0
XP
420
Country
Canada
Rydian said:
The people that would want to squeeze more games onto an SD are usually using 2GB ones, in which the space reserved for an uncompressed game could just fit more games in the first place. On something like a 4GB or 8GB card the savings add up a lot more, but people with those generally don't care about an extra game or two. I fall into the second category, I wouldn't even consider this.

I like the idea, but I don't think people would really use it, I don't want you to waste your time.

It just depends on the combination of games on your card. You may be able to fit one or two extra games on smaller cards if all your games compress small enough while leaving enough space for your biggest uncompressed rom (you can also trim and compress), while others with larger cards may notice the space gain a lot more with more games being compressed.
 

geoflcl

Permanent GBAtemp Newbie
Member
Joined
Jan 18, 2009
Messages
1,375
Trophies
0
Age
27
Website
loganderb.in
XP
489
Country
United States
What a novel idea!

Although something of this sort wouldn't be a concern for everybody (as Rydian said), I happen to be one of those lamers who use a 2GB card to cram all of their flashcart paraphernalia. Any saved space is quite beneficial, and I find the compression rates you've shown to be quite impressive!

Heck, even if you don't decide to go far with this idea, you've made me ecstatic just by this proof of concept. Wonderful work!

EDIT: I have just realized the problem of decompression free space for smaller cards. Oopse. Consider my above post exemplary of my stupidity.
(I really need to stop browsing the 'Temp while struggling to stay awake. :S )
 

gumbyx84

Well-Known Member
Member
Joined
Apr 23, 2010
Messages
562
Trophies
0
XP
399
Country
United States
This is interesting. While I agree with Rydian that its useless on smaller cards, I can see this being useful to people like me with 8GB cards. Would love to help you with this (coding and testing) if you choose to continue, but sadly I don't have a DSTWO.

PS Would it be possible to use this in DSi mode with the iEvo?
 

Boriar

Well-Known Member
Member
Joined
Sep 22, 2008
Messages
316
Trophies
0
Age
50
Location
Spain
XP
653
Country
I want it so much because:

1. I hate trim my roms backups. I can't explain it better or way but I dislike it
angry.gif

2. I like carry on my mSD all the roms I want to play but only play one to three simultaneusly, so the other ones may be compressed to preserve space.
yaynds.gif


Until now I use DSCompress v1.2 Author: Carpman http://voidptr.org but use dstwo processor will be great!!!

Edit: I can test it or translate something if you need it
 
D

Deleted_171835

Guest
I would use this!

I have a 2GB microSD card in my DSTWO and I can't fit every game I want to play even after I trim the rom. This would be great!
 

BassAceGold

Testicles
OP
Member
Joined
Aug 14, 2006
Messages
495
Trophies
0
XP
420
Country
Canada
gumbyx84 said:
This is interesting. While I agree with Rydian that its useless on smaller cards, I can see this being useful to people like me with 8GB cards. Would love to help you with this (coding and testing) if you choose to continue, but sadly I don't have a DSTWO.

PS Would it be possible to use this in DSi mode with the iEvo?

The code itself is fairly portable. Anything that has a zlib port can decompress/compress it, just depends on the system itself if the process will be fast or not.

QUOTE(SoulSnatcher @ Apr 15 2011, 04:21 PM) I would use this!

I have a 2GB microSD card in my DSTWO and I can't fit every game I want to play even after I trim the rom. This would be great!

Keep in mind that you do need to leave enough free space on your card to hold the largest uncompressed rom you have.
 

gumbyx84

Well-Known Member
Member
Joined
Apr 23, 2010
Messages
562
Trophies
0
XP
399
Country
United States
BassAceGold said:
gumbyx84 said:
This is interesting. While I agree with Rydian that its useless on smaller cards, I can see this being useful to people like me with 8GB cards. Would love to help you with this (coding and testing) if you choose to continue, but sadly I don't have a DSTWO.

PS Would it be possible to use this in DSi mode with the iEvo?

The code itself is fairly portable. Anything that has a zlib port can decompress/compress it, just depends on the system itself if the process will be fast or not.
... This really makes me want to get a DSTWO and iEvo. I would be all over helping you code or even test stuff for ya. Been wanting to get back in development for awhile and this sounds like fun. :-D
 

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
33,867
Trophies
2
Website
trastindustries.com
XP
22,598
Country
United Kingdom
A word on Tetris DS- if you picked the Japanese or US version the SDAT file has a meg or two of padding on it which when coupled with the regular trimming leads to that massive compression you saw.

Nice proof of concept though- if you are needing more code there is DSCompress as well.
 

BassAceGold

Testicles
OP
Member
Joined
Aug 14, 2006
Messages
495
Trophies
0
XP
420
Country
Canada
FAST6191 said:
A word on Tetris DS- if you picked the Japanese or US version the SDAT file has a meg or two of padding on it which when coupled with the regular trimming leads to that massive compression you saw.

Nice proof of concept though- if you are needing more code there is DSCompress as well.

I compressed untrimmed roms only. I might just test a bunch more roms to get a more accurate average rate of compression.

Updated first post with more results.
 

BassAceGold

Testicles
OP
Member
Joined
Aug 14, 2006
Messages
495
Trophies
0
XP
420
Country
Canada
Just posting a little bit based on the information in the first post. If you happened to have 512mb of free space, enough to hold all the games listed in the first post, and you compress them, you'd have 277.6 mb of free space left over. This space is enough for another compressed game (a small one) while still having enough room to decompress the largest rom (256mb), assuming you don't fill up the left over space with other things.
 

gumbyx84

Well-Known Member
Member
Joined
Apr 23, 2010
Messages
562
Trophies
0
XP
399
Country
United States
Those compression ratios are nice, as are the decompression times. I'm guessing you would make a PC app that would allow you to compress ROMs. Compression time would be much faster if done on PC.
 

BassAceGold

Testicles
OP
Member
Joined
Aug 14, 2006
Messages
495
Trophies
0
XP
420
Country
Canada
Added source for the compression / decompression routines if anyone wants to make something (sorry for the sparse comments, hopefully the code is self explanitory) of it in the first post. If anyone makes any modifications to the code just post it here so we don't get multiple different formats created.
 

Another World

Emulate the Planet!
Former Staff
Joined
Jan 3, 2008
Messages
10,578
Trophies
0
Age
45
Location
From Where???
Website
wiki.gbatemp.net
XP
5,305
Country
Colombia
my only concern is the extra writing being done to the card each time a file is decompressed. if i play that file 10 times in a day its essentially written to the card 10 times. i'm thinking about the limited amount of read/writes a msd has.

this is a really cool idea for nds roms, tho i'm not sure how practical it would be.

-another world
 

BassAceGold

Testicles
OP
Member
Joined
Aug 14, 2006
Messages
495
Trophies
0
XP
420
Country
Canada
Another World said:
my only concern is the extra writing being done to the card each time a file is decompressed. if i play that file 10 times in a day its essentially written to the card 10 times. i'm thinking about the limited amount of read/writes a msd has.

this is a really cool idea for nds roms, tho i'm not sure how practical it would be.

-another world
That's dependent on the menu using it. If the menu was smart, it wouldn't delete the uncompressed file unless a new compressed file was selected to unzip. So basically once you unzip a file, you wouldn't need to keep unzipping it until you select a new game to unzip.
 

twiztidsinz

Taiju Yamada Fan
Member
Joined
Dec 23, 2008
Messages
4,979
Trophies
0
Website
Visit site
XP
219
Country
United States
Another World said:
my only concern is the extra writing being done to the card each time a file is decompressed. if i play that file 10 times in a day its essentially written to the card 10 times. i'm thinking about the limited amount of read/writes a msd has.

this is a really cool idea for nds roms, tho i'm not sure how practical it would be.

-another world
http://en.wikipedia.org/wiki/Flash_memory#Memory_wear
They're rated to 100,000 writes, and using your 10 writes a day and doubling it for saves is 20 writes/day
100,000 / 20 = 5,000 / 365 = 13.6 years.

Add in something like 'wear leveling' that some cards use and you're going to get a longer life out of it.
 

Boriar

Well-Known Member
Member
Joined
Sep 22, 2008
Messages
316
Trophies
0
Age
50
Location
Spain
XP
653
Country
BassAceGold said:
Another World said:
my only concern is the extra writing being done to the card each time a file is decompressed. if i play that file 10 times in a day its essentially written to the card 10 times. i'm thinking about the limited amount of read/writes a msd has.

this is a really cool idea for nds roms, tho i'm not sure how practical it would be.

-another world
That's dependent on the menu using it. If the menu was smart, it wouldn't delete the uncompressed file unless a new compressed file was selected to unzip. So basically once you unzip a file, you wouldn't need to keep unzipping it until you select a new game to unzip.

Also, you can let user to set on/off a deleting option. In that way with on actived anyone can select a game and if it's already decompressed play it directly (not delete game at all) and with off activated only keep the decompressed one until a different game will be selected, saving more time/space like you choose.
rolleyes.gif
 
General chit-chat
Help Users
  • kenenthk @ kenenthk:
    3d porn ftw
    Gift
  • Veho @ Veho:
    Aww yiss, I have to have that boner really sticking out of the screen for me to get off.
    Gift
  • kenenthk @ kenenthk:
    I always wanted to swordfight a virtual boner
    Gift
  • Veho @ Veho:
    Beat Saber erotic
    sequel, Beat Boner.
    Gift
  • kenenthk @ kenenthk:
    Jds on that dev team
    Gift
  • S @ Shape:
    I've been playing Beat Boner since 1998, pretty sure it's a prequel.
    Gift
  • kenenthk @ kenenthk:
    98? Get gud since 95
    Gift
  • kenenthk @ kenenthk:
    Veho needs extra attachments for Beat boner at his age :tpi:
    Gift
  • S @ Shape:
    Get gud? i bet i can beat your Beat Boner score a
    ny day of the week :rofl2:
    Gift
  • kenenthk @ kenenthk:
    1v1 me bro no homo tho
    Gift
  • kenenthk @ kenenthk:
    Extra point when dickslang mp3 pops up
    Gift
  • S @ Shape:
    lets doooooo iiiiit
    Gift
  • S @ Shape:
    no homo
    Gift
  • kenenthk @ kenenthk:
    Sounds gay
    Gift
  • Veho @ Veho:
    ken is right, at my age I need some prep time before I can beat my boner.
    Gift
  • S @ Shape:
    i'm out of the lube it came with though so i need to go to the nintendo section at best buy real quick
    Gift
  • Veho @ Veho:
    Prep time and medication.
    Gift
  • kenenthk @ kenenthk:
    At psis age you need all the big pharma dlc you can get
    Gift
  • S @ Shape:
    God i can't even imagine the microtransactions on that
    Gift
  • kenenthk @ kenenthk:
    Well if you have no coverage you're fucked
    Gift
  • S @ Shape:
    mine is constantly like a brick though, i bet veho would have an easier time putting on the wii sleeve for the game
    Gift
  • kenenthk @ kenenthk:
    Lol if Veho used the sleeve he wouldn't be where he's at today
    Gift
  • S @ Shape:
    :rofl2:classic.
    Gift
  • kenenthk @ kenenthk:
    God ups is annoying with delivery city is literally 20 minutes away on the truck for 4 hours other peoples packages aren't important :angry:
    Gift
  • Gift
    Veho @ Veho: https://i.imgur.com/RNcO4zl.jpg