Hacking Nitro Zip

BassAceGold

Testicles
OP
Member
Joined
Aug 14, 2006
Messages
496
Trophies
1
XP
441
Country
Canada
So I've been playing around with zlib for a bit and finally managed to get it to output valid files with the DSTwo SDK. This got me thinking about creating a compressed rom type format. What I was thinking is that you'd have to just decompress the rom before playing. No on the fly decompression because that isn't really possible.

It would be fairly easy to incorporate such format into a menu like BAGplug (auto decompression and deletion of uncompressed files), if enough people were really into the idea.

Here are some (loosely accurate) tests of the format I have conducted on two opposite sides of the spectrum ( 16mb, 32mb, 64mb, 128mb, 256mb):
*R4DS Rom Trimmer 2.0 was used*
Tetris DS:
normal size - 16 mb
trimmed size - 13.1mb
compressed size - 6.4mb
compression time (on DS) - ~1 minute
decompression time - < 1 minute

Pokemon Soul Silver:
normal size - 128 mb
trimmed size - 120mb
compressed size - 58mb
compression time (on DS) - ~4minutes
decompression time - ~1 minute

Pokemon Black:
normal size - 256 mb
trimmed size - 256 mb
compressed size - 105 mb
compression time (on DS) - ~11 minutes
decompression time - ~1 and a half minutes

Dementium 2:
normal size - 64 mb
trimmed size - 54.2 mb
compressed size - 39.2 mb
compression time (on DS) - ~ 3 minutes
decompression time - ~1 minute

Brothers in Arms DS:
normal size - 32 mb
trimmed size - 30.9 mb
compressed size - 17.5 mb
compression time (on DS) - ~3 minutes
decompression time - < 1 minute

Club House Games:
normal size - 16 mb
trimmed size - 10 mb
compressed size - 8.3 mb
compression time (on DS) - < 1 minute
decompression time - < 30 seconds

Max compression level was used on the DSTwo, so faster compression times can be achieved using a lighter level, or if a pc application was made for compression.
The format itself is just a gzip file, with bits of the nds rom header appended to the end, so menus such as BAGplug or iMenu can still access rom information without having to unzip the rom itself.

So what my question is, is there really a need (or want) for this or are people just happy enough using trimmed roms? If there isn't much interest in this, then this idea won't go far.

Source code for the compression / decompression routines can be found here:
http://bag.nfnet.org/wp-content/uploads/2011/04/nzip-src.zip

Source was built using zlib 1.5.2
 

Rydian

Resident Furvert™
Member
Joined
Feb 4, 2010
Messages
27,880
Trophies
0
Age
36
Location
Cave Entrance, Watching Cyan Write Letters
Website
rydian.net
XP
9,111
Country
United States
The people that would want to squeeze more games onto an SD are usually using 2GB ones, in which the space reserved for an uncompressed game could just fit more games in the first place. On something like a 4GB or 8GB card the savings add up a lot more, but people with those generally don't care about an extra game or two. I fall into the second category, I wouldn't even consider this.

I like the idea, but I don't think people would really use it, I don't want you to waste your time.
 

BassAceGold

Testicles
OP
Member
Joined
Aug 14, 2006
Messages
496
Trophies
1
XP
441
Country
Canada
Rydian said:
The people that would want to squeeze more games onto an SD are usually using 2GB ones, in which the space reserved for an uncompressed game could just fit more games in the first place. On something like a 4GB or 8GB card the savings add up a lot more, but people with those generally don't care about an extra game or two. I fall into the second category, I wouldn't even consider this.

I like the idea, but I don't think people would really use it, I don't want you to waste your time.

It just depends on the combination of games on your card. You may be able to fit one or two extra games on smaller cards if all your games compress small enough while leaving enough space for your biggest uncompressed rom (you can also trim and compress), while others with larger cards may notice the space gain a lot more with more games being compressed.
 

geoflcl

Permanent GBAtemp Newbie
Member
Joined
Jan 18, 2009
Messages
1,375
Trophies
0
Age
30
Website
loganderb.in
XP
522
Country
United States
What a novel idea!

Although something of this sort wouldn't be a concern for everybody (as Rydian said), I happen to be one of those lamers who use a 2GB card to cram all of their flashcart paraphernalia. Any saved space is quite beneficial, and I find the compression rates you've shown to be quite impressive!

Heck, even if you don't decide to go far with this idea, you've made me ecstatic just by this proof of concept. Wonderful work!

EDIT: I have just realized the problem of decompression free space for smaller cards. Oopse. Consider my above post exemplary of my stupidity.
(I really need to stop browsing the 'Temp while struggling to stay awake. :S )
 

gumbyx84

Well-Known Member
Member
Joined
Apr 23, 2010
Messages
580
Trophies
1
XP
780
Country
United States
This is interesting. While I agree with Rydian that its useless on smaller cards, I can see this being useful to people like me with 8GB cards. Would love to help you with this (coding and testing) if you choose to continue, but sadly I don't have a DSTWO.

PS Would it be possible to use this in DSi mode with the iEvo?
 

Boriar

Well-Known Member
Member
Joined
Sep 22, 2008
Messages
316
Trophies
1
Age
52
Location
Spain
XP
695
Country
I want it so much because:

1. I hate trim my roms backups. I can't explain it better or way but I dislike it
angry.gif

2. I like carry on my mSD all the roms I want to play but only play one to three simultaneusly, so the other ones may be compressed to preserve space.
yaynds.gif


Until now I use DSCompress v1.2 Author: Carpman http://voidptr.org but use dstwo processor will be great!!!

Edit: I can test it or translate something if you need it
 
D

Deleted_171835

Guest
I would use this!

I have a 2GB microSD card in my DSTWO and I can't fit every game I want to play even after I trim the rom. This would be great!
 

BassAceGold

Testicles
OP
Member
Joined
Aug 14, 2006
Messages
496
Trophies
1
XP
441
Country
Canada
gumbyx84 said:
This is interesting. While I agree with Rydian that its useless on smaller cards, I can see this being useful to people like me with 8GB cards. Would love to help you with this (coding and testing) if you choose to continue, but sadly I don't have a DSTWO.

PS Would it be possible to use this in DSi mode with the iEvo?

The code itself is fairly portable. Anything that has a zlib port can decompress/compress it, just depends on the system itself if the process will be fast or not.

QUOTE(SoulSnatcher @ Apr 15 2011, 04:21 PM) I would use this!

I have a 2GB microSD card in my DSTWO and I can't fit every game I want to play even after I trim the rom. This would be great!

Keep in mind that you do need to leave enough free space on your card to hold the largest uncompressed rom you have.
 

gumbyx84

Well-Known Member
Member
Joined
Apr 23, 2010
Messages
580
Trophies
1
XP
780
Country
United States
BassAceGold said:
gumbyx84 said:
This is interesting. While I agree with Rydian that its useless on smaller cards, I can see this being useful to people like me with 8GB cards. Would love to help you with this (coding and testing) if you choose to continue, but sadly I don't have a DSTWO.

PS Would it be possible to use this in DSi mode with the iEvo?

The code itself is fairly portable. Anything that has a zlib port can decompress/compress it, just depends on the system itself if the process will be fast or not.
... This really makes me want to get a DSTWO and iEvo. I would be all over helping you code or even test stuff for ya. Been wanting to get back in development for awhile and this sounds like fun. :-D
 

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,284
Country
United Kingdom
A word on Tetris DS- if you picked the Japanese or US version the SDAT file has a meg or two of padding on it which when coupled with the regular trimming leads to that massive compression you saw.

Nice proof of concept though- if you are needing more code there is DSCompress as well.
 

BassAceGold

Testicles
OP
Member
Joined
Aug 14, 2006
Messages
496
Trophies
1
XP
441
Country
Canada
FAST6191 said:
A word on Tetris DS- if you picked the Japanese or US version the SDAT file has a meg or two of padding on it which when coupled with the regular trimming leads to that massive compression you saw.

Nice proof of concept though- if you are needing more code there is DSCompress as well.

I compressed untrimmed roms only. I might just test a bunch more roms to get a more accurate average rate of compression.

Updated first post with more results.
 

BassAceGold

Testicles
OP
Member
Joined
Aug 14, 2006
Messages
496
Trophies
1
XP
441
Country
Canada
Just posting a little bit based on the information in the first post. If you happened to have 512mb of free space, enough to hold all the games listed in the first post, and you compress them, you'd have 277.6 mb of free space left over. This space is enough for another compressed game (a small one) while still having enough room to decompress the largest rom (256mb), assuming you don't fill up the left over space with other things.
 

gumbyx84

Well-Known Member
Member
Joined
Apr 23, 2010
Messages
580
Trophies
1
XP
780
Country
United States
Those compression ratios are nice, as are the decompression times. I'm guessing you would make a PC app that would allow you to compress ROMs. Compression time would be much faster if done on PC.
 

BassAceGold

Testicles
OP
Member
Joined
Aug 14, 2006
Messages
496
Trophies
1
XP
441
Country
Canada
Added source for the compression / decompression routines if anyone wants to make something (sorry for the sparse comments, hopefully the code is self explanitory) of it in the first post. If anyone makes any modifications to the code just post it here so we don't get multiple different formats created.
 

Another World

Emulate the Planet!
Former Staff
Joined
Jan 3, 2008
Messages
10,579
Trophies
2
Age
48
Location
From Where???
Website
wiki.gbatemp.net
XP
5,533
Country
Colombia
my only concern is the extra writing being done to the card each time a file is decompressed. if i play that file 10 times in a day its essentially written to the card 10 times. i'm thinking about the limited amount of read/writes a msd has.

this is a really cool idea for nds roms, tho i'm not sure how practical it would be.

-another world
 

BassAceGold

Testicles
OP
Member
Joined
Aug 14, 2006
Messages
496
Trophies
1
XP
441
Country
Canada
Another World said:
my only concern is the extra writing being done to the card each time a file is decompressed. if i play that file 10 times in a day its essentially written to the card 10 times. i'm thinking about the limited amount of read/writes a msd has.

this is a really cool idea for nds roms, tho i'm not sure how practical it would be.

-another world
That's dependent on the menu using it. If the menu was smart, it wouldn't delete the uncompressed file unless a new compressed file was selected to unzip. So basically once you unzip a file, you wouldn't need to keep unzipping it until you select a new game to unzip.
 

twiztidsinz

Taiju Yamada Fan
Member
Joined
Dec 23, 2008
Messages
4,979
Trophies
0
Website
Visit site
XP
220
Country
United States
Another World said:
my only concern is the extra writing being done to the card each time a file is decompressed. if i play that file 10 times in a day its essentially written to the card 10 times. i'm thinking about the limited amount of read/writes a msd has.

this is a really cool idea for nds roms, tho i'm not sure how practical it would be.

-another world
http://en.wikipedia.org/wiki/Flash_memory#Memory_wear
They're rated to 100,000 writes, and using your 10 writes a day and doubling it for saves is 20 writes/day
100,000 / 20 = 5,000 / 365 = 13.6 years.

Add in something like 'wear leveling' that some cards use and you're going to get a longer life out of it.
 

Boriar

Well-Known Member
Member
Joined
Sep 22, 2008
Messages
316
Trophies
1
Age
52
Location
Spain
XP
695
Country
BassAceGold said:
Another World said:
my only concern is the extra writing being done to the card each time a file is decompressed. if i play that file 10 times in a day its essentially written to the card 10 times. i'm thinking about the limited amount of read/writes a msd has.

this is a really cool idea for nds roms, tho i'm not sure how practical it would be.

-another world
That's dependent on the menu using it. If the menu was smart, it wouldn't delete the uncompressed file unless a new compressed file was selected to unzip. So basically once you unzip a file, you wouldn't need to keep unzipping it until you select a new game to unzip.

Also, you can let user to set on/off a deleting option. In that way with on actived anyone can select a game and if it's already decompressed play it directly (not delete game at all) and with off activated only keep the decompressed one until a different game will be selected, saving more time/space like you choose.
rolleyes.gif
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
  • S @ salazarcosplay:
    @BigOnYa they ruined the gaming experience for me to be honest
  • S @ salazarcosplay:
    @BigOnYa Im not crazy about getting all of them, i feel like I have something to show for for the time put in
  • S @ salazarcosplay:
    @BigOnYa If you want to do rgh or 360 mod
  • S @ salazarcosplay:
    does it matter if you update your 360 or not before trying is it advisable or not
  • BigOnYa @ BigOnYa:
    Yea I don't pay attention to them really. Or do I try to 100% a game. I just play till story ends/ or I get the girl!
  • K3Nv2 @ K3Nv2:
    Bigonya uses his wiener to mod 360s
    +1
  • Xdqwerty @ Xdqwerty:
    Going to the water park, see ya
  • BigOnYa @ BigOnYa:
    You should update the 360 to newest dash before RGHing it yes. But not a big deal if you don't, you can install new dash/avatar updates after. It's just easier to do it auto online before, instead manual offline after.
  • BigOnYa @ BigOnYa:
    Have fun @Xdqwerty. If you see a chocolate candy bar floating in the water, don't eat it!
  • AncientBoi @ AncientBoi:
    :O:ohnoes: Y didn't U Tell ME that ALSO? @BigOnYa :ohnoes: 🤢🤮
    +1
  • BigOnYa @ BigOnYa:
    Does it taste like... chicken?
    +1
  • S @ salazarcosplay:
    @BigOnYa I wanted to ask you about your experience with seeing south park. Most of the people a bit younger like my younger brother and cousins that are a few younger than me that saw kids found south park funny because of the curse words, kids at school, that seemed like liking the show on a very basic level.

    I could not quite have a in depth discussion of the show.

    How was it for you? As an adult. What did you find the most interesting part about it. Did you relate to the parents of the kids and their situations. Was it satires, the commentary on society. The references on celebrities' and pop culture.
    +1
  • BigOnYa @ BigOnYa:
    I remember seeing the very first episode back in the day, and have watched every episode since. I used to set my VCR to record them even, shows how long ago.
  • BigOnYa @ BigOnYa:
    I just like any comedies really, and cartoons have always been a favorite of mine. Family guy, American Dad, Futurama, Cleveland Show, Simpsons - I like them all.
    +1
  • BigOnYa @ BigOnYa:
    South Park is great cause they always touch on relavent issues going on today, and make something funny out of it.
    +3
  • S @ salazarcosplay:
    @BigOnYa were you always up to date on the current events and issues of the time or were there issues that you first found out thru south park
  • BigOnYa @ BigOnYa:
    Most of the time yea I knew, I watch and read the news regularly, but sometimes the Hollywood BS stuff, like concerning actors slip by me. I don't follow most Hollywood BS (example: the Kardasians)
    +2
  • S @ salazarcosplay:
    @BigOnYa there were relevant issues before south park was made, that's why i think a south park prequel/spinoff would be great. Randy and his friends in their child hood
    +1
  • BigOnYa @ BigOnYa:
    Yea, like them running in high school together, getting into stuff, and how they got hitched and had kids. And how the town of South Park was back then compared to now. That would be cool to see.
  • BakerMan @ BakerMan:
    yeah
  • The Real Jdbye @ The Real Jdbye:
    @salazarcosplay if they made a prequel, it would still be about current issues, cause it doesn't make sense to make it about stuff that happened 30 years ago that nobody cares about anymore
  • The Real Jdbye @ The Real Jdbye:
    it's too late
  • The Real Jdbye @ The Real Jdbye:
    the older south park episodes about particular issues usually age poorly since the topic is no longer relevant
  • The Real Jdbye @ The Real Jdbye:
    an exception is giant douche vs turd sandwich, that's always relevant :P
    +1
    The Real Jdbye @ The Real Jdbye: an exception is giant douche vs turd sandwich, that's always relevant :P +1