Hacking Compressed ROMs

[Retal]

Is it not technically possible for ROMs to run out of zip archive or similar? If not, then why.

 

[D-Trogh]

With VisualBoyAdvance I think you CAN run ROMs out of a .ZIP archive..
Correct me when I'm wrong.. but I think you can..

But er.. talking about ROMs is not allowed.. right ?
Meh.. maybe you can talk about them, as it's more about an emulator here..
As long as you don't ask for them

 

[Maikel Steneker]

Is it not technically possible for ROMs to run out of zip archive or similar? If not, then why.

It is technically possible. The flashcard would have to unpack the rom onthefly. This would probably require memory in the flashcard though.

 

[TLSpartan]

You can talk about ROMS but not distrubute them(through links or otherwise). Emulators are fine

 

[Retal]

Can you quit posting utter bullshit please. The only person who made a sensible reply was maikelsteneker. This has NOTHING to do with emulators. In case you hadn't have already noticed, this is the R4 and flashcart forum and nowhere in my post did I mention emulators.

It is technically possible. The flashcard would have to unpack the rom onthefly. This would probably require memory in the flashcard though.

If flashcarts don't have enough or any memory then it is not technically possible. Do flashcarts have enough memory to run carts in this way or not?

 

[rest0re]

No its not possible atm

 

[arctic_flame]

It is technically possible. It would just take ages to load.

Steps:

1. Have enough free space on your card for the unzipped rom
2. Write an unzip program for your DS
3. Curse and swear at how slowly the unzip prgram runs.
4. Wait a bit more
5. Run the uncompressed rom.
6. Have fun playing
7. Delete Uncompressed rom whenever you want to play a different game.
8. Repeat steps 3-8.

EDIT: It would be impossible to run a compressed rom without uncompressing. DS cards require a fast-ish response time. So basically every game might load, but would freeze instantly.

 

[D-Trogh]

Can you quit posting utter bullshit please. The only person who made a sensible reply was maikelsteneker. This has NOTHING to do with emulators. In case you hadn't have already noticed, this is the R4 and flashcart forum and nowhere in my post did I mention emulators.

Can't you be a lil' more polite please - -' !!
I noticed we were in the R4-board when I was posting, so I couldn't change it anymore..
Ow, the edit button.. yeah.. I clicked it, but the page doesn't wanted to load.. so I went to another site..

But.. If you want an asnwer for the R4 - -'
The R4-team just need to write a compressor themselves and update the firmware version.
The compressor uses an own format (or a known one like .zip, .rar, ..) and the R4 just reads the compressed files.

Edit (whooh, it loaded!):
This will be quicker than an uncompressor on the R4

 

[FAST6191]

Various carts (the RAM based ones) use simple compression like LZ/huffman/RLE (which the GBA/DS can decode with BIOS functions) for GBA but seen as DS roms currently are up at 128 megs and do not often require a fast bit of memory then no. You could decompress to empty space on the NAND/?SD but time taken would be horrendous .

Not to mention about 60% of roms these days already have their internal code and/or pictures/sound.... compressed: compression on top of compression does not give great results: pull apart a DS rom and see how many files have LZ in their name or start of the file.

I dare say as most people maintain a clean set of roms the majority of compression seen is normally the junk bytes at the end of the rom (in various cases they can be almost half the rom: snowboard kids USA is 34 megs after trimming if memory serves).

 

[Retal]

Not to mention about 60% of roms these days already have their internal code and/or pictures/sound.... compressed: compression on top of compression does not give great results: pull apart a DS rom and see how many files have LZ in their name or start of the file.

I can positively assert that "additional" compression would not be a waste of time. I can typically reduce ROM sizes by well over 50% using various algorithms.

 

[rest0re]

Various carts (the RAM based ones) use simple compression like LZ/huffman/RLE (which the GBA/DS can decode with BIOS functions) for GBA but seen as DS roms currently are up at 128 megs and do not often require a fast bit of memory then no. You could decompress to empty space on the NAND/?SD but time taken would be horrendous .

Not to mention about 60% of roms these days already have their internal code and/or pictures/sound.... compressed: compression on top of compression does not give great results: pull apart a DS rom and see how many files have LZ in their name or start of the file.

I dare say as most people maintain a clean set of roms the majority of compression seen is normally the junk bytes at the end of the rom (in various cases they can be almost half the rom: snowboard kids USA is 34 megs after trimming if memory serves).

as far as i know trimming doesnt have anything to do with compression. its just simply removing "empty" junk data...

 

[FAST6191]

@Retal I agree LZ, RLE and huffman are not great space saving wise (they are fast though) and additional compression can have positive effects, it is just a bit less. I also contend that space saving does mainly come from the junk data being compressed (see below).

@rest0re trimming has nothing to do with compression I agree, when however the last 14 megabytes are all 0000 0000 0000...... which can be summed up in a few bytes you see a massive gain in compression for your 7z/zip/rar/whatever file.

 

[Retal]

Just to cite a few examples of trimmed roms that I have then compressed with RAR compression:

Castlevania - Portrait of Ruin (E) 50MB -> 25MB
Digimon World (U) 57MB -> 18MB
Phoenix Wright - Justice For All (E) 32MB -> 13MB
Picross (E) 33MB -> 4MB
Pokémon Diamond (U) 61MB -> 18MB
Sim City (E) 56MB -> 7MB

I think there's no more need to question whether compression would be a valuable space saving endeavour.

So back to the topic question, would it be feasible to support compression on flashcarts. As arctic_flame mentioned, one method is to decompress before execution which would take time. Alternatively, as maikelsteneker mentioned, it may be possible to decompress on-the-fly but would require memory on the flashcart.

 

[OSW]

well storage is always getting cheaper, most people these days wouldn't care much for compression features on a DS cart.

Basically summarising what other people have said, the space saving benefit does not outweigh the time disadvantage.

 

[Retal]

Figures.

 

[arctic_flame]

If you're desperate, I (anybody who can compile for DS) could have a go at porting UnRAR. to DS...

EDIT: z0mg typos!

 

[rest0re]

Just to cite a few examples of trimmed roms that I have then compressed with RAR compression:

Castlevania - Portrait of Ruin (E) 50MB -> 25MB
Digimon World (U) 57MB -> 18MB
Phoenix Wright - Justice For All (E) 32MB -> 13MB
Picross (E) 33MB -> 4MB
Pokémon Diamond (U) 61MB -> 18MB
Sim City (E) 56MB -> 7MB

I think there's no more need to question whether compression would be a valuable space saving endeavour.

So back to the topic question, would it be feasible to support compression on flashcarts. As arctic_flame mentioned, one method is to decompress before execution which would take time. Alternatively, as maikelsteneker mentioned, it may be possible to decompress on-the-fly but would require memory on the flashcart.

Dear sir, you are an idiot!
Seriously. You must understand what RAR does. It compress file with powerful algorhytm which replaces repeating sequences in with simpler algorhytm to describe them. Doing this "realtime" required much more power than DS has. See for example game needs to find information end of file or middle. It must compress file and it causes swapping at media. So it would not be A) Useful B) Requires more power than DS can handle. DS is now barely handling streaming games from your R4DS so imagine DS working streaming some hardcoded RAR package. Ask any #DSDEV channel coders and they laugh at your face. So... get real or live in fantasy world.

 

[pduin]

besides additional memory uncompressing needs a lot of computing power. you will have to put memory and a dedicated decompression chip on the cart to be able to run compressed roms. it is certainly possible but it will cost too much and take too much space on a cart.

 

[Glacius0]

This conversation is so pointless. But hey, I have nothing better to do, so:

A flash cart with GBA support may be able to run compressed roms as long as they're under 32mb (meaning they fit in the cart it's RAM). And even then, as mentioned before, you do indeed still need to use a simpler compression method than RAR. Let's say this saves you maybe optimistically 10 mb per 32 mb rom. Guessing from the loading of GBA roms, load times would be around a minute. Would anyone care for this?

 

[rest0re]

People whine and bitch and require all kind of things for flashcarts? What next, should flashcart take out your dog, make you coffee, fuck your girlfriend while you play Mario Kart? SNES backup times you could fit one game into backup device and you had to use disks to load it