Hacking DS2Compress: gzip and unzip with MORE JUICE

[Deleted member 319809]

If anyone here knows about DSCompress for compressing and decompressing their files on a DS, well I thought it was slow for the DSTwo and it updated its progress incorrectly, often wrapping to -100% multiple times while processing a file.

So I made:

DS2Compress 0.62! (with source on GitHub)

CAVEATS (should be gone as of 0.62, but see below for what 0.61 did, and 0.62 might still do)
* DO NOT unpack files without extensions (such as README and ChangeLog) from .zip files, or uncompress .gz files that have no extension before .gz. They corrupt the filesystem and crash applications that read your storage card!
* Unpacking some files creates a phantom file beside the real one. In certain circumstances, these phantom files may cause your computer to ask you to reformat your storage card. Do not reformat your card! In certain circumstances, the files can cause other real files to fail to be created correctly.
* Please make backups of files on your hard drive if you're not sure.

Relatedly: /NDS/Flashcarts and Accessories/Supercard/Supercard SDK/Does filesystem access just completely suck on the DS2?

What can I use this for?
You can use this to make ROMs you're not currently playing, or some real-time saves you don't need anymore, take up less space on your storage card. You can use this on real-time saves for the Super Nintendo and Game Boy Advance emulators, too! Basically, anything you want to move out of the way.

You can also use this to unpack DSTwo Real-Time Menu skins, in .zip format, so that they become the active skin. You need to do this every time you change skins in the DSTwo EOS (DS_GAME) if you want the RTM and cheats menu to match.

Timings
I have tested this application with a few files, and a good benchmark is NEW_MARIO_AD2E.NDS, freshly dumped (32,768 KiB). Here are some timings and compression ratios for each gzip compression level found in DS2Compress:

 

[Deleted member 319809]

DS2Compress 0.60 is now released. It adds the following:

* .zip: Skip archive members that end with a slash. This would cause DS2Compress 0.50 to crash.
* .zip: Show the name of the archive member being unpacked for progress indication.
* .zip: Make missing directories when unpacking a file with a path.
* .gz, .zip: Decompression speed improvements.

Version 0.60 is actually suitable for unpacking theme .zip files into /_dstwo/plug and /_dstwo/patch.

 

[Boriar]

Please, check Github. I have did a spanish translation but i can't download 0.60 to test it.

 

[Deleted member 319809]

How odd, I swear I updated the link... I'll update it again.

Edit: Done. You can test it first in the English block if you like, then I'll hook it into the Spanish block.

 

[Boriar]

Downloaded.
I will try to test it tomorrow but i think that in english block may change accentuated characters and "¿" like occurred me in CATSFC.

 

[Boriar]

Finally tested.
I have sent it on Github. Was a pain to fit that very long error messages into a little windows without much changes.
I hope you like it.

 

[Deleted member 319809]

If it was bad, then perhaps the best solution is for me to enlarge the message box image, because other languages will come along that are much longer than Spanish, such as Dutch and German. If you shortened messages a lot, I may ask you to "un-shorten" the messages and instead I'll enlarge smsgfr.bmp, after getting input from Aeter and SignZ.

 

[Boriar]

Why don't you introduce a scape character (something like \n) to let advance to next line but in showed text?
May be more convenient to adapt text instead of use spaces like i use by the moment or simple check if a word is break in two lines and then write complete in the next line.
With the first option i will be happy

 

[Deleted member 319809]

The code does refer to adding newlines (when reading the language file) and cutting text when a newline is encountered (when displaying text), but it doesn't seem to work. If it worked, you could put the two characters "\n" and then a newline in the language file, and it would do what you said. I'll need to investigate this.

 

[Deleted member 319809]

Why don't you introduce a scape character (something like \n) to let advance to next line but in showed text?
May be more convenient to adapt text instead of use spaces like i use by the moment or simple check if a word is break in two lines and then write complete in the next line.
With the first option i will be happy

DS2Compress 0.61, the Languages Edition, is now released. Your Spanish localisation is included, as well as SignZ's German localisation. The message boxes are enlarged, and centering text now works properly with multiline messages, centering the text on all lines.

Additionally, text centering now cuts the text around the last space that fits a line, and the three-character sequence "\" "n" <Enter> is now supported for language.msg.

So now, this message:
 
#MSG_DIALOG_OVERWRITE_EXISTING_FILE
El fichero de destino ya existe. ¿Sobrescribirlo?
 
looks like
 
  El fichero de destino ya existe.
          ¿Sobrescribirlo?
 
instead of
 
El fichero de destino ya existe. ¿Sob
rescribirlo?

The spaces are no longer needed to align things *or* cut them properly. I will implement this for CATSFC's next release too.

 

[Boriar]

Many, many thanks!!!
That simplifies my translations a lot

 

[Boriar]

A question.
Is it posible to let multiple files compressed in one archive? May be interesting to compress together the game and the save file for example.

 

[Deleted member 319809]

A question.
Is it posible to let multiple files compressed in one archive? May be interesting to compress together the game and the save file for example.

That would massively complicate the user interface. You'd have to have a button to add a file and continue, one to go back one directory, one to enter a directory, another to add the directory to the archive instead, one to make the archive and one to cancel archive creation. A help screen on the upper screen would be required in case of confusion, as well as a list of the files to be added. That's in case you don't remember whether you added a file to your archive already.

Key assignments could be:
A = Add file or add directory (confusing, because A is used to enter a directory in all similar file-selector interfaces in DS homebrew)
B = Go back one directory
Y = Enter directory
Start = Create archive
X = Cancel archive creation

-or-
A = Add file or enter directory (confusing, because Y is used for adding directories and A for adding files)
B = Go back one directory
Y = Add directory
Start = Create archive
X = Cancel archive creation

-or-
A = Enter directory
B = Go back one directory
Y = Add file or directory (slightly confusing, because A doesn't seem to "do anything", and it's usually supposed to be the action button)
Start = Create archive
X = Cancel archive creation

I think you'd be better off making zip files on your PC On a DS, .gz files are easier to select one file for.

 

[Boriar]

I don't think in something like this. Like you say, too complicate.
Only selecting file/s in the same directory. Save files usually remain near the game vinvulated

Edit:
X: select file (an asterisk)
Y: unselect file

When you have select all file, only need press A without need to be over a file to compress.
It is a really good option to put the name of selected files on upper screen with a message to indicate:
*A to compress only selected files

 

[Deleted member 319809]

Oh... that's much simpler then. Upon adding a file, a message could ask "Add another file to make a .zip archive?" with A as Yes and B as No, and bring up the file selector again if Yes is selected. Is that closer to what you thought?

 

[Boriar]

Sorry, i have edited my post at the same time you read it.
Please re-read again.

 

[Deleted member 319809]

Ah. Well, X is already used to cancel the current file selection altogether and return to the main menu. Perhaps Y to toggle selection, then. I still doubt .zip creation will be fast or user-friendly on a DS, though...

 

[Boriar]

(I do a new post because that is another issue)
Again, about the name of the zip, to simplifier you may let choose what of selected files would be the name of future zip when user press A to initiate the compression. It will be directly on upper screen with an arrow near the name and use direcctional buttons to move it. Of course with a little advise to the user, something like:
Please, select the name of file to use for zip archive

 

[The Real Jdbye]

* DO NOT unpack files without extensions (such as README and ChangeLog) from .zip files, or uncompress .gz files that have no extension before .gz. They corrupt the filesystem and crash applications that read your storage card!

You shouldn't release homebrew with bugs like that. I remember when DSOrganize corrupted my MicroSD and I lost all my saves. I was pissed off for months.
Edit: That's also a really weird bug. Must be something caused by your coding style.

 

[Deleted member 319809]

You shouldn't release homebrew with bugs like that. I remember when DSOrganize corrupted my MicroSD and I lost all my saves. I was pissed off for months.
Edit: That's also a really weird bug. Must be something caused by your coding style.

When I extract a file, I tack on the path containing the archive, plus a slash, plus the archive member's name, to a new string, then use the very regular 'fopen' function from the C library to open the file. If you'd like to check, see the ZipUncompress function in source/miniunz.c.

This seems like a bug in the Supercard SDK. For the longest time, people have reported corruption in the /CATSFC/GAMERTS directory with CATSFC's saved state creation; it has been mitigated somewhat by the fact that all of CATSFC's files have extensions.