ROM Hack New version of 'DSDecmp'

Barubary

Active Member
OP
Newcomer
Joined
Feb 27, 2009
Messages
28
Trophies
0
Website
github.com
XP
143
Country
United States
I figured this would be the best place to post this; I've made a new release of my decompressor of NDS compression formats, DSDecmp. (visit the project page, including download link, here)

With this new release the name has become somewhat misleading, since it can now also compress files back into any of the supported formats.

It's a command-line only tool, so I'll just copy/paste its usage here (which can be recalled at any time by not providing any arguments);
Code:
DSDecmp - Decompressor for compression formats used on the NDS - by Barubary

Usage:ÂÂDSDecmp (-c FORMAT FORMATOPT*) (-ge) input (output)

Without the -c modifier, DSDecmp will decompress the input file to the output
file. If the output file is a directory, the output file will be placed in
that directory with the same filename as the original file. The extension will
be appended with a format-specific extension.
The input can also be a directory. In that case, it would be the same as
calling DSDecmp for every non-directory in the given directory with the same
options, with one exception; the output is by default the input folder, but
with '_dec' appended.
If the output does not exist, it is assumed to be the same type as the input
(file or directory).
If there is no output file given, it is assumed to be the directory of the
input file.

With the -ge option, instead of a format-specific extension, the extension
will be guessed from the first four bytes of the output file. Only
non-accented letters or numbers are considered in those four bytes.

With the -c option, the input is compressed instead of decompressed. FORMAT
indicates the desired compression format, and can be one of:
--- formats built-in in the NDS ---
ÂÂÂÂlz10ÂÂ- 'default' LZ-compression format.
ÂÂÂÂlz11ÂÂ- LZ-compression format better suited for files with long repetitions
ÂÂÂÂlzovl - LZ-compression used in 'overlay files'.
ÂÂÂÂrleÂÂ - Run-Length Encoding 'compression'.
ÂÂÂÂhuff4 - Huffman compression with 4-bit sized data blocks.
ÂÂÂÂhuff8 - Huffman compression with 8-bit sized data blocks.
--- utility 'formats' ---
ÂÂÂÂhuffÂÂ- The Huffman compression that gives the bext compression ratio.
ÂÂÂÂnds*ÂÂ- The built-in compression format that gives the best compression
ÂÂÂÂÂÂÂÂÂÂÂÂratio. Will never compress using lzovl.
ÂÂÂÂgba*ÂÂ- The built-in compression format that gives the best compression
ÂÂÂÂÂÂÂÂÂÂÂÂratio, and is also supported by the GBA.

The following format options (FORMATOPT) are available:
lz10, lz11, lzovl, gba* and nds*:
ÂÂÂÂ-optÂÂ: employs a better compression algorithm to boost the compression
ÂÂÂÂÂÂÂÂÂÂÂÂratio. Not using this option will result in using the algorithm
ÂÂÂÂÂÂÂÂÂÂÂÂoriginally used to compress the game files.
ÂÂÂÂÂÂÂÂÂÂÂÂUsing this option for the gba* and nds* will only have effect on
ÂÂÂÂÂÂÂÂÂÂÂÂthe lz10, lz11 and lzovl algorithms.

If the input is a directory when the -c option, the default output directory
is the input directory appended with '_cmp'.

Supplying the -ge modifier together with the -c modifier, the extension of the
compressed files will be extended with the 'FORMAT' value that always results
in that particualr format (so 'lz11', 'rle', etc).
If the -ge modifier is not present, the extension of compressed files will be
extended with .cdat
 

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,321
Country
United Kingdom
I do like new compression tools and even before this yours was one of the best.

Would you rate your source as relatively easy to adapt/turn into a library?- as we delve deeper into the DS and GBA libraries custom compression and more often formats are starting to call for tools to be built rather than methods cobbled together from other tools and it is nice to have a thing to point at in such cases.
 

Barubary

Active Member
OP
Newcomer
Joined
Feb 27, 2009
Messages
28
Trophies
0
Website
github.com
XP
143
Country
United States
Arshes91 said:
hmm is just like an nds rom trimmer my friend? i like the tile
I believe an nds rom trimmer only removes empty space from roms, and does not alter the actual content of the files within.
The compression algorithms in DSDecmp will compress files into formats found in both GBA and NDS games (and some Wii ones too I think).

QUOTE(FAST6191 @ May 15 2011, 06:33 PM) I do like new compression tools and even before this yours was one of the best.

Would you rate your source as relatively easy to adapt/turn into a library?- as we delve deeper into the DS and GBA libraries custom compression and more often formats are starting to call for tools to be built rather than methods cobbled together from other tools and it is nice to have a thing to point at in such cases.
I as programmer interpreted that in two ways;
- with a small tweak in the build, I can just as easily make a .dll which can be accessed by other tools to do the (de)compressing.
- With this new version, I've expressly tried to set up the project/code to allow more formats to be included relatively easily. It's not 'plug-and-play' though; the executable must be rebuilt for new formats to be included.
 

xuliko

Well-Known Member
Member
Joined
Oct 5, 2010
Messages
100
Trophies
0
XP
99
Country
I have a problem. I opened your program but quickly closed quickly the MSDOS Window... I drop overlay (okamiden game) but nothing... I extracted that archives with CT2.
 

redfalcon

GBATemp Member
Member
Joined
Feb 19, 2009
Messages
250
Trophies
0
Website
code.google.com
XP
206
Country
Gambia, The
xuliko said:
I have a problem. I opened your program but quickly closed quickly the MSDOS Window... I drop overlay (okamiden game) but nothing... I extracted that archives with CT2.

You have to run it from a cmd since you need to hand over some arguments to the program.
 

xuliko

Well-Known Member
Member
Joined
Oct 5, 2010
Messages
100
Trophies
0
XP
99
Country
redfalcon said:
xuliko said:
I have a problem. I opened your program but quickly closed quickly the MSDOS Window... I drop overlay (okamiden game) but nothing... I extracted that archives with CT2.

You have to run it from a cmd since you need to hand over some arguments to the program.

Ok... I have solutioned that.
wink.gif
Thanks for all.
However, it doesn't work. I tried many functions.
 

Barubary

Active Member
OP
Newcomer
Joined
Feb 27, 2009
Messages
28
Trophies
0
Website
github.com
XP
143
Country
United States
xuliko said:
redfalcon said:
xuliko said:
I have a problem. I opened your program but quickly closed quickly the MSDOS Window... I drop overlay (okamiden game) but nothing... I extracted that archives with CT2.

You have to run it from a cmd since you need to hand over some arguments to the program.


Ok... I have solutioned that.
wink.gif
Thanks for all.
However, it doesn't work. I tried many functions.

What is it you are trying to do?
To decompress overlay files:
Code:
dsdecmp overlay_XXXX.bin overlay_XXXX_decompressed.bin
To compress them again (may not result in identical files when run on an unmodified decompressed file, unfortunately):
CODEdsdecmp -c lzovl overlay_XXXX_decompressed.bin overlay_XXXX.bin
or
CODE
dsdecmp -c lzovl -opt overlay_XXXX_decompressed.bin overlay_XXXX.bin
 

Bug_Checker_

Well-Known Member
Member
Joined
Jun 10, 2006
Messages
950
Trophies
0
XP
664
Country
United States
1st my sincerest apologies for necro-bumping this thread:
With code.google.com going the way of the dodo bird and Barubary going M.I.A, I felt (for the sake of history) that these links should be immortalized.

The new(fall of 2015) link "Automatically exported from code.google.com/p/dsdecmp" is:
https://github.com/barubary/dsdecmp
Source code:
https://github.com/Barubary/dsdecmp/archive/master.zip
Commits (or changes):
https://github.com/Barubary/dsdecmp/commits/master

Last known original and archived home page was at:
https://web.archive.org/web/20141223223529/https://code.google.com/p/dsdecmp/

Last known original and archived download page was at:
https://web.archive.org/web/20140406024405/https://code.google.com/p/dsdecmp/downloads/list

DSDecmp5alph2.zip 33.7 KB

Description:
Pre-release of DSDecmp v5, including game-specific formats for Luminous Arc and Golden Sun: Dark Dawn.
Also re-added support for NULL-'compression'.
There may still be some bugs.
SHA1 Checksum: a19e85d2ea6ac1645b37877b9f18b9fe59fc7b35

https://web.archive.org/web/2014040...nloads/detail?name=DSDecmp5alph2.zip&can=2&q=

DSDecmp5alph.zip 19.5 KB

Description:
Pre-release of DSDecmp v5. There may still be some bugs.
Be aware that the usage of the program has changed since v4. Provide no arguments to see the new usage.

(this release does not yet contain any game-specific plugins)
SHA1 Checksum: 3de71574083a4fa09f9bdbeab4844839c10c211c
https://web.archive.org/web/2014040...wnloads/detail?name=DSDecmp5alph.zip&can=2&q=

DSDecmp4.zip 14.9 KB

Description:
Contrary to previous versions, as of V4 DSDecmp can also compress back into any supported format.

Support for 'LZ-0x40' has been dropped, as it appeared to be a one-game-only format.

This is a build of r44.
SHA1 Checksum: 5f261bb50f60359ba3b454ffcd69ab63782f5678
https://web.archive.org/web/2014040...p/downloads/detail?name=DSDecmp4.zip&can=2&q=

A compressor and decompressor for the compression formats commonly used in games made for Nintendo consoles/handhelds.

Supports the following formats:

  • LZ77/LZSS (types 0x10, 0x11 and 'Overlay')
  • Huffman (only data lengths 8 and 4)
  • Run-Length Encoding


'Overlay' LZ compression is used in the 'overlay' files of a game, as well as its arm9 binary. If DSLazy is used to unpack the game, these files are called overlay_X.bin (with X any number, located in the overlay/ folder) are arm9.bin respectively.

Source is available for both Java and C#. However Overlay LZ do not currently have a Java-implementation, and only the C# implementation can compress files.

If you want to decompress files from Golden Sun: Dark Dawn, use version 3b, found under the deprecated downloads, or version 5 alpha.
 

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,321
Country
United Kingdom
Barubary responded to a PM back in August and gave me the links to the github that the google stuff exported from. Google code is not quite dead either, just went read only from what I have seen (you can still get to http://tiledggd.googlecode.com/svn/trunk/TiledGGD/ and http://dsdecmp.googlecode.com/svn/trunk/ for instance)

Others reading though dsdecmp is still a valuable tool and worth having in your kit and tileggd is as valuable as ever for compression a lot of people have moved to http://www.romhacking.net/utilities/826/ for the day to day stuff. The both do the same thing but cue's stuff handles failure and some edge cases a bit more gracefully, though there have also been a couple of cases where dsdecmp worked and that came up short.
 

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,321
Country
United Kingdom
It (and the Cue's stuff we suggest for most uses) are command line tools.

While you are probably used to clicking on things and getting a GUI then there are other ways of running code on Windows (and other operating systems). The classic way is the command line/command prompt (terminal on Linux systems).
Here you type commands and modifiers to them so as to run the program and have it do what you want it to. Some consider it old and outdated, those people clearly have never had to do a lot of work, very specific but repetitive tasks, and automate it/replicate it for 1000+ items (which is easy with a ROM) though.

Double clicking such programs would used to flash up a window saying this needs to be run in a command line but modern stuff will flash that up quicker than most will read it.

https://www.bleepingcomputer.com/tutorials/windows-command-prompt-introduction/

Most will extract the program to a directory (either in the unpacked ROM or copy the files from the unpacked ROM to that directory), run the command line program, navigate to that directory (this would be the CD, as in change directory, command) and then run the program they want (DSdecmp if you are going that way, what extras you can add are detailed in the downloads and opening post).

If you put a list of said commands in a text file and save it with the extension .bat then you can run the .bat (short for batch file, as in batch of commands to do) the computer will attempt to execute them all one after the other. Chain it with another tool ( https://web.archive.org/web/20170218180937/http://min.midco.net/cracker/filecutter.zip is one I often use with decompression efforts as it can slice out just the part I care about) and you can start doing quite complicated things very simply without having to program your own tool to do it.

Alternatively you could try Crystaltile2 or one of the various GUI programs. They will often have compression support built in you can decompress things by clicking (in the case of Crystaltile2, aka CT2, it will be after you open the ROM, click the little DS icon and in the following menu navigate to the file you want and right click on it, it will probably have a little icon saying LZ or something to do with compression) options from there should be fairly obvious (or at least there are two chances).
 
  • Like
Reactions: artsypencil

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
    SylverReZ @ SylverReZ: https://youtu.be/ngNaUaQOfPU