Codec LZSS DS released

Discussion in 'NDS - ROM Hacking and Translations' started by FAST6191, Jan 14, 2011.

Jan 14, 2011

Codec LZSS DS released by FAST6191 at 4:25 PM (9,800 Views / 0 Likes) 19 replies

  1. FAST6191
    OP

    Reporter FAST6191 Techromancer

    pip
    Joined:
    Nov 21, 2005
    Messages:
    21,727
    Country:
    United Kingdom
    Browsing around the internet I came across this and as we are often in need of capable LZSS decompression tools I thought I would link it up.

    For those not familiar the GBA and DS are fairly powerful consoles and as such can afford to use compression fairly extensively and as such they make extensive use of it.
    Probably the most popular compression type is LZSS and there are several variations upon it usually noted by a flag/hexadecimal magic stamp in the initial bytes of compression or just before.

    type 10 is the most common and oldest- if I am not mistaken it is the method that the BIOS SWI calls can handle. Most LZSS compression tools you will find geared towards GBA and DS stuff use this method.
    type 11 appeared some time back and was incompatible with type 10, crystaltile2 was among the first applications to support it.
    type 40 according to the NFO at least appeared recently in the DS release of Golden Sun.

    It supports both compression and decompression of these compression types.

    There are other types out there (Sylvanian Families had a writeup of the custom compression somewhere) and you should note that even if it is BIOS compatible compression the SWI calls will not always be used (they are a bit slow on occasion).

    I have not had time to put it through much but a new tool is always nice to have, if it saves having to load up and beat the options of crystaltile2 into shape that is even better- naturally it is fairly limited in scope so if you have say compressed files within an archive (I have seen several NARC files like this in the past and some of the big package formats used by some games also use it) but the header for the file is standard/uncompressed you might run into trouble- it is not hard to slice it up and feed it to the tool but if you were coming from some of the compression searching tools or are trying to use it on the GBA it might take a bit of getting used to.
    According to the NFO however it should support standard ARM9.bin and overlay compression methods.

    Basic screenshot
    [​IMG]

    Readme (In French)
    Code:
    Codec LZSS DS
    
    
    Auteur : Loki
    
    Version du : 26 / 12 / 2010
    
    
    Ce codec peut compresser et décompresser des fichiers avec les algorithmes LZSS de
    type 0x10, 0x11 et 0x40. Il peut aussi décompresser les fichiers arm9.bin et overlay.bin.
    
    Le type 0x40 n'est autre que la protection apparue pour la première fois le 28 octobre 2010
    dans la version japonaise du jeu Golden Sun: Dark Dawn / Obscure Aurore.
    
    
    Bonne traduction...
    Filetrip download
    http://filetrip.net/f23640-Codec-LZSS-DS-1-0.html

    My source
    http://nintendo-ds.logic-sunrise.com/news-...ichiers-ds.html
    Author/project homepage
    http://myth-project.legtux.org/
     
  2. Poryhack

    Member Poryhack GBAtemp Fan

    Joined:
    Oct 18, 2009
    Messages:
    330
    Country:
    United States
    Thanks for posting this FAST, it's good to finally find something other than CrystalTile2 to do overlay decompression with. I wonder if it would be possible to get the source; I have a friend who I think would like to implement something like this on Linux.
     
  3. loco365

    Member loco365 GBAtemp Guru

    Joined:
    Sep 1, 2010
    Messages:
    5,459
    If I could get the source, I could make an attempt to translate this to English for you all.
     
  4. FAST6191
    OP

    Reporter FAST6191 Techromancer

    pip
    Joined:
    Nov 21, 2005
    Messages:
    21,727
    Country:
    United Kingdom
    There is not really any point in a translation- this is the main screen
    [​IMG]

    , actually I might edit that into the first post.

    Only thing to note is it seems to want files with the extension LZSS or LZ (or you type *.* in the name box) but that is nothing new to rom hacking (many times do I force things past extension checks).

    As for source for porting purposes afraid I am not aware of any, still http://www.gbadev.org/tools.php?showinfo=56 has some source (mainly for type 10/GBA bios style), there is http://www.romhacking.net/utils/431/ , http://www.romhacking.net/utils/495 , http://www.romhacking.net/utils/511/ (this one has a measure of source and seems to be built around .net) and http://www.romhacking.net/docs/281/ (a nice document on the basics of LZ)
     
  5. Barubary

    Newcomer Barubary Member

    Joined:
    Feb 27, 2009
    Messages:
    28
    Country:
    United States
    If you just want to decompress, you can also use my tool, DSDecmp. All three LZ-formats are supported, as well as RLE and Huffman compression.
    It is command-line (or drag&drop) only, but the source is freely available.
     
  6. FAST6191
    OP

    Reporter FAST6191 Techromancer

    pip
    Joined:
    Nov 21, 2005
    Messages:
    21,727
    Country:
    United Kingdom
    Well this is embarrassing, I had forgotten about your tool Barubary (despite actually using it on several occasions).
     
  7. Poryhack

    Member Poryhack GBAtemp Fan

    Joined:
    Oct 18, 2009
    Messages:
    330
    Country:
    United States
    I didn't realize that tool was yours, excellent work! I've used it many times and I was about to bring that up in my earlier post until I saw that this new tools supports compressed ARM9 and ARM9 overlays. If you could add support for that in your tool it would be amazing.
     
  8. Phoenix Goddess

    Member Phoenix Goddess The Ninja's Protégée

    Joined:
    Apr 25, 2009
    Messages:
    3,809
    Location:
    Away from civilization.
    Country:
    United States
    If I could hug you for this FAST, I would.
    Thank you [​IMG]


    Barubary: This is the first time I'm seeing your tool and I wish I had spotted it sooner T_T
    Thank you for your work! It's amazing [​IMG]
     
  9. Barubary

    Newcomer Barubary Member

    Joined:
    Feb 27, 2009
    Messages:
    28
    Country:
    United States
    If anyone has a specification of the compression used in the ARM9 and overlay files, I'd gladly incorporate it.
    I know the compression used is the plain LZ (0x10?) but in reverse (read and write from end to start). However since only a part of the files are compressed I'll need to know the compression header format. I could try to figure it out myself, but it'd be much faster if there was already some documentation available =)
     
  10. Naxito

    Newcomer Naxito Advanced Member

    Joined:
    May 14, 2008
    Messages:
    63
    Location:
    Spain
    Country:
    Spain
    What about 0x30 in some files on Desktop Tower Defense?
     
  11. Poryhack

    Member Poryhack GBAtemp Fan

    Joined:
    Oct 18, 2009
    Messages:
    330
    Country:
    United States
    Turns out this tool was written in C#, so I decompiled it. You can download the source here. All the interesting stuff is happening in Form1.cs FYI.
     
  12. Barubary

    Newcomer Barubary Member

    Joined:
    Feb 27, 2009
    Messages:
    28
    Country:
    United States
    Your link is a 404, but I found a decompiler and did it myself. Since the format was so similar to 0x10 LZ, I've added support for it right away. =)

    EDIT: whoops, I was too eager. One or more small bugs are still present, which is/are currently being fixed.
     
  13. FAST6191
    OP

    Reporter FAST6191 Techromancer

    pip
    Joined:
    Nov 21, 2005
    Messages:
    21,727
    Country:
    United Kingdom
    Nice work both of you.

    I probably should kick this to another thread but
    Been doing a lot of reading today on various aspects of rom hacking, things have moved on a lot in recent months not to mention all the Chinese hacking work/docs that have appeared. In many ways it seems I have been caught sleeping and frankly I could not be more exited about it.

    Today's reading
    http://www.romhacking.net/forum/index.php/topic,8407 (I have linked it many times but a lot will has happened in the last couple of months).
    http://bbs.cntgb.com/read.php?tid=541 a bit but check the surrounding site.

    Also new version of crystaltile2 up on filetrip http://filetrip.net/f23649-CrystalTile2-2010-09-06.html , it seems some vestiges of the "registered" version were not removed from the initial return to normalcy, this should correct that (frankly though despite being one of my go to tools I had not noticed).

    Oh and if you all are still wanting compression stuff it is an old one but a good one
    http://members.iinet.net.au/~freeaxs/gbaco...sion%20on%20GBA

    Oh and for those mourning the loss of tahaxan's image format stuff http://llref.emutalk.net/docs/

    Short version- I need to do a lot more work on my rom hacking docs to have them stand side by side with current stuff.
     
  14. Barubary

    Newcomer Barubary Member

    Joined:
    Feb 27, 2009
    Messages:
    28
    Country:
    United States
    I've fixed the bugs in my decompression routine (at least the ones that I could find). There's also a 'hack' for files that have a value out of range. It may be that this hack is actually the proper way to do it (the files I've checked that have that problem do seem to come out properly), but I can't be sure. When using the command line, there will be a message 'DISP is too large [...]' if this happens. 'Codec LZSS DS' just crashes when that problem arises.
    Anyway, the project link is still the same.
     
  15. Poryhack

    Member Poryhack GBAtemp Fan

    Joined:
    Oct 18, 2009
    Messages:
    330
    Country:
    United States
    Oh sorry about that, not exactly sure how it happened but it got deleted from my Dropbox folder. It's back now in case anyone is still interested. Very glad to hear that you implemented it so quickly Barubary.
     
  16. Naxito

    Newcomer Naxito Advanced Member

    Joined:
    May 14, 2008
    Messages:
    63
    Location:
    Spain
    Country:
    Spain
    Any program from RLE? :S
     
  17. Barubary

    Newcomer Barubary Member

    Joined:
    Feb 27, 2009
    Messages:
    28
    Country:
    United States
     
  18. Naxito

    Newcomer Naxito Advanced Member

    Joined:
    May 14, 2008
    Messages:
    63
    Location:
    Spain
    Country:
    Spain
    I'm trying to descompres with option RLE but only copy the file :/

    Command: dscecmp -nr file43.ncgr "./descompressfolder"
     
  19. Barubary

    Newcomer Barubary Member

    Joined:
    Feb 27, 2009
    Messages:
    28
    Country:
    United States
    (although you don't need the "-s unless the path contains a space)
     
  20. Naxito

    Newcomer Naxito Advanced Member

    Joined:
    May 14, 2008
    Messages:
    63
    Location:
    Spain
    Country:
    Spain
    Thx!. And to recompress? ^^''
     

Share This Page