Hacking I need info about NDS compression

[BelugaDamiens]

Hi, I've been trying to unpack a custom file type (.ccb) which is used in a NDS ROM (Solatorobo). Literally almost all the ROM content is inside of that file type.
The thing is that I made a little program which reads those files, and give me a list of files contained in them, and their compressed data. And there is the problem... The compressed data.

I've heard that it's compressed using LZ compression of NDS, but I can't find any information about it. I searched on google, but I can only find LZW, LZSS, LZ77 among other types of similar compressions, and I am not so sure that they are the same which uses NDS (probably not, i guess). So I would be grateful, if someone could provide me information about the NDS compression...

I'd add a little compressed file hex content to this thread, but i'm not sure if I can (because rules and everyting that), i'm not used to write threads...

This is the output of the program I've made, with an example ccb file:

File size: 147144 Bytes
Number of files: 3
Files:
|
|--File name: s2air06bkg00.nsbmd
|--File type: 4
|------Compressed file size: 528 Bytes
|----------Compression type: 16
|------Uncompressed file size: 1056 Bytes
|
|--File name: s2air06bkg00.ccbcl
|--File type: 0
|------Compressed file size: 336 Bytes
|----------Compression type: 16
|------Uncompressed file size: 944 Bytes
|
|--File name: s2air06bkg00t.nsbtx
|--File type: 5
|------Compressed file size: 146176 Bytes
|----------Compression type: 16
|------Uncompressed file size: 262820 Bytes

Since I don't know how many types of compression types can use those files, and their names, I used the numeric id to show the compression type. I didn't see another compression type that is not 16, though.

Thanks in advance!!

 

[FAST6191]

Is 16 the hex conversion of 10h? Type 10 is common as you like (even in the BIOS) but 16 would be new to me (11 and 40 in some later games being the main LZ types).

https://www.romhacking.net/utilities/826/ has some source code to various formats (BLZ is the one used in the binaries/overlays).
http://members.iinet.net.au/~freeaxs/gbacomp/ https://problemkaputt.de/gbatek.htm#biosdecompressionfunctions if you wanted other links, I usually also link https://ece.uwaterloo.ca/~ece611/LempelZiv.pdf as a general overview of such things as well if you are new (despite the name it also serves as a nice intro to huffman).

The output there I think it reasonably clear. Looks like CCB is the archive format the devs of that game chose/made (rare on the DS but far from unheard of) for everything. Not sure why some devs did that (can't really be speed, could be poor understanding of the UI but I suppose in the end ours as the ROM hacker is to deal with it) and it has a NSBMD format file (common 3d model format), NSBTX (common 3d texture format) and not sure what ccbcl is but presumably another archive (cl possibly being an indicator of compression, or the full extension that got chopped in the rest of the game). Each being individually compressed makes sense (don't want to have to decompress all the contents to get only the one aspect).

 

[plasturion]

Found some info about CCS custom format used in this game somewhere else on the web:

Header
Offset Length Description
0x0 0x4 "CCB " (CyberConnect2Batch?)
0x4 0x2 Version
0x6 0x2 Number of files in archive
DATA File Table
DATA Archived files
File Table
0x0 0x18 Filename
0x18 0x1 File Type?
0x19 0x3 Compressed data size
0x1C 0x1 Compression type (NDS Compression)
0x1D 0x3 Uncompressed data size
Worked out by me, refined by Low Lines (of Console Tool fame). Yes, there's compression, but it's the standard built-in LZ.

 

[BelugaDamiens]

Is 16 the hex conversion of 10h? Type 10 is common as you like (even in the BIOS) but 16 would be new to me (11 and 40 in some later games being the main LZ types).

URL has some source code to various formats (BLZ is the one used in the binaries/overlays).
URL URL if you wanted other links, I usually also link [it doesn't let me reply with the url's xD] as a general overview of such things as well if you are new (despite the name it also serves as a nice intro to huffman).

The output there I think it reasonably clear. Looks like CCB is the archive format the devs of that game chose/made (rare on the DS but far from unheard of) for everything. Not sure why some devs did that (can't really be speed, could be poor understanding of the UI but I suppose in the end ours as the ROM hacker is to deal with it) and it has a NSBMD format file (common 3d model format), NSBTX (common 3d texture format) and not sure what ccbcl is but presumably another archive (cl possibly being an indicator of compression, or the full extension that got chopped in the rest of the game). Each being individually compressed makes sense (don't want to have to decompress all the contents to get only the one aspect).

Yeah, that 16 is decimal base, from 10h (I really have to change the program output to hex xD). Those files are from CyberConnect, I found their data structure some days ago in a forum which I don't remember rn... But it was so useful for doing the program xD
I'll check out those links, and see if I can make a LZ uncompressor...
About the nitro file types, I already messed with them before, so they shouldn't cause problems! (I hope so xD)
Thanks so much for all the info!!!

Post automatically merged: May 5, 2023

Found some info about CCS custom format used in this game somewhere else on the web:

Yeah, I already saw that, I used it for doing the program which I used to generate that output I've shown, indeed.
The problem is with the files compression inside the CCB.
Thank you anyway!!

 

[plasturion]

to decompress usualy i'm using tinke or crystaltile2 or compressors pack by CUE to use in batch wich contain sourcecode so you can check algorithm there are many variation of lz compression. i guess lzss could be the one you're looking for just extract compressed portion of data to seperate file and give a try if is valid it'll be decompressed.

 

[Reploid]

What do you want to do btw? Some cool mod?

 

[BelugaDamiens]

to decompress usualy i'm using tinke or crystaltile2 or compressors pack by CUE to use in batch wich contain sourcecode so you can check algorithm there are many variation of lz compression. i guess lzss could be the one you're looking for just extract compressed portion of data to seperate file and give a try if is valid it'll be decompressed.

I tried to create a separate file with the compressed data (extacted from the whole CBB file), and uncompress it with CUE, but it is not recognized as any LZ compressed file, which is strange because I read that it's the used compression...

Could it be because the CCB file is ommiting some part of the compressed data (like a header, magic word...), because it's always the same or something like that?

Anyway, i will check out the links which FAST6191 provided me in the answer, and I hope I find out something there XD

Post automatically merged: May 5, 2023

What do you want to do btw? Some cool mod?

I'm actually doing this because I wanted to rip some resources a while ago, but then I saw that it was all coded in those CCB, so now I want to extract them "for revenge" xD and for entertainment tbh...

Although it's possible that I try to modify something if I'm bored some day xD

 

[plasturion]

I see, that's what should be done. No rather not, magic word(number) seems like first byte of the compressed file and it's type. 10h is where the file starts (i gueess) and should be correspond to lzss compression type.. because in lzss.c we find:

#define CMD_CODE_10   0x10       // LZSS magic number

first i would comfirm if everything till the end will not be striped from any needed data (of that part of CCB file)
or if something lacks and have to be included inserted (header as you say) which seems to be strange to me.
another random link... here , if not work idk from this point.

 

[BelugaDamiens]

I see, that's what should be done. No rather not, magic word(number) seems like first byte of the compressed file and it's type. 10h is where the file starts (i gueess) and should be correspond to lzss compression type.. because in lzss.c we find:

#define CMD_CODE_10   0x10       // LZSS magic number

first i would comfirm if everything till the end will not be striped from any needed data (of that part of CCB file)
or if something lacks and have to be included inserted (header as you say) which seems to be strange to me.
another random link... here , if not work idk from this point.

Yes, I think that would be strange too... About that magic number which is a 0x10, since the FileTable of the CCB marks the data compression type as 0x10 too, and I noticed that all compressed data blocks starts with a 0x00, I tried to change that byte to a 0x10, but it didn't work, it still doesn't recognize it.
I've tought that another LZ type 10 compressed file would be good as reference, but I actually don't know where to find one with that specific type xD

 

[plasturion]

Yes, I think that would be strange too... About that magic number which is a 0x10, since the FileTable of the CCB marks the data compression type as 0x10 too, and I noticed that all compressed data blocks starts with a 0x00, I tried to change that bit to a 0x10, but it didn't work, it still doesn't recognize it.
I've tought that another LZ type 10 compressed file would be good as reference, but I actually don't know where to find one with that specific type xD

so you need reference of lz type 10, in example just compress any data like textfile, whatever ;]
lzss -evn [filename]

 

[BelugaDamiens]

so you need reference of lz type 10, in example just compress any data like textfile, whatever ;]
lzss -evn [filename]

Yeah, I did just that... I don't know how i didn't think that before xDDD

I've noticed that when I changed the first Byte to 0x10, CUE actually was trying to decode the file, but it didn't do it, because there had to be something wrong there in the file...

And... Finally got it!! The thing was that the compressed data inside the CCB, started with a 0x00 byte, which has to be replaced with the compression mode, and then after that byte, it have to be added four more bytes (little endian) which indicate the size of the uncompressed file... After doing that, CUE uncompressed it perfectly!!!

I tried with a nsbtx, and I opened it with a tool (Trifindo Pokemon DS Map Studio), it opened the file correctly!!

Thank you so much for the help!!

 

[FAST6191]

Never seen one with a chopped... let's call it header before. Interesting development, normally it is a custom/non standard flag, width or indicators.

Forgot to also say last time I did have a worked example in my hacking documentation, though Cue's source or dsdecmp source probably do better for most purposes there if you can already code.

 

[BelugaDamiens]

Never seen one with a chopped... let's call it header before. Interesting development, normally it is a custom/non standard flag, width or indicators.

Forgot to also say last time I did have a worked example in my hacking documentation, though Cue's source or dsdecmp source probably do better for most purposes there if you can already code.

I tried to add the CUE source code to my program, but it gives me problems because it uses some deprecated C code and the compiler doesn't want to work with it xD

Anyway i would have to do some changes on it for fitting it in my program, so I am finally porting the CUE source to my C++ code with all the necessary changes xDD