Hacking ROM Hack Help with unpacking an .LZ file

OffSetEyeBrow

Active Member
OP
Newcomer
Joined
Jun 16, 2018
Messages
35
Trophies
0
Age
21
Location
italy
XP
290
Country
Italy
I've been trying to get the map and enemy models from this game called "Kaijuu Busters Powered", it's a Japan-exclusive

image_2023-08-28_163220138.png

By checking the contents of the nds file, I realized that all the important files like textures and models are stored inside a ".lz" file, but I haven't been able to find any way to open it
LZip doesn't recognize the file as a normal LZ file that it uses
QuickBMS doesn't have any scripts that can open the said file, and I've run out of options
Does anyone know how I can visualize the contents of this file? I´ve included the lz file for map1 in the zip, if anyone wants to try it
 

Attachments

  • bg000a.zip
    23.6 KB · Views: 51

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,795
Trophies
5
XP
28,503
Country
United Kingdom
.lz is a common enough extension picked by devs presumably outputting a compressed file and needing an extension rather than being any kind of known file format (or formats in the case of things like .pac).

The DS (and GBA before it) feature compression algorithms in the BIOS which software can call upon, and most things use those or things compatible with them (some custom stuff does exist for the GBA but different discussion).

Many tools have been made
https://www.romhacking.net/utilities/826/
https://github.com/barubary/dsdecmp
Crystaltile2 (open it, click the DS icon on the top row, scroll down to the file you want, right click and options to decompress should be there, alternatively there are standalone tools within it if you play with the dropdown menus).

Once you have the decompressed file the real fun begins. Maybe it is a single file (kilobytes is small but could go either way) but if not and you have an archive then time to fiddle with such things.

I usually like to look for known formats in a hex editor ( https://www.romhacking.net/documents/[469]nds_formats.htm for an idea of how they tend to roll -- tend to get magic stamps/markers and file sizes) and then see if they relate to anything at the start or end of the file (you might expect file names, file sizes, locations of file, number of files contained within the archive). Alternatively open up in a tile editor and see what you can see.
 
  • Like
Reactions: OffSetEyeBrow

OffSetEyeBrow

Active Member
OP
Newcomer
Joined
Jun 16, 2018
Messages
35
Trophies
0
Age
21
Location
italy
XP
290
Country
Italy
.lz is a common enough extension picked by devs presumably outputting a compressed file and needing an extension rather than being any kind of known file format (or formats in the case of things like .pac).

The DS (and GBA before it) feature compression algorithms in the BIOS which software can call upon, and most things use those or things compatible with them (some custom stuff does exist for the GBA but different discussion).

Many tools have been made
https://www.romhacking.net/utilities/826/
https://github.com/barubary/dsdecmp
Crystaltile2 (open it, click the DS icon on the top row, scroll down to the file you want, right click and options to decompress should be there, alternatively there are standalone tools within it if you play with the dropdown menus).

Once you have the decompressed file the real fun begins. Maybe it is a single file (kilobytes is small but could go either way) but if not and you have an archive then time to fiddle with such things.

I usually like to look for known formats in a hex editor ( https://www.romhacking.net/documents/[469]nds_formats.htm for an idea of how they tend to roll -- tend to get magic stamps/markers and file sizes) and then see if they relate to anything at the start or end of the file (you might expect file names, file sizes, locations of file, number of files contained within the archive). Alternatively open up in a tile editor and see what you can see.
I tried using the Nintendo DS/gba compressor and crystaltile2 but neither really helped much
No programs on the ds/gba compressor worked with just .lz as a file extension, they required either .blz, .lze, .lzss, or other versions of the .lz format
crystaltile2 didn't give me any option to decompress by right-clicking, and when I tried to unzip the lz77 files that it had found on the ROM it would either crash or extract nothing (export just gave me the .lz file again)
1693313980616.png

I previously forgot to mention this, but I tried looking through the .lz file with
1693314213898.png
but I didn't find any resemblance of other file extensions, locations, sizes, or anything that made sense, just a mess of letters, symbols, and dots

I didn't try using dsdecmp because I honestly have no clue how to run the .cs or .java files. I know barely anything about programming, sorry. :(
 

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,795
Trophies
5
XP
28,503
Country
United Kingdom
Hmm. There were some binaries back when for dsdecmp, guess they got lost somewhere along the line. It is very rare I found anything that worked with it that Cue's tools did not.

Curious that nothing detects it.

Two main choices at this point (haven't looked at the game or your example files yet).

1) You have encountered some kind of custom compression. I am not sure I have examples before for the DS outside of files the devs left in games (had a few zip files and such filled with scripts, notes from devs, old files, test files, source code...) but it is well within reason still.

2) The compression is a bit further into the file -- sometimes files within archives will be compressed individually (saves having to decompress the lot and then chop down) and then the.
As most compression tools look for the telltale signs of compression then not encountering them will see it treat it like any other unknown file.

Assuming it is 2) you have a few options.
Said telltale signs are a file starting with 10h (sometimes referred to as type 10), 11h (type 11, 10 was the BIOS format seen since the GBA days but was improved a bit into the DS lilfetime and thus was born this) and 40h (first seen in Golden Sun and very rare after that too). It will then be followed by the compressed file size.

If said 10h/11h/40h happens a few bytes into the file (seen a few before with the file count and then I think it was file sizes but the general principles/things to expect in archive formats from the previous post is very much in play) then chop it out and feed it to the previously mentioned programs.
Alternatively those working back in the GBA days don't have a nice file system (like most old non optical non floppy disc consoles the games are one giant blob of data) so there were tools that would scan for the signs* and attempt to do things from there. This is also an option in crystaltile2 in one of the dropdown menus, though it is rather crash happy when you play with it (I think it mostly just searches for 10h and then tries to decompress it).

*mostly the same, though 10 was the only one in play and there might well have been some indicators otherwise**. What this aside is more about noting though is that BIOS logs exist (if you are calling the BIOS to handle it then easy enough to have an emulator note what it was and where it was going to in the ROM before feeding that log back into a tool).

**if you are curious about compression in general (this also being part of the answer to 1) above) I generally link people to https://ece.uwaterloo.ca/~ece611/LempelZiv.pdf (it also covers Huffman, the other big class of compression, in it despite the name).
https://www.romhacking.net/utilities/1770/ might also be of use. https://problemkaputt.de/gbatek.htm#biosdecompressionfunctions for more on the GBA/DS specifics if the source code of the various other links is not doing it for you.
The basic principle of LZ is after whatever fun with headers you have then the compressed file is so much this section is compressed, here is how far back to look from where we are at (it is called sliding window compression for a reason) and how much to grab as well as indicators/flags that this section is not compressed and skip to the next bit (a cheap way of compressing a custom compression then being just insert the "nothing compressed, move to next one" throughout the file/previously compressed and presumably now edited data).
I do have a worked example in the hacking documentation linked in my signature as well https://gbatemp.net/threads/gbatemp-rom-hacking-documentation-project-new-2016-edition-out.73394/
Anyway any element of this being changed quickly breaks tools hardcoded to only expect that, custom flags and a different split between the location and read back amounts (you can't just have arbitrarily long ones if you actually want to compress it after all, most things not being repetitive enough to warrant a huge amount of read this but you also don't want to crush it to when you can read megabytes of window but only a few bytes when you get there) being the main things to change.
 

OffSetEyeBrow

Active Member
OP
Newcomer
Joined
Jun 16, 2018
Messages
35
Trophies
0
Age
21
Location
italy
XP
290
Country
Italy
answered my self by accident, disregard this
Post automatically merged:

Hmm. There were some binaries back when for dsdecmp, guess they got lost somewhere along the line. It is very rare I found anything that worked with it that Cue's tools did not.

Curious that nothing detects it.

Two main choices at this point (haven't looked at the game or your example files yet).

1) You have encountered some kind of custom compression. I am not sure I have examples before for the DS outside of files the devs left in games (had a few zip files and such filled with scripts, notes from devs, old files, test files, source code...) but it is well within reason still.

2) The compression is a bit further into the file -- sometimes files within archives will be compressed individually (saves having to decompress the lot and then chop down) and then the.
As most compression tools look for the telltale signs of compression then not encountering them will see it treat it like any other unknown file.

Assuming it is 2) you have a few options.
Said telltale signs are a file starting with 10h (sometimes referred to as type 10), 11h (type 11, 10 was the BIOS format seen since the GBA days but was improved a bit into the DS lilfetime and thus was born this) and 40h (first seen in Golden Sun and very rare after that too). It will then be followed by the compressed file size.

If said 10h/11h/40h happens a few bytes into the file (seen a few before with the file count and then I think it was file sizes but the general principles/things to expect in archive formats from the previous post is very much in play) then chop it out and feed it to the previously mentioned programs.
Alternatively those working back in the GBA days don't have a nice file system (like most old non optical non floppy disc consoles the games are one giant blob of data) so there were tools that would scan for the signs* and attempt to do things from there. This is also an option in crystaltile2 in one of the dropdown menus, though it is rather crash happy when you play with it (I think it mostly just searches for 10h and then tries to decompress it).

*mostly the same, though 10 was the only one in play and there might well have been some indicators otherwise**. What this aside is more about noting though is that BIOS logs exist (if you are calling the BIOS to handle it then easy enough to have an emulator note what it was and where it was going to in the ROM before feeding that log back into a tool).

**if you are curious about compression in general (this also being part of the answer to 1) above) I generally link people to https://ece.uwaterloo.ca/~ece611/LempelZiv.pdf (it also covers Huffman, the other big class of compression, in it despite the name).
https://www.romhacking.net/utilities/1770/ might also be of use. https://problemkaputt.de/gbatek.htm#biosdecompressionfunctions for more on the GBA/DS specifics if the source code of the various other links is not doing it for you.
The basic principle of LZ is after whatever fun with headers you have then the compressed file is so much this section is compressed, here is how far back to look from where we are at (it is called sliding window compression for a reason) and how much to grab as well as indicators/flags that this section is not compressed and skip to the next bit (a cheap way of compressing a custom compression then being just insert the "nothing compressed, move to next one" throughout the file/previously compressed and presumably now edited data).
I do have a worked example in the hacking documentation linked in my signature as well https://gbatemp.net/threads/gbatemp-rom-hacking-documentation-project-new-2016-edition-out.73394/
Anyway any element of this being changed quickly breaks tools hardcoded to only expect that, custom flags and a different split between the location and read back amounts (you can't just have arbitrarily long ones if you actually want to compress it after all, most things not being repetitive enough to warrant a huge amount of read this but you also don't want to crush it to when you can read megabytes of window but only a few bytes when you get there) being the main things to change.
I tried searching for those pointers with HxD but I got 0 results, so it may be a custom compression.
While messing around in crystaltile2 I opened the ASM viewer and got something that looked to make some sense
Granted i don't know what I'm looking at, but does this mean its not a compressed file at all?

1693318694942.png
 

Kwyjor

Well-Known Member
Member
Joined
May 23, 2018
Messages
4,586
Trophies
1
XP
4,698
Country
Canada
No programs on the ds/gba compressor worked with just .lz as a file extension, they required either .blz, .lze, .lzss, or other versions of the .lz format
It's just a name. It is by no means beyond the realm of possibility that the programmers were using the same variant as someone else but decided to name the file ".lz" for some reason.

Granted i don't know what I'm looking at, but does this mean its not a compressed file at all?
You are looking at a blind attempt to interpret the data as binary instructions. It's the equivalent of a picture of random static.
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
    Psionic Roshambo @ Psionic Roshambo: https://en.m.wikipedia.org/wiki/Ke_Huy_Quan