ROM Hack Recompressing an overlay file?

[Bond697]

Hello,

I have an overlay from Pokemon White 2 that I've done a small assembly hack in. Every time I try to recompres it, however, the game blackscreens at the point that specific overlay should be loaded into memory. I'm sure that it's simply not compressing properly. I've tried all the lz77 options in Crystal Tile, dsdecmp, etc andhave had no success. There absolutely has to be a way to do this, but I'm not seeing it. Help?




Thanks,

Mat

 

[FAST6191]

Crystaltile2 struggles to recompress binaries and I thought dsdecmp had the option but http://gbatemp.net/topic/313278-nintendo-dsgba-compressors/ ( filetrip download http://filetrip.net/nds-downloads/utilities/download-cues-gba-ds-compressors-10-f29010.html )

The command will be something like BLZ -en overlay????.bin overlay????.bin.comp
It will spit out a file named overlay????.bin.comp which you can then try injecting.

I have not bothered to vet pokemon games but some games from Nintendo first and second party do odd things and might want most files injected back in properly so if BLZ does not work try http://www.no-intro.org/tools.htm (NDSTS) to inject it back in properly assuming it is smaller or same size (pad it out with 00 if smaller) and then move to something like tinke or nitroexplorer.

 

[Bond697]

Well, I finally got dsdecmp working and did lzovl, but still no luck. I'll try that link you provided next..

e: actually, i'm just reinserting with dsbuff to unpack and repack the rom. it works for b2w2 normally if you unpack then repack the rom, so i don't see why this wouldn't work.


that blz thing seems like it gets me very close. the game looks like it gets a little bit farther, but it still dies. i tried ndsts, but it dies with an out of memory error.

wait, nevermind. i got it working. thanks!

 

[Bond697]

I just realized that I never explained what the actual problem and solution were.

If you decompress an overlay with Crystal Tile 2, edit it, and recompress it with CT2 it doesn't work. If you decompress with CT2 and recompress with BLZ, it doesn't work. However, if you decompress and recompress with BLZ, the overlay works fine after being re-inserted. I had a set of overlays I was working with that I had decompressed with CT2, so I was getting confused when BLZ wasn't recompressing properly. Pulling them out again with dsbuff and properly decompressing them with BLZ worked perfectly. That's also probably why dsdecmp wasn't working for me.

Granted, no one seems to do any assembly hacking for Pokemon hacks here, but it's worth knowing just in case or for other games.

 

[Fenix 97]

hy,have someone idea of why blz doesn'd work to me?

 

[FAST6191]

What is not working with it? Is it the program itself that does nothing (and does it have any error messages) or the results of using it?
If you are running windows 7 on one of the more paranoid security modes it can dislike programs executing and modifying files so you might have to run it with admin privileges (in this case probably best to run a command line with admin privileges unless you want to make batch files all the time).

 

[Fenix 97]

i made a video with my problem,wath here http://www.youtube.com/watch?v=QM2Sn4tbdJU&feature=youtu.be

 

[Fenix 97]

fast 6191 are you there?

 

[FAST6191]

19 minutes is a bit short for bumping a topic, especially a technical one.

The reason it does that is because it is a command line program; for most user programs in windows double clicking them will get something done and you can play with the mouse and do things as well but there is a whole other method of running programs (I might even go so far as to say it is half of being able to operate a computer) that roughly corresponds to the command line/prompt (you might have had to edit shortcuts or put things in the run command- it flows from the same idea). When you double click it you have given it nothing to run with which means it will probably spit out the help command and as it has nothing to do it just closes, sometimes you can drag and drop files onto programs and it can do things to them but best not to in this case.

This is neither the time nor the place for a full command line tutorial though so the broad strokes

Afraid I am not quite sure what most of these will be in Italian but the ideas remain the same.

1) Press start and then press run
2) Type in CMD in the box and press enter
Alternatively press start -> programs -> accessories and click command prompt

Up will pop a probably black box with white text with something like
C:\documents and settings\Maurol>

Type
cd desktop and press enter
cd 29010-Nintendo_DS_Compressors-CUE and press enter. For the record when typing it once you have a few characters in there if you press tab it should autocomplete (linux will do this as will the more modern versions of windows but older windows not so much)
Theoretically you could have also put cd desktop\29010-Nintendo_DS_Compressors-CUE from the first line but you can learn about that when you learn about the command line in general. You can also paste things in there but rather than control and V right click and press paste.
Now you have the program there to run, I already had the rough command up above for compression but for the sake of not having to scroll up it will be something like
BLZ -d overlay????.bin
Now e
BLZ -en overlay????.bin overlay????.bin.comp

???? will want to be replaced with the number in the name of the overlay file.

You will want the overlay file in the same directory as the blz.exe file unless you want to start trying more advanced variations on the command line; I usually find it a lot easier if you copy the file to the directory the exe file is in and if nothing else if you screw up editing it you have have a backup.

On top of this the command line itself has a whole bunch of commands available to it
http://www.webtutoriales.com/articulos/comandos-windows
http://www.tecmundo.com.br/windows-xp/1755-lista-de-comandos-para-o-windows-xp.htm
You can probably do a better search for command line tutorials in Italian than I can though- all with all the commands there are also things like pipes, batch files and general directory handling.

 

[Fenix 97]

sorry i have some difficulties to understand what you say.now i show you what i have understand.i have to open the prompt and go on desktop and the folder wich contains blz.exe,i use tab and i select blz.exe,i open it and i have this message and now?

Microsoft Windows XP [Versione 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\Mauro>CD dESKTOP

C:\Documents and Settings\Mauro\Desktop>cd 29010-Nintendo_DS_Compressors-CUE

C:\Documents and Settings\Mauro\Desktop\29010-Nintendo_DS_Compressors-CUE>blz.ex
e

BLZ - (c) CUE 2011
Bottom LZ coding for Nintendo GBA/DS

Usage: BLZ command filename [filename [...]]

command:
-d .... decode 'filename'
-en ... encode 'filename', normal mode
-eo ... encode 'filename', optimal mode (LZ-CUE)

* multiple filenames and wildcards are permitted
* the original file is overwritten with the new file
* this codification is used in the DS overlay files

C:\Documents and Settings\Mauro\Desktop\29010-Nintendo_DS_Compressors-CUE>

 

[FAST6191]

It is actually telling you what you can try doing aka the usage. Indeed most command line programs if you just run them from the command line will tell you what the commands are or to type something like "programname /help" to get help/usage.

Still, in your case you want to type
blz.exe -d overlay????.bin

the blz.exe tells the prompt to try running a program called blz with the command the follows it
the -d is unique to this program (in this case it tells it to try decoding/decompressing the file) and the next part is where the file might be found. As it should now be in the same directory you can just type the name of file, again the ???? will want to be replaced with the numbers from the file you want to edit.

For this program the original file will be overwritten with the decompressed one unless you tell it not to (another reason for copying it out of the original directory), other decompression programs might do other things but we are not dealing with those right now so I will ignore them.

Again there are other things you can do and various operating systems work slightly differently but no need to confuse things at this point.

 

[Fenix 97]

ok don't worry it work now.thant you'r fantastic.if i could i'd make a gold monument to you

 

[Dirbaio]

There's absolutely no need to recompress overlays.

Bit 0 of byte 0x1F in the overlay table is 1 if the overlay is compressed, 0 otherwise.
So, reinsert the decompressed overlay as-is and clear bit 0 of byte 0x1f. (For example change it from 03 from 02)
This way the game won't decompress the overlay after it's loaded.

 

[Fenix 97]

ok anyway now it works and so i finally i have my overlay translated

 

[iamatmylimit]

Also something that may make things easier from now on. You can simply go to C:\WINDOWS\system32 and copy cmd.exe unto the desktop where you have the BLZ.exe


Then Drag & Drop BLZ.exe unto cmd.exe


The program will be added to cmd and all you do is type the commands to extract/decompress or whatever. Any program that needs to be run from command line can be used this way. Much easier than having to retype different locations for different files if you just have it in one convenient location like desktop where you can basically test things.

 

[Lyan53]

It is very complicated all these methods, just use a batch file (.bat) or command (.cmd)

It is just a simple .txt file where you put the command line (without the full path) and you rename the extension to .bat or .cmd

.bat or .cmd files will serve as executable file (.exe), it will do more than just double-click it to run the software


- Place your overlay and your .exe files in a folder

- Make a .txt file with your command line :

ex/ (with dsdecmp)

DSDecmp -c lzovl overlay_0001.bin overlay_0001.bin.ovl
pause

- Save this file and rename the extension .txt into .bat or .cmd

- Place your .bat/.cmd file in a same folder (without full path of files, it's useless)

- Double-click on .bat or .cmd file and that's all

................................................................

For dsdecmp, actually, this tool has a bug

At the end of overlay (re)compressed file, you can find the full size of compressed file (in little endian)

Highlighted in blue, it is information for compression (the header)
In the red box, it is the full size of the compressed file

dsdecmp made a miscalculation on it, because it does not take into account the size of this header informations

If you want your file works, it is necessary to fix this error by adapting to the full size of the compressed file (header included)

.......................................................................................................

It is also possible to bypass the compression of overlay files by editing the index of these, located in the file "y9.bin"

In this index file, each overlay is stored on 0x20 bytes

- In red box: number of the entry in the index/number of overlay file (these 2 values are the same)
- In green box: offset where the overlay file is loaded in RAM
- In blue box: full size of overlay file uncompressed (or decompressed if is compressed)
- In orange box: full size of overlay file compressed (if the overlay file is not compressed, this value is 0x000000 or maybe uncompressed size in some cases)
- In pink box: this is a compression byte, it can be 0x00 (for uncompressed file) or "0x01, 0x02, 0x03" for a compressed file (I don't know the difference between these 3 compression values)


If you change the compression byte and the compression size, you can by-pass easily the compression of your overlay file

ex/ (with an uncompressed overlay file)

See ya all

 

[Dirbaio]

That's what I said some posts ago. Clear bit 0 of the last byte and the overlay is uncompressed.
Also bit 1 means the overlay is digitally signed.

I mean, values are these:
0: Not compressed, not signed
1: Compressed, not signed
2: Not compressed, signed
3: Compressed and signed.

In theory the "right" way to do it is changing the last byte from 3 to 2. That's what NSMB Editor does when decompressing overlays and it works perfectly.
In practice the "digitally signed" bit doesn't matter. It's only used for DLP, and since you're modifying it the HMAC-SHA1 check will fail anyway.

The three bytes you marked in orange are unused so no need to change them.

 

[Lyan53]

Okay, thank you for these details Dirbaio