ROM Hack [Release] BinJ Editor - A Tool For Translating Dragon Quest Monsters Games

Ich_73

Well-Known Member
OP
Member
Joined
Aug 27, 2020
Messages
122
Trophies
0
XP
433
Country
Germany
Abstract
BinJ Editor lets you import, edit, save, share and export translations for games using the .binJ and .e file format to store messages. Those games include Dragon Quest Monsters: Terry's Wonderland 3D and Dragon Quest Monsters 2: Iru and Luca's Marvelous Mysterious Key.


Introduction
Hi, the last past weeks I was trying to find out how the binJ file format works. Simply because I wanted to change a few messages in Dragon Quest Monsters 2: Iru and Luca's Marvelous Mysterious Key. I thought this surely can't be that hard. The short answer is I was wrong, the long answer is hidden in the following spoiler.
The first thing I did to translate a text was opening the binJ file using Notepad++. Since it directly showed japanese characters I thought, I could directly replace them and expected it to work. Even though I was looking at a binary file, which you usually can't just read with a text editor. But when I copied some texts to Google Translate to find out what they mean, I was disappointed. The texts weren't real japanese. So I googled a bit and found this thread where they chatted about binJ files.

They wrote that these files use a custom encoding for japanese characters and characters in general. What you have to understand about binary files is that they are basically 1s and 0s. This is true for all files, but "normal text files" use a standardized encodings. This means that the byte "0110 0001" with a common encoding like ASCII or UTF-8 always stands for the character "A". But for the game I tried to translate a custom encoding is used. They encode the character "A" using the byte "0000 1101". Once I used the encoding table provided in the thread I was able to read the correct texts. Now I tried to change a single character by replacing the original bytes for this character with the bytes for the character I replaced it with. I sent the file to my 3DS and launched the game. It worked. Great.

Now I only needed to find a way to tell the individual texts appart and luckily the texts were always separated by a special two-byte long token. Thus I wrote a small programm that read the binary data, splitted the texts and decoded the bytes to japanese texts. And it worked great. This was the point where I started developing the first version of BinJ Editor. A thing that I haven't talked about yet is that with this method you have to keep the byte length of the original text and the translated text the same. Otherwise the game won't find the texts and just crashes. The problem with this method is that japanese is a lot denser than english. Normally two characters of japanese text are translated with about eight characters of english text. But since the encoding table they used is optimized for japanese text, a lot of common japanese characters are encoded using no more than a single byte. This means the english translation for a two character long japanese text is only allowed to have two bytes as well. As an example you could translate "耐性" (resistance) with "RE" but not "RES" and definitely not with "Resistance". That was a thing I was able to overcome much later.

But what I was able to do was shorten the texts by simply replacing not-needed characters with null bytes. And therefore I started developing the first version of BinJ Editor. After the first sketch was finished I started translating a few menu texts as a proof of concept. And this was the point were I realized all binJ files have a weird long "first text" that I never looked into. All first texts looked similar and at this point I thought maybe those bytes are the addresses of the texts in the binary file. I checked it and I was right. That means that given a list of texts (with their respective lengths) I was able to re-create this address block. Now I had two possibilites: rewrite my import function of binJ files to validate that the texts I obtain by looking at the separator tokens are correct OR just believe it works. And... I chose the second option. Which was the bad one. But at least I could now replace texts with translated texts of arbitrary length. So I rewrote a lot of parts of BinJ Editor and started translating more parts of the game. Until...

... there were problems. I translated some battle messages and whenever I entered a battle the game crashed. So I looked into the binary data once again and found out that this particular binJ file included the same address twice. So I had to look into the address block while parsing binJ files... I started rewriting the corresponding parts of BinJ Editor and at some point I realized that I always missed the first text of the translation before because the first text does not start with the separator token as all the other texts do. And did I mention one file contains a one-byte off pointer? Anyway... someday I got all of this to work and this basically is the whole story of how I got to decipher the binJ file format and how I wrote BinJ Editor. I later cleaned things up, added some convenient features, wrote documentation and... created this post.
If you simply want to know more about the binJ and e file formats check these out: BinJ Format and E Format


Presentation
91449697-b562e480-e87b-11ea-983f-1c6407ecef3a.png


To import a file choose "File > Import..." and select the file. This process may take a few seconds. Now you can start editing the file by entering text into the last column. When you finished editing you can save a project file by choosing "File > Save As..." or export an edited .binJ file by choosing "File > Export...".
A full explanation can be found on the Website of this project: BinJ Editor Manual

91468919-f74b5500-e892-11ea-8c2d-5d046eb6ac81.png


It also includes a very handy FTP Client for directly sending the edited files to your 3DS for Luma to patch.
More on that can be found in the Manual as well: BinJ Editor Manual

96344030-5c087c00-109a-11eb-9f66-37f79e5546f6.png


Another very useful tool is the Search Dialog which let's you search multiple binJ, e and other files at once.
More on this tool can be found in the Manual: BinJ Editor Manual


Examples
As an example I present you four screenshots from Dragon Quest Monsters 2: Iru and Luca's Marvelous Mysterious Key.
The first one is the title screen, the second one the loading screen and the third one the status screen of my buddy Pineapple. All texts are stored in binJ files.
The fourth screenshot shows a dialog with the father of the protagonists which is stored inside an e file.
Note that the first screenshot of each row is the original, the second one the translated (using BinJ Editor) and the third one the translated including a custom font and custom images (cannot be done using BinJ Editor).
91964127-0c026f80-ed0f-11ea-95da-69fc2f2858d3.png
Note: The monsternames are fixed, so my Slimey has a japanese name and the others have german nicknames.

91964134-0efd6000-ed0f-11ea-9341-e2057a400ede.png
Note: The game does weird things with the width of some texts, so "AGIL" displays unreadable and "A G I L" does the job.

91976011-9bfce500-ed20-11ea-801e-e9cde0104a54.png
96371242-0054f600-1161-11eb-9465-b48d31067c64.png


Download
The project is hosted on GitHub: BinJ Editor on GitHub
You can download the latest version at: Latest Version of BinJ Editor


I hope you like it
Ich_73
 
Last edited by Ich_73,

Ich_73

Well-Known Member
OP
Member
Joined
Aug 27, 2020
Messages
122
Trophies
0
XP
433
Country
Germany
Where'd you get that custom font from?
I took the original font and converted it to an image. I did the same thing with the font from the DQMT translation and merged them. Since the formats were a bit different I had to change some pixels manually. After that I converted the image to a bcfnt font and used that. It sounds more straightforward than it actually was, but that's basically it.
 

Helmax

Well-Known Member
Member
Joined
Feb 17, 2011
Messages
239
Trophies
0
XP
1,560
Country
United States
I took the original font and converted it to an image. I did the same thing with the font from the DQMT translation and merged them. Since the formats were a bit different I had to change some pixels manually. After that I converted the image to a bcfnt font and used that. It sounds more straightforward than it actually was, but that's basically it.

Was very curious because that looked like the font I built for DQM TW 3DS. Cool to hear you basically went through the same process as me
 
  • Like
Reactions: cearp

Ich_73

Well-Known Member
OP
Member
Joined
Aug 27, 2020
Messages
122
Trophies
0
XP
433
Country
Germany
Was very curious because that looked like the font I built for DQM TW 3DS. Cool to hear you basically went through the same process as me
I mean the latin alphabet part you see on the images is the part I took from that font you built. But since DQMT uses more symbols than DQM2 you can't just use the same bcfnt file.
 

Helmax

Well-Known Member
Member
Joined
Feb 17, 2011
Messages
239
Trophies
0
XP
1,560
Country
United States
The story dialog is in the demo and field files. They're just E extension files if I recall correctly
 

breaktemp

D-Sun who was called D-Moon
Member
Joined
Mar 17, 2008
Messages
1,061
Trophies
0
Location
TSU...no.1 in tha hood
Website
www.youtube.com
XP
801
Country
United States
@Ich_73

Really cool program
Seems like it can get the job done when it comes to the menus and such !
From Github, it looks like on the English side
You have completed a bit chunk of the game.
I do hope you can find a nice team of people who can support you with this.
 

Ich_73

Well-Known Member
OP
Member
Joined
Aug 27, 2020
Messages
122
Trophies
0
XP
433
Country
Germany
The story dialog is in the demo and field files. They're just E extension files if I recall correctly
Right, they're in ExtractedRomFS/data/Event/**/*.e.
It doesn't seem like Kuriimu is able to handle those files. Do you know a tool to do this, or have any other documentation of the file format? Or is it the same as with the binJ files and I would have to figure it out myself^^

Really cool program
From Github, it looks like on the English side
You have completed a bit chunk of the game.
Thanks! I tried to translate all the menus I could find, so yes, the first bit is done.
 
  • Like
Reactions: breaktemp

Helmax

Well-Known Member
Member
Joined
Feb 17, 2011
Messages
239
Trophies
0
XP
1,560
Country
United States
I had a team that was working on a full translation of this game. All of this work is already done. The problem is, the code.bin file in the exefs has buffer limits to prevent us from rendering more than 40 characters per dialog window, and I the English translation really blows this up in size
 

Ich_73

Well-Known Member
OP
Member
Joined
Aug 27, 2020
Messages
122
Trophies
0
XP
433
Country
Germany
I had a team that was working on a full translation of this game. All of this work is already done.
I know... Really sad you didn't make it in the end.

The problem is, the code.bin file in the exefs has buffer limits to prevent us from rendering more than 40 characters per dialog window, and I the English translation really blows this up in size
Interesting... I haven't noticed the problem yet, but I was able to recreate it right now, even though I could fit 65 characters.
Has anyone of your team had a look into the code whether it is just a hardcoded limit (because the developers thought, it would never exceed that length) or is that something worth investigating?
 

Ich_73

Well-Known Member
OP
Member
Joined
Aug 27, 2020
Messages
122
Trophies
0
XP
433
Country
Germany
I was the only one looking into it. Never figured it out. You get that figured out and we can release all our work
One person is not that many given the amount of people that would love to see another quality fan translation.
I'll try my best, but I haven't looked into the code section of hacking yet. I'll keep you informed.
 

JustDrew

Member
Newcomer
Joined
Apr 30, 2020
Messages
22
Trophies
0
Age
33
XP
259
Country
United Kingdom
its heartbreaking that the translation is done, but some line of code is preventing potentially the best game in the monster series from getting translated. So im guessing somewhere in that code.bin file there is a solution to all this...
 
Last edited by JustDrew,

Ich_73

Well-Known Member
OP
Member
Joined
Aug 27, 2020
Messages
122
Trophies
0
XP
433
Country
Germany
Release v2.0.0

Thanks to @Colmines92 who contacted me saying he can help decyphering .e files I could upgrade BinJ Editor with the functionality of editing those files.
E files are used in Dragon Quest Monsters: Terry's Wonderland 3D and Dragon Quest Monsters 2: Iru and Luca's Marvelous Mysterious Key to store cutscene and dialog data for the story.

96371682-2d0a0d00-1163-11eb-86f1-0811944ae3e5.png


If you want to know the details of this file format check out: E Format

I also took the time to implement some other useful features.
If you're interested take a look at the release notes: BinJ Editor v2.0.0.

I hope you like it
Ich_73
 
Last edited by Ich_73,

Ich_73

Well-Known Member
OP
Member
Joined
Aug 27, 2020
Messages
122
Trophies
0
XP
433
Country
Germany
@Helmax I simply quote myself to answer this:
I can now decompile and debug the code.bin file using IDA and Luma's Debugger. I can also edit the code on a byte level meaning I can change little things. I tested this by replacing some images though code and that worked. But at the moment I am not able to completely rewrite or add code and compile it myself. Because I don't know if that is even required in order to fix the word limit, I started looking for the right place in the code.bin file. I found the main loop and some other things but am still searching for the critical place where the text is loaded/queried/drawn/etc.

Thanks to @Colmines92 who contacted me about a week ago I can announce some exciting news on this project.
[...]
I myself have only used the code and all debugging features for translation related purposes, not for overcomming the character limit, but since he did such a nice job on the .e files I told him what I knew about this issue and he starting looking into it. So far he has found the place where text is rendered and he found out that all characters (more than seen on screen) are loaded. You can tell by listening to the text sounds as well. He assumes there is a character limit of 58 characters based on his observations. Also there does not seem to be a limit in the meaning of "if the text is longer than x characters, don't print it" but a general limitation to the rendering function (of the game engine?). He is trying his best to find out more.
 
Last edited by Ich_73,
General chit-chat
Help Users
  • No one is chatting at the moment.
    NeoGaming @ NeoGaming: you are sus