[Release] BinJ Editor - A Tool For Translating Dragon Quest Monsters Games

Abstract
BinJ Editor lets you import, edit, save, share and export translations for games using the .binJ and .e file format to store messages. Those games include Dragon Quest Monsters: Terry's Wonderland 3D and Dragon Quest Monsters 2: Iru and Luca's Marvelous Mysterious Key.


Introduction
Hi, the last past weeks I was trying to find out how the binJ file format works. Simply because I wanted to change a few messages in Dragon Quest Monsters 2: Iru and Luca's Marvelous Mysterious Key. I thought this surely can't be that hard. The short answer is I was wrong, the long answer is hidden in the following spoiler.

Spoiler: Backstory

The first thing I did to translate a text was opening the binJ file using Notepad++. Since it directly showed japanese characters I thought, I could directly replace them and expected it to work. Even though I was looking at a binary file, which you usually can't just read with a text editor. But when I copied some texts to Google Translate to find out what they mean, I was disappointed. The texts weren't real japanese. So I googled a bit and found this thread where they chatted about binJ files.

They wrote that these files use a custom encoding for japanese characters and characters in general. What you have to understand about binary files is that they are basically 1s and 0s. This is true for all files, but "normal text files" use a standardized encodings. This means that the byte "0110 0001" with a common encoding like ASCII or UTF-8 always stands for the character "A". But for the game I tried to translate a custom encoding is used. They encode the character "A" using the byte "0000 1101". Once I used the encoding table provided in the thread I was able to read the correct texts. Now I tried to change a single character by replacing the original bytes for this character with the bytes for the character I replaced it with. I sent the file to my 3DS and launched the game. It worked. Great.

Now I only needed to find a way to tell the individual texts appart and luckily the texts were always separated by a special two-byte long token. Thus I wrote a small programm that read the binary data, splitted the texts and decoded the bytes to japanese texts. And it worked great. This was the point where I started developing the first version of BinJ Editor. A thing that I haven't talked about yet is that with this method you have to keep the byte length of the original text and the translated text the same. Otherwise the game won't find the texts and just crashes. The problem with this method is that japanese is a lot denser than english. Normally two characters of japanese text are translated with about eight characters of english text. But since the encoding table they used is optimized for japanese text, a lot of common japanese characters are encoded using no more than a single byte. This means the english translation for a two character long japanese text is only allowed to have two bytes as well. As an example you could translate "耐性" (resistance) with "RE" but not "RES" and definitely not with "Resistance". That was a thing I was able to overcome much later.

But what I was able to do was shorten the texts by simply replacing not-needed characters with null bytes. And therefore I started developing the first version of BinJ Editor. After the first sketch was finished I started translating a few menu texts as a proof of concept. And this was the point were I realized all binJ files have a weird long "first text" that I never looked into. All first texts looked similar and at this point I thought maybe those bytes are the addresses of the texts in the binary file. I checked it and I was right. That means that given a list of texts (with their respective lengths) I was able to re-create this address block. Now I had two possibilites: rewrite my import function of binJ files to validate that the texts I obtain by looking at the separator tokens are correct OR just believe it works. And... I chose the second option. Which was the bad one. But at least I could now replace texts with translated texts of arbitrary length. So I rewrote a lot of parts of BinJ Editor and started translating more parts of the game. Until...

... there were problems. I translated some battle messages and whenever I entered a battle the game crashed. So I looked into the binary data once again and found out that this particular binJ file included the same address twice. So I had to look into the address block while parsing binJ files... I started rewriting the corresponding parts of BinJ Editor and at some point I realized that I always missed the first text of the translation before because the first text does not start with the separator token as all the other texts do. And did I mention one file contains a one-byte off pointer? Anyway... someday I got all of this to work and this basically is the whole story of how I got to decipher the binJ file format and how I wrote BinJ Editor. I later cleaned things up, added some convenient features, wrote documentation and... created this post.

If you simply want to know more about the binJ and e file formats check these out: BinJ Format and E Format


Presentation


To import a file choose "File > Import..." and select the file. This process may take a few seconds. Now you can start editing the file by entering text into the last column. When you finished editing you can save a project file by choosing "File > Save As..." or export an edited .binJ file by choosing "File > Export...".
A full explanation can be found on the Website of this project: BinJ Editor Manual



It also includes a very handy FTP Client for directly sending the edited files to your 3DS for Luma to patch.
More on that can be found in the Manual as well: BinJ Editor Manual



Another very useful tool is the Search Dialog which let's you search multiple binJ, e and other files at once.
More on this tool can be found in the Manual: BinJ Editor Manual


Examples
As an example I present you four screenshots from Dragon Quest Monsters 2: Iru and Luca's Marvelous Mysterious Key.
The first one is the title screen, the second one the loading screen and the third one the status screen of my buddy Pineapple. All texts are stored in binJ files.
The fourth screenshot shows a dialog with the father of the protagonists which is stored inside an e file.
Note that the first screenshot of each row is the original, the second one the translated (using BinJ Editor) and the third one the translated including a custom font and custom images (cannot be done using BinJ Editor).

Spoiler: Title Screen

Spoiler: Loading Screen

Note: The monsternames are fixed, so my Slimey has a japanese name and the others have german nicknames.

Spoiler: Status Screen

Note: The game does weird things with the width of some texts, so "AGIL" displays unreadable and "A G I L" does the job.

Spoiler: Conversation with Dad

Download
The project is hosted on GitHub: BinJ Editor on GitHub
You can download the latest version at: Latest Version of BinJ Editor


I hope you like it
Ich_73

 

I took the original font and converted it to an image. I did the same thing with the font from the DQMT translation and merged them. Since the formats were a bit different I had to change some pixels manually. After that I converted the image to a bcfnt font and used that. It sounds more straightforward than it actually was, but that's basically it.

 

I mean the latin alphabet part you see on the images is the part I took from that font you built. But since DQMT uses more symbols than DQM2 you can't just use the same bcfnt file.

 

Right, they're in ExtractedRomFS/data/Event/**/*.e.
It doesn't seem like Kuriimu is able to handle those files. Do you know a tool to do this, or have any other documentation of the file format? Or is it the same as with the binJ files and I would have to figure it out myself^^

Thanks! I tried to translate all the menus I could find, so yes, the first bit is done.

 

I know... Really sad you didn't make it in the end.

Interesting... I haven't noticed the problem yet, but I was able to recreate it right now, even though I could fit 65 characters.
Has anyone of your team had a look into the code whether it is just a hardcoded limit (because the developers thought, it would never exceed that length) or is that something worth investigating?

 

One person is not that many given the amount of people that would love to see another quality fan translation.
I'll try my best, but I haven't looked into the code section of hacking yet. I'll keep you informed.

 

Release v2.0.0

Thanks to @Locke-92 who contacted me saying he can help decyphering .e files I could upgrade BinJ Editor with the functionality of editing those files.
E files are used in Dragon Quest Monsters: Terry's Wonderland 3D and Dragon Quest Monsters 2: Iru and Luca's Marvelous Mysterious Key to store cutscene and dialog data for the story.



If you want to know the details of this file format check out: E Format

I also took the time to implement some other useful features.
If you're interested take a look at the release notes: BinJ Editor v2.0.0.

I hope you like it
Ich_73

 

@Helmax I simply quote myself to answer this: