1. Ich_73

    OP Ich_73 Advanced Member
    Newcomer

    Joined:
    Aug 27, 2020
    Messages:
    89
    Country:
    Germany
    Abstract
    BinJ Editor lets you import, edit, save, share and export translations for games using the .binJ and .e file format to store messages. Those games include Dragon Quest Monsters: Terry's Wonderland 3D and Dragon Quest Monsters 2: Iru and Luca's Marvelous Mysterious Key.


    Introduction
    Hi, the last past weeks I was trying to find out how the binJ file format works. Simply because I wanted to change a few messages in Dragon Quest Monsters 2: Iru and Luca's Marvelous Mysterious Key. I thought this surely can't be that hard. The short answer is I was wrong, the long answer is hidden in the following spoiler.
    The first thing I did to translate a text was opening the binJ file using Notepad++. Since it directly showed japanese characters I thought, I could directly replace them and expected it to work. Even though I was looking at a binary file, which you usually can't just read with a text editor. But when I copied some texts to Google Translate to find out what they mean, I was disappointed. The texts weren't real japanese. So I googled a bit and found this thread where they chatted about binJ files.

    They wrote that these files use a custom encoding for japanese characters and characters in general. What you have to understand about binary files is that they are basically 1s and 0s. This is true for all files, but "normal text files" use a standardized encodings. This means that the byte "0110 0001" with a common encoding like ASCII or UTF-8 always stands for the character "A". But for the game I tried to translate a custom encoding is used. They encode the character "A" using the byte "0000 1101". Once I used the encoding table provided in the thread I was able to read the correct texts. Now I tried to change a single character by replacing the original bytes for this character with the bytes for the character I replaced it with. I sent the file to my 3DS and launched the game. It worked. Great.

    Now I only needed to find a way to tell the individual texts appart and luckily the texts were always separated by a special two-byte long token. Thus I wrote a small programm that read the binary data, splitted the texts and decoded the bytes to japanese texts. And it worked great. This was the point where I started developing the first version of BinJ Editor. A thing that I haven't talked about yet is that with this method you have to keep the byte length of the original text and the translated text the same. Otherwise the game won't find the texts and just crashes. The problem with this method is that japanese is a lot denser than english. Normally two characters of japanese text are translated with about eight characters of english text. But since the encoding table they used is optimized for japanese text, a lot of common japanese characters are encoded using no more than a single byte. This means the english translation for a two character long japanese text is only allowed to have two bytes as well. As an example you could translate "耐性" (resistance) with "RE" but not "RES" and definitely not with "Resistance". That was a thing I was able to overcome much later.

    But what I was able to do was shorten the texts by simply replacing not-needed characters with null bytes. And therefore I started developing the first version of BinJ Editor. After the first sketch was finished I started translating a few menu texts as a proof of concept. And this was the point were I realized all binJ files have a weird long "first text" that I never looked into. All first texts looked similar and at this point I thought maybe those bytes are the addresses of the texts in the binary file. I checked it and I was right. That means that given a list of texts (with their respective lengths) I was able to re-create this address block. Now I had two possibilites: rewrite my import function of binJ files to validate that the texts I obtain by looking at the separator tokens are correct OR just believe it works. And... I chose the second option. Which was the bad one. But at least I could now replace texts with translated texts of arbitrary length. So I rewrote a lot of parts of BinJ Editor and started translating more parts of the game. Until...

    ... there were problems. I translated some battle messages and whenever I entered a battle the game crashed. So I looked into the binary data once again and found out that this particular binJ file included the same address twice. So I had to look into the address block while parsing binJ files... I started rewriting the corresponding parts of BinJ Editor and at some point I realized that I always missed the first text of the translation before because the first text does not start with the separator token as all the other texts do. And did I mention one file contains a one-byte off pointer? Anyway... someday I got all of this to work and this basically is the whole story of how I got to decipher the binJ file format and how I wrote BinJ Editor. I later cleaned things up, added some convenient features, wrote documentation and... created this post.
    If you simply want to know more about the binJ and e file formats check these out: BinJ Format and E Format


    Presentation
    [​IMG]

    To import a file choose "File > Import..." and select the file. This process may take a few seconds. Now you can start editing the file by entering text into the last column. When you finished editing you can save a project file by choosing "File > Save As..." or export an edited .binJ file by choosing "File > Export...".
    A full explanation can be found on the Website of this project: BinJ Editor Manual

    [​IMG]

    It also includes a very handy FTP Client for directly sending the edited files to your 3DS for Luma to patch.
    More on that can be found in the Manual as well: BinJ Editor Manual

    [​IMG]

    Another very useful tool is the Search Dialog which let's you search multiple binJ, e and other files at once.
    More on this tool can be found in the Manual: BinJ Editor Manual


    Examples
    As an example I present you four screenshots from Dragon Quest Monsters 2: Iru and Luca's Marvelous Mysterious Key.
    The first one is the title screen, the second one the loading screen and the third one the status screen of my buddy Pineapple. All texts are stored in binJ files.
    The fourth screenshot shows a dialog with the father of the protagonists which is stored inside an e file.
    Note that the first screenshot of each row is the original, the second one the translated (using BinJ Editor) and the third one the translated including a custom font and custom images (cannot be done using BinJ Editor).
    [​IMG]
    Note: The monsternames are fixed, so my Slimey has a japanese name and the others have german nicknames.

    [​IMG]
    Note: The game does weird things with the width of some texts, so "AGIL" displays unreadable and "A G I L" does the job.

    [​IMG]
    [​IMG]


    Download
    The project is hosted on GitHub: BinJ Editor on GitHub
    You can download the latest version at: Latest Version of BinJ Editor


    I hope you like it
    Ich_73
     
    Last edited by Ich_73, Oct 18, 2020
    Milozaki, cearp, Nutez and 2 others like this.
  2. Helmax

    Helmax GBAtemp Regular
    Member

    Joined:
    Feb 17, 2011
    Messages:
    234
    Country:
    United States
    Where'd you get that custom font from?
     
  3. Ich_73

    OP Ich_73 Advanced Member
    Newcomer

    Joined:
    Aug 27, 2020
    Messages:
    89
    Country:
    Germany
    I took the original font and converted it to an image. I did the same thing with the font from the DQMT translation and merged them. Since the formats were a bit different I had to change some pixels manually. After that I converted the image to a bcfnt font and used that. It sounds more straightforward than it actually was, but that's basically it.
     
  4. Helmax

    Helmax GBAtemp Regular
    Member

    Joined:
    Feb 17, 2011
    Messages:
    234
    Country:
    United States
    Was very curious because that looked like the font I built for DQM TW 3DS. Cool to hear you basically went through the same process as me
     
    cearp likes this.
  5. Ich_73

    OP Ich_73 Advanced Member
    Newcomer

    Joined:
    Aug 27, 2020
    Messages:
    89
    Country:
    Germany
    I mean the latin alphabet part you see on the images is the part I took from that font you built. But since DQMT uses more symbols than DQM2 you can't just use the same bcfnt file.
     
  6. Helmax

    Helmax GBAtemp Regular
    Member

    Joined:
    Feb 17, 2011
    Messages:
    234
    Country:
    United States
    The story dialog is in the demo and field files. They're just E extension files if I recall correctly
     
  7. breaktemp

    breaktemp D-Sun who was called D-Moon
    Member

    Joined:
    Mar 17, 2008
    Messages:
    1,029
    Country:
    United States
    @Ich_73

    Really cool program
    Seems like it can get the job done when it comes to the menus and such !
    From Github, it looks like on the English side
    You have completed a bit chunk of the game.
    I do hope you can find a nice team of people who can support you with this.
     
  8. Ich_73

    OP Ich_73 Advanced Member
    Newcomer

    Joined:
    Aug 27, 2020
    Messages:
    89
    Country:
    Germany
    Right, they're in ExtractedRomFS/data/Event/**/*.e.
    It doesn't seem like Kuriimu is able to handle those files. Do you know a tool to do this, or have any other documentation of the file format? Or is it the same as with the binJ files and I would have to figure it out myself^^

    Thanks! I tried to translate all the menus I could find, so yes, the first bit is done.
     
    breaktemp likes this.
  9. Helmax

    Helmax GBAtemp Regular
    Member

    Joined:
    Feb 17, 2011
    Messages:
    234
    Country:
    United States
    I had a team that was working on a full translation of this game. All of this work is already done. The problem is, the code.bin file in the exefs has buffer limits to prevent us from rendering more than 40 characters per dialog window, and I the English translation really blows this up in size
     
  10. Ich_73

    OP Ich_73 Advanced Member
    Newcomer

    Joined:
    Aug 27, 2020
    Messages:
    89
    Country:
    Germany
    I know... Really sad you didn't make it in the end.

    Interesting... I haven't noticed the problem yet, but I was able to recreate it right now, even though I could fit 65 characters.
    Has anyone of your team had a look into the code whether it is just a hardcoded limit (because the developers thought, it would never exceed that length) or is that something worth investigating?
     
  11. Helmax

    Helmax GBAtemp Regular
    Member

    Joined:
    Feb 17, 2011
    Messages:
    234
    Country:
    United States
    I was the only one looking into it. Never figured it out. You get that figured out and we can release all our work
     
  12. breaktemp

    breaktemp D-Sun who was called D-Moon
    Member

    Joined:
    Mar 17, 2008
    Messages:
    1,029
    Country:
    United States
    @Helmax

    Insert timely "Challenge Accepted"
    But wow, that is mighty generous of you
    To offer all of those years of work to be unlocked publicly !
    Praying that these fresh eyes can investigate and turn up a solution.
     
  13. limpbiz411

    limpbiz411 GBAtemp Fan
    Member

    Joined:
    Oct 12, 2007
    Messages:
    399
    Country:
    United States
    I wish we had a joker PRO 3 translation tool.
     
  14. Ich_73

    OP Ich_73 Advanced Member
    Newcomer

    Joined:
    Aug 27, 2020
    Messages:
    89
    Country:
    Germany
    One person is not that many given the amount of people that would love to see another quality fan translation.
    I'll try my best, but I haven't looked into the code section of hacking yet. I'll keep you informed.
     
  15. JustDrew

    JustDrew Member
    Newcomer

    Joined:
    Apr 30, 2020
    Messages:
    21
    Country:
    United Kingdom
    its heartbreaking that the translation is done, but some line of code is preventing potentially the best game in the monster series from getting translated. So im guessing somewhere in that code.bin file there is a solution to all this...
     
    Last edited by JustDrew, Sep 11, 2020
  16. Ich_73

    OP Ich_73 Advanced Member
    Newcomer

    Joined:
    Aug 27, 2020
    Messages:
    89
    Country:
    Germany
    Release v2.0.0

    Thanks to @Locke-92 who contacted me saying he can help decyphering .e files I could upgrade BinJ Editor with the functionality of editing those files.
    E files are used in Dragon Quest Monsters: Terry's Wonderland 3D and Dragon Quest Monsters 2: Iru and Luca's Marvelous Mysterious Key to store cutscene and dialog data for the story.

    [​IMG]

    If you want to know the details of this file format check out: E Format

    I also took the time to implement some other useful features.
    If you're interested take a look at the release notes: BinJ Editor v2.0.0.

    I hope you like it
    Ich_73
     
    Last edited by Ich_73, Nov 18, 2020
  17. breaktemp

    breaktemp D-Sun who was called D-Moon
    Member

    Joined:
    Mar 17, 2008
    Messages:
    1,029
    Country:
    United States
    Congrats on the update !
    I find it nice that your walkthrough instructions on Github are so clear and easy to follow.
    Looking forward to this project's developments.
     
    Ich_73 likes this.
  18. Helmax

    Helmax GBAtemp Regular
    Member

    Joined:
    Feb 17, 2011
    Messages:
    234
    Country:
    United States
    Any progress on the character limit @Ich_73 ?
     
  19. Ich_73

    OP Ich_73 Advanced Member
    Newcomer

    Joined:
    Aug 27, 2020
    Messages:
    89
    Country:
    Germany
    @Helmax I simply quote myself to answer this:
     
    breaktemp and chaos_envoys like this.
Draft saved Draft deleted
Loading...

Hide similar threads Similar threads with keywords - Translating, [Release], Monsters