ROM Hack Help with Hex Editing texts

[†Nømura]

Hey, guys, I am having a weird problem with hex editing and was wondering if anybody could help me figure out what's going on.

I am trying to translate the descriptions of players in IE3 The Ogre, which are located in (data_iz/logic/unitbase.STR)
The Hex Editor I'm using is MadEdit.

I have no problem with the hex editing itself. As you can see in the picture below, what happens is that Roman Alphabet characters do not show up in-game. Only Japanese characters do. In fact, I am able to succesfully replace the original Japanese text with other Japanese characters.

Seeing this, I immediately thought something was wrong with the font tables in the rom, and I went on to check them. It turns out that the player descriptions I'm trying to edit refer to the same font table used by story texts, which work fine, as you can see in this other picture:

In summary, in the player descriptions only characters from 0x8141 and forward in the font table show up during gameplay. All characters before that address don't work.

What could the problem be? Thank you in advance.

 

[FAST6191]

Sounds like you have figured out the problem already; the game for this part of the text only uses a sub section of the font/encoding, even apparently despite it using more of it elsewhere. It is not so common on newer systems, you used to see it, or the related font swap out, all the time on older systems though.

Choices are
Edit the game such that is uses a larger set, may come with memory issues and is generally annoying to do if you do not need the space.
Edit the game such that it references the lower set instead. Doable but still effort for no great gain.
Edit the font. If it is all Japanese and you are going to a Roman character set using language then just overwrite (part of) the Japanese set with another batch of Roman characters and use them instead. It is pretty arbitrary what the values are so it the game has it all already there then so it goes.

 

[†Nømura]

Choices are
Edit the game such that is uses a larger set, may come with memory issues and is generally annoying to do if you do not need the space.
Edit the game such that it references the lower set instead. Doable but still effort for no great gain.
Edit the font. If it is all Japanese and you are going to a Roman character set using language then just overwrite (part of) the Japanese set with another batch of Roman characters and use them instead. It is pretty arbitrary what the values are so it the game has it all already there then so it goes.

Thank you so much for confirming my doubts
There is one more problem, though.

It's true that I could overwrite the japanese set font with roman characters, but when hex editing, these new roman characters would occupy two bytes like the japanese ones. This means they would take too much room in the description file, and I wouldn't be able to make the translated descriptions fit in (I can't edit the pointers so as to gain more room either).

The only other option left would be to edit the refered set like you said. I'll have to look into where the section value is stored, though. I have no idea regarding where it could be.

Anyway, thank you again

 

[FAST6191]

To find the decode values then you pretty much have to reverse the text engine, or find where it reads the text string it wants and tells it what tiles to pull in from the font. I first thought you meant the earlier section had say 0020 and up as well as 08?? and beyond but the descriptions only did 08?? and up, similar enough and what I said still stands but an interesting twist. What you describe that you want to do is known as a 16bit to 8bit encoding conversion or some jumble of those words. Not so common on the DS as space is fairly plentiful (I know deufeufeu's Jump Ulitimate Stars patch did it anyway) but common enough on older systems where space meant something.
You might be able to get the game to use the other text decoding function that does support the characters you want, it is not without its own issues though.

It has been a while since I looked at the inazuma games but I don't think the font is nftr, and it behaving like this would also say it is probably not. If it was though then the encoding is held in the format unlike most other things which are all buried in the text handling engine, I mention that mainly so if someone searching the forums in years to come finds this then do pull apart the encoding that the font format itself carries.

That said what is the problem with changing the pointers? I recall the inazuma project I was somehow involved in had some issues with the place names being in the binary (and thus a complete nightmare to edit pointers for) but surely the descriptions are fairly conventional text with conventional pointers.

 

[†Nømura]

It has been a while since I looked at the inazuma games but I don't think the font is nftr, and it behaving like this would also say it is probably not. If it was though then the encoding is held in the format unlike most other things which are all buried in the text handling engine, I mention that mainly so if someone searching the forums in years to come finds this then do pull apart the encoding that the font format itself carries.

That said what is the problem with changing the pointers? I recall the inazuma project I was somehow involved in had some issues with the place names being in the binary (and thus a complete nightmare to edit pointers for) but surely the descriptions are fairly conventional text with conventional pointers.

Actually, the font is nftr, Does that mean I should look into the font file instead of pulling apart the text engine? That'd be awesome *_*

The pointers seem to work differently than those for the story text, and I haven't figured out how yet.

 

[DarthNemesis]

The letters in your translated screenshots don't match any of the 4 font files in data_iz/font; which font are they coming from? But as for the character descriptions, those are using FONT12.NFTR, which doesn't have any single-byte letters in it. I tried swapping in a different font that does have them, but the text engine still only displayed the double-byte english letters.

Edit:

The pointers seem to work differently than those for the story text, and I haven't figured out how yet.

The strings in unitbase.STR are all padded out to multiples of 0x20 before the next string starts, so I tried dividing the address of each string by 0x20 and those values do show up in unitbase.dat (e.g. the pointer for 0x022DC0 is 0x116E, and that shows up at 0x020526 in unitbase.dat).

 

[†Nømura]

The letters in your translated screenshots don't match any of the 4 font files in data_iz/font; which font are they coming from? But as for the character descriptions, those are using FONT12.NFTR, which doesn't have any single-byte letters in it. I tried swapping in a different font that does have them, but the text engine still only displayed the double-byte english letters.

I'm confident that the story texts I have translated DO use FONT12.NFTR. In fact, I tried edting one character (the lower case t) in this file and the change showed up, as you can see:

And also, the FONT12.NFTR I'm using does have single-byte letters in it, as you can see in the screenshot below. I'm not using the original FONT12.NFTR, I imported the one from Inazuma Eleven 1. This one has both single-byte roman characters (red box) and double-byte roman characters (blue box). I uploaded it in this message, it's inside the ZIP archive.

 

[DarthNemesis]

Ah, that's why the letters didn't match. Well, you should be able to use double-byte characters for the player descriptions and resize the strings as necessary. Do you have an inserter or are you actually editing all the pointers manually?

 

[†Nømura]

Ah, that's why the letters didn't match. Well, you should be able to use double-byte characters for the player descriptions and resize the strings as necessary. Do you have an inserter or are you actually editing all the pointers manually?

I'm doing it manually with a hex editor. There's no inserter for IE3 unfortunately /: Could I ask you how are you finding the address of the pointer in unitbase.dat?

Even by editing the pointers, though, if I were to use double-byte characters the spaces between words would take up two spots instead of one. I'll try to cram them into the description box.

Maybe there is a way to make the single-byte letters work. By looking into FONT12.NFTR and editing the character map, the letters displayed in the player descriptions change (although no single-byte letters...)

If only I could find the correct way to edit them...

Thank you for your support

 

[DarthNemesis]

I'm doing it manually with a hex editor. There's no inserter for IE3 unfortunately /:

That's a crazy amount of work. Tell me what you've figured out of the file formats so far and I should be able to write one.

Could I ask you how are you finding the address of the pointer in unitbase.dat?

Just did a search in a hex editor (with the bytes reversed since they're stored in little-endian order).

if I were to use double-byte characters the spaces between words would take up two spots instead of one. I'll try to cram them into the description box.

That's because 8140 takes up 10 pixels, but you can change the offset and next offset to 3 to match the size of 0020.

 

[†Nømura]

I can't seem to find the pointers for the few first players. The string of the second player listed in unitbase.dat starts at 0x00000080, so diving by 0x20 tells us the value of the pointer must be 0x4. But the only 04 there is near the player's section is not the pointer. Changing it, in fact, does nothing.

EDIT: Nevermind, I found them.

 

[†Nømura]

Sorry, one last thing. Do you know how the pointers for the story texts work? I was able to find those for address up to 0xFFF0 by searching for the same hex string in little-endian order in the evet.pkh file, but from 0x10000 and on this doesn't seem to work anymore...

 

[DarthNemesis]

The pointers in evet.pkh seem to point to 4 byes before the start of each string in evt.pkb (three 00 bytes and the length, for all the ones I checked).

 

[superfranci99]

It has been I while I haven't worked with this game...
After the abandoned italian translation I went back on it trying to figure out some issues I had found during the project.
I found out that this problem isn't caused by the font files or a particular encoding. It seems to be a bug on the asm code...
In few words, when the game wants to print a text on the screen, it can use more than one function... The problem is that one of this functions has an unnoticed (since the game itself just uses shift jis chars) bug that can't print single-byte chars. (for example the bugged function is used for the video subtitles too)
Anyway it seems I found the solution, but I still need to test it in the way to assume it works correctly (I'm actually still working on debug without making any change on the arm9 code)

Here you can find some informations about pkb and pkh files in Tinke source code.

 

[†Nømura]

Thanks to both for your precious help!