[Request Help] Latin characters in JP version of Diabolical Box

MRJPGames

Pretty great guy
OP
Member
Joined
Aug 17, 2013
Messages
1,199
Trophies
1
Location
The Netherlands
Website
fizazy.com
XP
1,686
Country
Netherlands
Hi, someone reached out to me if I could help them with the technical side of translating Diabolical Box JP (as the JP version has a special "Holiday" mode missing from the international version).

Though only having experience with CV I did set out to help, 'cause rom hacking is fun, and is something I want to encourage.

However I can't really seem to solve the core problem: English text is not rendered.

So what did I try:
1.
Simply modifying the .txt files in the rom to contain English/Dutch text.
This does not crash the game. however this results in empty text boxes.
1668525480862.png

2.
Using Shift_JIS encoding for "Full Width" or JASCII letters.
This sorta works, problem is it's "Full width" which makes it highly impractical, and it looks pretty terrible.
1668525444432.png

3.
Attempting to replace Japanse characters that do render with latin letters in the font used (font.NFTR), then using shift_jis encoding for those Japanse characters.
This provides more or less the same result, unless I can find symbols that internally already have the same parameters set for width, distance between chars etc. (Which yes I did update in the font.NFTR file, but from what I can tell either Tinke does not update the file correctly (unlikely as the changes persist within Tinke upon full reload, or more likely the game simply disregards this part of the font data for whatever reason...)
1668525671747.png

Main difference between this and the Full Width version that can be done without font modification is that the letters are smaller here, but as Japanese characters tend to be rather wide comparatively they end up taking up (about?) the same amount of space.


I was able to find this old thread https://www.romhacking.net/forum/index.php?topic=30895.0 which might be wholy unrelated, however it too seemed to be a case of regular ASCII (just like first attempt mentioned in this post) not working. However I have not been able to find much on what "2 byte" encoding would be, other than that it might be referring to Full width, in which case due to seeming lack of effective font editing would really not be a great solution...

If any additional info is needed in order to help, let me know!
 

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,796
Trophies
3
XP
28,412
Country
United Kingdom
2 byte encoding is what you dubbed full width -- the shiftJIS table ( http://rikai.com/library/kanjitables/kanji_codes.sjis.shtml ) does feature Roman characters (and Greek/Cyrillic a bit later) in the 82?? range which means 16 bits/2 bytes per character. Officially ShiftJIS should support 8 bit encodings for said Roman letters but most games that feature it do not bother hence this, if it is a NFTR using title though then you might be able to add them.


NFTRedit might be a better bet for a tool to play with the font as well
https://gbatemp.net/download/nftredit.29196/
Think that was the latest version/fork that worked with a revision found in some very much later games but if not there is a thread around here somewhere (same name/title), and the change was minor enough you could change and back again with a hex editor.

Equally on "impractical" then other than maybe not being the best spaced (a real concern don't get me wrong) is there much impractical with it? Some games might store things in memory and face limitations that way if you start writing reams of text but that is going to be rare for a puzzle game. If it is forcing it then various Japanese IME/word processor type things will be able to force it trivially. JWPCE being the method of choice for many (free, easy enough to use) https://www.at-it-translator.com/jwpce/ but probably a bit long in the tooth right now, njstar was popular as well. I forget the specific usage but along the top row will be some buttons (three or four of them, one was an A, others some Japanese symboles, one is normal ASCII/Windows IME, others will do Japanese according to whatever scheme it uses to input and a final one should force said 16 bit encodings in addition to various Japanese offerings but you don't much care there.
Though some also opt to use https://www.romhacking.net/utilities/504/ in conjunction with the above if saving out is going to be annoying.
You might be able to convert things across from ASCII (or at least U16 unicode) by addition as well easily enough if you wanted to go that way.
 
  • Like
Reactions: lattechan

MRJPGames

Pretty great guy
OP
Member
Joined
Aug 17, 2013
Messages
1,199
Trophies
1
Location
The Netherlands
Website
fizazy.com
XP
1,686
Country
Netherlands
Thanks for the reply!

Those are indeed what I dubbed "Full Width" (afaik that's what unicode calls them?), and those are the latin (roman?) symbols that work with the default font.

As for font editing, I gave that tool a try and it's quite nice, but seems to do the same thing Tinke does. I've since the original post been able to confirm Tinke changes the font correctly (and/or at least the same way NFTRedit does). The problem is that Diabolical Box seems to disregard the metadata(?) entirely. As an easy example I changed a character's properties to all 0s, so 0 length, 0 width and 0 offset. The result:
1668529229954.png
1668529713129.png

(I had 4 of that character at the beginning for testing)
This makes me suspect that the game always uses the 14px length, that every used character has by default (except the special "Friendly Ban" version stuff on top, which is a separate font and likely not really useful here.)
The games does seem to use the width parameter though, as if I set it to a value lower than the width of the character only part of it is drawn (eg. set it to 3 and only the first three pixels are drawn, but the next character remains unmoved).
I haven't tested with just changing offset, as it shouldn't really help me afaik.

As for impracticality, it's really just about the spacing in-game. It wouldn't be too much trouble for me to write a little script that takes all the txt files in normal ASCII and converts it to this custom "full width" encoding.

Currently the way it's looking to me is that the likely somewhat easy solution (as I'm not skilled in binary editing) would be to find where the game stores this constant width, replace it with a smaller constant width that could be used for all roman/latin letters and than change the "Full Width" part of the font to look about right with that mono spacing and do it that way? Is there anything I might be missing, or does that also seem the most likely compromise between easiness of the work and functionality?
Or alternatively just take the hit, use the wide characters and deal with it. Biggest problem there would really just be the puzzle texts often taking up most of the top-screen when properly formatted, thus it would require shortening the content quite a bit... Which might make the puzzle harder to solve... Than again the aforementioned hack would have the same problem, just to a lesser extend.
 
  • Like
Reactions: lattechan

mn1712trungson

Member
Newcomer
Joined
Oct 15, 2022
Messages
14
Trophies
0
Age
30
XP
282
Country
Vietnam
This problem can be solved with Nintyfont and NFTR Edit.

First, you need to use Nintyfont to add the ASCII or Latin character map (if needed).

About the range of Unicode blocks.
Starting from any character is fine, but must continue until the end of the block.
5.JPG


In the glyph properties box, press ADD and enter the code point value (0x20 to 0x7E for ASCII).
1.JPG

Caution: don't import BITMAP value or font will broken.

Next, we check the font with Tinke to confirm the character map has loaded (type 0 for proportional, any character in type 2 will skip the offset value and become full-width characters; I not test type1). More information in: problemkaputt.de/gbatek-ds-cartridge-nitro-font-resource-format.htm
2.JPG

(ASCII character map loaded correctly - Type 0) (First char = decimal value in Unicode chart)

3.JPG

(Double check on offset 0Ch - 0x2 - value = 00 00 (type 0))

Now, you can draw tile with NFTR Edit or Crystaltile and settings width value.
4.JPG


Warning: please always backup your font. Anytime NFTR Edit can't open your modified font, it means your font is broken.
 
Last edited by mn1712trungson,
  • Like
Reactions: lattechan

MRJPGames

Pretty great guy
OP
Member
Joined
Aug 17, 2013
Messages
1,199
Trophies
1
Location
The Netherlands
Website
fizazy.com
XP
1,686
Country
Netherlands
I've been trying to do this for ~an hour. I'm running into some issues... However before I detail those, is it even worth continuing provided the following:
The font provided with the international (EU specifically) version of the game, which uses the same ASCII characters (though seemingly indeed better properly included in the font, which seems not to be the case in the JP version). However even when just straight up using that font instead of the default font, it just won't render anything. As that font has no issues in the EU/US versions (duh) I have my doubts if any modifications to the JP font will be successful.

As for problems I ran into trying your approach:
- The "Add" button in NintyFonts does absolutely nothing for me (I'm using "Second Alpha" release from GitHub)

However the font already includes all non-special latin letters (so é etc. not included, but alphabet, numbers and punctuation is).
So I tried adding a character map for them (not present by default!), I tried using Tinke for this, which did not work (Cast error).
In adition Tinke is not able to modify Character Map type from 1 to 0 (same/similar cast error). (I was hoping to simply update the type of FULL WIDTH chars, as they DO display, and hope to either modify those tiles, or point towards the pre-included ASCII char tiles.)


I'll be doing some more playing around with things on my end. Will update this comment with any additional details (if I find any).
Post automatically merged:

So I did some more testing and it's making my rather confident that this is not an issue with the font but an issue with the way the game uses the font...
Here is a modified version of Diabolical Box JP and a modified version of Curious Village JP.
Modifications:
- Diabolical Box JP uses Curious Village JP's font.
- 1 Text box is changed in each game to:
"先生が、遺産相続騒動の解決を?
Roman Text Test!
Roman Text Test!"

This is the result:
1668791016852.png

It should look exactly the same, but it does not. For whatever reason DB-JP doesn't want to render the regular ASCII characters, while CV-JP has no issue with them (though still being quite w i d e)

However I still have a bit of hope left for a "custom" font that uses 2-byte chars encodings to encode all the ASCII characters. This will require me to force type 0 somewhere, or make a font somewhat from scratch... We'll see if I get around to it. Will post again if I do with my findings :)
 
Last edited by MRJPGames,

mn1712trungson

Member
Newcomer
Joined
Oct 15, 2022
Messages
14
Trophies
0
Age
30
XP
282
Country
Vietnam
Sorry for not telling you how to do it specifically for NintyFont.

When pressing the ADD key. You need to click in the last box to change the code point value.
Only Nintyfont, as far as I know, can create custom character maps.
Tinke was simply determining whether or not the character map was correct.

1.JPG


Finish and save the character map. Load font on Tinke and check character map value.
BTW, the "Roman Text Test!" you show.
It's actually half-width Japanese character set in the end Unicode chart.

2.JPG


And the reason why EU fonts do not appear on Japanese rom is straightforward.
Japanese rom encoding char will read the backwalk hex value (Byte 3-4 and Byte 1-2).
00 54 = T on Big Endian but on NDS it's become 54 00 Little Endian.


LE_string_value.JPG


If you need some help. I can modify the font (ASCII character map) for you.
Post automatically merged:

I spent some time debugging both games (EN) and JP.
I realized NFTR font in rom acts like a bitmap font + table (encoding in any format is fine).
So rom skip the next offset value and only take width value.
Bitmap font is drawn on RAM and pushed straight to BG1 (like NES or SNES games).

In JP rom, two-byte hex values serve as an opcode (0D - 0A for break lines, 40 = @ + number).
1.JPG

That means there must be a routine that reads the string in 2-byte format before processing the txt in 4-byte format.

They removed the 4-byte routine from EN ROM and processed the opcode + text in 2-byte format as well.
Of course, NFTR encoded Shift-JIS as well (next offset value used). Only change is txt file encodes ASCII.
 
Last edited by mn1712trungson,
  • Like
Reactions: MRJPGames

lps

man with no face
Newcomer
Joined
Sep 13, 2009
Messages
55
Trophies
1
Location
Crimea, UA
Website
tagteam.ru
XP
335
Country
You’ll need to decompress overlay9_002 and find font width table:
USA offset: 0x1EF58
EUR offset: 0x1F158
Try to find something similar for JAP rom.
 

MRJPGames

Pretty great guy
OP
Member
Joined
Aug 17, 2013
Messages
1,199
Trophies
1
Location
The Netherlands
Website
fizazy.com
XP
1,686
Country
Netherlands
Sorry for the late reply
Sorry for not telling you how to do it specifically for NintyFont.

When pressing the ADD key. You need to click in the last box to change the code point value.
Only Nintyfont, as far as I know, can create custom character maps.
Tinke was simply determining whether or not the character map was correct.

View attachment 338526

Finish and save the character map. Load font on Tinke and check character map value.
BTW, the "Roman Text Test!" you show.
It's actually half-width Japanese character set in the end Unicode chart.

View attachment 338527

And the reason why EU fonts do not appear on Japanese rom is straightforward.
Japanese rom encoding char will read the backwalk hex value (Byte 3-4 and Byte 1-2).
00 54 = T on Big Endian but on NDS it's become 54 00 Little Endian.


View attachment 338528

If you need some help. I can modify the font (ASCII character map) for you.
Post automatically merged:

I spent some time debugging both games (EN) and JP.
I realized NFTR font in rom acts like a bitmap font + table (encoding in any format is fine).
So rom skip the next offset value and only take width value.
Bitmap font is drawn on RAM and pushed straight to BG1 (like NES or SNES games).

In JP rom, two-byte hex values serve as an opcode (0D - 0A for break lines, 40 = @ + number).
View attachment 338626
That means there must be a routine that reads the string in 2-byte format before processing the txt in 4-byte format.

They removed the 4-byte routine from EN ROM and processed the opcode + text in 2-byte format as well.
Of course, NFTR encoded Shift-JIS as well (next offset value used). Only change is txt file encodes ASCII.
Sorry for the late reply, life kinda got in the way.

Regardless I have learned a lot about NFTR format, which is very nice. However I think editing the font won't help in the end.
I've been able to add new chars, to modify fonts to "move" ASCII to JASCII/Full Width encoding for them. However whatever I do the game will use static/constant spacing regardless.

Also you said that the mapping "type" as displayed in Tinke, and described here has something to do with proportionality. However it seems to me that it only controls how the character map is read, as described in the documentation of the format on problemkaput's website. So if I'm not mistaken it only controls how that mapping is done, is it a start at this tile and continue for entire range, is it a linear list of the entire range with the tile id's or is it a list of char encoding to tile id. None of which have much to do with proportionality (eg. type 2 will still be treated proportional in other game if I'm not mistaken)

As for the actual encoding/decoding used I think it's just proper shift_jis. 2 byte characters are processed seemingly normally, just not displayed. I believe this because introducing only one of such characters does not cause all characters afterwards to get jumbled up because they are no longer aligned. And as CRLF are done with 0x0D0A, both 2-byte chars, not the 4 bye variants you showed.

From all of this I'm concluding that my original suspicion that proportional width is simply being ignored by the rendering code is most likely correct?


You’ll need to decompress overlay9_002 and find font width table:
USA offset: 0x1EF58
EUR offset: 0x1F158
Try to find something similar for JAP rom.
I'll give it a shot, but my binary executable hacking skills are close to 0 (experience at least). But I'll see if I can get started with those US/EU offsets!
 
  • Like
Reactions: mn1712trungson

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
  • BigOnYa @ BigOnYa:
    Surprised they never made a second one.
  • K3Nv2 @ K3Nv2:
    It takes two was their newest different theme
  • BigOnYa @ BigOnYa:
    I didn't care for that one so much, but didn't play it much either.
  • BigOnYa @ BigOnYa:
    I just played little of "Still Wakes The Deep" , just came to gamepass today, its alright, kinda creepy. I'm not usually a horror game fan, but it looks really good graphic wise, esp w 4k. I admit it made me jump a few times.
  • LNLenost @ LNLenost:
    Anyone here using the 3DS?
  • LNLenost @ LNLenost:
    do u have pretendo?
  • LNLenost @ LNLenost:
    whats ur code?
  • B @ btjunior:
    i dont have pretendo sorry
  • LNLenost @ LNLenost:
    oh ok
  • LNLenost @ LNLenost:
    well if ur console isnt modded i suggest u to mod it
  • B @ btjunior:
    also, just curius, does the freinds app still work after the shutdown?
  • LNLenost @ LNLenost:
    yes
  • B @ btjunior:
    nice
  • LNLenost @ LNLenost:
    but u cant play online games anymore :(
  • LNLenost @ LNLenost:
    i miss going in other peoples town
  • B @ btjunior:
    yeah
  • LNLenost @ LNLenost:
    i have a friend named Pineapple (@pineappleJuice hi). we played a lot the day b4 the server shutdown
  • K3Nv2 @ K3Nv2:
    I preten to know doe
  • BigOnYa @ BigOnYa:
    @LNLenost how well is the pretendo servers, you ever get kicked off games when playing online?
    +1
  • LNLenost @ LNLenost:
    @BigOnYa not always you get kicked. it happens rarely to me.
    +1
  • LNLenost @ LNLenost:
    ofc they're not great, but props to them for making this
    +1
  • LNLenost @ LNLenost:
    btw, I decided to make the nintendo support team going insane
  • LNLenost @ LNLenost:
    i told them my badges disappeared after i opened the badge arcade after the shutdown, but actually they disappeared because i opened the badge arcade w/ pretendo
  • AcuteBulbasaurappears @ AcuteBulbasaurappears:
    Hey, is there anybody who knows how to create wuhb-files? :) I would like to create forwarders for two Wii-channels: https://gbatemp.net/threads/wuhb-to-vwii-channel-request.657253/#post-10441514
    AcuteBulbasaurappears @ AcuteBulbasaurappears: Hey, is there anybody who knows how to create wuhb-files? :) I would like to create forwarders...