ROM Hack How Do I Dump/Insert Text To Translate Japanese Ds Game Without Using A Hex Editor?

[HeyImAmethyst]

I am trying to translate a ds game named Tongari Boushi to Mahou no Omise. The text encoding is in UTF-8 and I already made a table for it. I was able to translate a few words in one of the files that stores the text that appears in the class selection menu but I have only been using Crystal Tile 2's hex editor to translate and it's slow doing it that way and changing the text in a text editor cause the text to glitch. I have seen that I can dump the text and then reinsert it after translating so I tried looking for was I can do this. I have seen a program called cartographer that dumps the text and Atlas the reinserts it and I saw one called abcde that does both but I don't know how to use them and I haven't seen any tutorials on them either. I am still trying figure out cartographer as I am typing this (its giving me this "Line 1: First character of the line is not a recognized table character" error and I'm not sure why).


Edit: I uploaded the original and my edited version of the file that I am editing in case anyone wants to take a look.

 

[FAST6191]

It is 2 in the morning as I write this so not really got much chance to look at things. For the general case though

Yeah hex editors are not much use for more than initial cursory analysis, some light typo fixing or simple find and replace. Anybody you see doing anything is likely doing a lot of work on the back end.

Crystaltile2 itself did start life way way way back as crystalscript so does have some limited abilities (well actually quite impressive in scope but buggy and tricky to use) in searching for and dumping text.

Atlas and cartographer are a combo that some use for extraction and insertion. They also lean more towards NES and SNES which can make things harder for the DS which tends to be more file level.

Kruptar7 is popular in Russia (there is a reason they have half a hundred translations and some very good people)
http://www.magicteam.net/index.php?page=programs
Tutorial of sorts
https://romhack.github.io/doc/kruptarPlugins/

Many however say forget it and make their own program. As far as things go if your programming language of choice has any kind of data handling (which because computers is most of them, though not all) then the harder part is usually figuring out the text itself.
Making it all a nice GUI is harder, however the most basic form for most is the just a tool to recalculate pointers -- if it is like many DS games and has a nice end of line indicator you can use those to calculate pointers and have it quickly recalculate pointers for you (search all say if the game uses 0000 then that and note locations, pointers should be directly related to that)

"changing the text in a text editor cause the text to glitch"
Crystaltile2's hex editor is not the most pleasant but it is usually solid. If you are getting glitches it is likely then with your edits themselves -- UTF8 is not something I have the most experience with on the DS but if it plays out like the shiftJIS and EUCJP stuff then many games will only encode what they need, and any lower value Roman character values will not be included (though might have alternatives within the game).

 

[HeyImAmethyst]

Atlas and cartographer are a combo that some use for extraction and insertion. They also lean more towards NES and SNES which can make things harder for the DS which tends to be more file level.

I seemed to have been able to get the text to dump from cartographer but I'm not sure if I configured the pointers right. At least for a few of the files I found that contain the text, the pointers are at the top of the file. With the ones that handle the text for the class selection there is a value before the pointers start. I am not sure if this value indicates the start of the pointers or if it is used for something else.

And I read in this post that to find the pointer for a hex string on a DS ROM I need to get it's address and convert it to little endian: http://www.romhacking.net/forum/index.php?topic=27415.0. There seems to be gaps in between the pointers too. Not sure if I am looking at them right but this is what it looks like for some of them:

Kruptar7 is popular in Russia (there is a reason they have half a hundred translations and some very good people)
http://www.magicteam.net/index.php?page=programs
Tutorial of sorts
https://romhack.github.io/doc/kruptarPlugins/

I saw this one while researching and I'm still confused on how to use it even with the tutorial. I'm still figuring it out though. It says in the tutorial that I can "specify places in ROM, which can be used for text storage". How do I do that? I think this is how to tell the program where the text files are but I don't see an option to do that.

Edit: I think I found it but I'm stuck at getting the text to display.

Crystaltile2's hex editor is not the most pleasant but it is usually solid. If you are getting glitches it is likely then with your edits themselves -- UTF8 is not something I have the most experience with on the DS but if it plays out like the shiftJIS and EUCJP stuff then many games will only encode what they need, and any lower value Roman character values will not be included (though might have alternatives within the game).

Oh. I was thinking that maybe because typing the letters in by using a text editor instead of editing the hex vaules shifts everything over which corrupts everything. At least in game when the text would appear as ****** so I was thinking that maybe the null values got shifted over to where the text needs to be.

 

[Rosa2709]

hola espero que puedas hacer este proyecto lo quiero jugar in english i hope you can do it!!!

 

[MrVtR]

I recommend Kruptar7 too, I'm working with it in some DSi/3DS/Switch games, once you learn how to use it ,it's easy to extract and inject texts

--------------------- MERGED ---------------------------

I seemed to have been able to get the text to dump from cartographer but I'm not sure if I configured the pointers right. At least for a few of the files I found that contain the text, the pointers are at the top of the file. With the ones that handle the text for the class selection there is a value before the pointers start. I am not sure if this value indicates the start of the pointers or if it is used for something else.

And I read in this post that to find the pointer for a hex string on a DS ROM I need to get it's address and convert it to little endian: http://www.romhacking.net/forum/index.php?topic=27415.0. There seems to be gaps in between the pointers too. Not sure if I am looking at them right but this is what it looks like for some of them:
View attachment 232975

View attachment 232976
View attachment 232977



I saw this one while researching and I'm still confused on how to use it even with the tutorial. I'm still figuring it out though. It says in the tutorial that I can "specify places in ROM, which can be used for text storage". How do I do that? I think this is how to tell the program where the text files are but I don't see an option to do that.

Edit: I think I found it but I'm stuck at getting the text to display.



Oh. I was thinking that maybe because typing the letters in by using a text editor instead of editing the hex vaules shifts everything over which corrupts everything. At least in game when the text would appear as ****** so I was thinking that maybe the null values got shifted over to where the text needs to be.

You must set the pointers configuration(like size,interval, reference to sum with the pointer and where it starts/ends), after that, the tool will load all the texts identified by the pointers

 

[HeyImAmethyst]

I recommend Kruptar7 too, I'm working with it in some DSi/3DS/Switch games, once you learn how to use it ,it's easy to extract and inject texts

--------------------- MERGED ---------------------------

You must set the pointers configuration(like size,interval, reference to sum with the pointer and where it starts/ends), after that, the tool will load all the texts identified by the pointers

Oh its been a while since I came back to this ^^'. I was able to get the text to show up but there is still this extra /00 at the end of each line. I have the ptReference still at the default value, Im not sure if thats the reason why. I'm still confused on what to put for the ptReference. You put that its reference to sum with the pointer and where it starts/ends but I'm confused on what you mean by that. Here is what I was able to get:

 

[TigerTiger]

Maybe @InochiPM knows? I guess it might be different on DS to 3DS but I feel like it would be helpful to compare

 

[MrVtR]

Oh its been a while since I came back to this ^^'. I was able to get the text to show up but there is still this extra /00 at the end of each line. I have the ptReference still at the default value, Im not sure if thats the reason why. I'm still confused on what to put for the ptReference. You put that its reference to sum with the pointer and where it starts/ends but I'm confused on what you mean by that. Here is what I was able to get:

View attachment 273786

This extra /00 won't dissappear unless you build a plugin to remove it and insert it when you run the script
It's absolutely normal and it represents the End of String on the file(That you stabilished on "String Terminator" inside kryptar7), no need to worry about it
For me, I make a lot of scripts for 3ds and switch, normally, the String terminator on it is 00 00 00, so, Kruptar ends all the strings with /00/00/00

--------------------- MERGED ---------------------------

Oh its been a while since I came back to this ^^'. I was able to get the text to show up but there is still this extra /00 at the end of each line. I have the ptReference still at the default value, Im not sure if thats the reason why. I'm still confused on what to put for the ptReference. You put that its reference to sum with the pointer and where it starts/ends but I'm confused on what you mean by that. Here is what I was able to get:

View attachment 273786

About the ptReference, it's an value to sum with the pointers values, this is called Relative Pointer
Like: ptReference is 3000 and first pointer have 0F value, so in 300F position it's supposed to start an text that's mapped to the 0F pointer
Normally, only few games use this kind of pointer, the normal case is the Absolute Pointer: ptReference with 0 and sum with the pointer value
If you were able to extract the texts without bugs with ptReference: 0, leave with this value

--------------------- MERGED ---------------------------

Oh its been a while since I came back to this ^^'. I was able to get the text to show up but there is still this extra /00 at the end of each line. I have the ptReference still at the default value, Im not sure if thats the reason why. I'm still confused on what to put for the ptReference. You put that its reference to sum with the pointer and where it starts/ends but I'm confused on what you mean by that. Here is what I was able to get:

View attachment 273786

Just for you to compare, this is an print from my Bravely Default translation project to Brazilian Portuguese
You can see that my String terminator is 000000, but all the texts ends with /00/00 only
This is an adjustment that I made on the Characters table, each char have 2 bytes, one for the letter and one 00 byte, so letter A would be 41 00. With this adjustment on the .tbl file, my kruptar project can work with only /00/00 on the end, cuz the letter before the string terminator already have an 00 byte on it, so if my phase ends with A, I would have in bytes: 41 00 00 00

About the ptReference, on Bravely Default, the game use in every file the Relative Pointers, so I can't leave ptReference with 0 value, however, the value that I must use is in hex header of the file(In little endian format on print 2), and you can check on ptReference from print1 that I put the exact hex code to work

 

[HeyImAmethyst]

This extra /00 won't dissappear unless you build a plugin to remove it and insert it when you run the script
It's absolutely normal and it represents the End of String on the file(That you stabilished on "String Terminator" inside kryptar7), no need to worry about it
For me, I make a lot of scripts for 3ds and switch, normally, the String terminator on it is 00 00 00, so, Kruptar ends all the strings with /00/00/00

— Posts automatically merged - Please don't double post! —

About the ptReference, it's an value to sum with the pointers values, this is called Relative Pointer
Like: ptReference is 3000 and first pointer have 0F value, so in 300F position it's supposed to start an text that's mapped to the 0F pointer
Normally, only few games use this kind of pointer, the normal case is the Absolute Pointer: ptReference with 0 and sum with the pointer value
If you were able to extract the texts without bugs with ptReference: 0, leave with this value

Oh ok! Thanks!

Just for you to compare, this is an print from my Bravely Default translation project to Brazilian Portuguese
You can see that my String terminator is 000000, but all the texts ends with /00/00 only
This is an adjustment that I made on the Characters table, each char have 2 bytes, one for the letter and one 00 byte, so letter A would be 41 00. With this adjustment on the .tbl file, my kruptar project can work with only /00/00 on the end, cuz the letter before the string terminator already have an 00 byte on it, so if my phase ends with A, I would have in bytes: 41 00 00 00

About the ptReference, on Bravely Default, the game use in every file the Relative Pointers, so I can't leave ptReference with 0 value, however, the value that I must use is in hex header of the file(In little endian format on print 2), and you can check on ptReference from print1 that I put the exact hex code to work

Thanks for the examples! How were you able to determine that the value you needed to use was in the header though? And how would I determine the ptReference value if I only know where the text is and not the pointer value but I know where the pointer table is? Would I have to guess the value or would the value be stored somewhere like with the value you used and how would I figure out where that value would be stored? Is there a set method for finding the values? Sorry if this is a lot of questions or if its confusing ^^'.

 

[MrVtR]

Oh ok! Thanks!



Thanks for the examples! How were you able to determine that the value you needed to use was in the header though? And how would I determine the ptReference value if I only know where the text is and not the pointer value but I know where the pointer table is? Would I have to guess the value or would the value be stored somewhere like with the value you used and how would I figure out where that value would be stored? Is there a set method for finding the values? Sorry if this is a lot of questions or if its confusing ^^'.

In the Bravely Default case an friend of mine could search the hex code and saw that pattern on the files, he ended up teaching me what I know about reverse engineering
So, in my case, the value is always declared on the hex code, but this iss the exception of the exception from what I've seen in 3ds and switch games at least, but, if this happens, the value i'll be stored in the file's header, so probably in the first lines of the hex code file

Normally, the ptReference is the first letter from the text table, so, if your first text table starts on 0xAAA, the ptReference is AAA and in most of the cases in relative pointers, it'll have low values on it, so e.g, one pointer have 02 00 00 00 on it in little endian, the text will be on AAA+02, so, AAC
I guess it doesn't have a method to find these values, it's a boring job of check the hex code and look for patterns on it, the most common patterns is some values with the file length, string length,where some area starts on the code(like in Bravely Default, that have some byte areas for the texture and texts), the ptReference,etc

 

[HeyImAmethyst]

In the Bravely Default case an friend of mine could search the hex code and saw that pattern on the files, he ended up teaching me what I know about reverse engineering
So, in my case, the value is always declared on the hex code, but this iss the exception of the exception from what I've seen in 3ds and switch games at least, but, if this happens, the value i'll be stored in the file's header, so probably in the first lines of the hex code file

Normally, the ptReference is the first letter from the text table, so, if your first text table starts on 0xAAA, the ptReference is AAA and in most of the cases in relative pointers, it'll have low values on it, so e.g, one pointer have 02 00 00 00 on it in little endian, the text will be on AAA+02, so, AAC
I guess it doesn't have a method to find these values, it's a boring job of check the hex code and look for patterns on it, the most common patterns is some values with the file length, string length,where some area starts on the code(like in Bravely Default, that have some byte areas for the texture and texts), the ptReference,etc

Oh ok! Thanks so much!