GBC Hacking

[NEO Ness]

I want to hack a Game Boy Color game that isn't a Pokemon game. All the tools that I've found are Pokemon Game tools. Does anyone know of any tools for text changes? In the game I will be changing Japanese text to English text. I'm a new hacker so I'm flying blind.

Any tips or suggestions would be greatly appreciated.

 

[FAST6191]

If there are no premade tools then you either get to make tools or use the more generic ones to do what needs to be done.

For a GB/GBC hardware reference I usually point people at http://problemkaputt.de/pandocs.htm

Other than that it is all pretty similar between systems
http://www.romhacking.net/start/
http://gbatemp.net/threads/gbatemp-rom-hacking-documentation-project-new-2014-edition-out.73394/

 

[NEO Ness]

Thank you for the links and information. I'm still new to all this so it's still a bit confusing, but I think I'm a little closer to solving my problem. There's a program called Poke Text for Pokemon games. How hard would it be to make that compatible with a different game? I'm no programmer, but would the task be too difficult to ask for help?

Making these tools seem to be my brick wall...

 

[FAST6191]

Text can be represented in a variety of ways in games, pokemon usually taking a slightly harder approach to things than many of its contemporary games on a platform. If you have the source code to one of the pokemon tools it could be a 30 second fix (in this game the text starts at ???? where in the pokemon game it is YYYY and the tool works accordingly), it could be so different as to not be worth altering the program or it could be somewhere in the middle.

You do not need to make tools of equivalent potency though -- figure out how the text is stored, make a table and you can often edit directly in the ROM or use tools like Cartographer to dump the script and atlas to put it back in and sort pointers.

 

[NEO Ness]

Okay, I'm catching on (slowly) but I think I'm catching on. I just used Hexplorer and made a table for the alphabet. Most of the guides you linked to me have outdated programs that are not compatible with my OS. I've got the alphabet down, now I need to find out how to find the dialogue so I can edit it. There's no Kana or Hira in the Hexplorer for some reason. I cannot find the Japanese texts so I cannot edit right now... I'll try the Cartographer and Atlas.

EDIT: I was not running it like a Japanese program, there is now Kana and Hira. Also I could not find Cartographer and Atlas. I'm still looking for a way to determine what is dialogue and what is not.

 

[FAST6191]

You managed to get those old pokemon tools to work (I tend to find they were made with some obscure version of visual basic and are a pain to get working with some installs of windows XP let alone newer) with a modern OS but not the tools linked?

"I've got the alphabet down, now I need to find out how to find the dialogue so I can edit it."
I could find a way to get the table without seeing the text, especially on more modern systems, but the vast vast majority of the time people work backwards from dialogue they found in the game to get the table.

Quick question. Would you view yourself as more of a translator? By all means continue to learn hacking (hacker-translators are rare and revered for a reason) and if nothing else certainly makes projects go smoother but you can probably find a hacker more easily than hackers find translators for their projects.

 

[NEO Ness]

I am a translator of sorts. I wouldn't say my Japanese is 'amazing' but it's decent enough for casual conversation. And I can read and write Kana and Hira flawlessly, with some Kanji to go with that. This is going to be a great challenge for me! I like challenges!

Anyways, the Poke Text worked on my Windows 7 just fine, and so did Hexplorer, but all the other ones I downloaded kept giving me an error message saying that it was not compatible with the current Windows Version. I tried compatibility mode with no luck. I never tried any other Pokemon-related software besides Poke Text.

"but the vast vast majority of the time people work backwards from dialogue they found in the game to get the table." I'm not sure what you mean by that. Do I need to approach the game differently? I found "STAROCEAN" in English, and I went from there filling in the blanks and double checking myself with Hexplorer.

"I could find a way to get the table without seeing the text, especially on more modern systems," You can? Please explain! Or do I need to provide you with more information?

 

[FAST6191]

If you have not read http://gbatemp.net/threads/densetsus-translation-toolbox.311523/ then I would, though it sounds like you could move to Japan tomorrow and do OK which means you can probably do OK here. Equally if it is what motivates you to learn more then carry on, obviously game text is largely informal but that probably will not matter in this instance.

"I found "STAROCEAN" in English..."
In that case you technically did work backwards from the text. I would urge a bit of caution there (English word/phrase in all caps in a Japanese game) as often you can put the time in to find you have a very clear understanding of the end credits, title screen or game menu and the game text is an entirely different matter.

As for finding a table without the text data/location then it is the considerably harder method in most cases. On systems like the DS there is a common font format called ntfr that has the encodings for a given character available and you can then type it, OCR it or do whatever it is you want to do. On older systems though you would probably find the font in memory, work backwards to find the code that rendered it in that order and then work through all the code that decodes the text.
Even if a hacker is doing that though then rather than work through the font/decode code then most people would then look at the text itself as you likely have the text location from the font stuff (part of it would have said copy this part of the ROM into memory and decode it using this, if you have the "copy this part of the ROM" thing then you have the text) and just fill in the blanks, tweak a few things to see what falls out or do whatever else you want to do.
All the above stuff is usually reserved for games which have things stopping the more traditional find the text (by corruption, relative search, pointer search, some statistical method, inferring/elimination.....) methods from working. Japanese games do see more of the complex stuff by virtue of relative search being less effective in Japanese but I would probably still not jump right to it.

On the matter of tools I have not been through the romhacking.net start stuff in the last couple of years but it should still have been OK for newer systems -- it is mainly game specific and really old stuff that would trouble newer versions of windows (usually be being visual basic based or possibly actually needing the 16 bit DOS sub systems). The tools I link up/cover in my guide should generally only maybe see some DLL errors, save perhaps a few of the cheat tools.

 

[NEO Ness]

"As for finding a table without the text data/location then it is the considerably harder method in most cases. On systems like the DS there is a common font format called ntfr that has the encodings for a given character available and you can then type it, OCR it or do whatever it is you want to do. On older systems though you would probably find the font in memory, work backwards to find the code that rendered it in that order and then work through all the code that decodes the text.
Even if a hacker is doing that though then rather than work through the font/decode code then most people would then look at the text itself as you likely have the text location from the font stuff (part of it would have said copy this part of the ROM into memory and decode it using this, if you have the "copy this part of the ROM" thing then you have the text) and just fill in the blanks, tweak a few things to see what falls out or do whatever else you want to do.
All the above stuff is usually reserved for games which have things stopping the more traditional find the text (by corruption, relative search, pointer search, some statistical method, inferring/elimination.....) methods from working. Japanese games do see more of the complex stuff by virtue of relative search being less effective in Japanese but I would probably still not jump right to it."

You lost me there (sorry!). I never saw a "Copy this part of the ROM", what program do I need for that? I'm using Hexplorer, is there no way to pull up dialogue/text? Will I have to manually find it? (I'm not sure I can... that's like 4 million lines of bytes to search through.)

Was writing down the alphabet with the corresponding code not making a table? If so, how do I make a proper table?

Thank you for being very informative.

 

[FAST6191]

Sorry I was talking about examining the game's code to figure out what it does on the GB/GBC CPU. At some level some of the things are basic, even if they fall under the umbrella of assembly hacking aka the thing considered the hardest part of ROM hacking, but that would involve some knowledge of how things work where the other techniques for fiddling with text that work in most cases can be done by people that know how to do a word search, crossword or similar type of puzzle.

Tables.
With very few exceptions computers only understand 0 and 1, 01010101001010 is a pain to read though so we use hexadecimal which is 4 binary numbers stacked up and that has 16 values (represented by 0 through F). 16 is not enough for basically any character set, much less one with a logographic component like Japanese, so usually between two (8 bit encoding -- 256 possible characters which is not enough for even the basic sets of kanji) and four (16 bit encoding -- 2^16 possible characters, more than enough but a pain on 8 bit systems like the GB/GBC) hex characters. There is no universal standard for this and games very much favour their own custom encoding setups. A table is a file that hackers use to make it so this custom encoding for characters can be shared and viewed in more generic tools. Find a table is then potentially grammatically iffy -- you do not find a table in a ROM as much as figure out the encoding by whatever means you can and then make a table with said data.

If the table you made corresponds to the characters in the game you need to decode and/or encode then you have made a proper table. I might be able to use some programming techniques to get to it faster or with less trial and error, you might be able to use linguistic techniques* or statistical techniques to figure things out that would see me jumping for the code to try to figure out. Probably best to stop short of a sequence that would fit in a mission impossible film to steal the source code for a game from a company but realistically if it gets you the data then it is a valid method.

*A character is meeting another after a period away and says お元??すか, I do not know Japanese and would have no clue but if you did then you would probably know the ?? section is 気で.

While I am at it as this comes up next.
Pointers.
With computers only understanding 1 and 0 they also do not do well with end of lines, end of sentences/paragraphs and instead tend to have to be told what starts where and how long it is. This is the job of a pointer which I usually like to describe as being like a contents section in a book, but for the computer in question. There are a few methods but most will be either basic maths (systems like the DS) or close to the hardware (most older systems that do not use a filesystem/CD) which is why I linked the pandocs hardware documentation in the earlier reply.

"is there no way to pull up dialogue/text"
Yeah you tell your hex editor to go to its location. The trick is figuring out where this is and it could be basically anywhere in a ROM. ROM hacking is pretty much finding the object of your interest and figuring out how the game devs stored it so you can either view it or edit it in a useful manner. Owing to the basic tenets of maths/computing (see halting problem) then this then means you get to learn lower and lower levels of code, though human laziness/computer predictability mean you do not always have to go there.

 

[Jayro]

I was changing the text in Link's Awakening with just a hex editor.

(Many games might not use plain text though)

 

[NEO Ness]

Hmmm,
maybe I'm doing something wrong? I can't seem to find a single line of readable text. It's all gibberish to me. What if the Kanji is also in a code? If that's the case then I'm not sure I'll be able to do this project after all



I just opened Pokemon Yellow with the Hexplorer and I didn't find any dialogue either. I'm beginning to wonder if I can even change dialogue this way?

Is there another way? Or am I missing something very crucial...

 

[FAST6191]

The text decode on the right hand side is using some other encoding method. The game is likely not using whatever that one is that (it is why we have tables), or it may be compressed, or it may be wrapped in other code/markup (probably not on the GB/GBC). You can edit anything you like with a hex editor, however it very quickly becomes a pointless exercise in abstraction so we use other methods to simplify it/render it somewhat more human readable.

Other than Crystlatile2 my personal favourite editor with table support worth speaking of is Translhextion http://www.romhacking.net/utilities/219/

 

[NEO Ness]

Transhextion didn't load the Japanese text. But surprisingly CrystalTile2 did! Even the Kanji!

You keep talking about Tables like they're a file. Do I need to make mine into a file? (I just listed the alphabet and wrote down the codes next to them, is there more to it than that?)

"You can edit anything you like with a hex editor, however it very quickly becomes a pointless exercise in abstraction so we use other methods to simplify it/render it somewhat more human readable." That is my goal! I've been trying to find something to allow me to read it normally without the code. Crystaltile2 and Translhextion were both Hex Editors. I'm not sure how to use them other than that. I can see how using only hex editors would become a very hard task, what else can I use? I feel like I've ran out of options...

EDIT: I don't think we're on the same page anymore. I just tried to search the Hex for a sentence that was in the opening cut-scene. No luck. Didn't even come close. So I'm guessing that the Hex isn't where I need to be to edit the Text in the game. (this one at least). I cant think of anything else to do...

 

[FAST6191]

A hex editor is a very dumb program, however that is sometimes what you need

The left column is the typically the address within the file. The middle section (which may be broken into columns of various widths) is the actual content of the file and the right hand column is whatever decoding you are using on the file -- it means nothing until you set it up and is just there because it is useful for many types of file. There are a series of common encodings (ASCII, shiftJIS, unicode, EBCDIC, euc_jp and many more besides) but games use their own for all sorts of reasons (some coding related, some ego on the part of the devs related, some a bit of both).
You can change this encoding the game uses later but for getting the initial text out you have to figure out what it uses. Figuring out what it uses is half the fun, so to speak. You store this encoding data in a table file and feed it to a hex editor that knows what they are to decode the hex via this custom encoding you have in the table file.

If the text is compressed you will probably have to decompress it and if the text has some markup in the middle (not all that likely on GB/GBC games but far from impossible or unheard of) it will not be pure text but close enough for most purposes.

You can make a table file in a basic text editor if you want but most would suggest you use something like http://www.romhacking.net/utilities/55/ or http://www.romhacking.net/utilities/709/ as they can do nice things like insert whole character sets at given points to save you the aggro.

As the hex editor is quite literally the file in all its glory you can note that maybe 33 hex = A (34=B....) but in the ASCII window it will display a 3 and then if you wanted to write CAB then you would change whatever it is to 353334 in hex or 534 in the ASCII window but that makes for a horrible workflow and is very prone to errors. This is why we then have hex editors that can read and decode things using table files so you can just type normally.

 

[NEO Ness]

Okay, so if I'm understanding you correctly I need to use a Table file with the Hex Editor in order to actually edit the text.

So where I went wrong was: I needed a Table file.

Now I need to use Oriton in order to make a accurate table file... I've been up all night trying to wrap my head around this. I really do appreciate the help!

Any advice on how to make an accurate Table file?

 

[FAST6191]

For the most part you only need the characters you are using -- maybe not so much when you are as constrained as you are on the GB/GBC but fonts can include any number of pieces of punctuation that both you or the game will probably never use. I mention it mainly as I see some new to this try to find everything where it might not be necessary.

Finding the table itself I had a whole chapter on in the hacking docs of mine I linked. It has pictures and worked examples for a lot of them. Without looking at this game specifically I would not know which approach will work best but it would be one of those covered, or usually some mix of them.

Personally I would start by finding the text in the game and then filling in the blanks. There are many methods again though some of the nicer ones in non Japanese games do not work so well with Japanese (relative searching being the big one). The entry level technique to find things would be corruption -- take your hex editor and invert or otherwise change part of the game, run it and see if the text has changed in an emulator (do not use savestates, possibly also consider either making a new ROM or closing the emulator and opening it again), repeat until you find some portion of the text has been corrupted. Once you have found the location then typically the rest of the text is nearby (as hopefully will be the pointers) but you can also fill in blanks, do more directed changes (if I know 30 is something then I can put a bunch of them and then 31 to see what 31 is).

 

[NEO Ness]

Okay, I think things are flowing again. But I think I've run into a bit of a problem...

The combination of 8,9,E, and F lines are a series of Kanji that depend on the order of the line. (ex: E0 E1 = 珮 E1 E0 = 矜) I don't know how to chart that. Or is it even necessary?



Did I do this correctly?

EDIT: Would knowing the game help? It's Star Ocean: Blue Sphere for the GBC.

 

[FAST6191]

E0E1 is 4 hex values or 16 bits of character (8 bits is 2^8 or 256 combinations which is not enough to even do the kana and some common use kanji) and those programs are aimed more at 8 bit stuff which is what most European language games use. Indeed converting Japanese games from 8 bit to 16 bit is a higher level but commonly performed hack on older systems.
I can not recall offhand how to do it in Oriton but in tablular there should be an option to insert or include 16 bit encodings for those things that need it. Hunt around the menus and there should be something like that in any table making program worth its salt.

Also in case you had not seen them then the two common older encodings used to represent Japanese (nowadays most are advised to go for unicode, even if it follows from these) are euc_jp
http://www.rikai.com/library/kanjitables/kanji_codes.euc.shtml
and shiftJIS
http://www.rikai.com/library/kanjitables/kanji_codes.sjis.shtml

shiftJIS is the more common in newer games but both have been observed in games. It will probably not be the complete set used by the game but it might well be in the same order, or share part of the encoding.

Yeah knowing the game would help if I was hacking the game but I was purposely keeping it somewhat generic. You mentioned star ocean earlier too, though I must confess I did not know there were any for the GBC.

 

[NEO Ness]

I read a good portion of that .PDF you linked me about Hacking Roms 2014.

I think I learned how to apply the table file to my Hex. But now I've run into some other issue, that I'm not sure how to fix (or if I can even fix it).

When I saved my Table I had the Kana alphabet decoded (right term?), then when I applied it, I could not find any Kana in the Hex. So I go back and check my Table by opening it with Oriton. I find this when I reopen it:

I feel like things are getting wrong and worse by the minute.