ROM Hack Handling differing string lengths when translating help?

[roseverdict]

As more of a "let's see if I can even do this" kind of project, I'm working on translating my copy of Picross DS into Esperanto. (Hopefully, if this goes well, I'm thinking I might try tackling translating Chocobo and the Magic Picture Books: The Witch, the Girl, and the Five Heroes into English. Probably just Google translate English for my personal use because I'm an impatient sonuvagun who doesn't know a lot of Japanese, but English nonetheless. Once my copy of that game gets delivered, anyway, since I couldn't find a single rom of it to save my life. And I looked.)

I'm fairly new to hex editing, though this isn't my first rodeo with programming in general, so I'm hoping I'll be able to pick up on things well enough that I don't immediately fall flat on my face. Something that's throwing me off, though- how do experienced translation romhackers deal with the fact that different languages take up different amounts of space to convey the same concepts? Like, "Quick Save Data found" in English turns into "Rapida Konservaj Datumoj trovitaj" in Esperanto, which. THAT'S A LOT MORE CHARACTERS. I'd been sorta squeezing past that issue when save data came up beforehand by just chopping out the konservaj/save part, since the rest of the sentence was able to more-or-less still stand on its own without it, but for this one I'm kinda screwed. I can barely imagine trying to work around the built-in character limits like that for an entire story-driven game!

...unless the character limits are all in my head and it's actually perfectly fine to alter the length drastically, but I don't think that's the case.

 

[FAST6191]

Most games will have pointers, which work much the same as they do in programming. For the DS typically for text it will be at the start of the file (some archive formats might have them in a separate file).
For those new to it entirely then I usually liken it to a contents page of a book, and similar things happen to the page counter robot when you rip out a bunch of pages or shove new ones in. As games change similarly to books once printed (which is to say not at all -- the read only part of ROM seeing to that) then no sense wasting resources calculating text sizes like it is some dynamic website and you note the sizes of the text, which the ROM hacker then has to sort and is the first real hurdle in learning ROM hacking.

Three broad styles, maybe four.

1) Standard pointers. At the start of the file will be a list of numbers (probably flipped because endianness) that correspond directly to the end of sections, end of lines in some cases (rarer on more modern devices compared to 8 and 16 bit stuff, though seen enough to note) or end of screen (say 3 lines but auto new line) or some other similar concept.

2) Offset pointers. At the start of the file will be a list of numbers but rather than starting counting at the start of the file they will start at some other point within the file, typically the start of the text section itself or the end of the pointer section.
You will tend to see these if you grab a list of the end of whatever (sometimes you have a nice 00 or other choice value to indicate it) and all the numbers have the same distance between them but are a fixed amount off from the real value.

3) Relative pointers. At the start of the file will be a list of numbers but to calculate the pointer you take the location of the pointer, the value the pointer holds and add them together. Made more sense in old school computing but the ROM hacker's job is not to reason why as much as deal with it, and technically does have

4) Sometimes lengths of text strings/files are used instead.

5) Rare for the DS but you might encounter it if you play on modern PCs or optical media based systems. Sector addressing. 32 bits (there is a reason it is called a 32 bit console) meaning 2^32 is a large number but still tops out around 4 gigabytes if you address directly. Call each section 2000 bytes or whatever then at the cost of some storage space as you waste what remains when you don't fill a sector completely you can simplify the pointer handling. Or if you have ever wondered why Windows says size of file and size on disc then here we are.

I have encountered other types of things out there, shifted pointers (some 3d formats, and Touch Detective), pointers might not only be pointers but also formatting for the, 2^32 as mentioned is 4 gigabytes so chop 1 bit off and you are left with 2 gigabytes (still larger than more games) but have a bit to do something with (the popular NARC archive format using it to indicate subdirectories), pointers don't have to be at the start and could be at the end/interspersed throughout the file (pointer, data section, next pointers, more data, pointers the third, oh no more data...), I once encountered a quasi scripting language (Riz Zowad/Wizard of Oz - Beyond the Yellow Brick Road) that had each entry be both the scripting which included a length value and the payload it carried... and there are many other ways things might get twisted around. Hopefully you are not dealing with anything like that for your first. Equally while 2^32, or even 2^16 for some more limited formats, might sound like a lot of data available and the ROM might even handle it then you can still run out of memory as things get copied to it to run, though cross that bridge as and when it becomes apparent it is a problem.

I have some worked examples in https://gbatemp.net/threads/gbatemp-rom-hacking-documentation-project-new-2016-edition-out.73394/ , and several others around the place as well.
My usual workflow is find something to key off in the text itself (most games I encounter have 00 or something unique to end a text section, such that you can search a file/section for that, note where it occurs, dump that in a spreadsheet and then recalculate after your changes).

 

[roseverdict]

Oooooh!!! That's super-helpful, and your rom hacking doc looks to be really comprehensive! I'm just gonna. Yoink that for future reference in case I get any other hubris-filled romhack ideas XD