Japanese Programming Madness

Discussion in 'NDS - ROM Hacking and Translations' started by Auryn, Feb 5, 2012.

Feb 5, 2012

Japanese Programming Madness by Auryn at 2:12 AM (6,736 Views / 0 Likes) 32 replies

  1. Auryn
    OP

    Member Auryn GBAtemp Advanced Fan

    Joined:
    Jul 21, 2011
    Messages:
    529
    Country:
    Switzerland
    While bored at work I started parsing my NDS japanese game collection and naturally take a look at the file structure.
    It didn't take long to find a game that had something strange: apparently no font.
    The game I am talking about is Kenshuui Tendo Dokuta (Japan); a sort of Trauma Center Clone.
    It's looking like this game has sections of text compressed in one file. This file contains a NCLR/NCGR and many NSCRs all without header and pointers.
    The NCGR is all letters that are needed for that section of text only and there is a NSCR for each screen of text.
    Here a pic of the first text screen and the NCGR (note the か that is reused):
    [​IMG]

    Do you have some other japanese programming crazyness to share??
     


  2. rastsan

    Member rastsan 8 baller, Death Wizard

    Joined:
    May 28, 2008
    Messages:
    963
    Location:
    toronto
    Country:
    Canada
    7th dragon... its 250+ font files intermixed in the one text file. There is the one block font file of the basic characters with then every file (with text) adds it own more complex letters onto that font table. could be one extra font grouping per file up to the aforementioned 250+. it goes text block font text block font. each text block has its own different new font table for the font that goes with that block. making translating it a tiresome process. As a new table needs to be made for every font just so that text can be dumped properly. where one font file and one table would have saved space and given the writers and programmers less of a headache.

    (can you imagine one letters value meaning over 400 different letters in one game? that value staying in use through the entire game but getting a new graphical letter for it depending on the text...)

    No disrespect to the programmers or owners of the copyright to this game was intended in this post.

    This conversation is strictly for research study purposes.
     
  3. FAST6191

    Reporter FAST6191 Techromancer

    pip
    Joined:
    Nov 21, 2005
    Messages:
    21,737
    Country:
    United Kingdom
    LifeSigns: Surgical Unit was the English sequel no?

    Anyhow so the "character encoding" is in fact a bitmap image reworked using a tile map (SDK formats no less) to make text. Interesting- I have pondered such a thing in years past when trying to think up a way to explain some of the graphics concepts and font handling but never really expected to actually see an implementation. Would also go some way to explaining the difficulty in finding text in the rom when I had a quick snout around it years ago.

    My favourite in recent times still has to be Wizard of Oz - Beyond the Yellow Brick Road http://gbatemp.net/topic/300953-pac-files-help/ (it gets a bit unreadable as things go on so apologies) which more or less has the script as a low rent scripting engine (I doubt it is Turing complete and certainly not in an obvious way but still fairly far reaching compared to your average text engine that might have placeholders if it is especially high end).

    Equally http://gbatemp.net/topic/305167-the-various-audio-formats-of-the-ds/ has a couple of things, I also liked the zombie daisuki game if only because it had some odd ideas (actual text for one- no pointers or fun things like that).

    I really should start pulling apart random roms again. Might even provide me with a bit of incentive to do something with my random roms hacking thread ( http://gbatemp.net/topic/282588-thread-of-a-thousand-rom-hacks/ ) I started and never did anything for.
     
  4. Auryn
    OP

    Member Auryn GBAtemp Advanced Fan

    Joined:
    Jul 21, 2011
    Messages:
    529
    Country:
    Switzerland
    @rastasan: a pain in the ass to extract but you will probably overwrite all the fonts with the same one when you will reinsert the translation...no??

    @[member='fast']: interesting topics


    Some more from Ace Attorney Investigations 2
    On AAI2 there are 3 fonts and all 3 have a tile width table at the beginning but for some reason, only 2 are used, the last one is just ignored and if you think that the one that are used are the one for the main text and the logic descriptions, and the one that is ignored is the one for the profiles/evidence descriptions, it's completely illogical.
    By the way we are still searching for some good hacker to fix little problems as this one.

    Other curious things is that AAI2 seems to like the bottom of every memory limits/bank and need to repoint almost everything that has to do with the text (even the text window) if expanded.

    The next fact is probably done because of translation but there a character in the game with up to 4 different name codes that make appear the same name tab but actually are the same characters in different situations (present time or flashback or even different case).
    Adding to this, maybe there are up to 2 more name codes for make appear the "???" before that the character is presented or he/she introduce herself O.o.

    The last one is probably not so exciting because very common in many games: there are leftovers of the demo version of the game and naturally unused text as well.
     
  5. Nagato

    Member Nagato GBAtemp Advanced Fan

    Joined:
    Jul 15, 2011
    Messages:
    529
    Country:
    United States
    Reminds me a lot of Nanashi no Game. Basically, there was a font file in each script archive. The scripts addressed each glyph it needed and the font file didn't duplicate things. Everything in the font file was also in the order it was encountered in the text, just like this. So instead of ここが今日から… it just wrote こが今日から…. If you look at the first script it points to 0x5a14 for the start of the scripting. You can find the string in the image with the indexes "0x0000 0x0000 0x0001 0x0002 0x0003 0x0004 0x0005 0x0006 0x0007 0x0008 0x0009 0x000A 0x000B 0x8000 0x8007 0x0001 0x000C 0x000D 0x000E 0x000F 0x0010 0x0011 0x0012 0x0013 0x8007 0x0000 0x0004 0x8001 0x8000" which comes out to be something along the lines of "ここが今日からお世話になる[n][color]聖命医大付属病院[/color]か[l][n]"

    Why? My guess is it's supposed to be some kind of memory optimization.
     
  6. rastsan

    Member rastsan 8 baller, Death Wizard

    Joined:
    May 28, 2008
    Messages:
    963
    Location:
    toronto
    Country:
    Canada
    @Auryn oh no I gave up on 7th dragon. I was on my x table file and going blind from identifying my 15 thousandth character.... Only to realize that I had three specific characters wrong throughout near every table and thus having to redump what I had dumped not quite re doing all the translation already inserted. that and the one person spamming my unlisted (unrelated to rom hacking in any way) email address with 7th dragon related "whens it gonna be done"... minimum 200 times a day.
    If i do it again I am gonna watch one specific area in memory with an assortment of breakpoints before each and every font, thus saving time by letting the game make the table(s) for me. Or re-writing the main font file completely... I say this with a big IF.
     
  7. Foxi4

    Reporter Foxi4 On the hunt...

    pip
    Joined:
    Sep 13, 2009
    Messages:
    22,732
    Location:
    Gaming Grotto
    Country:
    Poland
    The largest "Japanese Programming Madness" instance I can think of is Sigma Harmonics by Square Enix for the DS.

    The game's font is non-standard and encrypted to the point that it's physically impossible to translate it, not only for the fans but apparently for distributors aswell, since the game was never released abroad nor was it attempted by anyone in their right minds, all because the perspective is Portrait and the font goes from top to bottom vertically rather then horizontally (as it should in Japanese, but still).

    [yt]GxxqtdRlzJI[/yt]

    Check the ROM out - it's like it's black magic in .nds format.
     
  8. Nagato

    Member Nagato GBAtemp Advanced Fan

    Joined:
    Jul 15, 2011
    Messages:
    529
    Country:
    United States
    Sigma Harmonics really isn't that big of a clusterfuck. The font is in the pretty common NFTR format and there is no encryption. There are tools to extract the archives (including the official tool from the SDK) so unpacking and repacking isn't a problem either. Sigma Harmonics had *some* oddities but not any of the ones listed.

    Time for a little rant since it gets me every time people claim how impossible some game engines are.

    There is nothing that makes games played in portrait mode any harder to hack than normal games. People seem to jump to the font as being a big issue but it really isn't one. To understand that you must understand what a font really is. Speaking strictly in terms of what the NDS can do, a font is a collection of images for each character that will be displayed on screen. To display the sentence "Hello, world!" it will require an image for each unique character in the sentence ("Helo, wrd!"). A font file on the NDS is a collection of the character images and sometimes includes some extra data such as the width of the character depending on how complex the format is.

    Here's the font I used when I was messing with Sigma Harmonics to give you an idea about how the font would work:
    Warning: Spoilers inside!

    When turned the same way you would turn the DS to play Sigma Harmonics:
    Warning: Spoilers inside!

    No modifications needed to the font besides inserting rotated ASCII and/or remapping the characters to the normal ASCII range.

    Next is the vertical text problem. Generally (I'm sure someone can find something that is weird) when you are displaying text on the screen you will have an X and a Y position. Those correspond go the coordinates on the screen. At some point in the game's code there will be a part that calculates the position of the next character to be displayed. Assuming the DS is being held in standard mode, a game with horizontal text it will be modifying the X position to move it left or right. A game that wants to write the text vertically will modify the Y position instead. Now if we want to think about this in portrait mode you just flip the axes. In portrait mode, if you want to display the text vertically you would modify the X position. To display the text horizontally you would modify the Y position. If the game is displaying text vertically in portrait mode then you just need to modify the code so it's modifying the Y position instead of the X position to display horizontal text in portrait mode.

    The other problem is since the game displayed a border around all of the text, that must also be fixed. The frame itself is generated programmatically taking in the edge offsets and then filling in the rest. A pretty good rule to remember when dealing with games is "If it can be see it happening, it can be hacked". That includes things that you normally don't think of as seeing such as enemy AI. The difficulty involved ranges but if you see it happening, it can definitely be hacked. Everything the game uses to run must pass through somewhere at one point. All you need to do is find one of those places and trace it back far enough to find the code you want. Anything involving graphics is easier to find because you are guaranteed it will eventually end up in the VRAM.
     
  9. Foxi4

    Reporter Foxi4 On the hunt...

    pip
    Joined:
    Sep 13, 2009
    Messages:
    22,732
    Location:
    Gaming Grotto
    Country:
    Poland
    I did not know that, thank you very much. I was convinced that it was quite the clusterf*ck, hence no translation surfaced.

    I'm pretty sure you can't exactly edit the way the text window shows up though as it's not "a typical string" like it would be in the case of pre-set values for text but actual compiled code, so it would be more appropriet to "fool it" rather then change it. Example:

    Letter1Line1 Letter1Line2 etc EOL
    Letter2Line1 Letter2Line2 etc EOL

    Effectively ending up with the letters displayed horizontally rather then vertically as in the original. It's alot of work though, but I'm sure you get the gist of what I'm saying. Thing is, with that sort of thing employed, we'd likely end up not having enough space in the String anyways, so... yeah. Shame.
     
  10. Nagato

    Member Nagato GBAtemp Advanced Fan

    Joined:
    Jul 15, 2011
    Messages:
    529
    Country:
    United States
    The strings in the game are just typical strings. There's nothing fancy going on. It stores "Hello, world!" just like that. You don't need to do any fooling besides changing it from moving on the X axis to moving on the Y axis. Doing it the way you showed is just overcomplicating it since it's a problem to be solved in the game's code, not in the data.
     
  11. Sora de Eclaune

    Member Sora de Eclaune Baby squirrel, you's a sexy motherfucker.

    Joined:
    Feb 15, 2011
    Messages:
    2,785
    Location:
    123 Fake Street
    Country:
    United States
    ...Have I ever told you how much I love you?
     
  12. Auryn
    OP

    Member Auryn GBAtemp Advanced Fan

    Joined:
    Jul 21, 2011
    Messages:
    529
    Country:
    Switzerland
    Well thanks Nagato for explaining that even if I kwew it :P

    @[member='Foxi4']: I believe the % of games made by Square/Squaresoft/Square-enix that made it abroad was always less than 50% in any system but on the NDS I believe it's less than 30% or even 25%. Not because a game is difficult to translate that it doesn't get abroad (if so we would probably never got FF7 then :P).
    That game is sure not easy to translate (none are) but it's sure possible to do even if the changes it needs make it a game of medium-high difficulty to hack.
    As for the space, you see that the game doesn't use the whole height of the screen so not much different as the space you have horizontaly and if really needed, you can make the font a bit smaller as the original.
    By the way, the script of the game will look the same (horizontaly with normal line breaks as we are writing here) in the original and the translated version, only the graphic rappresentation on screen will change.
    I am sure that if somebody like Nagato would fix the text window and the text direction, somebody would sure pick it up very quickly.


     
  13. Nagato

    Member Nagato GBAtemp Advanced Fan

    Joined:
    Jul 15, 2011
    Messages:
    529
    Country:
    United States
    I actually did fix that stuff (sorta. I left it in a halfway state since I only did it to learn how to do stuff on the NDS) and handed it off to the person who was already working on the translation but I haven't heard anything about it since then. You can find that discussion here. The last thing I heard is that it was on hold since Valiarchon was working on some other projects. You can always ask him for an update.
     
  14. Auryn
    OP

    Member Auryn GBAtemp Advanced Fan

    Joined:
    Jul 21, 2011
    Messages:
    529
    Country:
    Switzerland
    Interesting Nagato. Do you have the site/source of where you learned the ASM stuff for the NDS??
     
  15. Nagato

    Member Nagato GBAtemp Advanced Fan

    Joined:
    Jul 15, 2011
    Messages:
    529
    Country:
    United States
    Unfortunately no. I wish I did since I've been asked that a few times. I've been reverse engineering games (almost exclusively x86 up until that point) since the mid 2000s so I had a lot of experience that I was able to apply to working with the NDS. The good thing about working with the NDS (and the GBA I suppose but I haven't messed with that) is that it uses the ARM architecture. ARM is a well documented architecture so I got a copy of no$gba debug version and, with the ARM documents in hand, whacked at it until I got it to work. I no longer work with no$gba since I find it doesn't meet my needs anymore but it worked well for what I needed back then. These days I use IDA Pro and Desmume with the GDB stub compiled since it's a much more powerful combination and I'm familiar with IDA Pro.

    ARM Architecture Reference Manual: http://www.scss.tcd.ie/~waldroj/3d1/arm_arm.pdf (Warning: huge PDF)
    ARM Instruction Set Reference: http://infocenter.arm.com/help/topic/com.arm.doc.qrc0001l/QRC0001_UAL.pdf
    GBATek (GBA/NDS-specific information): http://nocash.emubase.de/gbatek.htm
     
  16. rastsan

    Member rastsan 8 baller, Death Wizard

    Joined:
    May 28, 2008
    Messages:
    963
    Location:
    toronto
    Country:
    Canada
    but well worth the read...
    I got more from the arm architecture than any of the others... I can't remember if that one includes the arm7 too... which may seem pointless but there are games with arm7 programming.
     
  17. greator

    Member greator GBAtemp Regular

    Joined:
    Jun 14, 2008
    Messages:
    128
    Location:
    In your head...
    Country:
    Malaysia
    Isn't the big company who distribute this games in english are given all the tools needed to translate the game?
    Unlike us who have to reverse engineer the game.
     
  18. FAST6191

    Reporter FAST6191 Techromancer

    pip
    Joined:
    Nov 21, 2005
    Messages:
    21,737
    Country:
    United Kingdom
    One would think so greator but in practice they can often end up in a worse position than rom hackers do and end up doing many of the same tricks- http://www.loekalization.com/mistakes.html has a fair bit of insight.

    Granted some of the highest end ones and top tier companies will have nice methods by which to effect a translation and/or full source code (source code is perhaps surprisingly not necessarily the most useful thing you can give your translation/localisation team) but not all.


    As for assembly and related concepts I will also throw in http://www.coranac.com/tonc/text/toc.htm . It is for the GBA but most of it carries across.
     
  19. pleonex

    Member pleonex GBAtemp Regular

    Joined:
    Jan 16, 2009
    Messages:
    197
    Location:
    South of Spain
    Country:
    Spain
    Interesting post... I have found a lot of forgotten files in NDS games. Some of then very funny

    File name: "how_to_create_highlights.txt" in FIFA6, and many other gams of EASport
    Warning: Spoilers inside!


    Can anyone image what game has this image? :)
    http://dl.dropbox.com/u/3981393/Tinke/image.bmp


    Also, the game "Planet 51" has the images compressed in a common ZIP file (with folders too).

    Maybe later I can add more files like code files, DLLs, batch or executables (they like to forget remove files) :P

    NOTE: if this is against the rules, please tell me or/and remove it, I'm not very sure.
     
  20. rastsan

    Member rastsan 8 baller, Death Wizard

    Joined:
    May 28, 2008
    Messages:
    963
    Location:
    toronto
    Country:
    Canada
    "for research and discussion purposes" should cover this.
     

Share This Page