Hacking Need help with text extraction/insertion from file in PSP game

[Michiyuro]

Um, I'm trying to hack some otome PSP games, I manage to get all the files out, but the script files - meaning files containing the script text - are kinda troublesome for me, I don't know how to get the plain script text from them and insert them back. There's actually a tool for this. But it only supports some few versions of the script, if you try to edit anything unsupported, it'll stop working and the file will corrupt. You can see and extract plain text from the tool but it doesn't support inserting back. So it's a pretty great tool already, just not enough to use for patching/translating ><
If you need to see the script files I uploaded them here.
Knowing text hacking is really time-consuming and troublesome, I'd really appreciate it if someone can help me. My level is nowhere near breaking down bytes and hacking them to get the text whatsoever It'd be really great if you can do it, or maybe show me the steps on how to.
Thank you so much!

 

[Kelebek]

The name of the game would be really helpful. That github says it's Hakuouki but there's a bunch of games under that name.

 

[Michiyuro]

Oh, sorry, it's Norn9 by Otomate ;__;
Edit: It's made for Hakuouki but the tool works for some other game's scripts as well, but it has some problems as listed above and also if you edit the script directly instead of extracting it, the character's name can't be changed.

 

[thexyz]

There's a 80 bytes long COLLECTION_LINK section at the end which only contains size of script file. hkki does not support this and fails to update some other file sections (i.e. EXPORT_DATA) because of this.

 

[Michiyuro]

Um, then is there anything I should do about it? Is deleting that possible though?
But even with hkki I can't change the characters' name, or if I can make it work/editable by hkki I can just edit the names manually.
Anyhow, thanks to jjjewel I managed to edit it manually by replacing the Japanese text with the English one.
The problem is there's no wordwrap or auto line break, the font in-game is really big and wide also ;__;

Edited:

In-game look:

 

[jjjewel]

The font problem will probably need ASM hacking to fix, or if you're lucky, it might be solved if you can fix the font.
Some games force big space no matter how small the original font is. In this case you need ASM hacking to fix it. But some games have font with big space, so if you modify the font and make it narrower, you can fit more text in one line. (Anyway, I don't have the game so I can't tell which category this game belongs to.)

For manual line break, you can add 0x0A to the code. (If I remember correctly. That should force new line for Otomate games. Otherwise, try 0x0D or 0X0D0A.)

For example in your English script at address 0x000203EC, if you change 0x20 to 0x0A, it should force line break after "tell a" and have "good story to" in new line.

(This method worked when I tried with some DS games. I think it'll work for PSP games as well but I can't guarantee. You'll have to try and check.)

Anyway, it's better to find someone to work on text extractor/inserter especially for a long visual novel.

By the way, does the name always go vertical like the ？？？ in the pic? Try to check how many alphabets you can fit for the names. (There's one place in file 100 with クラスメイト as the speaker name. Try changing it to classmate or something longer and see if it shows up properly as the speaker name.) It might need ASM hacking or something to accommodate long names.

 

[Kelebek]

Can't get any of the script to breakpoint at all, which makes making an extractor incredibly difficult.

New lines are done in separate strings though, so I don't know if it would support newlines directly as well, but I hope so.

 

[Michiyuro]

It does support breaking lines manually, but the line break from the original script is still there though. Is there anyway to delete/replace it @@ Here's how the line from before look like after changing 20 to 0A. But it swallows the "g" in "good story" right after the break though, maybe I should put a space before the word then.

And I tried the name thingy too. Apparently the name's way too big as well, and yes names always show up vertically like that ;__;
There's a boy name Suzuhara Sorata (鈴原空汰) and the whole name's too long so I only type in Suzuhara (it fits perfectly in the space for japanese text, no more no less lol) to test, here's how it appear like:

The classmate thing also:

A screenshot of how the original Japanese name will look like if you need:

Oh and thank you jjjewel and Kelebek so much for being here to help me!

 

[jjjewel]

Can't get any of the script to breakpoint at all, which makes making an extractor incredibly difficult.

New lines are done in separate strings though, so I don't know if it would support newlines directly as well, but I hope so.

The explanation for hkki program at the end of page (the Code Section) should be sufficient to write an extractor.
https://github.com/lnz/hkki/

The problem would be the inserter as this file format needs a lot of pointer updates. (The codes are separated into blocks and there are absolute addresses inside the block that you need to update, plus the absolute addresses that point from other parts outside the blocks.)

You only need to look for opCode D4 for speaker, D2 for line(s) of dialog(s), and D3 for end of dialog.
In the spoiler is a sample of how the codes look. (Sorry that it looks a bit ugly. I did it with Paint. ) You can see where the code starts and ends, its opcode (in green circle) and the code's length underlined in pink.)

Spoiler

For a few other games by this company that used the same file format, I didn't have any problem combining multiple lines for D2 code to one big chunk with manual line break (0x0A.) But I didn't test this game.
Ex. In stead of
OpCode D2 - Line 1
OpCode D2 - Line 2
OpCode D2 - Line 3
OpCode D3 (End of dialog)

You can combine them to
OpCode D2 Line 1[0A]Line 2[0A]Line 3
OpCode D3 (End of dialog)

This way you have more freedom to control the length of text in each line.

You can also combine only two first lines and then null the third line with 0x00 if you don't want text in that line to show on screen.

The problem with current hkki is the Collection Link section as thexyz explained, and it seems like hkki assumes that OpCode D2 always follows OpCode D4 immediately. But that was the case for old DS games. In PSP, they added a bunch a of other codes like OpCode 7A for mouth movement in this game. That's why for some parts, hkki won't keep the speaker name and dialog together.

(*I might sound like I know a lot. It's because I've been messing with this company's games for years. But programming for something like hkki is still beyond my ability. T_T I still think it would be best to ask for a modification of hkki program to handle this game.)

 

[Kelebek]

Just looking for D2 isn't reliable though, you could end up with false positives or miss out some strings.

 

[jjjewel]

Just looking for D2 isn't reliable though, you could end up with false positives or miss out some strings.

Ah, you're right.

There are opCodes E7 for choices, D8 for chapter names (which usually appears when you save/load the game.)

Other than that, the switch opCodes (usually 0x33 or 0x34) will tell you where each choice leads to or sometimes when the scene branches somewhere else.

And then some background/CG/character sprite loading codes can be identified as well, but usually these parts aren't needed for the translation.

 

[Kelebek]

I didn't mean that, I just meant, look in the files for how many D2s you find, loads of them. Many of them having nothing to do with text at all, and used for absolute pointers for other opcodes or just as parameters for various opcodes etc. So then what do you do, search for D2000000 instead? Well you still find non-strings, because values can end with a D2 and then be followed by a 00000000 opcode or another parameter or something. So then 00D2000000? Well can you guarantee that every text string has a zero before it and isn't following something else, because whatever that is isn't a part of the D2 opcode at all?

There's no real way to determine what's a real D2 opcode without some heavy testing, that's what I mean.

 

[jjjewel]

The game codes go by block, so I didn't mean to look for D2 randomly anywhere in the script. What I meant was to check for OpCodes for each block and if it's D2, then it's a dialog text. And for this, I'm repeating what's explained in the readme of hkki on this page; https://github.com/lnz/hkki/ Following the instructions there, you'll always get the real opCodes, not random bytes in the scripts.

Anyway, I can't help with this project as the programming and the needs for ASM hacking are something I'm not capable of, so I'm just sharing what I know about the file format. I hope you or someone can help making tools for the project.

 

[CompCom]

The game's script format is a bit annoying but I think I have managed to create a tool that works with it (Thanks to the notes from the hkki tool).

It still needs a bit of testing but does appear to work fine. As for the character name I haven't found the code to modify that yet. However I have modified the ASM code to allow for the use of unencrypted cpk data on a real psp.

 

[SonYume]

Is there, somewhere a tutorial how to use that hkki? All I have are files without known extension. Actually I'm new with this so I don't know yet what program (except hex editor for translations) I need. Does anyone can help me, please?

 

[Miraeyoo]

Is there, somewhere a tutorial how to use that hkki? All I have are files without known extension. Actually I'm new with this so I don't know yet what program (except hex editor for translations) I need. Does anyone can help me, please?

I know this is an old post.. but did you manage to find a hkki tutorial? I have been searching forever.. i have a stcm2l .DAT script file that need to be converted to text..

 

[pingawashere]

Um, then is there anything I should do about it? Is deleting that possible though?
But even with hkki I can't change the characters' name, or if I can make it work/editable by hkki I can just edit the names manually.
Anyhow, thanks to jjjewel I managed to edit it manually by replacing the Japanese text with the English one.
The problem is there's no wordwrap or auto line break, the font in-game is really big and wide also ;__;

Edited:

In-game look:

is hkki the hex editor? if so how do i use it?