ROM Hack Developing PHP-based open source PokéSAV script

[MarkusE]

This should work:
$d = Math.floor(PokemonID Mod 4);
$c = (Math.floor(PokemonID/256)Mod 4) * 4;
$b = (Math.floor(PokemonID/65536)Mod 4) * 16;
$a = (Math.floor(PokemonID/16777216)Mod 4) * 64;
Unown = Math.floor(($a + $b + $c + $d) Mod 28);

Might have to change Mod to %'s

Also, for the Pokemon name hex-editing, you have to open the individual Pokemon .pkm files generated by PokeSAV, not the overall .sav file. Any hex editor should do.

 

[MarkusE]

For OT name, same byte-character mapping as in previous post except stored in 0x68-77 (yes, you can only have up to 8 chars)

So close to getting this whole thing deciphered.

 

[gamefreakfatty]

so, how is the PKM file getting imported into a SAV file?

 

[MarkusE]

We need to figure out how Pokemon Diamond/Pearl stores its checksums before we can proceed with the encryption.

0x04-07 of every Pokemon seems to be some kind of a checksum that involves PokeRus status (infected/cured), markings, ribbons, and date caught, whether it's a fateful encounter, whether it's an egg, and trainer gender. Changing any of these variables involves changes in their respective storage areas, and also changes to 0x04-07.

If you can try to help with this, that'd be very much appreciated...

 

[MarkusE]

136 bytes make up a Pokemon.
Unencrypted hex data in .pkm files generated by PokeSAV are arranged as follows:

AS YET UNRESOLVED - PLEASE HELP ME FIGURE THEM OUT IF YOU CAN
Specifically, 0x00-03, 04-07, 3C-3F, 5E, 82, 87-88

Update as of 11:43 pm PDT: Figured out byte 0x83 = Pokeball Caught

00 ATTRIBUTE
01 ATTRIBUTE
02 ATTRIBUTE
03 ATTRIBUTE

04 CHECKSUM
05 CHECKSUM
06 CHECKSUM
07 CHECKSUM

BLOCK STRUCTURE A BEGINS HERE

08 Pokemon Species ID
09 Pokemon Species ID

0A Held Item
0B Held Item

0C Trainer ID
0D Trainer ID
0E Secret ID
0F Secret ID

10 Experience Points
11 Experience Points
12 Experience Points
13 Experience Points

14 Friendship/Egg Steps to Hatch

15 Ability

16 Marks (01, 02, 04, 08, 10, 20)

17 Country (01 JP, 02 EN, 03 FR, 04 IT, 05 GE, 07 SP)

18 Effort Value - HP
19 Effort Value - Attack
1A Effort Value - Defense
1B Effort Value - Speed
1C Effort Value - Sp Atk
1D Effort Value - Sp Def

1E Cool
1F Beauty
20 Cute
21 Smart
22 Tough
23 Sheen

24 Ribbon
25 Ribbon
26 Ribbon
27 Ribbon

BLOCK STRUCTURE B BEGINS HERE

28 Move 1 ID
29 Move 1 ID
2A Move 2 ID
2B Move 2 ID
2C Move 3 ID
2D Move 3 ID
2E Move 4 ID
2F Move 4 ID

30 Move 1 PP
31 Move 2 PP
32 Move 3 PP
33 Move 4 PP

34 Move 1 PP Up
35 Move 2 PP Up
36 Move 3 PP Up
37 Move 4 PP Up

38 IV String
39 IV String
3A IV String
3B IV String

3C MYSTERY BYTE
3D MYSTERY BYTE
3E MYSTERY BYTE
3F MYSTERY BYTE

40 Fateful Encounter

41 Hidden Value
42 Hidden Value
43 Hidden Value
44 Hidden Value
45 Hidden Value
46 Hidden Value
47 Hidden Value

48 Pokemon Name - 1st Character
49 Pokemon Name
4A Pokemon Name - 2nd Character
4B Pokemon Name
4C Pokemon Name - 3rd Character
4D Pokemon Name
4E Pokemon Name - 4th Character
4F Pokemon Name
50 Pokemon Name - 5th Character
51 Pokemon Name
52 Pokemon Name - 6th Character
53 Pokemon Name
54 Pokemon Name - 7th Character
55 Pokemon Name
56 Pokemon Name - 8th Character
57 Pokemon Name
58 Pokemon Name - 9th Character
59 Pokemon Name
5A Pokemon Name - 10th Character
5B Pokemon Name
5C Pokemon Name - 11th Character
5D Pokemon Name

5E MYSTERY BYTE

5F Hometown

60 Contests Won
61 Contests Won
62 Contests Won

63 Hidden Value
64 Hidden Value
65 Hidden Value
66 Hidden Value
67 Hidden Value

68 OT Name - 1st Character
69 OT Name
6A OT Name - 2nd Character
6B OT Name
6C OT Name - 3rd Character
6D OT Name
6E OT Name - 4th Character
6F OT Name
70 OT Name - 5th Character
71 OT Name
72 OT Name - 6th Character
73 OT Name
74 OT Name - 7th Character
75 OT Name
76 OT Name - 8th Character
77 OT Name

78 Date Egg Received - Year (+2000)
79 Date Egg Received - Month
7A Date Egg Received - Day

7B Date Met - Year (+2000)
7C Date Met - Month
7D Date Met - Day

7E Egg Location
7F Egg Location

80 Met At Location
81 Met At Location

82 MYSTERY BYTE

83 PokéBall

84 Met At Level

85 Hidden Value
86 Hidden Value

87 MYSTERY BYTE
88 MYSTERY BYTE

 

[MarkusE]

Okay, upon playing with PokeSAV and No$ a bit more I don't think the mystery bytes matter a whole lot, except the checksum.

Next steps:
1. Figure out how the checksum is generated/stored
2. Figure out how the encryption is done.
and then I can start coding

As far as I know, bytes 0x08-87 are XOR encrypted when converting from .pkm to .sav We need to figure out what's XOR'ed with what and such.

Update:

Red = Checksum
Yellow Highlighted = Encrypted bits (0x28-2F has been relocated to 0x08-0F in the encrypted data)

 

[gamefreakfatty]

im sure you've probably found this. i saw it when trying to figure out the unkown shape. the bold part is what i think you're probably interested in.

-----------
17) Byte4-Byte7
-----------
e1dd6e3b Byte4-Byte7 ReverseOTrainerIDs

To get OTrainerIDs Reverse Byte order
e1dd6e3b ReverseOTrainerIDs
wwxxyyzz

3b6edde1 OTrainerIDs 997,121,505-Dec
zzyyxxww Byte7-Byte4

Used for:
OTrainerSecretID Byte7-Byte6
OTrainerID Byte5-Byte4
part of the Shiny Pokemon.
and part of the ABCDStructure encryptionKey.

zzyy High Order Word is OTrainerSecretID Byte7-Byte6
OTrainerSecretID Byte7-Byte6
OTrainerSecretID = Integer(OTrainerIDs/65536-Dec)
OTrainerSecretID = Integer(OTrainerIDs/10000-Hex)
3b6e = 15214-Dec

xxww Low Order Word is OTrainerID Byte5-Byte4
OTrainerID Byte5-Byte4
OTrainerID = OTrainerIDs Mod 65536-Dec
OTrainerID = OTrainerIDs Mod 10000-Hex
dde1 = 56801-Dec


---------------------------
18) ABCDStructure encryptionKey
---------------------------
encryptionKey I call XKey
is found by XOR'ing PokemonID with OTrainerIDs

ff47e89d PokemonID Byte3-Byte0
3b6edde1 OTrainerIDs Byte7-Byte4

ff47e89d Xor 3b6edde1 = C429357C = XKey

-------------
19) Shiny Pokemon
-------------
Found by XOR'ing (High Order of XKey) with (Low Order of XKey)
If XResult < 8 then Pokemon is Shiny

XKeyHigh = Integer(XKey/65536-Dec)
XKeyHigh = Integer(XKey/10000-Hex)

XKeyLow = XKey Mod 65536-Dec
XKeyLow = XKey Mod 10000-Hex

C429357C = XKey
wwxxyyzz

XKeyHigh = wwxx = C429
XKeyLow = yyzz = 357C

XKeyHigh Xor XKeyLow = XResult
C429 Xor 357C = F155 = XResult
F155 is bigger than 8, so Pokemon is NOT Shiny

-----------------------------------
How ABCD Structure Order is stored.
-----------------------------------
Thanks xyzman
ABCDOrder = Math.floor(PokemonID Mod 24-Dec)
4,282,902,685-Dec Mod 24-Dec = 13-Dec

ABCDOrder = Math.floor(PokemonID Mod 18-Hex)
ff47e89d Mod 18-Hex = D-Hex

00 = "ABCD"
01 = "ABDC"
02 = "ACBD"
03 = "ACDB"
04 = "ADBC"
05 = "ADCB"
06 = "BACD"
07 = "BADC"
08 = "BCAD"
09 = "BCDA"
0A = "BDAC"
0B = "BDCA"
0C = "CABD"
0D = "CADB"
0E = "CBAD"
0F = "CBDA"
10 = "CDAB"
11 = "CDBA"
12 = "DABC"
13 = "DACB"
14 = "DBAC"
15 = "DBCA"
16 = "DCAB"
17 = "DCBA"

0D = "CADB" is the order for ABCD in the ABCDStructure
ABCDOrder = "CADB"

im not sure if this helps or not?

What i saw that caught my eye with this is that there is an ABCDStructure encryption key which is what i think you're looking for.

 

[MarkusE]

I tried and it doesn't work.

It seems like D/P uses a slightly different encryption algorithm than Emerald

 

[gamefreakfatty]

maybe if we could contact COM (PokeSAV writer) then he could shed some light on the situation? use google translate maybe?

email for COM:
[email protected]

I have sent an email with a link to a Japanese translation. Just find a free host and put the email there as a txt or php or html file, go to Google Translate, put in the URL, then send him an email with the english letter and a translator URL at the bottom.

 

[MarkusE]

Thanks. let me know if he responds.

Maybe we can try to ask him for the source code?

 

[gamefreakfatty]

well, i dont know if he'll give source code, but we could ask him. i'll try asking him later, but i first wanna kinda try to become friends or something, so its not sounding like some random n00b asking for the source. i wonder what he uses for it though? probably VB6 cause i dont think you need the .NET runtimes for it. either that or C6/C++6. anyways, i'll ask him for source, but probably not yet.

i already asked him about the encryption of the save files, hopefully we can get at least that. source code is probably if we're lucky.

i just wonder if there is a way to translate my emails into Japanese upon sending them. if i can find one, i'll use that, because i dont know if he will understand English or if he'll understand to go to the link of the translation i provided in the email. another thing i really hope is that it wont go into Junk/Spam Email!!!

 

[MarkusE]

I'd suggest putting the translated Japanese text inline (within the email message).

 

[gamefreakfatty]

i dont know how to make the translated text show up as something besides ?'s. if anyone knows how, could they tell me.... PLEEAASE?

Edit: i think i might have figured it out

 

[MarkusE]

You need shift+JIS encoding. If you don't have it, it'll show as ???'s

Also, machine-translated text is often of questionable quality, so he may not be able to understand it...bah, it's worth a try anyway.

 

[gamefreakfatty]

thats the only way i can talk to him sadly... oh well, dont have much of a choice.

EDIT:
ok, i think i successfully sent him an email with Japanese text. i did use the Google Translator since i can NOT speak/write Japanese. No Japanese friends either, oh well. lets hope it works.
do you think Babelfish would have been better?

EDIT(again):
Warning: In order to see the Japanese text below, you have to have the font for Japanese text installed, which should come with your OS.
The letter to COM:

Using Google Translate to translate to Japanese:

Subject:
PokeSAVの暗号化の助け

Message Text:
残念、私達は日本語を話さない。
Googleはテキストを次に翻訳した。

com、私達はPokeSAVのタイプ適用を開発する助けを必要とする。 私達は保管ファイルの暗号化を用いる悩みを有している。 私達を助けて頂けますか。私達は暗号化を見つける方法を知ることを望む保管ファイルを正しく編集する方法を従って私達は調べてもいい。 ありがとう。

- gamefreakfatty

Using Google Translate to translate back to English:

Subject:
Help of the encoding PokeSAV

Message Text:
Regrettable, we do not speak Japanese.
Google translated the text next.

com, we need the help which develops the type application of PokeSAV. We have possessed the trouble which uses encoding the storage file. You can help us? As for us therefore we may inspect the method of compiling the storage file which desires the fact that the method of finding encoding is known just. Thank you.

- gamefreakfatty

I just hope that the email translates to Japanese better than it translated back to English!!

EDIT(again,again):
Ok, Babelfish translates back to English just the same, EXACTLY the same from what I can see.

Babelfish:

Regrettable, we do not speak Japanese.
Google translated the text next.

com, we need the help which develops the type application of PokeSAV. We have possessed the trouble which uses encoding the storage file. You can help us? As for us therefore we may inspect the method of compiling the storage file which desires the fact that the method of finding encoding is known just. Thank you.

- gamefreakfatty

Google Translate:

Regrettable, we do not speak Japanese.
Google translated the text next.

com, we need the help which develops the type application of PokeSAV. We have possessed the trouble which uses encoding the storage file. You can help us? As for us therefore we may inspect the method of compiling the storage file which desires the fact that the method of finding encoding is known just. Thank you.

- gamefreakfatty

Babelfish seems to be no different for translating back.

Lets see Babelfish translate to Japanese from English:

残念、私達は日本語を話さない。
Google はテキストを次に翻訳した。

COM 、私達はPokeSAV のタイプ適用を開発する助けを必要とする。私達は保管ファイルの暗号化を用いる悩みを有している私達を助けることができるか。暗号化を見つける方法を私達は知ることを望む従って保管ファイルを編集する方法を私達は正しく感謝する見つけることができる。

- gamefreakfatty

Compared to Google Translate's translation to Japanese to English.

残念、私達は日本語を話さない。
Googleはテキストを次に翻訳した。

com、私達はPokeSAVのタイプ適用を開発する助けを必要とする。 私達は保管ファイルの暗号化を用いる悩みを有している。私達を助けて頂けますか。私達は暗号化を見つける方法を知ることを望む保管ファイルを正しく編集する方法を従って私達は調べてもいい。 ありがとう。

- gamefreakfatty

I think there are only differences in the spacing of the Japanese text. Babelfish has less spaces.

-gamefreakfatty

 

[MarkusE]

Probably should mention that we're specifically asking about the .pkm-->.sav encryption for bytes $08-$87.

In broken Japanese:
$08-$87 の 暗号化 (.pkm-->.sav) アルゴリズムを教えて下さい

暗号化 = encryption, right? This is what Google Translator gave me and what I assume it was in your post.
アルゴリズム = algorithm

Thanks for your efforts.

 

[gamefreakfatty]

yes, i did have the word encryption in the email. i dont want to bombard him with emails, so the 4th will be the last until i get a response. is that ok? sorry if its rude, but if we bombard him with emails, he might get "p-o"ed and not want to help.

EDIT:
I sent him another email, but... SHOOOT!!!!! I forgot to ask for source. I'll send my 4th one asking for source code. then im done with emails for now.
i forgot to add this part too:

$08-$87 ? ??? (.pkm-->.sav) ?????????????

Here is what i sent, almost the same, but it should give enough info:
In English:

Subject:
PokeSAV Encryption Help Needed

Message:
Sorry, we do not speak Japanese.
My previous emails were incomplete.
Google translated text below.

COM, we need help developing a PokeSAV type application. we are having trouble with the encryption of the save files. can you help us? we would like to know how to find the encryption so we can find out how to edit the save files correctly. our main problem is that we are having trouble importing the pkm files into the save file. i am very sorry to send so many emails to you. thank you in advance.

-gamefreakfatty
In "Google Translated" Japanese:

Subject:
PokeSAV?????????????

Message:
???????????????
?????????????????
Google?????????????

com????PokeSAV???????????????????? ?????????????????????????? ?????????????????????????????????????????????????????????????????????????????????????pkm???????????????????????????????????????????????????? ??????????

- gamefreakfatty

i really hope that he replies! if he doesnt, we're doomed to painful agonizing work to "unencrypt" it if we still want to do the PHP thing.

EDIT: Another edit within 2 minutes, pathetic, eh?
Anyways. im sending an email with a source code request.

English:
QUOTESubject:
PokeSAV Source Code for Help

Message:
Sorry for bad Japanese, have to use Google Translate.

I am very sorry for so many emails! We are wondering if you can please let us look at your source code for PokeSAV. It would help out in our process to make the PHP script. If you can, please upload it somewhere and send me a link. It would be greatly appreciated. Thank you very much!
Here is a link to the progress of the PHP script:
http://gbatemp.net/index.php?showtopic=59118

-gamefreakfatty

Japanese:
QUOTE

Subject:
??????PokeSAV???????

Message:
????????????????Google?????????????

????????????????????????! ???PokeSAV???????????????????????????????????????????????????????????PHP???????????? ??????????????????????????????? ???????????????????!
????PHP????????????:
http://gbatemp.net/index.php?showtopic=59118

-gamefreakfatty

 

[gamefreakfatty]

bump!
srry, if the post gets too low, i wont be able to find it!

 

[MarkusE]

So no response, eh?

 

[gamefreakfatty]

ya, no response STILL! i don't think we're gonna get one. :/