Hacking Phantasy Star Portable 2 Infinity - English Translation

Kion · Sep 17, 2019

Looks like most of the models use a normal strip format.

Do you have the texture for the green drop, so I can try applying it? And I'm kind of curious where the textures are stored. It looks like .unt is the texture list, and then .uvr is the textures. But I'm not sure if I noticed essen's tool export any .uvr files, so I might have to take another look for that.

JamRules · Sep 17, 2019

henok12 said:
is it still far away from release?

There's no release date yet.

Kion said:
Looks like most of the models use a normal strip format.

View attachment 179650

Do you have the texture for the green drop, so I can try applying it? And I'm kind of curious where the textures are stored. It looks like .unt is the texture list, and then .uvr is the textures. But I'm not sure if I noticed essen's tool export any .uvr files, so I might have to take another look for that.

The textures are in the .nbl too, and yeah they're the .uvr files. Not looked into the .unt files before but that seems a good call.
Looks like the texture for that is xx_psp2_dpitem01.uvr.
I've added the others too just in case.

Kion · Sep 18, 2019

JamRules said:
There's no release date yet.

Out of curiosity, what's the planned method of release? I'm guessing some kind of patch program, or pre-patched .iso?

JamRules said:
The textures are in the .nbl too, and yeah they're the .uvr files. Not looked into the .unt files before but that seems a good call.
Looks like the texture for that is xx_psp2_dpitem01.uvr.
I've added the others too just in case.

The .unt files are pretty simply once you pop them open, there's not a lot there:

Code:

000000000: 4e55544c 48000000 24000000 00000000  NUTLH...$.......
000000010: 00000000 8c4c0000 04000100 00000000  .....L..........
000000020: 00000000 01000000 704c0000 736e5f6c  ........pL..sn_l
000000030: 78785f70 7370325f 64706974 656d3031  xx_psp2_dpitem01
000000040: 2e757672 00000000 00000000 00000000  .uvr............

For some reason I can't find any .uvr files from using essen's nbl tool. Could they be in the TMLL files? Can you include xx_psp2_dpitem01.uvr as an attachment?

And thanks for the textures! Was able to get a drop working with vertex colors and texture. Need to add another material for the ring effect on the outside.

Edit:

I went through and made a scan to look for Magic numbers in the file.fpb file to make sure I was grabbing everything, and these were the Magic number candidates and their frequencies.

Code:

{ NMLL: 11413,
  STD: 50404,
  NUIF: 104567,
  UNT: 33982,
  TMLL: 2066,
  NNVR: 37225,
  GBIX: 37290,
  PVRT: 18422,
  UNR: 6592,
  RIFF: 9266,
  REL: 24496,
  UNJ: 23307,
  UNA: 9649,
  UNM: 6233,
  UNV: 15743,
  UVRT: 18911,
  NXIF: 31951,
  XNR: 16644 }

I'm generally really surprised, I thought it was just archives stacked inside the file. Really surprised to see GBIX, PVRT, REL, and UNV occurring in the file.fpb file directly. I think their psudo-blowfish function in the archives mixes the bytes up enough that these don't seem to be false positives. I might also check to see where these occur. If there is a length declared after each number then it would be easier to look for specific file types, grab the length, and only pull out the files that I need as opposed to trying to pull out everything end-to-end the way the tool currently works.

JamRules · Sep 18, 2019

Kion said:
Out of curiosity, what's the planned method of release? I'm guessing some kind of patch program, or pre-patched .iso?

The .unt files are pretty simply once you pop them open, there's not a lot there:

Code:

000000000: 4e55544c 48000000 24000000 00000000 NUTLH...$....... 000000010: 00000000 8c4c0000 04000100 00000000 .....L.......... 000000020: 00000000 01000000 704c0000 736e5f6c ........pL..sn_l 000000030: 78785f70 7370325f 64706974 656d3031 xx_psp2_dpitem01 000000040: 2e757672 00000000 00000000 00000000 .uvr............

For some reason I can't find any .uvr files from using essen's nbl tool. Could they be in the TMLL files? Can you include xx_psp2_dpitem01.uvr as an attachment?

View attachment 179738

And thanks for the textures! Was able to get a drop working with vertex colors and texture. Need to add another material for the ring effect on the outside.

Edit:

I went through and made a scan to look for Magic numbers in the file.fpb file to make sure I was grabbing everything, and these were the Magic number candidates and their frequencies.

Code:

{ NMLL: 11413, STD: 50404, NUIF: 104567, UNT: 33982, TMLL: 2066, NNVR: 37225, GBIX: 37290, PVRT: 18422, UNR: 6592, RIFF: 9266, REL: 24496, UNJ: 23307, UNA: 9649, UNM: 6233, UNV: 15743, UVRT: 18911, NXIF: 31951, XNR: 16644 }

I'm generally really surprised, I thought it was just archives stacked inside the file. Really surprised to see GBIX, PVRT, REL, and UNV occurring in the file.fpb file directly. I think their psudo-blowfish function in the archives mixes the bytes up enough that these don't seem to be false positives. I might also check to see where these occur. If there is a length declared after each number then it would be easier to look for specific file types, grab the length, and only pull out the files that I need as opposed to trying to pull out everything end-to-end the way the tool currently works.

This is breifly mentioned in the first post but the current plan would be to distribute an .xdelta file that can be used with the original iso. We wouldn't distribute a prepatched iso.

Yeah, the unts don't seem to bad, just not had any reason to use them myself.

They are TMLL files I think, as generally the texture files are the only thing found in the TMLL section.

.uvr isn't allowed as an attachment apparently so I zipped it.

I would be careful about splitting at anything other than NMLL and RIFF as the other magic numbers are used in the .nbl header table or the content files. As far as I know NMLL, RIFF, BIND and PACK that contains them are the only ones that are referenced directly by the .fpb pointer table. Somewhat pointless information but the proper name for 8894.nbl is ga_CommonModelSu.nbl. I didn't see any other filenames that reference model.

Kion · Sep 19, 2019

JamRules said:
This is breifly mentioned in the first post but the current plan would be to distribute an .xdelta file that can be used with the original iso. We wouldn't distribute a prepatched iso.

I should learn to read.

JamRules said:
They are TMLL files I think, as generally the texture files are the only thing found in the TMLL section.

So TMLL isn't it's own file, it's a section inside of the NMLL file? I guess that means I need to add a few more debug comments into essen's nbl extractor if it's only seeing the NMLL part of the file and not finding the TMLL section.

JamRules said:
.uvr isn't allowed as an attachment apparently so I zipped it.

Thanks!

JamRules said:
I would be careful about splitting at anything other than NMLL and RIFF as the other magic numbers are used in the .nbl header table or the content files. As far as I know NMLL, RIFF, BIND and PACK that contains them are the only ones that are referenced directly by the .fpb pointer table. Somewhat pointless information but the proper name for 8894.nbl is ga_CommonModelSu.nbl. I didn't see any other filenames that reference model.

I forgot to look for a pointer table. In general I'm trying to think of the easiest way to grab the NMLL files out of the .fpb file to get quick access to the model and textures for the assets. I guess I could look for the NMLL offsets in the file, and then check the files to see where those offsets match. And then that way I could quickly pull out the NMLL files by going to the offset, checking the magic header, and then grabbing that offset until the next offset.

JamRules · Sep 19, 2019

Kion said:
I should learn to read.

So TMLL isn't it's own file, it's a section inside of the NMLL file? I guess that means I need to add a few more debug comments into essen's nbl extractor if it's only seeing the NMLL part of the file and not finding the TMLL section.

Thanks!

I forgot to look for a pointer table. In general I'm trying to think of the easiest way to grab the NMLL files out of the .fpb file to get quick access to the model and textures for the assets. I guess I could look for the NMLL offsets in the file, and then check the files to see where those offsets match. And then that way I could quickly pull out the NMLL files by going to the offset, checking the magic header, and then grabbing that offset until the next offset.

Yeah, the TMLL section should be part of the original NMLL/.nbl file. The header for the NMLL has values for the size of the TMLL section. Edit: I think this is detailed in essen’s notes on PSU mods.

Well, the pointer table is encrypted by default so you wouldn’t have found it.
Hence brute force scanning for magic number is easier.

Kion · Sep 20, 2019

JamRules said:
Yeah, the TMLL section should be part of the original NMLL/.nbl file. The header for the NMLL has values for the size of the TMLL section. Edit: I think this is detailed in essen’s notes on PSU mods.

I found the reason that the texture files weren't being exported from the nbl archives:

Looks like essen didn't finish the TMLL implementation and exits the program automatically when the TMLL file is compressed. I guess that means I'll have go dig around on PSU mods for clues.

JamRules said:
Well, the pointer table is encrypted by default so you wouldn’t have found it.
Hence brute force scanning for magic number is easier.

Scanning for magic numbers is pretty quick and easy approach. It's mostly a matter of being sure about having the right magic numbers to check for, since the nbl file header seems to expose the file type of the embedded files in the header. I checked and NMLL, BIND and RIFF seem to be a reliable enough list to pull out the nbl archives to work with. And after looking a little closer at the NMLL header, it looks like it declares the full length, the compressed length, and full length and compressed length of the TMLL (if exists). So I'm tempted to try the approach of look for NMLL, read the header, grab the byte length declared in the header, and then scan for the next one.

Edit:

After looking at the header for the NMLL file, it looks like it declares the length of the NMLL header, the NMLL payload, the pointer table, the TMLL header (if exists) and TMLL payload, but it doesn't seem to include any information about the padding in between these lengths. Being able to access the fpb's pointer table would probably be the quickest and most accurate approach of pulling out the nbl archives. Though barring the quickest way does indeed seem to be to check the magic numbers and pull from there.

Shadowth117 · Sep 24, 2019

I want to say first of all that this project is awesome and I'd really love to see it finished one day.

As far as this page in the thread, I was really surprised to see models being worked on. Very cool! It'd be interesting to work with those eventually.

HomeShineBrew · Oct 1, 2019

If anyone is still interested in what an upscale would look like in game I managed to get the original PSP to dump most visible textures in game and am currently upscaling them. It's done with a waifu2x pass then a Gigapixel AI pass. Should have some comparison pics up soon. It'll give an idea as to whether or not it's worth upscaling psp2i later. Not planning to release these textures either, don't want to get in trouble for copyright.

Sent from my PH-1 using Tapatalk

hagtorp · Oct 1, 2019

Interestingly, this year we will see the release?

HomeShineBrew · Oct 1, 2019

All right here's a couple comparison pics. Tried doing 20 comparisons worth but imgsli wont allow it. As you cant tell the UI doesn't stay upscaled, it seems to change texture addresses all the time in game so it's almost impossible to upscale all of it unless you can change it through the game file itself. IMO the upscaling doesn't do much, it would be better for those interested to manually replace textures in game with new stuff. Waifu2x was used first then Gigapixel AI. https://imgsli.com/NzIyMA/ EDIT: also here's Vivienne upscaled to show affect on character art https://imgsli.com/NzIyMQ

JamRules · Oct 2, 2019

HomeShineBrew said:
All right here's a couple comparison pics. Tried doing 20 comparisons worth but imgsli wont allow it. As you cant tell the UI doesn't stay upscaled, it seems to change texture addresses all the time in game so it's almost impossible to upscale all of it unless you can change it through the game file itself. IMO the upscaling doesn't do much, it would be better for those interested to manually replace textures in game with new stuff. Waifu2x was used first then Gigapixel AI. https://imgsli.com/NzIyMA/ EDIT: also here's Vivienne upscaled to show affect on character art https://imgsli.com/NzIyMQ

Thanks for the comparisons. Yeah, I wasn't expecting too much for just a plain upscale. Though there may be some areas that turn out better than others.
There is some hope that using PSU textures can help in some areas as these textures are double the size to begin with and so may have additional detail to them.
Ideally proper replacement textures would be made but this would be a huge undertaking.

In regards to the menu, you could maybe change it so it only use the pixel + clut hash and ignore the address. There are problems with this for 512x512 textures though, even with the half/reduced hash mode on I think.

hagtorp said:
Interestingly, this year we will see the release?

No release date yet

Kion · Oct 3, 2019

How do the upscaled textures work? I'm guessing that with PPSSPP you can increase the amount of vram and replace the textures?

HomeShineBrew · Oct 3, 2019

Kion said:
How do the upscaled textures work? I'm guessing that with PPSSPP you can increase the amount of vram and replace the textures?

Yeah. Using a 32-bit Opengl version of PPSSPP you can dump game textures under developer tools. Then you can run them through a program to upscale and put them in the games texture folder and PPSSPP will replace each texture according to the name (address) on the fly. It uses the VRAM of the computers GPU, I'm not sure that PPSSPP emulates the VRAM limit of the PSP cause you don't have to adjust anything in the settings to use more VRAM for the textures, it just works.

Sent from my PH-1 using Tapatalk

Kion · Oct 7, 2019

Not exactly on topic, but an update for the model format.

Screenshot_2019-10-07 Psp2 Infinity(2).png

Screenshot_2019-10-07 Psp2 Infinity(3).png

Screenshot_2019-10-07 Psp2 Infinity(4).png

Screenshot_2019-10-07 Psp2 Infinity(5).png

I've managed to parse the textures and most of simple common models. Now I guess i need to search around for the next step up like doors or weapons and stuff.

brunocar · Oct 9, 2019

im guessing this is far from release, so im not gonna ask for a date, but i just wanted to know, how far is this? are you guys still havent technical problems or is it progressing smoothly?

Weyu · Oct 12, 2019

Update:
We're making good progress with the patch.
I've finished typesetting and proofreading all weapons in the game (there's like a literal thousand of these).

We still have some smaller tech issues to work through and more internal QC.
Will update again when there's news.

Some screenshots: