ROM Hack RELEASE Smash Ultimate (Partial) Data.arc Extractor/Inject Tool

[FIX94]

So I think I have an idea of what happened, this may be a pretty big encoding bug on their part, I think they actually intended all files to be around 100kbps, because the blocks I recovered at the end of that file actually do appear to be at that bitrate, but for some reason, who knows, due to some encoder bug or game decoding issue, that data was then overwritten by a lower quality stream of only 67kbps, and this bit at the end with the last bit of the song may just be a leftover of what it was originally going to be quality wise, which is pretty sad if my theory here is correct... When comparing the bit at the end and a theoretical "overall" bitrate of the file, if it was all used that is, they would work out to be about the same.
edit: oh also its 64kbps to be perfectly exact here, because each block starts with 8 "header" bytes, subtracting those works out to exactly 64kbps cbr. The actually intended size was actually 96kbps cbr but it was seemingly overwritten at some point...

 

[Cozimo]

Sadly you cant just get more data out of something that was given to us like this, you cant make missing data suddenly appear out of thin air, that'd be pretty great eh After cutting the file to its true internal length it does very much appear to only be 67kbps, the math now sadly checks out.

Ah, I thought the source may have been higher due to the questions of potentially higher bitrate and the low opus bitrate being just a container issue.

A real shame, but it's not like it's terrible quality anyway. Well, thanks for your help!

--------------------- MERGED ---------------------------

So I think I have an idea of what happened, this may be a pretty big encoding bug on their part, I think they actually intended all files to be around 100kbps, because the blocks I recovered at the end of that file actually do appear to be at that bitrate, but for some reason, who knows, due to some encoder bug or game decoding issue, that data was then overwritten by a lower quality stream of only 67kbps, and this bit at the end with the last bit of the song may just be a leftover of what it was originally going to be quality wise, which is pretty sad if my theory here is correct... When comparing the bit at the end and a theoretical "overall" bitrate of the file, if it was all used that is, they would work out to be about the same.
edit: oh also its 64kbps to be perfectly exact here, because each block starts with 8 "header" bytes, subtracting those works out to exactly 64kbps cbr. The actually intended size was actually 96kbps cbr but it was seemingly overwritten at some point...

Wow, so there's literally nothing more that can be done, the files at the source are 2hat they are. Unless by some miracle there's a patch that sort of 'opens up' the game data a bit more or fixes said bug.

 

[Small Kat]

Someone did a SSBU mod by injecting files into the data.arc file.

Here's the first (afaik) texture edit in smash ultimate, texture edit by @KillzXGaming, injected into the game by me pic.twitter.com/58t0o59dhb

— jam1garner (@jam1garner) November 29, 2018

 

[FIX94]

Just did some more work on the main file structures, finally understood how linking works inside the arc, lots of files seem to basically have shortcuts to other files, so the same file can be in different folders, seems to be used quite a bunch. Also now it can also extract files that are in the last 1.3gb of the file which before was not possible because those files actually were referenced in a different way for some reason. So now I think it should be possible to actually extract everything if you know the filenames, still gotta find some way to get those names I guess

 

[Mat37]

This ? https://raw.githubusercontent.com/Thealexbarney/SmashNxTools/master/SmashNxTools/hashstrings.txt

 

[FIX94]

This ? https://raw.githubusercontent.com/Thealexbarney/SmashNxTools/master/SmashNxTools/hashstrings.txt

well something like that yes, though that isnt close to everything, seems like an attempt at brute forcing strings. A clean list would be pretty good...

 

[hamdan91]

what does the build.bat do?

 

[Notex]

what does the build.bat do?

Compiles the application.

 

[hamdan91]

is there any way we can inject the data.arc

--------------------- MERGED ---------------------------

well something like that yes, though that isnt close to everything, seems like an attempt at brute forcing strings. A clean list would be pretty good...

can we inject the data.arc file?

 

[FIX94]

is there any way we can inject the data.arc
?

if you know the name of the file and you run the tool, it'll display the absolute offset in the data.arc so thats I guess somewhat helpful if the filesize of the file you want to replace is the same or smaller.

Also I've been cracking away at one particular filename the last 2 days using my own little crc32 forcer tool cause I wanted to figure out where some things are stored and it finally finished, so now I know that for example the intro movie and cutscenes are located in:
ui/param/database/ui_movie_db.prc
and those are in full paths already, with stream:/ prefix and everything,
the music tracks are located in:
ui/param/database/ui_bgm_db.prc
those are just the base names, you still have to build them like:
stream:/sound/bgm/bgm_<name_from_that_list>.nus3audio
also the other movie files that show off how moves work and stuff are located in:
ui/param/database/ui_technic_db.prc
which again, you have to build them like:
stream:/movie/technique/<name_from_that_list>.webm

Also theres various other databases like ui_chara_db.prc, ui_clear_getter_db.prc, ui_spirit_db.prc, ui_howtoplay_db.prc, ui_item_db.prc, ui_popup_db.prc, ui_rule_db.prc and ui_stage_db.prc in the ui/param/database folder, and those arent all of them yet either, thats just how much I cracked into once I understood the base pattern.

With my tool programmed to just extract everything, I get a folder of over 135 thousand files so far, so there is still a LOT to understand and document

 

[hamdan91]

Yeah it's a pretty neat tool (Good job FIX94) I've been diving in the hashstring.txt file made by smashNXtools and I've been at least 2 hours or so looking for links texture files. As i was searching and hunting for it i got ideas from other hashes for eg. Link's bow texture is.
fighter/Link/model/...(cant Remember lol)
So i guessed it to be this
fighter/link/model/body/c00/def_link_001_col.nutexb.
And in a heartbeat it worked, for other links color alts is the the same thing but replace c00 with c01 or c02 and so on. Those hashes wasn't present in the hashstrings.txt

--------------------- MERGED ---------------------------

I'm just waiting for killzXgaming to fix an issue in his switchToolbox for nutexb format. I can export the nutexb file to multiple image types (Obviously the DDS file is the main format packed inside the nutexb) but i can't repack or replace the image back into the nutexb because the program gives me an Unhandled exception error. Until killzXgaming fixes the issue i can't do any texture editing.

--------------------- MERGED ---------------------------

if you know the name of the file and you run the tool, it'll display the absolute offset in the data.arc so thats I guess somewhat helpful if the filesize of the file you want to replace is the same or smaller.

yeah i can see the offset where the file is located, do i replace that offset with modded file in a hex editor?

--------------------- MERGED ---------------------------

if you know the name of the file and you run the tool, it'll display the absolute offset in the data.arc so thats I guess somewhat helpful if the filesize of the file you want to replace is the same or smaller.

yeah i can see the offset where the file is located, do i replace that offset with modded file in a hex editor?

 

[hamdan91]

FIX94 how do i replace the extracted files? I can see the offset the file is located but i don't know how to inject it back to the data.arc.

 

[minibar]

I didn't realize anyone was looking at the hashstrings.txt file. I've committed an updated version to GitHub

 

[hamdan91]

I didn't realize anyone was looking at the hashstrings.txt file. I've committed an updated version to GitHub

It really helps to find out the filenames but not everything is there on the txt file. I've been looking for links texture file and it wasn't available in the txt file, i just managed to find it by a lucky guessing. But now i just need to know how to inject the file back in the data.arc. I can see the offset where the file is packed but when i open the extracted file in the hex editor it's completely different HEX strings and obviously i can't just copy the entire HEX and paste it on the files offset in the data.arc. If you know FIX94 how to inject the file into the data.arc please tell me how.

 

[FIX94]

FIX94 how do i replace the extracted files? I can see the offset the file is located but i don't know how to inject it back to the data.arc.

I'll look into adding a replace function tomorrow I think.

 

[hamdan91]

I'll look into adding a replace function tomorrow I think.

Thanks FIX94

--------------------- MERGED ---------------------------

I'll look into adding a replace function tomorrow I think.

you're doing an amazing job keep up the good work.

 

[Deobulakenyo]

Xenoblade 2 had a similar system with an ARD and ARH file, but we ended up coming up with a way to inject modded files back into them. Also when DLC came out we found the DLC titles had the files bare inside of the NCA instead of in an archive and we can just use LayeredFS on the DLC title. Who knows if it will be the same

Can you share these XBC2 mods with us? Thanks

 

[minibar]

It really helps to find out the filenames but not everything is there on the txt file. I've been looking for links texture file and it wasn't available in the txt file, i just managed to find it by a lucky guessing. But now i just need to know how to inject the file back in the data.arc. I can see the offset where the file is packed but when i open the extracted file in the hex editor it's completely different HEX strings and obviously i can't just copy the entire HEX and paste it on the files offset in the data.arc. If you know FIX94 how to inject the file into the data.arc please tell me how.

94% of the file paths are in the txt file. The file you're looking for isn't one of these? https://pastebin.com/raw/cZQ7CRmz

Can you share these XBC2 mods with us? Thanks

There's not much to share. I quickly hacked a function together to replace existing files in the archive. https://github.com/Thealexbarney/XbTool/blob/master/XbTool/XbTool/Xb2/FileArchive.cs#L202

 

[hamdan91]

94% of the file paths are in the txt file. The file you're looking for isn't one of these? https://pastebin.com/raw/cZQ7CRmz

https://pastebin.com/raw/cZQ7CRmz
Wow! most of these files are not in the txt file

 

[minibar]

Wow! most of these files are not in the txt file

I think you didn't grab the new list when I said I updated it.