Hacking wwt+wit: Wiimms WBFS+ISO Tools

[Ebola16]

Ok, so I now tried

wit EXTRACT ssbb.iso ssbb --raw

then

wit MIX ssbb/UPDATE ssbb/DATA ssbb/P-HA8E ssbb/P-HA9E ssbb/P-HBAE ssbb/P-HBBE ssbb/P-HBCE ssbb/P-HBDE ssbb/P-HBEE ssbb/P-HBFE ssbb/P-HBGE ssbb/P-HBIE ssbb/P-HBKE ssbb/P-HBLE --dest new.iso --id RSBE01 --name SSBB --region USA

The ISO compiled without any errors but Dolphin refused to boot it and the MD5 didn't match CleanRip. I'm guessing that there isn't a way to make a 1:1 Brawl ISO from an extracted filesystem in WIT? Or am I missing something?

 

[Wiimm]

There are different reasons for the different checksums:
* Placement of partitions
* Trash data between partition differ
* Order and placement of files
* Trash data between files
* Fake signing of composed partitions

If you want to compare, use: wit diff IMAGE1 IMAGE2
or for file level comparison: wit fdiff IMAGE1 IMAGE2

 

[sideskroll]

Hi @Wiimm , I'm looking to switch from using wiibackupmanager to your tool mainly because of the issues with some games and their partitions.
The problem is that I'm having a hard time grasping the concept and usage of your tool. So I tried wiibackup fusion as a front-end but it doesn't give me the options I'm looking for.
So I thought I'd ask for a little help (OK, a LOT of help)
What I'm trying to accomplish: a setup that works exactly like WBM. meaning if I have an ISO on my PC and want to transfer it to my fat32 hdd, the tool takes care of the name of the folder (usb:/wbfs/Title [GameID]) of the file(s) name (Title.wbfs, Title.wbfs1 etc) split size (4gb) and removes ONLY the update partition. All in "one click". A standard setup for all games if you will.
Can it be done and if so, how?
And don't be afraid to explain it to me like you would a 3 year old child
It makes no difference to me to use either command line or a GUI (although a GUI would be much preferred) as long as it can do what I want it to do.
A *.bat file would be nice.
I'm on windows 10 x64 (if it makes any difference)
Thanks!

 

[Ebola16]

After some more testing with the questionable code, it appears to expect a file on the game disc to be at a certain offset. That explains all of the problems I've been having. Is there a way to define an offset for a file when compiling with WIT?

 

[Wiimm]

First, read the docu of https://wit.wiimm.de/ or at least the introduction and CLI part.
The tools have also a complete build-in help. Try for example: wit help copy

because of the issues with some games and their partitions.

I think, it is only SSBB with the extra partitions. I don't know an other game except SSBB that uses more than 1 partition table.

the tool takes care of the name of the folder (usb:/wbfs/Title [GameID]) of the file(s) name (Title.wbfs, Title.wbfs1 etc) split size (4gb) and removes ONLY the update partition.

I don't know, how WBM and the Gui's for my tools work. I use always my tools at command line, sometime as part of a script.

Instead of answering each question of everyone again, I have written the online docu. This are good points for you to start:
* https://wit.wiimm.de/info/iso-images.html#write
* https://wit.wiimm.de/opt/psel

 

[sideskroll]

First, read the docu of https://wit.wiimm.de/ or at least the introduction and CLI part.
The tools have also a complete build-in help. Try for example: wit help copy

I think, it is only SSBB with the extra partitions. I don't know an other game except SSBB that uses more than 1 partition table.

I don't know, how WBM and the Gui's for my tools work. I use always my tools at command line, sometime as part of a script.

Instead of answering each question of everyone again, I have written the online docu. This are good points for you to start:
* https://wit.wiimm.de/info/iso-images.html#write
* https://wit.wiimm.de/opt/psel

I know. I've read the docs. (That's actually how I got as far as to being able to copy the iso to wbfs) what I can't find the info for is to do the things I mention above. Specifically to have it move the wbfs to a certain place. Have it CREATE the wbfs folder if it's not found. have it split the file at the 4gb mark. Name the wbfs folder and file(s) based on the title.txt names (instead of me having to type it manually) things like that.
Like I said, I'd like to create a bat file for example. A template that I can use for EVERY GAME, EVER. And not having to bother with typing names, creating folders etc.
Like, for instance. I have a folder full of ISOs. I want the tool to:

• Copy them to my fat32 usb drive.
• 
  
• Remove the update partition.
• 
  
• Split them (if needed) into 4gb sized chunks.
• 
  
• Create the appropiate (Title [TitleID]) folder.
• 
  
• Name the wbfs file(s) apropiately.

Without me having to do it one by one.

A standard bat file.

Is not that I'm lazy or a bumbling idiot. It's just that your tool is complex. VERY complex. And even though I understand the basic concept of it and how to "make it work". What I'm trying to accomplish is out of my league (at the moment). But I'm a fast learner.
Could you help me with that?
Thanks.

 

[tswntk]

I know. I've read the docs. (That's actually how I got as far as to being able to copy the iso to wbfs) what I can't find the info for is to do the things I mention above. Specifically to have it move the wbfs to a certain place. Have it CREATE the wbfs folder if it's not found. have it split the file at the 4gb mark. Name the wbfs folder and file(s) based on the title.txt names (instead of me having to type it manually) things like that.
Like I said, I'd like to create a bat file for example. A template that I can use for EVERY GAME, EVER. And not having to bother with typing names, creating folders etc.
Like, for instance. I have a folder full of ISOs. I want the tool to:

• Copy them to my fat32 usb drive.
• 
  
• Remove the update partition.
• 
  
• Split them (if needed) into 4gb sized chunks.
• 
  
• Create the appropiate (Title [TitleID]) folder.
• 
  
• Name the wbfs file(s) apropiately.

Without me having to do it one by one.

A standard bat file.

Is not that I'm lazy or a bumbling idiot. It's just that your tool is complex. VERY complex. And even though I understand the basic concept of it and how to "make it work". What I'm trying to accomplish is out of my league (at the moment). But I'm a fast learner.
Could you help me with that?
Thanks.

I guess you should start writing the script (a bat file, basic/java script or autohotkey scripts etc) and ask for help on specific commandline parameters.
I think some of the functions is not within the scope of the wit tools - like creating folders/files with appropriate names - that you should retrieve the the game ID and look up the proper names from Wiidb or other sources. You might be able to get those info from the iso themselves but they are often ugly names.

Then again, SSBB is the only known game that WiiBackup Manager can't handle, really worth the effort?

 

[sideskroll]

I guess you should start writing the script (a bat file, basic/java script or autohotkey scripts etc) and ask for help on specific commandline parameters.
I think some of the functions is not within the scope of the wit tools - like creating folders/files with appropriate names - that you should retrieve the the game ID and look up the proper names from Wiidb or other sources. You might be able to get those info from the iso themselves but they are often ugly names.

Then again, SSBB is the only known game that WiiBackup Manager can't handle, really worth the effort?

Maybe, but there's also a right way and a wrong way to do things. And if taking my time to understand a new tool or concept can yield better results, why not?

OK, @Wiimm I managed to get as far as this:

wit copy "X:\path_to_iso\iso.iso" --wbfs --dest Z:/path_to_wbfs --psel -update --split --split-size 4G

Apparently, those parameters give me the results I'm looking for. Does it look right to you?
Now, how do I "automate" that? And, how do I rename the resulting files according to WIITDB info (not manually, of course)?
Thanks.

UPDATE:
Managed to create the folder and name of the file with these parameters:

wit copy "X:\path_to_iso\iso.iso" --wbfs --dest Z:/path_to_wbfs/%T[%I]/%I.wbfs --psel -update --split --split-size 4G

But I can't get it to create a space in between "%T" and "%I". It gives me an error. SO I either have to separate them with an underscore or just have em together like in the code line above.
Are my parameters correct? I've tried to read as much as I can.... And I think I'm beginning to understand the way the wools work but I'm far from knowledgeable.
Is the "--wbfs" parameter even necessary if the detination is already pointing to a file with wbfs extension?

 

[Wiimm]

Only some note to your last command:

--wbfs is not needed, if your destination file has the extension .wbfs. Also true for other file formats.

--DEST (short -D) is like --dest (short -d), but creates missing directories.

Use --overwrite (-o) to allow overwriting existing files.

use --test (-t) for a test/dry run without impact.

Use -vv (--verbose --verbose) to be more verbose with progress infos.

For Windows: use -E$ (--esc $) to change the escape character from % to $. This makes Windows life easier, because it interprets percent chars as special chars for string replacements.

Use always forward slashes instead of backward slashes. This make the life of Cygwin environment easier.

Use surrounding " if using special chars like spaces:
-vvE$ -D "Z:/path_to_wbfs/$T [$I]/$I.wbfs"
"-vvE$" is the same as: "-v -v -E$" is the same as: "--verbose --verbose --esc $"

Alternatively use "$Y" instead of "$T [$I]"
-vvE$ -D "Z:/path_to_wbfs/$Y/$I.wbfs"

--------------------- MERGED ---------------------------

Some more notes about split:

--split-size includes --split

If using --split without --split-size, the size is 4g (4000000000)

4G is to large, use --split-size 4G-32K instead.

Summary:
use either: --split
or: --split-size 4G-32K

 

[sideskroll]

Spoiler

Only some note to your last command:

--wbfs is not needed, if your destination file has the extension .wbfs. Also true for other file formats.

--DEST (short -D) is like --dest (short -d), but creates missing directories.

Use --overwrite (-o) to allow overwriting existing files.

use --test (-t) for a test/dry run without impact.

Use -vv (--verbose --verbose) to be more verbose with progress infos.

For Windows: use -E$ (--esc $) to change the escape character from % to $. This makes Windows life easier, because it interprets percent chars as special chars for string replacements.

Use always forward slashes instead of backward slashes. This make the life of Cygwin environment easier.

Use surrounding " if using special chars like spaces:
-vvE$ -D "Z:/path_to_wbfs/$T [$I]/$I.wbfs"
"-vvE$" is the same as: "-v -v -E$" is the same as: "--verbose --verbose --esc $"

Alternatively use "$Y" instead of "$T [$I]"
-vvE$ -D "Z:/path_to_wbfs/$Y/$I.wbfs"

--------------------- MERGED ---------------------------

Some more notes about split:

--split-size includes --split

If using --split without --split-size, the size is 4g (4000000000)

4G is to large, use --split-size 4G-32K instead.

Summary:
use either: --split
or: --split-size 4G-32K

Hi, thanks, I was JUST about to contact you again.
This is/was my latest attempt:

wit copy --progress --psel -update --split "D:\path\to\isos" D:\path\to\wbfs\%T_[%I]\%I.wbfs
pause

It works from a cmd window but NOT from a bat file. (now I know why, because of the %).
I figured the --split command before . Thanks for pointing it out though.
The reason I'm using .wbfs instead of the parameter --wbfs is because it doesn't work. I don't know if I'm doing something wrong or if it's a bug or something but if I use --wbfs the resulting file is a file with no extension (and it doesn't even split it. it just spits an incomplete extension-less file)
Also, is there a way to use the %x variable for the folder name? Every time I try it, it creates the name properly but with the -wbfs extension :S use $y instead. Got it.
I tried by removing the underscore (which is basically the same) but it won't work with a space in between the 2 variables.

This is what I ended up with:

wit copy --psel -update --split-size 4G-32K --overwrite -vvE$ "X:/path/to/iso/" -D "Z:/path/to/wbfs/%Y/%I.wbfs"
pause

Does it look good to you?
Two things though, Like I mentioned before:
Using

--wbfs

produces a extension-less, non-split file.

And

--split-size 4g-32K

produces the same exact file size that

--split

alone.

Should I be noticing any difference? Or is it internal (in the actual *.wbfs file)?

 

[sideskroll]

OK @Wiimm , I think I got it.

I noticed that the parameters could be anywhere on the command (except for --source/--recurse and -D which need to be entered before their respective lines), right?

So this is my FINAL bat (telle me if it looks good to you so I can start trying it out)

wit copy --recurse "X:/path/to/iso" -D "Z:/path/to/wbfs/$Y/$I.wbfs" --psel -update --split-size 4G-32K --overwrite -vvE$ --test
pause

Is that OK? Do I need to maybe specify a parameter after a command somewhere?
Also, I've tried time and time again and still, --wbfs does nothing. You might want to look into that.
Oh, and still --split/--split-size 4g-32K produce the same exact files.

 

[Wiimm]

1. looks good.

2. I have to prove --wbfs

3. If an option takes a param in must be follow immediately the option. For --DEST alias -D you have the following choices with same impact:
-D PATH
-DPATH (space not needed)
--DEST PATH
--DEST=PATH (no spaces, but equal sign)
--DE PATH (abbreviations are allowed as long as they are not ambiguous)

All these variants is not by me -- the system function getopt() does it!

 

[sideskroll]

1. looks good.

2. I have to prove --wbfs

3. If an option takes a param in must be follow immediately the option. For --DEST alias -D you have the following choices with same impact:
-D PATH
-DPATH (space not needed)
--DEST PATH
--DEST=PATH (no spaces, but equal sign)
--DE PATH (abbreviations are allowed as long as they are not ambiguous)

All these variants is not by me -- the system function getopt() does it!

Thanks for all your help.
I'm all set up now.
Yeah, take a look at the --wbfs command please cause it isn't working (at least for me)
Also, the --split/--split-size commands are outputting the same files. I'm not sure if that's by design or if there should be a difference.

Thanks again for everything. Specially for making such a versatile and powerful tool.
Since I have my little batch file now I won't be using WiiBackupManager again any time soon

 

[phenethylamine]

@Wiimm I was wondering if it would be possible for you to add a source tarball to the releases, since I'm trying to package the tools for the linux distribution I use.

It wouldn't be a massive issue if ViewVC didn't spit out a newly generated tarball every time you download the source, but since it does that it makes it impossible to verify the validity of the file, unfortunately.

 

[Wiimm]

It wouldn't be a massive issue if ViewVC didn't spit out a newly generated tarball every time

ViewVC wil give you always the up-to-date tarball. A release is always compiled from the most current tarball.

 

[phenethylamine]

ViewVC wil give you always the up-to-date tarball. A release is always compiled from the most current tarball.

It gives the up to date tarball, but since it's generating it live the timestamp is different making it impossible to verify the authenticity of the file using a checksum. If you download two tarballs of the exact same revision from ViewVC and run a diff on them, it will tell you they aren't the same file because of this.

Plus, it's really just fairly recommended to have source tarball releases alongside your binary releases.

Obviously you don't need to. There aren't rules for open source software exactly, but it's basically impossible to package your tools for a distro without one (unless they were to be packaged as nonfree, which is untrue)

 

[Wiimm]

Now I see what you mean. But I have no plans to chance the behavior.

 

[phenethylamine]

Now I see what you mean. But I have no plans to chance the behavior.

Alright. Thanks for the tools!

 

[Charix]

Hi, so I was trying to install WIT on Windows, but I keep getting the error "Recourse temporarily unavailable", any idea how to fix this?

* change dir to ./bin
* run bash script ./windows-install.sh

* setup
      1 [main] bash 31060 child_info_fork::abort: C:\Users\<redacted>\Downloads\wit-v3.01a-r7464-cygwin\bin\cygreadline7.dll: Loaded to different address: parent(0x180000) != child(0x170000)
./windows-install.sh: fork: retry: Resource temporarily unavailable
      1 [main] bash 31076 child_info_fork::abort: C:\Users\<redacted>\Downloads\wit-v3.01a-r7464-cygwin\bin\cygreadline7.dll: Loaded to different address: parent(0x180000) != child(0x170000)
./windows-install.sh: fork: retry: Resource temporarily unavailable
      1 [main] bash 31092 child_info_fork::abort: C:\Users\<redacted>\Downloads\wit-v3.01a-r7464-cygwin\bin\cygreadline7.dll: Loaded to different address: parent(0x180000) != child(0x170000)
./windows-install.sh: fork: retry: Resource temporarily unavailable
      1 [main] bash 31104 child_info_fork::abort: C:\Users\<redacted>\Downloads\wit-v3.01a-r7464-cygwin\bin\cygreadline7.dll: Loaded to different address: parent(0x180000) != child(0x170000)
./windows-install.sh: fork: retry: Resource temporarily unavailable
      1 [main] bash 31164 child_info_fork::abort: C:\Users\<redacted>\Downloads\wit-v3.01a-r7464-cygwin\bin\cygreadline7.dll: Loaded to different address: parent(0x180000) != child(0x170000)
./windows-install.sh: fork: Resource temporarily unavailable
--- done

 

[Wiimm]

any idea how to fix this?

Not really. It is either an Cygwin issue (I had similar issues, but never seen this message) or a anti-vir issue.

Which windows version do you use?

EDIT:
Searching for "cygwin fork: retry: Resource temporarily unavailable" I found a lot topics, nearly all with different solutiions. SOme more larger articles tells about memory conflicts with anti-vir.dll.