Hacking wwt+wit: Wiimms WBFS+ISO Tools

[mousex]

123

 

[W hat]

The above is for saving space on your PC hard drive. When you want to use the game, you would extract the archive and re-pack into ISO with WIT.

 

[mousex]

123

 

[Wiimm]

Ok. Sounded different. Sorry. Than Wiimm's suggestion (to extract the decrypted partition) will work great. You won't even have to build a new partition this way.

I have tested weeks ago decryption and compressing (bzip2) with 5 different games. The files are from 10% up to 45% smaller than without decryption. A next step would to clear all hash values and calculate them. The hashes use about 3% (exactly 1/32) of partition memory.

 

[W hat]

Good news everyone! I tried super compressing two games I have, and the results are promising. They are not scientifically rigorous yet, but when I get more time I'll try it with some bigger games. I wonder if the % saved will go up or down? 60% (edit: or 90%) of the size of WDF after the savings you already get with using WDF and removing updates and scrubbing the discs is amazing!

Game - Size (wdf) - Size (7-zip [ultra] decrypted) - % of WDF
Balloon Pop - 34 MB - 18.5 MB - 54%
Wii Play - 93.8 MB - 49.2 MB - 52%
SMG2 - 1.29 GB - 1 GB - 77.51%

Wiimm - Thanks for your awesome tool. If I plug in a WBFS formatted drive, how might I batch dump them into a folder on my PC? Could I do this without dumping every game, only like 10 or so?

If I have a folder of WDF games that I want to dump into their own fst folders, could I do that? What command would get me there?

I looked after finding it one time, but I didn't see it again on your new site. I think I read something about preserving original created / modified dates in the files. Could you explain that? I think it means preserve the created / modified dates on ex. ticket.bin on Wii Play.

edit: Didn't see the above post because I 'fast replied' from the first post.

 

[Wiimm]

Wiimm - Thanks for your awesome tool. If I plug in a WBFS formatted drive, how might I batch dump them into a folder on my PC? Could I do this without dumping every game, only like 10 or so?
CODE# extract all games to current dir
wwt -a extract +

# dumping only 10 games are a little tricky but no problem working with unix
# (ask if you don't understand)
wwt -a extract $( wwt id | head -n10 )

# wit can access the images by index:
for i in $(seq 10); do copy /dev/source/$i destdir; done
# or
for ((i=0;i

 

[W hat]

I suspect my problem here is my WIT and my games (WBFS) are on two different drives, so I can't execute this command on Windows CMD:

F:\wit\bin\>wit extract l:\RFBE01 -d -t f:\games\

Error #16 [can't open file]
can't open /cygdrive/l/RFBE01

edit: Even if it could, I think my syntax is wrong.

I don't know how to dump 10 games in Windows CMD either.

 

[Wiimm]

@W hat
You have to give the source file: l:\RFBE01 isn't a source file.

If you have a pool of discs you can also gibe that directory and use an include list:

wit extract base_dir_of_pool --include=id6 -d -t f:/games/%Y

Use a %-escape as destination to store all disc in different dirs.

You can use the --include multiple times to include more than one disc. It is also possible to say --include @file when file is a list of id6. And that might be a windows solution for you problem:

build a id list with "wit id6 >id.list" or "wwt id6 >id.list", edit abs split this list and use it with [email protected].

On the other hand:
Start an job with --update an wehen you think that's enough press CTRL-C once: wit+wwt will stop execution after finishing the current job.

 

[W hat]

Unrelated to the CMD talk going over my head
---

I plugged in my drive and checked it with WWT. I get this:

22 errors found
22 WBFS sectors are used by 2 or more discs!
15 discs are invalid!

I want to fix it, but the commands I try just tell me the above.

edit:

 * Check for disc block errors.
ÂÂ - #0=RFBE01/0: WBFS block #2 is used 2 times!
ÂÂ - #0=RFBE01/63: WBFS block #4 is used 2 times!
ÂÂ - #0=RFBE01/903: WBFS block #16 is used 2 times!
ÂÂ - #0=RFBE01/919: WBFS block #32 is used 2 times!
ÂÂ - #0=RFBE01/951: WBFS block #64 is used 2 times!
ÂÂ - #0=RFBE01/1015: WBFS block #128 is used 2 times!
ÂÂ - #2=RELEA4/517: WBFS block #512 is used 2 times!
ÂÂ - #2=RELEA4/1029: WBFS block #1024 is used 2 times!
ÂÂ - #7=RM3E01/99: WBFS block #2048 is used 2 times!
ÂÂ - #7=RM3E01/378: WBFS block #2326 is used 2 times!
ÂÂ - #9=RODE01/405: WBFS block #4096 is used 2 times!
ÂÂ - #11=RTZE08/638: WBFS block #4908 is used 2 times!
ÂÂ - #11=RTZE08/2176: WBFS block #22338 is used 2 times!
ÂÂ - #11=RTZE08/2177: WBFS block #18003 is used 2 times!
ÂÂ - #11=RTZE08/2178: WBFS block #4908 is used 2 times!
ÂÂ - #11=RTZE08/2179: WBFS block #18744 is used 2 times!
ÂÂ - #11=RTZE08/2180: WBFS block #2326 is used 2 times!
ÂÂ - #11=RTZE08/2182: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2183: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2184: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2185: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2186: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2187: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2188: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2189: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2190: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2191: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2192: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2193: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2194: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2195: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2196: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2197: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2198: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2199: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2200: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2201: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2202: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2203: WBFS block #257 is used 22 times!
ÂÂ - #11=RTZE08/2204: WBFS block #256 is used 2 times!
ÂÂ - #17=ROWE08/208: WBFS block #8192 is used 2 times!
ÂÂ - #33=RHDE8P/610: WBFS block #16384 is used 2 times!
ÂÂ - #39=RBYE78/1069: WBFS block #18003 is used 2 times!
ÂÂ - #40=RCDE52/884: WBFS block #18744 is used 2 times!
ÂÂ - #45=RGME5D/1097: WBFS block #22338 is used 2 times!
ÂÂ - #53=SXAE52/151: WBFS block #25616 is used 2 times!
ÂÂ - #54=SMNE01/1116: WBFS block #25616 is used 2 times!
ÂÂ - #63=RFBJ01/0: WBFS block #27712 is used 2 times!
ÂÂ - #63=RFBJ01/1116: WBFS block #28429 is used 2 times!
ÂÂ - #64=RFBP01/2: WBFS block #27712 is used 2 times!
ÂÂ - #66=DMHE08/62: WBFS block #2 is used 2 times!
ÂÂ - #66=DMHE08/63: WBFS block #4 is used 2 times!
ÂÂ - #66=DMHE08/64: WBFS block #16 is used 2 times!
ÂÂ - #66=DMHE08/65: WBFS block #32 is used 2 times!
ÂÂ - #66=DMHE08/66: WBFS block #64 is used 2 times!
ÂÂ - #66=DMHE08/67: WBFS block #128 is used 2 times!
ÂÂ - #66=DMHE08/68: WBFS block #256 is used 2 times!
ÂÂ - #66=DMHE08/69: WBFS block #512 is used 2 times!
ÂÂ - #66=DMHE08/70: WBFS block #1024 is used 2 times!
ÂÂ - #66=DMHE08/71: WBFS block #2048 is used 2 times!
ÂÂ - #66=DMHE08/72: WBFS block #4096 is used 2 times!
ÂÂ - #66=DMHE08/73: WBFS block #8192 is used 2 times!
ÂÂ - #66=DMHE08/74: WBFS block #16384 is used 2 times!
ÂÂ - #66=DMHE08/75: WBFS block #28429 is used 2 times!
* Check free blocks table.

* Summary of WBFS Check: 22 errors found:
ÂÂÂÂ22 WBFS sectors are used by 2 or more discs!
ÂÂ Total: 15 discs are invalid!

 

[W hat]

Based on the games I have and their file sizes, I'm going to do some compression testing on some games.

This post should also be where I post my results. For now it's just RAR and 7-zip. 7z ultra and Rar (best) are quickly becoming impractical. My slow PC and the law of diminishing returns means for 1 GB+ folders, I'm using normal settings, not ultra/best.

GameÂÂÂÂÂÂÂÂ|SizeÂÂ |SizeÂÂ |Size
ÂÂÂÂÂÂÂÂÂÂÂÂ|(WDF)ÂÂ|(Rar)ÂÂ|(7z)
Balloon Pop |34 MBÂÂ|20.7 MB|18.5 MB
Wii PlayÂÂÂÂ|93.8 MB|60.7 MB|49.2 MB
Wii SportsÂÂ|320 MB |219 MB |128 MB
AC: CFÂÂÂÂÂÂ|320 MB |ÂÂÂÂÂÂ |53.5 MB
GT ProÂÂÂÂÂÂ|660 MB |ÂÂÂÂÂÂ |215 MB
SMG2ÂÂÂÂÂÂÂÂ|1.29 GB|943 MB |1 GB
de BlobÂÂÂÂ |2 GBÂÂ |1.48 GB|1.38 GB
WL: Shake It|3.55 GB|ÂÂÂÂÂÂ |1.4 GB
MP3: Corrupt|
HotD: OverÂÂ|
List is a WIP

 

[giantpune]

ive just tried to create a .wbfs file from scratch. i have all my files in a folder and given the command

j@pc:/media/1TB_3GbS_2/ubuntu_wiishit$ '/media/1TB_3GbS_2/ubuntu_wiishit/wiims/bin/wit' CP '/media/1TB_3GbS_2/ubuntu_wiishit/homebrewISO' '/media/1TB_3GbS_2/ubuntu_wiishit/homebrewISO/damnISO.iso' 
*****ÂÂwit: Wiimms ISO Tool v0.47a r1106 x86_64 - Dirk Clemens - 2010-05-22ÂÂ*****
* wit SCRUB 1/1 FST:/media/1TB_3GbS_2/ubuntu_wiishit/homebrewISO -> ISO:/media/1TB_3GbS_2/ubuntu_wiishit/homebrewISO/damnISO.iso

and this creates a iso. i can open the iso in wiiscrubber and it appears correct. when i try the same command to create a .wbfs file i get an error.

j@pc:/media/1TB_3GbS_2/ubuntu_wiishit$ '/media/1TB_3GbS_2/ubuntu_wiishit/wiims/bin/wit' CP '/media/1TB_3GbS_2/ubuntu_wiishit/homebrewISO' '/media/1TB_3GbS_2/ubuntu_wiishit/homebrewISO/damnISO.wbfs' 
*****ÂÂwit: Wiimms ISO Tool v0.47a r1106 x86_64 - Dirk Clemens - 2010-05-22ÂÂ*****
* wit SCRUB 1/1 FST:/media/1TB_3GbS_2/ubuntu_wiishit/homebrewISO -> WBFS:/media/1TB_3GbS_2/ubuntu_wiishit/homebrewISO/damnISO.wbfs
!! wit: ERROR #23 [WBFS ERROR] in CopyToWBFS() @ src/lib-sf.c#1942
!!ÂÂÂÂÂÂError while creating disc [] @/media/1TB_3GbS_2/ubuntu_wiishit/homebrewISO/.damnISO.wbfs.y4fi4a
j@pc:/media/1TB_3GbS_2/ubuntu_wiishit$

is this something ive done wrong?

also, how does it flag the partitions? are they all flagged as data?

 

[Wiimm]

@W hat
You check log looks curious. Please try "wwt dump -lll" and look at the "WBFS Memory Map:" at the very end to verify this results. Overlapped areas are marked with "!" in the first column. Please post the log or send it via mail (I'll give you my mail address per PN).

@giantpune
Your command line is correct and you have found a bug.

 

[mattesno1]

Hi Wiimm,

I wrote a little bash script to process my Wii-ISOs.
With the following line wit converts my ISOs in a certain folder. Aafterwards it's supposed to delete the converted .iso files. Unfortunately it deletes the whole folder with al the newly created .wbfs-files, which I save in the same dir as the source ISOs.

wit copy -r "$DIR" %P/%I.%E -vBR --rdepth 3

Is there a way to process folders with WIT and deleting just the .isos and not the folder itself? Because now, all my games are gone

 

[Wiimm]

@mattesno1
i can't reproduce your error. I run you command 2 times an the same dir using linux/i386:
1.) All *.wdf are replaced by *.wbfs files
2.) I got error messages because an error of WBFS files as source. There is a naming bug but all files including non ISO files are still there.

One hint for playing not only with wit:
If I play with creating and removing (not modifying) files, I do a fast copy (hard links of files) with "cp -rl source dest" and run my scripts on on dest.

@giantpune:
Your bug is gone ... expect the next release.

 

[giantpune]

another bug in the ISO creation...
it seems that these ISOs are created with the jap region flag somewhere? i have the region in the TID set to E in the disc header, boot.bin, tmd, and ticket. the extracted files play fine in sneek on a 4.2U system menu. when i use wit to turn the files into a ISO and mount it in the wode, it shows up as jap region. turns out the u32 at 0x4e000 is still set to 0 (jap) in the wit-created ISO. obviously this isnt an issue for usb loaders, but its a big headache for me using the wode.

EDIT>>
also, 0x4fffc should be 0xC3F81A8E

 

[Wiimm]

another bug in the ISO creation...
it seems that these ISOs are created with the jap region flag somewhere? i have the region in the TID set to E in the disc header, boot.bin, tmd, and ticket. the extracted files play fine in sneek on a 4.2U system menu. when i use wit to turn the files into a ISO and mount it in the wode, it shows up as jap region. turns out the u32 at 0x4e000 is still set to 0 (jap) in the wit-created ISO. obviously this isnt an issue for usb loaders, but its a big headache for me using the wode.

EDIT>>
also, 0x4fffc should be 0xC3F81A8E

The region settings are not clear. I have compared some of my discs with the info on http://wiibrew.org/wiki/Wiidisc but i have found total different values. So I store the values in disc/region.bin and load again if the file exists while composing.

0x4fffc: simply forgotten

 

[giantpune]

simply setting 0x4000e to 00000001, the rest of the region bytes to 0 and 0x4fffc to c3f81a8e is enough to allow the wode to see the game as USA region and have system menu 4.2U able to read the image.

 

[Wiimm]

simply setting 0x4000e to 00000001, the rest of the region bytes to 0 and 0x4fffc to c3f81a8e is enough to allow the wode to see the game as USA region and have system menu 4.2U able to read the image.

Address 0x4000e is not reported in the wiki. I think you mean 0x4e000?!
And from where do I know that it is USA and not Europe/Jap/Kor?

Interesting: libwbfs scrubbing never copy 0x4fffc -> it's lost.

 

[giantpune]

yeah, i mean 0x4000e. i guess you can either make it a CLI switch to set a region, or you can get it from the TID in the tick/tmd.

it would be nice to have both. so if i didnt tell it any region, it will look at the TID and see E and say "ok, this is a USA game", but if i want to have a TID with E and force PAL, i can say

wit CP -REGION USA
or something like this

 

[Wiimm]

The wiki says:

0x4E000 ÂÂÂÂ 4 ÂÂÂÂ Region byte ÂÂÂÂ 0 = JAP, 1 = USA, 2 = EUR, 4 = KOR

It looks a little bit like a bit field. What happen if the value set to 7. Is it valid for all regions? What does wode do?

And yes, an additional option is good.

And again:

yeah, i mean 0x4000e.

really 0x4000e - it is part of the partition table (or am i too tired)