Hacking WIT: Wiimms ISO Tools / beta test

[Wiimm]

I have committed a new version. It supports patching options like --id --name --region --ios while copying. This version is not tested very well.

The source is published under the license of GPL2. Checkout the sources directly from the SVN repository or browse the source. This URLs are different from the product version. Visit http://wit.wiimm.de/ for more information.

 

[giantpune]

i gave it a test run and everything appears to be working

.wbfs -> .wbfs
.wbfs -> .iso
.iso -> .wbfs
patching ID, IOS, name in tmd, ticket, disc hdr, and part hdr


EDIT>>>
can i suggest something else for a future update? it would be nice to have an option to simply skip failed files when dumping a game to FST.
there are a few games that contain files with weirdly encoded names. these files are not necessarily needed to play the game, and are usually just config files for the games creators & testers. when trying to dump these games with WIT to a FAT filesystem, it will try to make the filename and will not be able to and just abort the entire game. instead, i have to dump the game to my ext4 drive and then copy the folder to my FAT drive and just skip the files that it cannot create in FAT.

EDIT again>>>

there appears to be an issue with patching the ID. i am starting with pikmin2 PAL as a .wbfs file. its ID shows up as R92P01. i copy it and try to patch the ID. the ID in the new disc header is right. but the ID in the boot.bin is incorrect. it is patching the first 4 characters in the boot.bin correctly. but it is messing up the last 2 characters. i am getting 52 39 32 45 4d 7f. the last 2 bytes should be 30 31.

 

[Wiimm]

I have committed again. The boot.bin/ID bug is fixed.

There is a new command: wit EDIT: It modify iso image directly.

@giantpune
I will change the extraction behavior. Can yo tell me some games with problems, perhaps I own one.


EDIT:
I have started my test scripts and i have found some problems while composing a disc.

 

[giantpune]

the 2 games i have that have these files with weird names are pikmin 2 (R92P01 ) and juon ( RJOP99 ).

 

[Wiimm]

comitted again. composing works now. extracting changed with error message after extracting all (untested because either i don't own such game or linux has no probs)

 

[TempusC]

Rather than ignoring names, chances are good they’re simply SJIS (or more rarely UTF-8) encoded. Why not implement some encodings like SJIS_X0213 and UTF-8 into the program?

 

[Wiimm]

UTF-8 is already there. And names with --name are used 1:1 independent from any encoding.

 

[Wiimm]

WIT: Wiimms ISO Tools v1.10a - 2010-07-24
Beta test version, for EXPERIMENTAL use only.

WIT (Wiimms ISO Tools) is a set of command line tools to manipulate Wii ISO images and WBFS containers. The two main tools are called wit (Wiimms ISO Tool) and wwt (Wiimms WBFS Tool, the WBFS manager).

For productive versions see the main WIT thread or visit http://wit.wiimm.de/.

Download this EXPERIMENTAL release:

• wit-v1.10a.beta-r1428-i386.tar.gz : linux/i386 version
• wit-v1.10a.beta-r1428-x86_64.tar.gz : linux/x86_64 version
• wit-v1.10a.beta-r1428-mac.tar.gz : Mac OS universal binaries (ppc, i386 and x86-64)
• wit-v1.10a.beta-r1428-cygwin.zip : cygwin (win32) version
• Other releases are available from http://wit.wiimm.de/download.html.

Highlights of this release:

• The commands "wit COPY", "wwt ADD" and some others accepts the patching options --id, --name, --ios, --region and --psel to modify the ISO image on the fly. If needed, the partitons will be truscha signed.
  .
• The new command "wit EDIT" modifies existing ISO images directly without copying.
  .
• The command "wit DUMP" is able to print much more info. There are new keywords for the option --show: USAGE, PATCH, RELOCATE. Try "--show all,-files --id A" to see the nearly full output.
  .
• The GUI QtWitGui is able to handle most of this.
  .

Change log:

wit v1.10a r1386 - 2010-07-15

- wiitdb.com: Support of new languages: ru, zhcn, zhtw
- wwt ANALYZE: If option --long is set then calculated values are printed too
ÂÂ if other values are available. If option --long is set twice calculated
ÂÂ values are always printed.
- wit+wwt: Option --trunc: Truncate created PLAIN ISO image to needed size.

wit v1.10a.beta r1428 - 2010-07-24

- Computed line breaks for error messages.
- "wit COPY" accepts the options --id --name --ios --region to patch the
ÂÂ image while copying.
- New command: wit EDIT: Patch ISO images directly without copying.

Missing docu:

- wit commands ILIST + EXTRACT
- Option --show => see built in help

The source is published under the license of GPL2. Checkout the sources directly from the SVN repository or browse the source. This URLs are different from the product version. Visit http://wit.wiimm.de/ for more information.

 

[giantpune]

im getting issues with the region setting again. i have used this to patch an ISO from region 1 to 4. when i do DUMP on the new iso, i get this

Region setting:ÂÂ67108864 / 00 00 00 00ÂÂ00 00 00 00

 

[Wiimm]

Hey giantpune, it looks like an endian problem. (think, fix is easy)
Extracting works now for you?


EDIT Fixed, update is coming today

 

[giantpune]

is there anything specific about extracting i should be testing?

 

[Blue-K]

I have a question...I want to convert all my games from .wbfs to SNEEK, so I use this command:

wit cp /home/Source -vvP --fst --pmode none --files =sneek -D /games/%T

I didn't wrote --sneek, because that would exclude the VC/Channel-Partitions, which I would like to keep, right?

Now, to the main problem...using the --test Command, I recieve Errors for two games. Like:

Spoiler

!! wit: ERROR #13 [INVALID WII DISC] in wd_load_part() @ src/libwbfs/wiidisc.c#1092
!! Invalid header in partition '0 [DATA]': /media/Data/wbfs/Marble Saga - Kororinpa [RK6E18].wbfs/#0
!! wit: ERROR #13 [INVALID WII DISC] in wd_load_part() @ src/libwbfs/wiidisc.c#1092
!! Invalid header in partition '0 [DATA]': /media/Data/wbfs/Marble Saga - Kororinpa [RK6E18].wbfs/#0
wit: WOULD EXTRACT WBFS:/media/Data/wbfs/Marble Saga - Kororinpa [RK6E18].wbfs/#0 -> /media/Data/games/Marble Saga - Kororinpa/

and

!! wit: ERROR #13 [INVALID WII DISC] in wd_load_part() @ src/libwbfs/wiidisc.c#1092
!! Invalid header in partition '45384148 ["HA8E"]': /media/Data/wbfs/Super Smash Bros. Brawl [RSBE01].wbfs/#0
!! wit: ERROR #13 [INVALID WII DISC] in wd_load_part() @ src/libwbfs/wiidisc.c#1080
!! Position of partition '45394148 ["HA9E"]' exceeds file size: /media/Data/wbfs/Super Smash Bros. Brawl [RSBE01].wbfs/#0
!! wit: ERROR #13 [INVALID WII DISC] in wd_load_part() @ src/libwbfs/wiidisc.c#1080
!! Position of partition '45414248 ["HBAE"]' exceeds file size: /media/Data/wbfs/Super Smash Bros. Brawl [RSBE01].wbfs/#0
!! wit: ERROR #13 [INVALID WII DISC] in wd_load_part() @ src/libwbfs/wiidisc.c#1080
!! Position of partition '45424248 ["HBBE"]' exceeds file size: /media/Data/wbfs/Super Smash Bros. Brawl [RSBE01].wbfs/#0
!! wit: ERROR #13 [INVALID WII DISC] in wd_load_part() @ src/libwbfs/wiidisc.c#1080
!! Position of partition '45434248 ["HBCE"]' exceeds file size: /media/Data/wbfs/Super Smash Bros. Brawl [RSBE01].wbfs/#0
!! wit: ERROR #13 [INVALID WII DISC] in wd_load_part() @ src/libwbfs/wiidisc.c#1080
!! Position of partition '45444248 ["HBDE"]' exceeds file size: /media/Data/wbfs/Super Smash Bros. Brawl [RSBE01].wbfs/#0
!! wit: ERROR #13 [INVALID WII DISC] in wd_load_part() @ src/libwbfs/wiidisc.c#1080
!! Position of partition '45454248 ["HBEE"]' exceeds file size: /media/Data/wbfs/Super Smash Bros. Brawl [RSBE01].wbfs/#0
!! wit: ERROR #13 [INVALID WII DISC] in wd_load_part() @ src/libwbfs/wiidisc.c#1080
!! Position of partition '45464248 ["HBFE"]' exceeds file size: /media/Data/wbfs/Super Smash Bros. Brawl [RSBE01].wbfs/#0
!! wit: ERROR #13 [INVALID WII DISC] in wd_load_part() @ src/libwbfs/wiidisc.c#1080
!! Position of partition '45474248 ["HBGE"]' exceeds file size: /media/Data/wbfs/Super Smash Bros. Brawl [RSBE01].wbfs/#0
!! wit: ERROR #13 [INVALID WII DISC] in wd_load_part() @ src/libwbfs/wiidisc.c#1080
!! Position of partition '45494248 ["HBIE"]' exceeds file size: /media/Data/wbfs/Super Smash Bros. Brawl [RSBE01].wbfs/#0
!! wit: ERROR #13 [INVALID WII DISC] in wd_load_part() @ src/libwbfs/wiidisc.c#1080
!! Position of partition '454b4248 ["HBKE"]' exceeds file size: /media/Data/wbfs/Super Smash Bros. Brawl [RSBE01].wbfs/#0
!! wit: ERROR #13 [INVALID WII DISC] in wd_load_part() @ src/libwbfs/wiidisc.c#1080
!! Position of partition '454c4248 ["HBLE"]' exceeds file size: /media/Data/wbfs/Super Smash Bros. Brawl [RSBE01].wbfs/#0
wit: WOULD EXTRACT WBFS:/media/Data/wbfs/Super Smash Bros. Brawl [RSBE01].wbfs/#0 -> /media/Data/games/Super Smash Bros. Brawl/

What does ERROR #13 mean? The games seem to work fine, so I dunno what's the problem.

 

[Wiimm]

@giantpune:
delayed error messages "can't create file" while extracting pikmin 2 or juon.

@Blue-K
ssbb: It looks like the dame incomplete dump as my first dump. try "wit d ssb_source" and compare the partitions offset and the file size.
Remember you tip: dump with cfg_loader (install_partitions=1:1 in config.txt)

My Kororinpa (RCPP18) have no probs. Which wit version did you use?

And yes: --sneek includes --psel data. use "--files :sneek"

 

[giantpune]

extracting works good. it has waited until the end of the process to say "oh shit, there was 10 files that could not be created, but i went ahead and did what i could"

 

[Wiimm]

ok & thanx for testing
Time to create the next productive release

 

[giantpune]

using r1431, i get some weird behavior when doing COPY wbfs -> FST with --sneek

it dumps what appear to be the correct files, but all the files that share the same name between partitions are the ones from the update partition

/home/j/c/wiimms_tools/1431/wit CP /media/WII_464GB_/wbfs/RSBE01_Super Smash Bros. Brawl/RSBE01.wbfs /media/WII_464GB_/games/SSBB_USA --fst --region=1 --sneek -v -v -v -v -L -L --overwrite --progress
wit: Wiimms ISO Tool v1.10a.beta r1431 x86_64 - Dirk Clemens - 2010-07-25

PROGRAM_NAMEÂÂ = wit
SEARCH_PATH[0] = /home/j/c/wiimms_tools/1431/
SEARCH_PATH[1] = /usr/local/share/wit/
SEARCH_PATH[2] = ./


! wit: Title file not found: titles.txt
! wit: Title file not found: titles.local.txt

Patching table of the disc:

ÂÂÂÂÂÂÂÂoffset .. offset endÂÂÂÂ sizeÂÂcomment
ÂÂ ----------------------------------------------
ÂÂÂÂÂÂÂÂ 4e000 ..ÂÂÂÂ 4e020ÂÂÂÂÂÂÂÂ20ÂÂregion: 1

Patching table of the disc:

ÂÂÂÂÂÂÂÂoffset .. offset endÂÂÂÂ sizeÂÂcomment
ÂÂ ----------------------------------------------
ÂÂÂÂÂÂÂÂ 4e000 ..ÂÂÂÂ 4e020ÂÂÂÂÂÂÂÂ20ÂÂregion: 1

!! wit: ERROR #20 [CAN'T CREATE FILE] in CreateFileFST() @ src/iso-interface.c#1803
!! File already created: /media/WII_464GB_/games/SSBB_USA/ticket.bin
!! wit: ERROR #20 [CAN'T CREATE FILE] in CreateFileFST() @ src/iso-interface.c#1803
!! File already created: /media/WII_464GB_/games/SSBB_USA/tmd.bin
!! wit: ERROR #20 [CAN'T CREATE FILE] in CreateFileFST() @ src/iso-interface.c#1803
!! File already created: /media/WII_464GB_/games/SSBB_USA/sys/boot.bin

!! wit: ERROR #20 [CAN'T CREATE FILE] in CreateFileFST() @ src/iso-interface.c#1803
!! File already created: /media/WII_464GB_/games/SSBB_USA/sys/bi2.bin
!! wit: ERROR #20 [CAN'T CREATE FILE] in CreateFileFST() @ src/iso-interface.c#1803
!! File already created:
!! /media/WII_464GB_/games/SSBB_USA/sys/apploader.img
!! wit: ERROR #20 [CAN'T CREATE FILE] in CreateFileFST() @ src/iso-interface.c#1803
!! File already created: /media/WII_464GB_/games/SSBB_USA/sys/main.dol
!! wit: ERROR #20 [CAN'T CREATE FILE] in CreateFileFST() @ src/iso-interface.c#1803
!! File already created: /media/WII_464GB_/games/SSBB_USA/sys/fst.bin

 

[Wiimm]

I had changed the --psel option. And --sneek implies --psel, but i have forgotten to change the code here.

The svn (beta) contains an update with a bug fix.

 

[Wiimm]

giantpune has nudged me and so the next extension of wit is on the way (idea only yet).

I plan a wit command, that takes partitions from any source and creates a new wii disc. The source partitions are only copied so that there is no need for a fake signing.

I plan the following syntax (greetings from sql):

COMMAND := 'wit' 'mix' SOURCE... --dest destination
SOURCEÂÂ:= sourcefile ['select' ptype] ['as' [ptab '.'] ptype]

-> ptype is the partition type (DATA,UPDATE,CHANNEL,number,ID4)
-> ptabÂÂis the partition table (0..3)

• If 'select' is not set, the DATA partition is selected.
• If 'as' is not set partition table #0 and the partition type of the source is used.
• Filenames should never written without a slash or point (prefix it by './' if necessary). This makes extensions compatible.

Examples:

# copy DATA partitions of 'wpl' and 'mkw' and store them in a WDF file
wit mix wpl.ciso mkw.wdf --dest result.wdf

# copy DATA partition of 'wpl' and CHANNEL partition of 'mkw'
# and store them in a PLAIN ISO file
wit mix wpl.ciso mkw.wdf select chan --dest result.iso

# copy DATA partition of 'wpl' and CHANNEL partition of 'mkw'
# and store both as DATA partition in a WBFS file
wit mix wpl.ciso mkw.wdf select chan as data --dest result.wbfs

DATA partitions are almost splitted into to parts: The beginning with the system files and then after a large hole all the real files are stored. It seems that Nintendo wants that the data is aligned at the end of a disc. This fact suggest me 2 options (implemented in a second step if ever):

• --trim : pack DATA (remove holes), fake signing is needed.
• --overlay : address the partitions so that the relevant (scrubbed) data is stored into the hole of the other data. This seems very experimental, but should work in theory.

Any more suggestions?

 

[Wiimm]

Some comments of 2 members arrived me per PN. Both like my ideas.
But this is the discussion forum, please post here!

 

[Wiimm]

I have committed a new version in the beta branch:

• I have made a first implementation of "wit mix" with some poor tests.

See "wit help mix" for info:
CODEwit MIX : Mix the partitions from different sources into one new Wii disc.

Syntax: wit MIX SOURCE... --dest|--DEST outfile
ÂÂÂÂÂÂÂÂwhere SOURCE := infile ['select' ptype] ['as' [ptab '.'] [ptype]]

Options:

ÂÂ-t --testÂÂÂÂÂÂÂÂÂÂRun in test mode, modify nothing.
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ >>> USE THIS OPTION IF UNSURE!