Hacking wwt+wit: Wiimms WBFS+ISO Tools

[Nickkk]

I think I found a bug when trying to convert a game from wbfs to wia with the argument --compression FAST. GOOD and BEST seem to work!

!! wit: ERROR #25 [CAN'T OPEN FILE] in SetupWriteWIA() @ src/lib-wia.c#2371
!! Can't open file stream: /Users/me/Desktop/hjk.wia
!! -> Bad file descriptor [9]
2012-11-06 15:02:36.589 Witgui[6805:303] !! wit: ERROR #41 [FATAL ERROR] in write_data() @ src/lib-wia.c#1483
!! ASSERTION FAILED !!!

 

[PsyBlade]

I would be gratefull if you could take a look at this file
http://www.mediafire.com/?wvsnucp6zvxrh7m
I would like to know if this is just fucked up bad (and maybe how it happened) or if this is a problem with our tools

It's a verbatim copy ot the first block of a WBFS
Submitter claimed that it worked but that wwt could not open the WBFS; seems inexperienced

I can open it with wwt just fine
The block mappings of some games look messed up bad but somehow seem strange
I think this MIGHT be some data format we don't understand

see http://gbatemp.net/threads/wbfs2fat-py.291320/page-8#post-4444055 and later for details

EDIT:
Appart from the common x-linking problem it seems some single layer games have uninitialized garbage for the second layer part of the block map instead of the usual zeros
Was able to work around the issue for THIS user with a particularly ugly hack
(skip 2. layer if block 22 is maped)
No idea how to detect this in general

 

[Wiimm]

@Nickkk+PsyBlade
I'm very busy at the moment I will do your tasks later.

 

[Nickkk]

Please forget the canceling issue, I tried again and now it works. Just to summarize my latest posts (and a couple of new issues):

1) Having individual downloads of wit, wwt etc would be great so that GUIs based on those can download them separately and don't have to ship them along everytime.

2) Converting from wbfs to win with the FAST compression mode doesn't seem to work, extracting from wia to wia works indeed.

3) Please update --scan-progress on wwt :-)

4) I noticed the CPU % getting used is sometimes really low ... is this normal? Or could conversions somehow run faster? Also the percent of work done when converting to wia jumps to 100% almost at the beginning and then back again a couple of times.

5) Canceling conversions to the iso format can take quite a lot of time ... is this normal, too?

Thank you for your patience and your answers! Your tools are really appreciated ;-)

 

[b0ring]

So I decided to finally try and convert my usb hard drive from WBFS to FAT32, but I'm not able to because of some problem games. Wii Backup Manager lists four potential problem games (they list ??? partitions).

I'm trying to figure out how to fix the problem but wwt keeps failing...

C:\Program Files\Wiimm\WIT>wwt verify -a -R -vvv

wwt: Wiimms WBFS Tool v2.10a r4118 cygwin - Dirk Clemens - 2012-10-10

One WBFS partition found.
* Summary of WBFS Check: 2 errors found:
2 WBFS sectors are used by 2 or more discs!
Total: 3 discs are invalid!


WBFSv1 #1/1 opened: /dev/sde1
>scan 1.0/70 DATA RODP01 WarioWare: Smooth Moves
>info TICKET & TMD are well signed. Partition is encrypted
and scrubbed.
+OK 1.0/70 DATA RODP01 WarioWare: Smooth Moves
>scan 2.0/70 DATA RHAP01 Wii Play
>info TICKET & TMD are well signed. Partition is encrypted
and scrubbed.
+OK 2.0/70 DATA RHAP01 Wii Play
!! wwt: ERROR #40 [OUT OF MEMORY] in my_malloc() @ src/debug.c#190
!! Out of memory while malloc() 3809215494 bytes (0xe30c0406)

I get the same thing, every time, with the exact same memory location. Any idea what my next move should be?

 

[Wiimm]

@psyblade:
I have spent only a little time but I can't see any curious thing.

@nickkk:
1.) Not planned
2.) Converting to "win"?
3.) For which operation? scanning of a wbfs drive is fast.
4.) Mostly the tools wait for disc io. For WIA it changes multiple times between de-/compressing and disc io.
5.) My tools will test the abort condition before each read/write call. It if the OS cleans the cache, there no chance to abort.

@b0ring:
I think it is an error because of wring data. I will create a new version in the next days to find the correct source code position to add some additional tests. Can you wait before removing the bad discs? Perhaps a raw copy of the related images and verifying the copied image will produce the same results.

 

[PsyBlade]

look for example at offset 0xD960
its in the middle of the block maping for R9JE69 (RBBeatles)
or the output of check -vv for that game

 

[b0ring]

@b0ring:
I think it is an error because of wring data. I will create a new version in the next days to find the correct source code position to add some additional tests. Can you wait before removing the bad discs? Perhaps a raw copy of the related images and verifying the copied image will produce the same results.

No problem, I'll try copying in the next few days when I have some time. Thanks for the quick reply.

 

[Wiimm]

@b0ring:
Here are some updates. They will print the source position more exact. That helps me to implement a guardian.
* http://wit.wiimm.de/download/bin/cygwin/wit-3194.exe
* http://wit.wiimm.de/download/bin/cygwin/wwt-3194.exe
Both are only the Cygwin binaries.

 

[b0ring]

@b0ring:
Here are some updates. They will print the source position more exact. That helps me to implement a guardian.
* http://wit.wiimm.de/download/bin/cygwin/wit-3194.exe
* http://wit.wiimm.de/download/bin/cygwin/wwt-3194.exe
Both are only the Cygwin binaries.

A quick run with the new files got me this, I hope it's of some use:

!! wwt: ERROR #40 [OUT OF MEMORY] in wd_load_part() @ src/libwbfs/wiidisc.c#2036

!! Out of memory while malloc() 3809215494 bytes (0xe30c0406)

 

[Wiimm]

That helped. And please now try:
* http://wit.wiimm.de/download/bin/cygwin/wit-4196.exe
* http://wit.wiimm.de/download/bin/cygwin/wwt-4196.exe
It should detect the wrong TMD size and abort only the one disc and continue with the others.

 

[Nickkk]

I meant wia, of course Converting to win with option FAST enabled throws an error. I meant adding --scan-progress to the LIST command as in wit. So, is there no way of using all CPU for a single conversion?

 

[Wiimm]

liblzma supports 2 CPUs, but because any reasons this didn't worked correct. But I made only a little try, because WIA is still my academic project and I never used it and so I will not waste my time.

 

[b0ring]

Thank you for these quick updates. They're much appreciated. Running it again, I got further but still hit problems.

I've put the output here: click me!

Now that clearly tells me that "Super Smash Bros. Brawl" is a problem, but that was backed up with seemingly no problem when I used Wii Backup Manager. And now I'm hitting an internal error?

Should I presume from all this that my hard drive is just fucked..?

 

[Wiimm]

If you mean this:

!! wwt: ERROR #14 [INVALID WII PARTITION] in wd_load_part() @ src/libwbfs/wiidi
c.c#2033
!!      Invalid TMD size (0xe30c0406) in partition '0x50384148 ["HA8P"]':
!!      /dev/sde1
!! wwt: ERROR #14 [INVALID WII PARTITION] in wd_load_part() @ src/libwbfs/wiidi

It's a wanted error message because the disc is incomplete. For SSBB it is normal, because the dumpers forget to copy the partitions of the second partition table. Prior wit crashed.

 

[b0ring]

Sorry, I was referring to the way the program terminated (Error #39).

I ran it again today with more success. The Error #14s from before became Error #16s this time. However, the program still terminated like this:

  !H1-DIF 48.0/69  DATA    R3MP01 Metroid Prime: Trilogy
  !H2-DIF 48.0/69  DATA    R3MP01 Metroid Prime: Trilogy
  !H1-DIF 48.0/69  DATA    R3MP01 Metroid Prime: Trilogy
  !H2-DIF 48.0/69  DATA    R3MP01 Metroid Prime: Trilogy
  !H1-DIF 48.0/69  DATA    R3MP01 Metroid Prime: Trilogy
  !H2-DIF 48.0/69  DATA    R3MP01 Metroid Prime: Trilogy
  !H1-DIF 48.0/69  DATA    R3MP01 Metroid Prime: Trilogy
  !H2-DIF 48.0/69  DATA    R3MP01 Metroid Prime: Trilogy
  !H3-ERR 48.0/69  DATA    R3MP01 Metroid Prime: Trilogy
  !H3-ERR 48.0/69  DATA    R3MP01 Metroid Prime: Trilogy
  !ABORT  48.0/69  DATA    R3MP01 Metroid Prime: Trilogy
- REMOVE disc [R3MP01] Metroid Prime: Trilogy
!! wwt: ERROR #39 [INTERNAL ERROR] in ExistsWDisc() @ src/wbfs-interface.c#4115
!!      Internal error

Can you offer any advice on what's happening to cause this sudden abort? I'm sorry for asking so many questions, but without knowing what I'm supposed to see I've no idea what I'm supposed to do...short of formatting the drive and starting again with the few games that successfully backed up.

 

[Wiimm]

Internal errors are really bad. here it is:

    if ( !w || !w->wbfs || !id6 || strlen(id6) != 6 )
    return ERROR0(ERR_INTERNAL,0);

'w' is the wdisc pointer. I assume, that the id6 was wrong (shorter than 6 chars because a NULL value.
This was to careful and I changed it now to strlen(id6) > 6

 

[PsyBlade]

before formating you could try to convert it regardless of errors
even in worst case you can simply format it and be off no worse

 

[b0ring]

Just thought I'd post an update. I ran 'verify' once more, it gave some errors during the check but this time it at least completed without aborting due to error. I went ahead an ran wbfs2fat anyway, and it all went smoothly. The USB Loader reads the games, and the four problem ones simply kick me back to the Wii menu.

So thanks both for the tools and support.

 

[Nickkk]

I'm trying to publish Witgui on the App Store. The only thing is that Apple requires wit to be signed with my Apple Developer Certificate ... Would I be allowed to build wit by myself and sign it? If the answer is yes, where can I download the source code?