Toggle sidebar

Hacking wwt+wit: Wiimms WBFS+ISO Tools

  • Thread starter Thread starter Wiimm
  • Start date Start date
  • Views Views 697,142
  • Replies Replies 1,984
  • Likes Likes 12
Wiimm said:
All is possible, it's always a point of time and fun.
Are there more --sections wishes?
Click to expand...

Now that you ask, it would be nice to have --sections in wwt format and wwt recover. Thank you in advance!

EDIT: wit/wwt remove already supports --sections, but the output doesn't seem to be formatted correctly.
 
I feel like a total dweeb but I having a problem with the fundamental step of getting my Linux distros to mount the Wii disk. I have tried in Arch and Ubuntu based distributions. Searching has not helped me. I did find others with the same problem like http://ubuntuforums.org/showthread.php?t=1974567.

Here is what I ran and the results:
wit LIST-LL /cdrom --titles /usr/local/share/wit/titles.txt --section

[summary]
total-discs=0
total-size=0
wit LIST-LL /dev/sr0 --titles /usr/local/share/wit/titles.txt --section
!! wit: ERROR #25 [CAN'T OPEN FILE] in SetupReadSF() @ src/lib-sf.c#423
!! Can't open file: /dev/sr0

program=wit
command=LIST-LL
code=25
name=CAN'T OPEN FILE
text=Can't open file
Is there something I am doing wrong or another command I need to run?
Thanks for the help.
 
What kind of DVD you put into the drive? A Wii DVD or a DVD with a standard filesystem (ext3 for example) which contains image(s)?

Some reasons for "!! Can't open file: /dev/sr0"
* User have no read access to the device.
* No DVD inserted
* DVD is unreadable.
 
Wiimm said:
What kind of DVD you put into the drive? A Wii DVD or a DVD with a standard filesystem (ext3 for example) which contains image(s)?

Some reasons for "!! Can't open file: /dev/sr0"
* User have no read access to the device.
* No DVD inserted
* DVD is unreadable.
Click to expand...

I am using a Wii game. I am trying to back it up. I tried running as root as well. I tried 3 different games on 2 different Linux installations. I have used the DVD drive for backing up movies so I know the drive works (playing a movie as I write this).
 
Do you know: Original Wii DVDs can't be read by the most DVD drives. Exception are only some old LG drives. So the DVD is for technical reasons unreadable.
 
Some strange thing happened when trying to repair my WBFS partition. The check before the repair seems to detect one error:


Code: 
[version]
prog=wwt
name=Wiimms WBFS Tool
version=2.25a
beta=0
revision=4825
system=mac
endian=1234 little
author=Dirk Clemens
date=2014-01-03
url=http://wit.wiimm.de/wwt
 
 
 
[check]
wbfs-file=/dev/rdisk1
n-slots=215
n-disc=0
column-header=          slot _id6__ index wbfs-block count
 
 
[check:summary]
err-blocks-marked-free=0
err-blocks-marked-used=1
err-blocks-multi-usage=0
err-blocks-invalid-reference=0
err-discs-without-blocks=0
total-errors=1
total-invalid-discs=0
REPAIR /dev/rdisk1
* Store fixed 'free blocks table' (512 bytes).
 
 
 
 
[error]
program=wwt
command=REPAIR
code=19
name=INVALID WBFS
text=Invalid WBFS
But when running another check afterwards (without repair), everything seems to be ok, and no errors are shown.
 
I searched the thread but I found no mention of this. I attempted to extract Metroid Prime 1 with wit and it seemed to work, however as I was working with the files I started to notice something strange: my tools weren't working, tools I knew for a fact worked. So I pulled out thakis' old gcmdump and found that wit was actually doing some odd data corruption on the pak files.

The headers were all correct, but the data was corrupted beyond repair, but that's not the strangest, the strangest part is that the data fit perfectly with the packs file descriptors.

I did a byte comparison in 010 and found:
wit_corruption.png

wit is A, gcmdump is B
 
Because of a bug after implementing the auto split detection in v2.25a, reading source images failed sometimes (if using file descriptor 0) . So please use an old version. I'll will update the tool soon.
 
More about the bug:
I have analysed it now. I have tested a file descriptor in a wrong way. A file descriptor is closed, if it is -1. But I tested it against 0 and 0 is stdin. And now, if stdin was used for parameter input (e.g. --include -), it was closed, the descriptor is free and used for reading an image. And at this point my tools fails.

Nickkk:
Since some versions, the free blocks table is always and automatically repaired on opening a WBFS. More exact: The free blocks table (1 bit for one block) is completely ignored and internal allocation table is created by analysing the block assignment. It takes only a part of a second. So the following check has no chance to see this error. On closing, a new table is created and written. I'll try to change this.

Antidote:
Ok, the files are different (and only at some random positions equal). But I don't understand the problem, because its not clear, what you have done.
 
Just extracted the file, and attempted to use my tools, I compared two Metroid1.pak files, one extracted from the GCM with your tool and the other with Thakis' tool available here: http://amnoid.de/gc/szstools.zip
Yours is incorrectly, either reading data, or is writing bad chunks. The data is bad randomly because that's how it was written by your tool. It has nothing to do with what I did.

just: $ wit x game.iso game

The screenshot is merely to demonstrate what I'm talking about, it's not meant to be diagnostic.
 
wiimms-tools.png


WIT: Wiimms ISO Tools v2.26a - 2014-01-22
A command line ISO+WBFS tool set for various os.

WIT (Wiimms ISO Tools) is a set of command line tools to manipulate Wii ISO images and WBFS containers. The two main tools are called wit (Wiimms ISO Tool) and wwt (Wiimms WBFS Tool, the WBFS manager).

Feature overview:
Visit http://wit.wiimm.de/features.html for more details.

Download of the current version:

There are also some GUI available:

Highlights of this release:
  • The highlight is the new, but experimental, GCZ support. See the change log for details.
    Code: 
    # wit info

File formats:

  name  description                 option  extensions  attributes
 ------------------------------------------------------------------------------------------
  ISO   Plain file                  --iso   .iso   -    read create extend -      loader
  WDF   Wii Disc Format             --wdf   .wdf   -    read create extend -      -
  CISO  Compact ISO                 --ciso  .ciso .wbi  read create modify nosize loader
  WBFS  Wii Backup File System      --wbfs  .wbfs  -    read create extend nosize loader
  WIA   Compressed Wii ISO Archive  --wia   .wia   -    read create -      compr  -
  GCZ   Dolphins GameCube Zip       --gcz   .gcz   -    read create -      compr  -
  FST   Extracted File System       --fst    -     -    read create -      fst    destedit
    .

Change log:

Code: 
wit v2.26a r4863 - 2014-01-22

 - Bug fix: Because of a bug after implementing the auto split detection,
   reading source images failed, if using stdin for parameters.
 - Support for Dolphins file format GCZ (GameCube Zip):
    - All commands detect and accept GCZ files as input file.
    - Creating of GCZ files is also supported, but EXPERIMENTAL until final
      tests have been done.
    - New option --gcz force GCZ output.
    - Patching of GCZ files is not possible, because the GCZ file structure
      doesn't allow modifications (size of compressed data must not change).
    - Composing an image to a GCZ file is not possible, because it needs
      patching checksums and header after writing the complete image.
   The GCZ support is very new, so please use it only with backups of your
   images and don't be anger, if it destroy something.

The source is published under the license of GPL2. Checkout the sources directly from the SVN repository or browse the source. Visit http://wit.wiimm.de/ for more information.
 
  • Like
Reactions: kimotori
I have tested the gcz conversion with 2 games. Dolphin has no problem to execute the images created by wit. Dolphin created the same images, only the compressed data differs, but if decompressing it is the same.

Here are some statistics, all measured on the same Win7 system:
Code: 
----------------------------------------
                      Animal    Legend
                    Crossing    Zelda
                      time      time
----------------------------------------
Dolphin v4, iso->gcz    79s      98s
wit v2.26a, iso->gcz    29s      54s 
wit v2.27a, iso->gcz    21s      43s
----------------------------------------

wit v2.26a is the current released version.

wit v2.27a, the next release, will have an optimization:
If creating a GCZ image, a blockwise z-compression is tried. If the compressed data is larger than 98.5%, the uncompressed data is stored. New is, that encrypted blocks are stored directly as uncompressed data, because encrypted are very bad compression candidates and the 98.5% test fails all the time. This makes GCZ creation faster.
 
Antidote
It's took a little bit more time, because I don't own the game and a friend helped finding and analyzing this bug. Thanx for pointing it.

BUG REPORT
==========

Bug fix: If extracting a GameCube image, files larger than 4 MiB are damaged behind this 4 MiB offset.
Click to expand...

wit uses an internal 4 MiB io-buffer for different things, and also for copying files. Wii images use 32-bit offsets, that must be multiplied by 4 to allow images >4 GiB; but GC not.

In the extract function I had forgotten the special GC case, so the offset-calculation of the next block were wrong. The follwing patch fix it:
Code: 
-           off4 += read_size >> 2;
+           if (part->part->is_gc)
+               off4 += read_size;
+           else
+               off4 += read_size >> 2;

The update will be released in the next days.
 
Wiimm said:
Antidote
It's took a little bit more time, because I don't own the game and a friend helped finding and analyzing this bug. Thanx for pointing it.

BUG REPORT
==========



wit uses an internal 4 MiB io-buffer for different things, and also for copying files. Wii images use 32-bit offsets, that must be multiplied by 4 to allow images >4 GiB; but GC not.

In the extract function I had forgotten the special GC case, so the offset-calculation of the next block were wrong. The follwing patch fix it:
Code: 
-          off4 += read_size >> 2;
+          if (part->part->is_gc)
+              off4 += read_size;
+          else
+              off4 += read_size >> 2;

The update will be released in the next days.
Click to expand...

have anticipated..
 
wiimms-tools.png


WIT: Wiimms ISO Tools v2.27a - 2014-01-31
A command line ISO+WBFS tool set for various os.

WIT (Wiimms ISO Tools) is a set of command line tools to manipulate Wii ISO images and WBFS containers. The two main tools are called wit (Wiimms ISO Tool) and wwt (Wiimms WBFS Tool, the WBFS manager).

Feature overview:
Visit http://wit.wiimm.de/features.html for more details.

Download of the current version:

There are also some GUI available:

Change log:

Code: 
wit v2.27a r4908 - 2014-01-31

 - Bug fix: If extracting a GameCube image, files larger than 4 MiB are
   damaged at beginning of this 4 MiB offset.

 - New feature: If creating a GCZ image, a blockwise z-compression is tried.
   If the compressed data is larger than 98.5%, the uncompressed data is
   stored. New is, that encrypted blocks are stored directly as uncompressed
   data, because encrypted are very bad compression candidates and the 98.5%
   test fails all the time. This makes GCZ creation faster. The new option
   --gcz-zip disables this optimization for encrypted data.

 - New option: --gcz-block=size: Define the block size for GCZ creation. The
   default size is 16K (also Dolphins default).

 - Tool 'wdf' supports info dumps of GCZ files to verify the GCZ creation.

 - New command for wit+wwt: FEATURES: Print a list of supported features. The
   output is machine readable. Scripts may use "wit features -qq" and check
   the exit status.

The source is published under the license of GPL2. Checkout the sources directly from the SVN repository or browse the source. Visit http://wit.wiimm.de/ for more information.
 
wiimms-tools.png


WIT: Wiimms ISO Tools v2.28a - 2014-03-01
A command line ISO+WBFS tool set for various os.

WIT (Wiimms ISO Tools) is a set of command line tools to manipulate Wii ISO images and WBFS containers. The two main tools are called wit (Wiimms ISO Tool) and wwt (Wiimms WBFS Tool, the WBFS manager).

Feature overview:
Visit http://wit.wiimm.de/features.html for more details.

Download of the current version:

There are also some GUI available:


Change log:

Code: 
wit v2.28a r4980 - 2014-03-01

 - Full WDF version 2 support:
    - WDFv2 files are a little bit smaller and support alignment.
    - Parts of the WDF library have been rewritten to support WDFv2 and
      alignment. A side effect is a more compact code and a better chunk
      management if modifiying WDF files.
    - Option --wdf forces WDF output, the version is definied automatically.
    - Option --wdf1 forces WDFv1 output.
    - Option --wdf2 forces WDFv2 output.
    - Option --align-wdf defines an alignment between 1 and 1GiB (power of 2)
      and optional the minimal hole size before creating a new chunk.
    - 'wit EDIT' supports --wdf1 and --wdf2 to allow version conversions.
   For technical details about WDF see: http://wit.wiimm.de/WDF
 - Support of split files of CleanRip: If reading a plain ISO file named
   '*.part0', the other parts are detected as continuation files.
 - Windows only: Cygwin update to v1.7.28 2014-02-09.

The source is published under the license of GPL2. Checkout the sources directly from the SVN repository or browse the source. Visit http://wit.wiimm.de/ for more information.
 
  • Like
Reactions: kimotori and GamerSince83
.wia is great because i got kirby epic yarn in 900MB when the wbfs is in 3.6GB thats really bueno :) the compression its so good
 
Thank you for keeping developing wit. It is a great tool, and will soon even be in debian stable.
I've decided to archive my wii dumps with the wia format and I've repeated your testing for one of my dumps which seem to give the same conclusion:

raw: 4699979776
wbfs: 1738539008
wbfs + xz -9: 1730461696
wia none: 1686332800
wia best: 1321414732
wia none+ rar -m5: 1431695454
wia none + xz -9: 1264884732

Since I'm not aspiring to backup the disks for preservation for humanity, but rater a fully working copy on minimal space I am just keeping the DATA partition which should have no practical disadvantage.

Two questions:
1. will wit VERIFY of the DATA partition check the nintendo signature of the internal checksums? So in other words if the partition passes the VERIFY check there is no possibility of the partition being tampered with without someone knowing the nintendo private key, which no one does except nintendo. Then if yes, except for preservation of unscrubbed iso files for the sake of humanity, there is no need for storing a checksum database for the DATA partitions since no unofficial games or incorrect dumps could pass the check.

2. How come an external compressor achieves the best result since xz implements the same lzma algorithm used in wit?

Thanks
 
  • Like
Reactions: kimotori

Site & Scene News

Go to forum More news

Popular threads in this forum

GTA VC WII (ReVC Wii) Wii Port

Homebrew  GTA III Wii Port

miniCDi - Philips CD-i emulator

Emulation Gaming  Wii Lightgun Emulation Compatibility Guide

Hacking  [RELEASE] vc64_240p — real 240p for N64 Virtual Console on the Wii (no deflicker blur)

Tutorial  How to play Call Of Duty World At War online on the Wii in 2026

desperate kids lost wii games from loader drive

Gaming Homebrew  Goldeneye Wii has it's multiplayer servers BACK ONLINE!!