Hacking wwt+wit: Wiimms WBFS+ISO Tools

[Wiimm]

With the current wit I see two possibilities:

1.) pipes: your_gecko_connector | wit ...
The problem is the not good pipe support in the current wit and that pipes are not seekable.

2.) sockets: your_gecko_connector ... & wit ... /path/to/socket; killproc your_gecko_connector
The '&' is here the shell sign for background.
The gecko connector creates a socket and deliver open/read/close operations to the wii.
I have never programmed sockets so i don't know if it really works.

 

[giantpune]

I have never programmed sockets so i don't know if it really works.

same here

 

[Wiimm]

3.) compile wit for the wii.
wit uses only standard POSIX functions (I believe it!)
In the next weeks I have less time to support this (project business is calling me)

 

[Wiimm]

WIT: Wiimms ISO Tools v1.15a - 2010-08-15
A command line ISO+WBFS tool set for various os.

WIT (Wiimms ISO Tools) is a set of command line tools to manipulate Wii ISO images and WBFS containers. The two main tools are called wit (Wiimms ISO Tool) and wwt (Wiimms WBFS Tool, the WBFS manager).

Feature overview:

• WIT (Wiimms ISO Toolset) is a set of command line tools.
• wit (Wiimms ISO tool) is the main Wii ISO manipulation tool.
• wwt (Wiimms WBFS tool) is a complete WBFS manager including FS checking and repairing, time stamps and multi WBFS support.
• wdf-cat and wdf-dump are WDF and CISO support tools.
• WIT supports 5 Wii disc file types (ISO, CISO, WDF, WBFS and FST) and can read from pipes (indirect RAR/ZIP support).
• wit can verify and compare (diff) Wii ISO images.
• wit can extract and compose Wii ISO images.
• wit can patch Wii ISO images while copying.
• wit can mix partitions from different sources into a new Wii ISO images.
• wit and wwt support many diagnostic commands.
• Advanced shell support.

Visit http://wit.wiimm.de/features.html for more details.

Download of the current version:

• wit-v1.15a-r1554-i386.tar.gz : Linux/i386 version
• wit-v1.15a-r1554-x86_64.tar.gz : Linux/x86_64 version
• wit-v1.15a-r1554-mac.tar.gz : Mac OS universal binaries (ppc, i386 and x86-64)
• wit-v1.15a-r1554-cygwin.zip : Cygwin (Windows) version
• Other releases are available from http://wit.wiimm.de/download.html.

There are also some GUI available:

• QtWitGui, manipulate and convert single ISO images.
  This QT based GUI focuses on manipulating single ISO images.
  .
• WiiJManager Multiplatform, a WBFS manager GUI
  A java GUI that acts like a WBFS manager. Converting of ISO images is also possible. (The project seems to be dead)

Change log:

wit v1.15a r1554 - 2010-08-15

- New "wit MIX" qualifer: HEADER: Copy the disc header (first 256 bytes of
ÂÂ disc) of the related source image to the destination image. Patching with
ÂÂ the options --id and --name is possible.
- New "wit MIX" qualifer: REGION: Copy the region infor (32 Bytes at offset
ÂÂ 0x4e000) of the related source image to the destination image. Patching
ÂÂ with the option --region is possible.
- Bug fix: "wit mix --name discname" didn't work.
- Bug fix for "wit mix --overlay".
- Writing a WBFS file is now part of the file layer and libwbfs is only used
ÂÂ as helper. Now every command, and also "wit MIX", is able to write to a
ÂÂ WBFS file like to any other formats without special code.
- "INSTALLER" is now an alternative name for the "UPDATE" partition, like
ÂÂ "GAME" for "DATA" since the epoch.
- To manage allowed and permitted options, 64 bit integer numbers had been
ÂÂ used. Now arrays will be used, because wwt have already reached the limit
ÂÂ of 64 command specific options. Hope that all run as before.
- Korean disc support implemented. (I can't test this by myself)
- New option: --common-key index: Change the common key index of all partitions
ÂÂ and fake sign the tickets.
- I have implemented a new "ID selection interface". It works with positive
ÂÂ ('+') and negative ('-' or '/') rules. A rule consist up to 6 characters,
ÂÂ case is ignored. Points ('.') are wildcards for a single character and mult
ÂÂ ('*') and plus ('+') for many characters. Each source ID6 is compared to
ÂÂ each rule of the list until a rules matches the ID. If it is a positive rule,
ÂÂ the ID is inserted into the workig list.
- The syntax of the commands "wit ID6" and "wwt ID6" have changed to use the
ÂÂ new "ID selection interface" (see above). More commands will follow.
- The commands 'wit ILIST/ILIST-L/ILIST-LL' are renamed to 'wit FILES/FILES-L
ÂÂ /FILES-LL'. The old command names are still allowed.
- The new command "wit FDIFF" is a shortcut for "wit DIFF --files +".

Known bugs:

- There are new problems while reading from non seekable sources (e.g. pipes).
ÂÂ The reason is the complete new and more intelligent wiidisc lib which makes
ÂÂ more lookups. I have to insert some code especially for pipes. My plans
ÂÂ that copying multiple partition discs works too.
ÂÂ ==> The best is to use a version 1.01c for pipes.

Missing docu:

- wit commands ILIST + EXTRACT + EDIT => see built in help
- Option --show => see built in help

The source is published under the license of GPL2. Checkout the sources directly from the SVN repository or browse the source. Visit http://wit.wiimm.de/ for more information.

 

[ChokeD]

I need some help. I haven't really had a use for this until now and I want to do away with my wbfs drive and go fat32. Trouble is I have NO idea where to begin, sorta. I was reading the documentation on wwt and I seen the EXTRACT command but I would like to extract the .wbfs as sneek format files and store them on another drive. Once there I will figure out some other commands on my own, I would like to verify them as I think 2 or 3 were a bad dump from a scratched disk I have but I'm not sure so I will be testing some stuff on my own I'm just sort of stuck. I'm not real command line smart but I'm not completely dumb to it. I learn fast. I just need shoved.

Could someone please help me with the proper commands needed to say,

extract multiple .wbfs from a wbfs partition on drive F:\ to sneek format files on ntfs\fat32 drive Z:\

If this is possible ?? Thank you very much for any help.

EDIT:
If nothing else I would like to extract the .wbfs as .wbfs from 1 wbfs drive to a fat32 drive.

EDIT2:
After reading for 5 or 6 hrs the wwt.txt and trying a bunch of different combos,

I thought I had it, I was wrong.

I don't normally give up this easy, but there is way to many commands and possible combos for me to waste my time finding the right combination of commands to get 1 simple process that I need done. The tools that are available here are of great value and highly respected by me but I just don''t have a long term use for them. Well, not long enough to spend days trying to figure out how to perform 1 task or memorize commands for every function.

I'll just pull them off with WBFS Manager as ISO's and convert them with NUSAD.

Thanks anyway,

ChokeD

EDIT 3: Well scratch that, My drive is jacked up now and isn't being read by WBFS Manager or wwt. wwt can't find it. This is so awesome. I love it. I should have never even tried to mess with wwt. I should have just pulled them off of there like I was going to and been done with it. Glad I got the retail discs for everything. Now I just have to dump them all over, at the most a week or 2 down the drain. No sense in me trying to REPAIR with wit cause that's how it got jacked to begin with. This is the 2nd time wbfs partitioning has screwed me. The first time it was USB Loader GX that code dumped somehow and corrupted the drive so that it wasn't readable with no way to recover. This will not happen again, I will never use wbfs ever again, ever. There just isn't enough gui support for a windows dummy like me I guess.

 

[giantpune]

thats the exact reason there is the "--test" option.

 

[ChokeD]

I did use --test command and have since recovered my drive not using wwt just by resetting my cpu and powering down the HD. wwt still isn't finding my drive though. WBFS Manager is though but now the manager won't do multiple game extraction or it draws errors.

Well, after messing with wwt for some reason probably mine the drive changed addresses and now I can find it with wwt. It was at /dev/sdj2 and now for some reason it's found at /dev/sdi2 . I am still stupid as to the parameters I need to just dump the files to another drive or repair the drive. The drive is messed up. I'll keep working with it. I have other drives so it's not like I'm in a hurry to get this one back up and running. It pisses me off so bad I want to figure it out now.

 

[markehmus]

thanks for the aid, pune. great app...love the QtWitGui part for easy use. i havent done any multi-transfers....but rebuilding SMNE(customs) and adding roms to that emulator disk was sure made simple by this.

i use versions of sneek on a regular basis , and now i can compile my sneek game data files back into an iso, something i found too dificult to do with partition builder,wiiscrubber or trucha signor.

im excited to see updates of this and thank you MEN for working on this great useful app that prolongs game life by alowing custom file replacement/addition/removal/transfer and basic abilitie to manipulate easily.

 

[ChokeD]

Well, I knew my drive was messed up, it's junk now. Don't know exactly what happen but I was using wwt on it to --repair the file system and it ran some tests and said 13 out of the 32 games I have were invalid. Yes I ran it with --test. I shut the pc and drive down to get some other stuff done around the house and around 3 hours later I turned the pc on then the drive and now the drive sits there and can not be found by windows or wbfs manager. It was a maxtor 1tb raid0 2x 500 gb exterior hdd. Toast. On to better things now. I opened up the enclosure hooked the drives up to the pc and they are good. Either this 1 year old drive was ready to go or the interface of the raid didn't like something that was being done. I'm not going to exclude anything. I just don't care really what it was. I got 2 500gb sata hd's now and I'm moving on.

Nice tools, much respect to ya Wiimm and you giantpune for QtWitGui but I'm done with this command line stuff for a while.

 

[Wiimm]

@ChokeD
I'm back from holiday and I have read you messages only fast. Are there any open questions?

 

[ChokeD]

@ChokeD
I'm back from holiday and I have read you messages only fast. Are there any open questions?

Thanks anyway, but it's to late to help me, my drive is junk now. I got to go by another exterior HD. The only possible thing I could ask or say would be do you think it is at all possible that someone could render an HD totally useless with your software when running an exterior HD that uses raid0 or raid1 software on the pcb of said drive enclosure ?? As stated before, I had a MAXTOR exterior HD enclosure with 2x 500gb drives running what is to be believed to be a raid0 array and I ran a --test with RECOVER with no error messages and it returned a butt-load of invalid discs so to fix I ran it without --test and my drive pooped. It wouldn't turn on or be seen by Windows or wwt or wbfs manager. Without the --test function I had only ran 3 unprotected commands on it, I used FIND, ANALYZE, RECOVER, with different options. The messed up thing about the whole situation is that when I started to --test with wwt the drive was found at /dev/sdj2 and after the first time of using RECOVER somehow my drives address changed to /dev/sdi2. I didn't have any other sticks or usb's or anything plugged of that nature that would have changed the address of the wbfs partition. The only thing I can think of is that wwt may have seen only 1 of the drives in the array and not the other, then seen something as invalid on that drive, when I attempted to RECOVER it, it fuxered the factory raid array software for the enclosure. Seriously, I ran wwt with a clear head and had done plenty of reading and research in the wwt.txt provided way before I started messing with the drive. Like I said before I am not very friendly with command line stuff but I have dealt with a lot of it before and I don't honestly believe that it was me that messed up here but I am willing to take the heat if it was. It may be nothing but I thought since you were asking I would reiterate what had happened.

Again, much respect to you and your tools, I just happen to be one of the less fortunate I guess, one way or the other.

 

[WiiPower]

Do these tools to allow to "anti-scrub" .isos, to pad the data to the outer edge of the disc to increase the reading speed on discs? I don't need it, but people who still use discs and have games that 1. are small and 2. would greatly benefit from faster reads(Metroid Prime 1/2/3, not MPT) could like it.

 

[Wiimm]

Do these tools to allow to "anti-scrub" .isos, to pad the data to the outer edge of the disc to increase the reading speed on discs? I don't need it, but people who still use discs and have games that 1. are small and 2. would greatly benefit from faster reads(Metroid Prime 1/2/3, not MPT) could like it.

As I learned from the others tools scrubbing means not "trimming and fake signing" but "ignoring unneded sectors". The WIT tools ignore that sectors too. For ISO files this means: leave areas with zeros that are candidate for the sparse effect. WDF, CISO and WBFS ignores such blocks completely, if they are large enough to fill a management block (WDF 20 bytes, CISO+WBFS: multi MiB).

Conclusion: The data of scrubbed disc is still at the same position as unscrubbed (at the end of the disc in the outer area of the physical disc for standard discs).



@ChokeD

First: My software is like every software not perfect. But I don't think, that raids are problematic. But it is problematic to access only one drive of the raid and not the whole raid. Under linux the nase partitions of the raid are still visible, but also the raid drives, mostly called /dev/md#. My tools don't know anything about raids, but working with /dev/md# should work (I would bet on that).

You have had a RAID0 (striping). I don't know your chunk size (I assume 1 MiB for my example). That means, that the first 1 MiB of the raid is stored on device #21 the next MiB on device #2, then #1, #2, ....

1 MiB is more then enough for the to store WBFS header and the inodes (not really inodes, but like) of about 107 discs. The WBFS layout have no other checksums or validation data. So all WBFS tools (not only WIT) are thinking that the first raid device is a true WBFS drive with corrupted data. And more: changing data on only one base device with any tool may corrupt the raid.

This all is only a problem for software raids. With hardware drives the raid hardware will hide the base partitions. Perhaps should scan /proc/mdstat for raid info to warn the the user. But this will only work for linux.

I know it's is time to write a repairing how to!
But I'm already working on the next extension .... shit docu steals time .....

 

[ChokeD]

@ChokeD
So all WBFS tools (not only WIT) are thinking that the first raid device is a true WBFS drive with corrupted data. And more: changing data on only one base device with any tool may corrupt the raid.

This all is only a problem for software raids. With hardware drives the raid hardware will hide the base partitions. Perhaps should scan /proc/mdstat for raid info to warn the the user. But this will only work for linux.

I know it's is time to write a repairing how to!
But I'm already working on the next extension .... shit docu steals time .....

Well, it seems to me that this is indeed what has happened. I'm not on Linux, The machine I was on at the time was XP.

About the how to, it would be nice to have but again to late for me. I stripped the HDD's out of the enclosure and put them both inside the kids cpu.

They needed the space more than I do. They work in Sony Vegas and do the YouTube gaming crap. I'll bet those docs do steal some serious time man but what goods a program or app if know one knows what options are available.

You do good work Wiimm, keep it up.

@pune

when, if ever, are you going to update qtwitgui to use an updated version of wit. ?? It would be nice if we could just choose a folder the converted games were in and view them after they had been converted. Forgive me if this option is there but I couldn't find it. I was just wondering cause I no you do good work and I'd trust anything you put out there.

Thanks for your time and effort both of you.

 

[giantpune]

qtwitgui uses wit r1554
http://code.google.com/p/qtwitgui/source/b.../mainwindow.cpp line 45

and what do you mean by this?

choose a folder the converted games were in and view them after they had been converted

what would this special function do differently than windows explorer/finder/nautilus? i dont see what you mean.

 

[ChokeD]

qtwitgui uses wit r1554
http://code.google.com/p/qtwitgui/source/b.../mainwindow.cpp line 45
I had the downloads page indexed, I didn't realize you were up to 112. Do I need anything special to compile or will the same devkit I got setup for sneek be good ??


and what do you mean by this?

choose a folder the converted games were in and view them after they had been converted

what would this special function do differently than windows explorer/finder/nautilus? i dont see what you mean.

Like the picture below it would just offer useful info and tell you what you have in any given folder. Instead of trying to figure out what /R8PP01 is it would offer up the info on the title. I wouldn't ask, but I can't use Wiijmanager anymore for some reason it don't want to work for me, but qtwitgui works great.

http://lh3.ggpht.com/_Biqcy9zWImI/S55gnWvL...iJManager04.jpg

 

[giantpune]

qtwitgui treats a folder as extracted files like a game that is all mashed together into 1 file. just drag the folder onto the gui or open it with ctrl + o or whatever. it will tell you what game it is. and there is no screen that says anything about more than 1 game at a time. but you dont need it.

there is no reason in the world you need to give your games names like "RSPE01". if you cant tell that that is wii sports, then rename the file to "RSPE01_wii sports". simple as that.

 

[ChokeD]

Right, I figured your retort would go something like that, telling me what I do and don't need. For those of us that use NUSAD with DiscEX for bulk conversion because your app doesn't and there are no other apps that do bulk conversion, we are left with no way to identify what game they are exactly without testing each and every one individually unless you know an easier way other than loading them all up 1 by 1 in qtwitgui. Is there an app with a similar function as the picture above that handles multiple games in a selective folder ??


Respectfully,
ChokeD

 

[giantpune]

just use wit/wwt LIST to list all the names of the games. if you need to see all the names of each game on a regular basis for some reason, then you should simply rename your games.

 

[W hat]

I'm guessing Wiimm knows the answer to this.

What is the average size of a Wii update partition? Is it significantly smaller / bigger on a NTSC-U disc, compared to a NTSC-J or PAL disc?