Hacking wwt+wit: Wiimms WBFS+ISO Tools

[mousex]

well in that case, the 64Gb RunCore SSD i use for swap and my OCZ SSD i use for / will speed up things abit

now if only i could afford another RunCore Pro to use with Wii

lol

Yeah, please test the write and read spped of your SSD on the Wii. I wasn't in the mood to FAT-Format mine yet as it's at HFS+ which is (in my opinion) still superior for OS X and the X360.

EDIT:
@wiimm: The help screen of wit (maybe other tools too) doens't have a hint how to set the split size, I found it only in the documentation. There is a --size parameter which confused me before but the right one to use for splitting was --split-size
And when talking about 4GB-32KB we're talking about 4294934528B, right?

 

[drh]

looks like you have the old original version of WBFS

This is not the the delete bug, is is an other bug that i found last week. The old libwbfs does not use all sectors. See "Inside WBFS" for more info.

Well MarioMadness is right, I did format the drive using the old USBLoader, I was not aware of any differences, which is quite annoying now because I am a few hours into copying ISO's to the new drive :-(

I tried to format the drive with wwt but it kept throwing up errors, so I thought it would be easier to just format the drive with the loader as I know that works simply and quickly.

iMac-1:wiimms drh$ ./wwt format /dev/disk4s1 --force
*****ÂÂwwt: Wiimms WBFS Tool v0.16b r431 mac - Dirk Clemens - 2009-11-18ÂÂ*****
FORMAT BLOCK DEVICE /dev/disk4s1 [sector size 512]
Segmentation fault
iMac-1:wiimms drh$ ./wwt format /dev/disk4s1 --force
*****ÂÂwwt: Wiimms WBFS Tool v0.16b r431 mac - Dirk Clemens - 2009-11-18ÂÂ*****
!! wwt: ERROR #18 [SEMANTIC ERROR] in cmd_format() @ wwt.c#1064
!!ÂÂÂÂÂÂOption --size needed to create a new WBFS file: /dev/disk4s1
** 0 files formatted.

Posts merged

Wiimm

@drh, If you send the signal USR2 to the process it will increase the verbose level. If level 2 is reached a progress counter is enabled.

@Wiimm, This could be written in chinese and I would understand it just as well :-)

 

[mousex]

@Wiimm, This could be written in chinese and I would understand it just as well :-)
Either you use the simple way: http://www.macupdate.com/info.php/id/26535 or in the terminal

sudo killall -USR2

will send USR2 to all processes. Or CODEkill -USR2 PID

(I'm not sure if it's -USR2 or just USR2) will send it to the process with the PID you get from ps. This is also helpfull to get the current status from dd because dd has about zero output if you don't ask it this way

 

[Wiimm]

@drv:
I don't know whats happen. But it seems that the block device is removed. But this should only possible for plain files, not for devices.

I can help you to repair this. But I need some infos:
1.) discover the id structure of the mac (file is a command): file /dev/disk*
2.) mknod --help


SIGNALS:
Linux knows killproc:

# send all 'wwt' processes signal USR2
killproc -USR2 wwt

 

[drh]

SIGNALS:
Linux knows killproc:

# send all 'wwt' processes signal USR2
killproc -USR2 wwt

Thanks Wiimm, I will keep this in mind for future reference (in case I forget to use the --progress switch)

I had aborted the process and restarted with --progress and it looks like I am getting 4.1Mib/sec.

I will post results of the other request when the last few files have copied over.

 

[drh]

@Wiimm

mknod --help give invalid option

iMac-1esktop drh$ mknod
usage: mknod [-F format] name [b | c] major minor
mknod [-F format] name [b | c] major unit subunit
mknod name [b | c] number
mknod name w

iMac-1esktop davehealey$ file /dev/disk*
/dev/disk0: block special
/dev/disk0s1: block special
/dev/disk0s2: block special
/dev/disk1: block special
/dev/disk1s1: block special
/dev/disk1s2: block special
/dev/disk2: block special
/dev/disk2s1: block special
/dev/disk2s2: block special
/dev/disk3: block special
/dev/disk3s1: block special
/dev/disk4: block special
/dev/disk4s1: block special
/dev/disk5: block special
/dev/disk5s0: block special
/dev/disk5s0s1: block special
/dev/disk5s0s2: block special


iMac-1esktop drh$ file /dev/disk4s1
/dev/disk4s1: block special

 

[Wiimm]

Sorry, not "file" but "ls -l /dev/disk*" is needed to see the special file scheme.

And /dev/disk4s1 exists. But the error message above says that /dev/disk4s1 not exists. Is the USB drive ok?

 

[drh]

I had aborted the process and restarted with --progress and it looks like I am getting 4.1Mib/sec.

After copying all the ISO's from the 320Gb to the 500Gb yesterday and then reading about the block size issues etc, I have kicked off wwt to pull off all the ISO's from the 500Gb drive in preparation to put back on the drive when the tools have been re-released. I have moved the drive to my new iMac 27" and I am getting 7.2Mib/sec, although I guess the 4.2Mib/sec I was getting yesterday was probably due to the files going from 1 usb to another.

 

[mousex]

As I understood the block size issue only affects .wbfs files and not WBFS formatted drives (because a smaller block size uses more memory but wbfs drives already use big blokc sizes).
by the way, if you want to format and it doesn't work. Did you try running as su or using rdisk4s1 instead of disk4s1? (I don't know what the difference is but both are the same device).

 

[Wiiwu]

Wiimm, can you add a md5 function into dump and also have it generate a md5 for copy operation. Please do a md5 for each and every partition it processes.

Wiimm, missed my suggestion, any comments?

 

[Wiimm]

Sorry Wiiwi, I read it at work and then I have not thought about it.

What do you mean with partition: partition inside an ISO?
If yes: It's possible but the whole ISO must be read. And this is time intensive.

 

[mousex]

Some time I read that there are checksums of the game partitions stored in the ISO. Reading and comparing them with the real data would be cool. I read that someone on IRC did it to "check scene releases" but I couldn't really find out how to do it.

 

[Wiiwu]

Sorry Wiiwi, I read it at work and then I have not thought about it.

What do you mean with partition: partition inside an ISO?
If yes: It's possible but the whole ISO must be read. And this is time intensive.

Yes, I meant for every game in iso or .wbfs file, there are update, game etc partition. I believe all data in each partition remains unchanged whatever format we convert to isowbfs etc. So ideally as tool, it will be great if it checksums (crc/md5) them and spit it out for the user.

If you tried abgx360 a tool for 360, you'll see it CRC every element in a 360 iso/disc.

It won't add much CPU time running a checksum while doing a COPY operation but I'm thinking its more useful as a addon option in the DUMP command (make it non-default) which understandably will take time as it read all partition. Will be most useful for integrity checking.

 

[Wiimm]

Question to the MAC users here:

Doe's the automatic search for WBFS drives work on the MAC?
What is shown when you type "wwt find" while a WBFS drive is plugged.

 

[mousex]

Question to the MAC users here:

Doe's the automatic search for WBFS drives work on the MAC?
What is shown when you type "wwt find" while a WBFS drive is plugged.

It didn't work for me. I didn't use wwt find but the tool didn't work without "-p /dev/disk4s1". I can't test it now (with wwt find) because I changed to FAT loading.

 

[Wiimm]

wwt: Wiimms WBFS Tool v0.18a - 2009-11-22
A command line WBFS tool set for various os.

I have updated the tool. The first post with more details is updated too.

Download:

• wwt-v0.18a-r465-i386.tar.gz : linux/i386 version
• wwt-v0.18a-r465-cygwin.zip : cygwin (win32) version

Highlights/bug fixes of this release:

• Bug fix: All scrubbing operations including adding to a WBFS could loose sectors because of a wrong calculation in the old libwbfs.
  .
• The new wwt commands RENAME and SETTTITLE change the ID6 and/or the disc title in the WBFS inode and/or in the ISO header.
  .
• wwt DUMP will now show ID6 and title of WBFS inode and ISO ehader.
  .
• wwt CHECK has a new meaning of the --long options.
  .

Last changes:

wwt v0.17a r454 - 2009-11-18

- wwt ADD --ignore: Ignores non exsting sources without message.
- wwt DUMP: show memory range of each inode.
- New command: wit DIFF: Compare ISO images in raw or scrubbed mode.
ÂÂ EXPERIMENTAL, because the DIFF command is not testet very well yet!
- wwt EXTRACT + wit COPY/DIFF: Handle '%'-escapes for output file name.
- New option --esc: Define an alternative escape character, default is '%'.
- Bug fix: Wrong name scheme for splitted WBFS files.
- Some minor bugs fixed.

wwt v0.18a r465 - 2009-11-22

- Bug fix: Wrong temp filename while creating an splitted WBFS.
- Default split size is now 4GiB-32KiB for WBFS and 4GB for all other files.
- wwt CHECK -ll: Print an additional dump if errors found.
- wit SCRUB: Allow type conversion.
- new commands: wwt RENAME+SETTITLE: Set ID and title of discs [EXPERIMENTAL].
- IMPORTAND BUG FIX in libwbfs/wiidisc: All scrubbing operations including
ÂÂ adding to a WBFS could loose sectors.

Sources comes with GPL2. Checkout the sources directly from the SVN repository or browse the source.

 

[adamsmasher]

Has anyone thought about making a simple GUI to go with this? On OS X there's the developer tools... or one could make something in Visual Basic as well (I made a simple program with that in the past, but that was 6 years ago).

 

[drh]

Question to the MAC users here:

Doe's the automatic search for WBFS drives work on the MAC?
What is shown when you type "wwt find" while a WBFS drive is plugged.

Hi Wiimm,

No, wwt find doesn't work, I have a drive plugged in now that is populated and recognised fine by "WBFS for Mac". If I do a wwt find, here is what happens:

CODEiMac:wdf drh$ wwt find
! wwt: WARNING in ScanPartitions() @ wbfs-interface.c#110
!ÂÂÂÂÂÂCan't open partition table: /proc/partitions
!ÂÂÂÂÂÂ-> No such file or directory
!! wwt: ERROR #8 [NO WBFS FOUND] in AnalysePartitions() @ wbfs-interface.c#312
!!ÂÂÂÂÂÂno WBFS partitions found -> abort
iMac:wdf drh$

I have "v0.17a r454 mac" compiled at the moment. Only the core wwt commands like add and extract work.

Format also fails with a "segmentation error", I tried to format a second drive yesterday with wwt, I had to use the gui manager "WBFS for Mac" in the end and then I used wwt to put the WDF files on that I extracted the other day.

 

[NReale3370]

Love this tool, I'm using on my DD-WRT router and it works like a charm for all my needs. I was wondering if there was any way to tell how big an ISO would be on a .wbfs file or partition (same thing right?) BEFORE converting it, either with this tool or another (besides just looking it up somewhere)

 

[Wiimm]

@NReale3370
It's easy: Let the file grow as long as needed. Then fix the internal data structure to new len.

@dry:
Under linux the file /proc/partitions lists all active partitions in the system. That's all I need for --auto.

My questions:
- Is there a file on mac which does the same?
- Whats about /dev/disk*? Do they exist all the time or are they only available if a device is connected?
It is possible to scan all /dev/disk* files for a WBFS.