Hacking wwt+wit: Wiimms WBFS+ISO Tools

[Wiiwu]

Read your post in the Configurable USB Loader thread that your tool can handle .wbfs files in fat partition.

Is there some example how-to commands specifically on the wbfs file management?

And if this can convert .iso to .wbfs, can add a feature to detect if partition is fat and if so, turn on automatic file splitting?

 

[Wiimm]

If the batch file was set up right, it should allow you to just drag an ISO onto it and ISO2WDF just does its thing... right? But before conversion you should wait until the next versions. The next step is that iso2wdf scrubs the ISO images. In the moment it makes a raw copy.

Read your post in the Configurable USB Loader thread that your tool can handle .wbfs files in fat partition.

Is there some example how-to commands specifically on the wbfs file management?
At the moment an automatic conversion of many files s like oggzees tool doe's is missing. I'm working ... and will create a bash script as workaround.

EDIT: here is a bash script (I think it runs also under cygwin):

#!/bin/bash

opt_init="--quiet --split --force --size=12g"
opt_add="--verbose --verbose --trunc"

while (($#))
do
ÂÂÂÂsrc="$1"
ÂÂÂÂshift

ÂÂÂÂstat=($(wwt filetype -l "$src"))
ÂÂÂÂtype="${stat[0]}"
ÂÂÂÂid6="${stat[1]}"
ÂÂÂÂif [[ $type = ISO || $type = WDF+ISO ]]
ÂÂÂÂthen
ÂÂÂÂÂÂÂÂwbfs="$id6.wbfs"
ÂÂÂÂÂÂÂÂprintf "\n** create %s from %s\n" "$wbfs" "$src"
ÂÂÂÂÂÂÂÂwwt init $opt_init "$wbfs"
ÂÂÂÂÂÂÂÂwwt addÂÂ$opt_add -p "$wbfs" "$src"
ÂÂÂÂelse
ÂÂÂÂÂÂÂÂprintf "\n!! wrong file type: %s %s\n" "$id6" "$src" >&2
ÂÂÂÂfi
done

QUOTE(Wiiwu @ Nov 1 2009, 04:00 AM)

And if this can convert .iso to .wbfs, can add a feature to detect if partition is fat and if so, turn on automatic file splitting?

That's an nice idea. But I don't know how to detect the maximal files size on a file system.

 

[Wiimm]

wwt: Wiimms WBFS Tool v0.11a - 2009-11-02
A command line WBFS tool set for various os.

I have updated the tool. The first post with more details is updated too.

WARNING: I have only done a few tests with the new scrubbing features (tools wit, wdf2iso, iso2wdf). It seems that scrubbing works.

Download:

• wwt-v0.11a-r339-i386.tar.gz (linux/386 version)
• wwt-v0.11a-r339-cygwin.zip (cygwin version)

Highlights of this release:

• Scrubbing is enabled in three tools:
  • wdf2iso and iso2wdf do scrubbing by default if the source is an iso. Both can read ISO and WDF. Other files are copied/converted directly.
  • wit has a new command: COPY|CP: Is copies one or much ISO files to a destination. While copying the files are converted from/to ISO/WBFS and scrubbed. A raw copy is also available.
• wwt DUMP: The DUMP command dumps now additional the memory layout of each disc (-ll) and of the whole WBFS (-lll). These dumps are good for controlling and debugging the WBFS.

Last changes:

wwt v0.11a r339 - 2009-11-02

- wwt "LIST --mixed": output format changed.
- New option --progress: enable progress counter independend of verbose level.
- wdf2iso + iso2wdf:
ÂÂÂÂ- Both accept any files as input. If the input is an ISO or a WDF+ISO than
ÂÂÂÂÂÂthe file is scrubbed (controlled by optiond --psel snd --raw).
ÂÂÂÂ- New option --preserve: preserves file times
- New command: wit COPY source... dest -> converting and scrubbing.
- wwt DUMP --long --long: Dump a memory layout map for each disc.
- wwt DUMP --long --long --long: Dump a memory layout map for whole WBFS.

Sources comes with GPL2. Checkout the sources directly from the SVN repository or browse the source with any browser.

 

[Wiimm]

BUG WARNING

I have found bugs in the libwbfs function wbfs_trim() (details).

Please do not use:

• wwt command ADD together with option --trunc
• wwt command TRUNCATE.

The bug will be fixed in the next release.
The command "wwt dump -lll wbfs_file" can be used for controlling wbfs files.

EDIT:
Bug 1. does not appear in WBFS files without any removed or overwritten discs.
Bug 2. does not appear if the WBFS has at least one free block.

 

[Blue-K]

Gosh...this Tool is getting better and better...and I'm still too stupid to use it

. I hope cojiro will soon find time and make a nice wrapper...

Anyways, thanks Wiimm for your Efforts, realy appriciated!
Dankeschön

!

 

[mousex]

Ok, since this has now the ability to copy all the Partitions from the ISO, I realy need this. And I think I have an Idea to make it a bit simpler for me (and others) which are still a bit afraid of the terminal...Nautilus-Scripts! So you only have to right-klick the ISO, and select the skript, which will do the rest. I did some research, and found that one:

#!/bin/bash
gnome-terminal --window-with-profile=default -x abgx360 -af3 “$1?

The underlined part is the part that has to be changed with the commands of the WBFS Tool, I guess...can anybody help me here a bit? Would be realy nice if we could get this to work...

Fow which system do you need the GUI? On linux (Gnome) this would be very easy to do (either Nautilus scripts or there was another toolkit that I used about one year ago that was perfect for this task, I just don't remember the name at the moment. It was just one programm that you called with different parameters to show a list/yes-no question, status bar and so on) with a nice looking GUI.
And well, on Windows, we saw how easily many people can make ugly looking GUIs and that only a few now how to make a nice working GUI

The fastet for you is learning how to use the tool (if you are really using Linux this shouldn't be too hard) and then writing a small Wrapper.

 

[Blue-K]

Fow which system do you need the GUI? On linux (Gnome) this would be very easy to do (either Nautilus scripts or there was another toolkit that I used about one year ago that was perfect for this task, I just don't remember the name at the moment. It was just one programm that you called with different parameters to show a list/yes-no question, status bar and so on) with a nice looking GUI.
And well, on Windows, we saw how easily many people can make ugly looking GUIs and that only a few now how to make a nice working GUI

The fastet for you is learning how to use the tool (if you are really using Linux this shouldn't be too hard) and then writing a small Wrapper.

Yeah, I threw Windows away and replaced it with Ubuntu, that's why I need a good Linux Manager

. I first had a look at Glade, with which it would be realy easy to make a nice GUI (I guess that's the App you've meant). But after some tries, I guess I wouldn't be able to code the Commands in. I then had a look at Zenity, and modified an existing Script. Here's the result:

Spoiler

#!/bin/bash
#
# You'll need to have wwt.bin correctly placed/installed!
# You'll also need zenity (Installed in Ubuntu by default).
#
zenity --info --title="Welcome to Wiimms WTT GUI Script!" --text="This Script lets you add an Wii ISO to your WBFS Drive. \n\nOriginal script written by by Nevon (www.blastfromthepast.se)\nModified by Blue-K\nWWT Tool by Wiimm \n\nClick OK and then select your ISO to continue."

FILENAME=$(zenity --file-selection --title="Please choose your ISO");

zenity --question --text="Would you like to add $FILENAME to the WBFS Drive?"
if [ $? = 0 ]; then

(wwt -a add "$FILENAME") |
zenity --progress --auto-close --title="Progress" --pulsate --text="Adding $FILENAME to the WBFS Drive.\n\nPlease be patient as this will take some time." --percentage=0

if [ "$?" = -1 ] ; then
zenity --error --text="Failed!"
exit
fi

zenity --info --title="Success!" --text="Game successfully added! Thanks for using this Script"
exit
fi
fi
fi

This one is a realy simple dirty Script, which can do one simple task: Add an ISO to a WBFS Drive (using the -a add command). I made this basicly for me, because I don't realy like the Terminal. It works for shure on Ubuntu, but should also work on other *nix Systems if you have zenity installed. You'll need to have wwt.bin correctly installed/placed (using the given Script in the official download). Download it, unpack it, right click on the file>Properties>Permissions>Check Allow executing file as program. Then simply double klick it and select run.

I might have now a closer look on Zenity...or use Nautilus Scripts..at least I can now finaly add ISOs with all partitions to my Drive

. YES!

 

[Skizzo]

Wiimm, is it true that WBFS does not support fragmented files? There is no fragmentation on a WBFS drive? I figured you'd be one of the ones to actually know the facts about this. Thanks for any info.

 

[redia]

hello,

am I wrong or there is no progress bar/counter in these tools ?
sorry but I can't seem to find it.

Cheers,
R

 

[oggzee]

Wiimm, is it true that WBFS does not support fragmented files? There is no fragmentation on a WBFS drive? I figured you'd be one of the ones to actually know the facts about this. Thanks for any info.

Who says there's no fragmentation on WBFS? Afaik it can occur, however the block size is large and you're dealing with huge 'files' so the negative effects of fragmentation will not show up. In other words, if you were adding and removing random games with different sizes, in the end they will be fragmented, but the speed difference should not be noticable because of the large block size.

 

[Elfish]

i was wondering if u could u add a partition resize function?

 

[W hat]

Fow which system do you need the GUI?

I would love a Windows GUI. It can look like total crap, function over form always.

 

[Bauldrick]

@ridia

I hate to ask, but could you possibly add ETA on add extract etc...? Like wbfs has, e.g :

ETA 4:59 [OOOo ]

Thanks again, I've converted 3/4 of my games to .wdf so far

multiple choice:
[ ] RTFM
[ ] -vv

I'm working on an environ var (WWT_OPT) for default settings.

 

[Skizzo]

Wiimm, is it true that WBFS does not support fragmented files? There is no fragmentation on a WBFS drive? I figured you'd be one of the ones to actually know the facts about this. Thanks for any info.

Who says there's no fragmentation on WBFS? Afaik it can occur, however the block size is large and you're dealing with huge 'files' so the negative effects of fragmentation will not show up. In other words, if you were adding and removing random games with different sizes, in the end they will be fragmented, but the speed difference should not be noticable because of the large block size.

Unless I'm mistaken, that's exactly what some wank stain by the name of D34DL1N3R (and who can argue with someone with a l33t name like that?) is arguing (no fragmentation) here http://gbatemp.net/index.php?showtopic=188...t&p=2367727

 

[Wiiwu]

i was wondering if u could u add a partition resize function?

If anyone need to resize a WBFS partition, then take this as a reason to move all your games onto a FAT partition and in the future with FAT, you can resize it anytime you like, with any tools you like.

Also any filesystem will have fragmentation, its just how aggressive the OS handles it and its always a balance with fragmenting files or fragmenting free space, eventually when low on space it will still fragment the files. But if fragmentation is still such a concern, then defrag it with the many defragging tool available.

 

[redia]

@ridia

I hate to ask, but could you possibly add ETA on add extract etc...? Like wbfs has, e.g :

ETA 4:59 [OOOo ]

Thanks again, I've converted 3/4 of my games to .wdf so far

multiple choice:
[ ] RTFM
[ ] -vv

I'm working on an environ var (WWT_OPT) for default settings.

damn... how did I miss that.
sorry & thanks Bauldrick

R

 

[Skizzo]

i was wondering if u could u add a partition resize function?

If anyone need to resize a WBFS partition, then take this as a reason to move all your games onto a FAT partition and in the future with FAT, you can resize it anytime you like, with any tools you like.

Also any filesystem will have fragmentation, its just how aggressive the OS handles it and its always a balance with fragmenting files or fragmenting free space, eventually when low on space it will still fragment the files. But if fragmentation is still such a concern, then defrag it with the many defragging tool available.

As I was trying to say, over in the thread I linked to, it appears the individual mentioned is saying that WBFS DOES NOT allow games to be fragmented. In other words, if I only have 4GB of free space left on my WBFS partition, but the free space comes from games that have been deleted over time, and thus the free space isn't contiguous (or 'consolidated') (say I have 1GB free, 15GB used by other games, 2.5GB free, 25GB used by games, and finally .5GB free) then is it true that the largest single game I can now install is however large the biggest chunk of free space is, or 2.5GB in the example above? Or could a game that is say 3.5GB in size still be installed, splitting the game into the first two chunks of free space in the example?

 

[oggzee]

As I was trying to say, over in the thread I linked to, it appears the individual mentioned is saying that WBFS DOES NOT allow games to be fragmented. In other words, if I only have 4GB of free space left on my WBFS partition, but the free space comes from games that have been deleted over time, and thus the free space isn't contiguous (or 'consolidated') (say I have 1GB free, 15GB used by other games, 2.5GB free, 25GB used by games, and finally .5GB free) then is it true that the largest single game I can now install is however large the biggest chunk of free space is, or 2.5GB in the example above? Or could a game that is say 3.5GB in size still be installed, splitting the game into the first two chunks of free space in the example?

I believe I already answered that, why the need to drag this further?
And wouldn't the thread 'Inside WBFS' be more appropriate to discuss wbfs fragmentation?

 

[Skizzo]

As I was trying to say, over in the thread I linked to, it appears the individual mentioned is saying that WBFS DOES NOT allow games to be fragmented. In other words, if I only have 4GB of free space left on my WBFS partition, but the free space comes from games that have been deleted over time, and thus the free space isn't contiguous (or 'consolidated') (say I have 1GB free, 15GB used by other games, 2.5GB free, 25GB used by games, and finally .5GB free) then is it true that the largest single game I can now install is however large the biggest chunk of free space is, or 2.5GB in the example above? Or could a game that is say 3.5GB in size still be installed, splitting the game into the first two chunks of free space in the example?

I believe I already answered that, why the need to drag this further?
And wouldn't the thread 'Inside WBFS' be more appropriate to discuss wbfs fragmentation?

Sorry, but the 'AFAIK' didn't exactly exude confidence in your answer, so I was seeking further input. So, you're saying the answer to the question above is that yes, as long as I have enough free space on the WBFS partition, even if that space is not contiguous, I CAN install the game. Thanks. And should I want to discuss the matter any further, I'll move it to the thread you suggest. Sorry about that too.

 

[Wiimm]

wwt: Wiimms WBFS Tool v0.12a - 2009-11-05
A command line WBFS tool set for various os.

I have updated the tool.

WARNING: I have only done a few tests with the new scrubbing features (tools wit, wdf2iso, iso2wdf). It seems that scrubbing works.

Download:

• wwt-v0.12a-r364-i386.tar.gz (linux/386 version)
• wwt-v0.12a-r364-cygwin.zip (cygwin version)

This is an experimental release!

Highlights of this release:

• The wit command DUMP has been improved. Also it can dump ISO images inside a WBFS.
  .
• I have rewritten the file subsystem. WBFS files with a single disc are interpreted as an ISO file and can be used as source. WBFS with more than 1 disc can be used together with a selector: /path/to/WBFS_FILE/SELECT. See the code block below.
  
  This is only true for the tool 'wit'. The tool 'wwt' uses the old file subsystem.
  .
• The tool 'wit' has 2 new commands: COPY and SCRUB: Both commands work in the same way, but for SCRUB the source and the destination is the same. Both can scrub and/or convert from/to ISO/WDF/WBFS and/or split/unsplit.
  
  To convert all files from a directory to single WBFS files just type:
  

  wit copy /path/to/dir -d dest_dir --wbfs --split

  
  To convert all files from a WBFS to single WBFS files just type:
  

  wit copy /path/to/wbfs -d dest_dir --wbfs --split

• Again: This is an experimental release!

///////////////////////////////////////////////////////////////////////////////
///////////////ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂProcessing ISO filesÂÂÂÂÂÂÂÂÂÂÂÂ ///////////////
///////////////////////////////////////////////////////////////////////////////

Command like "wwt ADD" or "wit COPY" uses ISO files as input. They accept
plain ISO files and ISO files in WDF and WBFS containers. In detail:

/path/to/PLAIN_ISO:
ÂÂÂÂA PLAIN ISO file is a 1:1 copy of a file. It is not compressed and not
ÂÂÂÂpart of any container. The standart extension is ".iso".

/path/to/WDF_ISO:
ÂÂÂÂThis is an ISO packed in a WDF container. WDF container may be used for
ÂÂÂÂany files and not only for ISO images. The standart extension is ".wdf".

/path/to/WBFS_FILE
ÂÂÂÂAn WBFS file/device with exact one ISO image is also accepted. The
ÂÂÂÂstandart extension for WBFS files is ".wbfs". Some commands accept the
ÂÂÂÂwhole WBFS with all ISO images as source. See the specific doc.

/path/to/WBFS_FILE/SELECTOR
ÂÂÂÂThis is a special construct. The ISO image of the WBFS_FILE selected
ÂÂÂÂby SELECTOR is used as source. The selector may be one of:

ÂÂÂÂ - 'ABCDEF' : ID6 of a disc (exaatc 6 charaters)

ÂÂÂÂ - indexÂÂÂÂ: The zero based index of the disc. The range goes from zero
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂup to the number of discs in the WBFS minus 1.
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ(decimal number, but not 6 digits)

ÂÂÂÂ - '#' slot : The decimal slot number within the WBFS. The range goes from
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂzero up to the maximal number of discs in the WBFS minus 1.

ÂÂÂÂExamples:
ÂÂÂÂÂÂ.../a.wbfs/rmcp01 : use "Mario Kart" from 'a.wbfs' as source.
ÂÂÂÂÂÂ.../a.wbfs/5ÂÂÂÂÂÂ: use the image with index #5 from 'a.wbfs' as source.
ÂÂÂÂÂÂ.../a.wbfs/#5ÂÂÂÂ : use the image at slot #5 from 'a.wbfs' as source.

When writing ISO files the option --wdf (default), --iso and --wbfs control
the output format. If writing a WBFS file this WBFS is truncated and contains
exactly one ISO image.

Last changes:

wwt v0.12a r364 - 2009-11-05

- Bug fix showing the progress counter.
- Bug fix in libwbfs/wbfs_trim() -> "ADD --trunc" and "TRUNCATE" failed.
- wit DUMP --long: Dump a memory map.
- wdf2iso + iso2wdf: print progress infos.
- New ISO source alternative: /path/to/WBFS_FILE/SELECTOR (see doc)
- New wit command: COPY: Copy and convert ISO images.
- New wit command: SCRUB: Scrubs ISO images.

Sources comes with GPL2. Checkout the sources directly from the SVN repository or browse the source with any browser.