Hacking wwt+wit: Wiimms WBFS+ISO Tools

[Wiimm]

Wiimm, I'm deeply impressed! Your tool becomes better each time I use it

I know that wit is very powerfull. You can list/compare/copy/scrub/split/rename (and more) many files with one little command line.

Source files are scanned (file names and extensions are irrelevant four source files) for iso format (PLAIN, WBFS, WDF, CISO). If the source is a directory all found ISOs in that directory are used as source. With option --recurse directories are searched recursively. The destination format are controlled by names (example: just put a .ciso at the end of dest file to force CISO) and by options. The dest filenames are controlled by %-escapes.

I know that a good FAQ is needed ... yes I know


QUOTE(Skeeve @ Apr 11 2010, 09:51 PM) You're german (too)?

Ja, ich bin ein alter Sack aus dem Ruhrgebiet.

 

[Wiimm]

@sylver78 & other mac developer

Since today I have access to a mac and hope that i can make mac distribs by myself. But the mac is missing the development tools

Can anyone of you explain what and how to install a development tool set for mac?

 

[giantpune]

i just installed x-code on my mac and nothing else and i was able to build your tool set. i had to sign up for something (free) to get access to the download link.

 

[drh]

http://developer.apple.com/technologies/xcode.html

 

[drh]

By the way, I formatted a disk the other day with a new block size using the command:

wwt format /dev/rdisk2s2 --hss 4096 --force

I filled the 1tb drive (why didn't I test it first with just 1 file) and the drive is not recognised by my loader :-(

I'm using WiiFlow, so I'm guessing its a problem with WiiFlow recognising the WBFS partition with the larger block size, because the drive worked fine before with the default block size :-(

 

[Wiimm]

http://developer.apple.com/technologies/xcode.html
Thanx, I have already found this.

QUOTE(drh @ Apr 13 2010, 07:33 PM) By the way, I formatted a disk the other day with a new block size using the command:

wwt format /dev/rdisk2s2 --hss 4096 --force

I filled the 1tb drive (why didn't I test it first with just 1 file) and the drive is not recognised by my loader :-(

I'm using WiiFlow, so I'm guessing its a problem with WiiFlow recognising the WBFS partition with the larger block size, because the drive worked fine before with the default block size :-(

Someone told me that GX can handle larger block size. Most loaders ignore the block size parameter and always load 512 bytes instead

 

[sylver78]

@sylver78 & other mac developer

Since today I have access to a mac and hope that i can make mac distribs by myself. But the mac is missing the development tools

Can anyone of you explain what and how to install a development tool set for mac?

drh said it : register on apple dev site (it's free), get xcode, install it (don't forget to enable 10.4 SDK) and the Mac will be ready to build wwt

BTW, when using my 10.4 makefile, building a ppc64 target is working great (but need someone to test it ...)

 

[wishmasterf]

Someone told me that GX can handle larger block size. Most loaders ignore the block size parameter and always load 512 bytes instead

Would be great if you could help them to implement the support for other block size than 512 bytes. They are all OSS so helping should be easy for you ;-)

 

[Wiimm]

Someone told me that GX can handle larger block size. Most loaders ignore the block size parameter and always load 512 bytes instead

Would be great if you could help them to implement the support for other block size than 512 bytes. They are all OSS so helping should be easy for you ;-)

Here is my help: http://gbatemp.net/index.php?showtopic=185428
But is there a real need to support this? Are 500 possible games not much more than enough?

 

[drh]

Are 500 possible games not much more than enough?

It is whilst the largest WD passport is still 1tb.

I could really do with a 1.5tb at the moment, but no-one makes one.
I don't want to go down the road of the larger form-factor drives or even non-usb powered drives, so come on WD the world is waiting.......

 

[giantpune]

can you not set the block size parameter when you create the partition?

 

[Wiimm]

can you not set the block size parameter when you create the partition?

With wwt use --hss= to define the "Hd Sector Size" (default=512) and/or --wss to define the "Wbfs Sector Size" (default is calculated, standard=2M,4M,8M, ...). The --wss is only needed for recovery.

 

[Skeeve]

If using --DEST (or -D) instead of --dest (or -d) then the missing directories are created automatically. (this is valid for all tools)
Unfortunately not:
CODE/usr/local/bin/wit copy --wbfs su-opoona.iso --DEST '/Volumes/HOMEBREW/wbfs/%T [%I]/'
*****ÂÂwit: Wiimms ISO Tool v0.40a r799M mac - Dirk Clemens - 2010-04-11ÂÂ*****
* wit SCRUB 1/1 ISO:su-opoona.iso -> WBFS:/Volumes/WIIDISK/wbfs/Opoona [RPOPC8]/RPOPC8.wbfs
!! wit: ERROR #17 [CAN'T CREATE DIRECTORY] in CreatePath() @ lib-std.c#3034
!!ÂÂÂÂÂÂCan't create directory:
!!ÂÂÂÂÂÂ-> No such file or directory

 

[Wiimm]

@Skeeve:
a bug?
try: --DEST '/Volumes/HOMEBREW/wbfs/%T [%I]' (without trailing slash)


EDIT:
There is a problem with absolute paths

EDIT 2: fixed in CreatePath()

--- lib-std.cÂÂ (revision 811)
+++ lib-std.cÂÂ (working copy)
@@ -3010,6 +3010,10 @@
ÂÂÂÂ char buf[PATH_MAX], *dest = buf;
ÂÂÂÂ StringCopyS(buf,sizeof(buf),fname);

+ÂÂÂÂ// skip root directory
+ÂÂÂÂwhile ( *dest == '/' )
+ÂÂÂÂÂÂ dest++;
+
ÂÂÂÂ for(;;)
ÂÂÂÂ {
ÂÂÂÂÂÂÂÂwhile ( *dest && *dest != '/' )

 

[Skeeve]

Thanks Wiimm! I'll update my working copy and report as soon as I see the fix in there.

 

[Wiimm]

I have modified the patch and use "while" instead of "if" because //Volume/a.b is also a valid path name.

 

[Skeeve]

But I don't see it in the repository yet, do I?

Update: I think you should put the while inside the for. Compare the difference when feeding a (stupid but legal) path like '//Volumes//WIIDISK//wbfs/test/path/file'.

While outside:

mkdir //Volumes
mkdir //Volumes/
mkdir //Volumes//WIIDISK
mkdir //Volumes//WIIDISK/
mkdir //Volumes//WIIDISK//wbfs
mkdir //Volumes//WIIDISK//wbfs/test
mkdir //Volumes//WIIDISK//wbfs/test/path

While inside:

mkdir //Volumes
mkdir //Volumes//WIIDISK
mkdir //Volumes//WIIDISK//wbfs
mkdir //Volumes//WIIDISK//wbfs/test
mkdir //Volumes//WIIDISK//wbfs/test/path

So I suggest:

--- lib-std.cÂÂ (revision 811)
+++ lib-std.cÂÂ (working copy)
@@ -3010,6 +3010,10 @@
ÂÂÂÂ char buf[PATH_MAX], *dest = buf;
ÂÂÂÂ StringCopyS(buf,sizeof(buf),fname);

ÂÂÂÂ for(;;)
ÂÂÂÂ {
+ÂÂÂÂÂÂÂÂ// skip double slashes
+ÂÂÂÂÂÂÂÂwhile ( *dest == '/' )
+ÂÂÂÂÂÂÂÂÂÂ dest++;
+
ÂÂÂÂÂÂÂÂwhile ( *dest && *dest != '/' )

 

[zon7]

Hi Wiimms.
I've tried today the linux version to add it to WiiJManager, but have come to a problem. My partition is not recognized on Ubuntu 9.10.
The disk has 2 partitions, one FAT and the other is WBFS formatted. Right now I have it recognized on windows and osx, but linux fail to recognize it. Any help?

BTW: I'll love to have a parameter in wit, when doing --recurse, to limit the tree exploration, in case the user choose /windows or something like this. It'll be useful to can limit it to for example 2-3 levels.
Thanks

 

[Wiimm]

But I don't see it in the repository yet, do I?
No. It's only in my private branch because developing extracting of files from ISO. And this is on work and not testet yet. A commit into the main trunk will come today or tomorrow.

And yes: I will move the while loop into the for loop. Thanx.

Hi Wiimms.
I've tried today the linux version to add it to WiiJManager, but have come to a problem. My partition is not recognized on Ubuntu 9.10.
The disk has 2 partitions, one FAT and the other is WBFS formatted. Right now I have it recognized on windows and osx, but linux fail to recognize it. Any help?
What is the output of: wwt find -l

QUOTE(zon7 @ Apr 16 2010, 09:06 AM)

BTW: I'll love to have a parameter in wit, when doing --recurse, to limit the tree exploration, in case the user choose /windows or something like this. It'll be useful to can limit it to for example 2-3 levels.
Thanks

I think about that request. A internal max_depth param already exists and set to 15 on --recurse and to 1 for others. So it should be easy to implement.

 

[Wiimm]

wwt: Wiimms WBFS Tool v0.41a - 2010-04-16
A command line WBFS tool set for various os.

I have updated the tool. The first post with more details is updated too.

Download of the current stable version:

• wwt-v0.41a-r824-i386.tar.gz : linux/i386 version
• wwt-v0.41a-r824-x86_64.tar.gz : linux/x86_64 version
• wwt-v0.41a-r824-mac.tar.gz : Mac OS universal binaries (ppc, i386 and x86-64)
• wwt-v0.41a-r824-cygwin.zip : cygwin (win32) version

Download of the previous stable version:

• wwt-v0.40a-r790-i386.tar.gz : linux/i386 version
• wwt-v0.40a-r790-x86_64.tar.gz : linux/x86_64 version
• wwt-v0.40a-r790-mac.tar.gz : Mac OS universal binaries (ppc, i386 and x86-64) by sylver78
• wwt-v0.40a-r790-cygwin.zip : cygwin (win32) version

There is also a GUI available:

• WiiJManager Multiplatform

Highlights/bug fixes of this release:

• The command "wit DUMP" is able to dump a file list. The list is controlled by the new option --files=.
  .
• The new command "wit ILIST" (or short "wit IL") will list all files of all given ISO images. Output can be controlled by --long and --no-header. The new option --files= controlls which files are listed. The option --psel selects partitions.
  .
• The new command "wit EXTRACT" (or short "wit X") will extract all files of all given ISO images. Define a destination directory with --dest or --DEST. The new option --files= controlls which files are extracted. The option --psel selects partitions.
  .
• The new wit option --files=list (-F list) define allow and deny rules for selecting files. The list is separatied by semicolons into items. Each item must be begin with as plus sign (allow) or a minus sign (deny) followed by a pattern. The patterns have rsync like style. For a detailed description see file "wit.txt" section "Selecting files with --files=rules".
  
  Examples:
  

  # list all files of all ISO in current directory
  wit il .
  
  # list all *.wad files of all ISO in current directory
  wit il . -F+.wad
  
  # list all IOS*.wad files of all ISO in current directory
  # don't print headers
  # (escape param to permit shell to interprete '*')
  wit il . -H '-F+IOS*.wad'

  .

Last changes:

wwt v0.40a r790 - 2010-03-31

ÂÂ- Cache handling (pipe support) improved. Discs with update partitons are
ÂÂÂÂproblematic (some discs needs a 256 MiB cache). Try --psel=game to ignore
ÂÂÂÂupdate partitions. The CYGWIN version has real problems with pipes.
ÂÂ- Bug fix: Option "--ciso" was not recognized.
ÂÂ- New test script: scripts/test-wwt+wit.sh
ÂÂÂÂ- test "wwt INIT" with different HD sector sizes
ÂÂÂÂ- test "wwt ADD" with ISO, CISO, WDF, WBFS
ÂÂÂÂ- test "wwt EXTRACT" to ISO, CISO, WDF, WBFS
ÂÂÂÂ- test "wit COPY" (convert) to ISO, CISO, WDF, WBFS

wwt v0.41a r824 - 2010-04-16

ÂÂ- New escape sequence for output file names:
ÂÂÂÂ%Y = Extended filename: A short cut for '%T [%I]' (no file extension)
ÂÂ- New command: wit ILIST: List all files from ISO images.
ÂÂ- New command: wit EXTRACT: Extract all files from ISO images.
ÂÂÂÂThis is a first implementation or better a proof of concept.
ÂÂ- New option: --files=rules: Define a rule list as selector for ISO
ÂÂÂÂfile operations. A semicolon separate single items. Each item must begin
ÂÂÂÂwith '+' (allow) or '-' (deny). Multiple usage of the option is allowed.
ÂÂÂÂThis options is used by the commands DUMP, ILIST and EXTRACT. See file
ÂÂÂÂ"wit.txt" section "Selecting files with --files=rules" for more details.
ÂÂ- Bug fix: Creating directories (--DEST) with absoulte paths will now work.
ÂÂ- New wit option: --rdepth: Set the maximum recurse depth for option
ÂÂÂÂ--recurse in the range 0..100. The default depth is 10.
ÂÂ- New param for wwt+wit LIST --section: used_blocks= (1 block = 32 KiB)

Sources comes with GPL2. Checkout the sources directly from the SVN repository or browse the source.