Hacking wwt+wit: Wiimms WBFS+ISO Tools

[Wiimm]

try-a (lower case) for --auto
But -A / --all should work .... mmmh (i can't test it on work)

 

[jivaros]

I'm am very glad to see a linux tool intensively developped !

Like i said before, a check function should be great, but i suggest another cool and unique feature :
importing an iso/wdf from a .rar (or .zip) file, without uncompressing the archive in a temporary directory ...

It seems with unrar-nonfree, it is possible to extract a file to stdin, but not with the unrar-free tool.

If you give me some pointers, (and if you wish) i could try to submit a patch for the rar2wbfs function !

 

[Wiimm]

I'm am very glad to see a linux tool intensively developped !

Like i said before, a check function should be great, but i suggest another cool and unique feature :
importing an iso/wdf from a .rar (or .zip) file, without uncompressing the archive in a temporary directory ...

It seems with unrar-nonfree, it is possible to extract a file to stdin, but not with the unrar-free tool.

If you give me some pointers, (and if you wish) i could try to submit a patch for the rar2wbfs function !

This is a hard job because the WBFS framework (written by others) needs random access and uses the seek() functions to address to the file. And thats mean that the image must be decompressed completely before the access.

On the other hand most (but not the header) of the seek access are at increased offsets: A seek simulation is possible when caching only a few (about 8 MiB) of data. In the next month i will go deeper into the WBFS framework.

 

[Wiimm]

wwt: Wiimms WBFS Tool v0.03a - 2009-10-08
A command line WBFS tool for linux.

I have updated the tool. The first post with more details is updated too.

I have tested this tool very well and the software do what I want but I can't give any warranty. I have made different import and export (ISO and WDF) tests with about 20 games and the resulting ISO images are absolute identical as the source files.

wwt is now Beta and I'm using it as replacement for the tool 'wbfs'.

Download: http://wiimm.de/x/./wwt-v0.03a-r200.tar.gz (Documentation is included)
More Infos about WDF: http://svn.wiimm.de/open/wii/trunk/wiimms-...WDF.txt?view=co

Last changes:

wwt V0.03a r200 - 2009-10-08

- Enhanced error management again.
- Misspelling in --help: The shortcut for 'DUMP' is 'D' and not 'DP'.
- Make: Support of environment variable 'XFLAGS' -> 'XFLAGS=-march=x86-64 make'
- File handling: Support of open() and fopen() as alternatives for testing.
- Script test-wdf-tools.sh: Positive test is now done with >1 million files.
- New option for 'wdf-dump': --chunk: Print chunk header table.
- WDF specification is complete.

@moparisthebest and others:
Just create a file named 'Makefile.local' and make you local modifications here.

 

[Wiimm]

Feature Request:

ÂÂREQUEST:
ÂÂÂÂI suggest another cool and unique feature:
ÂÂÂÂImporting an iso/wdf from a .rar (or .zip) file, without uncompressing the
ÂÂÂÂarchive in a temporary directory ...

ÂÂANSWER:
ÂÂÂÂThis is a hard job because the WBFS framework (written by others) needs
ÂÂÂÂrandom access and uses the seek() functions to address to the file. And
ÂÂÂÂthat means that the image must be decompressed completely before access. 

ÂÂÂÂOn the other hand most (but not the header) of the seek access are at
ÂÂÂÂincreased offsets: A seek simulation is possible. In the next month i will
ÂÂÂÂgo deeper into the WBFS framework.

ÂÂANALYSIS:
ÂÂÂÂI have quickly tested 3 iso images:
ÂÂÂÂIt seems to be possible when 2 file ranges are cached:
ÂÂÂÂ-ÂÂÂÂÂÂ 0x0.. 0x200000 (len=0x200000 = 2 MiB)
ÂÂÂÂ- 0xf800000..0xfe00000 (len=0x600000 = 6 MiB)
ÂÂÂÂAll other file access is in sequential order.

ÂÂIMPLEMENTATION:
ÂÂÂÂDone but experimental. Use option --cache to enable it.

Is there anybody who wants to test this feature with many game discs?

 

[patrick9999]

Hi,

Thanks you, i love and use it every day

 

[Wiimm]

wwt: Wiimms WBFS Tool v0.04a - 2009-10-11
A command line WBFS tool for linux.

I have updated the tool. The first post with more details is updated too.

I have tested this tool very well and the software do what I want but I can't give any warranty. I have made different import and export (ISO and WDF) tests with about 20 games and the resulting ISO images are absolute identical as the source files.

wwt is now Beta and I'm using it as replacement for the tool 'wbfs'.

Download: http://wiimm.de/x/./wwt-v0.04a-r212.tar.gz (Documentation is included)
More Infos about WDF: http://svn.wiimm.de/open/wii/trunk/wiimms-...WDF.txt?view=co

Last changes:
CODEwwt V0.04a r212 - 2009-10-11

- ADD: ID6 is now also recognized when source is a WDF.
- New command: FILETYPE: Print a status line for each given file.
- New script: test-ADD+EXTRACT.sh: ADD + EXTRACT test with various options.
- *EXPERIMENTAL*:
ÂÂÂÂ - Option --cache enables caching of well known file parts while reading
ÂÂÂÂÂÂ an ISO image. Perhaps cached areas must be expanded.
ÂÂÂÂ - Adding a game from a pipe if option --cache is set. Commands like
ÂÂÂÂÂÂ "wwt ... ADD --cache

 

[jivaros]

Awsome !
I'm gonna test this !

you deserve a standing ovation

:

\o/ \o/ \o/
| | |
/ \ / \ / \

Thank you !

 

[thza]

I really like this tool, but please follow the Linux Filesystem Hierarchy Standard (FHS). The titles.txt files should be stored under $PREFIX/share/wwt/titles.txt, not under $PREFIX/lib/titles.txt.

 

[jivaros]

- Adding a game from a pipe if option --cache is set. Commands like
"wwt ... ADD --cache

 

[Wiimm]

@jivaros
reading from stdin is not implemented yet
try: wwt .... add --cache

 

[Wiimm]

I really like this tool, but please follow the Linux Filesystem Hierarchy Standard (FHS). The titles.txt files should be stored under $PREFIX/share/wwt/titles.txt, not under $PREFIX/lib/titles.txt.

done.
(But the install script must be improved -- later)

 

[gizmomelb]

Hi Wiimm, this looks like a very interesting tool.

I just wanted to clarify if your scrub ISO option produces files the same as the cISO format please (ie: as ripped by any USB loader) or is there another option which would produce the same output?

Thank you.

 

[Wiimm]

Hi Wiimm, this looks like a very interesting tool.

I just wanted to clarify if your scrub ISO option produces files the same as the cISO format please (ie: as ripped by any USB loader) or is there another option which would produce the same output?

Thank you.

SCRUB is planned but not subject of interest in the moment. And answers about cISO are hard because I don't know what cISO is (missing technical info).

 

[Bauldrick]

How can I obtain the source, I wish to compile on mips architecture. I try svn but get:

svn: Repository moved permanently to '/open/wii/trunk/wiimms-wbfs-tool/'; please relocate

Many thanks

 

[Wiimm]

How can I obtain the source, I wish to compile on mips architecture. I try svn but get:

svn: Repository moved permanently to '/open/wii/trunk/wiimms-wbfs-tool/'; please relocate

Many thanks

This is only the web view to the open parts off the svn. The svn itself is closed (account is needed).
Please use the tarball (link at the bottom in every directory view).

 

[Bauldrick]

Hi Wiimm,

Thanks, I haven't tested it on my box yet, have only just got it to compile. The only way I could compile was to comment out

CFLAGSÂÂÂÂÂÂÂÂ += -mtune=generic

else I would get

cc1: error: unrecognized command line option "-mtune=generic"

This is on a mips cpu

Well, it appears to work - can find wbfs hdd and list games on it, great stuff...

-edit-

This is awesome work, many thanks for this! iso2wdf (moreover wdf) is great

 

[Wiimm]

wwt: Wiimms WBFS Tool v0.05a - 2009-10-14
A command line WBFS tool for linux.

I have updated the tool. The first post with more details is updated too.

I have tested this tool very well and the software do what I want but I can't give any warranty. I have made different import and export (ISO and WDF) tests with about 20 games and the resulting ISO images are absolute identical as the source files.

wwt is now Beta and I'm using it as replacement for the tool 'wbfs'.

Download: http://wiimm.de/x/./wwt-v0.05a-r247.tar.gz (Documentation is included)
More Infos about WDF: http://svn.wiimm.de/open/wii/trunk/wiimms-...WDF.txt?view=co

Last changes:

wwt V0.05a r247 - 2009-10-14

- Complete new interface to the WBFS subsystem:
ÂÂÂÂ- unused source files removed.
ÂÂÂÂ- improved error messages of libwbfs
- WBFS file handling rewritten. It supports now file (using open() function)
ÂÂ and stream (using fopen() function) methods. Use --io to select method.
- New option: --ignore: Ignore non existing files/discs without warnings.
ÂÂ Can be used with: ADD, EXTRACT, REMOVE (replaces --force) and FILETYPE.
- The lib path moved from '/usr/local/lib/' to '/usr/local/share/wwt/'.
- New option --sector-size: Set logical hd sector size larger than 512
ÂÂ while formatting WBFS => The max disc count of a WBSF will be raised.
- Title database implemented.
- New command: TITLES: Print the internal title database to stdout.
- ADD: ISO image is acepted from stdin if a file is named '-'. For pipes
ÂÂ the experimental option --cache must be set.

The --cache option is object to test. If adding a file from pipe fails then please use the binary wwt from 'debug/' folder and add the file again without piping and without --cache. The debug version create a file called '_trace.tmp' which much info. I need this file for analysis.

-----

EDIT:
WARNING: The last version of WWT v0.05a fails if formatting a block device (a real partition).

reason: playing with sector size ...

 

[Wiimm]

Hi Wiimm,

Thanks, I haven't tested it on my box yet, have only just got it to compile. The only way I could compile was to comment out

CFLAGSÂÂÂÂÂÂÂÂ += -mtune=generic

else I would get

cc1: error: unrecognized command line option "-mtune=generic"

This is on a mips cpu

Well, it appears to work - can find wbfs hdd and list games on it, great stuff...The option "-mtune=generic" has now moved to Makefile.local (a local makefile for each users.)

There is a script called scripts/test-ADD+EXTRACT.sh :
Just call it with a list of ISO or WDF files: it will add and extract the image with various options and compare the results. It needs much time ....

QUOTE(Bauldrick @ Oct 14 2009, 02:28 PM) This is awesome work, many thanks for this! iso2wdf (moreover wdf) is great

be carefull: iso2wdf does not scrubbing the image.

 

[Wiimm]

WARNING: The last version of WWT v0.05a fails if formatting a block device (a real partition).

reason: playing with sector size ...