Hacking wwt+wit: Wiimms WBFS+ISO Tools

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
Skizzo said:
BTW, is there any type of timestamp within WBFS where we could tell when the games were added? Just wondering if I could see how long it was up and running before the reboot.
I think about an extension in LIBWBFS: In each inode there is a copy of the first 256 bytes of the ISO. But only the first 128 bytes are used to mange WBFS. My idea: use the other 128 for some statistics like a timestamp and action counter. This makes repairing, undelete and recovering much easier. It's only a idea ...
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
wwt: Wiimms WBFS Tool v0.29a - 2010-01-22
A command line WBFS tool set for various os.


I have updated the tool. The first post with more details is updated too.

Download:
Last changes:
Code:
wwt v0.28a r596 - 2010-01-15

- Little optimation for libwbfs add_disc(): Read only needed wii sectors from
ÂÂ iso and fill others with zeros. This makes adding faster and avoids reading
ÂÂ behind end of file. This is needed for reading from a pipe.
- New error code: JOB_IGNORED: Set if a disc is ignored while adding a game.
- wwt ADD: adding to more than one WBFS failed since v0.27a.
- wit COPY+SCRUB+MOVE: Process files in alphabetic order and print iteration
ÂÂ counter.
- Bug fix: wwt LIST: Not all discs were shown if an invalid disc was found.
- Decompose unusual unicode characters (remove accents...) in title database.

wwt v0.29a r606 - 2010-01-22

- wwt ADD: Adding a complete WBFS is optimized and much faster now.
- Bug fix: Option --preserve didn't work.
- wwt+wit FILETYPE: Print reagion info if option --long is set twice.

Sources comes with GPL2. Checkout the sources directly from the SVN repository or browse the source.
 

zon7

Active Member
Newcomer
Joined
Oct 1, 2005
Messages
38
Trophies
0
XP
96
Country
Wiimm said:
Do yo mean the wwt messages like "PAL", "NTSC", "GERM", "RF"?
It's just a explanation of the fourth character of the ID. See my SVN and wiibrew.org for more info. But it is no problem to expand the output.

And if you like I will create a special option named --gui: It tells the progs that they are called by a gui and that they should print machine readable output.

The format is not as important, but I'll love to know how much space a iso game takes in wbfs/wdf. I don't know if it's possible.
BTW, a --gui option with tabbed output for each field will be great.
Hope to have the app finished this week
 

jeremy2491

Member
Newcomer
Joined
Jun 25, 2007
Messages
14
Trophies
0
XP
73
Country
Canada
Code:
CHECK /dev/sdb1


* Summary of WBFS Check: 792 errors found:
ÂÂ 247 used WBFS sectors marked as free!
ÂÂÂÂ13 free WBFS sectors marked as used!
ÂÂÂÂÂÂ Note: 13 errors are based on a bug in libwbfs v0.
ÂÂ 532 WBFS sectors are used by 2 or more discs!
ÂÂ Total: 2 discs are invalid!

how repair my WBFS?
 

angus528

Member
Newcomer
Joined
Aug 24, 2009
Messages
16
Trophies
0
XP
81
Country
United States
Just wondering if I am using the most efficient command to update drives. I have two drives and to update i use 'wwt -p /dev/sdb1 add /dev/sdc1 -u -v -P' but i just saw a post of someone using 'wwt UPDATE -p /dev/sdc2 /dev/sdb2 -vv'
Which is the most efficient or correct way to do it?
Also, the speed really tends to vary, sometimes i get ~8.5MiB per sec and other times i can get up to ~17MiB. Is there any cause for this or any way i can ensure i consistently get the higher speed for copying?

Thanks!
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
zon7 said:
WiiMM, can you tell me how to obtain the wbfs or wbf size of a iso file without making the file?
It's no possible yet. But it is perhaps a feature request for something like "wit STAT" (count the number of used disc sectors)
 

zon7

Active Member
Newcomer
Joined
Oct 1, 2005
Messages
38
Trophies
0
XP
96
Country
Wiimm said:
zon7 said:
WiiMM, can you tell me how to obtain the wbfs or wbf size of a iso file without making the file?
It's no possible yet. But it is perhaps a feature request for something like "wit STAT" (count the number of used disc sectors)

Ok. Thanks. I'll try without it
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
wwt: Wiimms WBFS Tool v0.30a - 2010-01-30
A command line WBFS tool set for various os.


I have updated the tool. The first post with more details is updated too.

Download:Highlights/bug fixes of this release:
  • The FILETYPE command of wwt and wit shows the scrubbed size of a ISO image too if the option --long is set at least twice.

    The calculate size is printed in MiB (like almost all size infos) and depends of the options --psel and --raw. This has a nice side effect: To find out the size of the update partition just use --psel=update.
    .
  • The new command wit ISOSIZE calculate ISO sizes in up to 4 aspects:
    • Number of used ISO blocks. (ISO block=32k)
    • Size in MiB.
    • Size of a wbfs file with a single ISO (assume WBFS block size = 2M)
    • Size of a ISO as part of a WBFS with 500g (assume WBFS block size = 8M)
    The calculate size depends of the options --psel and --raw. This has a nice side effect: To find out the size of the update partition just use --psel=update.
    .
Last changes:
Code:
wwt v0.29a r606 - 2010-01-22

- wwt ADD: Adding a complete WBFS is optimized and much faster now.
- Bug fix: Option --preserve didn't work.
- wwt+wit FILETYPE: Print region info if option --long is set twice.

wwt v0.30a r613 - 2010-01-30

- wwt+wit FILETYPE: Print scrubbed size if option --long is set twice.
- New command: wit ISOSIZE: Print scrubbed ISO sizes in up to 4 aspects.

Sources comes with GPL2. Checkout the sources directly from the SVN repository or browse the source.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
wwt: Wiimms WBFS Tool v0.31a - 2010-01-31
A command line WBFS tool set for various os.


I have updated the tool. The first post with more details is updated too.

Download:Highlights/bug fixes of this release:
  • "wwt FORMAT" has a new option: --recover. If this option is set, the WBFS will be formatted in recover mode:
    • Only the header of the WBFS is written. The inodes are not cleared.
    • All discs slots are marked as used.
    • All WBFS blocks are marked as used.
    • A silent check and repair is done:
      • Remove discs with invalid magic or without ID.
      • Free unused blocks in the free blocks table.
    • A verbose check is done (like "wwt check --verbose") to find and report other errors.
    .
Last changes:
Code:
wwt v0.30a r613 - 2010-01-30

- wwt+wit FILETYPE: Print scrubbed size if option --long is set twice.
- New command: wit ISOSIZE: Print scrubbed ISO sizes in up to 4 aspects.

wwt v0.31a r618 - 2010-01-31

- wwt+wit FILETYPE: Print table header.
- wwt FORMAT: new option: --recover: Format a WBFS in recover mode.
ÂÂ The recover feature is EXPERIMENTAL!

Sources comes with GPL2. Checkout the sources directly from the SVN repository or browse the source.
 

zon7

Active Member
Newcomer
Joined
Oct 1, 2005
Messages
38
Trophies
0
XP
96
Country
Wiimm said:
wwt: Wiimms WBFS Tool v0.30a - 2010-01-30
A command line WBFS tool set for various os.


I have updated the tool. The first post with more details is updated too.

Download:Highlights/bug fixes of this release:
  • The FILETYPE command of wwt and wit shows the scrubbed size of a ISO image too if the option --long is set at least twice.

    The calculate size is printed in MiB (like almost all size infos) and depends of the options --psel and --raw. This has a nice side effect: To find out the size of the update partition just use --psel=update.
    .
  • The new command wit ISOSIZE calculate ISO sizes in up to 4 aspects:
    • Number of used ISO blocks. (ISO block=32k)
    • Size in MiB.
    • Size of a wbfs file with a single ISO (assume WBFS block size = 2M)
    • Size of a ISO as part of a WBFS with 500g (assume WBFS block size = 8M)
    The calculate size depends of the options --psel and --raw. This has a nice side effect: To find out the size of the update partition just use --psel=update.
    .
Last changes:
Code:
wwt v0.29a r606 - 2010-01-22

- wwt ADD: Adding a complete WBFS is optimized and much faster now.
- Bug fix: Option --preserve didn't work.
- wwt+wit FILETYPE: Print region info if option --long is set twice.

wwt v0.30a r613 - 2010-01-30

- wwt+wit FILETYPE: Print scrubbed size if option --long is set twice.
- New command: wit ISOSIZE: Print scrubbed ISO sizes in up to 4 aspects.

Sources comes with GPL2. Checkout the sources directly from the SVN repository or browse the source.

Thanks a lot!
 

pudquick

Member
Newcomer
Joined
Jun 16, 2008
Messages
10
Trophies
0
XP
165
Country
United States

Hi Wiimm - I'll probably just PM you this, but wanted to toss it out there.

I downloaded and built from the SVN, r606. My system: Mac OS X 10.5.8, Intel Core2Duo MacBook Pro

No problems configuring or building! Ran make distrib, got a nice pre-packaged "wwt-v0.29a-r606-mac" ouput.

But when I attempt to run 'iso2wbfs', got some funny output:

mbp:bin myself$ time ./iso2wbfs -v geowars.iso

iso2wbfs v0.29a r606 mac - Dirk Clemens - 2010-01-28

* iso2wbfs: SCRUB geowars.iso -> RGLP7D.wbfs
!! iso2wbfs: ERROR #17 [READ FILE FAILED] in WrapperReadSector() @ lib-std.c#2764
!! Read failed [F=4,200+2600]: .RGLP7D.wbfs.3bvAAg
!! -> Bad file descriptor
363 MiB copied in 0:00, 1138.2 MiB/sec

real 0m12.490s
user 0m0.013s
sys 0m0.737s

While watching what the program does, it creates the ".RGLP7D.wbfs.3bvAAg" temp file, which in this case is 385,875,968 bytes, spends a little time, then moves the resulting file to "RGLP7D.wbfs".

However, the output file is blank / all zeros. Given that the app thinks it took 0:00 to do the copying but time says otherwise, I'm guessing it's erroring out somewhere before any real processing beyond measuring the size of the game partition. The "RGLP7D.wbfs" file output is also 385,875,968 bytes.

For comparison, I ran the same ISO through wbfs_file (v 2.8):

mbp:bin myself$ wbfs_file geowars.iso
Converting geowars.iso to .wbfs
Writing: gwgalaxieswii [RGLP7D]/RGLP7D.wbfs
adding RGLP7D gwgalaxieswii...
100.00% (-) ETA: 0:00:00 (366.00MB of 366.00MB ~ 89.75MB/s) time: 4.08s
Done in 0:00:04

In this case, the resulting output file is also 385,875,968 bytes in size - but has actual data in it (starts with 'WBFS', etc.).

... I was wondering if you had any ideas about what was going on?

OS X is POSIX compliant, so I would assume your file access methods should work. I did notice the test/debug 'io' parameter that looks like it could be enabled, but haven't gotten far enough around to tinkering with it yet.

The whole reason I was interested in your tool over wbfs_file is support for stdin as an input. This would allow be to do things like:
rar p -inul whatever.rar *.iso | iso2wbfs -
... and skip generating these useless 4.5GB .iso files that I just delete after it's in wbfs form.

Any help much appreciated!

Thank you so much for your work!
 

pudquick

Member
Newcomer
Joined
Jun 16, 2008
Messages
10
Trophies
0
XP
165
Country
United States
Wiimm said:
@pudquick
Please try for a test: ./wit copy -v geowars.iso RGLP7D.wbfs

To comapre the conmtents you can use: ./wit diff -v geowars.iso RGLP7D.wbfs

That did, indeed, work!

However - it looks like my planned use for your tool won't work.

I was excited about the prospect of using stdin to avoid creation of the .iso file from .rar extraction straight into your tool.

However, looking at your documentation, you've got examples like "wit copy -v - output.wbfs < (rar x whatever.rar)", and the < redirect is for files. Its not a pipe / stream, it's just a file handler.

When I tried doing: cat geowars.iso | wit copy -v - output.wbfs, I got an error telling me it couldn't seek on stdin - which only makes sense, because you can't seek on a pipe like that.

What's the reasoning for the seeking? Is it just to find the length of the .iso? If so, can that instead be passed as a parameter (I can get file length from the rar list information - plus Wii .iso files are generally all the same length anyways - 4699979776 bytes)? Or is seeking / going back and forth necessary during parsing to create the .wbfs?

Again - thanks for all your work on this wonderful utility.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
@pudquick
LIBWBFS needs random access to the ISO. But most of the data is readed in strict order. When reading from a pipe i will cache data that libwbfs will access randomly. Perhaps it is a missing area in the cache.

Please try: make clean debug all
... and run again. The debug option enables creating a trace file called '_trace_wit.tmp" I need this file for analyses.

EDIT:
pudquick said:
When I tried doing: cat geowars.iso | wit copy -v - output.wbfs, I got an error telling me it couldn't seek on stdin - which only makes sense, because you can't seek on a pipe like that.
What is the exact error message?
 

pudquick

Member
Newcomer
Joined
Jun 16, 2008
Messages
10
Trophies
0
XP
165
Country
United States
Wiimm said:
What is the exact error message?
Just downloaded and built your r618, with debug, per your suggestion.

Here's the error output:
cat geowars.iso | ./wit copy -v - output.wbfs

wit: Wiimms ISO Tool v0.31a r618 mac - Dirk Clemens - 2010-02-04

* wit SCRUB 1/1 ISO:- (stdin) -> WBFS:output.wbfs

***********************************************************************
***** It seems, that the caching area for the game is to small! *****
***** Please report this to the author. *****
***** Technical data: ID=RGLP7D OFF= 40020 *****
***********************************************************************

!! wit: ERROR #17 [READ FILE FAILED] in ReadSF() @ lib-sf.c#417
!! Seek failed [F=4,40020]: - (stdin)
!! -> Illegal seek
!! wit: ERROR #28 [FATAL ERROR] in disc_read() @ libwbfs/wiidisc.c#47
!! error reading disc (disc_read)
(Just a note: You've got a typo there - it should be "for the game is too small")

Here's the debug trace file: http://dl.dropbox.com/u/514357/_trace-wit.tmp
And here's the partial ~2MB ".output.wbfs.q7Kq9v" temp file it built: http://dl.dropbox.com/u/514357/output.wbfs.q7Kq9v

(Thanks again for all this)
 

pudquick

Member
Newcomer
Joined
Jun 16, 2008
Messages
10
Trophies
0
XP
165
Country
United States
Wiimm said:
Bug found:
I have forgotten to enable the cache for "wit COPY".

EDIT: iso2wbfs works too!

Glad to hear it! Let me know when the commit is up and I'll gladly test it
biggrin.gif
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
  • SylverReZ @ SylverReZ:
    Communism lol
  • SylverReZ @ SylverReZ:
    OUR products
  • The Real Jdbye @ The Real Jdbye:
    @LeoTCK actually good quality products are dying out because they can't compete with dropshipped chinese crap
    +2
  • BakerMan @ BakerMan:
    @LeoTCK is your partner the sascrotch or smth?
  • Xdqwerty @ Xdqwerty:
    Good morning
  • Xdqwerty @ Xdqwerty:
    Out of nowhere I got several scars on my forearm and part of my arm and it really itches.
  • AdRoz78 @ AdRoz78:
    Hey, I bought a modchip today and it says "New 2040plus" in the top left corner. Is this a legit chip or was I scammed?
  • Veho @ Veho:
    @AdRoz78 start a thread and post a photo of the chip.
    +2
  • Xdqwerty @ Xdqwerty:
    Yawn
  • S @ salazarcosplay:
    and good morning everyone
    +1
  • K3Nv2 @ K3Nv2:
    @BakerMan, his partner is Luke
  • Sicklyboy @ Sicklyboy:
    Sup nerds
    +1
  • Flame @ Flame:
    oh hi, Sickly
  • K3Nv2 @ K3Nv2:
    Oh hi flame
  • S @ salazarcosplay:
    @K3Nv2 what was your ps4 situation
  • S @ salazarcosplay:
    did you always have a ps4 you never updated
  • S @ salazarcosplay:
    or were you able to get new ps4 tracking it \
    as soon as the hack was announced
  • S @ salazarcosplay:
    or did you have to find a used one with the lower firm ware that was not updated
  • K3Nv2 @ K3Nv2:
    I got this ps4 at launch and never updated since 9.0
  • K3Nv2 @ K3Nv2:
    You got a good chance of buying a used one and asking the seller how often they used or even ask for a Pic of fw and telling them not to update
  • RedColoredStars @ RedColoredStars:
    Speaking of PLaystation. I see Evilnat put out a beta for PS3 CFW 4.91.2 on the 22nd.
  • K3Nv2 @ K3Nv2:
    Don't really see the point in updating it tbh
    K3Nv2 @ K3Nv2: Don't really see the point in updating it tbh