Hacking wwt+wit: Wiimms WBFS+ISO Tools

PsyBlade

Snake Charmer
Member
Joined
Jul 30, 2009
Messages
2,204
Trophies
0
Location
Sol III
XP
458
Country
Gambia, The
source: http://opensvn.wiimm.de/wii/trunk/wiimms-iso-tools/

yes, you are allowed to build it yourself (exact licence terms in gpl-2.0.txt)
signing should be ok too
but afaik apple does not allow open source programs on its app store

edit: apple has a problem with copyleft not open source

copyleft licences such as the GPL require you to give certain rights to your customer
e.g. you would need to grant him the right to distribute/modify the program
apples terms of service aparently do not allow you to grant those rights

see http://www.fsf.org/blogs/licensing/more-about-the-app-store-gpl-enforcement
 

Nickkk

Well-Known Member
Newcomer
Joined
May 25, 2012
Messages
61
Trophies
0
Website
desairem.altervista.org
XP
159
Country
Swaziland
Any idea on how to import the whole source into Xcode? I also noticed that when submitting an app to the Apple Store any executable should not support ppc ... which is included in the universal binaries of wit by now.
 

PsyBlade

Snake Charmer
Member
Joined
Jul 30, 2009
Messages
2,204
Trophies
0
Location
Sol III
XP
458
Country
Gambia, The
any decent ide should be able to import a full directory tree
but I never used xcode no idea what and how it can do

not much need to worry about ppc code
1) If you compile it yourself you can leave it out
2) apple will not accept it anyway
 

Krabbjuice

New Member
Newbie
Joined
Dec 20, 2012
Messages
4
Trophies
0
Age
33
XP
41
Country
Hey. When I try to use Witgui for Mac, this happens when I press the "WBFS Partition" button. What the fuck? Any help? The computer just freezes. I am on Mac OS 10.7.5

LGMb5.jpg
 

Pandamatak

New Member
Newbie
Joined
Jan 1, 2013
Messages
3
Trophies
0
Age
34
XP
51
Country
United States
When I open qtwitgui, what am I supposed to put for my wit path? I think I might have the right one (WIT/wit-console.bat) but the program freezes when I try to open an iso. What am I doing wrong?
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
In the last year, if have implemented the wit command FRAGMENTS to count the fragments of WBFS-Partitons and image files. The image counting will only working for the linux versions using ioctl(FIEMAP) and ioctl(FIBMAP), whatever is available. Now I'm looking for Mac and Windows/Cygwin functions to determine the fragments of a file. Can anyone tell me some resources?
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
In the last year, if have implemented the wit command FRAGMENTS to count the fragments of WBFS-Partitons and image files. The image counting will only working for the linux versions using ioctl(FIEMAP) and ioctl(FIBMAP), whatever is available. Now I'm looking for Mac and Windows/Cygwin functions to determine the fragments of a file. Can anyone tell me some resources?
Status: File mapping for Cygwin is now implemented, but no idea how to implement it for Mac.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
wiimms-tools.png


WIT: Wiimms ISO Tools v2.11a - 2013-02-09
A command line ISO+WBFS tool set for various os.

WIT (Wiimms ISO Tools) is a set of command line tools to manipulate Wii ISO images and WBFS containers. The two main tools are called wit (Wiimms ISO Tool) and wwt (Wiimms WBFS Tool, the WBFS manager).

Feature overview:
Visit http://wit.wiimm.de/features.html for more details.

Download of the current version:

There are also some GUI available:

Change log:

Code:
wit v2.11a r4233 - 2013-02-09

 - Windows version only: Update to Cygwin v1.7.17
 - Bug fix: A wrong TMD size is now detected while scanning image partitions.
 - Command 'wit FRAGMENTS' will now scan and print the filesystem mapping for
   Linux (like before) and Windows (that's new). Mac is still not suported.

The source is published under the license of GPL2. Checkout the sources directly from the SVN repository or browse the source. Visit http://wit.wiimm.de/ for more information.
 

PsyBlade

Snake Charmer
Member
Joined
Jul 30, 2009
Messages
2,204
Trophies
0
Location
Sol III
XP
458
Country
Gambia, The
I have been thinking a bit about wbfs corruption and how to fix it.
The problems I have seen in the last months were
a) the good old multiple games claim the same block (xlinking), mostly of the bad delete variety
and
b) bogous info in later half of the block map, leading to xlinking and claimed blocks being outside the fs


wwts options of xlinking it are either deleting all affected games or a time consumig full checksum check.
My current way is to give all affected games its own copy of the sector.
Neither of those strikes me as really good.

b) wasn't really solved by either


I think the best way of dealling with it would be:
make note of which wbfs blocks are contested or otherwise problematic
calculate the neded game sectors for each game and drop unneed but problematic blocks from the game's blocklist
store the list of needed sectors in problematic blocks for further use
this "rescrubbing" should take care of the bogous info part and might fix some other xlinking too (eg unscrubbed games)

then try to validate the checksums but only for the problematic sectors
games with failing sectors should then either be deleted or somehow marked as bad (naming perhaps?)
blocks that are still contested should be duplicated
(besides the "keep defect games" it might be possible that a block was partially neded by multipe games and still contained all needed sectors)

best of all worlds imho, any thoughts?
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
When I implemented the repairing part of my tools, I had similar thoughts.

Some preliminary ideas:
From 2009 I use only my tools. I will not say, that they are perfect, but I'll think, that they are really good and the best tools for managing WBFS partitions. I have 2 Wiis and use 2 harddisks with WBFS partitions. Both were formatted in 2010. Because of my Custom Mario Kart business I create each week several new images on both discs and I have *never* problems with corrupted images.

Then I know much about the old LIBWBFS. The delete bug was fixed very early. But there are several bugs that were never fixed. Some important are:
  • Wrong calculation of the the number of available WBFS sections in 1/32 while formatting.
  • Wrong calculation of the length of the 'free block table'.
  • Wrong usage of the 'free block table' (rarely, depends on geometry)
  • If disc becomes full while adding a game, LIBWBFS creates and leaves a wrong (random) block mapping for this game. If deleting the game, then a) used blocks can be marked free and b) writing behind the 'free block table' (write anywhere into main memory) because of no index overflow checking.
  • A wrong calculation of needed blocks of a scrubbed image (found by OmegaFrost (if I remember right); very rarely and perhaps only theoretical).
  • In general bad error handling of LIBWBFS.
Because of of these bugs, I have rewritten the LIBWBFS completely.


a) the good old multiple games claim the same block (xlinking), mostly of the bad delete variety
My first solution was the 'delete all' option, because it seems very easy to dump the game again. Later after implementing the VERIFY command, a second method to find bad images were born. I thought about only verify xlinked blocks, but I decline this option because of egoistic motives: I had never problems with it and I won't to spent time for this solution, because "dump the image again" is the solution.

b) bogous info in later half of the block map, leading to xlinking and claimed blocks being outside the fs
I had never seen this kind of bug (because I using only my tools?). But this may be the results of the "add game to full disc-"bug.
 

PsyBlade

Snake Charmer
Member
Joined
Jul 30, 2009
Messages
2,204
Trophies
0
Location
Sol III
XP
458
Country
Gambia, The
I didn't only want to propose that you implement this, I want to implement it too.
I'm thus interested in any kind of problems or improvements.

(Unfortunatly I can't try it out myself atm since I'm still struggling with determining the position and length of the "/sys/" files in the partition
What I found on the net (mostly wiibrew and posts from you) is incomplete and contradictory)
From 2009 I use only my tools. I will not say, that they are perfect, but I'll think, that they are really good and the best tools for managing WBFS partitions. I have 2 Wiis and use 2 harddisks with WBFS partitions. Both were formatted in 2010. Because of my Custom Mario Kart business I create each week several new images on both discs and I have *never* problems with corrupted images.
Unfortunatly one can't choose which the tools users used in the past, the only options are to deal with whatever mess was created or leaving them with it.
I had never seen this kind of bug (because I using only my tools?). But this may be the results of the "add game to full disc-"bug.
No I think ther'e some tools that only write a partial block map is any case.
The number of affected games (around 5-10 per user iirc) seemed a bit high for a full disk imho.
I got the impression that the corruption is limited to the second layer but have no way to verify this.
Maybe they assume that formating/deleting cleared the block map and thus write only up to the last block (or layer) they need and leave the rest as is.
The problems I posted about some months back was exactly that problem.
Iirc this user even had what was clearly a remnant fragment of a fat directory structure in there.
I can provide some wbfs headers which suffered this if you want to take a closer look.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
@PsyBlade
Perhaps you have found another bug in LIBWBFS:

The info data (copy of disk header + file mapping) is alloced by
Code:
    info = wbfs_ioalloc(p->disc_info_sz);
wbfs_ioalloc() must be declared by the tool creator and I remember I had seen simple malloc() calls without clearing the data. But there is a need to use calloc() or memalign() together with memset(). The add_disc() algorithm overwrites only map entries with copied or skipped data, but not the entries behind the last copied block.

That explains your impressions.

So a repair function should determine the last used wbfs block and clear the map behind the related entry. Another way is to calculate the scrubbing (the used blocks) again and clear all unused blocks in the map. Both way reduce raw copies to scrubbed copies.
 

danielko

Member
Newcomer
Joined
Sep 14, 2009
Messages
8
Trophies
0
XP
75
Country
United States
Do you mean automatic renaming using the first game name?
With a 2 disc game the first disc image will be renamed to the game name, the second one will fail with the error message saying the file name already exists. Isn't there a way for wit to tell disc 1 from disc 2, and use it in the names?

Also, gametdb has checksums, do you know how they are calculated? Is it from the raw image dump? Or from the file system contents taken in a specific order? Their FAQ seems to suggest it's the former.

Maybe you could persuade them to have a "disc data content" checksum field that is invariant to transformations like file alignment and scrubbing, and use that as a "verify", making it the de-facto "GoodNGC".
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
wiimms-tools.png


WIT: Wiimms ISO Tools v2.12a - 2013-03-14
A command line ISO+WBFS tool set for various os.

WIT (Wiimms ISO Tools) is a set of command line tools to manipulate Wii ISO images and WBFS containers. The two main tools are called wit (Wiimms ISO Tool) and wwt (Wiimms WBFS Tool, the WBFS manager).

Feature overview:
Visit http://wit.wiimm.de/features.html for more details.

Download of the current version:

There are also some GUI available:

Change log:

Code:
wit v2.12a r4272 - 2013-03-14

 - Bug fix: "wit EDIT" failed on WBFS files since v2.10a (Write privileges
   not longer needed for WBFS files).
 - New command: wwt SCRUB: Scrubs an image and free unused WBFS blocks. It can
   be combined with --psel= to clear unwanted image partitions. This command
   may help to repair wrong block maps.
 - New wfuse option: --allow-other (-O): This option is a short cut for
   '-o allow_other'. It enables re-exporting of the mounted file system for
   example by a samba server.

The source is published under the license of GPL2. Checkout the sources directly from the SVN repository or browse the source. Visit http://wit.wiimm.de/ for more information.
 
  • Like
Reactions: actar

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
  • K3Nv2 @ K3Nv2:
    Bigonya uses his wiener to mod 360s
    +1
  • Xdqwerty @ Xdqwerty:
    Going to the water park, see ya
  • BigOnYa @ BigOnYa:
    You should update the 360 to newest dash before RGHing it yes. But not a big deal if you don't, you can install new dash/avatar updates after. It's just easier to do it auto online before, instead manual offline after.
  • BigOnYa @ BigOnYa:
    Have fun @Xdqwerty. If you see a chocolate candy bar floating in the water, don't eat it!
  • AncientBoi @ AncientBoi:
    :O:ohnoes: Y didn't U Tell ME that ALSO? @BigOnYa :ohnoes: 🤢🤮
    +1
  • BigOnYa @ BigOnYa:
    Does it taste like... chicken?
    +1
  • S @ salazarcosplay:
    @BigOnYa I wanted to ask you about your experience with seeing south park. Most of the people a bit younger like my younger brother and cousins that are a few younger than me that saw kids found south park funny because of the curse words, kids at school, that seemed like liking the show on a very basic level.

    I could not quite have a in depth discussion of the show.

    How was it for you? As an adult. What did you find the most interesting part about it. Did you relate to the parents of the kids and their situations. Was it satires, the commentary on society. The references on celebrities' and pop culture.
    +1
  • BigOnYa @ BigOnYa:
    I remember seeing the very first episode back in the day, and have watched every episode since. I used to set my VCR to record them even, shows how long ago.
  • BigOnYa @ BigOnYa:
    I just like any comedies really, and cartoons have always been a favorite of mine. Family guy, American Dad, Futurama, Cleveland Show, Simpsons - I like them all.
    +1
  • BigOnYa @ BigOnYa:
    South Park is great cause they always touch on relavent issues going on today, and make something funny out of it.
    +3
  • S @ salazarcosplay:
    @BigOnYa were you always up to date on the current events and issues of the time or were there issues that you first found out thru south park
  • BigOnYa @ BigOnYa:
    Most of the time yea I knew, I watch and read the news regularly, but sometimes the Hollywood BS stuff, like concerning actors slip by me. I don't follow most Hollywood BS (example: the Kardasians)
    +2
  • S @ salazarcosplay:
    @BigOnYa there were relevant issues before south park was made, that's why i think a south park prequel/spinoff would be great. Randy and his friends in their child hood
    +1
  • BigOnYa @ BigOnYa:
    Yea, like them running in high school together, getting into stuff, and how they got hitched and had kids. And how the town of South Park was back then compared to now. That would be cool to see.
  • BakerMan @ BakerMan:
    yeah
  • The Real Jdbye @ The Real Jdbye:
    @salazarcosplay if they made a prequel, it would still be about current issues, cause it doesn't make sense to make it about stuff that happened 30 years ago that nobody cares about anymore
  • The Real Jdbye @ The Real Jdbye:
    it's too late
  • The Real Jdbye @ The Real Jdbye:
    the older south park episodes about particular issues usually age poorly since the topic is no longer relevant
  • The Real Jdbye @ The Real Jdbye:
    an exception is giant douche vs turd sandwich, that's always relevant :P
    +1
  • K3Nv2 @ K3Nv2:
    I was gone for like an hour and none of you thought to write or call pos
  • BigOnYa @ BigOnYa:
    We knew you were going to Sonic to get lunch.
  • K3Nv2 @ K3Nv2:
    Sonics fast I would've been home in 10 mins
  • BigOnYa @ BigOnYa:
    Meet and greet with AncientBoi then?
  • K3Nv2 @ K3Nv2:
    That would've gone slow he's old
    +1
    K3Nv2 @ K3Nv2: That would've gone slow he's old +1