wwt+wit: Wiimms WBFS+ISO Tools

Discussion in 'Wii - Hacking' started by Wiimm, Sep 28, 2009.

Sep 28, 2009
    • Member

    PsyBlade Snake Charmer

    Member Since:
    Jul 30, 2009
    Message Count:
    2,204
    Location:
    Sol III
    Country:
    Germany
    source: http://opensvn.wiimm.de/wii/trunk/wiimms-iso-tools/

    yes, you are allowed to build it yourself (exact licence terms in gpl-2.0.txt)
    signing should be ok too
    but afaik apple does not allow open source programs on its app store

    edit: apple has a problem with copyleft not open source

    copyleft licences such as the GPL require you to give certain rights to your customer
    e.g. you would need to grant him the right to distribute/modify the program
    apples terms of service aparently do not allow you to grant those rights

    see http://www.fsf.org/blogs/licensing/more-about-the-app-store-gpl-enforcement
    Last edited by PsyBlade, Dec 18, 2012


    • Newcomer

    Nickkk New Member

    Member Since:
    May 25, 2012
    Message Count:
    40
    Country:
    Switzerland
    Any idea on how to import the whole source into Xcode? I also noticed that when submitting an app to the Apple Store any executable should not support ppc ... which is included in the universal binaries of wit by now.
    • Member

    PsyBlade Snake Charmer

    Member Since:
    Jul 30, 2009
    Message Count:
    2,204
    Location:
    Sol III
    Country:
    Germany
    any decent ide should be able to import a full directory tree
    but I never used xcode no idea what and how it can do

    not much need to worry about ppc code
    1) If you compile it yourself you can leave it out
    2) apple will not accept it anyway
    • Newcomer

    Krabbjuice New Member

    Member Since:
    Dec 20, 2012
    Message Count:
    4
    Country:
    Sweden
    Hey. When I try to use Witgui for Mac, this happens when I press the "WBFS Partition" button. What the fuck? Any help? The computer just freezes. I am on Mac OS 10.7.5

    [IMG]
    • Newcomer

    Nickkk New Member

    Member Since:
    May 25, 2012
    Message Count:
    40
    Country:
    Switzerland
    But wait, here it seems like you already got the list of WBFS partitions ... so the computer froze after pressing OK?
    • Newcomer

    Pandamatak New Member

    Member Since:
    Jan 1, 2013
    Message Count:
    3
    Country:
    United States
    When I open qtwitgui, what am I supposed to put for my wit path? I think I might have the right one (WIT/wit-console.bat) but the program freezes when I try to open an iso. What am I doing wrong?
    • Newcomer

    BeastOfEden New Member

    Member Since:
    Jan 6, 2013
    Message Count:
    2
    Location:
    Austin, Texas
    Country:
    United States
    ....Nevermind.

    I figured out how to solve my issues..

    I'm retarded sometimes. haha
    Last edited by BeastOfEden, Jan 6, 2013
    • Member

    Wiimm Developer

    Member Since:
    Aug 11, 2009
    Message Count:
    1,809
    Location:
    Germany
    Country:
    Germany
    In the last year, if have implemented the wit command FRAGMENTS to count the fragments of WBFS-Partitons and image files. The image counting will only working for the linux versions using ioctl(FIEMAP) and ioctl(FIBMAP), whatever is available. Now I'm looking for Mac and Windows/Cygwin functions to determine the fragments of a file. Can anyone tell me some resources?
    • Member

    Wiimm Developer

    Member Since:
    Aug 11, 2009
    Message Count:
    1,809
    Location:
    Germany
    Country:
    Germany
    Status: File mapping for Cygwin is now implemented, but no idea how to implement it for Mac.
    • Member

    Wiimm Developer

    Member Since:
    Aug 11, 2009
    Message Count:
    1,809
    Location:
    Germany
    Country:
    Germany
    [IMG]

    WIT: Wiimms ISO Tools v2.11a - 2013-02-09
    A command line ISO+WBFS tool set for various os.

    WIT (Wiimms ISO Tools) is a set of command line tools to manipulate Wii ISO images and WBFS containers. The two main tools are called wit (Wiimms ISO Tool) and wwt (Wiimms WBFS Tool, the WBFS manager).

    Feature overview:
    Visit http://wit.wiimm.de/features.html for more details.

    Download of the current version:

    There are also some GUI available:

    Change log:

    Code:
    wit v2.11a r4233 - 2013-02-09
    
     - Windows version only: Update to Cygwin v1.7.17
     - Bug fix: A wrong TMD size is now detected while scanning image partitions.
     - Command 'wit FRAGMENTS' will now scan and print the filesystem mapping for
       Linux (like before) and Windows (that's new). Mac is still not suported.
    
    
    The source is published under the license of GPL2. Checkout the sources directly from the SVN repository or browse the source. Visit http://wit.wiimm.de/ for more information.
    • Member

    PsyBlade Snake Charmer

    Member Since:
    Jul 30, 2009
    Message Count:
    2,204
    Location:
    Sol III
    Country:
    Germany
    I have been thinking a bit about wbfs corruption and how to fix it.
    The problems I have seen in the last months were
    a) the good old multiple games claim the same block (xlinking), mostly of the bad delete variety
    and
    b) bogous info in later half of the block map, leading to xlinking and claimed blocks being outside the fs


    wwts options of xlinking it are either deleting all affected games or a time consumig full checksum check.
    My current way is to give all affected games its own copy of the sector.
    Neither of those strikes me as really good.

    b) wasn't really solved by either


    I think the best way of dealling with it would be:
    make note of which wbfs blocks are contested or otherwise problematic
    calculate the neded game sectors for each game and drop unneed but problematic blocks from the game's blocklist
    store the list of needed sectors in problematic blocks for further use
    this "rescrubbing" should take care of the bogous info part and might fix some other xlinking too (eg unscrubbed games)

    then try to validate the checksums but only for the problematic sectors
    games with failing sectors should then either be deleted or somehow marked as bad (naming perhaps?)
    blocks that are still contested should be duplicated
    (besides the "keep defect games" it might be possible that a block was partially neded by multipe games and still contained all needed sectors)

    best of all worlds imho, any thoughts?
    • Member

    Wiimm Developer

    Member Since:
    Aug 11, 2009
    Message Count:
    1,809
    Location:
    Germany
    Country:
    Germany
    When I implemented the repairing part of my tools, I had similar thoughts.

    Some preliminary ideas:
    From 2009 I use only my tools. I will not say, that they are perfect, but I'll think, that they are really good and the best tools for managing WBFS partitions. I have 2 Wiis and use 2 harddisks with WBFS partitions. Both were formatted in 2010. Because of my Custom Mario Kart business I create each week several new images on both discs and I have *never* problems with corrupted images.

    Then I know much about the old LIBWBFS. The delete bug was fixed very early. But there are several bugs that were never fixed. Some important are:
    • Wrong calculation of the the number of available WBFS sections in 1/32 while formatting.
    • Wrong calculation of the length of the 'free block table'.
    • Wrong usage of the 'free block table' (rarely, depends on geometry)
    • If disc becomes full while adding a game, LIBWBFS creates and leaves a wrong (random) block mapping for this game. If deleting the game, then a) used blocks can be marked free and b) writing behind the 'free block table' (write anywhere into main memory) because of no index overflow checking.
    • A wrong calculation of needed blocks of a scrubbed image (found by OmegaFrost (if I remember right); very rarely and perhaps only theoretical).
    • In general bad error handling of LIBWBFS.
    Because of of these bugs, I have rewritten the LIBWBFS completely.


    My first solution was the 'delete all' option, because it seems very easy to dump the game again. Later after implementing the VERIFY command, a second method to find bad images were born. I thought about only verify xlinked blocks, but I decline this option because of egoistic motives: I had never problems with it and I won't to spent time for this solution, because "dump the image again" is the solution.

    I had never seen this kind of bug (because I using only my tools?). But this may be the results of the "add game to full disc-"bug.
    • Member

    PsyBlade Snake Charmer

    Member Since:
    Jul 30, 2009
    Message Count:
    2,204
    Location:
    Sol III
    Country:
    Germany
    I didn't only want to propose that you implement this, I want to implement it too.
    I'm thus interested in any kind of problems or improvements.

    (Unfortunatly I can't try it out myself atm since I'm still struggling with determining the position and length of the "/sys/" files in the partition
    What I found on the net (mostly wiibrew and posts from you) is incomplete and contradictory)
    Unfortunatly one can't choose which the tools users used in the past, the only options are to deal with whatever mess was created or leaving them with it.
    No I think ther'e some tools that only write a partial block map is any case.
    The number of affected games (around 5-10 per user iirc) seemed a bit high for a full disk imho.
    I got the impression that the corruption is limited to the second layer but have no way to verify this.
    Maybe they assume that formating/deleting cleared the block map and thus write only up to the last block (or layer) they need and leave the rest as is.
    The problems I posted about some months back was exactly that problem.
    Iirc this user even had what was clearly a remnant fragment of a fat directory structure in there.
    I can provide some wbfs headers which suffered this if you want to take a closer look.
    • Member

    Wiimm Developer

    Member Since:
    Aug 11, 2009
    Message Count:
    1,809
    Location:
    Germany
    Country:
    Germany
    @PsyBlade
    Perhaps you have found another bug in LIBWBFS:

    The info data (copy of disk header + file mapping) is alloced by
    Code:
        info = wbfs_ioalloc(p->disc_info_sz);
    
    wbfs_ioalloc() must be declared by the tool creator and I remember I had seen simple malloc() calls without clearing the data. But there is a need to use calloc() or memalign() together with memset(). The add_disc() algorithm overwrites only map entries with copied or skipped data, but not the entries behind the last copied block.

    That explains your impressions.

    So a repair function should determine the last used wbfs block and clear the map behind the related entry. Another way is to calculate the scrubbing (the used blocks) again and clear all unused blocks in the map. Both way reduce raw copies to scrubbed copies.
    • Member

    Wiimm Developer

    Member Since:
    Aug 11, 2009
    Message Count:
    1,809
    Location:
    Germany
    Country:
    Germany
    That was easy to implement, the new command "wwt SCRUB" does exactly this:
    * http://wit.wiimm.de/changelog.html#r0
    * http://wit.wiimm.de/cmd/wwt/scrub
    • Newcomer

    danielko New Member

    Member Since:
    Sep 14, 2009
    Message Count:
    8
    Country:
    United States
    Any chance of getting "wit verify" to work on GC images?

    And getting "wit mv" to work on multi-disc GC games?
    • Member

    Wiimm Developer

    Member Since:
    Aug 11, 2009
    Message Count:
    1,809
    Location:
    Germany
    Country:
    Germany
    NO, because there are no checksums to verify.

    Do you mean automatic renaming using the first game name?
    • Newcomer

    danielko New Member

    Member Since:
    Sep 14, 2009
    Message Count:
    8
    Country:
    United States
    With a 2 disc game the first disc image will be renamed to the game name, the second one will fail with the error message saying the file name already exists. Isn't there a way for wit to tell disc 1 from disc 2, and use it in the names?

    Also, gametdb has checksums, do you know how they are calculated? Is it from the raw image dump? Or from the file system contents taken in a specific order? Their FAQ seems to suggest it's the former.

    Maybe you could persuade them to have a "disc data content" checksum field that is invariant to transformations like file alignment and scrubbing, and use that as a "verify", making it the de-facto "GoodNGC".
    Last edited by danielko, Feb 21, 2013
    • Member

    Wiimm Developer

    Member Since:
    Aug 11, 2009
    Message Count:
    1,809
    Location:
    Germany
    Country:
    Germany
    Sorry, but my interests in doing gamecube stuff are minimal and my gamecube image knowledge is minimal.
    Last edited by Wiimm, Feb 24, 2013. Reason: typo
    megazig likes this.
    • Member

    Wiimm Developer

    Member Since:
    Aug 11, 2009
    Message Count:
    1,809
    Location:
    Germany
    Country:
    Germany
    [IMG]

    WIT: Wiimms ISO Tools v2.12a - 2013-03-14
    A command line ISO+WBFS tool set for various os.

    WIT (Wiimms ISO Tools) is a set of command line tools to manipulate Wii ISO images and WBFS containers. The two main tools are called wit (Wiimms ISO Tool) and wwt (Wiimms WBFS Tool, the WBFS manager).

    Feature overview:
    Visit http://wit.wiimm.de/features.html for more details.

    Download of the current version:

    There are also some GUI available:

    Change log:

    Code:
    wit v2.12a r4272 - 2013-03-14
    
     - Bug fix: "wit EDIT" failed on WBFS files since v2.10a (Write privileges
       not longer needed for WBFS files).
     - New command: wwt SCRUB: Scrubs an image and free unused WBFS blocks. It can
       be combined with --psel= to clear unwanted image partitions. This command
       may help to repair wrong block maps.
     - New wfuse option: --allow-other (-O): This option is a short cut for
       '-o allow_other'. It enables re-exporting of the mounted file system for
       example by a samba server.
    
    
    
    The source is published under the license of GPL2. Checkout the sources directly from the SVN repository or browse the source. Visit http://wit.wiimm.de/ for more information.
    actar likes this.

SPONSORED LINKS
 

Share This Page