wwt+wit: Wiimms WBFS+ISO Tools

Discussion in 'Wii - Hacking' started by Wiimm, Sep 28, 2009.

  1. Wiiwu

    Wiiwu GBAtemp Regular

    Member
    209
    3
    Jan 21, 2009
    Hong Kong
    Asia
    Read your post in the Configurable USB Loader thread that your tool can handle .wbfs files in fat partition.

    Is there some example how-to commands specifically on the wbfs file management?

    And if this can convert .iso to .wbfs, can add a feature to detect if partition is fat and if so, turn on automatic file splitting?
     


  2. Wiimm
    OP

    Wiimm Developer

    Member
    2,156
    368
    Aug 11, 2009
    Gambia, The
    Germany
    That's an nice idea. But I don't know how to detect the maximal files size on a file system.
     
  3. Wiimm
    OP

    Wiimm Developer

    Member
    2,156
    368
    Aug 11, 2009
    Gambia, The
    Germany
    wwt: Wiimms WBFS Tool v0.11a - 2009-11-02
    A command line WBFS tool set for various os.

    I have updated the tool. The first post with more details is updated too.

    WARNING: I have only done a few tests with the new scrubbing features (tools wit, wdf2iso, iso2wdf). It seems that scrubbing works.

    Download:Highlights of this release:
    • Scrubbing is enabled in three tools:
      • wdf2iso and iso2wdf do scrubbing by default if the source is an iso. Both can read ISO and WDF. Other files are copied/converted directly.
      • wit has a new command: COPY|CP: Is copies one or much ISO files to a destination. While copying the files are converted from/to ISO/WBFS and scrubbed. A raw copy is also available.
    • wwt DUMP: The DUMP command dumps now additional the memory layout of each disc (-ll) and of the whole WBFS (-lll). These dumps are good for controlling and debugging the WBFS.
    Last changes:
    Code:
    wwt v0.11a r339 - 2009-11-02
    
    - wwt "LIST --mixed": output format changed.
    - New option --progress: enable progress counter independend of verbose level.
    - wdf2iso + iso2wdf:
    ÂÂÂÂ- Both accept any files as input. If the input is an ISO or a WDF+ISO than
    ÂÂÂÂÂÂthe file is scrubbed (controlled by optiond --psel snd --raw).
    ÂÂÂÂ- New option --preserve: preserves file times
    - New command: wit COPY source... dest -> converting and scrubbing.
    - wwt DUMP --long --long: Dump a memory layout map for each disc.
    - wwt DUMP --long --long --long: Dump a memory layout map for whole WBFS.
    Sources comes with GPL2. Checkout the sources directly from the SVN repository or browse the source with any browser.
     
  4. Wiimm
    OP

    Wiimm Developer

    Member
    2,156
    368
    Aug 11, 2009
    Gambia, The
    Germany
    BUG WARNING

    I have found bugs in the libwbfs function wbfs_trim() (details).

    Please do not use:
    • wwt command ADD together with option --trunc
    • wwt command TRUNCATE.
    The bug will be fixed in the next release.
    The command "wwt dump -lll wbfs_file" can be used for controlling wbfs files.

    EDIT:
    Bug 1. does not appear in WBFS files without any removed or overwritten discs.
    Bug 2. does not appear if the WBFS has at least one free block.
     
  5. Blue-K

    Blue-K No right of appeal.

    Member
    2,572
    1
    Jun 21, 2008
    Swaziland
    Helvetica
    Gosh...this Tool is getting better and better...and I'm still too stupid to use it [​IMG]. I hope cojiro will soon find time and make a nice wrapper...

    Anyways, thanks Wiimm for your Efforts, realy appriciated!
    Dankeschön [​IMG]!
     
  6. mousex

    mousex GBAtemp Advanced Fan

    Member
    987
    0
    Jan 23, 2009
    United States
    Fow which system do you need the GUI? On linux (Gnome) this would be very easy to do (either Nautilus scripts or there was another toolkit that I used about one year ago that was perfect for this task, I just don't remember the name at the moment. It was just one programm that you called with different parameters to show a list/yes-no question, status bar and so on) with a nice looking GUI.
    And well, on Windows, we saw how easily many people can make ugly looking GUIs and that only a few now how to make a nice working GUI [​IMG]
    The fastet for you is learning how to use the tool (if you are really using Linux this shouldn't be too hard) and then writing a small Wrapper.
     
  7. Blue-K

    Blue-K No right of appeal.

    Member
    2,572
    1
    Jun 21, 2008
    Swaziland
    Helvetica
    Yeah, I threw Windows away and replaced it with Ubuntu, that's why I need a good Linux Manager [​IMG]. I first had a look at Glade, with which it would be realy easy to make a nice GUI (I guess that's the App you've meant). But after some tries, I guess I wouldn't be able to code the Commands in. I then had a look at Zenity, and modified an existing Script. Here's the result:
    Warning: Spoilers inside!
    This one is a realy simple dirty Script, which can do one simple task: Add an ISO to a WBFS Drive (using the -a add command). I made this basicly for me, because I don't realy like the Terminal. It works for shure on Ubuntu, but should also work on other *nix Systems if you have zenity installed. You'll need to have wwt.bin correctly installed/placed (using the given Script in the official download). Download it, unpack it, right click on the file>Properties>Permissions>Check Allow executing file as program. Then simply double klick it and select run.

    I might have now a closer look on Zenity...or use Nautilus Scripts..at least I can now finaly add ISOs with all partitions to my Drive [​IMG]. YES!
     
  8. Skizzo

    Skizzo Banned

    Banned
    475
    0
    May 1, 2009
    United States
    Wiimm, is it true that WBFS does not support fragmented files? There is no fragmentation on a WBFS drive? I figured you'd be one of the ones to actually know the facts about this. Thanks for any info.
     
  9. redia

    redia GBAtemp Fan

    Member
    371
    1
    Mar 9, 2008
    Swaziland
    hello,

    am I wrong or there is no progress bar/counter in these tools ?
    sorry but I can't seem to find it.

    Cheers,
    R
     
  10. oggzee

    oggzee GBAtemp Addict

    Member
    2,335
    16
    Apr 11, 2009
    Slovenia
    Who says there's no fragmentation on WBFS? Afaik it can occur, however the block size is large and you're dealing with huge 'files' so the negative effects of fragmentation will not show up. In other words, if you were adding and removing random games with different sizes, in the end they will be fragmented, but the speed difference should not be noticable because of the large block size.
     
  11. Elfish

    Elfish GBAtemp Fan

    Member
    447
    3
    Sep 20, 2004
    Gambia, The
    i was wondering if u could u add a partition resize function?
     
  12. W hat

    W hat Rhythm Heaven Fan

    Member
    616
    110
    Feb 28, 2007
    United States
    I would love a Windows GUI. It can look like total crap, function over form always.
     
  13. Bauldrick

    Bauldrick GBAtemp Regular

    Member
    103
    1
    Jan 22, 2009
    @ridia

     
  14. Skizzo

    Skizzo Banned

    Banned
    475
    0
    May 1, 2009
    United States
    Unless I'm mistaken, that's exactly what some wank stain by the name of D34DL1N3R (and who can argue with someone with a l33t name like that?) is arguing (no fragmentation) here http://gbatemp.net/index.php?showtopic=188...t&p=2367727
     
  15. Wiiwu

    Wiiwu GBAtemp Regular

    Member
    209
    3
    Jan 21, 2009
    Hong Kong
    Asia
    If anyone need to resize a WBFS partition, then take this as a reason to move all your games onto a FAT partition and in the future with FAT, you can resize it anytime you like, with any tools you like.

    Also any filesystem will have fragmentation, its just how aggressive the OS handles it and its always a balance with fragmenting files or fragmenting free space, eventually when low on space it will still fragment the files. But if fragmentation is still such a concern, then defrag it with the many defragging tool available.
     
  16. redia

    redia GBAtemp Fan

    Member
    371
    1
    Mar 9, 2008
    Swaziland
    damn... how did I miss that.
    sorry & thanks Bauldrick

    R
     
  17. Skizzo

    Skizzo Banned

    Banned
    475
    0
    May 1, 2009
    United States
    As I was trying to say, over in the thread I linked to, it appears the individual mentioned is saying that WBFS DOES NOT allow games to be fragmented. In other words, if I only have 4GB of free space left on my WBFS partition, but the free space comes from games that have been deleted over time, and thus the free space isn't contiguous (or 'consolidated') (say I have 1GB free, 15GB used by other games, 2.5GB free, 25GB used by games, and finally .5GB free) then is it true that the largest single game I can now install is however large the biggest chunk of free space is, or 2.5GB in the example above? Or could a game that is say 3.5GB in size still be installed, splitting the game into the first two chunks of free space in the example?
     
  18. oggzee

    oggzee GBAtemp Addict

    Member
    2,335
    16
    Apr 11, 2009
    Slovenia
    I believe I already answered that, why the need to drag this further?
    And wouldn't the thread 'Inside WBFS' be more appropriate to discuss wbfs fragmentation?
     
  19. Skizzo

    Skizzo Banned

    Banned
    475
    0
    May 1, 2009
    United States
    Sorry, but the 'AFAIK' didn't exactly exude confidence in your answer, so I was seeking further input. So, you're saying the answer to the question above is that yes, as long as I have enough free space on the WBFS partition, even if that space is not contiguous, I CAN install the game. Thanks. And should I want to discuss the matter any further, I'll move it to the thread you suggest. Sorry about that too.
     
  20. Wiimm
    OP

    Wiimm Developer

    Member
    2,156
    368
    Aug 11, 2009
    Gambia, The
    Germany
    wwt: Wiimms WBFS Tool v0.12a - 2009-11-05
    A command line WBFS tool set for various os.

    I have updated the tool.

    WARNING: I have only done a few tests with the new scrubbing features (tools wit, wdf2iso, iso2wdf). It seems that scrubbing works.

    Download:This is an experimental release!


    Highlights of this release:
    • The wit command DUMP has been improved. Also it can dump ISO images inside a WBFS.
      .
    • I have rewritten the file subsystem. WBFS files with a single disc are interpreted as an ISO file and can be used as source. WBFS with more than 1 disc can be used together with a selector: /path/to/WBFS_FILE/SELECT. See the code block below.

      This is only true for the tool 'wit'. The tool 'wwt' uses the old file subsystem.
      .
    • The tool 'wit' has 2 new commands: COPY and SCRUB: Both commands work in the same way, but for SCRUB the source and the destination is the same. Both can scrub and/or convert from/to ISO/WDF/WBFS and/or split/unsplit.

      To convert all files from a directory to single WBFS files just type:
      Code:
      wit copy /path/to/dir -d dest_dir --wbfs --split
      To convert all files from a WBFS to single WBFS files just type:
      Code:
      wit copy /path/to/wbfs -d dest_dir --wbfs --split
    • Again: This is an experimental release!
    Code:
    ///////////////////////////////////////////////////////////////////////////////
    ///////////////ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂProcessing ISO filesÂÂÂÂÂÂÂÂÂÂÂÂ ///////////////
    ///////////////////////////////////////////////////////////////////////////////
    
    Command like "wwt ADD" or "wit COPY" uses ISO files as input. They accept
    plain ISO files and ISO files in WDF and WBFS containers. In detail:
    
    /path/to/PLAIN_ISO:
    ÂÂÂÂA PLAIN ISO file is a 1:1 copy of a file. It is not compressed and not
    ÂÂÂÂpart of any container. The standart extension is ".iso".
    
    /path/to/WDF_ISO:
    ÂÂÂÂThis is an ISO packed in a WDF container. WDF container may be used for
    ÂÂÂÂany files and not only for ISO images. The standart extension is ".wdf".
    
    /path/to/WBFS_FILE
    ÂÂÂÂAn WBFS file/device with exact one ISO image is also accepted. The
    ÂÂÂÂstandart extension for WBFS files is ".wbfs". Some commands accept the
    ÂÂÂÂwhole WBFS with all ISO images as source. See the specific doc.
    
    /path/to/WBFS_FILE/SELECTOR
    ÂÂÂÂThis is a special construct. The ISO image of the WBFS_FILE selected
    ÂÂÂÂby SELECTOR is used as source. The selector may be one of:
    
    ÂÂÂÂ - 'ABCDEF' : ID6 of a disc (exaatc 6 charaters)
    
    ÂÂÂÂ - indexÂÂÂÂ: The zero based index of the disc. The range goes from zero
    ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂup to the number of discs in the WBFS minus 1.
    ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ(decimal number, but not 6 digits)
    
    ÂÂÂÂ - '#' slot : The decimal slot number within the WBFS. The range goes from
    ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂzero up to the maximal number of discs in the WBFS minus 1.
    
    ÂÂÂÂExamples:
    ÂÂÂÂÂÂ.../a.wbfs/rmcp01 : use "Mario Kart" from 'a.wbfs' as source.
    ÂÂÂÂÂÂ.../a.wbfs/5ÂÂÂÂÂÂ: use the image with index #5 from 'a.wbfs' as source.
    ÂÂÂÂÂÂ.../a.wbfs/#5ÂÂÂÂ : use the image at slot #5 from 'a.wbfs' as source.
    
    When writing ISO files the option --wdf (default), --iso and --wbfs control
    the output format. If writing a WBFS file this WBFS is truncated and contains
    exactly one ISO image.
    Last changes:
    Code:
    wwt v0.12a r364 - 2009-11-05
    
    - Bug fix showing the progress counter.
    - Bug fix in libwbfs/wbfs_trim() -> "ADD --trunc" and "TRUNCATE" failed.
    - wit DUMP --long: Dump a memory map.
    - wdf2iso + iso2wdf: print progress infos.
    - New ISO source alternative: /path/to/WBFS_FILE/SELECTOR (see doc)
    - New wit command: COPY: Copy and convert ISO images.
    - New wit command: SCRUB: Scrubs ISO images.
    Sources comes with GPL2. Checkout the sources directly from the SVN repository or browse the source with any browser.