wwt+wit: Wiimms WBFS+ISO Tools

Discussion in 'Wii - Hacking' started by Wiimm, Sep 28, 2009.

  1. Wiimm
    OP

    Wiimm Developer

    Member
    2,121
    334
    Aug 11, 2009
    Gambia, The
    Germany
    try-a (lower case) for --auto
    But -A / --all should work .... mmmh (i can't test it on work)
     


  2. jivaros

    jivaros Newbie

    Newcomer
    8
    0
    Sep 29, 2009
    France
    I'm am very glad to see a linux tool intensively developped !

    Like i said before, a check function should be great, but i suggest another cool and unique feature :
    importing an iso/wdf from a .rar (or .zip) file, without uncompressing the archive in a temporary directory ...

    It seems with unrar-nonfree, it is possible to extract a file to stdin, but not with the unrar-free tool.

    If you give me some pointers, (and if you wish) i could try to submit a patch for the rar2wbfs function !
     
  3. Wiimm
    OP

    Wiimm Developer

    Member
    2,121
    334
    Aug 11, 2009
    Gambia, The
    Germany
    This is a hard job because the WBFS framework (written by others) needs random access and uses the seek() functions to address to the file. And thats mean that the image must be decompressed completely before the access.

    On the other hand most (but not the header) of the seek access are at increased offsets: A seek simulation is possible when caching only a few (about 8 MiB) of data. In the next month i will go deeper into the WBFS framework.
     
  4. Wiimm
    OP

    Wiimm Developer

    Member
    2,121
    334
    Aug 11, 2009
    Gambia, The
    Germany
    wwt: Wiimms WBFS Tool v0.03a - 2009-10-08
    A command line WBFS tool for linux.

    I have updated the tool. The first post with more details is updated too.

    I have tested this tool very well and the software do what I want but I can't give any warranty. I have made different import and export (ISO and WDF) tests with about 20 games and the resulting ISO images are absolute identical as the source files.

    wwt is now Beta and I'm using it as replacement for the tool 'wbfs'.


    Download: http://wiimm.de/x/./wwt-v0.03a-r200.tar.gz (Documentation is included)
    More Infos about WDF: http://svn.wiimm.de/open/wii/trunk/wiimms-...WDF.txt?view=co

    Last changes:
    Code:
    wwt V0.03a r200 - 2009-10-08
    
    - Enhanced error management again.
    - Misspelling in --help: The shortcut for 'DUMP' is 'D' and not 'DP'.
    - Make: Support of environment variable 'XFLAGS' -> 'XFLAGS=-march=x86-64 make'
    - File handling: Support of open() and fopen() as alternatives for testing.
    - Script test-wdf-tools.sh: Positive test is now done with >1 million files.
    - New option for 'wdf-dump': --chunk: Print chunk header table.
    - WDF specification is complete.
    @moparisthebest and others:
    Just create a file named 'Makefile.local' and make you local modifications here.
     
  5. Wiimm
    OP

    Wiimm Developer

    Member
    2,121
    334
    Aug 11, 2009
    Gambia, The
    Germany
    Feature Request:

    Code:
    ÂÂREQUEST:
    ÂÂÂÂI suggest another cool and unique feature:
    ÂÂÂÂImporting an iso/wdf from a .rar (or .zip) file, without uncompressing the
    ÂÂÂÂarchive in a temporary directory ...
    
    ÂÂANSWER:
    ÂÂÂÂThis is a hard job because the WBFS framework (written by others) needs
    ÂÂÂÂrandom access and uses the seek() functions to address to the file. And
    ÂÂÂÂthat means that the image must be decompressed completely before access. 
    
    ÂÂÂÂOn the other hand most (but not the header) of the seek access are at
    ÂÂÂÂincreased offsets: A seek simulation is possible. In the next month i will
    ÂÂÂÂgo deeper into the WBFS framework.
    
    ÂÂANALYSIS:
    ÂÂÂÂI have quickly tested 3 iso images:
    ÂÂÂÂIt seems to be possible when 2 file ranges are cached:
    ÂÂÂÂ-ÂÂÂÂÂÂ 0x0.. 0x200000 (len=0x200000 = 2 MiB)
    ÂÂÂÂ- 0xf800000..0xfe00000 (len=0x600000 = 6 MiB)
    ÂÂÂÂAll other file access is in sequential order.
    
    ÂÂIMPLEMENTATION:
    ÂÂÂÂDone but experimental. Use option --cache to enable it.
    Is there anybody who wants to test this feature with many game discs?
     
  6. patrick9999

    patrick9999 Member

    Newcomer
    32
    0
    Apr 8, 2009
    France
    Hi,

    Thanks you, i love and use it every day [​IMG]
     
  7. Wiimm
    OP

    Wiimm Developer

    Member
    2,121
    334
    Aug 11, 2009
    Gambia, The
    Germany
    wwt: Wiimms WBFS Tool v0.04a - 2009-10-11
    A command line WBFS tool for linux.

    I have updated the tool. The first post with more details is updated too.

    I have tested this tool very well and the software do what I want but I can't give any warranty. I have made different import and export (ISO and WDF) tests with about 20 games and the resulting ISO images are absolute identical as the source files.

    wwt is now Beta and I'm using it as replacement for the tool 'wbfs'.


    Download: http://wiimm.de/x/./wwt-v0.04a-r212.tar.gz (Documentation is included)
    More Infos about WDF: http://svn.wiimm.de/open/wii/trunk/wiimms-...WDF.txt?view=co

    Last changes:
    CODEwwt V0.04a r212 - 2009-10-11

    - ADD: ID6 is now also recognized when source is a WDF.
    - New command: FILETYPE: Print a status line for each given file.
    - New script: test-ADD+EXTRACT.sh: ADD + EXTRACT test with various options.
    - *EXPERIMENTAL*:
    ÂÂÂÂ - Option --cache enables caching of well known file parts while reading
    ÂÂÂÂÂÂ an ISO image. Perhaps cached areas must be expanded.
    ÂÂÂÂ - Adding a game from a pipe if option --cache is set. Commands like
    ÂÂÂÂÂÂ "wwt ... ADD --cache
     
  8. jivaros

    jivaros Newbie

    Newcomer
    8
    0
    Sep 29, 2009
    France
    Awsome !
    I'm gonna test this !

    you deserve a standing ovation [​IMG] :

    \o/ \o/ \o/
    | | |
    / \ / \ / \

    Thank you !
     
  9. thza

    thza Newbie

    Newcomer
    2
    0
    Sep 12, 2009
    United States
    I really like this tool, but please follow the Linux Filesystem Hierarchy Standard (FHS). The titles.txt files should be stored under $PREFIX/share/wwt/titles.txt, not under $PREFIX/lib/titles.txt.
     
  10. jivaros

    jivaros Newbie

    Newcomer
    8
    0
    Sep 29, 2009
    France
     
  11. Wiimm
    OP

    Wiimm Developer

    Member
    2,121
    334
    Aug 11, 2009
    Gambia, The
    Germany
    @jivaros
    reading from stdin is not implemented yet
    try: wwt .... add --cache
     
  12. Wiimm
    OP

    Wiimm Developer

    Member
    2,121
    334
    Aug 11, 2009
    Gambia, The
    Germany
    done.
    (But the install script must be improved -- later)
     
  13. gizmomelb

    gizmomelb GBAtemp Regular

    Member
    106
    1
    Jul 13, 2006
    Hi Wiimm, this looks like a very interesting tool.

    I just wanted to clarify if your scrub ISO option produces files the same as the cISO format please (ie: as ripped by any USB loader) or is there another option which would produce the same output?

    Thank you.
     
  14. Wiimm
    OP

    Wiimm Developer

    Member
    2,121
    334
    Aug 11, 2009
    Gambia, The
    Germany
    SCRUB is planned but not subject of interest in the moment. And answers about cISO are hard because I don't know what cISO is (missing technical info).
     
  15. Bauldrick

    Bauldrick GBAtemp Regular

    Member
    103
    1
    Jan 22, 2009
    How can I obtain the source, I wish to compile on mips architecture. I try svn but get:

    svn: Repository moved permanently to '/open/wii/trunk/wiimms-wbfs-tool/'; please relocate

    Many thanks
     
  16. Wiimm
    OP

    Wiimm Developer

    Member
    2,121
    334
    Aug 11, 2009
    Gambia, The
    Germany
    This is only the web view to the open parts off the svn. The svn itself is closed (account is needed).
    Please use the tarball (link at the bottom in every directory view).
     
  17. Bauldrick

    Bauldrick GBAtemp Regular

    Member
    103
    1
    Jan 22, 2009
    Hi Wiimm,

    Thanks, I haven't tested it on my box yet, have only just got it to compile. The only way I could compile was to comment out
    Code:
    CFLAGSÂÂÂÂÂÂÂÂ += -mtune=generic
    else I would get
    Code:
    cc1: error: unrecognized command line option "-mtune=generic"
    This is on a mips cpu

    Well, it appears to work - can find wbfs hdd and list games on it, great stuff...

    -edit-

    This is awesome work, many thanks for this! iso2wdf (moreover wdf) is great [​IMG]
     
  18. Wiimm
    OP

    Wiimm Developer

    Member
    2,121
    334
    Aug 11, 2009
    Gambia, The
    Germany
    wwt: Wiimms WBFS Tool v0.05a - 2009-10-14
    A command line WBFS tool for linux.

    I have updated the tool. The first post with more details is updated too.

    I have tested this tool very well and the software do what I want but I can't give any warranty. I have made different import and export (ISO and WDF) tests with about 20 games and the resulting ISO images are absolute identical as the source files.

    wwt is now Beta and I'm using it as replacement for the tool 'wbfs'.


    Download: http://wiimm.de/x/./wwt-v0.05a-r247.tar.gz (Documentation is included)
    More Infos about WDF: http://svn.wiimm.de/open/wii/trunk/wiimms-...WDF.txt?view=co

    Last changes:
    Code:
    wwt V0.05a r247 - 2009-10-14
    
    - Complete new interface to the WBFS subsystem:
    ÂÂÂÂ- unused source files removed.
    ÂÂÂÂ- improved error messages of libwbfs
    - WBFS file handling rewritten. It supports now file (using open() function)
    ÂÂ and stream (using fopen() function) methods. Use --io to select method.
    - New option: --ignore: Ignore non existing files/discs without warnings.
    ÂÂ Can be used with: ADD, EXTRACT, REMOVE (replaces --force) and FILETYPE.
    - The lib path moved from '/usr/local/lib/' to '/usr/local/share/wwt/'.
    - New option --sector-size: Set logical hd sector size larger than 512
    ÂÂ while formatting WBFS => The max disc count of a WBSF will be raised.
    - Title database implemented.
    - New command: TITLES: Print the internal title database to stdout.
    - ADD: ISO image is acepted from stdin if a file is named '-'. For pipes
    ÂÂ the experimental option --cache must be set.
    The --cache option is object to test. If adding a file from pipe fails then please use the binary wwt from 'debug/' folder and add the file again without piping and without --cache. The debug version create a file called '_trace.tmp' which much info. I need this file for analysis.

    -----

    EDIT:
    WARNING: The last version of WWT v0.05a fails if formatting a block device (a real partition).


    reason: playing with sector size ... [​IMG]
     
  19. Wiimm
    OP

    Wiimm Developer

    Member
    2,121
    334
    Aug 11, 2009
    Gambia, The
    Germany
    be carefull: iso2wdf does not scrubbing the image.
     
  20. Wiimm
    OP

    Wiimm Developer

    Member
    2,121
    334
    Aug 11, 2009
    Gambia, The
    Germany
    WARNING: The last version of WWT v0.05a fails if formatting a block device (a real partition).

    reason: playing with sector size ... [​IMG]