Hacking wwt+wit: Wiimms WBFS+ISO Tools

jivaros

Member
Newcomer
Joined
Sep 29, 2009
Messages
8
Trophies
0
XP
12
Country
France
I'm am very glad to see a linux tool intensively developped !

Like i said before, a check function should be great, but i suggest another cool and unique feature :
importing an iso/wdf from a .rar (or .zip) file, without uncompressing the archive in a temporary directory ...

It seems with unrar-nonfree, it is possible to extract a file to stdin, but not with the unrar-free tool.

If you give me some pointers, (and if you wish) i could try to submit a patch for the rar2wbfs function !
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,291
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,468
Country
Germany
jivaros said:
I'm am very glad to see a linux tool intensively developped !

Like i said before, a check function should be great, but i suggest another cool and unique feature :
importing an iso/wdf from a .rar (or .zip) file, without uncompressing the archive in a temporary directory ...

It seems with unrar-nonfree, it is possible to extract a file to stdin, but not with the unrar-free tool.

If you give me some pointers, (and if you wish) i could try to submit a patch for the rar2wbfs function !

This is a hard job because the WBFS framework (written by others) needs random access and uses the seek() functions to address to the file. And thats mean that the image must be decompressed completely before the access.

On the other hand most (but not the header) of the seek access are at increased offsets: A seek simulation is possible when caching only a few (about 8 MiB) of data. In the next month i will go deeper into the WBFS framework.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,291
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,468
Country
Germany
wwt: Wiimms WBFS Tool v0.03a - 2009-10-08
A command line WBFS tool for linux.

I have updated the tool. The first post with more details is updated too.

I have tested this tool very well and the software do what I want but I can't give any warranty. I have made different import and export (ISO and WDF) tests with about 20 games and the resulting ISO images are absolute identical as the source files.

wwt is now Beta and I'm using it as replacement for the tool 'wbfs'.


Download: http://wiimm.de/x/./wwt-v0.03a-r200.tar.gz (Documentation is included)
More Infos about WDF: http://svn.wiimm.de/open/wii/trunk/wiimms-...WDF.txt?view=co

Last changes:
Code:
wwt V0.03a r200 - 2009-10-08

- Enhanced error management again.
- Misspelling in --help: The shortcut for 'DUMP' is 'D' and not 'DP'.
- Make: Support of environment variable 'XFLAGS' -> 'XFLAGS=-march=x86-64 make'
- File handling: Support of open() and fopen() as alternatives for testing.
- Script test-wdf-tools.sh: Positive test is now done with >1 million files.
- New option for 'wdf-dump': --chunk: Print chunk header table.
- WDF specification is complete.

@moparisthebest and others:
Just create a file named 'Makefile.local' and make you local modifications here.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,291
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,468
Country
Germany
Feature Request:

Code:
ÂÂREQUEST:
ÂÂÂÂI suggest another cool and unique feature:
ÂÂÂÂImporting an iso/wdf from a .rar (or .zip) file, without uncompressing the
ÂÂÂÂarchive in a temporary directory ...

ÂÂANSWER:
ÂÂÂÂThis is a hard job because the WBFS framework (written by others) needs
ÂÂÂÂrandom access and uses the seek() functions to address to the file. And
ÂÂÂÂthat means that the image must be decompressed completely before access. 

ÂÂÂÂOn the other hand most (but not the header) of the seek access are at
ÂÂÂÂincreased offsets: A seek simulation is possible. In the next month i will
ÂÂÂÂgo deeper into the WBFS framework.

ÂÂANALYSIS:
ÂÂÂÂI have quickly tested 3 iso images:
ÂÂÂÂIt seems to be possible when 2 file ranges are cached:
ÂÂÂÂ-ÂÂÂÂÂÂ 0x0.. 0x200000 (len=0x200000 = 2 MiB)
ÂÂÂÂ- 0xf800000..0xfe00000 (len=0x600000 = 6 MiB)
ÂÂÂÂAll other file access is in sequential order.

ÂÂIMPLEMENTATION:
ÂÂÂÂDone but experimental. Use option --cache to enable it.
Is there anybody who wants to test this feature with many game discs?
 

patrick9999

Active Member
Newcomer
Joined
Apr 8, 2009
Messages
32
Trophies
0
XP
31
Country
France
Hi,

Thanks you, i love and use it every day
tongue.gif
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,291
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,468
Country
Germany
wwt: Wiimms WBFS Tool v0.04a - 2009-10-11
A command line WBFS tool for linux.

I have updated the tool. The first post with more details is updated too.

I have tested this tool very well and the software do what I want but I can't give any warranty. I have made different import and export (ISO and WDF) tests with about 20 games and the resulting ISO images are absolute identical as the source files.

wwt is now Beta and I'm using it as replacement for the tool 'wbfs'.


Download: http://wiimm.de/x/./wwt-v0.04a-r212.tar.gz (Documentation is included)
More Infos about WDF: http://svn.wiimm.de/open/wii/trunk/wiimms-...WDF.txt?view=co

Last changes:
CODEwwt V0.04a r212 - 2009-10-11

- ADD: ID6 is now also recognized when source is a WDF.
- New command: FILETYPE: Print a status line for each given file.
- New script: test-ADD+EXTRACT.sh: ADD + EXTRACT test with various options.
- *EXPERIMENTAL*:
ÂÂÂÂ - Option --cache enables caching of well known file parts while reading
ÂÂÂÂÂÂ an ISO image. Perhaps cached areas must be expanded.
ÂÂÂÂ - Adding a game from a pipe if option --cache is set. Commands like
ÂÂÂÂÂÂ "wwt ... ADD --cache
 

thza

New Member
Newbie
Joined
Sep 12, 2009
Messages
2
Trophies
0
XP
2
Country
United States
I really like this tool, but please follow the Linux Filesystem Hierarchy Standard (FHS). The titles.txt files should be stored under $PREFIX/share/wwt/titles.txt, not under $PREFIX/lib/titles.txt.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,291
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,468
Country
Germany
thza said:
I really like this tool, but please follow the Linux Filesystem Hierarchy Standard (FHS). The titles.txt files should be stored under $PREFIX/share/wwt/titles.txt, not under $PREFIX/lib/titles.txt.
done.
(But the install script must be improved -- later)
 

gizmomelb

Well-Known Member
Member
Joined
Jul 13, 2006
Messages
930
Trophies
0
XP
1,647
Country
Australia
Hi Wiimm, this looks like a very interesting tool.

I just wanted to clarify if your scrub ISO option produces files the same as the cISO format please (ie: as ripped by any USB loader) or is there another option which would produce the same output?

Thank you.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,291
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,468
Country
Germany
gizmomelb said:
Hi Wiimm, this looks like a very interesting tool.

I just wanted to clarify if your scrub ISO option produces files the same as the cISO format please (ie: as ripped by any USB loader) or is there another option which would produce the same output?

Thank you.
SCRUB is planned but not subject of interest in the moment. And answers about cISO are hard because I don't know what cISO is (missing technical info).
 

Bauldrick

Well-Known Member
Member
Joined
Jan 22, 2009
Messages
103
Trophies
0
XP
100
Country
How can I obtain the source, I wish to compile on mips architecture. I try svn but get:

svn: Repository moved permanently to '/open/wii/trunk/wiimms-wbfs-tool/'; please relocate

Many thanks
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,291
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,468
Country
Germany
Bauldrick said:
How can I obtain the source, I wish to compile on mips architecture. I try svn but get:

svn: Repository moved permanently to '/open/wii/trunk/wiimms-wbfs-tool/'; please relocate

Many thanks
This is only the web view to the open parts off the svn. The svn itself is closed (account is needed).
Please use the tarball (link at the bottom in every directory view).
 

Bauldrick

Well-Known Member
Member
Joined
Jan 22, 2009
Messages
103
Trophies
0
XP
100
Country
Hi Wiimm,

Thanks, I haven't tested it on my box yet, have only just got it to compile. The only way I could compile was to comment out
Code:
CFLAGSÂÂÂÂÂÂÂÂ += -mtune=generic

else I would get
Code:
cc1: error: unrecognized command line option "-mtune=generic"

This is on a mips cpu

Well, it appears to work - can find wbfs hdd and list games on it, great stuff...

-edit-

This is awesome work, many thanks for this! iso2wdf (moreover wdf) is great
smile.gif
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,291
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,468
Country
Germany
wwt: Wiimms WBFS Tool v0.05a - 2009-10-14
A command line WBFS tool for linux.

I have updated the tool. The first post with more details is updated too.

I have tested this tool very well and the software do what I want but I can't give any warranty. I have made different import and export (ISO and WDF) tests with about 20 games and the resulting ISO images are absolute identical as the source files.

wwt is now Beta and I'm using it as replacement for the tool 'wbfs'.


Download: http://wiimm.de/x/./wwt-v0.05a-r247.tar.gz (Documentation is included)
More Infos about WDF: http://svn.wiimm.de/open/wii/trunk/wiimms-...WDF.txt?view=co

Last changes:
Code:
wwt V0.05a r247 - 2009-10-14

- Complete new interface to the WBFS subsystem:
ÂÂÂÂ- unused source files removed.
ÂÂÂÂ- improved error messages of libwbfs
- WBFS file handling rewritten. It supports now file (using open() function)
ÂÂ and stream (using fopen() function) methods. Use --io to select method.
- New option: --ignore: Ignore non existing files/discs without warnings.
ÂÂ Can be used with: ADD, EXTRACT, REMOVE (replaces --force) and FILETYPE.
- The lib path moved from '/usr/local/lib/' to '/usr/local/share/wwt/'.
- New option --sector-size: Set logical hd sector size larger than 512
ÂÂ while formatting WBFS => The max disc count of a WBSF will be raised.
- Title database implemented.
- New command: TITLES: Print the internal title database to stdout.
- ADD: ISO image is acepted from stdin if a file is named '-'. For pipes
ÂÂ the experimental option --cache must be set.

The --cache option is object to test. If adding a file from pipe fails then please use the binary wwt from 'debug/' folder and add the file again without piping and without --cache. The debug version create a file called '_trace.tmp' which much info. I need this file for analysis.

-----

EDIT:
WARNING: The last version of WWT v0.05a fails if formatting a block device (a real partition).


reason: playing with sector size ...
frown.gif
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,291
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,468
Country
Germany
Bauldrick said:
Hi Wiimm,

Thanks, I haven't tested it on my box yet, have only just got it to compile. The only way I could compile was to comment out
Code:
CFLAGSÂÂÂÂÂÂÂÂ += -mtune=generic

else I would get
Code:
cc1: error: unrecognized command line option "-mtune=generic"

This is on a mips cpu

Well, it appears to work - can find wbfs hdd and list games on it, great stuff...The option "-mtune=generic" has now moved to Makefile.local (a local makefile for each users.)

There is a script called scripts/test-ADD+EXTRACT.sh :
Just call it with a list of ISO or WDF files: it will add and extract the image with various options and compare the results. It needs much time ....

QUOTE(Bauldrick @ Oct 14 2009, 02:28 PM) This is awesome work, many thanks for this! iso2wdf (moreover wdf) is great
smile.gif
be carefull: iso2wdf does not scrubbing the image.
 

Popular threads in this forum

General chit-chat
Help Users
  • No one is chatting at the moment.
  • Vetusomaru @ Vetusomaru:
    That Transgender Transformers parody later became canon
  • NinStar @ NinStar:
    lgbtqiapnhdr++
  • trepp0 @ trepp0:
    Bunjolio be no living on this website fr
    +1
  • Bunjolio @ Bunjolio:
    nuh uh
  • trepp0 @ trepp0:
    dawg, every time i have logged on, you have been active.
  • Bunjolio @ Bunjolio:
    yeah cuz im bored
  • K3N1 @ K3N1:
    Alot of us do tbh we just stop giving a shit about what others that don't know us think
  • redfire12 @ redfire12:
    nothing wrong with living "on site" lol, most folk do it on faceyB and insta
  • Absolute_Vile @ Absolute_Vile:
    Back 20 years ago, it was common and normal to spent a lot of time in forums, there was nothing more
  • K3N1 @ K3N1:
    Depends what you get out of it, if it's actual socializing, learning about people, trying to get along it can change your life indefinitely
    +1
  • K3N1 @ K3N1:
    Most users here have chatted so long they know the difference when they're just being a dipshit or actively struggling with something life brings
    +1
  • WiiQstnr @ WiiQstnr:
    does someone wanna join my new group? the link is https://gbatemp.net/forums/wiimodders.693/
  • Sonic Angel Knight @ Sonic Angel Knight:
    * peeks in chat * :ninja:
  • Bunjolio @ Bunjolio:
    anyone` know how to do that silly invisible avatar thing on pokemone on 3ds
  • Bunjolio @ Bunjolio:
    and is there any cheats for editing ur secret base customization item inventory fr pokemon omega ruby
  • Bunjolio @ Bunjolio:
    like the decortion for super ecret base
  • SylverReZ @ SylverReZ:
    Erect base lol
  • Bunjolio @ Bunjolio:
    lmao super erect base
  • Vetusomaru @ Vetusomaru:
    Most users here have chatted so long they know the difference when they're just being a dipshit or actively struggling with something life brings <---- not necessarily. there are people who have to deal with actual real life problems but they mostly avoid talking about them all the time or at all and instead they prefer venting about meaningless stuff like silly internet dramas to relief their stress
    . i talk about generally at the internet and not just here.
  • Vetusomaru @ Vetusomaru:
    nothing wrong with living "on site" lol, most folk do it on faceyB and insta <----- i never understood the appeal of Instagram
    . you cant even save photos directly.
  • Vetusomaru @ Vetusomaru:
    btw does GBATemp has their own official discord server?
  • DragonMals tempBOT:
    DragonMals has joined the room.
    DragonMals tempBOT: DragonMals has joined the room.