Hacking wwt+wit: Wiimms WBFS+ISO Tools

Monosodium gluta

Member
Newcomer
Joined
Jun 1, 2010
Messages
6
Trophies
0
XP
26
Country
United States
Wiimm said:
@Monosodium
I can't reproduce your bug (i have converted several iso images to wbfs)

Please do
1.) "make debug all"
2.) execute your command with the new tool again
3.) PN me the trace file named "_trace-iso2wbfs.tmp" (or send a download link)

I compiled the source (same version as the binary that produced the error) as you adviced.
Result: no errors. Now I have just one .wbfs file
CODE-rwx------ 1 yoshi root 1797259264 JunÂÂ1 19:20 RRBP41.wbfs
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
@Monosodium

What is if you do:
make clean+
make mac-distrib

Seems that I have to test it on a mac. But I'm only guest on the mac with a small amount of disc space.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
I have updated the svn repository (no time to build and test a release)

Code:
wit v0.48b r1164 - 2010-06-01 (repository only)

- Bug fix: If reading a very small WBFS file the old and buggy calculation
ÂÂ in libwbfs for the free tables size results in zero. Accessing the non
ÂÂ existent table forces a segmentation fault. For compatibility reasons
ÂÂ the calculation can't be changed.
- Bug fix: wit DUMP have always reported encrypted partitions.
- Composing bug fix: Copy only really needed data from boot.bin if file
ÂÂ header.bin is missed.
 

giantpune

Well-Known Member
Member
Joined
Apr 10, 2009
Messages
2,860
Trophies
0
XP
213
Country
United States
i have made a linux AMD64 for whoever needs it http://www.multiupload.com/IDVV70PZVW
i get some issue about permissions when i go to create this. i can make without any isues. but when i make distrib, i get a bunch of "permission denied". even though i have checked out the source to a folder in my home folder and i can edit all of the source.

if i do sudo make distrib it creates everything normally, but then i must take back ownership of the files from root. is there a way to make distrib this and not have to deal with root?

EDIT>>
this wit is still not marking 0x60 and 0x61 as 1 with the "disc" folder missing. these 2 bytes are needed to be 1 for the disc to be able to be read.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
@giantpune:
1,) It should work without root, perhaps some files are owned by root. Try "chown -R your_stduser ." in the main dir.

2.) The old wit set this values (offset 0x60 and 0x61) to 1 if the file ./disc/header.bin missed because boot.bin has this values also set.

The new wit does it not and I think its ok. (i read your mails later again, perhaps i'm wrong)


EDIT:
I have changes the makefile:
if calling "make install" after "make all" the first one do not create anything in the project directory. The following should now work better:
Code:
make all
sudo make install

"make help" is now up to date too.
 

Monosodium gluta

Member
Newcomer
Joined
Jun 1, 2010
Messages
6
Trophies
0
XP
26
Country
United States
Wiimm said:
@Monosodium

What is if you do:
make clean+
make mac-distrib

Seems that I have to test it on a mac. But I'm only guest on the mac with a small amount of disc space.

I compiled with
make clean+
make distrib (note there was no 'mac-distrib')

And ran the binary with:
CODE$ iso2wbfs --progress --verbose Rayman Raving Rabbids.iso
Result: One 1.7 GB .wbfs file (i.e. no problems)

I also ran the pre-compiled binary once more to test if I was just imagining previously
Result: The same problem as before: two 4.2 GB .wbfs files and one 800 Gb .wbfs file

So the latest mac-release is buggy, but if I compile it from sources by myself there are no problems.



By the way, when compiling with 'make distrib' I got the same permission issues as giantpune reported.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
for mac:
copy the file makefiles-local/Makefile.local to the base directory´and remove comment in the very last line
Code:
...
# Mac, universal binary
-include makefiles-local/Makefile.local.mac

This enables some special MAC things and also mac-distrib
 

Monosodium gluta

Member
Newcomer
Joined
Jun 1, 2010
Messages
6
Trophies
0
XP
26
Country
United States
Wiimm said:
for mac:
copy the file makefiles-local/Makefile.local to the base directory´and remove comment in the very last line
Code:
...
# Mac, universal binary
-include makefiles-local/Makefile.local.mac

This enables some special MAC things and also mac-distrib

Ok, thanks for the help. This worked. Now there was no permission issues after compiling... Btw, I'm curious what are the speacial Mac things?
smile.gif


Today I tried to feed the stdout of unrar to iso2wbfs. Here's what happened:
CODE$ unrar p -inul Sin.and.Punishment.2.Successor.to.the.Sky.rar *.iso | ~/Desktop/wiimms-iso-tools/wit-v0.48a-r0-mac/bin/iso2wbfs --progress --verbose --dest ~ -

iso2wbfs v0.48a r0 mac - Dirk Clemens - 2010-06-03

* iso2wbfs: SCRUB - -> /Users/yoshi/R2VP01.wbfs
ÂÂÂÂÂÂ 0% copied in 2:10 (2.0 MiB/sec) -> ETA 388:50:42ÂÂ
************************************************************************
*****ÂÂIt seems, that the caching area for the game is too small!ÂÂ*****
*****ÂÂPlease report this to the author.ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ *****
*****ÂÂTechnical data: ID=R2VP01, OFF=200000, SIZE=200000ÂÂÂÂÂÂÂÂÂÂ*****
************************************************************************

!! iso2wbfs: ERROR #20 [READ FILE FAILED] in ReadISO() @ src/lib-sf.c#842
!!ÂÂÂÂÂÂÂÂÂÂ Seek failed [F=3,200000]: - (stdin)
!!ÂÂÂÂÂÂÂÂÂÂ -> Illegal seek
!! iso2wbfs: ERROR #23 [WBFS ERROR] in wbfs_add_disc_param() @ src/libwbfs/libwbfs.c#1423
!!ÂÂÂÂÂÂÂÂÂÂ error reading disc
ÂÂÂÂÂÂ258 MiB copied in 2:10,ÂÂ2.0 MiB/secÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
!! iso2wbfs: ERROR #23 [WBFS ERROR] in CopyToWBFS() @ src/lib-sf.c#2185
!!ÂÂÂÂÂÂÂÂÂÂ Error while creating disc [] @/Users/yoshi/R2VP01.wbfs
 

giantpune

Well-Known Member
Member
Joined
Apr 10, 2009
Messages
2,860
Trophies
0
XP
213
Country
United States
why would you feed stdout to iso2wbfs? isnt the stdout of unrar just text output. it doesnt output the extracted file this way.
 

Monosodium gluta

Member
Newcomer
Joined
Jun 1, 2010
Messages
6
Trophies
0
XP
26
Country
United States
giantpune said:
why would you feed stdout to iso2wbfs? isnt the stdout of unrar just text output. it doesnt output the extracted file this way.
Because I want to save space and time and because wit supports itQUOTE(Wiimm @ Sep 28 2009, 03:13 PM) Feature overview:
  • ...
  • Read ISO from pipes (indirect ZIP/RAR support).
    ...

iso2wbfs seems to recognize the right ISO because it names it right (R2VP01.wbfs), so I must be doing at least something right.

If I'm feeding iso2wbfs with wrong kind of output, please correct me.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
If the source has to partitions and both should be copied and scrubbed mode is enabled then reading from pipe fails.

It's a simply reason:
For scrubbing all partitions must be analyzed first. And after scanning the second partition the first one is gone. Used --psel=data to avoid this.

I know this bug and it is on my private to do list.
 

Monosodium gluta

Member
Newcomer
Joined
Jun 1, 2010
Messages
6
Trophies
0
XP
26
Country
United States
Wiimm said:
If the source has to partitions and both should be copied and scrubbed mode is enabled then reading from pipe fails.

It's a simply reason:
For scrubbing all partitions must be analyzed first. And after scanning the second partition the first one is gone. Used --psel=data to avoid this.

I know this bug and it is on my private to do list.

Thanks! Now it works. However in the beginning there's a long pause and nothing seems to happen. The --progress switch is at 0% and CPU is usage is quite high. I guess it's extracting the rar to some cache???
CODEÂÂÂÂÂÂ0% copied in 1:16 (3.3 MiB/sec) -> ETA 1117:01:55
but after a while it starts to speed up.

And thanks for introducing me to the --psel switch in general. Now I can discard all the useless update-partitions etc. (Well, I hope they really are useless?
biggrin.gif
Am I right?)

Your work is really appreciated Wiimm!
 

patrick9999

Active Member
Newcomer
Joined
Apr 8, 2009
Messages
32
Trophies
0
XP
31
Country
France
Edit: i forgot the ' -d ' ... sorry
/**/
Hi,
i have an error :
Perhaps i do a wrong command ?

Code:
pos@buro:/data2/Iso$ iso2wdf *.iso /media/48397ef8-da34-49ca-a175-c2dcf9f5f21a/ --test
iso2wdf: WOULD SCRUB: Crash Of The Titans [RQJP7D].iso -> Crash Of The Titans [RQJP7D].wdf
iso2wdf: WOULD SCRUB: Pirates -  Le trésor de Barbe-Noire [RP7P52].iso -> Pirates -  Le trésor de Barbe-Noire [RP7P52].wdf
iso2wdf: WOULD SCRUB: Tetris Party Deluxe [STEETR].iso -> Tetris Party Deluxe [STEETR].wdf
iso2wdf: WOULD COPY:ÂÂ/media/48397ef8-da34-49ca-a175-c2dcf9f5f21a/ -> /media/48397ef8-da34-49ca-a175-c2dcf9f5f21a/.wdf

Code:
pos@buro:/data2/Iso$ iso2wdf -v *.iso /media/48397ef8-da34-49ca-a175-c2dcf9f5f21a/

iso2wdf v0.48a r1145 i386 - Dirk Clemens - 2010-05-29

* iso2wdf: SCRUB Crash Of The Titans [RQJP7D].iso -> Crash Of The Titans [RQJP7D].wdf
ÂÂÂÂ 3571 MiB copied in 2:42, 22.0 MiB/sec
* iso2wdf: SCRUB Pirates -  Le trésor de Barbe-Noire [RP7P52].iso -> Pirates -  Le trésor de Barbe-Noire [RP7P52].wdf
ÂÂÂÂÂÂ847 MiB copied in 0:39, 21.2 MiB/sec
* iso2wdf: SCRUB Tetris Party Deluxe [STEETR].iso -> Tetris Party Deluxe [STEETR].wdf
ÂÂÂÂÂÂ243 MiB copied in 0:09, 24.8 MiB/sec
* iso2wdf: COPYÂÂ/media/48397ef8-da34-49ca-a175-c2dcf9f5f21a/ -> /media/48397ef8-da34-49ca-a175-c2dcf9f5f21a/.wdf
!! iso2wdf: ERROR #20 [READ FILE FAILED] in ReadISO() @ src/lib-sf.c#842
!!ÂÂÂÂÂÂÂÂÂÂRead failed [F=3,0+1000]: /media/48397ef8-da34-49ca-a175-c2dcf9f5f21a/
!!ÂÂÂÂÂÂÂÂÂÂ-> Is a directory
 

giantpune

Well-Known Member
Member
Joined
Apr 10, 2009
Messages
2,860
Trophies
0
XP
213
Country
United States
can somebody familiar with OSx help me out? ive made myself a nice little GUI that simply envokes wit as a process. using this little bit everything works fine in linux
Code:
FILE *fp = popen( "./wit 2>&1 -h" , "r" );
ÂÂÂÂwhile ( fgets( buff, sizeof buff, fp ) != NULL )
ÂÂÂÂ{
ÂÂÂÂdo stuff
ÂÂÂÂ}
ÂÂÂÂpclose( fp );

using the same code in OSx, it opens the program, but it never enters the while() loop or reads any data. just opens and closes it. is there some other magic i need to do in OSx?
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
@patrick9999

What is this? /media/48397ef8-da34-49ca-a175-c2dcf9f5f21a/

Is this the destination? if yes then prefix it by a --dest. Only wit can do 'copy source dest' like other unix copy programs.

BTW: I have declared the three tools wdf2iso, iso2wdf, iso2wbfs as obsolete. There jobs can be done with the more powerful wit.


@giantpunt:
What happen if you remove "2>&1".
It's only an idea. I never used popen(), I use pipe(), fork() and exec() most in combination with threads instead.



@Monosodium glutamat

The progress counter knows about scrubbing and only needed sectors are counted.
But the file layer must read all data from the pipe because seeking doesn't work. And this takes additional time.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
giantpune said:
this wit is still not marking 0x60 and 0x61 as 1 with the "disc" folder missing. these 2 bytes are needed to be 1 for the disc to be able to be read.
I have discussed this with giantpune. It was a mistake of him (in boot.bin both are 0x01), the newest tools handles this two bytes correct and set them to 0x00 in the disc header.

Info: the first 128 bytes of the disc header and of boot.bin are very similar.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
WIT: Wiimms ISO Tools v0.49a - 2010-06-06
A command line WBFS tool set for various os.

WIT (Wiimms ISO Tools) is a set of command line tools to manipulate Wii ISO images and WBFS containers. The two main tools are called wit (Wiimms ISO Tool) and wwt (Wiimms WBFS Tool, the WBFS manager).

Feature overview:Download of the current version:There is also a GUI available:
Last changes:

Code:
wit v0.48b r1164 - 2010-06-01 (repository only)

- Bug fix: If reading a very small WBFS file the old and buggy calculation
ÂÂ in libwbfs for the free tables size results in zero. Accessing the non
ÂÂ existent table forces a segmentation fault. For compatibility reasons
ÂÂ the calculation can't be changed.
- Bug fix: wit DUMP have always reported encrypted partitions.
- Composing bug fix: Copy only really needed data from boot.bin if file
ÂÂ header.bin is missed.

wit v0.49a r1177 - 2010-06-06

- Built in help improved.

Missing docu:
ÂÂ- wit commands ILIST + EXTRACT

Sources comes with GPL2. Checkout the sources directly from the SVN repository or browse the source.
 

Little Baron

Active Member
Newcomer
Joined
May 26, 2010
Messages
26
Trophies
0
Website
Visit site
XP
151
Country
Gambia, The
One question:
"wit dump" show the content of some extracted files like boot.bin or fst.bin. But I'm missing the same for ticket.bin and tmd.bin. is this on your roadmap?
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
  • No one is chatting at the moment.
    SylverReZ @ SylverReZ: https://www.youtube.com/watch?v=ZokeA2lKB6o