Hacking wwt+wit: Wiimms WBFS+ISO Tools

[giantpune]

im not sure about the differing speeds people are getting, but i think it might be OS dependent. ive noticed that using good ole ubuntu, i get 71.58MBs. using the exact same PC, games, and hardware and running the "windows" version in XP, i get 12.XXMBs. Maybe it's cygin slowing it down or something.

 

[oggzee]

im not sure about the differing speeds people are getting, but i think it might be OS dependent. ive noticed that using good ole ubuntu, i get 71.58MBs. using the exact same PC, games, and hardware and running the "windows" version in XP, i get 12.XXMBs. Maybe it's cygin slowing it down or something.

You could try comparing it with the speed of wbfs_file, which is compiled using mingw to find out if cygwin is the problem.

 

[giantpune]

Holy hell. ubuntu 40 seconds, windows over 5 minutes. Both tests are reading the exact same data from a internal NTFS SATA HDD. Ubuntu is writing to a ext3 drive, while windows is writing to the same HDD it is reading from. But I doubt that the HDD being written to is what makes ubuntu go so much faster.

C:\Documents and Settings\Administrator>"C:\Documents and Settings\Administrator\Desktop\wbfs_file.exe" "C:\Doc
uments and Settings\Administrator\Desktop\RGHE52.wbfs" convert "C:\Documents and Settings\Administrator\Desktop
\RGHE52.iso"
Converting C:\Documents and Settings\Administrator\Desktop\RGHE52.wbfs to ISO
writing to C:\Documents and Settings\Administrator\Desktop\RGHE52.iso
100.00% (|) ETA: 0:00:00 (2922.00MB of 2922.00MB ~ÂÂ9.50MB/s) time: 307.70s
Done inÂÂ0:05:08

[email protected]:~/c/wbfs_to_files$ '/home/j/c/wbfs_file/wbfs_file' '/media/900GB_3GbS/BackUp_of_Wii_HDD/wbfs/RGHE52.wbfs' convert '/home/j/Azureus Downloads/GH3_test.iso'
Converting /media/900GB_3GbS/BackUp_of_Wii_HDD/wbfs/RGHE52.wbfs to ISO
writing to /home/j/Azureus Downloads/GH3_test.iso
100.00% (\) ETA: 0:00:00 (2922.00MB of 2922.00MB ~ 73.40MB/s) time: 39.81sÂÂ 
Done inÂÂ0:00:40

 

[tueidj]

You really don't consider all the seeking that occurs when you read and write to the same drive as being relevant?

 

[giantpune]

when i have windows write to a different HDD than it reads from, it goes 2x as fast. but still far slower than the ubuntu test.

 

[Wiimm]

wwt: Wiimms WBFS Tool v0.40a - 2010-03-31
A command line WBFS tool set for various os.

I have updated the tool. The first post with more details is updated too.

Download of the current stable version:

• wwt-v0.40a-r790-i386.tar.gz : linux/i386 version
• wwt-v0.40a-r790-x86_64.tar.gz : linux/x86_64 version
• wwt-v0.40a-r790-mac.tar.gz : Mac OS universal binaries (ppc, i386 and x86-64) by sylver78
• wwt-v0.40a-r790-cygwin.zip : cygwin (win32) version

Download of the previous stable version:

• wwt-v0.39a-r782-i386.tar.gz : linux/i386 version
• wwt-v0.39a-r782-x86_64.tar.gz : linux/x86_64 version
• wwt-v0.39a-r782-cygwin.zip : cygwin (win32) version

Attention to all MAC users:

• It would be nice if a MAC user can "make distrib" and send me the resulting "*.tar.gz." file. I will add it to the download directory.

There is also a GUI available:

• WiiJManager Multiplatform

Last changes:

wwt v0.39a r782 - 2010-03-29

- Bug fix: New CISO files > 4 GiB have been truncated (modulo 2^32 error).
- Character devices are allowed for WBFS.
- wwt --auto: If the file /proc/partitions is not found, than all devices
ÂÂ named '/dev/sd*' and '/dev/rdisk*' are scanned for WBFS. If scanning of
ÂÂ '/dev/rdisk*' fails then also files named '/dev/disk*' are scanned.
ÂÂ Using character devices '/dev/rdisk*' will speed up wwt on apple computers.

wwt v0.40a r790 - 2010-03-31

ÂÂ- Cache handling (pipe support) improved. Discs with update partitons are
ÂÂÂÂproblematic (some discs needs a 256 MiB cache). Try --psel=game to ignore
ÂÂÂÂupdate partitons. The CYGWIN version has real problems with pipes.
ÂÂ- Bug fix: Option "--ciso" was not recognized.
ÂÂ- New test script: scripts/test-wwt+wit.sh
ÂÂÂÂ- test "wwt INIT" with different HD sector sizes
ÂÂÂÂ- test "wwt ADD" with ISO, CISO, WDF, WBFS
ÂÂÂÂ- test "wwt EXTRACT" to ISO, CISO, WDF, WBFS
ÂÂÂÂ- test "wit COPY" (convert) to ISO, CISO, WDF, WBFS

Sources comes with GPL2. Checkout the sources directly from the SVN repository or browse the source.

 

[Wiimm]

when i have windows write to a different HDD than it reads from, it goes 2x as fast. but still far slower than the ubuntu test.

I can confirm that linux/ext* works much faster than windows/ntfs.
If you compare the speeds remember that my tools print out MiB/sec and that 1 MiB = 1024*1024 = 1048576 (add 5% to MiB value to compare with MB)

@all MAC users:
It would be nice if a MAC user can "make distrib" and send me the resulting "*.tar.gz." file. I will add it to the download directory.


@oggzee + others:
Have you a running cross compiler for mac that runs under linux?

 

[giantpune]

i went to checkout the source to compile it is OSx but a get denied

 

[oggzee]

Holy hell. ubuntu 40 seconds, windows over 5 minutes. Both tests are reading the exact same data from a internal NTFS SATA HDD. Ubuntu is writing to a ext3 drive, while windows is writing to the same HDD it is reading from. But I doubt that the HDD being written to is what makes ubuntu go so much faster.

Probably linux is still caching the data, try doing this:
time ( wbfs_file .... ; sync ; )
to make sure the caches are written

 

[Wiimm]

Sorry. Try this link for checkout: http://opensvn.wiimm.de/wii/trunk/wiimms-wbfs-tool

 

[giantpune]

the sync adds 7 seconds to it


EDIT>>
wwt-v0.40a-r792-mac.tar.gz
http://www.multiupload.com/nte3u2duck

im not sure about the inner workings of the mac and the compatibility between different versions. ive just made this in a vmware PC running OSx 10.6.2.

 

[sylver78]

the sync adds 7 seconds to it


EDIT>>
wwt-v0.40a-r792-mac.tar.gz
http://www.multiupload.com/nte3u2duck

im not sure about the inner workings of the mac and the compatibility between different versions. ive just made this in a vmware PC running OSx 10.6.2.

About the MacOS version, I guess that it's a i686 version ! The best would be to provide an universal binary (with PPC, i686 and x86-64 in it). I think that this could be done by modifying the Makefile a little bit ... I'll try to do that later.

BTW Wiimm, have you ever tested wwt built in 64 bits ? I've tested it on Mac OS snow leopard (by commenting out the march=x86-64 line, but I'm not sure that everything is 100% working and bug free ...

 

[Wiimm]

wwt-v0.40a-r792-mac.tar.gz
http://www.multiupload.com/nte3u2duck

im not sure about the inner workings of the mac and the compatibility between different versions. ive just made this in a vmware PC running OSx 10.6.2.
Thank you. If have uploaded this to my server and added the link to the announcement.

QUOTE(sylver78 @ Mar 31 2010, 11:06 AM) BTW Wiimm, have you ever tested wwt built in 64 bits ? I've tested it on Mac OS snow leopard (by commenting out the march=x86-64 line, but I'm not sure that everything is 100% working and bug free ...

linux/x86_64 is available since some versions. I have tested it with the new test script against 8 game => no problems. An universal binary would be nice

Where have you found march=x86-64 ? It is a relict. And perhaps it is in Makefile.local that is not part of the svn.

 

[sylver78]

linux/x86_64 is available since some versions. I have tested it with the new test script against 8 game => no problems. An universal binary would be nice

Where have you found march=x86-64 ? It is a relict. And perhaps it is in Makefile.local that is not part of the svn.
Great, will try to add the 64 bit version to the universal binary then

march=x86-64 is here in the file "Makefile" :
CODECFLAGSÂÂÂÂÂÂÂÂ= -fomit-frame-pointer -fno-strict-aliasing \
ÂÂÂÂÂÂÂÂÂÂ-Wall -Wno-parentheses -Wno-unused-function \
ÂÂÂÂÂÂÂÂÂÂ-O3 -Ilibwbfs -I.
#CFLAGSÂÂÂÂÂÂÂÂ+= -mtune=generic
#CFLAGSÂÂÂÂÂÂÂÂ+= -march=i686
#CFLAGSÂÂÂÂÂÂÂÂ+= -march=x86-64
CFLAGSÂÂÂÂÂÂÂÂ+= $(XFLAGS)
CFLAGSÂÂÂÂÂÂÂÂ:= $(strip $(CFLAGS))

 

[Wiimm]

But it's only a comment (and now a removed comment)

You can create "Makefile.local" for local settings and "Makefile.user" for user rules.

 

[drh]

Hi Wiimm,

The "Browse the Source" link is now broken since 0.40a was added.
I always used this link because there is a link "Download GNU Tarball" at the bottom to get the whole source in one file download.

The "SVN Repository" link just gives a list of all the individual files which is not too useful.

Edit:
Ah its ok, I used the "Browse the Source" link from page 34 that will links to the correct location:

http://opensvn.wiimm.de/viewvc/wii/trunk/wiimms-wbfs-tool/

 

[Wiimm]

Now add the SNEEK-Structure for games (Extract, .bin, etc...) and Punes FST Creator/Boot.bin Updater and WWT is finished for me

.

And where can I find the corresponding main.h?

@dry:
Thanx for the hint. The link is repaired (syntax error error in relocate script)

 

[fridgee]

You can find the main.h here

 

[giantpune]

if you are going to mess with the fst and extracted files to support sneek, it would probably be good to support part.bin files. as there is the 0xffff file limit in FAT and some games are several thousand. SSBB is over 5,000 files and guitar hero 3 is over 12,000 files. a part.bin is basically all the files that are found in the partition truncated into 1 file (starting at the boot.bin). so you dont use up 12,000 file entries where you can get away with 1. also, all the offsets for the fst.bin, main.dol, and every file in the fst.bin can be shifted down and aligned to 32 so you dont end up with a bunch of wasted space between files. basically you get a trimmed partition.

 

[sylver78]

I built a Mac OS universal binaries version (ppc, i386 and x86-64) for wwt v0.40a
It's available here : http://www.multiupload.com/9TSRUILY3T
I need someone with a ppc mac to test it (it's my first universal binary

)
PS : I failed building a ppc64 version so it's not included ...
PS2 : the install script is using wget to dowload a file on the internet, but wget is not installed on mac os by default ... I'm not sure what this file is for.