Hacking WDF = Wii Disc File

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,284
Trophies
0
Location
Germany
Website
wiimmfi.de
XP
1,388
Country
Germany
WDF = Wii Disc File

Wii ISO images has a size of about 4.4 GiB, but much space is unused and have to be filled with zeros. Some games need less than 100 MiB and that means about 4.3 GiB waste. The linux ext2/3 file system can handle such files very good because it do not allocate disc space for such (zero-)holes. Such files called "sparse files".

For me there is no need to store Wii ISO images in the old way filled with zeros. And the handling with large sparse files is not effective and needs always special options like 'ls -s' or 'cp --sparse=always'. Also it costs much time because the holes have to be scanned again.

One way is to compress the data. But compressing works only for the holes but not for the crypt data. And random access (needed by the WBFS framework) is very difficult for compressed data. The direct support of holes seems to be much better.

Because of this all I have defined a new file format, the WDF format. There are some main issue for me:[*] The WDF format should handles holes effectively to save disk space.[*] It should support splitted files because small files can sometimes be handled more effectively than whole large files.[*] There must be a WBFS tool to handle this kind of files. I have already implemented the WDF support im my 'WWT' (Wiimms WBFS Tool). The WWT package contains some more tools to handle WBFS.[*] Other WBFS manager should support the WDF too. And this means that the implementation must be done by the well known wrapper functions 'rw_sector_callback_t' and 'read_wiidisc_callback_t'. That makes transfering the code very easy.I have already implemented this (but not splitted WDF) in my WWT tools and it works fine. The tools are:
  • 'wwt' is a general WBFS manager that may import WDF files directly to a WBFS and may export discs from WBFS directly to WDF files.
  • 'wdf-dump' dumps the data structure of a WDF file.
  • 'iso2wdf' converts ISO images into WDF files.
  • 'wdf2iso' converts WDF files into ISO images.
  • 'wdf-cat' is a 'cat' like program with special handling of WDF.
The following table show the size of ISO images and WDF files of some games (my test suite).
CODE---------------------------------------------------------------------
id6 image size WDF size % name of disc
---------------------------------------------------------------------
RTYP01 4482 MiB 27 MiB
 

pepxl

GFX W!Z4RD
Member
Joined
Jun 19, 2009
Messages
3,263
Trophies
0
XP
302
Country
@ DarkMystery

No it doesnt cISO's are compressed ISO's so the file sizes are game size only and not full ISO size with all the zeros, plus its NTFS not NTSC
 

DjoeN

Captain Haddock!
Member
Joined
Oct 21, 2005
Messages
5,489
Trophies
0
Age
51
Location
Somewhere in this potatoland!
Website
djoen.dommel.be
XP
2,779
Country
Belgium
Never used cISO, but whatever program compresses the best without loss of speed/quality is good for me
biggrin.gif


I only wish a Windows command tool of your wwt tools came out
tongue.gif
, since i don't really like linux.
(Reads, i'm to lazy to work with linux now, after many failed attempts to)
 

olliepop2000

Well-Known Member
Member
Joined
Apr 4, 2009
Messages
457
Trophies
0
Age
42
Location
North West UK
XP
97
Country
Tests need to be made to compare WDF and CISO to compare ripping speeds and file sizes. If there is significant improvement then maybe this is the way to go.
 

Dack

Well-Known Member
Member
Joined
Aug 26, 2007
Messages
601
Trophies
0
Location
UK
XP
98
Country
CasperH said:
Isn't this the same as WiiScruber?

No - though it's had 'shrink' capability built in since about February (try using a resource file editor and enabling the hidden buttons on the 1.40 release
wink.gif
but I refined the format a bit after the release (which was one of the reasons I didn't activate the buttons))

Split files are an issue though as it's a bit pointless unless the IOSs support the split files format
 

Dack

Well-Known Member
Member
Joined
Aug 26, 2007
Messages
601
Trophies
0
Location
UK
XP
98
Country
It doesn't actually - it's quite a horrible format when you start looking into it.

(e.g. Due to the 16 bit limitation on number of clusters, for a 1 terrabyte disc you end up having 'clusters' that are 14 Meg in size - this means files are then rounded upto that nearest size).

BTW the Wii Chess size in the test is a bit of a con as thats the version that has the update partition removed, in fact, looking at the results - they all have the updates removed, in the case of Mario Kart you've also stripped the channel.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,284
Trophies
0
Location
Germany
Website
wiimmfi.de
XP
1,388
Country
Germany
tj_cool said:
ever heard of cISO ?heard? yes. But I never found a specification of it.
And ISO images aren't good candidates for compressing. Or means 'compressed' not something like zip/rar does?

I will make speed test later in the week. Here are the results of an quick test:
CODEAdding Zelda ISO:ÂÂÂÂ 31s
Adding Zelda WDF:ÂÂÂÂ 27s
Extracting Zelda ISO: 30s
Extracting Zelda WDF: 30s
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,284
Trophies
0
Location
Germany
Website
wiimmfi.de
XP
1,388
Country
Germany
I have made some test in the last days. Here are the results:

WDF does not compress but handles holes. And it is better in time and space as compression programs. I have compressed scrubbed ISOs (holes filled with 0x00) with gzip, bzi2, 7z and rar. The compression has taken much much time and the files are larger than the WDF files.

CODE----------------------------------------------------------------------------------
ÂÂÂÂÂÂÂÂÂÂÂÂWChÂÂÂÂ WPlÂÂÂÂ WMuÂÂÂÂ WSpÂÂÂÂ TLZÂÂÂÂ MKWÂÂÂÂ TRAÂÂÂÂ SSBÂÂÂÂ MPT
ÂÂÂÂÂÂÂÂÂÂ RTYP01ÂÂRHAP01ÂÂR64P01ÂÂRSPP01ÂÂRZDP01ÂÂRMCP01ÂÂRLRP4FÂÂRSBP01ÂÂR3MP01
----------------------------------------------------------------------------------
iso size:ÂÂ 4482ÂÂÂÂ4482ÂÂÂÂ4482ÂÂÂÂ4482ÂÂÂÂ4482ÂÂÂÂ4482ÂÂÂÂ4482ÂÂÂÂ7590ÂÂÂÂ8102
wdf size:ÂÂÂÂ 27ÂÂÂÂÂÂ99ÂÂÂÂ 137ÂÂÂÂ 635ÂÂÂÂ1101ÂÂÂÂ2647ÂÂÂÂ3171ÂÂÂÂ6907ÂÂÂÂ7563
wdf ratio:ÂÂ
 

wolffangalchemist

This is bat country!
Member
Joined
Nov 23, 2008
Messages
1,487
Trophies
0
Age
31
Location
middleofnowhere, AL
XP
1,038
Country
United States
while space is always a concern when storing data i don't see the point when it comes to a WBFS type of storage.
i have a 100gb wbfs partition and it has only 14 or so gigs used while some ppl might have more used if they copy every game know to existence to it(though that is kinda pointless).
simply because the amount if space each disc takes on A FS can be reduced by altering it doesn't mean you will save a considerable enough amount of space to justify it (meaning it's not gonna squeeze out enough space to make it so you can add a considerable amount of extra games.)

though carry on who knows it could do more than i think(increase load times speed and things of that nature) i'm always willing to test for things like this since i used Linux my self.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,284
Trophies
0
Location
Germany
Website
wiimmfi.de
XP
1,388
Country
Germany
wolffangalchemist said:
while space is always a concern when storing data i don't see the point when it comes to a WBFS type of storage.
i have a 100gb wbfs partition
To avoid misunderstandings: WDF will not replace WBFS. It is only a way to shrink ISO images on other file systems like ext2, FAT or NTFS. And while you are adding/extracting a WDF image to/from WBFS the transformation can be done on the fly without time costs.
 

bowser

Mwa ha ha ha!
Member
Joined
Sep 1, 2008
Messages
2,374
Trophies
0
Age
35
Location
GBAtemp ↑↑↓↓← → ← →BA
XP
2,192
Country
India
I think WDF is a great idea. Right now I'm using a lot of disk space on my computer storing ISOs of Wii games. This will help clear up a lot of space. Keep up the good work!
grog.gif


PS: I don't feel like using WiiScrubber and then compressing the ISOs because i'll have to unzip them again before adding them to WBFS.
 

nknave

Well-Known Member
Member
Joined
Jun 23, 2009
Messages
154
Trophies
0
XP
94
Country
Mexico
I think this will be good if the software was able to burn into a DVD-R directly from this WDF without any other conversion.
 
General chit-chat
Help Users
    kenenthk @ kenenthk: Least it's national tight end day