WDF = Wii Disc File

Discussion in 'Wii - Backup Loaders' started by Wiimm, Oct 4, 2009.

Oct 4, 2009

WDF = Wii Disc File by Wiimm at 9:56 AM (41,046 Views / 0 Likes) 62 replies

  1. Wiimm
    OP

    Member Wiimm Developer

    Joined:
    Aug 11, 2009
    Messages:
    2,051
    Location:
    Germany
    Country:
    Germany
    WDF = Wii Disc File

    Wii ISO images has a size of about 4.4 GiB, but much space is unused and have to be filled with zeros. Some games need less than 100 MiB and that means about 4.3 GiB waste. The linux ext2/3 file system can handle such files very good because it do not allocate disc space for such (zero-)holes. Such files called "sparse files".

    For me there is no need to store Wii ISO images in the old way filled with zeros. And the handling with large sparse files is not effective and needs always special options like 'ls -s' or 'cp --sparse=always'. Also it costs much time because the holes have to be scanned again.

    One way is to compress the data. But compressing works only for the holes but not for the crypt data. And random access (needed by the WBFS framework) is very difficult for compressed data. The direct support of holes seems to be much better.

    Because of this all I have defined a new file format, the WDF format. There are some main issue for me:[*] The WDF format should handles holes effectively to save disk space.[*] It should support splitted files because small files can sometimes be handled more effectively than whole large files.[*] There must be a WBFS tool to handle this kind of files. I have already implemented the WDF support im my 'WWT' (Wiimms WBFS Tool). The WWT package contains some more tools to handle WBFS.[*] Other WBFS manager should support the WDF too. And this means that the implementation must be done by the well known wrapper functions 'rw_sector_callback_t' and 'read_wiidisc_callback_t'. That makes transfering the code very easy.I have already implemented this (but not splitted WDF) in my WWT tools and it works fine. The tools are:
    • 'wwt' is a general WBFS manager that may import WDF files directly to a WBFS and may export discs from WBFS directly to WDF files.
    • 'wdf-dump' dumps the data structure of a WDF file.
    • 'iso2wdf' converts ISO images into WDF files.
    • 'wdf2iso' converts WDF files into ISO images.
    • 'wdf-cat' is a 'cat' like program with special handling of WDF.
    The following table show the size of ISO images and WDF files of some games (my test suite).
    CODE---------------------------------------------------------------------
    id6 image size WDF size % name of disc
    ---------------------------------------------------------------------
    RTYP01 4482 MiB 27 MiB
     


  2. tj_cool

    Supervisor tj_cool Stuff

    Joined:
    Jan 7, 2009
    Messages:
    9,941
    Location:
    This planet
    Country:
    Belgium
    ever heard of cISO ?
     
  3. HackerOfWii

    Banned HackerOfWii Banned

    Joined:
    Oct 2, 2009
    Messages:
    43
    Country:
    Australia
  4. KirovAir

    Member KirovAir Dutch Alcoholic Programmer

    Joined:
    Dec 7, 2006
    Messages:
    587
    Location:
    Netherlands
    Country:
    Netherlands
    cISO will keep the 4.4gig size on NTFS file systems?
     
  5. CasperH

    CasperH Newbie

    Isn't this the same as WiiScruber?
     
  6. pepxl

    Member pepxl GFX W!Z4RD

    Joined:
    Jun 19, 2009
    Messages:
    3,263
    Country:
    United Kingdom
    @ DarkMystery

    No it doesnt cISO's are compressed ISO's so the file sizes are game size only and not full ISO size with all the zeros, plus its NTFS not NTSC
     
  7. DjoeN

    Member DjoeN Captain Haddock!

    Joined:
    Oct 21, 2005
    Messages:
    4,925
    Location:
    Somewhere in this potatoland!
    Country:
    Belgium
    Never used cISO, but whatever program compresses the best without loss of speed/quality is good for me [​IMG]

    I only wish a Windows command tool of your wwt tools came out [​IMG], since i don't really like linux.
    (Reads, i'm to lazy to work with linux now, after many failed attempts to)
     
  8. olliepop2000

    Member olliepop2000 GBAtemp Fan

    Joined:
    Apr 4, 2009
    Messages:
    457
    Location:
    North West UK
    Country:
    United Kingdom
    Tests need to be made to compare WDF and CISO to compare ripping speeds and file sizes. If there is significant improvement then maybe this is the way to go.
     
  9. KirovAir

    Member KirovAir Dutch Alcoholic Programmer

    Joined:
    Dec 7, 2006
    Messages:
    587
    Location:
    Netherlands
    Country:
    Netherlands
    [​IMG] common typo of me.
     
  10. Dack

    Member Dack GBAtemp Advanced Fan

    Joined:
    Aug 26, 2007
    Messages:
    603
    Location:
    UK
    Country:
    United Kingdom
    No - though it's had 'shrink' capability built in since about February (try using a resource file editor and enabling the hidden buttons on the 1.40 release [​IMG] but I refined the format a bit after the release (which was one of the reasons I didn't activate the buttons))

    Split files are an issue though as it's a bit pointless unless the IOSs support the split files format
     
  11. Chaos Punk

    Member Chaos Punk GBAtemp Regular

    Joined:
    Apr 19, 2009
    Messages:
    224
    Location:
    Bakersfield, CA
    Country:
    United States
    WBFS (Wii Backup File Systeym) keeps the ISO sizes to their absolute minimums.
     
  12. Dack

    Member Dack GBAtemp Advanced Fan

    Joined:
    Aug 26, 2007
    Messages:
    603
    Location:
    UK
    Country:
    United Kingdom
    It doesn't actually - it's quite a horrible format when you start looking into it.

    (e.g. Due to the 16 bit limitation on number of clusters, for a 1 terrabyte disc you end up having 'clusters' that are 14 Meg in size - this means files are then rounded upto that nearest size).

    BTW the Wii Chess size in the test is a bit of a con as thats the version that has the update partition removed, in fact, looking at the results - they all have the updates removed, in the case of Mario Kart you've also stripped the channel.
     
  13. Wiimm
    OP

    Member Wiimm Developer

    Joined:
    Aug 11, 2009
    Messages:
    2,051
    Location:
    Germany
    Country:
    Germany
     
  14. Wiimm
    OP

    Member Wiimm Developer

    Joined:
    Aug 11, 2009
    Messages:
    2,051
    Location:
    Germany
    Country:
    Germany
    I have made some test in the last days. Here are the results:

    WDF does not compress but handles holes. And it is better in time and space as compression programs. I have compressed scrubbed ISOs (holes filled with 0x00) with gzip, bzi2, 7z and rar. The compression has taken much much time and the files are larger than the WDF files.

    CODE----------------------------------------------------------------------------------
    ÂÂÂÂÂÂÂÂÂÂÂÂWChÂÂÂÂ WPlÂÂÂÂ WMuÂÂÂÂ WSpÂÂÂÂ TLZÂÂÂÂ MKWÂÂÂÂ TRAÂÂÂÂ SSBÂÂÂÂ MPT
    ÂÂÂÂÂÂÂÂÂÂ RTYP01ÂÂRHAP01ÂÂR64P01ÂÂRSPP01ÂÂRZDP01ÂÂRMCP01ÂÂRLRP4FÂÂRSBP01ÂÂR3MP01
    ----------------------------------------------------------------------------------
    iso size:ÂÂ 4482ÂÂÂÂ4482ÂÂÂÂ4482ÂÂÂÂ4482ÂÂÂÂ4482ÂÂÂÂ4482ÂÂÂÂ4482ÂÂÂÂ7590ÂÂÂÂ8102
    wdf size:ÂÂÂÂ 27ÂÂÂÂÂÂ99ÂÂÂÂ 137ÂÂÂÂ 635ÂÂÂÂ1101ÂÂÂÂ2647ÂÂÂÂ3171ÂÂÂÂ6907ÂÂÂÂ7563
    wdf ratio:ÂÂ
     
  15. G0dLiKe

    Member G0dLiKe who needs a title ;)

    Joined:
    Aug 2, 2009
    Messages:
    1,674
    Country:
    United States
    Well, sounds good to me.

    We need a OSX and/ or Win XP program. Please.
     
  16. Omega Knight

    Member Omega Knight GBAtemp Regular

    Joined:
    Jul 29, 2007
    Messages:
    137
    Country:
    United States
    Great to see some linux love. Keep it up :3
     
  17. wolffangalchemist

    Member wolffangalchemist This is bat country!

    Joined:
    Nov 23, 2008
    Messages:
    1,406
    Location:
    middleofnowhere, AL
    Country:
    United States
    while space is always a concern when storing data i don't see the point when it comes to a WBFS type of storage.
    i have a 100gb wbfs partition and it has only 14 or so gigs used while some ppl might have more used if they copy every game know to existence to it(though that is kinda pointless).
    simply because the amount if space each disc takes on A FS can be reduced by altering it doesn't mean you will save a considerable enough amount of space to justify it (meaning it's not gonna squeeze out enough space to make it so you can add a considerable amount of extra games.)

    though carry on who knows it could do more than i think(increase load times speed and things of that nature) i'm always willing to test for things like this since i used Linux my self.
     
  18. Wiimm
    OP

    Member Wiimm Developer

    Joined:
    Aug 11, 2009
    Messages:
    2,051
    Location:
    Germany
    Country:
    Germany
    To avoid misunderstandings: WDF will not replace WBFS. It is only a way to shrink ISO images on other file systems like ext2, FAT or NTFS. And while you are adding/extracting a WDF image to/from WBFS the transformation can be done on the fly without time costs.
     
  19. bowser

    Member bowser Mwa ha ha ha!

    Joined:
    Sep 1, 2008
    Messages:
    2,177
    Location:
    GBAtemp ↑↑↓↓← → ← →BA
    Country:
    India
    I think WDF is a great idea. Right now I'm using a lot of disk space on my computer storing ISOs of Wii games. This will help clear up a lot of space. Keep up the good work! [​IMG]

    PS: I don't feel like using WiiScrubber and then compressing the ISOs because i'll have to unzip them again before adding them to WBFS.
     
  20. nknave

    Member nknave GBAtemp Regular

    Joined:
    Jun 23, 2009
    Messages:
    154
    Country:
    Mexico
    I think this will be good if the software was able to burn into a DVD-R directly from this WDF without any other conversion.
     

Share This Page