Reply to thread

Message: <blockquote data-quote="Wiimm" data-source="post: 3074890" data-attributes="member: 191758">I have developed a new WIi ISO disc format:WIA = Wii ISO ArchiveThis article describes the background and some aspects of the implementation.Classic File formatsA PLAIN ISO file stores the content of the DVD byte by byte. It has a usual size of 4.7 GB. Most WBFS and ISO tools are able to detect unused areas (=holes) and fill them with a zeros. This method is called scrubbing. There are 4 classic formats:<ul> <li data-xf-list-type="ul"> PLAIN ISO (1:1 copy)</li> <li data-xf-list-type="ul"> CISO (Compressed ISO, ignore unused blocks)</li> <li data-xf-list-type="ul"> WBFS (Multi ISO container, ignore unused blocks)</li> <li data-xf-list-type="ul"> WDF (Wii Disc File, mange holes directly)</li> </ul>A game like Animal crossing (my example for this article) has only about 350 MB real data. [code]-----------------------------------------------size   percent   creation time  format-----------------------------------------------352695380 100.00%       8.939 sec  WDF358645760 101.68%      14.946 sec  CISO360448000 102.19%       8.842 sec  WBFS4699979776   1332%       8.798 sec  PLAIN ISO-----------------------------------------------[/code]You see, that WDF, CISO and WBFS are similar small, because they do almost the same with different containers. WDF is the smallest format, but not playable by USB loaders. I set the WDF size to 100%.Compressing with BZIP2 and RARISO images can be compressed with with tools like bzip2 or rar. But this is very ineffective because most of the data is encrypted and encrypted data looks like random data and compressors can't compress random data. See this table:[code]------------------------------------------------------------size   percent   creation time  format------------------------------------------------------------352695380 100.00%       8.939 sec  WDF353345496 100.18%    6:24.597 m:s  WDF + RAR354147564 100.41%    6:04.515 m:s  PLAIN ISO + BZIP2354162098 100.41%    3:19.670 m:s  WDF + BZIP2355472608 100.78%   12:52.996 m:s  PLAIN ISO + RAR4699979776   1332%       8.798 sec  PLAIN ISO------------------------------------------------------------[/code]WDF is a little better as BIZP2 or RAR, but don't need as much time as the compressors. This is the main reason that I had developed WDF. WDF manage the holes and allow random access and is easy expandable.Decrypted images3-4 month ago I made test with decrypted images. If encrypting a decrypted images the result is the original ISO image. The advantage of decrypted images are that data don't looks like random data and that make decrypted images compressible:[code]------------------------------------------------------------size   percent   creation time  format------------------------------------------------------------84717686  24.02%    2:40.267 m:s  WDF/DECRYPT + RAR133009483  37.71%    2:40.254 m:s  WDF/DECRYPT + BZIP2330201884  93.62%      25.208 sec  WDF/DECRYPT352695380 100.00%       8.939 sec  WDF353345496 100.18%    6:24.597 m:s  WDF + RAR354162098 100.41%    3:19.670 m:s  WDF + BZIP2------------------------------------------------------------[/code]Since that time I know that decrypting and compressing is the best way to keep Wii discs small. Decrypted images contain all the hash data which takes 3% of the partition data. Hash data looks also like random data and is not compressible. But the hash values are reproducible from the war data. So the best is to throw it away.WIA = Wii ISO ArchiveWith all this knowledge I have designed a new Wii disc image format: WIA = Wii ISO Archive. My main interest is to find out, what is possible. I don't know If I ever convert my few games into WIA because the handling is much slower than WDF and I have enough space on my drives. The <a href="http://wit.wiimm.de/" target="_blank">WIT</a> tools already support this new format, long term tests are running and a first beta will be released in the next days.A WIA archive allows random access reading and is implemented in the file layer of the WIT tools. The WIA files can be read and written like the other supported formats (ISO,CISO,WDF,WBFS). Only the direct editing and patching is not possible. Converting a WIA into a PLAIN ISO restores all non scrubbed data including wrong hash values.I support 2 compression formats: Uncompressed and compressed with bzip2 (default). My decision for bzip2 was easy: Easy to use lib for all systems and good compression ratio for small (2MB) chunks.Here is the size statistic:[code]------------------------------------------------------------size   percent   creation time  format------------------------------------------------------------79942145  22.66%    2:41.197 m:s  WIA/NOCOMPRESS + RAR84717686  24.02%    2:40.267 m:s  WDF/DECRYPT + RAR127436075  36.13%    2:36.284 m:s  WIA/NOCOMPRESS + BZIP2127603521  36.17%    4:43.668 m:s  WIA + RAR127696363  36.20%    2:31.820 m:s  WIA128007670  36.29%    3:40.140 m:s  WIA + BZIP2133009483  37.71%    2:40.254 m:s  WDF/DECRYPT + BZIP2312976812  88.73%      27.094 sec  WIA/NOCOMPRESS330201884  93.62%      25.208 sec  WDF/DECRYPT352695380 100.00%       8.939 sec  WDF353345496 100.18%    6:24.597 m:s  WDF + RAR354162098 100.41%    3:19.670 m:s  WDF + BZIP2------------------------------------------------------------[/code]The WIA compression is really good and an additional compression with BZIP2 or RAR has no positive effect. The smallest images are made by creating uncompressed WIA images and then compressing it with an external tool. You see also that WIA/NOCOMPRESS is better than WDF/DECRYPT because of throwing away unneeded hash values.SummaryLast not least all formats in one table:[code]Summary of RUUP01, Animal Crossing: Let's Go to the City:------------------------------------------------------------size   percent   creation time  format------------------------------------------------------------79942145  22.66%    2:41.197 m:s  WIA/NOCOMPRESS + RAR84717686  24.02%    2:40.267 m:s  WDF/DECRYPT + RAR127436075  36.13%    2:36.284 m:s  WIA/NOCOMPRESS + BZIP2127603521  36.17%    4:43.668 m:s  WIA + RAR127696363  36.20%    2:31.820 m:s  WIA128007670  36.29%    3:40.140 m:s  WIA + BZIP2133009483  37.71%    2:40.254 m:s  WDF/DECRYPT + BZIP2312976812  88.73%      27.094 sec  WIA/NOCOMPRESS330201884  93.62%      25.208 sec  WDF/DECRYPT352695380 100.00%       8.939 sec  WDF353345496 100.18%    6:24.597 m:s  WDF + RAR354147564 100.41%    6:04.515 m:s  PLAIN ISO + BZIP2354162098 100.41%    3:19.670 m:s  WDF + BZIP2355472608 100.78%   12:52.996 m:s  PLAIN ISO + RAR358645760 101.68%      14.946 sec  CISO360448000 102.19%       8.842 sec  WBFS4699979776   1332%       8.798 sec  PLAIN ISO------------------------------------------------------------[/code]Other imagesAnimal crossing is an extreme example. The image is reducible to 23%. Other games are not so good for compression. The next table shows the WIA rates of some other games. And remember: 100% is the WDF size of a scrubbed image.[code]8054276  31.98%      24.213 sec  WIA  RTYP01, Wii Chess127696363  36.20%    3:00.483 m:s  WIA  RUUP01, Animal Crossing531722793  80.26%    5:25.284 m:s  WIA  RSPP01, Wii Sports153673062  86.74%    1:35.435 m:s  WIA  RHAP01, Wii Play1010766389  87.80%    9:36.026 m:s  WIA  RZDP01, Twilight Princess2567636439  92.60%   25:50.572 m:s  WIA  RMCP01, Mario Kart Wii[/code]</blockquote>

Verification

Search

Search

Reply to thread

Settings Notifications

Options