Hacking Inside WBFS

[charroux]

I have made some more small tests. With sector size 512 (default) a WBFS must be smaller as 2 TiB. But when increasing the sector size larger WBFS partitions seems possible. To create a test partition use WWT and type for example:

wwt init --size 4t --sector-size 1024 --split-size 2t --force a.wbfs

Splitting is needed because the maximal single file size.

Amazing stuff!

1) Is this currently compatible with any USB Loader?
2) What is the command line to format/init to allow more than 500 games on a partition? And is that compatible with any USB Loader?

 

[Wiimm]

@charroux

1.) No, because most (all) usb loader have the default sector size 512 hard coded.

2.) sector size > 512 needed. See the first post for details.

 

[charroux]

@charroux

1.) No, because most (all) usb loader have the default sector size 512 hard coded.

2.) sector size > 512 needed. See the first post for details.

Thanks. 1st post didn't help directly, but I get the idea that it will require a code change in a loader before it will work for most end users like me.

 

[r-win]

@charroux

1.) No, because most (all) usb loader have the default sector size 512 hard coded.

2.) sector size > 512 needed. See the first post for details.

Thanks. 1st post didn't help directly, but I get the idea that it will require a code change in a loader before it will work for most end users like me.

You could try USB Loader GX, since we applied the code change posted by Wiimm in his first post.

 

[Wiimm]

You could try USB Loader GX, since we applied the code change posted by Wiimm in his first post.

Great! I never noticed this before. I will play with it.

 

[Elfish]

When creating a WBFS or extracting a raw ISO file from a small game the disc usages is much less than the file size. But when copying the ISO this advantage is lost. This means: WDF is still the best solution for storing ISO images.

that's what i'M planning to do.
sadly there's no loader that loads wdf files directly

 

[Wiimm]

Free Blocks Table and Bugs

Introduction

A WBFS is divided into blocks with equal size. The block size is a power of 2. The calculation of of the block size is made when formatting a WBFS:[*]Use the minimal block size so that there are less than 2^16 blocks.[*]Adjust the block size so that all management data fits into one block. The block size power is stored into the WBFS header (struct wbfs_head) as member 'wbfs_sec_sz_s'.Based on this calculation only whole multiple of the sector size of the WBFS file or device can be used. The rest is not used. "wwt DUMP" shows the end of the used are in the memory map.

After formatting we have N blocks, addressed by 0..N-1. Block #0 contains the management data. All other blocks (1..N-1) are used to store the discs. A block is not used or exactly assigned to 1 disc.

The "free block table" (FBT) is located at the end of block #0, beginning at a hd sector (parameter "sector size"). For each block there is exact one bit reserved. Is the bit set (1), the block is free (not assigned to a disc). If the bit is unset (0), the block is used by a disc.

Each byte of the FBT contains the status of 8 blocks. The first byte is to manage the blocks 1..8, the next byte for blocks 9..15 and so on. As you see this assignments are NOT zero based! In the beginning of the WBFS there was a "delete bug": The function free_block() used a zero based calculation of the FBT and marked the wrong blocks for free (e.g. marked block 2 instead of block 1 for free). But there are some more bugs in libwbfs. And the more bugs are the reason of this article.

I plan to publish a patch for all discussed bugs later after some tests.


Formatting bug

While formatting the WBFS only the first N/8 bytes (N=number of WBFS blocks) of the FBT are set with the value 0xff. Remember: A set bit means "free block".
[*]If N is a multiple of 8 (N%8 == 0) then 1 block to much is marked as free. But because of the allocation bug (see below) this bug appears only, if N is a multiple of 32.[*]If N%8 > 1 then to less blocks are marked as free. But this means only that not the whole WBFS is used for discs.[*]If N%8 == 1 then the correct number of blocks is marked as free.
This formatting bugs can be fixed very easy:[*]Just unset the wrong bit when opening the WBFS.[*]When formatting or checking the disc then mark all blocks free. This can not be done if there is no space for the additional needed byte in block #0.

Allocation bug

libwbfs uses a run time optimization and scans the FBT in u32 (32 bits) steps. Because of this optimization only the first (N/32)*4 bytes of the FBT are used. And this means that up to to 4 bytes of the FBT are ignored. And this means that up to 30 WBFS blocks will not be used for storing discs. On a WBFS with about 500 GiB the WBFS block size is 2^23 -> up to 240 MiB wasted space.

A patch must be compatible to already existing WBFS. And this means, that only (N/8)*8 blocks can be used, because they are set correctly by the old formatting. The last upto 7 blocks should never be used.


Free-Block bug

The free block function free_block() does not make a check of the block number. If the block number is =0 or >=N wrong bits anywhere in the memory are set. No problem if the WBFS is valid.

But because of other errors in the WBFS file/device it is possible that there are wrong values in the disc memory maps. I have already seen dumps with such errors.


Add-Disc bug

If the WBFS becomes full while adding a disc the disc will not added to the WBFS, that's ok. But the already allocated blocks are not freed.

 

[charroux]

I had to give up, since Cygwin is denying me permission to wwt, and I have no idea how to proceed. Thanks for the info though.

 

[Wiimm]

I found a new small libwbfs bug:

In function wbfs_open_partition() the number of wii sectors is calculated by

ÂÂÂÂp->n_wii_sec = ( p->n_hd_sec / p->wii_sec_sz ) * p->hd_sec_sz;
ÂÂÂÂ// -> num_of_hd_sectors / wii_sector_size * hd_sector_size

If num_of_hd_sectors is not a multiple of wii_sector_size then the calulated result is to small because of rounding down.

This code looks better (multiply first):

ÂÂÂÂp->n_wii_sec = (u64)p->n_hd_sec * p->hd_sec_sz / p->wii_sec_sz;

To avoid a 64 bit calculation use this (it works until hd_sector_size is > wii_sec_sz because both are power of 2):

ÂÂÂÂp->n_wii_sec = p->n_hd_sec / ( p->wii_sec_sz / p->hd_sec_sz );

 

[Elfish]

hmmm interessting. seems like there are tons of bugs :/

 

[Wiimm]

hmmm interessting. seems like there are tons of bugs :/

Yes, I playing with WBFS since only 6 weeks and I have found a dozend bugs, most harmless but some harmfull. I have fixed all of them in my WWT.

My conclusions:[*] Do not trust WBFS manager or USB loaders that haven't fixed these errors.
.
[*] Do not trust me and my bug fixes because nobody has controlled me.
.
[*] Do not trust WWT because 2.) and because there are perhaps other bugs.
.
[*] Do not use a WBFS for backup, NEVER!

 

[mousex]

Do not use a WBFS for backup, NEVER!

This irony, don't use the Wii Backup File System for Backups

If you wouldn't care for compatibilty you could fix these errors, right?
What about a WBFS2 from you, tested until hell, developed in months and not some days, with real documentation and you send it to the USBLoader devs?

 

[sotra]

Hi,

I had an old 250GB Western Digital (model # WD2500B007 - caviar with neon blue and pink lights on the sides) that I used for Wii backup games exclusively, but then I got a 1TB Fantom Drives GreenDrive (model # GDP1000EU) that I chose to partition using EASEUS partition master (25GB FAT32, 25GB NTFS and the rest WBFS). No problems with either of these drives really, but one thing I noticed was that the sizes of the games shows up bigger in most cases on my new HDD.
Some examples: Wii Sports (324MB old drive vs. 512MB on new drive), Wii Fit Plus (1240Mb vs. 1472MB), The Beatles: Rock Band (3524MB vs. 3568MB), Wii Chess (32MB vs. 80MB) etc.

I've tried hooking the new HDD up to my Wii and loaded the games from my original discs through various USB loaders, used WBFS manager and WBFS intelligent GUI on my PC and copied from my old HDD directly to my new HDD using ncWBFSTool. Every single game came out bigger on my new HDD.
Any ideas why? I'd hate to take them all off my new HDD and reformat it a certain way to get them to come up smaller in size.
Thanks in advance for any replies.

Edit: If there's any sort of info about each HDD you want me to post then let me know through which program and how and I will do it ASAP. Thanks!

 

[Wiimm]

Do not use a WBFS for backup, NEVER!
This irony, don't use the Wii Backup File System for Backups

My thoughts too.

QUOTE(mousex @ Nov 13 2009, 04:13 PM)

If you wouldn't care for compatibilty you could fix these errors, right?
What about a WBFS2 from you, tested until hell, developed in months and not some days, with real documentation and you send it to the USBLoader devs?

I do only fixes that are compatible to WBFS. The idea of WBFS is good but it seems that nobody had made a code view.

 

[Wiimm]

The function alloc_block() have a serious bug: Depending of the value of 'n_wbfs_sec' a non existend block will be allocated. This patch fixes the bug:
CODE--- libwbfs-old.cÂÂÂÂÂÂ 2009-11-14 13:12:58.000000000 +0100
+++ libwbfs.cÂÂ 2009-11-14 13:14:12.000000000 +0100
@@ -450,7 +450,8 @@
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ if (v & (1

 

[Wiimm]

There is another bug in libwbfs:

If the WBFS becomes full while adding a game the WBFS becomes an invalid state:[*] The disc slot is marked as used, but the info header was not written and may contain illegal data of a previous removed disc or because there is any data on the harddisk.

Deleting this disc may free blocks of other discs because of the undefined data. Because of the free_block() bug, any data in the main memory can be modified == undefined behavior of the program.
.[*] The already alloced blocks are not freed.This is the original code:

bl = alloc_block(p);
if (bl == 0xffff)
{
ÂÂÂÂERROR("no space left on device (disc full)");
}

And this is my version. It's implemented in wwt since v.0.16a:
CODEbl = alloc_block(p);
if ( bl == WBFS_NO_BLOCK )
{
ÂÂÂÂ// free disc slot
ÂÂÂÂp->head->disc_table[discn] = 0;

ÂÂÂÂ// free already allocated blocks
ÂÂÂÂint j;
ÂÂÂÂfor ( j = 0; j < i; j++ )
ÂÂÂÂ{
ÂÂÂÂÂÂÂÂbl = wbfs_ntohs(info->wlba_table[j]);
ÂÂÂÂÂÂÂÂif (bl)
ÂÂÂÂÂÂÂÂÂÂÂÂfree_block(p,bl);
ÂÂÂÂ}
ÂÂÂÂwbfs_sync(p);
ÂÂÂÂWBFS_ERROR("no space left on device (disc full)");
}

 

[mousex]

So, with all these WBFS bugs. DO you think it is safer to store games in .wbfs files on a FAT partition and not on a WBFS drive? I guess if only one game is in a WBFS container most of the bugs can't happen, right?

 

[Wiimm]

So, with all these WBFS bugs. DO you think it is safer to store games in .wbfs files on a FAT partition and not on a WBFS drive? I guess if only one game is in a WBFS container most of the bugs can't happen, right?

yes
or
use a good wbfs manager (no idea if there is anyone, haha)

 

[Elfish]

i guess your tool is good huh?

Posts merged

i guess your tool is good huh?

 

[mousex]

It's the best. I don't know if it is already good (in the sense of bug free, sure it is good

) due to so many bugs in libwbfs which are maybe not yet found but it's the best if you want a sane WBFS.
I had 4 corrupt games because I deleted MP3 and installed the same game after this and 3 games later (in USBLoader 1.2). I thought MP3 would install at the same place as before but it caused three other games to use blocks at the same places (wwt deleted them during repair as I have the original discs).