Hacking Inside WBFS

charroux

Well-Known Member
Newcomer
Joined
May 23, 2009
Messages
68
Trophies
0
XP
73
Country
United States
Wiimm said:
I have made some more small tests. With sector size 512 (default) a WBFS must be smaller as 2 TiB. But when increasing the sector size larger WBFS partitions seems possible. To create a test partition use WWT and type for example:

Code:
wwt init --size 4t --sector-size 1024 --split-size 2t --force a.wbfs
Splitting is needed because the maximal single file size.
Amazing stuff!

1) Is this currently compatible with any USB Loader?
2) What is the command line to format/init to allow more than 500 games on a partition? And is that compatible with any USB Loader?
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,286
Trophies
0
Location
Germany
Website
wiimmfi.de
XP
1,442
Country
Germany
@charroux

1.) No, because most (all) usb loader have the default sector size 512 hard coded.

2.) sector size > 512 needed. See the first post for details.
 

charroux

Well-Known Member
Newcomer
Joined
May 23, 2009
Messages
68
Trophies
0
XP
73
Country
United States
Wiimm said:
@charroux

1.) No, because most (all) usb loader have the default sector size 512 hard coded.

2.) sector size > 512 needed. See the first post for details.
Thanks. 1st post didn't help directly, but I get the idea that it will require a code change in a loader before it will work for most end users like me.
 

r-win

Well-Known Member
Member
Joined
Oct 10, 2009
Messages
453
Trophies
0
XP
67
Country
Netherlands
charroux said:
Wiimm said:
@charroux

1.) No, because most (all) usb loader have the default sector size 512 hard coded.

2.) sector size > 512 needed. See the first post for details.
Thanks. 1st post didn't help directly, but I get the idea that it will require a code change in a loader before it will work for most end users like me.

You could try USB Loader GX, since we applied the code change posted by Wiimm in his first post.
 

Elfish

Well-Known Member
Member
Joined
Sep 20, 2004
Messages
451
Trophies
0
Age
37
Website
Visit site
XP
520
Country
Gambia, The
Wiimm said:
When creating a WBFS or extracting a raw ISO file from a small game the disc usages is much less than the file size. But when copying the ISO this advantage is lost. This means: WDF is still the best solution for storing ISO images.

that's what i'M planning to do.
sadly there's no loader that loads wdf files directly
frown.gif
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,286
Trophies
0
Location
Germany
Website
wiimmfi.de
XP
1,442
Country
Germany
Free Blocks Table and Bugs

Introduction

A WBFS is divided into blocks with equal size. The block size is a power of 2. The calculation of of the block size is made when formatting a WBFS:[*]Use the minimal block size so that there are less than 2^16 blocks.[*]Adjust the block size so that all management data fits into one block. The block size power is stored into the WBFS header (struct wbfs_head) as member 'wbfs_sec_sz_s'.Based on this calculation only whole multiple of the sector size of the WBFS file or device can be used. The rest is not used. "wwt DUMP" shows the end of the used are in the memory map.

After formatting we have N blocks, addressed by 0..N-1. Block #0 contains the management data. All other blocks (1..N-1) are used to store the discs. A block is not used or exactly assigned to 1 disc.

The "free block table" (FBT) is located at the end of block #0, beginning at a hd sector (parameter "sector size"). For each block there is exact one bit reserved. Is the bit set (1), the block is free (not assigned to a disc). If the bit is unset (0), the block is used by a disc.

Each byte of the FBT contains the status of 8 blocks. The first byte is to manage the blocks 1..8, the next byte for blocks 9..15 and so on. As you see this assignments are NOT zero based! In the beginning of the WBFS there was a "delete bug": The function free_block() used a zero based calculation of the FBT and marked the wrong blocks for free (e.g. marked block 2 instead of block 1 for free). But there are some more bugs in libwbfs. And the more bugs are the reason of this article.

I plan to publish a patch for all discussed bugs later after some tests.


Formatting bug

While formatting the WBFS only the first N/8 bytes (N=number of WBFS blocks) of the FBT are set with the value 0xff. Remember: A set bit means "free block".
[*]If N is a multiple of 8 (N%8 == 0) then 1 block to much is marked as free. But because of the allocation bug (see below) this bug appears only, if N is a multiple of 32.[*]If N%8 > 1 then to less blocks are marked as free. But this means only that not the whole WBFS is used for discs.[*]If N%8 == 1 then the correct number of blocks is marked as free.
This formatting bugs can be fixed very easy:[*]Just unset the wrong bit when opening the WBFS.[*]When formatting or checking the disc then mark all blocks free. This can not be done if there is no space for the additional needed byte in block #0.

Allocation bug


libwbfs uses a run time optimization and scans the FBT in u32 (32 bits) steps. Because of this optimization only the first (N/32)*4 bytes of the FBT are used. And this means that up to to 4 bytes of the FBT are ignored. And this means that up to 30 WBFS blocks will not be used for storing discs. On a WBFS with about 500 GiB the WBFS block size is 2^23 -> up to 240 MiB wasted space.

A patch must be compatible to already existing WBFS. And this means, that only (N/8)*8 blocks can be used, because they are set correctly by the old formatting. The last upto 7 blocks should never be used.


Free-Block bug


The free block function free_block() does not make a check of the block number. If the block number is =0 or >=N wrong bits anywhere in the memory are set. No problem if the WBFS is valid.

But because of other errors in the WBFS file/device it is possible that there are wrong values in the disc memory maps. I have already seen dumps with such errors.


Add-Disc bug


If the WBFS becomes full while adding a disc the disc will not added to the WBFS, that's ok. But the already allocated blocks are not freed.
 

charroux

Well-Known Member
Newcomer
Joined
May 23, 2009
Messages
68
Trophies
0
XP
73
Country
United States
I had to give up, since Cygwin is denying me permission to wwt, and I have no idea how to proceed. Thanks for the info though.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,286
Trophies
0
Location
Germany
Website
wiimmfi.de
XP
1,442
Country
Germany
I found a new small libwbfs bug:

In function wbfs_open_partition() the number of wii sectors is calculated by
Code:
ÂÂÂÂp->n_wii_sec = ( p->n_hd_sec / p->wii_sec_sz ) * p->hd_sec_sz;
ÂÂÂÂ// -> num_of_hd_sectors / wii_sector_size * hd_sector_size
If num_of_hd_sectors is not a multiple of wii_sector_size then the calulated result is to small because of rounding down.

This code looks better (multiply first):
Code:
ÂÂÂÂp->n_wii_sec = (u64)p->n_hd_sec * p->hd_sec_sz / p->wii_sec_sz;

To avoid a 64 bit calculation use this (it works until hd_sector_size is > wii_sec_sz because both are power of 2):
Code:
ÂÂÂÂp->n_wii_sec = p->n_hd_sec / ( p->wii_sec_sz / p->hd_sec_sz );
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,286
Trophies
0
Location
Germany
Website
wiimmfi.de
XP
1,442
Country
Germany
Elfish said:
hmmm interessting. seems like there are tons of bugs :/
Yes, I playing with WBFS since only 6 weeks and I have found a dozend bugs, most harmless but some harmfull. I have fixed all of them in my WWT.

My conclusions:[*] Do not trust WBFS manager or USB loaders that haven't fixed these errors.
.
[*] Do not trust me and my bug fixes because nobody has controlled me.
.
[*] Do not trust WWT because 2.) and because there are perhaps other bugs.
.
[*] Do not use a WBFS for backup, NEVER!
 

mousex

Well-Known Member
Member
Joined
Jan 23, 2009
Messages
986
Trophies
0
XP
114
Country
United States
Wiimm said:
Do not use a WBFS for backup, NEVER!
This irony, don't use the Wii Backup File System for Backups
tongue.gif

If you wouldn't care for compatibilty you could fix these errors, right?
What about a WBFS2 from you, tested until hell, developed in months and not some days, with real documentation and you send it to the USBLoader devs?
 

sotra

Member
Newcomer
Joined
Oct 9, 2009
Messages
9
Trophies
0
XP
2
Country
United States
Hi,

I had an old 250GB Western Digital (model # WD2500B007 - caviar with neon blue and pink lights on the sides) that I used for Wii backup games exclusively, but then I got a 1TB Fantom Drives GreenDrive (model # GDP1000EU) that I chose to partition using EASEUS partition master (25GB FAT32, 25GB NTFS and the rest WBFS). No problems with either of these drives really, but one thing I noticed was that the sizes of the games shows up bigger in most cases on my new HDD.
Some examples: Wii Sports (324MB old drive vs. 512MB on new drive), Wii Fit Plus (1240Mb vs. 1472MB), The Beatles: Rock Band (3524MB vs. 3568MB), Wii Chess (32MB vs. 80MB) etc.

I've tried hooking the new HDD up to my Wii and loaded the games from my original discs through various USB loaders, used WBFS manager and WBFS intelligent GUI on my PC and copied from my old HDD directly to my new HDD using ncWBFSTool. Every single game came out bigger on my new HDD.
Any ideas why? I'd hate to take them all off my new HDD and reformat it a certain way to get them to come up smaller in size.
Thanks in advance for any replies.

Edit: If there's any sort of info about each HDD you want me to post then let me know through which program and how and I will do it ASAP. Thanks!
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,286
Trophies
0
Location
Germany
Website
wiimmfi.de
XP
1,442
Country
Germany
mousex said:
Wiimm said:
Do not use a WBFS for backup, NEVER!
This irony, don't use the Wii Backup File System for Backups
tongue.gif

My thoughts too.

QUOTE(mousex @ Nov 13 2009, 04:13 PM)
If you wouldn't care for compatibilty you could fix these errors, right?
What about a WBFS2 from you, tested until hell, developed in months and not some days, with real documentation and you send it to the USBLoader devs?
I do only fixes that are compatible to WBFS. The idea of WBFS is good but it seems that nobody had made a code view.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,286
Trophies
0
Location
Germany
Website
wiimmfi.de
XP
1,442
Country
Germany
The function alloc_block() have a serious bug: Depending of the value of 'n_wbfs_sec' a non existend block will be allocated. This patch fixes the bug:
CODE--- libwbfs-old.cÂÂÂÂÂÂ 2009-11-14 13:12:58.000000000 +0100
+++ libwbfs.cÂÂ 2009-11-14 13:14:12.000000000 +0100
@@ -450,7 +450,8 @@
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ if (v & (1
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,286
Trophies
0
Location
Germany
Website
wiimmfi.de
XP
1,442
Country
Germany
There is another bug in libwbfs:

If the WBFS becomes full while adding a game the WBFS becomes an invalid state:[*] The disc slot is marked as used, but the info header was not written and may contain illegal data of a previous removed disc or because there is any data on the harddisk.

Deleting this disc may free blocks of other discs because of the undefined data. Because of the free_block() bug, any data in the main memory can be modified == undefined behavior of the program.
.[*] The already alloced blocks are not freed.This is the original code:
Code:
bl = alloc_block(p);
if (bl == 0xffff)
{
ÂÂÂÂERROR("no space left on device (disc full)");
}

And this is my version. It's implemented in wwt since v.0.16a:
CODEbl = alloc_block(p);
if ( bl == WBFS_NO_BLOCK )
{
ÂÂÂÂ// free disc slot
ÂÂÂÂp->head->disc_table[discn] = 0;

ÂÂÂÂ// free already allocated blocks
ÂÂÂÂint j;
ÂÂÂÂfor ( j = 0; j < i; j++ )
ÂÂÂÂ{
ÂÂÂÂÂÂÂÂbl = wbfs_ntohs(info->wlba_table[j]);
ÂÂÂÂÂÂÂÂif (bl)
ÂÂÂÂÂÂÂÂÂÂÂÂfree_block(p,bl);
ÂÂÂÂ}
ÂÂÂÂwbfs_sync(p);
ÂÂÂÂWBFS_ERROR("no space left on device (disc full)");
}
 

mousex

Well-Known Member
Member
Joined
Jan 23, 2009
Messages
986
Trophies
0
XP
114
Country
United States
So, with all these WBFS bugs. DO you think it is safer to store games in .wbfs files on a FAT partition and not on a WBFS drive? I guess if only one game is in a WBFS container most of the bugs can't happen, right?
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,286
Trophies
0
Location
Germany
Website
wiimmfi.de
XP
1,442
Country
Germany
mousex said:
So, with all these WBFS bugs. DO you think it is safer to store games in .wbfs files on a FAT partition and not on a WBFS drive? I guess if only one game is in a WBFS container most of the bugs can't happen, right?

yes
or
use a good wbfs manager
(no idea if there is anyone, haha)
 

mousex

Well-Known Member
Member
Joined
Jan 23, 2009
Messages
986
Trophies
0
XP
114
Country
United States
It's the best. I don't know if it is already good (in the sense of bug free, sure it is good
wink.gif
) due to so many bugs in libwbfs which are maybe not yet found but it's the best if you want a sane WBFS.
I had 4 corrupt games because I deleted MP3 and installed the same game after this and 3 games later (in USBLoader 1.2). I thought MP3 would install at the same place as before but it caused three other games to use blocks at the same places (wwt deleted them during repair as I have the original discs).
 
General chit-chat
Help Users
  • x65943 @ x65943:
    only 22lbs total tho - how much can you really exercise with that?
    Gift
  • x65943 @ x65943:
    I'm a pretty weak guy and even I use 50lb on each side for dumbbell bench and 30lb for each arm with dumbbell curl
    Gift
  • x65943 @ x65943:
    imo without at least a 30lbs dumbbell you won't get a lot of use out of em - and these max out at like 10lbs for each arm
    Gift
  • x65943 @ x65943:
    oh I see you actually get to pick which set you want - so you could buy the 88 lb one
    Gift
  • KenniesNewName @ KenniesNewName:
    Didn't notice in the title it says 44LB/66LB/88LB
    Gift
  • KenniesNewName @ KenniesNewName:
    I really want to start lifting again
    Gift
  • KenniesNewName @ KenniesNewName:
    Gym memberships here are like $40 a month so in like years worth of time I'd have enough just to buy a nice in home set
    Gift
  • x65943 @ x65943:
    that's true, gyms are expensive.
    Gift
  • KenniesNewName @ KenniesNewName:
    I'd rather just have my own equipment instead of not knowing whose sweaty balls were on it
    Gift
  • x65943 @ x65943:
    kenny I'm very concerned now that you may not be working out properly
    Gift
  • KenniesNewName @ KenniesNewName:
    My life isn't proper
    Gift
  • x65943 @ x65943:
    first of all, there should be exactly zero reasons to place your sweaty balls on any of the equipment - generally you are supposed to wear clothing at the gym
    Gift
  • KenniesNewName @ KenniesNewName:
    You aren't going to the right gym if you need to wear clothes
    Gift
  • KenniesNewName @ KenniesNewName:
    Ball sweat drops down like normal sweat if your balls are always dry I'd recommend a doctor
    Gift
  • KenniesNewName @ KenniesNewName:
    We're talking about exercise balls of course
    Gift
  • x65943 @ x65943:
    what to do if I am a doctor
    Gift
  • KenniesNewName @ KenniesNewName:
    Re-read WebMD articles
    Gift
  • KenniesNewName @ KenniesNewName:
    I got myself a cheapo steering wheel cover here's to hoping the sun doesn't burn my finger tips over hot steering wheels
    Gift
  • x65943 @ x65943:
    I have never experienced that
    Gift
  • x65943 @ x65943:
    just where do you live mate
    Gift
  • KenniesNewName @ KenniesNewName:
    Midwest weathers been like 90 all week
    Gift
  • KenniesNewName @ KenniesNewName:
    We don't have garage parking areas at the apartment either so my car bakes in the sun
    Gift
  • KenniesNewName @ KenniesNewName:
    Midwest weather is so bipolar in the summer
    Gift
  • x65943 @ x65943:
    also in midwest, and today at least was really nice - in the 70s with light drizzle and good cloud cover
    Gift
  • x65943 @ x65943:
    have been quite a few scorchers this year tho
    Gift
    x65943 @ x65943: have been quite a few scorchers this year tho