Hacking Inside WBFS

charroux

Well-Known Member
Newcomer
Joined
May 23, 2009
Messages
68
Trophies
0
XP
73
Country
United States
Wiimm said:
I have made some more small tests. With sector size 512 (default) a WBFS must be smaller as 2 TiB. But when increasing the sector size larger WBFS partitions seems possible. To create a test partition use WWT and type for example:

Code:
wwt init --size 4t --sector-size 1024 --split-size 2t --force a.wbfs
Splitting is needed because the maximal single file size.
Amazing stuff!

1) Is this currently compatible with any USB Loader?
2) What is the command line to format/init to allow more than 500 games on a partition? And is that compatible with any USB Loader?
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
@charroux

1.) No, because most (all) usb loader have the default sector size 512 hard coded.

2.) sector size > 512 needed. See the first post for details.
 

charroux

Well-Known Member
Newcomer
Joined
May 23, 2009
Messages
68
Trophies
0
XP
73
Country
United States
Wiimm said:
@charroux

1.) No, because most (all) usb loader have the default sector size 512 hard coded.

2.) sector size > 512 needed. See the first post for details.
Thanks. 1st post didn't help directly, but I get the idea that it will require a code change in a loader before it will work for most end users like me.
 

r-win

Well-Known Member
Member
Joined
Oct 10, 2009
Messages
453
Trophies
0
XP
67
Country
Netherlands
charroux said:
Wiimm said:
@charroux

1.) No, because most (all) usb loader have the default sector size 512 hard coded.

2.) sector size > 512 needed. See the first post for details.
Thanks. 1st post didn't help directly, but I get the idea that it will require a code change in a loader before it will work for most end users like me.

You could try USB Loader GX, since we applied the code change posted by Wiimm in his first post.
 

Elfish

Well-Known Member
Member
Joined
Sep 20, 2004
Messages
451
Trophies
1
Age
39
Website
Visit site
XP
564
Country
Gambia, The
Wiimm said:
When creating a WBFS or extracting a raw ISO file from a small game the disc usages is much less than the file size. But when copying the ISO this advantage is lost. This means: WDF is still the best solution for storing ISO images.

that's what i'M planning to do.
sadly there's no loader that loads wdf files directly
frown.gif
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
Free Blocks Table and Bugs

Introduction

A WBFS is divided into blocks with equal size. The block size is a power of 2. The calculation of of the block size is made when formatting a WBFS:[*]Use the minimal block size so that there are less than 2^16 blocks.[*]Adjust the block size so that all management data fits into one block. The block size power is stored into the WBFS header (struct wbfs_head) as member 'wbfs_sec_sz_s'.Based on this calculation only whole multiple of the sector size of the WBFS file or device can be used. The rest is not used. "wwt DUMP" shows the end of the used are in the memory map.

After formatting we have N blocks, addressed by 0..N-1. Block #0 contains the management data. All other blocks (1..N-1) are used to store the discs. A block is not used or exactly assigned to 1 disc.

The "free block table" (FBT) is located at the end of block #0, beginning at a hd sector (parameter "sector size"). For each block there is exact one bit reserved. Is the bit set (1), the block is free (not assigned to a disc). If the bit is unset (0), the block is used by a disc.

Each byte of the FBT contains the status of 8 blocks. The first byte is to manage the blocks 1..8, the next byte for blocks 9..15 and so on. As you see this assignments are NOT zero based! In the beginning of the WBFS there was a "delete bug": The function free_block() used a zero based calculation of the FBT and marked the wrong blocks for free (e.g. marked block 2 instead of block 1 for free). But there are some more bugs in libwbfs. And the more bugs are the reason of this article.

I plan to publish a patch for all discussed bugs later after some tests.


Formatting bug

While formatting the WBFS only the first N/8 bytes (N=number of WBFS blocks) of the FBT are set with the value 0xff. Remember: A set bit means "free block".
[*]If N is a multiple of 8 (N%8 == 0) then 1 block to much is marked as free. But because of the allocation bug (see below) this bug appears only, if N is a multiple of 32.[*]If N%8 > 1 then to less blocks are marked as free. But this means only that not the whole WBFS is used for discs.[*]If N%8 == 1 then the correct number of blocks is marked as free.
This formatting bugs can be fixed very easy:[*]Just unset the wrong bit when opening the WBFS.[*]When formatting or checking the disc then mark all blocks free. This can not be done if there is no space for the additional needed byte in block #0.

Allocation bug


libwbfs uses a run time optimization and scans the FBT in u32 (32 bits) steps. Because of this optimization only the first (N/32)*4 bytes of the FBT are used. And this means that up to to 4 bytes of the FBT are ignored. And this means that up to 30 WBFS blocks will not be used for storing discs. On a WBFS with about 500 GiB the WBFS block size is 2^23 -> up to 240 MiB wasted space.

A patch must be compatible to already existing WBFS. And this means, that only (N/8)*8 blocks can be used, because they are set correctly by the old formatting. The last upto 7 blocks should never be used.


Free-Block bug


The free block function free_block() does not make a check of the block number. If the block number is =0 or >=N wrong bits anywhere in the memory are set. No problem if the WBFS is valid.

But because of other errors in the WBFS file/device it is possible that there are wrong values in the disc memory maps. I have already seen dumps with such errors.


Add-Disc bug


If the WBFS becomes full while adding a disc the disc will not added to the WBFS, that's ok. But the already allocated blocks are not freed.
 

charroux

Well-Known Member
Newcomer
Joined
May 23, 2009
Messages
68
Trophies
0
XP
73
Country
United States
I had to give up, since Cygwin is denying me permission to wwt, and I have no idea how to proceed. Thanks for the info though.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
I found a new small libwbfs bug:

In function wbfs_open_partition() the number of wii sectors is calculated by
Code:
ÂÂÂÂp->n_wii_sec = ( p->n_hd_sec / p->wii_sec_sz ) * p->hd_sec_sz;
ÂÂÂÂ// -> num_of_hd_sectors / wii_sector_size * hd_sector_size
If num_of_hd_sectors is not a multiple of wii_sector_size then the calulated result is to small because of rounding down.

This code looks better (multiply first):
Code:
ÂÂÂÂp->n_wii_sec = (u64)p->n_hd_sec * p->hd_sec_sz / p->wii_sec_sz;

To avoid a 64 bit calculation use this (it works until hd_sector_size is > wii_sec_sz because both are power of 2):
Code:
ÂÂÂÂp->n_wii_sec = p->n_hd_sec / ( p->wii_sec_sz / p->hd_sec_sz );
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
Elfish said:
hmmm interessting. seems like there are tons of bugs :/
Yes, I playing with WBFS since only 6 weeks and I have found a dozend bugs, most harmless but some harmfull. I have fixed all of them in my WWT.

My conclusions:[*] Do not trust WBFS manager or USB loaders that haven't fixed these errors.
.
[*] Do not trust me and my bug fixes because nobody has controlled me.
.
[*] Do not trust WWT because 2.) and because there are perhaps other bugs.
.
[*] Do not use a WBFS for backup, NEVER!
 

mousex

Well-Known Member
Member
Joined
Jan 23, 2009
Messages
986
Trophies
0
XP
115
Country
United States
Wiimm said:
Do not use a WBFS for backup, NEVER!
This irony, don't use the Wii Backup File System for Backups
tongue.gif

If you wouldn't care for compatibilty you could fix these errors, right?
What about a WBFS2 from you, tested until hell, developed in months and not some days, with real documentation and you send it to the USBLoader devs?
 

sotra

Member
Newcomer
Joined
Oct 9, 2009
Messages
9
Trophies
0
XP
2
Country
United States
Hi,

I had an old 250GB Western Digital (model # WD2500B007 - caviar with neon blue and pink lights on the sides) that I used for Wii backup games exclusively, but then I got a 1TB Fantom Drives GreenDrive (model # GDP1000EU) that I chose to partition using EASEUS partition master (25GB FAT32, 25GB NTFS and the rest WBFS). No problems with either of these drives really, but one thing I noticed was that the sizes of the games shows up bigger in most cases on my new HDD.
Some examples: Wii Sports (324MB old drive vs. 512MB on new drive), Wii Fit Plus (1240Mb vs. 1472MB), The Beatles: Rock Band (3524MB vs. 3568MB), Wii Chess (32MB vs. 80MB) etc.

I've tried hooking the new HDD up to my Wii and loaded the games from my original discs through various USB loaders, used WBFS manager and WBFS intelligent GUI on my PC and copied from my old HDD directly to my new HDD using ncWBFSTool. Every single game came out bigger on my new HDD.
Any ideas why? I'd hate to take them all off my new HDD and reformat it a certain way to get them to come up smaller in size.
Thanks in advance for any replies.

Edit: If there's any sort of info about each HDD you want me to post then let me know through which program and how and I will do it ASAP. Thanks!
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
mousex said:
Wiimm said:
Do not use a WBFS for backup, NEVER!
This irony, don't use the Wii Backup File System for Backups
tongue.gif

My thoughts too.

QUOTE(mousex @ Nov 13 2009, 04:13 PM)
If you wouldn't care for compatibilty you could fix these errors, right?
What about a WBFS2 from you, tested until hell, developed in months and not some days, with real documentation and you send it to the USBLoader devs?
I do only fixes that are compatible to WBFS. The idea of WBFS is good but it seems that nobody had made a code view.
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
The function alloc_block() have a serious bug: Depending of the value of 'n_wbfs_sec' a non existend block will be allocated. This patch fixes the bug:
CODE--- libwbfs-old.cÂÂÂÂÂÂ 2009-11-14 13:12:58.000000000 +0100
+++ libwbfs.cÂÂ 2009-11-14 13:14:12.000000000 +0100
@@ -450,7 +450,8 @@
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ if (v & (1
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
There is another bug in libwbfs:

If the WBFS becomes full while adding a game the WBFS becomes an invalid state:[*] The disc slot is marked as used, but the info header was not written and may contain illegal data of a previous removed disc or because there is any data on the harddisk.

Deleting this disc may free blocks of other discs because of the undefined data. Because of the free_block() bug, any data in the main memory can be modified == undefined behavior of the program.
.[*] The already alloced blocks are not freed.This is the original code:
Code:
bl = alloc_block(p);
if (bl == 0xffff)
{
ÂÂÂÂERROR("no space left on device (disc full)");
}

And this is my version. It's implemented in wwt since v.0.16a:
CODEbl = alloc_block(p);
if ( bl == WBFS_NO_BLOCK )
{
ÂÂÂÂ// free disc slot
ÂÂÂÂp->head->disc_table[discn] = 0;

ÂÂÂÂ// free already allocated blocks
ÂÂÂÂint j;
ÂÂÂÂfor ( j = 0; j < i; j++ )
ÂÂÂÂ{
ÂÂÂÂÂÂÂÂbl = wbfs_ntohs(info->wlba_table[j]);
ÂÂÂÂÂÂÂÂif (bl)
ÂÂÂÂÂÂÂÂÂÂÂÂfree_block(p,bl);
ÂÂÂÂ}
ÂÂÂÂwbfs_sync(p);
ÂÂÂÂWBFS_ERROR("no space left on device (disc full)");
}
 

mousex

Well-Known Member
Member
Joined
Jan 23, 2009
Messages
986
Trophies
0
XP
115
Country
United States
So, with all these WBFS bugs. DO you think it is safer to store games in .wbfs files on a FAT partition and not on a WBFS drive? I guess if only one game is in a WBFS container most of the bugs can't happen, right?
 

Wiimm

Developer
OP
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
mousex said:
So, with all these WBFS bugs. DO you think it is safer to store games in .wbfs files on a FAT partition and not on a WBFS drive? I guess if only one game is in a WBFS container most of the bugs can't happen, right?

yes
or
use a good wbfs manager
(no idea if there is anyone, haha)
 

mousex

Well-Known Member
Member
Joined
Jan 23, 2009
Messages
986
Trophies
0
XP
115
Country
United States
It's the best. I don't know if it is already good (in the sense of bug free, sure it is good
wink.gif
) due to so many bugs in libwbfs which are maybe not yet found but it's the best if you want a sane WBFS.
I had 4 corrupt games because I deleted MP3 and installed the same game after this and 3 games later (in USBLoader 1.2). I thought MP3 would install at the same place as before but it caused three other games to use blocks at the same places (wwt deleted them during repair as I have the original discs).
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
  • Xdqwerty @ Xdqwerty:
    @btjunior, how old are you really?
  • Psionic Roshambo @ Psionic Roshambo:
    Especially a hacked one
  • Xdqwerty @ Xdqwerty:
    and idk if something will happen to my ps3 if i connect it to wifi with hen activated
  • Psionic Roshambo @ Psionic Roshambo:
    I remember when the Atari 2600 was new and exciting lol
  • Psionic Roshambo @ Psionic Roshambo:
    It could get banned from PSN but you can change your ID I think?
  • Xdqwerty @ Xdqwerty:
    also gonna install twilight menu in my r4 flashcard
  • Psionic Roshambo @ Psionic Roshambo:
    One thing that just occurred to me.... The sound on the 2600 sucked less back then the harsh sound we hear now is from infinitely better speakers we have now, back when the 2600 was new speakers produced a almost muffled sound, like CRTs made old graphics look slightly better.
  • Psionic Roshambo @ Psionic Roshambo:
    I wonder if I could recommend that to some emulation devs that perhaps the sound could use some smoothing out to simulate those old TVs
  • Psionic Roshambo @ Psionic Roshambo:
    I think a few of the early systems could benefit from that, at least up to the 8 bit generation, by the 16 bit generation I think TVs had gotten a lot better in almost every way
  • Xdqwerty @ Xdqwerty:
    i dont have an sd card adapter but I have an usb sd card adapter
  • K3Nv2 @ K3Nv2:
    Old people games
  • Xdqwerty @ Xdqwerty:
    its not the one that comes with the r4
  • Xdqwerty @ Xdqwerty:
    doesnt work (my flashcard is from r4isdhc.com)
  • Xdqwerty @ Xdqwerty:
    might install ysmenu first
  • Psionic Roshambo @ Psionic Roshambo:
    Try Wood firmware
  • Psionic Roshambo @ Psionic Roshambo:
    For your R4
  • Psionic Roshambo @ Psionic Roshambo:
    It's old but it's the best firmware out for DS stuff
  • Xdqwerty @ Xdqwerty:
    it says it only works for the original R4, R4i Gold (r4ids.cn), R4iDSN (r4idsn.com) and Acekard R.P.G.
  • Xdqwerty @ Xdqwerty:
    nvm it does support mine
  • Xdqwerty @ Xdqwerty:
    but why choose it over ysmenu @Psionic Roshambo?
  • Xdqwerty @ Xdqwerty:
    bc im stupid?
  • Xdqwerty @ Xdqwerty:
    yea ik im stupid
    Xdqwerty @ Xdqwerty: yea ik im stupid