Hacking WBFS defrag

Would you like to have a WBFS defrag util?

  • Yes

    Votes: 38 71.7%
  • No

    Votes: 15 28.3%

  • Total voters
    53

PsyBlade

Snake Charmer
OP
Member
Joined
Jul 30, 2009
Messages
2,204
Trophies
0
Location
Sol III
XP
458
Country
Gambia, The
edit:
Since this was linked in an more active topic is thought I update this thread

I abandond this project
because it was to hard to create a good algotithem to determin which part of which game should go where.
Good meaning ending with only 1 fragment for each game and few free space fragments,
while keeping the amount of copied data low.

If someone supplys one I might pick it up again.
The rest should be easy.


original:
I want to code some more and try to create a efficient defrag algorithm for wbfs.

Before I get going I would like to know how much interest is there for something like it.
I'm gonna need people willing to test it and compare the speed.
If there aren't any there is no point in even starting
I simply don't have access to enough fragmented drives for a meaningful trial.

I know that there is a chance that the improvement will be negligible,
but I'm not going to trust some arbitrary sounding definition that only fragments
 

Category

Trainee Romhacker
Member
Joined
Jan 29, 2008
Messages
252
Trophies
0
Location
Bournemouth
Website
masseffectzero.blogspot.co.uk
XP
477
Country
That would be a great way to get coding practice, but is it really a necessary program? Is WBFS fragmentation an issue that hits performance? I haven't heard of any cases of that myself.

And beyond that, how many people would have a use for it? With FAT32 loading and all the conveniences it brings, it shocks me people still format their drives as WBFS.
 

PsyBlade

Snake Charmer
OP
Member
Joined
Jul 30, 2009
Messages
2,204
Trophies
0
Location
Sol III
XP
458
Country
Gambia, The
Necessary? Nope definitely not.
But on the other hand, how much of the things people do for fun are?

How much it impacts performance is what I try to find out.

No I don't think there will be many users but some still prefer WBFS.

Personally I would do the switch immediately (again).
Even if it would still involve copying all games somewhere else and back again.
 

Taleweaver

Storywriter
Member
Joined
Dec 23, 2009
Messages
8,685
Trophies
2
Age
43
Location
Belgium
XP
8,066
Country
Belgium
I'm amazed that there are already 2 people willing to put their entire game collection at risk to test a tool for an obsolete partitioning system.


It's not my intention to bash WBFS, but come on...it's been nearly two years since FAT32 became possible, and all that time I heard nothing but bogus arguments in favor of WBFS. The reason it's still being used is because 1. the interweb has quite some outdated guides floating around and 2. because people don't want to change from whatever works.

"2." will be a culprit. Your beta testers (or even end users) will obviously need a backup of their backups*, and if they go through that trouble...why not just reformat it to FAT32 and use Wii Backup Manager to convert the games? Since it'll be a blank disk, it'll be completely defragged in the end!


So...yeah. I appreciate coding projects, and it IS something new. I just hope a program like this won't make people think WBFS can even be considered "equal" to FAT32.







*you gotta love how the word "backup" is used on gbatemp
wink.gif
 

Wiimm

Developer
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
Do you know, that some FAT/NTFS defragmentation tools calculates a fragmentation ration in fragments per MiB. Most of them say, that a ratio of =2 MiB (a 350 GB drive has chunks with 8 MiB). That means, that the ratio can never be larger than 0.5. And in the terms of above, a WBFS is never fragmented.

This is the theory, and here are some tests:
A year ago I have created 2 WBFS, one absolutely defragmented (1 disc behind the other) and one where every disc has many fragments (I think it was with 16 MiB fragments). Then I have made 2 tests:
a.) reading the data in line and randomly with an PC.
b.) playing some games (e.g. "Sims 3" do much loading in every scene change)
I have found only minimal differences for case a), but not for case b).

Will say: It is not worth it!


BTW:
It make more sense, to have an more intelligent block selection algorithm adding a new disc to a WBFS. I have already implemented one (together with some other new WBFS stuff) in wwt for a long time, but not activated, because it is not tested very well. I have stopped this development + testing, because of the tests and results above. You can enable it, if compiling my tools with "make new all".


The poll:
I have answered, that I don't need the tool, but I will test it. Linux+cli are welcome and I will never test a windows-click-solution. Full automated scripts are the best!

PsyBlade said:
Especially don't try to talk me out of it. It would be futile. Sorry for my try
wink.gif



Bladexdsl said:
not needed fat32 is pretty much the way now
Why using the worse variant, if WBFS works very fine and fast without fragmentation issues? FAT/NTFS is a good solution for users who can only manage their games with a mouse. For me, WBFS is fast and perfect and without any problems since nearly 2 years.

If developing Mario Kart Custom, I need only about 40 seconds to put an file system structure to my WBFS drive (conversion to WBFS on the fly). A FAT solutions need much longer and will fragment the drive.

Overall I see no reason to switch to FAT/NTFS, but some reasons to stay at WBFS.


QUOTE(Wever @ Oct 7 2011, 06:08 AM)
So...yeah. I appreciate coding projects, and it IS something new. I just hope a program like this won't make people think WBFS can even be considered "equal" to FAT32.
Why should he making WBFS worse to be equal to FAT?
 
  • Like
Reactions: tmv_josue

Taleweaver

Storywriter
Member
Joined
Dec 23, 2009
Messages
8,685
Trophies
2
Age
43
Location
Belgium
XP
8,066
Country
Belgium
Wiimm said:
Wever said:
So...yeah. I appreciate coding projects, and it IS something new. I just hope a program like this won't make people think WBFS can even be considered "equal" to FAT32.
Why should he making WBFS worse to be equal to FAT?
It isn't making WBFS worse. More or less the contrary: it would be making WBFS less worse than it is. However, it'll take much more of these kinds of programming projects before I would say the partitioning structure doesn't matter.

I'll throw in an example that happened just yesterday. I have a PC with 2 front end USB ports. While I was copying data on my hard drive, I also added some stuff on my SD card (which sat in a USB-to-SD cardreader). Since that data was much smaller, it finished first. So I used windows 7's "safely remove hardware and eject media" on it.

...and what works 99 out of 100 times, failed on me that one time. For maybe a split second, my USB drive went unpowered, causing the data flow to stop and the drive even not to be recognized anymore. It wouldn't work on my wii either.
However, when I reinserted it to my pc, windows immediately asked to scan and fix everything.

Well...I'm not the one to quickly praise microsoft's programs, but it did the job fine. And I bet there are at least a dozen decent freeware programs that would be able to do the same thing.
However, if I had had a WBFS-formatted drive, I most likely would have lost everything on the drive.

So...no. I'm not a fan of WBFS. And until there is a decent recovery program for when things screw up (things WILL do that at some point...it's called "life"
wink.gif
), I will discourage it (@Psyblade: please don't take it personally). At best, it would be trying to reinvent the wheel. But perhaps the better analogy would be to rearrange the deck chairs on the titanic.
 

Wiimm

Developer
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
When I unplug my wbfs drive while writing a wbfs using my wit tool, nothing happens, because the complete WBFS management data is hold in memory and only written, if adding the disc is finished. The only critical moment is while writing this management block. I have rewritten the LIBWBFS for this secure handling.
Oh, this is better than FAT handling!

My tools are very fast, because of writing the game data in linear order and storing the management data last. FAT(NTFS management data is to large and for block allocations, some non linear read+write operations are needed.
Oh, this is better and faster than FAT handling!

Before writing to a WBFS, my tools do a *complete* integrity check and repair small errors without user confirmation. This takes not a second and most users don't see it. Only for harder errors the program exits without writing.
Oh, this is better and much faster than FAT handling!

There are no fragmentation problems because of the very large chunks.
Oh, this is better than FAT handling!

All thinks above work only, because the management data of a WBFS is very small. FAT and NTFS can't do such optimized things.

And now please tell me again the technical advantage of a FAT system. The only I know is, that it is easier to copy/rename files if using mouse control. But this little advantage is blown if using cli.


I'm a WBFS fan. I have switched to FAT, saw all disadvantages, and switched back to WBFS.
 
  • Like
Reactions: tmv_josue

smf

Well-Known Member
Member
Joined
Feb 23, 2009
Messages
6,638
Trophies
2
XP
5,834
Country
United Kingdom
Wiimm said:
And now please tell me again the technical advantage of a FAT system. The only I know is, that it is easier to copy/rename files if using mouse control. But this little advantage is blown if using cli.
I'm a WBFS fan. I have switched to FAT, saw all disadvantages, and switched back to WBFS.

I only ever use the command line & I still find FAT32 easier to use.

At the time I switched there were major problems with WBFS partition corruption. Alot of the problems may have been fixed since, but it's too little too late.
The convenience of FAT32 is just too much compared to WBFS, all I need is wbfs_file.exe & xcopy.

WBFS only exists due to laziness. The PS2 usb loader had used FAT32 for years earlier.
 

Wiimm

Developer
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
For clearness:
All USB loader use *always* WBFS, either directly or as WBFS file in a FAT/NTFS file system. Therefor using FAT/NTFS adds only a new layer of file access. I think the WBFS support stays forever.
(btw: ULoader supports CISO, an alternative container for WBFS).

I think, everyone can make it's own decision which variant he/she use. But I don't like, if so many people without knowledge about WBFS say, that WBFS useless/old/bad/faulty. I see clear advantages for me and so I'm stay on it.
 

smf

Well-Known Member
Member
Joined
Feb 23, 2009
Messages
6,638
Trophies
2
XP
5,834
Country
United Kingdom
Wiimm said:
For clearness:
All USB loader use *always* WBFS, either directly or as WBFS file in a FAT/NTFS file system. Therefor using FAT/NTFS adds only a new layer of file access. I think the WBFS support stays forever.
(btw: ULoader supports CISO, an alternative container for WBFS).

I think, everyone can make it's own decision which variant he/she use. But I don't like, if so many people without knowledge about WBFS say, that WBFS useless/old/bad/faulty. I see clear advantages for me and so I'm stay on it.

There is ISO support in some loaders too, but you need to use NTFS for that.
The difference between WBFS as a file system and WBFS as a file format is convenience.

I have no problem with anyone preferring WBFS, I don't want to start a religious crusade or anything.
However there is still enough misinformation going around that people are still suprised that you can use >4gb games on FAT32.

As a single FAT32 partition is the best all round choice, it deserves at least making sure everyone is aware of it.
 

Lucif3r

Well-Known Member
Member
Joined
Sep 10, 2011
Messages
1,468
Trophies
0
XP
228
Country
The only reason you can use 4gb+ games on FAT32 is because the file gets splitted. So its not really a 4gb+ file anymore.


For pure Wii use, WBFS has a lot of advantages, but the compability of FAT32 across different platforms is hard to ignore.
Therefore Im using FAT32 and will most likely stick to it untill the Wii ends up in a closet lol.
 

Taleweaver

Storywriter
Member
Joined
Dec 23, 2009
Messages
8,685
Trophies
2
Age
43
Location
Belgium
XP
8,066
Country
Belgium
When I unplug my wbfs drive while writing a wbfs using my wit tool, nothing happens, because the complete WBFS management data is hold in memory and only written, if adding the disc is finished. The only critical moment is while writing this management block. I have rewritten the LIBWBFS for this secure handling.
Oh, this is better than FAT handling!

My tools are very fast, because of writing the game data in linear order and storing the management data last. FAT(NTFS management data is to large and for block allocations, some non linear read+write operations are needed.
Oh, this is better and faster than FAT handling!

Before writing to a WBFS, my tools do a *complete* integrity check and repair small errors without user confirmation. This takes not a second and most users don't see it. Only for harder errors the program exits without writing.
Oh, this is better and much faster than FAT handling!

There are no fragmentation problems because of the very large chunks.
Oh, this is better than FAT handling!

All thinks above work only, because the management data of a WBFS is very small. FAT and NTFS can't do such optimized things.

And now please tell me again the technical advantage of a FAT system. The only I know is, that it is easier to copy/rename files if using mouse control. But this little advantage is blown if using cli.


I'm a WBFS fan. I have switched to FAT, saw all disadvantages, and switched back to WBFS.
I must admit this information is all new to me (except maybe the 'no fragmentation problems', but all I've heard of that were vague, unsupported claims). But while on paper, WBFS may indeed be a superior system, the fact remains that most people use it as a means of just doing things with barely knowing what they're doing. That makes troubleshooting hard, since they're being vague about it.
 
  • Like
Reactions: 1 person

Wiimm

Developer
Member
Joined
Aug 11, 2009
Messages
2,292
Trophies
1
Location
Germany
Website
wiimmfi.de
XP
1,519
Country
Germany
I must admit this information is all new to me (except maybe the 'no fragmentation problems', but all I've heard of that were vague, unsupported claims). But while on paper, WBFS may indeed be a superior system, the fact remains that most people use it as a means of just doing things with barely knowing what they're doing. That makes troubleshooting hard, since they're being vague about it.
That's a main problem here. I know, that for most people FAT/NTFS management is much easier than WBFS. But the arguments are most wrong, because the most people here doesn't know, what WBFS is from a technical view point. On the other hand, many WBFS people using the old and buggy "WBSF Manager 3.0" (and other old and buggy tools) and if the tool makes an error they say: "shit WBFS" and not "shit WBFS manager".
 
  • Like
Reactions: tmv_josue

tueidj

I R Expert
Member
Joined
Jan 8, 2009
Messages
2,569
Trophies
0
Website
Visit site
XP
999
Country
The filesystem in use on a drive does not influence its hardware compatibility. A cios can either read sectors from it or not regardless of what those sectors contain.
 

smf

Well-Known Member
Member
Joined
Feb 23, 2009
Messages
6,638
Trophies
2
XP
5,834
Country
United Kingdom
Because some older drives only work with WBFS. For game loading, compatibility is WBFS > FAT32 > NTFS

When Hermes was the only CIOS that supported FAT32 & your hard drive only worked with Waninkoko's that may have been an issue.

CIOS d2x is the best choice right now, if your drive doesn't work with that then you should speak to them about it.
 

JoostinOnline

Certified Crash Test Dummy
Member
Joined
Apr 2, 2011
Messages
11,005
Trophies
1
Location
The Twilight Zone
Website
www.hacksden.com
XP
4,338
Country
United States
Because some older drives only work with WBFS. For game loading, compatibility is WBFS > FAT32 > NTFS

When Hermes was the only CIOS that supported FAT32 & your hard drive only worked with Waninkoko's that may have been an issue.

CIOS d2x is the best choice right now, if your drive doesn't work with that then you should speak to them about it.
Works fine for me, I just know a couple people who can't run FAT32 (idk about NTFS) on their HDD, no matter what cIOS is used. It always works fine for WBFS though.
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
  • No one is chatting at the moment.
  • ZeroT21 @ ZeroT21:
    it wasn't a question, it was fact
  • BigOnYa @ BigOnYa:
    He said he had 3 different doctors apt this week, so he prob there. Something about gerbal extraction, I don't know.
    +1
  • ZeroT21 @ ZeroT21:
    bored, guess i'll spread more democracy
  • LeoTCK @ LeoTCK:
    @K3Nv2 one more time you say such bs to @BakerMan and I'll smack you across the whole planet
  • K3Nv2 @ K3Nv2:
    Make sure you smack my booty daddy
    +1
  • LeoTCK @ LeoTCK:
    telling him that my partner is luke...does he look like someone with such big ne
    eds?
  • LeoTCK @ LeoTCK:
    do you really think I could stand living with someone like luke?
  • LeoTCK @ LeoTCK:
    I suppose luke has "special needs" but he's not my partner, did you just say that to piss me off again?
  • LeoTCK @ LeoTCK:
    besides I had bigger worries today
  • LeoTCK @ LeoTCK:
    but what do you know about that, you won't believe me anyways
  • K3Nv2 @ K3Nv2:
    @BigOnYa can answer that
  • BigOnYa @ BigOnYa:
    BigOnYa already left the chat
  • K3Nv2 @ K3Nv2:
    Biginya
  • BigOnYa @ BigOnYa:
    Auto correct got me, I'm on my tablet, i need to turn that shit off
  • K3Nv2 @ K3Nv2:
    With other tabs open you perv
  • BigOnYa @ BigOnYa:
    I'm actually in my shed, bout to cut 2-3 acres of grass, my back yard.
  • K3Nv2 @ K3Nv2:
    I use to have a guy for that thanks richard
  • BigOnYa @ BigOnYa:
    I use my tablet to stream to a bluetooth speaker when in shed. iHeartRadio, FlyNation
  • K3Nv2 @ K3Nv2:
    While the victims are being buried
  • K3Nv2 @ K3Nv2:
    Grave shovel
  • BigOnYa @ BigOnYa:
    Nuh those goto the edge of the property (maybe just on the other side of)
  • K3Nv2 @ K3Nv2:
    On the neighbors side
    +1
  • BigOnYa @ BigOnYa:
    Yup, by the weird smelly green bushy looking plants.
    K3Nv2 @ K3Nv2: https://www.the-sun.com/news/10907833/self-checkout-complaints-new-target-dollar-general-policies...