Hacking Wii backup Manager for Windows

[phiillh]

fig2k4

I cannot seem to get this to work with my setup.
I have my laptop set with a wbfs partition on its hard drive to allow me to put my iso's onto a wbfs drive and copy them to my usb drive.
The problem is your program see's the wbfs drive but says it is an unknown fs
Any idea's? or was it never meant to work like this?

 

[redia]

fig2k4

I cannot seem to get this to work with my setup.
I have my laptop set with a wbfs partition on its hard drive to allow me to put my iso's onto a wbfs drive and copy them to my usb drive.
The problem is your program see's the wbfs drive but says it is an unknown fs
Any idea's? or was it never meant to work like this?

Phiilh,

I do not understand what you are trying to do.
you have a WBFS partition (not a fat32)
you convert your ISO to this WBFS partition
and then you copy them to a USB drive ?
I am a little bit confused.

could you please clarify what you are doing.
I will try to help as soon as this is clear.
make sure you do not mix WBFS Partition and WBFS File

Cheers,
R

 

[fig2k4]

@a67543210

As phiillh said, the files are split at just under 2 gig. Every file in the set should be the same size, except the last one. If this isn't the case, let me know because it's a bug.

@phiillh

I haven't tested WBFS on a second partition. I have a feeling that the number of drive sectors that I'm passing to open the WBFS partition is for the whole drive. In the recent versions I turned off the "force" flag which would ignore the difference in sector count and still open the drive. I'll try this out with 2 partitions to check.

@Tristar and the wrong ISO size

After a quick look last night, I see that all the relevant data is written to the ISO, it's just the last empty part that's missing. I'm not sure why because my converted code looks identical. The extract ISO function is one of the most simple routines in libWBFS, strange...

Looks like the export game list function will be on the back burner until I fix these bugs...

 

[phiillh]

redia

yes that is what i do, put the iso onto a wbfs partition on my laptop and then copy the game from there to a wbfs partitioned usb drive.
if find this easier as my iso's are stored on a headless server.

the issue i have is that the program see's the wbfs partition on by laptop but says it is an unsupported fs and shows none of the games on there

p

 

[oggzee]

@Tristar and the wrong ISO size
After a quick look last night, I see that all the relevant data is written to the ISO, it's just the last empty part that's missing. I'm not sure why because my converted code looks identical. The extract ISO function is one of the most simple routines in libWBFS, strange...

Not sure if this is your problem, but, since wbfs does not contain the real source iso size, it is assumed to be that well known single layer size and so the file is pre-allocated to that size:
u64 single_size = (d->p->n_wii_sec_per_disc/2)*0x8000ULL;
u64 size = single_size;
file_truncate(fileno(f), size);
And then the contents are filled, which can be less than the full single layer size. In case of dual layer games it is written out as much as there is data and so the resulting iso will likely be smaller by some amount.
I am considering adding a tag and value to the disc header where the original iso size could be stored, what i have in mind is this: put in the last 8 bytes this: "SIZE"XXXX where "SIZE" is the literal ascii string and XXXX would be a hton(sectors) u32 value of the original iso size. So that dual layer discs could be extracted to a same size as it was in the original, but i'm not sure if it's worth the effort.

 

[redia]

redia

yes that is what i do, put the iso onto a wbfs partition on my laptop and then copy the game from there to a wbfs partitioned usb drive.
if find this easier as my iso's are stored on a headless server.

the issue i have is that the program see's the wbfs partition on by laptop but says it is an unsupported fs and shows none of the games on there

p

ok so if I got it right you have :
- your iso on a headless server
- a laptop with two (at least) partition, one of which is WBFS
- a usb key

and you are :
- converting your iso from the headless server to the laptop in the WBFS partition
- copying the game from the WBFS partition on the latptop to the USB key WBFS partition

that sounds like a lot of trouble simply to copy a game. why don't you directly put it on the usb key ? I don't get it (not being judgmental, I am simply trying to understand your setup)

is your WBFS partition on the laptop primary or extended ?
if the partition is only for temporary transfer, and you have an iso copy of your games I would advise you a few roads :
1. switch to WBFS file on fat32
this way you won't have a problem with the recognition of the partitions anymore
and if you REALLY want to keep the wbfs partition on your USB key. you can have only the laptop partition as WBFS file (don't forget it can be any filesystem fat or ntfs if it is not meant to be accessible from the WII)

or
2. delete and recreate the wbfs partition using the appropriate tools.
a. delete the partition
b. create a fat32 primary partition
c. format the partition using wiibackupmanager to WBFS
d. try accessing it again

what operating system are you running ?
how big is the wbfs partition on your laptop ?
what application are you normally using to access it ?

give us as much details as you can.

Cheers,
R

 

[phiillh]

ok here goes

the iso's are stored on a windows home server

my laptop runs win7 and connects wirelessly to the server
the laptop has an ntfs partition and a 100gb wbfs partition, set to primary iirc

the reason for my setup is that the file transfer speed to convert to iso to wbfs is slow over wireless anyway without adding the transfer via usb
also my usb drive won't hold all of the games so i use the wbfs partition on the laptop to quickly move games on and off the usb drive

i've been using wbfs 3.0 to convert the games to the laptop wbfs drive and using wbfssync to quickly move games between drives

i'm happy with the wbfs partitions at the moment as i use usb loader gx which doesn't support fat32 drive loading

my aim from this is the reduce the size of the files stored on my server using wbfs file or ciso so they take less room and are easier to transfer but at the same time only have to use one program to reduce the sizing, carry out the conversions and move the games between drives

overly complicated i know but its the best way for me, unless you have any other suggestions, other than getting a larger usb drive

 

[redia]

ok here goes

the iso's are stored on a windows home server

my laptop runs win7 and connects wirelessly to the server
the laptop has an ntfs partition and a 100gb wbfs partition, set to primary iirc

the reason for my setup is that the file transfer speed to convert to iso to wbfs is slow over wireless anyway without adding the transfer via usb
also my usb drive won't hold all of the games so i use the wbfs partition on the laptop to quickly move games on and off the usb drive

i've been using wbfs 3.0 to convert the games to the laptop wbfs drive and using wbfssync to quickly move games between drives

i'm happy with the wbfs partitions at the moment as i use usb loader gx which doesn't support fat32 drive loading

my aim from this is the reduce the size of the files stored on my server using wbfs file or ciso so they take less room and are easier to transfer but at the same time only have to use one program to reduce the sizing, carry out the conversions and move the games between drives

overly complicated i know but its the best way for me, unless you have any other suggestions, other than getting a larger usb drive

phiillh,

don't worry it is YOUR setup.. which means it is YOUR call.

in a scenario like that I would definitely recommend that you use .wbfs files on your laptop.
you can keep two partition (reformat the WBFS one as fat32, or NTFS it is up to you)
or you can have only one partition on your laptop (you can keep it NTFS)

if you are trying to store on a partition other than fat in the application make sure in options/setup make sure you have the "Allow any file system for .wbfs file storage" checked.

the first time you select your partition a WBFS folder will be created in the root.
all the .wbfs files will be stored there.

and if you still have the ability to transfer them to your WBFS usb key (as you are running GX)

I hope this helps,
R

 

[phiillh]

thanks redia

guess i'll just have to use either wbfs manager to extract the iso's as i don't have all of them on my server or use wbfssync to make syncscrub copies so i can delete the wbfs partition, unless the wbfs partition can be made useable as is.

phiillh

 

[redia]

thanks redia

guess i'll just have to use either wbfs manager to extract the iso's as i don't have all of them on my server or use wbfssync to make syncscrub copies so i can delete the wbfs partition, unless the wbfs partition can be made useable as is.

phiillh

phiillh,

better off extracting them as ciso.. they are compatible with wiibackupmanager (and smaller than iso)
this way you don't have to convert them from syncscrub back to iso before using them.

just my two cents.
or you can wait for omega frost to have some free time. I am pretty sure he will implement the .wbfs file format if he decide to keep on developping his app.

Cheers,
R

 

[phiillh]

thanks redia

guess i'll just have to use either wbfs manager to extract the iso's as i don't have all of them on my server or use wbfssync to make syncscrub copies so i can delete the wbfs partition, unless the wbfs partition can be made useable as is.

phiillh

phiillh,

better off extracting them as ciso.. they are compatible with wiibackupmanager (and smaller than iso)
this way you don't have to convert them from syncscrub back to iso before using them.

just my two cents.
or you can wait for omega frost to have some free time. I am pretty sure he will implement the .wbfs file format if he decide to keep on developping his app.

Cheers,
R

i think i'll end up staying the way things are at the moment as i cannot find one program which does what i want to acheive.

my ideal is to have a compressed version of the iso stored on my server and use a program on my laptop to access this compressed file and move it to my usb drive either to a wbfs partition or wbfs file.
another ideal would be to be able to access the iso on the server and compress it there without moving it to the laptop first

 

[gizmomelb]

fig2k4 - first off thank you for an excellent and easy to use program for managing our ISO, cISO, WBFS files!

I have a couple of minor suggestions / enhancement requests though please:

1. in the main Wii Backup Manager screen on the left hand side where it has the dropdowns for the Cover and Country, could it please be possible that the Country dropdown list only shows the selected countries in the same selected order as defined in the Options \ Settings \ Covers \ Country Download Order?

2. where the cover is displayed in the left hand panel, sure you can click the dropdown to select between the Flat, Disc, Fake 3D and Full covers - but could you please change it so that if you click in the panel where the cover is it then 'steps' through and shows the next type of cover please?
eg: showing the Fake 3D cover, click in the panel (or click on the cover if it's easy enough to check the 'hit box') and the cover then changes to the Full Cover display?

3. lastly (and no need for this at all), maybe mouse wheel up / down zooms in / zooms out for the display of the cover?

lastly this may be a user error, but I have my ISO images on my internal D: drive, if I select my internal E: drive it doesn't let me convert the ISO to CISO or WBFS file formats. Should I be able to?

Thank you once again.

 

[fig2k4]

@oggzee

I just tested with wbfs_file_1.2 and it behaves exactly like my app. ie. I converted to .wbfs then back to .iso and the file size was 4,685,037,568 like Tristar mentioned.

Did you add the file allocation stuff in a later version?


Also, I think I've found a way to determine if it should be dual or single layered. Basically, count the used blocks for the disc and check to see if the used data is larger than the single layer size.

It doesn't determine if the original disc was dual layered, but at least the ISO sizes will be correct.

The Delphi code...

function TWBFSDisc.UsedSectorCount: cardinal;
var
ÂÂÂÂi: integer;
begin
ÂÂÂÂresult := 0;
ÂÂÂÂfor i := 0 to fPartition.WBFSSectorsPerDisc - 1 do
ÂÂÂÂÂÂÂÂif ntohs(header.wlbatable[i]) > 0 then
ÂÂÂÂÂÂÂÂÂÂÂÂinc(result);
end;

Then in the ExtractISO() function you can calculate the file size to allocate.

usedcount := fActiveDisc.UsedSectorCount;

if usedcount * SrcNlb > cSingleLayerSectorCount * DstNlb then
begin
ÂÂFileSize := int64(cDoubleLayerSectorCount) * cWiiSectorSize;
ÂÂRealSectorsPerDisc := WBFSSectorsPerDisc;
end
else
begin
ÂÂFileSize := int64(cSingleLayerSectorCount) * cWiiSectorSize;
ÂÂRealSectorsPerDisc := WBFSSectorsPerDisc div 2;
end;

for i := 0 to RealSectorsPerDisc - 1 doÂÂ//this replaces p->n_wbfs_sec_per_disc
begin

I changed the variable names to be more readable in my code, but I'm sure you can see what I mean.


@gizmomelb
1. Sure.
2. I already implemented that but commented it out a while ago before I released it. I'll enable it again..
3. I plan on adding a zoom function. Maybe allow to maximize the cover view or something too so you can get a better look at the covers..
4. Does drive E: show the correct file system in the status bar? If not then go to options and tick "allow any filesystem..." and try again..

Direct ISOCISO isn't implemented yet. As a quick workaround I could add indirect convertion to .wbfs and then to CISO/ISO in one transparent step. Just until I can get back the the wbfs code and implement the direct drive to drive, iso to ciso etc.. It would take twice as long but it's still only one step for the user and multiple files can be batch processed.

 

[oggzee]

@oggzee
I just tested with wbfs_file_1.2 and it behaves exactly like my app. ie. I converted to .wbfs then back to .iso and the file size was 4,685,037,568 like Tristar mentioned.
Did you add the file allocation stuff in a later version?

Yes:
v1.3:
* Fixed extracted iso size on Windows
* Fixed ETA info for iso extraction

So to be more precise, i only fixed it so it works correctly on windows, on linux it was working right also in 1.2, the problem was with the fruncate function used on windows/mingw, which i replaced it with proper win32 apis in 1.3.

Also, I think I've found a way to determine if it should be dual or single layered. Basically, count the used blocks for the disc and check to see if the used data is larger than the single layer size.

It doesn't determine if the original disc was dual layered, but at least the ISO sizes will be correct.

The Delphi code...

[ CUT ]

Then in the ExtractISO() function you can calculate the file size to allocate.

[ CUT ]

I changed the variable names to be more readable in my code, but I'm sure you can see what I mean.

I have that implemented too:

CODE

libwbfs.c:
u32 wbfs_sector_used2(wbfs_t *p,wbfs_disc_info_t *di, u32 *last_blk)
{
ÂÂÂÂÂÂÂÂu32 tot_blk=0,j;
ÂÂÂÂÂÂÂÂfor(j=0;jn_wbfs_sec_per_disc;j++)
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂif(wbfs_ntohs(di->wlba_table[j])) {
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ*last_blk = j;
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂtot_blk++;
ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ}
ÂÂÂÂÂÂÂÂreturn tot_blk;
}

wbfs.c:
ÂÂÂÂ// check if the game is DVD9..
ÂÂÂÂ/*
ÂÂÂÂÂÂ u32 comp_blk;
ÂÂÂÂÂÂ u32 last_blk;
ÂÂÂÂÂÂ comp_blk = wbfs_sector_used2(d->p, d->header, &last_blk);
ÂÂÂÂÂÂ u64 real_size = (u64)last_blk * (u64)d->p->wbfs_sec_sz;
ÂÂÂÂÂÂ u64 dual_size = (d->p->n_wii_sec_per_disc)*0x8000ULL;
ÂÂÂÂÂÂ */
ÂÂÂÂÂÂÂÂu64 single_size = (d->p->n_wii_sec_per_disc/2)*0x8000ULL;
ÂÂÂÂÂÂÂÂu64 size = single_size;
ÂÂÂÂÂÂÂÂ//if (real_size > single_size) size = dual_size;

This will also let you determine if the disc was dual layered, as it checks which is the last real offset of a used block. But it doesn't help in any way so it is commented out. Because the disc is scrubbed you can't know the exact size the iso was, just the max used size, and another problem is, wbfs block size is 2mb (or more) while the wii disc sector size is 32kb and the full single or double layer size is not even a multiple of 2mb, so even if you would make a trick to store the last wbfs block always, it wouldn't help as you would get the size rounded to a 2mb size (up or down, depending on what you decide to be better)

 

[fig2k4]

Thanks again. It's sorted now. I just set the correct size after it's finished writing the disc data.

In Windows (not sure if it happens linux) when seeking past the file size and then writing to the file, there's a loooong freeze while the empty part of the file is allocated. I've fixed that too by just writing the zeros instead of letting the OS do it.

This will also let you determine if the disc was dual layered, as it checks which is the last real offset of a used block. But it doesn't help in any way so it is commented out. Because the disc is scrubbed you can't know the exact size the iso was, just the max used size, and another problem is, wbfs block size is 2mb (or more) while the wii disc sector size is 32kb and the full single or double layer size is not even a multiple of 2mb, so even if you would make a trick to store the last wbfs block always, it wouldn't help as you would get the size rounded to a 2mb size (up or down, depending on what you decide to be better)

I was just thinking of a way to output valid sizes, not rebuild the original ISO. If the used data is over the single layer size, then the ISO should be dual layered eh?

 

[oggzee]

Thanks again. It's sorted now. I just set the correct size after it's finished writing the disc data.
In Windows (not sure if it happens linux) when seeking past the file size and then writing to the file, there's a loooong freeze while the empty part of the file is allocated. I've fixed that too by just writing the zeros instead of letting the OS do it.
On windows it depends if the sparse file mode is enabled or not. You can check libwbfs_win32 to find the sparse enable mode code for windows. But I think it also depends on the filesystem, so yeah i guess on fat32 it will take a while. I was mostly testing and converting the files on NTFS and after the conversion was done i just copied the files over to the fat32 partition, so I haven't checked how fast will it behave if doing the conversion directly on the fat32, but I suspect your approach will probably be faster on fat32 and slower on ntfs. Which is probably better.

QUOTE(fig2k4 @ Nov 6 2009, 05:38 PM) I was just thinking of a way to output valid sizes, not rebuild the original ISO. If the used data is over the single layer size, then the ISO should be dual layered eh?

Yes, that's what my previously pasted code does, but as i said, the dual layer backup isos seem to be of various sizes and I don't know if the proper size should actually be 2x single layer? The single layer backups are always the same size...

 

[fig2k4]

Wow, I've never used sparse files before

I can write a 4 gig ISO in a couple of seconds with a small game.

Am I right in thinking that if a game is only 80 meg, the 4 gig ISO will only ever take up 80 meg of disk space? Given that the zero parts of the file will never be read.. When I copy the ISO in explorer, the new file does use 4 gig, but the original still only uses 80 meg. I can't see any reason not to enable sparse files for any file system that supports it in my app.

 

[oggzee]

Wow, I've never used sparse files before

I can write a 4 gig ISO in a couple of seconds with a small game.

Am I right in thinking that if a game is only 80 meg, the 4 gig ISO will only ever take up 80 meg of disk space? Given that the zero parts of the file will never be read..
That is correct.

When I copy the ISO in explorer, the new file does use 4 gig, but the original still only uses 80 meg.
That's an unfortunate implementation of copy in windows explorer

On linux, you can control the behaviour of copied sparse files with the --sparse=auto / --sparse=always / --sparse=never parameter.

QUOTE(fig2k4 @ Nov 6 2009, 06:46 PM)

I can't see any reason not to enable sparse files for any file system that supports it in my app.

I do that in wbfs_file

 

[fig2k4]

Updated to 0.1.8 a little early (See main post), but the ISO file sizes being wrong kinda forced it..

 

[redia]

Updated to 0.1.8 a little early (See main post), but the ISO file sizes being wrong kinda forced it..

mmm
I noticed a title sub-menu... including "update titles.txt"...
I do not have time to test anything right now.
but this sounds promissing.

thanks
R