Hacking Wii backup Manager for Windows

[fig2k4]

@zizer
Covers will be disabled by default.

@airline38
At the highest code level, getting the IOS from any of the formats is just the same for me. I won't be able to do it for archives though because that would need enough data to be extracted. Because the partitions aren't shuffled when removing updates etc, the data partition will still be around at least the 200 MB mark in the ISO. Extracting 200 MB would be quite pointless for such a small piece of info.

As a speed up for loading drives, I was thinking about creating a directory listing file in the WBFS folder. This file would contain all the needed info. eg. filename, title, scrubbed size, MD5hash. The IOS version could be stored there too.

When a drive is mounted, I'd read this listing file instead of having to parse all the WBFS files etc. Drives would be loaded almost instantly using that. The listing file could be quickly and easily updated when changes are made to the drive.

I'll be in touch when a test version is ready.

@madrat
I'm not sure why this error doesn't happen for everyone, including me. But I missed a space between the date and time when removing backup logs which should cause an error every time, for everyone.

It'll be fixed in the next version. Until then, you'll need to delete everything from the logs folder before starting the app.


Other known bugs.

Since changing to Unicode, the columns weren't saved correctly (even though the code looks fine) so the app crashed on start up. I worked round the problem by compressing the column data and storing that in the ini file. I'm guessing that nobody really moves or hides the columns or this would've been reported too.

WBFS files can't be used from inside archives. I get a Sector count mismatch error because it's reading a small temp file. I need to enable force mode when reading those. It should work after that, enough to get the Wii disc header. It's really pointless compressing WBFS files, but I did come across one.

 

[redia]

As a speed up for loading drives, I was thinking about creating a directory listing file in the WBFS folder. This file would contain all the needed info. eg. filename, title, scrubbed size, MD5hash. The IOS version could be stored there too.
When a drive is mounted, I'd read this listing file instead of having to parse all the WBFS files etc. Drives would be loaded almost instantly using that. The listing file could be quickly and easily updated when changes are made to the drive.
I love the idea of speeding up the loading process.
just a question ? why don't you include all these informations including hashes (created locally) in the DB ?

I'll be in touch when a test version is ready.

DIBS !!!!!! lol

@madrat
I'm not sure why this error doesn't happen for everyone, including me. But I missed a space between the date and time when removing backup logs which should cause an error every time, for everyone.

It'll be fixed in the next version. Until then, you'll need to delete everything from the logs folder before starting the app.
it happened to me once, but as the computer I was running the test was really in a bad shape, I simply thought I had a corruption, but not linked to your app...
since I am running it on another computer it is running flawlessly..


QUOTEOther known bugs.

Since changing to Unicode, the columns weren't saved correctly (even though the code looks fine) so the app crashed on start up. I worked round the problem by compressing the column data and storing that in the ini file. I'm guessing that nobody really moves or hides the columns or this would've been reported too.

I never tested the columns as the standard you established looks simply perfect to me

QUOTE

WBFS files can't be used from inside archives. I get a Sector count mismatch error because it's reading a small temp file. I need to enable force mode when reading those. It should work after that, enough to get the Wii disc header. It's really pointless compressing WBFS files, but I did come across one.

weird, I thought you could not access the wbfs file inside the archive do to sequential access... kinda similar to the wiiscrubber issue...
I really don't see the interest in compressing a WBFS file anyway... are there still people doing that ?

R

 

[fig2k4]

I love the idea of speeding up the loading process.
just a question ? why don't you include all these informations including hashes (created locally) in the DB ?I could do that, but I'd still need to search all the folders and gather a list of game IDs to look up the database. Having a simple text file in the WBFS folder with the file list means that I only need to open that one file to get all the info. If the text file doesn't exist then I'd just open the drive as normal, parse all the files and create it. It would be slow to mount the drive the first time, then after that it would literally be instant.

Another way would be to only store the filenames and IDs in the text file, but I might as well just store everything there since it would be so much quicker.

Edit: I'd need to have a "Refresh" function in the menu too, just in case you manually copy WBFS file in Windows explorer. That would just refresh the info by deleting the text file and remounting the drive normally.

it happened to me once, but as the computer I was running the test was really in a bad shape, I simply thought I had a corruption, but not linked to your app...
since I am running it on another computer it is running flawlessly..I think it might depend on your language settings in Windows because it's never happened to me, or the bug wouldn't be there.

QUOTE

weird, I thought you could not access the wbfs file inside the archive do to sequential access... kinda similar to the wiiscrubber issue...
I really don't see the interest in compressing a WBFS file anyway... are there still people doing that ?

That reminds me, I'll add the on the fly archive stuff to the todo list. So archives don't need to be fully extracted before transfers etc. I need to create a stream class that buffers data that usually accessed randomly, since it's only possible to read archives sequentially.

The only reason I can see for "storing" a WBFS file in a rar is for error checking. When it's successfully extracted, you know it's intact. If it's got a WBM MD5 hash though, there's no need for it at all because you can check if the relevant data is intact that way.

 

[madrat]

Thank you for the information and when will be there the next release?

 

[redia]

I could do that, but I'd still need to search all the folders and gather a list of game IDs to look up the database. Having a simple text file in the WBFS folder with the file list means that I only need to open that one file to get all the info. If the text file doesn't exist then I'd just open the drive as normal, parse all the files and create it. It would be slow to mount the drive the first time, then after that it would literally be instant.

Another way would be to only store the filenames and IDs in the text file, but I might as well just store everything there since it would be so much quicker.
I disagree.. but you love it when I do

having a text file AND a DB means to location for corruption/deletion, two ways of managing information, two files to handle..

plus if you put the information in the DB, it makes the file portable, at one point even maybe shareable...
my idea was to simply load the directory listing (shouldn't be long) and then lookup for the filename in the DB... only the unknown files would be parsed for further information.
all the files handled by your app have the correct name, so only files which have manually been renamed, or added would be parsed... only one time because then the information would be available in the DB...
once again, just my two cents

QUOTEThat reminds me, I'll add the on the fly archive stuff to the todo list. So archives don't need to be fully extracted before transfers etc. I need to create a stream class that buffers data that usually accessed randomly, since it's only possible to read archives sequentially.

The only reason I can see for "storing" a WBFS file in a rar is for error checking. When it's successfully extracted, you know it's intact. If it's got a WBM MD5 hash though, there's no need for it at all because you can check if the relevant data is intact that way.

wow... I did not think of that. smart thinking once again !!

R

 

[Wiimm]

BTW, regarding the IOS version of a game ,I don't know how difficult for u to check the tmd.bin
from a wbfs file . The IOS value is located at address '18b' of the tmd.bin (DATA partition).
QUOTE(fig2k4 @ Dec 31 2010, 10:59 AM) @airline38
At the highest code level, getting the IOS from any of the formats is just the same for me. I won't be able to do it for archives though because that would need enough data to be extracted. Because the partitions aren't shuffled when removing updates etc, the data partition will still be around at least the 200 MB mark in the ISO. Extracting 200 MB would be quite pointless for such a small piece of info.

airline38 asked me how wit+wwt do it:

wit+wwt have an internal virtual file layer that allow commands and functions to read+write with ISO offsets. If the ISO file is a WBFS this layer loads the "inode" data (header info in the first wbfs sector). This data contains a mapping table. With this map it's very easy to read the partition table and then the partition header including the tmd. This means after setup of the map there are only 2 random access read operations needed (size 1KB+2KB) to determine IOS num and many more info.

CISO, WDF and WIA are very similar, they only use other kinds of mapping tables.

BTW:
I have already extended the inode info. Each inode contains 0x100 bytes but only the first 0x60 are used from libwbfs (the wbfs subsystem). My tools use the area 0x80..0xff for extended infos like timestamps. Here is the data structure: http://opensvn.wiimm.de/viewvc/wii/trunk/w...iew=markup#l290
There is enough room to store more important values.

 

[Wiimm]

The only reason I can see for "storing" a WBFS file in a rar is for error checking. When it's successfully extracted, you know it's intact. If it's got a WBM MD5 hash though, there's no need for it at all because you can check if the relevant data is intact that way.

Images (WBFS+CISO+WDF) can only be compressed if they are decrypted. Here is some more info: http://wit.wiimm.de/info/wia.html#intro

 

[airline38]

@Wiimm, thanks for sharing these "know-how" of "wit+wwt" . (although most of these "know-how"
are "over my head")

@fig2k4 , hopefully u can find a similar way (or a new way) to easily extract IOS number
from a WBFS file.

 

[winny.uk]

@Wiimm: thanks for your input.

I'm always amazed at how quickly these ideas are considered and turned into reality

Stunning work!

 

[XFlak]

I could do that, but I'd still need to search all the folders and gather a list of game IDs to look up the database. Having a simple text file in the WBFS folder with the file list means that I only need to open that one file to get all the info. If the text file doesn't exist then I'd just open the drive as normal, parse all the files and create it. It would be slow to mount the drive the first time, then after that it would literally be instant.

Another way would be to only store the filenames and IDs in the text file, but I might as well just store everything there since it would be so much quicker.

Edit: I'd need to have a "Refresh" function in the menu too, just in case you manually copy WBFS file in Windows explorer. That would just refresh the info by deleting the text file and remounting the drive normally.

Just a simple thought from a simple batch programmer, lol, but what if after reading (& counting) the list of game IDs WBM did a simple count of the folders (or files) in the WBFS folder? Then if the numbers do not match, it would automatically refresh the drive without needing the user to press the "refresh" button.

I really like the idea of loading a HDD quicker, keep up the great work fig2k4!

 

[sleakbug]

Any chance of getting the ESRB rating in the column listing? I'm up several hundred titles and manually sorting thru to separate my kids games from mine would be easier if I were able to sort and select to extract wbfs to separate folders? BTW great work, I've been using WBM almost daily and love it, please keep up the great work!

 

[fig2k4]

@redia
plus if you put the information in the DB, it makes the file portable, at one point even maybe shareable..
That's the only point I could disagree with.

The database isn't really shareable unless it's exported to a generic format. The text file is already in that generic format that anyone can parse and use.

Since the database isn't really implemented yet, I could let you test with the text file. It would be much quicker and easier for me to do. It's probably only a few minutes work if I get into it. When I'm in a position to compare with the database method, we'll be able to see which is best to use.

@Wiimm
I can get the IOS version from my Wii disc class. When loading the WBFS though, I usually only read the WBFS disc header. Using the Wii disc would just take a little longer when there's a few hundred games.

Images (WBFS+CISO+WDF) can only be compressed if they are decrypted.
I was thinking about that. How well would the data be compressed if it wasn't encrypted. Does wit already do that to compress the data?

@XFlak
Good idea.

@sleakbug
That would be part of the database stuff. When that's finished, I can then add more columns to the other lists, with the ESRB rating etc.

 

[Cyan]

It seems your new FAT32 formater you programmed doesn't set the partition type in the MBR/EBR.
USBloaderGX was reading the MBR and detecting the old NTFS format instead of the new FAT32 format.

GX r1027 is now reading the partition header instead of the MBR to get the information, so it's ok.

Could you check about that issue ?
Thank you

 

[Wiimm]

Images (WBFS+CISO+WDF) can only be compressed if they are decrypted.
I was thinking about that. How well would the data be compressed if it wasn't encrypted. Does wit already do that to compress the data?

wit can decrypt an image with "wit copy --enc decrypt source dest" for a long time. The new file format WIA decrypt and compress it with LZMA (the 7zip algorithm). The already posted link http://wit.wiimm.de/info/wia.html#stat shows some pack ratios. The advantage of wia is the possible random access (in chunks). And here is the GBA discussion about WIA: http://gbatemp.net/index.php?showtopic=250617#entry3074890 . WIA is hard to implement but it shows the possible compression. I don't use it because access is to slow.

 

[redia]

@redia
plus if you put the information in the DB, it makes the file portable, at one point even maybe shareable..
That's the only point I could disagree with.

The database isn't really shareable unless it's exported to a generic format. The text file is already in that generic format that anyone can parse and use.

Since the database isn't really implemented yet, I could let you test with the text file. It would be much quicker and easier for me to do. It's probably only a few minutes work if I get into it. When I'm in a position to compare with the database method, we'll be able to see which is best to use.

the key word in my sentence was : maybe you just added the not.... lol

the idea behind my suggestion, is the more information is keep in one location, the "easier" it would be ultimately to share this information..
(obviously you are 100% right when you mention that the DB is not ready yet, implementing the txt file for test purpose is the way to go...)

As always I am at your service whenever you need to test this new feature.
I think the result my be relevant as I am moving my wbfs files around, so I am accessing some on a USB drive, some on an internal drive, some on a windows network drive and some on an unraid...
so really 4 different speed.
(why so many location ? simply because you told me you were going to change the size of the copy, so I wanted to have enough files for testing... I am currently consolidating, but it is no rush)

Cheers,
R

 

[toss01]

See "Options->Settings->Drives/DVD tab". Untick "Use mounted volumes only". That should do the trick.

Error I get is Valid drive but no WBFS folder Harddisk Volume 1 NTFS

in My Computer Manage drives it has

This NTFS as system reserved under Disk 0 so I dont think this is the drive

My USB harddrive Disk 1 Fat 32 as F: 85GB and 214 GB Healthy (Primary Position)

ncWBFStool lists it as WBFS@PhysicalDrive1Partition1

Hi fig2k4

Any help with this please

Thanks

 

[yixiar]

It's sound great. But i found a bug.

I use a 500G hard disk to storage this loader and games. My problem is:
1. Wii can't read disk while it is formated one 500G NTFS disk. But CFG can be run in a first fat(32) disk.
2.The wbfs game can't be read from the NTFS disk, but can be read from WBFS format disk.

the readme file say CFG run well with ntfs format, but in fact it can't be captible well.

I tested 0.3.6build48, the newest version.

Am I wrong, or the app has bugs?

 

[fig2k4]

@Cyan
In the other post I mentioned that FatFormat32 doesn't touch the MBR. That must be where the problem lies then, if the MBR needs updated. I have a look to see what API functions are available..

@Wiimm
Cheers for the info.

@redia
I'm still working away on this when I get the time, which is much less when the the kids are off school. By the time they go to bed at night, I'm so bloody exhausted that I can't be arsed doing anything, lol. It's getting there though..

@toss01
The only thing I can think of is that you've accidentally ticked the drive to make it hidden, in the settings. Try deleting the settings.ini and run the app again.

If that doesn't work, send me the WBM log and a screenshot of your drives in Windows Disk Management. I should be able to see what the problem is from that.

@yixiar
Right now I'm using a 500GB NTFS drive with CFG and it works fine.

I'm using IOS222, which is Hermes V5 installed with the default options. For CFG, I'm using the build that uses IOS222 by default (cfg??-222.dol).

Guitar hero etc. needs IOS223, also using defaults from the Hermes installer.

When I get round to some testing, I'll format a drive as NTFS and test, but it should work OK.

 

[Wabsta]

Hey, just wanted to stop by to thank you for making this awesome application, makes everything a lot easier!

Cheers.

 

[redia]

@redia
I'm still working away on this when I get the time, which is much less when the the kids are off school. By the time they go to bed at night, I'm so bloody exhausted that I can't be arsed doing anything, lol. It's getting there though..

LOL,

I hear you.
I wonder why we call that holiday

it should be theyarekillingmeday....

my favorite answer to this is that I can't wait to get back to the office so I can sleep...

but at the same time.. we can't get enough of the time we spend with them...

Cheers,
R