Homebrew USB Loader won't read dir after adding more files

[kevmic]

I've been experiencing some weird behaviour over last few days... searched the forum but can't find the same question.

I have USB Loader running successfully on my Wii. I backed up about 10 of my Wii games in to wbfs to test out... Put them on a USB Stick and worked great.

Then I tried a few Gamecube games... Put those on another stick... Worked fine.

Ran happily with those for a few weeks while the big backup of my game discs was going. 1200-ish Wii titles and 600-ish Gamecube.

Bought a new 4TB drive and split it 1TB Fat32 for GC and 3TB NTFS for Wii. By now I had about 250 GC available and 800 Wii ready, named and organised in the appropriate convention.

First I copied the rest of the Wii on (800 odd). Again, all fine. Everything working.

Next I dropped the GC on (250 ish). Still fine.

But the problem is when I add more titles on. After the full Wii set was ready, I added them on... But now USB Loader won't read the Wii titles at all. Zero titles loaded. The full set of GC (600 ish) on the Fat32 partition load fine.

So I started experimenting. Renamed the wbfs folder to something else... Made a new wbfs folder and moved 4 titles in to it. USB Loader read it fine.

Suspecting a bad file amongst the 1200... I moved just 100 of them in to wbfs. Now USB loader won't read the wbfs folder at all again.

So I repeated the 4 title test and it will read it. But as soon as I add another title... Even from my original set of 10 working titles... USB loader refuses to read the dir. Even after I remove the last added title and go back to the original 4, USB Loader refuses to read the dir.

Has anyone experience this before?

One complication is I could not use the same PC for all my backups. Could NTFS permissions be confusing things?

Sorry for long winded thread... Thought I'd explain every step.

 

[ber71]

I suspect that these new files got saved past the 2TB mark.
You've created a 3TB NTFS partition. The Wii can read it... up to the 2TB mark (from the beginning of the disk), not beyond that.
That's because the Wii uses 32bit adressing for logical sectors. Not because MBR or GPT, not because of fat or NTFS, as many people assume.
To be able to use more than 2TB of physical storage all partitions accounted, you have to use 4K logical sectors instead of the more common 512bytes sectors.
WD drives are the best for that purpose, but have to be reinitialized with that sector size. There is an official utility for this.

 

[jeannotte]

I've been experiencing some weird behaviour over last few days... searched the forum but can't find the same question.

I have USB Loader running successfully on my Wii. I backed up about 10 of my Wii games in to wbfs to test out... Put them on a USB Stick and worked great.

Then I tried a few Gamecube games... Put those on another stick... Worked fine.

Ran happily with those for a few weeks while the big backup of my game discs was going. 1200-ish Wii titles and 600-ish Gamecube.

Bought a new 4TB drive and split it 1TB Fat32 for GC and 3TB NTFS for Wii. By now I had about 250 GC available and 800 Wii ready, named and organised in the appropriate convention.

First I copied the rest of the Wii on (800 odd). Again, all fine. Everything working.

Next I dropped the GC on (250 ish). Still fine.

But the problem is when I add more titles on. After the full Wii set was ready, I added them on... But now USB Loader won't read the Wii titles at all. Zero titles loaded. The full set of GC (600 ish) on the Fat32 partition load fine.

So I started experimenting. Renamed the wbfs folder to something else... Made a new wbfs folder and moved 4 titles in to it. USB Loader read it fine.

Suspecting a bad file amongst the 1200... I moved just 100 of them in to wbfs. Now USB loader won't read the wbfs folder at all again.

So I repeated the 4 title test and it will read it. But as soon as I add another title... Even from my original set of 10 working titles... USB loader refuses to read the dir. Even after I remove the last added title and go back to the original 4, USB Loader refuses to read the dir.

Has anyone experience this before?

One complication is I could not use the same PC for all my backups. Could NTFS permissions be confusing things?

Sorry for long winded thread... Thought I'd explain every step.

Hello,

looking at this post from @ca032769 this might help you ==>https://gbatemp.net/threads/usb-loader-gx.149922/page-1129#post-6662953

 

[kevmic]

I suspect that these new files got saved past the 2TB mark.
You've created a 3TB NTFS partition. The Wii can read it... up to the 2TB mark (from the beginning of the disk), not beyond that.
That's because the Wii uses 32bit adressing for logical sectors. Not because MBR or GPT, not because of fat or NTFS, as many people assume.
To be able to use more than 2TB of physical storage all partitions accounted, you have to use 4K logical sectors instead of the more common 512bytes sectors.
WD drives are the best for that purpose, but have to be reinitialized with that sector size. There is an official utility for this.

@ber71 Thanks! I understand a bit about the ins and outs of disk formats (having been in the game for more years than I care to admit)... But this had not occured to me to try.

Considering the Wii's age, rather unexpected. Far as I remember AF was still debatable when purchasing drives around 2010? But then again... I suppose the sole format the Wii was built for was DVD which is 2048 bytes per sector... technically AF. So I guess this does make sense after all.

Anyway, thanks for mentioning this. It would have taken me weeks, if not more, to get down to this level of troubleshooting.


Also thanks to @jeannotte for linking @ca032769 's summary. Great work from @ca032769 ! I think we might not need to go through quite so many steps to get a drive in to the same state... so will try myself and post a slightly optimised procedure if it works.

Cheers. Will update in a day or so.

 

[kevmic]

Everything working great now! Really appreciate the advice.

Building on top of of @ca032769 's work, I have a slightly easier route to prepare a large drive for both GameCube and Wii using only Windows tools and no paid-for applications required.

Recording my findings and method here in case someone else needs this:

Requirements:
1 - Logical format
As mentioned by @ber71 above, the critical requirement is the HDD must be formatted with a sector size of 4096 bytes (aka Advanced Format if you want to read up on it). As a brief summary... all drives these days will be manufactured as 4096 bytes per sector, but most consumer level external HDD will present themselves as 512 bytes per sector for backwards compatibly and will be shipped that way. You need a drive which can be reformatted with a logical format of 4096 bytes per sector. I hear from these forums that WD drives have this feature. I had already bought a new 4TB Buffalo drive before knowing this, but luckily this brand also has a downloadable tool to do this. (My model is “HD-NRLD4.0U3-BA”. Not sure if Buffalo is a brand available in your country, it’s big in Japan where I am.) So maybe that gives people two brands known to support this.

2 - GameCube requires Fat32
GameCube titles must be on a Fat32 formatted partition. Traditionally the largest size of a Fat32 partition is 32GB, but we will work around this.

3 - NTFS better for Wii
The largest file size Fat32 can support is 4GB, so it would be nice to use NTFS for the Wii titles because they will often be greater than 4GB.


Setup
When setting up your drive partitions, there is no need to stick to MBR. GPT partition style is totally fine, and preferable if you want any of your partitions to be greater than 2TB.

In my example, I have 0.9TB of GameCube titles and 2.2 TB of Wii titles. So I would like to partition my 4TB drive as 1TB Fat32 for GameCube and 3TB NTFS for Wii.

After running your tool to reformat as 4096 bytes per sector, there will likely be one partition already created for the whole size of the disk. Let’s first wipe everything, and force it to GPT.

Step-by-step guide to using diskpart:
- Win-R to bring up run prompt and type “diskpart”
- In the new command prompt that opens, wait a moment for it to prepare itself and make cursor available.
- Type “list disk” and hit enter.
The output table lists all your disks and you can easily identify which one is your large drive (4TB in my case). The last column, GPT indicates with a star if it is GPT partition style or not. But regardless, with just a few commands we can force it and be sure.
- Type “select disk n”, n being the number of your disk to modify, the 4TB one in my case. (Drive #Zero will most likely be your C drive).
- Type “clean” to erase the partition table
- Type “convert gpt” to force the partition table style to GPT. If you saw a star in the output table it was already GPT, but if unsure just run this anyway.

Now we can create our partitions, using the basic inbuilt windows functions. In Computer Management go to Windows Disk Manager. You will see your large disk, shaded and saying “Unallocated”. The GameCube partition must be first on the disk, so let’s create that. Right-click on the Unallocated bar and select “New Simple Volume”. I need 1TB in my case, so created an NTFS partition as 1,000,000 MB, labeled “GC” and I assigned letter "G:" to it. Use any letter you like. Yes, I know… we said above we need Fat32 but created it as NTFS! We will change this soon.

Now let’s create our second partition for the Wii. Right click in the Unallocated section and again do "New Simple Volume" and create another NTFS partition with all of the remaining space. I labeled this partition “Wii” and assigned letter “W:” (again, does not matter what drive letter you use… whatever is convenient).

We created both partitions as NTFS because Windows doesn’t offer Fat32 on large volumes (over 32GB). But there is a free tool we can use to reformat the partition as Fat32. Download it from here: ridgecrop.co.uk / guiformat.exe

Run this to reformat the G: drive (or whichever letter you assigned to the GameCube partition) as Fat32. The appropriate allocation size will already default, in my case, for a 1TB drive its 32k (32768).



That’s it! Your drive is partitioned, formatted and ready. In the root of the GameCube drive create a folder “games”… In the root of the Wii drive create folder “wbfs”… and you should already know what to do from here.

I have 1250+ Wii titles (not split as using NTFS) and 600+ GameCube titles all successfully loaded as one big collection by USB Loader and working fine.

You will need to change a few settings on USB Loader to load both partitions, but this is easily figured out.

Oh... and with this many titles USB Loader takes a little while to respond when starting and when changing the sorting or layout. Be patient with it, it will respond after 10~15 seconds or so.

 

[blackb0x]

Oh... and with this many titles USB Loader takes a little while to respond when starting and when changing the sorting or layout. Be patient with it, it will respond after 10~15 seconds or so.

If you're using my updated version of the loader then take a look at the features menu and enable game header caching. It'll make things much faster, but you will need to manually reset the cache if you add more games directly to your HDD.

 

[kevmic]

If you're using my updated version of the loader then take a look at the features menu and enable game header caching. It'll make things much faster, but you will need to manually reset the cache if you add more games directly to your HDD.

Cheers! I am indeed using your updated version. Will have a try tomorrow