Hacking USB Loader GX

[dimok]

The standard libogc usbstorage implementation supports arbitrary sector sizes, so if it was copied properly into the cioses there should be no problem. The 512 byte limitation is only in the filesystem libraries like libfat.

oh. since when is that like that? I remember the time where USBStorage_ReadCapacity always returned 512 as sector size no matter of the physical sector size of the drive.
Anyway, thanks for the info tueidj. In that case there is nothing that should stop the implementation of different sector sizes (if the cIOS implementations are also supporting that).

 

[dexter222]

The 3tb seagate works as expected - The new Wii backup manager sees it as an actual 4k sector size disk and formats it WBFS accordingly meaning no 500 game limit

There are a couple of drawbacks but it seems to be working as expected.

CFG Loader v.63 works seamlessly with it however USB loader GX 2.0 simply counts down

The fact that it works amazes me because physical sector size other than 512 byte should not be supported by any of the current wii implementations of the usb handle. 4096 might be an exceptional sector size here. Even if read access works, i would not do a write attempt if i were you. The wii handle only write 512 bytes per sector and you will have a lot of sectors where you could write 4096 bytes to they will only hold 512 bytes wasting space.

Anyway if it works on CFG Loader than it will work on GX. Can you come for debugging with it to the IRC channel, to find the root of the problem.

Virtual sector sizes other than 512 bytes are not supported by any usb loader currently as far as I know.

I'm there on IRC but I don't see you... I'll hang there an hour now and will be back in a few hrs after that

 

[Cyan]

he usually is on IRC between 6PM and 10PM CET.
our time zone aren't very compatible in weekdays. I guess it will be easier to meet on IRC this weekend.

 

[EnigmaXtreme]

I dont know whats going on, but depending on what video mode i select all i get is a black screen. What option should I use?

 

[Cyan]

I guess it's when you try to play NTSC games.
just go to the main settings > Loader setting > video mode : Game default. (instead of Console default).

So that all games uses the default video signal used by the game, instead of setting it to PAL.

 

[Hielkenator]

Can someone compile the latest beta.dol?

 

[Kittymat]

already done, to avoid outside links use google

"gx loader downloads" on the first option select that page, then select

"Source changes at google codepage" all the Beta releases are here

If you want to see the latest reports on that beta release then select Project home in the top corner of that page, then updates in the left of the page that link takes you to!

Join the google page and you can post any issues you find

 

[dexter222]

Dimok started to work on the compatibility for WBFS sectors >512, but it seems there are few bugs in libwbfs, so he will have to fix that first.

It took a bit of debugging and some testing but dimok was successful:

USB Loader GX now supports the 3tb seagate drives (drives reporting 4k default sector size)

 

[Cyan]

Thanks for your help Dexter

also :
• The 4k sector drive MUST be fully WBFS. no other mixed partition format are allowed.
• 4k drives only accept WBFS, as NTFS lib is hardcoded to read 512k sector size.
• WBFS access speed is set to USB2.0 speed instead of USB3 speed.

 

[dimok]

• The 4k sector drive MUST be fully WBFS. no other mixed partition format are allowed.
I am not sure about this. Is that tested? I don't see why it can't be mixed.

QUOTE(Cyan @ Jan 30 2011, 10:49 AM) • 4k drives only accept WBFS, as NTFS lib is hardcoded to read 512k sector size.

That is actually easy to change. There is only one limitation I know that blocks that in libntfs. The problem is that fragmentation list seems to cause some problems with sector sizes other than 512 bytes. That's what i found out when testing with dexter. Usually GX does get fragment list on wbfs too now but that didn't seem to run any games for him. I was forced to change the game boot method for sizes other than 512 to the old method. I need to ask oggzee or look into the dip modules oggzee wrote. Maybe its some limitation in there.

 

[oggzee]

That is actually easy to change. There is only one limitation I know that blocks that in libntfs. The problem is that fragmentation list seems to cause some problems with sector sizes other than 512 bytes. That's what i found out when testing with dexter. Usually GX does get fragment list on wbfs too now but that didn't seem to run any games for him. I was forced to change the game boot method for sizes other than 512 to the old method. I need to ask oggzee or look into the dip modules oggzee wrote. Maybe its some limitation in there.

You're right, dip+frag uses a hardcoded 512 sector size.
We'll need to improve that.

 

[dexter222]

That WBFS supports 4k sector sizes is great but what really needs to happen is development for GUID/GPT support for the 3tb Western Digital and Hitachi drives. This is likely the direction drive development will take... or is taking.

Being able to use WBFS on the 3tb Seagate is a much appreciated stop-gap measure until we get GUID/GPT support for the Wii... and I don't see that happening overnight lol.

As far as multiple partitions for the 3tb Seagate... the 11th commandment should be "One Drive, One Partition"

but it may be possible to partition part of it FAT32/NTFS by overriding the default sector size when partitioning. I may test this later if anyone really wants to know.

 

[dimok]

That WBFS supports 4k sector sizes is great but what really needs to happen is development for GUID/GPT support for the 3tb Western Digital and Hitachi drives. This is likely the direction drive development will take... or is taking.

Being able to use WBFS on the 3tb Seagate is a much appreciated stop-gap measure until we get GUID/GPT support for the Wii... and I don't see that happening overnight lol.

As far as multiple partitions for the 3tb Seagate... the 11th commandment should be "One Drive, One Partition"

but it may be possible to partition part of it FAT32/NTFS by overriding the default sector size when partitioning. I may test this later if anyone really wants to know.

GUID/GPT is already supported by USB Loader GX, WiiXplorer and Wiiflow (maybe some more app). It's only a different partition table nothing special. It is the new standard for partition listing on a drive because the old Master Boot Record (MBR) / Extended Boot Record (EBR) is not sufficient anymore with partition sizes over 2TB and sector sizes 512 for example. Checkout wikipedia if you want more info. Every drive can have a GUID Partition Table.

 

[dexter222]

Dimok...

Are you telling me that, across the board, GUID is already supported??? That I can use a 3tb drive on the Wii using NTFS? That I went through all this 4k sector size drama for nothing??? LOL Dimok for someone so smart...

So you're telling me hermes 5.1 and cIOSx don't need to do anything special to support GUID/GPT? That I should just be able to partition and format the drive GPT and go back to using 512b sectors on a 3tb drive and it will just work?

Please feel free to be very specific with your answer

 

[tueidj]

No, the usbstorage driver only allows 32-bits for specifying the requested LBA.
32 + 9 (512 byte sectors) = 41, two to the power of 41 = 2,199,023,255,552 ~= 2TB of addressable storage if the sectors are 512 bytes.

 

[RavRob]

I love this loader but I have 2 problems I have since I upgraded.

1 - I lost my theme and can't find it on the new themes. The new ones are very light and the name of the games are hard to read. I have tried several but none work as good as the old themes. How can I get my old theme back?

2 - Although I had all my covers and disk art downloaded before the upgrade, I had to re-download all my arts. Now however I got all my cover art but still missing 42 disk art. How do I get them?

 

[dexter222]

No, the usbstorage driver only allows 32-bits for specifying the requested LBA.
32 + 9 (512 byte sectors) = 41, two to the power of 41 = 2,199,023,255,552 ~= 2TB of addressable storage if the sectors are 512 bytes.

Wait a minute lol - dimok just said that everything was good to go for GPT which translates to 9.4ZB - that's "Z" or, for you Canadians, "Zed"

It's a big number like 9.4 x 10 to the 21st. Lots of zeros lol lots more than meager terabytes! And that's at 512b sectors to boot! (no pun intended)

So... before I start investing in even more hardware, do any of the USB loaders support 9.4ZB @512b sectors like dimok says or are we still limited to the 2.19TB @512b sectors like tueidj says which means NOT GPT? It would be worth an investment to get away from the 4k sector size and WBFS.

The same thousand files takes over 150GB more on a 4k sector size WBFS than they do on a 512b sector size NTFS.

 

[tueidj]

The usbstorage code would need an overhaul to support more than 2^32 sectors, afaik the SCSI_READ_16/SCSI_WRITE_16 commands would be required. Not too difficult if you use the existing write/read functions as a base.

 

[oggzee]

No, the usbstorage driver only allows 32-bits for specifying the requested LBA.
32 + 9 (512 byte sectors) = 41, two to the power of 41 = 2,199,023,255,552 ~= 2TB of addressable storage if the sectors are 512 bytes.

yes, but if the sector size is 4k then you have 16TB addressable...

 

[dimok]

No, the usbstorage driver only allows 32-bits for specifying the requested LBA.
32 + 9 (512 byte sectors) = 41, two to the power of 41 = 2,199,023,255,552 ~= 2TB of addressable storage if the sectors are 512 bytes.

Wait a minute lol - dimok just said that everything was good to go for GPT which translates to 9.4ZB - that's "Z" or, for you Canadians, "Zed"

It's a big number like 9.4 x 10 to the 21st. Lots of zeros lol lots more than meager terabytes! And that's at 512b sectors to boot! (no pun intended)

So... before I start investing in even more hardware, do any of the USB loaders support 9.4ZB @512b sectors like dimok says or are we still limited to the 2.19TB @512b sectors like tueidj says which means NOT GPT? It would be worth an investment to get away from the 4k sector size and WBFS.

The same thousand files takes over 150GB more on a 4k sector size WBFS than they do on a 512b sector size NTFS.

Yes and no. GPT/GUID as you say is only a table list at the beginning of the drive. It lists all partitions on the drive. It not some magical thing that enables use of every drive size. It just has 64 bit sector offset for partitions and 64 bit sector counter for the size instead of 32 bit like on MBR/EBR. So it basically is supporting big sizes as you say. The problem that tueidj is talking about is the read/write code. It accepts only 32 bit sectors offsets inside the drive as input. So in other words: Yes you can use big sizes NTFS @ 512 bytes per sector but not completely. Like tueidj said only up to 2TB size or driver sector 4294967295 (max 32 bit number), since than the 32bit limit is reached AND you can only access the first 2TB of the drive. You can't access any partition behind the 2TB limit. That is a limitation in all current wii implementations as far as i know.

GPT/GUID was introduce with WinVista/Win7 (for Windows products) and already earlier for Linux/OSX. Windows XP 32bit and earlier do not support this at all (not even read).

To answer your question: The partition table itself is read (if exist on the drive) with USB Loader GX and other apps. It is not some special thing that enables special functions on a drive. It is just a new partition list method that replaces the old MBR/EBR method. Every drive can be converted to have GUID partitions entries.