Hacking USB Loader - Multidisk USB devices

Cameron87

New Member
OP
Newbie
Joined
Nov 25, 2011
Messages
4
Trophies
0
XP
1
Country
United States
I have a couple of USB devices, both are USB 3.0 compatible. One is a Buffalo HD-HX1.0TU3 (1TB external disk, with a "read-only removable" SD disk inside for drive tools and appears first in drive-list sequence) and can be AES 256-bit encrypted. The other device is a Kingston FCR-HS3 multi-card reader. I've been using cards through my WiiKey, and got the new reader to increase writespeeds, and eventually decided to repurpose the buffalo drive for the wii...

Thats when I discovered that the Cfg USB Loader only detects the first disk on the buffalo device, the tools disk. Just to be clear, its not a partition, they are distinct disks. Each disk only has one drive of maximum partition size. Also, the encryption on the buffalo is disabled so no software is needed, as the software tool to run to enable the AES chipset to decode-operate is Win32 only. There is an option to use encryption by giving the device the password to automatically decrypt without the software, but its not in that mode right now. I can keep the buffalo powered when switching from the PC to the Wii, ejecting the tools disk before I switch.. And the usb loader says something like no disk found; that is, when selecting devices, only the SD is detected.

The kingston device has 4 slots, with SD as Disk-3 and MicroSD as Disk 2. Disk 0 and 1 are CF and M2/MS/DUO, or vice-versa. I don't have anything but SD and MicroSD, and so the kingston behaves identically to the buffalo, on the usb loader, when I eject the buffalo's SD first.

So, I would like to know if there is anything that would allow me to load from a multidisk USB device, that isn't only on disk-0 in its drive-list. I hope all of that made sense. It would be nice to be able to use the bigger and soon, cheaper, SD cards that are out there, and those fast SDXC cards on the Wii and future devices. I'm currently using different disk drives (IDE and SATA) on a USB adapter, on my Wii, and that works great, but I don't have an extra 1TB laying around other than this Buffalo, and I'd rather use one drive than several.

*EDIT*
I've tried various formats as well (FAT32 of various sector sizes, and WBFS) and basically followed enough combinations of things to draw my conclusion that Cfg USB Loader can only find the first disk on a device.
 

Cameron87

New Member
OP
Newbie
Joined
Nov 25, 2011
Messages
4
Trophies
0
XP
1
Country
United States
Thanks for the reply. Maybe I'll have a look at the sourcecode and see for myself if there isn't something I can do to poll disks -- after all, its basically just linux. Do you have any experience working on this source? I'm thinking it would likely be in the kernel. Do you know of any other threads that relate to this?
 

tueidj

I R Expert
Member
Joined
Jan 8, 2009
Messages
2,569
Trophies
0
Website
Visit site
XP
998
Country
It's got nothing to do with how old the wii is. The USB storage code in libogc (which was "borrowed" for the USB loaders) used to only try to mount the first LUN (logical unit number) and would give up completely if there was an error (such as no disk inserted). It's since been changed to attempt mounting each LUN in sequence until one is successful. There's still no method to directly specify which LUN you want to mount but it wouldn't be hard to add one.
 

Cameron87

New Member
OP
Newbie
Joined
Nov 25, 2011
Messages
4
Trophies
0
XP
1
Country
United States
It's since been changed to attempt mounting each LUN in sequence until one is successful. There's still no method to directly specify which LUN you want to mount but it wouldn't be hard to add one.

That was my gut feeling. It would be nice if it could attempt to mount them all. What about the removable cards in a USB reader? Would the USB have to be reconnected for it to attempt to mount when a SD card is inserted, or could it trigger on its own from within the USB Loader when you choose to select a new drive?
 

1000Faces

Member
Newcomer
Joined
Jun 10, 2018
Messages
8
Trophies
0
XP
46
Country
United Kingdom
Necrobump! Did anything ever come of this?

I've got an ARM-based Linux box sitting next to my Wii U and rather than waste a whole extra drive on the console, I decided to use USB OTG to offer up an LVM volume from the Linux box as a USB Mass Storage device. This is possibly a world first, I've never found any mention of anyone else trying this. It's worked like a charm for a year now so I thought I'd see whether I could somehow add USB Loader to this.

As you know, you can't partition the Wii U drive. Even so much as enlarging the LVM volume size causes the console to no longer recognise it. I discovered that you can very easily add more LUNs to the device. I've done some experimentation and the Wii U only ever sees the first LUN. I had hoped USB Loader would be more flexible but this also only sees the first LUN. I've checked to make sure that USB Loader can see the second volume on its own. I can swap them without rebooting Linux but this is very dangerous and would almost certainly corrupt one or both.

I have looked at the libogc code and I can see it refers to LUNs a lot but the equivalent USB Loader code only refers to ports. They seem to do the low level calls quite differently so I can't figure out how to get USB Loader to deal with LUNs. Maybe I'm looking in the wrong place? Any pointers? With some help, I think I could contribute a patch.
 

Cyan

GBATemp's lurking knight
Former Staff
Joined
Oct 27, 2002
Messages
23,454
Trophies
2
Age
43
Location
Engine room, learning
XP
14,612
Country
France
how are LUN defined on the drive?
aren't LUN just partitions?

if Logical volume are not Partitions, then how are they defined on the drive if not stored on the sector 0 ? is it a firmware level information (like max usable size set with ioctl commands?)
 

1000Faces

Member
Newcomer
Joined
Jun 10, 2018
Messages
8
Trophies
0
XP
46
Country
United Kingdom
how are LUN defined on the drive?
aren't LUN just partitions?

No, they are presented as entirely separate drives and in many cases, they actually are entirely separate drives. USB can offer more than one endpoint per device. Smartphones can simultaneously offer a drive and ethernet, for example. LUNs are a SCSI concept and predate USB by a long way.
 

Cyan

GBATemp's lurking knight
Former Staff
Joined
Oct 27, 2002
Messages
23,454
Trophies
2
Age
43
Location
Engine room, learning
XP
14,612
Country
France
My NAS uses LUN, I thought it was a little like partitions, defined using info on first sector.
Thanks. I should read more about SCSI and see how it's defined and accessed when using the ehc interface.
 

1000Faces

Member
Newcomer
Joined
Jun 10, 2018
Messages
8
Trophies
0
XP
46
Country
United Kingdom
I took another look at this in case it was any clearer a second time around. I still can't find any information about the IOCTLs like USB_IOCTL_UMS_READ_SECTORS and why this is totally different to usbstorage.c in libogc. Please could you explain why USB Loader has its own implementation? I'm guessing these IOCTLs relate to some IOS, possibly IOS58? As you can see, I'm out of my depth here. :mellow: I tried to build USB Loader in the hope that I might learn more. It's the first time I've used any devkitPro stuff. I eventually managed to do it after a little grief but I'm still none the wiser.
 

Curtman

Member
Newcomer
Joined
Dec 31, 2015
Messages
24
Trophies
0
Age
44
XP
298
Country
Canada
I've got an ARM-based Linux box sitting next to my Wii U and rather than waste a whole extra drive on the console, I decided to use USB OTG to offer up an LVM volume from the Linux box as a USB Mass Storage device

Thanks for this necro.. I jumped in to the Wii softmod about a decade after everyone else did apparently. ;)

I searched around for a way to load backups via smb or nfs, or something like that to avoid using usb mass storage all together, and it seems like people had interest in this a long time ago and various people attempted and failed to make it work properly..

My idea is to use a Raspberry Pi Zero W ($10) in host mode (which emulates usb devices), to emulate a usb hard drive to the wii, and uses the superior wifi on the Pi to cache the backup on it's SD, or possibly even supply the data in realtime with a small cache in memory. In my imagination at least, the Pi would be able to assemble various directories and such (usb and/or network) into something the Wii could access as if it was a physical hard drive.

Anyway, sorry for the intrusion. I didn't find a more appropriate thread to spitball this idea.. It sounds like you are attempting a very similar thing. Anybody have any thoughts about whether or not this would work out?

I'm curious if instead of LVM, something like OverlayFS might work better.

An overlay filesystem combines two filesystems - an 'upper' filesystem
and a 'lower' filesystem. When a name exists in both filesystems, the
object in the 'upper' filesystem is visible while the object in the
'lower' filesystem is either hidden or, in the case of directories,
merged with the 'upper' object.

The lower filesystem can be any filesystem supported by Linux and does
not need to be writable. The lower filesystem can even be another
overlayfs.
 
Last edited by Curtman,

1000Faces

Member
Newcomer
Joined
Jun 10, 2018
Messages
8
Trophies
0
XP
46
Country
United Kingdom
I searched around for a way to load backups via smb or nfs, or something like that to avoid using usb mass storage all together, and it seems like people had interest in this a long time ago and various people attempted and failed to make it work properly..

My idea is to use a Raspberry Pi Zero W ($10) in host mode (which emulates usb devices), to emulate a usb hard drive to the wii, and uses the superior wifi on the Pi to cache the backup on it's SD, or possibly even supply the data in realtime with a small cache in memory. In my imagination at least, the Pi would be able to assemble various directories and such (usb and/or network) into something the Wii could access as if it was a physical hard drive.

Anyway, sorry for the intrusion. I didn't find a more appropriate thread to spitball this idea.. It sounds like you are attempting a very similar thing. Anybody have any thoughts about whether or not this would work out?
I've played with a Pi Zero W before, it's fun. I guess you're wanting to serve NFS from a desktop or something, otherwise you'd just connect a drive directly. It's a shame OTG is totally non-existent on the desktop, I don't know why that is. NFS won't work for Wii U games (unless you use Mocha?), you would need NBD for that, but it should work for Wii games. For caching, you could use FS-Cache, which I have played with before. For caching to memory, I guess you could use FS-Cache on tmpfs? I doubt the cache would help though, apart from wearing out the SD very fast, it seems unlikely the same small amount of data would be read repeatedly.

As for my adventures, I decided I had better things to waste time on and just bought a separate USB stick for the Wii games. It's plenty fast enough.
 

Curtman

Member
Newcomer
Joined
Dec 31, 2015
Messages
24
Trophies
0
Age
44
XP
298
Country
Canada
I've played with a Pi Zero W before, it's fun. I guess you're wanting to serve NFS from a desktop or something, otherwise you'd just connect a drive directly. It's a shame OTG is totally non-existent on the desktop, I don't know why that is. NFS won't work for Wii U games (unless you use Mocha?), you would need NBD for that, but it should work for Wii games. For caching, you could use FS-Cache, which I have played with before. For caching to memory, I guess you could use FS-Cache on tmpfs? I doubt the cache would help though, apart from wearing out the SD very fast, it seems unlikely the same small amount of data would be read repeatedly.

As for my adventures, I decided I had better things to waste time on and just bought a separate USB stick for the Wii games. It's plenty fast enough.

I don't have a Wii U, just the old classic Wii softmodded. I find it cumbersome to load backups on the USB drive. I don't really have any interest in playing the games, I have kids for that. Making old hardware do something it wasn't intended to do is my idea of fun.. I'm imagining a respberry pi that can appear as a single USB Mass Storage device partition, and serve backups loaded from an NFS export and/or CIFS/SMB and/or USB stick and/or whatever else. I think it could be useful on the Wii, Wii U, Xbox360, etc.. It would know what type of console it was plugged into and mount and concatenate the appropriate directories.

See my edit on that comment about OverlayFS..
 

1000Faces

Member
Newcomer
Joined
Jun 10, 2018
Messages
8
Trophies
0
XP
46
Country
United Kingdom
I'm curious if instead of LVM, something like OverlayFS might work better.
You seem to be confusing concepts. OverlayFS is a filesystem and LVM is for working with block devices. In either case, neither makes much difference to what the console sees. I put my Wii U "drive" in an LVM volume because it was convenient but it makes little difference in practise.
 

Curtman

Member
Newcomer
Joined
Dec 31, 2015
Messages
24
Trophies
0
Age
44
XP
298
Country
Canada
You seem to be confusing concepts. OverlayFS is a filesystem and LVM is for working with block devices.

Not at all. OverlayFS allows you to concatenate multiple filesystems into one. The idea would be to have an SD card as the "upper" read/write filesystem, with backups stored on any number of "lower" filesystems, which could be NFS, CIFS/SMB, physical disks, USB sticks, etc. The rPI would take that OverlayFS and make it appear to be a single FAT32 partition to the Wii as a mass storage device.
 

1000Faces

Member
Newcomer
Joined
Jun 10, 2018
Messages
8
Trophies
0
XP
46
Country
United Kingdom
Not at all. OverlayFS allows you to concatenate multiple filesystems into one. The idea would be to have an SD card as the "upper" read/write filesystem, with backups stored on any number of "lower" filesystems, which could be NFS, CIFS/SMB, physical disks, USB sticks, etc. The rPI would take that OverlayFS and make it appear to be a single FAT32 partition to the Wii as a mass storage device.
Okay but there won't be any writes, except when you initially install the games. I believe save data still goes onto the Wii NAND unless you use emuNAND?

Come to think of it, none of this will work anyway because g_mass_storage expects a block device, not a filesystem. Sorry, I should have realised this earlier. Your only option is NBD. There is DRBD but unless you're using the data in multiple places, there seems little point.
 
Last edited by 1000Faces,

Curtman

Member
Newcomer
Joined
Dec 31, 2015
Messages
24
Trophies
0
Age
44
XP
298
Country
Canada
Okay but there won't be any writes, except when you initially install the games. I believe save data still goes onto the Wii NAND unless you use emuNAND?

Come to think of it, none of this will work anyway because g_mass_storage expects a block device, not a filesystem. Sorry, I should have realised this earlier. Your only option is NBD. There is DRBD but unless you're using the data in multiple places, there seems little point.

The Raspberry Pi is a block device in OTG mode. It emulates a USB stick. Or an ethernet card, or a keyboard, or any other usb device or combo of devices you want it to pretend to be.

Example: (Hes making his Pi appear to be a USB stick to the device its plugged into, it also runs a Samba server so he can load files onto it without physically unplugging it and plugging it in somewhere else)

MAKE A PI ZERO W SMART USB FLASH DRIVE
 
Last edited by Curtman,

1000Faces

Member
Newcomer
Joined
Jun 10, 2018
Messages
8
Trophies
0
XP
46
Country
United Kingdom
The Raspberry Pi is a block device in OTG mode. It emulates a USB stick. Or an ethernet card, or a keyboard, or any other usb device or combo of devices you want it to pretend to be.

Example: (Hes making his Pi appear to be a USB stick to the device its plugged into, it also runs a Samba server so he can load files onto it without physically unplugging it and plugging it in somewhere else)
Haha, okay, I see. What he's doing here isn't safe. Most filesystems, including ext4 and FAT32, are not supposed to be mounted in more than one place at once, even if only one of those places is writing. You might get away with it but you're breaking the rules so you get to keep the pieces.
 

Curtman

Member
Newcomer
Joined
Dec 31, 2015
Messages
24
Trophies
0
Age
44
XP
298
Country
Canada
Haha, okay, I see. What he's doing here isn't safe. Most filesystems, including ext4 and FAT32, are not supposed to be mounted in more than one place at once, even if only one of those places is writing. You might get away with it but you're breaking the rules so you get to keep the pieces.

In his example, it "ejects" itself from the host that it's plugged into while you transfer files, and "reinitialises" afterward.. My plan is to have the Pi act as a bridge to other filesystems. There wouldn't be an actual FAT32 partition anywhere, except in the Pi would provide a FAT table and something that looks like a FAT32 partition to the host, but in reality that only exists in the RAM of the Pi. So there wouldn't be the same filesystem mounted from more than one place. That reinitialization would only need to occur when you want to make the Wii (or whatever) aware of new files that you've placed on the actual filesytems.

Flash drives aren't even good for the Wii, some work, some don't.

More accurately it would emulate a USB Mass Storage device, since that is done in software I would think that sort of thing would be easier to diagnose and troubleshoot than a real physical USB stick and its hardware usb microcontroller that you can't access.
 
Last edited by Curtman,
General chit-chat
Help Users
    Psionic Roshambo @ Psionic Roshambo: I would Photoshop movie stars kissing donkey balls lol