Homebrew WIP libusbhsfs - USB Mass Storage Class Host + Filesystem Mounter static library.

[blawar]

please dont forget your Lithium

Tinfoil and Lithium do not use this library, they use my own UMS implementation which I have to update to support NTFS. I will get to it eventually, but it is not a priority atm.

--------------------- MERGED ---------------------------

Finally, it's working good with my 2TB seagate NTFS hard drive.... thanks a lot

Don't thank me, all I did was recompile. Any idiot can do that. This is DMC's work.

 

[NickyRay]

Huge thanks to DMC for this work. Keep up the good work

 

[minomore]

thanks to all who worked on this amazing development, any chance of getting this on the key switch apps.... ie retroarch, pplay etc??

 

[DarkMatterCore]

thanks to all who worked on this amazing development, any chance of getting this on the key switch apps.... ie retroarch, pplay etc??

Next to no chance for RetroArch. Probably doable for pplay.

 

[lordelan]

Next to no chance for RetroArch.

Why? RetroArch for Wii U can load content from USB as well.

Edit: Amazing work btw. Just discovered this thread yet.

 

[DarkMatterCore]

Why? RetroArch for Wii U can load content from USB as well.

Edit: Amazing work btw. Just discovered this thread yet.

m4xw is against adding UMS device support because it may further encourage piracy in RetroArch.

 

[UnT7oh]

pPlay would be great but the developer seems completely inactive on Switch projects.

m4xw is against adding UMS device support because it may further encourage piracy in RetroArch.

Uh what? First of all that makes no sense at all, secondly USB support gives a huge boost in flexibilty, sharing your setup between different devices and so on.

 

[DarkMatterCore]

Uh what? First of all that makes no sense at all, secondly USB support gives a huge boost in flexibilty, sharing your setup between different devices and so on.

Nonetheless, that's his personal view on this matter, which I decided to respect. In the end, he's totally free to decide what to implement into RA.

In all fairness, he did say that even though he despises a feature like this, he might accept a PR if libusbhsfs becomes a devkitPro portlib, which is totally impossible because of fincs' stance on this matter.

 

[blawar]

Next to no chance for RetroArch. Probably doable for pplay.

pPlay requires a PHD in the dark arts and a blood sacrifice to compile. While technically it is possible, functionally I do not think anyone but the author who appears to have abandoned the project would try to compile that.

Retroarch is begging to be forked if they won't add it:

 

[ShroomKing]

@DarkMatterCore can you provide an aarch64 build of ntfs-3g please? i can't compile it due to some mac dependency checking problem. I have all dependencies but makepkg doesn't find them and i don't have a linux machine atm.

 

[blawar]

@DarkMatterCore can you provide an aarch64 build of ntfs-3g please? i can't compile it due to some mac dependency checking problem. I have all dependencies but makepkg doesn't find them and i don't have a linux machine atm.
View attachment 238567

If you want the entire libusbhsfs lib file, here it is:

unzip and place the .a file in /opt/devkitpro/portlibs/switch/lib/ I believe. Copy the h file from his repo to /opt/devkitpro/portlibs/switch/include/

--------------------- MERGED ---------------------------

@DarkMatterCore can you provide an aarch64 build of ntfs-3g please? i can't compile it due to some mac dependency checking problem. I have all dependencies but makepkg doesn't find them and i don't have a linux machine atm.
View attachment 238567

Here is the ntfs package though (unzip it, gbatemp wont let me upload xz files)

 

[DarkMatterCore]

Both ISC and GPLv2 redistributable packages of the library are available in the release page: https://github.com/DarkMatterCore/libusbhsfs/releases/latest (just in case). These are generated by Makefile rules.

 

[ShroomKing]

If you want the entire libusbhsfs lib file, here it is:

unzip and place the .a file in /opt/devkitpro/portlibs/switch/lib/ I believe. Copy the h file from his repo to /opt/devkitpro/portlibs/switch/include/

--------------------- MERGED ---------------------------



Here is the ntfs package though (unzip it, gbatemp wont let me upload xz files)

Thanks, libntfs-3g.a is what i was looking for.

Both ISC and GPLv2 redistributable packages of the library are available in the release page: https://github.com/DarkMatterCore/libusbhsfs/releases/latest (just in case). These are generated by Makefile rules.

Yes but those only have libusbhsfs.a.
If i understood correctly i also need to link the libntfs-3g if i want to use ntfs

 

[Imancol]

m4xw is against adding UMS device support because it may further encourage piracy in RetroArch.

I will never understand m4xw, it makes incredible port of emulators, but to say that the USB support is piracy, when the ports of the emulators are already it. Anyway, I hope someone else is encouraged to make their own port with this great addition to USB drives, which apparently works more and more wonderfully and that eventually is the solution to the bottlenecks suffered by some emulators to load ROMs from MicroSD .

--------------------- MERGED ---------------------------

pPlay requires a PHD in the dark arts and a blood sacrifice to compile. While technically it is possible, functionally I do not think anyone but the author who appears to have abandoned the project would try to compile that.

Retroarch is begging to be forked if they won't add it:

I think it's the only thing I can ask of you. Fork the code and create your own implementation. Sure, if you've worked with emulators before. Don't forget about PPSSPP

 

[DarkMatterCore]

Working on EXT2/3/4 support.

 

[DarkMatterCore]

2020-12-20 08:29:16 -> usbHsFsMountParseMasterBootRecordPartitionEntry: Found Linux partition entry with type 0x83 at LBA 0x800 (interface 1347682304, LUN 0).
2020-12-20 08:29:16 -> usbHsFsScsiReadLogicalUnitBlocks: Reading 0x2 block(s) from LBA 0x802 (0x400 byte[s]) (interface 1347682304, LUN 0).
2020-12-20 08:29:16 -> usbHsFsScsiSendRead10Command: Sending command (interface 1347682304, LUN 0).
2020-12-20 08:29:16 -> usbHsFsScsiSendCommandBlockWrapper: Data from CBW to send (interface 1347682304, LUN 0):
55534243630c316c0004000080000a28000000080200000200000000000000
2020-12-20 08:29:16 -> usbHsFsScsiReceiveCommandStatusWrapper: Data from received CSW (interface 1347682304, LUN 0):
55534253630c316c0000000000
2020-12-20 08:29:16 -> usbHsFsMountInspectExtSuperBlock: Found EXT superblock at LBA 0x802 (interface 1347682304, LUN 0).
2020-12-20 08:29:16 -> usbHsFsMountRegisterDevoptabDevice: Available device ID: 0 (interface 1347682304, LUN 0, FS 0).
2020-12-20 08:29:16 -> usbHsFsMountRegisterDevoptabDevice: Failed to get pointer to devoptab interface! (interface 1347682304, LUN 0, FS 0).

Only the EXT devoptab interface is missing now.

 

[TotalJustice]

2020-12-20 08:29:16 -> usbHsFsMountParseMasterBootRecordPartitionEntry: Found Linux partition entry with type 0x83 at LBA 0x800 (interface 1347682304, LUN 0).
2020-12-20 08:29:16 -> usbHsFsScsiReadLogicalUnitBlocks: Reading 0x2 block(s) from LBA 0x802 (0x400 byte[s]) (interface 1347682304, LUN 0).
2020-12-20 08:29:16 -> usbHsFsScsiSendRead10Command: Sending command (interface 1347682304, LUN 0).
2020-12-20 08:29:16 -> usbHsFsScsiSendCommandBlockWrapper: Data from CBW to send (interface 1347682304, LUN 0):
55534243630c316c0004000080000a28000000080200000200000000000000
2020-12-20 08:29:16 -> usbHsFsScsiReceiveCommandStatusWrapper: Data from received CSW (interface 1347682304, LUN 0):
55534253630c316c0000000000
2020-12-20 08:29:16 -> usbHsFsMountInspectExtSuperBlock: Found EXT superblock at LBA 0x802 (interface 1347682304, LUN 0).
2020-12-20 08:29:16 -> usbHsFsMountRegisterDevoptabDevice: Available device ID: 0 (interface 1347682304, LUN 0, FS 0).
2020-12-20 08:29:16 -> usbHsFsMountRegisterDevoptabDevice: Failed to get pointer to devoptab interface! (interface 1347682304, LUN 0, FS 0).

Only the EXT devoptab interface is missing now.

nice work as always mate

 

[Imancol]

2020-12-20 08:29:16 -> usbHsFsMountParseMasterBootRecordPartitionEntry: Found Linux partition entry with type 0x83 at LBA 0x800 (interface 1347682304, LUN 0).
2020-12-20 08:29:16 -> usbHsFsScsiReadLogicalUnitBlocks: Reading 0x2 block(s) from LBA 0x802 (0x400 byte[s]) (interface 1347682304, LUN 0).
2020-12-20 08:29:16 -> usbHsFsScsiSendRead10Command: Sending command (interface 1347682304, LUN 0).
2020-12-20 08:29:16 -> usbHsFsScsiSendCommandBlockWrapper: Data from CBW to send (interface 1347682304, LUN 0):
55534243630c316c0004000080000a28000000080200000200000000000000
2020-12-20 08:29:16 -> usbHsFsScsiReceiveCommandStatusWrapper: Data from received CSW (interface 1347682304, LUN 0):
55534253630c316c0000000000
2020-12-20 08:29:16 -> usbHsFsMountInspectExtSuperBlock: Found EXT superblock at LBA 0x802 (interface 1347682304, LUN 0).
2020-12-20 08:29:16 -> usbHsFsMountRegisterDevoptabDevice: Available device ID: 0 (interface 1347682304, LUN 0, FS 0).
2020-12-20 08:29:16 -> usbHsFsMountRegisterDevoptabDevice: Failed to get pointer to devoptab interface! (interface 1347682304, LUN 0, FS 0).

Only the EXT devoptab interface is missing now.

Which of all the partition types has the best speed?

 

[DarkMatterCore]

Which of all the partition types has the best speed?

It's probably between any supported FAT filesystem and any supported EXT filesystem, but I haven't done any proper benchmarks. Both FatFs and lwext4 offer very little I/O overhead, since they were both designed with microcontrollers in mind. Furthermore, lwext4 offers a built-in block cache system that may help speed up certain operations - this feature isn't available in FatFs.

NTFS-3G, on the other hand, is a full fledged driver designed with PC operating systems in mind, making it more complex to work with than both FatFs and lwext4. Considering the amount of data it processes from NTFS volumes, I'd say it's the supported library with the most overhead. It offers a way to define user-provided functions to handle block cache, which may help speed up write operations a bit, but I haven't looked too much into it.

 

[DarkMatterCore]