Homebrew WIP libusbhsfs - USB Mass Storage Class Host + Filesystem Mounter static library.

[duckbill007]

Try manually setting up USB3 descriptors

You mean using usbDsInterface_AppendConfigurationData ?

Should I first manually fill configuration description with all lengths, endpoint count etc?
I thought that this performed by usb:ds service (I mean Configuration descriptor) and I only need to append interface and endpoint descriptors.

 

[Imancol]

Some time ago, it was possible to use the "usb30_force_enabled" flag in Atmosphère's system_settings.ini to achieve this. However, HOS 9.0.0 introduced a hard dependency on the USB sysmodule in FS services, which means the USB sysmodule is now launched before Atmosphère is capable of parsing the INI file, rendering this flag completely useless.

Heck, no wonder the transfer was still the same as USB 2.0...I saw an option in the uLaunch created by @XorTroll that allows you to enable or disable this option, I didn't even know what it was for.

 

[duckbill007]

@DarkMatterCore, @SciresM Is there any way for homebrew to check at what speed interface connected especially get what is the wMaxPacketSize for connected endpoint? It can be different for HighSpeed and SuperSpeed.

 

[DarkMatterCore]

@DarkMatterCore, @SciresM Is there any way for homebrew to check at what speed interface connected especially get what is the wMaxPacketSize for connected endpoint? It can be different for HighSpeed and SuperSpeed.

Nope, there are no IPC calls to get such information (sadly).

If you're writing your own protocol, you just have to make the USB host report information about the selected device descriptor back to the console.

 

[duckbill007]

Unfortunately I am not writing my own protocol, but MTP implementation, and there bulk of data multiple to packet size should be ended by ZeroLengthPacket. And here is my issue: HighSpeed endpoint descriptor wMaxPacketSize must be 0x200 and SuperSpeed - 0x400 according to specification. If I set wMaxPacketSize to 0x200 for SS then switch can not be connected to PC due to incorrect descriptor error.

--------------------- MERGED ---------------------------

libmtp on linux fails to read data if there is ZLP after 512 bytes length packet while using USB3.

So, the only way right now I see is a user choice|config use USB3 or not and according to that configure interface as USB2only or USB3only which is not convenient.

 

[SciresM]

bulk of data multiple to packet size should be ended by ZeroLengthPacket.

use IDsEndpoint command 5 (SetZeroLengthTerminate) to configure the endpoint to send zero length packets appropriately.

 

[duckbill007]

Sending is only half a problem. Other half is receiving: there are commands with unknown final data length and if I read data with size multiple to 512 without MaxPacketLength I can not determine if it was a short packet on USB3 and I should stop reading or it was full packet on USB2 and I need to read following ZLP.

Or even simpler: I know that data that I should read is 0x600. I do read request for 0x600 bytes.
On USB3 last packet will be short and I do not need to do anything. On USB2 after that 3 packets there will be ZLP which I also need to read.

 

[DarkMatterCore]

If the ZLT feature is enabled and the size passed to usbDsEndpoint_PostBufferAsync() is aligned to the endpoint packet size, a ZLT packet will be issued.

Otherwise, no ZLT packet is sent even if the feature is enabled on the target endpoint - ZLT packets are not needed while handling unaligned block sizes according to the USB bulk transfer specs.

I just wanted to clear that up for anyone else reading this thread.

use IDsEndpoint command 5 (SetZeroLengthTerminate) to configure the endpoint to send zero length packets appropriately.

This may not always be a desirable behaviour - if the host device doesn't count for a ZLT packet being sent by the Switch, a timeout error will be triggered on the Switch - this is because most USB backends require you to use a bigger transfer length (+1 byte at least) if a ZLT packet is to be expected.

If you're dealing with file transfers and your transfer block size is aligned to the endpoint packet size, then you have to ask yourself if sending a ZLT packet in each loop iteration (except, probably, for the last one) is really the best way to go.

I faced this very same issue while developing the USB transfer protocol for the ongoing nxdumptool rewrite. The best solution I could come up with to avoid sending ZLT packets right after each and every aligned data block was making the host script report back the endpoint packet size from the selected device descriptor during the initial handshake.

This way, it is possible to accurately know when and where to send a ZLT. I agree there should be a way to know which device descriptor / speed was selected by the USB host, though.

 

[duckbill007]

First, I thought that there is such way: capmtp works in SS mode, but then I realized, that capmtp is readonly service and SetZeroLengthTerminate is enough for it.
I hope SetZLT can be called during transfer, because it should be sent only after last block if any.

 

[DarkMatterCore]

I hope SetZLT can be called during transfer, because it should be sent only after last block if any.

I can be called during a transfer process. Recall what I said in my previous post:

If the ZLT feature is enabled and the size passed to usbDsEndpoint_PostBufferAsync() is aligned to the endpoint packet size, a ZLT packet will be issued.

Otherwise, no ZLT packet is sent even if the feature is enabled on the target endpoint - ZLT packets are not needed while handling unaligned block sizes according to the USB bulk transfer specs.

PostBufferAsync takes care of sending the ZLT packet itself if you enabled the feature via SetZeroLengthTerminate. A ZLT packet isn't immediately sent whenever you call SetZeroLengthTerminate.

In other words, if your application determines a ZLT packet must be sent, this means you can call SetZeroLengthTerminate twice:

1. Right before PostBufferAsync to enable the ZLT feature, thus making PostBufferAsync send a ZLT packet.
2. RIght after PostBufferAsync to disable the ZLT feature.

 

[mrdude]

I'm getting these build errors now from the latest changes you put on github (previously it built just fine)

usbhsfs_utils.c
C:/msys64/home/MrDude/switch/libusbhsfs/source/usbhsfs_utils.c: In function 'usbHsFsUtilsCheckRunningServiceByName':
C:/msys64/home/MrDude/switch/libusbhsfs/source/usbhsfs_utils.c:415:14: error: implicit declaration of function 'tipcDispatchInOut'; did you mean 'serviceDispatchInOut'? [-Werror=implicit-function-declaration]
  415 |         rc = tipcDispatchInOut(smGetServiceSessionTipc(), g_smAtmosphereHasService, srv_name, tmp);
      |              ^~~~~~~~~~~~~~~~~
      |              serviceDispatchInOut
C:/msys64/home/MrDude/switch/libusbhsfs/source/usbhsfs_utils.c:415:32: error: implicit declaration of function 'smGetServiceSessionTipc'; did you mean 'smGetServiceSession'? [-Werror=implicit-function-declaration]
  415 |         rc = tipcDispatchInOut(smGetServiceSessionTipc(), g_smAtmosphereHasService, srv_name, tmp);
      |                                ^~~~~~~~~~~~~~~~~~~~~~~
      |                                smGetServiceSession
cc1.exe: all warnings being treated as errors
make[1]: *** [/opt/devkitpro/devkitA64/base_rules:19: usbhsfs_utils.o] Error 1
make: *** [Makefile:151: lib/libusbhsfs.a] Error 2

Any ideas?

 

[Chrisssj2]

Can we finally have USB HDD XCI loading on AMS?

 

[DarkMatterCore]

I'm getting these build errors now from the latest changes you put on github (previously it built just fine)

usbhsfs_utils.c
C:/msys64/home/MrDude/switch/libusbhsfs/source/usbhsfs_utils.c: In function 'usbHsFsUtilsCheckRunningServiceByName':
C:/msys64/home/MrDude/switch/libusbhsfs/source/usbhsfs_utils.c:415:14: error: implicit declaration of function 'tipcDispatchInOut'; did you mean 'serviceDispatchInOut'? [-Werror=implicit-function-declaration]
  415 |         rc = tipcDispatchInOut(smGetServiceSessionTipc(), g_smAtmosphereHasService, srv_name, tmp);
      |              ^~~~~~~~~~~~~~~~~
      |              serviceDispatchInOut
C:/msys64/home/MrDude/switch/libusbhsfs/source/usbhsfs_utils.c:415:32: error: implicit declaration of function 'smGetServiceSessionTipc'; did you mean 'smGetServiceSession'? [-Werror=implicit-function-declaration]
  415 |         rc = tipcDispatchInOut(smGetServiceSessionTipc(), g_smAtmosphereHasService, srv_name, tmp);
      |                                ^~~~~~~~~~~~~~~~~~~~~~~
      |                                smGetServiceSession
cc1.exe: all warnings being treated as errors
make[1]: *** [/opt/devkitpro/devkitA64/base_rules:19: usbhsfs_utils.o] Error 1
make: *** [Makefile:151: lib/libusbhsfs.a] Error 2

Any ideas?

You must manually build and install libnx from its master branch. TIPC serialization support for 12.x isn't available in the latest stable release.

Can we finally have USB HDD XCI loading on AMS?

Not with this library.

 

[mrdude]

You must manually build and install libnx from its master branch. TIPC serialization support for 12.x isn't available in the latest stable release.



Not with this library.

I got round that error by modding some code - now I get this error on intalling:

make[1]: Entering directory '/home/MrDude/switch/libusbhsfs/libntfs-3g'
warning: patch-2.7.6-1 is up to date -- skipping
error: failed to commit transaction (conflicting files)
==> ERROR: 'pacman' failed to install missing dependencies.
==> ERROR: Could not resolve all dependencies.
make[1]: *** [Makefile:9: all] Error 8
make[1]: Leaving directory '/home/MrDude/switch/libusbhsfs/libntfs-3g'
make: *** [Makefile:144: fs-libs] Error 2

EDIT: I ended up doing a manual install, which was kind of a waste - because when updated libnx - it wouldn't work with plutonium. So I reverted back to a backup I made of libnx.

 

[DarkMatterCore]

Just released v0.2.4 to address some small issues and tidy up the code a little bit more.

 

[DarkMatterCore]

Released v0.2.5 to fix a bug within lwext4 that could cause data abort crashes while mounting EXT volumes.

 

[mrdude]

Released v0.2.5 to fix a bug within lwext4 that could cause data abort crashes while mounting EXT volumes.

I can never get this to build - I keep getting an error that I have missing dependencies (older versions build ok though)

[email protected] MSYS ~/switch/libusbhsfs
# make BUILD_TYPE=GPL install
Installing ntfs-3g
make[1]: Entering directory '/home/MrDude/switch/libusbhsfs/libntfs-3g'
warning: patch-2.7.6-1 is up to date -- skipping
error: failed to commit transaction (conflicting files)
==> ERROR: 'pacman' failed to install missing dependencies.
==> ERROR: Could not resolve all dependencies.
make[1]: *** [Makefile:9: all] Error 8
make[1]: Leaving directory '/home/MrDude/switch/libusbhsfs/libntfs-3g'
make: *** [Makefile:144: fs-libs] Error 2

 

[DarkMatterCore]

I can never get this to build - I keep getting an error that I have missing dependencies (older versions build ok though)

[email protected] MSYS ~/switch/libusbhsfs
# make BUILD_TYPE=GPL install
Installing ntfs-3g
make[1]: Entering directory '/home/MrDude/switch/libusbhsfs/libntfs-3g'
warning: patch-2.7.6-1 is up to date -- skipping
error: failed to commit transaction (conflicting files)
==> ERROR: 'pacman' failed to install missing dependencies.
==> ERROR: Could not resolve all dependencies.
make[1]: *** [Makefile:9: all] Error 8
make[1]: Leaving directory '/home/MrDude/switch/libusbhsfs/libntfs-3g'
make: *** [Makefile:144: fs-libs] Error 2

If you're using git under msys to clone the repository, make sure that the automatic LF -> CRLF conversion feature while cloning is disabled.

 

[mrdude]

If you're using git under msys to clone the repository, make sure that the automatic LF -> CRLF conversion feature while cloning is disabled.

Thanks, What build of libnx is this using now - is it the latest?

 

[DarkMatterCore]

Thanks, What build of libnx is this using now - is it the latest?

Latest stable release, yeah.