Homebrew WIP libusbhsfs - USB Mass Storage Class Host + Filesystem Mounter static library.

duckbill007 · Mar 3, 2021

DarkMatterCore said:
Try manually setting up USB3 descriptors

You mean using usbDsInterface_AppendConfigurationData ?

Should I first manually fill configuration description with all lengths, endpoint count etc?
I thought that this performed by usb:ds service (I mean Configuration descriptor) and I only need to append interface and endpoint descriptors.

Imancol · Mar 4, 2021

DarkMatterCore said:
Some time ago, it was possible to use the "usb30_force_enabled" flag in Atmosphère's system_settings.ini to achieve this. However, HOS 9.0.0 introduced a hard dependency on the USB sysmodule in FS services, which means the USB sysmodule is now launched before Atmosphère is capable of parsing the INI file, rendering this flag completely useless.

Heck, no wonder the transfer was still the same as USB 2.0...I saw an option in the uLaunch created by @XorTroll that allows you to enable or disable this option, I didn't even know what it was for.

duckbill007 · Mar 6, 2021

@DarkMatterCore, @SciresM Is there any way for homebrew to check at what speed interface connected especially get what is the wMaxPacketSize for connected endpoint? It can be different for HighSpeed and SuperSpeed.

DarkMatterCore · Mar 6, 2021

duckbill007 said:
@DarkMatterCore, @SciresM Is there any way for homebrew to check at what speed interface connected especially get what is the wMaxPacketSize for connected endpoint? It can be different for HighSpeed and SuperSpeed.

Nope, there are no IPC calls to get such information (sadly).

If you're writing your own protocol, you just have to make the USB host report information about the selected device descriptor back to the console.

duckbill007 · Mar 6, 2021

Unfortunately I am not writing my own protocol, but MTP implementation, and there bulk of data multiple to packet size should be ended by ZeroLengthPacket. And here is my issue: HighSpeed endpoint descriptor wMaxPacketSize must be 0x200 and SuperSpeed - 0x400 according to specification. If I set wMaxPacketSize to 0x200 for SS then switch can not be connected to PC due to incorrect descriptor error.

--------------------- MERGED ---------------------------

libmtp on linux fails to read data if there is ZLP after 512 bytes length packet while using USB3.

So, the only way right now I see is a user choice|config use USB3 or not and according to that configure interface as USB2only or USB3only which is not convenient.

SciresM · Mar 6, 2021

duckbill007 said:
bulk of data multiple to packet size should be ended by ZeroLengthPacket.

use IDsEndpoint command 5 (SetZeroLengthTerminate) to configure the endpoint to send zero length packets appropriately.

duckbill007 · Mar 6, 2021

Sending is only half a problem. Other half is receiving: there are commands with unknown final data length and if I read data with size multiple to 512 without MaxPacketLength I can not determine if it was a short packet on USB3 and I should stop reading or it was full packet on USB2 and I need to read following ZLP.

Or even simpler: I know that data that I should read is 0x600. I do read request for 0x600 bytes.
On USB3 last packet will be short and I do not need to do anything. On USB2 after that 3 packets there will be ZLP which I also need to read.

DarkMatterCore · Mar 6, 2021

If the ZLT feature is enabled and the size passed to usbDsEndpoint_PostBufferAsync() is aligned to the endpoint packet size, a ZLT packet will be issued.

Otherwise, no ZLT packet is sent even if the feature is enabled on the target endpoint - ZLT packets are not needed while handling unaligned block sizes according to the USB bulk transfer specs.

I just wanted to clear that up for anyone else reading this thread.

SciresM said:
use IDsEndpoint command 5 (SetZeroLengthTerminate) to configure the endpoint to send zero length packets appropriately.

This may not always be a desirable behaviour - if the host device doesn't count for a ZLT packet being sent by the Switch, a timeout error will be triggered on the Switch - this is because most USB backends require you to use a bigger transfer length (+1 byte at least) if a ZLT packet is to be expected.

If you're dealing with file transfers and your transfer block size is aligned to the endpoint packet size, then you have to ask yourself if sending a ZLT packet in each loop iteration (except, probably, for the last one) is really the best way to go.

I faced this very same issue while developing the USB transfer protocol for the ongoing nxdumptool rewrite. The best solution I could come up with to avoid sending ZLT packets right after each and every aligned data block was making the host script report back the endpoint packet size from the selected device descriptor during the initial handshake.

This way, it is possible to accurately know when and where to send a ZLT. I agree there should be a way to know which device descriptor / speed was selected by the USB host, though.

duckbill007 · Mar 6, 2021

First, I thought that there is such way: capmtp works in SS mode, but then I realized, that capmtp is readonly service and SetZeroLengthTerminate is enough for it.
I hope SetZLT can be called during transfer, because it should be sent only after last block if any.

DarkMatterCore · Mar 7, 2021

duckbill007 said:
I hope SetZLT can be called during transfer, because it should be sent only after last block if any.

I can be called during a transfer process. Recall what I said in my previous post:

DarkMatterCore said:
If the ZLT feature is enabled and the size passed to usbDsEndpoint_PostBufferAsync() is aligned to the endpoint packet size, a ZLT packet will be issued.

Otherwise, no ZLT packet is sent even if the feature is enabled on the target endpoint - ZLT packets are not needed while handling unaligned block sizes according to the USB bulk transfer specs.

PostBufferAsync takes care of sending the ZLT packet itself if you enabled the feature via SetZeroLengthTerminate. A ZLT packet isn't immediately sent whenever you call SetZeroLengthTerminate.

In other words, if your application determines a ZLT packet must be sent, this means you can call SetZeroLengthTerminate twice:

1. Right before PostBufferAsync to enable the ZLT feature, thus making PostBufferAsync send a ZLT packet.
2. RIght after PostBufferAsync to disable the ZLT feature.

mrdude · Apr 29, 2021

I'm getting these build errors now from the latest changes you put on github (previously it built just fine)

Code:

usbhsfs_utils.c
C:/msys64/home/MrDude/switch/libusbhsfs/source/usbhsfs_utils.c: In function 'usbHsFsUtilsCheckRunningServiceByName':
C:/msys64/home/MrDude/switch/libusbhsfs/source/usbhsfs_utils.c:415:14: error: implicit declaration of function 'tipcDispatchInOut'; did you mean 'serviceDispatchInOut'? [-Werror=implicit-function-declaration]
  415 |         rc = tipcDispatchInOut(smGetServiceSessionTipc(), g_smAtmosphereHasService, srv_name, tmp);
      |              ^~~~~~~~~~~~~~~~~
      |              serviceDispatchInOut
C:/msys64/home/MrDude/switch/libusbhsfs/source/usbhsfs_utils.c:415:32: error: implicit declaration of function 'smGetServiceSessionTipc'; did you mean 'smGetServiceSession'? [-Werror=implicit-function-declaration]
  415 |         rc = tipcDispatchInOut(smGetServiceSessionTipc(), g_smAtmosphereHasService, srv_name, tmp);
      |                                ^~~~~~~~~~~~~~~~~~~~~~~
      |                                smGetServiceSession
cc1.exe: all warnings being treated as errors
make[1]: *** [/opt/devkitpro/devkitA64/base_rules:19: usbhsfs_utils.o] Error 1
make: *** [Makefile:151: lib/libusbhsfs.a] Error 2

Any ideas?

Chrisssj2 · Apr 29, 2021

Can we finally have USB HDD XCI loading on AMS?

DarkMatterCore · Apr 29, 2021

mrdude said:

I'm getting these build errors now from the latest changes you put on github (previously it built just fine)

Code:

usbhsfs_utils.c
C:/msys64/home/MrDude/switch/libusbhsfs/source/usbhsfs_utils.c: In function 'usbHsFsUtilsCheckRunningServiceByName':
C:/msys64/home/MrDude/switch/libusbhsfs/source/usbhsfs_utils.c:415:14: error: implicit declaration of function 'tipcDispatchInOut'; did you mean 'serviceDispatchInOut'? [-Werror=implicit-function-declaration]
  415 |         rc = tipcDispatchInOut(smGetServiceSessionTipc(), g_smAtmosphereHasService, srv_name, tmp);
      |              ^~~~~~~~~~~~~~~~~
      |              serviceDispatchInOut
C:/msys64/home/MrDude/switch/libusbhsfs/source/usbhsfs_utils.c:415:32: error: implicit declaration of function 'smGetServiceSessionTipc'; did you mean 'smGetServiceSession'? [-Werror=implicit-function-declaration]
  415 |         rc = tipcDispatchInOut(smGetServiceSessionTipc(), g_smAtmosphereHasService, srv_name, tmp);
      |                                ^~~~~~~~~~~~~~~~~~~~~~~
      |                                smGetServiceSession
cc1.exe: all warnings being treated as errors
make[1]: *** [/opt/devkitpro/devkitA64/base_rules:19: usbhsfs_utils.o] Error 1
make: *** [Makefile:151: lib/libusbhsfs.a] Error 2

Any ideas?

You must manually build and install libnx from its master branch. TIPC serialization support for 12.x isn't available in the latest stable release.

Chrisssj2 said:
Can we finally have USB HDD XCI loading on AMS?

Not with this library.

mrdude · Apr 29, 2021

DarkMatterCore said:
You must manually build and install libnx from its master branch. TIPC serialization support for 12.x isn't available in the latest stable release.

Not with this library.

I got round that error by modding some code - now I get this error on intalling:

Code:

make[1]: Entering directory '/home/MrDude/switch/libusbhsfs/libntfs-3g'
warning: patch-2.7.6-1 is up to date -- skipping
error: failed to commit transaction (conflicting files)
==> ERROR: 'pacman' failed to install missing dependencies.
==> ERROR: Could not resolve all dependencies.
make[1]: *** [Makefile:9: all] Error 8
make[1]: Leaving directory '/home/MrDude/switch/libusbhsfs/libntfs-3g'
make: *** [Makefile:144: fs-libs] Error 2

EDIT: I ended up doing a manual install, which was kind of a waste - because when updated libnx - it wouldn't work with plutonium. So I reverted back to a backup I made of libnx.

DarkMatterCore · Jun 3, 2021

Just released v0.2.4 to address some small issues and tidy up the code a little bit more.

DarkMatterCore · Jun 19, 2021

Released v0.2.5 to fix a bug within lwext4 that could cause data abort crashes while mounting EXT volumes.

mrdude · Sep 28, 2021

DarkMatterCore said:
Released v0.2.5 to fix a bug within lwext4 that could cause data abort crashes while mounting EXT volumes.

I can never get this to build - I keep getting an error that I have missing dependencies (older versions build ok though)

Code:

MrDude@MrDude MSYS ~/switch/libusbhsfs
# make BUILD_TYPE=GPL install
Installing ntfs-3g
make[1]: Entering directory '/home/MrDude/switch/libusbhsfs/libntfs-3g'
warning: patch-2.7.6-1 is up to date -- skipping
error: failed to commit transaction (conflicting files)
==> ERROR: 'pacman' failed to install missing dependencies.
==> ERROR: Could not resolve all dependencies.
make[1]: *** [Makefile:9: all] Error 8
make[1]: Leaving directory '/home/MrDude/switch/libusbhsfs/libntfs-3g'
make: *** [Makefile:144: fs-libs] Error 2

DarkMatterCore · Sep 28, 2021

mrdude said:

I can never get this to build - I keep getting an error that I have missing dependencies (older versions build ok though)

Code:

MrDude@MrDude MSYS ~/switch/libusbhsfs
# make BUILD_TYPE=GPL install
Installing ntfs-3g
make[1]: Entering directory '/home/MrDude/switch/libusbhsfs/libntfs-3g'
warning: patch-2.7.6-1 is up to date -- skipping
error: failed to commit transaction (conflicting files)
==> ERROR: 'pacman' failed to install missing dependencies.
==> ERROR: Could not resolve all dependencies.
make[1]: *** [Makefile:9: all] Error 8
make[1]: Leaving directory '/home/MrDude/switch/libusbhsfs/libntfs-3g'
make: *** [Makefile:144: fs-libs] Error 2

If you're using git under msys to clone the repository, make sure that the automatic LF -> CRLF conversion feature while cloning is disabled.

mrdude · Sep 28, 2021

DarkMatterCore said:
If you're using git under msys to clone the repository, make sure that the automatic LF -> CRLF conversion feature while cloning is disabled.

Thanks, What build of libnx is this using now - is it the latest?

DarkMatterCore · Sep 29, 2021

mrdude said:
Thanks, What build of libnx is this using now - is it the latest?

Latest stable release, yeah.

Homebrew WIP libusbhsfs - USB Mass Storage Class Host + Filesystem Mounter static library.

Well-Known Member

Otak Productions

Well-Known Member

Finding my light.

Well-Known Member

Developer

Well-Known Member

Finding my light.

Well-Known Member

Finding my light.

Developer

Well-Known Member

Finding my light.

Developer

Finding my light.

Finding my light.

Developer

Finding my light.

Developer

Finding my light.

Similar threads

Popular threads in this forum