Homebrew WIP libusbhsfs - USB Mass Storage Class Host + Filesystem Mounter static library.

[blawar]

Yeah. It's the same reason why more than one FAT volume isn't being mounted right now - the way it handles multiple partitions expects you to know how many partitions there are for each available drive beforehand.

EXT support is also in my plans.

Are there a lot of people who use external hard drives formatted to EXT*?

I uploaded snippets of the MBR/GPT code I wrote for Tinfoil, I hope it saves you some time if you intend to parse the MBR/GPT manually: https://gist.github.com/blawar/98635098baa9ae8ba502dfd62a5c8417

note that EXFAT and NTFS both share the same partition code (annoying). The only way to differentiate them is to actually read from the partition itself ti try to figure out the file system.

 

[DarkMatterCore]

@blawar Thanks! I'll check that out in a minute.

Not a whole lot of people uses EXT in their external drives, I reckon that. But I want to offer that option anyway. Some developers running a Linux distro most likely do, as well as some end users (I know a few who do) - and since this is aimed at developers in the first place, I just feel like it may be a good thing in the long run.

I've been around since the Wii era, and EXT support was something big back then. I see no reason to, at least, not try to surpass what people achieved back then.

 

[ploggy]

@blawar Thanks! I'll check that out in a minute.

Not a whole lot of people uses EXT in their external drives, I reckon that. But I want to offer that option anyway. Some developers running a Linux distro most likely do, as well as some end users (I know a few who do) - and since this is aimed at developers in the first place, I just feel like it may be a good thing in the long run.

I've been around since the Wii era, and EXT support was something big back then. I see no reason to, at least, not try to surpass what people achieved back them.

Where were you during the WiiU days ? I still don't think we have proper WiiU hdd support for HB lol

 

[DarkMatterCore]

Where were you during the WiiU days ? I still don't think we have proper WiiU hdd support for HB lol

In college. I always wanted to get a Wii U, but I just couldn't back then. And the total obliteration of my country's economical system didn't make things easier (still doesn't).

 

[ploggy]

In college. I always wanted to get a Wii U, but I just couldn't back then. And the total obliteration of my country's economical system didn't make things easier (still doesn't).

oof yeah that'll do it but if you ever do find yourself with a WiiU we could really use your talents heh

 

[peteruk]

@Ghost92 Please stop. I just don't want the thread to be derailed.

Each developer is free to implement stuff like this in whichever way they want. This is just another option.

If they can do it better than us, then so be it. This is not a competition.

Love you DKM - sending positive vibes to you <3

 

[DarkMatterCore]

oof yeah that'll do it but if you ever do find yourself with a WiiU we could really use your talents heh

I'll get on it if I ever get one, for sure.

 

[blawar]

Could you please implement SX's USB api on the backend, so developers who use this library will automatically work on both atmosphere and sxos with no code changes? The relevant code is here: https://github.com/blawar/tinleaf/blob/master/source/nx/usbhdd.cpp

It is not currently working in SXOS though because the code in Tinleaf is low quality, and I do not believe their API supports relative paths.

 

[DarkMatterCore]

Could you please implement SX's USB api on the backend, so developers who use this library will automatically work on both atmosphere and sxos with no code changes? The relevant code is here: https://github.com/blawar/tinleaf/blob/master/source/nx/usbhdd.cpp

It is not currently working in SXOS though because the code in Tinleaf is low quality, and I do not believe their API supports relative paths.

Sure. If it can be done, I'll look into it. Doesn't really matter if they support relative paths or not, because I reimplemented that in a way similar to how libnx handles it - which is allocating a cwd buffer per mounted volume and just pass a full, absolute path to the destination filesystem library. Check the dev branch - those funny public functions for (un)setting a default device are no more.

I'm currently working on MBR / GPT / SFD (VBR) parsing + fixing some issues with the BOT driver (someone broke usb:hs using a Vita in UMS mode, somehow). lol

--------------------- MERGED ---------------------------

@blawar Would you give permission to license the SX service init code under ISC? Nevermind, the license left by TX is permissive enough.

 

[blawar]

Sure. If it can be done, I'll look into it. Doesn't really matter if they support relative paths or not, because I reimplemented that in a way similar to how libnx handles it - which is allocating a cwd buffer per mounted volume and just pass a full, absolute path to the destination filesystem library. Check the dev branch - those funny public functions for (un)setting a default device are no more.

I'm currently working on MBR / GPT / SFD (VBR) parsing + fixing some issues with the BOT driver (someone broke usb:hs using a Vita in UMS mode, somehow). lol

--------------------- MERGED ---------------------------

@blawar Would you give permission to license the SX service init code under ISC? Nevermind, the license left by TX is permissive enough.

I was gong to say, their license is pretty permissive. So I downloaded those files from the website years ago, and then I ported them to the latest libnx when they did that big service api rewrite. However any code that I wrote in that tinleaf repo you are free to use and re-license in any way you see fit.

--------------------- MERGED ---------------------------

Also, ensure it is actually SXOS and not reinx or AMS before you try to initialize TX services or it will crash the user's switch. The code used for detecting SXOS on that file I linked works.

 

[DarkMatterCore]

@blawar Is it absolutely necessary to initialize a handle to the tx service beforehand?

 

[blawar]

@blawar Is it absolutely necessary to initialize a handle to the tx service beforehand?

I just tested it, and no, tx.h, tx.c, txInitialize(), and txExit() do not appear to be needed.

 

[DarkMatterCore]

I just tested it, and no, tx.h, tx.c, txInitialize(), and txExit() do not appear to be needed.

https://github.com/DarkMatterCore/libusbhsfs/commit/75f53bf707053f6b070ec3c8fca53b9ef9ccece5

There you go. It's still untested (no Switch, no SX OS). Feel free to build it and try it.

It should work exactly like it does under Atmosphère. If successful, usbHsFsGetMountedDeviceCount() should always return 1 after the user event is signaled, though.

 

[blawar]

https://github.com/DarkMatterCore/libusbhsfs/commit/75f53bf707053f6b070ec3c8fca53b9ef9ccece5

There you go. It's still untested (no Switch, no SX OS). Feel free to build it and try it.

It should work exactly like it does under Atmosphère. If successful, usbHsFsGetMountedDeviceCount() should always return 1 after the user event is signaled, though.

I compiled it and ran your example app in SXOS with USBHDD attached and got this:

USB MAss Storage status change event triggered!
Mounted USB Mass Storage device count: 1
Failed to list mounted USB Mass Storage devices!

 

[blawar]

This fixes the error, there was a logic error

u32 usbHsFsListMountedDevices(UsbHsFsDevice *out, u32 max_count)
{
    mutexLock(&g_managerMutex);
 
    u32 device_count = 0, ret = 0;
 
    if (g_isSXOS)
    {
        if(g_sxOSDeviceAvailable)
        {
            /* Copy device data, update return value and jump to the end. */
            memcpy(out, &g_sxOSDevice, sizeof(UsbHsFsDevice));
            ret = 1;
        }
        goto end;
    }
 
    if (!g_usbHsFsInitialized || !g_driveCount || !g_driveContexts || (!g_isSXOS && !(device_count = usbHsFsMountGetDevoptabDeviceCount())) || !out || !max_count)
    {
        USBHSFS_LOG("Invalid parameters!");
        goto end;
    }
 
    for(u32 i = 0; i < g_driveCount; i++)
    {
        UsbHsFsDriveContext *drive_ctx = &(g_driveContexts);
    
        for(u8 j = 0; j < drive_ctx->lun_count; j++)
        {
            UsbHsFsDriveLogicalUnitContext *lun_ctx = &(drive_ctx->lun_ctx[j]);
        
            for(u32 k = 0; k < lun_ctx->fs_count; k++)
            {
                UsbHsFsDriveLogicalUnitFileSystemContext *fs_ctx = &(lun_ctx->fs_ctx[k]);
            
                /* Fill device element. */
                UsbHsFsDevice *device = &(out[ret++]);  /* Increase return value. */
                usbHsFsFillDeviceElement(drive_ctx, lun_ctx, fs_ctx, device);
            
                /* Jump out of the loops if we have reached a limit */
                if (ret >= max_count || ret >= device_count) goto end;
            }
        }
    }
 
end:
    mutexUnlock(&g_managerMutex);
 
    return ret;
}

--------------------- MERGED ---------------------------

This is the output from your test program on SXOS. IIRC SX's api does not support renaming files on the hard drive which may be the cause of that error.

 

[DarkMatterCore]

Yeah. Errors like that were to be expected. I stubbed certain calls from its devoptab device interface (chdir, rename, dirreset, utimes and others).

--------------------- MERGED ---------------------------

Hmm, gotta take another good look at statvfs(). It is supported.

 

[blawar]

Yeah. Errors like that were to be expected. I stubbed certain calls from its devoptab device interface (chdir, rename, dirreset, utimes and others).

--------------------- MERGED ---------------------------

Hmm, gotta take another good look at statvfs(). It is supported.

I updated Tinleaf to use use your dev branch, and released a binary: https://github.com/blawar/tinleaf/releases/tag/1.4.1

I tested it on both SXOS and AMS, works great.

 

[DarkMatterCore]

I updated Tinleaf to use use your dev branch, and released a binary: https://github.com/blawar/tinleaf/releases/tag/1.4.1

I tested it on both SXOS and AMS, works great.

Nice! I'm looking into the statvfs() issue atm, but I fail to see what could have gone wrong here:

Ah, the return value. lol I should get some sleep.

 

[blawar]

Nice! I'm looking into the statvfs() issue atm, but I fail to see what could have gone wrong here:

static int usbfsdev_statvfs(struct _reent *r, const char *path, struct statvfs *buf)
{
    Result rc = 0;
    u64 freespace = 0, totalspace = 0;
    int ret = -1;
 
    rc = usbFsStatFilesystem(&totalspace, &freespace);
    if (R_SUCCEEDED(rc))
    {
        memset(buf, 0, sizeof(struct statvfs));
        buf->f_bsize   = 1;
        buf->f_frsize  = 1;
        buf->f_blocks  = totalspace;
        buf->f_bfree   = freespace;
        buf->f_bavail  = freespace;
        buf->f_files   = 0;
        buf->f_ffree   = 0;
        buf->f_favail  = 0;
        buf->f_fsid    = 0;
        buf->f_flag    = ST_NOSUID;
        buf->f_namemax = 0;
    } else {
        r->_errno = EINVAL;
    }
 
    return ret;
}

They don't even take an input path.

It might be a bug with SXOS's driver. I remember when they first released the homebrew library, I had a graph at the top for hdd free space but it was very buggy and locked up my app for like 30 seconds so I removed it.

Perhaps a further down the road thing (since it is 99% there already), however SXOS' server lets you read and write directly from the storage. It is possible for you to just issue raw storage read/write commands and unify the code even more. Then you could implement the things you stubbed.

Thanks for the quick work. This is a really great thing that there is a standard interface for accessing HDD's on all CFW's.

 

[DarkMatterCore]

Perhaps a further down the road thing (since it is 99% there already), however SXOS' server lets you read and write directly from the storage. It is possible for you to just issue raw storage read/write commands and unify the code even more. Then you could implement the things you stubbed.

Thanks for the quick work. This is a really great thing that there is a standard interface for accessing HDD's on all CFW's.

I wouldn't have done it if you didn't give me permission to relicense the code I needed, so thank you.

About the raw logical block commands, I did consider it but I only saw "ReadRaw" - no "WriteRaw" available as part of the IPC command list.

--------------------- MERGED ---------------------------

There's also no command to retrieve the logical block size and the total logical block count.