Homebrew WiiU FTP Server (use WUT, fix inject failures and increase bandwith)

Laf111

Well-Known Member
OP
Member
Joined
Mar 19, 2019
Messages
139
Trophies
0
XP
1,010
Country
France
Based on ftpiiu but with the following issues fixed :
  • connections failures and crashs
  • file's dates (timestamps)
  • file injection (add rights on files received using IOSUHAX lib)
  • remove the one slot limitation on upload and unlock up to 8 simultaneous transfers (download/upload)
  • much more faster than the original

Few games such as WWHD check the save files'rights and refuse to import them if permissions rights are not set using IOSUHAX_FSA_ChangeMode.


WiiuFtpServer comes also with some extra features :

  • support Wiiu PRO and Wiimote controllers
  • you can choose to disable or enable the power saving feature
  • enable / disable VERBOSE mode on server side
  • mount NAND paths only if you ask for it
  • a network unbrick feature
  • a CRC calculation on server side and a tool to verify the CRC of files transferred

Download and details : https://github.com/Laf111/WiiUFtpServer


When using Ethernet (Wii-U adapter and Ethernet 5) :

 
Last edited by Laf111,

ploggy

WAKA! WAKA!
Member
Joined
Aug 29, 2007
Messages
4,326
Trophies
1
XP
6,238
Country
United Kingdom
Thank you for this :)
Is there a reason why you haven't released a readymade installable wup of the Channel? and is the Channel a forwarder?
If it is a forwarder I wont have to reinstalling the Chan every time there's an update :P
 

ploggy

WAKA! WAKA!
Member
Joined
Aug 29, 2007
Messages
4,326
Trophies
1
XP
6,238
Country
United Kingdom
@Laf111 your Chan files seem to be messed, Possibly the tga files are wrong?

Edit: got the Chan to work but once you load the app it doesn't exit so.. kinda pointless :/
 
Last edited by ploggy,

Laf111

Well-Known Member
OP
Member
Joined
Mar 19, 2019
Messages
139
Trophies
0
XP
1,010
Country
France
Sweet! No rush :) and you too

Hey,

V2 is out.

Took me more time than expected because i had to modify the code for the channel behaviour.
The channel version shutdown the Wii-U when exiting (bug with WUT:;procui = fail to go back to the Wiiu Menu...).

Have a nice day
 
Last edited by Laf111,
  • Like
Reactions: Masana and ploggy

ploggy

WAKA! WAKA!
Member
Joined
Aug 29, 2007
Messages
4,326
Trophies
1
XP
6,238
Country
United Kingdom
Hey,

V2 is out.

Took me more time than expected because i had to modify the code for the channel behaviour.
The channel version shutdown the Wii-U when exiting (bug with WUT:;procui = fail to go back to the Wiiu Menu...).

Have a nice day
Thank you. i'll check it out :)
 

V10lator

Well-Known Member
Member
Joined
Apr 21, 2019
Messages
974
Trophies
0
Age
34
XP
2,342
Country
Germany
Did you ever check the return value of https://github.com/Laf111/WiiUFtpServer/blob/main/src/net.c#L67 ? It should be 0, else it didn''t work. I'm asking this as I'm pretty sure it returns a none-zero value as you can't specify a 128 KB buffer without specifying a userspace buffer before (max kernel space is 63 KB). Have a look at these codes for how to set a userspace buffer: https://github.com/V10lator/NUSspli/blob/master/src/downloader.c#L229-L266 - Important: somemopt() will block untill socket_lib_finish() gets called, that's why you need to run it in its own thread).

Other than that you need to tune storage I/O, too. It doesn't help if you're able to transfair data faster than you're able to read and write it. The important part for high-speed is to specify a I/O buffer here, too. just like this: https://github.com/V10lator/NUSspli/blob/master/src/ioThread.c#L243 - If you don't do that WUT/libIOSUHAX will use a new buffer for each read()/write(), making it horrible slow. Note that the buffer needs to be 0x40 aligned and that you need to free() it after fclose().
 
Last edited by V10lator,
  • Like
Reactions: Laf111

Laf111

Well-Known Member
OP
Member
Joined
Mar 19, 2019
Messages
139
Trophies
0
XP
1,010
Country
France
Did you ever check the return value of https://github.com/Laf111/WiiUFtpServer/blob/main/src/net.c#L67 ? It should be 0, else it didn''t work. I'm asking this as I'm pretty sure it returns a none-zero value as you can't specify a 128 KB buffer without specifying a userspace buffer before (max kernel space is 63 KB). Have a look at these codes for how to set a userspace buffer: https://github.com/V10lator/NUSspli/blob/master/src/downloader.c#L229-L266 - Important: somemopt() will block untill socket_lib_finish() gets called, that's why you need to run it in its own thread).

Other than that you need to tune storage I/O, too. It doesn't help if you're able to transfair data faster than you're able to read and write it. The important part for high-speed is to specify a I/O buffer here, too. just like this: https://github.com/V10lator/NUSspli/blob/master/src/ioThread.c#L243 - If you don't do that WUT/libIOSUHAX will use a new buffer for each read()/write(), making it horrible slow. Note that the buffer needs to be 0x40 aligned and that you need to free() it after fclose().

Thank you !
I'll have a look.
 
  • Like
Reactions: V10lator

V10lator

Well-Known Member
Member
Joined
Apr 21, 2019
Messages
974
Trophies
0
Age
34
XP
2,342
Country
Germany
Great work. :) Still it seems to be slower than it could be. Maybe the next speed improvement could be multitasking? I didn't look how much is implemented already but you could do it like this:
  • Network I/O on core 0.
  • UI on core 1.
  • Storage I/O on core 2.
This needs an I/O queue but such a queue is a good thing anyway.

//EDIT: One question: Is there a reason to embed WUT and libIOSUHAX? I mean you depend on devkitPro anyway and with devkitPro getting these deps is as simple as
Code:
pacman -S wut wut-tools wiiu-libiosuhax
 
Last edited by V10lator,
  • Like
Reactions: Laf111

Laf111

Well-Known Member
OP
Member
Joined
Mar 19, 2019
Messages
139
Trophies
0
XP
1,010
Country
France
Great work. :) Still it seems to be slower than it could be. Maybe the next speed improvement could be multitasking? I didn't look how much is implemented already but you could do it like this:
  • Network I/O on core 0.
  • UI on core 1.
  • Storage I/O on core 2.
This needs an I/O queue but such a queue is a good thing anyway.

//EDIT: One question: Is there a reason to embed WUT and libIOSUHAX? I mean you depend on devkitPro anyway and with devkitPro getting these deps is as simple as
Code:
pacman -S wut wut-tools wiiu-libiosuhax

I struggled to find a version of FtpiiU that compiles... Then i removed all uneeded dependencies (libfat, ogc), choose WUT instead of dynamic libs. So i want to save some time to other user who want to rebuild it.
 
  • Like
Reactions: Cyan

Laf111

Well-Known Member
OP
Member
Joined
Mar 19, 2019
Messages
139
Trophies
0
XP
1,010
Country
France
Is the embedded WUT latest git master or latest release? I'm asking cause master has socket API changes you want to avoid (I'm using this WUT fork cause of that: https://github.com/V10lator/wut ).

Sorry, i was not connected since this morning.

I got this version with the pacman command when i installed devKitPro (early days of april).

I began to have a look to your socket optimization (i mean use user buffers). Thank you for that !
If i understand correctly, i need to use the release version of WUT ? ?

But i wanted to fix the timestamps first (this fix should be in libIOSUHAX. That's also why i compile the lib : i was pretty sure that the fix should be there).

In fact, i achieved my initial goals (fix injection + timestamp) needed for Cemu2WiiU to work (mainly injection, timestamp is only for a display purpose since the FTP server does not implement the MTDM function) and in that use, slow transferts are not a problem since save files represent only few MB to transfert.
 
Last edited by Laf111,

Laf111

Well-Known Member
OP
Member
Joined
Mar 19, 2019
Messages
139
Trophies
0
XP
1,010
Country
France
Funny, I get much slower speeds with your version.
I get around 950KB/s with FTPiiu-everywhere and 250KB/s with this one.

Damn, you're right !!!
I must have used a WUT version of ftp-everwhere for my bench...

https://1drv.ms/u/s!Apr2zdKB1g7fgnKD7aZwKpN5oDrU

The WUT socket API seems worse that dynamic_libs one.
It proves that "newer" does not always mean "better"...

Using the ftp-everywhere CBHC from the homebrew app store :

https://1drv.ms/u/s!Apr2zdKB1g7fgnPZAdR-Exd2Z43F

WiiuFtpServer using dynamic_libs + optim :

https://1drv.ms/u/s!Apr2zdKB1g7fgnTqtJOgwjaAal0M

This version will be available in v4-2. i'm currently tying to make a channel with it.

Thanks for your feeback.
 
Last edited by Laf111,

piratesephiroth

I wish I could read
Member
Joined
Sep 5, 2013
Messages
3,447
Trophies
1
Age
101
XP
3,131
Country
Brazil
Last edited by piratesephiroth,
General chit-chat
Help Users
  • No one is chatting at the moment.
    DEMONGreninjaPG @ DEMONGreninjaPG: kennies your honry