Hacking WiiUTablet - use the Gamepad as PC drawing tablet

PewnyPL

Well-Known Member
OP
Member
Joined
Feb 2, 2014
Messages
745
Trophies
0
XP
1,786
Country
Poland
A 2-part application, a HTML5 webpage to run on WiiU and a Java server program to run on target PC. Currently it's in very early alpha, but it can already establish conenction between Wii U and a PC and click+drag the cursor around.

To run it, two things must be changed:
  • host variable in WiiUTablet.js so it points to the target PC
  • resX and resY static variables in Main.java so they correspond to PC's screen resolution
First the server must be started, then the webpage loaded on the Wii U. If for whatever reason the connection is lost, simply refreshing the page will do the trick.

However, for some unknown reason there seems to be some lag on the mouse movement. While not a single packet is lost, the smooth and quick movement of stylus on the gamepad (ie. drawing a circle) creates a jagged line. It seems to be a problem of Java's AWT Robot class, but I may be wrong, could anyone check this?
Currently the jagginess is under control, sadly, the latency between Wii U and PC started to be noticeable (a split second delay between touch on the gamepad and the click showing on PC), this however seems impossible to fix.

Here is the link to Github: https://github.com/PewnyPL/WiiUTablet

Note: This project was only tested on Windows, however it should work on every Java-enabled system. Also, the pre-compiled binary attached here is designed with 1920x1080 resolution in mind, if your computer has a lower one it may cause strange behavior when moving beyond certain point on the touchscreen.

EDIT: Reuploaded the file, apparently I forgot the library.
 

Attachments

  • release1.zip
    82.8 KB · Views: 2,663
  • release2.zip
    88.2 KB · Views: 2,527

laudern

Well-Known Member
Member
Joined
Nov 14, 2010
Messages
720
Trophies
0
Location
Queensland, Australia
XP
1,594
Country
Australia
Would anyone actually want this? Sure I can understand the challenge of accomplishing such a feat, but really? A relatively small, touch resistive, single touch screen, is that the ideal setup? Just go buy a pen input device or a MS surface.
 

PewnyPL

Well-Known Member
OP
Member
Joined
Feb 2, 2014
Messages
745
Trophies
0
XP
1,786
Country
Poland
Would anyone actually want this?
Someone who already has a Wii U but doesn't want to spend money on a drawing tablet (yet). In some parts of the world it's not easy to get one, as either the only ones readily available are the more expensive ones, or if you want to order one from abroad, the shipping costs are ridiculous.
 

yuyuyup

Well-Known Member
Member
Joined
Apr 30, 2006
Messages
3,717
Trophies
1
Location
USA MTN timezone
Website
Visit site
XP
2,516
Country
United States
A 2-part application, a HTML5 webpage to run on WiiU and a Java server program to run on target PC. Currently it's in very early alpha, but it can already establish conenction between Wii U and a PC and click+drag the cursor around.

To run it, two things must be changed:
  • host variable in WiiUTablet.js so it points to the target PC
  • resX and resY static variables in Main.java so they correspond to PC's screen resolution
First the server must be started, then the webpage loaded on the Wii U. If for whatever reason the connection is lost, simply refreshing the page will do the trick.

However, for some unknown reason there seems to be some lag on the mouse movement. While not a single packet is lost, the smooth and quick movement of stylus on the gamepad (ie. drawing a circle) creates a jagged line. It seems to be a problem of Java's AWT Robot class, but I may be wrong, could anyone check this?

Here is the link to Github: https://github.com/PewnyPL/WiiUTablet

Note: This project was only tested on Windows, however it should work on every Java-enabled system.
Believe it or not, this sounds great to me (I guess it's just not good enough for some people.) Are you aware of http://libdrc.org/
 
  • Like
Reactions: CORE

endoverend

AKA zooksman
Member
Joined
Jun 6, 2013
Messages
2,846
Trophies
0
Website
zooksman.com
XP
2,869
Country
United States
I get the javascript to run on the Wii U browser, but unfortunately nothing happens to the pointer on the PC even after changing the host variable to the PC's local IP and running the .jar file. Maybe I need to change the port?
 

PewnyPL

Well-Known Member
OP
Member
Joined
Feb 2, 2014
Messages
745
Trophies
0
XP
1,786
Country
Poland
I get the javascript to run on the Wii U browser, but unfortunately nothing happens to the pointer on the PC even after changing the host variable to the PC's local IP. Maybe I need to change the port?

If the Java app compiles and starts without throwing an exception, then it should work. Make sure it's not blocked by a firewall, and that on the Wii U gamepad you see the touchscreen variables.
 
D

Deleted User

Guest
Whoa, cool for sharing something like this.
Though, er, how do I exactly run the java files?
I have no experience with Java.
 

endoverend

AKA zooksman
Member
Joined
Jun 6, 2013
Messages
2,846
Trophies
0
Website
zooksman.com
XP
2,869
Country
United States
If the Java app compiles and starts without throwing an exception, then it should work. Make sure it's not blocked by a firewall, and that on the Wii U gamepad you see the touchscreen variables.

I double click the java_websocket.jar file, and absolutely nothing happens. I get the variables on the gamepad screen. Is there something I'm doing wrong?
 

PewnyPL

Well-Known Member
OP
Member
Joined
Feb 2, 2014
Messages
745
Trophies
0
XP
1,786
Country
Poland
I double click the java_websocket.jar file, and absolutely nothing happens. I get the variables on the gamepad screen. Is there something I'm doing wrong?

java_websocket.jar is just a library. I did not add a pre-compiled binary there, adding it now to the first post.

Edit: Uploaded
 

endoverend

AKA zooksman
Member
Joined
Jun 6, 2013
Messages
2,846
Trophies
0
Website
zooksman.com
XP
2,869
Country
United States
Was able to compile & run it through eclipse, thanks for the advice. However this is impossible to use for drawing in its current iteration (lines are VERY jagged and you have to take a guess as to where you're actually putting the mouse on the screen lol), but it's still cool and a good idea.
 

PewnyPL

Well-Known Member
OP
Member
Joined
Feb 2, 2014
Messages
745
Trophies
0
XP
1,786
Country
Poland
Was able to compile & run it through eclipse, thanks for the advice. However this is impossible to use for drawing in its current iteration (lines are VERY jagged and you have to take a guess as to where you're actually putting the mouse on the screen lol), but it's still cool and a good idea.

Yeah, I know, I have no idea what causes those jagged lines as, again, no packets are dropped, everything sent from Wii U gets to the PC, it's like the mouse movement methods can't keep up. As for the 'guessing', as far as I know all drawing tablets work exactly like that, if you press the pen on a point in 1/4th of the X and Y location on the tablet, it's like if you pressed it in 1/4th X and Y pixel on your screen, so it's replicating that.
 

endoverend

AKA zooksman
Member
Joined
Jun 6, 2013
Messages
2,846
Trophies
0
Website
zooksman.com
XP
2,869
Country
United States
As for the 'guessing', as far as I know all drawing tablets work exactly like that, if you press the pen on a point in 1/4th of the X and Y location on the tablet, it's like if you pressed it in 1/4th X and Y pixel on your screen, so it's replicating that.

Lol, I've never tried a real drawing tablet, so I wouldn't know, but that seems really imprecise to me. What I meant to say is that there's no way to move the cursor without clicking. I heard most draw tablets allow you to hover over them with the pen to move without clicking, but that's obviously impossible with the gamepad.
 

Bu2d85

Well-Known Member
Member
Joined
Nov 6, 2014
Messages
1,039
Trophies
0
Age
37
XP
1,055
Country
United States
Lol, I've never tried a real drawing tablet, so I wouldn't know, but that seems really imprecise to me. What I meant to say is that there's no way to move the cursor without clicking. I heard most draw tablets allow you to hover over them with the pen to move without clicking, but that's obviously impossible with the gamepad.
If the program recognizes button presses it could be setup to not click when holding the left trigger or something. That way you could move the mouse without clicking.
 

endoverend

AKA zooksman
Member
Joined
Jun 6, 2013
Messages
2,846
Trophies
0
Website
zooksman.com
XP
2,869
Country
United States
If the program recognizes button presses it could be setup to not click when holding the left trigger or something. That way you could move the mouse without clicking.

It can't recognize button presses, it's just a web browser and the buttons are the gamepad are assigned for other things. A JS application wouldn't be able to detect that.
 

PewnyPL

Well-Known Member
OP
Member
Joined
Feb 2, 2014
Messages
745
Trophies
0
XP
1,786
Country
Poland
It can't recognize button presses, it's just a web browser and the buttons are the gamepad are assigned for other things. A JS application wouldn't be able to detect that.

Actually, JS can recognize every button on the Wii U gamepad, even the home button (even tho it immediately exits the app), I just didn't implement that yet in this.
 
  • Like
Reactions: pelago

PewnyPL

Well-Known Member
OP
Member
Joined
Feb 2, 2014
Messages
745
Trophies
0
XP
1,786
Country
Poland
Managed to get the jagginess under control, sadly, the latency between Wii U and PC started to be noticeable (a split second delay between touch on the gamepad and the click showing on PC). It may be impossible to fix however, so this project may get scrapped...
Github and first post have been updated.
 

Pelayo

Member
Newcomer
Joined
May 5, 2015
Messages
6
Trophies
0
Age
31
XP
29
Country
Well, I don't know exactly which file do I have to run to start the server... I tried running WiiUTablet.jar(nothing appears), but I just can't get in the page on the Wii U browser. It gives me an error. What am I doing wrong? I already changed the text in the .js and .JAVA files... Is it maybe the location of the .jar file(I did put it in the main folder)?
Anyways, this looks great, have you thought of adding this to USendMii as a feature?
 

PewnyPL

Well-Known Member
OP
Member
Joined
Feb 2, 2014
Messages
745
Trophies
0
XP
1,786
Country
Poland
Well, I don't know exactly which file do I have to run to start the server... I tried running WiiUTablet.jar(nothing appears)
That's weird, it should start without any problems, unless the port used by it is already taken. Maybe you have something wrong with Java?
but I just can't get in the page on the Wii U browser. It gives me an error.
What error does it give you? Did you host the .html and .js file on a webhosting? The .jar file doesn't start any web servers.
Anyways, this looks great, have you thought of adding this to USendMii as a feature?
If it was open source, I'd try.
 
General chit-chat
Help Users
  • No one is chatting at the moment.
    KenniesNewName @ KenniesNewName: Mini Smartphone iLight 11 Pro The World's Smallest 11 Pro Android Mobile Phone, Super Small...