Hacking WiiUTablet - use the Gamepad as PC drawing tablet

PewnyPL

Well-Known Member
OP
Member
Joined
Feb 2, 2014
Messages
771
Trophies
1
XP
2,121
Country
Poland
A 2-part application, a HTML5 webpage to run on WiiU and a Java server program to run on target PC. Currently it's in very early alpha, but it can already establish conenction between Wii U and a PC and click+drag the cursor around.

To run it, two things must be changed:
  • host variable in WiiUTablet.js so it points to the target PC
  • resX and resY static variables in Main.java so they correspond to PC's screen resolution
First the server must be started, then the webpage loaded on the Wii U. If for whatever reason the connection is lost, simply refreshing the page will do the trick.

However, for some unknown reason there seems to be some lag on the mouse movement. While not a single packet is lost, the smooth and quick movement of stylus on the gamepad (ie. drawing a circle) creates a jagged line. It seems to be a problem of Java's AWT Robot class, but I may be wrong, could anyone check this?
Currently the jagginess is under control, sadly, the latency between Wii U and PC started to be noticeable (a split second delay between touch on the gamepad and the click showing on PC), this however seems impossible to fix.

Here is the link to Github: https://github.com/PewnyPL/WiiUTablet

Note: This project was only tested on Windows, however it should work on every Java-enabled system. Also, the pre-compiled binary attached here is designed with 1920x1080 resolution in mind, if your computer has a lower one it may cause strange behavior when moving beyond certain point on the touchscreen.

EDIT: Reuploaded the file, apparently I forgot the library.
 

Attachments

  • release1.zip
    82.8 KB · Views: 2,777
  • release2.zip
    88.2 KB · Views: 2,658

laudern

Well-Known Member
Member
Joined
Nov 14, 2010
Messages
719
Trophies
1
Location
Queensland, Australia
XP
1,620
Country
Australia
Would anyone actually want this? Sure I can understand the challenge of accomplishing such a feat, but really? A relatively small, touch resistive, single touch screen, is that the ideal setup? Just go buy a pen input device or a MS surface.
 

PewnyPL

Well-Known Member
OP
Member
Joined
Feb 2, 2014
Messages
771
Trophies
1
XP
2,121
Country
Poland
Would anyone actually want this?
Someone who already has a Wii U but doesn't want to spend money on a drawing tablet (yet). In some parts of the world it's not easy to get one, as either the only ones readily available are the more expensive ones, or if you want to order one from abroad, the shipping costs are ridiculous.
 

yuyuyup

Well-Known Member
Member
Joined
Apr 30, 2006
Messages
3,809
Trophies
2
Location
USA MTN timezone
Website
Visit site
XP
3,248
Country
United States
A 2-part application, a HTML5 webpage to run on WiiU and a Java server program to run on target PC. Currently it's in very early alpha, but it can already establish conenction between Wii U and a PC and click+drag the cursor around.

To run it, two things must be changed:
  • host variable in WiiUTablet.js so it points to the target PC
  • resX and resY static variables in Main.java so they correspond to PC's screen resolution
First the server must be started, then the webpage loaded on the Wii U. If for whatever reason the connection is lost, simply refreshing the page will do the trick.

However, for some unknown reason there seems to be some lag on the mouse movement. While not a single packet is lost, the smooth and quick movement of stylus on the gamepad (ie. drawing a circle) creates a jagged line. It seems to be a problem of Java's AWT Robot class, but I may be wrong, could anyone check this?

Here is the link to Github: https://github.com/PewnyPL/WiiUTablet

Note: This project was only tested on Windows, however it should work on every Java-enabled system.
Believe it or not, this sounds great to me (I guess it's just not good enough for some people.) Are you aware of http://libdrc.org/
 
  • Like
Reactions: CORE

endoverend

AKA zooksman
Member
Joined
Jun 6, 2013
Messages
2,846
Trophies
0
Website
zooksman.com
XP
2,905
Country
United States
I get the javascript to run on the Wii U browser, but unfortunately nothing happens to the pointer on the PC even after changing the host variable to the PC's local IP and running the .jar file. Maybe I need to change the port?
 

PewnyPL

Well-Known Member
OP
Member
Joined
Feb 2, 2014
Messages
771
Trophies
1
XP
2,121
Country
Poland
I get the javascript to run on the Wii U browser, but unfortunately nothing happens to the pointer on the PC even after changing the host variable to the PC's local IP. Maybe I need to change the port?

If the Java app compiles and starts without throwing an exception, then it should work. Make sure it's not blocked by a firewall, and that on the Wii U gamepad you see the touchscreen variables.
 
D

Deleted User

Guest
Whoa, cool for sharing something like this.
Though, er, how do I exactly run the java files?
I have no experience with Java.
 

endoverend

AKA zooksman
Member
Joined
Jun 6, 2013
Messages
2,846
Trophies
0
Website
zooksman.com
XP
2,905
Country
United States
If the Java app compiles and starts without throwing an exception, then it should work. Make sure it's not blocked by a firewall, and that on the Wii U gamepad you see the touchscreen variables.

I double click the java_websocket.jar file, and absolutely nothing happens. I get the variables on the gamepad screen. Is there something I'm doing wrong?
 

PewnyPL

Well-Known Member
OP
Member
Joined
Feb 2, 2014
Messages
771
Trophies
1
XP
2,121
Country
Poland
I double click the java_websocket.jar file, and absolutely nothing happens. I get the variables on the gamepad screen. Is there something I'm doing wrong?

java_websocket.jar is just a library. I did not add a pre-compiled binary there, adding it now to the first post.

Edit: Uploaded
 

endoverend

AKA zooksman
Member
Joined
Jun 6, 2013
Messages
2,846
Trophies
0
Website
zooksman.com
XP
2,905
Country
United States
Was able to compile & run it through eclipse, thanks for the advice. However this is impossible to use for drawing in its current iteration (lines are VERY jagged and you have to take a guess as to where you're actually putting the mouse on the screen lol), but it's still cool and a good idea.
 

PewnyPL

Well-Known Member
OP
Member
Joined
Feb 2, 2014
Messages
771
Trophies
1
XP
2,121
Country
Poland
Was able to compile & run it through eclipse, thanks for the advice. However this is impossible to use for drawing in its current iteration (lines are VERY jagged and you have to take a guess as to where you're actually putting the mouse on the screen lol), but it's still cool and a good idea.

Yeah, I know, I have no idea what causes those jagged lines as, again, no packets are dropped, everything sent from Wii U gets to the PC, it's like the mouse movement methods can't keep up. As for the 'guessing', as far as I know all drawing tablets work exactly like that, if you press the pen on a point in 1/4th of the X and Y location on the tablet, it's like if you pressed it in 1/4th X and Y pixel on your screen, so it's replicating that.
 

endoverend

AKA zooksman
Member
Joined
Jun 6, 2013
Messages
2,846
Trophies
0
Website
zooksman.com
XP
2,905
Country
United States
As for the 'guessing', as far as I know all drawing tablets work exactly like that, if you press the pen on a point in 1/4th of the X and Y location on the tablet, it's like if you pressed it in 1/4th X and Y pixel on your screen, so it's replicating that.

Lol, I've never tried a real drawing tablet, so I wouldn't know, but that seems really imprecise to me. What I meant to say is that there's no way to move the cursor without clicking. I heard most draw tablets allow you to hover over them with the pen to move without clicking, but that's obviously impossible with the gamepad.
 

Bu2d85

Well-Known Member
Member
Joined
Nov 6, 2014
Messages
1,042
Trophies
0
Age
38
XP
1,271
Country
United States
Lol, I've never tried a real drawing tablet, so I wouldn't know, but that seems really imprecise to me. What I meant to say is that there's no way to move the cursor without clicking. I heard most draw tablets allow you to hover over them with the pen to move without clicking, but that's obviously impossible with the gamepad.
If the program recognizes button presses it could be setup to not click when holding the left trigger or something. That way you could move the mouse without clicking.
 

endoverend

AKA zooksman
Member
Joined
Jun 6, 2013
Messages
2,846
Trophies
0
Website
zooksman.com
XP
2,905
Country
United States
If the program recognizes button presses it could be setup to not click when holding the left trigger or something. That way you could move the mouse without clicking.

It can't recognize button presses, it's just a web browser and the buttons are the gamepad are assigned for other things. A JS application wouldn't be able to detect that.
 

PewnyPL

Well-Known Member
OP
Member
Joined
Feb 2, 2014
Messages
771
Trophies
1
XP
2,121
Country
Poland
It can't recognize button presses, it's just a web browser and the buttons are the gamepad are assigned for other things. A JS application wouldn't be able to detect that.

Actually, JS can recognize every button on the Wii U gamepad, even the home button (even tho it immediately exits the app), I just didn't implement that yet in this.
 
  • Like
Reactions: pelago

PewnyPL

Well-Known Member
OP
Member
Joined
Feb 2, 2014
Messages
771
Trophies
1
XP
2,121
Country
Poland
Managed to get the jagginess under control, sadly, the latency between Wii U and PC started to be noticeable (a split second delay between touch on the gamepad and the click showing on PC). It may be impossible to fix however, so this project may get scrapped...
Github and first post have been updated.
 

Pelayo

Member
Newcomer
Joined
May 5, 2015
Messages
6
Trophies
0
Age
33
XP
49
Country
Well, I don't know exactly which file do I have to run to start the server... I tried running WiiUTablet.jar(nothing appears), but I just can't get in the page on the Wii U browser. It gives me an error. What am I doing wrong? I already changed the text in the .js and .JAVA files... Is it maybe the location of the .jar file(I did put it in the main folder)?
Anyways, this looks great, have you thought of adding this to USendMii as a feature?
 

PewnyPL

Well-Known Member
OP
Member
Joined
Feb 2, 2014
Messages
771
Trophies
1
XP
2,121
Country
Poland
Well, I don't know exactly which file do I have to run to start the server... I tried running WiiUTablet.jar(nothing appears)
That's weird, it should start without any problems, unless the port used by it is already taken. Maybe you have something wrong with Java?
but I just can't get in the page on the Wii U browser. It gives me an error.
What error does it give you? Did you host the .html and .js file on a webhosting? The .jar file doesn't start any web servers.
Anyways, this looks great, have you thought of adding this to USendMii as a feature?
If it was open source, I'd try.
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
  • No one is chatting at the moment.
  • BigOnYa @ BigOnYa:
    Offer to clean house, wash windows, walk dogs, anything. You gotta have drive in life to get anything. Nothing is just given to you, atleast when you get older.
  • ZeroT21 @ ZeroT21:
    it's gud pocket money
  • BigOnYa @ BigOnYa:
    I remember when I was 14, I lived by a Coffee delivery service, and I asked the owner if I could work, and he had me wash the trucks on weekends to make money.
  • ZeroT21 @ ZeroT21:
    i did carpet cleaning jobs, it was weird but doable
    +1
  • ZeroT21 @ ZeroT21:
    especially with all the tools available nowadays
  • ZeroT21 @ ZeroT21:
    back then all i had were brushes, cleaning agents and lots of elbow grease to work with
  • BigOnYa @ BigOnYa:
    Most kids don't want to work nowdays, they want everything handed to them. And to think you're poor just cause you don't own a Switch is ridiculous.
  • K3Nv2 @ K3Nv2:
    @BigOnYa, Give me money
    +2
  • ZeroT21 @ ZeroT21:
    some would say, being poor and healthy is fine, rather than being rich, sick and dying
    +1
  • ZeroT21 @ ZeroT21:
    need food? just grow some potatoes or whatever veggie. But society just messes shit up and make us work for paper to buy food
  • The Real Jdbye @ The Real Jdbye:
    being poor and healthy is the worst case scenario because you have the longest amount of time to "enjoy" being poor
  • wolffangalchemist @ wolffangalchemist:
    Yeah back in my late teens I used to mow about 10 lawns every two weeks for $35 a yard. It was stupid big yards I knew where paying a lot more than that to lawn care companies in a some what up tier gated community a few blocks from the area I lived. Was great getting $700 a month with no bills at the time and good excercise too, now I have bills so I never have any money to buy things.
  • K3Nv2 @ K3Nv2:
    Yet kids are always like I WANNA BE ADULT
    +1
  • BigOnYa @ BigOnYa:
    I wish I could go back to 14, living at home free and little to no worries.
  • K3Nv2 @ K3Nv2:
    Kids still have bs getting manipulated easily, believing dumb youtube content etc
    +1
  • BigOnYa @ BigOnYa:
    If I could go back, I'd be more of a horn ball than I am now. I'd be chasing all the girls I never had the balls to back then
  • K3Nv2 @ K3Nv2:
    I'd drop out of school sooner
  • wolffangalchemist @ wolffangalchemist:
    Yeah I would probably drop out sooner too, the country ass high school I went to wouldn't know intelligence if it hit the up side the head.
  • BigOnYa @ BigOnYa:
    I used to play those goonies games on NES, used to piss me off
  • wolffangalchemist @ wolffangalchemist:
    I think the most frustrating nes game as a kid to me was contra.
  • BakerMan @ BakerMan:
    well, i got school tomorrow, night yall
    BakerMan @ BakerMan: well, i got school tomorrow, night yall