WiiUTablet - use the Gamepad as PC drawing tablet

Discussion in 'Wii U - Hacking & Backup Loaders' started by PewnyPL, May 3, 2015.

  1. PewnyPL
    OP

    PewnyPL GBAtemp Advanced Fan

    Member
    591
    254
    Feb 2, 2014
    Poland
    A 2-part application, a HTML5 webpage to run on WiiU and a Java server program to run on target PC. Currently it's in very early alpha, but it can already establish conenction between Wii U and a PC and click+drag the cursor around.

    To run it, two things must be changed:
    • host variable in WiiUTablet.js so it points to the target PC
    • resX and resY static variables in Main.java so they correspond to PC's screen resolution
    First the server must be started, then the webpage loaded on the Wii U. If for whatever reason the connection is lost, simply refreshing the page will do the trick.

    However, for some unknown reason there seems to be some lag on the mouse movement. While not a single packet is lost, the smooth and quick movement of stylus on the gamepad (ie. drawing a circle) creates a jagged line. It seems to be a problem of Java's AWT Robot class, but I may be wrong, could anyone check this?
    Currently the jagginess is under control, sadly, the latency between Wii U and PC started to be noticeable (a split second delay between touch on the gamepad and the click showing on PC), this however seems impossible to fix.

    Here is the link to Github: https://github.com/PewnyPL/WiiUTablet

    Note: This project was only tested on Windows, however it should work on every Java-enabled system. Also, the pre-compiled binary attached here is designed with 1920x1080 resolution in mind, if your computer has a lower one it may cause strange behavior when moving beyond certain point on the touchscreen.

    EDIT: Reuploaded the file, apparently I forgot the library.
     

    Attached Files:

  2. laudern

    laudern GBAtemp Advanced Fan

    Member
    611
    469
    Nov 14, 2010
    Australia
    Queensland, Australia
    Would anyone actually want this? Sure I can understand the challenge of accomplishing such a feat, but really? A relatively small, touch resistive, single touch screen, is that the ideal setup? Just go buy a pen input device or a MS surface.
     
  3. PewnyPL
    OP

    PewnyPL GBAtemp Advanced Fan

    Member
    591
    254
    Feb 2, 2014
    Poland
    Someone who already has a Wii U but doesn't want to spend money on a drawing tablet (yet). In some parts of the world it's not easy to get one, as either the only ones readily available are the more expensive ones, or if you want to order one from abroad, the shipping costs are ridiculous.
     
  4. yuyuyup

    yuyuyup GBAtemp Psycho!

    Member
    3,337
    774
    Apr 30, 2006
    United States
    USA MTN timezone
    Believe it or not, this sounds great to me (I guess it's just not good enough for some people.) Are you aware of http://libdrc.org/
     
  5. PewnyPL
    OP

    PewnyPL GBAtemp Advanced Fan

    Member
    591
    254
    Feb 2, 2014
    Poland
    I am aware of this and I SO would rather use it, but this is sadly a Linux only solution, and while I have no problems using it myself, I'd preffer to do it on Windows.
     
  6. endoverend

    endoverend AKA zooksman

    Member
    GBAtemp Patron
    endoverend is a Patron of GBAtemp and is helping us stay independent!

    Our Patreon
    2,847
    3,560
    Jun 6, 2013
    United States
    I get the javascript to run on the Wii U browser, but unfortunately nothing happens to the pointer on the PC even after changing the host variable to the PC's local IP and running the .jar file. Maybe I need to change the port?
     
  7. PewnyPL
    OP

    PewnyPL GBAtemp Advanced Fan

    Member
    591
    254
    Feb 2, 2014
    Poland
    If the Java app compiles and starts without throwing an exception, then it should work. Make sure it's not blocked by a firewall, and that on the Wii U gamepad you see the touchscreen variables.
     
  8. MasterF0x

    MasterF0x GBAtemp Regular

    Member
    111
    135
    Jun 18, 2014
    United States
    Whoa, cool for sharing something like this.
    Though, er, how do I exactly run the java files?
    I have no experience with Java.
     
  9. endoverend

    endoverend AKA zooksman

    Member
    GBAtemp Patron
    endoverend is a Patron of GBAtemp and is helping us stay independent!

    Our Patreon
    2,847
    3,560
    Jun 6, 2013
    United States
    I double click the java_websocket.jar file, and absolutely nothing happens. I get the variables on the gamepad screen. Is there something I'm doing wrong?
     
  10. PewnyPL
    OP

    PewnyPL GBAtemp Advanced Fan

    Member
    591
    254
    Feb 2, 2014
    Poland
    java_websocket.jar is just a library. I did not add a pre-compiled binary there, adding it now to the first post.

    Edit: Uploaded
     
  11. endoverend

    endoverend AKA zooksman

    Member
    GBAtemp Patron
    endoverend is a Patron of GBAtemp and is helping us stay independent!

    Our Patreon
    2,847
    3,560
    Jun 6, 2013
    United States
    Was able to compile & run it through eclipse, thanks for the advice. However this is impossible to use for drawing in its current iteration (lines are VERY jagged and you have to take a guess as to where you're actually putting the mouse on the screen lol), but it's still cool and a good idea.
     
  12. PewnyPL
    OP

    PewnyPL GBAtemp Advanced Fan

    Member
    591
    254
    Feb 2, 2014
    Poland
    Yeah, I know, I have no idea what causes those jagged lines as, again, no packets are dropped, everything sent from Wii U gets to the PC, it's like the mouse movement methods can't keep up. As for the 'guessing', as far as I know all drawing tablets work exactly like that, if you press the pen on a point in 1/4th of the X and Y location on the tablet, it's like if you pressed it in 1/4th X and Y pixel on your screen, so it's replicating that.
     
  13. endoverend

    endoverend AKA zooksman

    Member
    GBAtemp Patron
    endoverend is a Patron of GBAtemp and is helping us stay independent!

    Our Patreon
    2,847
    3,560
    Jun 6, 2013
    United States
    Lol, I've never tried a real drawing tablet, so I wouldn't know, but that seems really imprecise to me. What I meant to say is that there's no way to move the cursor without clicking. I heard most draw tablets allow you to hover over them with the pen to move without clicking, but that's obviously impossible with the gamepad.
     
  14. Bu2d85

    Bu2d85 GBAtemp Advanced Fan

    Member
    GBAtemp Patron
    Bu2d85 is a Patron of GBAtemp and is helping us stay independent!

    Our Patreon
    945
    265
    Nov 6, 2014
    United States
    If the program recognizes button presses it could be setup to not click when holding the left trigger or something. That way you could move the mouse without clicking.
     
  15. endoverend

    endoverend AKA zooksman

    Member
    GBAtemp Patron
    endoverend is a Patron of GBAtemp and is helping us stay independent!

    Our Patreon
    2,847
    3,560
    Jun 6, 2013
    United States
    It can't recognize button presses, it's just a web browser and the buttons are the gamepad are assigned for other things. A JS application wouldn't be able to detect that.
     
  16. PewnyPL
    OP

    PewnyPL GBAtemp Advanced Fan

    Member
    591
    254
    Feb 2, 2014
    Poland
    Actually, JS can recognize every button on the Wii U gamepad, even the home button (even tho it immediately exits the app), I just didn't implement that yet in this.
     
    pelago likes this.
  17. PewnyPL
    OP

    PewnyPL GBAtemp Advanced Fan

    Member
    591
    254
    Feb 2, 2014
    Poland
    Managed to get the jagginess under control, sadly, the latency between Wii U and PC started to be noticeable (a split second delay between touch on the gamepad and the click showing on PC). It may be impossible to fix however, so this project may get scrapped...
    Github and first post have been updated.
     
  18. Pelayo

    Pelayo Newbie

    Newcomer
    6
    0
    May 5, 2015
    Well, I don't know exactly which file do I have to run to start the server... I tried running WiiUTablet.jar(nothing appears), but I just can't get in the page on the Wii U browser. It gives me an error. What am I doing wrong? I already changed the text in the .js and .JAVA files... Is it maybe the location of the .jar file(I did put it in the main folder)?
    Anyways, this looks great, have you thought of adding this to USendMii as a feature?
     
  19. PewnyPL
    OP

    PewnyPL GBAtemp Advanced Fan

    Member
    591
    254
    Feb 2, 2014
    Poland
    That's weird, it should start without any problems, unless the port used by it is already taken. Maybe you have something wrong with Java?
    What error does it give you? Did you host the .html and .js file on a webhosting? The .jar file doesn't start any web servers.
    If it was open source, I'd try.
     
  20. NhgrtPlayer

    NhgrtPlayer Member

    Newcomer
    10
    0
    May 6, 2015
    France
    Hi, how are you supposed to use this please ?