Hacking Joy-Con Unity Library

[wormyrocks]

Hi all,

If you're a game developer and you want to build a Mac or PC game that uses the Joy-Con, I wrote a Unity library that lets you do that.

It parses accel/gyro data, lets you control HD Rumble, and of course reads the buttons and stick.

For now it only supports one Joy-Con at a time, as I have been working on the Joy-Con as a potential future control scheme for the HoloPlayer One by Looking Glass, where I work. Yep, I actually got paid to do this, it's awesome.

It should be noted that my sensor fusion code, which handles Joy-Con position estimation, is basically hot garbage right now. The Joy-Con doesn't have a magnetometer, so full 6dof position estimation is impossible. I'm aiming to get position estimation accuracy on par with that treasure chest game in 1-2-Switch. The Unity package comes by default with position estimation disabled, but will still return raw IMU data converted to Gs (accelerometer) and dps (gyroscope). I use the calibration values stored in the Joy-Con SPI flash, so the gyro values are actually pretty accurate.

If you want to help improve my position estimation, I'd welcome pull requests. The code I have right now is currently built on top of this guide.

Thanks a lot to CTCaer for all the help.

Um, I'm a new user, so I had to scrub this post of links before the forum would let me put it up. If someone can tell me how many posts I have to have before I can add links that'd be great.

Here is a link to the library:
github.
com/Looking-Glass/JoyconLib

Here is the guide I used for sensor fusion code:
starlino.
com/imu_guide.html

 

[BlastedGuy9905]

You need 10 posts.

 

[BORTZ]

Yeah sorry we have had some major spammer problems so when you first create a new account your privileges are limited.

 

[wormyrocks]

Thanks so much! If you could linkify those it'd be much appreciated.

 

[Xellar]

Looks nice

I'm going to check it out i may play with it a little.

 

[mfosse]

My sensor fusion code is also hot garbage, but feel free to look at it in my driver on github (mfosse) it uses a basic complementary filter, I didn't bother with the kalman filter and it doesn't work properly past 90° pitch for whatever reason

 

[wormyrocks]

Hi friends - my sensor fusion code is now better!
i. imgur.co m/Ye5qp1n.gif
Get the new version here: g ithub.c om/Looking-Glass/JoyconLib/releases

--------------------- MERGED ---------------------------

Hi friends - my sensor fusion code is now better!
i. imgur.co m/Ye5qp1n.gif
Get the new version here: g ithub.c om/Looking-Glass/JoyconLib/releases

(I would really appreciate it if a mod could image-ify and link-ify those. Seriously, I can't overstate how much the position tracking is improved in this version.)

 

[BlastedGuy9905]

Hi friends - my sensor fusion code is now better!
i. imgur.co m/Ye5qp1n.gif
Get the new version here: g ithub.c om/Looking-Glass/JoyconLib/releases

--------------------- MERGED ---------------------------


(I would really appreciate it if a mod could image-ify and link-ify those. Seriously, I can't overstate how much the position tracking is improved in this version.)

HOLY SHIT. That's amazing! Also, how do you make your Joy-Con's HOME Button light up?

 

[wormyrocks]

lol, I was using CTCaer's Joycon Utility to test out the output of some raw values and I think he put in some functionality to make it light up.

 

[kitzuki]

Nice Work.
For those lazy Here was the link http://i.imgur.com/Ye5qp1n.gif

 

[TotalInsanity4]

Hi friends - my sensor fusion code is now better!
i. imgur.co m/Ye5qp1n.gif
Get the new version here: g ithub.c om/Looking-Glass/JoyconLib/releases

--------------------- MERGED ---------------------------


(I would really appreciate it if a mod could image-ify and link-ify those. Seriously, I can't overstate how much the position tracking is improved in this version.)

So, out of curiosity, how much work would it require to rewrite this as a general driver?

 

[BlastedGuy9905]

lol, I was using CTCaer's Joycon Utility to test out the output of some raw values and I think he put in some functionality to make it light up.

okay, downloaded the tool, dumped the spi, what to do?

 

[wormyrocks]

okay, downloaded the tool, dumped the spi, what to do?

Um, mine just lights up as soon as I connect it to JC Toolkit. I assumed that was an Easter egg or something. If it doesn't work for you, you can always use JC Toolkit to send subcommand 0x38 as described on (this page)*, but... it's a light, it's really not that interesting.

So, out of curiosity, how much work would it require to rewrite this as a general driver?

How do you mean? The Joy-Con already shows up natively as a controller on Windows. There already is a (vJoy driver)** and (C driver)***, both in progress, on which I based the bulk of my work. To my knowledge, the only new/unique thing I did with this library was the position estimation. I also read and process input packets in separate threads - this approach, or something like it, is pretty vital if you want to avoid serious input lag, as the Joy-Con pushes packets every 15 milliseconds and will build up an enormous backlog if they are read any slower than that.

If you are just wondering how portable this code is - (it is quite straightforward)**** and you are welcome to look through it (and I'm happy to answer questions).

* github (dot) com (slash) dekuNukem/Nintendo_Switch_Reverse_Engineering/blob/master/bluetooth_hid_subcommands_notes.md
** github (dot) com (slash) mfosse/JoyCon-Driver
*** github (dot) com (slash) riking/joycon
**** github (dot) com (slash) Looking-Glass/JoyconLib/blob/master/Assets/JoyconLib_scripts/Joycon.cs

P.S. I freakin hate this ^^^^

 

[TotalInsanity4]

How do you mean? The Joy-Con already shows up natively as a controller on Windows. There already is a (vJoy driver)** and (C driver)***, both in progress, on which I based the bulk of my work. To my knowledge, the only new/unique thing I did with this library was the position estimation. I also read and process input packets in separate threads - this approach, or something like it, is pretty vital if you want to avoid serious input lag, as the Joy-Con pushes packets every 15 milliseconds and will build up an enormous backlog if they are read any slower than that.

If you are just wondering how portable this code is - (it is quite straightforward)**** and you are welcome to look through it (and I'm happy to answer questions).

* github (dot) com (slash) dekuNukem/Nintendo_Switch_Reverse_Engineering/blob/master/bluetooth_hid_subcommands_notes.md
** github (dot) com (slash) mfosse/JoyCon-Driver
*** github (dot) com (slash) riking/joycon
**** github (dot) com (slash) Looking-Glass/JoyconLib/blob/master/Assets/JoyconLib_scripts/Joycon.cs

P.S. I freakin hate this ^^^^

I was wondering specifically about the gyroscopic stuff (and HD rumble, should you choose to pursue that later). I might peruse your code, but I'm not particularly advanced and probably wouldn't know what I'm looking at half the time lol

(Also if you put "www." in front of the address it'll automatically link-ify it)

 

[APartOfMe]

this is insane! good work!

 

[BlastedGuy9905]

Um, mine just lights up as soon as I connect it to JC Toolkit. I assumed that was an Easter egg or something. If it doesn't work for you, you can always use JC Toolkit to send subcommand 0x38 as described on (this page)*, but... it's a light, it's really not that interesting.

it does light up, but cant i make it automatically light up as soon as it connects to my switch?

 

[APartOfMe]

it does light up, but cant i make it automatically light up as soon as it connects to my switch?

i didn't even know it lit up until a few days ago. it's a shame that ninty doesn't make it light up anywhere

 

[BlastedGuy9905]

i didn't even know it lit up until a few days ago. it's a shame that ninty doesn't make it light up anywhere

they are waiting until they release the new online with usernames and then it will light up if you recieve a notification.

 

[APartOfMe]

they are waiting until they release the new online with usernames and then it will light up if you recieve a notification.

cool

 

[Manurocker95]

Lol, this is something I was looking for xD Gonna try this now!