Hacking Infrared (night vision) and thermal imaging camera home-brew?

[Nincompoopdo]

People have recently found out that the Switch has an infrared and thermal camera in the right joy-con. Is it possible to make a home-brew app to enable the camera? There's a LABO app that uses the Infrared and thermal camera functions, but it displays the image in a very small frame. It will be cool to use the joycon as a remote night vision camera.

 

[KiiWii]

IR: yes
Thermal: rly?

 

[Nincompoopdo]

IR: yes
Thermal: rly?

Yes there's a thermal sensor, I didn't know until recently. They demo a LABO app that makes the drone follow the heat source (hand):

 

[Alex119098]

I would hope so. It seems like a cool thing to mess around with

 

[jurai]

are you sure that is actually a thermal sensor and not just a thermal looking rendering of the IR depth

 

[KiiWii]

Wow, thats a really odd thing for ninty to include

 

[Technicmaster0]

I'm pretty sure that it only has an IR camera and not a thermal camera.
There is an example on how to use the IR camera in libnx and I made a remake of the remote bug toy-con a few weeks ago which uses the IR camera aswell:
https://gbatemp.net/threads/labo-like-remote-controlled-vehicle-made-with-joy-cons.498861/

 

[notimp]

1st order of business - please get those full frame animated gifs out of this thread.

2nd: It wasn't recently found out.

3rd: Read this: https://gbatemp.net/threads/can-som...ndo-not-enabled-it-in-the-firmware-ye.499335/

The nro in there gives you a webcam like video stream, and the Switch screenshot function works in it as well. So you can already use it to take images - today. (Currently 3.0.0 users only. Because those are the only ones that can run homebrew apps.)

edit: Afair, you have to detach the controllers for the app to "start working".

 

[Mnecraft368]

If you sync the joycon R to a pc and download Joycon-Toolkit, you can see through the IR camera, although its pretty slow (that might just be my crap bluetooth connection though.

 

[Nincompoopdo]

1st order of business - please get those full frame animated gifs out of this thread.

2nd: It wasn't recently found out.

3rd: Read this: https://gbatemp.net/threads/can-som...ndo-not-enabled-it-in-the-firmware-ye.499335/

The nro in there gives you a webcam like video stream, and the Switch screensot function works in it as well. So you can already use it to take images - today. (Currently 3.0.0 users only. Because those are the only ones that can run homebrew apps.)

edit: Afair, you have to detach the controllers for the app to "start working".

I didn't know there's a thread about this already. Glad to see someone doing it!

--------------------- MERGED ---------------------------

are you sure that is actually a thermal sensor and not just a thermal looking rendering of the IR depth

According to this article at Kotaku, the camera can be switched to a heat-sensing mode:

https://www.kotaku.com.au/2018/02/the-switch-joy-cons-infrared-sensor-is-cooler-than-we-thought/

 

[notimp]

According to this article at Kotaku, the camera can be switched to a heat-sensing mode:

https://www.kotaku.com.au/2018/02/the-switch-joy-cons-infrared-sensor-is-cooler-than-we-thought/

According to common sense....

...Kotaku is linkfarm owned by Gawker Media, the mother of all clickfarms.

And entry level physics is not their strong side.
https://physics.stackexchange.com/q...fference-between-thermal-and-infrared-imaging

 

[Nincompoopdo]

If you sync the joycon R to a pc and download Joycon-Toolkit, you can see through the IR camera, although its pretty slow (that might just be my crap bluetooth connection though.

I use a Mac, can't try it.

 

[notimp]

Framerate on the (Switch "native") homebrew app isn't great either. Also - did you expect your bluetooth video streams to rival the crappy audio you get from your airpods? Because one of the two doesnt even have to stream video feeds, and still is pretty lackluster...

I extend my "not good at basic physics" description to also include gbatemp.

 

[Nincompoopdo]

According to common sense....

...Kotaku is linkfarm owned by Gawker Media, the mother of all clickfarms.

And entry level physics is not their strong side.
https://physics.stackexchange.com/q...fference-between-thermal-and-infrared-imaging

I think it's either Nintendo misleading the journalists who attended the demo or there really is a heat-sensor in the joy-con. Nintendo uses thermal stickers to guide the drone around the table.

We'll just have to wait until next week when LABO is released to find out.

 

[Mnecraft368]

I use a Mac, can't try it.

Does it work using Wine (if macs have Wine)

 

[notimp]

I think it's either Nintendo misleading the journalists who attended the demo or ...

And what I meant is, that its obvious, that this thing only has an IR camera in it, which does record light in the same wavelength spectrum, you'd be interested, in to do thermal imaging - if your baseline is room temperature.

Take the camera. Put it close to your face, or another warm object. It shows it bright and blown out - without external light in the room.

Or the other way around - it doesnt take a genius to extrapolate that Nintendo is probably not doing low light amplification, because by then they would produce professional night vision devices for 15k USD a pop.

You can also read the wikipedia Article about nightvision and count how often "infrared" can be found in there:
https://en.wikipedia.org/wiki/Night_vision_device

Or guesstimate how much this statement makes sense to you "any manufacturer of consumer goods in this age is just visiting chinese manufacturing lines, and ticking off boxes on a pre-existing sheet of paper, regarding that factory lines usual capabilities, and in the end "their" product comes out".

Likelyhood of N shoving a cheap sensor in there to enable "creative use cases": 100%
Likelyhood of this being anything else than a cheap 3-5 USD implementation of a CMOS sensor without an IR filter: 0%

(On the other hand, ... those marketing folks sound so trustworthy.. )

I'd say every tech journalist should know that. But what do I know..

 

[Mnecraft368]

You can also read the wikipedia Article about nightvision and count how often "infrared" can be found in there:

12 times if anybody was wondering

 

[Nincompoopdo]

Does it work using Wine (if macs have Wine)

I doubt it will work, WINE is very unreliable especially for driver related softwares.

--------------------- MERGED ---------------------------

And what I meant is, that its obvious, that this thing only has an IR camera in it, which does record light in the same wavelength spectrum, you'd be interested, in to do thermal imaging - if your baseline is room temperature.

Take the camera. Put it close to your face, or another warm object. It shows it bright and blown out - without external light in the room.

Or the other way around - it doesnt take a genius to extrapolate that Nintendo is probably not doing low light amplification, because by then they would produce professional night vision devices for 15k USD a pop.

You can also read the wikipedia Article about nightvision and count how often "infrared" can be found in there:
https://en.wikipedia.org/wiki/Night_vision_device

Or guesstimate how much this statement makes sense to you "any manufacturer of consumer goods in this age is just visiting chinese manufacturing lines, and ticking off boxes on a pre-existing sheet of paper, regarding that factory lines usual capabilities, and in the end "their" product comes out".

Likelyhood of N shoving a cheep sensor in there to enable "creative use cases": 100%
Likelyhood of this being anything else than a cheap 3-5 USD implementation of a CMOS sensor without an IR filter: 0%

(On the other hand, ... those marketing folks sound so trustworthy.. )

I'd say every tech journalist should know that. But what do I know..

LOL, I don't think anyone is expecting a high grade night vision/thermal device from Nintendo. Of all companies, Nintendo is known for using cheap components. I think they are using toy-grade components - not amazing but they serve their purpose. Just like PSVR, it's not as good as Vive or Rift but it's good enough - to remain cheap for consumers.

 

[TotalInsanity4]

Nintendo uses thermal stickers to guide the drone around the table.

No they don't, they use reflective stickers that the IR camera is sensitive to

 

[guily6669]

Thermal camera costs quite a damn lot even without any quality, even a cheap crap from China I bet would cost more than the whole joy-cons...

There's a caterpillar phone with it now, which is more aimed for workers to check electricity or heat pipes or whatever...