Hardware What could the IR camera be used for?

Nerdtendo

Your friendly neighborhood idiot
OP
Member
Joined
Sep 29, 2016
Messages
1,762
Trophies
1
XP
4,592
Country
United States
*this is post is inspired by the (extremely late) news article about Switch's underused features*

So, the Switch has been out for a little more than 9 months and there is one feature we haven't seen any use out of. That feature is of course the infrared camera at the bottom of the right joycon. Aside from one mini game in the overpriced but misunderstood "1, 2, Switch", EDIT: and reloading in RE (thanks for the tip @duffmmann), it has done absolutely nothing. This raises the question of what it will be used for in the future.* We can rule out wii-like pointing because of its positioning. The fact that "Little Inferno" used point controls without the camera further confirms this. The most plausable theory I've heard is joycon expansions ala the circle pad pro. What potential ways do you think it will be used in upcoming games or additional hardware?

*For purposes of this discussion, we will assume that the camera will be used eventually.
 
Last edited by Nerdtendo,

duffmmann

Well-Known Member
Member
Joined
Mar 11, 2009
Messages
3,966
Trophies
2
XP
2,305
Country
United States
Resident Evil Revelations 1 and 2 use it for reloading your gun. Just like this (start at about 50 seconds):
Its pretty neat, completely optional as you can just push the button instead, but it adds an extra level of immersion that I enjoy especially when you use the joy con with motion control for these games for aiming and knifing.
 
Last edited by duffmmann,

Nerdtendo

Your friendly neighborhood idiot
OP
Member
Joined
Sep 29, 2016
Messages
1,762
Trophies
1
XP
4,592
Country
United States
Resident Evil Revelations 1 and 2 use it for reloading your gun. Just like this (start at about 50 seconds):
Its pretty neat, completely optional as you can just push the button instead, but it adds an extra level of immersion that I enjoy especially when you use the joy con with motion control for these games for aiming and knifing.

Cool. I'm not into horror games so I never knew that. It's a neat usage. I edited OP to include that
 

duffmmann

Well-Known Member
Member
Joined
Mar 11, 2009
Messages
3,966
Trophies
2
XP
2,305
Country
United States
Well I'd definitely like to see the option to reload like RE for any other first or third person shooter (that requires reloading) that comes to the Switch like Wolfenstein or patched into Doom. Its a clever use that works well, while still remaining completely optional.

Also, this is some of the info the patent for the IR blaster contains. While it's all just what could be done with the tech for gaming on the system, I think it may give some hints as to what else we might be able to expect (worth noting is that the patent also intended for the controller/system to also be able to project images, which as we know isn't the case with the Switch):
20160073033_4uvr1y.jpg

FIGS. 10A to 10D are each a diagram showing an example of a process based on a gesture input using the infrared camera 4. As shown in FIG. 10A, when the user makes their right hand into “rock” at a predetermined position in the right side surface direction of the portable electronic apparatus 1 while holding the portable electronic apparatus 1 with their left hand, a character 21 is displayed on the display 2. As shown in FIG. 10B, when, from the state of FIG. 10A, the user moves their right hand toward the right side surface of the portable electronic apparatus 1 while their right hand is kept as “rock”, the character 21 displayed on the display 2 is enlarged.

Furthermore, when, from the state shown in FIG. 10B, the user makes their right hand into “paper”, the facial expression of the character 21 on the display 2 changes (FIG. 10C). For example, as shown in FIG. 10C, the character 21 changes to have a smile expression. Moreover, as shown in FIG. 10D, when the user moves their right hand toward the portable electronic apparatus 1 while their right hand is kept as “paper”, the displayed character 21 is further enlarged, and the facial expression of the character 21 changes. At that time, the vibrator 11 may operate to vibrate the portable electronic apparatus 1, and a sound may be outputted from the speaker 13.

20160073033_57hrnh.jpg

FIGS. 12A and 12B are each a diagram showing another example of a process based on a gesture input using the infrared camera 4. In this example, a calculation question is displayed on the display 2. The user answers the question displayed on the display 2, by using their right hand. For example, when a character string “1+4=?” is displayed as shown in FIG. 12A, if the user performs, by using their right hand, a gesture input (a gesture of “paper”) indicating “5” which is the correct answer of this question, a display indicating that the answer is correct is performed on the display 2. At that time, a sound indicating that the answer is correct may be outputted, or the vibrator 11 may be operated in a pattern indicating that the answer is correct, to vibrate the portable electronic apparatus 1.

When a character string “1+1=?” is displayed as shown in FIG. 12B, if the user performs, by using their right hand, a gesture input (an input of “5” by a gesture) different from the correct answer of this question, a display indicating the answer is incorrect is performed on the display 2. At that time, a sound indicating that the answer is incorrect may be outputted, or the vibrator 11 may be operated in a pattern indicating that the answer is incorrect.

20160073033_65npnx.jpg

Next, an example of a process using the distance measuring sensor 5 will be described. FIG. 13 is a diagram showing the example of the process using the distance measuring sensor 5. As shown in FIG. 13, when the user puts their right hand at a predetermined position in the right side surface direction of the portable electronic apparatus 1 while holding the portable electronic apparatus 1 with their left hand, a snake character 22 appears from the right edge of the display 2. When the user moves their right hand toward the portable electronic apparatus 1, the snake character 22 extends leftward from the right edge of the display 2. Specifically, the portable electronic apparatus 1 calculates the distance between the portable electronic apparatus 1 and the right hand of the user on the basis of information from the distance measuring sensor 5, and sets a length of the snake character 22 in accordance with the calculated distance. Then, the portable electronic apparatus 1 displays the snake character 22 on the display 2. In addition, depending on the distance, the portable electronic apparatus 1 may operate the vibrator 11 to vibrate the portable electronic apparatus 1.

20160073033_8fvozg.jpg

As shown in (A) of FIG. 22, a batter is displayed on the display 2. When the user makes a gesture of “paper” with their right hand, the portable electronic apparatus 1 recognizes the gesture on the basis of an image from the infrared camera 4, and projects an image of a ball onto the right hand of the user by using the projector 8. Next, when the user makes a gesture of swinging their right hand, the image of the ball projected on the right hand of the user disappears, and the ball appears from the right edge of the display 2 and moves in the leftward direction of the screen ((B) of FIG. 22). When the user presses the input button 6B at the timing at which the ball reaches a predetermined range of the batter, the batter hits back the ball ((C) of FIG. 22). The hit-back ball moves in the rightward direction of the screen. The user makes a gesture of “rock” at predetermined timing (timing at which the hit-back ball reaches the right hand) after the ball reaches the right edge of the display 2. Accordingly, an image of the ball is projected onto the right hand of the user as if the user caught the hit-back ball ((D) of FIG. 22). At that time, characters “OUT” indicating that the user is successful in catching the ball may be displayed on the display 2, and a sound may be outputted, or the portable electronic apparatus 1 may vibrate. The portable electronic apparatus 1 may recognize the position of the right hand of the user (a position on a plane parallel to the right side surface of the portable electronic apparatus, and/or a position in a direction perpendicular to the right side surface) on the basis of an image from the infrared camera 4 or a distance measured by the distance measuring sensor 5, and may project an image of the ball onto the right hand of the user in accordance with a result of the recognition. When the user does not make a gesture of “rock” at the above-described predetermined timing or the timing at which the gesture of “rock” is made deviates from the above-described predetermined timing, the user fails in catching the ball, and the image of the ball as shown in (D) of FIG. 22 is not projected onto the right hand. For example, an image different from the ball may be projected onto the right hand, or characters “ERROR” or “HIT” may be displayed on the display 2. When a gesture made by the user cannot be recognized with the infrared camera 4, the same process as described above may be performed by detecting movement of the hand with the distance measuring sensor 5

q0rLjhB.png

For example, when an attachment including an operation section (hereinafter referred to as an "extension controller") is attached to the right controller 4, the main unit 2 can detect an operation performed on the operation section based on the image-capturing results from the infrared image-capturing section 123. Therefore, the main unit 2 can execute an information process in accordance with an operation performed on the operation section by using the image-capturing results.

Specifically, the extension controller can be attached to and detached from the right controller 4, and include buttons and a movable operation section such as a stick, for example. The extension controller includes, inside the housing, a movable portion that moves (herein, the movement includes rotation) in response to an operation performed on an operation section. For example, the movable portion is a member configured to move in response to a button, which is an operation section, being pressed down. Here, the extension controller is attached to the right controller 4 so that the infrared image-capturing section 123 can capture an image of the movable portion inside the housing. For example, the housing of the extension controller is provided with a window portion, and the infrared image-capturing section 123 can capture an image of the movable portion via the window portion, with the extension controller attached to the right controller 4. Then, the main unit 2 can identify an operation performed on an operation section of the extension controller based on the position and/or the attitude of the movable portion in the image captured by the infrared image-capturing section 123. Thus, by using captured images, the main unit 2 may execute an information process in response to an operation performed on the operation section.

I mean basically what we see here is that the intentions for the IR blaster (based on these patents anyways) seems to be to function in a similar (albeit far less sophisticated) fashion as the camera of the Kinect.
 

duffmmann

Well-Known Member
Member
Joined
Mar 11, 2009
Messages
3,966
Trophies
2
XP
2,305
Country
United States
how about a basic universal remote (like wiiu had) i can totally see holding the joycon upside-down with no issue for that

I want this too, but think about how it could possibly work. Either utilizing the right Joycon alone, where you're restricted to basic controls like channel up/down(X/B), volume up/down (A,Y) and maybe an input or menu button could be tied to the plus button, and then you can navigate those menus with the joystick and pushing the joystick in or any of the triggers to select (Keep in mind you'll have to be holding the right joycon upside down if this were to work), and I'm not entirely sure how if separated you could tell the joycon that its supposed to be in Universal remote mode now and not Switch controller mode, though I'm sure they could find a way. Not ideal, but this could work. Else, you could use the entire Switch as smart controller with the screen giving you options much like the Wii U gamepad worked as a smart universal remote. But this would require you to hold the entire Switch upside down because of where the IR blaster is located (but hey maybe if your Switch sensed that it was upside down it would automatically go into universal remote mode and the display would pop up on your screen upisde down from normal so that it appears right side up in this mode, this could work, but I think it would be pretty awkward). Point is, while I loved this aspect of the Wii U Gamepad, even though the Switch too might be capable of it, it would be not nearly as good of an implementation of the feature because of the design of the Switch compared to the Wii U Gamepad. Hell, the 3DS is better suited to be a universal remote than the Switch is at this point.
 
Last edited by duffmmann,

Nerdtendo

Your friendly neighborhood idiot
OP
Member
Joined
Sep 29, 2016
Messages
1,762
Trophies
1
XP
4,592
Country
United States

jt_1258

Ella
Member
Joined
Aug 21, 2016
Messages
3,051
Trophies
2
Age
24
XP
4,847
Country
United States
Doesn't count. It used gyro to mimick a pointer. Not the IR camera.
still, his point that it's not possible to wii pointer at all was the point of his I intended to hit, also, I don't think it would be that hard for nintendo to release a usb sensor bar for the switch anyway, it's possible
 

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,284
Country
United Kingdom
*this is post is inspired by the (extremely late) news article about Switch's underused features*

How is it late? Had it been about the PS2 then it would be, here though there are likely a few years of it having games be designed and built from scratch aka still current and thus a perfectly viable question. I am genuinely puzzled and do not get the confusion over this one. The only thing I can think is people are so conditioned to get such discussions at the very launch/run up of a console's launch (or addon's launch) that having one a bit later is somehow odd.

Anyway I am going to have to look at the range, refresh rate and resolution of this one, and maybe how it might be augmented by the other features it has. If it is a webcam with an IR pass/with IR filters then that is one thing, if it is a glorified heat sensor from a burglar alarm then that is more limiting and if it is a remote control receiver then eh. I will probably then split it between homebrew and commercial uses -- for some homebrew fun I will litter my room with IR LEDs but no dev is going to do that commercially, slim possible exception if amiibos or something contained IR LEDs.

Littering the place I can then do a maze in the dark (or invisible in the light) which could be fun.
Hiding an IR LED in something could then make for an amusing puzzle game where you get to point at the correct item after following some clues and the one with a suitable LED (blinking in a pattern or something) does whatever it does in games. I will give that the same thing can be done on a screen and has been able to be done for years but people do seem to enjoy breaking whatever the gaming equivalent of the fourth wall is.

I imagine it would make a for a very poor multiplayer game of "laser" tag/quest/whatever. Equally do I imagine enough people having switches out in the world to make it viable -- 3ds streetpass and DS wifi was good for it but the switch install base would make it tricky, and it is not even like there is the

Is it something that can decode a TV remote signal? I always liked the things in eternal darkness where it would drop the "volume" of my TV (though at the time I was using a nice wood panel effect 70s behemoth that had nothing resembling an on screen display). Granted HDMI CEC is probably the better candidate here and if there is no IR sender...

Thus for games you have a sensor bar or what looked like fairly close to skin there (though if it was to prevent false positives then I shall await results on the three Rs above).
 

Nerdtendo

Your friendly neighborhood idiot
OP
Member
Joined
Sep 29, 2016
Messages
1,762
Trophies
1
XP
4,592
Country
United States
What are the mini-games for 1-2-Switch before we keep going here.
It is a single mini game where you hold the camera to your mouth and open/close it really fast to eat sandwiches.

How is it late? Had it been about the PS2 then it would be, here though there are likely a few years of it having games be designed and built from scratch aka still current and thus a perfectly viable question. I am genuinely puzzled and do not get the confusion over this one. The only thing I can think is people are so conditioned to get such discussions at the very launch/run up of a console's launch (or addon's launch) that having one a bit later is somehow odd.

9 months after a console's release seems pretty late for a prediction post. We pretty much know our answer. We've seen good use out of every feature advertised except for the IR camera.
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
    Xdqwerty @ Xdqwerty: I'm sleepy good night