To gauge general interest.

Meteor7

Guess where this thumb goes.
OP
Member
Joined
Jun 9, 2014
Messages
1,336
Trophies
1
Location
a fit of spasms and accidental black magic
XP
4,643
Country
United States
Hello all. I've been recently thinking about starting a video series on YouTube where different methods of playing games across different forms of hardware and software are tested in-depth for how much input lag each would produce and under which circumstances. The results would be presented in the YouTube video itself, featuring recorded footage, full numerical data on input lag, light statistical analysis, and detailed descriptions of steps and specific tools for repeatibility. The working title right now is "Input Lag Police", or ILP, but I'm really not too keen on that title. Maybe someone has a better suggestion?

So, this is a resource that I would love to have had as a gamer who uses different emulators on different devices, but I'd like to know if this is something others would find useful. Ideas for videos include testing of the Raspberry Pi 3 and Retropie's emulators, Retroarc's emulators on the 3DS, PSP, and Vita, Adrenaline's PSP compatibility, and any other requests people might make. The ultimate goal of the show would be to collect, process, and report the data on input lags in various settings in a way that is succinct and entertaining while being easy to compare quantitatively between various methods. Would anyone like to watch something like that?
 

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,321
Country
United Kingdom
Is input lag a great problem in the modern world? It would be more compelling if it was. From where I sit it is largely a solved issue.

How are you planning to test it? If you are just going to flick a camera to 240fps and get it and a screen (preferably a low latency one) in the screen at the same time before going framewise then peh. If you are going to have fun with an oscilloscope, fly wires from controllers to thing,, and hack software things to display an output state then I will watch the setup part at least.
 
D

Deleted User

Guest
So, this is a resource that I would love to have had as a gamer who uses different emulators on different devices, but I'd like to know if this is something others would find useful. Ideas for videos include testing of the Raspberry Pi 3 and Retropie's emulators
RetroPie is on Raspberry Pi.
If you would make a interesting project, then i would watch because RPi is mostly used in various projects.
Personally, i would still watch you without that input lag nonsense. You could make 2-3 videos about it but in a long term it will fail.
 
Last edited by ,

Meteor7

Guess where this thumb goes.
OP
Member
Joined
Jun 9, 2014
Messages
1,336
Trophies
1
Location
a fit of spasms and accidental black magic
XP
4,643
Country
United States
From where I sit it is largely a solved issue.
That's actually one of the reasons I wanted to do the project. How much input lag would you guess Retropie's PSX and NES emulators have on average? Would you describe the NES and SNES emulators on the Wii as perfectly responsive? I think there's a lot of room for improvement that people either can't feel or don't know they've been feeling, and my hunch is that reporting the actual numbers would surprise them.
How are you planning to test it? If you are just going to flick a camera to 240fps and get it and a screen (preferably a low latency one) in the screen at the same time before going framewise then peh. If you are going to have fun with an oscilloscope, fly wires from controllers to thing,, and hack software things to display an output state then I will watch the setup part at least.
*shifts nervously in seat* ...Maybe a text-based report would be better. A video with voice is generally more easy to find and digest for most people, but I guess it really isn't going to add informational quality to the project, so there might not be enough to justify that kind of presentation. At the same time, though, I wouldn't want the data to get overlooked because of TL;DR.
 

Tom Bombadildo

Dick, With Balls
Member
Joined
Jul 11, 2009
Messages
14,575
Trophies
2
Age
29
Location
I forgot
Website
POCKET.LIKEITS
XP
19,226
Country
United States
I think what FAST means is how exactly you're going to gauge input lag between the various platforms? Cuz that's what I'm curious about, too. It's a bit hard to really accurately figure out how much input lag something has on it's own when there are a lot of variable factors that can vastly change your results. Things like just a shit TV, or a poor quality controller, or even software issues if it's an emulator and such like that.
 
  • Like
Reactions: Subtle Demise

Meteor7

Guess where this thumb goes.
OP
Member
Joined
Jun 9, 2014
Messages
1,336
Trophies
1
Location
a fit of spasms and accidental black magic
XP
4,643
Country
United States
I think what FAST means is how exactly you're going to gauge input lag between the various platforms? Cuz that's what I'm curious about, too. It's a bit hard to really accurately figure out how much input lag something has on it's own when there are a lot of variable factors that can vastly change your results. Things like just a shit TV, or a poor quality controller, or even software issues if it's an emulator and such like that.
That's the sort of beauty in this particular type of experiment. All the extraneous variables like controllers, TVs, etc. are compatible in many different combinations and can be swapped out and tested in different settings to weed out their influence. For example, when I was measuring on Retropie's PSX emulator, I used an official PS3 controller both wired and using bluetooth, an unofficial PS3 controller in both ways, and an 8bitdo NES pro controller to try and get a good profile independent of the controller. On its own, I feel like this would be a moderately effective way to try and identify the controller's influence, but then I was also able to test the controllers' input lags with an official PS3 and a CFW PS3 running the same game on the same TV with the same controllers. Because of how much cross-compatibility the hardware used with gaming has, I feel like weeding out those variables by swapping out TV's, controllers, cables, and the like is much easier here than with an average experiment.
 
Last edited by Meteor7,

Tom Bombadildo

Dick, With Balls
Member
Joined
Jul 11, 2009
Messages
14,575
Trophies
2
Age
29
Location
I forgot
Website
POCKET.LIKEITS
XP
19,226
Country
United States
Right, but you're still not really answering how you're getting your measurements, which is what I'm curious about :P Are you just magically counting "1 2 3 4 5 microseconds!" or are you recording things at a high framerate and then counting the frames, are you using actual instruments like an oscilloscope (as FAST mentioned) to get the absolute numbers, how are you getting your results?
 
  • Like
Reactions: RHOPKINS13

gman666

Cubicle Expert
Member
Joined
Jun 4, 2011
Messages
381
Trophies
0
Location
a cubicle
XP
1,001
Country
United States
Hello all. I've been recently thinking about starting a video series on YouTube where different methods of playing games across different forms of hardware and software are tested in-depth
I think this would be a lot more interesting.. With input lag it would be a niche viewership and you would be limiting your potential. Input lag could still be incorporated but don't make it your only thing.
 
  • Like
Reactions: Deleted User

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,321
Country
United Kingdom
Actually have not used most of the emulators mentioned, we mainly did streets of rage on the wii and before that I had an original xbox with all it had. I can't rule out my powers of prediction and muscle memory being unknowingly good, however I did try a gamecube with donkey konga on an awful TV (the people that had it believed the people at the shop were there to help them select a good TV...) and did abysmally at it compared to other places so I doubt that.

On responsive then have you considered the later parts of https://arstechnica.com/gaming/2011...-3ghz-quest-to-build-a-perfect-snes-emulator/ ? I am sure you will probably have some more elementary tests that are not so timing sensitive but it is something else to ponder.

Regarding the testing setup if you think a little montage with an oscilloscope and wires going everywhere is not compelling footage then I may be further gone than I thought... though going round all the machinist channels on youtube I see comments all the time of "I am not but it was cool watching". I was not saying film everything from five angles and give raw data, you can if you want but fading it a bit and overlaying things would be how I would go for general interest purposes.

I was once watching a Ben Heck video on some controller mod he did (might have been a controller for a disabled guy) and he made an offhand comment about how he mods controllers like that for latency testing for devs of games. Being devs though they could have real time memory info, change sprites to given colours, display things on the screen at the same time. From there they could more accurately calculate input latency and go from there.
If your plan however was to turn a controller side on, get it on screen with a character in a game, set your camera to silly fast record speed and say "the button is fully depressed here and ** frames later (which is ?? milliseconds) the response happens, rinse and repeat for other devices, possibly also find an old CRT and compare it to panels of today" then that is of limited value to me, and... there is a reason it is pretty much only digital foundry that you will be reading about the sorts of things they do on there and why other people tend towards "it feels responsive" in a review.
If you are going to make a thinking man's turbo controller/cheat controller, fire it with a microcontroller and measure it all with an oscilloscope, or something that would yield that kind of accuracy then we can talk more.
 
  • Like
Reactions: Subtle Demise

Meteor7

Guess where this thumb goes.
OP
Member
Joined
Jun 9, 2014
Messages
1,336
Trophies
1
Location
a fit of spasms and accidental black magic
XP
4,643
Country
United States
Right, but you're still not really answering how you're getting your measurements, which is what I'm curious about :P Are you just magically counting "1 2 3 4 5 microseconds!" or are you recording things at a high framerate and then counting the frames, are you using actual instruments like an oscilloscope (as FAST mentioned) to get the absolute numbers, how are you getting your results?
Oh, right. I've been using a Nikon D5300 camera at 1080p 60fps pointed at a screen, just like FAST implied (only at a much lower framerate). Because the resolution of the camera's measurements is 1 frame (obviously), the measurements have the standard +<1 frame as potential error from the measurement. This manifests itself as some ambiguity as to where "contact" actually happens. I've made a rule of not counting the contact frame at where the noise occurs (where there is typically a blur between "on" and "off" the button) and instead where I can see solid contact (which is always the frame after). This could potentially mean my measurements are consistently as much as, but not up to, one frame (out of 60fps) too low, but without better fps, I wouldn't be able to resolve the measurement further. Considering how the measurements will only be impactful to the reader if they come out higher than they expected, I've decided to keep erring on the low side as to make "high" measurements all the more reliable.

The reason I would consider such a rough measurement as useful is because of just how many frames I actually record of input lag on average. Even something as innocuous as the PSP's native PSX support can produce an average of 8 frames of input lag using this recording method, which again, errs on the side of being one frame too low. The lowest measurement I've ever recorded was 3 frames from a model ags-101 GBASP, which gives a non-trivial potential error of 33% on the measurements, but this is the only example where I'd call a higher fps for more accurate measurements "necessary".
 
Last edited by Meteor7,

The Real Jdbye

*is birb*
Member
Joined
Mar 17, 2010
Messages
23,293
Trophies
4
Location
Space
XP
13,850
Country
Norway
Oh, right. I've been using a Nikon D5300 camera at 1080p 60fps pointed at a screen, just like FAST implied (only at a much lower framerate). Because the resolution of the camera's measurements is 1 frame (obviously), the measurements have the standard +<1 frame as potential error from the measurement. This manifests itself as some ambiguity as to where "contact" actually happens. I've made a rule of not counting the contact frame at where the noise occurs (where there is typically a blur between "on" and "off" the button) and instead where I can see solid contact (which is always the frame after). This could potentially mean my measurements are consistently as much as, but not up to, one frame (out of 60fps) too low, but without better fps, I wouldn't be able to resolve the measurement further. Considering how the measurements will only be impactful to the reader if they come out higher than they expected, I've decided to keep erring on the low side as to make "high" measurements all the more reliable.

The reason I would consider such a rough measurement as useful is because of just how many frames I actually record of input lag on average. Even something as innocuous as the PSP's native PSX support can produce an average of 8 frames of input lag using this recording method, which again, errs on the side of being one frame too low. The lowest measurement I've ever recorded was 3 frames from a model ags-101 GBASP, which gives a non-trivial potential error of 33% on the measurements, but this is the only example where I'd call a higher fps for more accurate measurements "necessary".
One frame at 60FPS is a whole 16.67ms, which is definitely enough to screw up your accuracy in certain games. You'll want a better camera if you want to measure input lag accurately. I think ones that can do 200 fps are pretty cheap these days. Plus, you can have a lot of fun with a high speed camera. :P
 

Tom Bombadildo

Dick, With Balls
Member
Joined
Jul 11, 2009
Messages
14,575
Trophies
2
Age
29
Location
I forgot
Website
POCKET.LIKEITS
XP
19,226
Country
United States
Yeah, not sure how accurate those measurements will be if we're talking ms (which you should be for input lag).

Neat idea though, if you could get some better equipment to test I'd watch some comparisons.
 

Meteor7

Guess where this thumb goes.
OP
Member
Joined
Jun 9, 2014
Messages
1,336
Trophies
1
Location
a fit of spasms and accidental black magic
XP
4,643
Country
United States
If you are going to make a thinking man's turbo controller/cheat controller, fire it with a microcontroller and measure it all with an oscilloscope, or something that would yield that kind of accuracy then we can talk more.
Got to admit, this part went right over my head. I'm not even sure what a microcontroller is. Do you mean have the oscilloscope measuring the phase difference of the signal at the start of the controller and at the end of the cable? Probably not, as that would only measure the time it takes for the signal to get from the controller to the console, but it's the only experience where I remember using an oscilloscope to measure delay.

--------------------- MERGED ---------------------------

Yeah, not sure how accurate those measurements will be if we're talking ms (which you should be for input lag).
Even when talking about lags as high as 133.33ms? Besides 16.66ms is the maximum potential error with this method. I think even if the true value were 116.66ms, that would still be significant. That's not to say that I'm against a higher fps camera, of course. :P I'm also interested in soldering an LED to the contacts of a good test controller to have a more accurate reading of the "contact" frame.
 
Last edited by Meteor7,

Sonic Angel Knight

Well-Known Member
Member
Joined
May 27, 2016
Messages
14,403
Trophies
1
Location
New York
XP
12,946
Country
United States
I know that some emulation tools people have managed to determine if there is input lag in some games, that is totally odd. Some how someone managed to find out why the cheat code for debug mode in sonic 3 (When the sega logo disappears and sonic starts to appear on the title screen, press up up down down up up up up) Someone said there is random lag that the game accepts inputs only on some frames during the process which is why years ago it was so difficult and no one knew why, when they tried it they felt if they didn't get it, it was a lie and rumor.

There is some games with actual lag that i felt need addressing rather than the common games we expect to just function like normal, also are there any plans to discuss ports of games if there any difference in lag, especially since new consoles keep coming out and moved from wired controls to wireless either wifi or bluetooth, there also people who do that i guess, though i can't really back up their believe if they do notice a difference and why. Like a original nes and the NES classic if some games are difficult cause timing is off. Usually i just play games, adapt and get better. :P
 

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,321
Country
United Kingdom
"thinking man's turbo controller/cheat controller"

As nobody likes pressing switches to test a board when you have 20000 of the things then most controllers have test pads or other points you can put a signal into and thus test inputs. Hackers and people that make all those auto reload and game specific controller mods then also use these to input signals of their choosing. Turbo is just that and you input as many pulses as you need*, something with a bit more timing then takes an extra device to pull it off. If your chosen device lacks these pads you get to find somewhere else to solder to. Here is a website that covers a whole bunch of them for different systems
http://www.slagcoin.com/joystick/pcb_wiring.html
There are other interesting things you can do with this like split controllers and have two people operate one game, make foot pedals, for handhelds you can give them external controllers, if you need to sort something to help work around a disability you can do that, you can take the output of one game controller and feed it into another (an expensive way of making a controller adapter perhaps but it will work) and much more besides.

*modern game companies don't like this so you might want to mix it up and make it look slightly human rather than precise 200ms pulses every time if you fear getting banned.

Microcontrollers then. These are little chips you can program to do your bidding (send pulses, do maths, do logic on things like if this leg is high and so is this then do this). If you have ever heard of an arduino ( https://www.arduino.cc/ ) then that is a higher level one of them. As part of the basic introduction you will get it to send a signal to an onboard LED which in reality is just a leg on the chip, step two will be get it to light or flash a pattern when a button is pressed. For this controller testing you want it to output a signal when a button is pressed (or regularly as it does not really matter in the end)... it is that easy. In doing this you eliminate the guesswork of when the button hit bottom, if the button is sticky or if the switch bounced a bit, all of which help make the button and screen in frame tests less useful than they might be.

You wire up the scope probe to the output of your arduino so when the pulse happens it triggers capture on the scope. You need some means to tell when something has happened on the screen and this is probably the harder part. This second part will have another probe on it so you can measure the time delay.

Were I doing it with what I know I would probably change the sprite and use some kind of optical detector, I might also change the sprite in the game to be a black square, or I might change the whole screen to be black or something (or maybe all pink and get a pink filter). Sounds hard but have you ever wondered how duck hunt works, and also why shooting a lamp tricks it? There are any number of light dependent things you can use to take signals in, earlier today I watched a video in which someone took apart a air freshener that detects movement and it looked like that was something that did that. You might not need to mod games either if you can pick a suitable test (good contrast, obvious that something is happening and so forth).

As a halfway house and still quite useful you could find a quick light LED or something and trigger that at the same time as you fire the input pulse. With a camera on the screen and seeing the LED light you can get better timings than simply using a normal button.

If you want to do something fancy then you can also put a signal into the controller and tell how long it takes to spit something out the other side. On the face of it I would agree it is purely for the cool, however it would also allow you to tell what component of the lag is attributable to the controller and what the emulator/hardware and screen setup might contribute. If it is posing a problem (and there are all sorts of ways of checking inputs) don't get too hung up on it.

For a screen test then there are fancy devices that fancy screen testing sites use, I would say they are expensive but they even more than that. Most of us though just get a CRT screen (they are not without lag but it is so low that most consider it as good as it gets, not to mention one of the reasons many light gun games do not work on modern screens) and a LCD screen and wire them up to the same graphics card to display the same thing. You then film it as it goes through a (fast) sequence like counting up through numbers and then you can literally read off the lag by picking any frame once the test has started.
 

Meteor7

Guess where this thumb goes.
OP
Member
Joined
Jun 9, 2014
Messages
1,336
Trophies
1
Location
a fit of spasms and accidental black magic
XP
4,643
Country
United States
Microcontrollers then. These are little chips you can program to do your bidding (send pulses, do maths, do logic on things like if this leg is high and so is this then do this). If you have ever heard of an arduino ( https://www.arduino.cc/ ) then that is a higher level one of them. As part of the basic introduction you will get it to send a signal to an onboard LED which in reality is just a leg on the chip, step two will be get it to light or flash a pattern when a button is pressed. For this controller testing you want it to output a signal when a button is pressed (or regularly as it does not really matter in the end)... it is that easy. In doing this you eliminate the guesswork of when the button hit bottom, if the button is sticky or if the switch bounced a bit, all of which help make the button and screen in frame tests less useful than they might be.

You wire up the scope probe to the output of your arduino so when the pulse happens it triggers capture on the scope. You need some means to tell when something has happened on the screen and this is probably the harder part. This second part will have another probe on it so you can measure the time delay.
I love that idea, but I nave neither an oscilloscope nor confidence in my electrical knowledge enough to go through with that project. If I were, though, I'd probably stick another LED between the controller and the chip just in case I can measure the delay between the chip receiving the signal and putting out the pattern.

Were I doing it with what I know I would probably change the sprite and use some kind of optical detector, I might also change the sprite in the game to be a black square, or I might change the whole screen to be black or something (or maybe all pink and get a pink filter). Sounds hard but have you ever wondered how duck hunt works, and also why shooting a lamp tricks it? There are any number of light dependent things you can use to take signals in, earlier today I watched a video in which someone took apart a air freshener that detects movement and it looked like that was something that did that. You might not need to mod games either if you can pick a suitable test (good contrast, obvious that something is happening and so forth).
I had never thought of that! At first I thought simplifying graphical elements might affect how quickly systems and emulators process the game and spit out the image, but if I could just modify something like a walking sprite to be a large black square so that it only appeared when the character began to move and use a CRT, you might be able to make a little light-tight enclosure around the sensor and stick it to the screen. Then the chip could put out a signal to the oscilloscope whenever the sensor stops detecting light. Unfortunately, my only experience with sprite editing is when using tools made by others explicitly for that purpose. On the subject of picking a suitable test, 2D games always have definite sprites used when animating the character, so I usually just tap a button and watch for when the sprite switches over, but for 3D games, menus usually have enough of a binary element to them that I can measure states from those.

As a halfway house and still quite useful you could find a quick light LED or something and trigger that at the same time as you fire the input pulse. With a camera on the screen and seeing the LED light you can get better timings than simply using a normal button.
This is probably the only thing in my wheelhouse as of right now. The only problem would be when doing tests on handhelds, as I don't want to LED-mod every single one of my consoles.

For a screen test then there are fancy devices that fancy screen testing sites use, I would say they are expensive but they even more than that. Most of us though just get a CRT screen (they are not without lag but it is so low that most consider it as good as it gets, not to mention one of the reasons many light gun games do not work on modern screens) and a LCD screen and wire them up to the same graphics card to display the same thing. You then film it as it goes through a (fast) sequence like counting up through numbers and then you can literally read off the lag by picking any frame once the test has started.
I intended to use a CRT whenever possible, just for how reliable they are in terms of input lag. I was also going to report the make and model of any screen I used. I was going to focus less on testing the actual screens for lag as there seems to be a pretty thorough repository of numbers for screen lags already, and since it's generally easy to optimize a screen for gaming, I'd assume people already knew just how much lag their display was/was not giving them.

These are all really great ideas, though, so thank you. I don't have the means to achieve the kind of precision that requires an oscilloscope, and frankly I don't have the confidence that I won't accidentally fry it if I were to buy one (and they're pretty expensive). It kind of makes me wish I was still in school, though, as we had access to arduinos, oscilloscopes, and light sensors all in the lab. Too little too late, I guess. Even still, I have the feeling that even rough measurements could be relevant if the measured lag is high enough.
 
Last edited by Meteor7,

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,321
Country
United Kingdom
If you go looking then for not a lot ($30 maybe, ebay has them for $35 buy it now http://www.ebay.com/itm/Philips-PM-...ual-Channel-Analog-Oscilloscope-/112374585596 ) you can get an older analogue scope, find someone selling one and tell them what it is for and you might be able to do better still (anybody that has one probably knows what it is and if it is going to someone that will use it rather than just flip it then they generally like that idea). I lucked out and got a nice one (still CRT though) with memory and everything for not a lot but if you have a camera then you can record the scope and tell that way if you lack memory options.
You can indeed blow up a scope but you won't if you make sure you don't try and measure something silly or do something silly like try to measure relative voltages for an earthed device.


For the arduino stuff I was not kidding when I said the first tutorial is blinking a light, aka sending a timed signal to a pin
http://www.ladyada.net/learn/arduino/lesson1.html
Should you have to wire up all 10 buttons or something for the average controller to make an automatic playing machine then it might be cause for a little slowdown to consider, if it is just one button you can hopefully figure something out and maybe even without solder.

On ROM hacking then yeah it could be tricky, however people have come before you and written things down
http://datacrystal.romhacking.net/wiki/Category:Games
If you wanted to hack some random anime game from the 80s (happens all the time we get people asking that sort of thing) then yeah you would probably be out of luck, here though you should be able to find a game suitable for task, or smack one hard enough that it becomes suitable, find the location it says the sprites are at (in that you might have to click ROM map on the right hand side) and overwrite things to do what needs doing.

Also yeah I miss school labs. Some of the things they had all throughout my educational meanderings I have since found out cost much more than I thought they might and I miss having access to such things.
 
  • Like
Reactions: Meteor7

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
    Xdqwerty @ Xdqwerty: @salazarcosplay, I heard herbert stopped appearing on the show