What is a good method for upscaling emulated game videos?

GAPMan

Well-Known Member
OP
Newcomer
Joined
Sep 8, 2021
Messages
60
Trophies
0
Age
36
XP
255
Country
United States
I mostly deal with emulated games specifically games from consoles like Super Nintendo and Sega Genesis. While I had received feedback and advice from various places over the years, I am still mostly confused and a little afraid. I am looking for a method that will allow me to upscale games from various consoles especially consoles from the Fifth Generation (PlayStation, Sega Saturn, Nintendo 64, etc). I do acknowledge not every method is ideal for every game as not every game is pre rendered or in 3D on those consoles. I am just looking for methods that allow me to accommodate for those games especially games like:













I will admit that these might not be the greatest examples but there are so many games that require specific methods of upscaling for sites YouTube, archive.org etc. that I am afraid of getting artifacts. Can anyone help me upscale these games? Or at least point me in the right directions so that I can upscale the games in a high quality and accuracy? Should I use neural networks? Or should I use Non Linear Editors? Or linear editors?
 

Sonic Angel Knight

Well-Known Member
Member
Joined
May 27, 2016
Messages
14,397
Trophies
1
Location
New York
XP
12,921
Country
United States
Hopefully I can help.

For 3D games, some emulators have a internal resolution upscale feature to improve the polygons and textures enough to look cleaner and "HD." The games that include FMV won't be effected by this feature, so using bicubic filter would help with that. It blends the pixels in a way that supposed to be slightly advanced compared to bilinear filter which most common video players do including on the internet like Youtube, Twitch, Discord, etc.

For 2D games (The ones using only pixel graphics) use nearest neighbor. It helps preserve the sharp pixel look when upscaled to higher resolution. Also make sure to use interger scaling. That means which ever the horizontal and vertical resolution is, use multiples of it. An example is Sega Megadrive, which has 320 horizontal resolution (or 256 sometimes both depending on the game) and 224 vertical resolution for NTSC and 240 for PAL. Multiplying both by 5 would be
320x224 = 1600x1120
256x240 = 1280x1200

Most people say to do 5X interger scale, for 1080P video, but that would result in an overscan issue where the top and bottom would need to be cropped due to the vertical resolution being higher than 1080 for most systems. (NTSC consoles used 224 vertical resolution while PAL used 240) 4X would be able to fit the whole video into 1080 with underscan but most video players (including all the ones mentioned previously) will just force everything to fit inside the box with no cropping. It also upscales any video using bilinear filter which hides sharp piexls.

Two options exist for such situations. If using interger scale ends up with overscan, you may have to crop top and bottom. This usually is safe to do as some of the important game information would not be lost. Such as HUD elements. But that may depend on the game or if it bothers you enough that you don't like it. The other option would be to underscan the upscale but add a frame to fill out the video to 1080. This way there will be a border around the video but nothing cropped. Of course I don't think anyone does that on youtube but perhaps it doesn't bother them as much.

Lastly, there is some games that just will use different resolutions. Normally consoles like NES will only have one resolution to use, later consoles like SNES and Sega megadrive will have more than just one. Snes resolution can use 256x224 (240 in pal) and 512x448 (480 in pal) and as mentioned already, Megadrive has 256 and 320 widescreen. The games on the console can use any of these, and even swap between within the same game. Secret of Mana for example uses higher res mode in menus but low res during gameplay. Playstation is notorious for stuff like this which makes game capture using capture cards with those games problematic without the right setup. If you're using an emulator, you should be able to force one resolution at all times. Otherwise get ready for more work. :ninja:
 
  • Like
Reactions: Gep_Etto

danny19901

Well-Known Member
Member
Joined
Feb 12, 2016
Messages
633
Trophies
0
Age
32
XP
1,569
Country
1st thing you could do is capture gameplay from a higher resolution of emulation many Emulators enhance graphics based on resolution specially with Duckstation, PPSSPP, Redream, Project64, Retroarch and so on

2nd You could also use an AI Upscaler to enhance the detail of the videos stuff like Cupscale, Waifu2x or GigaPixel

If you need anymore advice or suggestions let me know I will be happy to help

Sent from my M2011K2G using Tapatalk
 

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,311
Country
United Kingdom
Linear editing and non linear editing is a different matter entirely -- it refers to the underlying ethos in the UI/approach to editing rather than anything at all to do with scaling. Equally the vast majority of things you will be looking at for editing these days will be non linear/nle approaches and you will have to go out of your way to find linear stuff.

There are multiple approaches, all with their own upsides and downsides that vary with what you are doing. If you are going for accuracy then that limits you in many ways, if you want the shiniest video then that allows other things. If you fancy using particularly fancy emulators and methods thereof you have other means again. I will also note that what people want to see varies as well, and the idea that people don't much care what they see but what they hear so if it sounds like you are recording down the bottom of a well you can have 4k native HDR footage version on a supercomputer and nobody will want to watch it/engage with it.

We also have *spits* interlacing to contemplate here but I will put that off for a minute. If you can use a progressive video mode then in 95% of cases (there are some ones where interlaced might technically be a higher resolution or notable frame rate boost).

Accuracy wise then you have one choice. It is called nearest neighbour (though different editors can call it different things) and will require you to be a multiple of 2 increase to your base resolution. Fortunately most things are multiples of two away from the vertical resolution at least.
How you fill the borders is up to you. Some will go with basic black, some will have a banner, some will split the video as it is there, drop the saturation (or indeed overlay over a single colour), blur it a bit and call that a fancy professional effect. I am not overly fond of the latter but I will not dismiss it either.

After this it gets fun. Many more approaches here
1) If the emulator will dump your layers into separate video streams then that can be a thing, even if only to scale the UI elements (often still 2d affairs) separately to the background 3d elements (which might benefit from a slightly softened approach or filters added that will make the 2d UI look horrible).
2) As noted above some emulators will virtually increase their 3d resolution (polygons is polygons and rendered internally which means you can increase things there, maybe even do a subsurf) and/or do widescreen as part of that, and possibly even do texture replacement on top of that (though this will be artistic).
2d can't be as easily scaled outside of specific use cases -- the 2d world is probably what you are seeing on the screen with nothing beyond that compared to 3d where the position of most things is known. There are some things that try it on (usually variations on the theme of take savestate, move player all different directions, take shots of each, stitch back together and give back to player to continue on, works well until you encounter a wall, a NPC spawn, random battle or similar, some might incorporate a walk through walls cheat as well or similar) 2d however has long had filters -- sai, super said, eagle and all that other jazz.
2d tile replacement is a thing however.
There are some game specific filters designed for use with one game -- a lot of NES and SNES mario things will be this where filters are twisted to work well with that and maybe not others.
3) Many games these days have ports out on other systems. Can be worth looking here, and might even be available in hacked form on your base console to do something nice with (or indeed might be a translation patch if say the Japanese version was the best for some reason -- usually PAL got shafted or needed some extra hacks to do things, though not always and sometimes it had better bug fixes).
4) Conventional scaling approaches, which would include some of the more exotic ones.
Your basic filters are bilinear and lanczos, and will probably be good for most things. Interlaced video can have some perks for itself here as you are scaling between two known points ( http://avisynth.nl/index.php/Nnedi3 for an older take, and arguably bringing us into the smart scaling world).
Truly smart scalers I rarely like to use for video stuff -- the seam carving approaches work fine for static images but trying to constrain them for video is much harder. To that end you have some weaker ones, ones that may focus on edge detection, movement detection and adjust accordingly to try to keep things sharper. For game footage it is harder than some live action or animated stuff. Rendering will probably also not be anything like real time where nearest neighbour for 2x video size is take pixel, spread it to three sides to make 4x4 "pixel", repeat for all other pixels in image, and can be done on a calculator. Not so bad if you are doing a 40 minute cut up video but trying to go fancy for a 16 hour lets play, or livestream, gets harder.


On top of this we also have ideas of CRT emulation
http://bogost.com/games/a_television_simulator/
To say nothing of trying to fix problems with games themselves -- see all those early GBA games that were forcibly brightened to work with the unlit screen of the original GBA.
 
  • Like
Reactions: Gep_Etto

GAPMan

Well-Known Member
OP
Newcomer
Joined
Sep 8, 2021
Messages
60
Trophies
0
Age
36
XP
255
Country
United States
1st thing you could do is capture gameplay from a higher resolution of emulation many Emulators enhance graphics based on resolution specially with Duckstation, PPSSPP, Redream, Project64, Retroarch and so on

2nd You could also use an AI Upscaler to enhance the detail of the videos stuff like Cupscale, Waifu2x or GigaPixel

If you need anymore advice or suggestions let me know I will be happy to help

Sent from my M2011K2G using Tapatalk

I had been using a method recommended to me where essentially do two resizes in Virtualdub2. The first resize is Nearest Neighbor with Same as source with relative percentage being mostly even (400,600,800). The other resize is Lanzcos3 where you select Compute height from ratio as to correct the aspect ratio. This method helped me out for the most part but I think it might not work for all games. What do you think of this method? If this is not a good method to use, what alerantives do you offer? I also have HitFilm Express, MeGUI, PowerDirector 365, Handbrake, Filmora and Shotcut. Also, what AI Upscaler do you suggest? FFMPEG or AVISynth?
 

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,311
Country
United Kingdom
I had been using a method recommended to me where essentially do two resizes in Virtualdub2. The first resize is Nearest Neighbor with Same as source with relative percentage being mostly even (400,600,800). The other resize is Lanzcos3 where you select Compute height from ratio as to correct the aspect ratio. This method helped me out for the most part but I think it might not work for all games. What do you think of this method? If this is not a good method to use, what alerantives do you offer? I also have HitFilm Express, MeGUI, PowerDirector 365, Handbrake, Filmora and Shotcut. Also, what AI Upscaler do you suggest? FFMPEG or AVISynth?
So you chain two resizes together? Nearest neighbour/point resize to get it most of the way there (the nearest neighbour is only ever going to be a multiple of two different to the source) and then a lanczos (one of the more effective of the basic scaling methods, possibly best for avoiding the mild blurring that tends to be associated with such things and at the same time takes some of the harsher edges off from pixel methods).
It will be a method to get it to a full size modern video (give or take aspect ratio concerns but I mentioned either borders, overlay on a background or the fancy expand the current video and do washed out version of that as a background image that many fancier channels seem to go in for) for things that prefer a 1080p or whatever source. The lanczos will give you some softening but nearest neighbour first will minimise that. A reasonable compromise from where I sit.

FFMPEG and AVISynth are not upscalers in and of themselves. They might have some built in, and AVISynth is traditionally where I used to find the fanciest external filters (often years before adobe et al came out with prototypes for their equivalents). I don't know if they will share something of a codebase currently but unless there is a bug you will probably get the same from their inbuilt options as the algorithms are general concepts.

Depending upon what level of editing you are doing I might actually suggest you build a script with avisynth as for the most part same console will produce same output, or series of outputs* that will save you some clicks. This can mean you have a premade script for redoing videos. I like avisynth for editing in general ( https://gbatemp.net/threads/be-a-great-video-maker-and-replicate-this-video-effect.360509/ ) but if you either used avisynth as in its traditional frameserver role and have the other video editor you like grab the avisynth file which it just thinks is a video or just fed it to a lossless encoder if that is going to be tricky (don't know what goes for some of the modern encoders and how well they handle frameserver so eh, if you have the hard drive space then OK but again 16 hours of let's play is a different matter. Megui was my go to for use with avisynth and it uses it underneath it all to do things anyway).

*script will probably look something like

Code:
a=directshowsource("yourvideo.mp4")
a.pointresize(Width*2,Height*2).LanczosResize(somevalue,1080)
plus any convert to different colour space if needed ("converttoyv12()" maybe, http://avisynth.nl/index.php/Internal_filters#Color_conversion_and_adjustment_filters has them), plus sort any interlacing issues if you face those (depending upon how accurate your emulator is you may get that, and if you capture from a real device one day you almost certainly will).
somevalue can probably be calculated if you really wanted ( http://avisynth.nl/index.php/Clip_properties ) as it should be mathematically related (1080/originalheight=ratio and number to multiply to, should be a reasonable number rather than a crazy irrational one but who knows what you get to do as far as rounding) but I would probably just set it to the best one manually, and rather than 2 you could have 4 or 6 if it gets it somewhere closer to 1080 or whatever you are using. I would also do the blurred background, borders or overlay effect here as well (some examples in link earlier on replicating effects). If you have a webcam, timer (simple or something more complicated like I see the speedrun people use) or other image you want to overlay or overlay the video on then that would also be a good point for it.
From there I would probably do any trims, title cards and subtitles which lands you at 90% of the way for editing that most people do for their videos but I like avisynth as previously mentioned. Also by definition your project is backed up so you can redo things at a later date if you still have the source files and decide you want to tweak the resize options.
 
  • Like
Reactions: Gep_Etto

GAPMan

Well-Known Member
OP
Newcomer
Joined
Sep 8, 2021
Messages
60
Trophies
0
Age
36
XP
255
Country
United States
This is long read but I will give your method a try. I will admit that I am new AVISynth but I am willing to try anything once.

Edit: In regards to "copying this effect", do just put the whole video in AVIsynth or just a few clips?
 
Last edited by GAPMan,

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,311
Country
United Kingdom
Sorry. Missed the edit.
Depends what you are doing - some of those were logo replications (don't think I have the GBA boot logo sendup there, though it is in the public aspects of the forum somewhere) which you would probably be better off rendering out as lossless and importing anew, though after the system grinder of some of those it should settle down into basic rendering if you did go there.

I also note I made a mistake above. I kept saying multiple of two when I meant whole number multiple -- the turn one pixel into 4 works just as well when you go another step and turn it into 9, 16 is in the pattern so next 25... you can do the rest of the area of increasing squares. If that gets you to within a couple of hundred pixels (you might even try going over and scale back down) then that might be better to go for one of the more arbitrary resolution scaling options mentioned elsewhere.

As far as avisynth in general it can be quite a painful initial hurdle to get over where something like avidemux or kdenlive if sticking with open source fun you can have really fun effects going on by clicking on obvious UI elements a few times. I first came to it when converting videos -- convert a few DVDs that are basically whole as is (maybe a trim for the credits) and it is fine (might even do a few versions for different resolutions -- MPEG4 ASP aka xvid to play on the crappy DVD player, another for the PSP and a nice 720p version in MPEG4 AVC aka H264 for long term storage/watching on my PC with nice screen but also not gobbling up all my storage), move to a few MPEG2 transport streams of TV shows from various digital broadcasts and you can chop adverts out (few trim commands here and there, plus the resolutions thing from before), get said streams but segmented (if the transport stream ripper had some kind of advert detection they can stop the encoding there, encode what is done so far, pick up again when the show comes back and not have to encode 40 something minutes or more of TV show after it is all done and win the race to be first on the internet, at the cost of possibly some quality with the hard cuts and non continuous video) and that is a few appending videos (+ symbol) which is most of video editing done for most purposes, add some borders to get around my CRT at the time being awful (grey borders rather than black with no choice, cropped, stretched or that weird stretch where the outside of the screen is more stretched compared to the middle) and now we are off to the races. If you are going to try to jump into tightly edited video a la your favourite youtube video channel that is possibly paying a full time editor, title card artist, colour correction, studio camera operator, footage source creator/stock footage finder and has corporate backing then... yeah.
 

GAPMan

Well-Known Member
OP
Newcomer
Joined
Sep 8, 2021
Messages
60
Trophies
0
Age
36
XP
255
Country
United States
Funny, I assumed that you could put videos in programs like MeGUI and AVSmod to learn what edits were used. Apparently, I am wrong as I am not that astute to know that edits were used unless I use programs like HitFilm Express, Shotcut, PowerDirector 365 or DaVinci Resolve. I assumed that if I put a video like this:



That it will give the script was used to make that video. Looks like I was dead wrong and it looks I need to more about AVISynth as well as editing to get the result I need. This also applies to how I want o upscale and make sure that some fifth generation games presentable as those games are usually 3D or 2D. Is there more "copying" the effects of than what was posted?
 

Jayro

MediCat USB Dev
Developer
Joined
Jul 23, 2012
Messages
12,962
Trophies
4
Location
WA State
Website
ko-fi.com
XP
16,954
Country
United States
Always upscale with a 1:1 pixel ratio. Non-square pixels have no business being in front of my eyes, unless it's Sega GameGear. (Which stretched the pixels non-square from 160x144 to 4:3 mode)
 

GAPMan

Well-Known Member
OP
Newcomer
Joined
Sep 8, 2021
Messages
60
Trophies
0
Age
36
XP
255
Country
United States
Always upscale with a 1:1 pixel ratio. Non-square pixels have no business being in front of my eyes, unless it's Sega GameGear. (Which stretched the pixels non-square from 160x144 to 4:3 mode)

And that be done in any video editor? Or just ones like DaVinci Resolve?
 

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,311
Country
United Kingdom
Yeah you can no more tell what edits someone did than you can what previous content someone might have cut out from a document that they copy pasted from a word processor to an internet post.
You might be able to figure out what codecs and programs were used to make it, and I will skip video forensics (some of the more interesting stuff coming from emulator vs hardware or did they cut the best times for given levels in a whole game speedrun debates) as it is not going to be much help for you in this.
Some of those programs might have a project file that notes some of the things that were done (they are not always as complete as avisynth, or at least the undo history only gets you so far if you did want to go right back and change a source clip) but they are usually kept (or often not) by the original video creator and not shared.

That video is a TAS video so while someone might have gone manual they would also likely know what state the game was in (level, map, otherwise) to drop the second screen or shrink it depending and could note timings accordingly. That said would not be the hardest thing in the world to note frame numbers and chop and change video accordingly as it looks like you have three main choices for the video format in that (whole screen, side by side and three panels with the gameplay taking the bulk*)

*while it is just black in the example above the third in other speedrun videos maybe including a timer, or camera on the runner's face, or pulse monitor, for some TAS stuff input lists/internal logic as well...

If we are sticking with avisynth then the DS is something of a special case (most games are not multi screen, though there is some playstation stuff and PC stuff, usually for driving games or space sims) compared to most where you are more concerned with resolution and adding borders. If I was to do it in Avisynth... there are two approaches.
1) You do something with filters and overlay (I recall doing something similar for the thread I linked earlier) and with the list of time codes you get from scanning through the video for the right frame numbers (assuming your TAS did not give them) you could probably generate a nice long list of things likely along the lines of a.fullfilter.trim(blah,blah2)+a.levelfilter.trim(blah3,blah4)+a.mapfilter.trim(blah5,blah6) where a is your video, the blahs are your frame counts and the various things named filter are your filters that scale and stack (see below)
This is probably better for doing lots of videos of a similar style as you can reuse things at the cost of a bit of upfront work. You can still reuse lots of below as well but it would be a bit more fiddly.
2) You get introduced to stackvertical and stackhorizontal, maybe generate a blank clip with black to act as the filler, calculate what sizes things need to be (we already covered scaling methods) and adjust accordingly such that the output is 1080p or whatever you want there. Modern video being widescreen where the DS was not makes things a bit easier as you have some free real estate to play with but in the end it is all just multiplication/basic maths, or if you are a heretic (don't letterbox things) then also some cropping.
I would probably make three virtual video clips being the three styles seen there (full screen for play screen, side by side for map section and the shrunken play with black box and map screen) and cut between them with my lists of times (in my case assuming the TAS setup did not provide things for me* I would probably grab them from virtualdub with avisynth providing the input as it is nice for frame grabbing purposes but use whatever you like that gives you frame numbers, including adding video info to the avisynth script and pausing/frame advancing on any player that can handle that).

*should be easy enough to have the lua most of those use to output to a text file when a given position in RAM corresponds to a given state in the game along with a time code/frame count of some form. However that is getting into programming TAS programs so I will leave that for now.
 

GAPMan

Well-Known Member
OP
Newcomer
Joined
Sep 8, 2021
Messages
60
Trophies
0
Age
36
XP
255
Country
United States
That was exactly my problem. I thought by putting in a document I could at figure what settings were used to make that video. I didn't want the whole video, just the crops and setting used to make the game presentable. Also, 3D games are not extatly like pixel art so I cannot just use PointResize either.
 

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,311
Country
United Kingdom
You can still figure things out.

Download it however you will (I like youtube-dl myself at time of writing. https://github.com/ytdl-org/youtube-dl and there is a GUI which is good for multiple things but I find setting that up annoying), you can probably even do screenshots if you wanted (might need full screen ones or further maths if it is a scaled image).
Measure it using whatever measuring tools your image editor has available (Gimp has them, even the humble area select in Windows Paint usually told you the size of the thing you are selecting). In this case you have three distinct styles of image, and depending upon how you want to view it then two (or three if we include the black segment) video sources (one for each screen, though some emulators might store it all as one, and some setups might include the hinge gap as some games did adjust accordingly to make animations carry on through the imaginary hinge area as though the screen was continuous.) and create settings accordingly. Cropping is probably going to be minimal (again letterboxing, the term for cropping a video to fit in a different aspect ratio, is considered a crime against video for most, and very dubious for games where even for a TAS you want your viewers to be able to see what is coming) so it is mostly going to be resizing, in this case for the large main gameplay stuff then whatever it takes to get the main gameplay screen to be full vertical res and then use whatever remains to house the map screen and black section (assuming it is not done as an overlay to a black background or giant black border added to the top of the map section to make height). Side by side is probably just going to be whatever size we can use to keep both the same size and within the width limit (for 1920x1080 aka 1080p that is 1920/2 and that forms the horizontal width, or max value thus you pick the next lower multiple if sticking with nearest neighbour)

As far as nearest neighbour/point resize with 3d (which NSMB DS technically is -- pull apart that game and you will find 3d models and animations, it is just projected from the side in the main game, 3d actually tending to be more visible in the map section) you can still use it, indeed accuracy if you are going for that demands it.
 

GAPMan

Well-Known Member
OP
Newcomer
Joined
Sep 8, 2021
Messages
60
Trophies
0
Age
36
XP
255
Country
United States
3D is truly a mystery to me but I will have to study AVIsynth more as you said.

Edit: Also, what do you when you encounter FMV video? I know that every game has FMV but it seems like animated FMVs needs to handled differently than FMV art.
 

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,311
Country
United Kingdom
You are over complicating things for yourself. Nearest neighbour methods are the only way of accurately upscaling video this side of crazy AI methods (and there are some basics available to you), basic scaling methods (bilinear/lanczos/those given in basic emulators) will blur things a bit but sometimes a bit of blur is nice for 3d to soften things a bit or indeed act as a crude form of anti aliasing so some like to use them, and as nearest neighbour has limits as far as needing to be a multiple to have a sensible output then some might seek to use another method to get the rest of the way to a modern resolution if adding borders/sticking as an overlay is not acceptable for their purposes. If you want to try some of the interpolation upscaling methods, and methods that try to be motion aware (this also brings up ideas of motion blur and framerate increases) you can but at that point it gets a bit questionable -- do it for your family videos, news reel, or fancy film by all means but if the footage is intended to even be vaguely representative of the game itself... questionable at best.
If the emulator can internally increase the resolution (3d space is 3d space, just because the original console says give me a 576i 4:3 image of this virtual world does not mean the emulator has to do that as the world/texture data is all there), or some replace the textures/sprites wholesale with external works to include manual recreations at high resolutions (artist created, art leaks or ported from a higher resolution port it really does not matter), or you can get a native version (4k Mario 64 is possible with that recent port for the same reasons the emulator stuff is possible, just a lot more efficient and also able to do things like increase draw distance trivially) then if your audience will accept that then some opt for those.

FMV in games is only really tricky in that you are probably double compressing video, and most game consoles don't use high quality codecs/standards/settings in their FMV which means your video in turn handles all the compression artifacts which makes it even less compressible*. Depending upon the system it might also be quite low resolution as well and technically drop down to a lower than the menus/gameplay mode for it, or maybe a lower framerate that is not a simple multiple**.

*if your FMV using basic motion jpeg (which stuck around to the Wii) or something patent dodging (they don't just not use avi xvid or mp4 H264 because they are silly) and equally dubious or designed for low power machines descends into a mess of macro blocks (if you have ever seen a heavily compressed video/scene, usually a high action one, turn into almost a tile puzzle this is that) or compressed JPG noise the encoding you do for your let's play does not know the macro blocks and noise are an error/compression artifact and dutifully has to encode them and their apparent radical visual change as though it was thing the original video author intended, which could in turn risk maxing out the available storage it has for that section of video if you have a max bitrate feature set in your encoder.

**might use say 20fps as that is just about when people stop seeing things as a slideshow and that saves a lot of data over 60fps (video being a space hog compared to most other things). For games and modern stuff you are probably going to be spared the pulldown issue that DVD people might face with CGI sections as the emulator/capture will likely just be dumping current video output (and audio and maybe button presses if that is a thing for your setup) every console level frame (mostly going to be 60fps but some newer things go higher, some things could be 30fps, and older PAL stuff could be 50fps and 25fps depending upon their setups as there was also 60Hz PAL at times as PAL TVs generally could do that)

Video because marginally relevant to some of the asides at least
 

The Real Jdbye

*is birb*
Member
Joined
Mar 17, 2010
Messages
23,249
Trophies
4
Location
Space
XP
13,802
Country
Norway
I mostly deal with emulated games specifically games from consoles like Super Nintendo and Sega Genesis. While I had received feedback and advice from various places over the years, I am still mostly confused and a little afraid. I am looking for a method that will allow me to upscale games from various consoles especially consoles from the Fifth Generation (PlayStation, Sega Saturn, Nintendo 64, etc). I do acknowledge not every method is ideal for every game as not every game is pre rendered or in 3D on those consoles. I am just looking for methods that allow me to accommodate for those games especially games like:













I will admit that these might not be the greatest examples but there are so many games that require specific methods of upscaling for sites YouTube, archive.org etc. that I am afraid of getting artifacts. Can anyone help me upscale these games? Or at least point me in the right directions so that I can upscale the games in a high quality and accuracy? Should I use neural networks? Or should I use Non Linear Editors? Or linear editors?

Some emulators and video plugins allow you to HQx upscale textures in 3D games with varying results. Most Project64/Mupen plugins allow this. Haven't seen any that do xBRZ upscaling though which would be slightly better but HQx is also pretty decent. Emulators for 2D systems also commonly allow upscaling with HQx and sometimes xBRZ (Kega Fusion, a Sega Genesis/Mega Drive emulator has xBRZ upscaling available as a plugin, in most other cases if the emulator doesn't have it already built in then you can't add it as 2D emulators usually don't support plugins but Kega Fusion is a notable exception here)
HQx/xBRZ upscaling on 2D games doesn't tend to work quite as well as it does on 3D games, it tends to do strange things to sharp single pixel width edges, like the borders around sprites. 3D games tend not to have a lot of sharp edges in their textures though so it works better for those in my experience. I can live with the imperfections and I tend to enable HQx or xBRZ whenever possible. But purists will hate me for saying this. It is very far from the original experience after all.
 

GAPMan

Well-Known Member
OP
Newcomer
Joined
Sep 8, 2021
Messages
60
Trophies
0
Age
36
XP
255
Country
United States
Some emulators and video plugins allow you to HQx upscale textures in 3D games with varying results. Most Project64/Mupen plugins allow this. Haven't seen any that do xBRZ upscaling though which would be slightly better but HQx is also pretty decent. Emulators for 2D systems also commonly allow upscaling with HQx and sometimes xBRZ (Kega Fusion, a Sega Genesis/Mega Drive emulator has xBRZ upscaling available as a plugin, in most other cases if the emulator doesn't have it already built in then you can't add it as 2D emulators usually don't support plugins but Kega Fusion is a notable exception here)
HQx/xBRZ upscaling on 2D games doesn't tend to work quite as well as it does on 3D games, it tends to do strange things to sharp single pixel width edges, like the borders around sprites. 3D games tend not to have a lot of sharp edges in their textures though so it works better for those in my experience. I can live with the imperfections and I tend to enable HQx or xBRZ whenever possible. But purists will hate me for saying this. It is very far from the original experience after all.

Funny I was always told the opposite as not to use those features when making videos of those games.

You are over complicating things for yourself. Nearest neighbour methods are the only way of accurately upscaling video this side of crazy AI methods (and there are some basics available to you), basic scaling methods (bilinear/lanczos/those given in basic emulators) will blur things a bit but sometimes a bit of blur is nice for 3d to soften things a bit or indeed act as a crude form of anti aliasing so some like to use them, and as nearest neighbour has limits as far as needing to be a multiple to have a sensible output then some might seek to use another method to get the rest of the way to a modern resolution if adding borders/sticking as an overlay is not acceptable for their purposes. If you want to try some of the interpolation upscaling methods, and methods that try to be motion aware (this also brings up ideas of motion blur and framerate increases) you can but at that point it gets a bit questionable -- do it for your family videos, news reel, or fancy film by all means but if the footage is intended to even be vaguely representative of the game itself... questionable at best.
If the emulator can internally increase the resolution (3d space is 3d space, just because the original console says give me a 576i 4:3 image of this virtual world does not mean the emulator has to do that as the world/texture data is all there), or some replace the textures/sprites wholesale with external works to include manual recreations at high resolutions (artist created, art leaks or ported from a higher resolution port it really does not matter), or you can get a native version (4k Mario 64 is possible with that recent port for the same reasons the emulator stuff is possible, just a lot more efficient and also able to do things like increase draw distance trivially) then if your audience will accept that then some opt for those.

FMV in games is only really tricky in that you are probably double compressing video, and most game consoles don't use high quality codecs/standards/settings in their FMV which means your video in turn handles all the compression artifacts which makes it even less compressible*. Depending upon the system it might also be quite low resolution as well and technically drop down to a lower than the menus/gameplay mode for it, or maybe a lower framerate that is not a simple multiple**.

*if your FMV using basic motion jpeg (which stuck around to the Wii) or something patent dodging (they don't just not use avi xvid or mp4 H264 because they are silly) and equally dubious or designed for low power machines descends into a mess of macro blocks (if you have ever seen a heavily compressed video/scene, usually a high action one, turn into almost a tile puzzle this is that) or compressed JPG noise the encoding you do for your let's play does not know the macro blocks and noise are an error/compression artifact and dutifully has to encode them and their apparent radical visual change as though it was thing the original video author intended, which could in turn risk maxing out the available storage it has for that section of video if you have a max bitrate feature set in your encoder.

**might use say 20fps as that is just about when people stop seeing things as a slideshow and that saves a lot of data over 60fps (video being a space hog compared to most other things). For games and modern stuff you are probably going to be spared the pulldown issue that DVD people might face with CGI sections as the emulator/capture will likely just be dumping current video output (and audio and maybe button presses if that is a thing for your setup) every console level frame (mostly going to be 60fps but some newer things go higher, some things could be 30fps, and older PAL stuff could be 50fps and 25fps depending upon their setups as there was also 60Hz PAL at times as PAL TVs generally could do that)

Video because marginally relevant to some of the asides at least


I guess you are right about that. The thing is, I am very fearful that I will get artifacts and it will mess up. I had no idea of stuff that goes into editing those videos especially those games that 2D animation in their FMVs. I guess I am making things to complicated but I wonder if I was better using PowerDirector or some other editor? If it really is that simple, I might need to reassess how I do things as I do not really have the best stuff. What would be a simple way to upscale games like Sonic CD or Mega man x4 to HD for example? Or games like FFVII for example?
 

FAST6191

Techromancer
Editorial Team
Joined
Nov 21, 2005
Messages
36,798
Trophies
3
XP
28,311
Country
United Kingdom
There will be artifacts, not much you can do about that. I would focus on making an entertaining and informative video instead, plus the audio thing mentioned earlier if you are going to talk on it as a notable part of it.

As far as choice of editor. I already mentioned my slippery slope into avisynth so might be that you are overthinking it there as well. Indeed most editing can be covered with cuts, title cards and maybe a subtitle somewhere. Scaling tends to be covered and I don't tend to find programs lack nearest neighbour as it is useful for editors to have to blow up footage to see things themselves. I fully expect the bigger aggravation being sourcing the footage (there is a reason many channels will source elsewhere, loop things over while they are talking or cut to static images/ken burns effect images), categorising it, contemplating creative choices and the like.

You don't need to go all

for a gameplay based video, be it a review, let's play, retrospective or whatever.


As far as upscaling itself.
Different consoles, regions and even within games if they are TV based can be different resolutions at different points (I did not resize anything for https://gbatemp.net/threads/found-t...-quite-different-to-na-pizza-hut-demo.556304/ ). Handhelds tend to be a bit more consistent here.
There is the problem of pixel aspect ratio but that is mostly going to be a gamegear thing (hold a magnifying glass over a screen, you will see the pixels, there are different arrangements possible that will in turn affect the effective aspect ratio).
Anyway so you know the resolution, and for this we are assuming no deinterlacing issues either.
You want the resolution to be 1080p say (720p and computer 4k are multiples of that as well so no great issues with the underlying concept).
Your choice then with an eye to quality is
nearest neighbour multiply to the vertical resolution whole numbers up.
If this multiple is not possible for whatever reason (maybe you have a full size PAL game without black borders added) you can either
Go next multiple lower than the 1080 vertical res, add borders/overlay on a background image/do the fancy desaturated effect covered earlier
Go lower than 1080 vertical res and do a more adaptable (but usually introduces some blur) to get it the rest of the way.
Go higher than 1080 vertical and do a more adaptable scale to scale it back down.
Go higher and crop to fit, aka letterboxing, aka a sin in video editing circles unless you are personally doing it as a cinematographer/director for an effect (close up on eyes or something).

If the horizontal does not fill (likely as modern widescreen aspect ratios is not most consoles) add a border there, overlay the video on an image, or do that effect covered elsewhere if you want to follow that trend.

We will skip the multiple layers and whatnot thing for now as it is probably not in play anyway.

Depending upon your video editor program this might mean you have to render out an intermediate stage if it will not allow multiple resize, however this is what avisynth is supposed to be for -- avisynth is supposed to fake being an AVI file as far as the program opening it is concerned so you can take your source, do the scale in that and then do whatever in the main video without it gobbling all your hard drive.
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
    Veho @ Veho: Apply snorgle to pinfor.