Hacking Fixing vWii Mode

[Clector]

Yes, there's nothing really wrong per say with sharpness on the Wii U vWii scaling on 720p and 1080p, only 480p. But the Wii fares well against things, I don't see the horrible messed up blurried mess that everyone says on Internet; but probably the output on a console affected by 480p video bug will be worse?
As for the less bleeding, I'm not sure, should be the Width I guess; the game doesn't use deflicker on progressive (thanks HAL!), but I think that shouldn't affect the chroma issue on Wii U?
For the colors, yes it is set to RGB, but if you set the YPbPr colors to Rec 601 the OSSC converts them correctly, there's the option to output on YCbCr thought.

 

[NoobletCheese]

I believe the colour encoding should be correct when Wii U is set to output HDMI 480p. The reason I didn't pick up on this originally is because my TV incorrectly decodes 480p with HD luma coefficients (it thinks 480p is HD). So I incorrectly thought the colours were still bad in 480p mode, but actually they are almost certainly not. Most TV's should decode 480p with the SD coefficients and colours should be accurate.

So the issue is to make vWii automatically switch to 480p, and I couldn't find a way to do this. I'm sure ULGX can force it to 480p, so perhaps an update to ULGX would do the trick.

That leaves the colour bleeding issue. If this could be solved, I imagine it would take place in the IOS code. In that case ULGX seems to be the likely candidate for forcing our custom IOS with the fix, assuming such a fix is possible.

 

[Nomado]

I believe the colour encoding should be correct when Wii U is set to output HDMI 480p. The reason I didn't pick up on this originally is because my TV incorrectly decodes 480p with HD luma coefficients (it thinks 480p is HD). So I incorrectly thought the colours were still bad in 480p mode, but actually they are almost certainly not. Most TV's should decode 480p with the SD coefficients and colours should be accurate.

So the issue is to make vWii automatically switch to 480p, and I couldn't find a way to do this. I'm sure ULGX can force it to 480p, so perhaps an update to ULGX would do the trick.

That leaves the colour bleeding issue. If this could be solved, I imagine it would take place in the IOS code. In that case ULGX seems to be the likely candidate for forcing our custom IOS with the fix, assuming such a fix is possible.

You might want to check hbl2hbc thread there is a forwarder that puts vWii to 480p automatically when running it. It can be configured to show several hbc to launch so that user can select which to launch. Gamepad can be turned off when launching or it can be gamepad only.

 

[Clector]

If someone is going to use the vWii, I recommend to use 720p or 1080p, 480p has the most overscan and is kind of softer than it should be, maybe because of the scaling.

 

[NoobletCheese]

If someone is going to use the vWii, I recommend to use 720p or 1080p, 480p has the most overscan and is kind of softer than it should be, maybe because of the scaling.

Yes vWii does have that global underscan downscaling (artificial black border around the edges) which would lower resolution beneath 480p for the 480p mode only. My TV doesn't allow disabling overscan for 480p so I can't confirm it though.

Another issue I recall is that Wii U needs to be set to 4:3 as well, otherwise 4:3 games get pillarboxed into 480p resulting in less horizontal resolution.

 

[Clector]

Here are some old captures, but they should show the overscan difference.
480p:

720p:

1080p:


As for 4:3, you can change the video settings with homebrew, esentially letting you run the vWii on settings that you normally can like 4:3 on 720p or 1080p for example; USBLoaderGX can also force a game to 4:3 or 16:9 if you need to.

 

[NoobletCheese]

Curious, on my system I'm seeing a black border around the edges in the 1080p mode with my TV's overscan disabled, which is not present in your screenshot. I've seen other comments reporting the black border as well, so I can't explain why your screenshot doesn't have it.

edit: perhaps it's because your screenshot is of a GameCube game.

 

[Clector]

Luigi's Mansion internal and external resolution is the same I think, that may be why there's no borders?

 

[NoobletCheese]

Luigi's Mansion internal and external resolution is the same I think, that may be why there's no borders?

Yes that's true there are many games which don't use the full vertical and horizontal resolution and have vertical and/or horizontal borders.

And if I recall correctly they do that for the same reason -- to prevent wasting precious pixels in the TV's overscan area.

But I still think that's a separate issue as for example the vWii system menu has the black border but Wii hardware does not.

 

[Clector]

The Wii has borders on the System Menu, so it's not the Wii U fault:

 

[Fesku]

I happened to find a fix to remove color-bleeding (at least in Gamecube games). The only thing you additionally need is: The Legend of Zelda: Skyward Sword.
I suspect that Zelda: Skyward Sword changes graphical system settings, that persist after the game closes, the graphical changes cause the problem mentioned here in the thread, the "color-bleeding" is fixed.

How to fix the color-bleeding issue:
1. You own The Legend of Zelda: Skyward Sword on disc and launch it on the vWii menu after inserting it into the CD drive.
2. After reaching the title screen, you can go back to the home menu (the graphical changes have been made).
3. You start a Gamecube game of your choice on Ninendont and can play it without color-bleeding.

Comparison images, on the one site(before) the Gamecube games that were started "normally" and on the other site(after) the Gamecube games that were started after Skyward-Sword:

 

[NoobletCheese]

@Fesku

Great find! Perhaps we can figure out what Skyward Sword is doing and see if we can reproduce it.

Previously I noted that patching viWidth/viXOrigin to certain values for certain games would sometimes fix it, but I wasn't able to reproduce it for other games & couldn't glean any underlying logic to it.

Skyward Sword's main dol contains 4 definitions for 480p modes with the following widths:

fbWidth 608; viWidth 670; viXOrigin 25
fbWidth 608; viWidth 686; viXOrigin 17
fbWidth 640; viWidth 670; viXOrigin 25
fbWidth 640; viWidth 686; viXOrigin 17

Assuming one of the above triggers the fix, we could try patching games so they apply those values before applying their own defaults.

 

[Ambidextrose]

I wasn't able to replicate the Skyward Sword trick. Did you use a nintendont forwarder channel or just homebrew channel?

 

[Fesku]

I wasn't able to replicate the Skyward Sword trick. Did you use a nintendont forwarder channel or just homebrew channel?

It shouldn't matter if you start Nintendont in the homebrew channel or in the forwarder, I started it in the homebrew channel.
What matters are your settings on Nintendont, don't change the videowidth setting, it should stay on "auto".
I tested Super Mario Sunshine PAL&NTSC and Super Smash Bros. Melee PAL&NTSC and in both games, both versions, the fix worked.

Additionally, my video settings: Videomode: Force NTSC, WiiU Widescreen: On

 

[NoobletCheese]

I am able to replicate the Skyward Sword fix, and I am launching it from ULGX.

But after exiting the game, vWii sysmenu becomes afflicted with the chroma bug (prior to launching Skyward Sword it was not afflicted). This does not give me confidence that it would work for all games.

On the other hand, Homebrew Launcher's GUI shows the Skyward Sword fix is still in effect (I'm using Snes9x's icon.png to judge... it has a red circle with a black outline). And ULGX's GUI shows the fix is still in effect (using the cyan circle outlines and some gamecover art to check).

However, even with the Skyward Sword fix in effect, the chroma still isn't 100% correct. It seems there are 2 bugged chroma modes on vWii:

1. Red bleeding to the right by ~2px and a 1px green edge on the left
2. Red bleeding to the right by ~1px and no green edge on left

Skyward Sword trick appears to result in 2, and my original Wii console does not have this.

Still, 2 is a lot better than 1, and depending on your TV's processor it may be smoothing it out to an acceptable degree since the error is only about 1px (this kind of thing can happen anyway if your TV's processor downsamples chroma to 4:2:2 internally).

I'm not confident the Skyward Sword trick will work for all games, however I will test a bunch of games and see how well it sticks between games and reboots, or whether vWii needs to be hard reset every time.

I think the issue might be better solved by patching vWii to output RGB instead of YCbCr, as this appears to be an issue with the CbCr channels having the wrong alignment. But if the RGB is derived from the bugged YCbCr then I don't suppose it would work.

 

[Deleted member 42501]

Interesting development.

The linked may also intrigue as it boots vWii into 480p automatically which gives a much better picture for me. Whilst not perfect its a lot nicer than what the Wii U mangles it to in 1080p usually:

https://gbatemp.net/threads/wiiu-video-mode-changer.490721/page-2#post-9480282

https://gbatemp.net/threads/hbl2hbc...hbl-into-vwii-hbc.455673/page-11#post-9059814

https://gbatemp.net/threads/vwii-in-480p-automatically.530221/page-2#post-8909578

https://shmups.system11.org/viewtopic.php?p=1208264#p1208264

Food for thought as fixing this issue would make the Wii U pretty much perfect as it really mashes the vWii output in standard mode. A firmware mod would fix all of these issues but a case by case patch would also be welcome either way...

 

[NoobletCheese]

After some more testing it seems you can get the "good" chroma for Wii titles if you hard reboot between each game launch. And for GameCube games a hard reboot followed by Skyward Sword trick.

But honestly that's a lot of hard rebooting which takes a long time & would be a dealbreaker for me. And the colour decoding is still wrong (reds are orangey & desaturated cause WiiU doesn't convert Wii's SD colours to HD colours... unless you take that out of the equation by using @mive 's app to force 480p and your TV treats 480p as SD colours...mine doesn't -- doh!).

These are the available modes according to mive's app:

const char *outPortStr[] = { "HDMI", "Component", "Composite/S-Video", "Composite/SCART" };

const char *outResStr[] = { "720p (fallback?)", "576i PAL50", "480i", "480p", "720p", "720p 3D?", "1080i", "1080p", "720p (again?)", "720p (No Signal from Component?)", "480i PAL60", "720p (yet again?)", "720p (50Hz, glitchy gamepad)", "1080i (50Hz, glitchy gamepad)", "1080p (50Hz, glitchy gamepad)" };

I wonder if any of these would result in RGB for vWii mode? If they do, then it might solve the issue. Need to try them one by one with a hard reboot in between, for both Gamecube and Wii titles.

I've got to say, if we could get this chroma issue fixed, vWii would be my preferred way to play Wii. Apart from the chroma issue the HDMI output seems sharper than Wii's analogue signal and is kind of what I'm after. Mayflash's Wii HDMI adapter is just as sharp but has black crush and "jailbars" on solid colour fields like blue. "So close yet so far".

 

[NoobletCheese]

It seems the chroma bug can be cleared for Wii games by opening and closing "Wii Menu Manual" in between game launches. I'll need to do more testing to make sure it works for all games.

Trying to find a similar thing for Gamecube titles. I've tried launching a bunch of homebrew apps to try and clear it by setting different video modes in 240p test suite, media player etc. but none of them had any effect.

 

[NoobletCheese]

Well it seems I was confused about the Skyward Sword trick, as it's not working for me anymore.

It seems the reason I thought it worked was because I had Nintendont set to video width to a wider value which greatly reduces the chroma bleeding, even though Mario Sunshine doesn't actually obey Nintendont's width.

But launching Skyward Sword does fix the chroma bleeding in Homebrew Launcher png icons, and instantiates the bug in sysmenu, so it's definitely doing something.

Also I think libogc has something to do with this -- launching Wii titles from ULGX results in different result compared to launching from game disc channel, regardless of whether I perform the "Wii Menu Manual" trick to clear the bug at the sysmenu. Will keep experimenting...

 

[Deleted member 42501]

The optimal solution lies in the DMCU:

You may have noticed that good old VI (the display controller / Video Interface of ye olde GameCube and Wii) is gone. Huh? How does that work? Turns out they’re emulating that in software and translating its configuration to the R7xx’s CRTC registers. But where does this emulation software run? Sneaky: they added a special microcontroller to the Latte just for this purpose. The DMCU is a 68HC11 compatible 8-bit CPU whose sole purpose is to perform VI emulation. It has a frontend hardware shim that looks like VI to the Wii software, and behind the scenes it translates those registers to the Radeon’s, including upscaling configuration. cafe2wii loads the DMCU’s firmware (the DMCU doesn’t seem to be active in Wii U mode). It has its own dedicated RAM and access to both the faux-VI back side and the Radeon’s register area. There is also a little mailbox to talk to it from vWii mode: the System Menu and IOS use this to configure the 4:3 stretch mode for Virtual Console games (which is why The Homebrew Channel accidentally ends up in the wrong mode: after a recent update, the System Menu thinks it’s a Virtual Console title due to its title ID starting with ‘L’ and sets up that special mode). Presumably it also got an update when gamepad support for vWii showed up, though I haven’t looked at the firmware again since then.

https://fail0verflow.com/blog/2014/console-hacking-2013-omake/


As this is whats obviously introducing the issues when it comes to interfacing vWii outputs to Wii U. Looking into patching its calls could be quite productive. Not sure how easy dumping its content would be or whether its reflashable but just entertaining variables here...

The 480p as mentioned previously definitely gives me a much better image with far more accurate colors but then you've got to switch it over. @Maschell can Tiramisu have any influence on these factors based on your insight and experience?