Homebrew Accessing the DRC/TV video buffer

[::Phoenix::]

Hi there, I am writing an homebrew which would hopefully allow to take screenshots at anytime while the Wii U executes. I already managed to get VPADRead hooked thanks to function patcher from loadiine's code and I am also able to hook such function whenever a title gets loaded.

The only thing I am missing is a way of accessing the raw video buffer of the DRC and/or TV in order to be saved to sd card after a specific button press (which wil be checked thanks to the VPADRead hook). Is accessing the video buffer something that can be done with kernel privileges or do I need IOSU access?

It seems the browser is able to access this data when sharing screens on miiverse, so there must be a way even without an exploit, but don't know how...

-EDIT- Maybe I found the right function, should be GX2SetTV/DRCBuffer.

 

[shinyquagsire23]

I'd disassemble GX2CopyColorBufferToScanBuffer, that's what actually ends up copying buffers to the displays. For some games there might only be one color buffer copied to both screens. There's probably an official function used somewhere else though, I've never really looked.

 

[::Phoenix::]

I'd disassemble GX2CopyColorBufferToScanBuffer, that's what actually ends up copying buffers to the displays. For some games there might only be one color buffer copied to both screens. There's probably an official function used somewhere else though, I've never really looked.

Thank you! I will check it out. As for know I am looking for something in gx2.rpl which allows to copy buffers from one format to another. It seems the TV/DRC buffer can be initialized to many formats (with the most frequent one being R8G8B8A8, I think), but it would be a pain in the ass to implement the conversion routine from scratch.

 

[Maschell]

In HID to VPAD I'm also hooking into GX2CopyColorBufferToScanBuffer to draw a cursror, but it doesn't work with all games. (demo)

https://github.com/Maschell/hid_to_vpad/blob/master/src/patcher/function_hooks.cpp#L49

To hook easily into functions, you can use
https://github.com/Maschell/function-patcher-example

I started to make screeenshot tool serval month ago, if you're interested in it, I can share the source code. My goal was to "record" the gamepad screen by making 30 screenshots/second.
http://gbatemp.net/threads/wii-u-hacking-homebrew-discussion.367489/page-936#post-6255919
https://vid.me/WIkU

 

[DeslotlCL]

In HID to VPAD I'm also hooking into GX2CopyColorBufferToScanBuffer to draw a cursror, but it doesn't work with all games. (demo)

https://github.com/Maschell/hid_to_vpad/blob/master/src/patcher/function_hooks.cpp#L49

To hook easily into functions, you can use
https://github.com/Maschell/function-patcher-example

I started to make screeenshot tool serval month ago, if you're interested in it, I can share the source code. My goal was to "record" the gamepad screen by making 30 screenshots/second.
http://gbatemp.net/threads/wii-u-hacking-homebrew-discussion.367489/page-936#post-6255919
https://vid.me/WIkU

Can't believe i didn't see that shovel knight footage you captured with screenshots only, that is really amazing!

Does it slowdown the gameplay while capturing screenshots for the video? It would be cool to record online gameplay footage with it.

 

[Maschell]

Does it slowdown the gameplay while capturing screenshots for the video? It would be cool to record online gameplay footage with it.

Yes, thats the reason why I tested it with Shovel Knight. This was the only game I found that could keep 25-30fps
I also spent a few days on porting a mp4 codec to the wiiu (from the wii), but gave up. Everything should still sleep somewhere on my HDD, if somebody is really interested in continuing it, I can search/share it.

btw. Don't try to write many small screenshots to your SD Card, the nintendo API is killing your sd card (Won't recognize it until you format everything)

 

[DeslotlCL]

Yes, thats the reason why I tested it with Shovel Knight. This was the only game I found that could keep 25-30fps
I also spent a few days on porting a mp4 codec to the wiiu (from the wii), but gave up. Everything should still sleep somewhere on my HDD, if somebody is really interested in continuing it, I can search/share it.

btw. Don't try to write many small screenshots to your SD Card, the nintendo API is killing your sd card (Won't recognize it until you format everything)

Still, you made an impressive job dude!

 

[::Phoenix::]

In HID to VPAD I'm also hooking into GX2CopyColorBufferToScanBuffer to draw a cursror, but it doesn't work with all games. (demo)

https://github.com/Maschell/hid_to_vpad/blob/master/src/patcher/function_hooks.cpp#L49

To hook easily into functions, you can use
https://github.com/Maschell/function-patcher-example

I started to make screeenshot tool serval month ago, if you're interested in it, I can share the source code. My goal was to "record" the gamepad screen by making 30 screenshots/second.
http://gbatemp.net/threads/wii-u-hacking-homebrew-discussion.367489/page-936#post-6255919
https://vid.me/WIkU

I am already using your function patcher. But started from scratch instead of that example though. My goal is less ambitious, I would like to take single screenshots in png format after a button combination has been pressed and do that from anywhere in the Wi U. This seems to be working thanks to the EXIT_RELAUNCH_ON_lOAD feature of HBL, which allows me to let the hooks work within the currently running title. What I am insterested to know is if you spent some time implementing conversion routines for all the possible buffer formats in order to save them to jpeg, or you found tha all games use plain simple RGBA8?

 

[Maschell]

I only implemented the conversion for like 2 or 3 texture formats. I can send you my compression/conversion code when I'm home)
Maybe @AboodXD has conversions for all textures formats?

 

[::Phoenix::]

I only implemented the conversion for like 2 or 3 texture formats. I can send you my compression/conversion code when I'm home)
Maybe @AboodXD has conversions for all textures formats?

As far as I can see only some of the formats defined in the enumeration are meant for the screen buffer and they are simple variations of uncompressed rgba. It should not be difficult to write down some on the fly conversion.

Your source code would be very helpful!

 

[::Phoenix::]

@Maschell I suppose that if I want to access the SD card on any title launch I will need iosuhax, unless I write a loader a LA loadiine for running the title inside mii maker, right?

 

[gudenau]

It is not hard to access the FB, just replace the buffer call and read from there. I've done it.

 

[::Phoenix::]

It is not hard to access the FB, just replace the buffer call and read from there. I've done it.

Thay's what I am doing right know. I am also able to save the whole raw buffer to SD. Didn't have much time to continue further. I just need to convert the raw data to PNG and move the saving process on a separate thread for not making the game stuck while saving.

Strangely the video buffer seems scrambled, with some swizzle-like algorithm, did nt expect it for the raw video buffer. Any tips about that?

 

[gudenau]

Thay's what I am doing right know. I am also able to save the whole raw buffer to SD. Didn't have much time to continue further. I just need to convert the raw data to PNG and move the saving process on a separate thread for not making the game stuck while saving.

Strangely the video buffer seems scrambled, with some swizzle-like algorithm, did nt expect it for the raw video buffer. Any tips about that?

I had a raw buffer, and PNG will take a decent amount of processing because of it's format. BMP would just require a header.

 

[starlander]

Phoenix, are you still working on this project?


Sent from my iPhone using Tapatalk

 

[BullyWiiPlaza]

Phoenix, are you still working on this project?

I also still need this for the TCP Gecko Installer...

 

[::Phoenix::]

Phoenix, are you still working on this project?


Sent from my iPhone using Tapatalk

My Job takes much of my time. Unfortunately, I am not able towork on homebrew recently. I hope I will be able to come back to at some point.

 

[BullyWiiPlaza]

Alright so I did some work on the screenshot thing and I managed to hook the GX2CopyColorBufferToScanBuffer function and print to the UDP log using the following hook:

static volatile int executionCounter = 0;

DECL(void, GX2CopyColorBufferToScanBuffer, const GX2ColorBuffer *colorBuffer, s32 scan_target) {
    if (executionCounter > 120) {
        GX2Surface surface = colorBuffer->surface;
        log_printf("GX2CopyColorBufferToScanBuffer {surface width:%d, height:%d, image size:%d, image data:%x}\n", surface.width, surface.height, surface.image_size, surface.image_data);

        executionCounter = 0;
    }

    executionCounter++;
    real_GX2CopyColorBufferToScanBuffer(colorBuffer, scan_target);
}

This runs the hook every 120 executions since the function is called like 120 times per second to avoid spam in the log file. It dumps out some fields from the GX2ColorBuffer's GX2Surface for information. The log file looks like this:

GX2CopyColorBufferToScanBuffer {surface width:1280, height:720, image size:3686400, image data:f5c7b800}
GX2CopyColorBufferToScanBuffer {surface width:854, height:480, image size:1658880, image data:f5ae6000}
GX2CopyColorBufferToScanBuffer {surface width:1280, height:720, image size:3686400, image data:f5c7b800}
GX2CopyColorBufferToScanBuffer {surface width:854, height:480, image size:1658880, image data:f5ae6000}
...

As you can see, the entries oscillate between two image resolutions and the image size/image data address stays the same. I went ahead and dumped the image data using JGecko U but it mainly contains 0xFF and 0x00 bytes which doesn't seem useful. How would you go and convert that to an actual JPEG or PNG now so you have a viewable screenshot on your computer?

 

[Sono]

I went ahead and dumped the image data using JGecko U but it mainly contains 0xFF and 0x00 bytes which doesn't seem useful. How would you go and convert that to an actual JPEG or PNG now so you have a viewable screenshot on your computer?

it must be raw image data. you must first get the framebuffer mode (you can guess the pixel size with (image_size / (width * height))) then you can use ImageMagick or a handwritten program to make a png for you from it.