Hacking Question about Wii U Emulation

[Noctosphere]

also whats the difference between thread and core? my cpu is tetracore and octothread

 

[mutinize]

also whats the difference between thread and core? my cpu is tetracore and octothread

Okay look I see you posting a bunch about clocks speeds, threads, cores, etc. What processor do you have. AMD or Intel. What video card do you have. AMD, Nvidia, or integrated. Typically emulation relies on the single thread performance of a CPU (that GHz label) and performs ... lackluster to say the least on AMD CPUs. Sometimes you can pass shit off onto the GPU (video card) to improve performance but if (big if) this emulator is legit I'm expecting it to be bare bones and poorly optimized for now. There is no "magic conversion rate" to emulators, some are very well optimized (dolphin) and can run on low specs and some are not which require a higher amount of resources.

Also you're just going to confuse yourself if you keep asking questions here. People are accurate but this is not the place for it. There are literally thousands of webpages on the internet dedicated to teaching you about these things.

I recorded some footage early September. I uploaded it here. There are a lot more graphical glitches compared to the current build. Also, speed was increased to 30 FPS. The original capture ran at 5-8 FPS.

Thank you for the video. Between that and the artifacting in the previous images I'm starting suspect this is actually happening. Which would be absolutely amazing. Now make the gamepad work on PC

Dolphin is the only good Gcn emulator. It's the only goodnWii emulator. Why not continue this and make it the only good Wii U emulator?

Dolphin has GCN and Wii support because their architecture is so similar they are basically the same console from an operating standpoint. The Wii to Wii U is such a difference in how they operate (night and day) that there's no chance of it being bundled into Dolphin unless someone is starting over from scratch. And at that point it might as well be a new thing.

 

[Bahax FZ]

Duplicated.

 

[Bahax FZ]

I initiated the project about 2 years ago, but counting away all the weeks inbetween where nothing happend the effective development time is about 6 months. Right now, I am the only developer on the project, although I had help in the past from two friends. There are plans to recruit more long-term members in the future.


HLE for all the Wii U libraries and the PPC Cafe OS kernel. As a result, IOSU emulation is not necessary. We decided for this approach because it's easier and more visible when progress is made. Somewhere down the line we may move away from HLE where possible.


In short, I believe the project does not profit from being open-sourced. Not at this stage at least. We are trying to take a different approach which will hopefully lead to more speedy development. I'll share more details in the future. At present, getting the emulator ready for the release is the top priority.


Started from scratch since there isn't really anything that could have been reused. The only reusable component would be the PPC interpreter, but for this I used my own PPC interpreter from a different unreleased emulator project (this interpreter is based on PearPC, but was completely rewritten. iirc Dolphins interpreter has a similar history?)


It's there but has not really had any noticeable implications so far. At this stage we use 1 thread to emulate all three cores. Changing it to one thread per core shouldn't be too difficult. The shared core clock timer might be annoying to emulate, but thats about it. Everything else is abstracted away inside API functions. I have yet to encounter any game which relies on accurate cache behavior.


I recorded some footage early September. I uploaded it here. There are a lot more graphical glitches compared to the current build. Also, speed was increased to 30 FPS. The original capture ran at 5-8 FPS.

I love you.

 

[oumoumad]

It is legit, you can see that he has some image problems maybe in alpha channel, when you look at the trees in SMBU : https://imgur.com/a/t53lB

So I definitely believe him, and I hope his goal for a release late this month will workout well.

 

[TecXero]

It is legit, you can see that he has some image problems maybe in alpha channel, when you look at the trees in SMBU : https://imgur.com/a/t53lB

So I definitely believe him, and I hope his goal for a release late this month will workout well.

That's not hard to fake, just need gimp, basic photo editing skills, and a basic knowledge of how images are displaying and effects rendered.

 

[neobrain]

I initiated the project about 2 years ago, but counting away all the weeks inbetween where nothing happend the effective development time is about 6 months.

That sounds familiar, huh

Good luck on the project, I suppose

 

[oumoumad]

That's not hard to fake, just need gimp, basic photo editing skills, and a basic knowledge of how images are displaying and effects rendered.

I no think someone would bother that much to actually separate each layer just to make them look as if he had a problem with the alpha channel... Just at the end to make a couple people believe him
I am a vfx artist, and no, it doesn't require JUST basic knowledge with GIMP to be able to separate layers very well and clean that way.

 

[TecXero]

I no think someone would bother that much to actually separate each layer just to make them look as if he had a problem with the alpha channel... Just at the end to make a couple people believe him
I am a vfx artist, and no, it doesn't require JUST basic knowledge with GIMP to be able to separate layers very well and clean that way.

He could directly edit the rendered image, he wouldn't have to edit the layers beforehand. Just a screenshot and editing that screenshot. It's not uncommon for people to go that far for a bit of internet fame, assuming it's that innocent. There have been worse cases where someone has led people on with convincing screenshots (even sometimes some really well done video, but that's rare) then finally release the "emulator" later, which just turns out to be malware or something else malicious.

I'm just saying remain skeptical until it's out and working.

 

[mustafag32g]

Is the wii U fully documented in the wiki or what? How the hell could one man suddenly emulate a wii U and make it run a game ?

Ci

 

[oumoumad]

He could directly edit the rendered image, he wouldn't have to edit the layers beforehand. Just a screenshot and editing that screenshot. It's not uncommon for people to go that far for a bit of internet fame, assuming it's that innocent. There have been worse cases where someone has led people on with convincing screenshots (even sometimes some really well done video, but that's rare) then finally release the "emulator" later, which just turns out to be malware or something else malicious.

I'm just saying remain skeptical until it's out and working.

It's up to you to be skeptical or not ^^ it doesn't change anything anyway.
For me I don't really have a reason to be skeptical and also I really really don't think someone would take that much effort and work hours tracing every single thing the same way very clean,pixel exact and change the colors and separate them and also faking the aliasing JUST so he can get some temporal fame from few people . Also from his comments, the guy knows what he's talking about when it comes to technical information about Wii U :

I initiated the project about 2 years ago, but counting away all the weeks inbetween where nothing happend the effective development time is about 6 months. Right now, I am the only developer on the project, although I had help in the past from two friends. There are plans to recruit more long-term members in the future.


HLE for all the Wii U libraries and the PPC Cafe OS kernel. As a result, IOSU emulation is not necessary. We decided for this approach because it's easier and more visible when progress is made. Somewhere down the line we may move away from HLE where possible.


In short, I believe the project does not profit from being open-sourced. Not at this stage at least. We are trying to take a different approach which will hopefully lead to more speedy development. I'll share more details in the future. At present, getting the emulator ready for the release is the top priority.


Started from scratch since there isn't really anything that could have been reused. The only reusable component would be the PPC interpreter, but for this I used my own PPC interpreter from a different unreleased emulator project (this interpreter is based on PearPC, but was completely rewritten. iirc Dolphins interpreter has a similar history?)


It's there but has not really had any noticeable implications so far. At this stage we use 1 thread to emulate all three cores. Changing it to one thread per core shouldn't be too difficult. The shared core clock timer might be annoying to emulate, but thats about it. Everything else is abstracted away inside API functions. I have yet to encounter any game which relies on accurate cache behavior.


I recorded some footage early September. I uploaded it here. There are a lot more graphical glitches compared to the current build. Also, speed was increased to 30 FPS. The original capture ran at 5-8 FPS.

 

[mustafag32g]

Reveal more pictures please ! I would like to see how sm 3d world looks like1

 

[TotalInsanity4]

Reveal more pictures please ! I would like to see how sm 3d world looks like1

Given that it's a full 3D rendered game and not 2.5D like the others shown, probably like sh*t lol

 

[Hykem]

To be honest, I've seen much more elaborated hoaxes than this.
I'm also surprised that no one mentioned that he/she could very well be running the games on a real Wii U and making random changes in RAM to cause glitches and graphical bugs. Especially taking into account the author's alleged involvement in WarCraft and StarCraft hacks: https://web.archive.org/web/2010112...ums/f25/[warcraft-iii]-wutil-maphack-365.html

Anyway, this is something extremely easy to prove and no source or proof of this "emulator" is even necessary. Considering that @Exzap is, currently, the sole developer of this project, it's logical to deduce that he/she must know the CafeOS kernel inside out, right? It's impossible to take the HLE approach without knowing every detail of what we are trying to emulate.
Well, what better piece of evidence could be provided than a nice write-up or even just a few reverse-engineered functions from the kernel? It's all public knowledge and no piece of said "emulator" would have to be shared.
At the same time, the author would be verifying his/hers claims without the need of posting more pictures or footage.

 

[oumoumad]

To be honest, I've seen much more elaborated hoaxes than this.
I'm also surprised that no one mentioned that he/she could very well be running the games on a real Wii U and making random changes in RAM to cause glitches and graphical bugs. Especially taking into account the author's alleged involvement in WarCraft and StarCraft hacks: https://web.archive.org/web/20101127205306/http://skillhackers.com/forums/f25/[warcraft-iii]-wutil-maphack-365.html

Anyway, this is something extremely easy to prove and no source or proof of this "emulator" is even necessary. Considering that @Exzap is, currently, the sole developer of this project, it's logical to deduce that he/she must know the CafeOS kernel inside out, right? It's impossible to take the HLE approach without knowing every detail of what we are trying to emulate.
Well, what better piece of evidence could be provided than a nice write-up or even just a few reverse-engineered functions from the kernel? It's all public knowledge and no piece of said "emulator" would have to be shared.
At the same time, the author would be verifying his/hers claims without the need of posting more pictures or footage.

Reasonable enough ^^. As the guy said, we'll see soon

 

[gamesquest1]

tbh i was actually thinking....that looks like the windwaker gecko codes you can add a whole bunch of weirdness via gecko.....plus its very odd he went from

Heard it will be released later this month...

to him being the only developer.....why would you hear something if your the only dev working on it unless you talk to yourself about your planned release date

 

[Exzap]

Reveal more pictures please ! I would like to see how sm 3d world looks like1

SMW3D doesn't run yet, it crashes on boot.

Just a disclaimer since I have noticed people started to hype this up. Don't expect too much from the first release. The three screenshots I posted were cherry picked from the few games that actually run trough the intro, none of them goes ingame yet.

Anyway, this is something extremely easy to prove and no source or proof of this "emulator" is even necessary. Considering that @Exzap is, currently, the sole developer of this project, it's logical to deduce that he/she must know the CafeOS kernel inside out, right? It's impossible to take the HLE approach without knowing every detail of what we are trying to emulate.

Yeah that would be the logical conclusion, but in practice thats not the case. I'll give you an example:

coreinit.FSOpenFile(Async) can be implemented without even looking at the assembly. You can guess the relevant parameters and ignore the not-so-important ones (like FSClient* and FSCmdBlock*). Needless to say, implementing it this way is inaccurate, but almost all games don't care about that. How should they even be able to tell whats going on internally? That being said, I prefer reimplementing stuff 1:1 in HLE if the advantages outweigh the additional time cost.
There are dozens of other such cases, I have a rough understanding of the whole system, but no particular in-depth knowledge on any of the components. It's simply not necessary at this point.

Well, what better piece of evidence could be provided than a nice write-up or even just a few reverse-engineered functions from the kernel? It's all public knowledge and no piece of said "emulator" would have to be shared.
At the same time, the author would be verifying his/hers claims without the need of posting more pictures or footage.

Have some random code instead:
GX2 color control and related code (API side only)
Some hacky Mii stuff. Games crash if Mii data is invalid.
I know you would prefer to see kernel code, but except the hacky, ugly and unpresentable threading code there is no kernel related code. Systemcalls are not emulated yet. Stuff like interrupts, MMU and memory-mapped I/O are not emulated because applications run in userspace and not supporting these features can boost performance a lot.

plus its very odd he went from to him being the only developer.....why would you hear something if your the only dev working on it unless you talk to yourself about your planned release date

I was trying to be nebulous for the fun of it. It backfired so I decided to stick to the facts.

 

[ExJam]

SMW3D doesn't run yet, it crashes on boot.
coreinit.FSOpenFile(Async) can be implemented without even looking at the assembly. You can guess the relevant parameters and ignore the not-so-important ones (like FSClient* and FSCmdBlock*). Needless to say, implementing it this way is inaccurate, but almost all games don't care about that. How should they even be able to tell whats going on internally? That being said, I prefer reimplementing stuff 1:1 in HLE if the advantages outweigh the additional time cost.
There are dozens of other such cases, I have a rough understanding of the whole system, but no particular in-depth knowledge on any of the components. It's simply not necessary at this point.


Have some random code instead:
GX2 color control and related code (API side only)
Some hacky Mii stuff. Games crash if Mii data is invalid.

Did you actually need to make async fs calls asynchronous? ATM I just call the the callback instantly from within the function but I'm worried that might run into issues later with locks in the callers code or something
https://github.com/exjam/wiiu-emu/blob/master/src/modules/coreinit/coreinit_fs.cpp#L344-L365

It's interesting that you bother with emulating the GPU register and command buffers. Is there any particular reason you took this approach over just directly mapping stuff to dx/opengl calls? Seems like a waste of effort to try replicate the buffers exactly when you end up translating the calls anyway. Our GX2 stuff is super early in progress but for example we just translate it directly: (for display lists we will just use dx12 command lists anyway so this works fine)
https://github.com/exjam/wiiu-emu/blob/master/src/modules/gx2/dx12/gx2dx12_draw.cpp#L17-L31
https://github.com/exjam/wiiu-emu/blob/master/src/modules/gx2/dx12/gx2dx12_display.cpp#L206-L231

How are you handling shaders? Are you decompiling them with proper control flow like if statements & while loops and such, or just keeping it simple for now?

How did you manage to do 1 thread for all 3 cores, surely switching between the threads is a nightmare! We use a host fiber per WiiU thread, and a host thread per WiiU core, I highly recommend that approach .

ps. you should open source your code

 

[Exzap]

@ExJam
For a majority of games it shouldn't matter if FS callbacks are done instantly or delayed. I handle them asynchronously because there is no reliable way to do PPC callbacks inside HLE functions in my interpreter implementation.

I don't know much about DX12 but OpenGL doesn't have command buffers, at least not in a way that would be useful. But that's not the real issue. When I started working on the GX2 functions I was worried that games could compile display lists offline and submit them via GX2CallDisplayList/GX2CopyDisplayList/etc. Not very fun to handle. Turns out, this isn't just a theoretical problem. Nintendo added functions and tools for exactly that. I know Yoshi's Woolly World uses it. Look for GX2PatchDisplayList() in the most recent gx2.rpl.
Additionally, what if you encounter commands which can not be translated to DX12's command buffers? It seems risky enough to be worth the extra mile.
Btw. the open source AMD driver in the mesa library contains lots of functions, constants and register offsets which match with GX2/GPU7. There is no need to reverse engineer everything from scratch.

Regarding shaders, our r700 Assembly -> GLSL converter makes a lot of assumptions. It ignores the whole control-flow stack thing and just assumes that JUMP instructions are part of a if or loop and that there is a PRED instruction just above which defines the condition. One of the bigger problems is that GLSL doesn't properly support integer and float types in the same variable, unlike the input assembly. But luckily there exists intBitsToFloat() and floatBitsToInt() in GLSL and we use it heavily. The output for a medium-complex vertex shader looks like this. It's disgusting enough to trigger bugs on older NVidia drivers and produces random output on Intel drivers. Untested on AMD hardware.

Lastly, I tried switching to Fibers not too long ago. But it was a debugging nightmare. Maybe I'll give it a try sometime again.

 

[Bahax FZ]

@Exzap

What games in its current build, runs at least the intro? besides you can post more videos of other games? (New Mario Bros U, DK Tropical Freeze for example, in the current build). With fraps showing the fps in realtime. Would be quite interesting and exciting, in addition to increasing your credibility.