DS emulation gets an augmented reality proof of concept

Screenshot from 2023-06-23 14-42-31.png

Homebrew and romhacking developers are always working on great things to bring forth new ways to experience great classics for older consoles. One such example is in DS emulation, which has seen it's fair share of attention these past years, like the introduction of a new emulator in the form of melonDS, which seems to be in a fast track to becoming the defacto DS emulator, and today it seems like a big breakthrough in DS emulation has appeared based on the work of melonDS.

Software developer Zhuowei Zhang has developed a proof of concept of an augmented reality feature for DS emulation, in which a holographic 3D image of the current game floats above the controller the player uses for the emulator.



As detailed in the tweet, the proof of concept works with melonDS and the melonDS core, utilizing a live 3D model extraction called MelonRipper, and the augmented reality development is possible using iOS' RealityKit. The prototype currently has some downsides, which Zhang goes into detail in the GitHub repository for the project:
  • the MelonRipper->RealityKit converter doesn't work very well (e.g. doesn't handle transparency)
  • there's a terrible memory leak (might be this?) that crashes the app after a few minutes
  • only tested with the camera position used by Mario Kart DS.
    • (for example, Pokemon Black and HeartGold use a different camera angle, and I had to remove the shader that crops the model for anything to show up)
  • no way to select rom/touchscreen input/etc. The ROM name is hardcoded to "rom.nds" in the app's folder in Files/iTunes.
Despite these downsides, the project as it stands is already outstanding, giving players some glimpses of the augmented reality with some DS titles like Mario Kart DS, Pokemon Heart Gold and Pokemon Black, giving a floating image of the games with great detail (aside from the transparency issues mentioned in the repository).

The project is currently open source and available in Zhang's GitHub repository linked below.

:arrow: DSReality GitHub Repository
 

Lostbhoy

HEY YOU GUYS!
Member
Joined
Aug 23, 2009
Messages
2,525
Trophies
2
Age
44
Location
Scotland
XP
5,259
Country
United Kingdom
I don't get this... Ar on a ds but they using a switch and obviously another camera source??

I dont understand what they are playing on, how they are viewing it, what device im watching in the video and my... it looks god awful... What is the point??

Im not trying to bash it here, just understand the what and why as personally it seems a pointless, overly convoluted way to play a ds game.

Can anyone explain a useful reason for this as I can't think of any ds game I wanna play like this!
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
  • Xdqwerty @ Xdqwerty:
    @SylverReZ I read that non-binary people identify as a blend of male and female. The point is i think i'm a blend of those and thus non binary
  • SylverReZ @ SylverReZ:
    @Xdqwerty, Non-binary people are those who don't identify themselves exclusively as male or female.
  • cearp @ cearp:
    I don't identify as anything, unless I need to get on a plane then I show ID
    +3
  • AncientBoi @ AncientBoi:
    👈 [homo, queer, fag, coksuqr, etc......]
    +3
  • Xdqwerty @ Xdqwerty:
    @SylverReZ, I meant that I read that they can also identify as what i mentioned
    +2
  • BigOnYa @ BigOnYa:
    I identify as Long Rod Johnson, cause a cat called me that once.
    +2
  • AncientBoi @ AncientBoi:
    [takes a 👀 in your pants] Liar :angry:
    +3
  • Xdqwerty @ Xdqwerty:
    @SylverReZ, and thus that would make me non binary since i feel like a blend of both male and female
    +1
  • AncientBoi @ AncientBoi:
    @Xdqwerty only time will tell, what/whom you are. At 14, its a bit early to "Define" yourself. :)
    +3
  • Psionic Roshambo @ Psionic Roshambo:
    I only recently have learned who I am and how much I am capable of and I'm 50 lol
    +3
  • Psionic Roshambo @ Psionic Roshambo:
    A thousand epiphanies and unbelievable pain and growth.
    +2
  • Psionic Roshambo @ Psionic Roshambo:
    I wish I could sum it up for children to understand but it truly takes living a lifetime of learning
    +2
  • Psionic Roshambo @ Psionic Roshambo:
    It's more than knowledge it's understanding
    +1
  • Psionic Roshambo @ Psionic Roshambo:
    Both are required
  • Xdqwerty @ Xdqwerty:
    @Psionic Roshambo, ask chatgpt or Google gémini to summarize it
  • Psionic Roshambo @ Psionic Roshambo:
    Lol summarize life's lessons
  • BigOnYa @ BigOnYa:
    My wifey tells me all the time "You need to grow the fuck up." Which I don't understand, cause I'm full grown already.
    +2
  • Xdqwerty @ Xdqwerty:
    @BigOnYa, she means grow more
    +1
  • Xdqwerty @ Xdqwerty:
    Yawn
  • AncientBoi @ AncientBoi:
    farrrrrrrrrrrt
    +1
  • BigOnYa @ BigOnYa:
    @AncientBoi I imagine it don't make the fart sound anymore, probably sounds more like letting go of a balloon you just blew up.
    +3
  • SylverReZ @ SylverReZ:
    @BigOnYa, Tell her to grow a pair. :rofl2:
    +3
  • Xdqwerty @ Xdqwerty:
    @SylverReZ, you played any Sega genesis games?
    Xdqwerty @ Xdqwerty: @SylverReZ, you played any Sega genesis games?