Homebrew DS-native GBA conversion

[Wyatt]

When I heard that the DSi was removing the Slot-2. I didn't shed many tears. "After all," I told myself, "I don't have many GBA games anyway, and I can emulate those on my x (x=whatever has an emulator) if need be." But then, I found this the thread on "The truth about GBA emulation" or something (http://gbatemp.net/index.php?showtopic=72623) and read it. I came to realize that while actual emulation of GBA code is likely impossible at a performant level, there is the possibility that another method could be used, which I will present here for interested parties.

To put it simply, the idea is to disassemble Game Boy Advance ROMs and rebuild them in an optimized form to be transferred and run as DS-native code using the LLVM framework (http://llvm.org).

I won't lie, it would likely be a substantial amount of work. One (or more likely, a group) would have to:
* write a disassembler frontend for LLVM to transform ARM7TDMI ASM to the LLVM IR
** LLVM's IR is very robust and one of the main features that elevates it to the fore of modern compiler theory.
** Does disassembly carry caveats from a legal standpoint? Such things are ever the hindrance of innovation.
* Write a compilation strategy for the DS hardware (ARM9+ARM7) that encapsulates:
** Sequentiality - It mustn't jump around too much or try to load too much at once. This is a doozy and is where the optimization flexibility of LLVM will really shine.
** IO reassignment - It's all different, though this might be the easiest step. Some stuff might be able to be done on the ARM7 anyway (the special non-ARM9 instructions I saw mentioned in the other thread)
** Reentrancy - I could be wrong on this, but as I recall, the DS was mentioned to run simultaneously on both ARMs. I believe that this creates an interesting requirement on thread safety.

Note, too that while I speak of a GBA frontend and a DS backend specifically, all backends use the same IR and one could potentially target any platform, giving performant GBA gaming to many machines and architectures.

Cheers,
Wyatt

 

[xxRAG3]

I'm not going to say TL;DR because your a genius. Could you dumb it down a little for me though ;P

 

[granville]

@xxRAG3: he wants to take apart gba games and recode them to run on the DSi.

We need two things in regards to gba without slot 1:

1- a thing it the registry process that requires the user to type "there is no way to run GBA roms without a slot-2 card" and without using copy/paste.

2- the permission to rickroll anyone who asks this

 

[MicShadow]

@xxRAG3: he wants to take apart gba games and recode them to run on the DSi.

We need two things in regards to gba without slot 1:

1- a thing it the registry process that requires the user to type "there is no way to run GBA roms without a slot-2 card" and without using copy/paste.

2- the permission to rickroll anyone who asks this

If you read up about it, it IS possible. But such thing would require a hypervisor, which Wyatt was talking about a compiler approach to it.
But as it's been said before, performance would be severly limitied without huge optimisation (thinking 1-5 fps if you're lucky)


Wyatts solution makes sense, but who will do that? and will it really be worth it, as its highly unlikely that it will work (well).

In short we tell the noobs that you CAN NEVER run slot-1 GBA, so they dont make a million threads asking if someone can do it.

 

[anoNL]

How are you going to fit an emulator + 32MB GBA ROM into 4MB of RAM?

 

[raulpica]

How are you going to fit an emulator + 32MB GBA ROM into 4MB of RAM?

Well, the DSi has 16MB of RAM.

A lot more possible, but still...
At this point we don't even know if it's possible to run Homebrew in native DSi mode (I'm confident it'll be possible... the question is when), so making speculations right now is prolly useless.

 

[FAST6191]

Caveat: I know almost nothing about LLVM. I dare say my first thought though is that it is the return of the FPGA hype: FPGAs are amazing but not practicable for general work at a very "high" level (I never know how to phrase such thing but I mean beyond signal swapping/tweaking).
This is also assuming total control over the DSi functions/hardware.
I know I will flip flop between proof of concept and full blown replacement but bear with me.

* write a disassembler frontend for LLVM to transform ARM7TDMI ASM to the LLVM IR

You appear to be proposing a form of dynamic recompilation using a virtual machine of sorts (LLVM) as a frontend and while it does have large foothold in emulation and the hardware is very close making things that little bit easier I am not convinced.
This being said the rule of thumb of 10x processor speed does seem to have been met (which with the similarities....).

** LLVM's IR is very robust and one of the main features that elevates it to the fore of modern compiler theory.
No comment until I do some reading. I assume it is similar to the idea behind java bytecode or the earlier versions of visual basic compared to the later ones but more efficient but you want it done in realtime (almost turning the emulation into an abstraction layer).

** Does disassembly carry caveats from a legal standpoint? Such things are ever the hindrance of innovation.
I know of no emulator that has been taken down on such a point (and several use similar techniques: see the N64 and gamecube emulators), BIOS copying/distribution yes but not this.

* Write a compilation strategy for the DS hardware (ARM9+ARM7) that encapsulates:
** Sequentiality - It mustn't jump around too much or try to load too much at once. This is a doozy and is where the optimization flexibility of LLVM will really shine.
DSi code- that is up to the designer. GBA code, that is not going to happen (have a go at tracing for a few minutes, here is a guide: http://www.romhacking.net/docs/361/ (site is down for me right now, search for VBA-SDL-h if you want a tracing tool though, I should have a copy kicking around on my machine if you really want it))

** IO reassignment - It's all different, though this might be the easiest step. Some stuff might be able to be done on the ARM7 anyway (the special non-ARM9 instructions I saw mentioned in the other thread)
I assume you talk of the slight differences between the DS and GBA BIOS instructions and similar. I should imagine making an interpreter for this would be the easiest step in all this.

** Reentrancy - I could be wrong on this, but as I recall, the DS was mentioned to run simultaneously on both ARMs. I believe that this creates an interesting requirement on thread safety.
Yes both ARMs are available and have been used to great effect in emulation already (sound hardware mainly) and other high performance tasks (the wavelet based video player DSVideo for the file system).
Race conditions huh, I too can not see most GBA code surviving such a problem.


Re: lack of space. I would hate to do it but an abstraction layer for DLDI and the 08XXXXXX-09XXXXX and higher waitstate mirrors is an idea I have toyed with. The main problem is lack of speed (ask any GBA slot supercard owner or a user of the GBC hardware cards used on the GBA and now multiply it by 10). A combination of speedpatches and extensive testing/reverse engineering* of roms may help alleviate this, also I toss in the idea of using generic data and replacing it as and when it is available (think you seeing the material colour of a 3d item before the textures load)

*as a rom hacker I can tell you this is a very long winded process and especially so for the GBA. On the other hand there have been several cases of game specific emulators (zelda on the GC and the virtual console as well as some DS homebrew like I think it was International Karate, in that case we are talking about the C64 if memory serves).


End:
You have many ideas that to my knowledge have not been tried in full in emulation yet (parts yes but see the difference between a logic gate and a processor) and I am very interested to see.
Can I instead suggest a more general proof of concept; we have hyper accurate emulators but they take lots of resources to use: BSNES ( http://byuu.cinnamonpirate.com/bsnes/ ) takes a fairly modern machine to run the snes at full speed) or could I suggest you talk to the MAME team, some of the later arcade boards could definitely benefit from this sort of thing.

 

[Wyatt]

In short we tell the noobs that you CAN NEVER run slot-1 GBA, so they dont make a million threads asking if someone can do it.
I'll admit, I was hesitant to actually post on this for that reason-- the other thread had that issue already. But in the end I figured that by properly labeling it as explicitly not emulation that it would be sufficient.

Anyway, to clarify some things:
I propose nothing like a hypervisor or emulator.
This is not dynamic recompilation: this is the automated translation of static GBA programs to static DS programs. In this method, LLVM takes the role of a static compiler and there is no non-native code being executed by the DS. Optimization is enabled by the nature of the LLVM IR (seriously, if you develop software and care about your compiler at all, you should read up on it; it's really neat), and code generation is tailored to the platform by the compilation strategy used on the backend.

As far as its ability to work well is concerned, it will be as performant as any other native code for the DS. Or any other platform: this is not for the DS or DSi specifically. It's a nice feature of compilers in general that the IR isn't specific to any particular language or platform: the backend is where the magic happens and a DS backend should be of interest to anyone trying to write code for the DS.

Note that I originally used sequentiality to refer to a specal case of locality of reference. You can think of the performance penalty as similar to a cache miss on x86. Compilers (LLVM included) are already designed to make this kind of optimisation.

QUOTEI assume you talk of the slight differences between the DS and GBA BIOS instructions and similar. I should imagine making an interpreter for this would be the easiest step in all this.

Going back to the other thread, yes, it was Normatt that mentioned that initially:
"the trig functions (e.g. missing bios functions) shouldn't be hard as you can use your own custom bios on the arm9 so essentially you could use the gba bios itself."
I'm not familiar enough with the issues at work there to comment, but I wonder if that might become a non-issue, given the nature of the optimisations that would be made?

 

[FAST6191]

OK it seems I missed the boat there.

Static conversion then.
I did a bit of reading on llvm and I have no doubt it is a step in the right direction but I can barely get a disassembled program to assemble again (and this accounting for headers, compression and whatever else "by hand" and with the aid of a tracing emulator) and all decompilers I have ever seen other than those languages that "support" decompilation tend to either operate about the standard libraries or essentially make a crude program with a lot of inline assembler (which is even further from being able to be compiled).
I will have to bow out as I either use ASM or something ridiculously high level like autoit (I plan on fixing that but until I do it would not be right to comment) and compiler theory is not something I am overly familiar with.

Sequentiality, I appear to have missed the elephant in the room when I spoke of that. Interrupts on the GBA are damn hard to get working properly and not just because of the syntax, requirements, timing and priority of them. Believe me when I say they are used all the time (the GBA is somewhat analogous to real mode of the x86 line) as well which would magnify your issues something chronic.

On the other hand while I do not think automated is all that possible I am inclined to say "conversion projects" similar to the translation ones might be able to bring a few games over.

 

[XRacer101]

This will never happen. Why? It'll take up way too much time and effort plus the maximum that you will get out of this is what like 5 fps?

 

[cupajoe]

This will never happen. Why? It'll take up way too much time and effort plus the maximum that you will get out of this is what like 5 fps?

I think if they used the method above it would run pretty close to full speed. But the effort doesn't match the outcome. Nobody will do all this work to get a single GBA game working on DS when you can play it by other means.

 

[dinofan01]

If you want to try, all power to you. But I still doubt it will happen.

 

[Heran Bago]

Do not reply without reading. Not emulation.

There are already dissemblers for ARM instructions. IDA Pro, right? I would think the harder parts would be making the ARM9 side for the DS, and the IO. The GBA's quite designed for its own screen resolution (for example) and the two options are to try a painful conversion or make some kind of shell to run it in (room on ARM9 side) but then it's not so much a port as a... feat of programming and hacks.

 

[Normmatt]

Static recompilation cannot be done for gba to nds, as the gba rom needs to be all in memory somewhere as its a ROM and not read from the cart itself like the ds, this means for static recompilation you would need to manually go through it and sort out exactly what does what and what exactly everything is and recode it to load only the parts of the game that it needs into memory, this would be a massive undertaking and would probably be easier to do a complete recode.