I (along with Lordus) have been thinking about all of this for a while to see if there was any way it could work out. We ended up asking sgstair, who actually did present us with something that could be feasible. So I have to take back what I said, I was wrong to throw it down so easily.
Then again, it's uncertain how well it'd work. There would probably be mild to serious compatibility issues. To paraphrase sgstair, "if such a thing worked it'd by thanks to nothing but luck."
Here's the way it works: when the MPU raises a prefetch abort code code is loaded into RAM for that block of ROM, and then the PC is changed to this block in RAM, for a small block size. This is then protected by another MPU region which will trap any branches or PC-relative references outside the block. When it exits the block, the block region window is changed to fit the block it moves to, or it's loaded into memory if it's not there (eventually evicting something that is there). For those familiar with translation lookaside buffers, this is basically using the MPU to create a 1-entry version. This part covers code virtualization and some data virtualization.
Other data accesses that are not PC-relative to the code block loaded in would end up being trapped (including those to indirect addresses back on ROM). There wouldn't be a data TLB, per se, so these would all have to be virtualized entirely for each access.
All I/O accesses would have to be trapped; you can't single them out because the granularity on the MPU isn't low enough for that. DMA would have to be emulated and redirected where possible.
The BIOS would be loaded in ITCM at the beginning. It'd probably benefit from some customization to make memory transfers faster.
The block size to choose is not an easy thing and I'm sure different games would work better with different things. Large block size means more code will be kept resident at a time (less TLB misses), but will mean that less of the working set will be captured due to internal fragmentation, and less code will be mappable overall due to MPU segment/size restrictions. For this to have any hope at working well I'm sure it'd have to be tweaked per-game for many case.
Positives:
- When code does run uninterrupted it'd be fast, assuming that it stays in icache enough of the time. Not only is DS's ARM9 4x the CPU speed of GBA's ARM7, it also has faster load/store instructions.
- Not a lot of effort would be needed to get this off the ground at first, it's not like writing an emulator.
- It might be possible to use the two rendering engines on both screens to capture + scale the output to fullscreen, although it'd lag a screen behind and it'd look pretty ugly anyway since it wouldn't be filtered.
- Could hack in things like savestates, rapidfire, screenshots, standard menu options, etc
Negatives:
- 16MB ROMs have a potential hazard since they expect a 16MB ROM address space to be accessible and might accidentally read from or branch to main RAM in a PC-relative fashion, causing the game to break. Branches are much more likely to happen than loads of this nature, meaning it can be alleviated by making the RAM portion non-execute and holding all necessary code in the upper half of ITCM (if it'll fit, there's only 16KB). The downside is that it'll break games that have code in EWRAM (it's not common but they do exist)
- Things are worse for 32MB ROMs, since they can't fit in the RAM space at all. If they try to branch from the lower half of the ROM to the upper half they'll end up in the middle of IWRAM and crash.
- 1 TLB entry is very unreliable. GBA code tends to jump around all over the place so it'll miss a lot on function calls at whatnot. What's even worse is if you get caught in a tight loop that crosses the boundary of the segment; if this happens you can kiss performance goodbye.
- If the game accesses an array at an indirect address in ROM via the CPU as opposed to using DMA or the GBA BIOS call for this they'll be very slow, since every access will have to be trapped and emulated.
- Some indirect branches have to be trapped too. For instance, if the code goes uses a lookup table (like for implementing switches). Returns from interrupts should usually work okay since the return addresses were stored implicitely.
- Sound is really tricky. The ARM9 can't access the audio hardware, GBA games tend to stream audio from buffers in IWRAM, which must be mapped to the ARM9 and not the ARM7. So audio DMA writes will have to be trapped and sent to the ARM7 in some way (either directly or via buffers). There are usually two streams updating for stereo sound so this is a lot of overhead. Games that rely on sound DMA IRQs are going to do even worse; the IRQs will have to be faked (I THINK this is perfectly doable, but costs you even more). These games aren't that common but do exist (Sword of Mana comes to mind)
- Games that have heavy raster effects will usually write to I/O a few times per scanline and this will add a lot of overhead.
- Because the display hardware doesn't match up in terms of line count HDMA will have to be emulated using HIRQs or else the game will glitch out. This adds more overhead.
- If the game relies on vblank happening at a certain line counter value then it's going to be in for a surprise. Faking vblank by faking the IRQ then forcefully blanking the screen might work, but sounds risky. Enter: more work to do per HIRQ.
- To handle all these VMM IRQs the IRQ handler is going to be a bit more bloated than the standard GBA BIOS one; I suppose the clock speed difference makes up for this.
- Enabling cache on IWRAM and VRAM will cause some games to glitch out or break entirely (inconsistencies with DMA, self modifying code, etc). Unfortunately, the IWRAM and VRAM as seen by the DS's CPU is actually slower than it is on GBA - you can expect code that runs from it to be almost twice as slow. If the game requires it to be uncached (like Golden Sun and other Camelot games would) and executes in IWRAM a significant percentage of the time (like most GBA games do, 40% is what I measured over some games) then they're going to suffer for this.
- GBA ROMs in general might not be that friendly with cache because they weren't written with it in mind. They jump all over the place and might introduce way collisions. Cache misses are worse than going through RAM, but for non-sequential loads/stores DS is actually slower than GBA's, I'm talking GBA's ROM. It makes up for this somewhat by issuing an N32 fetch for two Thumb opcodes at once (essentially getting a MUCH faster S16 second fetch out of this, where GBA would need to do two N16's - if you consider this DS is a bit faster). If this happens to volatile data it's a much worse story, the DS's RAM is significantly slower than the GBA's EWRAM. On the other hand, if it does stay cached then things will be in pretty good shape.
- Some games might thrash the thing's ROM cache by bouncing around data. GBA games tend to have pretty small working sets though.
So that's pretty much my take on it. Do I think it might be possible? Perhaps. Do I think someone actually did it? Maybe, but I have some doubts. Do I think it's worth trying? Yes, from a technical standpoint it's pretty interesting and not a huge amount of work to get some results. Do I think it'd be a suitable overall replacement for running games off of slot 2? I doubt it.
Then again, it's uncertain how well it'd work. There would probably be mild to serious compatibility issues. To paraphrase sgstair, "if such a thing worked it'd by thanks to nothing but luck."
Here's the way it works: when the MPU raises a prefetch abort code code is loaded into RAM for that block of ROM, and then the PC is changed to this block in RAM, for a small block size. This is then protected by another MPU region which will trap any branches or PC-relative references outside the block. When it exits the block, the block region window is changed to fit the block it moves to, or it's loaded into memory if it's not there (eventually evicting something that is there). For those familiar with translation lookaside buffers, this is basically using the MPU to create a 1-entry version. This part covers code virtualization and some data virtualization.
Other data accesses that are not PC-relative to the code block loaded in would end up being trapped (including those to indirect addresses back on ROM). There wouldn't be a data TLB, per se, so these would all have to be virtualized entirely for each access.
All I/O accesses would have to be trapped; you can't single them out because the granularity on the MPU isn't low enough for that. DMA would have to be emulated and redirected where possible.
The BIOS would be loaded in ITCM at the beginning. It'd probably benefit from some customization to make memory transfers faster.
The block size to choose is not an easy thing and I'm sure different games would work better with different things. Large block size means more code will be kept resident at a time (less TLB misses), but will mean that less of the working set will be captured due to internal fragmentation, and less code will be mappable overall due to MPU segment/size restrictions. For this to have any hope at working well I'm sure it'd have to be tweaked per-game for many case.
Positives:
- When code does run uninterrupted it'd be fast, assuming that it stays in icache enough of the time. Not only is DS's ARM9 4x the CPU speed of GBA's ARM7, it also has faster load/store instructions.
- Not a lot of effort would be needed to get this off the ground at first, it's not like writing an emulator.
- It might be possible to use the two rendering engines on both screens to capture + scale the output to fullscreen, although it'd lag a screen behind and it'd look pretty ugly anyway since it wouldn't be filtered.
- Could hack in things like savestates, rapidfire, screenshots, standard menu options, etc
Negatives:
- 16MB ROMs have a potential hazard since they expect a 16MB ROM address space to be accessible and might accidentally read from or branch to main RAM in a PC-relative fashion, causing the game to break. Branches are much more likely to happen than loads of this nature, meaning it can be alleviated by making the RAM portion non-execute and holding all necessary code in the upper half of ITCM (if it'll fit, there's only 16KB). The downside is that it'll break games that have code in EWRAM (it's not common but they do exist)
- Things are worse for 32MB ROMs, since they can't fit in the RAM space at all. If they try to branch from the lower half of the ROM to the upper half they'll end up in the middle of IWRAM and crash.
- 1 TLB entry is very unreliable. GBA code tends to jump around all over the place so it'll miss a lot on function calls at whatnot. What's even worse is if you get caught in a tight loop that crosses the boundary of the segment; if this happens you can kiss performance goodbye.
- If the game accesses an array at an indirect address in ROM via the CPU as opposed to using DMA or the GBA BIOS call for this they'll be very slow, since every access will have to be trapped and emulated.
- Some indirect branches have to be trapped too. For instance, if the code goes uses a lookup table (like for implementing switches). Returns from interrupts should usually work okay since the return addresses were stored implicitely.
- Sound is really tricky. The ARM9 can't access the audio hardware, GBA games tend to stream audio from buffers in IWRAM, which must be mapped to the ARM9 and not the ARM7. So audio DMA writes will have to be trapped and sent to the ARM7 in some way (either directly or via buffers). There are usually two streams updating for stereo sound so this is a lot of overhead. Games that rely on sound DMA IRQs are going to do even worse; the IRQs will have to be faked (I THINK this is perfectly doable, but costs you even more). These games aren't that common but do exist (Sword of Mana comes to mind)
- Games that have heavy raster effects will usually write to I/O a few times per scanline and this will add a lot of overhead.
- Because the display hardware doesn't match up in terms of line count HDMA will have to be emulated using HIRQs or else the game will glitch out. This adds more overhead.
- If the game relies on vblank happening at a certain line counter value then it's going to be in for a surprise. Faking vblank by faking the IRQ then forcefully blanking the screen might work, but sounds risky. Enter: more work to do per HIRQ.
- To handle all these VMM IRQs the IRQ handler is going to be a bit more bloated than the standard GBA BIOS one; I suppose the clock speed difference makes up for this.
- Enabling cache on IWRAM and VRAM will cause some games to glitch out or break entirely (inconsistencies with DMA, self modifying code, etc). Unfortunately, the IWRAM and VRAM as seen by the DS's CPU is actually slower than it is on GBA - you can expect code that runs from it to be almost twice as slow. If the game requires it to be uncached (like Golden Sun and other Camelot games would) and executes in IWRAM a significant percentage of the time (like most GBA games do, 40% is what I measured over some games) then they're going to suffer for this.
- GBA ROMs in general might not be that friendly with cache because they weren't written with it in mind. They jump all over the place and might introduce way collisions. Cache misses are worse than going through RAM, but for non-sequential loads/stores DS is actually slower than GBA's, I'm talking GBA's ROM. It makes up for this somewhat by issuing an N32 fetch for two Thumb opcodes at once (essentially getting a MUCH faster S16 second fetch out of this, where GBA would need to do two N16's - if you consider this DS is a bit faster). If this happens to volatile data it's a much worse story, the DS's RAM is significantly slower than the GBA's EWRAM. On the other hand, if it does stay cached then things will be in pretty good shape.
- Some games might thrash the thing's ROM cache by bouncing around data. GBA games tend to have pretty small working sets though.
So that's pretty much my take on it. Do I think it might be possible? Perhaps. Do I think someone actually did it? Maybe, but I have some doubts. Do I think it's worth trying? Yes, from a technical standpoint it's pretty interesting and not a huge amount of work to get some results. Do I think it'd be a suitable overall replacement for running games off of slot 2? I doubt it.
