Homebrew Making use of external memory (memory pak and SuperCard)

[Archerite]

While trying to port my homebrew remake of the BatteryCheck platform game to the DS I am patching the code to work around the limitations. One thing that is going to be required though is A LOT of memory to convert graphics on the DS itself. The reason I need to do this has to do with the license of the original game and that I do not own the graphics. The lowest memory usage I have measured was around 16-20MB but that was on my PC running in Linux using special tools. For the DS I also need to resize the graphics which lowers the required RAM quite a bit I think, but I still need to load the original in RAM first and allocate a second buffer to store the results in.

I have been reading a lot of examples and trying so many things to make use of the official "Memory Expansion Pak" and the "SuperCard miniSD" which both need a unlock sequence to make them writable in the GBA ROM space. It took me a while to figure out this was not working because I forgot to set the ARM9 as the SLOT-2 owner but after that it worked great! Or at least so I thought. I wrote a tiny 512 byte palette buffer into the external RAM and used DMA to copy it into VRAM. The bitmap was still stored in main ram though so I changed a few things to move that into external RAM too, and to make sure that worked I had to find a way to test it on an emulator. While desmume-cli is not perfect by any means it's what allows me to specify an R4 card with directory containing files and in the source I found the Memory expansion pak emulation option, except no way to enable it! Some digging was required to figure out it only selects the 8MB RAM in SLOT-2 when the opera ROM is loaded! So to be able to test my builds I modded the source to always enable it no matter what! And hey, it works!......except on real hardware

After many trial and error I think I have found the issue but hopefully some of you here know more about it. I think SLOT-2 can only be read and/or written in 16 bit mode! I know it's a 16bit bus ofcourse but hoped the hardware would just do it's magic in working out if I request the high or low byte. My earlier test worked with a uint16_t array so that was no problem. I wrote a small memory tester that writes a few numbers looping though an uint8_t array and it kept failing on hardware. In my modded desmume it worked fine! Until I made it an uint16_t array, and now it works on hardware too!!! While I am happy it works and for my own code I might be ok with this limitation, I also need to use zlib to decompress the original game files. And it's ZLIB that is giving me a ZLIB_DATA_ERROR each and every time I give it buffers located in external RAM.

So my question is: Does the external memory interface ONLY work in 16bit mode or do I need to setup some register to allow 8bit read and write to it?

 

[Alexander1970]

While trying to port my homebrew remake of the BatteryCheck platform game to the DS I am patching the code to work around the limitations. One thing that is going to be required though is A LOT of memory to convert graphics on the DS itself. The reason I need to do this has to do with the license of the original game and that I do not own the graphics. The lowest memory usage I have measured was around 16-20MB but that was on my PC running in Linux using special tools. For the DS I also need to resize the graphics which lowers the required RAM quite a bit I think, but I still need to load the original in RAM first and allocate a second buffer to store the results in.

I have been reading a lot of examples and trying so many things to make use of the official "Memory Expansion Pak" and the "SuperCard miniSD" which both need a unlock sequence to make them writable in the GBA ROM space. It took me a while to figure out this was not working because I forgot to set the ARM9 as the SLOT-2 owner but after that it worked great! Or at least so I thought. I wrote a tiny 512 byte palette buffer into the external RAM and used DMA to copy it into VRAM. The bitmap was still stored in main ram though so I changed a few things to move that into external RAM too, and to make sure that worked I had to find a way to test it on an emulator. While desmume-cli is not perfect by any means it's what allows me to specify an R4 card with directory containing files and in the source I found the Memory expansion pak emulation option, except no way to enable it! Some digging was required to figure out it only selects the 8MB RAM in SLOT-2 when the opera ROM is loaded! So to be able to test my builds I modded the source to always enable it no matter what! And hey, it works!......except on real hardware

After many trial and error I think I have found the issue but hopefully some of you here know more about it. I think SLOT-2 can only be read and/or written in 16 bit mode! I know it's a 16bit bus ofcourse but hoped the hardware would just do it's magic in working out if I request the high or low byte. My earlier test worked with a uint16_t array so that was no problem. I wrote a small memory tester that writes a few numbers looping though an uint8_t array and it kept failing on hardware. In my modded desmume it worked fine! Until I made it an uint16_t array, and now it works on hardware too!!! While I am happy it works and for my own code I might be ok with this limitation, I also need to use zlib to decompress the original game files. And it's ZLIB that is giving me a ZLIB_DATA_ERROR each and every time I give it buffers located in external RAM.

So my question is: Does the external memory interface ONLY work in 16bit mode or do I need to setup some register to allow 8bit read and write to it?

I think @Robz8 maybe can help,his TWiLight Menu and some Pokemon Games uses/supports the Slot 2 Memory Expansion on the DS/DS Lite.

 

[RocketRobz]

Maybe you could try using the space left in main RAM, and DMA copy the data to the Memory Expansion Pak.

 

[Archerite]

Maybe you could try using the space left in main RAM, and DMA copy the data to the Memory Expansion Pak.

Thanks for the suggestion, I think that is indeed how I need to handle this on the DS. No matter what I try it seems zlib is unable to use the external RAM directly for whatever reason.

In simple words this is what I am doing:
1 - allocate "packed" buffer for compressed stream
2 - allocate "unpacked" buffer for uncompressed stream
3 - read compressed stream from file into "packed" with fread() directly
4 - give the buffers and sizes to zlib::uncompress()
5 - check uncompress() returns Z_OK or something else

When using main memory this works fine on the DS just like on every other platform I have tried. But when I give zlib::uncompress() a buffer in the external memory pak region it seems to fail on hardware. The frustrating part is that on my modded desmume emulator the exact same build runs just fine! It must be something stupid that makes it fail when running on hardware but not in the emulator. You know any accurate emulator that supports R4 emulation and the expansion pak? Since the decompression is in an abstracted function in my game engine I might just make it decompress in main memory and then return a buffer in external memory to the calling function. Then I do not have to mess around with it in the rest of my sourcecode. Like I said it might just be some stupid register that needs to be set in a certain way that will solve this.

Maybe I should write a simple testing app that tries multiple ways and just say OK or FAIL when run. Now I keep messing up my source code and recompile every time

 

[Archerite]

I have just tried adding the following steps:
6 - allocate "buffer" in external memory (with my own dumb and stupid allocator)
7 - memcpy "unpacked" into "buffer" (should change that to dmaCopy I guess)
8 - free "packed"
9 - free "unpacked"
10 - return pointer to "buffer" (instead of "unpacked")

At least for the code that reads a bitmap and converts the palette into DS format this finally works on hardware too!! There are 6 bitmaps in the data file which are 300kb each and I tried loading them all in ExtRAM....3MB allocated and a little slow...but it works!!!