Homebrew emulating DLDI | reading from cartridge

seagreensys

Member
OP
Newcomer
Joined
Feb 16, 2021
Messages
10
Trophies
0
Age
23
XP
48
Country
Antarctica
Hello all, I've been working on a baremetal colorforth compiler on the DS in my spare time for the last few months.
However I've run into a relatively big roadblock namely reading from anything larger than the available ram.
I have 2 options: either DLDI or reading directly from the cart.
So far I've got DLDI working more or less on real hardware but havent had much success getting it to work on a PC emu.
On the other hand, reading from the cart seems like the universal option as its natively supported but I dont really understand how does the key1 key2 thing works, also the eeprom is limited to 8mb max which in itself is not a huge problem for forth when the entire compiler source is 20k tops and the editor should be another 10k but I want to leave things neat for whomever uses it in the future.
I was wondering if anyone could point me in the right direction or had a better understanding of the cart mechanism
 

seagreensys

Member
OP
Newcomer
Joined
Feb 16, 2021
Messages
10
Trophies
0
Age
23
XP
48
Country
Antarctica
You could check TWLMenu++ or GodMode9i's source code for the card code.
Thanks, I forgot to mention I'm writing for vanilla NDS so the i variants SD card is out of the picture(and my skill) at least for now. So far I've checked the libnds code for cart reading but it seems a bit convoluted.
 

Coto

-
Member
Joined
Jun 4, 2010
Messages
2,979
Trophies
2
XP
2,565
Country
Chile
Thanks, I forgot to mention I'm writing for vanilla NDS so the i variants SD card is out of the picture(and my skill) at least for now. So far I've checked the libnds code for cart reading but it seems a bit convoluted.

Yes, you can use DLDI in DESMUME by reading:
https://gbatemp.net/download/tgds-ramdisk_dldi.36517/

Every devkitARM built homebrew works with it (except homebrew using DS hardware behaviour that is unemulated). For example, yesterday I got to work SSEQPlayer with it, without resorting to work with real hardware.

32M only, because of that's how much SLOT-2 has mapped. Hmm, maybe I could add some paging of some sort to extend the DLDI , but AFAIK, when testing DS homebrew, 32MB is everything I need.
 
  • Like
Reactions: RocketRobz

seagreensys

Member
OP
Newcomer
Joined
Feb 16, 2021
Messages
10
Trophies
0
Age
23
XP
48
Country
Antarctica
Yes, you can use DLDI in DESMUME by reading:
tgds-ramdisk_dldi

Every devkitARM built homebrew works with it (except homebrew using DS hardware behaviour that is unemulated). For example, yesterday I got to work SSEQPlayer with it, without resorting to work with real hardware.

32M only, because of that's how much SLOT-2 has mapped. Hmm, maybe I could add some paging of some sort to extend the DLDI , but AFAIK, when testing DS homebrew, 32MB is everything I need.
Thanks, this is something I'll definitely try out.
Am not using devkitpro to build the program though, so I'll have to fiddle around with it.
So far as I understand I have to patch the .nds file with this and use the read/writeblk address like I would with normal DLDI or do I have to read directly from the gba memory?
 

Coto

-
Member
Joined
Jun 4, 2010
Messages
2,979
Trophies
2
XP
2,565
Country
Chile
Thanks, this is something I'll definitely try out.
Am not using devkitpro to build the program though, so I'll have to fiddle around with it.
So far as I understand I have to patch the .nds file with this and use the read/writeblk address like I would with normal DLDI or do I have to read directly from the gba memory?

Ah no, my bad, I explained badly. You need a .NDS binary, supporting DLDI.
The TGDS DLDI (in the link) is already compiled for your convenience. Then:

1)
You need to create a disk image, using the software (in the link). A 32MB disk image should be created/updated.
Here you can add all the data required into the disk image. Remember to set the option: "save them on shutdown".
Then rename it into "RAMDiskDLDI.img". You will need a 128K .SAV file along with it. Here, I added a dummy .SAV file I use. Unzip it in the same dir the disk image is!

2)
Patch the .NDS file with the TGDS DLDI (in the link). Use dlditool32 and select the TGDS RAMDISK DLDI from the list.

3)
Then, in DESMUME, you need to choose the 32MB disk image. Config > Slot 2 (GBA Slot) > GBA Cartridge > Browse... and the disk image file is selected. If everything was done correctly (and DESMUME doesn't crash), you should have DLDI up and running in the desired NDS homebrew!
 

Attachments

  • RAMDiskDLDI.zip
    18.7 KB · Views: 142
Last edited by Coto,
  • Like
Reactions: RocketRobz

seagreensys

Member
OP
Newcomer
Joined
Feb 16, 2021
Messages
10
Trophies
0
Age
23
XP
48
Country
Antarctica
Ah no, my bad, I explained badly. You need a .NDS binary, supporting DLDI.
The TGDS DLDI (in the link) is already compiled for your convenience.
Thanks Coto, was looking at the .dldi binary and it seems that I'll just need to include it when building the main image, so hopefully it'll work out in desmume.
Also, is the sav file needed? Because I still dont have any code to access the eeprom.
 

Coto

-
Member
Joined
Jun 4, 2010
Messages
2,979
Trophies
2
XP
2,565
Country
Chile
Thanks Coto, was looking at the .dldi binary and it seems that I'll just need to include it when building the main image, so hopefully it'll work out in desmume.
Also, is the sav file needed? Because I still dont have any code to access the eeprom.

That's a DESMUME limitation. Since we are mapping a 32M file into GBA address (0x08000000 +-32M),
SRAM (0x0E000000 + 64K)is also required, or DESMUME crashes.
The SRAM file is completely ignored when dealing with DLDI filesystem. So you may as well use your own SRAM file.
 

seagreensys

Member
OP
Newcomer
Joined
Feb 16, 2021
Messages
10
Trophies
0
Age
23
XP
48
Country
Antarctica
That's a DESMUME limitation. Since we are mapping a 32M file into GBA address (0x08000000 +-32M),
SRAM (0x0E000000 + 64K)is also required, or DESMUME crashes.
The SRAM file is completely ignored when dealing with DLDI filesystem. So you may as well use your own SRAM file.
Oh that makes sense, I would probably be left scratching my head if that had happened.
 

Coto

-
Member
Joined
Jun 4, 2010
Messages
2,979
Trophies
2
XP
2,565
Country
Chile
Oh, also I didn't mention: Since the DLDI is rewritable, it's completely OK to ship your NDS homebrew with TGDS DLDI by default. Cart loaders will likely put their own DLDI code when launching the DS homebrew. In fact, all DS homebrew by default is shipped with a stubbed out DLDI file.

If you patch the DLDI homebrew in the Makefile (right after the ndstool recipe), you get a ready-to-run DS homebrew in DESMUME everytime you build it.
 
  • Like
Reactions: RocketRobz

seagreensys

Member
OP
Newcomer
Joined
Feb 16, 2021
Messages
10
Trophies
0
Age
23
XP
48
Country
Antarctica
Oh, also I didn't mention: Since the DLDI is rewritable, it's completely OK to ship your NDS homebrew with TGDS DLDI by default. Cart loaders will likely put their own DLDI code when launching the DS homebrew. In fact, all DS homebrew by default is shipped with a stubbed out DLDI file.

If you patch the DLDI homebrew in the Makefile (right after the ndstool recipe), you get a ready-to-run DS homebrew in DESMUME everytime you build it.

Well I tried it and it patched fine but it seems that my system is too alien for desmume since it just seems to go off and never return when I try to read a disk sector.
I'll just have to bite the bullet and implement direct cart reads and write and leave DLDI for last, at least for the moment but thanks for all the help man.
 

Coto

-
Member
Joined
Jun 4, 2010
Messages
2,979
Trophies
2
XP
2,565
Country
Chile
Well I tried it and it patched fine but it seems that my system is too alien for desmume since it just seems to go off and never return when I try to read a disk sector.
I'll just have to bite the bullet and implement direct cart reads and write and leave DLDI for last, at least for the moment but thanks for all the help man.

Huh? That's weird. Maybe it's the DESMUME version?

Coto88 / toolchaingenericds-template — Bitbucket

You can see the screenshot here, mounted files properly through TGDS Ramdisk DLDI.
DESMUME Version I use: 0.9.11 x64 (Windows 8 x64)
 
Last edited by Coto,

seagreensys

Member
OP
Newcomer
Joined
Feb 16, 2021
Messages
10
Trophies
0
Age
23
XP
48
Country
Antarctica
Huh? That's weird. Maybe it's the DESMUME version?

Coto88 / toolchaingenericds-template — Bitbucket

You can see the screenshot here, mounted files properly through TGDS Ramdisk DLDI.
DESMUME Version I use: 0.9.11 x64 (Windows 8 x64)

Might be because am doing mostly everything on the arm7 in thumb mode which is a nono in most cases, it works IRL and in no$gba but it just hangs on some versions of desmume and melonds.
 

Coto

-
Member
Joined
Jun 4, 2010
Messages
2,979
Trophies
2
XP
2,565
Country
Chile
Might be because am doing mostly everything on the arm7 in thumb mode which is a nono in most cases, it works IRL and in no$gba but it just hangs on some versions of desmume and melonds.

ARM7 stacks should be at least 512 bytes for each processor mode. (0x200), and as usual, SYS/USR PSR bits are using the same stacks.

Also, the DLDI (through DLDI flags) maps the GBA Cartridge slot to ARM9 processor. Thumb code doesn't matter as long the toolchain and newlib is compiled against ARM code (ARM mode that is, not Thumb). DLDI code never touches ARM7 processor except for these bits. Also I've had much better compatibility with DESMUME than no$gba when debugging emulators/coding homebrew stuff. Really weird.

Or do you mean ARM7 DLDI code? I've done some of that...
Coto88 / gbarunner2 / gbarunner2 / common / dldigba.cpp — Bitbucket

Coto88 / gbarunner2 / gbarunner2 / arm9 / source / setup.s — Bitbucket

Coto88 / gbarunner2 / gbarunner2 / arm7 / source / main.cpp — Bitbucket

Of course this thing doesn't even boot in DESMUME, as it requires ARM7 interrupts, CART IO ports emulation, also mapped to ARM7, to handle read / write sector DLDI commands.
 

seagreensys

Member
OP
Newcomer
Joined
Feb 16, 2021
Messages
10
Trophies
0
Age
23
XP
48
Country
Antarctica
ARM7 stacks should be at least 512 bytes for each processor mode. (0x200), and as usual, SYS/USR PSR bits are using the same stacks.

Also, the DLDI (through DLDI flags) maps the GBA Cartridge slot to ARM9 processor. Thumb code doesn't matter as long the toolchain and newlib is compiled against ARM code (ARM mode that is, not Thumb). DLDI code never touches ARM7 processor except for these bits. Also I've had much better compatibility with DESMUME than no$gba when debugging emulators/coding homebrew stuff. Really weird.

Or do you mean ARM7 DLDI code? I've done some of that...
Of course this thing doesn't even boot in DESMUME, as it requires ARM7 interrupts, CART IO ports emulation, also mapped to ARM7, to handle read / write sector DLDI commands.

Well thats the thing, am not using anything related to the standard toolchain but doing everything from scratch on my own assembler/compiler, with the arm7 being the main processor and the arm9 being a slave that responds to FIFO irqs.
So for example I included your dldi binary near the base of main ram and as expected it gets hooked on real hardware and the read/writesectors function pointers get plugged in like they would normally, so I would just read that address put the parameters on the correct registers and do a BX call which would switch the processor to ARM state and read or write a sector on disk.
Also my stacks are really shallow the data stack is max 32 bytes or 8 words deep and the return stack is a little bit larger with 256 bytes, but they each grow into each other so I guess whats really happening could very well be some really bad stack corruption.
So I guess the design is way too different from the standard and something gets clobbered along the way, but at the moment I havent implemented better debugging tools to see whats happening exactly. I had similar problems with the PC version of the same compiler(the one am using to build the ds version) with opengl corrupting my stack but I never thought it would be an issue on baremetal.
 

Coto

-
Member
Joined
Jun 4, 2010
Messages
2,979
Trophies
2
XP
2,565
Country
Chile
Well thats the thing, am not using anything related to the standard toolchain but doing everything from scratch on my own assembler/compiler, with the arm7 being the main processor and the arm9 being a slave that responds to FIFO irqs.
So for example I included your dldi binary near the base of main ram and as expected it gets hooked on real hardware and the read/writesectors function pointers get plugged in like they would normally, so I would just read that address put the parameters on the correct registers and do a BX call which would switch the processor to ARM state and read or write a sector on disk.
Also my stacks are really shallow the data stack is max 32 bytes or 8 words deep and the return stack is a little bit larger with 256 bytes, but they each grow into each other so I guess whats really happening could very well be some really bad stack corruption.
So I guess the design is way too different from the standard and something gets clobbered along the way, but at the moment I havent implemented better debugging tools to see whats happening exactly. I had similar problems with the PC version of the same compiler(the one am using to build the ds version) with opengl corrupting my stack but I never thought it would be an issue on baremetal.

Well, I can save you a few years (5 so far) of your lifetime, and maybe... you may want to use TGDS homebrew? It's an alternative to devkitARM. Licensed code is GPL v2(or +). So you don't have to waste time on little nuisances and get to work straight into what you want to achieve.

You are asking these things to the correct guy I tell you. The ARM ABI is a standard, thus, by following it, it ensures ARM cores to behave correctly. Make sure you read :
The Status register - ARMwiki (heyrick.eu)
PSR and conditional execution (heyrick.eu)

Also you have better debugger in TGDS homebrew... yours truly.. TGDS-gdbstub!
Coto88 / toolchaingenericds-gdbstub-example — Bitbucket

If you add that code into any TGDS homebrew, you can remote debug memory through a GDB stub! I use:
affinic gdb debugger, which is a frontend UI for win32, hiding GNU GDB Debugger. You can inspect realtime EWRAM, ITCM, DTCM contents. through WIFI, between the DS and a PC.

Also I added that as an exception handler. So if you access, say, invalid DTCM area, a memory protection unit, data abort will trigger... calling a GDB debug session! So you can inspect what is in the stack at the time the exception took place.


Also i'm not sure you're using 32 bytes as stack (processor stack)? 32 bytes is 8 nested ARM opcodes! That will crash instantly. ARM cores have full descending stacks (which means the stack pointer BASE address, set in the SP register, r13, will always substract n registers (4 bytes each) from the current stack and save them in ascending order, starting from base new address, until the original r13 stack address, 4 bytes each).

This means:

0x02400000 <-- top EWRAM, mirrored.
irq_stack_mode: 0x02400000 - 512 bytes; //irq mode now has 512 bytes available
svc_stack_mode: irq_stack_mode - 512 bytes; //svc / supervisor now has 512 bytes available
usrsys_stack_mode: svc_stack_mode - 512 bytes; //User / System mode now has 512 bytes available

Thus, when the ARM core enters any mode, and enters a function, registers need to be banked/saved, thus by pushing them onto the stack will SUBSTRACT as many registers required, from the base address, into the r13 stack pointer.
When return from such function, you need to restore them from the r13 stack pointer, by retrieving them then ADDING internally the base address into what originally it was when the function was about to be called through a branch opcode.

Also it uses Clang for C/C++ code so it builds much better code than GCC. You can reverse engineer TGDS binaries and it's like reading a book.
 
Last edited by Coto,
  • Like
Reactions: RocketRobz

seagreensys

Member
OP
Newcomer
Joined
Feb 16, 2021
Messages
10
Trophies
0
Age
23
XP
48
Country
Antarctica
That sounds really nice so I'll probably check it out for an upcoming project since you can do DLDI properly in an emu, but atm I've already done most of the work on the colorforth compiler and the only thing I really need before moving to an on-system editor was reading from external memory.

I dont know if you're familiar with the forth language as its somewhat obscure and the variant am using is even more obscure but yeah you can do pretty much everything with very little code, for example lets take the stack, in forth we use 2 stacks to avoid making local variables, so whatever you need you put on the data stack and then pull it out later or you can push it to the return stack to have some more extra space.

The way I implemented mine was at the start of the arm7 wram, the data stack at 0x3800000 and the return stack at 0x38000100 so the data stack would grow to 3800004..8..C and so on while the return stack would go down, thats why I said I used 32 bytes for data since I just keep it shallow and the return stack can eat into the memory because they are in the same range.

The compiler am working on already works and compiles directly from the arm7 to both itself and the arm9, I used to have it in reverse as per nintendo's guidelines but found that without moving stuff to the TCM right off the bat the entire thing slowed to a crawl due to the memory waits for the arm9 and caching is not friendly to self modifying code so arm7.

I'll release the whole thing and some tutorials as soon as I write the editor and get the cart reading code done.
 
  • Like
Reactions: RocketRobz

Coto

-
Member
Joined
Jun 4, 2010
Messages
2,979
Trophies
2
XP
2,565
Country
Chile
Yeah that sounds quite the experience as a learning course. It's very frustrating when things do not work properly, but it's even worse when the compiler generates wrong code. I had a lot of issues with GCC code, sometimes it doesn't generate proper code and there is undefined behaviour as the program runs... that is the worst.

Clang is either: everything works or everything fails. So the development cycle has been cut to half, and now I can focus on better homebrew. (and limitations of self modifier code implementation, such as TGDS-multiboot which should allow to add new stuff I plan to write).

Needless to say, alpha programs (such as writing a compiler) will help when debugging/documenting blindly future usable homebrew, but for that you need to use something entirely PC-sided and very little of real NDS, unless you have a dev unit, which speeds up a ton debugging.
 

seagreensys

Member
OP
Newcomer
Joined
Feb 16, 2021
Messages
10
Trophies
0
Age
23
XP
48
Country
Antarctica
Hey man, just wanted to say thanks. I finally managed to get your DLDI driver working!
Turns out the arm9 had to give the arm7 control over the cart bus, thats why I was calling readSectors and it read garbage:yay:
 
Last edited by seagreensys,
  • Like
Reactions: RocketRobz

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
    HiradeGirl @ HiradeGirl: :discuss: