Homebrew DS(i) Mode hacking progress thread

[piratesephiroth]

I didn't even hear about that, I kind of gave up on mgba a while ago... Is this official mgba or retroarch?

oficial

https://forums.mgba.io/showthread.php?tid=425

 

[metroid maniac]

Streaming data?

And I see... That's kinda a bummer, but it was to be expected D: I was hoping it would help supersede DS flashcards as a whole, tbh. SD loading would be more of a community effort (testing, reporting bugs...) Compared to depending on flashcard teams.

Thanks for the info c:

Think games like GTA Chinatown Wars which continuously get data from the cartridge. Loading times are going to be significantly increased so these games will suffer when it comes to performance.

 

[piratesephiroth]

yeah, if only the slower ARM7 core can use the SD card any game that streams data continuously will be unplayable.

 

[FrozenDragon150]

I see ;w;

Well, thanks for your help :3

 

[metroid maniac]

yeah, if only the slower ARM7 core can use the SD card any game that streams data continuously will be unplayable.

I'm not sure about unplayable. It very much depends on each individual case. Keep in mind that it's possible that the ARM7 is largely idle in some games.

 

[zoogie]

Yellows8 just released 3ds_dsiwarehax_installer officially
https://github.com/yellows8/3ds_dsiwarehax_installer/releases

Still need k11 sploit to install on 11.0-11.1 systems.

 

[emmanu888]

Video Game Heroes works on 0,0,3 but IkuReader does not, it stops at a "can't find fonts" error

 

[Alex658]

Streaming data?

And I see... That's kinda a bummer, but it was to be expected D: I was hoping it would help supersede DS flashcards as a whole, tbh. SD loading would be more of a community effort (testing, reporting bugs...) Compared to depending on flashcard teams.

Thanks for the info c:

ARM7 sends data packages to ARM9 via streaming, ARM9 by itself doesn't have access to the SD card, ARM7 does. AFAIK, correct me if i'm wrong.

 

[metroid maniac]

ARM7 sends data packages to ARM9 via streaming, ARM9 by itself doesn't have access to the SD card, ARM7 does. AFAIK, correct me if i'm wrong.

That's correct but also not what I was on about.

 

[FrozenDragon150]

I'm actually pretty curious about the whole thing, lol.

Could someone explain, in very broad terms xD how actual flashcards work and how SD redirection is planned on being implemented?

 

[Sankie]

Yellows8 just released 3ds_dsiwarehax_installer officially
https://github.com/yellows8/3ds_dsiwarehax_installer/releases

Still need k11 sploit to install on 11.0-11.1 systems.

Sorry, what can you do with this? In simple words i can understand

And @Alex658 thanks for the help but couldnt make it work :/

 

[RocketRobz]

Sorry, what can you do with this? In simple words i can understand

And @Alex658 thanks for the help but couldnt make it work :/

@Alex658 <-^ tagging fixed

 

[zoogie]

Sorry, what can you do with this? In simple words i can understand

And @Alex658 thanks for the help but couldnt make it work :/

It installs dsiwarehax saves to your dsiware. This is of no use for non-cfw people at this time.
You must wait for k11 hax before you can use it on clean 11.x systems.

 

[Deleted User]

I apologize if this has been answered before, but how is it that homebrew NDS ROMs are able to be run through the launcher, yet official ROMS just lead to a white screen? Is there some flag being set in official ROMs that isn't being set in homebrew ROMs, or vice versa?

 

[smileyhead]

I apologize if this has been answered before, but how is it that homebrew NDS ROMs are able to be run through the launcher, yet official ROMS just lead to a white screen? Is there some flag being set in official ROMs that isn't being set in homebrew ROMs, or vice versa?

Homebrew reads from SD card.
Games try to read off the Game Cart, but fail to do so, resulting in a white screen.
This is why we need to figure out how to patch retail games to read off the SD.

 

[RocketRobz]

I apologize if this has been answered before, but how is it that homebrew NDS ROMs are able to be run through the launcher, yet official ROMS just lead to a white screen? Is there some flag being set in official ROMs that isn't being set in homebrew ROMs, or vice versa?

Retail ROMs attempt to read it's data from the Slot-1 cartridge, instead of the SD card.

 

[metroid maniac]

I'm actually pretty curious about the whole thing, lol.

Could someone explain, in very broad terms xD how actual flashcards work and how SD redirection is planned on being implemented?

Very simply - when a real game is running, the flashcard talks the same language a real game card does. When the game running on the console asks to load some new information from the flashcard, the flashcard replies in exactly the same way a real game card would. Therefore none of the actual game code is being patched and as far as the game is concerned, everything is functioning like normal.

If you want to get DS games loading from the SD, then you've got a lot of hurdles to overcome.
There is nothing "in between" that you can use to intercept communication between the running code and the SD card. You have to get stuck in the game's code and rewrite every command that used to ask for data from the cartridge to instead ask for data from the SD card. This is quite delicate and could easily fall apart.

If you remember USB Loaders, Nintendont or Dios MIOS on the Wii, then this is much harder. The Wii had a chip known as the "Starlet" that managed communications between the game and all the other hardware like the disc and USB. By taking over the Starlet you could make the game load from a USB drive without it ever knowing. The PS2 also has a similar design with its IOP, which is why HDD Loaders exist for that console.
That's just not possible on the DS because of the way it's built; the games talk directly to the game card and directly to the SD card, with nothing in between to hijack.

Furthermore there are some hardware quirks that get in the way. The DS has two main processors - an ARM9 for doing most of the heavy lifting, and an ARM7 for doing some background tasks. Only the ARM7 is capable of accessing the SD card, whereas both processors are able to access the card slot.
This is a real tricky one. Not only do you have to patch game code running on the ARM7 to get data from the SD card, but you also need to set up another "server" on the ARM7 so that the ARM9 can ask it to fetch data from the SD card on its behalf. Now it's getting even more complicated.

Just to put this into context - back in the early days of DS homebrew, each flashcard had a different way of letting homebrew retrieve data from the embedded memory card's files and folders (NOT the same thing as getting the data from within the ROM file. That works the same no matter what).
To get around this all the homebrew developers and flashcard manufacturers started using DLDI. Pretty much any DS hombrew that uses files and folders on the memory card uses DLDI. DLDI means that whenever you make a homebrew for the DS, all the instructions for getting files from the memory card are left intentionally blank. That way you load the homebrew on the flashcard, you can just fill the gaps automatically with whatever language the homebrew needs to speak in order for the homebrew to be able to ask the flashcard to get files from the memory card.
Now you've got a futureproof way of being able to get DS homebrews to load files from just about anywhere you can imagine, right?

nds_bootstrap implements DLDI patching for DS homebrews so they can load from the SD card, and despite the futureproofing intended in DLDI there are still lots of glitches in a lot of homebrews. Some aren't working at all or have major lag when loading stuff. This is just because it's so complicated trying to get the ARM7 and ARM9 to communicate and pass data around in a timely and accurate manner.
Real games are only going to be harder to patch than DLDI homebrews. DLDI is made to be easy for homebrew developers to make their own interface for, regular DS games aren't going to be so kind to us.

tl;dr The DS wasn't designed for this and it's not making it easy.

 

[Deleted User]

Very simply - when a real game is running, the flashcard talks the same language a real game card does. When the game running on the console asks to load some new information from the flashcard, the flashcard replies in exactly the same way a real game card would. Therefore none of the actual game code is being patched and as far as the game is concerned, everything is functioning like normal.

If you want to get DS games loading from the SD, then you've got a lot of hurdles to overcome.
There is nothing "in between" that you can use to intercept communication between the running code and the SD card. You have to get stuck in the game's code and rewrite every command that used to ask for data from the cartridge to instead ask for data from the SD card. This is quite delicate and could easily fall apart.

If you remember USB Loaders, Nintendont or Dios MIOS on the Wii, then this is much harder. The Wii had a chip known as the "Starlet" that managed communications between the game and all the other hardware like the disc and USB. By taking over the Starlet you could make the game load from a USB drive without it ever knowing. The PS2 also has a similar design with its IOP, which is why HDD Loaders exist for that console.
That's just not possible on the DS because of the way it's built; the games talk directly to the game card and directly to the SD card, with nothing in between to hijack.

Furthermore there are some hardware quirks that get in the way. The DS has two main processors - an ARM9 for doing most of the heavy lifting, and an ARM7 for doing some background tasks. Only the ARM7 is capable of accessing the SD card, whereas both processors are able to access the card slot.
This is a real tricky one. Not only do you have to patch game code running on the ARM7 to get data from the SD card, but you also need to set up another "server" on the ARM7 so that the ARM9 can ask it to fetch data from the SD card on its behalf. Now it's getting even more complicated.

Just to put this into context - back in the early days of DS homebrew, each flashcard had a different way of letting homebrew retrieve data from the embedded memory card's files and folders (NOT the same thing as getting the data from within the ROM file. That works the same no matter what).
To get around this all the homebrew developers and flashcard manufacturers started using DLDI. Pretty much any DS hombrew that uses files and folders on the memory card uses DLDI. DLDI means that whenever you make a homebrew for the DS, all the instructions for getting files from the memory card are left intentionally blank. That way you load the homebrew on the flashcard, you can just fill the gaps automatically with whatever language the homebrew needs to speak in order for the homebrew to be able to ask the flashcard to get files from the memory card.
Now you've got a futureproof way of being able to get DS homebrews to load files from just about anywhere you can imagine, right?

nds_bootstrap implements DLDI patching for DS homebrews so they can load from the SD card, and despite the futureproofing intended in DLDI there are still lots of glitches in a lot of homebrews. Some aren't working at all or have major lag when loading stuff. This is just because it's so complicated trying to get the ARM7 and ARM9 to communicate and pass data around in a timely and accurate manner.
Real games are only going to be harder to patch than DLDI homebrews. DLDI is made to be easy for homebrew developers to make their own interface for, regular DS games aren't going to be so kind to us.

tl;dr The DS wasn't designed for this and it's not making it easy.

This was very helpful and informative. Thank you!

Now you've got me curious; how would one set up a "server", as you described, to read from the SD Card? Is there any source worth looking at?

Also, what would the latency be like on games read from the SD Card?

 

[RocketRobz]

This was very helpful and informative. Thank you!

Now you've got me curious; how would one set up a "server", as you described, to read from the SD Card? Is there any source worth looking at?

Also, what would the latency be like on games read from the SD Card?

This source code might be worth looking at.

iirc, @ahezard said that a cache system would be implemented in the future, just not on the first release where we can run retail ROMs from SD.
The cache system would use 12MB of the DSi RAM when running DS-mode games.

 

[metroid maniac]

This was very helpful and informative. Thank you!

Now you've got me curious; how would one set up a "server", as you described, to read from the SD Card? Is there any source worth looking at?

Also, what would the latency be like on games read from the SD Card?

ahezard's nds-bootstrap currently patches the game homebrew's arm7 binary so the server is running alongside the regular arm7 code. Glancing back at ahezard's original post describing his implementation it's pretty clear that this isn't trivial to implement.

According to the github page, nds-bootstrap is currently able to load stuff about as fast as an old slot 2 device. That's still rather slow but it's a lot better than the earlier releases.

--------------------- MERGED ---------------------------

This source code might be worth looking at.

iirc, @ahezard said that a cache system would be implemented in the future, just not on the first release where we can run retail ROMs from SD.
The cache system would use 12MB of the DSi RAM when running DS-mode games.

The additional DSi main RAM cannot be used because it overrides an uncached mirror of the DS's original 4MB of RAM.
Though, the DSi does have that extra 768KB of WRAM. I'm not sure if that causes any conflicts, or if it could be useful.