Another in the very long list of "nice idea but probably not much effort put in on my part" threads but what the hey.
The short version is I want database of every AP laden, "simple" bug sporting or uses custom hardware DS game to have a commented patch to fix it. I hold this is useful as previously there was not so much in the way of this and what there was usually got fixed by the scene or other entity and a patch released for all to play with where the DS had it fall to closed source loving flash card teams.
Unlike most consoles before it the DS has a lot of games with extensive antipiracy, "simple" bugs and hardware tweaks. For the most part there are some patches and emulators kind of work around it but having witnessed a far lesser version of it on the GBA I thought I might float the idea to prevent it, especially now DS releases have basically dried up. More problematically most of the proper patches are tied up in closed source flash carts where previous systems had scene or group driven patches, a problem already noted by several scene members if you read the discussions that led to various new standards and similar such things.
What few anti piracy removal patches there are could be useful and I do have a copy of pre lockdown (June 2010 vintage) WOOD/AKAIO source code with a lot of patches in (including Catan, Castlevania Portrait of Ruin and Combat of Giants insects). The question of whether to dump a cheat database (many early patches could be applied as a handful of cheats), said copy of AKAIO/Wood RPG's patch database and whatever else and move sideways would want to be addressed.
Similarly there are a few games with odd hardware like guitar hero, the arkanoid paddle controller (though for the most part this is optional and most are more likely to want to add it into a game). Most of these have patches already but they could possibly stand to be better or at least able to be modified to suit the whims of a given user (we already had several for guitar hero). Likewise though patches existed at the time on the GBA for boktai few would be caught using anything but http://gbatemp.net/threads/boktai-solar-sensor-patch-kit.245127/ today, and even there they got customised as well.
I am thinking about pokewalker IR patches for save dumping, pokepatch for PAL park and similar such things as part of this, if only because of the amount of "but pokemon emerald RTC" stuff I have seen.
On bugs I am mainly looking at the things mentioned above like Portrait of Ruin crashing or the Catan bug. I love improvement patches as much as anybody and will happily lend whatever assistance I can for someone wanting to do that but it really has to be outside the scope of this project, probably the point where a second project is attempted in these cases. For a first go at defining something "if it crashes the game, breaks a mode or causes slowdown*" then it plays.
*the mere act of removing the AP has been seen to improve various speeds of various things.
I would be quite harsh here and even things like language mods/unlocks for those games with it already in would be a separate project or at best an addendum database/table within it.
I would say for this project that comments are to be where it is at. Naturally you do not have to comment every "this is a below 8000h read" but if something is obfuscated then that would be nice for those to know. The AKAIO source which I will probably post up before long (or you can see the castlevania version of http://gbatemp.net/threads/which-flashcart-for-castlevania-por.343239/page-2#post-4559271 ) would be a good example, though this project would probably be less C style code than that.
Finally this is of some importance as ROM hacking tends to trip a lot of these protections and having a would be hacker have to fix it before getting on with their project, we see this sometimes in megadrive hacking but here at least game genie mastercodes can usually help out there (to say nothing of simpler nature and lower performance of things restricting what could be done).
To the best of my knowledge there are 6 types of AP and related oddities to tackle here.
Below 8000
Checksums
Save timing
Daigassou! Band Brothers DX unique save stamps.
DSi workarounds.
iQue games.
More on each of these at the end of the post.
I am not sure save type changing needs to be tackled, GTA was probably the most notable of them for this. The only real issue is likely to be one of the Daigassou! Band Brothers DX anyway.
All have examples of simple things and if my conversations with various flash cart makers and other relevant people are anything to go by some of the later games had nice obfuscated checks, checks buried deep in overlays, checks only loaded in the late game and "subtle effects" AP rather than crashes, whitescreens, demo modes, lockscreens and other obvious effects.
My personal preference is to work backwards from the last releases so people with lesser cards can benefit earlier but by its very nature the project would be open for anybody to have a look at.
As many that did the flash card dance this time around will be able to tell you there are "generic" patches or patching engines that work for some games (same publisher, dev, series, region dupes), I am not adverse to this though in the hypothetical database I would probably try to see if we can spin off a patch for every instance it might matter in but note it as "one of the ? AP variant games".
Projects of similar scope have been done before. Probably the most notable was the attempt to "train" every original xbox game which largely did what it set out to do.
The protections then.
Below 8000 hex.
Under normal conditions the data below 8000 hex on a game cart can not be read. Part of this is the header which is grabbed from memory and part of it is also the "secure area" but neither matters for this. There is also some extra stuff for later DS games where some area of the header previously thought 00 filled got some utterly useless to any hacker, flash cart maker or anybody really data (it technically includes the DSi/3ds signing so they can use a whitelist.... http://hackmii.com/2010/02/lawsuit-coming-in-3-2-1/ ). This is why you saw a lot of "redumps" over the last year or two and why nobody really gave a damn about them.
If the game tries to read it the cart actually returns according to a set pattern, see the "get data" command in
http://nocash.emubase.de/gbatek.htm#dscartridgeprotocol
No game ever used it for a "real" purpose and everything you see here was a check.
Certain emulators would note these reads in their error logs, you have options for static disassembly and more focused methods as well.
Checksums.
Here the binary, which should be static in most games as they were compiled (a few games used various forms of lua and other "scripting" languages, not sure what goes in the way of commercial emulators and dynamic recompilation on the DS), has a checksum done of it or possibly some section of it. This checksum would then be changed by games changing the save type (why savelist cards dodged it for a while), by cheats in general (the binary is in memory and cheats edit memory, many good cheat makers then edited a binary as part of a cheat) and by adding cheat engines/soft reset (why the special/clean/ghost/safe/whatever modes of various flash carts would prevent cheats, soft reset and other such functionality from happening).
Update. Apparently COP the recruit actually restored original binary sections that flash carts would have fiddled with during runtime.
Various checking methods though none quite so nice as below 8000 hex. Later games favoured heavy obfuscation, hundreds of checks (and not just a function call either), sometimes burying things in overlays only loaded in late game or in the ARM7 and other such niceties. That said the only integrity checking that should be going on is the usual anti cheat stuff (mirrored, complementary.... all the stuff you learn of once you get past the basic "how to have infinite lives/ammo" type cheats), save checksums, those needed by some network communications and the like.
Save timing.
Arguably the first AP method on the DS. Here you will often find the EEPROM chip or whatever chip was originally on the commercial release was slower and being a difference it can be used as a check. One instance I know of and it was a fairly obscure Japanese game which I have since forgotten the name of and did not note in my ROM hacking docs for some reason. It was patched later so if you go through the older AKAIO/WoodRPG changelogs (and the EZ5 ones which is where I learned of it) then you will see it.
Daigassou! Band Brothers DX unique save stamps.
DSi games and homebrew aside this is probably the only game to feature DLC give or take the "daily puzzles" on various versions of picross or how you might want to look at some of the pokemon bonuses.
Here you could download extra songs over and above what the original cart came with. I am not sure if this was a licensing thing or what. Either way the commercial release shipped with a 64Mbit flash chip for saves (flash 8 Mbit was the next step down though Warioware DIY and Jam with the Band had another type that variously gets termed NAND saves or Flash 256 Mbit). It is impractical for a commercial game release to make unique ROM versions for each game but saves are a different matter entirely so it seemed every save shipped had a unique marker which was sent to the DLC server.
All the DLC songs were dumped as far as I know and injection of songs (custom, DLC or otherwise) into the game was figured out early on.
Not sure what goes with the "Jam with the Band" which was roughly the European release of the game (it never made it to North America).
DSi workarounds.
No idea here. I just saw it in the WoodRPG sources I was looking at for the other patches so I mention it here. If it needed patches to work on a DS I do really have to include it though.
Apparently the commonly used injection point for flash cart stuff was zeroed out by the games, probably of little real interest for this project then.
iQue games.
Including demos and a probably pointless proper there are a grand total of 7 of these and chances are if you see a Chinese game it is probably a hacked version of a Japanese game or similar (the Chinese translation scene is nothing if not prolific). China does get Nintendo hardware though a company/arrangement sees them branded as the iQue whatever and in this case it was the iQueDS. The hardware plays all the same games and does all the same things but Japanese on the main menu and in return it gets a reasonable line in simplified Chinese to use in pictochat. As games carry their own fonts this is basically it. However the few official games for some reason want to only be played on the iQue, the "protection" was a very simple firmware check and never anything obfuscated. http://nocash.emubase.de/gbatek.htm#dsfirmwareserialflashmemory has more on this.
For all the above binaries and overlays do have compression, it is a custom form of LZ usually called something like DS binary/overlay compression though technically/if speaking in a more general compression sense it is a type of BLZ (backwards LZ) and it is used on the ARM9 and its overlays (the ARM7 tends to be left alone). For the likes of AKAIO and other runtime things this is less problematic as things have to be decompressed to run after all. Encryption has never been seen in DS game binaries and data though.
The short version is I want database of every AP laden, "simple" bug sporting or uses custom hardware DS game to have a commented patch to fix it. I hold this is useful as previously there was not so much in the way of this and what there was usually got fixed by the scene or other entity and a patch released for all to play with where the DS had it fall to closed source loving flash card teams.
Unlike most consoles before it the DS has a lot of games with extensive antipiracy, "simple" bugs and hardware tweaks. For the most part there are some patches and emulators kind of work around it but having witnessed a far lesser version of it on the GBA I thought I might float the idea to prevent it, especially now DS releases have basically dried up. More problematically most of the proper patches are tied up in closed source flash carts where previous systems had scene or group driven patches, a problem already noted by several scene members if you read the discussions that led to various new standards and similar such things.
What few anti piracy removal patches there are could be useful and I do have a copy of pre lockdown (June 2010 vintage) WOOD/AKAIO source code with a lot of patches in (including Catan, Castlevania Portrait of Ruin and Combat of Giants insects). The question of whether to dump a cheat database (many early patches could be applied as a handful of cheats), said copy of AKAIO/Wood RPG's patch database and whatever else and move sideways would want to be addressed.
Similarly there are a few games with odd hardware like guitar hero, the arkanoid paddle controller (though for the most part this is optional and most are more likely to want to add it into a game). Most of these have patches already but they could possibly stand to be better or at least able to be modified to suit the whims of a given user (we already had several for guitar hero). Likewise though patches existed at the time on the GBA for boktai few would be caught using anything but http://gbatemp.net/threads/boktai-solar-sensor-patch-kit.245127/ today, and even there they got customised as well.
I am thinking about pokewalker IR patches for save dumping, pokepatch for PAL park and similar such things as part of this, if only because of the amount of "but pokemon emerald RTC" stuff I have seen.
On bugs I am mainly looking at the things mentioned above like Portrait of Ruin crashing or the Catan bug. I love improvement patches as much as anybody and will happily lend whatever assistance I can for someone wanting to do that but it really has to be outside the scope of this project, probably the point where a second project is attempted in these cases. For a first go at defining something "if it crashes the game, breaks a mode or causes slowdown*" then it plays.
*the mere act of removing the AP has been seen to improve various speeds of various things.
I would be quite harsh here and even things like language mods/unlocks for those games with it already in would be a separate project or at best an addendum database/table within it.
I would say for this project that comments are to be where it is at. Naturally you do not have to comment every "this is a below 8000h read" but if something is obfuscated then that would be nice for those to know. The AKAIO source which I will probably post up before long (or you can see the castlevania version of http://gbatemp.net/threads/which-flashcart-for-castlevania-por.343239/page-2#post-4559271 ) would be a good example, though this project would probably be less C style code than that.
Finally this is of some importance as ROM hacking tends to trip a lot of these protections and having a would be hacker have to fix it before getting on with their project, we see this sometimes in megadrive hacking but here at least game genie mastercodes can usually help out there (to say nothing of simpler nature and lower performance of things restricting what could be done).
To the best of my knowledge there are 6 types of AP and related oddities to tackle here.
Below 8000
Checksums
Save timing
Daigassou! Band Brothers DX unique save stamps.
DSi workarounds.
iQue games.
More on each of these at the end of the post.
I am not sure save type changing needs to be tackled, GTA was probably the most notable of them for this. The only real issue is likely to be one of the Daigassou! Band Brothers DX anyway.
All have examples of simple things and if my conversations with various flash cart makers and other relevant people are anything to go by some of the later games had nice obfuscated checks, checks buried deep in overlays, checks only loaded in the late game and "subtle effects" AP rather than crashes, whitescreens, demo modes, lockscreens and other obvious effects.
My personal preference is to work backwards from the last releases so people with lesser cards can benefit earlier but by its very nature the project would be open for anybody to have a look at.
As many that did the flash card dance this time around will be able to tell you there are "generic" patches or patching engines that work for some games (same publisher, dev, series, region dupes), I am not adverse to this though in the hypothetical database I would probably try to see if we can spin off a patch for every instance it might matter in but note it as "one of the ? AP variant games".
Projects of similar scope have been done before. Probably the most notable was the attempt to "train" every original xbox game which largely did what it set out to do.
The protections then.
Below 8000 hex.
Under normal conditions the data below 8000 hex on a game cart can not be read. Part of this is the header which is grabbed from memory and part of it is also the "secure area" but neither matters for this. There is also some extra stuff for later DS games where some area of the header previously thought 00 filled got some utterly useless to any hacker, flash cart maker or anybody really data (it technically includes the DSi/3ds signing so they can use a whitelist.... http://hackmii.com/2010/02/lawsuit-coming-in-3-2-1/ ). This is why you saw a lot of "redumps" over the last year or two and why nobody really gave a damn about them.
If the game tries to read it the cart actually returns according to a set pattern, see the "get data" command in
http://nocash.emubase.de/gbatek.htm#dscartridgeprotocol
No game ever used it for a "real" purpose and everything you see here was a check.
Certain emulators would note these reads in their error logs, you have options for static disassembly and more focused methods as well.
Checksums.
Here the binary, which should be static in most games as they were compiled (a few games used various forms of lua and other "scripting" languages, not sure what goes in the way of commercial emulators and dynamic recompilation on the DS), has a checksum done of it or possibly some section of it. This checksum would then be changed by games changing the save type (why savelist cards dodged it for a while), by cheats in general (the binary is in memory and cheats edit memory, many good cheat makers then edited a binary as part of a cheat) and by adding cheat engines/soft reset (why the special/clean/ghost/safe/whatever modes of various flash carts would prevent cheats, soft reset and other such functionality from happening).
Update. Apparently COP the recruit actually restored original binary sections that flash carts would have fiddled with during runtime.
Various checking methods though none quite so nice as below 8000 hex. Later games favoured heavy obfuscation, hundreds of checks (and not just a function call either), sometimes burying things in overlays only loaded in late game or in the ARM7 and other such niceties. That said the only integrity checking that should be going on is the usual anti cheat stuff (mirrored, complementary.... all the stuff you learn of once you get past the basic "how to have infinite lives/ammo" type cheats), save checksums, those needed by some network communications and the like.
Save timing.
Arguably the first AP method on the DS. Here you will often find the EEPROM chip or whatever chip was originally on the commercial release was slower and being a difference it can be used as a check. One instance I know of and it was a fairly obscure Japanese game which I have since forgotten the name of and did not note in my ROM hacking docs for some reason. It was patched later so if you go through the older AKAIO/WoodRPG changelogs (and the EZ5 ones which is where I learned of it) then you will see it.
Daigassou! Band Brothers DX unique save stamps.
DSi games and homebrew aside this is probably the only game to feature DLC give or take the "daily puzzles" on various versions of picross or how you might want to look at some of the pokemon bonuses.
Here you could download extra songs over and above what the original cart came with. I am not sure if this was a licensing thing or what. Either way the commercial release shipped with a 64Mbit flash chip for saves (flash 8 Mbit was the next step down though Warioware DIY and Jam with the Band had another type that variously gets termed NAND saves or Flash 256 Mbit). It is impractical for a commercial game release to make unique ROM versions for each game but saves are a different matter entirely so it seemed every save shipped had a unique marker which was sent to the DLC server.
All the DLC songs were dumped as far as I know and injection of songs (custom, DLC or otherwise) into the game was figured out early on.
Not sure what goes with the "Jam with the Band" which was roughly the European release of the game (it never made it to North America).
DSi workarounds.
Apparently the commonly used injection point for flash cart stuff was zeroed out by the games, probably of little real interest for this project then.
iQue games.
Including demos and a probably pointless proper there are a grand total of 7 of these and chances are if you see a Chinese game it is probably a hacked version of a Japanese game or similar (the Chinese translation scene is nothing if not prolific). China does get Nintendo hardware though a company/arrangement sees them branded as the iQue whatever and in this case it was the iQueDS. The hardware plays all the same games and does all the same things but Japanese on the main menu and in return it gets a reasonable line in simplified Chinese to use in pictochat. As games carry their own fonts this is basically it. However the few official games for some reason want to only be played on the iQue, the "protection" was a very simple firmware check and never anything obfuscated. http://nocash.emubase.de/gbatek.htm#dsfirmwareserialflashmemory has more on this.
For all the above binaries and overlays do have compression, it is a custom form of LZ usually called something like DS binary/overlay compression though technically/if speaking in a more general compression sense it is a type of BLZ (backwards LZ) and it is used on the ARM9 and its overlays (the ARM7 tends to be left alone). For the likes of AKAIO and other runtime things this is less problematic as things have to be decompressed to run after all. Encryption has never been seen in DS game binaries and data though.
