I probably should not start a new project right now but I thought I would lay the groundwork at least.
Most people doing anything on multiple DS roms will encounter the SDK formats soon enough ( http://www.romhacking.net/documents/469/ , http://llref.emutalk.net/docs/ , http://kiwi.ds.googlepages.com/sdat.html , http://kiwi.ds.googlepages.com/nsbmd.html , http://nvwr.googlecode.com/svn/trunk/libs/formats/ and so on) and they make a lot of DS hacking somewhat easier (presumably programming as well) but certain games have been observed to do things differently leading to the whole affair coming crashing down.
Now this is a well known problem in reverse engineering and indeed many of those that define proprietary formats will subtly shift the format which breaks reverse engineered parsers but I have yet to see anything resembling a kind of V1, v2, custom tweak analysis similar to the likes of the big reverse engineering projects other than maybe when people were looking at different region saves (not sure about v00 and v01 or early pre wifi and later wifi versions).
Monster tale appeared to have something extra in the SSEQ (tracker/sequence audio format common in SDAT audio which most DS roms use) files but I have not confirmed that http://gbatemp.net/index.php?showtopic=292661
Pokemon has done lots of this over the years with the most notable being the nameless narc files which led to narctool 0.1-p and I am told some of the graphics formats there ignore the layout information and have it defined somewhere else. Pokemon has a history of being a pain to hack so whether this falls under different formats, incomplete specification or obfuscation is probably open to debate.
Phoenix Wright was supposed to use some graphics formats differently as well ( http://gbatemp.net/topic/303529-tinke-072/page__st__225 ) or at least different to current implementation from tinke.
Sands of Destruction had some graphics formats that might not be standard http://gbatemp.net/topic/329423-unpacking-an-archive-in-a-ds-rom-iear/
NSBMD... I do not know if this is differences in formats or just that the format has not been completely reverse engineered but I am sure we have all seen the existing tools fall short. Certainly I have seen texture animations and similar done manually and the NSBCA/NSBTX formats ignored). I grabbed the old Metroid Prime Hunters model viewer the other day ( http://filetrip.net/nds-downloads/utilities/download-dsgraph-10-f29517.html ) and although popular thought has it as the precursor to NSBMD (certainly nsbmdtool appears to owe a nod to the graphviewer program for metroid and both are open source so comparing should be easy enough) I have not seen anybody confirm it.
NFTR (font format) had a few minor tweaks as time went on ( http://gbatemp.net/topic/105060-nftr-editor/page__st__135 ).
VX video format from ActImagine/mobiclip later became MODS and there appear to have been a couple of versions of this ( http://gbatemp.net/topic/125374-player-of-vx-file/ ) but the format itself has not really be reverse engineered yet.
BMG (a text format but one usually only seen in first and second party games or download play in my experience) I am not sure I have seen a complete reverse engineering of yet although unicode and fairly basic examples of the elementary pointer types does not leave a lot of room to change much.
Custom compression? As far as most common types are concerned (type 10, type 11, type 40....) most of it is nothing major (a different flag type, a different range....) if you know how the compression works in the first place. I might have some proper custom compression though (El Tigre Make my mule) but I do have to confirm that (it might just be "headerless").
Procyon Audio (you probably know it by the extension SAD)- not strictly an SDK format as far as I know but common enough to be noted. There is however a disparity between the luminous arc stuff and the professor layton version of the format which wins it a spot on the list.
ADX- ADX, AHX, ADH. Most of that is done but apparently there is a ADX2 format although that seems to be seen in the PSP rather than the DS.
I am not sure if I want to try relating the likes of NSBMD and (N)ARC to the GC ( http://hitmen.c02.at/files/yagcd/yagcd/chap15.html#sec15 ) and Wii equivalents at this point but that is a possible future direction. Going down that leads to comparing formats between games either on the same system (Super Princess Peach and New Super Mario Brothers probably want to be compared at some point if they have not already) or different systems (Mario Kart DS being a fairly big focus here on various levels). I am not really looking at non SDK formats myself but if you have an interesting one or want to compare games in a franchise/from a given developer (the ph/pb format used in Inazuma 11 changed between versions for example) do feel free to post an analysis.
I probably should note somewhere in this that either parts of larger files have been seen by themselves (the NTFP section of a palette was often seen by itself in earlier games and not in what would later be the NCLR palette format) or could be precursors to formats (same example from a moment ago probably applies).
Also if you have any good developer leftovers that explain a format and might give a clue there ( http://gbatemp.net/topic/320192-japanese-programming-madness/ has some examples and links to them and the SMAP file from Zoids Saga DS), we all love levels, audio and graphics that did not make the final cut but unless part of that is directly useful to reverse engineering (that is to say it is composed of primitives and example layouts in the case of a level) probably better to find another thread.
Anyone got anything worth noting in this arena?
Most people doing anything on multiple DS roms will encounter the SDK formats soon enough ( http://www.romhacking.net/documents/469/ , http://llref.emutalk.net/docs/ , http://kiwi.ds.googlepages.com/sdat.html , http://kiwi.ds.googlepages.com/nsbmd.html , http://nvwr.googlecode.com/svn/trunk/libs/formats/ and so on) and they make a lot of DS hacking somewhat easier (presumably programming as well) but certain games have been observed to do things differently leading to the whole affair coming crashing down.
Now this is a well known problem in reverse engineering and indeed many of those that define proprietary formats will subtly shift the format which breaks reverse engineered parsers but I have yet to see anything resembling a kind of V1, v2, custom tweak analysis similar to the likes of the big reverse engineering projects other than maybe when people were looking at different region saves (not sure about v00 and v01 or early pre wifi and later wifi versions).
Monster tale appeared to have something extra in the SSEQ (tracker/sequence audio format common in SDAT audio which most DS roms use) files but I have not confirmed that http://gbatemp.net/index.php?showtopic=292661
Pokemon has done lots of this over the years with the most notable being the nameless narc files which led to narctool 0.1-p and I am told some of the graphics formats there ignore the layout information and have it defined somewhere else. Pokemon has a history of being a pain to hack so whether this falls under different formats, incomplete specification or obfuscation is probably open to debate.
Phoenix Wright was supposed to use some graphics formats differently as well ( http://gbatemp.net/topic/303529-tinke-072/page__st__225 ) or at least different to current implementation from tinke.
Sands of Destruction had some graphics formats that might not be standard http://gbatemp.net/topic/329423-unpacking-an-archive-in-a-ds-rom-iear/
NSBMD... I do not know if this is differences in formats or just that the format has not been completely reverse engineered but I am sure we have all seen the existing tools fall short. Certainly I have seen texture animations and similar done manually and the NSBCA/NSBTX formats ignored). I grabbed the old Metroid Prime Hunters model viewer the other day ( http://filetrip.net/nds-downloads/utilities/download-dsgraph-10-f29517.html ) and although popular thought has it as the precursor to NSBMD (certainly nsbmdtool appears to owe a nod to the graphviewer program for metroid and both are open source so comparing should be easy enough) I have not seen anybody confirm it.
NFTR (font format) had a few minor tweaks as time went on ( http://gbatemp.net/topic/105060-nftr-editor/page__st__135 ).
VX video format from ActImagine/mobiclip later became MODS and there appear to have been a couple of versions of this ( http://gbatemp.net/topic/125374-player-of-vx-file/ ) but the format itself has not really be reverse engineered yet.
BMG (a text format but one usually only seen in first and second party games or download play in my experience) I am not sure I have seen a complete reverse engineering of yet although unicode and fairly basic examples of the elementary pointer types does not leave a lot of room to change much.
Custom compression? As far as most common types are concerned (type 10, type 11, type 40....) most of it is nothing major (a different flag type, a different range....) if you know how the compression works in the first place. I might have some proper custom compression though (El Tigre Make my mule) but I do have to confirm that (it might just be "headerless").
Procyon Audio (you probably know it by the extension SAD)- not strictly an SDK format as far as I know but common enough to be noted. There is however a disparity between the luminous arc stuff and the professor layton version of the format which wins it a spot on the list.
ADX- ADX, AHX, ADH. Most of that is done but apparently there is a ADX2 format although that seems to be seen in the PSP rather than the DS.
I am not sure if I want to try relating the likes of NSBMD and (N)ARC to the GC ( http://hitmen.c02.at/files/yagcd/yagcd/chap15.html#sec15 ) and Wii equivalents at this point but that is a possible future direction. Going down that leads to comparing formats between games either on the same system (Super Princess Peach and New Super Mario Brothers probably want to be compared at some point if they have not already) or different systems (Mario Kart DS being a fairly big focus here on various levels). I am not really looking at non SDK formats myself but if you have an interesting one or want to compare games in a franchise/from a given developer (the ph/pb format used in Inazuma 11 changed between versions for example) do feel free to post an analysis.
I probably should note somewhere in this that either parts of larger files have been seen by themselves (the NTFP section of a palette was often seen by itself in earlier games and not in what would later be the NCLR palette format) or could be precursors to formats (same example from a moment ago probably applies).
Also if you have any good developer leftovers that explain a format and might give a clue there ( http://gbatemp.net/topic/320192-japanese-programming-madness/ has some examples and links to them and the SMAP file from Zoids Saga DS), we all love levels, audio and graphics that did not make the final cut but unless part of that is directly useful to reverse engineering (that is to say it is composed of primitives and example layouts in the case of a level) probably better to find another thread.
Anyone got anything worth noting in this arena?