I have had this idea for a while but I have had increasingly little time to be able to get on with this so I could not kick it off as I might have liked to (mainly with a bunch of example hacks).
Still
<b>Thread of a thousand rom hacks</b>
The one sentence summary of this thread is “Contained within is information on the internals of DS roms”.
<b>Introduction</b>
In this thread you will find the results of many games being pulled apart and the formats that make them up being figured out as well as data regarding it all. If discussion of the thread and the like could be kept to a minimum saving this for the data itself that would be great. By all means if you see something you are curious about quote it query something.
Hopefully it should also serve as the oft requested example hack/project.
<b>Background</b>
The inspiration for it comes from several sources but there are two main ones
1)We saw the 5th anniversary of the first DS full commercial DS rom being dumped back in June and September saw the 5th anniversary being able to run commercial roms from sources other than commercial carts.
At the time many of us were still using GBA era flash carts with expensive NOR memory chips- a cart could easily cost around $100USD for 32 megabytes; even in relative terms that was only a few games- on the GBA the largest sets of roms were 32 megabytes and most were 8 or 16 megabytes not taking into account trimming. On the DS now it was worse; even among the first 100 roms released there were only 10 roms that were 16 megabytes in size.
One of the great feature of flash carts is the ability to have multiple games on one device but before that in the release of }{ain's NDSpatcher was a readme that included a method to get Castlevania to work on a 32 megabyte cart (it was very crude and quite literally involved chopping the rom in half or more to get it to fit), not long after that the homebrew tool “ndstool” (used to this day as the backend to DSBuff and Dslazy) was found to work with commercial roms (it could pull them apart and rebuild them and they would still work). Following this people would take DS roms and strip them of things to have them fit on these small carts (even those that had the space for a full rom would do it so as to be able to squeeze another game onto it).
While this provides a nice jumping off point for the start of DS rom hacking (SDAT is a very commonly used sound format on the DS- this was well known and exploited by rom rippers long before DJ Bouche and loveemu would sort out basic tools and specifications for the format) and indeed is what afforded my entry into the field it is not the point that I wanted to highlight.
So before you say thanks for the history lesson it should be pointed out that once basic batch scripts were provided for ndstool many of the people making these rips were not master hackers but simply flash cart users who wanted to get a few more games on their cart (if anyone wanted to read about it here is a/”the” thread I made in December 2005: <a href="http://ezflash.sosuke.com/viewtopic.php?f=10&t=457&start=0" target="_blank">http://ezflash.sosuke.com/viewtopic.php?f=...457&start=0</a> – while they are a bit more refined than the Castlevania hack they omit techniques as simple/obvious as relinking the “removed” file names- on the PSP this is a common occurrence and people will often take a small song and point all the other songs at it and in doing so remove all the other data from the disc to save space whereas on the DS (working with video now- see most DS games using SDAT) you might find the smallest video and copy that 5 times (at 3 megs a hit it might save 20 megs overall but still cost you 15 megs when 3 might do). Fastforward to today and a comment/impression I get when talking to those wanting to get into hacking or just starting out is that while they are not intimidated by the hackers who work with some of the big translations who can take a DS rom, figure out the obscure compression format used and then give the developers pointers on how to improve their decompression speed in the time it takes someone to have lunch (or if you prefer a more hacking oriented example the time it takes for them to figure out what a given file/directory or files does in the game) they have the idea that posting up a table for a rom is not worth the time.
This I feel is very wrong and any data that can be gathered about DS roms is useful- for instance take the SDAT format that was already mentioned. Originally it was but a few roms (less than 10 in all and less than 5 if you ignore region dupes) that were known not to use it, some people pulled apart a few roms that were fairly obscure (read not of great interest to hackers/translation crews) to find that the ADX/AHX format from cri ( <a href="http://www.cri-mw.com/product/lineup/audio/crivibe15/index.html" target="_blank">http://www.cri-mw.com/product/lineup/audio...be15/index.html</a> ) had appeared on the DS. Today nobody really knows but that number is somewhat higher (a quick scan of <a href="http://www.cri-mw.com/product/adoption/platform/ds.html" target="_blank">http://www.cri-mw.com/product/adoption/platform/ds.html</a> says 81 roms* or just over 1.5% of all DS roms have some CRI tech inside them including some region dupes- data just gathered for this very thread). If you look at the names it does include some big games but also some very small/obscure ones.
*there were 81 instances of “date of sale” when you copy and paste the text into a text editor and get it to count the instances.
2)A somewhat shorter reason but <a href="http://www.romhacking.net/forum/index.php/topic,8407.html" target="_blank">http://www.romhacking.net/forum/index.php/topic,8407.html</a> among many many others on the subject of rom hacking detail the little steps in pulling something apart and/or documenting a format. To draw and analogy to more traditional programming pursuits above all else it is often the case that just getting the code to compile/assemble/pack into something that can execute that causes problems to those just joining. For instance the DS has a generic header format <a href="http://llref.emutalk.net/nds_formats.htm#Generic" target="_blank">http://llref.emutalk.net/nds_formats.htm#Generic</a> and can be seen on that link and <a href="http://llref.emutalk.net/docs/" target="_blank">http://llref.emutalk.net/docs/</a> is also good used not only by nitroSDK formats but many ones altered or created anew by companies for their games and once a hacker knows this it can be quite easy for them to register it, use it and spit out a specification (or start of one) for the format. A simple action but one that can easily unravel an entire unknown format very quickly- fantastic but all too often these threads will start with the format pretty much all sorted as if by some near superhuman feat of mental agility when in fact it was a combination of a simple logical thought process and maybe a bit of experience.
I also have a hope that this will spark some more interest in areas of hacking not immediately to do with translation (translation is great and hacking would not be half of what it is today without it but it is far from the only field in rom hacking). Such hacks can be done quickly and without the need for a team- see the likes of the remodelling/move set alterations of super smash brothers on the wii and the editing of cars in various 360 racing titles.
<b>The “rules”, examples and suggestions-</b>
Usual forum rules* but on top of that in this thread only data about roms and the surrounding explanation of it all (that is to say if you wanted to contrast the midi format with the SSEQ format or one consoles image storing format with another that is fine). If you are adding to the knowledge of DS rom formats or operating methods your posts are welcome in this thread. Try to keep general discussion/"nice idea" posts to a minimum if you can (you can snag a placeholder if you like though).
*rules regarding excerpts from commercial games are such that generally you only post relevant data. There is scope for interpretation though- you probably could get away with posting an entire header for a file even if it is not strictly relevant. Your hex editor does have address numbers for a reason though so chopping portions out of a file are often not as deleterious as chopping out sections of a string and can even make the resulting post better/easier to read/follow.
I and I presume others are going to focus on DS roms but if you have a PSP, wii or 360 game you want to pull apart you can (something like <a href="http://gbatemp.net/t229989-a-bit-of-doom-2-xbla-hacking" target="_blank">http://gbatemp.net/t229989-a-bit-of-doom-2-xbla-hacking</a> or <a href="http://gbatemp.net/t72013-wii-decryption-tool-released?view=findpost&p=942490" target="_blank">http://gbatemp.net/t72013-wii-decryption-t...st&p=942490</a> * is more than welcome). Indeed as translation is not such a great focus on those there is a lot of work going into more regular game modification.
*I mainly linked that thread as it is a great example of what I want to happen here- a bunch of people just pulling things apart and documenting it (it would be about a month from that date until we could actually run any of those changes with the release of the “trucha” signing application and that was very much not anticipated to happen that quickly).
It is recognised that the person who does the work has done work but rom hacking as with most areas of technology is all about building on that which came before and to this end by posting it here you can fully expect someone else will come along then take the work what you have done and work it bit more (this also can include fixing something if you got it wrong). Afterwards the results of this thread will be made available for everyone to interpret however they like.
Failed/incomplete hacks- also good. People could see and fix a mistake you made- who has not had a program fail because someone did not include a bracket somewhere in it. Similarly absolute failure in science is very rare and history is replete with examples of unforeseen consequences to “failures”. For a hacking specific example the first versions of narctool (a tool designed to pull apart the DS format known as NARC and the compressed cousin in CARC) lacked support for subdirectories and nameless files (primarily as they were not in the files used to reverse engineer the specification- a constant concern of the hacker but it carries on anyway). In a sense narctool was then incomplete but few would have it not released in light of that.
While most of the thread has dealt with file formats you are more than welcome to detail the memory layout of a game- cheats can be made this way to do some very interesting things. You might also get some help if it turns out that the cheat required would actually be a more complex pointer or assembly grade cheat.
If you want to detail the save format of a game you can (save editing has been done many times on the DS for many games).
A halfway point between cheats and saves is savestate editing where a savestate/memory snapshot is edited in lieu of a “proper” save.
It could also have further implications with some of the more low level hacks- some of the SDAT hacks to change games are very small changes all in all (smaller than the area changed by some cheats even). Often the headers (or relevant sections thereof) are stored in ram for easy access/use in building instructions- that is to say for something like SDAT the file locations might be stored in ram and once running the game fetches the address from the ram rather than the rom- if this turns out to be the case you could change the sounds on a rom with a simple cheat.
On the subject of cheats and game saves- it is often easier to hack a game to ignore any save checks/hashing than it is to reimplement a check in standalone tool (better yet the hacked game might redo the hashing for you) so if you wanted to make a combination hack that is great too.
If you want to publish a tool/script/plugin you can. We got a very nice tool for pulling apart and reassembling roms the other day <a href="http://gbatemp.net/t280476-romulan-data-extractor-injector" target="_blank">http://gbatemp.net/t280476-romulan-data-extractor-injector</a>
If you want to detail and example hack/method (thinking something like <a href="http://crackerscrap.com/docs/sfchacktut.html" target="_blank">http://crackerscrap.com/docs/sfchacktut.html</a> or <a href="http://crackerscrap.com/docs/dshooking.html" target="_blank">http://crackerscrap.com/docs/dshooking.html</a> ) then that is fine to.
<a href="http://nocash.emubase.de/gbatek.htm" target="_blank">http://nocash.emubase.de/gbatek.htm</a> is a great document for the inner workings of the DS however in various points it can lack things or indeed not have been updated for new developments. Same with any other relevant technical document/file specification. If you want to pin out a controller or something like that then do so.
If you wanted to try a new technique or build one up you could- the GBA slot of the DS is mapped to memory which gives you a full 32 megabytes of read only memory to play with. Not of great use to many DS hacks given you can just extend files at will thanks to the file system but if you wanted to try something as a proof of concept there is nothing stopping you.
Equally if you wanted to publish a proof of concept hack to try and find a team that is OK too (do note though that translations are better served with other threads- this thread is set to be somewhat fast paced).
Many formats are already known- nothing stopping you from reworking one or providing another spin on it (don't like the spec described by/for a C programmer- do something about it).
Equally if you want to hack one game you can, if you want to hack a common format across many games you can, if you want to contrast a game series or region dupes you can. If you want to do a statistical analysis of a bunch of roms then you can do that as well.
If you want to publish/make a “full” hack you can*, if you want to detail a format you can do that as well
*the reasoning/thinking here is if I say detail the SDAT format and leave it at that you can come along and change your favourite game to always play or skip over a given song in a given section or something along those lines then publish that.
<b>The tools-</b>
Anything you want to use you can but if you want some suggestions. With the exception of some hex editors at the end pretty much everything is free and often even open source and there is always a free alternative in the tools I link up here. In the following list no real attention will be given to game specific tools for obvious reasons.
ndstool or some a way to pull DS roms apart/parse the nitro rom file system.
<a href="http://ezflash.sosuke.com/viewtopic.php?f=10&t=457&start=0" target="_blank">http://ezflash.sosuke.com/viewtopic.php?f=...457&start=0</a> (or search for DS buff and DS lazy)
Crystaltile2- this program is one of the best general purpose hacking programs available anywhere.
<a href="http://filetrip.net/f23649-CrystalTile2-2010-09-06.html" target="_blank">http://filetrip.net/f23649-CrystalTile2-2010-09-06.html</a>
A quick guide to the program can be found here: <a href="http://gbatemp.net/t73394-gbatemp-rom-hacking-documentation-project-wip?view=findpost&p=2641950" target="_blank">http://gbatemp.net/t73394-gbatemp-rom-hack...t&p=2641950</a> )
<a href="http://www.romhacking.net/forum/index.php/topic,8407.html" target="_blank">http://www.romhacking.net/forum/index.php/topic,8407.html</a> and <a href="http://llref.emutalk.net/downloads/consoleTool_v02b.zip" target="_blank">http://llref.emutalk.net/downloads/consoleTool_v02b.zip</a>
<a href="http://filetrip.net/f5658-NDS-Editor-0-1.html" target="_blank">http://filetrip.net/f5658-NDS-Editor-0-1.html</a>
ndsts can also be used for basic file extraction
<a href="http://www.no-intro.org/tools.htm" target="_blank">http://www.no-intro.org/tools.htm</a>
The DS is known to also have several archive formats that get stacked on top of the DS file system (indeed it could be argued many of the formats to be pulled apart are archives of some form/varying levels of specialisation). Some like NARC and SDAT have tools that can pull these archive types apart again to reveal the files inside them.
<a href="http://treeki.shacknet.nu/" target="_blank">http://treeki.shacknet.nu/</a> has a tool called narc explorer. An older program called narctool also exists in various forks <a href="http://members.cox.net/dexter0/DSTools/narctool.shtml" target="_blank">http://members.cox.net/dexter0/DSTools/narctool.shtml</a> and <a href="http://www.pipian.com/ierukana/hacking/ds_files.html" target="_blank">http://www.pipian.com/ierukana/hacking/ds_files.html</a>
Crystaltile2 has many formats inbuilt/able to be extracted.
SDAT <a href="http://www.4shared.com/file/68276816/8092229e/ndssndext_v04.html" target="_blank">http://www.4shared.com/file/68276816/80922...sndext_v04.html</a> and <a href="http://www.4shared.com/file/76520093/5a299676/VGMTrans.html" target="_blank">http://www.4shared.com/file/76520093/5a299676/VGMTrans.html</a> , <a href="http://hcs64.com/vgm_ripping.html" target="_blank">http://hcs64.com/vgm_ripping.html</a> and <a href="http://hcs64.com/files/VGMTrans_src_and_bin_9_29_09.zip" target="_blank">http://hcs64.com/files/VGMTrans_src_and_bin_9_29_09.zip</a> and crystaltile2 also features in this but not quite as well.
Known formats- in no way is this intended to be a complete or even largely complete list, it is just some examples.
<a href="http://www.pipian.com/ierukana/hacking.html" target="_blank">http://www.pipian.com/ierukana/hacking.html</a> (a great example site pertaining to pokemon games on the DS)
<a href="http://llref.emutalk.net/nds_formats.htm" target="_blank">http://llref.emutalk.net/nds_formats.htm</a> and a newer version <a href="http://llref.emutalk.net/docs/" target="_blank">http://llref.emutalk.net/docs/</a> (general formats and more)
<a href="http://kiwi.ds.googlepages.com/sdat.html" target="_blank">http://kiwi.ds.googlepages.com/sdat.html</a> (SDAT sound)
<a href="http://kiwi.ds.googlepages.com/nsbmd.html" target="_blank">http://kiwi.ds.googlepages.com/nsbmd.html</a> (DS 3d- the console tool from lowlines mentioned several times already has taken it much further than this document has gone and indeed has fixed some of the troubles along the way)
<a href="http://gbatemp.net/index.php?showtopic=60809&st=0&start=0" target="_blank">http://gbatemp.net/index.php?showtopic=608...t=0&start=0</a> (some work on Jump Ultimate Stars)
<a href="http://wiki.xentax.com/index.php/Game_File_Format_Central" target="_blank">http://wiki.xentax.com/index.php/Game_File_Format_Central</a> (a big site for many formats from many games across many platforms).
Compression is no longer the problem it once was. It is still an annoying extra step but it can be dealt with by anyone able to operate some tools rather than requiring someone familiar with assembly coding.
Several tools exist and it should be noted that many of the GBA tools will work for the DS as well.
<a href="http://gbatemp.net/t274472-codec-lzss-ds-released" target="_blank">http://gbatemp.net/t274472-codec-lzss-ds-released</a>
<a href="http://code.google.com/p/dsdecmp/" target="_blank">http://code.google.com/p/dsdecmp/</a>
<a href="http://filetrip.net/f11736-BatchLZ77-1-3.html" target="_blank">http://filetrip.net/f11736-BatchLZ77-1-3.html</a>
<a href="http://www.romhacking.net/utils/511/" target="_blank">http://www.romhacking.net/utils/511/</a>
<a href="http://www.romhacking.net/utils/495/" target="_blank">http://www.romhacking.net/utils/495/</a>
<a href="http://www.romhacking.net/utils/485/" target="_blank">http://www.romhacking.net/utils/485/</a>
<a href="http://gbadev.org/tools.php?showinfo=56" target="_blank">http://gbadev.org/tools.php?showinfo=56</a>
Again crystaltile2 has support for compression searching and extraction as well as quite good file system level detection (it will tell you when you parse the DS file system inside the program if it thinks the file is compressed and offer decompression options.
A guide to compression including the long standing tool “GBAcrusher” (great for GBA and DS BIOS compatible compression) can be found <a href="http://members.iinet.net.au/~freeaxs/gbacomp/" target="_blank">http://members.iinet.net.au/~freeaxs/gbacomp/</a>
a tile editor- tiled2002 and crystaltile2 are the big 2 DS tile editors
<a href="http://home.arcor.de/minako.aino/TilEd2002/" target="_blank">http://home.arcor.de/minako.aino/TilEd2002/</a> (GBA 4bpp and GBA8bpp correspond to the DS 4bpp and 8bpp)
emulator- desmume has grown into a very capable hacking grade emulator and unlike no$gba debug is actually free/available at present. Granted no$gba is still a bit ahead of desmume in some areas but not that many.
Such things have been taken quite far but the cheat making contingent as opposed to the more regular rom hacking crowd so if you are not so familiar with some of the tools/methods or want more check out:
<a href="http://cheats.gbatemp.net/forum/tutorials/" target="_blank">http://cheats.gbatemp.net/forum/tutorials/</a>
<a href="http://desmume.org/download/" target="_blank">http://desmume.org/download/</a>
This leads onto assembly for which I direct you to:
<a href="http://crackerscrap.com/docs/dshooking.html" target="_blank">http://crackerscrap.com/docs/dshooking.html</a> as a starter of sorts. Be aware roms can have compressed binaries and overlays- crystaltile2 can handle the compression along with some of the other compression programs above (mainly ones aimed at the DS)
Disassemblers are available in emulators like desmume, no$gba debug if you have it has some great abilities in this area beyond disassembly and into full on debugging.
Crystaltile2 has some disassembly features
<a href="http://hp.vector.co.jp/authors/VA018359/nds/ndshack.html" target="_blank">http://hp.vector.co.jp/authors/VA018359/nds/ndshack.html</a> (ndsdis2) is an older disassembler.
<a href="http://www.openrce.org/downloads/browse/IDA_Plugins" target="_blank">http://www.openrce.org/downloads/browse/IDA_Plugins</a> has some plugins for IDA (a high end disassembler used by many people outside of DS/consoles and into fields like security- mainly focused on the paid for version though)
Script display/extraction is an odd issue (romhacking has moved to game specific stuff for the most part) but tables have not gone out of style yet.
A relative searcher is part of that:
Again crystaltile2 features such a function/ability.
<a href="http://www.romhacking.net/utils/513/" target="_blank">http://www.romhacking.net/utils/513/</a> (monkey moore) is a great little tool for such things and can even make tables.
Hex editors- you can do anything with a hex editor but the general idea of all the tools that preceded this is to make things easier.
Crystaltile2 has a hex editor inbuilt with many nice hacking features like table support (including built in shiftJIS and others) and the ability to switch between the tile editor and hex editor (and other areas of the program) as well as DS file system support (you can jump to files without having to extract them).
<a href="http://mh-nexus.de/en/hxd/" target="_blank">http://mh-nexus.de/en/hxd/</a>
<a href="http://www.x-ways.net/winhex/index-m.html" target="_blank">http://www.x-ways.net/winhex/index-m.html</a> (has a paid for version and a free one)
<a href="http://www.hexworkshop.com/" target="_blank">http://www.hexworkshop.com/</a> (paid for but used by many rom hackers owing to the features it has)
<a href="http://www.romhacking.net/?category=13&Platform=&game=&author=&os=&level=&perpage=50&page=utilities&utilsearch=Go&title=&desc=" target="_blank">http://www.romhacking.net/?category=13&...itle=&desc=</a> has many hacking specific hex editors (ones able to support tables
<a href="http://www.chmaas.handshake.de/delphi/freeware/xvi32/xvi32.htm" target="_blank">http://www.chmaas.handshake.de/delphi/free...xvi32/xvi32.htm</a> (a free editor used in many guides, not so many features but a solid editor none the less).
General hacking forum niceties/BBcode
Most of you are familiar with forums but hacking work like this is subtly different
To keep your hex editor excerpt from being messed with by the forum it is generally best you stick it in a code section
There is also a secondary code section (that allows a bit of formatting as well) called a codebox- this is like a code section but scrolls inside the page itself
pictures- gbatemp has a picture service <a href="http://gbatemp.net/?act=module&module=pix" target="_blank">http://gbatemp.net/?act=module&module=pix</a> that you can put your files on and share with anyone on the internet.
<a href="http://filetrip.net/" target="_blank">http://filetrip.net/</a> is available for tools but for general single files pertaining to rom hacking it is best you use another site. You can use one of the big file hosters if you want but places like 4shared.com have a great way to store files of your choosing and maintain some control over it all.
<a href="http://gbatemp.net/?act=module&module=ws" target="_blank">http://gbatemp.net/?act=module&module=ws</a> is good for storing chunks of data if you need to.
All that over onto the thread itself.
Still
<b>Thread of a thousand rom hacks</b>
The one sentence summary of this thread is “Contained within is information on the internals of DS roms”.
<b>Introduction</b>
In this thread you will find the results of many games being pulled apart and the formats that make them up being figured out as well as data regarding it all. If discussion of the thread and the like could be kept to a minimum saving this for the data itself that would be great. By all means if you see something you are curious about quote it query something.
Hopefully it should also serve as the oft requested example hack/project.
<b>Background</b>
The inspiration for it comes from several sources but there are two main ones
1)We saw the 5th anniversary of the first DS full commercial DS rom being dumped back in June and September saw the 5th anniversary being able to run commercial roms from sources other than commercial carts.
At the time many of us were still using GBA era flash carts with expensive NOR memory chips- a cart could easily cost around $100USD for 32 megabytes; even in relative terms that was only a few games- on the GBA the largest sets of roms were 32 megabytes and most were 8 or 16 megabytes not taking into account trimming. On the DS now it was worse; even among the first 100 roms released there were only 10 roms that were 16 megabytes in size.
One of the great feature of flash carts is the ability to have multiple games on one device but before that in the release of }{ain's NDSpatcher was a readme that included a method to get Castlevania to work on a 32 megabyte cart (it was very crude and quite literally involved chopping the rom in half or more to get it to fit), not long after that the homebrew tool “ndstool” (used to this day as the backend to DSBuff and Dslazy) was found to work with commercial roms (it could pull them apart and rebuild them and they would still work). Following this people would take DS roms and strip them of things to have them fit on these small carts (even those that had the space for a full rom would do it so as to be able to squeeze another game onto it).
While this provides a nice jumping off point for the start of DS rom hacking (SDAT is a very commonly used sound format on the DS- this was well known and exploited by rom rippers long before DJ Bouche and loveemu would sort out basic tools and specifications for the format) and indeed is what afforded my entry into the field it is not the point that I wanted to highlight.
So before you say thanks for the history lesson it should be pointed out that once basic batch scripts were provided for ndstool many of the people making these rips were not master hackers but simply flash cart users who wanted to get a few more games on their cart (if anyone wanted to read about it here is a/”the” thread I made in December 2005: <a href="http://ezflash.sosuke.com/viewtopic.php?f=10&t=457&start=0" target="_blank">http://ezflash.sosuke.com/viewtopic.php?f=...457&start=0</a> – while they are a bit more refined than the Castlevania hack they omit techniques as simple/obvious as relinking the “removed” file names- on the PSP this is a common occurrence and people will often take a small song and point all the other songs at it and in doing so remove all the other data from the disc to save space whereas on the DS (working with video now- see most DS games using SDAT) you might find the smallest video and copy that 5 times (at 3 megs a hit it might save 20 megs overall but still cost you 15 megs when 3 might do). Fastforward to today and a comment/impression I get when talking to those wanting to get into hacking or just starting out is that while they are not intimidated by the hackers who work with some of the big translations who can take a DS rom, figure out the obscure compression format used and then give the developers pointers on how to improve their decompression speed in the time it takes someone to have lunch (or if you prefer a more hacking oriented example the time it takes for them to figure out what a given file/directory or files does in the game) they have the idea that posting up a table for a rom is not worth the time.
This I feel is very wrong and any data that can be gathered about DS roms is useful- for instance take the SDAT format that was already mentioned. Originally it was but a few roms (less than 10 in all and less than 5 if you ignore region dupes) that were known not to use it, some people pulled apart a few roms that were fairly obscure (read not of great interest to hackers/translation crews) to find that the ADX/AHX format from cri ( <a href="http://www.cri-mw.com/product/lineup/audio/crivibe15/index.html" target="_blank">http://www.cri-mw.com/product/lineup/audio...be15/index.html</a> ) had appeared on the DS. Today nobody really knows but that number is somewhat higher (a quick scan of <a href="http://www.cri-mw.com/product/adoption/platform/ds.html" target="_blank">http://www.cri-mw.com/product/adoption/platform/ds.html</a> says 81 roms* or just over 1.5% of all DS roms have some CRI tech inside them including some region dupes- data just gathered for this very thread). If you look at the names it does include some big games but also some very small/obscure ones.
*there were 81 instances of “date of sale” when you copy and paste the text into a text editor and get it to count the instances.
2)A somewhat shorter reason but <a href="http://www.romhacking.net/forum/index.php/topic,8407.html" target="_blank">http://www.romhacking.net/forum/index.php/topic,8407.html</a> among many many others on the subject of rom hacking detail the little steps in pulling something apart and/or documenting a format. To draw and analogy to more traditional programming pursuits above all else it is often the case that just getting the code to compile/assemble/pack into something that can execute that causes problems to those just joining. For instance the DS has a generic header format <a href="http://llref.emutalk.net/nds_formats.htm#Generic" target="_blank">http://llref.emutalk.net/nds_formats.htm#Generic</a> and can be seen on that link and <a href="http://llref.emutalk.net/docs/" target="_blank">http://llref.emutalk.net/docs/</a> is also good used not only by nitroSDK formats but many ones altered or created anew by companies for their games and once a hacker knows this it can be quite easy for them to register it, use it and spit out a specification (or start of one) for the format. A simple action but one that can easily unravel an entire unknown format very quickly- fantastic but all too often these threads will start with the format pretty much all sorted as if by some near superhuman feat of mental agility when in fact it was a combination of a simple logical thought process and maybe a bit of experience.
I also have a hope that this will spark some more interest in areas of hacking not immediately to do with translation (translation is great and hacking would not be half of what it is today without it but it is far from the only field in rom hacking). Such hacks can be done quickly and without the need for a team- see the likes of the remodelling/move set alterations of super smash brothers on the wii and the editing of cars in various 360 racing titles.
<b>The “rules”, examples and suggestions-</b>
Usual forum rules* but on top of that in this thread only data about roms and the surrounding explanation of it all (that is to say if you wanted to contrast the midi format with the SSEQ format or one consoles image storing format with another that is fine). If you are adding to the knowledge of DS rom formats or operating methods your posts are welcome in this thread. Try to keep general discussion/"nice idea" posts to a minimum if you can (you can snag a placeholder if you like though).
*rules regarding excerpts from commercial games are such that generally you only post relevant data. There is scope for interpretation though- you probably could get away with posting an entire header for a file even if it is not strictly relevant. Your hex editor does have address numbers for a reason though so chopping portions out of a file are often not as deleterious as chopping out sections of a string and can even make the resulting post better/easier to read/follow.
I and I presume others are going to focus on DS roms but if you have a PSP, wii or 360 game you want to pull apart you can (something like <a href="http://gbatemp.net/t229989-a-bit-of-doom-2-xbla-hacking" target="_blank">http://gbatemp.net/t229989-a-bit-of-doom-2-xbla-hacking</a> or <a href="http://gbatemp.net/t72013-wii-decryption-tool-released?view=findpost&p=942490" target="_blank">http://gbatemp.net/t72013-wii-decryption-t...st&p=942490</a> * is more than welcome). Indeed as translation is not such a great focus on those there is a lot of work going into more regular game modification.
*I mainly linked that thread as it is a great example of what I want to happen here- a bunch of people just pulling things apart and documenting it (it would be about a month from that date until we could actually run any of those changes with the release of the “trucha” signing application and that was very much not anticipated to happen that quickly).
It is recognised that the person who does the work has done work but rom hacking as with most areas of technology is all about building on that which came before and to this end by posting it here you can fully expect someone else will come along then take the work what you have done and work it bit more (this also can include fixing something if you got it wrong). Afterwards the results of this thread will be made available for everyone to interpret however they like.
Failed/incomplete hacks- also good. People could see and fix a mistake you made- who has not had a program fail because someone did not include a bracket somewhere in it. Similarly absolute failure in science is very rare and history is replete with examples of unforeseen consequences to “failures”. For a hacking specific example the first versions of narctool (a tool designed to pull apart the DS format known as NARC and the compressed cousin in CARC) lacked support for subdirectories and nameless files (primarily as they were not in the files used to reverse engineer the specification- a constant concern of the hacker but it carries on anyway). In a sense narctool was then incomplete but few would have it not released in light of that.
While most of the thread has dealt with file formats you are more than welcome to detail the memory layout of a game- cheats can be made this way to do some very interesting things. You might also get some help if it turns out that the cheat required would actually be a more complex pointer or assembly grade cheat.
If you want to detail the save format of a game you can (save editing has been done many times on the DS for many games).
A halfway point between cheats and saves is savestate editing where a savestate/memory snapshot is edited in lieu of a “proper” save.
It could also have further implications with some of the more low level hacks- some of the SDAT hacks to change games are very small changes all in all (smaller than the area changed by some cheats even). Often the headers (or relevant sections thereof) are stored in ram for easy access/use in building instructions- that is to say for something like SDAT the file locations might be stored in ram and once running the game fetches the address from the ram rather than the rom- if this turns out to be the case you could change the sounds on a rom with a simple cheat.
On the subject of cheats and game saves- it is often easier to hack a game to ignore any save checks/hashing than it is to reimplement a check in standalone tool (better yet the hacked game might redo the hashing for you) so if you wanted to make a combination hack that is great too.
If you want to publish a tool/script/plugin you can. We got a very nice tool for pulling apart and reassembling roms the other day <a href="http://gbatemp.net/t280476-romulan-data-extractor-injector" target="_blank">http://gbatemp.net/t280476-romulan-data-extractor-injector</a>
If you want to detail and example hack/method (thinking something like <a href="http://crackerscrap.com/docs/sfchacktut.html" target="_blank">http://crackerscrap.com/docs/sfchacktut.html</a> or <a href="http://crackerscrap.com/docs/dshooking.html" target="_blank">http://crackerscrap.com/docs/dshooking.html</a> ) then that is fine to.
<a href="http://nocash.emubase.de/gbatek.htm" target="_blank">http://nocash.emubase.de/gbatek.htm</a> is a great document for the inner workings of the DS however in various points it can lack things or indeed not have been updated for new developments. Same with any other relevant technical document/file specification. If you want to pin out a controller or something like that then do so.
If you wanted to try a new technique or build one up you could- the GBA slot of the DS is mapped to memory which gives you a full 32 megabytes of read only memory to play with. Not of great use to many DS hacks given you can just extend files at will thanks to the file system but if you wanted to try something as a proof of concept there is nothing stopping you.
Equally if you wanted to publish a proof of concept hack to try and find a team that is OK too (do note though that translations are better served with other threads- this thread is set to be somewhat fast paced).
Many formats are already known- nothing stopping you from reworking one or providing another spin on it (don't like the spec described by/for a C programmer- do something about it).
Equally if you want to hack one game you can, if you want to hack a common format across many games you can, if you want to contrast a game series or region dupes you can. If you want to do a statistical analysis of a bunch of roms then you can do that as well.
If you want to publish/make a “full” hack you can*, if you want to detail a format you can do that as well
*the reasoning/thinking here is if I say detail the SDAT format and leave it at that you can come along and change your favourite game to always play or skip over a given song in a given section or something along those lines then publish that.
<b>The tools-</b>
Anything you want to use you can but if you want some suggestions. With the exception of some hex editors at the end pretty much everything is free and often even open source and there is always a free alternative in the tools I link up here. In the following list no real attention will be given to game specific tools for obvious reasons.
ndstool or some a way to pull DS roms apart/parse the nitro rom file system.
<a href="http://ezflash.sosuke.com/viewtopic.php?f=10&t=457&start=0" target="_blank">http://ezflash.sosuke.com/viewtopic.php?f=...457&start=0</a> (or search for DS buff and DS lazy)
Crystaltile2- this program is one of the best general purpose hacking programs available anywhere.
<a href="http://filetrip.net/f23649-CrystalTile2-2010-09-06.html" target="_blank">http://filetrip.net/f23649-CrystalTile2-2010-09-06.html</a>
A quick guide to the program can be found here: <a href="http://gbatemp.net/t73394-gbatemp-rom-hacking-documentation-project-wip?view=findpost&p=2641950" target="_blank">http://gbatemp.net/t73394-gbatemp-rom-hack...t&p=2641950</a> )
<a href="http://www.romhacking.net/forum/index.php/topic,8407.html" target="_blank">http://www.romhacking.net/forum/index.php/topic,8407.html</a> and <a href="http://llref.emutalk.net/downloads/consoleTool_v02b.zip" target="_blank">http://llref.emutalk.net/downloads/consoleTool_v02b.zip</a>
<a href="http://filetrip.net/f5658-NDS-Editor-0-1.html" target="_blank">http://filetrip.net/f5658-NDS-Editor-0-1.html</a>
ndsts can also be used for basic file extraction
<a href="http://www.no-intro.org/tools.htm" target="_blank">http://www.no-intro.org/tools.htm</a>
The DS is known to also have several archive formats that get stacked on top of the DS file system (indeed it could be argued many of the formats to be pulled apart are archives of some form/varying levels of specialisation). Some like NARC and SDAT have tools that can pull these archive types apart again to reveal the files inside them.
<a href="http://treeki.shacknet.nu/" target="_blank">http://treeki.shacknet.nu/</a> has a tool called narc explorer. An older program called narctool also exists in various forks <a href="http://members.cox.net/dexter0/DSTools/narctool.shtml" target="_blank">http://members.cox.net/dexter0/DSTools/narctool.shtml</a> and <a href="http://www.pipian.com/ierukana/hacking/ds_files.html" target="_blank">http://www.pipian.com/ierukana/hacking/ds_files.html</a>
Crystaltile2 has many formats inbuilt/able to be extracted.
SDAT <a href="http://www.4shared.com/file/68276816/8092229e/ndssndext_v04.html" target="_blank">http://www.4shared.com/file/68276816/80922...sndext_v04.html</a> and <a href="http://www.4shared.com/file/76520093/5a299676/VGMTrans.html" target="_blank">http://www.4shared.com/file/76520093/5a299676/VGMTrans.html</a> , <a href="http://hcs64.com/vgm_ripping.html" target="_blank">http://hcs64.com/vgm_ripping.html</a> and <a href="http://hcs64.com/files/VGMTrans_src_and_bin_9_29_09.zip" target="_blank">http://hcs64.com/files/VGMTrans_src_and_bin_9_29_09.zip</a> and crystaltile2 also features in this but not quite as well.
Known formats- in no way is this intended to be a complete or even largely complete list, it is just some examples.
<a href="http://www.pipian.com/ierukana/hacking.html" target="_blank">http://www.pipian.com/ierukana/hacking.html</a> (a great example site pertaining to pokemon games on the DS)
<a href="http://llref.emutalk.net/nds_formats.htm" target="_blank">http://llref.emutalk.net/nds_formats.htm</a> and a newer version <a href="http://llref.emutalk.net/docs/" target="_blank">http://llref.emutalk.net/docs/</a> (general formats and more)
<a href="http://kiwi.ds.googlepages.com/sdat.html" target="_blank">http://kiwi.ds.googlepages.com/sdat.html</a> (SDAT sound)
<a href="http://kiwi.ds.googlepages.com/nsbmd.html" target="_blank">http://kiwi.ds.googlepages.com/nsbmd.html</a> (DS 3d- the console tool from lowlines mentioned several times already has taken it much further than this document has gone and indeed has fixed some of the troubles along the way)
<a href="http://gbatemp.net/index.php?showtopic=60809&st=0&start=0" target="_blank">http://gbatemp.net/index.php?showtopic=608...t=0&start=0</a> (some work on Jump Ultimate Stars)
<a href="http://wiki.xentax.com/index.php/Game_File_Format_Central" target="_blank">http://wiki.xentax.com/index.php/Game_File_Format_Central</a> (a big site for many formats from many games across many platforms).
Compression is no longer the problem it once was. It is still an annoying extra step but it can be dealt with by anyone able to operate some tools rather than requiring someone familiar with assembly coding.
Several tools exist and it should be noted that many of the GBA tools will work for the DS as well.
<a href="http://gbatemp.net/t274472-codec-lzss-ds-released" target="_blank">http://gbatemp.net/t274472-codec-lzss-ds-released</a>
<a href="http://code.google.com/p/dsdecmp/" target="_blank">http://code.google.com/p/dsdecmp/</a>
<a href="http://filetrip.net/f11736-BatchLZ77-1-3.html" target="_blank">http://filetrip.net/f11736-BatchLZ77-1-3.html</a>
<a href="http://www.romhacking.net/utils/511/" target="_blank">http://www.romhacking.net/utils/511/</a>
<a href="http://www.romhacking.net/utils/495/" target="_blank">http://www.romhacking.net/utils/495/</a>
<a href="http://www.romhacking.net/utils/485/" target="_blank">http://www.romhacking.net/utils/485/</a>
<a href="http://gbadev.org/tools.php?showinfo=56" target="_blank">http://gbadev.org/tools.php?showinfo=56</a>
Again crystaltile2 has support for compression searching and extraction as well as quite good file system level detection (it will tell you when you parse the DS file system inside the program if it thinks the file is compressed and offer decompression options.
A guide to compression including the long standing tool “GBAcrusher” (great for GBA and DS BIOS compatible compression) can be found <a href="http://members.iinet.net.au/~freeaxs/gbacomp/" target="_blank">http://members.iinet.net.au/~freeaxs/gbacomp/</a>
a tile editor- tiled2002 and crystaltile2 are the big 2 DS tile editors
<a href="http://home.arcor.de/minako.aino/TilEd2002/" target="_blank">http://home.arcor.de/minako.aino/TilEd2002/</a> (GBA 4bpp and GBA8bpp correspond to the DS 4bpp and 8bpp)
emulator- desmume has grown into a very capable hacking grade emulator and unlike no$gba debug is actually free/available at present. Granted no$gba is still a bit ahead of desmume in some areas but not that many.
Such things have been taken quite far but the cheat making contingent as opposed to the more regular rom hacking crowd so if you are not so familiar with some of the tools/methods or want more check out:
<a href="http://cheats.gbatemp.net/forum/tutorials/" target="_blank">http://cheats.gbatemp.net/forum/tutorials/</a>
<a href="http://desmume.org/download/" target="_blank">http://desmume.org/download/</a>
This leads onto assembly for which I direct you to:
<a href="http://crackerscrap.com/docs/dshooking.html" target="_blank">http://crackerscrap.com/docs/dshooking.html</a> as a starter of sorts. Be aware roms can have compressed binaries and overlays- crystaltile2 can handle the compression along with some of the other compression programs above (mainly ones aimed at the DS)
Disassemblers are available in emulators like desmume, no$gba debug if you have it has some great abilities in this area beyond disassembly and into full on debugging.
Crystaltile2 has some disassembly features
<a href="http://hp.vector.co.jp/authors/VA018359/nds/ndshack.html" target="_blank">http://hp.vector.co.jp/authors/VA018359/nds/ndshack.html</a> (ndsdis2) is an older disassembler.
<a href="http://www.openrce.org/downloads/browse/IDA_Plugins" target="_blank">http://www.openrce.org/downloads/browse/IDA_Plugins</a> has some plugins for IDA (a high end disassembler used by many people outside of DS/consoles and into fields like security- mainly focused on the paid for version though)
Script display/extraction is an odd issue (romhacking has moved to game specific stuff for the most part) but tables have not gone out of style yet.
A relative searcher is part of that:
Again crystaltile2 features such a function/ability.
<a href="http://www.romhacking.net/utils/513/" target="_blank">http://www.romhacking.net/utils/513/</a> (monkey moore) is a great little tool for such things and can even make tables.
Hex editors- you can do anything with a hex editor but the general idea of all the tools that preceded this is to make things easier.
Crystaltile2 has a hex editor inbuilt with many nice hacking features like table support (including built in shiftJIS and others) and the ability to switch between the tile editor and hex editor (and other areas of the program) as well as DS file system support (you can jump to files without having to extract them).
<a href="http://mh-nexus.de/en/hxd/" target="_blank">http://mh-nexus.de/en/hxd/</a>
<a href="http://www.x-ways.net/winhex/index-m.html" target="_blank">http://www.x-ways.net/winhex/index-m.html</a> (has a paid for version and a free one)
<a href="http://www.hexworkshop.com/" target="_blank">http://www.hexworkshop.com/</a> (paid for but used by many rom hackers owing to the features it has)
<a href="http://www.romhacking.net/?category=13&Platform=&game=&author=&os=&level=&perpage=50&page=utilities&utilsearch=Go&title=&desc=" target="_blank">http://www.romhacking.net/?category=13&...itle=&desc=</a> has many hacking specific hex editors (ones able to support tables
<a href="http://www.chmaas.handshake.de/delphi/freeware/xvi32/xvi32.htm" target="_blank">http://www.chmaas.handshake.de/delphi/free...xvi32/xvi32.htm</a> (a free editor used in many guides, not so many features but a solid editor none the less).
General hacking forum niceties/BBcode
Most of you are familiar with forums but hacking work like this is subtly different
To keep your hex editor excerpt from being messed with by the forum it is generally best you stick it in a code section
There is also a secondary code section (that allows a bit of formatting as well) called a codebox- this is like a code section but scrolls inside the page itself
pictures- gbatemp has a picture service <a href="http://gbatemp.net/?act=module&module=pix" target="_blank">http://gbatemp.net/?act=module&module=pix</a> that you can put your files on and share with anyone on the internet.
<a href="http://filetrip.net/" target="_blank">http://filetrip.net/</a> is available for tools but for general single files pertaining to rom hacking it is best you use another site. You can use one of the big file hosters if you want but places like 4shared.com have a great way to store files of your choosing and maintain some control over it all.
<a href="http://gbatemp.net/?act=module&module=ws" target="_blank">http://gbatemp.net/?act=module&module=ws</a> is good for storing chunks of data if you need to.
All that over onto the thread itself.
