ROM Hack Extracting Assets From Spore Creatures

[Kapsey]

Hello! I'm looking to extract graphic assets (such as maps, models, sprites, etc) from Spore Creatures, as the game is not well documented.

I've managed to unpack the game, which has the following files:

The overlay folder contains files titled overlay_0000 through overlay_0006, which I'm not sure are relevant.

In the data folder are the following files. I was able to access the contents of the dwc folder using MKDSCM, and it seems to just be a bunch of stuff related to the Nintendo Wi-fi connection in the game.

My current guess is that the assets are stored in rom.bin and indexed with romFileNames.bin, but so far I've been completely unable to open either.

How would I go about extracting the graphic assets from these files? Please note that I'm a bit new to ROM hacking.

Thank you!

 

[FAST6191]

Rundown of what such things generally are and what I imagine they will be.

ARM7.bin. In commercial DS games this is basically a library file -- you can usually swap them with other games made around the same time and be just fine, indeed sometimes it was even a fix for certain flash carts.
ARM9.bin. In commercial DS games this is the main piece of the code that is responsible for the game.
Overlays (mostly for the ARM9). The DS does not have a lot of memory (about 4 megabytes give or take) which is has to store the binaries, all data about the state of the game, anything it is using at the time... and as the DS is also not a PC that has a nice OS to handle things then Nintendo made it such that a section of memory was set aside and then overlays could be loaded in and out at will.

Binaries in general, including the overlays, can contain anything that the game can need. It is generally considered bad form to do this as a programmer but with many DS games being afterthoughts by game companies then it happened anyway, indeed some games have thousands of overlays and every file within one.

Anyway
The folder DWC is usually a download play component (whatever they send to another DS that does not have a copy of the game to play a cut down version). Will typically have a file called utility.bin, others might have "[something].srl" instead. This is actually another DS ROM in its own right you can unpack with whatever you used to get this far. Sometimes it has useful info (many times the download play is a cut down version of the ROM that only has what it necessary) but other times it can be radically different.

sound_data.sdat is the standard name for the DS sound format (usually called SDAT). There are other formats but SDAT is in all but probably about 30 games in the whole DS library.

You would likely be right about rom.bin and romFileNames.bin. It has been seen on several occasions that rather than leave things as individual files like the ones you see there that devs put everything in another archive format/layer/whatever of their own design. Often names and locations of files within the bigger archive will be in a separate small file (easier to handle small files if you are only a 66MHz ARM9 processor with 4 megs of RAM) but it could be a red herring.
To that end you will probably want to figure out the format. This is usually not so bad to do -- most archive formats have some combo of file names, file locations, file sizes, any info about compression or subdirectories and not a lot else, many will even just be name and location or even just location (pity when it is just location as names can be quite helpful).
I normally look to see if the data in either the name file has locations, or the start of the big file has locations of files within it. You can either search for common formats, note their location and see if there is something that might correspond to that, or go the other way and look at what might be locations and then see if the values there match what looks like the start and end of files.
From there make a list of locations, use a tool like file cutter (https://web.archive.org/web/20170218180937/http://min.midco.net/cracker/filecutter.zip little command line tool popular around here that can will chop sections out of a file for your, quite useful in these sorts of scenarios, especially as you can use the list of locations and a spreadsheet to make a batch file, even more so if you don't care about getting the files back into the ROM) or make your own extraction program.

If there are nice known formats ( http://www.romhacking.net/documents/469/ ) then great, hopefully tools like tinke and crystaltile2 will make short work of them. If not then you get to figure out the formats yourself, usually they will be close to the hardware formats though (again 66MHz and 4 megs of RAM, no time to be converting files for no reason) http://problemkaputt.de/gbatek.htm http://members.iinet.net.au/~freeaxs/gbacomp/ http://www.coranac.com/tonc/text/video.htm and I guess the hacking guide stickied above/in my signature).

 

[Kapsey]

First of all, I really appreciate your thorough response! It has been super helpful in understanding the files.

In romFileNames.bin, the file names start around 00008EF0. I'm not sure what the data is above it. What would the locations look like in memory?

For an idea, this is the general format of the data above the name addresses.

And the name addresses are written out like this.

Thank you so much for your help!

Rundown of what such things generally are and what I imagine they will be.

ARM7.bin. In commercial DS games this is basically a library file -- you can usually swap them with other games made around the same time and be just fine, indeed sometimes it was even a fix for certain flash carts.
ARM9.bin. In commercial DS games this is the main piece of the code that is responsible for the game.
[...]

 

[FAST6191]

Locations might not be in memory but within the files themselves. In this case the numbers there look like they count upwards after a flip (in the shot there all those 8F followed by 90 and then 91... is usually a dead giveaway).
You flip things because of endianess -- you can read about the big endian vs little endian holy wars on your own time but suffice to say in computers some CPUs use big endian (X86, aka the PC, being the most notable) where others (ARM and many other things) use little endian. https://www.freecodecamp.org/news/what-is-endianness-big-endian-vs-little-endian/ . Consider it basically the computing equivalent of year-month-day, day-month-year and the got dropped on their head as a baby alternative of month-day-year (even if technically it does make sorting by month a bit easier in computers).


Added bonus the first number on the pointers list is 8EFC when flipped which is when the names in the second shot start. Quite often see that where the first entries or near the start is a length of the whole section (helps the DS navigate a bit if it does not have to find/guess where the end of the file name list starts and can just read the first byte, take that number, add it and go to the result, possibly before finding the name it wants and then the number that is in the list multiplied by however long the addresses are to find the matching address).
That said decoding a few more they are quite close together there so that might actually be pointers just for the names so you can split them up more easily that way, and decoding a few appears to follow that pattern.
Might be more pointers after the names, or there might be pointers at the start of the rom.bin (have certainly seen the names, which are not that essential for normal use in a game, be a separate file but the file locations be in the main file before in another DS game that did everything in one big archive).

Seeing ntft in there. It is not one of the more popular formats but it is ringing a bell. Does it change later in the list?

Pointers in general (you will probably meet them again when you start pulling out files) come in three main flavours on the DS, though there are a couple of others.
I usually describe them like the contents page of a book.

1) Standard.
Take the value of the pointer, start counting from the start of the file.
2) Offset.
Take the value of the pointer, start counting from some point in the file, usually the end of the header section/start of the section you care about. Continuing with the book analogy if you started counting physical pages but ignored that the numbering started later than that, with front matter using Roman numerals or something, then this is that.
3) Relative.
Take the value of the pointer, add it to the location of the pointer and you have the location. This is done as inside a CPU this can be an easier thing to do. You will tend to find this obvious if each pointer is an additional fixed distance "out" from where it should be (say pointer at address 0 is OK, address one is 4 bytes out, address 2 is 8 bytes, address 3 is 12... and wouldn't you know each pointer is 4 bytes apart).

The other two are
Sometimes it is just a length value so you get to add up everything before then. Might sound tedious but actually quite nice as the contents page analogy comes back -- in the numbered things above imagine what would happen had you ripped out a bunch of pages, or stuffed a bunch of new ones in and then went back to counting physical pages...
Sectors/pages. Sectors is more for hard drives and optical drives, pages for RAM and older consoles (where you might also meet it as banks). Anyway for optical drives then if you need to store however many gigabytes of files then you need a number that can address all that (32 bits is about 4 gigabytes, and is no coincidence that 32 bit Windows tops out at 4 gigs of RAM, single layer DVD is 4.35...) so instead things are broken into sectors instead. You say sector 500 and bytes 10 through 35 then you might waste some space, but when it is basically free nobody cares, waste bytes back in the NES days though and you get fired... probably from a cannon.

 

[Kapsey]

That said decoding a few more they are quite close together there so that might actually be pointers just for the names so you can split them up more easily that way, and decoding a few appears to follow that pattern.

I don't know if knowing this would have been helpful or not before I divided up the names, lmfao. I ended up just figuring out all the extensions and find/replacing them with newline characters at the end so I could put them into Excel.

Seeing ntft in there. It is not one of the more popular formats but it is ringing a bell. Does it change later in the list?

As I said above, I found all the extensions in splitting the names, so now I have a list!

They are: .ntft, .fizzdb, .pal, .sporeta, .leganim, .nsbmd, .gmb, .tfb, .2lz, .dat, .bsp, .nsbca, .nsbta, .lookup, .bin, .gab, .xdb

A lot of those are known DS file types, though a lot of them are also strange filetypes associated with things such as DS emulators (GMB), Apple II OS (TFB) and the Quake Engine (BSP)? I also have absolutely no idea what .sporeta is, but I'm assuming its unique to the game since it has the name in it.

Might be more pointers after the names, or there might be pointers at the start of the rom.bin (have certainly seen the names, which are not that essential for normal use in a game, be a separate file but the file locations be in the main file before in another DS game that did everything in one big archive).

I'll try and take a look. This is the part that is most out of my comfort zone; I'm not super familiar with hex, pointers, addresses and all that stuff. Do you know of any good tutorials or videos that would be a nice introduction for a beginner?

Thank you so much for your help, I really appreciate it!

 

[FAST6191]

If the file type is something so random as apple emulators or quake then I would usually assume coincidence.

As far as looking after the names section. Scroll down through the names, at some point they will end. What happens then? If it is the end of the file then oh well.

As far as possibly being in rom.bin then open that up in a hex editor. See what there is at the start. Chances are the ntft file, and most other files, will start with a so called magic stamp (or maybe in the case that of that file with LZ in the extension then 10 or 11 as that is usually an indicator of compression). To that end if it starts immediately with what looks like the ntft file then probably not pointers at the start, if it has a section about as long as the pointers in for the names in the file name file looked at earlier (going to be about the same length, or maybe double if the pointers are longer, and presumably the amount of files in the main rom.bin is the same as the number of names, technically does not have to be but it is rarely going to be too radically different).

As far as guides.
I do have
https://gbatemp.net/threads/gbatemp-rom-hacking-documentation-project-new-2016-edition-out.73394/
some like
http://gbatemp.net/topic/291274-the-ultimate-nintendo-ds-rom-hacking-guide/

http://www.romhacking.net/start/ has taught many.

https://doc.kodewerx.org/index.html is for cheats, and I would usually flank it with https://web.archive.org/web/20080309104350/http://etk.scener.org/?op=tutorial . Familiarity with cheats and the sorts of things seen in memory when finding them has helped some.

http://wiki.xentax.com/index.php/Game_File_Format_Central details a whole bunch of formats found in games, as does https://wiki.multimedia.cx/index.php?title=Category:Game_Formats

Hex is just a numbering system -- computers use binary underneath it all, as 3 digits in length is not very useful then 4 (which means 2 to the power 4 combinations, or 16 combinations) gets used. As normal numbers don't do 16 combinations without repeating then people picked A through F to make up the difference before it starts counting round again. I often find new hackers attribute some kind of special skill status to it when it really is not. Data representation, general approaches that programming dictates or at least makes the most efficient, other files observed on the system in the same game or others, basic operations (shift, rotate, all the boolean operations)... that is what allows hackers to make sense of what is in a hex editor, and hex editors are generally the least worst tool for exploring a file initially but by no means good. Or more generally a hex editor is somewhat like a word processor -- you can be a word processor savant but it is not going to help much in writing a novel.

ROM hacking video tutorials are a bit thin on the ground as nobody has really ever found a good way of conveying information via them.

 

[Kapsey]

Thank you so much for the info and resources! I'll be sure to look into them.

Also, I apologize for the delayed response! I was away from home for a while.

As far as looking after the names section. Scroll down through the names, at some point they will end. What happens then? If it is the end of the file then oh well.

Unfortunately, there's nothing in the file after the names end.

As far as possibly being in rom.bin then open that up in a hex editor. See what there is at the start. Chances are the ntft file, and most other files, will start with a so called magic stamp (or maybe in the case that of that file with LZ in the extension then 10 or 11 as that is usually an indicator of compression). To that end if it starts immediately with what looks like the ntft file then probably not pointers at the start, if it has a section about as long as the pointers in for the names in the file name file looked at earlier (going to be about the same length, or maybe double if the pointers are longer, and presumably the amount of files in the main rom.bin is the same as the number of names, technically does not have to be but it is rarely going to be too radically different).

The first thing in the rom.bin file is the pointer (8EFC) to the first entry in romFileNames.bin. Unfortunately, I have no idea what the data after it is. The only information I can find about NTFT files is that they're usually 4 or 8 bits, nothing about a magic stamp.

I'm at a loss for how rom.bin is organized. It seems like the pointers to the filenames do exist in it, but I have no idea if they're followed by the data, a pointer to it, or something that's beyond me.

The only way I could think to try and solve that puzzle was by looking for the only obvious data in rom.bin, which is for en_badwords.bin. Since it's also a bin file, it shows as a list of curse words written plainly in ASCII. I got the pointer for the name in romFileNames.bin, which is A7E802. The data seems to start in rom.bin at 0x517B38 (though I'm unsure if there's a header for the file). However, although I found a few references in rom.bin to the name pointer, I can't find anything nearby it that corresponds to where the data starts.

Here is a screenshot of the start of the data in rom.bin. At the start, it references the pointer to 1BTPatC1_cmp4.ntft (FC8E), but I don't know what the data following is.

This is the data around the start of en_badwords.bin. I highlighted where the data appears to start at 0x517B38. The pointer to the name in romFileNames.bin is A7E802. I have no idea if I should expect to find somewhere in either file that corresponds the name location to this data's location, but I haven't seen anything like that so far.