A little bit of 3ds ROM hacking

Discussion in '3DS - Flashcards & Custom Firmwares' started by FAST6191, Aug 27, 2012.

Aug 27, 2012
  1. FAST6191
    OP

    Reporter FAST6191 Techromancer

    pip
    Joined:
    Nov 21, 2005
    Messages:
    21,738
    Country:
    United Kingdom
    Following a conversation the other day about developer games and their using different encryption I set off down the internet highway in an attempt to procure said developer games and after a cheesy montage sequence my reward was a copy of the files (sadly no binary but that looks to be kept somewhat further apart) from the pyramids demo in plaintext. I have only looked at it for a few hours thus far but it appears to be similar enough to the DS to allow people to jump across yet it will probably be hex editors for the time being.

    File name listing

    Many things of interest in there, the basic TAR and have a look did not net anything of great interest but what looks like the classic LZ fingerprints are there (a bit more on that later) which probably made for that result.

    Sample from overlay.bcsklaX
    Code:
    0000000000 1180 1100 0043 4746 58FF FE14 0000 0000 .....CGFX.......
    0000000010 0005 8011 0000 4101 2009 4441 5441 6C20 ......A. .DATAl 
    0000000020 0BD0 0370 0130 5334 0220 4E44 4943 5443 ...p.0S4. NDICTC
    0000000030 2C60 3FFF FFFF FF30 9B70 9541 3660 5994 ,`?....0.p.A6`Y.
    0000000040 1000 0004 20BD 0943 414E 4D20 C406 8420 .... ..CANM ... 
    0000000050 0F62 8820 1350 2BC8 4202 20D9 0CBA E053 .b. .P+.B. ....S
    0000000060 3C20 ED30 17F0 533D 40F2 0234 005A 2053 < .0..S=@..4.Z S
    0000000070 3111 3A40 BB02 0041 5320 6318 0800 0028 1.:@...AS c....(
    0000000080 2125 5F3E 206F 3A20 7331 3020 12C0 5B51 !%_> o: s10 ..[Q
    0000000090 3120 C842 7139 B254 343F 3B00 6E7C 3DFD 1 .Bq9.T4?;.n|=.
    00000000A0 6D7C 3D86 C020 0B31 644D 5634 3F93 4716 m|=.. .1dMV4?.G.
    00000000B0 7B3D 5620 0322 200B 3178 6600 5934 3F5F {=V ." .1xf.Y4?_
    00000000C0 0C79 3D23 B020 033A 200B 318C E15D 343F .y=#. .: .1..]4?
    00000000D0 05CC C775 3D90 2003 B720 0B80 31A0 A363 ...u=. .. ..1..c
    00000000E0 343F FD84 712C 3DC3 2003 7820 0B31 B489 4?..q,=. .x .1..
    00000000F0 6A00 343F 144F 6C3D DB4E 186C 3D5E 200B j.4?.Ol=.N.l=^ .
    0000000100 31C8 7172 3400 3F37 3166 3DFF 3066 303D 1.qr4.?71f=.0f0=
    0000000110 4620 0B31 DC35 7B34 3F05 8336 5F3D 4C20 F .1.5{4?..6_=L 
    0000000120 030B 200B 9031 F0B0 8420 3B6A 573D DF0C .. ..1... ;jW=..
    0000000130 6957 3D84 200B 3204 B68E 0034 3F0B D74E iW=. .2....4?..N
    0000000140 3DD9 D618 4E3D 8B20 0B32 1822 9934 023F =...N=. .2.".4.?
    0000000150 8A88 453D 5920 03F7 4098 50C7 C9A3 343F ..E=Y ..@.P...4?
    0000000160 B589 163B 3D87 2003 9F20 0B32 4085 00AE ...;=. .. .2@...
    0000000170 343F B4E5 303D 89B0 2003 5A20 0B32 542D 4?..0=.. .Z .2T-
    0000000180 B934 3F05 B3A7 253D 8A20 0303 200B 8032 .4?...%=. .. ..2
    0000000190 689B C334 3FE1 DA19 2C3D BB20 0371 200B h..4?...,=. .q .
    00000001A0 327C ABCD 0134 3F72 8A0D 3D4F 2003 6080 2|...4?r..=O .`.
    00000001B0 200B 3290 39D7 343F A20B C100 3D82 2003  .2.9.4?....=. .
    
    Even with compression (although it starts with 11 the few files tested seemed incompatible with the type 11 DS compression) the few magic stamps are evident and 1180 hex is a good length for the file not to mention 8011 appearing several bytes later which is a pattern repeated by the other files looked at.

    It will have to be investigated further but even the smallest files (usually good candidates for skipping compression owing to a lack of gains achieved) seem to be compressed. The compression looks like it will have to be manually stepped through.
    The question of whether arcX is the new NARC/ARC/CARC has all signs point to yes- multiple compressed magic stamps and what could be names follow in the file.

    The loc files has some ASCII in them that appeared to correspond to the names of the languages they come from although they appeared to be closer to variable names and developer notes than game text.

    The thing many eyes would have been drawn to though is the sound stuff with BCSAR seemingly being the main file and first of all yes it looks very much like SDAT (although maybe a bit lighter on the header definitions of things to follow). As an added bonus there appears to be both an XML file that might have been fed into something and what might be the intermediate format said tools spat out- developer left extras before we can even decrypt 3DS ROM images.

    Sample of the XML file, initial header split across lines to help the forum.

    Code:
    <?xml version="1.0"?>
    <SoundArchiveBinary xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xmlns:xsd="http://www.w3.org/2001/XMLSchema" Name="classics"
    xmlns="NintendoWare.SoundFoundation.FileFormats.NintendoWareBinary"> 
    <Sounds>
    <StreamSound ID="01000000" Name="STRM_MUSIC_BHURMA" FileID="00000000" />
    <StreamSound ID="01000001" Name="STRM_PYRAMID_MENU" FileID="00000001" />
    <StreamSound ID="01000002" Name="STRM_PYRAMID_CREDITS" FileID="00000002" />
    <WaveSound ID="01000003" Name="WSD_PYRAMIDS_GRABAMMO" />
    <WaveSound ID="01000004" Name="WSD_PYRAMIDS_ALLTREASURES" />
    <WaveSound ID="01000005" Name="WSD_PYRAMIDS_GUN" />
    <WaveSound ID="01000006" Name="WSD_PYRAMIDS_IMPACT" />
    <WaveSound ID="01000007" Name="WSD_PYRAMIDS_CRACHEURDEATH" />
    <WaveSound ID="01000008" Name="WSD_PYRAMIDS_CREATEBRICK" />
    <WaveSound ID="01000009" Name="WSD_PYRAMIDS_HIGHJUMPIMPACT" />
    <WaveSound ID="0100000A" Name="WSD_PYRAMIDS_DEATH" />
    <WaveSound ID="0100000B" Name="WSD_PYRAMIDS_CLOCK" />
    <WaveSound ID="0100000C" Name="WSD_PYRAMIDS_DESTROYBRICK" />
    <WaveSound ID="0100000D" Name="WSD_PYRAMIDS_TRIGGER" />
    ..
    ..
    ..
    ..
    ..
    ..
    <WaveSound ID="01000027" Name="WSD_PYRAMIDS_SCARABBUZZLOOP" />
    <WaveSound ID="01000028" Name="WSD_PYRAMIDS_SCARABDEATH" />
    </Sounds>
    <SoundGroups>
    <SoundGroup ID="02000000" Name="WSDSET_0" FileID="00000003" />
    </SoundGroups>
    <SoundSetBanks />
    <Players>
    <Player ID="04000000" Name="PLAYER_0" />
    </Players>
    <WaveArchives>
    <WaveArchive ID="05000000" Name="WSDSET_0_WaveArchive@AutoGenerated" />
    </WaveArchives>
    <Groups />
    <Files>
    <InternalFile ID="00000000" Name="stream/STRM_MUSIC_BHURMA.bcstm" />
    <InternalFile ID="00000001" Name="stream/STRM_PYRAMID_MENU.bcstm" />
    <InternalFile ID="00000002" Name="stream/STRM_PYRAMID_CREDITS.bcstm" />
    <InternalFile ID="00000003" Name="../cache/WSDSET_0.bcwsd" Size="5832" />
    <InternalFile ID="00000004" Name="../cache/WSDSET_0_WaveArchive@AutoGenerated.bcwar" Size="803072" />
    </Files>
    <ElementMap>
    <Root Address="00000000" Size="816896" Name="SoundArchiveBinaryFile">
    <Items>
    <ElementMapItem Address="00000000" Size="56" Name="BinaryHeader" />
    <ElementMapItem Address="00000040" Size="3196" Name="StringBlock" />
    <ElementMapItem Address="00000CC0" Size="4656" Name="InfoBlock" />
    <ElementMapItem Address="00001F00" Size="808960" Name="FileBlock">
    <Items>
    <FileElementMapItem Address="00001F20" Size="5832" Name="../cache/WSDSET_0.bcwsd" FileID="00000003" />
    <FileElementMapItem Address="00003600" Size="803072" Name="../cache/WSDSET_0_WaveArchive@AutoGenerated.bcwar" FileID="00000004" />
    </Items>
    </ElementMapItem>
    </Items>
    </Root>
    </ElementMap>
    </SoundArchiveBinary>
    Sample of the CSID
    Code:
    // SoundIDs
    static const unsigned int STRM_MUSIC_BHURMA = 0x01000000;
    static const unsigned int STRM_PYRAMID_MENU = 0x01000001;
    static const unsigned int STRM_PYRAMID_CREDITS = 0x01000002;
    static const unsigned int WSD_PYRAMIDS_GRABAMMO = 0x01000003;
    static const unsigned int WSD_PYRAMIDS_ALLTREASURES = 0x01000004;
    static const unsigned int WSD_PYRAMIDS_GUN = 0x01000005;
    static const unsigned int WSD_PYRAMIDS_IMPACT = 0x01000006;
    static const unsigned int WSD_PYRAMIDS_CRACHEURDEATH = 0x01000007;
    ..
    ..
    ..
    ..
    ..
    static const unsigned int WSD_PYRAMIDS_QRCODESUCCESS = 0x01000026;
    static const unsigned int WSD_PYRAMIDS_SCARABBUZZLOOP = 0x01000027;
    static const unsigned int WSD_PYRAMIDS_SCARABDEATH = 0x01000028;
    
    // SoundGroupIDs
    // WaveSoundSetIDs
    static const unsigned int WSDSET_0 = 0x02000000;
    // SequenceSoundSetIDs
    
    // BankIDs
    
    // PlayerIDs
    static const unsigned int PLAYER_0 = 0x04000000;
    
    // WaveArchiveIDs
    
    // GroupIDs
    

    First few results of a strings search in the BCSAR file

    The file is 0C7700 long which might point at C being the new location for the lengths (the DS usually has it as 8 hex).

    Still sample
    Code:
    0000000000 4353 4152 FFFE 4000 0000 0002 0077 0C00 CSAR..@[member='......']w..
    0000000010 0300 0000 0020 0000 4000 0000 800C 0000 ..... ..@[member='......'].
    0000000020 0120 0000 C00C 0000 4012 0000 0220 0000 . ......@.... ..
    0000000030 001F 0000 0058 0C00 0000 0000 0000 0000 .....X..........
    0000000040 5354 5247 800C 0000 0024 0000 1000 0000 STRG.....$......
    0000000050 0124 0000 C805 0000 2B00 0000 011F 0000 .$......+.......
    0000000060 0802 0000 1200 0000 011F 0000 1A02 0000 ................
    0000000070 1200 0000 011F 0000 2C02 0000 1500 0000 ........,.......
    0000000080 011F 0000 4102 0000 1600 0000 011F 0000 ....A...........
    0000000090 5702 0000 1A00 0000 011F 0000 7102 0000 W...........q...
    00000000A0 1100 0000 011F 0000 8202 0000 1400 0000 ................
    00000000B0 011F 0000 9602 0000 1B00 0000 011F 0000 ................
    00000000C0 B102 0000 1900 0000 011F 0000 CA02 0000 ................
    00000000D0 1C00 0000 011F 0000 E602 0000 1300 0000 ................
    00000000E0 011F 0000 F902 0000 1300 0000 011F 0000 ................
    00000000F0 0C03 0000 1A00 0000 011F 0000 2603 0000 ............&...
    0000000100 1500 0000 011F 0000 3B03 0000 1400 0000 ........;.......
    0000000110 011F 0000 4F03 0000 1200 0000 011F 0000 ....O...........
    0000000120 6103 0000 1800 0000 011F 0000 7903 0000 a...........y...
    0000000130 1A00 0000 011F 0000 9303 0000 1200 0000 ................
    0000000140 011F 0000 A503 0000 1600 0000 011F 0000 ................
    0000000150 BB03 0000 1600 0000 011F 0000 D103 0000 ................
    0000000160 1700 0000 011F 0000 E803 0000 1700 0000 ................
    0000000170 011F 0000 FF03 0000 1200 0000 011F 0000 ................
    0000000180 1104 0000 1800 0000 011F 0000 2904 0000 ............)...
    0000000190 1400 0000 011F 0000 3D04 0000 1600 0000 ........=.......
    
    The raw import of STRM_PYRAMID_MENU.bcstm in audacity did a fair attempt at removing hearing as a sense but in there seems to be various flavours of PCM or ADPCM.

    Next step is probably reverse engineer the compression (it looks to be a fairly basic LZSS implementation without a terribly big gap between flags but hand decoding LZ is tedious when you have the original file there and know the method used in that implementation) although music is probably worth looking at.
     
    GHANMI, nukeboy95, yuyuyup and 5 others like this.
  2. Fudge

    Banned Fudge Remember that death is not the end, but only a tra

    Joined:
    Aug 26, 2009
    Messages:
    2,655
    Location:
    New York
    Country:
    United States
    Awesome info, how exactly did you obtain the binary, if you don't mind me asking??
     
  3. FAST6191
    OP

    Reporter FAST6191 Techromancer

    pip
    Joined:
    Nov 21, 2005
    Messages:
    21,738
    Country:
    United Kingdom
    Nintendo seem to have found someone with a vague idea about security for the 3ds and the whole ROM is encrypted hence there not really being anything yet. However the developer stuff (which this was from) can do what they like and uses keys other than the official one(s) with thread being what followed my wandering around some of the random Chinese hacking sites (sadly it was in private mode so I do not have links). By the looks of things on 3dbrew and such the binary itself (which is to say actual executable code) sits aside from the general data although I am sure there will be some overlap somewhere along the way so all I have to look at is the data itself but that still provides things to do.

    There may well be more out there but having got my hands on a copy of something built with the SDK I was happy enough to get started with that.
     
  4. porkiewpyne

    Member porkiewpyne Report-er

    Joined:
    Jun 8, 2008
    Messages:
    1,941
    Country:
    Australia
    You'd better put up a BIG (red) sign saying that the 3DS is still not fully hacked yet before the n00bz come in asking "HAO TO PLAY 3DS ROMZ plox kthxbai". Just sayin'

    Here lemme help you put one here first.

    THE 3DS IS NOT FULLY HACKED YET AND WILL NOT PLAY 3DS ROMS!

    Bah who am I kidding?


    On a more serious note, what does this actually mean if you don't mind me asking? :)
     
    2 people like this.
  5. Fishaman P

    Member Fishaman P Speedrunner

    Joined:
    Jan 2, 2010
    Messages:
    3,176
    Location:
    Wisconsin
    Country:
    United States
    GAR! This isn't Smogon, we're not geeky enough to handle this!
     
  6. FAST6191
    OP

    Reporter FAST6191 Techromancer

    pip
    Joined:
    Nov 21, 2005
    Messages:
    21,738
    Country:
    United Kingdom
    What it means- next to nothing as far as getting code running (either on hardware or via emulation of some form) but Nintendo had a fairly complete SDK with lots of premade formats/data structures for the GBA and especially DS ( http://llref.emutalk.net/docs/ and http://www.romhacking.net/documents/469/ ) as well as the GC and Wii with most of them being interrelated in some way. Knowing these formats has allowed DS, GC and Wii hacking to blossom (the GBA is not bad either if you see the sappy sound format work) and in many cases have unknown games be open to those that can get around a GUI program rather than just those that are capable of being elbow deep in a hex editor all day.

    This is then something of a little peak at some potential formats for the 3ds (and a very cursory one at that) but it is the sort of thing we deal with around here (the equivalent for the Wii- http://gbatemp.net/topic/72013-wii-decryption-tool-released/page__st__15 ) so it is here.
     
    1 person likes this.
  7. Lucifer666

    Member Lucifer666 all the world needs is me

    Joined:
    Apr 22, 2011
    Messages:
    1,393
    Location:
    The Fourth Dimension
    Country:
    Antarctica
    Interesting find buddy. Followed this topic.
    Hoping it doesn't end up like the decrypted firmware thread and its noob flood.
     
  8. Coto

    Member Coto GBAtemp Addict

    Joined:
    Jun 4, 2010
    Messages:
    2,278
    Country:
    Chile
    While this may be completely unfounded..

    https://docs.google....2DV9cmenCl_oNNQ

    CSAR (document by IBM, ORACLE oriented), as I saw here:

    Code:
    0000000000 4353 4152 FFFE 4000 0000 0002 0077 0C00 CSAR ..@[member='......']w..
    0000000010 0300 0000 0020 0000 4000 0000 800C 0000 ..... ..@[member='......'].
    0000000020 0120 0000 C00C 0000 4012 0000 0220 0000 . ......@.... ..
    0000000030 001F 0000 0058 0C00 0000 0000 0000 0000 .....X..........
    0000000040 5354 5247 800C 0000 0024 0000 1000 0000 STRG.....$......
    0000000050 0124 0000 C805 0000 2B00 0000 011F 0000 .$......+.......
    0000000060 0802 0000 1200 0000 011F 0000 1A02 0000 ................
    0000000070 1200 0000 011F 0000 2C02 0000 1500 0000 ........,.......
    0000000080 011F 0000 4102 0000 1600 0000 011F 0000 ....A...........
    0000000090 5702 0000 1A00 0000 011F 0000 7102 0000 W...........q...
    00000000A0 1100 0000 011F 0000 8202 0000 1400 0000 ................
    00000000B0 011F 0000 9602 0000 1B00 0000 011F 0000 ................
    00000000C0 B102 0000 1900 0000 011F 0000 CA02 0000 ................
    00000000D0 1C00 0000 011F 0000 E602 0000 1300 0000 ................
    00000000E0 011F 0000 F902 0000 1300 0000 011F 0000 ................
    00000000F0 0C03 0000 1A00 0000 011F 0000 2603 0000 ............&...
    0000000100 1500 0000 011F 0000 3B03 0000 1400 0000 ........;.......
    0000000110 011F 0000 4F03 0000 1200 0000 011F 0000 ....O...........
    0000000120 6103 0000 1800 0000 011F 0000 7903 0000 a...........y...
    0000000130 1A00 0000 011F 0000 9303 0000 1200 0000 ................
    0000000140 011F 0000 A503 0000 1600 0000 011F 0000 ................
    0000000150 BB03 0000 1600 0000 011F 0000 D103 0000 ................
    0000000160 1700 0000 011F 0000 E803 0000 1700 0000 ................
    0000000170 011F 0000 FF03 0000 1200 0000 011F 0000 ................
    0000000180 1104 0000 1800 0000 011F 0000 2904 0000 ............)...
    0000000190 1400 0000 011F 0000 3D04 0000 1600 0000 ........=.......
    
    

    CSAR (1st bit 4353h) seems to be a signed zip file using a custom compression, yet encryption.

    "A CSAR is a container file, i.e. it contains multiple files of possibly different file types. These files are
    typically organized in several subdirectories each of which contains related files (and possibly other
    subdirectories etc). The current version of the specification supports CSARs that are zip files, typically
    compressed.

    Each CSAR must contain a subdirectory called Meta-Inf. This subdirectory must contain a so-called
    manifest file. This file is named MANIFEST and has the file extension .MF. It represents metadata
    of the other files in the CSAR. These metadata are given in the format of name/value pairs. These
    name/value pairs are organized in blocks. Each block provides metadata of a certain artifact of the
    CSAR. An empty line separates the blocks in the manifest file."

    "The first block of the manifest file (Block_0 in the figure) provides metadata of the CSAR itself (e.g.
    its version, creator etc). Each other block begins with a name/value pair that points to an artifact
    within the CSAR by means of a pathname. The remaining name/value pairs in a block are the proper
    metadata of the pointed to artifact. For example, a corresponding name/value pair specifies the
    MIME-type of the artifact."

    "A CSAR or a selective artifact within a CSAR may be signed. When signing an artifact of a CSAR the
    digest of this artifact as well as the public key of the entity signing the artifact is included in the CSAR
    together with a corresponding certificate; all of this is included in the so-called signature block file
    for the signed artifact. The signature block file has an extension dependent on the algorithm used
    for computing the digest (e.g. .RSA, .DSA), has the name of the signed artifact and is stored in the /
    Meta-Inf directory."

    Please don't min this too much as it may be completely unrelated but... the way the CSAR file inside that bcsar handles stuff from xml-like-arguments would be too much of a coincidence.
     
  9. 3DSGuy

    Member 3DSGuy No longer in scene

    Joined:
    May 22, 2012
    Messages:
    345
    Country:
    United States
    BTW, just for reference, these assets would be found in the RomFS of the application's CXI. So guys, you're looking at the file formats which exist in the RomFS.
     
    1 person likes this.
  10. Pippin666

    Member Pippin666 SSF43DE Master

    Joined:
    Mar 30, 2009
    Messages:
    1,663
    Location:
    Montreal, Qc
    Country:
    Canada
    BOOM, hopes out !!

    Pip
     

Share This Page