Could someone explain what the different 3DS software formats are...

Discussion in '3DS - ROM Hacking, Translations and Utilities' started by Hiccup, Oct 26, 2015.

  1. Hiccup
    OP

    Hiccup GBAtemp Advanced Fan

    Member
    900
    270
    Nov 21, 2009
    ...the different ways they can be decrypted/encrypted/extracted/packed? (question 1)
    2. And if there are any ever any in-game differences between the different formats of games (e.g. if they fixed a bug for the eShop release, but the bug still exists with the cart version)
    I'd just like to understand it :P
     
    Last edited by Hiccup, Oct 26, 2015
  2. FAST6191

    FAST6191 Techromancer

    pip Reporter
    23,154
    8,895
    Nov 21, 2005
    From the top then.
    The typical 3ds ROM is encrypted with keys we do not know. You can however use a 3ds that can run homebrew to generate a means to decrypt them, the files it produces are the same size as the ROM and as they are XORed against the ROM they became known as XORpads.
    Running a modified ROM can be done in a variety of ways but the big two that most gateway style flash cart users will see are 0 key and just encrypted with the xorpad again. 0 key is as it sounds and the ROM is encrypted with an all 0 key. Recently the so called hans stuff released by smea allows a kind of side loading of this.
    GBAtemp does not allow sharing of the xorpads and many places will also not allow it, even if they otherwise allow keys to be shared. Afraid I have not been out in the http/torrent/ROM download site world to know what goes.

    The 3ds ROM has two sections. The binary section which houses the code for the processor and the rest of it which houses all the files. As ever though the binary could contain some text and other things depending upon how it is programmed. Respectively they are known as the ExeFS and ROMFS, I shall leave you to ponder the reasoning behind those name choices. This applies whether you are doing the NCSD (or the cci aka .3ds ROM) or NCCH (aka .cxi) which usually is seen and known to the average 3ds user around here as CIA files, or roughly the 3ds equivalent of WAD files on the wii (small files that contain whole games and are usually associated with downloadable games and some lower level aspects of the system). The binaries can be extended by the so called CRO0 files, they are a bit more modern in their thinking than the DS approach of using overlays but I am not sure I would pit it against a true dynamically linked library setup.
    3ds ROMs have a header unique to each cart installed, said header is used to allow access to online services and some took to calling these ROMS with header .3dz. Apparently Nintendo's online security division was hitting the crack pipe a bit hard when they implemented the system though as any valid header from any game (including non online ones) will work for any and all games, so some people buy a throwaway game, dump the header and use it on all their other games. Bans have happened (kind of) and using public headers (headers which were dumped and released to the world) is not a great plan.

    http://3dbrew.org/wiki/Category:File_formats has a lot on the higher level formats and a few sub ones. I will spare you the discussion of banners, icons and such like; many do play with them for various hacks and custom themes but it is nothing terribly interesting if you have ever messed with anything like that before. That said the banner stuff is a fairly simple nice introduction to the basic graphics format ( CBMD ) and cwav audio format, more on those in a little bit.

    "if they fixed a bug for the eShop release, but the bug still exists with the cart version"
    There might be some regional differences, however 3ds games can support updates (the .CSU format) so that might not happen. I do not think any are nerfed to fit in the download. Some had pondered what could be done using the updates but the HANS stuff appears to have sidestepped much of that.

    The 3ds does have actual 3d hardware that resembles an actual piece of 3d hardware/something familiar to the more embedded side of opengl (in this case an older version with some of the newer stuff for textures/shaders bolted on). It will be a bit of a slog but http://gbatemp.net/threads/extracting-models-and-textures-from-3ds-games.370788/ has some good stuff in it. If you just want a search term then bcmdl is the main model format, it will be flanked by animations, textures and other things depending upon the game but that is the model format and a good search term. I do not have a good link for shaders but http://gbatemp.net/threads/gpu-re-need-everyones-help.379633/ will hopefully cover a few choice things.

    Following the wii, GC, probably Wii U, GBA and DS it has its own sound format used by an awful lot of games. In this case it is called BCSAR, which in turn breaks down into CSTM, CWSD, and CWAV. Though there is also BRSTM which most will first encounter if they want to go into modding smash brothers.

    Like the GBA and DS before it the 3ds uses compression and a variation on the theme of LZ is the more popular. It is not backwards compatible with the GBA and DS implementations and annoyingly uses some of the header values (11 in this case) so 3ds type 11 LZ would have to be the thing rather than the DS where a simple type 11 LZ would say things people needed to know. Afraid I am not up on the current 3ds compression supporting tools. Archive formats are also a thing still for whatever reason, the szs format seen on the wii appeared in the exact same format as well.

    If you are more used to the DS then there is nothing quite like tinke or crystaltile2 to play with, however https://gbatemp.net/threads/release-every-file-explorer.373615/ gets close and exceeds those at times. Most seem to gravitate towards that and it can get a lot done, I am not such a fan of premade tools and would rather understand things from the ground up.

    So yeah the 3ds is a halfway modern system in the software side of things, messed about by cretins and implemented by fools but the underlying principles shine though. Pity there are so few games for it.
     
  3. Hiccup
    OP

    Hiccup GBAtemp Advanced Fan

    Member
    900
    270
    Nov 21, 2009
    Thats quite helpful thanks.

    Let me make sure though:
    .cci = .3ds = software card rom image ?
    .cxi = .cia = format used to install software to a 3ds, these are downloaded from the servers and installed to the 3DS when you use the eShop or used by devs/homebrewers to install software?


    More questions (+ the not fully answered ones from the first post):
    3. when you extract a decrypted ROM, are there actually any folders, or are they made up? (I know the ones in the RomFS are real)
    4. Where is the banner and home menu icon stored in a software? How do you extract it?
    5. same as 4 but with the emanual
    5. What differences are there between .cia and .3ds? other than use?
    6. difference between software and title?
    7. Is it possible to decrypt 3DS roms without xorpads?
     
    Last edited by Hiccup, Oct 26, 2015
  4. FAST6191

    FAST6191 Techromancer

    pip Reporter
    23,154
    8,895
    Nov 21, 2005
    1 and 2. Yeah pretty much.

    4a) .BNR and .ICN are stuck in the ExeFS 4b) http://3dbrew.org/wiki/NCCH#CFA has a bit more, afraid I have not messed with manuals much.
    5. Technically they are a different class of file in most instances, though so interrelated it is but a minor distinction. If you were heading towards file size limits or something (CIA can only support ? gigabytes where .3ds could theoretically... sort of thing)
    6. By title I assume you mean http://3dbrew.org/wiki/Title_Data_Structure and software being the CIA/3ds and related stuff, afraid I would just be reading you the contents of 3dbrew at this point. Theoretically most things could go in most places without getting too esoteric, I do not think any will give you any especially useful differences in terms of file sizes available*, read times, hardware support/lockout, reliability... if that is where you were heading (sticking with Nintendo handhelds thinking some of the DSi stuff where some would not have wifi but would have camera and another might not have camera but would have wifi) then though 3ds hacks are not especially focused right now they own enough of the system for that not to matter/apply all that much.
    *I guess if you wanted to stick 10 gigs of files in there to port some PC game, video player or something then you might run into some problems somewhere with some formats, however anybody that can run one method will probably be able to do the rest if they cared to -- some custom firmwares are a bit limited, and the sky3ds only has any homebrew/region free/ROM hacking support if it is chained with a custom firmware or similar exploit (if you are familiar with the 360 then difference between JTAG/RGH and DVD mods being a good example).
    7. Ish. Prior to the xorpad method being developed then you could snatch snippets of ROM code out of memory (the earlier CWAV stuff being a good example https://gbatemp.net/threads/3ds-cwav-dumper.361437/ ) and I guess you could build a program to manually dump files to a SD cart/network connection using a 3ds. If you mean is there a way to get a standard .3ds, no xorpad and just fire it through a simple PC tool like ndstool or something then no and there is no ETA on such a thing either (the keys are buried pretty deep, hence the oracle abuse nature of the decryption). There are some websites that do offer already decrypted ROMs but I tend not to have to visit such sites so I do not know how easy it is out there if you are heading down that path.

    I have to run so I will leave it there for now. I should also say we are touching upon the limits of my knowledge of this area -- I follow the 3ds mainly to keep on top of it as it is a console we cover here and another system that can be hacked, I am not following the minutiae of it all like several others around here as the 3ds does not interest me at a baser level.
     
    Hiccup likes this.
  5. piratesephiroth

    piratesephiroth I wish I could read

    Member
    2,838
    1,386
    Sep 5, 2013
    Brazil
    yeah, the games can come in these two containers
    CCI = CTR Cartridge Image (also known as NCSD). It contains some sort of unique ID to be used online.

    CIA = CTR Importable Archive. It includes a "ticket" file to be installed to the 3DS, that's used to confirm ownership everytime the game is started.

    Then the content comes in NCCH files.
    These are mainly the CXI for the main application (CTR eXecutable image) and the CFA (CTR File Archive) for the remaining contents (update, manual, download play)
    The CXI contains the ExeFS, which has the game's executable, banner and icon.
    All the remaining files are packed in the RomFS.

    more info here: http://3dbrew.org/wiki/Category:File_formats
     
    Last edited by piratesephiroth, Oct 26, 2015
    Hiccup likes this.
  6. Hiccup
    OP

    Hiccup GBAtemp Advanced Fan

    Member
    900
    270
    Nov 21, 2009
    Thanks
    What is the best (accurate, easy to use) software (3DS or Windows) for:
    8. Dumping 3DS cart ROM
    9. Decrypting a ROM with an xorpad
    10. Extracting files out of a decrypted ROM (into folders, if applicable like with RomFS). Preferably with support for different ROM "types".

    Also:
    11. What is the difference between a decrypted ROM, and a zero-keyed ROM?
     
  7. piratesephiroth

    piratesephiroth I wish I could read

    Member
    2,838
    1,386
    Sep 5, 2013
    Brazil
    The best software for dumping cartridges is Gateway's menu.
    The best software for decrypting is Decrypt9WIP: http://gbatemp.net/threads/download-decrypt9-wip-3dsx-launcher-dat.388831/

    The best software for extracting the files from decrypted roms is ctrtool.

    A "zero-keyed" rom uses development console encryption.
    A decrypted rom uses no encryption at all.
     
    Last edited by piratesephiroth, Oct 26, 2015
  8. Hiccup
    OP

    Hiccup GBAtemp Advanced Fan

    Member
    900
    270
    Nov 21, 2009
    Thanks.

    12. Does gateway's menu produce good dumps?
    13. Is "Uncart" better?
     
  9. piratesephiroth

    piratesephiroth I wish I could read

    Member
    2,838
    1,386
    Sep 5, 2013
    Brazil
    Gateway menu produces perfect dumps.
    I never used uncart but it requires the homebrew launcher so it's clearly not as easy to use.
     
    Hiccup likes this.
  10. Asia81

    Asia81 In my Ecchi World <3

    Member
    4,844
    2,340
    Nov 15, 2014
    France
    Albi
    Uncart is so slow for me...
     
  11. Hiccup
    OP

    Hiccup GBAtemp Advanced Fan

    Member
    900
    270
    Nov 21, 2009
    14. Did gateway's dumper used to make bad dumps?
     
  12. piratesephiroth

    piratesephiroth I wish I could read

    Member
    2,838
    1,386
    Sep 5, 2013
    Brazil
    I never heard of that
     
  13. Hiccup
    OP

    Hiccup GBAtemp Advanced Fan

    Member
    900
    270
    Nov 21, 2009
    It seems you can correct the dumps with the gateway ROM patcher.

    15. What is the best PC (or browserhax/menuhax homebrew) tool for applying xorpads to .3DSs.

    16. What is the best PC (or browserhax/menuhax homebrew) tool for packing romfs/exefs extracted into a folder, into a romfs/exefs that works with HANS?