Hacking Recent eShop downloads are not dumping through GM9/installing through FBI

Zaphod77 · Jan 30, 2019

if encrypting doesn't help, then they must have changed the cdn file structure somehow.

we need to compare a working super mario bros.cia and a non working one. you have the non working one.

an07her · Jan 30, 2019

You should compare what's so different with existing dump and yours.
Try ctrtool -x -t cia YOURCIA.cia --tmd=tmd --tik=tik --certs=certs to extract some files and compare.
Also ctrtool -y YOURCIA.cia should show all the info about it.

Zaphod77 · Jan 30, 2019

once that comparison is done we can figure out what assumptions the cia installers are making that are wrong.

PrincessLillie · Jan 30, 2019

OP: Have you tried with DevMenu?
Please do not ask for a download link, as we cannot provide one due to the copyrighted nature of DevMenu. Google is your friend.

Zaphod77 · Jan 30, 2019

i believe someone said that devmenu DID NOT work, but NASA 1.6 did for one of them. need to extract one of these bad ones and compare with good.

Deleted-236924 · Jan 31, 2019

The 3DS doesn't care whether CIAs are encrypted or not (to be more precise, the only time where it does care, is for (some?) NAND titles, which does not include DSiWare.)

They can't just add new DRM on digital titles without also updating the 3DS firmware to support those changes + likely requiring users to redownload all of their games (hasn't happened, will not happen.)

Comparing tmd and ticket is not very useful. tmd will not be legit unless you got it from the CDN, because methods for dumping SD content to CIA don't tend to maintain tmd signatures, and you have no idea how the CIAs you'll find on warez sites were made. Not all methods necessarily alter the tmd in the same way.
(If you need evidence of this, get a CIA with legit tmd, from the CDN, install it and then dump it back to CIA through GM9, and see what it looks like.)

Ticket also possibly might not tell you much. There is no accepted standard for how fake tickets are formatted, everyone does it differently, and fact of the matter is that there is no objectively correct way of doing this. Two tickets for the same title can differ and still be perfectly functional. Title key used to matter in a way, but that's not a concern anymore because your fake tickets won't be letting you download from the CDN even if the title key matches anyway, so you can have whatever you want there and it doesn't make the CIA invalid.

GM9 had changes made back in October regarding the way it handles universal (standard) CIA creation, more specifically the way it handles tickets. In my case the issue was related to duplicate tickets in ticket.db, which I'd imagine likely isn't the case for you here, but it's always possible that there's some really weird specific issue that no-one has really ever run into before which is causing these problems.

There hasn't been a stable release since August, therefore if you just download the latest from Github like most people do, you don't have those recent changes. I highly suggest trying the latest build just in case that might happen to also fix your issue. Otherwise I would suggest opening your own issue on Github to try and figure it out (https://github.com/d0k3/GodMode9/issues)

I can assure you that nothing has changed with the CDN, I've bought games during the same time-frame as you have, plus some DLC, along with downloading updates off the eShop. I've made my own backups for all of those titles through ctrcdnfetch and those CIAs are perfectly functional.

Tuiridh · Jan 31, 2019

NASA 1.6 also did not work on the recent eShop downloads I mentioned. I may have found someone with alternate working .cias of those games, in the meantime I am passing it to someone else who can make sense out of the comparison (if even helpful at all) as I know no code.

Ammako said:
There hasn't been a stable release since August, therefore if you just download the latest from Github like most people do, you don't have those recent changes. I highly suggest trying the latest build just in case that might happen to also fix your issue. Otherwise I would suggest opening your own issue on Github to try and figure it out (https://github.com/d0k3/GodMode9/issues)

I will try this out, although I believe I installed cfw after August. Thank you!

--------------------- MERGED ---------------------------

GodMode9 v1.7.1 and FBI v2.6.0 are my current versions.

Deleted-236924 · Feb 1, 2019

Tuiridh said:
I will try this out, although I believe I installed cfw after August. Thank you!

You wouldn't have newer commits of GM9 unless you specifically went and downloaded them. Only the stable numbered releases are uploaded to Github (not that the newer commits are inherently unstable, but they haven't been as thoroughly tested.)

The latest is v1.7.1-35-ga04c2beb

It may not help, because your issue is something different altogether, but it's worth a try because changes have been made since.

This is relevant: https://github.com/Steveice10/FBI/issues/393

Also, it's possible that the issue is not on GM9's end at all, but even when that is the case I've found that d0k3 tends to help troubleshoot the issue regardless, so it's worth posting on Github about it if needed.

Tuiridh · Feb 4, 2019

Okay, a friend of mine has compared my copies of non-working .cias with functional copies and created a hex image chart, these are his findings/results:

Nintendo didn't make a new security feature. Both eshop and freeshop copies of a-train have all the same extracted files hashwise, except for the ticket. When godmode9 made the two games, it copied tickets from ticket.db that are formatted wrong. fbi wont install because it does not recognize the lower half of those tickets. The ticket.db found on the 3ds drive is damaged. The ticket.db was likely corrupted from libretro or retroarch cias that crashed in mid-install or forced the 3ds to turn off. You can fix the tickets with simple hex editing.

Deleted-236924 · Feb 4, 2019

Doubtful. If your ticket.db had been corrupted in such a way that your ticket for those titles appeared like this, you would have had many more problems than a simple failed CIA creation. For one, those games + any affected titles would not run (even one single byte changed on a ticket in ticket.db after installation results in the application not being able to run.)

I doubt this is how your ticket actually looks inside your ticket.db, because if your ticket were broken in that way, 1) the eShop would have noticed missing tickets and repaired them by re-downloading and re-installing them, 2) assuming that the ticket was indeed broken like this while browsing the eShop, you wouldn't have been able to re-download the game, because the eShop only accepts valid signed tickets, and a corrupted ticket like this wouldn't have a valid signature. Also, 3) I have a hunch that you did not attempt a reinstall of Libretro NXEngine after each time that you reinstalled and redownloaded those games from the eShop, so there's no possible way your eShop tickets would have gotten corrupted by it like this.

Check 1:/dbs/ticket.db --> Ticket.db options... --> Mount image to drive, press A to enter path when prompted, go into eShop folder, and look for 0004000000155700.XXXXXXXX.tik (XXXXXXXX being your console ID), press A on it and do "Show in Hexeditor" and compare with what's in the image above.

What might have happened? Nobody knows exactly how ticket.db works. We can install and delete tickets via am services (along with installing CIAs, which also installs the ticket), and we can parse and read its contents, but that's about all we can do right now. Nobody has bothered researching the whole ticket.db format (how to fully delete tickets, for example.)

Chances are, you had a previous ticket for those titles installed, and it was later "deleted" (note that deleted tickets are not completely removed, they are only marked as inactive. FBI doesn't show tickets marked as deleted, but GM9 sees them still.) What happens is that the ticket remains in db, until maybe it gets overwritten by something else. In this case I guess it got partially overwritten by a ticket for Libretro NXEngine at a later time during install. No, interrupting a CIA install does not cause this, unless maybe you somehow managed to time it just right so you interrupted it at the time ticket.db was being written to. But that's such a short window of time, it's unlikely, and you would have had so many more problems with your console if ticket.db did get corrupted, anyway.

How does GM9 work? Pre-October 23 2018, all it did was take a 1:1 copy of the ticket it found in ticket.db, and zero out console ID, eShop account ID, and ticket ID, and used that for creating the CIA (Source, in red, or, for the colorblind, the lines with a - on them.) GM9 also does not differentiate between deleted/inactive tickets, and I know from experience that, if duplicate tickets were present in ticket.db, GM9 could not properly tell which ticket is the currently "active" one, and which ones were deleted tickets. It just picked whichever it found first, which could end up being the wrong one (maybe even a partially overwritten ticket. I don't *think* GM9 validates most of the lower half of the ticket when you don't specifically tell it to validate the ticket signature. Though feel free to correct me if I'm wrong.)

With the changes from October 23 2018, it doesn't do that anymore. Now it just uses a generic fake ticket (the same one that is used for creating CIAs from cartridges, among other things), inserts into it the title ID and title key from your ticket in ticket.db, and uses that for creating the CIA. This means that even if it finds the wrong ticket in ticket.db, it still finds the right title ID, which is the only bit of data that actually matters (titlekey doesn't matter on decrypted CIAs, which is what GM9 outputs, and the rest of the generic ticket is perfectly functional for any title that's ever been thrown at it.) As you can tell this is much more error-proof than the previous way of handling things.

Simply looking at this picture, and knowing how GM9 handles CIA creation, I could tell you for a fact that if you simply used the latest commit of GM9 as I've linked to you earlier, you would be able to build proper working CIAs for those titles. There's no possible way a ticket like the one shown on the right would happen after the October 23 2018 commit (also, unimportant, but for the sake of information, the purple and orange blocks on the right are one single block of data, and not two separate ones; the four 00 bytes separating them are actually from GM9 overwriting the eShop account ID with 00s.)

As for how you could have gotten duplicate tickets like this to happen in your db, that's not something I can answer, because I wasn't the one who was using your 3DS.

TurdPooCharger · Feb 5, 2019

Tuiridh said:
Okay, a friend of mine has compared my copies of non-working .cias with functional copies and created a hex image chart, these are his findings/results:

Oh wow, it was all due to messed up tickets? I tried something like your friend's idea but in reverse by taking a clean *.cia of F-ZERO SNES VC and editing its ticket to resemble that bad dump. In GodMode9, mine was decrypted, hash verifiable, and image mountable. In FBI, I got the same error:

Next, I tried editing the bad ticket in my 1:/dbs/ticket.db. A hex search of 00 04 00 00 0F 70 19 00 found two (2) instances of the F-ZERO (fake) ticket. Due to the previous failed CIA install, F-ZERO was not on HOME Menu. I installed a non-edited, clean copy to bring it back. Hex searching the ticket.db again, I found a third (3rd) ticket copy was added.

@Tuiridh, I think your ticket.db most likely have at least two (2) tickets for both A-Train 3D City Simulator and Super Mario Bros NES VC. Your 3DS native firmware formally recognizes the good copies of those tickets. To add to your friend's analysis, I surmised the ticket mishap for A-Train happened when you were using RetroArch and tried switching over to the NXengine core. While it was installing that core's CIA for the first time, the 3DS system must have panicked and failed to complete its install. In this rare scenario, your first ticket instance of A-Train ended up being like what we see in that picture. You may have found that game to be missing and went on Nintendo eShop to redownload it again. If true, a similar event would have occurred for SMB.

If I had to guess why GodMode9 dumped those two games with bad tickets despite using Build CIA (standard), it was probably trying to pull something called the 0x10 bytes "encrypted title key" out of the first ticket instances it found. This action was probably done for preservation purposes, probably a leftover of pre-CDN server block where one may want to re-download a game from eShop or freeShop using a fake ticket. You would still need that game's specific encTitleKey to decrypt its downloaded contents.

Going by the above assumption, GM9 must have copied those damaged tickets out of ticket.db, treating them as fake tickets without noticing they're corrupt. There may have been a screw-up trying to distinguish the legit tickets versus the damaged leftovers, so it defaulted on finding the first instance that matched that upper half description: 00 01 00 04 ... Root-CA00000003-XS0000000c ... 0004000000155700. Once it found this, it copied the full 0x350 bytes of that ticket and stuff that bad boy into the CIA, calling it a day.

***

In order to fix your ticket.db with a clean slate,

Backup your HOME Menu layout.
Backup the saves + extdata for 3DS games with Checkpoint or JKSM.
Backup GBA VC Saves.
Backup the current SysNAND *.bin for safety: Creating a NAND Backup
Individually extract all legit tickets:
- GodMode9 → [1:] SYSNAND CTRNAND/dbs/ticket.db → Ticket.db options... → Mount image to drive.
- While in the T: drive, you should see four (4) folders: eshop, hidden, system, unknown.
- Hold (L)-shoulder button and press (RIGHT) d-pad to highlight select all folders in yellow. Let go of (L)-shoulder button.
- Hold (R)-shoulder button and press (A), and then let go of (R)-shoulder button. → Copy to 0:/gm9/out → <A> yes → <A> to continue.
- This will copy the folders with all the *.tik files (individual tickets).
Download the CTRTransfer *.bin image that matches your system's region and series type (o3DS / n3DS).
- Do not follow the CTRTransfer guide. All that you want is the 11.5.0-38X_ctransfer_x3ds.bin. Put this file at sdmc:/gm9/out.
GodMode9 → 0:/gm9/out/11.5.0-38X_ctransfer_x3ds.bin → CTRNAND options... → Mount image to drive.
While in the 7:/dbs directory, hover over, highlighting white ticket.db. Press (Y) on the file to [CLIPBOARD] select it (see bottom screen).
- If your choice is incorrect, press (SELECT) to reset the selection.
Back out with (B) button. Go into the [1:] SYSNAND CTRNAND/dbs/ directory.
Press (Y) → Copy path(s) → <A> yes → button combo → Overwrite files(s) → <A> yes.
- This will inject or replace with an empty ticket.db.
While hovering on, highlighting white ticket.db, press (A) → Calculate CMAC → <A> yes → <A> yes → button combo → <A> yes.
- This is to fix mismatch in the hashes.
Press (START) to boot to HOME Menu. You will notice all your games will be missing... This is because their tickets are missing.
- You can check for this by going to: System Settings → Data Management → Nintendo 3DS → Software
- Titles with missing tickets will have greyed out X's.
FBI → SD/gm9/out. Inside each of the four (4) folders, you see should see a bunch of *.tik files.
Press (A) on <current directory> → Install all tickets. Do this for all four folders.
- This will restore ~~all~~ most officially obtained Nintendo eShop such as your purchased games and DLCs.
Re-download A-Train 3D City Simulator, Super Mario Bros NES Virtual Console, and any other missing legit titles you bought from Nintendo eShop.
For titles you acquired from those sites, restore their fake tickets with faketik.
- Tickets that weren't restored with FBI and faketik will require using fake tickets found at that ISO site, or the following guide:
- I made a NAND backup awhile back, now had to recover, missing games.
To unwrap multiple games at once, use Cthulhu.
- HOME Menu software management → Unwrap all HOME Menu software.
Restore your HOME Menu layout.

Deleted-236924 · Feb 6, 2019

There is nothing to fix, because nothing is broken. It's simply Godmode9 handling things improperly, which has been fixed since October.

Deleted-236924 · Feb 6, 2019

Here's a peek at what the ticket.db inside the 11.5.0-38U_ctrtransfer_o3ds.bin from the guide looks like (you can check this yourself by getting CDN-FX, as well):

(Note that all the tickets with a console ID of 00E74E01 here were legit tickets from legitimate purchases made by the owner of the 3DS from which this CTRTransfer image came from.)

If you mount this ticket.db in GM9, you'll find that only a fraction of these actually show up (something everyone here can also verify, as well):

Note that GM9 only displays tickets which are properly signed by Nintendo. Any changes to a ticket invalidates the signatures, and thus they won't show up in GM9.

If you look inside the ticket.db in a hex editor, at 0x1E2818 you can find a ticket for Super Mario Bros. VC. This ticket looks complete, but the eShop account ID has been zeroed out by the system. At 0x13AAC10, you can find another ticket for Super Mario Bros. VC, this one has been partially overwritten (half of the signature is missing.)

If you scroll around in this area, you can see a handful other tickets that have been partially overwritten in that way, mixed in between a few tickets that are complete. I haven't looked through the entire file to find all examples (waste of time), but you get the point.

(PS: do not upload your ticket.db here for people to look at; it contains all of your tickets, including all of your legitimate tickets, which should not be shared here because they could be used by anyone for downloading through ctrcdnfetch. This could also get your 3DS banned if Nintendo notices too many downloads from all over the world using your tickets, or if DLC you didn't buy is being downloaded. If they don't just invalidate your tickets altogether, but I'm not sure if they would do that.)

Point is, tickets getting partially overwritten is normal functioning of the system. The console doesn't care about what is in ticket.db aside from the tickets which are currently in use (it will not overwrite tickets which have not been marked as deleted, either.) FBI may be able to tell currently active and undeleted tickets apart from the inactive/duplicate ones (it can display them, after all, and hide them once you "delete" them), but GM9 doesn't have access to all the ARM11 functions that FBI does; all it can do is parse the contents of ticket.db directly.

Homebrew authors don't know all the inner details of how the system works, only Nintendo does. I mean, I guess it could be possible for someone to disassemble all of the system files to figure out exactly how everything works and document everything, but this would be a huge waste of time given that we can already achieve everything that actually matters by just using Nintendo's built-in functions, with minimal effort.

Furthermore, I am going to repeat myself: the eShop re-downloads and re-installs all missing tickets (or corrupted tickets, sure, why not) for your purchased content when you enter it, if it detects that the currently active ticket for these titles is not the one from your eShop account/NNID. This is easily verifiable by restoring a ticket.db (or a complete nand backup) which did not have the tickets for one or more of your downloads, and entering the eShop.

This is also why pirated DLC may disappear when you enter the eShop. Simply accessing a game's DLC shop (or the Theme shop) will grant your account a legit DLC ticket for it. It will also show up in data management, though it doesn't actually contain any of the DLC.
If you installed pirated DLC afterwards, upon visiting the eShop later, it notices that the ticket currently in use for this title is not the one from your eShop account, and it overwrites it. The only way to stop this from happening is to delete your eShop account or unlink your NNID, if all of the purchases and DLC shop accesses were made with an NNID. Though note that by deleting your eShop account, you will of course lose all of your purchases, so this is not something anyone should do lightly (you won't lose anything if you only have to unlink your NNID, though you won't have access to your stuff while your NNID is unlinked either.)

tl;dr this is fine, you have nothing to gain from replacing your ticket.db and reinstalling your tickets, you would only be wasting your time if you did so. Especially if nothing is actually malfunctioning on your system while you're using it (all games and system titles functional, etc.) The issue was with GM9's improper handling of tickets, because we don't know exactly how everything in ticket.db works, so it can be difficult (impossible) for GM9 to tell exactly which ticket is the current one in use given that it doesn't have access to all the ARM11 functions that FBI does to differentiate between tickets.

GM9 should not fail to build working CIAs past the October 23 2018 commit. A CIA is basically a ticket, a tmd, and contents. The TMD and contents are as-is on SD, there is no way for GM9 to get that wrong. As for the ticket, it has to scan through ticket.db and find the relevant information in there. This should not fail with the newer commits, because GM9 uses a pre-constructed ticket, with only the title key and title ID being replaced with data from ticket.db (technically also the common keyY, but fake tickets from cdn downloaders don't seem to care about that and it worked just fine for them regardless.) As long as 0x140-0x1BB (issuer) and 0x1DC-0x1E3 (titleID) in a ticket GM9 detects within ticket.db are intact (which they always should be, if GM9 is detecting it as being a ticket for your title), 0x1BF-0x1CE (title key) is necessarily also intact, meaning that any data detected by GM9 within ticket.db as being a ticket for your title will result in a proper fake ticket being built. Though note that if a duplicate ticket had a wrong title key, this could still result in your built CIA having the wrong title key, but this should not cause any problems because the title key is only used when downloading from the eShop/CDN, and since you can't download from the CDN anymore without having a completely valid signed ticket, it doesn't really matter if your custom-built CIAs have an incorrect title key (it only matters for CIAs generated from CDN content.)

The only reason anyone would want to replace their ticket.db with a clean one to remove all traces of past tickets, is if they're really obsessed with keeping everything as "clean" and "proper" as possible (note: I am; I've got an old backup with a relatively clean ticket.db that I've restored more than once to several different 3DSes, because I have an unhealthy obsession with keeping everything as "clean" and "proper" as possible. I'm still kicking myself for not making a clean factory backup of my 2DS when I got it, lol.) This is of course a waste of time though, because your 3DS will work just fine even if the ticket.db isn't completely empty of old ticket remnants, and Nintendo doesn't scan the contents of your ticket.db for detecting traces of homebrew/CFW usage either.

Hacking Recent eShop downloads are not dumping through GM9/installing through FBI

Well-Known Member

Well-Known Member

Well-Known Member

(Future) VTuber

Well-Known Member

Deleted-236924

Guest

Banned!

Deleted-236924

Guest

Banned!

Attachments

Deleted-236924

Guest

Meh.

Deleted-236924

Guest

Deleted-236924

Guest

Similar threads

Popular threads in this forum