Hacking Discussion Info on SHA-256 hashes on FS patches

[DarkMatterCore]

@DarkMatterCore knows how to make the ES Patches maybe he will make a script or maybe @mrdude will

I can't compare ES byte sequences on my own, so I can't really make any automated patches. I don't have a Switch anymore.

I can, however, provide insight and assist with the process. I'm familiar with all these formats thanks to working on nxdumptool.

 

[LyuboA]

I can't compare ES byte sequences on my own, so I can't really make any automated patches. I don't have a Switch anymore.

I can, however, provide insight and assist with the process. I'm familiar with all these formats thanks to working on nxdumptool.

thats great if @mrdude is up for another script for complete patches

 

[mrdude]

thats great if @mrdude is up for another script for complete patches

I think crckd is looking into it, he says on a previous post he is. If he needs help, then he can post in here and then we can help him. In the meantime, I have bigger fish to fry . I do agree with you that it would be good if we also have this ability to make es patches, but I've not really got much time to look into it just now and it takes a long time to do all the 'investigating' before scripts can even be started.

 

[LyuboA]

I can't compare ES byte sequences on my own, so I can't really make any automated patches. I don't have a Switch anymore.

I can, however, provide insight and assist with the process. I'm familiar with all these formats thanks to working on nxdumptool.

since you dont have a switch anymore what does this mean for nxdumptool ??

I think crckd is looking into it, he says on a previous post he is. If he needs help, then he can post in here and then we can help him. In the meantime, I have bigger fish to fry . I do agree with you that it would be good if we also have this ability to make es patches, but I've not really got much time to look into it just now and it takes a long time to do all the 'investigating' before scripts can even be started.

when you have time you've already done more then most Thank you and Thanks to @DarkMatterCore and @crckd you guys are great for this community

 

[LyuboA]

i tried installing several NSP games non of them works without ES patches

 

[DarkMatterCore]

since you dont have a switch anymore what does this mean for nxdumptool ??

Nothing at all. I sold it around August - September last year.

There are people willing to help me by testing stuff - that's all I need.

 

[LyuboA]

Nothing at all. I sold it around August - September last year.

There are people willing to help me by testing stuff - that's all I need.

thats great thanks btw if you need more testers i have several Switch consoles

 

[ZachyCatGames]

There's something weird.
The one you mention is for 11.0.0, yet that one's without exFat.
There's no mention of 11.0.1 at all in neither the patches.ini nor the IPS patches.

kips weren't touched in 11.0.1, they're identical to 11.0.0.

 

[DarkMatterCore]

I updated my scripts to adjust the upper NCA file size limit. I also made them generate a SD card ready directory tree layout for both IPS patches and Hekate patches.

 

[peteruk]

(I fear I could get shot down here, but am willing to take one for the team)

Are we likely to see a GUI tool version of this when everything is figured out and finished ? PC / Linux / Mac ?

Figured I'd ask, thanks in advance.

 

[mrdude]

(I fear I could get shot down here, but am willing to take one for the team)

Are we likely to see a GUI tool version of this when everything is figured out and finished ? PC / Linux / Mac ?

Figured I'd ask, thanks in advance.

Yes, that can easily be done. No point in doing it though until we figure out ES the patches though. The program can be used to make loader/fs/es patches. Also there would be no need for python to be installed as we can make python files into stand alone executables. See attached MakeIPS.py has been converted into an exe.

 

[ShadowOne333]

Yes, that can easily be done. No point in doing it though until we figure out ES the patches though. The program can be used to make loader/fs/es patches. Also there would be no need for python to be installed as we can make python files into stand alone executables. See attached MakeIPS.py has been converted into an exe.

And we might have an insight into the ES patches, thanks to an anonymous source.
The source gave quite a lot of neat info, which might be what's required to finalize the set of signature patches to make an open source solution that everyone can run and create the patches from.

Here's the intel provided:

These are the patterns found between different versions of the ES NSO:

03E4EB5556B98B327D1353E8AA2C7ADF2C544470 [10.0.0]:
    2DC1C: 20010034 -> 09000014 [1F 90 01 31 28 92 80 52] [pattern found in more than one segment]
    2E9D8: E0030036 -> 1F2003D5 [C0 72 40 F9 E1 93 00 91]
    2DC70: 61000054 -> 1F2003D5 [F3 03 1F AA 02 00 00 14] [pattern found in more than one segment] [not found in 11.0.0+]

5AA09E1AF740A91D0F73ADFAE81A63E8AC0610D2 [10.1.0]:
    2DC30: 20010034 -> 09000014 [1F 90 01 31 28 92 80 52] [pattern found in more than one segment]
    2E9F8: E0030036 -> 1F2003D5 [C0 72 40 F9 E1 93 00 91]
    2DC84: 61000054 -> 1F2003D5 [F3 03 1F AA 02 00 00 14] [pattern found in more than one segment] [not found in 11.0.0+]

3B8BF56DBEC7225D2EE666B009C42C0DC4552010 [11.0.0]:
    2D094: 80020034 -> 14000014 [1F 90 01 31 28 92 80 52] [pattern found in more than one segment]
    2DBBC: 36D28152 -> 1F2003D5 [C0 72 40 F9 E1 93 00 91]
    2D0F4: C80000B4 -> 06000014 [E0 23 00 91 45 EE FF 97] [11.0.0+ only pattern]

The hex string at the start of each section represents the build ID for that version of the ES NSO (which is also the filename used for the IPS patches).

Each indented line starts with the patch offset (taken from the corresponding IPS patch), followed by the original and modified AArch64 instructions, as well as the 8 bytes after that. In order words, this is somewhat the format being used to document these findings:

{build_ID} [{version}]:
    {patch_offset}: {original_sequence} -> {patch_sequence} [{byte_sequence_after_patch}]
    ...

 

[mrdude]

And we might have an insight into the ES patches, thanks to an anonymous source.
The source gave quite a lot of neat info, which might be what's required to finalize the set of signature patches to make an open source solution that everyone can run and create the patches from.

Here's the intel provided:

It would be handy to know what files are being patched, and how they are extracted from firmware files - can your source also supply that info?

 

[DarkMatterCore]

It would be handy to know what files are being patched, and how they are extracted from firmware files - can your source also supply that info?

I'm sure they're just unpacking the Program NCA from the ES sysmodule and decompressing its main NSO.

 

[mrdude]

I'm sure they're just unpacking the Program NCA from the ES sysmodule and decompressing its main NSO.

Ok maybe I should be clearer - what file in the latest firmware folder (11.0.1) is the ES Sysmodule?

For example is it: 0a34b913d4cfee8b15ef0c7cd6661697.nca, and how would one unpack it?

 

[DarkMatterCore]

They skip the eTicket RSA signature verification. FS + Loader patches skip NCA RSA signature verification.

--------------------- MERGED ---------------------------


The filename for ES patches comes from the module/build ID found at 0x40 in the ES main NSO. It's a 0x20 byte-long field, but only 0x14 bytes are used - iirc it's a SHA-1 hash calculated over a section from the NSO while it's being built, and not a full file hash unlike FS patches.

Keep in mind NSOs usually use LZ4 compression - you need to use a NSO decompressor before creating/applying IPS patches:

hactool -t nso --disablekeywarns --uncompressed=main_dec main

By the way, looking for a specific, extracted file/path won't help you in this case because all ExeFS sections hold at least the main NSO and the main.npdm. You'll really need to check the NCA header or the main.npdm file.

If you go down the NCA header route, you can follow my previous instructions. The byte at 0x205 must match 0x00 (Program NCA) and the unsigned 64-bit integer at 0x210 must match 0x0100000000000033 (ES sysmodule).

If you go down the main.npdm route, the NULL-terminated string at 0x20 must match "es".

@mrdude This info should be handy. I shared it some pages ago.

The same command we've been using to unpack NCAs should do the trick - just replace "--romfsdir" with "--exefsdir".

 

[mrdude]

@mrdude This info should be handy. I shared it some pages ago.

The same command we've been using to unpack NCAs should do the trick - just replace "--romfsdir" with "--exefsdir".

I tried unpacking ALL the files, I got thousands of files/folders but none of them were called 'main', so that's where I failed on that info you supplied.

So I tried a little batch file like this, found the files in about 1 second

for %%f in (firmware/*.*) do (
hactool.exe -k keys.dat -t nca --exefsdir=dumped/ firmware/%%f
if exist dumped/main.npdm (
EXIT /B
)
)

I take it nearly every file has these; main + main.npdm files, these should be dumped into separate folders and then scanned?
So batch needs to look like this to dump:

for %%f in (firmware/*.*) do (
mkdir dumped/%%f
hactool.exe -k keys.dat -t nca --exefsdir=dumped/%%f/ firmware/%%f
rmdir dumped/%%f /Q
)

 

[DarkMatterCore]

I take it nearly every file has these; main + main.npdm files, these should be dumped into separate folders and then scanned?

That's right, yeah. The only way to determine if you're dealing with the right NCA is by peeking into the NCA header or the main.npdm file.

 

[mrdude]

That's right, yeah. The only way to determine if you're dealing with the right NCA is by peeking into the NCA header or the main.npdm file.

Ok I will try and write a python script to do that as I don't know how to do it in dos command line.

 

[crckd]

Ok I will try and write a python script to do that as I don't know how to do it in dos command line.

I looped on all files inside the folder until i get the nca with title id 0100000000000033.
remaining would be finding the addresses.

import os
import sys
import subprocess

if len(sys.argv) == 1:
   print("no argv")
   sys.exit(1)

ES_NCA = ""
FIRMWARE_DIR = sys.argv[1]

print("Checking files in " + FIRMWARE_DIR + " folder.")
for filename in os.listdir(FIRMWARE_DIR):
   if filename.endswith(".nca"):
 
       outlines = subprocess.check_output(['hactool', '--disablekeywarns', FIRMWARE_DIR + '/' + filename])

       for line in outlines.splitlines():
           line = line.decode('ascii').replace(" ","")
           if line.startswith("TitleID:0100000000000033") and not filename.endswith(".cnmt.nca"):
               print("Found! Filename : " + filename)
               ES_NCA = filename
               break
       if ES_NCA:
           print("Using hactool to extract exefsdir")
           subprocess.run(["hactool","-t nca","--exefsdir=.","--disablekeywarns", FIRMWARE_DIR + '/' + filename], stdout=subprocess.DEVNULL)
           if os.path.exists("main"):
               outlines = subprocess.check_output(['hactool','--keyset=prod.keys','--intype=nso','--disablekeywarns','--uncompressed=main_dec','main'])
               print("Using hactool to uncompress main")
               for line in outlines.splitlines():
            
                   line = line.decode('ascii').replace(" ","")
                   if line.startswith("BuildId:"):
                       print("Found Build ID : " + line.replace("BuildId:","")[0:40])
           break