Discussion Info on SHA-256 hashes on FS patches

DarkMatterCore · Jan 27, 2021

LyuboA said:

@DarkMatterCore knows how to make the ES Patches maybe he will make a script or maybe @mrdude will
Click to expand...

I can't compare ES byte sequences on my own, so I can't really make any automated patches. I don't have a Switch anymore.

I can, however, provide insight and assist with the process. I'm familiar with all these formats thanks to working on nxdumptool.

LyuboA · Jan 27, 2021

DarkMatterCore said:

I can't compare ES byte sequences on my own, so I can't really make any automated patches. I don't have a Switch anymore.

I can, however, provide insight and assist with the process. I'm familiar with all these formats thanks to working on nxdumptool.
Click to expand...

thats great if @mrdude is up for another script for complete patches

mrdude · Jan 27, 2021

LyuboA said:

thats great if @mrdude is up for another script for complete patches
Click to expand...

I think crckd is looking into it, he says on a previous post he is. If he needs help, then he can post in here and then we can help him. In the meantime, I have bigger fish to fry :-). I do agree with you that it would be good if we also have this ability to make es patches, but I've not really got much time to look into it just now and it takes a long time to do all the 'investigating' before scripts can even be started.

LyuboA · Jan 27, 2021

DarkMatterCore said:

I can't compare ES byte sequences on my own, so I can't really make any automated patches. I don't have a Switch anymore.

I can, however, provide insight and assist with the process. I'm familiar with all these formats thanks to working on nxdumptool.
Click to expand...

since you dont have a switch anymore what does this mean for nxdumptool ??

mrdude said:

I think crckd is looking into it, he says on a previous post he is. If he needs help, then he can post in here and then we can help him. In the meantime, I have bigger fish to fry :-). I do agree with you that it would be good if we also have this ability to make es patches, but I've not really got much time to look into it just now and it takes a long time to do all the 'investigating' before scripts can even be started.
Click to expand...

when you have time you've already done more then most Thank you and Thanks to @DarkMatterCore and @crckd you guys are great for this community

LyuboA · Jan 27, 2021

i tried installing several NSP games non of them works without ES patches

DarkMatterCore · Jan 27, 2021

LyuboA said:

since you dont have a switch anymore what does this mean for nxdumptool ??
Click to expand...

Nothing at all. I sold it around August - September last year.

There are people willing to help me by testing stuff - that's all I need.

LyuboA · Jan 27, 2021

DarkMatterCore said:

Nothing at all. I sold it around August - September last year.

There are people willing to help me by testing stuff - that's all I need.
Click to expand...

thats great thanks btw if you need more testers i have several Switch consoles

ZachyCatGames · Jan 27, 2021

ShadowOne333 said:

There's something weird.
The one you mention is for 11.0.0, yet that one's without exFat.
There's no mention of 11.0.1 at all in neither the patches.ini nor the IPS patches.
Click to expand...

kips weren't touched in 11.0.1, they're identical to 11.0.0.

DarkMatterCore · Jan 27, 2021

I updated my scripts to adjust the upper NCA file size limit. I also made them generate a SD card ready directory tree layout for both IPS patches and Hekate patches.

peteruk · Jan 27, 2021

(I fear I could get shot down here, but am willing to take one for the team)

Are we likely to see a GUI tool version of this when everything is figured out and finished ? PC / Linux / Mac ?

Figured I'd ask, thanks in advance.

mrdude · Jan 29, 2021

peteruk said:

(I fear I could get shot down here, but am willing to take one for the team)

Are we likely to see a GUI tool version of this when everything is figured out and finished ? PC / Linux / Mac ?

Figured I'd ask, thanks in advance.
Click to expand...

Yes, that can easily be done. No point in doing it though until we figure out ES the patches though. The program can be used to make loader/fs/es patches. Also there would be no need for python to be installed as we can make python files into stand alone executables. See attached MakeIPS.py has been converted into an exe.

ShadowOne333 · Jan 29, 2021

mrdude said:

Yes, that can easily be done. No point in doing it though until we figure out ES the patches though. The program can be used to make loader/fs/es patches. Also there would be no need for python to be installed as we can make python files into stand alone executables. See attached MakeIPS.py has been converted into an exe.
Click to expand...

And we might have an insight into the ES patches, thanks to an anonymous source.
The source gave quite a lot of neat info, which might be what's required to finalize the set of signature patches to make an open source solution that everyone can run and create the patches from.

Here's the intel provided:
Quoted from Anonymous:
These are the patterns found between different versions of the ES NSO:
Code:
03E4EB5556B98B327D1353E8AA2C7ADF2C544470 [10.0.0]:
    2DC1C: 20010034 -> 09000014 [1F 90 01 31 28 92 80 52] [pattern found in more than one segment]
    2E9D8: E0030036 -> 1F2003D5 [C0 72 40 F9 E1 93 00 91]
    2DC70: 61000054 -> 1F2003D5 [F3 03 1F AA 02 00 00 14] [pattern found in more than one segment] [not found in 11.0.0+]

5AA09E1AF740A91D0F73ADFAE81A63E8AC0610D2 [10.1.0]:
    2DC30: 20010034 -> 09000014 [1F 90 01 31 28 92 80 52] [pattern found in more than one segment]
    2E9F8: E0030036 -> 1F2003D5 [C0 72 40 F9 E1 93 00 91]
    2DC84: 61000054 -> 1F2003D5 [F3 03 1F AA 02 00 00 14] [pattern found in more than one segment] [not found in 11.0.0+]

3B8BF56DBEC7225D2EE666B009C42C0DC4552010 [11.0.0]:
    2D094: 80020034 -> 14000014 [1F 90 01 31 28 92 80 52] [pattern found in more than one segment]
    2DBBC: 36D28152 -> 1F2003D5 [C0 72 40 F9 E1 93 00 91]
    2D0F4: C80000B4 -> 06000014 [E0 23 00 91 45 EE FF 97] [11.0.0+ only pattern]
The hex string at the start of each section represents the build ID for that version of the ES NSO (which is also the filename used for the IPS patches).

Each indented line starts with the patch offset (taken from the corresponding IPS patch), followed by the original and modified AArch64 instructions, as well as the 8 bytes after that. In order words, this is somewhat the format being used to document these findings:
Code:
{build_ID} [{version}]:
    {patch_offset}: {original_sequence} -> {patch_sequence} [{byte_sequence_after_patch}]
    ...
Click to expand...

mrdude · Jan 29, 2021

ShadowOne333 said:

And we might have an insight into the ES patches, thanks to an anonymous source.
The source gave quite a lot of neat info, which might be what's required to finalize the set of signature patches to make an open source solution that everyone can run and create the patches from.

Here's the intel provided:
Click to expand...

It would be handy to know what files are being patched, and how they are extracted from firmware files - can your source also supply that info?

DarkMatterCore · Jan 29, 2021

mrdude said:

It would be handy to know what files are being patched, and how they are extracted from firmware files - can your source also supply that info?
Click to expand...

I'm sure they're just unpacking the Program NCA from the ES sysmodule and decompressing its main NSO.

mrdude · Jan 29, 2021

DarkMatterCore said:

I'm sure they're just unpacking the Program NCA from the ES sysmodule and decompressing its main NSO.
Click to expand...

Ok maybe I should be clearer - what file in the latest firmware folder (11.0.1) is the ES Sysmodule?

For example is it: 0a34b913d4cfee8b15ef0c7cd6661697.nca, and how would one unpack it?

DarkMatterCore · Jan 29, 2021

DarkMatterCore said:
They skip the eTicket RSA signature verification. FS + Loader patches skip NCA RSA signature verification.

— Posts automatically merged - Please don't double post! —

The filename for ES patches comes from the module/build ID found at 0x40 in the ES main NSO. It's a 0x20 byte-long field, but only 0x14 bytes are used - iirc it's a SHA-1 hash calculated over a section from the NSO while it's being built, and not a full file hash unlike FS patches.

Keep in mind NSOs usually use LZ4 compression - you need to use a NSO decompressor before creating/applying IPS patches:
Code:
hactool -t nso --disablekeywarns --uncompressed=main_dec main
Click to expand...
DarkMatterCore said:

By the way, looking for a specific, extracted file/path won't help you in this case because all ExeFS sections hold at least the main NSO and the main.npdm. You'll really need to check the NCA header or the main.npdm file.

If you go down the NCA header route, you can follow my previous instructions. The byte at 0x205 must match 0x00 (Program NCA) and the unsigned 64-bit integer at 0x210 must match 0x0100000000000033 (ES sysmodule).

If you go down the main.npdm route, the NULL-terminated string at 0x20 must match "es".
Click to expand...

@mrdude This info should be handy. I shared it some pages ago.

The same command we've been using to unpack NCAs should do the trick - just replace "--romfsdir" with "--exefsdir".

mrdude · Jan 29, 2021

DarkMatterCore said:

@mrdude This info should be handy. I shared it some pages ago.

The same command we've been using to unpack NCAs should do the trick - just replace "--romfsdir" with "--exefsdir".
Click to expand...

I tried unpacking ALL the files, I got thousands of files/folders but none of them were called 'main', so that's where I failed on that info you supplied.

So I tried a little batch file like this, found the files in about 1 second :-)

for %%f in (firmware/*.*) do (
hactool.exe -k keys.dat -t nca --exefsdir=dumped/ firmware/%%f
if exist dumped/main.npdm (
EXIT /B
)
)
Click to expand...

I take it nearly every file has these; main + main.npdm files, these should be dumped into separate folders and then scanned?
So batch needs to look like this to dump:

for %%f in (firmware/*.*) do (
mkdir dumped/%%f
hactool.exe -k keys.dat -t nca --exefsdir=dumped/%%f/ firmware/%%f
rmdir dumped/%%f /Q
)
Click to expand...

DarkMatterCore · Jan 29, 2021

mrdude said:

I take it nearly every file has these; main + main.npdm files, these should be dumped into separate folders and then scanned?
Click to expand...

That's right, yeah. The only way to determine if you're dealing with the right NCA is by peeking into the NCA header or the main.npdm file.

mrdude · Jan 29, 2021

DarkMatterCore said:

That's right, yeah. The only way to determine if you're dealing with the right NCA is by peeking into the NCA header or the main.npdm file.
Click to expand...

Ok I will try and write a python script to do that as I don't know how to do it in dos command line.

crckd · Jan 29, 2021

I looped on all files inside the folder until i get the nca with title id 0100000000000033.
remaining would be finding the addresses.

Code:

import os
import sys
import subprocess

if len(sys.argv) == 1:
   print("no argv")
   sys.exit(1)

ES_NCA = ""
FIRMWARE_DIR = sys.argv[1]

print("Checking files in " + FIRMWARE_DIR + " folder.")
for filename in os.listdir(FIRMWARE_DIR):
   if filename.endswith(".nca"):
 
       outlines = subprocess.check_output(['hactool', '--disablekeywarns', FIRMWARE_DIR + '/' + filename])

       for line in outlines.splitlines():
           line = line.decode('ascii').replace(" ","")
           if line.startswith("TitleID:0100000000000033") and not filename.endswith(".cnmt.nca"):
               print("Found! Filename : " + filename)
               ES_NCA = filename
               break
       if ES_NCA:
           print("Using hactool to extract exefsdir")
           subprocess.run(["hactool","-t nca","--exefsdir=.","--disablekeywarns", FIRMWARE_DIR + '/' + filename], stdout=subprocess.DEVNULL)
           if os.path.exists("main"):
               outlines = subprocess.check_output(['hactool','--keyset=prod.keys','--intype=nso','--disablekeywarns','--uncompressed=main_dec','main'])
               print("Using hactool to uncompress main")
               for line in outlines.splitlines():
            
                   line = line.decode('ascii').replace(" ","")
                   if line.startswith("BuildId:"):
                       print("Found Build ID : " + line.replace("BuildId:","")[0:40])
           break

Useful Searches

DarkMatterCore I like turtles. Developer Level 10

LyuboA Unknown Entity Member Level 5

mrdude GBAtemp Maniac Member Level 12

LyuboA Unknown Entity Member Level 5

LyuboA Unknown Entity Member Level 5

DarkMatterCore I like turtles. Developer Level 10

LyuboA Unknown Entity Member Level 5

ZachyCatGames GBAtemp Psycho! Member Level 12

DarkMatterCore I like turtles. Developer Level 10

Attached Files:

peteruk GBAtemp Addict Member Level 15

mrdude GBAtemp Maniac Member Level 12

Attached Files:

ShadowOne333 QVID PRO QVO Developer Level 22

mrdude GBAtemp Maniac Member Level 12

DarkMatterCore I like turtles. Developer Level 10

mrdude GBAtemp Maniac Member Level 12

DarkMatterCore I like turtles. Developer Level 10

mrdude GBAtemp Maniac Member Level 12

DarkMatterCore I like turtles. Developer Level 10

mrdude GBAtemp Maniac Member Level 12

OP crckd Member Newcomer Level 2

Attached Files:

Hide similar threads Similar threads with keywords - patches, hashes,

Share this thread:

DarkMatterCore I like turtles.
Developer
Level 10

LyuboA Unknown Entity
Member
Level 5

mrdude GBAtemp Maniac
Member
Level 12

LyuboA Unknown Entity
Member
Level 5

LyuboA Unknown Entity
Member
Level 5

DarkMatterCore I like turtles.
Developer
Level 10

LyuboA Unknown Entity
Member
Level 5

ZachyCatGames GBAtemp Psycho!
Member
Level 12

DarkMatterCore I like turtles.
Developer
Level 10

peteruk GBAtemp Addict
Member
Level 15

mrdude GBAtemp Maniac
Member
Level 12

ShadowOne333 QVID PRO QVO
Developer
Level 22

mrdude GBAtemp Maniac
Member
Level 12

DarkMatterCore I like turtles.
Developer
Level 10

mrdude GBAtemp Maniac
Member
Level 12

DarkMatterCore I like turtles.
Developer
Level 10

mrdude GBAtemp Maniac
Member
Level 12

DarkMatterCore I like turtles.
Developer
Level 10

mrdude GBAtemp Maniac
Member
Level 12

OP crckd Member
Newcomer
Level 2