Fixed to put correct sha256 into app_header section:
Offending line typo:
wrong - sha256_rommenu = hashlib.sha256(bytes[0x610:0x1AD610]).hexdigest()
correct - sha256_rommenu = hashlib.sha256(bytes[0x610:0x1AC610]).hexdigest()
This still doesn't fix sxos based apps from working, probably this needs fixed by adding another patch to stage2, maybe
@Reacher17 can fix this as he's far more advanced with this stuff than 99.99% of most people.
kudos
I spent the night looking into several things regarding all this and finally understood some of the quirks... one of them is why core and lite are not acepting the boot.dat...
https://gist.github.com/hexkyz/cef102e45cea2cfba1350c7c42199983#file-tx_unpack-py-L20
this is the best script to understand latest things boot.dat; it is only unpacking and does not use all header fields but seeing hdr defintion and using an hex editor with a couple of different boot.dat versions did the trick.
the thing is core and lite check a 2048bit signature ( not just a hash ) that is stored after stage2 and pointed to by boot_dat_hdr_sig_offset field.
now, this is probably an RSA2048+SHA256 signature so we don't have the private key so we can't generate a valid signature for the changed header. Options:
a) crack the RSA signature check in sx modchip firmware
b) change the RSA public key to a given one, either one for all users or one per each user; the corresponding RSA private key would be provided when creating a boot.dat
c) I think the fingerprint can be spoofed in this same firmware and we would need nothing else, no need to mess with boot.dat. There's a 16 byte ID in the firmware and the fingerprint is either it itself or generated from it. So we can just set it to the same value as the donor one.
d) if we don't want to go this route we can do another thing which would also open other possibilities like having a C source code, confortable, sideloader to do 11.0.1 and future compatibilizations and feature additions easier. The idea is to use sxgear 1.1 and make a small chainloader that loads the original, unmodified boot.dat by respecting the boot.dat file format, ignoring the header signature ( or, if we want, we could sign it with a given private key and make this check the signature with its corresponding public key ), using the keys we use to create our boot.dat to decrypt the stages and jumping to the entrypoint of the last stage. The idea is we could also add changes just before jumping to entrypoint, things like sigpatches or hooks to adapt to new firmwares or new features, etc. Of course there are possible variants here, like just loading or embedding an unencrypted last stage only. It is just an adhoc chainloader written in C, easier to maintain and bridging the gap from sxgear to last stage only or full boot.dat and adding our own fixes and additions just before transferring it control. Think of it as a superlightweight hekate to fill this gap. Some sort of sigpatches compatibility could even be possible...
EDIT: I forgot to say that it might be the case that the signature key is the same used to decrypt the stages ( this would mean the signature would not be using asymetric crypto, it would not be RSA2048+SHA256 ). That would make it all easier and posible to sign boot.dat header properly. But still, that d) option still feels like a god idea to me...
EDIT2: I'll try contacting Spacecraft-NX creator to ask him about an SX modchip firmware dump. These modchips use ARM Cortex-M microcontrollers and this will easily give us some answers. OR I will read the tx modchip myself...