Hacking WIP Python - XCI to LayeredFS

[ChainedHope]

Wrote this up pretty quickly to help people having issues with using hactool. [Also I realize I put this in the wrong section... If a mod would move it that would be cool... I wasn't paying attention when I posted]
Requirements

• Windows OS
• Python 2.7
• Tkinter (Python library, google it)
• hactool
• keys.dat

Setup

1. Put the python file, hactool, and your keys.dat (don't ask me for it) in the same directory.
2. Run 'python xci_to_lfs.py' in cmd or powershell (while in the file directory)
3. Use the GUI to set your TitleID and pick your XCI
4. Wait a minute. I didn't put anything in the script to notify you when its done. For a 2GB game it takes about a minute. YMMV. The last thing it does is delete the tmp folder, so when its gone you are good to go.
5. Copy the Atmosphere folder to the root of your SD Card
6. Follow the rest of the tutorial found in the tutorials section [How to use LayeredFS for Backup Loading] (Can't post links)

Note that YMMV.

DOWNLOAD: https://mega.nz/#!Y6ByFbxI!3-whFid8d0UwMuDMEqrwM879d33SRZ9-GOuHwlpCkZU

FAQ (But you just posted o.O ... Yeah but I know you are going to ask)
Q: I'm getting a Python not found error
A: Install python and make sure it is in your PATH variable. Google is your friend

Q: WhErE Do I GEt KEYS.dat
A: Find it yourself or follow the tutorial in the tutorials section

Q: Why isn't there a romfs.bin or romfs.romfs?!?
A: Folder works too

Q: Do I need to edit NPDM?
A: Nope.

Q: Game X doesnt work for Game Y!?!
A: Go check the tutorial mentioned above for a list of working redirects.

Q: WHERES THE DOWNLOAD
A: ... Read up.

Q: Why not Python 3?
A: Because I use Python 2 for compatibility with older software. @rkk seems to have a working Python 3 script handy on post #16 (https://gbatemp.net/threads/python-xci-to-layeredfs.507562/#post-8057198).


Changelog
V1.0.2 -- Made the files extract into game folders (same name as .xci) for use with Game Redirector (PyNX)
V1.0.1 -- Added notifications, changed the NPDM writing to not use hardcoded offsets
V1.0.0 -- Rough script. Needs cleaned and organized but seems to work as a POC for now.

 

[Shirpie]

Great, even more ways to streamline the process. Not a moment too late.
Thanks for this.

 

[Deleted User]

https://mega.nz/#!w34zmB4b!6T8lFXJgrlb5P3THvfflbZIn2svk2QG2HMpUkzEX6Uo

 

[linga]

I get an error

 

[mattytrog]

Well done OP.

Will come in handy for the newbies I`m sure.

Top banana!

 

[ChainedHope]

I get an error

You probably have a unicode character in your path. If that blurred out part has a unicode character then thats the problem. [I didn't bother with encoding, this method will probably only be useful for a few days anyways]

 

[isoisoiso]

Thanks desu

 

[Gnarmagon]

maybe you should FINALLY switch to python3 ? or deliver a compiled version....

 

[phn]

Trying this method with DK tropical freeze (8gb) takes a looong time to copy to tmp the biggest NCA. I know, MMMV. Also I got a couple of errors with tkinter don't know why. When I get it to work I'll tell ya, but anyways thanks a lot for the effort.

 

[jakkal]

-snip-

Can you update it to notify when it's done copying?

 

[studio1b]

why do we have to fight. let the world be a loving place.
I don't understand why people always need to throw up the OH U WANT FREE THIS AND FREE THAT who cares.

 

[linga]

You probably have a unicode character in your path. If that blurred out part has a unicode character then thats the problem. [I didn't bother with encoding, this method will probably only be useful for a few days anyways]

Thank you for solving my problem

 

[naamloosNL]

why do we have to fight. let the world be a loving place.
I don't understand why people always need to throw up the OH U WANT FREE THIS AND FREE THAT who cares.

 

[Deleted User]

I get this error

Modifying NPDM
Exception in Tkinter callback
Traceback (most recent call last):
File "C:\Python27\lib\lib-tk\Tkinter.py", line 1541, in __call__
return self.func(*args)
File "xci_to_lfs.py", line 79, in do_the_thing
with open(path+'/ExeFs/main.npdm', 'rb') as f:
IOError: [Errno 2] No such file or directory: u'atmosphere/titles/010025400AECE000//ExeFs/main.npdm'

 

[jakkal]

Worked perfectly on my first try

 

[Deleted-451877]

Wrote this up pretty quickly to help people having issues with using hactool. [Also I realize I put this in the wrong section... If a mod would move it that would be cool... I wasn't paying attention when I posted]
Requirements

• Windows OS
• Python 2.7
• Tkinter (Python library, google it)
• hactool
• keys.dat

Setup

1. Put the python file, hactool, and your keys.dat (don't ask me for it) in the same directory.
2. Run 'python xci_to_lfs.py' in cmd or powershell (while in the file directory)
3. Use the GUI to set your TitleID and pick your XCI
4. Wait a minute. I didn't put anything in the script to notify you when its done. For a 2GB game it takes about a minute. YMMV. The last thing it does is delete the tmp folder, so when its gone you are good to go.
5. Copy the Atmosphere folder to the root of your SD Card
6. Follow the rest of the tutorial found in the tutorials section [How to use LayeredFS for Backup Loading] (Can't post links)

Note that YMMV.

DOWNLOAD: aHR0cHM6Ly9tZWdhLm56LyMhdzM0em1CNGIhNlQ4bEZYSmdybGI1UDNUSHZmZmxiWkluMnN2azJRRzJITXBVa3pFWDZVbw==

(Can't post links. So its a base64 encoded link. use "base64decode . org" to get the download)

FAQ (But you just posted o.O ... Yeah but I know you are going to ask)
Q: I'm getting a Python not found error
A: Install python and make sure it is in your PATH variable. Google is your friend

Q: WhErE Do I GEt KEYS.dat
A: Find it yourself or follow the tutorial in the tutorials section

Q: Why isn't there a romfs.bin or romfs.romfs?!?
A: Folder works too

Q: Do I need to edit NPDM?
A: Nope.

Q: Game X doesnt work for Game Y!?!
A: Go check the tutorial mentioned above for a list of working redirects.

Q: WHERES THE DOWNLOAD
A: ... Read up.


Changelog
V1.0.0 -- Rough script. Needs cleaned and organized but seems to work as a POC for now.

Your way of patching the npdm is pretty unclear. I think you're hardcoding the offset, which isn't correct since it varies from file to file.
Here is a script I made for myself (python 3 though), that has (I think) a cleaner way to do it:

import sys, os
from binascii import hexlify as hx, unhexlify as uhx
from struct import pack as pk, unpack as upk

hactoolPath =

# directory = input("Input directory? ")
directory = os.path.dirname(__file__)

for root, dirs, items in os.walk(directory):
    for item in items:
        if item.endswith(".xci"):
            # TID to write
            newTID = input("New TID to write in " + item + "? ")
            
            outputFolder = root + "\\atmosphere\\titles\\" + newTID
            os.makedirs(outputFolder, exist_ok=True)
        
            # Extract the secure partition
            filePath = os.path.join(root, item)
            commandLine = hactoolPath + " -t xci " + filePath + " --securedir=" + outputFolder + " 1>nul 2>nul"
            print("Extracting " + filePath + "...")
            os.system(commandLine)
            
            # Find the biggest nca
            biggestSize = 0
            biggestNCA  = ""
            for file in os.listdir(outputFolder):
                if file.endswith(".nca"):
                    size = os.path.getsize(os.path.join(outputFolder, file))
                    if size > biggestSize:
                        biggestSize = size
                        biggestNCA  = file
            
            # Decrypt it
            filePath = os.path.join(outputFolder, biggestNCA)
            commandLine = hactoolPath + " " + filePath + " --exefsdir=" + outputFolder + "\\exefs --romfs=" + outputFolder + "\\romfs.bin 1>nul 2>nul"
            print("Decrypting " + filePath + "...")
            os.system(commandLine)
            
            # Delete NCAs
            print("Deleting NCA files...")
            for file in os.listdir(outputFolder):
                if file.endswith(".nca"):
                    os.remove(os.path.join(outputFolder, file))
            
            # Patch main.npdm
            with open(outputFolder + "\\exefs\\main.npdm", "rb+") as npdm:
                # Pack TID
                newTID = pk("<Q", int(newTID, 16))
            
                # TID offset & length
                hexa = bytearray(npdm.read())
                TIDoffset = hexa.find(b"ACI0") + 0x10
                TIDlength = 0x8
                
                # Replace TID
                print("Writing new TID...")
                hexa[TIDoffset:TIDoffset+TIDlength] = newTID
                npdm.seek(0)
                npdm.write(hexa)
                print("Done\n")

 

[ChainedHope]

Your way of patching the npdm is pretty unclear. I think you're hardcoding the offset, which isn't correct since it varies from file to file.
Here is a script I made for myself (python 3 though), that has (I think) a cleaner way to do it:

import sys, os
from binascii import hexlify as hx, unhexlify as uhx
from struct import pack as pk, unpack as upk

hactoolPath =

# directory = input("Input directory? ")
directory = os.path.dirname(__file__)

for root, dirs, items in os.walk(directory):
    for item in items:
        if item.endswith(".xci"):
            # TID to write
            newTID = input("New TID to write in " + item + "? ")
         
            outputFolder = root + "\\atmosphere\\titles\\" + newTID
            os.makedirs(outputFolder, exist_ok=True)
     
            # Extract the secure partition
            filePath = os.path.join(root, item)
            commandLine = hactoolPath + " -t xci " + filePath + " --securedir=" + outputFolder + " 1>nul 2>nul"
            print("Extracting " + filePath + "...")
            os.system(commandLine)
         
            # Find the biggest nca
            biggestSize = 0
            biggestNCA  = ""
            for file in os.listdir(outputFolder):
                if file.endswith(".nca"):
                    size = os.path.getsize(os.path.join(outputFolder, file))
                    if size > biggestSize:
                        biggestSize = size
                        biggestNCA  = file
         
            # Decrypt it
            filePath = os.path.join(outputFolder, biggestNCA)
            commandLine = hactoolPath + " " + filePath + " --exefsdir=" + outputFolder + "\\exefs --romfs=" + outputFolder + "\\romfs.bin 1>nul 2>nul"
            print("Decrypting " + filePath + "...")
            os.system(commandLine)
         
            # Delete NCAs
            print("Deleting NCA files...")
            for file in os.listdir(outputFolder):
                if file.endswith(".nca"):
                    os.remove(os.path.join(outputFolder, file))
         
            # Patch main.npdm
            with open(outputFolder + "\\exefs\\main.npdm", "rb+") as npdm:
                # Pack TID
                newTID = pk("<Q", int(newTID, 16))
         
                # TID offset & length
                hexa = bytearray(npdm.read())
                TIDoffset = hexa.find(b"ACI0") + 0x10
                TIDlength = 0x8
             
                # Replace TID
                print("Writing new TID...")
                hexa[TIDoffset:TIDoffset+TIDlength] = newTID
                npdm.seek(0)
                npdm.write(hexa)
                print("Done\n")

I questioned the offset myself but on discord I had a few people try it and most had it working. Figured the NPDM was just standard and didn't change a lot between files. I even made my script keep a backup of the original file incase it failed so you could shove the npdm into the website editor someone made. Like I said originally, It was a rough script thrown together in about an hour. I don't think this method will be used for much longer so I'm not sure I will be going back and updating it.

--------------------- MERGED ---------------------------

maybe you should FINALLY switch to python3 ? or deliver a compiled version....

Python 2 is still a valid platform for compatibility with older software. @rkk has a cleaner python 3 script a few posts up (post # linked to in the FAQ for people who want to use Python3 over Python2)

 

[Deleted-451877]

I questioned the offset myself but on discord I had a few people try it and most had it working. Figured the NPDM was just standard and didn't change a lot between files. I even made my script keep a backup of the original file incase it failed so you could shove the npdm into the website editor someone made. Like I said originally, It was a rough script thrown together in about an hour. I don't think this method will be used for much longer so I'm not sure I will be going back and updating it.

According to switchbrew[dot]org/index[dot]php?title=NPDM it does. But indeed I don't think we'll be manually patching npdm's for long.

 

[Deleted member 420418]

Pretty good stuff.

 

[ChainedHope]

According to switchbrew[dot]org/index[dot]php?title=NPDM#ACI0 it does. But indeed I don't think we'll be manually patching npdm's for long.

Good to know. I'll update my code a bit then just incase that might be some users error. Also I linked your post number in the OP for a python 3 script.
... I should probably add a notification at the very least...