Hacking WIP Python - XCI to LayeredFS

[Eastonator12]

Wrote this up pretty quickly to help people having issues with using hactool. [Also I realize I put this in the wrong section... If a mod would move it that would be cool... I wasn't paying attention when I posted]
Requirements

• Windows OS
• Python 2.7
• Tkinter (Python library, google it)
• hactool
• keys.dat

Setup

1. Put the python file, hactool, and your keys.dat (don't ask me for it) in the same directory.
2. Run 'python xci_to_lfs.py' in cmd or powershell (while in the file directory)
3. Use the GUI to set your TitleID and pick your XCI
4. Wait a minute. I didn't put anything in the script to notify you when its done. For a 2GB game it takes about a minute. YMMV. The last thing it does is delete the tmp folder, so when its gone you are good to go.
5. Copy the Atmosphere folder to the root of your SD Card
6. Follow the rest of the tutorial found in the tutorials section [How to use LayeredFS for Backup Loading] (Can't post links)

Note that YMMV.

DOWNLOAD: https://mega.nz/#!w34zmB4b!6T8lFXJgrlb5P3THvfflbZIn2svk2QG2HMpUkzEX6Uo

FAQ (But you just posted o.O ... Yeah but I know you are going to ask)
Q: I'm getting a Python not found error
A: Install python and make sure it is in your PATH variable. Google is your friend

Q: WhErE Do I GEt KEYS.dat
A: Find it yourself or follow the tutorial in the tutorials section

Q: Why isn't there a romfs.bin or romfs.romfs?!?
A: Folder works too

Q: Do I need to edit NPDM?
A: Nope.

Q: Game X doesnt work for Game Y!?!
A: Go check the tutorial mentioned above for a list of working redirects.

Q: WHERES THE DOWNLOAD
A: ... Read up.

Q: Why not Python 3?
A: Because I use Python 2 for compatibility with older software. @rkk seems to have a working Python 3 script handy on post #16 (https://gbatemp.net/threads/python-xci-to-layeredfs.507562/#post-8057198).


Changelog
V1.0.0 -- Rough script. Needs cleaned and organized but seems to work as a POC for now.

i get this after start

Exception in Tkinter callback
Traceback (most recent call last):
File "C:\Python27\lib\lib-tk\Tkinter.py", line 1541, in __call__
return self.func(*args)
File "C:\Users\Easton\Desktop\hak\xci_to_lfs.py", line 76, in do_the_thing
print(flip_donor + ' <-> ' + self.original_id)
TypeError: coercing to Unicode: need string or buffer, NoneType found

 

[dovah]

Your way of patching the npdm is pretty unclear. I think you're hardcoding the offset, which isn't correct since it varies from file to file.
Here is a script I made for myself (python 3 though), that has (I think) a cleaner way to do it:

TIDoffset = hexa.find(b"ACI0") + 0x10

the cleanest way is to get the ACI0 offset from the proper location (0x70 to 0x74) and not searching it by the magic packet ("ACI0"), imo.

 

[coolbird22]

Can someone elaborate what this does ?

 

[Deleted-451877]

the cleanest way is to get the ACI0 offset from the proper location (0x70 to 0x74) and not searching it by the magic packet ("ACI0"), imo.

Feel free to replace the relevant line with

npdm.seek(0x70)
TIDoffset = upk("<I", npdm.read(0x4))[0] + 0x10

or similar then. In my opinion searching for the MAGIC is clearer to the outsider.

--------------------- MERGED ---------------------------

Can someone elaborate what this does ?

It extracts XCI cartridges dumps, and patches the main.npdm so that you can run it in lieu of another game.

 

[phn]

I'm not able to load DK Tropical Freeze. I don't know if it has something to do with the folders or something. I can't try putting the whole file inside the sdcard because I got it on fat32. Trying now arms!

 

[ChainedHope]

Updated to version 1.0.1
Now has a notification when starting the process and when it is finished. Also changed the NPDM to better match @rkk 's python 3 script. His method was a lot better than mine (hardcoded) and there isn't much point in reinventing the wheel on this one.

As a side note I'm working on a homebrew app that will let you swap games into a donor id. You'll just need to put your games in a generic '/games' folder on the switch and tell it which donor id you are using (It will include all the free games in the tutorial section as donors)

[I also looked into doing the xci conversion on the switch itself but unless someone has a version of hactool that works I think I will pass on it]

 

[phn]

Arms works flawlessly, thanks man! I'm gonna try with the updated version to play DK...
The error it gives me is just after patching the NPDM. It says the "tmp" dir isn't empty and it is because there are 2 files (one ticket and another one).

Sorry but cannot post imgs https://imgur.com/a/AMixCTs

 

[Thetoto]

Updated to version 1.0.1
Now has a notification when starting the process and when it is finished. Also changed the NPDM to better match @rkk 's python 3 script. His method was a lot better than mine (hardcoded) and there isn't much point in reinventing the wheel on this one.

As a side note I'm working on a homebrew app that will let you swap games into a donor id. You'll just need to put your games in a generic '/games' folder on the switch and tell it which donor id you are using (It will include all the free games in the tutorial section as donors)

[I also looked into doing the xci conversion on the switch itself but unless someone has a version of hactool that works I think I will pass on it]

Yeah great idea. So you just have to move files from /games/ to /atmosphere/ID/ in your homebrew (and maybe edit .npdm?) Hyped on your homebrew

 

[ChainedHope]

Yeah great idea. So you just have to move files from /games/ to /atmosphere/ID/ in your homebrew (and maybe edit .npdm?) Hyped on your homebrew

You would just have a folder on the SD card with /games/'game name'/atmosphere/...
So you would run mine or some other tool to get the atmosphere directory and put them on the SD card in the generic games folder. The homebrew app would just store what game was currently being used and swap the files back and forth on the SD card for the chosen titleID. Very limited by your SD card tho. I've managed to get games to move around the SD card, but there is no UI right now.

--------------------- MERGED ---------------------------

Arms works flawlessly, thanks man! I'm gonna try with the updated version to play DK...
The error it gives me is just after patching the NPDM. It says the "tmp" dir isn't empty and it is because there are 2 files (one ticket and another one).

Sorry but cannot post imgs https://imgur.com/a/AMixCTs

[Not my script in image.... but] Didn't know that some XCIs would export those. I've not tested any new revision carts (like DK). You should be fine to just delete those files tho. Deleting the folder is the last step in the process and done after everything else. Also it can be a folder. It does not have to be a bin/romfs file. This is in the FAQ.

 

[phn]

Now DK works, don't know if it has to be with your update or what, but I tried like 8 times this afternoon and only got to work after you pushing 1.1. Thanks again!!

 

[ao3pixel]

Traceback (most recent call last):
  File "xci_to_lfs.py", line 13, in <module>
    import tkFileDialog as filedialog
  File "C:\Python27\lib\lib-tk\tkFileDialog.py", line 43, in <module>
    from tkCommonDialog import Dialog
  File "C:\Python27\lib\lib-tk\tkCommonDialog.py", line 11, in <module>
    from Tkinter import *
  File "C:\Python27\lib\lib-tk\Tkinter.py", line 38, in <module>
    import FixTk
  File "C:\Python27\lib\lib-tk\FixTk.py", line 68, in <module>
    import _tkinter
ImportError: DLL load failed: %1 is not a valid Win32 application.

Any help on this? Tried looking up Tkinter on google but apparently it comes preinstalled.

 

[ChainedHope]

Traceback (most recent call last):
  File "xci_to_lfs.py", line 13, in <module>
    import tkFileDialog as filedialog
  File "C:\Python27\lib\lib-tk\tkFileDialog.py", line 43, in <module>
    from tkCommonDialog import Dialog
  File "C:\Python27\lib\lib-tk\tkCommonDialog.py", line 11, in <module>
    from Tkinter import *
  File "C:\Python27\lib\lib-tk\Tkinter.py", line 38, in <module>
    import FixTk
  File "C:\Python27\lib\lib-tk\FixTk.py", line 68, in <module>
    import _tkinter
ImportError: DLL load failed: %1 is not a valid Win32 application.

Any help on this? Tried looking up Tkinter on google but apparently it comes preinstalled.

Looks like Tkinter is broken.. Try reinstalling and making sure your PATH variable is correct. You can also try the other tool here in this forum section. (Its a little bit easier than dealing with Python issues)

 

[ChainedHope]

Updated to V1.0.2 for use with GameRedirector (PyNX Homebrew, will be up soon).

 

[salamandrusker]

Wrote this up pretty quickly to help people having issues with using hactool. [Also I realize I put this in the wrong section... If a mod would move it that would be cool... I wasn't paying attention when I posted]
Requirements

• Windows OS
• Python 2.7
• Tkinter (Python library, google it)
• hactool
• keys.dat

Setup

1. Put the python file, hactool, and your keys.dat (don't ask me for it) in the same directory.
2. Run 'python xci_to_lfs.py' in cmd or powershell (while in the file directory)
3. Use the GUI to set your TitleID and pick your XCI
4. Wait a minute. I didn't put anything in the script to notify you when its done. For a 2GB game it takes about a minute. YMMV. The last thing it does is delete the tmp folder, so when its gone you are good to go.
5. Copy the Atmosphere folder to the root of your SD Card
6. Follow the rest of the tutorial found in the tutorials section [How to use LayeredFS for Backup Loading] (Can't post links)

Note that YMMV.

DOWNLOAD: https://mega.nz/#!Y6ByFbxI!3-whFid8d0UwMuDMEqrwM879d33SRZ9-GOuHwlpCkZU

FAQ (But you just posted o.O ... Yeah but I know you are going to ask)
Q: I'm getting a Python not found error
A: Install python and make sure it is in your PATH variable. Google is your friend

Q: WhErE Do I GEt KEYS.dat
A: Find it yourself or follow the tutorial in the tutorials section

Q: Why isn't there a romfs.bin or romfs.romfs?!?
A: Folder works too

Q: Do I need to edit NPDM?
A: Nope.

Q: Game X doesnt work for Game Y!?!
A: Go check the tutorial mentioned above for a list of working redirects.

Q: WHERES THE DOWNLOAD
A: ... Read up.

Q: Why not Python 3?
A: Because I use Python 2 for compatibility with older software. @rkk seems to have a working Python 3 script handy on post #16 (https://gbatemp.net/threads/python-xci-to-layeredfs.507562/#post-8057198).


Changelog
V1.0.2 -- Made the files extract into game folders (same name as .xci) for use with Game Redirector (PyNX)
V1.0.1 -- Added notifications, changed the NPDM writing to not use hardcoded offsets
V1.0.0 -- Rough script. Needs cleaned and organized but seems to work as a POC for now.

it works in 3.0.0?

 

[Deleted-451877]

it works in 3.0.0?

Read the FAQ, it doesn't.
Mine does though, but it doesn't have a GUI. By default, it decrypts every XCI it can find in its root location and subsequent subfolders.
You'll need to edit in the full path to your hactool executable, as well as have your keys located in C:\Users\[...]\.switch\prod.keys (or edit the script a little bit).
Here's an updated version of it btw, I ran into some problems when working on my SD, that I solved using dirty quote and backslash shenanigans.

import os
from struct import pack

hactoolPath =

# directory = input('Input directory? ')
directory = os.path.dirname(__file__)

for root, dirs, items in os.walk(directory):
    for item in items:
        if item.endswith('.xci'):
            # TID to write
            newTID = input('New TID to write in ' + os.path.join(root, item) + '?\n')
            
            outputFolder = root + '\\atmosphere\\titles\\' + newTID
            os.makedirs(outputFolder, exist_ok=True)
        
            # Extract the secure partition
            filePath = os.path.join(root, item)
            print('Extracting ' + filePath + '...')
            commandLine = hactoolPath + ' -t xci "' + filePath + '" --securedir="' + outputFolder + '" 1>nul 2>nul'
            os.system(commandLine)
            
            # Find the biggest nca
            biggestSize = 0
            biggestNCA  = ''
            for file in os.listdir(outputFolder):
                if file.endswith('.nca'):
                    size = os.path.getsize(os.path.join(outputFolder, file))
                    if size > biggestSize:
                        biggestSize = size
                        biggestNCA  = file
            
            # Decrypt it
            filePath = os.path.join(outputFolder, biggestNCA)
            print('Decrypting ' + filePath + '...')
            commandLine = hactoolPath + ' "' + filePath + '" --exefsdir="' + outputFolder + '\\exefs" --romfs="' + outputFolder + '\\romfs.bin" 1>nul 2>nul'
            os.system(commandLine)
            
            # Delete NCAs
            print('Deleting NCA files...')
            for file in os.listdir(outputFolder):
                if file.endswith('.nca'):
                    os.remove(os.path.join(outputFolder, file))
            
            # Patch main.npdm
            with open(outputFolder + '\\exefs\\main.npdm', 'rb+') as npdm:
                # TID offset & length
                hexa = bytearray(npdm.read())
                TIDoffset = hexa.find(b'ACI0') + 0x10
                TIDlength = 0x8
                
                # Replace TID
                print('Writing new TID...')
                hexa[TIDoffset:TIDoffset+TIDlength] = pack('<Q', int(newTID, 16))
                npdm.seek(0)
                npdm.write(hexa)
                print('Done\n')