RELEASE XCI BUILDER

Discussion in 'Switch - ROM Hacking, Saves, Translations & Tools' started by JRoad, Sep 2, 2018.

  1. JRoad
    OP

    JRoad GBAtemp Fan

    Member
    4
    May 18, 2018
    Spain
    EDIT:
    I'll open soon a new thread for NX-Batchbuilder which will integrate all my current tools plus will feature a reverse mode to change back to nsp the files converted and fix the old conversions that needed the [lc].nsp transforming them in the ones NUT_cleaner does.

    23/09/2018. Added NUT_BATCH_CLEANER v0.30 . Check github for latest changes.
    v0.30- Added new options in zconfig.
    > Option 4: Way the items are organized.
    * line: scattered in output folder
    * folder: in folders according to game name
    > Option 5 and option 6: Output folders for xci and nsp. As default using
    a common folder for both.
    > Option 7: Preserve stripped files from the nsp as a .zip file for future
    restore function. (True by default)
    > Option 8: Replace original file with clean/converted files.As default false.
    TAKE CARE WITH THIS ONE. BETTER CHECK ALL WORKS FINE BEFORE ACTIVATING IT.
    Added option to erase list.txt in manual mode.


    21/09/2018. Added NUT_BATCH_CLEANER v0.21 with some fixes:
    v0.21- Fixed issue with manual mode not detecting the user repacking input correctly
    Added better check for dlcs and updates, according to id
    Added another config parameter to "nsp_cleaner_options.cmd" (xnsp_ifnotgame)
    > This parameter serves to repack dlcs and updates as nsp if processing list as xci only.
    Options are:
    * true: repack dlcs and updates as nsp
    * false: skip dlc and update repacking when exporting as xci.

    20/09/2018. Added NUT_BATCH_CLEANER v0.2 with xci_repacking and more:

    https://github.com/julesontheroad/NUT_BATCH_CLEANER/releases/tag/0.2
    Well may as well post this since I won't do a new thread till I integrate it with current builder:
    Automator for nut by Blawar to make nsp's without titerights encryption and tickets. If you use this kind of nspfiles with the builder you'll get xci files that doesn't require ticket installations aka [lc].nsp
    https://github.com/julesontheroad/NUT_BATCH_CLEANER
    As allways an automator of a process you can achive manually with NUT from blawar:
    https://github.com/blawar/nut

    XCI BUILDER (v0.7)

    This and it's brother NX-Trimmer are meant to highlight the excelent work done by LucaFraga with hacbuild which is actually fully functional with a small issue if you now how to use it.
    The issue with hacbuild is that it can only build xci files with 4nca in the secure partition. This is probably because a parameter in the header of the xci which points to the number of files in the secure partition. Probably was thought as a fixed value since the normal partition only have more than 4 files in rare occasions. This was solved via a compromise (in these rare occasions, as described bellow, but requires a better approach by identifying the issue and editing hacbuild)
    You're wellcome to modify, repack, make a gui.. etcera of this as long as you give credits to at least to LucaFraga, SciresM and CVFireDragon wich made the hardest work, especially Luca and Scires. Mine is just a workflow facilitator of something you can do manually with these programs.
    More in
    https://github.com/julesontheroad/XCI_Builder

    Note1: Replaced ztools according to the advice by @Bigjokker. It now features a hacbuild version rebuilt by him. (REV2). All included in rev2 file but keeping Bigjokker file linked as a way of showing gratitude for his help.
    more info in https://gbatemp.net/threads/nx-trimmer-v-0-31-an-update-partition-trimmer-based-in-hacbuild.516870/

    Note 2 (important): Ok, so for big games conversion hacbuild doesn't seem to be the most stable program. I'm getting instances i which it completes the process flawlesly and instances in which it stops while building from root.hfs0 (only really big games like RE Revelations 2). I'm gonna change the batch from auto-xci building to manual xci building by steps. It should fix this issue, i'll probably updload it next week to make some proper testing.
    Also if you have a root.hfs0 in your final folder the xci file is probably not completed so erase the xci file. I'm gonna make another batch to help you build those root.hfs0 in an xci so you don't have to repeat the process from the start. I'll probably upload it on sunday as i'll be kinda bussy all day tomorrow.


    1. Description
    This tool purpose is to convert nsp files to xci files. This is a batch application which serves to automate the workflow between the following programs:
    This is a batch application which serves to automate the workflow between the following programs:
    a.) hacbuild: Program meant to create xci files from nca files, made by LucaFraga.
    https://github.com/LucaFraga/hacbuild/
    b.) hactool: Program which function is give information, decrypt and extract a lot of different kind of files us by the NX System.
    Hactool was made by SciresM
    https://github.com/SciresM/hactool/
    c.) nspBuild: Program meant to create nsp files from nca files.
    https://github.com/CVFireDragon/nspBuild/
    NX-Trimmer was also inspired by "A Simple XCI, NCA, NSP Extracting Batch file (Just Drag and Drop) with Titlekey decrypt"
    by Bigjokker and published in gbatemp:

    2. Requirements
    - A computer with a Window's OS is needed
    - You'll need to complete keys.txt in ztools with the keys needed by hactool.
    - Optionally complete header_key.txt with xci_header_key
    - You'll need to have Python installed for nspbuild to work correctly
    - You'll need to have at least .net frameworks 4.5.2 installed so hacbuild can work correctly.

    3. Functions
    - Conversion from nsp files to xci files
    - Creation of small nsp files (normally less than 1mb) to install game titlekey in the console.
    - "game_info" files extraction from xci files
    - The xci files created incorporate a blank update and normal partition which makes them smaller than
    tipicall xci files.

    4. Changelog
    **CHANGED header_key.txt to header_key(optional).txt ***
    If you want to use the header_key.txt fill the text file and erase the (optional) mark

    header_key(optional).txt > fill the "xci_header_key" value > change name to header_key.txt to use it.

    v0.6.5 - Lot of changes made to previous version:
    I - Added compatibility with games with more than 5 nca. (Games that include an html manual)
    The manual was stripped from the game, this doesn't impede game execution. And game is fully functional.
    If you try to access the manual in these kind of games this action won't get any result and you will be able to keep playing.
    Games are fully funtional without manual nca and there are few games that includes it to begin with.
    II - Output route was moved to "output_xcib" so everything is better organiced XCI_Builder can be in the same folder as NX-Trimmer,
    which now uses el usará "output_nxt" as output folder.
    III - Reformed ztools folder, erasing not needed files.
    IV - Little mod to hacbuild.exe so it takes "xci_header_key" from ztools. "xci_header_key" it's actually not needed everything works without it
    but hacbuild gives a warning wich was ugly for the batch output
    V - Added template for keys.txt and header_key.txt in ztools
    VI - Added tag system for the files output. It goes as follows:
    a) Se eliminan las tags [] de los ficheros. Para eliminar cosas como trimmed.
    b) Se eliminan los caracteres _ (más que nada porque no me gusta como quedan)
    c) Se añade las siguientes tags a la salida.
    [xcib] xci converted with XCI_Builder
    [nm] "no manual", manual was erased to get the resulting xci to work
    [lc] In output nsp refers to the small nsp needed to get the xci to work.
    VII - Added options in batch header. Edit with notepad++ to select them
    a) "preservemanual" By default 0. Set at 1 if you want the manual nca to not be deleted. If set at 1 the manual will be sent to output folder.
    NOTE1: Is advised to ¡¡NOT INSTALLED the manual .nca!! as the game will give error if you try to access the game manual.
    NOTE2: The xci without manual is completely functional and if manual is selected it won't happened anything.
    NOTE3: This option is thought for a future xci to nsp converter that will revert the process.
    b) "delete_brack_tags" Erase [] tags like [trimmed]- By default at 1
    c) "delete_pa_tags" Erase tags () like (USA) - By default at 0. If activated it could also erase [] tags
    NOTE: If upgrading from previous version replace ztools folder hactool version was upgraded and an small modification was made in hacbuild code.
    v0.5.0 - First launch in elotrolado.net
    v0.5.1 - v0.5.5.1 - Little fixes

    5. Limitations
    - xci files only work with SX OS
    - To load the xci files you'll need to get the titlekey which I call license in your NX console. It can be
    achieved in various ways.
    a.) Installing the original game previously via eshop
    b.) Installing the original nsp previously via devmenu/SX OS/Tinfoil...
    (It doesn't need to be installed in the current moment)
    c.) Installing the license file obtained with the programo "game_name[lc].nsp"
    This file needs to be installed with SX OS installer.
    d.) For nsp files with more than 5nca is needed to delete the nca manual
    e.) Processing times for more than 4gb games is longer than it shoul be proporcionallu. Probably linked to hacbuild fix number 5 regarding the overflow
    error in this kind of files. (Could be interesting to investigate if a fastest processing approach is possible)
    f.) The symbol "!" gives error when passed to hacbuild. So rename snipperclips file

    6. Use of the application
    I.- First fill "keys.txt" in ztools folder so hactool can work properly.
    More info: https://github.com/SciresM/hactool
    II.- Optionally fill the file header_key.txt in ztools with xci_header_key value
    III.- To convert a nsp to xci dragg the nsp file over "XCI_Builder_v0.6.x.bat" and wait for the system window to close
    IV.- You'll get a folder with the name of the game in the output folder. Inside you'll find two files.
    - "game"[xcib].xci -> Converted nsp
    - "game"[lc].nsp -> License file to ge titlekey in the system.
    V.- For game_info.ini you can edit the preset or give it as it is. I'm getting 100% compatibility as it is.
    VI.- Install file [lc].nsp with SX OS custom installer or older tinfoil versions
    VII. - Load the xci file and enjoy

    7. Compatibility
    With current changes and accepting the described limitations this method should be compatible with all current xci files.
    At least I didn't find any issues.

    8. Thanks and credits to

    LucaFraga, SciresM and CVFireDragon
    Also thanks to all members from gbatemp and elotrolado.net
     

    Attached Files:

    Last edited by JRoad, Sep 23, 2018
  2. TunaKetchup

    TunaKetchup GBAtemp Regular

    Member
    5
    Mar 20, 2018
    Vietnam
    Thanks
     
  3. JRoad
    OP

    JRoad GBAtemp Fan

    Member
    4
    May 18, 2018
    Spain
    Added this other one as the time limit for threads was lifted. It's better for the two lines of application of hacbuild to be split.

    I want to say this:
    All people are wellcome to redistribute, modify, add a gui or whatever they want as long as they give credits at least to LucaFraga (hacbuild), SciresM (hactool) and CVFiredragon (nspbuild). Especially LucaFraga who is the creator of hacbuild in which all of this is based.
    Mine is a workflow facilitator between the three of them for a process that can be manually achieved. Only to make mine and other people live easier in regards of completing the process.
    Also please try to respect the limitations and usage described in the readme. If you do this you're good with me.

    Also let me express my gratitude to @Bigjokker as his batch application was to say the least a big inspiration for this one:
    https://gbatemp.net/threads/a-simpl...t-drag-and-drop-with-titlekey-decrypt.513300/

    All said above is true for NX-Trimmer and XCI-Builder
     
    Last edited by JRoad, Sep 2, 2018
  4. Bigjokker

    Bigjokker GBAtemp Regular

    Member
    3
    Jan 4, 2015
    United States
    To make it smaller you can remove the sfk.exe tool in ztools. As it is not used in your bat.
     
  5. JRoad
    OP

    JRoad GBAtemp Fan

    Member
    4
    May 18, 2018
    Spain
    Thanks I missed that one cleaning my working folder.

    Edit:
    It's a little late here in Spain so I'll update ztools folder tomorrow in here and in both projects git.

    Thx again.
     
    Last edited by JRoad, Sep 2, 2018
  6. Leozairus

    Leozairus Advanced Member

    Newcomer
    2
    Jun 15, 2018
    Brazil
    thanks! amazing work
     
  7. loiuyt1207

    loiuyt1207 Member

    Newcomer
    3
    Apr 14, 2015
    Help me how to repack xci file (or nsp) after modified. I want to mod some games
     
  8. JRoad
    OP

    JRoad GBAtemp Fan

    Member
    4
    May 18, 2018
    Spain
    For that you'll need to repack it to nsp first to use this directly or to use hacbuild by LucaFraga via command line. If using hacbuild you create first a folder for your game, then inside you create 3 folders: normal, secure and update, then you move your nca files to secure, you then take my "game_info_preset.ini" and rename it to "game_info.ini" and put it in the game folder. Then you call -> hacbuild xci-auto "game folder" "output.xci" or repackthe nca files in an nsp first an drag them in my bat.

    To rebuild to nsp I use https://github.com/CVFireDragon/nspBuild/releases from CVFireDragon.

    BTW I tried the SDK based builders and with Simon's builder but I didn't have many luck in SX OS with it's generated files. With the SDK ones the file will launch but freeze after the Nintendo logo, for me at least.
    I had luck though with the "My hero Academia: One's justice" patched nsp from @Biff627 . I don't know if he did something special to rebuild the nca files but it works in SX OS after being patched.
    Also didn't tinkered much with this yet so I could be making some mistakes when rebuilding the nca files.
     
  9. Bigjokker

    Bigjokker GBAtemp Regular

    Member
    3
    Jan 4, 2015
    United States
    I made a fork of your XCI_Builder github. In it I cleaned up the CMD window for the bat and gave recognition to the program builders. Also the nspBuild code was not working for me. Not making a [lc].nsp so I changed it to work on my computer.

    If you want to take a look at it https://github.com/bigjokker/XCI_Builder
     
  10. JRoad
    OP

    JRoad GBAtemp Fan

    Member
    4
    May 18, 2018
    Spain
    Ok, fine by me. We all would benefit from better tools. ¿Do you now if nspbuild error was Python version related?
     
  11. Bigjokker

    Bigjokker GBAtemp Regular

    Member
    3
    Jan 4, 2015
    United States
    I had both versions of python but still was not working although I think it needs Python 2.7. Seemed the code was adding backslashes to the arguments (i.e C:\\desktop\\ ztools\\*.nca) on my computer.
     
  12. JRoad
    OP

    JRoad GBAtemp Fan

    Member
    4
    May 18, 2018
    Spain
    I have 3.7 installed supposedly according to it's git it should work with both 3.x and 2.x python with the 3.0 Beta
     
  13. Bigjokker

    Bigjokker GBAtemp Regular

    Member
    3
    Jan 4, 2015
    United States
    OK working with 3.7 now on 3.0 Beta. I had to reinstall it. Thanks!
     
  14. spupok100

    spupok100 Newbie

    Newcomer
    1
    Aug 19, 2018
    Moldova
    Is possible to merge main nsp and update nsp to one xci?
     
    Moko likes this.
  15. JRoad
    OP

    JRoad GBAtemp Fan

    Member
    4
    May 18, 2018
    Spain
    Yes and no.
    In the normal (official) way it's possible but right now hacbuild prevents it. The issue is it's only allowing 4 files in the secure partition (something related to the global header, as the partition header it's built correctly)
    In a non standard way you can, you need to do a custom nsp (look pySDK as it's the only thing we can talk in temp). For SX OS custom nsp files will work but only if made from a cartridge (xci) game. It seems to be related to the ACID signature in eshop game being calculated differently as in cartridge games. Well you figure out how to mix the ROMFS folder in a LayeredFS way and then you figure out how to make a custom nsp. You then can convert it to xci with the converter.
    Right now it's a big pain in the ass of a process.
     
  16. spupok100

    spupok100 Newbie

    Newcomer
    1
    Aug 19, 2018
    Moldova
    I can't find a method to convert modifed romFS folder to *.nsp.
    Can you help?
    I tryied to use NCABuilder but have no luck.
     
  17. JRoad
    OP

    JRoad GBAtemp Fan

    Member
    4
    May 18, 2018
    Spain
  18. Bigjokker

    Bigjokker GBAtemp Regular

    Member
    3
    Jan 4, 2015
    United States
    @JRoad I was looking at HACbuild and figured out how to put more than 4 nca's in a XCI.

    I built a new batch because didn't need nspbuild anymore. Also I have not tested it as I don't have SX OS.
     
    Type_O_Dev likes this.
  19. JRoad
    OP

    JRoad GBAtemp Fan

    Member
    4
    May 18, 2018
    Spain
    Sadly it didn't work. I might explained myself badly. You could allways get more than 4 files in secure, in fact it builds. The problem is when you build with more than 4 files in secure SX OS will either give a cartridge error or freeze in the SX menu launcher. I did limit the error to more than 4 files in secure.
    I'm sure the issue is in one of the fixed values in the global header. At first I realized the problem with rev games, and thought to be an issue with the lack of order in the secure partition (see snipperclips rev 1, for example, with 10 files in secure) but I did forced it to built with correct order and it didn't work. I made exact hf0 files for secure, normal and update and the only part different was the header and it didn't work.
    For now I limited the issue to the global header and more than 4 files in secure but I don't know exactly the motivation.
    Also, I'm building everything as card one since I tried building in card 2 at first and I had freezes that way, instead building even card 2 games as card 1 works and they run without issues.
     
    Bigjokker likes this.
  20. Mikep90

    Mikep90 GBAtemp Regular

    Member
    3
    May 29, 2018
    United States
    when using this its not outputting game"[lc].nsp. Im only getting the xci.
     
Loading...