RELEASE NSC_Builder. Nintendo Switch Cleaner and Builder. (Game+updates+dlc in a single xci)

Discussion in 'Switch - ROM Hacking, Saves, Translations & Tools' started by JRoad, Nov 4, 2018.

Loading...
  1. JRoad
    OP

    JRoad GBAtemp Advanced Fan

    Member
    7
    May 18, 2018
    Spain
    IMPORTANT NOTES:
    Get new version on:

    https://github.com/julesontheroad/NSC_BUILDER/releases

    Direct links for last versions:

    - NSC_BUILDER v0.91b

    32 bits: https://github.com/julesontheroad/NSC_BUILDER/releases/download/v0.91b/NSB_091bx86.zip
    64 bits: https://github.com/julesontheroad/NSC_BUILDER/releases/download/v0.91b/NSB_091bx64.zip
    Many errors are related of not having all the needed keys in your keys.txt or to have a wrong format for them. Please stick to the formate in the keys_template.txt
    --> Just paste them like this: xci_header_key = FFFFFFFFFFFFFFFFFFFFFFFFFFFF (Not the real key)
    Alternatively notepad++ has an option to convert end of lines in files from linux(UNIX) to windows format.
    If you have issues getting a link detected try to change any weird characters in the file's name, for example "&" or '
    If the program does something unexpected go to the configuration and input "d" I could let on some option in the config file when testing instead of the intended defaults.
    If you use Atmosphere or Kosmos you'll need the patches from the4n:
    https://gbatemp.net/threads/i-heard-that-you-guys-need-some-sweet-patches-for-atmosphere.521164/
    To launch them download latest fusee-primary-0.8.2.bin and add this line to the config if you use kosmos -> payload=bootloader/payloads/fusee-primary-0.8.2.bin
    To install multicontent nsp your best option is to use dz-tinfoil by blawar or SX OS Installer (rom-menu):
    dz-tinfoil: https://github.com/digableinc/tinfoil
    If you need to activate unsigned code in dz-tinfoil the password is the konami code.
    For new SX installer is the same since is actually dz-tinfoil. You'll need to activate unsigned code with the konami code.
    Always try to do conversions or file modifications yourself and to get originals from trusted sources.

    NBA 2k19 goes above 32gb with the update. There aren't 64gb cartridges in the market currently or flags in the SDK so it won't work. Not sure if the limitation is actually in Horizon current capabilities or in SX OS code. The error is kinda the same as before they released they 32gb cartridges patch.

    This one takes the mantle of Nut Cleaner and XCI_builder. Don't use XCI_Builder or any of it's variations anymore. The code is archived and outdated.


    ---END OF IMPORTANT NOTES---

    Changelog:
    For changelogs see the second post in this thread.

    1. Description
    NSC_Builder is the merged Project that continues xci_builder and Nut_Batch_Cleaner.
    NSC_Builder is based both in the works of Blawar’s nut.py and Luca Fraga’s hacbuild and powered by “squirrel” a nut’s fork with added functions that removes the CDN based functions from nut while tweaks the title-rights modification functions and adds some useful ones for file management.
    From version v0.8 the program doesn’t rely on hacbuild for xci generation and new code was made for a better integration on squirrel.
    Squirrel will get a new github repository soon and be packed as exe for NSCB from beta v0.8. Old squirrel code can be seen in the NSCB main repository, new code will be published in it’s own repository at the end of NSCB beta phase after some cleanup it’s done on it’s code.

    2. What’s the meaning of “REMOVING TITLE RIGHTS”.
    When you remove the titlerights encryption from nsp files you can install the games without any need of tickets, which leaves a smaller trackable footprint on your console, providing you aren’t sending telemetry data to Nintendo.
    It also helps in the conversion from nsp to xci files allowing to not install tickets externally.

    3. What can I do with this program?
    Current version of the program allows you to:
    1.- Make multi-content xci or nsp files.
    2.- Erase titlerights encryption from nsp files.
    3.- Build xci files without the “update partition” which means they take less space on your storage.
    4.- Take off deltas from updates
    5.- Split multi content back into xci or nsp files
    6.- Change the packing of the content between xci and nsp
    7.- Lower the Required System Version to the actual encryption of the game.
    8.- Lower the masterkey needed to decrypt a game.
    9.- Check out information from a xci and nsp, including the Firmware needed to be able to execute it, the game info, the size of the nca content…
    10.- Check data from nacp and cnmt files without extracting them from nsp\xci
    11.- Repack xci and nsp content in formats compatible with fat32
    12.- Mass build xci files and nsp files in single and multi content format
    13.- Rename nsp,xci files to match it's content
    14.- Verify nsp, nsx, xci y nca files
    15.- Output information in text format
    16.- Extract content of nsp files and secure partition of xci files
    17.- Set jobs for later in multi mode
    18.- Separate jobs by based-titleid in multi mode
    19.- Remove bad characters from filenames (sanitize) or convert asian names to romaji
    20.- Extract nca file contents for base games and dlcs or extract ncas as plaintext
    21.- Joiner for xc*,ns* and *0 fat32 files

    4. Batch modes:
    The program has 2 modes:
    - auto mode: you drag nsp files individually or folders with several files over the batch to enter in auto mode.
    - manual mode: you double click the batch and you can build a list of files to process.
    The behavior of the auto-mode is configured trough the “Configuration menu in manual mode”.

    5. Important

    This program attempts to modify the minimum data possible in nsp and xci files, due to that reason it requires signature patches to ignore both signatures at NCA headers. Firmwares that already include them are:
    - SX OS
    - ReiNX
    https://github.com/Reisyukaku/ReiNX/releases
    - For Kosmos use joonie86 sigpatches and Hekate5.0 or joonie86 Hekate Mod "a.k.a J"
    https://github.com/Joonie86/hekate/releases/tag/5.0.0J
    - For atmosphere use the4n sigpatches
    https://gbatemp.net/attachments/2-0-0-8-1-0-zip.170607/
    To install multi-nsp you need a installer compatible with them. Reported compatible installers are:
    - SX OS rom-menu
    - SX OS installer
    - Blawar’s tinfoil:
    https://github.com/digableinc/tinfoil
    - Blawar’s lithium:
    https://github.com/blawar/lithium

    6. Requirements
    - A computer with a Window's OS is needed
    - Fill keys_template.txt on the ztools folder and rename it to keys.txt
    You can get a full keyset with Lockpick if your console is at FW6.2 or
    A friend can lend you the needed keys.
    If you want to add the xci_header_key a friend will need to lend it to you.
    https://github.com/shchmue/Lockpick/releases

    7. Limitations
    - You can't make multi-content xci files with more than 8 games. It'll give error when loading in horizon. I suspect it may be a qlauncher limitation so it could work with theme mods but INTRO didn't test it.
    Note: This means “games”, updates and dl car not hold by that limitation.
    - Title-rights remove dlcs give a message prompt of incomplete content for some games from 6.0 onwards, that message can be skipped and the dlcs will work fine despite the prompt.
    -- For more information read the changelogs and the updated readme --

    8. Thanks and credits to

    NSC_Builder is based on:
    a.) Nut: Without the work of "blawar" one of the most talented Switch sceners nothing of this would be possible at this point.
    https://github.com/blawar/nut
    b.) Hacbuild: The xci repacking functions are based on hacbuild's code, made by LucaFraga
    - Original hacbuild: https://github.com/LucaFraga/hacbuild) by LucaFraga
    - Revised hacbuild by me: https://github.com/julesontheroad/hacbuild
    c.) Big thx to 0Liam for his constant help.
    d.) pyNCA3,pyNPDM,pyPFS0,pyRomFS libraries adapted from pythac (made by Rikikooo)
    Also thanks to:
    AnalogMan. He made splitNSP.py, figured the needed block size for Horizon format splitted nsps (wich differs from the splitted xci block size) and the need to archive the folders)
    https://github.com/AnalogMan151/splitNSP/releases
    Thx to MadScript77 his great suggestions,specially the idea of profiles for the batch.
    Thx to 0mn0 and the old SH crew for always being helpful.
    Thx to evOLved, Cinnabar and a certain dragon for their help and good suggestions.
    Also thanks to all members from gbatemp, elotrolado.net and my friends at discord ;)

    Video tutorials by @grabman
    Warning: Spoilers inside!
     
    Last edited by JRoad, Sep 30, 2019
  2. JRoad
    OP

    JRoad GBAtemp Advanced Fan

    Member
    7
    May 18, 2018
    Spain
    NOTE: Added at the end of the post a list with the patchable and unpatchable linked account games currently
    NSC_BUILDER v0.91b. Introducing NSCB File_Info:
    - Current functions are:
    * Game Information. Combines data read from file with eshop data from nutdb
    * Description: Description from the eshop (nutdb)
    * Image Gallery: Pictures from the eshop (nutdb)
    * BaseID File-Tree: Shows the lates version for dlcs and updates associated to the game (nutdb)
    * Titles: Advanced Files List from NSCB-new
    * NACP Reader from NSCB
    * NPDM Reader from NSCB
    * CNMT Reader from NSCB
    * Verification from NSCB, till Level 2 so it's loaded fast. Use normal NSCB for hashing for now.
    See Instructions in github:
    https://github.com/julesontheroad/NSC_BUILDER/releases/tag/v0.91
    NSCB v0.90c- Bugfixes:
    - Fixed issue with renamer where it didn't delete processed items from list when using one worker
    - Disable threaded renames since for some reason the changes i did in 0.90b broke it apparently. Threaded database processing is working fine, i'll activate the renamer when i have time to debug the issue.
    - Updated the offline nutdb zip. Remember those files are set to update once a year, i'll add a manual updater in 0.91 but if you want to force the update of the files you can delete the json files in the db folder, also you don't need the zip, the normal NSCB is setup to redownload the json files every 24h, this can be modified editing "NUT_DB_TITLES_URL.txt" and "NUT_DB_REGIONS_URL.txt" as described in v0.90 release notes.


    NSCB v0.90b- Bugfixes:
    - Fixes issue with name detection for dlcs non listed in the nutdb json files

    - Adds nsx format to allowed for the info functions. Except for the npdm reader the reason is that without a titlekey you can't read the npdm file

    - Fixes routes for the download of the nut json files for exe releases. Now the json get download correctly to zconfig\DB

    - Fixes thread support for renaming and DB operations

    - Fixes the option 14 not working on the configuration

    - Note: For offline support on NSCB you need to download the json files and add a big refresh value to NUT_DB_REGIONS_URL and NUT_DB_REGIONS_URL.txt, then you can update them manually. I'll add support to manual update from the program on next release.

    • For now a nutdb.zip that has been added to the release, download it if you want to use the previously described scheme. The zip already includes the 4 nutdb json files that are being currently used, it also includes a value of 9999 hours in the refresh time, that's more than a year.
    • Download that file, and extract it in NSCB route. It already includes the zconfig folder. Overwrite the contents when asked by your system.
    NSCB v0.90- Several Bugfixes and aditions:
    * Fixed another error that prevented the creation of titlerights removed multi-nsps
    * Expanded xci trimming functions in indidual mode, after selecting trimming now there is these options:
    - Supertrimmer. Removes ending padding, padding between partitions, logo partition, firmware from update partition and content from normal partition. Also removes game updates from the xci but keeps dlc. Respects certificates. Meant for xci installation on tinfoil while keeping file verification
    - Supertrimmer respecting updates. Supertrimm that keeps game updates.
    - Normal trimming: Removes "FFFF" end padding
    - Untrimmer: Adds back "FFFF" end padding
    * Fixed dlc naming. Current approach will use the nut json files. The json files are downoaded to the DB folder for speed on demand. The url and refresh time of the json files is controlled by "NUT_DB_TITLES_URL.txt" and "NUT_DB_REGIONS_URL.txt" in zconfig.
    The refresh time has 3 parameters, representing hours, minutes and seconds. The defaulted time is 24h for each json used.
    Be aware that the json files are downloaded by demand. Titles json will be downloaded for dlc renames and US region for descriptions, genres, etc... The json will only refresh themselves when the refresh period is reached and the json is needed.
    * Fixed error in joiner mode
    * Fixed false positive in decription verification for the updates included in 5 recent xci which exefs doesn't include a main.npdm.
    For aditions look at: https://github.com/julesontheroad/NSC_BUILDER/releases/tag/v0.90


    Previous versions changelog:
    Warning: Spoilers inside!
    Some pictures:
     

    Attached Files:

    Last edited by JRoad, Sep 30, 2019
  3. FMCore

    FMCore Advanced Member

    Newcomer
    4
    Jul 10, 2018
    Canada
    Thanks mate!
     
    ChurrosBacon likes this.
  4. LucioDragon

    LucioDragon Advanced Member

    Newcomer
    3
    Jun 21, 2018
    Chile
    Now nsp can be mixed with dlc and updates to build then an xci?
     
    Mylink5 likes this.
  5. JRoad
    OP

    JRoad GBAtemp Advanced Fan

    Member
    7
    May 18, 2018
    Spain
    Yep. The config is set for auto-process folders like that.
    You can also access to manual mode. Option 2 multi-pack mode for this.
     
    LucioDragon likes this.
  6. huma_dawii

    huma_dawii GBAtemp Psycho!

    Member
    10
    Apr 3, 2014
    United States
    Planet Earth
    I think you should add to the title this (UPDATE+DLC+BASE merger) or something that is easy to tell people WHAT this does.
     
    matias3ds likes this.
  7. JRoad
    OP

    JRoad GBAtemp Advanced Fan

    Member
    7
    May 18, 2018
    Spain
    Ah, the think is you can put games in it. 8 games as maximum according to my tests.

    But if you think it would be better to have that in the title we can do that. Thing is, do I need to ping a mod to change the title?
     
    Last edited by JRoad, Nov 4, 2018
  8. huma_dawii

    huma_dawii GBAtemp Psycho!

    Member
    10
    Apr 3, 2014
    United States
    Planet Earth
    You probably do need to do that! :) People get confused by the title of the homebrew, I think is very important to be clear what your software does and the purpose, many people is very interested in the merging update + DLC to a base XCI and if the title of your thread doesn't mention any of that then it will get overlook and wont get as much attention (There are some people lost in the wild looking for this but they didn't even know you were working on it, you can contact @Quantumcat and see if you can get this in the FRONT PAGE because this is a BIG RELEASE for everyone).
     
    Quantumcat and Moko like this.
  9. kingraa777

    kingraa777 boom!

    Member
    6
    Apr 17, 2015
    ception: object of type 'NoneType' has no len()
    The system cannot find the file C:\Users\PC\Downloads\switch\SWITCH.
    The system cannot find the file C:\Users\PC\Downloads\switch\SWITCH.


    is this normal ?

    EDIT: the programme files need to be in this directory /SWITCH it seeems
     
    Last edited by kingraa777, Nov 4, 2018
  10. Firemanx

    Firemanx Member

    Newcomer
    3
    Nov 4, 2018
    United States
    @JRoad , first and foremost, tha I you so very much for your time on this project, it is very generous to the community.

    Repackaging an nsp into an xci, when launching the xci for the first time, does the packed update run and install, or does the total package run as if it is a dummy updated cartridge which doesn't need to install any local files?
     
  11. kingraa777

    kingraa777 boom!

    Member
    6
    Apr 17, 2015
    also my converted games dont show logos?
    edit now im getting 1kb files made with an nsp i know works?
     
    Last edited by kingraa777, Nov 4, 2018
  12. JRoad
    OP

    JRoad GBAtemp Advanced Fan

    Member
    7
    May 18, 2018
    Spain
    Not really mine are in F:\NSCB. I don't really now what your issue is related to, you can try and set absolute routes in your config folder but that error seems from the input, do you have a dot in your folder named "SWITCH"?
     
  13. huma_dawii

    huma_dawii GBAtemp Psycho!

    Member
    10
    Apr 3, 2014
    United States
    Planet Earth
    The games when you eject the card, the icon that stays in the system only shows the ORIGINAL VERSION without the update, is that normal? I merged XC2 Torna and when the XCI is loaded shows 1.0.1 and when I reboot the console it shows me the 1.0.0
     
  14. JRoad
    OP

    JRoad GBAtemp Advanced Fan

    Member
    7
    May 18, 2018
    Spain
    Ah no, it's like a rev game. It just loads the content inside the xci as if it was in the installed in the sd. All revision games work like that.
     
  15. kingraa777

    kingraa777 boom!

    Member
    6
    Apr 17, 2015
    not dot in that directory i was getting this error when the programme was in a different directory now i put the programme files in tht directory in the error message i dont get the message also what would cause the output files to have either a blank logo when converted or the programme just to return a small kb file ?
     
  16. FMCore

    FMCore Advanced Member

    Newcomer
    4
    Jul 10, 2018
    Canada
    I get a permission denied error
    upload_2018-11-4_14-7-21.

    and it hangs there.
     
  17. AnalogMan

    AnalogMan ultraSuMoFramework Dev

    Member
    12
    Apr 20, 2007
    United States
    So, if I merge an update into an XCI, the system doesn’t install that update internally? I know with games that have updates on them, when you insert the game it prompts you to install the update that was included on the cart to either the SD card or NAND.
     
  18. JRoad
    OP

    JRoad GBAtemp Advanced Fan

    Member
    7
    May 18, 2018
    Spain
    Torna as dlc or as game? If I understand you the meta loaded when the game is not mounted is the one from the game. Doesn't happen with mine really. If you have access to devmenu you can check if everything loaded correctly, also didn't check xenoblade yet, remember it had issues being cleaned.
    I plan to add back the [lc].nsp method for those cases. Also as soon as I add tools.bat you will be able to generate placeholders, that can force your meta\control to update to last version.

    — Posts automatically merged - Please don't double post! —

    Well it isn't working properly for you that's for sure. Can you make a logo? Do you remember how it was done?
    CD \d "program folder"
    NSCB.bat "ROUTE TO FILE\game.nsp" >log.txt 2>&1
     
  19. huma_dawii

    huma_dawii GBAtemp Psycho!

    Member
    10
    Apr 3, 2014
    United States
    Planet Earth
    Is the original XCI game + NSP update! D: unmounted shows just the 1.0 but when I mount it shows 1.0.1 I don't know if that affect in something, I don't have DevMenu don't know if it works on 6.0.1
     
  20. JRoad
    OP

    JRoad GBAtemp Advanced Fan

    Member
    7
    May 18, 2018
    Spain
    I don't really know why it tries to find something in your bin.
     
  21. FMCore

    FMCore Advanced Member

    Newcomer
    4
    Jul 10, 2018
    Canada
    It's not just the recycle bin

    upload_2018-11-4_14-17-15.
     
Quick Reply
Draft saved Draft deleted
Loading...