Homebrew RELEASE NSZ - Homebrew compatible NSP/XCI compressor/decompressor

[nicoboss]

I've got a folder of old NSPZ files i'd like to convert back to nsp, but they don't show up in the GUI. I read they should be supported?

Cool that you made use of my old file and long depreciated file format. Converting from NSPZ to NSP is quite easy however NSZ is the wrong tool for that. My old file format is implemented inside nsZip which you will find under https://github.com/nicoboss/nsZip. It has a nice GUI that will let you decompress your NSPZ files back to NSP. Once you have them as NSP you can use NSZ to compress them to the new NSZ file format if you want. NSZ is the successor of NSPZ and is an entirely different file format based on the same ideas. NSPZ files are only supported for file organization and extraction tasks with the new NSZ tool. If want to know more or have any questions about nsZip I recommend visiting it's gbatemp thread under https://gbatemp.net/threads/nszip-nsp-compressor-decompressor-to-reduce-storage.530313/

 

[zxr750j]

Thanks @nicoboss , I think I used nszip to convert them back then! I also remember why I needed to shrink the files: I'll have to do the conversion to nsz in batches due to lack of storage...

 

[mathew77]

I've got a folder of old NSPZ files i'd like to convert back to nsp, but they don't show up in the GUI. I read they should be supported?

Another solution: just delete them all and redownload from your usual sites.

 

[wontell]

I'm trying to run the program in Windows but I can't. The released binary on github doesn't work well (gui crashes) so I tried the master but I kept getting

stderr: AttributeError: 'arguments' object has no attribute 'alwaysParseCnmt' so I tried to use pip and now I get: TypeError: 'int' object is not subscriptable

Can someone help me? I'm running python 3.6 and Windows 10

 

[nicoboss]

I'm trying to run the program in Windows but I can't. The released binary on github doesn't work well (gui crashes) so I tried the master but I kept getting

stderr: AttributeError: 'arguments' object has no attribute 'alwaysParseCnmt' so I tried to use pip and now I get: TypeError: 'int' object is not subscriptable

Can someone help me? I'm running python 3.6 and Windows 10

It is well known that the GUI is broken on latest release but this should be long fixed in latest master. I just tested latest master and indeed was able to replicate your "AttributeError: 'arguments' object has no attribute 'alwaysParseCnmt'" issue. Luckily it was very easy to fix. Just get latest master again and it will work. Sorry that this wasn’t caught during testing as the CI/CD pipeline only tests the CLI. I will try to create a new release as soon as possible. I was extremely busy with University in the past year but luckily I'm in my last semester.

 

[wontell]

It is well known that the GUI is broken on latest release but this should be long fixed in latest master. I just tested latest master and indeed was able to replicate your "AttributeError: 'arguments' object has no attribute 'alwaysParseCnmt'" issue. Luckily it was very easy to fix. Just get latest master again and it will work. Sorry that this wasn’t caught during testing as the CI/CD pipeline only tests the CLI. I will try to create a new release as soon as possible. I was extremely busy with University in the past year but luckily I'm in my last semester.

thank you!!

 

[Chrisssj2]

This app often doesnt work and it hangs on certain files without any error msg leaving you to guess which files were done successfully or which caused the freeze.... Plz can we get an update?

 

[Dust2dust]

This app often doesnt work and it hangs on certain files without any error msg leaving you to guess which files were done successfully or which caused the freeze.... Plz can we get an update?

He doesn't update the first message, but the github is still updated. Last one was december 13th (v4.2.1).

 

[Chrisssj2]

He doesn't update the first message, but the github is still updated. Last one was december 13th (v4.2.1).

Thats the one im using but, like i said it is having trouble with a good amount of files.

 

[Dust2dust]

Thats the one im using but, like i said it is having trouble with a good amount of files.

Oh, OK. Have you tried just using the cli? I never used the GUI, personally, and never had problems. I'm on Linux, maybe that makes a difference, or maybe not.

 

[Chrisssj2]

Oh, OK. Have you tried just using the cli? I never used the GUI, personally, and never had problems. I'm on Linux, maybe that makes a difference, or maybe not.

Idk what you are saying with CLi but im not a technical person. so yeah. im on windows 11

 

[Dust2dust]

Idk what you are saying with CLi but im not a technical person. so yeah. im on windows 11

CLI=Command Line Interface. On linux, I would enter the command "python3 nsz --level 20 -C -V /home/switch" to compress all nsp files in the switch folder and then verify them. The Windows command is probably similar, but I haven't checked. It's probably in the documentation.

 

[nicoboss]

This app often doesnt work and it hangs on certain files without any error msg leaving you to guess which files were done successfully or which caused the freeze.... Plz can we get an update?

I actually planed on releasing an update tomorrow adding Bit-identical PFS0 recreation to comply with the no-intro standard but this won't fix your issue as you never reported it under Issues. I will delay this release to next weekend so I can investigate your issue and fix it if possible.

Feel free to provide more information regarding your issue here if you don't have a GitHub account.

In order to recreate and fix your issue please provide the following information:

• What games are failing for you? Can you please provide some excamples?
• What error do you get? Please post the log containing the error message. The error lof is shown as white text on a black window at the very end once all your games are processed. NSZ always gives you a summary of all errors that occurred once everything got processed.
• Are you trying to compress games from the eShop (.nsp) or a Cartage (.xci)?
• Are you using any NSZ settings different than the default?
• What tool and settings are you using to dump your games?
• You mentioned NSZ is freezing. This should never happen. Make sure you don't run out of RAM or Storage. Some large games can take up to an hour to compress (depending on your PC and compression settings). Make sure to wait long enough and check CPU usage. If it asks you to press Enter to exit this means that it's done and if you don't see any errors above that every game you requested it to process was processed successfully.

I saw a lot of users having similar issues because they forgot to dump thair latest keys using Lockpick_RCM. Make sure your keys inside %userprofile%/.switch/prod.keys are dumped from the firmware currently running on your Nintendo Switch and is not outdated. You can only compress games encrypted with keys you have dumped

NSZ is currently used by hundred thousand of persons and nobody ever reported this issue so far so you must do something different than the vast majority of NSZ users.

He doesn't update the first message, but the github is still updated. Last one was december 13th (v4.2.1).

Oh, you are right. I indeed neglected GBATemp as this thread got so inactive I thought nobody is looking at it anymore. NSZ is still in active development as a spare time project and my last commit was last weekend. I will update the first page again once I do the 4.3 release next weekend.

 

[Chrisssj2]

I actually planed on releasing an update tomorrow adding Bit-identical PFS0 recreation to comply with the no-intro standard but this won't fix your issue as you never reported it under Issues. I will delay this release to next weekend so I can investigate your issue and fix it if possible.

Feel free to provide more information regarding your issue here if you don't have a GitHub account.

In order to recreate and fix your issue please provide the following information:

• What games are failing for you? Can you please provide some excamples?
• What error do you get? Please post the log containing the error message. The error lof is shown as white text on a black window at the very end once all your games are processed. NSZ always gives you a summary of all errors that occurred once everything got processed.
• Are you trying to compress games from the eShop (.nsp) or a Cartage (.xci)?
• Are you using any NSZ settings different than the default?
• What tool and settings are you using to dump your games?
• You mentioned NSZ is freezing. This should never happen. Make sure you don't run out of RAM or Storage. Some large games can take up to an hour to compress (depending on your PC and compression settings). Make sure to wait long enough and check CPU usage. If it asks you to press Enter to exit this means that it's done and if you don't see any errors above that every game you requested it to process was processed successfully.

I saw a lot of users having similar issues because they forgot to dump thair latest keys using Lockpick_RCM. Make sure your keys inside %userprofile%/.switch/prod.keys are dumped from the firmware currently running on your Nintendo Switch and is not outdated. You can only compress games encrypted with keys you have dumped

NSZ is currently used by hundred thousand of persons and nobody ever reported this issue so far so you must do something different than the vast majority of NSZ users.


Oh, you are right. I indeed neglected GBATemp as this thread got so inactive I thought nobody is looking at it anymore. NSZ is still in active development as a spare time project and my last commit was last weekend. I will update the first page again once I do the 4.3 release next weekend.

These files for example don't want to seem to convert. they are all nsp files. Whenever the tool meets such file the entire que gets f*ked. and you wont be able to tell which files were done properly, getting me to redo the whole thing over again to be sure just to find the problematic files and ensure the other files are done properly. If I don't see the "DONE" message. i dont trust any work that may have been done.
I have 32 GB ram. top of the line computer. I do wait long enough, and check harddisk activity and cpu usage, thats why i say freeze, because it is not doing anything and the program "acts" like it is still busy and is counting time, but nothing is happening. These are files that are smaller then 1 gigabyte.... shouldnt take long normally. and i have let it stay on for hours a few times when this happens. I use max compression settings, other then that nothing changed


This is the error I get:

Also if I change the name of the file to something else i can't even seem to add it to the GUI list for example if I change the filename to "kut.nsp" i cant even add it to the list... I got the feeling something is up with the filenames somehow

 

[nicoboss]

I just released NSZ 4.3.0:

• Bit-identical PFS0 recreation to comply with the no-intro standard
  • The hash of the original NSP before compression and the hash of the NSP resulting from decompressing the NSZ will match as they are bit-identical.
  • Use the newly introduced --remove-padding option to make existing NSP/NSZ files nxdumptool/no-intro complient.
  • Use the newly introduced --keep-delta option to keep all useless delta fragments (NDV0) during compression so the NSP (PFS0) can be recreated bit-identical during decompression.
  • PFS0 hash verification is now part of the --verify process. --verify now requires --keep-delta when used during compression or it will detect removed NDV0 fragments as errors. Use --quick-verify to skip the PFS0 hash verification. --quick-verify only verifies NCA hashes and so does not require --keep-delta when used during compression. For compatibility reasons --quick-verify will be automatically used instead when specifying --verify without --keep-delta to match the command line argument and verification behavior of previous NSZ versions.
  • This fixes #98, fixes #101 and fixes #116
• Added console output warning about missing and invalid keys
• Added support for pyinstaller as discussed in #119 (pyinstaller builds for macOS)
• Added crc32 hash of master_key_0f. Thanks to @drizzt for adding it!

@Chrisssj2 Can you please retest on NSZ 4.3.0. Your issue should be fixed now. I think it cause was you not having master_key_0f inside your prod.keys in combination with previous NSZ version handling missing keys really poorly. If you still experience this issue please poste another screenshot of the terminal. There should now be much better error reporting regarding missing or invalid keys

He doesn't update the first message, but the github is still updated. Last one was december 13th (v4.2.1).

With the release of NSZ v4.3.0 I finally updated the initial post again adding all missing changelog entries. I will try to keep the initial post of this thread updated in the future.


Edit: Hotfix 07.07.2023: If you downloaded nsz_v4.3.0_win64_portable.zip in the first 24 hours after the release please redownload. This fixes #133 (nsz 4.3.0 nsz_v4.3.0_win64_portable won't open).

I forgot to include python.exe inside nsz_v4.3.0_win64_portable.zip. I just released a hotfix patch fixing this. I recommend redownloading nsz_v4.3.0_win64_portable.zip from the latest NSZ 4.3.0 release if you downloaded nsz_v4.3.0_win64_portable.zip in the first 24 hours after the release. Because all devices I tested this on already had Python installed and added to PATH I completely missed this during testing. This issue only affects the portable Windows version of NSZ. Everyone that installs NSZ using the PIP Package or is running it from source is unaffected.

 

[Nynrah]

For some reason I can't seem to load my Pikmin 4 XCI into the NSZ GUI to compress it while I was able to load the larger Zelda Tears of the Kingdom game in the GUI. Anyone knows why this is the case?

Also, can all DLC NSP files be compressed without issues? I heard rumors that DLC unlockers don't work as NSZ files. I know it doesn't matter for size, but I'd like to know anyway.

 

[nicoboss]

For some reason I can't seem to load my Pikmin 4 XCI into the NSZ GUI to compress it while I was able to load the larger Zelda Tears of the Kingdom game in the GUI. Anyone knows why this is the case?

Also, can all DLC NSP files be compressed without issues? I heard rumors that DLC unlockers don't work as NSZ files. I know it doesn't matter for size, but I'd like to know anyway.

What's the error you get when trying to load Pikmin 4 into NSZ? Make sure to dump the latest prod.keys as that's usually the reason why some newly released games don't work.

All NSP files no matter if base game, update and DLC can be compressed using NSZ. I never heard of such rumors but if you find an NSP that doesn't work please create an issue on https://github.com/nicoboss/nsz/issues/new and I will fix it.

 

[Nynrah]

What's the error you get when trying to load Pikmin 4 into NSZ? Make sure to dump the latest prod.keys as that's usually the reason why some newly released games don't work.

All NSP files no matter if base game, update and DLC can be compressed using NSZ. I never heard of such rumors but if you find an NSP that doesn't work please create an issue on https://github.com/nicoboss/nsz/issues/new and I will fix it.

It appears it is because of the (title) keys. I hadn't seen the console output as I was a bit in a rush at the time and I assumed my keys were up to date. Sorry for the inconvenience...

 

[nicoboss]

I just released NSZ 4.4.0:

• Android CLI support
  • To make NSZ Android compatible all uses of shared memory and locks had to be replaced with Android compatible alternatives
  • How to install NSZ on Android
    • Install "Pydroid 3" and the "Pydroid repository plugin" from the Play Store
    • Open "Pydroid 3" and navigate to "Pip"
    • Enter "nsz" and unselect "use prebuild" then press install
    • Navigate to "Terminal" to use the "nsz" command
    • The first time it will tell you where to copy your prod.keys which you should do using the "cp" command
    • Use any command line arguments you want like "nsz -D file.nsz" to decompress your game
• Fixed #134 (OverflowError: can't convert negative int to unsigned)
  • This was fixed inside #135 (Removal of stringTable padding)
  • NSP/NSZ files with a missing stringTable padding are now handled properly
  • This fixes some rare edge cases where bit-identical PFS0 recreation silently failed
• Updated all libraries inside the portable Windows version

 

[Nynrah]

@nicoboss: Hey there, one quick question about compressing to and decompressing from NSZ/XCZ. I understand that this is lossless. Does that mean that a decompressed file is 100% identical to the uncompressed NSP/XCI file? Just double-checking if I understood this right before I start compressing my library for HDD storage.