nsZip is an open source PC tool to lossless compress/decompress NSP files in order to save a lot of storage while all NCA files keep their exact same hash. [prebreak]1[/prebreak]
But it’s more than just a compression. A lot of data like NDV0 partitions or fragment NCA3 files will be removed while compressing and exactly recreated when decompressed which saves even more space especially on updates higher than v65536.
In addition, the NSZ format was designed with emulators in mind so adding NSZ support to Yuzu will be possible in the future and because NSZ contains decrypted NCAs no keys would be needed to only extract game files. As compression algorithm Zstandard is used to multithreaded compress 256 KB chunks while uncompressible chunks are stored uncompressed. That way NSPZ/XCIZ allows random read access. Zstandard has a 43MB/s compression and 7032MB/s decompression speed on an 8 threaded CPU at level 18 while having one of the best compression ratios compared to other compression algorithms.

How compressing works: NSP => extracted NSP => decrypted NCAs => trim fragments => compressing to NSZ => verify correctness => repacking NSPZ file
How decompressing works: NSPZ file => extracted NSPZ => decompress NSZ => untrim fragments => encrypt NCAs => verify correctness => repack as NSP

Check out my GitHub page to report bugs, follow this project, post your suggestions or contribute.
GitHub link: https://github.com/nicoboss/nsZip/releases

For the new homebrew compatible NSZ format please see https://github.com/nicoboss/nsz
NSZ thread: https://gbatemp.net/threads/nsz-homebrew-compatible-nsp-xci-compressor-decompressor.550556/
Until NSZ support is implemented in nsZip I don't see why you want to use it instead of NSZ

Block compressed NSZ is very similar to the beautiful NSPZ format just without all the unnecessarily complexity which made NSPZ unfeasible for other software to implement. I might remove NSPZ/XCIZ support in the future if I see that there's no point in keeping it to not confuse people.

Differences between NSZ and NSPZ:

NSPZ/XCIZ:
- GitHub Project: https://github.com/nicoboss/nsZip
- Always uses Block compression allowing random read access to play compressed games in the future
- Decrypts the whole NCA
- Trims NDV0 fragments to their header and reconstructs them
- Only supported by nsZip and unfortunately doesn't really have a future

NSZ/XCZ:
- GitHub Project: https://github.com/nicoboss/nsz
- Uses solid compression by default. Block compression can be enabled using the -B option. Block compression will be the default for XCZ
- Decrypts all sections while keeping the first 0x4000 bytes encrypted. Puts informations needed to encrypt inside the header.
- Deleted NDV0 fragments as they have no use for end users as they only exist to save CDN bandwidth
- Already widely used. Supported by Tinfoil, SX Installer v3.0.0 and probably a lot of other software in the future


nsZip v2.0.0 preview 2:

• Fixed XCI to XCIZ compression

nsZip v2.0.0 preview 1:

• New UI made using WPF instead of WinForms
  
• Huge performance improvements
  • Directly decrypt the NSP without extracting it first
  • Directly decompressing NSPZ files without extracting them first
  • Huge encrypting speed increase
  • Huge SHA-256 verification speed increase
  • Enhanced compressing multithreading
  • Support for up to 400 CPU Cores
• Improved game compatibility
• Command Line support
• More settings

Version 1.1.0 Changelog:

• Added support for over multiple NCAs splitted NDV0 fragment trimming

Version 1.0.1 Changelog:

• Let the user continue when detecting a yet unimplemented multifile NDV0 fragment. I'll add proper support for that unimplemented multifile NDV0 fragment format in the following days.
• The compression level can now be changed (tradeoff between speed and compression ratio)
• The block size can now be changed (tradeoff between random access time and compression ratio)

Future planes:

• NSZ implementation

FAQ:
Q: It prod.keys not found!
A: Dump them using Lockpick and copy them to %userprofile%/.switch/prod.keys
Q: How much storage does this save?
A: It depends on the game. The compressed size is usually around 40-80% of the original size.
Q: What about XCI?
A: Compression to XCIZ is supported, but XCIZ to XCI recreation is still in development.
Q: The program throws an error or seems to behaves not as intended!
A: Open an issue on my GitHub page where you give me exact steps how to reproduce.
Q: Does this compress installed games?
A: Not now but it's planned for the far future.

Screenshot:

 

[Arnold101]

thanks

btw just crashed after this

Skiped fragemt trimming as no patch Cnmt was found!
=> probably no dlc/update over v65536

impossible load dll libzstd

--------------------- MERGED ---------------------------

fixed it

 

[Nazosan]

That would be the DLL file from the x86 or x64 folders. Try putting it in the same folder as nsZip.exe and see if that helps.

BTW, is it necessary that this be distributed as an installer of some sort? A simple tool like this seems like the kind of thing that should just be zipped up to me.

 

[dreambottle]

I agree, it would be great to release a "portable" version. The installer looks scary for those who don't want to clutter the system.

 

[Nazosan]

I didn't go right out and say it because I'm not terribly familiar with how this particular software deployment system actually works and I did go ahead and "install" (I must put that in quotes because it seems to have some weird deployment setup that isn't a true normal software installation -- almost temporary) it first, but as far as I can determine, the .deploy files are simply renamed ordinary files -- no compression or anything. Gather them all together in one folder and rename to take the .deploy off and I believe you have a portable setup. Though I could only say for sure if it were tested this way on a system that had not ever had it "installed" in the first place. The only thing that needs to be done as far as I can determine is to select between the 32-bit and 64-bit compiles of the libzstd library DLL. (It may be the 32-bit version is fine either way? I honestly don't know.)

Obviously that's a bit of a pain to deal with. I would say it could be distributed both ways, but honestly, I just don't see why it needs an installer at all in its current state and amount of use. A simple portable, ready to use fileset seems sufficient as things stand right now.

 

[proffk]

This tool is good but too many crashes. When ever I select run the gui freezes and not responding. No text in the box and Prod.key is in the switch folder. I just left it running and it seems to be doing something now. I think the not responding may be a small bug after selecting run but annoying as it happens everytime. After all that I've found out nspz is a new format and can't be installed yet. Should of done more reading.

 

[Deleted member 413168]

This tool is good but too many crashes. When ever I select run the gui freezes and not responding. No text in the box and Prod.key is in the switch folder. I just left it running and it seems to be doing something now. I think the not responding may be a small bug after selecting run but annoying as it happens everytime. After all that I've found out nspz is a new format and can't be installed yet. Should of done more reading.

While the UI freezes, it doesn’t actually crash. If you open task manager whilst it’s running you will see it’s still working.

 

[mrmole]

@nicoboss any news on the future updates?

 

[aytug25]

when i use this program for compress nsp games, will it be compressed only nsp files or after installed it, games will have less memory on sd card ?

 

[nicoboss]

@nicoboss any news on the future updates?

Let's just release a preview of nsZip Version 2.0.0 as the only missing future is the update notification for nsZip portable and its already very well tested. Expect the final v2.0.0 to be released in the following days.

https://github.com/nicoboss/nsZip/releases/tag/v2.0.0p1
nsZip v2.0.0 preview 1:

• New UI made using WPF instead of WinForms
• Huge performance improvements
  • Directly decrypt the NSP without extracting it first
  • Directly decompressing NSPZ files without extracting them first
  • Huge encrypting speed increase
  • Huge SHA-256 verification speed increase
  • Enhanced compressing multithreading
  • Support for up to 400 CPU Cores
• Improved game compatibility
• Command Line support
• More settings

There's even better news to tell: I sucessfully ported ZStandard to the Nintendo Switch so nsZipNX might get reality in the future. At least Tinfoil/Goldleaf support would now definitely be doable but even running compressed games might be possible in the far future.

 

[SaulFemm]

Successfully compressed my entire library using the CLI version. If any
one is interested, I can share the powershell script I used to recursively go through my library and to place the compressed files back in the same folder as the original NSP.

 

[Deleted User]

would be helpful when i could encrypt and decrypt NSPs directly on my Switch

 

[Deathscreton]

Successfully compressed my entire library using the CLI version. If any
one is interested, I can share the powershell script I used to recursively go through my library and to place the compressed files back in the same folder as the original NSP.

I'd like to try this. Right now I'm just storing my files in a Winrar archive, but I'd like them loose and on the drive. Your script would help immensely.

 

[skinnyBIGGS]

Great app cant wait to see it take flight and able to play games without decompress... thanks for the share

 

[SaulFemm]

I'd like to try this. Right now I'm just storing my files in a Winrar archive, but I'd like them loose and on the drive. Your script would help immensely.

Get-ChildItem D:\Switch\Games\ -Recurse -Filter *.nsp |
Foreach-Object {
.\nsZip.exe -i $_.FullName -o $_.DirectoryName -t D:\
}

It's not much. Of course, you'll need to edit the paths to suit your needs. I also never bothered to make it delete the .nsp files afterwards, but that should be easy enough.

 

[Nazosan]

I'd like to try this. Right now I'm just storing my files in a Winrar archive, but I'd like them loose and on the drive. Your script would help immensely.

Why did you pack them in RAR files? They usually won't compress at all, so it just makes things more complicated for you that way.

 

[Deathscreton]

Why did you pack them in RAR files? They usually won't compress at all, so it just makes things more complicated for you that way.

Huh. I guess I never really noticed that.

 

[Nazosan]

Yeah, it's kind of sad, but generally you won't get anything out of them because even if the actual package isn't encrypted, the actual data is I think. Either way, you usually end up with a slightly larger file or, if it is smaller, not by enough to make it even remotely worth the wasted resources.

And... That's where nsZip comes in. It solves that problem by dealing with the encryption -- removing for compression, restoring for decompression. That's why we need this to be taken up by as many as possible on the Switch and given plenty of support.

 

[LanzoThunder]

Why does every file that I try in cli mode it say that's not a NCA3 file?

Also, if i try to use it in GUI mode on the console I can see it stops saying that it cannot access to xciMeta.dat because it's in use from another process.

Any fix to these problems?

 

[Deathscreton]

I'm unpacking my files as I speak, gonna run it over night then give that powershell script a try.

Why does every file that I try in cli mode it say that's not a NCA3 file?

Also, if i try to use it in GUI mode on the console I can see it stops saying that it cannot access to xciMeta.dat because it's in use from another process.

Any fix to these problems?

You on Win10? Did you try running as an admin?

EDIT: Also, this program is amazing. When can we expect future updates?

 

[nicoboss]

Why does every file that I try in cli mode it say that's not a NCA3 file?

Also, if i try to use it in GUI mode on the console I can see it stops saying that it cannot access to xciMeta.dat because it's in use from another process.

Any fix to these problems?

There is a bug inside nsZip v2.0.0 preview 1 that breaks XCI to XCIZ compression. Thanks a lot for reporting it. I fixed it in the latest version!

https://github.com/nicoboss/nsZip/releases/tag/v2.0.0p2
nsZip v2.0.0 preview 2 Changelog:
- Fixed XCI to XCIZ compression