Homebrew RELEASE Ryjin: A NSZ Converter Mod

[GnK23]

----I attached a copy of the full bat command. I do not wish to update anymore the thread.------


RYJIN 2.5 NSZ Compressor Mod FINAL
by Gnk23 and Rylen101 of Pinoy Gaming Hustlers
Special thanks to blawar, nicoboss, and taorn for the NSZ compressor codes
The-4n for the 4nxci converter
DOWNLOAD LINK: https://gofile.io/?c=d2ogP4
---------------------------
Main Feature:
A. Ryjin Automatic.
- Sets threads to maximum possible units minus 1 based on your PC specs.

B. Ryjin Manual
- Shows how many threads your pc can use for conversion
- You can select how much threads you want to use but does not allow you to exceed more than what your pc can handle.

C. COMPRESSION TYPES
1. SOLID COMPRESSION
The size of the game may be reduced up to 80% depending on the title. Solid compressed NSZ although smaller in size from their original NSP is faster to install.
However please be informed that the installed size of solid compression is the same as that of its original NSP.

2. BLOCK COMPRESSION
This feature is supported by TINFOIL 6+ only. Block compression technically supports playing compressed games in the future as mentioned by @nicoboss.
More info on tinfoil here: https://tinfoil.io/Download#download

DL LINK: https://gofile.io/?c=m4tOJr

----------------------------

Changelog 2.5:
-Enabled Threads for Faster Conversion
-Enabled Automatic Batch Conversion
-Applied commit #ef6ec5f from https://github.com/nicoboss/nsz by dafault.
-Now with custom input/output for NSP compression
-NSP can now be deleted automatically after compression
-Added SOLID TO BLOCK NSZ conversion
-Added pop-up folder selector for easier access.
----------------------------
OLD MOD
RYJIN 2.5 MOD: REMOVED FILE CHECK (11-26-19)
I have made one last MOD for ryjin. This modded version skips the file checks at the beginning of each process that allows faster compression especially when dealing with large number of files.
Take note however of the following:
1. This will not check if you have a bad nsp or nsz so don't assume it will fix your problem. Always use working nsp or nsz during convertion. If your file is busted it's not the fault of the converter
2. This will definitely overwrite anything inside the folder that has already been converted if you plan to reconvert them.
https://gofile.io/?c=YjNG0J

Source:
https://github.com/nicoboss/nsz

 

[jkjj]

thanks, will try this in a virtual machine. nsc_builder is a guess as to how many threads to use

 

[GnK23]

thanks, will try this in a virtual machine. nsc_builder is a guess as to how many threads to use

Nope the threads is based on your pc core capacity. We used a simple cmd program to determine the threads. The program automatically detects your pc core threads and sets them as default. You can also choose the manual so you can see how many threads your pc can handle.

 

[jkjj]

ah i made the assumption based on your features list, without looking too deeply into nsc_builder functions, only having used it in a vm on linux, as windows sux the big one for me

 

[Xyphoseos]

Just to be sure, if I convert/compress nsp to nsz, can I uncompress the nsz file to nsp later ?

 

[jkjj]

Just to be sure, if I convert/compress nsp to nsz, can I uncompress the nsz file to nsp later ?

i havent run this yet specifically (definitely is my assumption with this mod) but yes it is possible with multiple tools for this, the python scripts for both compress/decompress by blawar/nicoboss
github.com/nicoboss/nsz
additionally nsc_builder has this in a batch program (terminal UI app).
if you haven't used nsc_builder you really should try it.

 

[lordelan]

Splatoon 2 NSP: 3281 MB
Splatoon 2 NSZ: 2600 MB

Not bad. Repacking Breath of the Wild now.
And when I'm home I can finally test whether NSZ files can be easily installed with the amazing Tinson + Tinfoil duo.

 

[Xyphoseos]

Splatoon 2 NSP: 3281 MB
Splatoon 2 NSZ: 2600 MB

Not bad. Repacking Breath of the Wild now.
And when I'm home I can finally test whether NSZ files can be easily installed with the amazing Tinson + Tinfoil duo.

Is the process long ?

 

[lordelan]

Is the process long ?

Splatoon 2 took a few minutes. Zelda about half an hour I think.

By the way...
Zelda NSP: 14136 MB
Zelda NSZ: 11204 MB

Saved 3 gigs, not bad.

 

[Xyphoseos]

Splatoon 2 took a few minutes. Zelda about half an hour I think.

By the way...
Zelda NSP: 14136 MB
Zelda NSZ: 11204 MB

Saved 3 gigs, not bad.

Can I install nsz with quark/goldleaf ?

 

[linuxares]

Can I install nsz with quark/goldleaf ?

No

EDIT: Probably never will https://github.com/XorTroll/Goldleaf/issues/417

 

[Xyphoseos]

No

EDIT: Probably never will https://github.com/XorTroll/Goldleaf/issues/417

Too bad

Also, is there a difference between this and winrar compress ?
I mean nsz and nsp.rar

 

[linuxares]

Too bad

Also, is there a difference between this and winrar compress ?
I mean nsz and nsp.rar

Winrar is just a tool. The rar format is probably different compared to the Z format. https://www.lifewire.com/z-file-2622674
I don't know if this is what's used, but I just pull a guess out of my behind. @blawar knows more.

 

[Xyphoseos]

Winrar is just a tool. The rar format is probably different compared to the Z format. https://www.lifewire.com/z-file-2622674
I don't know if this is what's used, but I just pull a guess out of my behind. @blawar knows more.

I would mean about the compress
Above TLOZ has 3 giga in less using nsz, how many with win rar ?

 

[linuxares]

I would mean about the compress
Above TLOZ has 3 giga in less using nsz, how many with win rar ?

Just to try. As far as I know LZMA2 with 7zip is the best compression so far, until something new comes along.

EDIT: https://en.wikipedia.org/wiki/Brotli is apparently the new kid on the block

EDIT2: Zstandard is faster than Brotli but Brotli is made to compress larger files. But it's negtable. So Zstandard 22 should compress as hard as it can. Compression is slow on both Zstandard 22 and Brotli but fast to decompress.

 

[GnK23]

Winrar is just a tool. The rar format is probably different compared to the Z format. https://www.lifewire.com/z-file-2622674
I don't know if this is what's used, but I just pull a guess out of my behind. @blawar knows more.

Well I agree that winrar is different from nsz. using nsz decreases not only the size of the stored game but also increases the installation speed. However take note that onced installed it will resume it's original size.
It would much of a hassle everytime you need to extract the game from winrar if ever you are going to install it.
Nsz fixes this problem not only with the space but with the installation procedure.

--------------------- MERGED ---------------------------

Is the process long ?

It all depends on your processor and number of threads.
Im using i5 7th gen with 4 threads.
A 14gb takes around 12-20min to convert

 

[linuxares]

Well I agree that winrar is different from nsz. using nsz decreases not only the size of the stored game but also increases the installation speed. However take note that onced installed it will resume it's original size.
It would much of a hassle everytime you need to extract the game from winrar if ever you are going to install it.
Nsz fixes this problem not only with the space but with the installation procedure.

--------------------- MERGED ---------------------------


It all depends on your processor and number of threads.
Im using i5 7th gen with 4 threads.
A 14gb takes around 12-20min to convert

Also your harddrive matters a lot! A fast SSD will be plenty faster than a slow SSD or a mechanical. Read and write is very important in this scenario. But still 20min is nothing if you can 22 lvl compress it.

 

[GnK23]

Also your harddrive matters a lot! A fast SSD will be plenty faster than a slow SSD or a mechanical. Read and write is very important in this scenario. But still 20min is nothing if you can 22 lvl compress it.

We tested the level 22 compression before and found it unstable so by default we used the level 17.

 

[linuxares]

We tested the level 22 compression before and found it unstable so by default we used the level 17.

17 is plenty good.

 

[nicoboss]

Too bad

Also, is there a difference between this and winrar compress ?
I mean nsz and nsp.rar

Just to try. As far as I know LZMA2 with 7zip is the best compression so far, until something new comes along.

EDIT: https://en.wikipedia.org/wiki/Brotli is apparently the new kid on the block

EDIT2: Zstandard is faster than Brotli but Brotli is made to compress larger files. But it's negtable. So Zstandard 22 should compress as hard as it can. Compression is slow on both Zstandard 22 and Brotli but fast to decompress.

There went a lot of thought and discussion into why zStandard was chosen for booth NSZP and NSZ as compression algorithm. Most thought went into how to allow random read access for emulators like Yuzu and potentially CFW in the future. To do this we need to split the game into a lot of small indecently compressed chunks without losing much compression ratio so we decided for a non-dictionary-based compression. To keep the chunk size as large as possible to further lower its compression ratio impact while still offering a lag free game experience the decompression speed must be as high as possible to increase random read access time. We wanted to achieve all of this with as good of a compression ratio as possible while keeping compression time and RAM usage reasonable. This is one of the first tests used to made this decision 9 month ago: https://gist.github.com/nicoboss/064e37753de007ea95a29f7fcf8bb3d0
As you can see zStandard is the clear winner. Only the dictionary-based LZMA compression would give better compression ratios but without reasonable compression speed and extremely high RAM usage with high dictionary sizes. But what probably killed it in the end was its poor decompression speed of around 20 MB/s slowing down game installations instead of speeding them up. Unlike NSPZ with NSZ solid compression without random read access is the default option and even there we used zStandard which shows how great that compression algorithm really is.
Edit: Please note NSPZ and NSZ booth unpack the NSP, Decrypt the NCAs, Compress the NCAs and Repack the NSP as otherwise no compression would be possible as encrypted data can't be compressed.

--------------------- MERGED ---------------------------

17 is plenty good.

zStandard level 17 kind of sucks as compared to 18 its compression ratio/compression speed ratio is very bad. 18 is the default for nsZip and 17 just randomly chosen by blawar for no real reason and will probably be changed to 18 soon.