Homebrew RELEASE Ryjin: A NSZ Converter Mod

GnK23

Well-Known Member
OP
Member
Joined
Apr 6, 2016
Messages
168
Trophies
0
Age
32
XP
638
Country
United States
----I attached a copy of the full bat command. I do not wish to update anymore the thread.------


RYJIN 2.5 NSZ Compressor Mod FINAL
by Gnk23 and Rylen101 of Pinoy Gaming Hustlers
Special thanks to blawar, nicoboss, and taorn for the NSZ compressor codes :grog:
The-4n for the 4nxci converter
DOWNLOAD LINK: https://gofile.io/?c=d2ogP4
---------------------------
Main Feature:
A. Ryjin Automatic.
- Sets threads to maximum possible units minus 1 based on your PC specs.

B. Ryjin Manual
- Shows how many threads your pc can use for conversion
- You can select how much threads you want to use but does not allow you to exceed more than what your pc can handle.

C. COMPRESSION TYPES
1. SOLID COMPRESSION
The size of the game may be reduced up to 80% depending on the title. Solid compressed NSZ although smaller in size from their original NSP is faster to install.
However please be informed that the installed size of solid compression is the same as that of its original NSP.

2. BLOCK COMPRESSION
This feature is supported by TINFOIL 6+ only. Block compression technically supports playing compressed games in the future as mentioned by @nicoboss.
More info on tinfoil here: https://tinfoil.io/Download#download

DL LINK: https://gofile.io/?c=m4tOJr

----------------------------

Changelog 2.5:
-Enabled Threads for Faster Conversion
-Enabled Automatic Batch Conversion
-Applied commit #ef6ec5f from https://github.com/nicoboss/nsz by dafault.
-Now with custom input/output for NSP compression
-NSP can now be deleted automatically after compression
-Added SOLID TO BLOCK NSZ conversion :gun:
-Added pop-up folder selector for easier access.
----------------------------
OLD MOD
RYJIN 2.5 MOD: REMOVED FILE CHECK (11-26-19)
I have made one last MOD for ryjin. This modded version skips the file checks at the beginning of each process that allows faster compression especially when dealing with large number of files.
Take note however of the following:
1. This will not check if you have a bad nsp or nsz so don't assume it will fix your problem. Always use working nsp or nsz during convertion. If your file is busted it's not the fault of the converter
2. This will definitely overwrite anything inside the folder that has already been converted if you plan to reconvert them.
https://gofile.io/?c=YjNG0J

Source:
https://github.com/nicoboss/nsz
 

Attachments

  • Untitled.png
    Untitled.png
    25.2 KB · Views: 1,137
  • FULL BAT COMMAND.txt
    11.5 KB · Views: 462
Last edited by GnK23,

GnK23

Well-Known Member
OP
Member
Joined
Apr 6, 2016
Messages
168
Trophies
0
Age
32
XP
638
Country
United States
thanks, will try this in a virtual machine. nsc_builder is a guess as to how many threads to use
Nope the threads is based on your pc core capacity. We used a simple cmd program to determine the threads. The program automatically detects your pc core threads and sets them as default. You can also choose the manual so you can see how many threads your pc can handle.
 

jkjj

Well-Known Member
Newcomer
Joined
Jan 16, 2019
Messages
76
Trophies
0
Age
41
XP
247
Country
United States
ah i made the assumption based on your features list, without looking too deeply into nsc_builder functions, only having used it in a vm on linux, as windows sux the big one for me
 

jkjj

Well-Known Member
Newcomer
Joined
Jan 16, 2019
Messages
76
Trophies
0
Age
41
XP
247
Country
United States
Just to be sure, if I convert/compress nsp to nsz, can I uncompress the nsz file to nsp later ?

i havent run this yet specifically (definitely is my assumption with this mod) but yes it is possible with multiple tools for this, the python scripts for both compress/decompress by blawar/nicoboss
github.com/nicoboss/nsz
additionally nsc_builder has this in a batch program (terminal UI app).
if you haven't used nsc_builder you really should try it.
 

lordelan

Well-Known Member
Member
Joined
Jan 4, 2015
Messages
5,233
Trophies
1
Age
41
XP
4,942
Country
Germany
Splatoon 2 NSP: 3281 MB
Splatoon 2 NSZ: 2600 MB

Not bad. Repacking Breath of the Wild now.
And when I'm home I can finally test whether NSZ files can be easily installed with the amazing Tinson + Tinfoil duo. :)
 
  • Like
Reactions: Broduskii and GnK23

linuxares

I'm not a generous god!
Global Moderator
Joined
Aug 5, 2007
Messages
9,135
Trophies
1
XP
10,589
Country
Sweden
  • Like
Reactions: GnK23

Xyphoseos

Hack or no games
Member
Joined
Jun 29, 2016
Messages
1,783
Trophies
0
Age
21
Location
Rivière
XP
2,444
Country
France

linuxares

I'm not a generous god!
Global Moderator
Joined
Aug 5, 2007
Messages
9,135
Trophies
1
XP
10,589
Country
Sweden
I would mean about the compress
Above TLOZ has 3 giga in less using nsz, how many with win rar ?
Just to try. As far as I know LZMA2 with 7zip is the best compression so far, until something new comes along.

EDIT: https://en.wikipedia.org/wiki/Brotli is apparently the new kid on the block

EDIT2: Zstandard is faster than Brotli but Brotli is made to compress larger files. But it's negtable. So Zstandard 22 should compress as hard as it can. Compression is slow on both Zstandard 22 and Brotli but fast to decompress.
 
Last edited by linuxares,
  • Like
Reactions: Xyphoseos

GnK23

Well-Known Member
OP
Member
Joined
Apr 6, 2016
Messages
168
Trophies
0
Age
32
XP
638
Country
United States
Winrar is just a tool. The rar format is probably different compared to the Z format. https://www.lifewire.com/z-file-2622674
I don't know if this is what's used, but I just pull a guess out of my behind. @blawar knows more.
Well I agree that winrar is different from nsz. using nsz decreases not only the size of the stored game but also increases the installation speed. However take note that onced installed it will resume it's original size.
It would much of a hassle everytime you need to extract the game from winrar if ever you are going to install it.
Nsz fixes this problem not only with the space but with the installation procedure.

--------------------- MERGED ---------------------------

Is the process long ?
It all depends on your processor and number of threads.
Im using i5 7th gen with 4 threads.
A 14gb takes around 12-20min to convert
 
Last edited by GnK23,

linuxares

I'm not a generous god!
Global Moderator
Joined
Aug 5, 2007
Messages
9,135
Trophies
1
XP
10,589
Country
Sweden
Well I agree that winrar is different from nsz. using nsz decreases not only the size of the stored game but also increases the installation speed. However take note that onced installed it will resume it's original size.
It would much of a hassle everytime you need to extract the game from winrar if ever you are going to install it.
Nsz fixes this problem not only with the space but with the installation procedure.

--------------------- MERGED ---------------------------


It all depends on your processor and number of threads.
Im using i5 7th gen with 4 threads.
A 14gb takes around 12-20min to convert
Also your harddrive matters a lot! A fast SSD will be plenty faster than a slow SSD or a mechanical. Read and write is very important in this scenario. But still 20min is nothing if you can 22 lvl compress it.
 
  • Like
Reactions: GnK23

GnK23

Well-Known Member
OP
Member
Joined
Apr 6, 2016
Messages
168
Trophies
0
Age
32
XP
638
Country
United States
Also your harddrive matters a lot! A fast SSD will be plenty faster than a slow SSD or a mechanical. Read and write is very important in this scenario. But still 20min is nothing if you can 22 lvl compress it.
We tested the level 22 compression before and found it unstable so by default we used the level 17.
 

nicoboss

Well-Known Member
Member
Joined
Feb 1, 2019
Messages
120
Trophies
0
Age
24
XP
985
Country
Switzerland
Too bad

Also, is there a difference between this and winrar compress ?
I mean nsz and nsp.rar
Just to try. As far as I know LZMA2 with 7zip is the best compression so far, until something new comes along.

EDIT: https://en.wikipedia.org/wiki/Brotli is apparently the new kid on the block

EDIT2: Zstandard is faster than Brotli but Brotli is made to compress larger files. But it's negtable. So Zstandard 22 should compress as hard as it can. Compression is slow on both Zstandard 22 and Brotli but fast to decompress.
There went a lot of thought and discussion into why zStandard was chosen for booth NSZP and NSZ as compression algorithm. Most thought went into how to allow random read access for emulators like Yuzu and potentially CFW in the future. To do this we need to split the game into a lot of small indecently compressed chunks without losing much compression ratio so we decided for a non-dictionary-based compression. To keep the chunk size as large as possible to further lower its compression ratio impact while still offering a lag free game experience the decompression speed must be as high as possible to increase random read access time. We wanted to achieve all of this with as good of a compression ratio as possible while keeping compression time and RAM usage reasonable. This is one of the first tests used to made this decision 9 month ago: https://gist.github.com/nicoboss/064e37753de007ea95a29f7fcf8bb3d0
As you can see zStandard is the clear winner. Only the dictionary-based LZMA compression would give better compression ratios but without reasonable compression speed and extremely high RAM usage with high dictionary sizes. But what probably killed it in the end was its poor decompression speed of around 20 MB/s slowing down game installations instead of speeding them up. Unlike NSPZ with NSZ solid compression without random read access is the default option and even there we used zStandard which shows how great that compression algorithm really is.
Edit: Please note NSPZ and NSZ booth unpack the NSP, Decrypt the NCAs, Compress the NCAs and Repack the NSP as otherwise no compression would be possible as encrypted data can't be compressed.

--------------------- MERGED ---------------------------

17 is plenty good. :)
zStandard level 17 kind of sucks as compared to 18 its compression ratio/compression speed ratio is very bad. 18 is the default for nsZip and 17 just randomly chosen by blawar for no real reason and will probably be changed to 18 soon.
 
Last edited by nicoboss,
General chit-chat
Help Users
    Julie_Pilgrim @ Julie_Pilgrim: *with friend* friend: what you listening to me: bladee friend: whats that me: *plays bladee*...