RELEASE Ryjin: A NSZ Converter Mod

Discussion in 'Switch - Emulation, Homebrew & Software Projects' started by GnK23, Oct 15, 2019.

  1. GnK23
    OP

    GnK23 GBAtemp Regular

    Member
    3
    Apr 6, 2016
    United States
    RYJIN 2.5 NSZ Compressor Mod FINAL
    by Gnk23 and Rylen101 of Pinoy Gaming Hustlers
    Special thanks to blawar, nicoboss, and taorn for the NSZ compressor codes :grog:

    ---------------------------
    Main Feature:
    A. Ryjin Automatic.
    - Sets threads to maximum possible units minus 1 based on your PC specs.


    B. Ryjin Manual
    - Shows how many threads your pc can use for conversion
    - You can select how much threads you want to use but does not allow you to exceed more than what your pc can handle.

    C. COMPRESSION TYPES
    1. SOLID COMPRESSION
    The size of the game may be reduced up to 80% depending on the title. Solid compressed NSZ although smaller in size from their original NSP is faster to install.
    However please be informed that the installed size of solid compression is the same as that of its original NSP.

    2. BLOCK COMPRESSION
    This feature is supported by TINFOIL 6+ only. Block compression technically supports playing compressed games in the future as mentioned by @nicoboss.
    More info on tinfoil here: https://tinfoil.io/Download#download

    Instruction :
    -Place hactool compatible keys.txt inside the Ryjin folder
    -Run the Ryjin 2.5 and follow on screen instructions
    ----------------------------
    Changelog:
    -Enabled Threads for Faster Conversion
    -Enabled Automatic Batch Conversion
    -Applied commit #ef6ec5f from https://github.com/nicoboss/nsz by dafault.
    -Now with custom input/output for NSP compression
    -NSP can now be deleted automatically after compression
    -Added SOLID TO BLOCK NSZ conversion :gun:
    -Added pop-up folder selector for easier access.
    ----------------------------

    DOWNLOAD LINK: https://gofile.io/?c=Djgth0

    Source:
    https://github.com/nicoboss/nsz
     

    Attached Files:

    Last edited by GnK23, Nov 13, 2019
    CapFed, rogras, merlin555 and 6 others like this.
  2. jkjj

    jkjj Advanced Member

    Newcomer
    2
    Jan 16, 2019
    United States
    thanks, will try this in a virtual machine. nsc_builder is a guess as to how many threads to use
     
    Zukov and GnK23 like this.
  3. GnK23
    OP

    GnK23 GBAtemp Regular

    Member
    3
    Apr 6, 2016
    United States
    Nope the threads is based on your pc core capacity. We used a simple cmd program to determine the threads. The program automatically detects your pc core threads and sets them as default. You can also choose the manual so you can see how many threads your pc can handle.
     
  4. jkjj

    jkjj Advanced Member

    Newcomer
    2
    Jan 16, 2019
    United States
    ah i made the assumption based on your features list, without looking too deeply into nsc_builder functions, only having used it in a vm on linux, as windows sux the big one for me
     
  5. Xyphoseos

    Xyphoseos Hack or no games

    Member
    7
    Jun 29, 2016
    France
    Rivière
    Just to be sure, if I convert/compress nsp to nsz, can I uncompress the nsz file to nsp later ?
     
  6. jkjj

    jkjj Advanced Member

    Newcomer
    2
    Jan 16, 2019
    United States
    i havent run this yet specifically (definitely is my assumption with this mod) but yes it is possible with multiple tools for this, the python scripts for both compress/decompress by blawar/nicoboss
    github.com/nicoboss/nsz
    additionally nsc_builder has this in a batch program (terminal UI app).
    if you haven't used nsc_builder you really should try it.
     
    lordelan and Xyphoseos like this.
  7. lordelan

    lordelan GBAtemp Psycho!

    Member
    12
    Jan 4, 2015
    Germany
    Splatoon 2 NSP: 3281 MB
    Splatoon 2 NSZ: 2600 MB

    Not bad. Repacking Breath of the Wild now.
    And when I'm home I can finally test whether NSZ files can be easily installed with the amazing Tinson + Tinfoil duo. :)
     
    Broduskii and GnK23 like this.
  8. Xyphoseos

    Xyphoseos Hack or no games

    Member
    7
    Jun 29, 2016
    France
    Rivière
    Is the process long ?
     
  9. lordelan

    lordelan GBAtemp Psycho!

    Member
    12
    Jan 4, 2015
    Germany
    Splatoon 2 took a few minutes. Zelda about half an hour I think.

    By the way...
    Zelda NSP: 14136 MB
    Zelda NSZ: 11204 MB

    Saved 3 gigs, not bad.
     
    Xyphoseos likes this.
  10. Xyphoseos

    Xyphoseos Hack or no games

    Member
    7
    Jun 29, 2016
    France
    Rivière
    Can I install nsz with quark/goldleaf ?
     
  11. linuxares

    linuxares I'm not a generous god!

    Moderator
    17
    Aug 5, 2007
    Sweden
    Xyphoseos likes this.
  12. Xyphoseos

    Xyphoseos Hack or no games

    Member
    7
    Jun 29, 2016
    France
    Rivière
  13. linuxares

    linuxares I'm not a generous god!

    Moderator
    17
    Aug 5, 2007
    Sweden
    Winrar is just a tool. The rar format is probably different compared to the Z format. https://www.lifewire.com/z-file-2622674
    I don't know if this is what's used, but I just pull a guess out of my behind. @blawar knows more.
     
    GnK23 likes this.
  14. Xyphoseos

    Xyphoseos Hack or no games

    Member
    7
    Jun 29, 2016
    France
    Rivière
    I would mean about the compress
    Above TLOZ has 3 giga in less using nsz, how many with win rar ?
     
  15. linuxares

    linuxares I'm not a generous god!

    Moderator
    17
    Aug 5, 2007
    Sweden
    Just to try. As far as I know LZMA2 with 7zip is the best compression so far, until something new comes along.

    EDIT: https://en.wikipedia.org/wiki/Brotli is apparently the new kid on the block

    EDIT2: Zstandard is faster than Brotli but Brotli is made to compress larger files. But it's negtable. So Zstandard 22 should compress as hard as it can. Compression is slow on both Zstandard 22 and Brotli but fast to decompress.
     
    Last edited by linuxares, Oct 15, 2019
    Xyphoseos likes this.
  16. GnK23
    OP

    GnK23 GBAtemp Regular

    Member
    3
    Apr 6, 2016
    United States
    Well I agree that winrar is different from nsz. using nsz decreases not only the size of the stored game but also increases the installation speed. However take note that onced installed it will resume it's original size.
    It would much of a hassle everytime you need to extract the game from winrar if ever you are going to install it.
    Nsz fixes this problem not only with the space but with the installation procedure.

    — Posts automatically merged - Please don't double post! —

    It all depends on your processor and number of threads.
    Im using i5 7th gen with 4 threads.
    A 14gb takes around 12-20min to convert
     
    Last edited by GnK23, Oct 15, 2019
  17. linuxares

    linuxares I'm not a generous god!

    Moderator
    17
    Aug 5, 2007
    Sweden
    Also your harddrive matters a lot! A fast SSD will be plenty faster than a slow SSD or a mechanical. Read and write is very important in this scenario. But still 20min is nothing if you can 22 lvl compress it.
     
    GnK23 likes this.
  18. GnK23
    OP

    GnK23 GBAtemp Regular

    Member
    3
    Apr 6, 2016
    United States
    We tested the level 22 compression before and found it unstable so by default we used the level 17.
     
  19. linuxares

    linuxares I'm not a generous god!

    Moderator
    17
    Aug 5, 2007
    Sweden
    17 is plenty good. :)
     
    GnK23 likes this.
  20. nicoboss

    nicoboss Advanced Member

    Newcomer
    4
    Feb 1, 2019
    Switzerland
    There went a lot of thought and discussion into why zStandard was chosen for booth NSZP and NSZ as compression algorithm. Most thought went into how to allow random read access for emulators like Yuzu and potentially CFW in the future. To do this we need to split the game into a lot of small indecently compressed chunks without losing much compression ratio so we decided for a non-dictionary-based compression. To keep the chunk size as large as possible to further lower its compression ratio impact while still offering a lag free game experience the decompression speed must be as high as possible to increase random read access time. We wanted to achieve all of this with as good of a compression ratio as possible while keeping compression time and RAM usage reasonable. This is one of the first tests used to made this decision 9 month ago: https://gist.github.com/nicoboss/064e37753de007ea95a29f7fcf8bb3d0
    As you can see zStandard is the clear winner. Only the dictionary-based LZMA compression would give better compression ratios but without reasonable compression speed and extremely high RAM usage with high dictionary sizes. But what probably killed it in the end was its poor decompression speed of around 20 MB/s slowing down game installations instead of speeding them up. Unlike NSPZ with NSZ solid compression without random read access is the default option and even there we used zStandard which shows how great that compression algorithm really is.
    Edit: Please note NSPZ and NSZ booth unpack the NSP, Decrypt the NCAs, Compress the NCAs and Repack the NSP as otherwise no compression would be possible as encrypted data can't be compressed.

    — Posts automatically merged - Please don't double post! —

    zStandard level 17 kind of sucks as compared to 18 its compression ratio/compression speed ratio is very bad. 18 is the default for nsZip and 17 just randomly chosen by blawar for no real reason and will probably be changed to 18 soon.
     
    Last edited by nicoboss, Oct 15, 2019
    ioann1s, tiliarou, linuxares and 3 others like this.
Quick Reply
Draft saved Draft deleted
Loading...