RELEASE NSZ Title Compression Results

Discussion in 'Switch - Emulation, Homebrew & Software Projects' started by blawar, Oct 10, 2019.

  1. godreborn

    godreborn GBAtemp Guru

    Member
    12
    Oct 10, 2009
    United States
    nm, I think I got it. somehow some of the scripts got corrupted.
     
  2. godreborn

    godreborn GBAtemp Guru

    Member
    12
    Oct 10, 2009
    United States
    @blawar , how do I know if something is verified? is there a txt file? must I compress, then decompress or is it also verified when an nsp is compressed?
     
  3. godreborn

    godreborn GBAtemp Guru

    Member
    12
    Oct 10, 2009
    United States
    for some reason, the compression window keeps crashing. it took over an hour to do 3GBs at level 22. I've tried doing this on another computer, but whenever I type in nsz.py, it shows an outline of a squirrel and says "fin."
     
  4. Taorn

    Taorn GBAtemp Regular

    Member
    5
    May 27, 2017
    United States
    You didn't download the file but the web page it's one. Click on the "raw" button on the top left if you want to download files from Github directly.
     
  5. godreborn

    godreborn GBAtemp Guru

    Member
    12
    Oct 10, 2009
    United States
    I can get it to work on my computer, but my computer is too slow for it. my other computer keeps erroring on everything. same files, same version of python. it doesn't make any sense.
     
  6. PatrickD85

    PatrickD85 GBAtemp Advanced Fan

    Member
    7
    Jul 4, 2012
    Netherlands
    in front of a screen...
    Impressive work.
    This reminds of the good old trimmed days back with the Wii.
    So for now it is good for storage .. but who knows what the future will bring!
     
  7. godreborn

    godreborn GBAtemp Guru

    Member
    12
    Oct 10, 2009
    United States
    so are nsz and nspz the same thing? this is where I'm getting really confused. nspz seems to have better compression.
     
  8. Nincompoopdo

    Nincompoopdo GBAtemp Fan

    Member
    7
    May 20, 2017
    United States
    If only XCI can be compressed and works like PSP's compressed ISO, it will save so much space on external drive and Sdcard.
     
  9. Jayro

    Jayro MediCat USB and Mini Windows 10 Developer

    Member
    15
    GBAtemp Patron
    Jayro is a Patron of GBAtemp and is helping us stay independent!

    Our Patreon
    Jul 23, 2012
    United States
    Octo Canyon
    Those are some impressive-ass results. Does it use the LZMA2 compression algorithm, or something else?
     
  10. godreborn

    godreborn GBAtemp Guru

    Member
    12
    Oct 10, 2009
    United States
    on that second computer, I get errors about modules being missing. I don't see how that can be, because it mirrors the computer that the scripts work on. it's way too slow on my computer. it would take weeks to compress 1TB, so I thought I'd use a different computer. nicoboss' nszip works well, and it's pretty fast, but I'm getting nspz, which I have no idea how it relates to nsz. I can't believe the compression, because with nszip, I get about 4KBs as opposed to 120KBs for the different costumes in contra. nsz was like 52KBs, so I don't know if something is wrong. I don't own a hacked switch, but I wanted to be prepared for it. suffice it to say, I can't test these things, only ask questions about them.
     
    Last edited by godreborn, Oct 12, 2019
  11. JRoad

    JRoad Developer

    pip Developer
    8
    May 18, 2018
    Spain
    If your issue are modules i added compression in option 8 of nsc_builder and there is a x86 and a x64 version compiled as exe:
    https://github.com/julesontheroad/NSC_BUILDER/releases/tag/v0.93
    Also if you want speed try the lower levels is where you get the most significant compression bump.
    I saw you want verification too. I'll add something easier but for now you can decompress and use mode 4: File-Info, then option 8 "verify". That will verify the decompressed file.
     
    Last edited by JRoad, Oct 12, 2019
    godreborn likes this.
  12. godreborn

    godreborn GBAtemp Guru

    Member
    12
    Oct 10, 2009
    United States
    I'll try it. what exactly is an nspz/xciz? they don't appear to be the same as nsz. in the release thread of nszip, it says that it goes from nsz to nspz. nspz isn't compatible with something like sx os, is it? I think they added support recently for nsz. what's the point in taking it one step further if nothing supports it? the compression seems to be even more insane with nspz. I am very confused by these terms.
     
  13. JRoad

    JRoad Developer

    pip Developer
    8
    May 18, 2018
    Spain
    nspz was the old format from Nicoboss, nsz and xcz are normal nsp or xci files with compressed nca files called nsz. Only big nca types are compressed because that's where the gain and not compressing the control and meta nca files allows to keep it compatible with reading file information, for example i could add support to most of my file information functions quickly with nsz.
    Nicoboss can probably give you more information about his standard since i only checked the code for his first revision. Either way i'll add the xcz format in NSC_Builder soon.
    Either way this format thinks about xci and nsp files as a container modifying only part of the ncas inside.
     
    godreborn likes this.
  14. godreborn

    godreborn GBAtemp Guru

    Member
    12
    Oct 10, 2009
    United States
    alright, I have them compressing. I just hope cmd doesn't crash. this happened the last couple times I used the python scripts. I was/am using 22 as the compression ratio. at this speed, I think it might take several days to compress over 100 games
     
  15. JRoad

    JRoad Developer

    pip Developer
    8
    May 18, 2018
    Spain
    Yeah, it'll depend on your aim. Level22 is slow also don't use threads at level22 because it may give you memory errors, at least that happens for me with most of the files. I'd say threads are mostly good at lower levels even if they hurt compression a little.
     
    godreborn likes this.
  16. godreborn

    godreborn GBAtemp Guru

    Member
    12
    Oct 10, 2009
    United States
    yeah, I put d. can this ever cause a game to become corrupted? that's my fear, especially when using level 22. I've spent a lot of time fixing everything, like those nsps that have part of the file cut off.
     
  17. JRoad

    JRoad Developer

    pip Developer
    8
    May 18, 2018
    Spain
    No, tinfoil decrypts it on the console. Either way i'll add my verification pre-compression and after. Basically what can make something compress badly is if the titlekey or masterkeys the user has have a wrong value which the verification will solve.
    For decompression\installation i'd say blawar made it simple enough that it won't give issues.
     
    godreborn likes this.
  18. godreborn

    godreborn GBAtemp Guru

    Member
    12
    Oct 10, 2009
    United States
    do you mean that the files are verified after compression? or, should I just decompress them and check if the files are the right size?
     
  19. godreborn

    godreborn GBAtemp Guru

    Member
    12
    Oct 10, 2009
    United States
    for some reason, some of my nsz files are larger than their nsp counterparts, like the contra shirts. nsp = 120KBs; nsz = 155KBs. this only happens on really small nsp files.
     
  20. JRoad

    JRoad Developer

    pip Developer
    8
    May 18, 2018
    Spain
    DLCs maybe? I guess the zstandard adds some data to decompress the file. Also those really small dlcs are normally a txt file with some lines (txt,json,.data) I saw one that just said "unlock content number 4).
    It was proposed to me to add an skip option for files smaller than X size or just dlcs smaller than that size, I'll do in next version.
     
    godreborn likes this.
Quick Reply
Draft saved Draft deleted
Loading...