RELEASE NSZ Title Compression Results

Discussion in 'Switch - Emulation, Homebrew & Software Projects' started by blawar, Oct 10, 2019.

  1. blawar
    OP

    blawar Developer

    pip Developer
    11
    Nov 21, 2016
    United States
    As many of you already know, I executed @nicoboss ' NSP compression idea in Tinfoil 5.00. The main reason I implemented it, was to increase install speed over wifi. However it also saves storage space, and reduces network transfer times from cloud storage.

    Most NSZ's will install faster over wifi, than Goldleaf can install the NSP over USB.

    Average compression was 58% across the library. I believe most of these are at level 17 compression, not max 22.

    Here is a CSV file with individual title compression results:

    I am so happy to be apart of this welcoming, inclusive scene, where people could put away petty differences to come together to accomplish awesome things. Together we can move mountains!

    edit: link to info and sample scripts for compressing and decompressing: https://github.com/nicoboss/nsz

    edit2: updated stats zip to include most data saved.


    Mario Kart 8 Deluxe [0100152000022000][v0].nsz,-2.1 GB
    GRANDIA HD Collection [0100E0600BBC8000][v0].nsz,-2.1 GB
    Anima Gate of Memories [0100706005B6A000][v0].nsz,-2.1 GB
    Please Teach Me Onedari Shogi [0100F2C00EED4000][v0].nsz,-2.1 GB
    Monster Boy and the Cursed Kingdom [01006F7001D10000][v0].nsz,-2.1 GB
    Forgotton Anne [010059E00B93C000][v0].nsz,-2.1 GB
    Mulaka [0100211005E94000][v0].nsz,-2.1 GB
    Aqua Moto Racing Utopia [0100D0D00516A000][v0].nsz,-2.2 GB
    Nights of Azure 2 Bride of the New Moon [0100628004BCE000][v0].nsz,-2.2 GB
    EARTHLOCK [01006E50042EA000][v0].nsz,-2.2 GB
    Fall of Light Darkest Edition [01005A600BE60000][v0].nsz,-2.2 GB
    PLANET ALPHA [010003C0099EE000][v0].nsz,-2.2 GB
    Monster Hunter Generations Ultimate [0100770008DD8000][v0].nsz,-2.2 GB
    Nippon Marathon [0100EF5008FC4000][v0].nsz,-2.2 GB
    Star Story The Horizon Escape [0100D7000AE6A000][v0].nsz,-2.2 GB
    Robot Squad Simulator [01001B800D742000][v0].nsz,-2.2 GB
    Bulletstorm Duke of Switch Edition [01003DD00D658000][v0].nsz,-2.2 GB
    Dragon's Dogma Dark Arisen [010032C00AC58000][v0].nsz,-2.3 GB
    Disgaea 1 Complete [01004B100AF18000][v0].nsz,-2.3 GB
    Serial Cleaner [0100818008004000][v0].nsz,-2.3 GB
    Baobabs Mausoleum Ep 2 1313 Barnabas Dead End Drive [010079300E976000][v0].nsz,-2.3 GB
    Azure Reflections [01006FB00990E000][v0].nsz,-2.3 GB
    Atelier Lulua The Scion of Arland [0100B1400CD50000][v0].nsz,-2.4 GB
    GRIS [0100E1700C31C000][v0].nsz,-2.4 GB
    Pumped BMX Pro [01009AE00B788000][v0].nsz,-2.4 GB
    Crossing Souls [0100B1E00AA56000][v0].nsz,-2.4 GB
    Baobabs Mausoleum Ep 1 Ovnifagos Don't Eat Flamingos [0100425009FB2000][v0].nsz,-2.4 GB
    Bendy and the Ink Machine [010074500BBC4000][v0].nsz,-2.4 GB
    MARVEL ULTIMATE ALLIANCE 3 The Black Order [010060700AC50000][v0].nsz,-2.5 GB
    FINAL FANTASY XII THE ZODIAC AGE [0100EB100AB42000][v0].nsz,-2.5 GB
    Alien Cruise [0100A2E00D0E0000][v0].nsz,-2.5 GB
    Crazy Strike Bowling EX [01006BC00C27A000][v0].nsz,-2.5 GB
    Furi [01009D3008D20000][v0].nsz,-2.5 GB
    My Friend Pedro Blood Bullets Bananas [010031200B94C000][v0].nsz,-2.6 GB
    Gal Metal [010048600B14E000][v0].nsz,-2.6 GB
    Detective Gallo [01009C0009842000][v0].nsz,-2.6 GB
    Disgaea 5 Complete [01005700031AE000][v0].nsz,-2.6 GB
    Jotun Valhalla Edition [0100340009736000][v0].nsz,-2.6 GB
    Little Friends Dogs & Cats [010079A00D9E8000][v0].nsz,-2.7 GB
    SENRAN KAGURA Peach Ball [0100D1800D902000][v0].nsz,-2.8 GB
    Mother Russia Bleeds [0100D9C00AA52000][v0].nsz,-2.8 GB
    MachiKnights -Blood bagos- [0100F2400D434000][v0].nsz,-2.8 GB
    NARUTO SHIPPUDEN Ultimate Ninja Storm Trilogy [0100EC800800C000][v0].nsz,-2.8 GB
    Omensight Definitive Edition [0100CDC00C40A000][v0].nsz,-2.8 GB
    Amnesia Collection [01003CC00D0BE000][v0].nsz,-2.8 GB
    Apocalipsis Wormwood Edition [01008FC00C5BC000][v0].nsz,-2.8 GB
    Blasphemous [0100698009C6E000][v0].nsz,-2.9 GB
    Phantaruk [0100DDD00C0EA000][v0].nsz,-2.9 GB
    KILL la KILL -IF [0100FB400D832000][v0].nsz,-2.9 GB
    Anima Gate of Memories - Arcane Edition [010033F00B3FA000][v0].nsz,-3.1 GB
    Shelter Generations [01009EB004CB0000][v0].nsz,-3.1 GB
    Nelly Cootalot [01005F000B784000][v0].nsz,-3.1 GB
    Mr Shifty [010031F002B66000][v0].nsz,-3.1 GB
    Hollow Knight [0100633007D48000][v0].nsz,-3.1 GB
    Masquerada Songs and Shadows [0100113008262000][v0].nsz,-3.2 GB
    Hard West [0100ECE00D13E000][v0].nsz,-3.2 GB
    Atelier Meruru The Apprentice of Arland DX [0100ADD00C6FA000][v0].nsz,-3.3 GB
    Fear Effect Sedna [01005AC0068F6000][v0].nsz,-3.3 GB
    Hover [0100F6800910A000][v0].nsz,-3.3 GB
    Big Crown Showdown [010088100C35E000][v0].nsz,-3.4 GB
    Silence [0100F1400B0D6000][v0].nsz,-3.4 GB
    BLAZBLUE CROSS TAG BATTLE [0100B61008208000][v0].nsz,-3.4 GB
    Shift Happens [0100B1000AC3A000][v0].nsz,-3.5 GB
    Perception [0100563005B70000][v0].nsz,-3.5 GB
    Resident Evil 4 [010099A00BC1E000][v0].nsz,-3.5 GB
    Exception [0100F2D00C7DE000][v0].nsz,-3.6 GB
    Petoons Party [010044400EEAE000][v0].nsz,-3.6 GB
    GIRLS UND PANZER DREAM TANK MATCH DX [01006DD00CC96000][v0].nsz,-3.6 GB
    PC Building Simulator [010072800CBE8000][v0].nsz,-3.6 GB
    OKAMI HD [0100276009872000][v0].nsz,-3.6 GB
    Atelier Totori The Adventurer of Arland DX [01009BC00C6F6000][v0].nsz,-3.7 GB
    R B I Baseball 18 [01005CC007616000][v0].nsz,-3.8 GB
    Shadow Blade Reload [0100D5500DA94000][v0].nsz,-3.8 GB
    Away Journey To The Unexpected [01002F1005F3C000][v0].nsz,-3.8 GB
    Guts & Glory [01005E5009EF2000][v0].nsz,-3.9 GB
    DOOM 3 [010029D00E740000][v0].nsz,-3.9 GB
    Atelier Lydie & Suelle The Alchemists and the Mysterious Paintings [010009900947A000][v0].nsz,-3.9 GB
    Atelier Arland series Deluxe Pack [010088600C66E000][v0].nsz,-3.9 GB
    Hunting Simulator [0100C460040EA000][v0].nsz,-3.9 GB
    Ginger Beyond the Crystal [0100C50007070000][v0].nsz,-4.0 GB
    Gear Club Unlimited [010065E003FD8000][v0].nsz,-4.2 GB
    LEGRAND LEGACY Tale of the Fatebounds [0100E7500BF84000][v0].nsz,-4.3 GB
    Assassins Creed III Remastered [01007F600B134000][v0].nsz,-4.6 GB
    South Park The Fractured but Whole - Standard Edition [01008F2005154000][v0].nsz,-4.9 GB
    Devil May Cry [0100E8000D5B8000][v0].nsz,-5.0 GB
    Onimusha Warlords [0100416008A12000][v0].nsz,-5.0 GB
    Meow Motors [0100EBE00D5B0000][v0].nsz,-5.1 GB
    Go Vacation [0100C1800A9B6000][v0].nsz,-5.1 GB
    Night in the Woods [0100921006A04000][v0].nsz,-5.3 GB
    ARK Survival Evolved [0100D4A00B284000][v0].nsz,-5.8 GB
    Fire Emblem Warriors [0100F15003E64000][v0].nsz,-5.9 GB
    Pillars of Eternity Complete Edition [0100D6200E130000][v0].nsz,-6.0 GB
    SENRAN KAGURA Reflexions [0100E0C00ADAC000][v0].nsz,-6.0 GB
    Hellblade Senua's Sacrifice [010044500CF8E000][v0].nsz,-6.1 GB
    Hyrule Warriors Definitive Edition [0100AE00096EA000][v0].nsz,-6.3 GB
    Golem Gates [01003C000D84C000][v0].nsz,-6.8 GB
    Lust for Darkness [0100F3100EB44000][v0].nsz,-6.8 GB
    SINNER Sacrifice for Redemption [0100B16009C10000][v0].nsz,-7.4 GB
    L A Noire [0100830004FB6000][v0].nsz,-7.7 GB
    SmuggleCraft [0100C4200B820000][v0].nsz,-8.3 GB
    Nelke & the Legendary Alchemists ~Ateliers of the New World~ [01006ED00BC76000][v0].nsz,-8.9 GB
    Observer [01002A000C478000][v0].nsz,-12.3 GB
     

    Attached Files:

    Last edited by blawar, Oct 11, 2019
  2. Vaatu

    Vaatu Advanced Member

    Newcomer
    3
    Sep 18, 2018
    United States
    My Room
  3. aerik

    aerik Member

    Newcomer
    3
    Jan 6, 2004
    Brazil
    Congratulations to all involved!

    Sent from my ONEPLUS A6003 using Tapatalk
     
  4. mrdude

    mrdude GBAtemp Advanced Fan

    Member
    7
    Dec 11, 2015
    @blawar - I hope you somehow don't manage to fook this project up by adding some DRM crap that nobody wants or needs. However thanks for your input.
     
    one-piece and wurstpistole like this.
  5. thaikhoa

    thaikhoa GBAtemp Addict

    Member
    9
    Sep 16, 2008
    Australia
    Be a bad guy for those who need you. Thanks heaps for updating Lithium for custom XCI support.:grog:
     
    gizmomelb likes this.
  6. CrisFTW

    CrisFTW GBAtemp Maniac

    Member
    8
    Jan 10, 2016
    United States
    Nice stuff! Keep it up Blawar
     
  7. eyeliner

    eyeliner Has an itch needing to be scratched.

    Member
    7
    Feb 17, 2006
    Portugal
    Is the percentage the space gained or the compression achieved?
    If it is the resulting file size, holy damn, some games became miniscule.

    Good job!
     
  8. chrisfand

    chrisfand Newbie

    Newcomer
    2
    Sep 10, 2016
    United States
    Only the best title installers have this feature :yay:
     
  9. blawar
    OP

    blawar Developer

    pip Developer
    11
    Nov 21, 2016
    United States
    it’s percentage of original. 100% = no compression, 0% = infinite compression
     
    lordelan and eyeliner like this.
  10. eyeliner

    eyeliner Has an itch needing to be scratched.

    Member
    7
    Feb 17, 2006
    Portugal
    Some impressive results. Thanks for the tool.
     
  11. CyrusVN

    CyrusVN Advanced Member

    Newcomer
    2
    Jun 14, 2006
    Doing the lord's work thank you brah
     
  12. darkxex

    darkxex Advanced Member

    Newcomer
    2
    Jun 13, 2016
    Cote d'Ivoire
    Blawar, I'm your fan, you're my Trump, build a great wall for those who use vanilla atmosphere. And make atmosphere great again.
     
  13. JRoad

    JRoad GBAtemp Advanced Fan

    Member
    7
    May 18, 2018
    Spain
    It's really impressive, blawar never stops surprising me with his software.
     
  14. linuxares

    linuxares I'm not a generous god!

    Moderator
    17
    Aug 5, 2007
    Sweden
    Nicely done! That's some impressive stats for sure!
    Now we need a way to load the compressed files directly :P (I know it's not possible with the standard OS. This is a joke)
     
  15. Reploid

    Reploid GBAtemp Advanced Maniac

    Member
    10
    Jan 20, 2010
    Serbia, Republic of
    a joke. For now.
     
  16. linuxares

    linuxares I'm not a generous god!

    Moderator
    17
    Aug 5, 2007
    Sweden
    Hopefully for now! I think it might be fezable but it's probably a bigger chance to get a XCZ working.

    I took the liberty to split the results in to collums so it's easier to read. Just resize your collums and the odd numbers should fix themselves (Except on the IDs for some reason... I didn't care enough to try to fix it :P)
     

    Attached Files:

    lordelan likes this.
  17. duckbill007

    duckbill007 Advanced Member

    Newcomer
    4
    May 5, 2011
    Russia
    I read proposed ncz file format and do not understand - in what cases there can be more than one section? I thought all nca content is encrypted with the same key. (or not encrypted at all).
     
  18. masagrator

    masagrator File digger

    Member
    10
    Oct 14, 2018
    Poland
    NCZ is just compressed NCA with untouched header. Encryption is still there untouched.
     
  19. JRoad

    JRoad GBAtemp Advanced Fan

    Member
    7
    May 18, 2018
    Spain
    The header uses the header key and the body the keyblock for standard crypto and the decrypted titlekey for titlerights crypto.
    The sections are 2pfs0s in base program ncas and a romfs. 1 pfs0 is the logo section which is decrypted, the other is the exefs encrypted with AES ctr using the keyblock\titlekey, the romfs is encrypted the same. While the encryption is the same the way to plaintext those sections is different.
    Other ncas have less sections that can be one or another and updates use bktr which is a pain in the ass, which is the reason why it's taking a little longer for the spec to support them.
    You can read most of this information in switchbrew if you're interested.
     
  20. lordelan

    lordelan GBAtemp Psycho!

    Member
    12
    Jan 4, 2015
    Germany
    4 % of original NSP size? How is that even possible, wtf.
     
Quick Reply
Draft saved Draft deleted
Loading...