Wii U image (WUD) compression tool

Discussion in 'Wii U - Hacking & Backup Loaders' started by Exzap, Sep 22, 2015.

  1. Exzap
    OP

    Exzap GBAtemp Regular

    Member
    154
    1,445
    Sep 19, 2015
    Netherlands
    I made a little tool to compress Wii U images/dumps.

    How does it work?
    It detects duplicate sectors and only stores them once. Essentially this leads to all the empty ranges being shaved off, storing only ranges which contain file or filesystem data.
    Although the algorithm is simple, it works really well. In almost all cases it will reduce the size by more than 50% (12GB+!)

    Examples (uncompressed size is always 23.3GB):
    Super Mario World 3D -> 2.61GB (20.69GB saved)
    DKC Tropical Freeze -> 11.7GB (11.6GB saved)
    Zelda WW HD -> 2.70GB (20.7GB saved)
    Darksiders 2 -> 9.43GB (13.87GB saved)
    Mario Party 10 -> 4.01GB (19.29GB saved)

    It is lossless, meaning you can compress and uncompress a file and the outcome will still be the same. This is different to tools like WiiScrubber which actually modify the data.

    Why is this better than using RAR/ZIP/etc.?
    It's faster and tools can natively read it (obv. assuming they support it). At least my future releases will support compressed images directly.

    Download (includes source)
    Download mirror

    The easiest way to use the tool is to just drag the file you want to compress/decompress onto the .exe

    Note for developers:
    I tried to make it easy to integrate into other projects. If you want to implement it, just include wud.cpp and wud.h and you are good to go (for reading only, code for compression is in main.cpp). API should be self-explanatory. Small adjustments are needed to make it work for non-Windows platforms.
     


  2. Xuman

    Xuman GBAtemp Fan

    Member
    395
    245
    Jan 26, 2012
    United States
    Will mosdef try this asap. Good job! Small things like these are what helps out in the long run. I myself am glad the Wii U scene is healthy lately.
     
    canariobr likes this.
  3. fernas

    fernas Newbie

    Newcomer
    2
    1
    Nov 9, 2012
    Mexico
    Wow, excellent work!
     
    Margen67 likes this.
  4. davokazaki

    davokazaki GBAtemp Regular

    Member
    108
    36
    Aug 13, 2014
    Excelent work :)
     
    Margen67 likes this.
  5. Reecey

    Reecey Mario 64 (favorite game of all time)

    Member
    4,152
    1,078
    Mar 7, 2010
    At Home :)
    @Exzap . Good work just a thought is this in connection with loadiine cause I see it puts out a WiiU.wux file, not WiiU.wud, any connection with @golden45 ? interesting seems about the right time a compressor has popped up all of a sudden.
     
    Last edited by Reecey, Sep 22, 2015
  6. EclipseSin

    EclipseSin FullMental Trollemist

    Member
    1,813
    1,287
    Apr 1, 2015
    United States
    Vegeta's Hyrule
    Loadiine doesn't work with WUDs as far as I know. It works with extracted files. And I'd assume .wux is just the extension he chose for compressed wud... Could name it .iso for the same effect.
     
  7. Exzap
    OP

    Exzap GBAtemp Regular

    Member
    154
    1,445
    Sep 19, 2015
    Netherlands
    It's just a coincidence. Probably not useful for loadiine due to the reason already stated by EclipseSin.
     
    canariobr and DarkFlare69 like this.
  8. Reecey

    Reecey Mario 64 (favorite game of all time)

    Member
    4,152
    1,078
    Mar 7, 2010
    At Home :)
    I just thought .pux .wux files? I don't know.
     
  9. gamesquest1

    gamesquest1 Nabnut

    Member
    14,100
    9,433
    Sep 23, 2013
    seeing as the loader made by @golden45 uses the extracted files i doubt this will be of any use except for archiving "backup", but tbh the raw extracted file system offers the same space saving and means you dont have to extract it several times to access the actual file system :P
     
  10. Reecey

    Reecey Mario 64 (favorite game of all time)

    Member
    4,152
    1,078
    Mar 7, 2010
    At Home :)
    So what your saying is, this loader can only load official back ups that are extracted before hand, it cant load any .wud files etc..? or files in the wild. but then again once they have been extracted I would imagine they would get posted up somewhere in the correct format.
     
    Last edited by Reecey, Sep 22, 2015
  11. gamesquest1

    gamesquest1 Nabnut

    Member
    14,100
    9,433
    Sep 23, 2013
    well seeing as its based on cafiine almost certainly, not that i have access to it or know exactly what it uses, but he has stated it started as a modified cafiine to load the rpx file from the network too.....which suggests its extracted files only

    but you can extract the files from a wud file using Uwizard too using the titlekey that is included in the nfo on most releases, so they aren't exactly useless and can still be used, just that the extracted files are more manageable than trying to stream the entire encrypted image over a network
     
    Last edited by gamesquest1, Sep 22, 2015
  12. Philip3ds

    Philip3ds GBAtemp Regular

    Member
    183
    42
    May 13, 2015
    I knew that something like this would be made.
     
    Margen67 and EclipseSin like this.
  13. the_randomizer

    the_randomizer The Temp's official fox whisperer

    Member
    20,797
    9,793
    Apr 29, 2011
    United States
    Dr. Wahwee's castle
    A nice little tool, too bad there's no way to make unofficial backups yet :P
     
    CandraSoftware and Margen67 like this.
  14. Vegeta

    Vegeta GBAtemp Regular

    Member
    188
    9
    Nov 8, 2002
    Margen67, Maximilious and EclipseSin like this.
  15. kprovost7314

    kprovost7314 GBAtemp's Official Bara Master

    Member
    1,679
    864
    Dec 24, 2014
    United States
    In that bara manga ( ͡° ͜ʖ ͡°)
    Maybe this will be combined into Uwizard. @Mr. Mysterio Great work still. I was surprised that the MK8 ISO was 23GB. :lol:
     
    Last edited by kprovost7314, Sep 22, 2015
    TotalInsanity4 likes this.
  16. Mr. Mysterio

    Mr. Mysterio Super Genius

    Member
    661
    856
    Sep 16, 2014
    United States
    Rosalina's Comet Observatory
    I actually designed my own WUD compression algorithm, but the reason I didn't add it to Uwizard is that DiscU is closed source and it can't read compressed games.
     
    TotalInsanity4, Margen67 and Irastris like this.
  17. Irastris

    Irastris Just another fag with a suggestive avatar

    Member
    1,005
    570
    May 3, 2015
    United States
    Was it just due to how your technique worked? I'd hate to compress the .wuds and not be able to easily use them anymore. (But this 186GBs is seriously killing me, I should invest in a better HDD)
     
  18. Mr. Mysterio

    Mr. Mysterio Super Genius

    Member
    661
    856
    Sep 16, 2014
    United States
    Rosalina's Comet Observatory
    I don't know what you mean. Could you please explain the quoted text a little better?
     
  19. Margen67

    Margen67 Dirty entited pirate

    Banned
    1,100
    4,583
    Nov 3, 2014
    United States
    0:10
     
  20. Irastris

    Irastris Just another fag with a suggestive avatar

    Member
    1,005
    570
    May 3, 2015
    United States
    You say your algorithm made them incompatible with DiscU, does that mean it worked differently than Exzap's program does? He claims his program allows them to still be compatible, so I'd assume it's different, but I'd still like to be sure before I go around compressing my WUD folder.