Yet Another BlueDump MOD (YABDM).

Discussion in 'Wii - Hacking' started by DarkMatterCore, Sep 16, 2013.

  1. DarkMatterCore
    OP

    DarkMatterCore I like turtles.

    Member
    890
    199
    May 30, 2009
    Venezuela
    Cabimas, Zulia, Venezuela.
    This is a modification of BlueDump I made in August during my vacation, which fixes a lot of problems present in the original application. This is not the first mod I have done to BlueDump; there was a little one I did back in 2011 here, but I lost its source, so I had to do everything again from scratch (plus some other things that actually help in the WAD dumping process).

    It is able to dump any title to a WAD file and saves in their unencrypted form, just like the original BlueDump. The main features that differentiate it from its predecessor are a lower memory usage, ability to dump titles bigger than 47MB (even the "Wii + Internet" Channel!), compatibility with USB storage devices, GCN controllers and Classic Controllers, full hardware access with vWii support (using libruntimeiospatch, so big thanks to both damysteryman and Excelsiior) and the ability to convert content.bin files saved in sd:/private/wii/title to WADs.

    I personally want to thank JoostinOnline, because he really did help me out in the latest revisions. Most code optimizations, the new controller input code, the new icon.png and even support for meta.xml arguments were entirely done by him. :)

    Here's the full changelog since BlueDump Alpha 3:

    Warning: Spoilers inside!
    Subversion repository at Google Code: http://code.google.com/p/bluedump-mod/source/list.

    You can always get the latest version here (with icon.png and meta.xml). Currently on v1.8.
     
    Razor83, Fishaman P, actar and 8 others like this.


  2. flamepanther

    flamepanther GBAtemp Regular

    Member
    158
    39
    Apr 16, 2011
    United States
    I think this is much needed. I could have used this months ago when the unmodified version kept crashing when dumping certain games I bought from the shop. I ended up having to dump my entire NAND and use ShowMiiWads to extract the games from it. Now maybe a few others can avoid the same chore.
     
    DarkMatterCore likes this.
  3. DarkMatterCore
    OP

    DarkMatterCore I like turtles.

    Member
    890
    199
    May 30, 2009
    Venezuela
    Cabimas, Zulia, Venezuela.
    Well, not saying that you should, but you would do me a great favor if you actually tried to dump the titles that made the original BlueDump crash in your Wii, using YABDM of course. Literally, the only people besides me that have tested the application are JoostinOnline and stomp_442; most of the bugs I found were fixed because I actually reproduced them in my countless tests.

    There are a few things I'm interested to test in other consoles:

    * General application performance VS original BlueDump performance.
    * DLC name reading AND dumping.
    * Content.bin conversion process to WAD files. In all my tests, the hashes from the converted files matched those from the WADs created through the main title browser, but I haven't had the opportunity to test this in a different Wii.
    * Most importantly, actual vWii compatibility! (includes everything mentioned above)

    I also want to improve the USB compatibility, though I don't really know where to start. It shouldn't be necessary to try out 45MB+ channel dumping; the mere fact that I was able to dump the "Wii + Internet" channel proves that it works OK. :)
     
  4. Maxternal

    Maxternal Peanut Gallery Spokesman

    Member
    5,210
    2,073
    Nov 15, 2011
    Deep in GBAtemp addiction
    Nice work again on this. I'll have to finally test it sometime.
     
  5. flamepanther

    flamepanther GBAtemp Regular

    Member
    158
    39
    Apr 16, 2011
    United States
    I wouldn't object to doing this, in theory. In reality there are three problems:
    -I don't remember which they were.
    -It wasn't consistent. A title would crash it on one try, but work on the next try. I was dumping a lot of games, so I got tired of it.
    -I don't have any games on my real NAND anymore.

    Maybe this weekend I can make time to grab some old purchases from the shop channel and just sort of see if it's more reliable in general, and try out the USB support though.
     
    DarkMatterCore likes this.
  6. DarkMatterCore
    OP

    DarkMatterCore I like turtles.

    Member
    890
    199
    May 30, 2009
    Venezuela
    Cabimas, Zulia, Venezuela.
    Yeah, that was probably because the original BlueDump allocated a memory block to copy the whole content file to MEM2. Doing that isn't very safe, considering the small amount of memory the Wii has. If you do test it, please, let me know. :)

    BTW, seeing that a lot people is now using emulated NAND filesystems, I might consider adding support for them; after all, the whole content.bin compatibility was a suggestion, not something I really planned to do from the beginning.

    The only question I have is, what do you use nowadays to do NAND emulation? I'm not very familiar with it. If they are *NEEK based modules, do they now allow the use of ISFS commands? If I recall correctly, they were broken under *NEEK.

    I want to keep it as simple as possible: just basic title dumping capabilities, not something major like the BlueDump MOD by scooby; I always said that if you want to do a full NAND FS dump, you can use BootMii + nand.bin extractors, FS ToolBox or Simple FS Dumper. No need to reinvent the wheel.
     
  7. Iwatasan

    Iwatasan Newbie

    Newcomer
    3
    3
    Sep 24, 2013
    United States
    DarkMatterCore, just wanted to chime in and say this is a fantastic application. I'm setting up my new Wii U and decommissioning my old Wii, but I wanted backups of the channels I had on it. I tried so freaking many things to make that happen but only YABDM worked reliably. I would say it worked 100% but I have one issue, and since this release is so new, no amount of searching has helped me find an answer.

    I'm trying to back up Photo Channel 1.1 as a WAD. I'm on System Menu 4.2U. From DOP-Mii or NUSD I can download Photo Channel 1.0 and Photo Channel 1.1. I can also download Photo Channel 1.1 from the Shop Channel, but Shop Channel doesn't offer 1.0 anymore. Installing different combinations of these three yields different results. The only combination that seems to work as expected is installing Photo Channel 1.1 from both DOP-Mii/NUSD and Shop Channel simultaneously. Installing either one on their own doesn't work.

    DOP-Mii/NUSD downloads an 8.13MB WAD, but installing it only makes it so DOP-Mii reports "v3" installed while in fact nothing appears on the System Menu or in Wii Data Management. Shop Channel downloads a file only 3 blocks in size, and installing it creates a Photo Channel-looking channel on the System Menu, except it doesn't have a "Start" button, and when selected, it displays "To continue using the Photo Channel, you will need to perform a Wii system update." So that one's more of a placeholder channel, but at least it shows up in Wii Data Management and can be erased that way.

    Using YABDM I was able to discover the 3 block file was a nameless title under "Disc Savedata" with Title ID 48415a41. Dumping this as savedata yields a 520 byte file, "title.tmd." I understand YABDM can restore this file as well as back it up, but it seems that if the Photo Channel 1.1 (downloaded from the Shop Channel) is erased via Wii Data Management>Channels, YABDM has no reference point to restore the TMD to, and therefore can't do it.

    Other than updating to System Menu 4.3U, is there a way around this? Is there a homebrew app other than YABDM that can install TMD files where a savedata title doesn't already exist? If not, would there be a simple way to add such a feature to the next release of YABDM?

    I'll probably try updating to System Menu 4.3 to see if this issue resolves itself, but this seems like an interesting enough confounder on the part of Nintendo that perhaps it isn't the only time they've used such a thing. If that's true, maybe a "novel savedata 'restore'/install" feature would come in handy for more than Photo Channel 1.1 and for more people than just myself. Of course that's up to you and depends entirely on how simple it would be to code.
     
    DarkMatterCore likes this.
  8. DarkMatterCore
    OP

    DarkMatterCore I like turtles.

    Member
    890
    199
    May 30, 2009
    Venezuela
    Cabimas, Zulia, Venezuela.
    First of all, I'm glad you found this application useful! It really took me weeks to get it to work just the way I wanted; it feels awesome to know it was handy for someone else, too. :)

    Regarding the whole Photo Channel problem, as far as I know, when both versions are available, the second one doesn't entirely replace the first one; instead, just its "placeholder" in the System Menu gets changed. If you delete v1.1 from the Data Management menu, you should be able to use v1.0 again. That's the way it was originally meant to work. Actually, as you may have already noticed, they're installed to different directories, thus having different TIDs.

    Just like you're saying, it's completely possible for YABDM to restore the title.tmd file you mention if the title directory already exists, but it would be hard to do that once it gets deleted for two reasons:

    1. The "reference point", namely, the directory itself, doesn't exist. YABDM strongly relies on it to do its operations.

    2. If a TMD exists, most likely the Ticket file for the title is lying around in /ticket, too. When the title gets removed, usually, the Ticket file is also deleted; why is the title.tmd file sometimes left behind, beats me. In this particular case, the application has no way to get the Ticket and reinstall it. Remember: if a title exists, it HAS to have a Ticket.

    It would be "feasible" to do with ES_* calls, but again, I don't want to add title installing functions to YABDM. There are already a lot of applications that excel at doing this.

    It's a very weird problem, anyway. I have messed around with this channel in countless opportunities and never had anything like this. I'll look into it and see what can I do for you. ;)
    _____________________________________________

    EDIT: Here are my findings about this title. Even though it gets installed to /title/00010000 (like any common Wii save data), it DOES have a content file, 00000000.app. I succeeded in creating its WAD file by first copying the "channel" to the SD Card using the Data Management menu, and then converting the content.bin file using YABDM.

    It seems that this title is the one responsible for replacing the Photo Channel v1.0 placeholder with the v1.1 data. So yeah, when you delete it, v1.0 becomes usable again (if it's installed, of course). Better yet, it does have an internal name, it is just not being read properly because YABDM treats it as a savedata and thus, tries to open its banner.bin (which doesn't exist).

    I'll release a new version as soon as possible.
    _____________________________________________

    EDIT #2:

    Download: http://www13.zippyshare.com/v/31660912/file.html.

    Please, try it out! :)
     
    Iwatasan likes this.
  9. Lord Coolman
    This message by Lord Coolman has been removed from public view by raulpica, Sep 25, 2013, Reason: Your post is -rp.
    Sep 25, 2013
  10. Iwatasan

    Iwatasan Newbie

    Newcomer
    3
    3
    Sep 24, 2013
    United States
    Even though you came at it from a slightly different angle, you still solved my problem. In your initial response, you focus on the issue of having Photo Channel 1.0 and Photo Channel 1.1 at the same time. Re-reading my first post I realize I did not make clear enough that I was focusing on the issue of having Photo Channel 1.1 from NUSD/DOP-Mii (8.13MB version) and "Photo Channel 1.1" from Shop Channel (520 byte version) installed at the same time, WITHOUT Photo Channel 1.0 being installed (I removed it with DOP-Mii). But, the main issue was just dumping that little 520 byte title to WAD, which was a point that apparently got across!

    Furthermore, you solved the problem much more gracefully than the way I suggested. After reading your replies and thinking harder about the problem, it's pretty clear this little 520 byte title is just an unusual answer to the unusual question of, "How should Nintendo update the Photo Channel in a way that allows users to downgrade back to the original version if they try 1.1 but don't like it?" Unusual, and I think, most likely unique. Adding a title installing function to YABDM to solve the problem of restoring a backed-up copy of such a unique title.tmd would most definitely be overkill since it's almost certainly the only example of Nintendo doing things this way. Making an update that specially handles just this unique title may also seem overkill to some, but I think it's awesome that you did it, even though there was already a workaround method I had overlooked!

    I could have done exactly what you did by transferring the BIN to SD and converting it to WAD with YABDM, but I didn't think of it. Why didn't I think of doing that? I hadn't tried converting a BIN to WAD via YABDM yet. In fact, I have seen posts on various forums where noobs get flamed for asking the ridiculous question of "Can a BIN be converted to WAD?" I am quite surprised YABDM can do it, and this on top of directly backing up channels as expertly as it does!

    Judging by your edits, though, I'm quite certain you're talking about the exact same "Photo Channel 1.1" from Shop Channel (520 byte version) that I was principally addressing in my first post. So it's called "title 00010000-HAZA" is it? Has a nice ring to it don't you think? No? Haha. Perhaps you can send me that WAD since that's really what I've been after and it would save me the trouble of digging my Wii back out of the closet to mess with it? Can we arrange that?

    In your second edit, perhaps it should read:
    I'm sorry such a minor gripe required a whole new release just to correct this one little unusual/unique glitch, but I certainly appreciate the effort. And since you had to update a couple other things too I hope I shouldn't feel like I put you out too much! Haha. Anyway, really great fuckin' work man! You rock!!! Love the app. THANK YOU!
     
    DarkMatterCore likes this.
  11. DarkMatterCore
    OP

    DarkMatterCore I like turtles.

    Member
    890
    199
    May 30, 2009
    Venezuela
    Cabimas, Zulia, Venezuela.
    What's so unusual regarding this title is that it gets installed to /title/00010000 (savedata dir)... If it was installed, for example, in /title/00010001 (installed channel dir), an update to YABDM shouldn't have been necessary. The problem was mainly related to the way the application handles each type of title; a proper workaround would be to check the "content" dir for the currently selected savedata and see if there are any APP files on it. But, just like you said, this would be overkill... this is probably the only case of a channel being installed as a savedata, so the whole fix for it was just adding an exception in the dump_menu() function (yep, a hardcoded solution).

    I think it is pretty clear now that there actually exist three different titles referred to as "Photo Channel":
    • v1.0.
    • v1.1.
    • "Swap" module installed in the savedata directory.
    It was fun to do this fix, I actually didn't know about the existence of this third title. :)

    I wouldn't worry too much, after all my English is not that good and I may have misunderstood some of the information. But after doing my tests, it was pretty clear to me what you wanted to do.

    Lord Coolman was the one who suggested this in the first case. It wasn't really hard, the content.bin format was already documented on Wiibrew; all I had to do was adding a way of reading the console-unique key (PRNG Seed) from the OTP, and of course write new code to handle the contents stored inside the BIN file.

    The first version of the code was very ugly, since I was decrypting the part A of the file (which has a fixed length of 0x640 bytes) to get the size of the part B, then I sought the combined length of both parts to get to the part C (which, by the way, also has a fixed length of 0x80 bytes), where I then read the size of the TMD (part D). Once I got to the TMD offset, I was able to actually do the conversion process, since everything else after it (part E) is essentially the encrypted channel contents.

    The current code now searches for the "Bk.." magic word, which can be used to identify the offset of the part C.

    Yup, "HAZA" is the ASCII representation of the hexadecimal Title ID you posted earlier, "48415a41". It's easier to remember IDs this way than having to memorize their hex counterpart.

    Anyway, check your PMs.
     
    Lord Coolman and Iwatasan like this.
  12. Iwatasan

    Iwatasan Newbie

    Newcomer
    3
    3
    Sep 24, 2013
    United States
    Yep you've got that straight. I like the name "Photo Channel Swap Module" too, good choice. Has a better ring to it than "00010000 HAZA." Haha. Well, judging by the effort I put into trying to find information about it, I'd say hardly anyone knows about its existence. I wish I could say I knew about it all along, but it was definitely stumbled upon purely by accident. Would have been very easy to miss.

    Here's the explanation of what happens under every combination of these three titles:

    If Photo Channel 1.1 is installed on its own, a channel will not be displayed for this app on the System Menu; installing the Photo Channel Swap Module is required for it to properly display and function.

    If Photo Channel 1.0 is installed on its own or alongside Photo Channel 1.1 without the Photo Channel Swap Module, Photo Channel 1.0 is used, and Photo Channel 1.1 is not.

    If the Photo Channel Swap Module is installed on its own or alongside Photo Channel 1.0 without Photo Channel 1.1, Photo Channel 1.0 will not be used; instead, an unusable placeholder channel for Photo Channel 1.1 will be displayed, telling you to update your Wii to the latest version of the System Menu.

    If Photo Channel 1.0, Photo Channel 1.1, and the Photo Channel Swap Module are all installed simultaneously, Photo Channel 1.0 will not be used, but Photo 1.1 will be used and properly displayed.
    Haha, I get the gist of your technical explanation (thanks for dumbing it down for me too, haha), but yeah the details are over my head. A hacker who winds up here from a Google search is more likely to be able to fully appreciate the details as well as the skill it must have taken to pull off the previously impossible. As for me I just appreciate how approachable you've been.

    So, thanks for the app, thanks for the update, thanks for the WAD, and thanks for giving me the time of day. Haha, I think that about covers it. Resume original thread.

    Your English is flawless down to the proper usage of semicolons, by the way. No need to even bring that up.
     
    DarkMatterCore likes this.
  13. bezem

    bezem GBAtemp Regular

    Member
    211
    60
    Dec 15, 2012
    United States
    I also wanted to chime in and say I found this program mighty useful! Some background - I have a special place in my heart for the Wii; it's the first console I bought on my own on launch day and I played that thing into the ground. So with the Wii U's Wii mode I was ecstatic to finally have my Wii Menu back, as I assumed the Wii U would just have some form of integrated backwards compatibility like the Wii did for Gamecube. I personally prefer having the Wii Menu in its own sandbox. As a sentimental fool, I also wanted my Wii Menu to look exactly as it did on my old Wii. That meant the ability to install and have all the channels that are missing from the Shop Channel on the Wii U. I had successfully installed all but three - the latest versions of Netflix, Amazon, and Youtube. For some reason when I'd use w?dder and ShowmiiNand with my old Bootmii backups, I couldn't repack these as they'd just give me an error. I also couldn't download them from NUSD as they didn't have a ticket. I tried going in with a Hex editor but my lack of hexperience left me unsuccessful. Using this on an actual Wii, I was able to make WADs of all three, and the Netflix and Youtube both installed! The Amazon threw out a -1022 error when I tried it, so I'm still working on it. Otherwise this app has been exactly what I needed and I thank you and anyone who has ever worked on it!
     
    DarkMatterCore likes this.
  14. actar

    actar GBAtemp Fan

    Member
    376
    76
    Feb 3, 2011
    Italy
    Torino (Italy)
    Thanks vety much! great application!!
     
    DarkMatterCore likes this.
  15. DarkMatterCore
    OP

    DarkMatterCore I like turtles.

    Member
    890
    199
    May 30, 2009
    Venezuela
    Cabimas, Zulia, Venezuela.
    bezem If you send me a copy of the Amazon Channel dump you created with YABDM, I may be able to fix that behaviour. Error -1022 is usually related to the hash of a given content not matching the one saved on the TMD. This might or might not be a bug in the application itself, I can only be sure of that once I get to examine the output file.

    I haven't been very active lately because of my university studies, but as soon as I get my holiday vacations, I'll be back into action. Thank you very much for your comments, I really do appreciate them! :)

    And remember, if you have any other suggestions, let me know about them.
     
    bezem likes this.
  16. bezem

    bezem GBAtemp Regular

    Member
    211
    60
    Dec 15, 2012
    United States
    I'll be sure to send it over to you; I'll just have to make a new one. I actually tried w?dder and showmiinand again and was able to successfully get the latest Amazon wadded and installed on vWii. That said, I'm more than happy to make an Amazon WAD again from my original Wii and send it over to you for testing. It's my wife's birthday today so it won't be today, but I promise I will!
     
  17. W hat

    W hat Rhythm Heaven Fan

    Member
    619
    112
    Feb 28, 2007
    United States
    If I have Kirby's Adventure (NES Virtual Console), purchased from the store, and so does a friend, and we both dump it using Bluedump, will our WADs be the same? Will they be identical to a scene release WAD for that game?
     
  18. DarkMatterCore
    OP

    DarkMatterCore I like turtles.

    Member
    890
    199
    May 30, 2009
    Venezuela
    Cabimas, Zulia, Venezuela.
    No problem at all, I can wait. :lol:

    If both of you fakesign the Ticket, the output WADs should be exactly equal (remember that the Ticket has console-specific information that can only be removed by fakesigning it). Though I doubt the file will be the same as the scene release, because they tend to use a slightly different certificate chain; I don't know the reason behind this. Plus, they don't always add the WAD footer; and if they do, sometimes it isn't aligned to a 64-byte boundary. That's enough to make them fairly different to the ones generated by YABDM.

    Another problem I have found is that some WAD packers align each individual content file to a 64-byte boundary before encrypting, whilst YABDM first aligns them to a 16-byte boundary, encrypts the data and THEN aligns the output data to a 64-byte boundary. That's what Nintendo does; if you compare any encrypted content generated by the application (e.g. by using a hex editor), they should be the same as the ones stored on NUS. In fact, the original BlueDump was doing this. ;)

    I tend to be very picky with this kind of stuff, so it's only natural the application is aimed to properly dump the data in a way that it complies with the currently available WAD format documentation. I never took the scene releases as a starting point; as a reference point, yes, but no more than that.
     
  19. JoostinOnline

    JoostinOnline Certified Crash Test Dummy

    Member
    10,966
    3,754
    Apr 2, 2011
    United States
    The Twilight Zone
    Tell her you're doing something important, but that she can just wait for you in the bedroom.
     
  20. bezem

    bezem GBAtemp Regular

    Member
    211
    60
    Dec 15, 2012
    United States
    Sorry to necro-bump but I thought I'd follow-up on my promise. Amazon Instant Video actually had an update so I tried making a WAD again using YABDM and it worked flawlessly. So whatever caused WAD creation to fail previously for Amazon is no more.
     
    DarkMatterCore likes this.
  21. DarkMatterCore
    OP

    DarkMatterCore I like turtles.

    Member
    890
    199
    May 30, 2009
    Venezuela
    Cabimas, Zulia, Venezuela.
    I have been updating the application in the last few days. The changes aren't really that big deal (since there's nothing much left to do, apart from maybe doing a GUI), but here are them, anyway:

    You can find the download on the first post.
     
    bezem, Cyan, Maxternal and 1 other person like this.