1. EyeKey

    OP EyeKey GBAtemp Regular
    Member

    Joined:
    Feb 10, 2017
    Messages:
    232
    Country:
    Israel
    After a lot of research and reversing of the proprietary file system, I finally released wfsdump - a tool for dumping the content of WFS devices. (MLC or USB). wfsdump is a is a quick command line tool that I created that wraps wfslib, the C++ library for WFS that I released among it.
    Right now this library only support reading files. Modifying WFS will require much more work and isn't planned right now, but may happen in the future. (I don't know if or when I will work on it).

    I don't have any plan to release GUI util right now. Feel free to make one using this library.


    The only thing that isn't supported right now are links, which are ignored currently by wfsdump.

    Download (Source)
    see also: wfs-fuse



    Usage
    wfsdump
    Code:
    wfsdump --help
    Code:
    Usage: wfsdump --input <input file> --output <output directory> --otp <opt path> [--seeprom <seeprom path>] [--mlc] [--usb] [--dump-path <directory to dump>] [--verbos]
    Allowed options:
      --help                 produce help message
      --input arg            input file
      --output arg           ouput directory
      --otp arg              otp file
      --seeprom arg          seeprom file (required if usb)
      --dump-path arg (=/)   directory to dump (default: "/")
      --mlc                  device is mlc (default: device is usb)
      --usb                  device is usb
      --verbos               verbos output
    Example
    Dump mlc from backup
    Code:
    wfsdump --input mlc.full.img --output dump_dir --otp otp.bin --mlc
    Dump USB device under Windows
    (Needed to be run with administrator previliges, so run from privileged command line)
    Code:
    wfsdump --input \\.\PhysicalDrive3 --output dump_dir --otp otp.bin --seeprom seeprom.bin
    You need to replace PhsyicalDrive3 with the correct device, you can figure it out with this PowerShell command
    Code:
    Get-WmiObject Win32_DiskDrive
    Dump USB device under Linux
    Code:
    sudo wfsdump --input /dev/sdb --output dump_dir --otp otp.bin --seeprom seeprom.bin

    Build
    Linux
    Install the requirements
    Code:
    sudo apt-get install g++ make libboost-dev libboost-system-dev libboost-filesystem-dev libboost-program-options-dev libcrypto++-dev
    Run the makefile
    Code:
    make wfsdump
    (or make if you want to build all the tools, requires also libfuse-dev)

    Visual Studio
    Visual Studio 2015 project file is provided. This project depends on the libraries boost and Crypto++. Configuration of those libraries include path and lib path is required.
     
    Last edited by EyeKey, Jul 27, 2017
  2. iAqua

    iAqua
    Member

    Joined:
    Dec 7, 2015
    Messages:
    2,848
    Country:
    United Kingdom
    Naisu!
     
    leerz and Jayro like this.
  3. Peligropowers

    Peligropowers Newbie
    Newcomer

    Joined:
    Jul 16, 2017
    Messages:
    4
    Country:
    Spain
    Great news. Good work. Thanks.

    Enviado desde mi ALE-L21 mediante Tapatalk
     
  4. proflayton123

    proflayton123 Sakura思い
    Member

    Joined:
    Jan 11, 2016
    Messages:
    6,026
    Country:
    Japan
    Wonderful to see!

    Thanks and nice works!~
     
  5. VinsCool

    VinsCool Persona Secretiva Felineus
    Moderator

    Joined:
    Jan 7, 2014
    Messages:
    14,066
    Country:
    Canada
    Fantastic work! That will be really useful!
     
  6. The Real Jdbye

    The Real Jdbye Always Remember 30/07/08
    Member

    Joined:
    Mar 17, 2010
    Messages:
    20,060
    Country:
    Norway
    Awesome work, I know lots of people have been waiting for a tool like this.
     
  7. Deleted User

    Deleted User Newbie

    Awesome. Thanks for your work!
     
  8. piratesephiroth

    piratesephiroth I wish I could read
    Member

    Joined:
    Sep 5, 2013
    Messages:
    3,430
    Country:
    Brazil
    beautiful
     
  9. nastys

    nastys ナースティス
    Member

    Joined:
    Aug 5, 2014
    Messages:
    1,715
    Country:
    Italy
    A FUSE filesystem based on this would be awesome!
     
  10. C0mm4nd_

    C0mm4nd_ Aspirant Wii U homebrew dev :P
    Member

    Joined:
    Oct 9, 2016
    Messages:
    697
    Country:
    Italy
    Cool :)
     
    Anthropod likes this.
  11. Valery0p

    Valery0p GBAtemp Fan
    Member

    Joined:
    Jan 16, 2017
    Messages:
    426
    Country:
    Italy
    Congratulations for your awesome program!
    This is the first step for something bigger on last fw...
    Sorry to bother you, but the MLC key is a per console unique key, right?
     
  12. C0mm4nd_

    C0mm4nd_ Aspirant Wii U homebrew dev :P
    Member

    Joined:
    Oct 9, 2016
    Messages:
    697
    Country:
    Italy
    yep
     
  13. Jayro

    Jayro MediCat USB and Malwarebytes Bootable Developer
    Developer

    Joined:
    Jul 23, 2012
    Messages:
    8,291
    Country:
    United States
    This would be so cool implimented into a disc dumper, ripping games right to the USB harddrive, instead of SD card first, then having to install SD to USB.
     
    VinsCool and Deleted User like this.
  14. piratesephiroth

    piratesephiroth I wish I could read
    Member

    Joined:
    Sep 5, 2013
    Messages:
    3,430
    Country:
    Brazil
    @EyeKey Could this be adapted to reencrypt the files (without modification) with a different key?
    I mean, could you skip the additional troublesome operations by keeping the the metadata (hashes, owners, permissions, IVs, etc) in a separate file and use that to reencrypt the dumped files?
     
    Last edited by piratesephiroth, Jul 24, 2017
  15. EyeKey

    OP EyeKey GBAtemp Regular
    Member

    Joined:
    Feb 10, 2017
    Messages:
    232
    Country:
    Israel
    I am not planning to do it. (it is open-source, so if someone else want to do it..).
    And right now I don't even parse everything in the filesystem, the Area object is pointing to few more metadata blocks that I don't read or parse at all, and they need to be reencrypted too, and I have no idea if they point to more metadata pages, they probably do.

    And another small thought, you don't even need to know the correct IV of each block to decrypt and reencrypt it correctly, because it is CBC, but you do need to know where each block start and end, and there are different sizes of blocks (0x1000, 0x2000, 0x8000). (So you need to parse the filesystem anyway...)
     
    Last edited by EyeKey, Jul 24, 2017
    piratesephiroth likes this.
  16. EyeKey

    OP EyeKey GBAtemp Regular
    Member

    Joined:
    Feb 10, 2017
    Messages:
    232
    Country:
    Israel
    Added Linux support.
     
    ihaveamac, C0mm4nd_ and nastys like this.
  17. ihaveamac

    ihaveamac GBAtemp Guru
    Member

    Joined:
    Apr 20, 2015
    Messages:
    5,922
    Country:
    United States
    thanks for your work on the wfs format, I appreciate it.

    any chance of proper macOS support? I was trying to build this on 10.12.6 but ran into some errors... I used Homebrew to get the dependencies ("brew install boost cryptopp"), then tried to build wfslib/wfslib, and this is the output (note g++ is actually clang)
    Code:
    g++ -c -Wall -Werror -std=c++14 Area.cpp -o Area.o
    In file included from Area.cpp:8:
    In file included from ./Area.h:13:
    ./Block.h:43:15: error: implicit instantiation of undefined template 'std::__1::basic_string<char,
          std::__1::char_traits<char>, std::__1::allocator<char> >'
                    std::string msg;
                                ^
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/iosfwd:193:33: note:
          template is declared here
        class _LIBCPP_TYPE_VIS_ONLY basic_string;
                                    ^
    In file included from Area.cpp:9:
    ./MetadataBlock.h:19:15: error: 'MetadataBlock::Fetch' hides overloaded virtual function [-Werror,-Woverloaded-virtual]
            virtual void Fetch(bool check_hash = true);
                         ^
    ./Block.h:31:15: note: hidden overloaded virtual function 'Block::Fetch' declared here: different number of parameters
          (0 vs 1)
            virtual void Fetch();
                         ^
    2 errors generated.
    make: *** [Area.o] Error 1
    
    to use actual g++ (from brew, replacing CXX with g++-7 in wfslib/wfslib/Makefile) results in no errors. but then wfslib/wfsdump doesn't build for some reason (regardless of compiler)
    Code:
    g++-7 -I../ -c -Wall -Werror -std=c++14 main.cpp -o main.o
    g++-7 main.o -L../ -lwfs -lboost_system -lboost_filesystem -lboost_program_options -lcryptopp -o wfsdump
    Undefined symbols for architecture x86_64:
      "boost::program_options::to_internal(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)", referenced from:
          std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > boost::program_options::to_internal<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) in main.o
      "boost::program_options::options_description::options_description(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int, unsigned int)", referenced from:
          _main in main.o
      "boost::program_options::arg[abi:cxx11]", referenced from:
          boost::program_options::typed_value<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, char>::name() const in main.o
      "boost::program_options::detail::cmdline::set_additional_parser(boost::function1<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>)", referenced from:
          boost::program_options::basic_command_line_parser<char>::extra_parser(boost::function1<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>) in main.o
      "boost::program_options::detail::cmdline::cmdline(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&)", referenced from:
          boost::program_options::basic_command_line_parser<char>::basic_command_line_parser(int, char const* const*) in main.o
      "boost::program_options::validate(boost::any&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, int)", referenced from:
          boost::program_options::typed_value<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, char>::xparse(boost::any&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) const in main.o
      "boost::program_options::operator<<(std::basic_ostream<char, std::char_traits<char> >&, boost::program_options::options_description const&)", referenced from:
          _main in main.o
      "boost::program_options::abstract_variables_map::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const", referenced from:
          boost::program_options::variables_map::operator[](std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const in main.o
      "boost::program_options::value_semantic_codecvt_helper<char>::parse(boost::any&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, bool) const", referenced from:
          vtable for boost::program_options::typed_value<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, char> in main.o
    ld: symbol(s) not found for architecture x86_64
    collect2: error: ld returned 1 exit status
    make: *** [wfsdump] Error 1
    
    I may just be missing something, I don't know.
     
  18. EyeKey

    OP EyeKey GBAtemp Regular
    Member

    Joined:
    Feb 10, 2017
    Messages:
    232
    Country:
    Israel
    Thank for your report, I have fixed those error, now you can compile this project with clang.
    About the second one, check out this: https://stackoverflow.com/questions...gram-options-on-mac-os-x-with-g-4-8-mac-ports
    you need to get the right variant of boost (but you should probably use clang)

    If you don't want to compile the new fuse app (will fail on mac?) just run "make wfsdump" to compile only wfsdump.
     
    Last edited by EyeKey, Jul 27, 2017
  19. ihaveamac

    ihaveamac GBAtemp Guru
    Member

    Joined:
    Apr 20, 2015
    Messages:
    5,922
    Country:
    United States
    wfslib builds with clang now (no errors/warnings), but wfsdump now produces some new errors.
    Code:
    $ make wfsdump
    /Applications/Xcode.app/Contents/Developer/usr/bin/make -C wfslib
    make[1]: Nothing to be done for `all'.
    /Applications/Xcode.app/Contents/Developer/usr/bin/make -C wfsdump
    g++ -I../ -c -Wall -Werror -std=c++14 main.cpp -o main.o
    In file included from main.cpp:10:
    In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/fstream:170:
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/istream:295:26: error:
          implicit instantiation of undefined template 'std::__1::ctype<unsigned char>'
                    if (!__ct.is(__ct.space, *__i))
                             ^
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/istream:1230:16: note:
          in instantiation of member function 'std::__1::basic_istream<unsigned char, std::__1::char_traits<unsigned char>
          >::sentry::sentry' requested here
            sentry __sen(*this, true);
                   ^
    main.cpp:37:12: note: in instantiation of member function 'std::__1::basic_istream<unsigned char,
          std::__1::char_traits<unsigned char> >::read' requested here
                                    stream.read(&*data.begin(), std::min(data.size(), to_read));
                                           ^
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__locale:429:53: note:
          template is declared here
    template <class _CharT> class _LIBCPP_TYPE_VIS_ONLY ctype;
                                                        ^
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__locale:187:54: error:
          implicit instantiation of undefined template 'std::__1::ctype<unsigned char>'
        return static_cast<const _Facet&>(*__l.use_facet(_Facet::id));
                                                         ^
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/istream:291:41: note:
          in instantiation of function template specialization 'std::__1::use_facet<std::__1::ctype<unsigned char> >'
          requested here
                const ctype<_CharT>& __ct = use_facet<ctype<_CharT> >(__is.getloc());
                                            ^
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/istream:1230:16: note:
          in instantiation of member function 'std::__1::basic_istream<unsigned char, std::__1::char_traits<unsigned char>
          >::sentry::sentry' requested here
            sentry __sen(*this, true);
                   ^
    main.cpp:37:12: note: in instantiation of member function 'std::__1::basic_istream<unsigned char,
          std::__1::char_traits<unsigned char> >::read' requested here
                                    stream.read(&*data.begin(), std::min(data.size(), to_read));
                                           ^
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__locale:429:53: note:
          template is declared here
    template <class _CharT> class _LIBCPP_TYPE_VIS_ONLY ctype;
                                                        ^
    2 errors generated.
    make[1]: *** [main.o] Error 1
    make: *** [wfsdump] Error 2
    
    I did also just try to build wfs-fuse and some of the errors seemed similar
    Code:
    $ make wfs-fuse
    /Applications/Xcode.app/Contents/Developer/usr/bin/make -C wfslib
    make[1]: Nothing to be done for `all'.
    /Applications/Xcode.app/Contents/Developer/usr/bin/make -C wfs-fuse
    g++ -I../ -c -Wall -Werror -std=c++14 -D_FILE_OFFSET_BITS=64 wfs-fuse.cpp -o wfs-fuse.o
    In file included from wfs-fuse.cpp:12:
    In file included from ../wfslib/WfsLib.h:11:
    In file included from ../wfslib/FileDevice.h:12:
    In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/iostream:40:
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/istream:295:26: error:
          implicit instantiation of undefined template 'std::__1::ctype<unsigned char>'
                    if (!__ct.is(__ct.space, *__i))
                             ^
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/istream:1411:16: note:
          in instantiation of member function 'std::__1::basic_istream<unsigned char, std::__1::char_traits<unsigned char>
          >::sentry::sentry' requested here
            sentry __sen(*this, true);
                   ^
    wfs-fuse.cpp:81:9: note: in instantiation of member function 'std::__1::basic_istream<unsigned char,
          std::__1::char_traits<unsigned char> >::seekg' requested here
            stream.seekg(offset, stream.beg);
                   ^
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__locale:429:53: note:
          template is declared here
    template <class _CharT> class _LIBCPP_TYPE_VIS_ONLY ctype;
                                                        ^
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__locale:187:54: error:
          implicit instantiation of undefined template 'std::__1::ctype<unsigned char>'
        return static_cast<const _Facet&>(*__l.use_facet(_Facet::id));
                                                         ^
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/istream:291:41: note:
          in instantiation of function template specialization 'std::__1::use_facet<std::__1::ctype<unsigned char> >'
          requested here
                const ctype<_CharT>& __ct = use_facet<ctype<_CharT> >(__is.getloc());
                                            ^
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/istream:1411:16: note:
          in instantiation of member function 'std::__1::basic_istream<unsigned char, std::__1::char_traits<unsigned char>
          >::sentry::sentry' requested here
            sentry __sen(*this, true);
                   ^
    wfs-fuse.cpp:81:9: note: in instantiation of member function 'std::__1::basic_istream<unsigned char,
          std::__1::char_traits<unsigned char> >::seekg' requested here
            stream.seekg(offset, stream.beg);
                   ^
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__locale:429:53: note:
          template is declared here
    template <class _CharT> class _LIBCPP_TYPE_VIS_ONLY ctype;
                                                        ^
    In file included from wfs-fuse.cpp:12:
    In file included from ../wfslib/WfsLib.h:11:
    In file included from ../wfslib/FileDevice.h:12:
    In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/iostream:40:
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/istream:1230:16: error:
          no matching constructor for initialization of 'std::__1::basic_istream<unsigned char,
          std::__1::char_traits<unsigned char> >::sentry'
            sentry __sen(*this, true);
                   ^     ~~~~~~~~~~~
    wfs-fuse.cpp:82:9: note: in instantiation of member function 'std::__1::basic_istream<unsigned char,
          std::__1::char_traits<unsigned char> >::read' requested here
            stream.read(reinterpret_cast<unsigned char*>(buf), size);
                   ^
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/istream:267:5: note:
          candidate constructor not viable: requires 1 argument, but 2 were provided
        sentry(const sentry&); // = delete;
        ^
    3 errors generated.
    make[1]: *** [wfs-fuse.o] Error 1
    make: *** [wfs-fuse] Error 2
    
     
    Last edited by ihaveamac, Jul 27, 2017
  20. EyeKey

    OP EyeKey GBAtemp Regular
    Member

    Joined:
    Feb 10, 2017
    Messages:
    232
    Country:
    Israel
    I think that I fixed it, can you try again?
     
Draft saved Draft deleted
Loading...

Hide similar threads Similar threads with keywords - wfsdump, dumping, devices