[Release+Mini Tutorial] AIO.bat: All-In-1 batch file for compiling 3DS stuff

Discussion in '3DS - Flashcards & Custom Firmwares' started by Madridi, May 22, 2016.

  1. Madridi
    OP

    Madridi Card Collector

    Member
    3,163
    976
    May 9, 2008
    Qatar
    Doha
    Hey guys,

    A friend and I were playing with this a few weeks back. It wasn't meant to be publicly released for multiple reasons (mainly, because of the requirements needed for each build). It was just meant to be for quick, private use. Anyway, we thought some people might get a kick out of it so I decided to share it and hopefully it would be beneficial.

    Don't expect anything fancy. It's just supposed to "do the job" in the easiest way possible. It was tested on Windows 7 and Windows 10, 64-bit. No idea if it works on any other OS (It shouldn't I think..)

    Here are a few screenshots of the program:
    Screenshots

    As I said earlier, the reason that it wasn't going to be publicly released was mainly due to the fact that each build has different requirements. As a result, please research the requirements for each repo you are trying to compile. The batch file's purpose is to automate the build process. It won't magically build something if you don't have the necessary requirement installed. However, I'll try to include whatever I can remember in the requirement section below.

    - Requirement:
    • Install "git"
    • Install "Python 2.7": Once you get to the "Customize Python 2.7" screen, make sure you select "add python.exe to patch".
    • Install "Devkitpro": While we only need to install devkitarm, I suggest installing everything.
    • Install "Make"
    • Install "Visual C++ Runtime Libraries x86": Make sure you install x86, not x64
    • Install "ImageMagicK": This version I linked to (6.9.2) is old. Version 6.9.3 had a problem. The current latest is 7.0.1, but I have no idea if that works ok. So better use this link to avoid any problems.
    • Install "Info-Zip"
    • Install "CMake"
    • Install "Python 3.5": Personally, I customized installation, and chose to install for all users, and to add python to environment variables. After installation, go the installed folder and rename "Python.exe" to "Python3.exe"
    • Install "Visual C++ Compilers For Python 2.7": Restart PC after installing.
    • Install "wget Binaries & Dependencies": unzip the wget.exe and dll files from the 2 links to "C:\devkitpro\devkitarm\bin\" folder, assuming C: drive is where you have your devkitpro folder.
    • Install "curl": Unzip curl.exe to "C:\devkitpro\devkitarm\bin\" folder, assuming C: drive is where you have your devkitpro folder.
    • Install "bin2c": Unzip bin2c.exe to "C:\devkitpro\devkitarm\bin\" folder, assuming C: drive is where you have your devkitpro folder.
    • Install "MinGW": After installing, right-click the package "mingw32-base" and choose "mark for installation", open the "installation" tab and choose "apply changes".
    • Install "Cygwin": Make sure you select developer mode in installation. Installation will take A LONG TIME. Make sure the references in PATH (mention in section below) are after MinGW, or you'll have problems
    • Install "PyCrypto": After installing, go to C:/Python27/scripts folder. Hold shift + right-click and open command prompt. Type "easy_install pycrypto" and click enter
    • Install "3dstools": Unzip all 3 files to "C:\devkitpro\devkitarm\bin\" folder, assuming C: drive is where you have your devkitpro folder. Replace the existing ones. This is necessary for windows 7 users, or else FBI building WILL crash!
    • Install "armips" (Attached): Unzip, and place the .exe inside in ""C:\devkitPro\msys\bin" folder, assuming C: drive is where you have your devkitpro folder. If one already exists, replace it.
    • Install "makerom" (Attached): Unzip, and place the .exe inside in ""C:\devkitPro\devkitarm\bin" folder, assuming C: drive is where you have your devkitpro folder. If one already exists, replace it.
    • Install "Picasso" (Attached): Unzip, and place the .exe inside in ""C:\devkitPro\devkitarm\bin" folder, assuming C: drive is where you have your devkitpro folder. If one already exists, replace it.
    • Install "HBKB" (Attached): Unzip, and place the hbkb.h inside in "C:\devkitPro\libctru\include\" folder, and place the libhbkb.a inside in ""C:\devkitPro\libctru\lib\" folder, assuming C: drive is where you have your devkitpro folder.
    • Install files required to "Zip & Unzip" (Attached): Unzip, and place both .exe inside in ""C:\devkitPro\devkitarm\bin" folder, assuming C: drive is where you have your devkitpro folder.
    • Install "PyYAML-3.11-cp35-cp35m-win32": In the same folder you have this file. right-click, open cmd, and type "pip install PyYAML-3.11-cp35-cp35m-win32.whl". If pip is out of date, it will give you instruction on how to update it, then run the above command again to install it.

    - Creating MKLink:

    Create MKLink

    - PATH config:

    After installing all of the above, you need to make sure sure that all of the above is included in PATH. To do this, open cmd, and type "PATH"

    You will probably see a lot of stuff. What you need to make sure is that the following are there:
    PATH Check

    IMPORTANT: DO NOT DELETE ANYTHING FROM PATH! Just add whatever is missing..

    The above assumes the correct location of each entry. If you have them somewhere else, you need to modify the above.

    To modify PATH, do the following:
    PATH Modify

    - Update Libs:
    • Download the latest release of AIO, unzip and fetch the AIO.bat
    • Create a folder where you want to compile repos.
    • Run the AIO.bat, and press "D" to go to Libs menu
    IMPORTANT: Make sure the folder and it's entire path does not have a space. You might run into problems otherwise.
    • Install Libctru (Smea's fork)
    • Install Citrus
    • Install Citro3d
    • Install Portlibs

    - Final Notes & Credits:
    • I may have missed some stuff in the requirements, but this is the majority of them. If a specific source is not compiling for you. Please check it's respective repository for what's required before posting here.
    • A9LH forks that require input files such as OTP, are not fully automated. It will pause and ask you to supply these files before continuing.
    • Sometimes, FBI will not build with the setup above, as it sometimes requires Steveice10's libctru. That branch of libctru is included in AIO, but you can either have Smea's branch or Steveice10's branch. Since his branch is only used for FBI, we chose to go with Smea's branch. At the moment, Steveice10's libctru is private as FBI builds fine with Smea's branch.
    • You might need to update libctru regularly. More likely than not, when something fails to build, it's because of an outdated lib - which in most cases is libctru. However, do not update if things are working fine. In some cases, the update breaks some other projects, until that projects updates the usage of latest libctru as well.
    • I am not a programmer. This was just something I was playing with. So if you feel there is something that could be done in a better way, feel free to modify it.
    • As usual, USE THIS AT YOUR OWN RISK!!
    Thanks goes to the following people:
     

    Attached Files:

    Last edited by Madridi, Jul 31, 2016
    fr3quency, Joel16, pandavova and 17 others like this.
  2. astronautlevel

    astronautlevel Finding a reason, waiting for a miracle

    Member
    4,046
    5,095
    Jan 26, 2016
    United States
    That Nightly Site™
    If you have python 2 and 3 installed on a windows computer, you can choose which one to run by using py -2 for python 2.7 and py -3 for python 3.5. That being said I have no idea how this builds cakes because I haven't looked at the source yet so it may or may not be an option.
     
  3. Madridi
    OP

    Madridi Card Collector

    Member
    3,163
    976
    May 9, 2008
    Qatar
    Doha
    Could you elaborate on that? Where would we do that choice?
     
  4. klear

    klear Member

    Newcomer
    28
    4
    Mar 26, 2015
    Italy
  5. DjoeN

    DjoeN Captain Haddock!

    Member
    5,207
    1,503
    Oct 21, 2005
    Belgium
    Somewhere in this potatoland!
    few extra notes (correct me if i'm wrong):

    - For the moment FBI can be build with latest ctrulib from smealum
    - Citrus will not build with latest ctrulib (you need an older version)
    - If you have Python 2.7 installed, just install python 3.5, it's no problem to have them side by side.

    - To compile CakesFW (CakesForEveryWan) on a Windows computer you need to edit the makefile (cause it specific needs python 3 to build):

    - PYTHON := python3
    change this to
    - PYTHON := py -3
    (I don't know how to configure python or whatever to use the original line like used in the makefile for cakes)

    You also need PyYaml to build CakesFW, if you use Python 3.5, you need to install the unofficial PyYamml Wheel: PyYAML-3.11-cp35-cp35m-win32.whl
    (You need latest pip 8 to install the wheel, if you have an older version pip will ask you for an update)

    [EDIT 1]
    If you have 7z errors, those can be ignored or you could download 7z commandline
    (InfoZip didn't solve the 7z error)

    - http://www.7-zip.org/download.html -> Download .7z x86 / x64 7-Zip Extra: standalone console version, 7z DLL, Plugin for Far Manager
    - Extract the *.dll and *.exe files into devkitpro\devkitarm\bin\ (since that's already in your path) (If you use Windows x64 go into the x64 folder and use those files)
    - Rename 7za.exe to 7z.exe
    - This should solve your 7z errors at the end of a build with the batch script and make ready to distribute *.7z of the build

    [EDIT 2]
    - Seems SaltFW is updated to be compatible with latest ctrulib :)

    [EDIT 3]
    - To build CIAngel you need HBKB lib
    - You also need to change a few things before you can build HBKB:
    After this go to \HBKB\Library\hbkb\ and run make from a commandline window
    When building HBKB is done, you need to manually move the files to ctrulib:
    - \HBKB\Library\hbkb_include_header\hbkb.h to c:\devkitPro\libctru\include\
    - \HBKB\Library\hbkb\lib\libhbkb.a to c:\devkitPro\libctru\lib\
     
    Last edited by DjoeN, May 22, 2016
  6. Rohul1997

    Rohul1997 GBAtemp Fan

    Member
    349
    351
    Dec 7, 2015
    Giant Chasm
    Thanks for the info
    So doing this 100% works?
     
  7. DjoeN

    DjoeN Captain Haddock!

    Member
    5,207
    1,503
    Oct 21, 2005
    Belgium
    Somewhere in this potatoland!
    It works for me :)
    I have Python 2.7.x installed and also Python 3.5.x

    Using the unmodified makefile results in a python 3 error, changing that line works
     
    Rohul1997 likes this.
  8. Rohul1997

    Rohul1997 GBAtemp Fan

    Member
    349
    351
    Dec 7, 2015
    Giant Chasm
    OK cool
    I'll try this out later and I have an idea in mind
     
  9. Madridi
    OP

    Madridi Card Collector

    Member
    3,163
    976
    May 9, 2008
    Qatar
    Doha
    Thanks, I was able to get them both installed, and got cakes compiled. Now I need to test other sources to check that nothing broke.

    Glad to hear that this is the current situation. Sometimes @Steveice10 ends up using his fork though until it's merged with smea's branch. So it's a good idea to have it in the bat if users want to have it. It's also why Smea's is the preferred set up
    Yeah I saw your comment on that. I assume @Steveice10 will fix that soon. I'll add it to the notes though for now.
    That worked, and I don't think it messed up anything. I'll do some tests and if all goes well, I'll see about including this in the main post. Thanks.
    @mid-kid : Would it break anything from your side if you change python3 to py -3? It certainly would make building on windows much easier.

    Will add this to the instructions.
    Yeah I forgot about that one, and it's not something I could include in the bat because of the manual changes. I'll see about adding this info in the main post. Thanks for the help :)
     
  10. astronautlevel

    astronautlevel Finding a reason, waiting for a miracle

    Member
    4,046
    5,095
    Jan 26, 2016
    United States
    That Nightly Site™
    AFAIK it would break linux support.
     
    Madridi likes this.
  11. Madridi
    OP

    Madridi Card Collector

    Member
    3,163
    976
    May 9, 2008
    Qatar
    Doha
    Damn.. Well, I hope a solution can be found for this..
     
  12. Madridi
    OP

    Madridi Card Collector

    Member
    3,163
    976
    May 9, 2008
    Qatar
    Doha
    Alright, I just updated the main post and the batch file with several points:
    - Added a workaround for cakes. Requires manual intervention though until its fixes..
    - Changed cakes command from make to make release
    - Added HBKB to the attachements
    - Added Zip and unzip exe files for sources that zip up their releases
    - Added a few things under notes
     
  13. WhoAmI?

    WhoAmI? PASTA's dirty animal

    Member
    1,273
    1,009
    Mar 15, 2015
    Poké Ball
    If you like my original script, you should've seen the one me and @nastys were working on - it installs ALL the required dependencies to compile pretty much any CFW, for Windows and Linux (Nastys made a GUI, too).

    Unfortunately, at the time of making it (~3-4 months ago), rxTools was being a bitch about compiling on Windows, so we aborted the project as we were unsure of what the reasons were (these were later fixed AFAIK).

    If I and Nastys find the source (it's somewhere..), I'll see if it can be completed and uploaded to GitHub, or I'll somehow merge it with your script.
     
    Rohul1997, DjoeN and Madridi like this.
  14. Madridi
    OP

    Madridi Card Collector

    Member
    3,163
    976
    May 9, 2008
    Qatar
    Doha
    Holy shit dude that's amazing! Can't wait to see what you come up with :D
     
    WhoAmI? likes this.
  15. Madridi
    OP

    Madridi Card Collector

    Member
    3,163
    976
    May 9, 2008
    Qatar
    Doha
    Updated once again (more like, a revert..)

    Cakes no longer needs manual intervention. All you need to do is rename python.exe inside the python 3.5 folder to python3.exe, and cakes will compile just fine
     
    Rohul1997 and DjoeN like this.
  16. DjoeN

    DjoeN Captain Haddock!

    Member
    5,207
    1,503
    Oct 21, 2005
    Belgium
    Somewhere in this potatoland!
    Ok, here's another one to add @Madridi
    - JK Save Manager ;)

    Building JK Save Manager
    - JKSM

    External Libs/Apps required:
    - sf2dlib
    - Portlibs Zlib (needed for freetype and linpng)
    - Portlibs FreeType2
    - Portlibs libpng
    - sftdlib
    - MakeRom

    HowTo:
    Important!!!
    Follow the order to build!
    you always need ctrulib to be installed

    - Place MakeRom in your "\devkitPro\devkitARM\bin\" folder
    - Build and install sf2dlib (you only need ctrulib to build this)
    - Build and install Portlibs Zlib (Needed to build Portlibs FreeType2 and Portlibs LibPNG)
    - Build and install Portlibs FreeType2 (Needs Portlibs Zlib to be installed)
    - Build and install Portlibs LibPNG (Needs Portlibs Zlib to be installed)
    - Build and install sftdlib (Needs sf2dlib and FreeType2 to be installed)
    - Build JKSM with "make cia" (Needs sf2dlib/FreeType2/libpng/sftdlib to be installed)

    You now should have JKSM.3dsx / JKSM.smdh / JKSM.cia (and other files) in your JKSM folder

    [EDIT]
    Sorry, did all manually, so no batch script used yet.
    Will try to write it later after irl/family stuff (food/household stuff etc...)
     
    Last edited by DjoeN, May 22, 2016
    Madridi and Rohul1997 like this.
  17. Madridi
    OP

    Madridi Card Collector

    Member
    3,163
    976
    May 9, 2008
    Qatar
    Doha
    Oh cool he started using his github? At the time I looked at it, he didnt. It just had old code + CIA, I'll add this later tonight (and add libs as well)
     
    Rohul1997 likes this.
  18. DjoeN

    DjoeN Captain Haddock!

    Member
    5,207
    1,503
    Oct 21, 2005
    Belgium
    Somewhere in this potatoland!
    @Madridi
    Check PM for batch scripts to build 3ds-portlibs (zlib/freetype/libpng) / sf2dlib and sfdlib
     
  19. Madridi
    OP

    Madridi Card Collector

    Member
    3,163
    976
    May 9, 2008
    Qatar
    Doha
    Thanks, I started adding them already, but I'll have to modify each one to fit the choice menu. It's kinda annoying lol
     
  20. DjoeN

    DjoeN Captain Haddock!

    Member
    5,207
    1,503
    Oct 21, 2005
    Belgium
    Somewhere in this potatoland!
    well, i did some last edits :P
     
    Madridi likes this.