Building Atmosphère on Windows using MSYS2

Sometimes you need to build the binaries for Atmosphère, or you want to change something on the source code, for building your starting point should be: https://github.com/Atmosphere-NX/Atmosphere/blob/master/docs/building.md
On that document you can read:
Building Atmosphère is a very straightforward process that relies almost exclusively on tools provided by the devkitPro organization.​
Dependencies:
  • devkitA64
  • devkitARM
  • Python 2 (Python 3 may work as well, but this is not guaranteed)
  • LZ4
  • PyCryptodome (optional)
  • Hactool
Setup a DevKitPro environment: (https://devkitpro.org/wiki/Getting_Started)
Install the following packages via (dkp-)pacman:​
  • switch-dev
  • switch-glm
  • switch-libjpeg-turbo
  • devkitARM
  • devkitarm-rules
  • hactool
Install the following library via python's package manager pip, required by exosphere:​
  • lz4
Finally, clone the Atmosphère repository and run make under its root directory.​

That instructions are a little plain for we the noobs.

First you need Install a MSYS environment:
The latest instructions for install the MSYS environment are on https://gbatemp.net/threads/install-msys-environment.652234/ if you don't have one working, use them.

After you have the MSYS environment working you need to install DevKitPro
For that you can follow the instructions on the tutorial https://gbatemp.net/threads/setup-a-devkitpro-environment-on-windows.652238/

Now you can install DevKitPro packages:

You Need base DevKitPro packages:
Bash:
pacman -S --noconfirm switch-dev devkitA64 dkp-toolchain-vars libnx switch-tools switch-mesa switch-libdrm_nouveau switch-sdl2

And the Atmosphère prerequisites
Bash:
Pacman -Sy --noconfirm devkitA64 devkitARM devkitarm-rules hactool mingw-w64-x86_64-python mingw-w64-x86_64-python-lz4 mingw-w64-x86_64-python-pycryptodome switch-dev switch-glm switch-libjpeg-turbo mingw-w64-x86_64-python-pip

I had to add zip package because even if wasn't explicit indicated on the build page, the code needs it.

You also will need the packages git make lz4 for all to work, use the command:

Bash:
Pacman -Sy --noconfirm zip git make lz4


All packages on a combined command:​

Bash:
pacman -S --noconfirm switch-dev devkitA64 dkp-toolchain-vars libnx switch-tools switch-mesa switch-libdrm_nouveau switch-sdl2 devkitA64 devkitARM devkitarm-rules hactool mingw-w64-x86_64-python mingw-w64-x86_64-python-lz4 mingw-w64-x86_64-python-pycryptodome switch-dev switch-glm switch-libjpeg-turbo mingw-w64-x86_64-python-pip zip git make lz4


The package hactool need you to copy your own prod.keys file on $HOME/.switch folder. (c:\msys64\home\USERNAME\.switch if you use the default installation folder)


LibNX Dependency
Atmosphère depends heavily on LibNX, generally when a major version of firmware is released or great changes on the code are made, changes to LibNX are also made.
The Atmosphère team works on their own repo of LibNX (https://github.com/Atmosphere-NX/libnx) and forward Pull Request to official SwitchBrew LibNX repo (https://github.com/switchbrew/libnx).


So ONLY if there are changes on LibNX repository, no yet committed to DevKitPro release, you need to build it and install on your setup.
If you haven't done already, setup the environment variables
You can test it running the command
export
You should have this:
Bash:
export DEVKITPRO=/opt/devkitpro
export DEVKITARM=${DEVKITPRO}/devkitARM
export DEVKITPPC=${DEVKITPRO}/devkitPPC
export PATH=${DEVKITPRO}/tools/bin:$PATH


Then go to the Atmosphere-NX/LibNX repo and look what is the new branch for the current firmware, by example, for Firmware 16.0.0 the branch name is "1600_Support"

1677718224337.png

Then go to your MSYS MinGW64 window and clone the Atmosphere-NX/LibNX repo

Bash:
cd ~
git clone --recursive https://github.com/Atmosphere-NX/libnx.git
cd libnx

And checkout the needed branch:

Bash:
git checkout 1600_support

Then build and install the new LibNX libraries

Bash:
make install -j$(nproc)

Now you can build Atmosphère with latest LibNX code; remember if you update the pacman LibNX package the changes get reversed and you need to build LibNX again.

Now is time to build the binaries
Clone the repository:
Bash:
cd ~
git clone --recursive https://github.com/Atmosphere-NX/Atmosphere

Go to new created Atmosphère folder
Bash:
cd Atmosphere

Be sure you have defined the environment variables for DevKitPro
You can test it running the command
export
You should have this:
Bash:
export DEVKITPRO=/opt/devkitpro
export DEVKITARM=${DEVKITPRO}/devkitARM
export DEVKITPPC=${DEVKITPRO}/devkitPPC
export PATH=${DEVKITPRO}/tools/bin:$PATH


COPY YOUR PROD.KEYS FILE TO THE .SWITCH FOLDER:

Copy prod.keys file on $HOME/.switch
Bash:
mkdir ~/.switch
cp "c:\YOUR\CURRENT\PATH\TO\KEYS\PROD.KEYS" ~/.switch/prod.keys

In the case of MSYS for using localhost paths you must use "" by example in my case I use the commands:
Bash:
mkdir ~/.switch
cp "z:\Emulation\NSwitch\keys\switch\prod.keys" ~/.switch/prod.keys

Obviously you also can copy the file manually on windows to the equivalent path to ~/.switch/ which by default is c:\msys\home\USERNAME\.switch Where c:\msys is the installation folder and USERNAME is the current name for the user.


Atmosphère is a Python 2 project, even when python 2 has been deprecated by almost 5 years now. some * insist on never update it so as far as that is going, you need to change some files in order to work on python 3, on the files:
  • Atmosphere\fusee\program\update_mtc_tables.py
  • Atmosphere\fusee\program\update_sdram_params.py
The function XRANGE must to be changed to RANGE, edit the files and change xrange( to range( save the files and continue.

And now you are ready to start the building process:
Bash:
make -j$(nproc)

I do use the "-j$(nproc)" parameter to "bleed" all the available CPU to run the compiler, without that the whole process could take up to 1 hour.

Also Note:, if you use make -j is possible a Error 2 can be generated, run make again without parameters. The process will continue where failed, since the two latest versions of DevKitPro using the -j parameters is not synced and the make process can fail by timeouts.


I want to thank a lot to @binkinator and @godreborn for all the help and inspiration to make this guide, and want to ask you for help improving this guide.

Also special thanks to @godreborn for been a support of the users replies and suggestions to the OP.

Remember, as you are getting a new set of binaries, you need new set of sigpatches if you use it. So the great tool from @mrdude comes in hand, the latest release can be downloaded from: https://github.com/mrdude2478/IPS_Patch_Creator/releases/; you use the program to generate a new set of patches for your recently build package3 file.


Latest IPS Patch Creator:
https://gbatemp.net/download/ips-patch-creator-1-5-9.38850/




-Edited: add sigpatches step and clean up a little-
-Edit 2, add LibNX build-
 

Attachments

  • 1695782853186.png
    1695782853186.png
    62.7 KB · Views: 217
  • 1695782902542.png
    1695782902542.png
    6.4 KB · Views: 206
  • Sigpatch-IPS-Creator_1.5.7_Yandex.png
    Sigpatch-IPS-Creator_1.5.7_Yandex.png
    3.3 KB · Views: 136
Last edited by impeeza,
lz4 is installed. 🤔
Try pip3 install lz4
Post automatically merged:

you need to see if it's using python2 or python3. pip3 is python3. pip could be python2 or python3, best way to check is python --version. it will tell you the python version. to exit, do exit ()
 
Last edited by godreborn,
  • Love
Reactions: impeeza
Try pip3 install lz4
Post automatically merged:

you need to see if it's using python2 or python3. pip3 is python3. pip could be python2 or python3, best way to check is python --version. it will tell you the python version. to exit, do exit ()
I try it when I am Home
Post automatically merged:

Try pip3 install lz4
Post automatically merged:

you need to see if it's using python2 or python3. pip3 is python3. pip could be python2 or python3, best way to check is python --version. it will tell you the python version. to exit, do exit ()
1674745827168.png

Post automatically merged:

I try it when I am Home
Post automatically merged:


1674745827168.png
Same Issue : (
 
Last edited by TT-ghost_semir,
I try it when I am Home
Post automatically merged:


View attachment 349491
Post automatically merged:


Same Issue : (
what does python --version say?
Post automatically merged:

I just now thought of this, but do make clean, then make again. often times, once an error has been reached when compiling, you have to start over. I think that's what @impeeza meant in one of the previous posts. sometimes you do have to redo git clone (if make clean doesn't clear everything, and you don't know what to delete, but that's not that often where that happens).
 
Last edited by godreborn,
Hi, please note what this guide is for use MinGW, so all command should be run INSIDE a MSYS MinGW command line:
1674828545800.png


Not Windows Command Line (CMD), nor Powershell, nor Windows Terminal

also this guide doesn't cover MinGW outside of MSYS.

as you can install GIT for Windows, on a CMD command line it appears to work, but the prerequisites are very difficult to setup on a Windows Machine natively, that's why I use MSYS as an self-contained environment where all can be easy setup.
 
I think I know what the problem is with his setup, now that it hit me he's using cmd prompt. I've managed to build atmosphere using cmd prompt, so I know it's possible. the problem with his setup is that he needs to add environmental variable to the python scripts folder, which should be on the root of his python folder.
Post automatically merged:

1674846892857.png


it goes in the users section, under path.
 
I think I know what the problem is with his setup, now that it hit me he's using cmd prompt. I've managed to build atmosphere using cmd prompt, so I know it's possible. the problem with his setup is that he needs to add environmental variable to the python scripts folder, which should be on the root of his python folder.
Post automatically merged:

View attachment 349698

it goes in the users section, under path.
exactly, and you get on a mess of environment variables, versioning, different prerrequisites, that's why I do preffer MSYS MinGW.
 
exactly, and you get on a mess of environment variables, versioning, different prerrequisites, that's why I do preffer MSYS MinGW.
I normally use cmd prompt for Nintendo stuff unless something must be installed that windows isn't compatible with or a command is used that isn't on Windows. then, I'll use Linux or sometimes msys. sometimes you have to use something else, because something will not compile for whatever reason.
 
I think I know what the problem is with his setup, now that it hit me he's using cmd prompt. I've managed to build atmosphere using cmd prompt, so I know it's possible. the problem with his setup is that he needs to add environmental variable to the python scripts folder, which should be on the root of his python folder.
Post automatically merged:

View attachment 349698

it goes in the users section, under path.
This is my

1674868539858.png

Post automatically merged:

what does python --version say?
Post automatically merged:

I just now thought of this, but do make clean, then make again. often times, once an error has been reached when compiling, you have to start over. I think that's what @impeeza meant in one of the previous posts. sometimes you do have to redo git clone (if make clean doesn't clear everything, and you don't know what to delete, but that's not that often where that happens).
1674868662695.png
 
Try whereis python in the terminal.
Post automatically merged:

To clarify, it's

whereis python

It will list the locations of all pythons installed. I think k it might not be reading the one from msys, but regular windows.
 
Last edited by godreborn,
@TT-ghost_semir , I'm still debating whether I want to continue with the scenes, but I thought I'd give you this information. here's an image of me successfully compiling atmosphere with msys (I've already compiled it with cmd prompt and linux). I used @impeeza 's tutorial to install dependencies, since they seem to be gone from github. anyway, I think your setup is looking in the c drive for python. type which python, which is better than whereis, to see if it's detecting the msys one. then, which lz4. it will tell you which python and which lz4 is being used with the python cmd, like this:

1674929891129.png

1675031473410.png
 
@TT-ghost_semir , I'm still debating whether I want to continue with the scenes, but I thought I'd give you this information. here's an image of me successfully compiling atmosphere with msys (I've already compiled it with cmd prompt and linux). I used @impeeza 's tutorial to install dependencies, since they seem to be gone from github. anyway, I think your setup is looking in the c drive for python. type which python, which is better than whereis, to see if it's detecting the msys one. then, which lz4. it will tell you which python and which lz4 is being used with the python cmd, like this:

View attachment 350089
View attachment 350094
Thank you 🙏
I try it now.
Sorry for the Wait 🙈
Post automatically merged:

@TT-ghost_semir , I'm still debating whether I want to continue with the scenes, but I thought I'd give you this information. here's an image of me successfully compiling atmosphere with msys (I've already compiled it with cmd prompt and linux). I used @impeeza 's tutorial to install dependencies, since they seem to be gone from github. anyway, I think your setup is looking in the c drive for python. type which python, which is better than whereis, to see if it's detecting the msys one. then, which lz4. it will tell you which python and which lz4 is being used with the python cmd, like this:

View attachment 350089
View attachment 350094
this say msys:
1675262932237.png
 
Last edited by TT-ghost_semir,
  • Like
Reactions: impeeza
Thank you 🙏
I try it now.
Sorry for the Wait 🙈
Post automatically merged:
alright, look in /var/lib/pacman/local . verify that lz4 is listed in there and its folder isn't empty. also, check if it's located in /usr/lib/python3.11/site-packages . verify it's not empty. change the python part to whichever python you're using, which I think you said was python 3.11.
 
alright, look in /var/lib/pacman/local . verify that lz4 is listed in there and its folder isn't empty. also, check if it's located in /usr/lib/python3.11/site-packages . verify it's not empty. change the python part to whichever python you're using, which I think you said was python 3.11.
Now i am get this error
1675354872690.png


1675354895944.png


1675354906124.png

Post automatically merged:

I'm using Python 3.11 (latest).
I was able to fix the LZ4 error, the LZ4 folder was not in /usr/lib/python3.11/site-packages
 
Dunno how I missed this thread (actually, I do… I was deployed and can’t access this site when that happens ), but there’s a whole lot of improvement to be had. I mean, it ends up working (for most… I guess… haven’t read the whole thread, but it’s almost 2am and I have to work in the morning), but some steps are missing (like editing .bashrc to get your path set correctly and those exports declared simply by launching mingw or whatever). Under “Install DEVKITPRO”, both of your pacman -S commands contain duplicates on top of duplicates (things repeated between the first and second, on top of switch-dev in the second, which installs things individually listed in the first).

I should have a good look at this and post my suggested edits in a month or so. My wish is sooner, but the idea is that I’m about to build a new PC from scratch and I’m waiting to get my hands on a 7950x3d to complete it. No telling when that will *really* happen.

or I may get bored and do it this weekend on my current setup.
Post automatically merged:

I seem to have the latest libnx already installed:

View attachment 320432
Why are you trying to do this from cmd.exe? That’s your first problem. You need to start from scratch, and run everything from a mingw shell (installing devkitpro and all…). Or otherwise launch a bash shell if you’re trying to use WSL.
 
Last edited by urherenow,
  • Like
Reactions: impeeza
Dunno how I missed this thread (actually, I do… I was deployed and can’t access this site when that happens ), but there’s a whole lot of improvement to be had. I mean, it ends up working (for most… I guess… haven’t read the whole thread, but it’s almost 2am and I have to work in the morning), but some steps are missing (like editing .bashrc to get your path set correctly and those exports declared simply by launching mingw or whatever). Under “Install DEVKITPRO”, both of your pacman -S commands contain duplicates on top of duplicates (things repeated between the first and second, on top of switch-dev in the second, which installs things individually listed in the first).

I should have a good look at this and post my suggested edits in a month or so. My wish is sooner, but the idea is that I’m about to build a new PC from scratch and I’m waiting to get my hands on a 7950x3d to complete it. No telling when that will *really* happen.

or I may get bored and do it this weekend on my current setup.
Post automatically merged:


Why are you trying to do this from cmd.exe? That’s your first problem. You need to start from scratch, and run everything from a mingw shell (installing devkitpro and all…). Or otherwise launch a bash shell if you’re trying to use WSL.
I can compile it in cmd prompt.
Post automatically merged:

Now i am get this error
View attachment 350696

View attachment 350697

View attachment 350701
Post automatically merged:

I'm using Python 3.11 (latest).
I was able to fix the LZ4 error, the LZ4 folder was not in /usr/lib/python3.11/site-packages
Site packages should be a series of scripts, not the lz.exe.
Post automatically merged:

You could try pip install encodings, but I think you might have a broken python install.
 
Last edited by godreborn,
  • Like
Reactions: impeeza
I can compile it in cmd prompt.
Post automatically merged:


Site packages should be a series of scripts, not the lz.exe.
Post automatically merged:

You could try pip install encodings, but I think you might have a broken python install.
Can does not mean SHOULD. Even Microsoft's own WSL keeps mostly separate from the rest of the system. People may have python installed for Windows, that wasn't meant for msys/WSL. and a partial (depending on how much stuff is in your $PATH, not all necessarily makes it in) Windows path is appended to your path within msys/mingw, causing both conflicts and causing things not to work at all, if they get added to the END of your path (which again, may not be included in its entirety).


msys/python3 behaves like Python on Linux (with some patches) and mingw64/mingw-w64-x86_64-python3 is similar to Python for Windows. It is a little outdated because of amount patches that need to be updated/reworked. Using Python for Windows without MSYS2 specific patches may lead to non-oblivious errors.

That is just 1 example. Whether or not you frankenstined your setup to *seemingly* work, you're doing it WRONG.
 
  • Like
Reactions: impeeza

Site & Scene News

Popular threads in this forum