Building Atmosphère on Windows using MSYS2

Sometimes you need to build the binaries for Atmosphère, or you want to change something on the source code, for building your starting point should be: https://github.com/Atmosphere-NX/Atmosphere/blob/master/docs/building.md
On that document you can read:
Building Atmosphère is a very straightforward process that relies almost exclusively on tools provided by the devkitPro organization.​
Dependencies:
  • devkitA64
  • devkitARM
  • Python 2 (Python 3 may work as well, but this is not guaranteed)
  • LZ4
  • PyCryptodome (optional)
  • Hactool
Setup a DevKitPro environment: (https://devkitpro.org/wiki/Getting_Started)
Install the following packages via (dkp-)pacman:​
  • switch-dev
  • switch-glm
  • switch-libjpeg-turbo
  • devkitARM
  • devkitarm-rules
  • hactool
Install the following library via python's package manager pip, required by exosphere:​
  • lz4
Finally, clone the Atmosphère repository and run make under its root directory.​

That instructions are a little plain for we the noobs.

First you need Install a MSYS environment:
The latest instructions for install the MSYS environment are on https://gbatemp.net/threads/install-msys-environment.652234/ if you don't have one working, use them.

After you have the MSYS environment working you need to install DevKitPro
For that you can follow the instructions on the tutorial https://gbatemp.net/threads/setup-a-devkitpro-environment-on-windows.652238/

Now you can install DevKitPro packages:
Bash:
pacman -S --noconfirm switch-dev devkitA64 dkp-toolchain-vars libnx switch-tools switch-mesa switch-libdrm_nouveau switch-sdl2

And Atmosphère prerequisites
Bash:
Pacman -Sy --noconfirm devkitA64 devkitARM devkitarm-rules hactool mingw-w64-x86_64-python mingw-w64-x86_64-python-lz4 mingw-w64-x86_64-python-pycryptodome switch-dev switch-glm switch-libjpeg-turbo mingw-w64-x86_64-python-pip

For me I had to add zip package because even if wasn't explicit indicated on the build page, the code needs it.

Additional you need the packages git make lz4 for all to work, use the command:

Bash:
Pacman -Sy --noconfirm zip git make lz4


The package hactool need you to copy your own prod.keys file on $HOME/.switch folder. (c:\msys64\home\USERNAME\.switch if you use the default installation folder)

LibNX Dependency
Atmosphère depends heavily on LibNX, generally when a major version of firmware is released or great changes on the code are made, changes to LibNX are also made.
The Atmosphère team works on their own repo of LibNX (https://github.com/Atmosphere-NX/libnx) and forward Pull Request to official SwitchBrew LibNX repo (https://github.com/switchbrew/libnx).

So if there are changes on LibNX you need to build it and install on your setup.
If you haven't done already, setup the environment variables

Bash:
export DEVKITPRO=/opt/devkitpro
export DEVKITARM=${DEVKITPRO}/devkitARM
export DEVKITPPC=${DEVKITPRO}/devkitPPC
export PATH=${DEVKITPRO}/tools/bin:$PATH
Then go to the Atmosphere-NX/LibNX repo and look what is the new branch for the current firmware, by example, for Firmware 16.0.0 the branch name is "1600_Support"

1677718224337.png

Then go to your MSYS MinGW64 window and clone the Atmosphere-NX/LibNX repo

Bash:
cd ~
git clone --recursive https://github.com/Atmosphere-NX/libnx.git
cd libnx

And checkout the needed branch:

Bash:
git checkout 1600_support

Then build and install the new LibNX libraries

Bash:
make install -j$(nproc)

Now you can build Atmosphère with latest LibNX code; remember if you update the pacman LibNX package the changes get reversed and you need to build LibNX again.

Now is time to build the binaries
Clone the repository:
Bash:
cd ~
git clone --recursive https://github.com/Atmosphere-NX/Atmosphere

Go to new created Atmosphère folder
Bash:
cd Atmosphere

Never forget to create the environment variables for DevKitPro
Bash:
export DEVKITPRO=/opt/devkitpro
export DEVKITARM=/opt/devkitpro/devkitARM
export DEVKITPPC=/opt/devkitpro/devkitPPC

And now you are ready to start the building process:
Bash:
make -j$(nproc)

I do use the "-j$(nproc)" parameter to "bleed" all the available CPU to run the compiler, without that the whole process could take up to 1 hour.

I want to thank a lot to @binkinator and @godreborn for all the help and inspiration to make this guide, and want to ask you for help improving this guide.

Also special thanks to @godreborn for been a support of the users replies and suggestions to the OP.

Remember, as you are getting a new set of binaries, you need new set of sigpatches if you use it. So the great tool from @mrdude comes in hand, the latest release can be downloaded from: https://github.com/mrdude2478/IPS_Patch_Creator/releases/; you use the program to generate a new set of patches for your recently build package3 file.


Latest IPS Patch Creator:
Sigpatch-IPS-Creator_1.5.7_Yandex.png





-Edited: add sigpatches step and clean up a little-
-Edit 2, add LibNX build-
 

Attachments

  • 1695782853186.png
    1695782853186.png
    62.7 KB · Views: 46
  • 1695782902542.png
    1695782902542.png
    6.4 KB · Views: 48
Last edited by impeeza,

godreborn

Welcome to the Machine
Member
Joined
Oct 10, 2009
Messages
38,471
Trophies
3
XP
29,116
Country
United States
lz4 is installed. 🤔
Try pip3 install lz4
Post automatically merged:

you need to see if it's using python2 or python3. pip3 is python3. pip could be python2 or python3, best way to check is python --version. it will tell you the python version. to exit, do exit ()
 
Last edited by godreborn,
  • Love
Reactions: impeeza

TT-ghost_semir

Member
Newcomer
Joined
Aug 17, 2022
Messages
15
Trophies
0
Location
Gronau (Westfalen)
XP
130
Country
Germany
Try pip3 install lz4
Post automatically merged:

you need to see if it's using python2 or python3. pip3 is python3. pip could be python2 or python3, best way to check is python --version. it will tell you the python version. to exit, do exit ()
I try it when I am Home
Post automatically merged:

Try pip3 install lz4
Post automatically merged:

you need to see if it's using python2 or python3. pip3 is python3. pip could be python2 or python3, best way to check is python --version. it will tell you the python version. to exit, do exit ()
1674745827168.png

Post automatically merged:

I try it when I am Home
Post automatically merged:


1674745827168.png
Same Issue : (
 
Last edited by TT-ghost_semir,

godreborn

Welcome to the Machine
Member
Joined
Oct 10, 2009
Messages
38,471
Trophies
3
XP
29,116
Country
United States
I try it when I am Home
Post automatically merged:


View attachment 349491
Post automatically merged:


Same Issue : (
what does python --version say?
Post automatically merged:

I just now thought of this, but do make clean, then make again. often times, once an error has been reached when compiling, you have to start over. I think that's what @impeeza meant in one of the previous posts. sometimes you do have to redo git clone (if make clean doesn't clear everything, and you don't know what to delete, but that's not that often where that happens).
 
Last edited by godreborn,

impeeza

¡Kabito!
OP
Member
Joined
Apr 5, 2011
Messages
6,312
Trophies
3
Age
46
Location
At my chair.
XP
18,551
Country
Colombia
Hi, please note what this guide is for use MinGW, so all command should be run INSIDE a MSYS MinGW command line:
1674828545800.png


Not Windows Command Line (CMD), nor Powershell, nor Windows Terminal

also this guide doesn't cover MinGW outside of MSYS.

as you can install GIT for Windows, on a CMD command line it appears to work, but the prerequisites are very difficult to setup on a Windows Machine natively, that's why I use MSYS as an self-contained environment where all can be easy setup.
 

godreborn

Welcome to the Machine
Member
Joined
Oct 10, 2009
Messages
38,471
Trophies
3
XP
29,116
Country
United States
I think I know what the problem is with his setup, now that it hit me he's using cmd prompt. I've managed to build atmosphere using cmd prompt, so I know it's possible. the problem with his setup is that he needs to add environmental variable to the python scripts folder, which should be on the root of his python folder.
Post automatically merged:

1674846892857.png


it goes in the users section, under path.
 

impeeza

¡Kabito!
OP
Member
Joined
Apr 5, 2011
Messages
6,312
Trophies
3
Age
46
Location
At my chair.
XP
18,551
Country
Colombia
I think I know what the problem is with his setup, now that it hit me he's using cmd prompt. I've managed to build atmosphere using cmd prompt, so I know it's possible. the problem with his setup is that he needs to add environmental variable to the python scripts folder, which should be on the root of his python folder.
Post automatically merged:

View attachment 349698

it goes in the users section, under path.
exactly, and you get on a mess of environment variables, versioning, different prerrequisites, that's why I do preffer MSYS MinGW.
 

godreborn

Welcome to the Machine
Member
Joined
Oct 10, 2009
Messages
38,471
Trophies
3
XP
29,116
Country
United States
exactly, and you get on a mess of environment variables, versioning, different prerrequisites, that's why I do preffer MSYS MinGW.
I normally use cmd prompt for Nintendo stuff unless something must be installed that windows isn't compatible with or a command is used that isn't on Windows. then, I'll use Linux or sometimes msys. sometimes you have to use something else, because something will not compile for whatever reason.
 

TT-ghost_semir

Member
Newcomer
Joined
Aug 17, 2022
Messages
15
Trophies
0
Location
Gronau (Westfalen)
XP
130
Country
Germany
I think I know what the problem is with his setup, now that it hit me he's using cmd prompt. I've managed to build atmosphere using cmd prompt, so I know it's possible. the problem with his setup is that he needs to add environmental variable to the python scripts folder, which should be on the root of his python folder.
Post automatically merged:

View attachment 349698

it goes in the users section, under path.
This is my

1674868539858.png

Post automatically merged:

what does python --version say?
Post automatically merged:

I just now thought of this, but do make clean, then make again. often times, once an error has been reached when compiling, you have to start over. I think that's what @impeeza meant in one of the previous posts. sometimes you do have to redo git clone (if make clean doesn't clear everything, and you don't know what to delete, but that's not that often where that happens).
1674868662695.png
 

godreborn

Welcome to the Machine
Member
Joined
Oct 10, 2009
Messages
38,471
Trophies
3
XP
29,116
Country
United States
Try whereis python in the terminal.
Post automatically merged:

To clarify, it's

whereis python

It will list the locations of all pythons installed. I think k it might not be reading the one from msys, but regular windows.
 
Last edited by godreborn,

godreborn

Welcome to the Machine
Member
Joined
Oct 10, 2009
Messages
38,471
Trophies
3
XP
29,116
Country
United States
@TT-ghost_semir , I'm still debating whether I want to continue with the scenes, but I thought I'd give you this information. here's an image of me successfully compiling atmosphere with msys (I've already compiled it with cmd prompt and linux). I used @impeeza 's tutorial to install dependencies, since they seem to be gone from github. anyway, I think your setup is looking in the c drive for python. type which python, which is better than whereis, to see if it's detecting the msys one. then, which lz4. it will tell you which python and which lz4 is being used with the python cmd, like this:

1674929891129.png

1675031473410.png
 

TT-ghost_semir

Member
Newcomer
Joined
Aug 17, 2022
Messages
15
Trophies
0
Location
Gronau (Westfalen)
XP
130
Country
Germany
@TT-ghost_semir , I'm still debating whether I want to continue with the scenes, but I thought I'd give you this information. here's an image of me successfully compiling atmosphere with msys (I've already compiled it with cmd prompt and linux). I used @impeeza 's tutorial to install dependencies, since they seem to be gone from github. anyway, I think your setup is looking in the c drive for python. type which python, which is better than whereis, to see if it's detecting the msys one. then, which lz4. it will tell you which python and which lz4 is being used with the python cmd, like this:

View attachment 350089
View attachment 350094
Thank you 🙏
I try it now.
Sorry for the Wait 🙈
Post automatically merged:

@TT-ghost_semir , I'm still debating whether I want to continue with the scenes, but I thought I'd give you this information. here's an image of me successfully compiling atmosphere with msys (I've already compiled it with cmd prompt and linux). I used @impeeza 's tutorial to install dependencies, since they seem to be gone from github. anyway, I think your setup is looking in the c drive for python. type which python, which is better than whereis, to see if it's detecting the msys one. then, which lz4. it will tell you which python and which lz4 is being used with the python cmd, like this:

View attachment 350089
View attachment 350094
this say msys:
1675262932237.png
 
Last edited by TT-ghost_semir,
  • Like
Reactions: impeeza

godreborn

Welcome to the Machine
Member
Joined
Oct 10, 2009
Messages
38,471
Trophies
3
XP
29,116
Country
United States
Thank you 🙏
I try it now.
Sorry for the Wait 🙈
Post automatically merged:
alright, look in /var/lib/pacman/local . verify that lz4 is listed in there and its folder isn't empty. also, check if it's located in /usr/lib/python3.11/site-packages . verify it's not empty. change the python part to whichever python you're using, which I think you said was python 3.11.
 

TT-ghost_semir

Member
Newcomer
Joined
Aug 17, 2022
Messages
15
Trophies
0
Location
Gronau (Westfalen)
XP
130
Country
Germany
alright, look in /var/lib/pacman/local . verify that lz4 is listed in there and its folder isn't empty. also, check if it's located in /usr/lib/python3.11/site-packages . verify it's not empty. change the python part to whichever python you're using, which I think you said was python 3.11.
Now i am get this error
1675354872690.png


1675354895944.png


1675354906124.png

Post automatically merged:

I'm using Python 3.11 (latest).
I was able to fix the LZ4 error, the LZ4 folder was not in /usr/lib/python3.11/site-packages
 

urherenow

Well-Known Member
Member
Joined
Mar 8, 2009
Messages
4,760
Trophies
2
Age
48
Location
Japan
XP
3,656
Country
United States
Dunno how I missed this thread (actually, I do… I was deployed and can’t access this site when that happens ), but there’s a whole lot of improvement to be had. I mean, it ends up working (for most… I guess… haven’t read the whole thread, but it’s almost 2am and I have to work in the morning), but some steps are missing (like editing .bashrc to get your path set correctly and those exports declared simply by launching mingw or whatever). Under “Install DEVKITPRO”, both of your pacman -S commands contain duplicates on top of duplicates (things repeated between the first and second, on top of switch-dev in the second, which installs things individually listed in the first).

I should have a good look at this and post my suggested edits in a month or so. My wish is sooner, but the idea is that I’m about to build a new PC from scratch and I’m waiting to get my hands on a 7950x3d to complete it. No telling when that will *really* happen.

or I may get bored and do it this weekend on my current setup.
Post automatically merged:

I seem to have the latest libnx already installed:

View attachment 320432
Why are you trying to do this from cmd.exe? That’s your first problem. You need to start from scratch, and run everything from a mingw shell (installing devkitpro and all…). Or otherwise launch a bash shell if you’re trying to use WSL.
 
Last edited by urherenow,
  • Like
Reactions: impeeza

godreborn

Welcome to the Machine
Member
Joined
Oct 10, 2009
Messages
38,471
Trophies
3
XP
29,116
Country
United States
Dunno how I missed this thread (actually, I do… I was deployed and can’t access this site when that happens ), but there’s a whole lot of improvement to be had. I mean, it ends up working (for most… I guess… haven’t read the whole thread, but it’s almost 2am and I have to work in the morning), but some steps are missing (like editing .bashrc to get your path set correctly and those exports declared simply by launching mingw or whatever). Under “Install DEVKITPRO”, both of your pacman -S commands contain duplicates on top of duplicates (things repeated between the first and second, on top of switch-dev in the second, which installs things individually listed in the first).

I should have a good look at this and post my suggested edits in a month or so. My wish is sooner, but the idea is that I’m about to build a new PC from scratch and I’m waiting to get my hands on a 7950x3d to complete it. No telling when that will *really* happen.

or I may get bored and do it this weekend on my current setup.
Post automatically merged:


Why are you trying to do this from cmd.exe? That’s your first problem. You need to start from scratch, and run everything from a mingw shell (installing devkitpro and all…). Or otherwise launch a bash shell if you’re trying to use WSL.
I can compile it in cmd prompt.
Post automatically merged:

Now i am get this error
View attachment 350696

View attachment 350697

View attachment 350701
Post automatically merged:

I'm using Python 3.11 (latest).
I was able to fix the LZ4 error, the LZ4 folder was not in /usr/lib/python3.11/site-packages
Site packages should be a series of scripts, not the lz.exe.
Post automatically merged:

You could try pip install encodings, but I think you might have a broken python install.
 
Last edited by godreborn,
  • Like
Reactions: impeeza

urherenow

Well-Known Member
Member
Joined
Mar 8, 2009
Messages
4,760
Trophies
2
Age
48
Location
Japan
XP
3,656
Country
United States
I can compile it in cmd prompt.
Post automatically merged:


Site packages should be a series of scripts, not the lz.exe.
Post automatically merged:

You could try pip install encodings, but I think you might have a broken python install.
Can does not mean SHOULD. Even Microsoft's own WSL keeps mostly separate from the rest of the system. People may have python installed for Windows, that wasn't meant for msys/WSL. and a partial (depending on how much stuff is in your $PATH, not all necessarily makes it in) Windows path is appended to your path within msys/mingw, causing both conflicts and causing things not to work at all, if they get added to the END of your path (which again, may not be included in its entirety).


msys/python3 behaves like Python on Linux (with some patches) and mingw64/mingw-w64-x86_64-python3 is similar to Python for Windows. It is a little outdated because of amount patches that need to be updated/reworked. Using Python for Windows without MSYS2 specific patches may lead to non-oblivious errors.

That is just 1 example. Whether or not you frankenstined your setup to *seemingly* work, you're doing it WRONG.
 
  • Like
Reactions: impeeza

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
    Psionic Roshambo @ Psionic Roshambo: The Real Jdbye I am going AMD this gen still debating 7900XT or XTX