building devkitARM: calico

  • Thread starter Thread starter mrparrot2
  • Start date Start date
  • Views Views 3,505
  • Replies Replies 27
  • Likes Likes 3

mrparrot2

Well-Known Member
Member
Joined
Nov 29, 2021
Messages
128
Reaction score
158
Trophies
1
Age
31
Location
SP, Brazil
XP
748
Country
Brazil
I am writing this post as a way to express my frustration about building the recent version of devkitARM from sources. The buildscripts provided here are not enough for you to get it up and running, and you need to find out some missing pieces: general-tools, ndstool, libgba, libnds and libfat. Once you got the barebones of devkitARM built, all those libraries can be built and installed without issues if you are used with CMake or GNU Autotools.

Now enters calico. Calico is the devkitpro shinny new OS-like library for the Nintendo DS(i) which seems to be a new glue for the various chips in the console, plus now having support for threads. One would expect it to be built like any other library so far, but no. This thing relies on Catnip and this is where all problems starts. If you read its README.md file you will notice there is no instructions about how to install it. Nor instructions about any of its dependencies. So I opened an issue about it mentioning the missing files catnip is referencing just for it to be immediatelly closed with a reference for the buildscripts page, which doesn't help at all.

So I started looking at the packman repositories, which always returns a "File Not Found" error, until I eventually found this page, which contains a history of devkitpro packages. There I installed (i.e. extracted to /opt) the following packages:
  • devkitarm-cmake-1.2.2-1-any.pkg.tar.zst
  • dkp-cmake-common-utils-1.5.2-1-any.pkg.tar.zst
  • gba-cmake-1.0.1-1-any.pkg.tar.zst
  • nds-cmake-1.3.0-6-any.pkg.tar.zst
  • nds-pkg-config-0.28-4-any.pkg.tar.xz
Then I was able to get catnip to work. Those packages do not contain binary files, just CMake and bash scripts, so one can check it how it works. Then I was able to get calico to build with:

$ catnip -S $PWD build

and install it with
$ catnip -S $PWD install

Now here is the catch: how I am supposed to know this? Catnip page should be clear what is necessary for it to work.
 
DevKitPro as an organization has not historically given value to neither data and tool archival nor supporting what they perceive as legacy code and libraries in favor of shipping up-to-date software and enforcing what they perceive as modern paradigms. It's great that you're archiving all this, but it's a bit antithetical to the goals of the designers of the software from what I understand. The buildscripts wiki informs you that the nds-dev target exists and will install all those for you as binary installations. If you're interested in a more conventional toolchain for the NDS target I suggest BlocksDS -- it doesn't move quite as fast while breaking things.
 
DevKitPro as an organization has not historically given value to neither data and tool archival nor supporting what they perceive as legacy code and libraries in favor of shipping up-to-date software and enforcing what they perceive as modern paradigms.
Having cutting-edge version of something breaking compatibility with old software is usual, if not expected. For example, libLLVM doesn't even try to keep compatibility with software linked with older versions of it. It is your responsibility to ensure that your software links fine with libLLVM, say, 16, 17, 18 or 19. If you are stuck with an old version, it is your responsibility to
backport the fixes that you need, and it is just how things goes. They just don't want the trouble, and I don't see a problem in that.

It's great that you're archiving all this, but it's a bit antithetical to the goals of the designers of the software from what I understand.

What is anthithetical to their goals? Be able to build it from sources and not use the binary they are shipping? Sorry, but that seems a very suspicious behavior for me, and very anthithetical to the free software way of doing things. Plus, what if I want to have the toolchain built for a PowerPC host?
 
  • Like
Reactions: Moseii
Actually, not. I am not trying to Frankenstein it with a bunch of old versions combined with new versions of the libraries. I am cloning things from their own git repo.
devkitpro as an organisation wants users to use the prebuilt binaries of the latest versions of their toolchain, so they are not interested in making it easy to build it yourself
 
  • Like
Reactions: Moseii
I've seen DevKitPro staff say on GitHub issues that other toolchains in the space are "illegal forks" (their words). The issue in question was removed when I went to find it to quote it. This begs the question of if they understand or respect FOSS ideals. This is also scary to see. From what I understand the company and branding are a vehicle to support the organization owners for the purpose of income.

I would seek another software organization to invest in. Your troubles with their staff are a symptom.
 
Last edited by linux86,
  • Like
Reactions: Moseii
I've seen DevKitPro staff say on GitHub issues that other toolchains in the space are "illegal forks" (their words). The issue in question was removed when I went to find it to quote it. This begs the question of if they understand or respect FOSS ideals. This is also scary to see. From what I understand the company and branding are a vehicle to support the organization owners for the purpose of income.

I would seek another software organization to invest in. Your troubles with their staff are a symptom.
Most libraries seems to be licensed under GPL or similar. If they keep with this sort of behavior it will eventually backfire and a fork will take place.
 
  • Like
Reactions: Moseii and linux86
Most libraries seems to be licensed under GPL or similar. If they keep with this sort of behavior it will eventually backfire and a fork will take place.

Yes, that's why I created BlocksDS. It's a fork of all DS-related libraries that had an open source license in devkitARM at the start of 2023. There were things I couldn't fork, like NitroFS, because there is no license, and there is no way to reach the original author. Also, the devkitPro maintainers didn't seem to be worried about that. https://github.com/devkitPro/libfilesystem/issues/3

As opposed to the devkitPro maintainers, I've documented 100% of the release process here https://blocksds.skylyrac.net/docs/design/release_process/ and all my binaries are built in a GitHub pipeline https://github.com/blocksds/packages/blob/master/.github/workflows/build.yml so the process is as transparent as possible. Even the documentation and docker images are built in GitHub actions. The only things I don't build myself are the compiler, libc, libstdc++... You can get all information here. https://github.com/WonderfulToolchain/wonderful-packages/tree/main

In fact, my repository of packages has a bunch of libraries that other people created and can be useful, particularly to build old projects, like uLibrary, PAlib, libwoopsi, Nitro Engine, NFlib... As opposed to the devkitPro philosophy of just using what they offer and nothing else.

And well, they can say forks of their libraries are illegal as much as they want (is that really the word they used?). The mere fact that they use that word means they don't understand open source licenses. EDIT: It has been clarified they didn't call the forks illegal.

They also have the tendency to delete GitHub issues that put them in uncomfortable situations regarding their behaviour. https://web.archive.org/web/20241005010626/https://github.com/devkitPro/ndstool/issues/14
 
Last edited by AntonioND,
Yes, that's why I created BlocksDS. It's a fork of all DS-related libraries that had an open source license in devkitARM at the start of 2023. There were things I couldn't fork, like NitroFS, because there is no license, and there is no way to reach the original author. Also, the devkitPro maintainers didn't seem to be worried about that. https://github.com/devkitPro/libfilesystem/issues/3

As opposed to the devkitPro maintainers, I've documented 100% of the release process here https://blocksds.skylyrac.net/docs/design/release_process/ and all my binaries are built in a GitHub pipeline https://github.com/blocksds/packages/blob/master/.github/workflows/build.yml so the process is as transparent as possible. Even the documentation and docker images are built in GitHub actions. The only things I don't build myself are the compiler, libc, libstdc++... You can get all information here. https://github.com/WonderfulToolchain/wonderful-packages/tree/main

In fact, my repository of packages has a bunch of libraries that other people created and can be useful, particularly to build old projects, like uLibrary, PAlib, libwoopsi, Nitro Engine, NFlib... As opposed to the devkitPro philosophy of just using what they offer and nothing else.

And well, they can say forks of their libraries are illegal as much as they want (is that really the word they used?). The mere fact that they use that word means they don't understand open source licenses.

They also have the tendency to delete GitHub issues that put them in uncomfortable situations regarding their behaviour. https://web.archive.org/web/20241005010626/https://github.com/devkitPro/ndstool/issues/14

How in pair is BlockDS to 2021 version of devkitARM? I may just migrate to BlockDS. I don't use NitroFS in my homebrew and I may contribute to BlockDS myself.
 
  • Like
Reactions: Moseii
How in pair is BlockDS to 2021 version of devkitARM? I may just migrate to BlockDS. I don't use NitroFS in my homebrew and I may contribute to BlockDS myself.
It started as a fork of libnds, maxmod and DSWiFi from early 2023, and there are a few additions that Calico doesn't have, like DSi camera support, experimental DSi DSP support, local multiplayer WiFi mode without a router, faster filesystem access than devkitARM, documentation and actual licenses everywhere, and I'm probably forgetting important things. The main 2 missing things compared to calico are NAND and DSi WiFi (WPA support and such).
 
  • Like
Reactions: Moseii
Having cutting-edge version of something breaking compatibility with old software is usual, if not expected. For example, libLLVM doesn't even try to keep compatibility with software linked with older versions of it. It is your responsibility to ensure that your software links fine with libLLVM, say, 16, 17, 18 or 19. If you are stuck with an old version, it is your responsibility to
backport the fixes that you need, and it is just how things goes. They just don't want the trouble, and I don't see a problem in that.



What is anthithetical to their goals? Be able to build it from sources and not use the binary they are shipping? Sorry, but that seems a very suspicious behavior for me, and very anthithetical to the free software way of doing things. Plus, what if I want to have the toolchain built for a PowerPC host?
Documentation has never been great for the devkitpro tools and libraries. It sounds like you were able to build everything from source though - or is that not correct? Not to disparage blockds, but would blockds have happened without devkitpro efforts?
I don’t necessarily disagree with your complaints. The calico changes do make the setup/install/update process quite complex as it can be quite confusing when calico is not found. Maybe my overall expectations are lower for free software.
Most of the complaints I have seen from devkitpro are regarding support requests for modified devkitpro tools and libraries. This can be a big time sink for them. Often times this is caused by outdated videos and forks/downloads that are not clearly marked/identified.
To be clear - I am not affiliated with devkitpro in any capacity. I am just a user of their products. And these are just my opinions and observations.
 
It doesn't matter who enabled what to me, honestly. I only care about who's doing the right thing.

When DevKitPro pushed the recent, what was it, r21 update all macOS copies of NDS and GBA projects including their own examples would not compile on any Mac. It was pushed untested. A trivial test would be to leverage the GitHub CI/CD infra they already have to run `make` on their examples. There appears to be no investment in automated testing infrastructure before a release.

DevKitPro does not support rolling back or using an older version of the toolchain. They actively thwart archival attempts using litigation covering their trademark. While trademark protection is valid and good, there's no interest in providing a backup from the organization themselves. I was completely locked out of my projects with no way back other than a personal disk image.

I opened an issue on GitHub and was shut down, told to go to the forums. I went to the forums and my post was automatically deleted due to having too much text. I summarized the issue. An update to the binary installations was pushed a day or three later. No associated sources were updated -- I checked every repo looking for an explanation of the fix.

It's a very bad look to ship GCC updates without pushing updated source code or documenting the changes. What happened there? Doesn't matter to me, I switched away from the platform and ported Wonderful Toolchain to macOS myself for my own personal needs.

None of these are about modified tools or libraries.

A user figuring out how to build GPL3 software on their own without help from the developers of the software who provided binaries of the software is completely antithetical to FOSS. You're supposed to provide a means to reproduce your builds. Taking donations and feeding yourself with GPL3 software without teaching others how it's done is a departure from what it's all about.

I opened an issue on catnip asking for clarification of the license -- there is none. It was silently moved to a private repository, deleting it. It seems the organization is reserving all rights to legal action against anyone who clones catnip and tries to modify it or ship their own. Catnip is dead out of the door.
 
Documentation has never been great for the devkitpro tools and libraries. It sounds like you were able to build everything from source though - or is that not correct?
No, there are missing pieces. I can't get nds-examples to compile because the crt0 shipped with the buildscripts references removed symbols from libnds. That must somehow reference calico instead, but where is the updated crt0?

But the point I made is that I got it to build despite their almost non-existing documentation because I have written numerous Makefiles, C, C++ and autotools stuff in my life.
 
Documentation has never been great for the devkitpro tools and libraries. It sounds like you were able to build everything from source though - or is that not correct? Not to disparage blockds, but would blockds have happened without devkitpro efforts?
I don’t necessarily disagree with your complaints. The calico changes do make the setup/install/update process quite complex as it can be quite confusing when calico is not found. Maybe my overall expectations are lower for free software.
Most of the complaints I have seen from devkitpro are regarding support requests for modified devkitpro tools and libraries. This can be a big time sink for them. Often times this is caused by outdated videos and forks/downloads that are not clearly marked/identified.
To be clear - I am not affiliated with devkitpro in any capacity. I am just a user of their products. And these are just my opinions and observations.
Would devkitPro have happened without the dozens of people that have contributed to all their libraries and tools? The maintainers of devkitPro have definitely done a lot of work there, and that's why they are still credited properly in BlocksDS, but they wouldn't have been able to create devkitPro without the work of a lot of people. That's what FOSS is about, after all. You take the work of others, improve it, other people improve your work, etc. I don't feel bad about forking their libraries because they weren't theirs to begin with. I'm only going to list what I know about the NDS libraries, and only the major contributors, but you can find similar lists for other platforms.

libnds started of as ndslib, by Dovoto and Joat: https://sourceforge.net/projects/ndslib/

DSWifi was created by sgstair: https://www.akkit.org/dswifi/

Maxmod and mmutil were created by Mukunda: https://mukunda.com/projects.html

libfat was created by Chishm: https://www.chishm.com/libfat/index.html

libfilesystem was originaly called NitroFS, they removed the original license header and renamed it to libfilesystem. https://github.com/SVatG/TheStrongestDemo/blob/03ed510589d85606f3e606e08bc156d391a5d042/nitrofs.h -> https://github.com/devkitPro/libfil...a17daa0ddd2c76eb92165161481d/source/nitrofs.c (I can't find the original website of nitrofs right now!)

ndstool was partly written (and created?) by Darkfader: https://darkfader.blogspot.com/2005/

grit was created by Cearn: https://www.coranac.com/category/proj/grit/

So yes, they pretty much did the same thing I've done: They saw libraries and tools made by good developers, integrated them in devkitPro and started to distribute them. Then they improved them, of course. I've seen commits by WinterMute everywhere. But you can also find commits by me everywhere in BlocksDS.

My point is: FOSS is about collaboration. You need to document things, be transparent, license the code properly, and communicate with your community about changes that may affect them (like breaking changes). If not, your project isn't FOSS, it's just "source available".

Compare that with BlocksDS:

- Documentation: https://blocksds.skylyrac.net/docs/ There are also lots of comments in the source code now: https://github.com/blocksds/libnds/...884cd92ef5868a/source/arm9/system/mpu_setup.s (there were zero comments in this file when I forked the library)
- Transparency about how everything gets built and uploaded to the servers: https://github.com/blocksds/packages/blob/master/.github/workflows/build.yml https://blocksds.skylyrac.net/docs/design/release_process/
 
Last edited by AntonioND,
If you look at Dave Murphy (WinterMute)'s personal blog, he uses dehumanizing metaphors that compare users of his software to animals -- specifically a herd of cats he is managing.

The metaphor I ended up with was intended to describe behaviour that I find absurd, painful and more than a little cruel and it was done in the abstract in an attempt to avoid pointing out specific people and/or projects.

I'm not sure he's seeing people anymore on the GitHub issues, just problems he's above solving.

The calico and catnip names for his new software paradigm are explicitly designed to herd the cats into compliance again. The names and this blog tell you all you need to know.
 
I've seen DevKitPro staff say on GitHub issues that other toolchains in the space are "illegal forks" (their words).

Calling all other toolchains "illegal" without evidence is a heavy and potentially defamatory accusation. I have seen no evidence of devkitPro's developers making such a specific statement, and I personally trust them to know better; if those were not their literal words, you may want to correct that.
 
Here is the direct link.

The person you refer to is hosting illegal copies of many binaries

I mistook "fork" for "copies". This appears to relate to the trademark enforcement litigation. Thank you for seeking clarification. I see you were there when it was written.

Rehosting a GPL3 binary is not inherently illegal. If the state of the software leads to a state where litigation can be performed from this action, it seems the software binaries were engineered against the GPL3. Here we are -- GPL3 software that can lead to illegal states after complying with the text file provided to the user. Any machinations that enabled this are tools that destroy FOSS ideals.

Normally I would assume ignorance and not malice, but given your take that the organization members are intelligent and know how to conduct themselves appropriately, I can only presume malice.
 
Last edited by linux86,
Here is the direct link.



I mistook "fork" for "copies". This appears to relate to the trademark enforcement litigation. Thank you for seeking clarification. I see you were there when it was written.

Rehosting a GPL3 binary is not inherently illegal. If the state of the software leads to a state where litigation can be performed from this action, it seems the software binaries were engineered against the GPL3. Here we are -- GPL3 software that can lead to illegal states after complying with the text file provided to the user. Any machinations that enabled this are tools that destroy FOSS ideals.

Normally I would assume ignorance and not malice, but given your take that the organization members are intelligent and know how to conduct themselves appropriately, I can only presume malice.
100% agree - I have no problems with blockds. In my head that is what I meant by “not to disparage blockds”.
I was trying to say that I felt devkitpro has made valuable contributions to making these platforms accessible. I know I would not have been able to develop anything for ds/dsi/3ds without devkitpro.
Post automatically merged:

If you look at Dave Murphy (WinterMute)'s personal blog, he uses dehumanizing metaphors that compare users of his software to animals -- specifically a herd of cats he is managing.



I'm not sure he's seeing people anymore on the GitHub issues, just problems he's above solving.

The calico and catnip names for his new software paradigm are explicitly designed to herd the cats into compliance again. The names and this blog tell you all you need to know.
Yeah … that was not great. I took that as a person with poor personal communications skills trying to express frustration in a humorous way. My take away from that is that some people ask for help and then ignore the offered help - which is their prerogative, but some people take this to extremes and get abusive about the offered help. I may have read a lot into it …
 
Last edited by elhobbs,

Site & Scene News

Popular threads in this forum