Hacking Automatic builds of popular 3DS CFW/Tools

[GBHAKC75]

Hi everyone.

I've configured a build infrastructure for 3DS Tools and CFWs, using Jenkins.
For every code change on these projects, they're automatically built and ready to be used.

Download server : https://jenkins.hakujou.fr/

PLEASE BE CAREFUL : THE BUILDS CAN BE DANGEROUS IF THERE'S A MISTAKE ON CODE
ONLY USE THIS IF YOU KNOW THE RISKS​

Supported projects :

• Luma3DS (also w/ Legacy)
• FBI
• Decrypt9
• GodMode9 (+ SafeMode9)
• Hourglass9
• sysDowngrader
• SafeA9LHInstaller
• TWLoader
• ReiNAND
• SafeB9SInstaller
• PKSM
• MultiUpdater
• Anemone3DS

Supported exploits :

• arm9loaderhax
• boot9strap
• ntrboot-flasher
• SoundHax
• FastHax
• SafeHax
• WaitHax

Removed :

• ReiNAND+ (Deprecated)
  
• Luma3DS-Kecleon (Deprecated)
• Themely (Closed-source, controversial)

FAQ :
"Why ? We already have nightly builds for those"
While that's true, this is a bit different. Existing websites with nightlies are using bash scripts to perform the builds and add them to websites. There's nothing wrong with that, but when there's enterprise-grade tools to do precisely this kind of tasks (named CI : Continuous Integration), I don't really see the point of using a script which may break against a real CI tool. So I use Jenkins, a very popular open-source CI tool used even in enterprises.

"I'd like <SomeHomebrew> to be added. Is it possible ?"
Sure ! Just asks me, with Git link of the project.
Please note I can't add Windows-only software (such as PKHeX) as they're using C# with Windows proprietary libs and therefore can't be built on Linux servers.

 

[gnmmarechal]

Nice stuff! But shouldn't you name Corbenik as Skeith? You were compiling it as such, correct? As for how I'm pulling the files...

Well... It's simple, I currently just download the field from a URL set in a file in my server. I just need a static URL to Corbenik (Skeith, I guess).

About compiling other stuff..... Well, users know, or should know, that nightlies aren't stable builds and can indeed contain bugs. So IMO, it shouldn't matter too much.

 

[Arubaro]

I'm not a big fan of nightlies, but is a fact that this helps debugging apps until the next stable version is released, so I'm grateful

 

[GBHAKC75]

Nice stuff! But shouldn't you name Corbenik as Skeith? You were compiling it as such, correct? As for how I'm pulling the files...

Well... It's simple, I currently just download the field from a URL set in a file in my server. I just need a static URL to Corbenik (Skeith, I guess).

About compiling other stuff..... Well, users know, or should know, that nightlies aren't stable builds and can indeed contain bugs. So IMO, it shouldn't matter too much.

Thanks. I asked myself the very same question, but as @chaoskagami said, he doesn't plan to do releases for now and Skeith is just the name for Corbenik nightlies. The way I see it it's still Corbenik. I can change it pretty easily if that's a problem anyway

So you need an URL like "https://dl.hakujou.fr/Luma3DS/latest" ? I can do that with symlinks if needed, that's not a problem.

I understand, but some people just don't understand that and want to try the latest software anyway. While it should be avoided, I still fear the fact that someone will try to install some new A9LH and brick his 3DS... I'll focus on less "risky" softwares for now (such as FBI) and see later for more risky ones

I'm not a big fan of nightlies, but is a fact that this helps debugging apps until the next stable version is released, so I'm grateful

It's pretty much the opinion that I have. Jenkins is really great because it provides console output for every builds so it's easy to see what breaks, which commit breaks it and why (which is the point of CI), that's my main reason to have done this project.
I'd like to open Jenkins to everyone but it's connected to my AD setup and I don't really want to expose my users to everyone :/

 

[gnmmarechal]

Thanks. I asked myself the very same question, but as @chaoskagami said, he doesn't plan to do releases for now and Skeith is just the name for Corbenik nightlies. The way I see it it's still Corbenik. I can change it pretty easily if that's a problem anyway

So you need an URL like "https://dl.hakujou.fr/Luma3DS/latest" ? I can do that with symlinks if needed, that's not a problem.

I understand, but some people just don't understand that and want to try the latest software anyway. While it should be avoided, I still fear the fact that someone will try to install some new A9LH and brick his 3DS... I'll focus on less "risky" softwares for now (such as FBI) and see later for more risky ones



It's pretty much the opinion that I have. Jenkins is really great because it provides console output for every builds so it's easy to see what breaks, which commit breaks it and why (which is the point of CI), that's my main reason to have done this project.
I'd like to open Jenkins to everyone but it's connected to my AD setup and I don't really want to expose my users to everyone :/

Something like EasyRPG Player would be great, if you did something like:
http://dl.hakujou.fr/Corbenik/latest-build.zip for the build
and something like
http://dl.hakujou.fr/Corbenik/latestver for the actual commit/version name, I suppose?

 

[GBHAKC75]

Something like EasyRPG Player would be great, if you did something like:
http://dl.hakujou.fr/Corbenik/latest-build.zip for the build
and something like
http://dl.hakujou.fr/Corbenik/latestver for the actual commit/version name, I suppose?

Do you have any examples for EasyRPG ? I could do something similar so it'd be easier for you to use.

 

[gnmmarechal]

Do you have any examples for EasyRPG ? I could do something similar so it'd be easier for you to use.

So, EasyRPG Player 3DS is here:
https://ci.easyrpg.org/job/player-3ds/
I get the CIA, for example, from "https://ci.easyrpg.org/job/player-3ds/lastSuccessfulBuild/artifact/builds/3ds/easyrpg-player.cia". I get the version from "https://ci.easyrpg.org/job/player-3ds/lastSuccessfulBuild/buildNumber".

 

[GBHAKC75]

Okay, I finally exposed Jenkins, it's easier and I managed to separate my users.
It's the exact same structure for me : https://jenkins.hakujou.fr

The dev can also see console output this way, it might be useful to them.

Have fun !

 

[gnmmarechal]

Okay, I finally exposed Jenkins, it's easier and I managed to separate my users.
It's the exact same structure for me : https://jenkins.hakujou.fr

The dev can also see console output this way, it might be useful to them.

Have fun !

Awesome, thanks! This will be implemented soon (this afternoon), and then, after getting someone to test it (as I didn't bring my 3DS with me and won't have it until the 3rd of January x.x), I'll add it to the stable updater.

--------------------- MERGED ---------------------------

By the way, would it be possible to, instead of buildNumber, somehow get the commit?

 

[GBHAKC75]

Awesome, thanks! This will be implemented soon (this afternoon), and then, after getting someone to test it (as I didn't bring my 3DS with me and won't have it until the 3rd of January x.x), I'll add it to the stable updater.

--------------------- MERGED ---------------------------

By the way, would it be possible to, instead of buildNumber, somehow get the commit?

You could parse it from here : https://jenkins.hakujou.fr/job/3DS/job/Corbenik/6/git/

But a "cleaner" way to handle this would be to use Jenkins API, like this : https://jenkins.hakujou.fr/job/3DS/job/Corbenik/lastBuild/api/xml?xpath=//lastBuiltRevision/SHA1

 

[keyra]

i must be stupid but how do i know if your luma is older than the one one alex's builds?

 

[gnmmarechal]

i must be stupid but how do i know if your luma is older than the one one alex's builds?

huh, it shouldn't be. It compiles every commit, so it may actually be more recent?

 

[GBHAKC75]

i must be stupid but how do i know if your luma is older than the one one alex's builds?

You can compare against commit number. Look in GitHub which one is the latest and which site has it.
But as said, it builds automatically (and very quickly) the project at any commit so the latest version should always be there.

 

[keyra]

hmmmz i see the commit version on alex but not on yours that"s why

 

[GBHAKC75]

hmmmz i see the commit version on alex but not on yours that"s why

Check here : https://jenkins.hakujou.fr/job/3DS/job/Corbenik/lastBuild/
It's the "revision" part.

 

[gnmmarechal]

Actually, I might just use your Luma3DS URLs in StarUpdater-UN. I'll update Corbenik CFW Updater: RE, and then I'll look into Luma.

--------------------- MERGED ---------------------------

Check here : https://jenkins.hakujou.fr/job/3DS/job/Corbenik/lastBuild/
It's the "revision" part.

Did it go down?

EDIT: nvm, my internet's crap.

 

[GBHAKC75]

Actually, I might just use your Luma3DS URLs in StarUpdater-UN. I'll update Corbenik CFW Updater: RE, and then I'll look into Luma.

--------------------- MERGED ---------------------------


Did it go down?

EDIT: nvm, my internet's crap.

You scared me !

IMHO you really should considering using Jenkins API for those projects. It's waaaaaay cleaner than parsing project window.

 

[gnmmarechal]

All right, there's one thing. Jenkins doesn't provide a zip directly, does it?

--------------------- MERGED ---------------------------

Other than
https://jenkins.hakujou.fr/job/3DS/job/Corbenik/lastSuccessfulBuild/artifact/out/*zip*/out.zip

that is.

--------------------- MERGED ---------------------------

Would it be possible to make Jenkins provide the zip directly, with the same structure as Corbenik's/Skeith's?

 

[GBHAKC75]

All right, there's one thing. Jenkins doesn't provide a zip directly, does it?

--------------------- MERGED ---------------------------

Other than
https://jenkins.hakujou.fr/job/3DS/job/Corbenik/lastSuccessfulBuild/artifact/out/*zip*/out.zip

that is.

No it doesn't, but I can zip some files through build process if needed.
Is there a problem with that zip, though ? Looks fine to me.

EDIT : Oh, the problem is the "out" folder in it ? I'll look into it.

 

[gnmmarechal]

No it doesn't, but I can zip some files through build process if needed.
Is there a problem with that zip, though ? Looks fine to me.

EDIT : Oh, the problem is the "out" folder in it ? I'll look into it.

Yep. The updater function to extract both skeith and corbenik is exactly the same, with only the skeith directory changing, so this zip having a different structure would require me to make another function for Skeith.
The zip should have the same structure as Corbenik/Skeith, with everything directly inside it.