I've decided to just copy/paste the relevant section from the IRC. I find the discussion interesting in that it talks about the difficulties in maintaining NTR, so I just wanted to share.
Mirror: https://hastebin.com/uxekajibut.txt
Code:
<wedr__> attempting to add and integrate Visual Studio 2017 into cell9's NTR fork.
<TuxSH> corbenik was a script kiddie trap
<wedr__> Alright, with VS setup, let me try to create a Makefile for this NTR thing... I do admit, the Python method seems to be of a cross-platform non-Make solution... :/
<@profi200> It's borked beyond repair. Not sure you will get any standard build system working with it.
<wedr__> Damn... Should've heeded the discussion earlier, but I was at work
<wedr__> Well I guess we can't use straight off use the Makefile templates
<wedr__> /use straight off/straight off
<wedr__> Hmmmmm
<wedr__> So the toolchain params are all hardcoded...
<wedr__> Ain't that fun, no wonder cell9 is reluctant on making it open source, because it's not... easily tweaked...
<wedr__> Alright, first PR for NTR. https://github.com/44670/NTR/pull/1
<wedr__> Because I can't do anything. :(
<wedr__> Hi cell9
<cell9> hey
<wedr__> cell9: Please wait for a 2nd PR
<Pieman> hello everyone :3
<wedr__> cell9: Before I do a PR, can you check to see if this Makefile works for you? https://github.com/tommai78101/NTR/commit/144ac050ea6dc3c92a7618b104d1ea5c9daf8b61
<wedr__> I sort of triiiiiiiiiied my best to make a good Makefile. :/
<wedr__> oh yeah, forgot that PRs can be reviewed...
<cell9> ntr.o3ds.bin should be build with -Os
<cell9> ntr.n3ds.bin should be built with -O3 (which increases the file size but much faster)
<wedr__> cell9: I don't see ntr.n3ds.bin in your build.py
<cell9> it was in the build-jpeg.py
<wedr__> ohhh
<wedr__> I'll have to make a few more changes.
<cell9> so writing a proper makefile may be not very easy
<wedr__> yeah, it's really not, but I think I can manage...
<cell9> I guess object files built for different release type should be seperated to different paths
<wedr__> ok
<wedr__> cell9: By the way, are you ok with me adding Visual Studio solution files to the upcoming PR?
<wedr__> 'Cause VS2017 supports Python
<wedr__> Having it integrated works really well, in my opinion
<cell9> better not
<cell9> tbh ntr is developed with VS2015
<cell9> however before opensourcing I have removed the deps for vs, which may be better for other developers
<wedr__> Ah...
<wedr__> I'll remove them from my PR then.
<wedr__> cell9: Can you review the Makefile? Thanks. https://github.com/44670/NTR/pull/2
<cell9> will source/jpeg be built?
<cell9> btw there include/gen.h should be auto generated from the build option
<cell9> *the
<wedr__> source/jpeg should be built....
<wedr__> I need to check where gen.h is being produced...
<wedr__> cell9: Where did you get that include/gen.h from?
<wedr__> OHhhh
<wedr__> if using build-jpeg, you define a macro
<wedr__> Else, you don't...
<wedr__> Gotta sleep, I've believed I have the Makefile ready now, I need someone else to review it.
<cell9> .
<cell9> 0
<cell9> .00000000
<cell9> oops
<wedr__> If anyone wanna help out, by all means go ahead: https://github.com/44670/NTR/pull/2 I'm stuck and tired, so if anyone knows how to fix the Makefile, go ahead and PR me and cell9
<wedr__> leaving
== KindOne_ has changed nick to KindOne
== KindOne- has changed nick to KindOne
== KindOne_ has changed nick to KindOne
== KindOne_ has changed nick to KindOne
== lameboy2 has changed nick to lameboy
<Namidairo> not really related, but what are people's opinions on cmake
<@fincs> It is shit
<@fincs> And sadly it's been spreading its cancer lately
<Namidairo> lol
== Alta has changed nick to Altaz
<wedr_> Is make the same as cmake?
<wedr_> Or cmake is just "cross-platform make"?
<wedr_> I'm just here, nervous about what to do with the makefile I created for NTR.
<wedr_> Because I couldn't get it to build, as expected, despite my efforts to try and recreate the A files using the default dka installation.
<@WntrMute> cmake is a makefile generator
<wedr_> Ah, so the "c" is "Create Make"?
<wedr_> Well that's easy enough.
<@WntrMute> pfft
<@WntrMute> if it was "easy enough" there wouldn't be bjam & cmake & meson & autotools & a hundred & one other shitty build system generators
<@WntrMute> recreate what a files?
<wedr_> WntrMute: For some weird reason, the NTR project comes with 3 A files. I'm guessing you are to require the A files to compile the NTR?
<wedr_> WntrMute: Then again, when building, it would say, oh, svc_blahblahblah function not found.
<@WntrMute> those are part of the compiler, they shouldn't be in the repo
<wedr_> Well, it's in the official NTR project
<wedr_> the one cell9 uploaded
<@WntrMute> I know
<@WntrMute> libc.a libgcc.a and libm.a should *not* be in the repo
<wedr_> Then I should've made the clean command remove the A files.
<wedr_> But if I remove those files, I wouldn't be able to know how to generate them back.
<@WntrMute> they're part of the toolchain, they should remain in the toolchain and the binary should be linked *properly*
<wedr_> Ah, so the build scripts cell9 uses is actually wrong?
<@WntrMute> yes
<wedr_> dang, I wasted my time rewriting the makefile. :(
<wedr_> WntrMute: Do we have a makefile that uses any of those files somewhere, so I can see how it is written?
<@WntrMute> you're asking the wrong questions
<wedr_> No, I mean, a Makefile that generates those files. How, where do they come from?
<wedr_> They have to at least be build from somewhere...
* WntrMute headdesks
<wedr_> :( I'm not used to the Linux toolchains. :/
<@WntrMute> not by you, not by this project and certainly not by anyone wanting to compile this project
<@WntrMute> those files should not be in the repo, they should not be explicitly linked and you shouldn't have to care about them in any capacity
<wedr_> ah I see now
<@WntrMute> the link command is basically just wrong
<Linkle> Oh, sorry
<Linkle> lolhi WntrMute
<@WntrMute> ultimately those ridiculous python scripts need to be examined, the commands analysed and the intended behaviour determined
<@WntrMute> like what in fucks name is this? https://github.com/44670/NTR/blob/master/build-jpeg.py#L17-L18
<wedr_> WntrMute: I'm guessing the inserted macro is used elsewhere in the source/jpeg code
<@WntrMute> don't guess
<wedr_> ok
<@WntrMute> look at it, say "what in fuck's name is this lunatic doing?"
<wedr_> Well at least the first thing on the TODO list is to fix the build system.
* wedr_ is optimistic.
<@WntrMute> examine this line and wonder at the lunacy that brought it into existence
<@WntrMute> https://github.com/44670/NTR/blob/master/build-jpeg.py#L32
<wedr_> WntrMute: cell9 is compiling all of the C files with -O3 for the n3ds. It's probably how it's done the pythonic way.
<@WntrMute> actually, both of those lines :/
<wedr_> I don't know for the second line. :/
<wedr_> What the *.s files are for...
<@WntrMute> heh, "pythonic way"
<@WntrMute> you appear to be labouring under the misapprehension that this build process makes any fucking sense
<@WntrMute> let me help with that
<wedr_> Yes... I admit that.
<@WntrMute> it fucking doesn't
<@WntrMute> cell9, heh, hi
<cell9> hey
<@WntrMute> cell9, why are you building this stuff like this?
<cell9> huh?
<@WntrMute> https://github.com/44670/NTR/blob/master/build-jpeg.py#L32
<wedr_> cell9: We were just discussing about your python build scripts for NTR.
<cell9> ah
<wedr_> cell9: And I had trouble actually starting the complicated build, because I assumed the build scripts were working.
<@WntrMute> am I missing something or is there an actual output specified for that?
<cell9> pardon?
<cell9> the output files should be *.o
<@WntrMute> hmm
<Linkle> Anyone get their miiverse download yet?
<cell9> then it was linked together by ld command
<wedr_> Linkle: It will happen a few weeks from now
<Linkle> Thanks wedr_, i had taken a lot of MM screenshots for documentation
<wedr_> Linkle: You should go visit the web archive
<Linkle> My browser has a miiverse page open with 6 notifications...I'll never get to read them
<wedr_> It's 45 million posts
<Kirtide> rip
<wedr_> cell9: Previously, both WntrMute and I were discussing about the -l flags you put here. https://github.com/44670/NTR/blob/master/build-jpeg.py#L35
<wedr_> cell9: And then WntrMute told me the .a files shouldn't be in the repo from the start.
<cell9> -lc -> link to libc.a
<cell9> there was a libc.a in the repo
<wedr_> cell9: yeah, but... aren't those the toolchain files?
<cell9> that's right
<@WntrMute> oh I see, there's a -c hidden right down the end
<@WntrMute> cell9, you shouldn't link to those explicitly like that
<cell9> what -c ?
<cell9> because --nostdlib was set
<wedr_> cell9: You have -c at the end of the -I;s
<@WntrMute> run(CC+ " -O3 -s -g -I include -I include/jpeg -I/c/devkitPro/portlibs/armv6k/include " + allFile('source/dsp/*.c') + allFile('source/jpeg/*.c') + allFile('source/ns/*.c') + allFile('source/*.c') + allFile('source/libctru/*.c') + " -c -march=armv6 -mlittle-endian -fomit-frame-pointer -ffast-math -march=armv6k -mtune=mpcore -mfloat-abi=hard ");
<wedr_> -I's*
<cell9> ah
<@WntrMute> you shouldn't be using --nostdlib and then linking to stdlibs
<cell9> haha
<@WntrMute> makes no sense
<cell9> maybe I have met some problem when linking to the stdlib by default
<@WntrMute> yeah, it's called messing with how stuff links for no reason :p
<cell9> which will link to crtmain and some other stuffs
<cell9> but ntr has its own entry
<@WntrMute> depends on what you tell it to link
<cell9> hmm
<@WntrMute> basically you just shouldn't be doing this like this, it has a tendency to cause screwed up things to happen
<cell9> and the libc was picked from another toolchain
<@WntrMute> heh
<@WntrMute> that's just insanity
<cell9> maybe I have met some issue while using devkitarm's one, I couldn't rememver
<cell9> *remember
<@WntrMute> yeah, I know exactly what issue you ran into
<cell9> wow
<wedr_> With that said, I'm surprised NTR worked...
<@WntrMute> at least I have a fairly good idea
<cell9> another reason is that ntr needs relocate itself on startup
<@WntrMute> I'm not sure if this is actually fixable
<cell9> ntr.bin is not a elf/3dsx/ncch file
<@fincs> Did someone say relocation? ( ͡° ͜ʖ ͡°)
<@WntrMute> cell9, yeah, neither is .3dsx
<@WntrMute> I'm kind of surprised this stuff works at all
<cell9> because it needs to be attached to running processes
<@WntrMute> you can't just go building stuff with one toolchain & linking with libs from another one
<@WntrMute> how much libc stuff are you actually using?
<cell9> minimal
<cell9> no io
<cell9> no printf
<cell9> only memcpy/strcpy stuffs
<cell9> and some math functions(for jpeg)
<@WntrMute> then you shouldn't really have any issues using devkitARM libs
<@WntrMute> are you using libjpeg from portlibs?
<cell9> nope
<@WntrMute> then you shouldn't be using the headers
<cell9> some patches has made on the jpeg library
<@WntrMute> or is that what this build-jpeg python script is about?
<cell9> nope, build.py is just a build script without jpeg
<cell9> oh, did you mean the -I/c/devkitPro/portlibs/armv6k/include
<cell9> +?
<@WntrMute> yes
<cell9> hmm
<@WntrMute> you have zlib in your source tree as well
<@fincs> What changes were made to the jpeg lib? ARM ASM optimizations?
<cell9> nope
<@fincs> Removing malloc/free usage?
<cell9> yeah
<cell9> it was redirected to rpMalloc
<cell9> a insane allocator :P
<@fincs> Then you don't need a custom version of libjpeg
<@WntrMute> you don't need to modify libjpeg to do that
<cell9> and some modules could be deleted
<cell9> to make the binary size smaller
<@fincs> (and also function/data sections + gc sections)
<cell9> maybe not enough
<cell9> some path won't be called if you don't use that option by calling the apis
<@fincs> Then you have to fix the code
<@WntrMute> don't do this crap https://github.com/44670/NTR/blob/master/build.py#L13
<@WntrMute> how old is this stuff? https://github.com/44670/NTR/tree/master/source/libctru
<cell9> I guess, -I/c/devkitPro/portlibs/armv6k/include could be removed
<cell9> hmm, ntr was released on 2014
<@WntrMute> using a Makefile instead of this python weirdness would be a good start
<cell9> yep
<cell9> I guess -I/c/devkitPro/portlibs/armv6k/include could be removed
<cell9> maybe I have been using portlibs, I couldn't remember
<cell9> anyway now it does't use any library from portlibs
<cell9> *doesn't
<@WntrMute> do you have zlib source hidden in here somewhere?
<cell9> zlib is not used anymore
<@WntrMute> still have the headers
<cell9> yep, it could be removed
<@profi200> Then get rid of it.
<wedr_> So I guess my PR #2 is going to be scrapped...?
* wedr_ is sad for wasting 3 hours.
* wedr_ plays sad violin
<@WntrMute> sometimes you have to throw stuff away unfortunately
<wedr_> Yep
<wedr_> but still, the emotions cannot be overstated.
<@WntrMute> not the best of ideas to get emotionally attached to code
<@profi200> This all needs careful thinking. And code changes. It will very likely break by switching to dkA libs so that needs to be fixed.
<@WntrMute> it's not switching to dkA libs that will break it
<cell9> I guess it's using a clib from a toolchain for microcontrollers
<cell9> https://developer.arm.com/open-source/gnu-toolchain/gnu-rm/downloads
<cell9> might be this one, not sure
<wedr_> <cell9> I guess it's using a clib from a toolchain for microcontrollers <---- the libc.a?
<cell9> yep
<@profi200> If you avoid big libc functions you will have no problem using the one from dkA.
<cell9> probably
<cell9> however at 2014 the dkA may not support 3ds
<cell9> so maybe I have met some problems at that time
<@profi200> Pretty sure it did at that point.
<cell9> wow
<cell9> then I couldn't remember the actual reason
<@WntrMute> when did I sort the armv6k multilibs?
<@fincs> 2015 I think
<@WntrMute> although I doubt if you were using armv6k before I did that
<@fincs> Actually November 2014
<@WntrMute> still, not that it really matters
<@fincs> Err, nope
<@fincs> February 2015
<@fincs> https://devkitpro.org/viewtopic.php?f=13&t=8409
<@WntrMute> using libc from one toolchain with code compiled with a different one is a fairly daft thing to be doing
<wedr_> NTR is... an amazing feat...
<cell9> yep it is kinda dangerous
<cell9> I guess GNU Arm Embedded Toolchain should be used
<cell9> or the libc should be replaced with dkA
<wedr_> or both? :)
<@WntrMute> or you could just stop doing things completely the wrong way :p
<cell9> :/
<wedr_> Didn't cell9 just started doing things the right way?
<@WntrMute> nostdlib isn't the option you need
<@booto> "PRs accepted?"
<wedr_> PR #1 was accepted. :P
<wedr_> Now how do I close/delete a PR?
<wedr_> Ah found it
<wedr_> nvm
<wedr_> cell9: I'll be closing PR #2, given that, well, some drastic measures must be taken first before Makefiles can be made.
<cell9> unfortunately yes
<@WntrMute> I'd be less inclined to be procrastinating if this wasn't using python as some ghetto build system :/
<wedr_> cell9: closed
<cell9> :(
<wedr_> But it's still there, you can use the Makefile if you want to.
<wedr_> But that has to happen after the libraries are sorted out
<@WntrMute> yeah, that's not a Makefile
<wedr_> ok... a bad attempt of a make
<wedr_> (I thought I'm a genius at using macros...)
<@profi200> lol
<wedr_> (but oh well)
<@profi200> Use the template makefile from 3ds-examples. Everything else is wasted time.
<wedr_> To be fair, I was using the Python build script as the template. :/
<wedr_> At the time...
<cell9> and a build.sh or build.py is also required
<cell9> because gen.h needs to be generated
<@WntrMute> pfft
<wedr_> It's a simple @ >
<@WntrMute> why?
<@profi200> Template as in base. Still need to adjust it to what the script did.
<wedr_> WntrMute: source code uses the macro
<@WntrMute> so what?
<@WntrMute> never heard of setting flags before you run Make?
<cell9> are there any proper ways to do that with makefile?
<wedr_> cell9: yes
<cell9> I guess a ./configure script is required
<wedr_> cell9: no need I feel
<cell9> which should be a proper way to do that
<@WntrMute> CFLAGS=-DHAVE_JPEG=1 make
<cell9> hmm
<wedr_> yeah that
<@WntrMute> or have a target that sets the flag
<wedr_> target is easier
<cell9> maybe that's ok at this time
<wedr_> because it can be chained with other targets
<@WntrMute> depends on what you want to do
<@WntrMute> meh
<@WntrMute> this is why I never get anything done though :/Mirror: https://hastebin.com/uxekajibut.txt
Last edited by delete12345,
