Homebrew Is there a max amount of code in a 3dsx file?

[mashers]

Hi everyone

I've got a weird problem with my homebrew launcher. As I add code and features, the launcher will stop exiting when I try to launch something (i.e. it freezes). The problem goes away if I remove other, unused, code.

For example, I got to a point where the launcher was working, and the only change I made was to add a definition for a boolean variable. This variable at the time had no effect on anything as it had only been defined. After adding this definition, the launcher stopped exiting when selecting something. When I removed an unused graphics function it started working again with the boolean variable definition still there.

I have been having this problem throughout, where I can add features up to a certain point before having to find something unused to remove. The actual filesize of the 3dsx does not seem to be the issue as I can add rather large data files which get incorporated into the 3dsx and do not cause this problem. It is literally only if I add code to the source.

I created a function which logs to a text file on the SD card. I then put calls to this all through main.c, and the last log entry was just before the call to sdmcExit(). The log immediately after this line was not present in the log file, indicating that sdmcExit() is hanging. I can't, however, work out why adding code to the project (which might not even have anything to do with the filesystem) would cause this hang.

Thanks in advance for any help!

 

[Tjessx]

There is a max file size, how big is your 3dsx?
Are you sure you closed all files before sdmcexit is called

 

[nop90]

I had such a problem in the past.

Mine was caused by having a global array and overflowing it. In this case the code doesn't crash because you are writing in allocated memory for other global vars, but the code behaviour is unpredictable

 

[Tjessx]

I had such a problem in the past.

Mine was caused by having a global array and overflowing it. In this case the code doesn't crash because you are writing in allocated memory for other global vars, but the code behaviour is unpredictable

This just solved a bug i had. Thanks

 

[mashers]

There is a max file size, how big is your 3dsx?

The file is 1.6MB currently. Is that too big?

Are you sure you closed all files before sdmcexit is called

I've checked and every call I have made to fopen() is followed by a fclose() when the file is finished with. Is that enough?

I had such a problem in the past.

Mine was caused by having a global array and overflowing it. In this case the code doesn't crash because you are writing in allocated memory for other global vars, but the code behaviour is unpredictable

Hmm ok, could be on to something. I've got some C++ files included which have globally defined vectors and maps. Could those be causing the problem?

 

[nop90]

Using pointers you can do everything and everything can happen if you miss some boundary check.

Without pointers there would have not been any code hacking (complex verb tenses, hope it's not too wrong! ).

You are not alone trying to find strange bugs. There aren't other solutions than debugging the code line by line. Good luck.

 

[mashers]

Well it turns out the problem is to do with using a C++ map object. This object was originally defined globally so for testing I commented out the definition and also the content of the few methods that used it (it's only for loading and saving the config). The launcher then worked perfectly. However, even using a map defined as an object within a method seems to have the same effect. I'm going to try an implementation of the same functionality using a vector instead as I've used vectors elsewhere with no problems.

 

[Tjessx]

Well it turns out the problem is to do with using a C++ map object. This object was originally defined globally so for testing I commented out the definition and also the content of the few methods that used it (it's only for loading and saving the config). The launcher then worked perfectly. However, even using a map defined as an object within a method seems to have the same effect. I'm going to try an implementation of the same functionality using a vector instead as I've used vectors elsewhere with no problems.

Is it a multidemensional array? cause i had the same problem with a 2dimensional array yesterday

 

[mashers]

Is it a multidemensional array? cause i had the same problem with a 2dimensional array yesterday

It was a map, which I believe is a two dimensional vector with additional methods for accessing the content of one vector (the values) through the values in the other (the keys). I've been trying to implement the same functionality using C arrays (char[][]) but it's a nightmare, so I might just try using two one-dimensional C++ vectors with one containing the keys and the other the values, and iterate through the keys vector to get the index of the key to access the value in the same position in the values vector. Less than ideal, but preferable to using plain C arrays which are horrible.

--------------------- MERGED ---------------------------

Hmm, interestingly std::vector<char*>keys works whereas std::vector<std::string>keys causes the app to hang when launching...

Edit - even std::map<char *, char *> data works. So it seems the problem is caused by having a vector or a map with a C++ style string inside it, but using a C-style char* works. Weird!

 

[elhobbs]

I think there may be a limit as I am having an issue adding csnd to a project that is at that same size - 1.6 MB. If I link csnd it never finishes the homebrew menu launch sequence - prior to even calling any code in my project. I tested this by adding a sleep call after creating a console and printing some text. If add code that touches csnd then it never reached this point.

 

[Tjessx]

It was a map, which I believe is a two dimensional vector with additional methods for accessing the content of one vector (the values) through the values in the other (the keys). I've been trying to implement the same functionality using C arrays (char[][]) but it's a nightmare, so I might just try using two one-dimensional C++ vectors with one containing the keys and the other the values, and iterate through the keys vector to get the index of the key to access the value in the same position in the values vector. Less than ideal, but preferable to using plain C arrays which are horrible.

--------------------- MERGED ---------------------------

Hmm, interestingly std::vector<char*>keys works whereas std::vector<std::string>keys causes the app to hang when launching...

Edit - even std::map<char *, char *> data works. So it seems the problem is caused by having a vector or a map with a C++ style string inside it, but using a C-style char* works. Weird!

you could also do something like this
typedef struct Data {
int key;
char* content;
}Data;

And then you can create a Data array like this:
Data data[20];
you can both safe the key and value in the array like this:
data[0].key = 1;
data[0].key = "test";

of course you would need to change the variable type of the key and content for your needs.

 

[TheCruel]

I'm working with homebrew ~4MB atm (3MB without RomFS), so your size isn't the problem. Also I use std::map plenty too.

I think I've encountered similar problems. I have a demo homebrew right now that crashes if you remove an unused variable declaration in the class definition. It seems to me to be a memory alignment issue. If you ever recreate this problem on a minimal scale, it would be nice to use that as a test case for debugging. We were talking in #3dsdev not too long ago about how linearally allocating something very small seemed to break mem alignment used by GPU in some cases and crash things.

But it is worth noting that there is a file limit for Ninjhax 2.x though I'm not sure what it is. You'll know when it doesn't launch your game anymore lol.

 

[Tjessx]

But it is worth noting that there is a file limit for Ninjhax 2.x though I'm not sure what it is. You'll know when it doesn't launch your game anymore lol.

And the homebrew launcher team said something like: "we don't see homebrew bigger then .... coming anytime soon and you'll have to do a lot of effort to reach this size".
I'm not sure, but i think it is the amount of free ram of the 3DS which you can use, because before launching the 3DSX it load it into the ram (i think)

 

[mashers]

Thanks for the additional info guys. So, are we thinking this is a ctrulib issue?

 

[TheCruel]

Thanks for the additional info guys. So, are we thinking this is a ctrulib issue?

Well, if there is an instance where merely declaring an unused variable determines whether your app runs or not, then I'd say that's likely a ctrulib bug.

 

[elhobbs]

I'm working with homebrew ~4MB atm (3MB without RomFS), so your size isn't the problem. Also I use std::map plenty too.

I think I've encountered similar problems. I have a demo homebrew right now that crashes if you remove an unused variable declaration in the class definition. It seems to me to be a memory alignment issue. If you ever recreate this problem on a minimal scale, it would be nice to use that as a test case for debugging. We were talking in #3dsdev not too long ago about how linearally allocating something very small seemed to break mem alignment used by GPU in some cases and crash things.

But it is worth noting that there is a file limit for Ninjhax 2.x though I'm not sure what it is. You'll know when it doesn't launch your game anymore lol.

How much of that 3 MB is executable code vs initialized/embedded data?
When you say linear allocs cause problems with gpu code - do you mean even when using linearMemAlign to make sure you meet documented alignment requirements?

 

[TheCruel]

How much of that 3 MB is executable code vs initialized/embedded data?
When you say linear allocs cause problems with gpu code - do you mean even when using linearMemAlign to make sure you meet documented alignment requirements?

I only have a couple embedded files (shader and font) which are about 60kB. Everything els eI load using RomFS. But I have some const arrays also in .text section (most of my filesize) for encoding and such. I'm using a very verbose port of OpenGL which has a lot of code not currently necessary. Plus I use C++ STL stuff, so there is some bloat there.

Regarding memalign, I think that may have been the issue on IRC. Someone wasn't aligning their textures to 0x80 properly. But I still think there may be memory management/alignment issues elsewhere, even if you align everything as documented. I don't doubt there's still something not documented properly.