Homebrew Avoiding memory addresses shifts

[BullyWiiPlaza]

I've been working on the TCP Gecko Installer and noticed that declaring global variables shifts everything downwards so all cheat codes will break. How can this be prevented? It's really important otherwise everything was pretty much

#include <string.h>
#include <stdio.h>
#include <malloc.h>
#include "common/common.h"
#include "main.h"
#include "dynamic_libs/socket_functions.h"
#include "dynamic_libs/gx2_functions.h"
#include "kernel/syscalls.h"
#include "dynamic_libs/fs_functions.h"
#include "common/fs_defs.h"

// ###### CULPRITS (but I need them to be allocated by default to cut down on boilerplate code) #######
void *client;
void *commandBlock;

struct pygecko_bss_t {
    int error, line;
    void *thread;
    unsigned char stack[0x6F00];
};

#define CHECK_ERROR(cond) if (cond) { bss->line = __LINE__; goto error; }
#define errno (*__gh_errno_ptr())
#define MSG_DONTWAIT 32
#define EWOULDBLOCK 6
#define FS_BUFFER_SIZE 0x1000
#define DATA_BUFFER_SIZE 0x5000

[...]

Any trick I could use? Apparently, global variables are allocated on the heap but the heap is in the 4XXXXXXX Wii U memory area. Yet the 1XXXXXXX area is shifted

@Maschell
@gudenaurock
@QuarkTheAwesome
@dimok
@wj44

 

[QuarkTheAwesome]

That's a weird one. What do you get out of &client? I'm assuming this is a HBL ELF; so it really shouldn't affect anything outside the 0x00800000 range.
Have you tried setting the variables to be in .data?

 

[BullyWiiPlaza]

That's a weird one. What do you get out of &client? I'm assuming this is a HBL ELF; so it really shouldn't affect anything outside the 0x00800000 range.
Have you tried setting the variables to be in .data?

Yes, it's a homebrew launcher ELF. The client is for the file system. How would you set the variables in the .data section like you proposed? The memory shift is 0x2200 but client and commandBlock only take up 0x1700 + 0xA80 = 0x2180 space (once allocated) so it kinda matches up.

 

[QuarkTheAwesome]

Yes, it's a homebrew launcher ELF. The client is for the file system. How would you set the variables in the .data section like you proposed? The memory shift is 0x2200 but client and commandBlock only take up 0x1700 + 0xA80 = 0x2180 space (once allocated) so it kinda matches up.

Oh, do you malloc() them? That's probably not the best idea. libdimok's (that's what I'm calling it now) malloc falls through to MEMAllocFromDefaultHeapEx, which does exactly what it says on the tin. You're allocating space from the console's main heap which it uses for everything, including games.
The Wii U's allocation functions also take an alignment variable (so you can say "give me an address that ends in 00 (nearest 0x100 bytes)" or whatever) which explains the small size discrepancy you've noticed.

I'd solve this either by allocating from somewhere else (annoying and could break other stuff) or putting what you need in the .data section, effectively moving it to 0x00800000 (has a size limitation).

Here's how you'd put the above in .data:

char client[FS_BUFFER_SIZE /*or whatever*/] __attribute__ ((section (".data")));

You could then use it like so:

FunctionThatTakesClientAsVoidPointer((void*)client);

Alternatively, this would keep client as a void*:

char client_r[FS_BUFFER_SIZE /*or whatever*/] __attribute__ ((section (".data")));
void* client = (void*)client_r;

The big difference is that your code above declares a void pointer (a memory address) that you later fill in with malloc(). Here, however, we set up a char array (char allows us to give a size in bytes) that's in the actual ELF. client (or client_r) is now a pointer to that array, so we can cast it as we like and off we go!

I will note that in this case buffer overflows will be far more devastating than normal. Be careful with your sizing.

 

[BullyWiiPlaza]

Yep, that works. The memory is no longer shifted.
But calling a file system function (like read directory) crashes the console now. It basically gets stuck. Not sure why :/

char client[FS_CLIENT_SIZE] __attribute__ ((section (".data")));
char commandBlock[FS_CMD_BLOCK_SIZE] __attribute__ ((section (".data")));

void initializeFileSystem()
{
    // Initialize the file system
    int status = FSInit();
    CHECK_FUNCTION_FAILED(status, "FSInit")

    // Allocate the client
    // client = malloc(FS_CLIENT_SIZE);
    CHECK_ALLOCATED(client, "Client")

    // Register the client
    status = FSAddClientEx(client, 0, -1);
    CHECK_FUNCTION_FAILED(status, "FSAddClientEx")

    // Allocate the command block
    // commandBlock = malloc(FS_CMD_BLOCK_SIZE);
    CHECK_ALLOCATED(commandBlock, "Command block")

    FSInitCmdBlock(commandBlock);
}

 

[QuarkTheAwesome]

Hm, not sure. I'd try an exception handler and asking someone who knows more about the FS functions.

 

[BullyWiiPlaza]

Uh, this is kinda annoying. If I do the following:

void* client __attribute__ ((section (".data")));
void* commandBlock __attribute__ ((section (".data")));
client = malloc(FS_CLIENT_SIZE);
commandBlock = malloc(FS_CMD_BLOCK_SIZE);

It is basically the same as the "original" declarations (FS functions work but the memory is shifted):

void* client;
void* commandBlock;

Allocating it via

void* client[FS_CLIENT_SIZE] __attribute__ ((section (".data")));
void* commandBlock [FS_CMD_BLOCK_SIZE] __attribute__ ((section (".data")));

and not using malloc() does not shift but crashes when an FS function is called.

void* client __attribute__ ((section (".data")));
void* commandBlock __attribute__ ((section (".data")));
client = (void *) OSAllocFromSystem(FS_CLIENT_SIZE, 4);
commandBlock = (void *) OSAllocFromSystem(FS_CMD_BLOCK_SIZE, 4);

FS functions work but shifts the memory.

Using

static

also changes nothing.

Any other suggestions?
I need both to work out at once

 

[QuarkTheAwesome]

I really do have no idea. I will note that using "void* x[num]" like above will give an array of void*s, each 4 bytes big (x becomes a pointer to the first one; a void**!). However; that shouldn't affect much around functionality.
At this point all I can recommend is an exception handler. My favorite is obviously my one but you could get away with Dimok's too. It should really zero in on the exact problem and you can work backwards pretty easily from there.

 

[BullyWiiPlaza]

But how does it help? I don't think it will do anything useful if it dumps registers and such. I have to find a solution to this problem anyway but I'm running out of ideas D:

 

[LawnMeower]

I hope this can be solved. it's annoying making all cheats above 0x11000000 being pointer codes due to this issue

 

[QuarkTheAwesome]

But how does it help? I don't think it will do anything useful if it dumps registers and such. I have to find a solution to this problem anyway but I'm running out of ideas D:

If you can get an exception log; we can figure out why the FS functions don't like having stuff in .data.

 

[BullyWiiPlaza]

If you can get an exception log; we can figure out why the FS functions don't like having stuff in .data.

So I would just do

installExceptionHandler()

with your code from here and exceptions will be caught?

Or this by dimok

I'll go with dimok's since the code is already in the project but unused.

---

Well, the game is put in a semi-frozen state (the music keeps repeating but the game still runs). Pressing home freezes it completely. Nothing is printed on-screen as exception. Also, JGecko U freezes and the server is stuck. Re-connecting is impossible obviously.

char client_r[FS_CLIENT_SIZE] __attribute__ ((section (".data")));
void *client = (void *) client_r;

char commandBlock_r[FS_CMD_BLOCK_SIZE] __attribute__ ((section (".data")));
void *commandBlock = (void *) commandBlock_r;

The memory is not shifted though but the FS functions are not functioning. Maybe you can take a look at the C code? Maybe there is something wrong with it though but it works fine when the global variables are allocated "normally" on the heap. How strange. We need some other genius to help out

 

[BullyWiiPlaza]

I fixed it now. It was easier than expected xD

All I had to do was delay the allocation till the file system is actually needed and allocate just once. Basically like this:

// Still global variables
void *client;
void *commandBlock;

void considerInitializingFileSystem() {
    if (!client) {
        // Allocate everything and set up the memory...
    }
}

case 0x52: { /* cmd_read_file */
    char file_path[FS_MAX_FULLPATH_SIZE] = {0};
    receiveString(bss, clientfd, file_path, FS_MAX_FULLPATH_SIZE);

    considerInitializingFileSystem(); // The trick to avoid the shift

    int handle;
    int status = FSOpenFile(client, commandBlock, file_path, "r", &handle, -1);
    // A lot more code...

Done. Not shifted, good code style and file system functions working. This is a base to work with for further features

 

[LawnMeower]

I fixed it now. It was easier than expected xD

All I had to do was delay the allocation till the file system is actually needed and allocate just once. Basically like this:

// Still global variables
void *client;
void *commandBlock;

void considerInitializingFileSystem() {
    if (!client) {
        // Allocate everything and set up the memory...
    }
}

case 0x52: { /* cmd_read_file */
    char file_path[FS_MAX_FULLPATH_SIZE] = {0};
    receiveString(bss, clientfd, file_path, FS_MAX_FULLPATH_SIZE);

    considerInitializingFileSystem(); // The trick to avoid the shift

    int handle;
    int status = FSOpenFile(client, commandBlock, file_path, "r", &handle, -1);
    // A lot more code...

Done. Not shifted, good code style and file system functions working. This is a base to work with for further features

you're my hero

 

[HackingNewbie]

soooo... is this code gonna be used to update the codehandler or something? Will I have to download a new version of tcpgecko installer or jgecko u or something else to use codes without memory shifting and all that stuff?
Anyway, awesome work Bully!

 

[BullyWiiPlaza]

soooo... is this code gonna be used to update the codehandler or something?

Yes, to add file system support. It's already there in JGecko U. There might be more if I have good ideas and get it working since barely anyone else helps implementing something

 

[HackingNewbie]

So I just have to download a new version of the codehandler (when it's made) and nothing else?

 

[BullyWiiPlaza]

So I just have to download a new version of the codehandler (when it's made) and nothing else?

No. TCP Gecko Installer but it's already made.

 

[HackingNewbie]

K, so I'll download it from CosmoCourtney's wii u code thread and boom, no memory shifting?

 

[LawnMeower]

K, so I'll download it from CosmoCourtney's wii u code thread and boom, no memory shifting?

The codehandler has no affect on memory shifting. The shifting is caused by TCPGecko.elf especially when it declares variables. Bully's TCPGecko improvement now fixes the memory shifting.
So no need to re-download the codehandler unless it experiences a new update