As usual this one got long and technical....possibly boring....just so you know 
When starting out on the Wii at the end of 2018 I ran into a problem I had not ever really had while programming. I knew the term big endian en little endian and that it had something to do with how bytes are stored in memory in what order they are read by the CPU. It was actually in the exact same post where I released preview v0.1 that I also explained the endian issue....just for convenience I will repeat it a little here:
Now if this was the only struct that I would have to define it might not be so bad....but there are 33 event's in Batterycheck where some of them have 8 properties!! And those need to be specified TWICE in the same way as I explained above: One time for little-endian machines (x86, MIPS, ARM) and one time in big-endian machines (POWERPC, Motorola 68000?? I think ??) in reverse order! It's just confusing and easy to get wrong....and I was not really looking forward to implement the enormous list of Jazz Jackrabbit 2 events by hand!!! The original Jazz2 has nearly 253 events in total if I remember correctly, in any case it's a very big lot more than the 33 of Batterycheck.
Now to explain my "Terrible solution" I might have to explain a little bit how C++ macro's work.
With that example from libgba in my head I had started experimenting with replacing a couple of things in my own source code. And I think I will end up using it very extensively over the next few weeks to replace parts of the code that need to be configurable for certain platforms or other type of games that I want my engine to be used for. For example the MIPS CPU in the PS2 and PSP are very sensitive to memory alignment and I could simply write a macro function to allocate a block of memory having that macro be slightly different for these systems taking care of the alignment for me. That way I do not have to make ugly #ifdef's to check for certain platforms all over the code and only the platform drivers might need them! Making the code far more readable in the process.
Hopefully this is enough intro to give an idea of how I have solved the problem I was facing with the typedef's and writing it twice in reverse order! Since it's a long block of text and code I hide it away in it's own little spoiler block to not scare away anyone immediately
Finally we come to the bit where the magic is happening...sort of speak. If you look closely at how I define my structs now using the macro's you notice it's only specified a single time right? And not twice like it used to be in the first example from 2018. That has been my ultimate goal for a very long time!!! Even when I do not really like the uglyness of the code for the macro's being used....actual usage of them makes the code really clean and understandable I think!
It's the strange looking macro's called like "JJ2_eventConfig_x" with the a,b,c,d,...etc arguments where this double definition is actually solved and avoided at the same time! If you look closely they are the ones that are defined twice and wrapped in an #ifndef GEKKO condition, where "GEKKO" is the name of the CPU that's inside the GameCube but the value is also true when compiling for the Wii. The main difference between the two sets is when compiling for a non-gekko platform the a,b,c,d arguments are just placed like that in the order they logically are placed in. But when it is compiling for GEKKO the order of the arguments after expanding the macro is reversed!!!
I might need to replace all the GEKKO checks later actually to something more general that checks for any BIG-ENDIAN platform that requires the same reverse ordering of struct elements. It's not just for the Wii and GameCube but also if I might one day add ports for the WiiU, PS3 and XBOX360....as they all have the same PowerPC architecture. Not sure if they all run in BIG-ENDIAN though but the WiiU does. And also the motorola 68k that's inside the Sega Genesis/Mega Drive is Big-Endian if I remember correctly! Yeah I know...way to many platforms on my list but that's half the fun of my project for me!
Hopefully the wall of text above is somewhat understandable, even though I tried segmenting it with spoiler blocks a little. Just for those that did not understand any of it...or did not read it at all....I guess the short version would be: I have finally found a solution to hide a complex and confusing part of my sourcecode by using a technique called function macro's in C++. The resulting macro functions them self are not really easy to read and that's why I called it a "terrible solution"....but at the same time it prevents the "horrible problem" of having to write the same code twice when compiling the code for a Big-Endian architecture like the GameCube and Wii. And these two platform are important to me as that is what I have started this adventure on.
Thank you, and until next time
When starting out on the Wii at the end of 2018 I ran into a problem I had not ever really had while programming. I knew the term big endian en little endian and that it had something to do with how bytes are stored in memory in what order they are read by the CPU. It was actually in the exact same post where I released preview v0.1 that I also explained the endian issue
....just for convenience I will repeat it a little here:I have finaly fixed the endian bug for Wii and Gamecube! I found some helpfull info explaining how bitfields work differently on big-endian machines. For some reason not only the bytes are swapped but also the packing/ordering of the bitfields! To give an idea of what that means...and what I needed to understand it myself...here is an example for one of the objects
Now that is solved I can continue digging into the 3DS texture loading and the animated tiles that skip frames and such. In the mean time I also started looking into what I need for adding PSP and PS2 ports so I installed both devkits with docker. Only need some examples to see how to initialize these systems and access the input, gpu and storage...sound comes later.
Code:
#pragma pack(1)
typedef struct {
#ifndef GEKKO
uint8_t event_id:8;
uint8_t misc:4;
uint8_t trigger_id:6;
int8_t speed:5;
uint8_t length:4;
uint8_t duration:5;
#else
uint8_t duration:5;
uint8_t length:4;
int8_t speed:5;
uint8_t trigger_id:6;
uint8_t misc:4;
uint8_t event_id:8;
#endif
} Event47;
#pragma pack()Now that is solved I can continue digging into the 3DS texture loading and the animated tiles that skip frames and such. In the mean time I also started looking into what I need for adding PSP and PS2 ports so I installed both devkits with docker. Only need some examples to see how to initialize these systems and access the input, gpu and storage...sound comes later.
Now if this was the only struct that I would have to define it might not be so bad....but there are 33 event's in Batterycheck where some of them have 8 properties!! And those need to be specified TWICE in the same way as I explained above: One time for little-endian machines (x86, MIPS, ARM) and one time in big-endian machines (POWERPC, Motorola 68000?? I think ??) in reverse order! It's just confusing and easy to get wrong....and I was not really looking forward to implement the enormous list of Jazz Jackrabbit 2 events by hand!!! The original Jazz2 has nearly 253 events in total if I remember correctly, in any case it's a very big lot more than the 33 of Batterycheck.
Now to explain my "Terrible solution" I might have to explain a little bit how C++ macro's work.
To take a simple example from LIBGBA and LIBNDS they use macro's to setup the registers for the DMA capabilities of those machines. In simple terms DMA (DirectMemoryAccess) is a super fast way to copy memory from one location to another without the CPU being used to do it. And since the GBA and DS have kind of slow CPU's they make use of DMA for nearly everything. SO here is how the maco's are defined:
With that last function macro you can just do something like: DMA_Copy(2,0x00100,0x08100,0x00); (just filed in a few values for illustration only!). This tells the macro you want to use DMA Channel 2, and copy from location 0x00100 to 0x08100.....and while writing this I thought.....how does it know how many bytes you want to copy??? No idea as I have not used it myself just yet....and that is not the point here since I only want to use the example and where I got my idea for the "terrible solution" from
The advantage is that the registers that are defined above that function macro are in turn pointing to a specific memory address. Using those lables is a LOT easier than using the value behind them every time you might want to use the DMA controller. And using them should make the C++ compiler translate that into very efficient assembly code. Having the macro function defined instead of a regular function will have the compiler replace the "function call" into just those three register address being set to the given values. This might be just be directly translated into three instruction's...but I am not sure if that is actually happening. It could also first move the values into an internal CPU register and have that copied to the memory location.
Code:
#define REG_DMA0SAD *(vu32*)(REG_BASE + 0x0b0)
#define REG_DMA0DAD *(vu32*)(REG_BASE + 0x0b4)
#define REG_DMA0CNT *(vu32*)(REG_BASE + 0x0b8)
#define REG_DMA1SAD *(vu32*)(REG_BASE + 0x0bc)
#define REG_DMA1DAD *(vu32*)(REG_BASE + 0x0c0)
#define REG_DMA1CNT *(vu32*)(REG_BASE + 0x0c4)
#define REG_DMA2SAD *(vu32*)(REG_BASE + 0x0c8)
#define REG_DMA2DAD *(vu32*)(REG_BASE + 0x0cc)
#define REG_DMA2CNT *(vu32*)(REG_BASE + 0x0d0)
#define REG_DMA3SAD *(vu32*)(REG_BASE + 0x0d4)
#define REG_DMA3DAD *(vu32*)(REG_BASE + 0x0d8)
#define REG_DMA3CNT *(vu32*)(REG_BASE + 0x0dc)
#define DMA_Copy(channel, source, dest, mode) {\
REG_DMA##channel##SAD = (u32)(source);\
REG_DMA##channel##DAD = (u32)(dest);\
REG_DMA##channel##CNT = DMA_ENABLE | (mode); \
}With that last function macro you can just do something like: DMA_Copy(2,0x00100,0x08100,0x00); (just filed in a few values for illustration only!). This tells the macro you want to use DMA Channel 2, and copy from location 0x00100 to 0x08100.....and while writing this I thought.....how does it know how many bytes you want to copy??? No idea as I have not used it myself just yet
....and that is not the point here since I only want to use the example and where I got my idea for the "terrible solution" fromThe advantage is that the registers that are defined above that function macro are in turn pointing to a specific memory address. Using those lables is a LOT easier than using the value behind them every time you might want to use the DMA controller. And using them should make the C++ compiler translate that into very efficient assembly code. Having the macro function defined instead of a regular function will have the compiler replace the "function call" into just those three register address being set to the given values. This might be just be directly translated into three instruction's...but I am not sure if that is actually happening. It could also first move the values into an internal CPU register and have that copied to the memory location.
With that example from libgba in my head I had started experimenting with replacing a couple of things in my own source code. And I think I will end up using it very extensively over the next few weeks to replace parts of the code that need to be configurable for certain platforms or other type of games that I want my engine to be used for. For example the MIPS CPU in the PS2 and PSP are very sensitive to memory alignment and I could simply write a macro function to allocate a block of memory having that macro be slightly different for these systems taking care of the alignment for me. That way I do not have to make ugly #ifdef's to check for certain platforms all over the code and only the platform drivers might need them! Making the code far more readable in the process.
Hopefully this is enough intro to give an idea of how I have solved the problem I was facing with the typedef's and writing it twice in reverse order! Since it's a long block of text and code I hide it away in it's own little spoiler block to not scare away anyone immediately
Code:
#define PRAGMA(X) _Pragma(#X)
#ifndef GEKKO
#define JJ2_eventConfig_2(a,b) a b
#define JJ2_eventConfig_3(a,b,c) a b c
#define JJ2_eventConfig_4(a,b,c,d) a b c d
#define JJ2_eventConfig_5(a,b,c,d,e) a b c d e
#define JJ2_eventConfig_6(a,b,c,d,e,f) a b c d e f
#define JJ2_eventConfig_7(a,b,c,d,e,f,g) a b c d e f g
#define JJ2_eventConfig_8(a,b,c,d,e,f,g,h) a b c d e f g h
#else
#define JJ2_eventConfig_2(a,b) b a
#define JJ2_eventConfig_3(a,b,c) c b a
#define JJ2_eventConfig_4(a,b,c,d) d c b a
#define JJ2_eventConfig_5(a,b,c,d,e) e d c b a
#define JJ2_eventConfig_6(a,b,c,d,e,f) f e d c b a
#define JJ2_eventConfig_7(a,b,c,d,e,f,g) g f e d c b a
#define JJ2_eventConfig_8(a,b,c,d,e,f,g,h) h g f e d c b a
#endif
#define JJ2___typedef_struct(id,fields) \
PRAGMA(pack(1)) \
typedef struct { \
fields \
} Event##id;
#define JAZZ2_EVENT_STRUCT___2(id, a, b) JJ2___typedef_struct(id, JJ2_eventConfig_2(a,b); );
#define JAZZ2_EVENT_STRUCT___3(id, a, b, c) JJ2___typedef_struct(id, JJ2_eventConfig_3(a,b,c); );
#define JAZZ2_EVENT_STRUCT___4(id, a, b, c, d) JJ2___typedef_struct(id, JJ2_eventConfig_4(a,b,c,d); );
#define JAZZ2_EVENT_STRUCT___5(id, a, b, c, d, e) JJ2___typedef_struct(id, JJ2_eventConfig_5(a,b,c,d,e); );
#define JAZZ2_EVENT_STRUCT___6(id, a, b, c, d, e, f) JJ2___typedef_struct(id, JJ2_eventConfig_6(a,b,c,d,e,f); );
#define JAZZ2_EVENT_STRUCT___7(id, a, b, c, d, e, f, g) JJ2___typedef_struct(id, JJ2_eventConfig_7(a,b,c,d,e,f,g); );
#define JAZZ2_EVENT_STRUCT___8(id, a, b, c, d, e, f, g, h) JJ2___typedef_struct(id, JJ2_eventConfig_8(a,b,c,d,e,f,g,h); );
#define _u8(name,size) uint8_t name:size;
#define _u16(name,size) uint16_t name:size;
#define _u32(name,size) uint32_t name:size;
#define EVENT__1__ONE_WAY 1
#define EVENT__17__END_OF_LEVEL 17
#define EVENT__29__START_POSITION 29
#define EVENT__33__BATTERY_HOLDER 33
#define EVENT__35__LIFERAFT 35
#define EVENT__38__ELEVATOR 38
//left the rest of them out of this example ;)
//======================================================================================================================
JAZZ2_EVENT_STRUCT___3(
EVENT__1__ONE_WAY,
_u8 ( event_id , 8 ),
_u8 ( misc , 4 ),
_u32 ( unused , 20 )
);
Code:
//======================================================================================================================
JAZZ2_EVENT_STRUCT___3(
EVENT__17__END_OF_LEVEL,
_u8 ( event_id , 8 ),
_u8 ( misc , 4 ),
_u32 ( unused , 20 )
);
//======================================================================================================================
JAZZ2_EVENT_STRUCT___3(
EVENT__29__START_POSITION,
_u8 ( event_id , 8 ),
_u8 ( misc , 4 ),
_u32 ( unused , 20 )
);
//======================================================================================================================
JAZZ2_EVENT_STRUCT___5(
EVENT__33__BATTERY_HOLDER,
_u8 ( event_id , 8 ),
_u8 ( misc , 4 ),
_u8 ( trigger_id , 6 ),
_u8 ( direction , 1 ),
_u16 ( unused , 13 )
);
//======================================================================================================================
JAZZ2_EVENT_STRUCT___4(
EVENT__35__LIFERAFT,
_u8 ( event_id , 8 ),
_u8 ( misc , 4 ),
_u8 ( water_id , 6 ),
_u16 ( unused , 14 )
);
/======================================================================================================================
JAZZ2_EVENT_STRUCT___8(
EVENT__38__ELEVATOR,
_u8 ( event_id , 8 ),
_u8 ( misc , 4 ),
_u8 ( trigger_id , 6 ),
_u8 ( moves , 3 ),
_u8 ( direction , 3 ),
_u8 ( speed , 4 ),
_u8 ( _moves , 3 ),
_u8 ( unused , 3 )
);I don't blame anyone if you don't have a clue about what any of this stuff is doing here. It's also a little complex for me to explain since macro's are kind of new to me aswell....but I'll try anyway. As a basic wrapper there is the JAZZ2_EVENT_STRUCT macro that comes in a couple of variations to allow for a different number of elements inside. There are ways to detect it but they nearly triple the amount of helper macro's and I think this is already bad enough as it is.
The first parameter is the EventID that comes from a list of define's I made to have an identifier that includes both the event ID and a descriptive label all in one. I think it explains itself already but just in case tight. While you might think that the "event_id" is also a property already...so why do I specify it here too? Well for that you will have to wait a little longer. Then there is always the "misc" field that has 4 bits with a few options....forgot what they were but it's explained in the JJ2 file format documentation I am sure
Most of the struct's in my example here have a field called "unused" since every config value is a fixed 4 byte word and I need/want all my structs to be exactly 32bit's in sice for that reason. I had to manually count how many of them were unused for each event though....also something I am not really looking forward doing for Jazz2 yet....but that's far in the future so let's not worry about it yet. It's actually only the last three structs that have more properties and as you can see the elevator has a lot of them!!
You might have noticed I also have three shortcut macro's to define 8, 16, or 32 bit access into the bitfields that also allow me to specify it's name and number of bit's to use. They are really just wrappers that expand into the correct syntax to use for defining those elements in a struct. The result of those wrapper macro's are passed on to the JAZZ2_EVENT_STRUCT() macro and depending on the number of elements from 2-8 a differently named version is used.
So looking at this JAZZ2_EVENT_STRUCT wrapper it sends it's parameters to another wrapper macro "JJ2___typedef_struct" that defines the actual struct definition! I am not really happy with how the macro's are named yet and I might change that later but for now this is what it is. If you look at this struct generator macro it's not really complicated but this is where the EventID is actually used right there at the end of it! The numeric ID is appended to generate the name of the struct. So for the elevator that would be "Event38"! I might also end up with a different naming convention for these structs but for now I leave it as it is.
As a basic wrapper there is the JAZZ2_EVENT_STRUCT macro that comes in a couple of variations to allow for a different number of elements inside. There are ways to detect it but they nearly triple the amount of helper macro's and I think this is already bad enough as it is.The first parameter is the EventID that comes from a list of define's I made to have an identifier that includes both the event ID and a descriptive label all in one. I think it explains itself already but just in case tight.
While you might think that the "event_id" is also a property already...so why do I specify it here too? Well for that you will have to wait a little longer. Then there is always the "misc" field that has 4 bits with a few options....forgot what they were but it's explained in the JJ2 file format documentation I am sureMost of the struct's in my example here have a field called "unused" since every config value is a fixed 4 byte word and I need/want all my structs to be exactly 32bit's in sice for that reason. I had to manually count how many of them were unused for each event though....also something I am not really looking forward doing for Jazz2 yet....but that's far in the future so let's not worry about it yet. It's actually only the last three structs that have more properties and as you can see the elevator has a lot of them!!
You might have noticed I also have three shortcut macro's to define 8, 16, or 32 bit access into the bitfields that also allow me to specify it's name and number of bit's to use. They are really just wrappers that expand into the correct syntax to use for defining those elements in a struct. The result of those wrapper macro's are passed on to the JAZZ2_EVENT_STRUCT() macro and depending on the number of elements from 2-8 a differently named version is used.
So looking at this JAZZ2_EVENT_STRUCT wrapper it sends it's parameters to another wrapper macro "JJ2___typedef_struct" that defines the actual struct definition! I am not really happy with how the macro's are named yet and I might change that later but for now this is what it is. If you look at this struct generator macro it's not really complicated but this is where the EventID is actually used right there at the end of it! The numeric ID is appended to generate the name of the struct. So for the elevator that would be "Event38"! I might also end up with a different naming convention for these structs but for now I leave it as it is.
Finally we come to the bit where the magic is happening...sort of speak
. If you look closely at how I define my structs now using the macro's you notice it's only specified a single time right? And not twice like it used to be in the first example from 2018. That has been my ultimate goal for a very long time!!! Even when I do not really like the uglyness of the code for the macro's being used....actual usage of them makes the code really clean and understandable I think! It's the strange looking macro's called like "JJ2_eventConfig_x" with the a,b,c,d,...etc arguments where this double definition is actually solved and avoided at the same time! If you look closely they are the ones that are defined twice and wrapped in an #ifndef GEKKO condition, where "GEKKO" is the name of the CPU that's inside the GameCube but the value is also true when compiling for the Wii. The main difference between the two sets is when compiling for a non-gekko platform the a,b,c,d arguments are just placed like that in the order they logically are placed in. But when it is compiling for GEKKO the order of the arguments after expanding the macro is reversed!!!
I might need to replace all the GEKKO checks later actually to something more general that checks for any BIG-ENDIAN platform that requires the same reverse ordering of struct elements. It's not just for the Wii and GameCube but also if I might one day add ports for the WiiU, PS3 and XBOX360....as they all have the same PowerPC architecture. Not sure if they all run in BIG-ENDIAN though but the WiiU does. And also the motorola 68k that's inside the Sega Genesis/Mega Drive is Big-Endian if I remember correctly! Yeah I know...way to many platforms on my list but that's half the fun of my project for me!
Hopefully the wall of text above is somewhat understandable, even though I tried segmenting it with spoiler blocks a little
. Just for those that did not understand any of it...or did not read it at all....I guess the short version would be: I have finally found a solution to hide a complex and confusing part of my sourcecode by using a technique called function macro's in C++. The resulting macro functions them self are not really easy to read and that's why I called it a "terrible solution"....but at the same time it prevents the "horrible problem" of having to write the same code twice when compiling the code for a Big-Endian architecture like the GameCube and Wii. And these two platform are important to me as that is what I have started this adventure on. Thank you, and until next time
