Homebrew Textures block format

[cebolleto]

For a few days I have been trying to load textures into memory without luck

Looking into the code of 3dscraft I found a script named texconv.py which seems to do the job. I have translated that into C++ code and it seems to work, but there are a few issues:
- Textures are upside down
- Only square textures seems to be loaded properly

Could anyone explain how textures are supposed to be stored into memory?
Also just for curiosity I'd like to understand why the orders of pixels in the GPU is this weird...

Thanks!

 

[elhobbs]

For a few days I have been trying to load textures into memory without luck

Looking into the code of 3dscraft I found a script named texconv.py which seems to do the job. I have translated that into C++ code and it seems to work, but there are a few issues:
- Textures are upside down
- Only square textures seems to be loaded properly

Could anyone explain how textures are supposed to be stored into memory?
Also just for curiosity I'd like to understand why the orders of pixels in the GPU is this weird...

Thanks!

the width and height appear to be reversed in libctru. That still leaves stuff upside down. I was wondering if that has something to do with the rotated frame buffer.

 

[cebolleto]

Thanks, that almost did the job... I am still having issues with some resolutions
Does anyone know why the 3ds uses this format? I imagine it adds some kind of optimization but cannot image what...

 

[elhobbs]

Thanks, that almost did the job... I am still having issues with some resolutions
Does anyone know why the 3ds uses this format? I imagine it adds some kind of optimization but cannot image what...

The dimensions need to be powers of two. I have not had issues within those constraints. If you are referring to the block format I think I saw something somewhere about increasing cache hits when reading the texture.

 

[themperror]

For a few days I have been trying to load textures into memory without luck

Looking into the code of 3dscraft I found a script named texconv.py which seems to do the job. I have translated that into C++ code and it seems to work, but there are a few issues:
- Textures are upside down
- Only square textures seems to be loaded properly

Could anyone explain how textures are supposed to be stored into memory?
Also just for curiosity I'd like to understand why the orders of pixels in the GPU is this weird...

Thanks!

Textures should always be a power of 2 in any dimension, 8x8, 16x16,32x32,64x64,128x128, 256x256, 512x512 (higher probably won't be needed on the 3ds (also considering speed)), in OpenGL and DirectX they CAN be rectangular, in the old days they had to be square though.. so 512x256 was allowed, But I don't know if it's allowed on the 3ds.

Textures are normally loaded in VRAM, in case of CPU rendering it's just a block of linear memory. so an array with all pixels in there will suffice.

 

[neobrain]

Textures are stored upside-down in memory indeed, i.e. the v=1 coordinate corresponds to the first data row in memory at the texture source address, whereas the v=0 coordinate corresponds to the last row in memory.

The same applies to framebuffers, fyi.

 

[minexew]

Does anyone know why the 3ds uses this format? I imagine it adds some kind of optimization but cannot image what...

It's called the "native PICA format" and it either significantly simplifies the hardware design or helps a lot with cache utilization and thus performance (2nd possibility seems more likely IMO because the PICA uses some kind of block-based rendering).
Can't think of any other reason why they would use it.

 

[neobrain]

The block reordering is used to achieve better memory access locality, i.e. better cache hit rate. Cf. http://en.wikipedia.org/wiki/Z-order_curve .

 

[minexew]

The block reordering is used to achieve better memory access locality, i.e. better cache hit rate. Cf. http://en.wikipedia.org/wiki/Z-order_curve .

That's a really good article, by the way

 

[elhobbs]

Textures should always be a power of 2 in any dimension, 8x8, 16x16,32x32,64x64,128x128, 256x256, 512x512 (higher probably won't be needed on the 3ds (also considering speed)), in OpenGL and DirectX they CAN be rectangular, in the old days they had to be square though.. so 512x256 was allowed, But I don't know if it's allowed on the 3ds.

Textures are normally loaded in VRAM, in case of CPU rendering it's just a block of linear memory. so an array with all pixels in there will suffice.

Rectangle textures are allowed. My experience has been that GPU_SetTexture has the width and the height reversed.

 

[cebolleto]

The block reordering is used to achieve better memory access locality, i.e. better cache hit rate. Cf. http://en.wikipedia.org/wiki/Z-order_curve .

Thanks a lot. This is exactly what I was looking for

 

[cebolleto]

Sorry I am still having problems with this... I can load square textures and rectangle textures but only if theis high is bigger then their width. When the width is bigger (64x32 for example) it doesn't look ok

Could anyone post the C++ code? That would be very helpful

 

[elhobbs]

I am sure this could be optimized but it take an 8 bit source and converts it to 16 bit. But it does show the proper tile ordering. It also flips the texture vertically. There is also a fair bit of dead code in there. Primarily for cross compiling for windows. I can confirm that it works with wide and narrow textures. There is a scale call but it just ensures power of 2 dimensions.

https://github.com/elhobbs/spectre3ds/blob/master/source/sys_textures.cpp#L329

 

[cebolleto]

I have been taking a look, but still the same problem. It doesn't seem to work with textures that are 32x64 or 64x32

 

[cebolleto]

OK... so after swapping width and height in parseTileTrans16 everything seems to work!!! Thanks a lot!!

 

[elhobbs]

OK... so after swapping width and height in parseTileTrans16 everything seems to work!!! Thanks a lot!!

The width height swap was mentioned earlier in the thread. It appears that ctrulib has them backwards swapped.

 

[cebolleto]

Yes, I know... I was already swapping the width and height when calling GPU_SetTexture. The weird thing is that I also need to do it when calling parseTileTrans16 which is not what you had there

 

[elhobbs]

Yes, I know... I was already swapping the width and height when calling GPU_SetTexture. The weird thing is that I also need to do it when calling parseTileTrans16 which is not what you had there

That is bizarre. I do not do that and it works fine for me. I am curious. Do you have any code that I could look at to see how you are using this?

 

[cebolleto]

No, yeah... you were right... after cleaning my code a bit it turned out I was swapping width and height on a previous call and so I had to swap again
Now everything makes sense

Thanks a lot for your help elhobbs. Now I can continue working on this

 

[cebolleto]

Sorry, I forgot to post the final code for this for anyone who can need it

void ReorderImageData(u8* src, u8* palette, u8* dst, int width, int height, void(copyFunc)(u8*, u8*, int, u8*, int))
{
    static int tile_order[] = {
         0,  1,  8,  9,  2,  3, 10, 11,
        16, 17, 24, 25, 18, 19, 26, 27,
         4,  5, 12, 13,  6,  7, 14, 15,
        20, 21, 28, 29, 22, 23, 30, 31,

        32, 33, 40, 41, 34, 35, 42, 43,
        48, 49, 56, 57, 50, 51, 58, 59,
        36, 37, 44, 45, 38, 39, 46, 47,
        52, 53, 60, 61, 54, 55, 62, 63
    };


    int idx = 0;
    int i, j;
    for(int y = 0; y < height; y += 8)
    {
        for(int x = 0; x < width; x += 8)
        {
            for(int k = 0; k < 64; ++ k)
            {
                i = (tile_order[k] % 8);
                j = (tile_order[k] - i) / 8;

                int src_idx = (height - (y + j) - 1) * width + (x + i);

                copyFunc(src, palette, src_idx, dst, idx);
       
                idx ++;
            }
        }
    }
}

where copyFunc is a pointer to a function similiar to these

void copyRGB(u8* src, u8* palette, int src_idx, u8* dst, int dst_idx)
{
    dst[dst_idx * 3 + 0] = src[src_idx * 3 + 2];
    dst[dst_idx * 3 + 1] = src[src_idx * 3 + 1];
    dst[dst_idx * 3 + 2] = src[src_idx * 3 + 0];
}

void copyRGBA(u8* src, u8* palette, int src_idx, u8* dst, int dst_idx)
{
    dst[dst_idx * 4 + 0] = src[src_idx * 4 + 3];
    dst[dst_idx * 4 + 1] = src[src_idx * 4 + 2];
    dst[dst_idx * 4 + 2] = src[src_idx * 4 + 1];
    dst[dst_idx * 4 + 3] = src[src_idx * 4 + 0];
}