Hacking Does filesystem access just completely suck on the DS2?

[Deleted member 319809]

Alright. I'm coding a "sequel" to DSCompress, called DS2Compress, and it chokes on so many files it's just not funny.

I am using sources provided by BassAceGold in the "libds2a.a unofficial" thread - it was a 404, but he may have updated the link recently after I requested the sources. In the thread he states that there's some fcntl support, some unistd support, a more recent working zlib and a few more odds and ends. However it doesn't fix the problem I'm encountering, which is also present in the original official SDK (0.13) and the improved official SDK (1.2).

When extracting files that have no extension, such as README and CHANGELOG, the short name becomes corrupt and contains a NUL. fsck on Linux complains that README is incorrectly called README.<space><space><space>\000 and core-dumps while attempting to fix it. DS2Compress itself also crashes hard when asked to display the file name in the file selector.

When compressing or extracting files that have a long file name (>8+3), such as 0_Pokemon HG.RTS or 0_Pokemon HG.RTS.gz, the destination file is sometimes written as 0_Pokemo䊋 -- 0_Pokemo[Unicode U+428B] -- or similar, 0 bytes, just beside the real file, written correctly. The file is ignored by the file selector, but appears as a truncated filename on Linux and possibly messes with Windows and Mac OS. fsck on Linux then complains that two files use the same short name, which in this example would look like 0POKE~02.RTS.

What gives? And can we try to fix this somehow?

If you guys need any "client code" that uses the file access functions, I can post my DS2Compress sources early, but there is already such a problem in CATSFC when writing saved states. Regardless I'll post my sources because it's based on many GPLv2 things.

 

[BassAceGold]

Have you tried a different SD card to rule out any fat corruption? Does this problem exist using standard fat functions outside of your extraction code (Ruling out any problems within zlib)?

 

[Deleted member 319809]

Well, the zip extraction code directly uses

/* given */ char outfile [MAX_PATH + 1];
fopen(outfile, "wb");

which is mapped to fat_fopen via the #define in stdio.h, so I think it's a correct call.
* For gzip compression, the input file is opened, its name unmodified, with fopen. The output file is opened, after using

strcat(outfile, ".gz");

, with gzopen.
* For gzip decompression, the input file is opened, its name unmodified, with gzopen. The output file is opened, after using

outfile[strlen(outfile) - strlen(".gz")] = '\0';

, with fopen.
* For zip decompression, the input file is opened, its name unmodified, with unzOpen (source/unzip/*.c from CATSFC has this too). Output files are opened with fopen, their names relative to the archive's directory generated with:

char Path [MAX_PATH + 1];
strcpy(Path, infile);
char* slash = strrchr(Path, '/');
if (!slash)
    return EINVAL; // or so
else
    *slash = '\0'; // Remove from the last slash onwards, including the slash
char outfile [MAX_PATH + 1]; // for each file
strcpy(outfile, Path);
strcat(outfile, "/");
strcat(outfile, archive_member.filename);

I've tested with the 128 MB card, after reformatting it cleanly with the size adjustment (1 MB at the start of the card is unusable for the first partition) and FAT16, then putting test files on the card with a computer. Test files were .gz, .zip and uncompressed, and it messed up during both compression and decompression.