FATPlow: A fix for slow loading times for DSi's with huge SD cards.

[BerelyBecky]

Howdy y'all

If you are using a very large SD card in your DSi, you may have noticed a tendency for first boots of a game to be very slow. This happens because it has to create new files, and in order to do that it needs to find free clusters on the FAT32 file system of your SD card. To find free clusters, the search begins from the very beginning of the file system, checking one cluster after the other, until it hits a free one. The more filled your SD card is, the longer this search will take. FAT32 does provide a way to speed this up by remembering where the last cluster is each time it writes a new file, but it seems this isn't implemented on TWI/ndsbootstrap.

However, there is a solution to this problem! We can simply move all our data to clusters at the end of the SD card, and leave all the room at the start free, so that the DSi can immediately find the clusters it needs. I've written a tool that does just that, and it seems reasonably stable that I feel comfortable sharing it. It's a command line tool and I've only been able to make a Linux build, but if anyone else wants to port it to windows, do feel free!

https://github.com/MerelyMezz/fatplow

! WARNING WARNING WARNING !
Using this tool directly modifies the underlying data of your file system. It is possible that the data on your SD card may be destroyed, so make sure to make a backup first, so that you can restore it later after reformatting.

How to use:
After downloading the binary make sure it has executable permissions. Make sure your SD card is not mounted, then run in the terminal:

./fatplow [Filename of your SD card block device] move-clusters-to-end

 

[Coderkei]

Would this tool need to be re-run any time changes are made to the files on the SD card?

 

[ber71]

This is something I noted on the wii. Backups takes a long time to load, because of some disk writes here and there, and a FAT table size too large for the limited wii RAM.

The workaround I found was:

Move the roms to another disk, in order to get it empty.
Copy some large dummy files to the disk. For example, a movie collection.
Restore the roms.
Remove the dummy files.
The disk has now a lot of space at the beginning and games are launching fast.

 

[BerelyBecky]

Would this tool need to be re-run any time changes are made to the files on the SD card?

Not necessarily. New files may probably be written to the start clusters again, but unless it's such a big amount that it slows things down again, you probably don't need to run it again. The tool does come with a subcommand "print-cluster-use" to inspect the cluster usage, if you wanna know whether running it might make sense again.

This is something I noted on the wii. Backups takes a long time to load, because of some disk writes here and there, and a FAT table size too large for the limited wii RAM.

The workaround I found was:

Move the roms to another disk, in order to get it empty.
Copy some large dummy files to the disk. For example, a movie collection.
Restore the roms.
Remove the dummy files.
The disk has now a lot of space at the beginning and games are launching fast.

This is another valid way of solving the problem. It works best on a freshly formatted system, because the Nxt_Free hint will be pointing to the start at the file system. If the file system has been in use for a while, the hint may be pointing at somewhere in the middle, so your dummy file isn't guaranteed to be allocated where you want. Though I think this depends very much on what system you are using and how it treats Nxt_Free.

 

[master801]

Doing this is going to degrade the SD card even quicker, forewarning to anyone out there.

This is essentially defragmenting but done in reverse.

 

[BerelyBecky]

Doing this is going to degrade the SD card even quicker, forewarning to anyone out there.

This is essentially defragmenting but done in reverse.

It's not really defragmenting. The tool does not particularly care about keeping clusters contiguous, it just moves them to the first free cluster near the end. Each cluster is moved at most once. Also, if your card is filled over half, most of the data won't even be rewritten anyway. Clusters near the beginning get moved first, so if the free space at the end is filled up, all the remaining files simply stay where they are.

 

[Evilengine]

Interesting. I guess "huge SD card" starts at 64gb?

 

[Apache Thunder]

I'm curious if this has any impact on 3DS. I do recall 3DS consoles have longer boot times on large mSD cards. Anything past 32GB seems to have a noticeable impact on boot times.

Anyways it seems this is a Linux program? (the github release page just has an elf file.....)

You should think about making a Windows build...though perhaps it might be simpler to make a NDS version you can run on the DSi console itself. (though I suppose that'd take longer then doing it on PC. )

 

[BerelyBecky]

Interesting. I guess "huge SD card" starts at 64gb?

I've been using a 512GB card, which when filled would take up to 3 minutes to boot, oof.

I'm curious if this has any impact on 3DS. I do recall 3DS consoles have longer boot times on large mSD cards. Anything past 32GB seems to have a noticeable impact on boot times.

Anyways it seems this is a Linux program? (the github release page just has an elf file.....)

You should think about making a Windows build...though perhaps it might be simpler to make a NDS version you can run on the DSi console itself. (though I suppose that'd take longer then doing it on PC. )

I don't have a 3DS, so I can't really say for sure, but I wouldn't be surprised if it suffered from the same flaw.

Yea, I only provided a Linux build, because I don't have a Windows system and don't know how you would even access the SD card on there. The source is right there if anyone wants to port it to windows. Maybe WSL can run it?

 

[redunka]

For long boot times on 3DS there's this experimental patch: https://gbatemp.net/threads/beta-re...p-boot-speed-on-high-capacity-sd-card.673415/

 

[Apache Thunder]

Oh neat. I'll give that a try....once it no longer has that nag screen. I tested it and the warning message comes up after every boot...I understand the need to warn people since it's still in testing but would have been nice to make that warning only appear once...