Hi at all,

it is a well known fact (at least since Samsung's infamous 840 EVO scandal) that data on NAND is volatile. That means it gets harder for a controller to read "old" data.
That is the case for our SD cards in nintendo switch consoles. Last year in October 2022 I bought a large 512GB SanDisk Ultra card (NO FAKE) for 40 bucks. It worked well. Until I updated Mario Kart 8 to its last update a few days ago and was astonished by the long track loading times (40 secs). On my other switch (512GB Samsung drive) there was 9 second loading.
So I switched cards to make sure it was not the SD reader or something else. Same result. Then I refreshed the data on the faulty card (DiskFresh freeware tool by Pugan Software). Of course you can copy your data, then quick format and paste your data back which has the exact same effect: Every sector is written again.
It took me a whole day to refresh the sectors. After that the card was as fast as usual. It maxes out the switch's SD reader for sure. Since I have come by several threads complaining about slow SD cards and those being fake... this is NOT always the case as shown here. simply refresh your data once in a while and it restores your SD card speeds to maximum.

tl;dr:

Not only things in your refrigerator get old as time goes by. DATA GETS OLD AND SLOW. Please refresh your data once in a while. Since we all install our games once and never write data to that spot ever again, it gets slow over time. This is not a huge deal for SSDs (SATA or Nvme, since those have firmware which handle refreshing data accordingly). But SD cards are dumb and the switch obviously does not manage data to keep it fresh.


Best wishes!

 

[Stellar]

Edit: after a cursory read on bitrot, OP is probably right.

 

[Nynrah]

Edit: after a cursory read on bitrot, OP is probably right.

I'm aware of bitrot, but while refreshing an SD card is not difficult to do, there's one thing I don't understand and what the OP just doesn't seem to respond to. What about embedded memory like eMMC? Why does the firmware of a device (e.g. your 10-year-old 3DS for example) last as long as it does? I think this is a fair point that @Nazosan and others raised. I've already tried to ask it twice, but the OP hasn't seen fit to address this point yet.

If it IS that bad, it is a huge problem. SD cards can be refreshed, but you can't refresh firmware just like that. Is it pure coincidence that firmware hasn't bitrotted yet because of a firmware update every now and then? Is there something else going on here (in the case of eMMC, like in a Wii U, it is basically just an embedded SD card). What's the point of it all if your SD card survives while your device rots away at the same pace? What can even be done if it's this bad?

I am inclined to believe the folks who disparage the OP's alarmistic message unless someone can provide a solid explanation to the contrary.

 

[Stellar]

I don't know if eMMC suffers from the same issues, but the simple fact that we write to it often means it will die eventually. If you have a V1 Switch you should back up your NAND and if the worst happens, you can literally just boot into emuNAND out of an SD card made from your NAND backup.

 

[ghjfdtg]

I mentioned it earlier but eMMC is optimized for the use case as internal storage. Good OSs will even regularly run trim/erase commands on them to keep them healthy and fast. SD cards are seen as disposable and removable short time storage as bad as that may sound. eMMC data loss is catastrophic in most cases. SD card data loss is not because you are supposed to make backups and it's replaceable.

 

[SkullHex2]

What would be the safest way to refresh the microSD? Alternatives are:

• Hekate's UMS, possibly stressing out the card reader (you don't usually write the entire microSD all at once) and the screen (could be an issue for OLED models)
• take out the microSD, and we all know how touchy the card reader is

 

[AnthonyXeno]

How do you "Refresh the Data"? can't say i've heard of that before

 

[FarmYard-Gaming]

I've only ever had one SD card fail on me so far (mind, it wasn't a fun time), but I can understand the concern behind it all, especially without cloud saves

 

[PityOnU]

Before anyone gets too freaked out here, let's just refocus the conversation from "regularly perform labor intensive and high-wear maintenance on your hardware" to just "don't trust SD cards."

--------------------------------------------------

Here is just a brief aside to frame the context of the conversation and my above comment. I have my Ph.D. in computer engineering, have worked at Intel and Microsoft Xbox, and have many colleagues whose work primarily involves designing controllers for flash memory. I say this so you know that these are at least relatively informed generalizations based on my experience and background. Specifics may vary, though, and I'm not going to write a research paper here on GBATemp at 9:00AM, so please, do your own due diligence w.r.t. your hardware of choice.

Most people on these forums haven't really lived long enough to notice/be impacted by this yet, but yes, all physical media eventually wears out and breaks. Cassettes, CD's, HDD's, SSD's, vinyls, tapes... everything. Even if entirely unused and unplugged, data gets corrupted, motors seize up, things just plain wear out. You can throw around terms like "bit rot" and the rest, but at the end of the day, the reality is simply that nothing lasts forever. The only thing different from one technology to another is the timescales at which this occurs.

With regard to timescales, different storage media and target customer (home use vs. professional vs. enterprise) do affect what a particular piece of hardware is designed (not guaranteed) to achieve. Typically larger, simpler, and more rugged (i.e. enterprise) storage mediums will last the longest, but they are also almost always less performant and far more expensive than their more consumer-focused equivalents.

NAND (flash) memory is an interesting medium in that it can actually be VERY reliable and safe for data storage when compared to others just from the fact that it essentially has no moving parts (at least, above the quantum scale). However, over the years (as with all other storage mediums, as well), we have squeezed more and more space and performance out of the technology by pushing it to its absolute limits. The way we store data on flash is actually extremely fragile now, and in response to this, modern flash controllers are actually extremely complicated. However, this is something that has evolved slowly over a number of years, so things usually end up working fine. Usually. We are now a long way away from the numerous, buggy growing pains that were a thing when SSD's were first becoming popular.

Most serious flash storage mediums, whether you realize it or not, are also over-provisioned. As a simple example, a 1TB SSD may actually have 1.25TB of flash memory on the drive. Your computer can only access 1TB of it because the controller on the card tracks the health of the storage on the drive and will automatically use parts of the extra 0.25TB when parts of the 1TB (expectedly!) start to break down. You can also do a lot of interesting tricks with this spare 0.25TB to increase performance of the drive.

To circle the conversation back around the SD cards...

--------------------------------------------------

SD cards are just about the most fragile, error-prone, low-lifespan, consumer-grade storage medium you can possibly get your hands on. They are not meant to be anything other than cheap, tiny, and convenient. Never ever put any data on an SD card that you actually care about and want to stick around. As other users have said, make backups elsewhere!

With regard to read performance degradation over time: I think that's going to be an extremely YMMV issue that's more related to the controller on the card (as well as how it is being used) than is being suggested here. While I'm not disputing the OP's (academically interesting) experience here, I would be very reluctant to start making blanket recommendations that everyone rewrite their SD cards once every few months. That's really not something that should be generally necessary for preserving data. My personal guess here is that the issue OP experienced was more likely caused by influences at the OS level (e.g. data fragmentation and then performance throttling) than it was by "bit rot," especially after only a handful of months.

 

[ghjfdtg]

Exactly. Format your SD card correctly and keep using it. Maybe once or twice a year try and read out all the data. The controller will correct errors it encounters. Rewriting regularly will just wear out the flash even faster.

edit:
I almost forgot. If you already read out all the data you can combine this with making backups. You should make backups. "No backup, no pity!"

 

[Nynrah]

Exactly. Format your SD card correctly and keep using it. Maybe once or twice a year try and read out all the data. The controller will correct errors it encounters. Rewriting regularly will just wear out the flash even faster.

edit:
I almost forgot. If you already read out all the data you can combine this with making backups. You should make backups. "No backup, no pity!"

By reading out you mean just plugging the thing in so it's powered or use a particular tool (Winows here, by the way)?

 

[ghjfdtg]

Just do backups of all the data. That will ensure the controller will try to access all the used blocks and do error correction. If something happens to corrupt anyway (for example because of the shitty driver in HOS) you can always restore/reformat with the backups you made.

I mentioned it earlier but using fstrim on SD cards can also help to restore performance and help with wear leveling. Unfortunately almost no device makes use of it by default. You can use fstrim booting to Linux or Android on your Switch.

 

[PityOnU]

By reading out you mean just plugging the thing in so it's powered or use a particular tool (Winows here, by the way)?

Just do backups of all the data. That will ensure the controller will try to access all the used blocks and do error correction. If something happens to corrupt anyway (for example because of the shitty driver in HOS) you can always restore/reformat with the backups you made.

I mentioned it earlier but using fstrim on SD cards can also help to restore performance and help with wear leveling. Unfortunately almost no device makes use of it by default. You can use fstrim booting to Linux or Android on your Switch.

What @ghjfdtg is suggesting here is that, during the process of accessing the data you have stored, the controller for whatever storage medium you are using may go through the trouble of correcting and re-writing potentially problematic data chunks.

I say may here because there is no guarantee that this is the case, and depends entirely on the complexity/implementation details of the controller for whatever storage device you are using. There is no way to tell for sure if this happens or not, and there may be cases where not doing this would be preferable (maximizing read performance, for e.g.).

There are cases where just "plugging in" your device may be enough to have optimizations run on it. Windows, for example, will automatically run TRIM and TRIM-like operations on connected devices when they are not actively under use and the system is idle (determined based on a number of different factors). But again, there is no real way to (easily) tell for sure what is happening when, and to what extent.

At the end of the day, all of this hand-waving, estimation, concerns, recommendations, etc. comes down to timescales. For the average user (statistically, probably you), there is an extremely high likelihood that you will either stop using your specific device, or will upgrade to a better one, long before you need to worry about data degradation. With SD cards, specifically, it is also far more likely that the entire device will just fail long before specific data you have stored on it "rots."

If you put your DS/3DS/WiiU/Switch in a closet for 20 years and then try to play it again, maybe data loss would be a concern. But I expect you will have far better things to worry about by then.

 

[ghjfdtg]

What @ghjfdtg is suggesting here is that, during the process of accessing the data you have stored, the controller for whatever storage medium you are using may go through the trouble of correcting and re-writing potentially problematic data chunks.

I mean, at some point it has to correct these errors because otherwise eventually it fails to recover the data and the controller has no other choice as to report ECC errors. The SD standard has an error bit in the card status for them. Search for "CARD_ECC_FAILED" in the part 1 simplified specification here: https://www.sdcard.org/downloads/pls/

edit:
On the other hand this has happened before but it's rare. See this.
https://github.com/Atmosphere-NX/Atmosphere/issues/2257#issuecomment-1868430058

 

[PityOnU]

I mean, at some point it has to correct these errors because otherwise eventually it fails to recover the data and the controller has no other choice as to report ECC errors. The SD standard has an error bit in the card status for them. Search for "CARD_ECC_FAILED" in the part 1 simplified specification here: https://www.sdcard.org/downloads/pls/

Well, as soon as you start referencing standards specifications you're really opening a whole other can of worms. You can immediately see right from that page that there are at least 3 layers of abstraction between the actual memory and whatever you're using to try and interface with it, and that's to say nothing of the many many many layers of abstraction present in any OS accessing any kind of storage medium these days. Also, unfortunately, while standards do specify how things are expected to present themselves, they make no demands regarding actual under-the-sheets implementation, so in practice things may look very different. As an example of this, wireless AC essentially only exists because wireless N became such a huge mess with different manufacturers implementing the "standard" differently that it was just easier to hit the reset button than it was to actually fix everything to work together.

Anyway, we're getting a bit distracted here. My original point remains: data loss from "rot" really isn't something you should be investing a lot of your time into worrying about in the context of data for a game console. In my opinion, at least, there are far better uses for your time. Making backups of important data also bypasses the concern entirely, while simultaneously addressing many other more likely ones.

 

[naddel81]

Thanks for all of the contributors to the thread. To conclude:

Please check your data once in a while. Especially on your "cheap" and "dirty" SD Cards (DS/3DS/WiiU/Switch/PSP etc.).
They will get slow until they become unreadable (data loss).
For me the SD card got very slow within 12 months. It was still usable, but noticeably slower. Refreshing does not really hurt, when you do it once every year. Even on QLC NAND the lifespan is more than enough to refresh data when needed.

Again thanks for participating. I would never have imagined a few weeks ago that this thread would get more that 10 000 views.

 

[kevin corms]

I bet you are on exfat, don't use exfat. By refreshing, do you mean trim?

Post automatically merged: Feb 1, 2024

This is a very dramatic reaction to an uncommon issue.

It happened to him, so its going to happen to everyone all the time! Common reaction on the internet.

 

[SylverReZ]

Thanks for all of the contributors to the thread. To conclude:

Please check your data once in a while. Especially on your "cheap" and "dirty" SD Cards (DS/3DS/WiiU/Switch/PSP etc.).
They will get slow until they become unreadable (data loss).
For me the SD card got very slow within 12 months. It was still usable, but noticeably slower. Refreshing does not really hurt, when you do it once every year. Even on QLC NAND the lifespan is more than enough to refresh data when needed.

Again thanks for participating. I would never have imagined a few weeks ago that this thread would get more that 10 000 views.

The more you format, the more times it'll be slower to transfer files over. Also varies what filesystem you formatted the partition as.

 

[LeyendaV]

I would say this is kinda extreme. Yes, physical media deteriorates over time, that's just natural. And yes, flash storage devices are more likely to have problems than mechanical storage devices when it comes to data retention.
The fact that you have issues doesn't mean, by any means, that someone else will have it. Let alone everybody out there. ALWAYS remember, both "it works for me" and "it doesn't work for me" are NEVER valid arguments.

 

[kevin corms]

I would say this is kinda extreme. Yes, physical media deteriorates over time, that's just natural. And yes, flash storage devices are more likely to have problems than mechanical storage devices when it comes to data retention.
The fact that you have issues doesn't mean, by any means, that someone else will have it. Let alone everybody out there. ALWAYS remember, both "it works for me" and "it doesn't work for me" are NEVER valid arguments.

Mechanical drives fail much more and quicker than flash does. However when flash dies, its really hard to recover anything. When flash memory was less mature it was less reliable, but its 2024 now. Moving parts == more chance for failure. There are actual stats online to show the actual difference, im too lazy to look them up for you.

 

[LeyendaV]

Mechanical drives fail much more and quicker than flash does. However when flash dies, its really hard to recover anything.

If you read my message again, carefully, you'll see I never said flash drives fail often. I said they tend to have more problems.
Cells on a flash drive start to die after some time, and they do it quickly if you don't use them. Mechanical drives, by standard, must assure, at minimum, 10 years of flawless functioning.
Also, when a drive is "dead", it may be impossible to recover anything at all, regardless if it's a mechanical or flash unit.