Hardware How I fixed 160-0103 system memory error

SDIO

Well-Known Member
OP
Member
Joined
Feb 13, 2023
Messages
1,973
Trophies
0
Age
28
XP
1,266
Country
Germany
DISCLAIMER if your console bricked because of Cold Boot Haxchii (CBHC), then none of this applies to you. If you deleted the DS Title used by Haxchi or did a Factory reset with CBHC installed, UDPIH with the recovery menu is your solution -> https://gbatemp.net/threads/udpih-usb-host-stack-exploit-recovery-menu.613369/page-31#post-10097502

*Update* Since this post was written things envoled rapidly, we now have multiple ways of fixing this, even without soldering. Please see the >>> Ultimate Wii U Troubleshooting Guide <<< to find the right solution for you

Hi all,
my early 32GB Wii U started crashing with: "There is a Problem with the system memory. For help, make a note of the error code and visit support.nintendo.com. Error Code 160-0103". It happened at random times, but got worse over time and did not only affect one game. I tried to work around this, by reinstalling everything, that seemed corrupted but it only got worse and finally the screen just stayed black after turning it off and on again after such an error. This error is known and affects early models of the "Premium" black 32GB Wii U. At some point Nintendo repaired these units for free, even after warranty, so if you have the same problem you might want to try this first.
I will explain in this post how I got my Wii U back to fully working condition, by replacing the defective eMMC memory chip with an SDHC card and replacing corrupted files. Also I learned a few valuable lessons, that I want to share with you.

Findings
let me start with my findings, as they might be relevant to most of the readers:
  • The SLC has a cache (scfm.img) for the MLC (eMMC), which means when restoring the MLC, you need also to restore the SLC, or else the file system on the MLC will be inconsistent, which causes all sorts of problems. You won't be happy for long if you only restore one without the other, as more curruption will creep in.
  • The Wii U will accept a 32GB SDHC card in place of the eMMC but not a 64GB SDXC card.
  • The file system doesn't seem to use all the available space, so you should be fine if the SD card is slightly smaller than the eMMC
  • It is a Hardware Problem, and there is no way around replacing the eMMC, it will die sooner or later
  • You should act, before the Wii U bricks, so you don't have to flash back the SLC, which requires a teensy and lots of fine soldering.
  • Make multiple complete NAND Backups if you suspect a problem with the MLC

Short Story
I cut the CLK trace between the pads, to disable the eMMC (I would recommend grounding it on the eMMC side to not have it floating). I used magnet wire to solder a SD to Micro SD adapter to testpoints near the eMMC (the ones usally used for the Hardmod to dump/restore the emmc). I also added a 100nF cap to the SD Adapter for decoupling (not sure if needed). For 3,3V and GND I used thicker wire and got it from some other components around. I also kept the wires as short as possible for signal integrity. I also used hot glue as a strain relief on the magnet wires I soldered to the tiny testpads.

Then I used a Tennsy with NANDWay (signal booster edition) to write back a backup of the SLC. While the SLC flashing was going on I grabbed an 32GB micro SDHC card and wrote the MLC backup and ignored the error at the end, that the SD card was running out of space.
I also extracted the MLC backup with wfs-extract, which have me an Idea which files are corrupted.

After removing the teensy and inserting the micro SD card into the adapter I soldered in place of the eMMC, the system booted and allowed me to start the browser, which allowed me to use Homebrew.
FTPiiU everywhere gave me access to the whole file system. I went trough the list of errors wfs-extract gave me in the system titles and deleted the files. After deleting I made sure they don't come back as 0 byte files (by refreshing) and if they came back, I deleted them again. After making sure they are really gone, I replaced them with a copy from a good backup I extracted from a different Wii U (but you should also be able to get them from NUS). Then I deleted corrupted files in the saves and reinstalled updated and eshop titles that got courrputed.
At the end I downloaded the whole mlc over ftp to make sure there are no corrupt files left.
In Filezilla corrupted files caused Filezilla to try do download the same file twice (and thus asked to replace it, usually with a smaller or 0 byte version). lftp (command line tool) either got stuck in an endless loop on corrupted files or showed an I/O Error.

Long Story
Of course it didn't went so smooth as described in the short story. First of all I didn't want to solder the wires for restoring the SLC and I didn't even know it was necessary, because I didn't know of the SLC cache. So in my first round of attempts I focused only on the eMMC. I did the Hardmod for the eMMC and used the second SDIO Inferface (on the GPIOs) on my Raspberry Pi to access the eMMC. Using the Raspberry had several advantages: I could power it from the Wii Us USB Ports, which eliminated the risk of potential differentials and also gave an additional GND connection. Also it gives more control than an USB-Cardreader. For example I could configure it to reduce the clk or the width of the databus. Also it supports blkdiscard.

Using the Raspberry I first wanted to get a picture of the state the eMMC was in. At this point I didn't know if the problem was Hardware, Software (File System on the Wii U) or something in between, like the the Firmware of the eMMC chip. I read out the eMMC multiple times and no two images matched each other. I tried it 6 times in total. Looking at the distribution of the differences over all of the Images, I foudn that it were mostly the same ~million bytes (not much compared to the gigabytes) that where different. Also they clustered together along their addresses. Looking into the bytes it seems at it were even just single bits that read one time as a 1 and the next as a 0. This gave me confidence that the problem was not with my wiring but with the eMMC itself. This is a little strange, as normally the controller inside the eMMC should detect these errors, and correct them if possible or return an error if not possible.
My hope now was than that I could just write back a Image or an backup, which would allow the eMMC to replace the bad blocks with reserve blocks and maybe work again (hoping the problem would not get worse and just effect a few cells). But then I noticed the eMMC didn't accept writes. It allway errored out. So the emmc had some obvious problems, which gave me hope.

My next idea was to run blkdiscard on the eMMC, which tells the controller, that it can forget everything and start from scratch. An this showed success, the command went trough and the eMMC was writable again. I wrote back one of the Images I made before. After writing back and disconnecting the Raspberry the Wii U started to show signs of life again: It produced a picture, but then after a few seconds crashed with the 160-0103 error again. But at least some progress.
Not knowing of the SLC cache, I restored a MLC backup, I created when the Wii U was still working to the eMMC and I got a little but further: The wii bootet to the user selection screen, but after selecting a user it just froze. This was back in 2020, before UDPIH existed. So all that I could try with my knowledge back than was trying to fix one of my backups or images with wfs-inject, but I was never successful with that. After many tries, which meant lots of writes on the eMMC the eMMC finally died. It is still readable but not writable anymore. Not even blkdiscard could bring it back. The screen also stayed black then.I then tired to replace the eMMC with an SDXC card, because the 32GB SDHC card were all to small but the screen just stayed dark. At this point it was just shooting at the dark and I had no real Idea why it doesn't like the SD card (bad soldering, wires to long, maybe the wii you doesn't like sdcards or it needs to be the exact same size) and even if I got it to work with the SD, the backup still would be broken. I ordered an new eMMC, but when it arrived my motivation in this project was gone and the tiny solder balls and the problem of the broken backup that was waiting for me after that finally killed my motivation and I shelved that project.

Fast forwarding to 2023: I now had a logic analyzer and my brother was asking my about the Wii U. So thought I might give it another shot. The logic analyzer showed me that the Wii U did the handshake with the SD Card according to the spec. This showed that 1. the wii U had still life in it, it understands sdcards and my soldering seemed write. The first thing the Wii U did after the handshake was to initialte multiple block write and the sdcard responded with an OK response code and I could see data on the databus, but only zeros (I belive, the analyzer had some trouble with that). The strange thing was, that the Wii U never stopped the multiple block write, at some point it just stuck, with the SD Card beeing ready and waiting for more data. Very strange. I then tried an 32GB sdcard, but the same happened.
I then connected the eMMC again and removed the SD Card and I could see the handshake with the eMMC (which is a little different) but after that the same MULTIPLE_BLOCK_WRITE, which indicates the problem was not with the SD card.

Confused by that and having exhausted my options on the eMMC side, the only option left from my perspective was to dump the SLC and take a look at the logs, maybe they could give me a hint. Also at this point I had nothing to lose. After doing all the soldering and dumping the SLC I could finally take a look at the logs and they were just underwhelming. Absolutely nothing helpful. But having the teensy connected I flashed back the oldest backup I had of the Wii U (Firmware 5.5.3). After the flash I still had the eMMC connected. When I turned the wii U on (didn't even connect a monitor), I sampled again with the logic analyer and what I saw was very nice suprise: a lot was going on especially reads. Excited I connected a monitor and saw the Wii U booting into the error again.
I then flashed the mlc of the 5.5.3 backup on the 32gb card and it also booted to the memory error. Not perfect but more than I was expecting. I then flashed the 5.5.3 mlc backup to the card and it bootet to the user selection (still with the 5.5.3 slc). I was expecting it to freez as soon as I select a user. To my surprise the Systemmenu loaded, I could connect to the Internet, run homebrew and even install 5.5.5. Everything was looking fine. I was replacing the corrupted files, reinstalled games from the eShop , everything worked. I created a new Backup with 5.5.5 and everything seemingly working.
But then I hit the first problem: I could not reinstall Dr Kawashima's Brain Training. With FTPIiU I found that I could not delete some files from that game, they were just comming back. I also ran into similar problems with another game. Also renaming the directory didn't work. I then moved the directory to the tmp directory and rebootet. This caused the Wii U to brick again. I then flashed back my backup of the mlc, but it was still bricked, which was confusing, because I only changed stuff on the mlc. After some more experimentation (involving reflashing the SLC) and noticing that the scfm.img changes very dramatically between backups, so more things where hinting to it. In anaother thread I could just find that it is just some sort of cache. But with more googeling I could piece together that this must be a (block level?) cache for the MLC. The problems I saw were probably caused by me using the 5.5.4 mlc together with the 5.5.3 slc.
After flashing back the SLC and the MLC from the 5.5.4 backup at the same time, these problems where gone, and we arrived at the "short story" from above.


What should I do if I get the 160-0103 error?
(Update: see https://gbatemp.net/threads/160-0103-error.636361/ )

Are you getting the error at random times and in different Apps? This hints to a bad eMMC. If you are getting the error only with one save game, than that doesn't necessarily mean that you got a problem with the MLC. Maybe the Wii U just turned off in a bad moment and corrupted the savegame or something like that. Bonus points hinting to a bad eMMC: if it is an early 32GB model.
If you still suspect it is a bad eMMC talk to Nintendo, maybe they are still doing the free repair.
If Nintendo declines or you don't want to send it to Nintendo, I would suggest, that you replace the eMMC before the Wii U bricks. Because if the system is still working, you can just clone the eMMC to a new eMMC or SD Card and don't need to flash the SLC. If your wii is too far gone and the eMMC is already to corrupted, so you have to use an older MLC backup, you will also need to flash back the SLC, or you will run into the problems I mentioned in my long story. With UDPIH it might be possible to fix a not completly booting system after the replacement. In theory it could be possible to get the system booting far enough with an older MLC backup and then flash the SLC in software, but as far as I know there does not exist any tool for that at the moment.

Other Remarks
I used an SD to micro SD Adapter and and SD card, because that was what I had lying around. The cleanest solution would be to replace the eMMC with another eMMC, but that would require BGA soldering and there are also components on the other side of the board, which would make it extra tricky. So that was above my skill level.
You can also get eMMC soldered to a breakout board or an micro SD adapter, this can then be easily done with a soldering iron. Also there are micro SD slots available with a small breakout board. You could use one of these in place of the SD to micro SD adapter I used. The advantage of the adapter is, that it is out of plastic, so you don't run into the risk of shorting anything out.

In theory eMMC should give you better performance than an SD card, but I couldn't measure any difference between a faster and a slower SD card. Maybe because of the SLC cache, maybe because the Wii U is so slow because of other reasons....
You can find the pinouts for the SD card and the testpoints on the Wii u board easily online if you search for Wii U emmc.

Good flux makes soldering so much easier. With enough flux it will solder itself. (You should still practice on something else you don't care for first).

Create multiple NAND Backups and save them to the PC. I was lucky to have a second one, because the first one was too corrupted.
Update: we are now also able to rebuild the mlc without an existing backup: https://gbatemp.net/threads/how-to-upgrading-rebuilding-wii-u-internal-memory-mlc.636309/

I attached a picture of my work, you can ignore the DuPont connectors, that's where I connected the logic analyzer.


EDIT: now that I can post links here are a few:

Links
"/dev/scfm - SLC cache for MLC": https://wiiubrew.org/wiki/IOSU#IOS-FS
scfm.img: https://gbatemp.net/threads/release-wii-u-nand-tools.465386/post-7202380
EMMC and SD pinouts: https://wiiubrew.org/wiki/Hardware/EMMC_NAND
Wii U Hardmod: https://gbatemp.net/threads/successfully-dumped-wiiu-emmc-nand-with-hardmod.457165/
wfs-tools (wfs-extract wfs-inject): https://github.com/koolkdev/wfs-tools
Replacement by Nintendo (3 years ago)
DIscussion about Nintendos repairs: https://gbatemp.net/threads/wii-u-memory-failure-160-0103-never-modded.603986/post-9681470
Seems to be a problem with Hynix eMMC: https://hackmd.io/d12Fq9g-QlCjN2HJp7Yvew?view
mdinfo (shows what eMMC your console has): https://github.com/GaryOderNichts/mdinfo/releases/tag/v2

EDIT: This post https://gbatemp.net/threads/possible-emmc-to-sd-replacement-for-wii-u.628448/post-10102071 by @GaryOderNichts makes me believe the problem is not SDHC vs SDXC but really just the different capacity. It suggests there are different capacity bins. As everything below ~32GB is treated the same you should be able to use either a 8GB or 16GB card with a 8GB console. But for a 32GB console you need a card with at least 32GB but smaller than 64GB, so 32GB is the only option.

Going by this post with the exact sizes, a replacement card for a 32GB console needs at least 31239176192 bytes. A replacement for the 8GB needs at least 7717519360 bytes

There are 32GB sdcards, which are to small. So make sure to check the exact size first.

Cards which worked:
Cards which didn't work:

The point where I get the 3V3 from in the picture is problematic, you should get it from a different point, see this post from @Lazr1026 : https://gbatemp.net/threads/how-i-fixed-160-0103-system-memory-error.626448/post-10119405
 

Attachments

  • sdemmc.jpeg
    sdemmc.jpeg
    1.3 MB · Views: 72
Last edited by SDIO,

KleinesSinchen

GBAtemp's Backup Reminder + Fearless Testing Sina
Member
GBAtemp Patron
Joined
Mar 28, 2018
Messages
4,366
Trophies
2
XP
14,705
Country
Germany
Not yet through the complete text… but I already can say: Sounds awesome!
Doing something to preserve the pretty rare (compared to other consoles) Wii U units.

The Wii U will accept a 32GB SDHC card in place of the eMMC but not a 64GB SDHC card.
Bummer. I'd always hoped for some internal solution with more memory (however resizing the file system would be done). Normally SDXC are in no way different to SDHC (other than the standard file system – which is of no importance here).
======

Edit:
I'm impressed after reading through. Really good work!
A lot of information in one place.

Thanks for sharing!
 
Last edited by KleinesSinchen,

SDIO

Well-Known Member
OP
Member
Joined
Feb 13, 2023
Messages
1,973
Trophies
0
Age
28
XP
1,266
Country
Germany
Yes I was also suprised that SDXC is not working, because the Wii U supports SDXC in the normal SD slot and I assume they are using the same driver for both. The Wii U is dooing the handshake like on the SDHC card, but then just stops after that. The last command ist SD Status. After that the Reads start on the SDHC card but not on the SDXC. Maybe the problem isn't SDXC itself but the Wii U ist doing some checking for the size? It would be interesting to see how a 8GB Wii U would behave with a 16 oder 32GB card.
And yes growing the file system would be a problem. Maybe one could modify the routines for formatting USB disks and change them to use the MLC key and after formatting copying everything over. I also thought about writing the mlc key too the seeprom, but it seems to be a little bit more complicated. But if the Wii U isn't accepting bigger cards anyway there is no real point in this endeavor. (except maybe fixing other wii Us where to file system is too corrupted).

Maybe a 64GB eMMC would work, but I don't feel like spending 50€ for a slim chance and more work.
I also did comparisons with an USB Stick and the USB Stick was not slower, sometimes slightly faster. So there is not much incentive in upgrading the internal storage, if you can just plug in a USB Stick and call it a day.
 
  • Like
Reactions: KleinesSinchen

michalt

Well-Known Member
Newcomer
Joined
Jun 28, 2020
Messages
74
Trophies
0
Age
55
XP
273
Country
Netherlands
It is a pity we cannot make WiiU run fully from external SD card without using the NAND. It would be a nice solution as, in the end, all those internal SSD drives will die and those are not easy to replace :-(
 
  • Like
Reactions: KleinesSinchen

CrazySquid

Well-Known Member
Member
Joined
May 27, 2017
Messages
190
Trophies
0
XP
796
Country
Amazing work! Interesting that the Wii U doesn't like higher than 32GB SD Cards, maybe Cafe OS is hard limited to 32GB max, who knows.
I'm a bit worried because I have my Wii U since launch, so it's a 32GB launch unit, still works perfectly fine without a single issue, but I don't now how longer will the NAND work.
Also, I did read that you couldn't get better performance out of a faster SD Card? That's interesting, I remember when I was using redNAND with a high class SDCard, the Wii U Menu loading and the apps felt faster.

By the way, an eMMC is supposed to have a better read/write life cycle right than SDCards right? Although, I guess that after you do this mod, it will just be easier to replace a dying SDCard than a new dying eMMC (But the eMMC should last another 10 years I guess, it's probable that other Wii U component will begin to fail first).
 
  • Like
Reactions: KleinesSinchen

V10lator

Well-Known Member
Member
Joined
Apr 21, 2019
Messages
2,585
Trophies
1
Age
36
XP
5,366
Country
Germany
The way I read this is that you tried the 64 GB card one time only and that you did this at a time the 32 GB card also didn't work / showed the exact same errors.
Am I right with this and if so: Could you try to clone the 32 GB card to the 64 GB one and try this, please? From a technical standpoint the Wii U knows 2 MMC sizes: 8 GB and not 8 GB. All not 8 GB is expected to be 32 GB, so using a 64 GB card should work (but waste space).
 
  • Like
Reactions: KleinesSinchen

michalt

Well-Known Member
Newcomer
Joined
Jun 28, 2020
Messages
74
Trophies
0
Age
55
XP
273
Country
Netherlands
I have read several posts regarding the NANDs issues (or at least it seems so), but its difficult for me to draw conclusions which WiiU are affected, what is the % of consoles affected, etc.

Im not an expert, but I think the mechanical parts (e.g. optical drive and mechanical disk drives) or modern flash based drives (like in WiiU) will probably die faster than rest of the console chips/components. Optical drive is not an issue, but NAND is if we cannot easily replace it in the future. I looking from the perspective of using my old Atari from the end of 70ties.
 

godreborn

Welcome to the Machine
Member
Joined
Oct 10, 2009
Messages
38,471
Trophies
3
XP
29,103
Country
United States
I have read several posts regarding the NANDs issues (or at least it seems so), but its difficult for me to draw conclusions which WiiU are affected, what is the % of consoles affected, etc.

Im not an expert, but I think the mechanical parts (e.g. optical drive and mechanical disk drives) or modern flash based drives (like in WiiU) will probably die faster than rest of the console chips/components. Optical drive is not an issue, but NAND is if we cannot easily replace it in the future. I looking from the perspective of using my old Atari from the end of 70ties.
it's the moving parts. plus, electronics are not made the way they used to be. they're cheaply made.
 
  • Like
Reactions: michalt

CrazySquid

Well-Known Member
Member
Joined
May 27, 2017
Messages
190
Trophies
0
XP
796
Country
I see, I was wondering because my Wii U is a bit old now (launch unit, so over 10 years old). It still works like the first day, but you know, I started to see a lot of systems that started to fail due to dying MLCs or data corruption (although those were Hynix units, mine is a Samsung), so... yeah, I just wanted to take care of it.

Anyway, despite this article being more recent than the manufactering date of the Wii U's NAND, is the best I have found:
https://www.anandtech.com/show/9248/the-truth-about-ssd-data-retention

Basically, it says that for brand-new SSDs, data retention is not a realistic problem.
But, for those SSDs that are too old and have exceeded manufacturer endurance specs (which I MAY have, since I filled my MLC a few times...) then data retention is for about a year.
So, I guess I will just power on my Wii U before a year passes, just to avoid data corruption.

View attachment 349213

"Remember that the figures presented here are for a drive that has already passed its endurance rating, so for new drives the data retention is considerably higher, typically over ten years for MLC NAND based SSDs."
I'm quoating a post of mine from another thread, but I'm seeing some people are starting to have NAND corruption with the typical 160-0103 error, the article is from 2015, but it suggests that for older SSDs you should power it before a year passes since the last time you use your device, so I suggest everyone to power on their Wii Us every 8 months or so (whatever the time you want, just make sure isn't close to a year).
 

Intro_to_tech

Member
Newcomer
Joined
Jul 2, 2021
Messages
15
Trophies
0
Age
31
XP
106
Country
United States
I've heard multiple things:
- This information is pretty old and just now getting traction
- It only impacts Hynix emmc's
- Time isn't necessarily the issue since it has to do with the emmc's ability to write data to failed blocks

Would OP be willing to address those issues? Also are you planning on releasing a tutorial for replacing the emmc with an SD card? I could follow the picture, but I want to ensure I'm getting the correct connections and pinholes.
 
Last edited by Intro_to_tech,

V10lator

Well-Known Member
Member
Joined
Apr 21, 2019
Messages
2,585
Trophies
1
Age
36
XP
5,366
Country
Germany
It only impacts Hynix emmc's
That's not correct: It looks like Hynix has the highest chance of failing but others fail, too.

Last time I checked statistical data on this was low but this was months ago. @GaryOderNichts could you share your current data set with us, please? To other readers: Gary gets the most failed NAND reports thanks to UDPIH and mdinfo.
 
  • Like
Reactions: CrazySquid

Intro_to_tech

Member
Newcomer
Joined
Jul 2, 2021
Messages
15
Trophies
0
Age
31
XP
106
Country
United States
That's not correct: It looks like Hynix has the highest chance of failing but others fail, too.

Last time I checked statistical data on this was low but this was months ago. @GaryOderNichts could you share your current data set with us, please? To other readers: Gary gets the most failed NAND reports thanks to UDPIH and mdinfo.
Thanks for the clarification, that's why I asked it if was true or not
 

GaryOderNichts

Well-Known Member
Member
Joined
Aug 9, 2018
Messages
791
Trophies
1
XP
5,449
Country
Germany
That's not correct: It looks like Hynix has the highest chance of failing but others fail, too.

Last time I checked statistical data on this was low but this was months ago. @GaryOderNichts could you share your current data set with us, please? To other readers: Gary gets the most failed NAND reports thanks to UDPIH and mdinfo.
@Lazr1026 created a hackmd for the list here.
 

Ettino

Well-Known Member
Member
Joined
Oct 12, 2022
Messages
498
Trophies
0
XP
917
Country
Canada
Goddamn. Precision German engineering strikes again! My main Wii U is still working fine (knock wood) and the nand is empty as I launch everything from the external HDD. But I doubt I would even attemp to do all this, I still have 2 back up consoles as well.

But well done, OP.
 

michalt

Well-Known Member
Newcomer
Joined
Jun 28, 2020
Messages
74
Trophies
0
Age
55
XP
273
Country
Netherlands
Does it make sense to move all the games from NAND to external drive to protect the console in this context?
 

V10lator

Well-Known Member
Member
Joined
Apr 21, 2019
Messages
2,585
Trophies
1
Age
36
XP
5,366
Country
Germany
@michalt We don't know the internal details of the Wii Us implementation so this is guesswork but yes.
Keeping your NAND as free as possible means more free cells which translates to better wear leveling. Also you will reduce writes (save files and stuff) as these files will move with the game.
 

Site & Scene News

Popular threads in this forum

General chit-chat
Help Users
  • No one is chatting at the moment.
    K3Nv2 @ K3Nv2: Sorry for accidentally bending over