Try a litmus test by taking a spare or different SD card than the main one you're current using.
- Copy everything off the secondary SD card onto a computer.
- Reformat in FAT32 + 32 KB cluster size using Windows File Explorer or guiformat (if bigger than 32 GB).
- Full Write + Verify the empty card with H2testw. To ensure clean test results, do not skip this step.
- If the card passes, delete the *.h2w test files. However, replace the card if an error was reported.
- A hardware failed card must be replaced. Go back to step 1 or 2 in the event the current card is bad.
- Build a temporary CFW setup. Hold off from using files from your main setup.
- For convenience, you can use the starter kit here.
- Add a 3DS or CIA rom for one of the expanded memory games onto the card.
- It is recommended that the chosen game has matching region with the 3DS firmware.
- Insert the card into the o3DS, turn on the (POWER), and hold (SELECT) to access Luma3DS v9.1 configuration.
- Use only (x) Show NAND or user string in System Settings
- (Conditional) If the game is 3DS format, convert it into CIA in GodMode9.
- Verify that the CIA rom is a clean dump.
- GodMode9 → 0:/.../example.cia → CIA image options... →Decrypt file(...) → Decrypt inplace.
- You want: File is not encrypted or Decryption success
- GodMode9 → 0:/gm9/out/example.cia → CIA image options... → Verify file.
- You want: Verification success
- Install the game with FBI. If you're not familiar with Rosalina Menu injection, watch this:
- Look in [GitHub, ihaveamac] SeedDB list. If your game is known to require seed,
- With Internet connection,
FBI → Titles → Game → Import Seed
- Play the game and attempt recreating the returning to HOME Menu error.
Depending how your o3DS reacts in this temporary setup, this test is to pinpoint whether there's something wrong with the 3DS firmware or the main SD card.
Okokok
So I did a stupid
Turns out the issue was my luma config
Not sure why
I had tried turning on and off settings, but never replaced the file itself
I looked on platilecs guide and noticed that I talked about black screen errors after installing b9s
It said to wipe luma config
I also noticed that the new copy of luma I threw on my SD was an empty file lol, so it was using the config on my nand from when I got it off my SD earlier
So after putting a new good luma on my SD, and resetting config, I got through initial setup
Then after noting that everything worked fine I planned to restore the malfunctioned nand, grab all the stuff I need for a ctrtransfer, and then install that stuff to it once I formatted again
So after restoring to the old nand; I thought I'd try an expanded memory game again
And it worked, I was able to exit home and everything
So it was either me resetting luma config.
Or the act of formatting/restoring the nand
I then tried this with the luma file on my nand (which still had the same config copied from off my SD from before) and I got the error
So tldr; it was a simple fix that would have taken 30 seconds but I spent the better part of an entire day trying to troubleshoot
i seriously appreciate the help though, this community never ceases to be great!