That's strange.. I'm not sure how to solve this problem. On discord CTCaer replied
well that's weird, since init fails normally are the connector pins (or in modded cases could be solder junk shorting the vdd rails)j ust check your connector pins that they are not broken, missing or on the air but anyway, that's a hw issue the only difference is that it's peculiar. and having only the big ones get affected immediately it points more to power issues
All my diode readings on the power rails seem ok. I think I'll resolder some wires to the pico to double check it's not that.
Check your SD card flex pins.