The problem is you are trying to do anything with a supercard SD. The supercard GBA line is not a great thing -- they cheaped out on memory and save and thus made them basically pointless.
That said sounds like you might have a battery on the way out, which makes sense given how old the SP is these days. Saving engages other aspects of the GBA which might draw more current and thus cause it to flake out, and leaving it on for a while might see something similar if it drops below cut off voltage.