If you want to keep the spring and just slide in and out after putting the spring into the R4 instead of clicking it just remove it (the spring should stay in place and allow you to use it by pulling it and pushing it in.)
The best way to keep the spring so it will be ok is to take out the card when not in use and it will keep your spring nice and springy. The spring being compressed causes a lot of pressure on each side of the spring, the spring being to strong for the clip than holds the micro sd card in place is the whole problem.
Summery, to use the spring and to keep it in good condition, place the MDS card in the holder instead of the R4 when not in use.
It would be a dumb idea to leave the spring compressed because it would cause more problems than initially possible.