Try formatting your SD (with a special program such as SDFormatter, don't use Windows tool) and your Nintendo 3DS, and start from the begining. I use a n3DS too and downgraded recently and I didn't have problems at all.
I fear of a brick on my console, need confirmation of people who succefuly...