For everyone getting constant soft-bricks on N3DS when downgrading: The only fix so far appears to be doing a system format. Some users have lucked out and managed to downgrade after a /couple/ of soft-bricks, but for those that have like 10-12 times, then formatting is most likely the solution. Take it from someone who's managed to downgrade after that awful crap.
My guess is that there's something left over on the NAND that isn't being cleanly downgraded or overwritten, and is freaking out the lower FW. The downgrades themselves (provided everything is right) are fine. This is evident that it works immediately first try after a system format.
Back your stuff up, and do a system format if you really care about downgrading before 10.4. Unless a specific fix comes out for it, chances are you'll always have the problem.