Possible? Yes. Feasible or practical? Not so much.
There have been plenty of reports from people who've successfully downgraded N3DS/XL units using sysUpdater and the only failures or bricks reported so far have been due to user error. Personally I've managed to downgrade a N3DS XL from 10.3E to 9.2E. But you need to take into account that sysUpdater has a huge launch failure rate on New 3DSes. I've attempted to launch or otherwise start the downgrade process at least 50 times repeating the exact same steps, out of which only 1 attempt was successful. Whether or not it launches is purely down to luck, as odd as that sounds for something computer-based. The failed attempts usually were down to the exploit being not properly executed, which prevented sysUpdater from even launching or starting the downgrade process in the first place. This is something that happens by random. Once the exploit does load and the process begins, usually it's downhill from there.
It takes a lot of time and patience and certainly isn't for the faint of heart. If downgrading isn't that urgent for you, I'd strongly suggest you wait for KTM to be finalised since it'll make the process that much smoother and safer.