So I'm going through this, trying to find a solution too. I have a similar card, a Sandisk 128GB bought from a major chain store, multiple tests indicating it's good. Speeds are even fairly consistent when running tests. Yet FBI, after a little while, consistently, will go from a reasonable 1.5 MiB/s to around 250 KiB/s or so. The only difference for me is the network connection, but I have the same problem on a local install of a CIA.
The funny thing is: this is consistently a problem across all MicroSDs of many different sizes I've used on all 3DSes I've used, and it seems like it happens after a similar amount of time installing CIAs, every time. A reboot fixes this issue, but it's a bit annoying to have to reboot each time. And this has only been a problem recently to my memory. Installation used to be fairly consistent with similarly consistent speeds, so I'm beginning to think that this is either an issue with FBI, or a limitation of MicroSD cards in general. This means that I somehow doubt OP's problem is related to their MicroSD. I wish that there was an alternative way of installing CIAs so this could be tested properly, but unfortunately, as far as I'm aware, there's no current alternative. Unless somebody knows of one?