Been having the same issue. Are you using the official latest version for the controller for BT? 1.06
I also updated the adapter to 1.04 and tried 2.4ghz, but that didn't work for me either. Though I'd prefer BT so I can easily swap between my desktop and switch.