I have absolutely no idea how I installed vJoy, but I do remember it being a pain as well...
I think I tried multiple forks, and one of them finally did install.
Although one thing to remember is that for some reason the UAC prompt shows up silently behind the installer, and it takes a REALLY LONG TIME for the UAC prompt to show up. I think I watched a 20min youtube video, and it showed up SILENTLY on the taskbar after the end of the video, and didn't activate until I clicked on it.
I'd say try doing a reboot (do NOT use the shutdown option, I do indeed meant to use the reboot option), and try to install that way.
Also I don't think you need to unplug anything at all for the installer to work.
Otherwise the Xbox feeder does work, and I have even used it to play GTA5.
My DualShock 2 is more comfortable though, and it also has vibration support as well, and analog triggers (yes, on a DualShock 2 almost all buttons are analog), so I'm using that instead of the 3DS. An unmodified CStick sucks to use...