Thanks for the theory, but the wireless connection on my modded console works just fine. I install plenty of games over wifi from my home PC, so if anything was the matter with the wifi connection, I should have noticed some other issues cropping up by now. Sure, the modded console aggressively blocks nintendo's servers, but I don't think that's it. Now that I think about it, I actually don't have to bother blocking nintendo, the console is banned from online play anyways, lol I just realized how pointless it is. Maybe they really do have to be on the same FW version, I don't know. I can't upgrade the modded console to 19.0.1 at the moment to rule that out, but once I do upgrade it, it will be one of the first things I'm going to test out.
As far as I understand it, "local wireless play" just sets up an ad-hoc wireless network, broadcasts the lobby, and lets any other console pick up the lobby and join it. Perhaps I should disconnect from the wifi network completely and try again.
Edit: I have confirmed that the two consoles can talk to each other. When going to my user profile on the consoles, and selecting "search for local user" in the add friend menu, the other console's user and his Mii shows up. Adding him as a friend fails of course, since one console is banned. At least I know the consoles can communicate with each other. Now to solve that Mario Kart issue...