Quick option: Delete and reinstall YouTube app from eShop. Later, repeat TubeHax load method, and please be careful selecting your FW correct version.
Edit: OK! Reading again, I understand you can't ever load Smealum's FW selector webpage... Your other option will be using another Internet connection (possibly, your ISP could be blocking access to that webpage, or maybe is using a proxy server, IDK).
An alternative way also you can probe if webpage works is changing DNS on your PC (network card options) for that used by TubeHax method (107.x.y.z, whatever), and navigating to m.youtube.com. If you see the FW selector on your PC browser, then connection is OK!