Hi,
I found a way to theoretically do just this, disable all extra features of the gamepad like sound and the display and use it just as a controller, some time ago.
You can read more about it here:
https://gbatemp.net/threads/wii-u-hacking-homebrew-discussion.367489/page-514#post-5643453
Unfortunately I can't test or debug my homebrew on my console as it's running 5.4.0, which is why I asked for anyone interested with a console on 5.3.2 to contact me. Nobody did though, so I thought there wasn't that much interest in exploring this. In case you have a console on 5.3.2 and would like to test that with me, just drop me a message!