Well good news, I slammed it really hard multiple times on the table edge, and it seems to be working pretty well now (knock on wood, heheh) definitely a ton better then before (works everytime I press it, and even works when I hold it down).
Thanks for the advice, it worked!