I don't think the controls are really all that bad personally, but my sister had a hell of a time with her swing last night. I must have been how she was holding the remote but sometimes when she went for her backswing it would quickly do her foreswing slicing the ball underpowered. My dad and I had few problems though. The graphics are pretty horrible but the audio environment is great and the announcer will rip you to shreds if you play poorly. "Just take your shot already!"
strummer12, I think you and your friends will still find enjoyment with this game. Give it a shot, and the ability to easily play with a beer or J in hand should be reason enough to try it. I haven't played it more than a few holes but so far so good.