I don't think an angry face will detract from any overall like totals or anything like that. The closest thing to a dislike on the site is the downvotes on threads with questions (I think these threads get the voting thing) and posts in this forum.
Again I don't think these downvotes are detrimental to any user stats, they just serve a purpose in their relevant uses.