I generally think they add replay value to a game and if I really like a game, I'll often go back to get achievements I missed after beating a game, if they aren't too difficult to get. Some games make achievements way too easy to get, some make them near impossible to get requiring you to be a professional speedrunner or hiding them in really obscure places in a long game that you can't go back and get, you have to start the entire 50+ hour game over from the beginning. Don't have patience for that.
It's a shame Nintendo consoles don't have achievements, some games implement achievements in the game engine anyway, but the achievements are localized to that game (and your save file) which makes it less fun to me. I like being able to compare my achievements to others' on Steam.