Some I speak to seem to have a "I was raised and then I reached the age of reason" story. In my case, and in the words of a favourite song, I started out with nothing, still got most of it left.
Anyway I quite like this life business, as do most that experience it, and like it to be the best it can be for me. Praying and religious morality then seemed far less effective than trying to understand how the world works and the rather less contradictory and easier to understand morality of the modern world. Worse still was the controlling bent that many religions seem to go in for. To finish it off I learned something of psychology, economics and general argumentation and it seemed the religions that survived employed suspect methods or were artefacts of all three -- for economics a classic one is noted as those religions with many gods split their population and thus are primed to be overtaken with those with a singular god and a singular mission and singular set of overheads.
Extremism, and possibly fundamentalism as they tend to go hand in hand, is a major problem but I would not say it is the major problem. The low level moralistic undercurrent a lot of Christians and recovering Christians (same for many other religions, though the effects and approaches vary somewhat) exhibit and its effects on raising kids. This guy (
https://www.youtube.com/user/TheraminTrees/videos ) does lots of stuff I would +1 here -- raising kids to believe, the nature of gods if they are supposed to be all loving, all knowing and all powerful, the creation of less than sceptical mindsets (it is often noted that father christmas/santa claus is kind of god lite, I notice similar things for other aspects).