The general advice is that if you don't know what you're doing, you're better off not doing it. That said, everybody has to start somewhere.
I don't know specifically for overclocking AMD, as Intel has greatly simplified the process by making the primary variables the clock multiplier and the voltage. Most motherboards that support overclocking generally have some sort of preloaded overclocking settings (such as "Level Up" in Asus motherboards) that give a small but safe and stable overclock. Beyond that you usually have to start increasing the base frequency, clock multiplier, FSB (for RAM) and voltages up in small increments.
For those without the intuition to discern a safe overclock, there are two places to look for guidance. Either look at the spec sheet for your CPU to see what the factory limits are (e.g. what it will do automatically using TurboBoost if it's Intel), or look online for the settings people used in their "stable" overclocks. Never go straight for the top - always start modest then slowly increase until you start getting signs of instability (like crashes in benchmarks).
Also, the reason why I'm not doing a step by step walkthrough on this is because no two CPUs (even if they're the same model), motherboards, PSUs or coolers are the same. Some are more stable or efficient than others, and your specific environment (e.g. how "clean" the AC supply is) can also affect how well things go. Some motherboards are better overclockers than others (a good sign is to look at how they cool their VRMs, which will always get toasty with increasing overclocks) and if you have a cheap motherboard, you're better off not even trying.
The only other piece of advice I can think to give... voltage is a matter of scale, and never pump it up too much. A small overclock (as in 100-300Mhz) can often be obtained without any voltage increase. Increasing the voltage is mainly to add to the stability, however increasing voltage will always increase the heat generated and wear out the CPU and VRMs quicker. I often see people working to the 3rd or 4th decimal when working out the "sweetspot" for voltage, because it can be that sensitive.
