First make sure it is nothing like stickykeys: they are set to appear when you press shift/control and other keys which games tend to have as run, duck, reload and such.
After that a few ideas:
first clean out the fan, heatsink (plus any heat pipes and whatnot you may have) and move all the cables to allow for good airflow (you will probably need some electrical tape and cable ties for this one).
second check the fan, if it is just started the fan could be on the way out, also consider reseating the heatsink.
third if you do not mind a bit of noise it seems like your fan is set to dynamic, either put it on full blast or set the base level to something higher: it is easier to prevent cooling but if you have a spinup time then it makes life a bit harder.
4th find an overclocking guide and follow it until you actually get to the overclocking as it will better set up your bios. I suggest you do not use load based clocking as well. Another guide:
http://www.tweakguides.com/TGTC.html
Try a proper test, I applaud your reasoning but something like prime95 for every core you have is a good idea. Failing that a lot of stuff allows overlays in game (I have NVTray which sticks FPS, memory usage and temps for my graphics cards).