If the fan is positioned at the back of the case (i.e. behind the CPU heatsink) then you should have the fan sucking air out of the case (i.e. an exhaust fan). If it's positioned at the front or side of the case, you should have it sucking air into the case (i.e. an intake fan). Ideally, you want both to create air-flow that always sucks in fresh air to cool the parts and sucks the hot air back out.
Word of warning, if you have an exhaust fan but no intake vents (i.e. holes in the front or bottom that let air in) then you may just suffocate the parts, which won't help cooling at all. Likewise, if you have an intake fan but no exhaust vents (nearly every case should have some grills on the back that act the same), then you'll just be sucking dust into the computer and not actually cooling it. That's typically why both are needed.
If you find the fans you're using too noisy, consider buying fans with lower speeds. Scythe do case fans for all sorts of situations.