Whether or not a new PSU fits in the case depends on the size of the case and the size of the PSU. "Slim" cases generally can't fit the more powerful PSUs, although bog-standard cases can fit any non-extended PSU (which usually means anything up to 750W or so).
Cases can be relatively cheap, but also quite expensive, depending on what you want in it. I can probably pick up a functional case for $20 to build mid-ranged computers, which leaves another $40-60 for a decent PSU. Of course, that also means that you won't get so much money for the GPU.
EDIT: In response to maxlwin, cost for a Core i3 upgrade kit (with RAM and motherboard): ~$250-300. Cost for a complete mid-ranged computer (assuming HD5770 and 1TB HDD), ~$600-700. Cost for replacing the case and PSU alone, ~$50-150 (depending on how good you need them to be). It's all relative. The motherboard, CPU and RAM make up the "guts" of the computer. The Case and PSU make up "body" of the computer. The drives are only added on once the rest are sorted out, and the graphics card should be the last part bought for the computer because the longer you wait on it, the more money you can save up, the cheaper the prices get and generally the stronger the GPU you get in the end.
Cuz all of this is confusin' the hell out of me, for real.
