I'm hunting for a PSP as well, so I'll lend some of my info to you.
I'd go with a PSP 1000. They're 100% hackable (right?), so you can buy one in confidence without having to worry about motherboards and the like.
You'll probably also need a Pandora battery to reset the firmware to an earlier, hackable version. You can get a really cheap Pandora from Dealextreme for $7. However, assuming you're US-based (according to your flag), if you've got cash to spend, you can buy one from ModChipCentral.com for about $25. Sure, it's more than 3x the price of DX, but they're a lot quicker. They ship from Winnpeg (which is like northeastern Canada, or is my geography brain farting?) and they delivered my R4 (which I bought from them) within a week. And I live in New Jersey.
You'll also need a decent sized memory stick. Sony brand ones are the most reliable, so you may even want to consider getting a crappy-sized one just purely for the hacking, as some of the non-branded ones don't work with it (well, I think). Then you can either buy a different memory stick (price varies depending on the brand) or you can buy a memory stick MicroSD adaptor and buy a giant sized MicroSD card. That's a good solution if you use a flashcart on your DS and are looking for a PSP.
Personally, my plans so far are...
- PSP 1000: $100 (at Gamestop, refurbished)
- Pandora battery: $25 (I'm an impatient bastard)
- Sony 4GB Memory stick (which can fit roughly 3 games, I think): $20.
So around $150 overall, probably a little more. And seeing as I have around $190 to spend, I may consider a bigger memory stick or buying some official PSP games. Or I could go with Dealextreme and buy some PSP games to wait out the shipping of the Pandora.