Yeah, I have a phat PS2 as well. It would've been an awesome idea to just install a hard drive, and I'd be away. But my evil PS2 stopped playing CDs and movie DVDs a long time ago. (It still plays games, but takes a long time to load on startup).
To start up the whole thing you need to use a CD if I'm correct? That renders my PS2 useless. Maybe I should see if I can fix the laser though. Anyways, I'll stop moaning.
And as for the hard drive, yeah, you do have to go inside, but it's only a small partition of the PS2 at the back. So you don't really have to do much. Apparently it's as easy as pie.
Anyways, another drawback on the whole thing is it's incredibly time-consuming to set up, and you won't exactly find any of the hard-drives for cheap. I searcched on eBay, and they were all at least $100. Not good.
If I ever get the urge to play through all the old classic PS2 games, I'll just buy a second hand, already modded PS2. Not trying to put you off the softmod or anything lol, but I just felt it was unnecessarily hard to carry out.
And you just need to Google "PS2 softmod" and you'll probably find some guides.