If you are thinking you can get a full audio workstation almost at a laptop for not the most money these days where before it would have taken a whole room and tens of thousands then you are right. A stock machine of any type is probably not going to work for high end, for recording the average podcast or online video though it could sort of work.
Software is not really a problem, though a software with a workflow or more options than you care for is not going to help and as much as I like audacity it does not hold a candle to proper digital audio workstation (DAW) software, there are a few various free or open source ones but proper paid for stuff goes for a lot of money. There is also adobe audition.
For most the issue will come from hardware, fortunately there are external capture devices you can use to bypass this. You may also want to look into latency (
https://www.clicktorelease.com/code/speech-jammer/ or have a go with whatever speech jammer you like) for certain types of music.
Microphones are split in many ways, and that is before you get to wireless ones, but bus powered (the thing you plug into the red port on your computer probably gets its power from that, hence bus powered) and externally powered (USB as well sort of thing) are two distinctions and digital vs analogue is the other big one. Obviously you can go into connector type (most audio peeps would not be seen dead using 3.5mm jack and it is XLR at minimum). If you have a USB microphone though you can dodge much of this, the entry level but worth the effort stuff tends to be the snowball and blue yeti type things but there is a lot of other stuff out there.
A basic entry level electric guitar amp is the one they bundled with the guitarsmith game, if you have another amp and can feed an output from that into a more conventional external sound card then even better.
If you are feeling really fancy there are also things like
http://marantzpro.com/products/view/pmd661mkii and this could go on for a while (got any closed back/monitor headphones? They are really nice but tend to have a price tag to match).
There are drum beat (see drum pad controller) and other things you can get if you fancy a mixing board rather than using one on screen, you could possibly even build one (
http://tomashg.com/?p=975 ).
Looking at this I realise I am confusing things a bit and it is probably a bit too late for typing this sort of thing so I will leave it there. Hopefully I have at least given you some search terms. Short version is the little red ports on your computer are fine for skype and games and maybe some online video in a pinch but to do anything real you are going to want proper capture gear and an external sound card.
