You can use any USB HDD and any SD card, but some brands are prfered over others. If you have one currently, you will be able to use that. Backups are stored on a different type of filesystem than windows files, so you will need to seperate whatever storage you are using into 2 partitions. Keep in mind though that games are usually around 2-4 GB, so a small crad will work, but only for small games.
And not all HDDs are over $100... I got a 500GB one for $87!