I moved from a 256 to 400 just because memory prices for the sd has been slashed heavily, I find it difficult to fill up the space. Just make sure you go with the sandisk / samsung to be sure.
When you are hoarding all the games and especially with the great collections of homebrew emulators and andrenaline psx/psp you won't play anything. I spend most of my time at the home screen looking for a game I want to play.