"64 bit" itself will not give you any kind of performance boost. 64 bit applications do not run faster than their 32 bit counterparts. The only benefit to "upgrading" to a 64 bit OS is that you can have more than 3 GiB RAM. Generally it is adviced to stay on 32 bit as long as you don't need that extra RAM, as 32 bit applications and drivers are usually more stable and faster.
In addition, there is very little performance difference between 2 GB RAM and more, for general computer use.
Yes, it does matter. Compatibility issues are a thing of the past - it's a myth and it should be ostracized. Everyday applications take full advantage of a 64bit system - it's the games that sometimes have issues because users actually believe that for some reason 32bit computing is more stable. It's not. Fair play, 3GB RAM is the recommended value, but not the minimal one. It'll work perfectly fine on 2 if you apply tweaks.
EDIT: Just to add a little bit of Layman's perspective, a processor that was manufactured to work with a 64bit system (and not the first-generation ones, mind you - 64bit has been around for a while now) is like a double-decker bus. Working in 32bit mode, you're "driving it" while only using the bottom floor for passengers, and for what? Because the chances of it toppling over are smaller? Nah.
Let's say that a program wants to pass a 64bit-long word. In 32bit mode, it has to divide it into a few operations rather than doing it in one go - hence the performance difference. The trade-off is a higher RAM requirement, but hey! Welcome to the future - progress requires sacrifices. All in all, it *is* a performance improvement.