Glad someone who knows what they're talking about is commenting here.
Just for the record, no USB wall charger that's anywhere near spec will output MORE than 5 volts. The spec for USB, and for its wall chargers, is ALWAYS 5 volts. The current (amperage) varies, but the voltage is ALWAYS 5 volts.
And as far as the current goes, a power source is rated for how much current something can "draw" from it. That is, the power source isn't "forcing" a higher current and cannot fry something that's rated for less than the power source's current rating. In fact, you never want to use a power source rated for less than the device states — you will probably break something or start a fire.
All models of 3DS and New 3DS (as well as DS and DS Lite) are certainly capable of charging at 5 volts, and a cell phone charger would be more than adequate. The fact that it's rated for 4.6 volts officially isn't an issue because (and ONLY because) its integrated voltage regulators can handle a much wider range than "just" 4.6 volts. What WOULDN'T be adequate would be, for example, a bluetooth headset charger that outputs less than 500 mA — you'll probably damage or burn something, because the required current isn't high enough. Or someone mentioned their speakers' AC adapter outputs 9 volts which would also certainly burn a 3DS. But I can guarantee that such an adapter wouldn't be within USB spec, and should not have a USB port on it in the first place.
The 3DS's charging circuit is well-designed, and minor flaws and issues regarding charging voltage won't make a difference. Don't screw around with the power you provide to just any charging circuit, though. A poorly designed lithium ion battery charger can certainly cause a big, big fire.