Hypothetically speaking, a Photo exploit would be possible, but you must take into consideration the following variables.
It primarily depends on whether or not you are able to create a buffer overflow, and totally freeze the system. This also assumes that the Photo viewing app/program is NOT sandboxed. A true sandboxed app/program would be somewhat protected from such exploits, because it cannot freeze the entire system.
After all of that, the picture must contain ARM executable assembly code that is specific to Nintendo's proprietary variant of the ARM11 processor. This requires RAM research. Said RAM research is going to be much more difficult for TeamTwiizers (or whoever else makes an attempt), because unlike the TSOP chip package of the DSi, the RAM module within the 3DS is a BGA package. That means, the contacts to the chip are actually under the chip itself, and requires a complete removal of the chip via a hot air rework station, and a re-solder once the desired contacts are connected. Scanlime and the other folks over at TeamTwiizers have much experience working with TSOP packages. But, never have I witnessed them pull off a project that involves BGA packages, on a difficulty level such as this. Sure, they have desoldered BGA chips. Sure, they performed a RAM traces on TSOP chips. But have they ever performed a resolder of a BGA chip? Not that I know of....
Not to discourage anyone here, but although it is possible hypothetically speaking, to "hack" the 3DS in many different ways, I am sorry to say from my personal point of view that, this task simply doesn't seem possible, practically speaking.