N64 emulation is totaly possible, it has just enough ressources for a decent emulation (of course without texture packs and graphics upgrades like antialias, fancy filters, ...)
128MB FCram, dualcore arm + dedicated GPU + 4MB vram is enough, poeple proved that it is possible to push the hardware to its limits with enough patience
Wii64 is a nice example, and the wii had even less memory (3MB vram, 88MB ram (mem1 + mem2). And refering to the gpu, i think it is pretty much equivalent. Ok it is using a set of proprietary instructions to achieve some specific effects, but i think it has enough guts to handle a decent N64 emulation (not in 3D IMO) 30 fps is still enough for majority of games, only a few like f-zero X hit the 60 (with in counterpart very few textures & polygons)
But that's it, don't even think about dreamcast emulation or higher console. And I honestly doubt a vita has the guts to handle a gamecube emulation, but that's just my impression; the hardware is not final yet, and we know so little about it to have a serious topic
IMO which is likely possible to emulate (in a few years) :
Home Consoles :
NES,SNES (Famicon/S.Famicom), MasterSystem (Mark III), Megadrive (Genesis), Playstation, (no stable & optimized Saturn core yet, don't even think about it), N64, NeoGeo, TurboGrafX
Handhelds :
GB,GBC,GBA, (DS is useless) GameGear , NeoGeo Pocket, Lynx
Arcade :
CPS1, CPS2 (weak compatibility, memory is still short for some title), lots 80's systems allready supported by DS,
The possibles ports of well known games (ported from their open source code)
Quake, Quake2, (Quake 3 maybe but will be hell on memory), Doom (and mods), Duke 3D (build engine, possible port of Eduke32), and other GPL engines i forgot to mention which can be handled by the hardware
Systems that cannot be emulated decently (except from a god guru coder)
Home & Handhelds :
Dreamcast, PS2, Xbox, GameCube, Wii, PSP (and off course none of current gen, no PS3/ X360, don't be stupid)
Arcade Systems :
CPS3 (memory shortage), Triforce, Naomi (same as dreamcast), and other systems i forgot to mention for obvious reasons
Ports impossible because of their proprietary status (no code released)
Half-life & mods like Counter-Strike (the "CS" on PSP is a mod of quake, not the same engine, not the same feeling), morrowind & other closed source games even beyond 3DS capabilities.
P.S : sorry for the syntax / grammar mistakes, i tried my best, and i'm not using any trad tools
