1) lolicon will give you better fps when overclocking cpu only. gpu is not needed for any emulator on vita as for now. Depends on glvita is supported on not, only some homebrew games may support it. Don't expect a big performance up as retroarch is already overclocking a little, so depends on games may be usefull to cap some games at 60fps or remove some frameskip.
2) Mame 2000 was the better core, until mame 2003 plus. Romset is not that different from mame 2003 but support more games, performance and bug fix. It's still an active project :
https://github.com/libretro/mame2003-plus-libretro
3) FBA is better for some games (cpsX neogeo some shoot'them up ....) as mame is better for classic games (arkanoid rtype ... )
For fba core on RA, you have :
-2012 (0.2.97.30)
-2018 (0.2.97.43)
-fbalpha (0.2.97.44)
-fbneo (0.2.97.44)
2012 and 2018 are the latest version on said romset and not improved anymore.
fbalpha is stopped for fbneo since a few days ago. So fbalpha is now a snapshot of 0.2.97.44 and daily improvements goes to fbneo.
Performance wise, 2012 and 2018 seems to be the best, unless you want an unsupported game or an accuracy fix.
Latest version of pfba support 0.2.97.43, same as fbalpha2018. So you can use same romset for both. Pfba have better filters/user friendly(maybe depends on taste) but frame skipping is buggy on some games (makes your control crazy makes the game unplayable) where fb2018 frame skip is better. So pfba and RA fba2018 core are kind of equal and can have both without worring for space.
Still RA mame core is needed for classic games, prolly because of sound. If you disable sound on fba/pfba or lowering sample rate, it's playing fine)
4) Try mame 2003-plus , TMNT works fine with analog stick by default
Nice. I appreciate it. Now I have another question. Can I use both analog stick as normal with Retroarch, or is that only possible through Adrenaline?
Default mapping is left analog or d-pad for movement, right analog is not used by default so may be mapped for some custom key (load save snap hello)