Should be possible, the Vita has enough power for that. The idea of the dreamcast back in the years was a 3dfx console 'Katana' (openGL based VitaGL). The Vita can handle it. Its all perfect, only the framrate is just to slow. It depends on the console and on the coder, truly has the Vita better ports than the PS3 console. DC is a 128bit console, but works different like PS2 or XboX or N64 with expansion. And other example is DaedulusX64 its not working good on the Vita, but a other N64 emulator works great on Wii that has less power than a Vita. Its all depends on the coder and the console himself.
edit:
The coder rinnegatamante is powerfull and has the knowledge that we dont have. He is currently prove himself with coding Doom 3 PC version to the VITA system. In this fact Dreamcast emulation should be kindergarten. He ports so many stuff in the same time, so we have to wait when he comes back for this project. I think he will find out with Doom 3 what is the limit for the Vita of porting games. This is profe of concept but better than we can imagine. The most trouble console was Sega Saturn, currently only the PS4 is able to emulate it. I am wonder what for magic has the vita for saturn ?
This emulator is near perfect, would be sad when rinnegatamante stop it to fix the framerate. Maybe we need 15-20fps more thats all. No graphics fix needed so far for testing. We could play with this one the alternate version from Soldier of Fortune or Unreal Tournament without porting the PC version.