Headphones that have an in-line mic have a slightly different connector than the ordinary stereo minijack.
If either the connector on your regular headphones are damaged or the internal socket is damaged, this is a typical result.
The PS Vita is designed to utilize inline mics like cell phones do (their official Vita headphones have an inline mic) but I don't have any problem getting stereo output on my hi-fi 'phones which do not have a microphone built in.
Sadly, the fix is to try another pair of headphones to see if the Vita's stereo minijack socket is damaged, or try a headphone jack splitter (or extension cord) if the headphones in question are valuable (I had a $130 Bose set for like 4 or 5 years before it was unusable, "just replacing them" would have been absurd if a headphone jack splitter or extension cable would have stabilized the connection).