its easy to see why.
1st- the price, it was preety damn high for a portable at a time, people complained the 3ds released one year prior was already too expensive for a portable at 250 thats why nintendo 6 months latter droped the price to 170$ and then vita came one year later at 300(basicaly almost double the price of 3ds), heck it costed more than preety much any other console at a time portable or not.
2-Memory cards, sony yet again instead of using regular micro or sd cards like nintendo did made up their own memory cards and they are more expensive than games, with little storage really and vita basicaly needing them since almost every game installs a huge chunk of it on internal memory, unlike 3ds wich is just updates and very rarely needs an install outside of the cartrige, you would end up paying over 80$ for a 32gb card its ridiculous i remenber 4gb cards were like 20$ and that could fit maybe 3 games if they didnt install much ...
3-Exclusive games are rare and very poor quality, seriouly if you can name 5 good exclsuive games that were released outside of japan you will be listing already crappy games, vita only had indie games and some ports here and there, there isnt a single game you go out and say i need to own a vita to play this game.