The non-removable battery argument seems to get strong in the event that the factory battery is as worthless as it is on the Vita and such.
People (including myself, and even OPM back then) LOL'd at the ridiculously bad battery life that the original PSP-1000 had, especially when using it the way Sony intended us to use it (everything off UMD). There were already extended battery accessories being released before the system came out. But at least that one was user replaceable.
The 3DS battery from Nintendo is also laughably bad, especially when compared to how epically long the DS lite would last on a single charge. But at least Nintendo also factored in being able to replace its battery fairly easy.
Non-removable batteries on a device isn't as bad if the device itself can last a long time between charges. But the less it lasts, the more the public will see it as unacceptable.
As I said, the reason that my friend's Droid RAZR MAXX is so lauded for its battery life, is that at least Motorola made up for not having a user-replaceable battery by giving it such a large battery in the first place, helping mitigate much of these concerns. Apple seems to finally have their head together regarding the battery with the latest gen of iProducts, with them lasting ten or more hours per charge. For the majority of users and usage patterns, that's basically a full day's worth of usage.
If Sony had bothered to not half-ass the Vita's design, and granted a large enough battery for that level of usage per charge cycle, then the criticism of it having a non-removable battery (well, without voiding warranty that is), wouldn't be as bad as it is. But as consumers, it is in our interests to voice these concerns, even to a company that seems to hold its user base in such contempt as $ony so readily does.
As for the proprietary memory issue, that's entirely $ony's fault, and is nothing more than a blatant cash grab. If Nintendo's 3DS games can get by without needing storage on the SD card, why can't Sony's Vita game card have space to store at least its save data? It isn't a fair comparison to the iPhone because the Vita already costs as much as Sony's home console, and is meant to be a gaming device first and foremost. An iPhone user can make due with less music and mp3's on their iDevice. It's entirely different to tell a gamer that they have to start deleting save data that they've worked weeks and months on to make room to store yet another oversized file.