I would probably draw a line somewhere between being unable to see what is going on around you, definitely if there is big movement involved. Chucking up say a virtual chess board to a set of glasses that shines a light in your eye I can't see as being all that different to my playing chess on my GBA, and more "classical" sci fi versions of augmented reality would also be there. Of course someone would then turn around and do full bore immersion but with a real world map and that would get hard.
For what I imagine the initial forays to be though then if I can't turn my head and look at you then probably not. I would however not have any objection to someone sitting on a bus unable to see what is going on around them.
I imagine most future VR headsets will be of the transparent variety, to be more versatile, offer AR and VR.
The portable/"handheld" mode, when not used for AR, would show the game in a frame on the glasses, probably in the lower part of the glasses while leaving the rest transparent, so you could peer over it at people like you're wearing those old school bifocals. If the headset has headtracking it could "hang" the game frame as an object in space and it would hang there when you look away (Hololens-style).
For full immersion you would just stick a sleep mask on the whole thing.
