Some of the late Halo games had something like this, as well as the option to customise things. Halo Reach was probably the first with a mode it called firefight, though I am told later games did more here (never really played those).
The "whored mode" is likely a pun on horde mode, something Gears of War somewhat popularised (though Halo mentioned above does things here). If you do want that then I see Gears of War everywhere there is cheap games. It is also a reasonable search term for the style of game
https://www.giantbomb.com/horde-mode/3015-3021/games/ . Some also call them survival mode but that is a harder term to use, though wave survival might be one.
I don't know what ROM hackers have here these days -- the idea of a boss rush is very old news at this point but endless waves is a bit different. PC game mod makers might be a different matter.
Some dungeon crawlers (though we have now departed anything resembling shooting game, unless you mean twin stick efforts) have challenge runs that function much like this, and some probably have endless survival or can be twisted to it.
Most PC games these days got bitten by the tower defence bug so will include various token efforts there, however I did play a plants vs zombies shooter that was surprisingly good.
Depending upon what you want to do you might be able to do something with cheats -- I have been playing Far Cry 3 lately and it has some challenge options within it that if you gave yourself infinite time you would be able to do something like what you want.
