Are you talking about the original gameboy? Well for the games that save data to the game pak, Pokemon, Gameboy Camera, etc. so lose the ability to save your progress.
For the Gameboy Advance most games use EPROM and don't have a battery to worry about, but the Metroid games do use the same SRAM saving as the above. Pokemon on GBA only uses the battery for the real time clock that keeps track of berry growth and the day/night cycle.