I assume this is for the original EZ4, the various lite models will differ here but the logic is the same (replace like for like unless you can improve it and want to).
The earlier models of the EZ4 had a rechargeable battery. A panasonic VL2020.
Later models dropped the rechargeable battery. Unknown reasons really, probably cost. CR2020 is the usual battery of choice.
At no point did the EZ4 (or anything before it) have proper recharge circuitry. To that end any ??2020 battery* that puts out 3V will do, you want one with tags on opposite sides though. Anybody that can claim to solder electronics should be able to replace it. You might be able to squeeze a slightly larger battery in there, though check before you order things online.
*it varies a bit between manufacturers but the two letters tend to indicate a model type and chemistry, the first two numbers are the width in mm, the second two numbers are the height with a point in the middle (20=2.0mm)
http://www.batterystore.com/Charts/CoinCell.htm has more if you want to get the idea of the concept.
On saving. There are four broad classes of save including no save at all and each of those has in the order of 20 sub classes. No flash cart emulated this save type exactly and instead every game is patched (it is a simple process) to use a type of SRAM that is present on the cart. SRAM, like many types of RAM, needs power to hold data which is what the battery is for.
You save as you would normally in the game and then upon the next reboot the save is taken from the SRAM and put onto the miniSD card. The saves are then taken from the miniSD and loaded back into SRAM for the game to use when you launch the game.
The battery will eventually die, it should take years before that happens though (do note though that your cart may be years old before it comes to you). When it does you just replace it. No workarounds to try to increase the battery life other than if you have a rechargeable in there have it in a powered GBA slot once in a while to keep it topped up. As far as "save and turn off" type games go it means nothing as far as the programming, electronics and signals side of things go. You can however save in the game and if in the game it allows you to exit to menu and load a save back up then that is quite fine.
If you did see something about soft reset workarounds in your reading that is not so much for battery life but something you can do between noticing the battery is on the way out and getting a new one installed. The idea runs that you patch a game with soft reset functionality, save normally in the game and then soft reset back into the menu. At no point should the SRAM have been without power so your save will still be kept. Soft reset is not quite available for all games and I tend not to use it, but if it is a big RPG ish thing where saves matter then chances are it will have the option to patch it for soft reset.