No EZFlash device has had slowdown issues with the GBA. Supercards, supercard clones and some of the really oldest GBA carts had it but everything else works fine (give or take a save patch). The EZ2 then representing one of the most compatible flash carts out there.
At this point I imagine the battery is toast. You can try leaving it in the cradle/linker for a few hours to charge it and see whether it will keep them. Technically you could try the SMS thing (savegame management system, not sega master system which does have GBA emulators) but don't, or play games that don't care about saves.
The battery is a panasonic VL2020 and you will want tabs on yours unless you have a tab welder already. Sadly they are also used in BMW key fobs so replacement costs are often high (though you don't need the BMW version and a normal tabbed version will do).
If you can find another suitable rechargeable battery then great (there is not much in the way of charge management other than some basic current limiting).
You will probably want an XP install (or a virtual machine with it in) to operate the program -- I don't think we have anything like FAS1 for the EZ1/EZ2 at this point and while there is some source code I don't think much has been done yet.
Your main limitation then will be that 512Mbit (that is 64 megabytes for others playing along at home) does indeed mean that and with many games being 128MBit and things topping out at 256Mbit (
http://gbatemp.net/threads/buying-a-gba-flash-cart-in-2013.341203/page-18#post-4756995 has a list) you have sharply limited space.
I should also note you can probably sell the setup if you were that way inclined. While a nice 8Gbit EZ3 would be more desirable* and the market has faded a bit since EZFlash and Everdrive decided to drop their more recent efforts there are still those that want the legacy/vintage stuff and a 512 EZ2 is a really nice score we don't see come up often (might not fund an everdrive but you could fund an omega and maybe have a bit left over, or someone might do a straight swap).
*the ability to shove GBA carts in the linker and read them and possibly also read/write saves as well off in the PC... it is a nice ability to have, never mind having a solid little flash cart that probably also means you dodge pirate syndrome to play with as well.
