Something is worth what someone else is willing to pay for it.
I could also link up
The everdrive has the ability to drag and drop (vs using a simple program/one of a selection of programs to patch), has faster loading (seconds), not sure about save emulation (the reason why pokepatch exists -- the patching changes games to use SRAM vs EEPROM or flash which they might have used instead) and real time clock (mainly a concern for pokemon fans). That is better than the alternatives presently available so it at least does not fail out of the gate by being more expensive and offering less. There has been some question as to the quality of newer EZ4 batches, mine has been fine but I was sent it by the team so I can't rule out being cherry picked one. My original EZ4 is good however many years on this is (I got it shortly after it was launched) and my EZ2 is also good.
In the end it is up to you. You ask whether you can play basically all GBA games (
http://gbatemp.net/threads/buying-a-gba-flash-cart-in-2013.341203/page-18#post-4756995 is the whole list of trouble games and workarounds) with the EZ4 and yes is the answer. There is not an awful lot to GBA flash carts unlike say the SNES which has all the special chips to deal with.
Anyway GBA flash carts that you will find out in the world.
New
EZ4. I don't want to go over the types again right now but EZ4 original, EZ4 lite, EZ4 lite compact (limited GBA support, don't get it for GBA), EZ4 lite deluxe (256Mbit of PSRAM and 384 of NOR, great little device if you can find one but it will likely not be cheap), EZ4 new batch (basically the same as EZ4 original) and EZ4 microSD which is the same as the original but has a microSD slot instead of a miniSD one.
Everdrive. See thread linked before.
EZ 3 in 1, which really needs a DS and DS slot flash cart to manage. Of the useful ones this will be the cheapest if you want the DS lite sized one. The repro makers and some electronics people rinsed the world of normal GBA cart sized ones so what is out there in GBA size is more expensive whenever I see it.
You can still find fire cards. These are old NOR carts and when they say 128MBit they mean it and you only have 16 megabytes of space. There are 256Mbit ones available but they are even harder to come by. Most find these occasionally on aliexpress and the like. These are likely the cheapest out there.
You can still find supercards. These have not been made in years as far as I know but they have lousy GBA compatibility and slowdowns and urgh. As DS carts they were supported for longer but nobody uses GBA slot carts for DS games any more so that is why they are still around.
Occasionally you might see some neoflash carts of various forms (the originals were rebaged XG2 GBA flash carts and actually not bad at GBA if you can get it sorted), some m3 expansion pack carts (not as flexible as the 3 in 1 if you don't have the m3 that it works with, though you can still play GBA games on it, if you do have the M3 DS real I think it was then you have some real time clock and more) and very rarely you might find an old GBA era thing for sale on wherever you find second hand stuff.