When that time comes is some indeterminate point in the future. There have been revisions of lithium battery (never mind nicd and nimh before it) that would in turn necessitate whole new setups for charging and management thereof.
Having wires pre welded with a fancy connector on the end would at some level mean a drop in replacement for a dud battery is a thing that can be done if you don't want to weld your own or fabricate some more conventional spring connectors or something. Though again you would be subject to whatever requirements that system has.
Anyway there are three aspects for the setup I would note
The setup needs to deliver a suitable voltage (and presumably current but most things are going to manage that OK) to the relevant pins on the GBA. This is two AA batteries (normally considered around 3V but AA batteries go lower than that before they are considered spent so you get to figure out a cutoff there if you want).
Most lithium batteries are considerably beyond 3V (what the upper limit for the GBA is we might have to debate, though generally 10% or 20% above stated is what most electronics projects go for, that is to say 3.6V is probably getting a bit high) so probably some form of regulation will be needed. Whether you burn it off with resistors, diodes, regulators or do something a bit more flash with more modern DC-DC conversion options is up to you.
Lithium is generally noted as being rather finicky as to how it is charged -- constant current sections, temperature sensing maybe, top up charge vs more bulk charging and that is ignoring any deep discharge recovery and ultimate battery lifetime considerations (fast charge will be fast charge but comes at a cost of total number of charges in a lifetime).
To this end you want a charge management controller.
Part of this will also presumably be any short circuit protection, thermal runaway, overcharge (will putting in a wall charger do something?) and all the rest. Some batteries will come with this preinstalled, that Amazon link claiming multiple such features. Others will want the charge management device to handle it.
Most charge management devices these days will have a USB socket in them (kind of obvious to really).
Higher capacity. Future developments are future developments (there is a whole lot of chemistry, physics, manufacture, materials and all things besides going on in that world). Generally though energy density is fairly fixed right now, so may end up physically larger and all that it entails. If you wish to further modify the case (an eminently achievable act) or have it stick out further (I have a vintage GBA battery pack here that is over double the size of a normal GBA flush fit battery for instance) then play that one as you will, though note that lithium cells are not overly fond of being pierced, bent, struck or otherwise mechanically damaged (as in fire and smoke and lots of it).
Assuming you replacement is of a compatible chemistry and physical size then do also be sure the connector is wired the same way round -- reversing the polarity can lead to all sorts of fun.
That said assuming you found rather than a 1500mAh that it comes with in the first link a 2000mAh battery (and broke out the 3d printer or cutting tools to enlarge the battery space) of the same/compatible chemistry then most charge management setups will still charge it. It is not like trying to charge a car battery with a wall wart that normally only powered a LED torch even if too was nominally the same voltage.
That said replacement battery is not a great feat of engineering in this sort of thing. Make battery produce 3V at enough current, using DC voltage changing options if necessary. Include proper charge management with battery. To that end if in 5-10 years that battery no longer does the business you can generally turn around and buy whatever the current hotness is off the shelf and put it together.
The $25 in the opening post is a bit steep for my taste but at the same time I will not knock someone that just wants a presized and generally assured to work drop in replacement for them.
hmmm I'm a terrible pirate lol
