A curious set of skills there (generally find most that can resolder a resistor or capacitor will know this sort of thing) but OK.
I hate buying batteries from China. It is a bit like buying SD cards, however things can also set on fire (though usually it is just it is returned for not working and you get to eat it as you are not returning anything).
Anyway that is a charge management controller for lithium batteries (apparently li-po but you will be hard pressed to find older stuff these days) and some parts mention a boost converter as well (though I don't think it comes).
The general principle is fairly basic -- voltage is voltage in case (see Thevenin equivalent circuits if you want some textbook type headings to search) so if you have a suitable voltage output you wire the positive end of the controller output to the positive end of the battery compartment and the negative to the negative (both will be going into the board, the little jumpers to make you have 2-4 batteries in series are bits of metal and not much else).
AA batteries are typically rated at 1.5V each so if you have 4 you need 6V out and if you have two then 3V is more like it (stuffing 5V in probably being bad news). That said batteries don't stay the same voltage throughout their use and thus 5V in the case of 4 AA batteries might do something but you might also have to test that (this is why we have bench power supplies).
https://data.energizer.com/PDFs/E91.pdf
In some systems they will step the voltage down after the batteries to do their own thing with it and in those cases you might be able to remove the voltage dropper and go after that with whatever you have. For now I would probably just stick with replacing cells.
Boost converters are slightly newer devices in the electronics hobby space (technically been around for a long time but prices, output currents and more have only more recently become good enough for things like this).
They take a DC voltage (as in what you get from most batteries) and boost it to a higher level for things that need it. They are not 100% efficient (though quite good) so if you can get a battery within range then it is best to otherwise you are wasting power (and so play time) just getting it powered. With one of these you might then not have to put two batteries in series (which could well require two controllers) to make it reach higher voltages.
Lithium batteries are what most rechargeable batteries are now -- for their size and cost they store the most energy and have the nicest abilities. However you don't just apply a voltage and wait a bit like on some more basic setups, you have to do all sorts of fun things if you want said lithium battery to last for many charges and that is where the charge management device you linked comes in. Some batteries will have onboard options but others will want management.
Short version.
You need to find what voltage the devices you want to make battery kits for operate at.
You need then to buy batteries and a suitable charge controller.
If stepping up voltage then a boost converter that puts out a suitable voltage. If stepping down (can be done with things as simple as a diode)
Solder the outputs of the battery (or maybe in the case of the thing above the charge management chip) to where the normal AA batteries or wall adapter will go in.
Fit all this inside the battery slot of whatever device you have, something you put on the back, part of the empty shell or cut a section out and use that. Also find some way of taking power in so the person can plug it in and charge it without having to take it apart.
Here is a project someone did for the original GBA
https://gbatemp.net/threads/integrated-rechargable-gba-mod.465889/
I don't know if any of the people doing screen replacement kits, speaker mods and whatnot will do a premade battery replacement but I would have a look as it is a reasonably common thing for people to want.
