The limits tend to be more for browsing folders using the internal file viewer.
As long as you don't have to go browsing the save folder you should be OK. It might take a bit longer to save if it has to hunt through all the files in the file allocation table to get the right one but should still work for quite a bit more than that. I don't know what the limit will truly be but you would probably have to work hard to get there.
As ever though do make sure you grab backups from time to time if you are playing at that sort of level just in case there is some unforeseen consequence.
That said you are far from the first person to use an EZ4 to carry a sizeable chunk of the GBA library around with you at all points. They seem to manage it OK.