AIO should never flash EZ4 NOR, it has an older/different NOR part ID so it should never detect the same as a 3in1. Me and smiths worked through this already in AIO 1.2 or so, I can say my own EZ4 deluxe is working perfectly. It starts in DS mode EZ4 menu when I select the GBA cart in AIO's main menu and press L+A, and starts in EZ4 GBA mode menu when I just press A.
The problem is actually quite simple, for some reason your EZ4 is being detected as a EZ3in1. Are you using the latest 1.41+updated loaders from yesterday AKAIO? (as I said, this was corrected a couple versions ago around AIO 1.2 or so, anything older will display similar issues to those you describe, no idea how official firmware will react.) How long ago did you get your EZ4 (if it was manufactured very close to when the first 3in1 came out there is a slight chance they shared the NOR part until EZ4 was discontinued, my own deluxe was supplied by EZ team when they initially started selling.)
If you dig around for some of my old stuff on sosuke you should be able to find a series of small nds homebrew in the EZ4 section called "EZReset" which will take you to GBA mode without reinsert, and possible "EZResetDS" though that one never had the proper command to fully switch back to OS NOR and doesn't appear to work with the current AIO (white screens.) I may revisit these homebrew if your issue isn't an outdated AIO, just let me know and I will do what I can to make them compatible with latest AIO.
The solution itself is simple though, at the end of 3in1 check if 3in1 is not found (the check cannot be vague, it should check the NOR ID against known chip list) there should be a
SetRomPage(0x8000);
Make an ini option that forces this to be done regardless of the 3in1 check, and people with the issue could use the ini option. What is currently happening is, if it misdetects as 3in1 and doesn't execute this page set then AIO sees the NOR memory instead of the OS/menu memory. If someone digs back, I also gave a secondary check that can be performed directly before the 3in1 check that requires only reading a few bytes off GBA bus to see if EZ3/4 OS is there prior to doing any page sets for 3in1 check.