I'm pretty sure that you can't exit out of any homebrew because the R4 won't use soft reset with homebrew. If they made an option in homebrew so that you could, it might work like moonshell.
Unless the homebrew program has support for soft-reset, you cannot do it. Moonshell has reset code, and there is a homebrew "rebootlib" library available, but very few programs support it.
wasn't it like Start Select R L (could be start select A B X Y) It works for some games, some doesn't work. And there are times when it doesn't work suddenly, but most of the times it works. Doesn't work for much homebrew, read the other replies. Make sure the soft reset (the green button, where if you touch it, it becomes red) is green
