I don't think I quite understand, my games are not installed as .nsp games, they are installed in the retroarch library and I can't press and/or hold R because it acts as a "down button", the menu won't stay still when I try to run a game. I always overclock to play n64 and none of them run too smooth.
I think you are misunderstanding.
First off, usage of SX OS with RetroArch is highly discouraged, because most of the feedback and support is being done through Atmosphere only.
Second, when we say NSP, we mean no homebrew app NSPs, as in no NSP forwarders or anything of sorts.
When we say that you should launch RetroArch through a game, that's literally what we mean.
If you have an installed game like Pokemon Quest or any other, if you are using Atmosphere, you can press and hold R when you launch the game (Pokemon Quest for this example), and instead of opening up the game, the Switch will open up the Homebrew Launcher alongside its list of apps. This is called "title redirection".
From here, you need to go to RetroArch and play like this.
The reason why we are using this method to run RetroArch is because if you don't, RetroArch won't have the required amount of RAM needed for certain cores to work properly, like N64, PSX and PPSSPP.
That's why we are telling you to use Atmosphere alongside title redirection, because otherwise heavy 3D cores won't work decently at all.