Wood seems to look at the game id and the arm7 load address, I'm not sure about other loaders. And indeed, if those commands are supported at all is really flashcard dependent. A fact is though that accessing romfs the Nintendo way is often way faster than the libnitrofs way of doing it, especially with large nds files.Huh. That seems a bit arcane and maybe even flashcard-dependent.
Do you have any clues about what causes the flashcard to decide if a particular .nds is homebrew or retail? It doubt it's something as simple as the presence of a NitroFS.