Define "custom soundtracks"
Are we talking about playing some MP3s in a media player, making some sort of tune play in the R4 menu, changing the soundtrack in a DS game, or what here?
You can play Mp3s in a media homebrew called Moonshell.
There's no way to play music on the R4 menu.
The only way to get a custom soundtrack on an individual DS game is through difficult rom hacking.