You have effectively just asked to be taught the entirety of DS SDAT audio hacking which is a bit outside the scope of a reply to a forum post. However you seem to have many of the concepts behind it all already so I would like to think we can point you in the right direction.
SDAT audio is the most common audio format on the (it is not the only one and I try to have information on the others-
http://gbatemp.net/threads/the-various-audio-formats-of-the-ds.305167/ although that thread does include several choice links for SDAT hacking) and it is something of an all encompassing audio format.
SDAT houses three classes of audio file
SSEQ which is a tracker/midi style format
STRM which is basically a streaming format (indeed it uses almost raw PCM and ADPCM audio).
Sound effects, these come in a variety of formats although they are interrelated. Within reason any format can house things although developers do usually stick to would be restrictions.
On top of this SSEQ especially has several helper formats to define and hold instruments for it to use. Non sseq stuff is usually just a flavour of PCM or ADPCM give or take a small header and that is easily sorted so I will ignore it for the time being.
I cover various SDAT hacks in my hacking documents-
http://gbatemp.net/threads/gbatemp-rom-hacking-documentation-project-rewritten-for-2012.73394/ . You are going to want the PDF versions rather than the forum post versions right now though (they should be right at the start of the thread).
Still a list of tools and some quite explanations
Until fairly recently audio hacks took some effort with a hex editor or chancing that some of the existing tools functioned properly. Nowadays we have tinke-
http://gbatemp.net/threads/tinke-0-7-2.303529/ which can repoint files and adjust all relevant file sizes for you. You might still have to edit the occasional thing in a hex editor but it will be very guided rather than you having to figure out much.
On making audio
The original conversion software was called sseq2midi and there are several versions out there. When it came time to convert back the logical name was midi2sseq for which there are also a few versions but
http://gbatemp.net/threads/ds-code-sseq-player-and-new-mid-to-sseq-converter.301382/ is my preferred version these days (as it is by most doing such hacks). Related to that was the process of audio looping- theoretically the new midi2SSEQ supports various looping flags which will get converted into the SSEQ looping methods, before people had to go manual.
That is not the whole story though for as you say custom instruments are not only possible but used a lot. You can convert files to be played back on the PC more or less as they would on hardware with tools like
http://sourceforge.net/projects/vgmtoolbox/ which spits out 2sf format collections, however that is not so useful for the would be audio editor. Sadly there are no tracker tools like openmpt, awave studio or anvil studio that support sseq directly.
To that end we have tools like VGMtrans (
http://filetrip.net/nds-downloads/utilities/download-vgmtrans-92909-f27960.html ) which will export midi files and the popular soundbank format DLS, I will note that despite openmpt technically claiming support for the DLS format I have only had limited success using VGMtrans exported DLS files in it where awave studio will quite happily go along with it. I have limited experience with anvil studio though many audio hackers do like it a lot.
sseq2midi programs (used behind the scenes in a lot of programs) will also spit out a text listing of the commands being used and VGMtrans has a somewhat broken but still useful visual representation/decode of the format.
On editing SDAT.
Tinke takes care of file replacements, you should not have to go manual these days unless you are repointing, editing files directly (SSEQ is a crude scripting language of a sort so commands can be edited) or editing relationships.
However it lacks finer grain support as indeed do all other known tools, however the VGMtoolbox program I kind of dismissed a moment ago will spit out SMAP files. SMAP was originally a format used by the Nintendo SDK for the DS to allow developers to check relationships at a glance. Developers failed to remove SMAP files from a few games (most notably Zoids DS) and it led to a lot of reverse engineering work on the SDAT format, today it is a hacker standard and coupled with the specifications for the format itself is the thing that will help you most. Still it is not necessary in all cases as
http://filetrip.net/nds-downloads/utilities/download-crystaltile2-20100906-f23649.html will also pull information from the SDAT files in real time as it were sparing the need to generate the data as long as you know what you are looking at.
On specifications
http://kiwi.ds.googlepages.com/sdat.html and
http://www.romhacking.net/documents/469/
The former is slightly incomplete but definitely worth having and the latter is somewhat more readable (although it references the format a lot).
Now you have some tools to play with the two line summary of what you will want to do is find the file you want to replace, replace it with the relevant SDAT and then replace the bank for that file. If the file has multiple banks replace those and if those banks are shared between several SSEQ files then either suffer broken audio for other files (not so bad if the others are different language audio) or change the banks the file thinks it wants to ones you can afford to sacrifice (adding new banks is possible but a right pain to do- fortunately you are not usually forced into that situation) and replace those instead.
