Music/Sound for DS Hombrew

[Project68K]

I am working on a homebrew game for the Nintendo DS with C++ and Nightfox Lib, and I am having a little trouble implementing the sound properly. Using NF_LoadRawSound, and NF_PlayRawSound is simple enough, but I want a way to be able to check if the sound or music has stopped playing. Is there a way to implement this, or is there no hope?

 

[NotImpLife]

Sound on DS can make you scratch your head sometimes. Here is a more technical approach.

As you want to play raw sound, that means each piece of sound would use exactly one out the 16 DS audio channels. The channels's id is told by the return value of NF_PlayRawSound, which internally calls the libnds function soundPlaySample.

Now, take a look at SOUNDxCNT register, where you replace "x" with the channel id previously obtained. It is a 32-bit register which contains useful playback data such as the volume, pan, loop, and last but not least, the state of the channel. Bit 31 of SOUNDxCNT is set to 1 if the sound is playing, and 0 if it is not.

Therefore, the strategy would be something like this:

int nf_slot_id = /*some value of choice*/;
NF_LoadRawSound("my_sound", nf_slot_id,  my_freq, my_format);

bool loop = false;
int channel_id = NF_PlayRawSound(nf_slot_id, my_volume, my_pan, loop, 0);

if(!sound_playing(channel_id))
{
    // sound has stopped, do something...
}


////////////////////////////////////////////////////////

bool sound_playing(int channel_id)
{  
    // get bit31 of the word at 40004x0h, x=channel id
    return *((int*)(0x04000400 | (channel_id<<4))) & 0x80000000;
}

However, this solution raises even more questions related to the DS low-level behavior. libnds defines the 40004x0h registers as volatile (meaning the system can change them without programmer's knowledge). It is also defined as part of the ARM7 binary, not ARM9 (which exaplains the "ugly" way of accessing it in the above code). My attempt of reading the sound reister always returned 0 (you can also try it for yourself and hope for the best), and it seems like the 0x04xxxxxx memory region is different for the ARM9 and ARM7. With the emulator's memory viewer, you can see that on ARM7, the sound register is indeed something like 0xD0400040, while ending the sound changes it to 0x50400040 (note the bit 31 being reset to 0). Therefore, this is worse than I initially imagined. You may need a FIFO to have the ARM7 communicate the sound state to the ARM9 and then the problem is solved.


PS: There is an even simpler method which requires DS timers. It is as follows:
- you have to know the sound duration beforehand, assuming you don't change the playback frequency in the middle of the sound
- start both the sound and the timer
- when timer exceeds the sound's duration, it means the sound has finished, unload the sound, stop the timer and make the code your dreams become reality.

Related on how to use the timer, also check this post and libnds documentation.

You may need to be very patient to solve sound issues on NDS, I truly hope maybe someone in the future will come up with smarter and easier tricks to achieve the same thing.

Good luck! Please keep us updated with your work!

 

[Project68K]

Sound on DS can make you scratch your head sometimes. Here is a more technical approach.

As you want to play raw sound, that means each piece of sound would use exactly one out the 16 DS audio channels. The channels's id is told by the return value of NF_PlayRawSound, which internally calls the libnds function soundPlaySample.

Now, take a look at SOUNDxCNT register, where you replace "x" with the channel id previously obtained. It is a 32-bit register which contains useful playback data such as the volume, pan, loop, and last but not least, the state of the channel. Bit 31 of SOUNDxCNT is set to 1 if the sound is playing, and 0 if it is not.

Therefore, the strategy would be something like this:

int nf_slot_id = /*some value of choice*/;
NF_LoadRawSound("my_sound", nf_slot_id,  my_freq, my_format);

bool loop = false;
int channel_id = NF_PlayRawSound(nf_slot_id, my_volume, my_pan, loop, 0);

if(!sound_playing(channel_id))
{
    // sound has stopped, do something...
}


////////////////////////////////////////////////////////

bool sound_playing(int channel_id)
{  
    // get bit31 of the word at 40004x0h, x=channel id
    return (0x04000400 | (channel_id<<4)) & 0x80000000;
}

However, this solution raises even more questions related to the DS low-level behavior. libnds defines the 40004x0h registers as volatile (meaning the system can change them without programmer's knowledge). It is also defined as part of the ARM7 binary, not ARM9 (which exaplains the "ugly" way of accessing it in the above code). My attempt of reading the sound reister always returned 0 (you can also try it for yourself and hope for the best), and it seems like the 0x04xxxxxx memory region is different for the ARM9 and ARM7. With the emulator's memory viewer, you can see that on ARM7, the sound register is indeed something like 0xD0400040, while ending the sound changes it to 0x50400040 (note the bit 31 being reset to 0). Therefore, this is worse than I initially imagined. You may need a FIFO to have the ARM7 communicate the sound state to the ARM9 and then the problem is solved.


PS: There is an even simpler method which requires DS timers. It is as follows:
- you have to know the sound duration beforehand, assuming you don't change the playback frequency in the middle of the sound
- start both the sound and the timer
- when timer exceeds the sound's duration, it means the sound has finished, unload the sound, stop the timer and make the code your dreams become reality.

Related on how to use the timer, also check this post and libnds documentation.

You may need to be very patient to solve sound issues on NDS, I truly hope maybe someone in the future will come up with smarter and easier tricks to achieve the same thing.

Good luck! Please keep us updated with your work!

I think I understand some of the code with the bits that you were using. Let me see if I can paraphrase it in one sentence, you are isolating the 31st bit, and returning it. There are only two things that confuse me:

- The first being why did we specifically move it 4 bits.
- The second being what the 0x800... stands for.

 

[NotImpLife]

I think I understand some of the code with the bits that you were using. Let me see if I can paraphrase it in one sentence, you are isolating the 31st bit, and returning it. There are only two things that confuse me:

- The first being why did we specifically move it 4 bits.
- The second being what the 0x800... stands for.

1. If channel if is for example 8, we need to check the register at 04000480h. The register X (for any X ranging from 0 to 15 (0 to F in hexadecimal)) is located at address 4000400+ X0h. X0h can be rewritten as X*10h = X*16. Left shift by 4 has the same effect as multiplication with 2^4=16, hence the formula 4000400h + (X<<4). We can also observe that in this case we can safely replace addition with the bitwise or, hence the formula above. Assuming the compiler optimizations are disabled, 4000400h | (X<<4) would produce faster code than 4000400h + 16*X, which basically lead to the same result.

2. 0x8000_0000 is 1000_0000_0000_0000_0000_0000_0000_0000 in binary ( one followed by 31 zero-s). 0x80000000 is a bit mask that identifies the bit 31 in any binary (int) number. The result of X & 0x80000000 can either be 0x80000000 or 0x00000000, depending on whether that bit 31 is active or not. Now that I think better, 1<<31 would be a better expression of the number 0x80000000 in that context.

I remind you that the computations I made are just an example of a method that tells the value of bit 31 and can be done in other different ways.

Later edit: I deeply apologise for my mistake in the above code! I wasn't indented to check bit 31 of the address, but of the value it lies at that address. Please let me repair that:

#define BIT_31_OF(x) (x>>31)

int* sound_register_address = (int*)(0x04000400 + channel_id*0x10);

int sound_register_value = *sound_register_address; // I forgot dereferenciating

return BIT_31_OF(sound_register_value) == 1;

I also modified it in the previous post.

 

[NotImpLife]

I managed to create a working example starting from the libnds FIFO template provided by devkitPro. The idea is as follows:

- ARM9 requests the sound register value from the ARM7
- AMR7 sends back the response(*)
- ARM9 receives the register, checks its bit 31 and reacts accordingly

(*) ARM7 may send value 0 right after starting the sound, which has bit 31 equal to 0 so it is interpreted as a false-positive flag that sound has stopped (though it hasn't even started). So it seems like it takes some time for the registers to be updated. Also, after a sound stopeed playing, the sound register does not change its value to 0. You need to manually do that on ARM7 if you want to. To solve that, I chose to store the actual sound state in a custom bool variable "sound_playing" and do some extra checks to make sure the sound stop event is fired exactly once. Please check out the code below which plays ~15 seconds of audio and then displays a "Sound stopped" message.

 

[Project68K]

I managed to create a working example starting from the libnds FIFO template provided by devkitPro. The idea is as follows:

- ARM9 requests the sound register value from the ARM7
- AMR7 sends back the response(*)
- ARM9 receives the register, checks its bit 31 and reacts accordingly

(*) ARM7 may send value 0 right after starting the sound, which has bit 31 equal to 0 so it is interpreted as a false-positive flag that sound has stopped (though it hasn't even started). So it seems like it takes some time for the registers to be updated. Also, after a sound stopeed playing, the sound register does not change its value to 0. You need to manually do that on ARM7 if you want to. To solve that, I chose to store the actual sound state in a custom bool variable "sound_playing" and do some extra checks to make sure the sound stop event is fired exactly once. Please check out the code below which plays ~15 seconds of audio and then displays a "Sound stopped" message.

Is it possible to do all this in a for loop? It seems like a pretty dumb question, but I tried implementing this in a for loop. Basically, my idea was to using the NF_LoadRawSound and NF_PlayRawSound to play through an array of raw music files, which all together would make a full song. However, it seems like it's skipping everything else and going to the end.

 

[NotImpLife]

Is it possible to do all this in a for loop? It seems like a pretty dumb question, but I tried implementing this in a for loop. Basically, my idea was to using the NF_LoadRawSound and NF_PlayRawSound to play through an array of raw music files, which all together would make a full song. However, it seems like it's skipping everything else and going to the end.

While the implementation of this fact is completely up to you, I would suggest an event-based idea which uses the sound stop "callback" as the perfect opportunity to launch the next song. Note that ARM7's only job is to continuously communicate the sound register's value, so the only changes we have to do is on the ARM9 side:

// index of the current song to play
int song_id = 0;
// playing the sound - replace this with the NFlib equivalent
int channel_id = soundPlaySample(sound[song_id], SoundFormat_16Bit, bgm_bin_size, 32000, 64, 64, 0, 0);  
bool sound_playing = true;

int vblank_cnt = 0;
while(1) {      
    fifoSendValue32(FIFO_USER_01,0);

    int val = fifoGetValue32(FIFO_USER_01);      

    goto_xy(1,3);      
    iprintf("VBLANKCNT = %i\n", vblank_cnt++);
    goto_xy(1,4);
    iprintf("SOUND0CNT = %08X\n",val);

    if(val!=0 && sound_playing && (val >> 31)==0)
    {          
        goto_xy(0,6);      
        iprintf("Sound stopped!\n");          
        soundKill(channel_id);
        sound_playing = false;
       
        song_id++; // go to the next sound      
        if(song_id < soungs_count)
        {
            // start the sound
            channel_id = soundPlaySample(sound[song_id], SoundFormat_16Bit, bgm_bin_size, 32000, 64, 64, 0, 0);  
            sound_playing = true;      
        }
        else
        {
            // all songs have been played
        }
    }

    swiWaitForVBlank();
    scanKeys();
    if (keysDown() & KEY_START) break;
}

You can also do this using a "for"-loop, which contains a while loop that waits for the song to be finished:

// kinda like this
for(int song_id = 0; song_id < song_count; song_id++)
{
    int channel_id = soundPlaySample(songs[song_id], SoundFormat_16Bit, bgm_bin_size, 32000, 64, 64, 0, 0);      
    bool sound_playing = true;

    int vblank_cnt = 0;
    while(1) {      
        fifoSendValue32(FIFO_USER_01,0);
        int sound_reg0 = fifoGetValue32(FIFO_USER_01);      

        if(sound_reg0!=0 && sound_playing && (sound_reg0 >> 31)==0)
        {                      
            soundKill(channel_id);
            sound_playing = false;
            break; // exit while loop when song ends
        }

        swiWaitForVBlank();      
    }
}

 

[Deleted member 591971]

nflib has that function for sound, convert the audio to raw 8bit signed pcm using audacity and that should work. if the sound effects are in wav format you can use maxmod.



now if you are talking about music then you are completely screwed unless your music is in mod format, use maxmod, if your music is in wav format then you can stream the data from sd/nitrofs, i never got this working and that is why i still use a custom version of PAlib (please do not use it.) if you are wondering about loading the music as a sound effect on nflib, you can't do that because music files are huge, and nflib size limit is pretty small.

 

[Project68K]

While the implementation of this fact is completely up to you, I would suggest an event-based idea which uses the sound stop "callback" as the perfect opportunity to launch the next song. Note that ARM7's only job is to continuously communicate the sound register's value, so the only changes we have to do is on the ARM9 side:

// index of the current song to play
int song_id = 0;
// playing the sound - replace this with the NFlib equivalent
int channel_id = soundPlaySample(sound[song_id], SoundFormat_16Bit, bgm_bin_size, 32000, 64, 64, 0, 0);
bool sound_playing = true;

int vblank_cnt = 0;
while(1) {    
    fifoSendValue32(FIFO_USER_01,0);

    int val = fifoGetValue32(FIFO_USER_01);    

    goto_xy(1,3);    
    iprintf("VBLANKCNT = %i\n", vblank_cnt++);
    goto_xy(1,4);
    iprintf("SOUND0CNT = %08X\n",val);

    if(val!=0 && sound_playing && (val >> 31)==0)
    {        
        goto_xy(0,6);    
        iprintf("Sound stopped!\n");        
        soundKill(channel_id);
        sound_playing = false;
     
        song_id++; // go to the next sound    
        if(song_id < soungs_count)
        {
            // start the sound
            channel_id = soundPlaySample(sound[song_id], SoundFormat_16Bit, bgm_bin_size, 32000, 64, 64, 0, 0);
            sound_playing = true;    
        }
        else
        {
            // all songs have been played
        }
    }

    swiWaitForVBlank();
    scanKeys();
    if (keysDown() & KEY_START) break;
}

You can also do this using a "for"-loop, which contains a while loop that waits for the song to be finished:

// kinda like this
for(int song_id = 0; song_id < song_count; song_id++)
{
    int channel_id = soundPlaySample(songs[song_id], SoundFormat_16Bit, bgm_bin_size, 32000, 64, 64, 0, 0);    
    bool sound_playing = true;

    int vblank_cnt = 0;
    while(1) {    
        fifoSendValue32(FIFO_USER_01,0);
        int sound_reg0 = fifoGetValue32(FIFO_USER_01);    

        if(sound_reg0!=0 && sound_playing && (sound_reg0 >> 31)==0)
        {                    
            soundKill(channel_id);
            sound_playing = false;
            break; // exit while loop when song ends
        }

        swiWaitForVBlank();    
    }
}

How did you get the bgm_bin_size? Is there an equivalent for RAW files?

 

[Project68K]

OK, so I tried implementing some of the things that were suggested in this thread. So far, the first segment of the song is able to play without any issues. However, it does not seem to continue playing the rest of the segments. This is what my code looks like so far:

#include <nds.h>
#include <nds/arm9/sound.h>
#include <nf_lib.h>
#include <iostream>
#include <vector>

using namespace std;

void InitSystem()
{
    NF_Set2D(0,0);
    NF_Set2D(1,0);
    NF_InitRawSoundBuffers();
    NF_SetRootFolder("NITROFS");
}

int main() {
    InitSystem();
    soundEnable();
    const char* music[4] = {

        "dsSongSegment2",
        "dsSongSegment3",
        "dsSongSegment4",
        "dsSongSegment5"
    };

    //jPlayer.playMusic("music.txt",0,22050,0);
    NF_LoadRawSound(music[0],0,22050,0);
    NF_LoadRawSound(music[1],1,22050,0);
    NF_LoadRawSound(music[2],2,22050,0);
    NF_LoadRawSound(music[3],3,22050,0);
    int slotNumber = 0;
    int channel_id = NF_PlayRawSound(slotNumber,127,64,0,0);
    bool sound_playing = true;
    consoleDemoInit();
    while (1)
    {
        cout << slotNumber << endl;
        fifoSendValue32(FIFO_USER_01,0);

        int val = fifoGetValue32(FIFO_USER_01);

        if (val!=0 && sound_playing && (val >> 31) == 0)
        {
            soundKill(channel_id);
            sound_playing = false;


            slotNumber++;

            if (slotNumber < 4)
            {
                channel_id = NF_PlayRawSound(slotNumber,127,64,false,0);
                sound_playing = true;
            }
            else
            {

            }
        }

        swiWaitForVBlank();
    }
    return 0;
}

 

[Project68K]

nflib has that function for sound, convert the audio to raw 8bit signed pcm using audacity and that should work. if the sound effects are in wav format you can use maxmod.



now if you are talking about music then you are completely screwed unless your music is in mod format, use maxmod, if your music is in wav format then you can stream the data from sd/nitrofs, i never got this working and that is why i still use a custom version of PAlib (please do not use it.) if you are wondering about loading the music as a sound effect on nflib, you can't do that because music files are huge, and nflib size limit is pretty small.

Aight. I think I figured it out. It's not completely perfect, but it is near perfect. Here's what I have done

We have a an array of const chararacters that will have the name of the raw files, in sequence. In the main structure, we define the song id, which will be 0. It is the first index of the array. We also load the first file by saying:

NF_LoadRawSound(arr[song_id], 0,22050,0).

After this, we play the sound file by assigning an int variable (which will be the channel the sound is playing on). After this, we define our two most important variables: the soundTimer, and the sound Delay.

The sound timer will increase by 1 * the delta time (in this case, it will be 1/60, which gives 0.0166666...) in the while loop. We set sound timer to 0 for now.

The sound Delay is the max time allowed for sound timer to increase. This, and the sound timer, will be a float , and the delay is a bit tricky to assume, and is really found by a lot of trial and error. Since the sound timer will be a float, you will need to do two things:

- Know how long a song segment is (10 seconds in my case)
- Use an emulator and print the sound Timer to the screen to roughly estimate where the song segment stops.

Once you've done that, we are on to the main loop. We increase the sound timer as we said before, and if the sound timer is greater than or equal to the sound delay, then we immediately unload the raw sound on slot 0, and kill all sound on the current channel using the channel variable we talked about before.

We increase the song_id, and we check if the song_id is less than the amount of items on the array. If it is, then we load arr[song_id] into the SAME slot, and then play it again.

What we essentially are doing is loading up a queue of RAW segments into 1 single slot. This way, we don't have to use multiple slots for one song. I was actually shocked it somewhat worked like it did. It sort of uses the idea suggested in NotImpLife's most recent post in this thread, but without using the more low-level, and seemingly unreliable aspects of the DS sound register and arm7.

Here's the code:

#include <nds.h>
#include <nds/arm9/sound.h>
#include <nf_lib.h>
#include <iostream>

using namespace std;


void InitSystem()
{
    NF_Set2D(0,0);
    NF_Set2D(1,0);
    NF_InitRawSoundBuffers();
    NF_SetRootFolder("NITROFS");
}

int main() {
    InitSystem();
    soundEnable();
    const char* music[5] = {

        "ds_song1",
        "ds_song2",
        "ds_song3",
        "ds_song4",
        "ds_song5"
    };

    int song_id = 0;
    NF_LoadRawSound(music[song_id],0,22050,0);
    
    int channel_id = NF_PlayRawSound(0,64,64,0,0);
    float soundTimer = 0;
    float soundDelay = 10.0167;
    
    consoleDemoInit();

    while (1)
    {
        
        soundTimer += 1 * 0.01666666666;
        if (soundTimer >= soundDelay)
        {
            NF_UnloadRawSound(0);
            soundKill(channel_id);
            soundTimer = 0;
            
            song_id++;

            if (song_id < 5)
            {
                NF_LoadRawSound(music[song_id],0,22050,0);
                channel_id = NF_PlayRawSound(0,64,64,0,0);
            }
            else
            {
            
            }
        }
        swiWaitForVBlank();
    }
    return 0;
}

I have also added a download link to the actual project. Hopefully, someone is able to make this a lot better than it currently is: