ROM Hack Need some help reversing the DSE audio formats

[psy_commando]

So, I've been slowly reversing the Digital Sound Element http://www.procyon-studio.co.jp/dse/ format that's used by several Nintendo DS games, like Pokemon Mystery Dungeon Explorers of Time/Darkness/Sky.
I've got most of the most important stuff figured out, but I'm still not certain how to do a few things...

• For one, I'm not sure how the system computes the volume for each instruments/programs.. I've traced the execution a few times, but its really hard to follow what's going on.. All I know is that all the volumes for the master, the track, the program, and the sample split are combined together, and the result is a very large integer that is shifted several times over and placed into the NDS's sound control registers..
• And also, I was wondering if anyone had a suggestion on how to export the music from those games? I've been using sounfonts and midis to do it, but soundfonts lack the proper volume envelope parameters. DSE uses an envelope with an initial attack volume, and a fade-out while the key is still held, but its impossible to do that in SF. And the volume envelopes are linear in DSE, but they're logarithmic in the soundfont format. Also, the DSE format supports up to 4 "LFO" based effects per program, with an arbitrary high rate limit, depth parameter, delay, and effect fade-out. Each effect can affect the pitch, pan, volume, or a 4th parameter that could be cutoff. And there are several waveforms to chose from. But the soundfont format has 2 LFOs, there's no obvious way to affect the pan using those, and they're limited to 100Hz max as rate, and the depth is logarithmic...
  
• Also I lack audio knowledge, and I'm not sure why some basic synth wave samples sound extremely off-pitch when the midis are played back.. Even though the root key seems ok, and it sounds correctly when playing a note using that soundfont + sample. The samples are very short, and the waves repeats only 2-4 times.

So far, I wrote a WIP tool that does a decent job at ripping the tracks. It can rip the samples as-is, but the envelopes not being linear make a significant difference. Or it can rip them and "bake" the envelope into them, however that causes issues when the sounfont player resamples the sounds past a certain note.. And the release phase of the envelope just doesn't sounds right because it has to be handled by the soundfont.. So that's why now I'm asking around for help because I'm not sure how to fix everything or finish documenting the format..

I filled a bunch of wiki pages with my notes at project pokemon :
http://projectpokemon.org/wiki/Digital_Sound_Elements

But, yeah, any help would be welcome !

 

[FAST6191]

You have gone further than anybody else I have seen, though I have not checked HCS64 of late. I would probably know this format more as the pb audio format or possibly SED SWD and SMD if you are using extensions/magic stamps.

We did find a build script for this format in one of the Inazuma 11 projects once ( http://gbatemp.net/threads/the-various-audio-formats-of-the-ds.305167/ has it) but looking at your previous posts I said as much in those, and I have not done anything since on the matter. Not to mention anything I would have done would have been just to confirm it was a proper sequencer/tracker format, some file swaps, some internal file system fiddling and some general header parsing and not a lot else. Thanks for dev page link though, I added it to thread.

 

[loco365]

It seems there's been a sudden explosion of interest in this particular sound format. Perhaps I'll point you towards this thread on The PokéCommunity Forums, it may be of use for you to talk to the person there that's doing the exact same thing to a large degree of success.

 

[psy_commando]

(I posted this message in a bit of a hurry, so hopefully there isn't too much missing..)

You have gone further than anybody else I have seen, though I have not checked HC64 of late. I would probably know this format more as the pb audio format or possibly SED SWD and SMD if you are using extensions/magic stamps.

We did find a build script for this format in one of the Inazuma 11 projects once ( http://gbatemp.net/threads/the-various-audio-formats-of-the-ds.305167/ has it) but looking at your previous posts I said as much in those, and I have not done anything since on the matter. Not to mention anything I would have done would have been just to confirm it was a proper sequencer/tracker format, some file swaps, some internal file system fiddling and some general header parsing and not a lot else. Thanks for dev page link though, I added it to thread.

Honestly, I had no clue it was called DSE until someone linked me the official page XD
I didn't call it by extension, because games like zombie daisuki and professor layton hid the smdl and swdl files deep into another file format, so its not immediately obvious. And so I just assumed it must be the case for most non Chunsoft games using DSE.

And the last thing I saw on HC64 about DSE was about using a minidump to play DSE-using game soundtracks. There might have been some new stuff since then though XD

Yeah, all the leads and links I found were inconclusive, or just touching the surface :/
But the format is surprisingly simple and straightforward.
So, I basically learned to read disassembler output in a few months and disassembled the audio processing manually using cheat engine and No$GBA's free debugger. But, there's some stuff happening in there that I'd need to hear and explanation for from an audio expert to even know what's going on ^^;

And no problems!
Its far from complete though, a good chunk of the events are still unresearched yet. But, most of the ones actually used in music tracks are understood, except event 0xBE which I still can't figure out at all.. Doesn't seem to change much though.. And of course, the SEDL format for sequenced sound effects is just completely untouched at this point.

Me and a few others also documented a lot of the other file formats in PMD2. If that's of any use XD
We got this thread going on with most of the details. Its basically my pmd2 tools thread but it turned into a PMD2 research thread after a while :
http://projectpokemon.org/forums/sh...eon-2-Psy_commando-s-Tools-and-research-notes

And I just posted the latest version of my "audioutil" if anyone wants to hear the problem firsthand. I've been holding it back for a while since its still kinda rough around the edges, but it might help getting some attention on the issues I'm having with the pitch and volume and etc..

Speaking of which, I didn't even link an example.. >_<
Here's probably the track that's the most affected by the pitch issues right now. It sounds absolutely awful right now XD : https://dl.dropboxusercontent.com/u...ed_with_audioutil/Dec7/52_B_DUN_HASSAM_01.mp3
That's a MIDI + soundfont ripped from the game using my tool being played in Foobar2000. A lot of other tracks are really close to the real thing, unlike the one in the mp3 above, but I won't link those, because idk if that's against the rules around here. I posted a few examples on the project pokemon thread though.

It seems there's been a sudden explosion of interest in this particular sound format. Perhaps I'll point you towards this thread on The PokéCommunity Forums, it may be of use for you to talk to the person there that's doing the exact same thing to a large degree of success.

An explosion ? You mean there's more than just me and that guy ?
It could be because of the new mystery dungeon that just came out?

Thanks for the tip! I'll try to drop by on that thread later on today.
Ironically, I've been debating on posting my stuff on PC for a while. But since they're all mostly console applications, and PC isn't really all that into ROM research, the amount of nagging for a GUI I'd probably have gotten might have been a bit too much to handle when I want to focus on functionality for now XP

 

[loco365]

An explosion ? You mean there's more than just me and that guy ?
It could be because of the new mystery dungeon that just came out?

At the end of that thread are some links to other people's research as well as a link to a github request for VGMTrans to support the DSE SND format.

 

[FAST6191]

"was about using a minidump to play DSE-using game soundtracks."
That is how a lot of the SDAT stuff is done.

Also yeah no$gba debug going free was a good thing for a lot of ROM hacking. You asked for help with the technical stuff and I suppose I could help there (though the DS is not especially complex or even all that limited like the older consoles and even the GBA), if however you are going to want someone like you see on HCS64 that can spot and know that something has a too slow decay or something then I am not going to be much help at all. Actually come to think of it I am not even going to be much help narrowing down/telling when things are the wrong pitch either.

 

[psy_commando]

Sorry for the late reply ^^;

At the end of that thread are some links to other people's research as well as a link to a github request for VGMTrans to support the DSE SND format.

Those are pretty old though. I know TruePikachu, he helped out with reversing the format. Though idk what happened with him, he just vanished :/
I kinda feel bad about it too, because I think he might have misinterpreted something I said.. ^^;

And I commented in that vgmtrans request, but I never heard back from iloveemu. So idk if there's truly that much interest. XD

I wish DSE stuff would grab the attention of people like iloveemu or kode54. It would probably make things progress much faster XD

"was about using a minidump to play DSE-using game soundtracks."
That is how a lot of the SDAT stuff is done.

Well, you can pretty much handle all game audio formats that way more or less I guess. But I'm looking to eventually be able to import stuff back into DSE format too so, its not all that helpful to stumble on people taking the "easy" way XP

Also yeah no$gba debug going free was a good thing for a lot of ROM hacking. You asked for help with the technical stuff and I suppose I could help there (though the DS is not especially complex or even all that limited like the older consoles and even the GBA), if however you are going to want someone like you see on HCS64 that can spot and know that something has a too slow decay or something then I am not going to be much help at all. Actually come to think of it I am not even going to be much help narrowing down/telling when things are the wrong pitch either.

Well, its not that complex, but some of the details on the way audio is handled on the NDS are pretty unclear..
There's no info the dynamic range of the system, like what maximum loudness it can get to. Or how the volume value in the control register is obtained or what it actually means :/
And well, there are mentions that, the NDS7 runs the audio, but nothing else. And also, I'm a bit confused because I'm never sure whether I'm debugging code on the NDS9 or NDS7 in no$gba.

And, that's fine. I'll try to get some help from HCS64 for that, but this far I haven't attracted much attention to my reply. I might need to make a new thread..

 

[loco365]

And, that's fine. I'll try to get some help from HCS64 for that, but this far I haven't attracted much attention to my reply. I might need to make a new thread..

I have a thread on there for some of my GBA GAX work, as well as someone finding my recent bits of work and posting a thread seeing if anyone would pick it up, both of which have garnered little interest. I'd love to find some place that could actually help. I've considered Xentax, but I'm not too sure yet.

 

[psy_commando]

I have a thread on there for some of my GBA GAX work, as well as someone finding my recent bits of work and posting a thread seeing if anyone would pick it up, both of which have garnered little interest. I'd love to find some place that could actually help. I've considered Xentax, but I'm not too sure yet.

That sucks :/
And xentax isn't all that great ^^;
I've been mostly lurking around there a while and I don't think I have ever seen anyone messing with audio formats. :/

There must be a reddit or something for audio programming .. :/
Maybe the chiptune scene might have some forums dedicated to that though..