DSPADPCM (.dsp audio) Encoding Made Easy!

Discussion in 'Wii - Hacking' started by jackoalan, Jun 6, 2015.

  1. jackoalan
    OP

    jackoalan Newbie

    Newcomer
    9
    8
    May 12, 2015
    United States
    Hello everyone!

    I'm a game reverse-engineering and modding enthusiast focusing on games by Retro Studios (i.e. the Metroid Prime games and recent Donkey Kong titles). For some time, I've been wanting to insert custom music and SFX by implementing a DSPADPCM encoder (which many titles seem to use for audio compression).

    Nintendo provides developers with DSPADPCM.EXE and dsptool.dll to perform the encoding. This tool and codec is available for GameCube, Wii and Wii U systems. The codec is based on an IEEE-defined ADPCM predictor combined with 8 coefficient-pairs selected via filtered discrete-fourier-transform. The result is 8-byte ADPCM frames with 1-byte header at 4 bits-per-sample. A mono 16-bit PCM stream ends up being ~28.5% the original size in DSPADPCM.

    @delroth pointed me to this implementation in BrawlTools which appears to contain reverse-engineered versions of the original algorithms:
    https://code.google.com/p/brawltools2/source/browse/trunk/BrawlLib/Wii/Audio/AudioConverter.cs

    I spent much time trying to grok the algorithms, identifying the mathematical basis and transforming the reverse-engineered code into something more readable and optimized:
    https://github.com/jackoalan/gc-dspadpcm-encode/blob/master/grok.c

    I'm pleased with the results! Generated streams sound great when injected into actual game images. To make the encoding process easier, I've created a fork of Audacity to import/export .dsp files as well as .dsp derivatives used by Retro. Part of the .dsp format includes metadata for a loop-region as well; this is visualized as a label-track in Audacity. The encoder can easily be applied to layouts used by other games as well, so long as they derive from DSPADPCM:
    [​IMG]
    Check it out here:
    https://github.com/jackoalan/audacity/releases
     
    Last edited by jackoalan, Jun 7, 2015


  2. Antidote

    Antidote GBAtemp Regular

    Member
    108
    38
    Jul 13, 2011
    United States
    Nice work jack :D DSPADPCM turned out to be surprisingly easy once we had an idea where to go with it.
     
    Usuario-X likes this.
  3. jackoalan
    OP

    jackoalan Newbie

    Newcomer
    9
    8
    May 12, 2015
    United States
    Ah yes! Due thanks to @Antidote and Parax from M2K2 for supplying test material and plenty of background information on relevant file formats.
     
  4. Shrinefox

    Shrinefox GBAtemp Regular

    Member
    113
    56
    Sep 5, 2013
    United States
    This is awesome, made my life a lot easier modding sound files for Gamecube games.

    However, Audacity (seemingly at random) decides not to load anything when I click on a dsp sometimes, even though it's listed in the recent files after I do so.
    I don't know if this is an Audacity glitch or maybe something wrong/incompatible with the files (maybe too short in length?)

    I even tried going into my appdata and temp audacity folders and clearing it out but it made no difference trying to open some of them.
     
  5. jackoalan
    OP

    jackoalan Newbie

    Newcomer
    9
    8
    May 12, 2015
    United States
    If you notice it happen consistently for certain individual files, then you might be dealing with a different container type.

    Not all .dsp files are made with Nintendo's usual utility. In most cases the same codec is used, but it's also arbitrarily split for stereo or other multi-track usage. For instance Metroid Prime 2 comes with .dsp files that begin with 'RS03' and uses ~2sec per-channel interleaving for fluid use with the DVD -> ARAM DMA mechanic in the GameCube specifically.

    This fork is specifically made for the containers used by Retro Studios (found in the Prime Trilogy and both new DKC games).

    Depending on the game you're dealing with, the container will need to be added to the fork.
     
    Shrinefox likes this.
  6. koz

    koz Advanced Member

    Newcomer
    82
    23
    Jul 7, 2015
    Does your DSPADPCM code allow streaming to a wii remote for sound output.

    This would be a great achievement if you can make it work.
     
  7. jackoalan
    OP

    jackoalan Newbie

    Newcomer
    9
    8
    May 12, 2015
    United States
    According to this http://wiibrew.org/wiki/Wiimote#Sound_Data_Format, the Wiimote uses Yamaha ADPCM.

    It should be noted that DSPADPCM is not well-suited for real-time encoding (which is how the Wiimote stream works). The entire music/sfx track needs to be mixed and encoded ahead of time, using some rather complex computations.
     
  8. Shrinefox

    Shrinefox GBAtemp Regular

    Member
    113
    56
    Sep 5, 2013
    United States
    What's odd is that they all came from the same file in the same game, though (Thousand Year Door). It'd be strange if they made different sound effects with a different utility, so maybe there's just something variable about that specific type of the format that only breaks sometimes. Thanks for the heads up though, didn't know it was made for a specific game.
     
  9. jackoalan
    OP

    jackoalan Newbie

    Newcomer
    9
    8
    May 12, 2015
    United States
    Yea, it's common for SFX and other event cues to be short, standard mono .dsp files

    On the other hand, GameCube music will often be a fully-produced stereo mix, requiring a stereo container of some sort (not standardised by Nintendo)
     
  10. Antidote

    Antidote GBAtemp Regular

    Member
    108
    38
    Jul 13, 2011
    United States
    Retro cheated a little by having both left and right channels as separate files in MP1.
     
  11. jackoalan
    OP

    jackoalan Newbie

    Newcomer
    9
    8
    May 12, 2015
    United States
    Last edited by jackoalan, Mar 27, 2016
    Shrinefox likes this.
  12. TGE

    TGE Member

    Newcomer
    38
    12
    Jan 10, 2013
    Netherlands
    jackoalan, would you mind implementing the Paper Mario: The Thousand Year Door's .stm stereo dspadpcm container? It's a pretty basic container without much added to it.
    The structure is as follows:
    Code:
    // size = 64 bytes
    struct STMHeader
    {
        u16 field00;                // 0x00, always 512, possibly a version number of 2.00
        u16 sampleRate;             // 0x02, usually 32000
        u32 numChannels;            // 0x04, usually 2 (could also be initial offset)
        u32 adpcmData2Offset;       // 0x08, byte offset to the adpcm data of the second channel (relative to the end of the this header + all channel headers)
        u32 adpcmLoopStartOffset;        // 0x0C, byte loop start offset, equal to 0xFFFFFFFF if not used (relative to the start of the data itself, not the headers)
        u32 adpcmData2OffsetAux1;   // 0x10, same value as adpcmData2Offset
        u32 adpcmData2OffsetAux2;   // 0x14, same value as adpcmData2Offset
        u32 adpcmLoopStartOffsetAux1;    // 0x18, if adpcmLoopOffset != 0xFFFFFFFF then same value else value is 0
        u32 adpcmLoopStartOffsetAux2;    // 0x1C, same as adpcmData2OffsetAux1
        u8  padding[32];            // 0x20, 32 bytes unused
    }
    
    // after this numChannels * standard dspadpcm header follows
    // right after that the adpcm data of the first channel starts
    // if there's a second channel, then you must seek to the adpcmData2Offset to get the data for the second channel
    // there appears to be 32 byte alignment between the adpcm data blocks
    // and some unknown alignment (or just a fixed amount of padding) at the end of the file
    // what's missing from this is which value indicates the loop end offset
    // so far, all the files in the game loop from the loop start point to the end so
    // there's a chance that adpcmData2OffsetAux1 or adpcmData2OffsetAux2 is the loop end offset
    
    The game seems to completely ignore the looping contexts of the embedded dsps and instead uses the values specified in the stm header.
    Samples (includes a looped and non-looped one) http://puu.sh/nVH16.7z
     
    Last edited by TGE, Mar 28, 2016
    Shrinefox likes this.
  13. jackoalan
    OP

    jackoalan Newbie

    Newcomer
    9
    8
    May 12, 2015
    United States
    Here's what i've got so far for .stm support:
    https://github.com/jackoalan/audacity/releases/tag/v2.1.3-dev-stmtest

    I'm a bit confused by the two files though. Both seem to set 'adpcmLoopStartOffset' as 0xffffffff. I'm assuming this means that both of these are non-looping files.

    Judging by each channel's .dsp header, 'btl_die1_32k.stm' loops, but it's the whole file as the loop region.

    I implemented looping from the .stm based on your struct comments. Let me know if this functions correctly with files that definitely loop.
     
  14. TGE

    TGE Member

    Newcomer
    38
    12
    Jan 10, 2013
    Netherlands
    Ah, great work. Thanks a lot.
    Sorry about them both not looping, I accidentally put the wrong file in the archive. Even still, it works perfectly for actually looped tracks.
    One weird issue though. Testing it ingame gave me some trouble. Basically, if adpcmData2OffsetAux1 & 2 aren't exactly equal to adpcmData2Offset, the channels will desync in stereo and produce garbage audio in mono. Same goes for the other 2 aux variables.
    Seeing as how none of the tracks in the game have the loop end point not set to the end I'm not sure if this is a format limitation.
    But aside from that, the files work perfectly ingame.
     
  15. jackoalan
    OP

    jackoalan Newbie

    Newcomer
    9
    8
    May 12, 2015
    United States
  16. libertyernie

    libertyernie Advanced Member

    Newcomer
    89
    72
    Apr 6, 2011
    United States
    I'm looking at the differences between your grok.c and AudioConverter.cs (mostly as a way for me to practice C/C++, honestly) and I noticed this part in particular:
    Code:
            /* Set initial scale */
            for (scale=0 ; (scale<=12) && ((distance>7) || (distance<-8)); scale++, distance>>=1)
                scale = (scale <= 1) ? -1 : scale - 2;
    
    BrawlLib has a semicolon right after the for loop, so the loop gets run until it stops, and then it runs the line afterwards. Your code treats that next line as the body of the loop. In comparing the audio files, I don't see much of a difference, so I was wondering which is correct (or at least better.)

    Unrelated to the above: I also converted this code back to C# and slotted it into BrawlLib - see https://github.com/libertyernie/brawltools/commit/e1b40ddc383dfa487320e294e172d8b8d1406d92 for the side-by-side comparison.
    Trying it out, it seems like there's more difference between the original and encoded audio in this version than in the original BrawlLib code, but I can't tell by listening - only by subtracting the original waveform from the encoded-and-then-decoded waveform in Audacity (Invert on one, then Mix and Render) and comparing them.

    EDIT: I think I've got it working better now. Here's the changes I made: https://github.com/libertyernie/brawltools/commit/d6708a356d3cc3b9d133c37e94bfb5721e8593e2
    I'm guessing the pass-by-reference issue was when I ported it to C# and implanted tvec as a struct, and the pcmHistBuffer issue was also a change I made since I couldn't allocate arrays in the same way (so your code is probably fine).
     
    Last edited by libertyernie, Apr 4, 2016
  17. jackoalan
    OP

    jackoalan Newbie

    Newcomer
    9
    8
    May 12, 2015
    United States
    Ah cool! Thanks for the eagle-eyed catch! I'll merge the changes in my using repos soon
     
  18. Elijah6133

    Elijah6133 Member

    Newcomer
    12
    0
    Dec 3, 2015
    Canada
    for some reason it wont open on my laptop it keeps giving the following error The application was unable to start correctly (0xc000007b) Ill keep trying to figure it out, but if you know of a way to fix this could you let me know?
     
  19. Elijah6133

    Elijah6133 Member

    Newcomer
    12
    0
    Dec 3, 2015
    Canada
    I still cant get it to work. is it because i have a x64 operating system?
     
  20. Elijah6133

    Elijah6133 Member

    Newcomer
    12
    0
    Dec 3, 2015
    Canada
    ok got it to open now, but is there any chance you could make a tutorial on youtube showing all the different featuresÉ