29.Jan.15

Milan Nankov

Milan Nankov

Universal Audio Component – Decoding Audio With Media Foundation

In the first blog post of this series, we created a universal XAudio2 audio component that can play multiple audio sample at the same time. We used this component in our universal app that runs on Windows Phone and Windows devices. This time we are going to address a limitation of our component - it can only play WAVE files. If an app heavily relies on audio, making use of compressed formats like MP3 will definitely help to keep the size in the acceptable range.

For the sake of keeping our UniversalAudioPlayer  as simple as possible, we are not going to introduce any changes to it and it will keep doing what it does best - playing WAVE files. The app will be responsible for decoding MP3s into a format that our player understands by using a decoder class that we are going to build. The process looks like this:

playback-steps


Fortunately, there is this thing called Microsoft Media Foundation that will do the heavy lifting of decoding our MP3 files (or any other of the supported formats for that matter) into WAVE format. Let's explore decoding audio with Media Foundation.

Microsoft Media Foundation

Microsoft Media Foundation is one of the multimedia frameworks provided by Microsoft. It has some commonalities with XAudio2 but it is geared towards audio and video playback of high-definition conent, content protection, and efficient video acceleration. You can think of Media Foundation as the core of multimedia players like Windows Media Player and the likes.

Since our universal audio player already has a much more powerfull audio playback capabilities, we are only going to use the audio decoding functionality of Media Foundation. This functionality that we are interested in is provided by IMFSourceReader. The decoder class that we are about to build is based on a Media Foundation tutorial called Decoding Audio.

The AudioDecoder class

This is the class that our universal app will use to decode compressed audio files into WAVE which our audio player can play.

The only public method here is Array<byte>^ Decode(IRandomAccessStream^ audioStream) which is pretty straightforward - we pass a stream that contains a compressed audio and get the bytes of the WAVE representation of the input file. Here is the implementation.

The snippet above provides an overview of the process of decoding an audio file. If we take a look at DecodeAsWav we will see the following sequence:

  1. Configure audio stream and get the audio type - info about the format of the decoded audio
  2. Decode audio data
  3. Use decoded audio data to build a valid WAVE audio file
I am going to skip the configuration part because it's boring and I will go straight to the part where the actual decoding takes place.

Get decoded audio data

This is the place where we use IMFSourceReader. We use IMFSourceReader::ReadSample to read one sample at a time and write each sample to a buffer. What ReadSample does is read a sample from the input audio stream (MP3, for example) and convert this sample into the WAVE format. Once we are done with all input samples, we have our WAVE audio data - dataBytes. The last item on our list is to package this data into the RIFF format that have talked about last time.

Construct a valid WAVE file

This is where we package the decoded audio data into a valid WAVE file with all of the required chunks - RIFF, data, and fmt.

Nothing too fancy here - we just gather the data that we have produced so far and package it into a RIFF format.

One final step - use the decoder in our app.

The App

universalaudioappThe app is pretty much unchanged.  We only need a couple of lines to decode and make use of our MP3 files. Here's what's changed:

With the help of Microsoft Media Foundation we have added a very important feature to our universal audio component. The ability to play compressed audio files like MP3s is a core requirement for many applications and our component is now in line with those requirements.

Download the full source code of the updated audio player and the app.

Cheers!

Resources:

Need consulting on this topic?

Yes No