Gapless MP3 Audio on iOS

Daniel Sperl on May 23, 2012

Kostya Teterin, the developer who brought us Caribbean Treasures, joins us this time with an article about a cool audio library he has developed for his latest game.

Gapless MP3 Audio

As every game developer knows, background music is an important part of any game. You can use it to build up tension, to set up the right mood, let the player relax, etc. To create a dynamic soundtrack, you need a flexible way of combining different audio tracks.

If you’ve ever tried to loop and join mp3s seamlessly, you’ll have found out the hard way that it’s not as easy as one would expect; there are a lot of problems that need to be solved. The library I will describe in this article will simplify that task immensely.

Gapless MP3 technology allows you to create music from a set of mp3 fragments that you can join in any order on the fly. Here’s how a gapless mp3 soundtrack might look like:

There are many scenarios where gapless looping comes in handy. For example, you can play a short loop while the tutorial dialog is open and then continue to the main theme once it’s closed. Or you can play intense music fragments during fights and calmer fragments in-between. There are many exciting things that you can do with gapless mp3s by playing fragments continuously in any order.

First you need to create gapless mp3 tracks. The technical side of how this is done is described in this article at compuphase.com.

For us software developers, however, it is more important to learn how to play them on iOS devices. In this article, you will first learn how to do that manually; then we’ll have a look at a wrapper class that simplifies this process immensely.

Gapless Audio with the “Audio Queue Services”

The common way to play mp3 files is to use the AVAudioPlayer class (Sparrow uses this class behind the scenes, too). It can stream mp3 files instead of loading them into the memory, it has all necessary methods and properties to handle the playback, and it uses a hardware mp3 decoder.

Another approach is to use “Audio Queue Services”, the low-level foundation of AVAudioPlayer. Audio Queue Services are more challenging to use, but give you more control. However, even with that approach, some problems remain:

  • Both AVAudioPlayer and Audio Queue Services are very imprecise. When you start to play the music, there’s always a tiny delay up to 300 milliseconds. A value big enough to be heard.
  • The hardware mp3 decoder can process only one audio session (audio queue) at a time. If you start a new session while the hardware decoder is busy, it will use software codecs that drain the battery, or might even throw a “Hardware in use” error on older devices.
  • When an audio session is done, it needs some time to release its resources. During this time, you can’t start another session with the hardware decoder.

If you try to play gapless mp3 music in the traditional way, you will have to wait until the previous fragment releases its resources, and there will be a short delay before the new fragment can start to play. The length of these delays is unpredictable and it is even longer on older devices.

The only way to avoid these delays is to play all mp3 fragments within the same audio session. With Audio Queue Services, you can achieve that.

Since an audio queue is initialized with the configuration of the decoder, all your mp3 fragments must be encoded with the same codec, properties and parameters. There is no way to reconfigure the decoder during the session.

All “dirty” work is done by the callback that reads the portion of audio data from a file and fills the audio buffers. The default callback stops the session when there’s no more data to read. Our custom callback will begin to fill the buffers with the data from the next mp3 file instead.

The “Gapless MP3 Player” Library

This solution is already implemented in the Gapless MP3 Audio Player for iOS. You can download the complete sources from Github: Gapless-MP3-Player on GitHub

The demo application uses gapless mp3 music written by Dan Reynolds for my upcoming game Fix The Leaks.

Gapless MP3 Player is completely free and you can use it in your own projects. While it does not depend on it, it’s especially easy to use with Sparrow, because it contains a special “SPGAudioPlayer” class to be used with the game engine. This class handles the internal audio system events and has a very simple interface.

The following example adds two audio fragments to the queue and plays them:

SPGAudioPlayer *player = [[SPGAudioPlayer alloc] init];
[player addSoundFromFile:@"intro_fragment.mp3"];
[player addSoundFromFile:@"main_loop_fragment.mp3" loop:-1];
[player playQueue];

The first fragment is not looped, while the second one loops endlessly (-1 = endless loop, 0 = no loop, and numbers above 0 define how many time to play the fragment in a loop).

If you want to set up another queue of mp3 fragments with the same player object, you can call “[player clearQueue]” to clear the queue.

You can also call “[player breakLoop]” to move to the next fragment when the current looped fragment is done. Let me show you another example:

SPGAudioPlayer *player = [[SPGAudioPlayer alloc] init];
[player addSoundFromFile:@"tutorial_fragment.mp3" loop:-1];
[player addSoundFromFile:@"intro_fragment.mp3"];
[player addSoundFromFile:@"main_loop_fragment.mp3" loop:-1];
[player playQueue];

// Tutorial window is displayed, tutorial_fragment.mp3 is playing in endless loop
...
// Player tap the “Close” button on tutorial window
[player breakLoop];
// tutorial_fragment.mp3 is done playing, and queue is moved to intro_fragment.mp3

The following methods to control the playback are also available:

[player stop] - stop the playback
[player pause] - pauses the playback (call [player playQueue] again to resume)

Gapless mp3 music technology can advance your game audio to the next level and help you to immerse the player deeper into the game. And now you can easily use it in your own iOS games based on the Sparrow Framework!

Thanks a lot for your hard work on this library, Kostya! BTW, to all Sparrow users: don’t forget to check out Fix the Leaks when it’s available: it’s definitely one of the coolest and most impressive Sparrow games I’ve seen so far!