Music Machinery

This week  we plan to release a new version of the Echo Nest Python remix API that will add the ability to manipulate tempo and pitch along with all of the other manipulations supported by remix.   I’ve played around with the new capabilities a bit this weekend, and its been a lot of fun.   The new pitch and tempo shifting features are integrated well with the API, allowing you to adjust tempo and pitch at any level from the tatum on up to the beat or the bar.  Here are some  quick examples of using the new remix features to manipulate the tempo of a song.

A few months back I created some plots that showed the tempo variations for a number of drummers.  One plot showed the tempo variation for The Beatles playing ‘Dizzy Miss Lizzy’.  The peaks and valleys in the plot show the tempo variations over the length of the song.

You got me speeding Miss Lizzy

With the tempo shifting capabilities of the API we can now not only look at the detailed tempo information – we can manipulate it.  For starters here’s an example  where we use remix to gradually increase the tempo of Dizzy Ms. Lizzy so that by the end, its twice as fast as the beginning.  Here’s the clickplot:

And here’s the audio

I don’t notice the acceleration until about 30 or 45 seconds into the song.  I suspect that a good drummer would notice the tempo increase much earlier.   I think it’d be pretty fun to watch the dancers if a DJ put this version on the turntable.

As usual with remix – the code is dead simple:

 st = modify.Modify()
 afile = audio.LocalAudioFile(in_filename)
 beats = afile.analysis.beats
 out_shape = (2*len(afile.data),)
 out_data = audio.AudioData(shape=out_shape, numChannels=1, sampleRate=44100)
 total = float(len(beats))
 for i, beat in enumerate(beats):
     delta =  i / total
     new_ad = st.shiftTempo(afile[beat], 1 + delta / 2)
     out_data.append(new_ad)
 out_data.encode(out_filename)

You got me Quantized Miss Lizzy!

For my next experiment, I thought it would be fun if we could turn Ringo into a more precise drummer – so I wrote a bit of code that shifted the tempo of each beat to match the average overall tempo – essentially applying a click track to Ringo.  Here’s the resulting click plot, looking as flat as most drum machine plots look:

Compare that to the original click plot:

Despite the large changes to the timing of the song visibile in the click plot,  it’s not so easy to tell by listening to the audio  that Ringo has been turned into a robot – and yes there are a few glitches in there toward the end when the beat detector didn’t always find the right beat. 

The code to do this is straightforward:

 st = modify.Modify()
 afile = audio.LocalAudioFile(in_filename)
 tempo = afile.analysis.tempo;
 idealBeatDuration = 60. / tempo[0];
 beats = afile.analysis.beats
 out_shape = (int(1.2 *len(afile.data)),)
 out_data = audio.AudioData(shape=out_shape, numChannels=1, sampleRate=44100)
 total = float(len(beats))
 for i, beat in enumerate(beats):
     data = afile[beat].data
     ratio = idealBeatDuration / beat.duration
     # if this beat is too far away from the tempo, leave it alone
     if ratio < .8 and ratio > 1.2:
         ratio = 1
     new_ad = st.shiftTempoChange(afile[beat], 100 - 100 * ratio)
     out_data.append(new_ad)
 out_data.encode(out_filename)

Is is Rock and Roll or Sines and Cosines?

Here’s another tempo manipulation – I applied a slowly undulating sine wave to the overall tempo – its interesting how more disconcerting it is to hear a song slow down than it does hearing it speed up.

I’ll skip the code for the rest of the examples as an exercise for the reader, but if you’d like to see it just wait for the upcoming release of remix – we’ll be sure to include these as code samples.

The Drunkard’s Beat

Here’s a random walk around the tempo – my goal here was an attempt to simulate an amateur drummer – small errors add up over time resulting in large deviations from the initial tempo:

Here’s the audio:

Higher and Higher

In addition to these tempo related shifts, you can also do interesting things to shift the pitch around.  I’ll devote an upcoming post to this type of manipulation, but here’s something to whet your appetite.  I took the  code that gradually accelerated the tempo of a song and replaced the line:

     new_ad = st.shiftTempo(afile[beat], 1 + delta / 2)

with this:
     new_ad = st.shiftPitch(afile[beat], 1 + delta / 2)
in order to give me tracks that rise in pitch over the course of the song.

Wrapup

Look for the release that supports time and pitch stretching in remix this week. I’ll be sure to post here when it is released.   To use the remix API you’ll need an Echo Nest API key, which you can get for free from developer.echonest.com.
The click plots were created with the Echo Nest BPM explorer – which is a Java webstart app that you can launch from your browser at The Echo Nest BPM Explorer site.