Music Machinery

This morning, while eating my Father’s day bagel, I got to play some more with the video aspects of the Echo Nest remix API.  The video remix is pretty slick.  You use all of the tools that you use in the audio remix, except that the object you are manipulating has a video component as well.    This makes it easy to take an audio remix and turn it into a video remix.  For instance, here’s the remix code to create a remix that includes the first beat of every bar:

 audiofile = audio.LocalAudioFile(input_filename)
 collect = audio.AudioQuantumList()
 for bar in audiofile.analysis.bars:
     collect.append(bar.children()[0])
 out = audio.getpieces(audiofile, collect)
 out.encode(output_filename)

To turn this into a video remix, just change the code to:

 av = video.loadav(input_filename)
 collect = audio.AudioQuantumList()
 for bar in av.audio.analysis.bars:
     collect.append(bar.children()[0])
 out = video.getpieces(av, collect)
 out.save(output_filename)

The code is nearly identical, differing in loading and saving, while the core remix logic stays the same.

To make a remix of a YouTube video, you need to save a local copy of the video.   I’ve been using KeepVid to save local flv (flash video format) of any Youtube video.

Today I played with the track ‘Boom Boom Pow’ by the Black Eyed Peas.  It’s a fun song for remix because it has a very strong beat, and already has a remix feel to it.  And since the song is about digital transformation, it seems to be a good target for remix experiments.  (and just maybe they won’t mind the liberties I’ve taken with their song).

Here’s the original (click through to YouTube to watch it since embedding is not allowed):

Just Boom

The first remix is to only include the first beat of every measure.   The code is this:

    for bar in av.audio.analysis.bars:
         collect.append(bar.children()[0])

Just Pow

Change the beat included from beat zero to beat three, and we get something that sounds very different:

Pow Boom Boom

Here’s a version with the beats reversed.  The core logic for this transformation is one line of code:

av.audio.analysis.beats.reverse()

The 5/4 Version

Here’s a version that’s in 5/4 – to make this remix I duplicated the first beat and swapped beats 2 and 3.  This is my favorite of the bunch.

These transformations are of the simplest variety, taking just a couple of minutes to code and try out.   I’m sure some budding computational remixologist could do some really interesting things with this API.

Note that the latest video support is not in the main branch of remix.  If you want to try some of this out you’ll need to check out the bl-video branch from the svn repository.     But this is guaranteed to be rolled into the main branch before the upcoming Music Hackday. Update: the latest video support is now part of the main branch.  If you want to try it out, check it out from the trunk of the SVN repository. So download the code, grab your API key and start remixing.

Update: As Brian pointed out in the comments there was some blocking on the remix renders. This has been fixed, so if you grab the latest code, the video output quality is as good as the input.