This morning, while eating my Father’s day bagel, I got to play some more with the video aspects of the Echo Nest remix API. The video remix is pretty slick. You use all of the tools that you use in the audio remix, except that the object you are manipulating has a video component as well. This makes it easy to take an audio remix and turn it into a video remix. For instance, here’s the remix code to create a remix that includes the first beat of every bar:
audiofile = audio.LocalAudioFile(input_filename) collect = audio.AudioQuantumList() for bar in audiofile.analysis.bars: collect.append(bar.children()) out = audio.getpieces(audiofile, collect) out.encode(output_filename)
To turn this into a video remix, just change the code to:
av = video.loadav(input_filename) collect = audio.AudioQuantumList() for bar in av.audio.analysis.bars: collect.append(bar.children()) out = video.getpieces(av, collect) out.save(output_filename)
The code is nearly identical, differing in loading and saving, while the core remix logic stays the same.
To make a remix of a YouTube video, you need to save a local copy of the video. I’ve been using KeepVid to save local flv (flash video format) of any Youtube video.
Today I played with the track ‘Boom Boom Pow’ by the Black Eyed Peas. It’s a fun song for remix because it has a very strong beat, and already has a remix feel to it. And since the song is about digital transformation, it seems to be a good target for remix experiments. (and just maybe they won’t mind the liberties I’ve taken with their song).
Here’s the original (click through to YouTube to watch it since embedding is not allowed):
The first remix is to only include the first beat of every measure. The code is this:
for bar in av.audio.analysis.bars: collect.append(bar.children())
Change the beat included from beat zero to beat three, and we get something that sounds very different:
Pow Boom Boom
Here’s a version with the beats reversed. The core logic for this transformation is one line of code:
The 5/4 Version
Here’s a version that’s in 5/4 – to make this remix I duplicated the first beat and swapped beats 2 and 3. This is my favorite of the bunch.
These transformations are of the simplest variety, taking just a couple of minutes to code and try out. I’m sure some budding computational remixologist could do some really interesting things with this API.
Note that the latest video support is not in the main branch of remix. If you want to try some of this out you’ll need to check out the bl-video branch from the svn repository. But this is guaranteed to be rolled into the main branch before the upcoming Music Hackday. Update: the latest video support is now part of the main branch. If you want to try it out, check it out from the trunk of the SVN repository. So download the code, grab your API key and start remixing.
Update: As Brian pointed out in the comments there was some blocking on the remix renders. This has been fixed, so if you grab the latest code, the video output quality is as good as the input.
#1 by brian on June 21, 2009 - 3:07 pm
there is a built in function for downloading movies from a youtube URL, just FYI, you don’t need keepvid. Instead of video.loadav do video.loadavfromyoutube(“http://youtube.com/…”) and it’ll take care of the rest.
Also, intense viewers may notice the extreme DCT blocks in the remix renders– we’re on that, should be fixed in a couple of days.
#2 by Carolyn on June 21, 2009 - 10:02 pm
I hope you enjoyed your bagel and had a happy Father’s Day.
Also, for the record, here I am on the computer and checking your blog. I also checked my email, which contained 17 spam messages. Hooray!
#3 by Torley on June 22, 2009 - 2:47 pm
Found out about this from Waxy! Wow, the possibilities. Gotta familiarize myself with Echo Nest nao.
#4 by plamere on June 22, 2009 - 2:54 pm
Great! Can’t wait to see one of your awesome videos about the Echo Nest on torley.com!
#5 by Oblomov on June 23, 2009 - 6:52 am
The first two make it sound like a Daft Punk song ;-)
#6 by carmen on June 29, 2009 - 2:35 pm
wheres the pow indeed. big up the broken plugin boxes. gues it isnt HTML5
#7 by zazi on July 15, 2009 - 9:47 am
Yes, a very interesting approach. Looking forward to new and interesting video with the Remix API.
#8 by canyon cody on July 22, 2009 - 2:17 pm
this is amazing, im showing this to my 4th grade students in computer class tomorrow!