Archive for category remix
Last week at the Echo Nest 4 year anniversary party we had two renown DJs keeping the music flowing. DJ Rupture was the featured act – but opening the night was the Echo Nest’s own DJ API (a.k.a Ben Lacker) who put together a 30 minute set using the Echo Nest remix.
I was really quite astounded at the quality of the tracks Ben put together (and all of them apparently done on the afternoon before the gig). I asked Ben to explain how he created the tracks. Here’s what he said:
1. ‘One Thing’ – featuring Michael Jackson’s (dj api’s rip)
I found a half-dozen a cappella Michael Jackson songs as well as instrumental and a cappella recordings of Amerie’s “One Thing” on YouTube. To get Michael Jackson to sing “One Thing”, I stitched all his a cappella tracks together into a single track, then ran afromb: for each segment in the a cappella version of “One Thing”, I found the segment in the MJ a cappella medley that was closest in pitch, timbre, and loudness. The result sounded pretty convincing, but was heavy on the “uh”s and breath sounds. Using the pitch-shifting methods in modify.py, I shifted an a cappella version of “Ben” to be in the same key as “One Thing”, then ran afromb again. I edited together part of this result and part of the first result, then synced them up with the instrumental version of “One Thing.”
2. One Thing (dj api’s gamelan version)
I used afromb again here, this time resynthesizing the instrumental version of “One Thing” from the segments of a recording of a Balinese Gamelan Orchestra. I synced this with the a cappella version of “One Thing” and added some kick drums for a little extra punch
3. Billie Jean (dj api screwdown)
First I ran summary on an instrumental version of Beyoncé’s “Single Ladies (another YouTube find) to produce a version consisting only of the “ands” (every second eighth note). I then used modify.shiftRate to slow down an a cappella version of “Billie Jean” until its tempo matched that of the summarized “Single Ladies”. I synced the two, and repeated some of the final sections of “Single Ladies” to follow the form of “Billie Jean
Ryan Berdeen has been doing some cool things with Flash and the Echo Nest. He’s been making it the Echo Nest track analysis work with audio mixing functionality of Flash 10, effectively giving remix capabilities to the Flash programmer. He’s put together a simple demo that lets you upload a track and perform some manipulations of it. Check it out here: Flash Remix Demo (and the source is here). The demo uses Ryan’s new Echo Nest Flash API. Cool Stuff!
This week we plan to release a new version of the Echo Nest Python remix API that will add the ability to manipulate tempo and pitch along with all of the other manipulations supported by remix. I’ve played around with the new capabilities a bit this weekend, and its been a lot of fun. The new pitch and tempo shifting features are integrated well with the API, allowing you to adjust tempo and pitch at any level from the tatum on up to the beat or the bar. Here are some quick examples of using the new remix features to manipulate the tempo of a song.
A few months back I created some plots that showed the tempo variations for a number of drummers. One plot showed the tempo variation for The Beatles playing ‘Dizzy Miss Lizzy’. The peaks and valleys in the plot show the tempo variations over the length of the song.
You got me speeding Miss Lizzy
With the tempo shifting capabilities of the API we can now not only look at the detailed tempo information – we can manipulate it. For starters here’s an example where we use remix to gradually increase the tempo of Dizzy Ms. Lizzy so that by the end, its twice as fast as the beginning. Here’s the clickplot:
And here’s the audio
I don’t notice the acceleration until about 30 or 45 seconds into the song. I suspect that a good drummer would notice the tempo increase much earlier. I think it’d be pretty fun to watch the dancers if a DJ put this version on the turntable.
As usual with remix – the code is dead simple:
st = modify.Modify() afile = audio.LocalAudioFile(in_filename) beats = afile.analysis.beats out_shape = (2*len(afile.data),) out_data = audio.AudioData(shape=out_shape, numChannels=1, sampleRate=44100) total = float(len(beats)) for i, beat in enumerate(beats): delta = i / total new_ad = st.shiftTempo(afile[beat], 1 + delta / 2) out_data.append(new_ad) out_data.encode(out_filename)
You got me Quantized Miss Lizzy!
For my next experiment, I thought it would be fun if we could turn Ringo into a more precise drummer – so I wrote a bit of code that shifted the tempo of each beat to match the average overall tempo – essentially applying a click track to Ringo. Here’s the resulting click plot, looking as flat as most drum machine plots look:
Compare that to the original click plot:
Despite the large changes to the timing of the song visibile in the click plot, it’s not so easy to tell by listening to the audio that Ringo has been turned into a robot – and yes there are a few glitches in there toward the end when the beat detector didn’t always find the right beat.
The code to do this is straightforward:
st = modify.Modify() afile = audio.LocalAudioFile(in_filename) tempo = afile.analysis.tempo; idealBeatDuration = 60. / tempo; beats = afile.analysis.beats out_shape = (int(1.2 *len(afile.data)),) out_data = audio.AudioData(shape=out_shape, numChannels=1, sampleRate=44100) total = float(len(beats)) for i, beat in enumerate(beats): data = afile[beat].data ratio = idealBeatDuration / beat.duration # if this beat is too far away from the tempo, leave it alone if ratio < .8 and ratio > 1.2: ratio = 1 new_ad = st.shiftTempoChange(afile[beat], 100 - 100 * ratio) out_data.append(new_ad) out_data.encode(out_filename)
Is is Rock and Roll or Sines and Cosines?
Here’s another tempo manipulation – I applied a slowly undulating sine wave to the overall tempo – its interesting how more disconcerting it is to hear a song slow down than it does hearing it speed up.
I’ll skip the code for the rest of the examples as an exercise for the reader, but if you’d like to see it just wait for the upcoming release of remix – we’ll be sure to include these as code samples.
The Drunkard’s Beat
Here’s a random walk around the tempo – my goal here was an attempt to simulate an amateur drummer – small errors add up over time resulting in large deviations from the initial tempo:
Here’s the audio:
Higher and Higher
In addition to these tempo related shifts, you can also do interesting things to shift the pitch around. I’ll devote an upcoming post to this type of manipulation, but here’s something to whet your appetite. I took the code that gradually accelerated the tempo of a song and replaced the line:
new_ad = st.shiftTempo(afile[beat], 1 + delta / 2) with this:new_ad = st.shiftPitch(afile[beat], 1 + delta / 2)
in order to give me tracks that rise in pitch over the course of the song.
Look for the release that supports time and pitch stretching in remix this week. I’ll be sure to post here when it is released. To use the remix API you’ll need an Echo Nest API key, which you can get for free from developer.echonest.com.
The click plots were created with the Echo Nest BPM explorer – which is a Java webstart app that you can launch from your browser at The Echo Nest BPM Explorer site.
Check out Adam Lindsay’s latest post on Dissociated Mixes. He’s got a pretty good collection of automatically shuffled songs that sound interesting and eerily different from the original. One example is this remixed audio/video of Beck’s Record Club cover of “Waiting for my Man” by The Velvet Underground and Nico:
This morning, while eating my Father’s day bagel, I got to play some more with the video aspects of the Echo Nest remix API. The video remix is pretty slick. You use all of the tools that you use in the audio remix, except that the object you are manipulating has a video component as well. This makes it easy to take an audio remix and turn it into a video remix. For instance, here’s the remix code to create a remix that includes the first beat of every bar:
audiofile = audio.LocalAudioFile(input_filename) collect = audio.AudioQuantumList() for bar in audiofile.analysis.bars: collect.append(bar.children()) out = audio.getpieces(audiofile, collect) out.encode(output_filename)
To turn this into a video remix, just change the code to:
av = video.loadav(input_filename) collect = audio.AudioQuantumList() for bar in av.audio.analysis.bars: collect.append(bar.children()) out = video.getpieces(av, collect) out.save(output_filename)
The code is nearly identical, differing in loading and saving, while the core remix logic stays the same.
To make a remix of a YouTube video, you need to save a local copy of the video. I’ve been using KeepVid to save local flv (flash video format) of any Youtube video.
Today I played with the track ‘Boom Boom Pow’ by the Black Eyed Peas. It’s a fun song for remix because it has a very strong beat, and already has a remix feel to it. And since the song is about digital transformation, it seems to be a good target for remix experiments. (and just maybe they won’t mind the liberties I’ve taken with their song).
Here’s the original (click through to YouTube to watch it since embedding is not allowed):
The first remix is to only include the first beat of every measure. The code is this:
for bar in av.audio.analysis.bars: collect.append(bar.children())
Change the beat included from beat zero to beat three, and we get something that sounds very different:
Pow Boom Boom
Here’s a version with the beats reversed. The core logic for this transformation is one line of code:
The 5/4 Version
Here’s a version that’s in 5/4 – to make this remix I duplicated the first beat and swapped beats 2 and 3. This is my favorite of the bunch.
These transformations are of the simplest variety, taking just a couple of minutes to code and try out. I’m sure some budding computational remixologist could do some really interesting things with this API.
Note that the latest video support is not in the main branch of remix. If you want to try some of this out you’ll need to check out the bl-video branch from the svn repository. But this is guaranteed to be rolled into the main branch before the upcoming Music Hackday. Update: the latest video support is now part of the main branch. If you want to try it out, check it out from the trunk of the SVN repository. So download the code, grab your API key and start remixing.
Update: As Brian pointed out in the comments there was some blocking on the remix renders. This has been fixed, so if you grab the latest code, the video output quality is as good as the input.
Ben Lacker, one of our leading computational remixologists here at the Echo Nest has been improving the video remix capabilities of the Echo Nest remix API. On Friday, he remixed this mind blower. It’s Coldplay’s music video for ‘The Scientist’ – beat reversed, which means that song is played in reverse order beat by beat (but each beat is still played in forward order). Since Coldplay’s video is already shot in reverse order, the resulting video has a story that unfolds in proper chronological order, but where every second of video runs backwards, while the music unfolds in reverse chronological order while every beat runs forward. I get a little bit of a stomachache watching this video.
Ben has committed the code for this remix to the Echo Nest remix code samples so feel free to check it out and hack on it. I hope to see some more interesting music and video remixes coming out of the upcoming Music Hackday.
Remix is a sophisticated tool to allow you to quickly, expressively, and intuitively chop up existing audio content and create new content based on the old. It allows you to reach inside the music, and let the music’s own musical qualities be your — or your computer’s — guide in finding something new in the old. By using Remix’s knowledge of a given song’s structure, you can render the familiar strange, or the strange slightly more familiar-sounding. You can create countless parameterized variations of a given song — or one of near-limitless length — that respect or desecrate the original, or land on any of countless steps in between.
This release as concentrated on making it easier to install. We now have install instructions for Linux, Mac and Windows. We also now use the FFMpeg encoder/decoder instead of mad and lame. This has a number of advantages; it makes it easier to install, it supports a larger number of file formats, and perhaps most importantly, it is the same decoder that the Echo nest Analyze uses. This ensures that audio segment boundaries fall exactly on zero-crossings.
Remix is really fun to play with, and the results are always interesting and sometimes even musical. Here’s an example of a song released in the last year (can you guess it?) that has been remixed to include only the first beat of each measure.
Doug Repetto, researcher at Columbia University (and founder of dorkbot), has been taking the Echo Net Remix API for a spin. Doug is interested in how beat displacement and re-ordering affects the perception of different kinds of music. To kick off his research, he’s created some really interesting beat rotation experiments. Here’s a couple of examples.
Rich Skaggs & Friends playing Bill Monroe, “Big Mon”, rotated so that the beats are in order 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1:
The same song rotated so that the beats are in order 1 3 4 2 1 3 4 2 1 3 4 2 1 3 4 2:
This time rotated so that the beats are in order 1 3 4 2 1 2 3 4 1 3 4 2 1 2 3 4:
All 3 versions are musically interesting and sound different. I’m amazed at how music that sounds so complex can be manipulated so simply to give such interesting results. Doug has lots more examples of his experiments: rotational energy and centripetal force. If you are interested in computational remixology, it is worth checking out.
Version 1.0 of the Echo Nest remix has been released. Echo Nest Remix is an open source SDK for Python that lets you write programs that manipulate music. For example, here’s a python function that will take all the beats of a song, and reverse their order:
def reverse(inputFilename, outputFilename): audioFile = audio.LocalAudioFile(inputFilename) chunks = audioFile.analysis.beats chunks.reverse() reversedAudio = audio.getpieces(audioFile, chunks) reversedAudio.encode(outputFilename)
When you apply this to a song by The Beatles you get something that sounds like this:
which is surprisingly recognizable, musical – and yet different from the original.
This is my jam lets you create mini-mixes to share with people.
And where would the web be without the ability to add more cowbell to any song.
There’s lots of good documentation already for remix. Adam Lindsay has created a most excellent overview and tutorial for remix. There’s API documentation and there’s documentation for the underlying Echo Nest web services that perform the audio analysis. And of course, the source is available too.
So, if you are looking for that fun summer coding project, or if you need an excuse to learn Python, or perhaps you are a budding computational remixologist download remix, grab an API key from the Echo Nest and start writing some remix code.
Here’s one more example of the fun stuff you can do with remix. Guess the song, and guess the manipulation:
Aaron Meyer’s issued a challenge for someone to remix 79 versions of the song Popcorn. So I fired up one of the remix applications that Tristan and Brian wrote a while back that uses our remix API to stitch all 79 versions of Popcorn together into one 12 minute track – songs are beat matched, tempos are stretched and beats are aligned to form a single seamless (well, almost seamless) version of the Hot Buttered classic. I’m interested to hear what some of the other computational remixologists could do with this challenge. Everyone, stop writing your thesis, and make some popcorn!
Download: A Kettle of Echo Nest Popcorn.