Archive for category fun
In search of the click track
Posted by Paul in code, fun, Music, The Echo Nest on March 2, 2009
Sometime in the last 10 or 20 years, rock drumming has changed. Many drummers will now don headphones in the studio (and sometimes even for live performances) and synchronize their playing to an electronic metronome – the click track. This allows for easier digital editing of the recording. Since all of the measures are of equal duration, it is easy to move measures or phrases around without worry that the timing may be off. The click track has a down side – some say that songs recorded against a click track sound sterile, that the missing tempo deviations added life to a song.
I’ve always been curious about which drummers use a click track and which don’t, so I thought it might be fun to try to build a click track detector using the Echo Nest remix SDK ( remix is a Python library that allows you to analyze and manipulate music). In my first attempt, I used remix to analyze a track and then I just printed out the duration of each beat in a song and used gnuplot to plot the data. The results weren’t so good – the plot was rather noisy. It turns out there’s quite a bit of variation from beat to beat. In my second attempt I averaged the beat durations over a short window, and the resulting plot was quite good.
Now to see if we can use the plots as a click track detector. I started with a track where I knew the drummer didn’t use a click track. I’m pretty sure that Ringo never used one – so I started with the old Beatle’s track – Dizzy Miss Lizzie. Here’s the resulting plot:
This plot shows the beat duration variation (in seconds) from the average beat duration over the course of about two minutes of the song (I trimmed off the first 10 seconds, since many songs take a few seconds to get going). In this plot you can clearly see the beat duration vary over time. The 3 dips at about 90, 110 and 130 correspond to the end of a 12 bar verse, where Ringo would slightly speed up.
Now lets compare this to a computer generated drum track. I created a track in GarageBand with a looping drum and ran the same analysis. Here’s the resulting plot:
The difference is quite obvious, and stark. The computer gives a nice steady, sterile beat, compared to Ringo’s.
Now let’s try some real music that we suspect is recorded to a click track. It seems that most pop music nowadays is overproduced, so my suspicion is that an artist like Britney Spears will record against a click track. I ran the analysis on “Hit me baby one more time” (believe it or not, the song was not in my collection, so I had to go and find it on the internet, did you know that it is pretty easy to find music on the internet?). Here’s the plot:
I think it is pretty clear from the plot that “Hit me baby one more time” was recorded with a click track. And it is pretty clear that these plots make a pretty good click track detector. Flat lines correspond to tracks with little variation in beat duration. So lets explore some artists to see if they use click tracks.
First up: Weezer:
Nope, no click track for Weezer. This was a bit of a surprise for me.
How about Green Day?
Yep – clearly a click track there. How about Metallica?
No click track for Lars! Nickeback?
update: fixed nickleback plot labels (thanks tedder)
No surprise there – Nickleback uses a click track. Another numetal band (one that I rather like alot) is Breaking Benjamin:
It is clear that they use a click track too – but what is interesting here is that you can see the bridge – the hump that starts at about 130 seconds into the song.
Of course John Bonham never used a click track – but lets check for fun:
So there you have it, using the Echo Nest remix SDK, gnuplot and some human analysis of the generated plots it is pretty easy to see which tracks are recorded against a click track. To make it really clear, I’ve overlayed a few of the plots:
One final plot … the venerable stairway to heaven is noted for its gradual increase in intensity – part of that is from the volume and part comes from in increase in tempo. Jimmy Page stated that the song “speeds up like an adrenaline flow”. Let’s see if we can see this:
The steady downward slope shows shorter beat durations over the course of the song (meaning a faster song). That’s something you just can’t do with a click track. Update – as a number of commenters have pointed out, yes you can do this with a click track.
The code to generate the data for the plots is very simple:
def main(inputFile):
audiofile = audio.LocalAudioFile(inputFile)
beats = audiofile.analysis.beats
avgList = []
time = 0;
output = []
sum = 0
for beat in beats:
time += beat.duration
avg = runningAverage(avgList, beat.duration)
sum += avg
output.append((time, avg))
base = sum / len(output)
for d in output:
print d[0], d[1] - base
def runningAverage(list, dur):
max = 16
list.append(dur)
if len(list) > max:
list.pop(0)
return sum(list) / len(list)
I’m still a poor python programmer, so no doubt there are better Pythonic ways to do things – so let me know how to improve my Python code.
If any readers are particularly curious about whether an artist uses a click track let me know and I’ll generate the plots – or better yet, just get your own API key and run the code for yourself.
Update: If you live in the NYC area, and want to see/hear some more about remix, you might want to attend dorkbot-nyc tomorrow (Wednesday, March 4) where Brian will be talking about and demoing remix.
Update – Sten wondered (in the comments) how his band Hungry Fathers would plot given that their drummer uses a click track. Here’s an analysis of their crowd pleaser “A day without orange juice” that seems to indicate that they do indeed use a click track:
Update: More reader contributed click plots are here: More on click tracks ….
Update 2: I’ve written an application that lets you generate your own interactive click plots: The Echo Nest BPM Explorer
The Echo Nest Remix SDK
Posted by Paul in fun, Music, The Echo Nest on February 28, 2009
One of the joys of working at the Echo Nest is the communal music playlist. Anyone can add, rearrange or delete music from the queue. Of course, if you need to bail out (like when that Cindi Lauper track is sending you over the edge) you can always put on your headphones and tune out the mix. The other day, George Harrison’s “Here Comes the Sun” started playing, but this was a new version – with a funky drum beat, that I had never heard before – perhaps this was a lost track from the Beatle’s Love? Nope, turns out it was just Ben, one of the Echo Nest developers, playing around with The Echo Nest Remix SDK.
The Echo Nest Remix SDK is an open source Python library that lets you manipulate music and video. It sits on top of the Echo Nest Analyze API, hides all of the messy details of sending audio back to the Echo Nest, and parsing the XML response, while still giving you access to the full power of the API.
remix – is one of The Echo Nest’s secret weapons – it gives you the ability to analyze and manipulate music – and not just audio manipulations such as filtering or equalizing, but the ability to remix based on the hierarchical structure of a song. remix sits on top of a very deep analysis of the music that teases out all sorts of information about a track. There’s high level information such as the key, tempo time signature, mode (major or minor) and overall loudness. There’s also information about the song structure. A song is broken down into sections (think verse, chorus, bridge, solo), bars, beats, tatums (the smallest perceptual metrical unit of the song) and segments (short, uniform sound entities). remix gives you access to all of this information.
I must admit that I’ve been a bit reluctant to use remix – mainly because after 9 years at Sun Microsystems I’m a hard core Java programmer (the main reason I went to Sun in the first place was because I liked Java so much). Every time I start to use Python I get frustrated because it takes me 10 times longer than it would in Java. I have to look everything up. How do I concatenate strings? How do I find the length of a list? How do I walk a directory tree? I can code so much faster in Java. But … if there was ever a reason for me to learn Python it is this remix SDK. It is just so much fun – and it lets you do some of the most incredible things. For example, if you want to add a cowbell to every beat in a song, you can use remix to get the list of all of the beats (and associated confidences) in a song, and simply overlap a cowbell strike at each of the time offsets.
So here’s my first bit of Python code using remix. I grabbed one of the code samples that’s included in the distribution, had the aforementioned Ben spend two minutes walking me through the subtleties of Audio Quantum and I was good to go. My first bit of code just takes a song and swaps beat two and beat three of all measures that have at least 3 beats.
def swap_beat_2_and_3(inputFile, outputFile):
audiofile = audio.LocalAudioFile(inputFile)
bars = audiofile.analysis.bars
collect = audio.AudioQuantumList()
for bar in bars:
beats = bar.children()
if (len(beats) >= 3):
(beats[1], beats[2]) = (beats[2], beats[1])
for beat in beats:
collect.append(beat);
out = audio.getpieces(audiofile, collect)
out.encode(outputFile)
The code analyzes the input, iterates through the bars and if a bar has more than three beats, swaps them. (I must admit, even as a hard core Java programmer, the ability to swap things with (a,b) = (b,a) is pretty awesome) and then encodes and writes out a new audiofile. The resulting audio is surprisingly musical. Here’s the result as applied to Maynard Ferguson’s “Birdland”:
This is just great programming fun. I think I’ll be spending my spare coding time learning more Python so I can explore all of the things one can do with remix.
The Led Zeppelin Graph
Posted by Paul in fun, Music, The Echo Nest on February 26, 2009
I’ve been pretty busy figuring out the lay of the land at the new job, so I haven’t had too much time for recreational programming. However, last night, while my lovely wife was watching Dr. House demonstrate his excellent interpersonal skills, I got a chance to write a little bit of code to generate an artist graph using the Echo Nest Developer API.
The idea is to generate a graph that shows the artist similarity space in a fashion that can encourage exploration of the artist space. To do this, I simply use the Echo Nest get_similar call to walk the artist graph. Instead of getting bogged down in some graphics library to create the visualization, I just output ‘.DOT’ commands and render the whole thing using graphviz. Graphviz does all the hard work figuring out how to layout the graph. Here’s a tiny example of some graphviz output:
One of the problems with making these sort of graphs is that they can get extremely complicated, very quickly. Even after just a few steps away from the seed artist in the crawl of the artist graph there can be 100s of artists and 1000s of connections. Without some care, the graph quickly turns into an unreadable tangle. However, since we want to use these graphs for exploration of the artist space we can make a simplification that eliminates much of the complexity. For exploration, people tend to start from a known artist, and then proceed to lesser known artists. If we make our graph work in the same way, we will eliminate a large number of extraneous connections. Instead of connecting all artists that we encounter in our crawl of the artist graph, we only connect new artists to more popular artists that are already in the graph. This gives us an easy to manage directed, acyclic graph that flows from very familiar artists to unknown artists.
The pseudocode to do this is very simple:
add a seed artist to the work queue
while the work queue is not empty
curArtist <= the next artist from the queue
for each artist similar to curArtist
if similar artist less familiar than curArtist
plot link to similar artist
add similar artist to workqueue
The real java code is not much more complicated:
while (workQueue.size() > 0) {
Artist artist = workQueue.remove(0);
List<Scored<Artist>> simArtists = echoNest.getSimilarArtists(artist, 0, 6);
float familiarity = echoNest.getFamiliarity(artist);
for (Scored<Artist> scoredArtist : simArtists) {
Artist similarArtist = scoredArtist.getItem();
float simFamiliarity = echoNest.getFamiliarity(similarArtist);
if (simFamiliarity < familiarity) {
out.printf("\"%s\" -> \"%s\";\n", artist.getId(), similarArtist.getId());
if (!plottedSet.contains(similarArtist)) {
workQueue.add(similarArtist);
plottedSet.add(similarArtist);
out.printf("\"%s\" [label=\"%s\"]\n", similarArtist.getId(), similarArtist.getName());
}
}
}
}
This yields some fun graphs. Here’s a detail from a graph created using Led Zeppelin as the see artist:
And the full graph in all its glory is here:
I can think of all sorts of things to add to this artist graph. We could size the nodes based upon the familiarity of the artist. We could color the artists based upon how ‘hot‘ the artist is. We could replace the graphviz with a real graphing library like prefuse and make the whole graph interactive – so you could actively explore the artist space, click on a node, read reviews about the artist, listen to their music, watch their videos.
Astute readers may have noticed that I’m making calls using an EchoNest library. That’s one of the things I’ve been working on in the last week – building a Java client library for the EchoNest developer API. I’ll be releasing this soon, once I figure out the best way to release an open source client library here at The Echo Nest. I should hopefully get something released by the end of this week. If you are interested in a sneak preview of the Java client library, let me know.
A recommender comic …
Posted by Paul in fun, recommendation on February 23, 2009
Damn you, Mr. Bezos!
Posted by Paul in fun, Music, recommendation on February 22, 2009
Once again, I was blind-sided by the Amazon recommender. I was placing an order for a few books that my wife wanted. Easy enough, and it would only take 5 minutes. But while I was adding Marie’s books to the shopping cart, a recommendation for a new Keith Emerson CD caught my eye. The last thing I bought by KE was not so good, but the reviews for this CD were rather positive – and so I added it to the cart. And then another Keith Emerson Anthology CD was recommened “just for me” – which has some songs I haven’t listened to for years and are still sitting on vinyl in my attic. That 2 CD set found its way into my shopping cart too.

And then, while at the final checkout, the new Ben Fry / Processing book was sitting there, with 13 excellent reviews. How could I pass that up? And so with an extra $80 removed from my wallet, I finally checked out of the store. Really, that should be illegal. But I’m looking forward to the new tunes and the new book.
In which I am ridiculed for my music tastes …
Posted by Paul in fun, Music, recommendation on February 16, 2009
I was giving a talk last week about music recommendation at a local college. I was explaining how some of the various online music recommenders work when I noticed that some of the students were chuckling and laughing. I had checked my fly right before I started talking so I knew it wasn’t that. Then some wise guy in the front row made it all clear: “Do you really like to listen to Hilary Duff?”. After a moment of confusion, I realized that I was showing my Pandora radio stations that included the second most infamous Hilary.

My Pandora Radio stations
I tried to explain that I sometimes listen to my Pandora radio with my 13 year-old daughter. I’m not sure that they really believed that.
This evening, my daughter and I were having dinner and talking about music. She’s past the Hilary Duff and Hannah Montana phase. She’s moved onto the Veronicas (check out her latest review) – so we listend to a bit of Veronicas’ radio on Last.fm – which has now been faithfully scrobbled as part of my listening history forever:

I do like listening to music with my daughter. She knows all of the artists, and (seemingly more important), all the back stories, interconnections, failures and gossip about the artists. That seemed to be as important as the music itself. And although it is fun to listen to with my daughter, the music is not really to my taste. I do want to make it clear to anyone, whether it is a class at the local college or a potential future employer that I’m not really that into bubblegum pop.
A song for Valentine’s day
(via Chris M. )












