Music Machinery

I spent last weekend in Cannes, participating in the MIDEM Hack Day – an event where music hackers from around the globe gather to hack on music. My hack is called The Drop Machine.  It is a toy web app that plays nothing but the drops.  Here’s a video demo of it:

The interesting bit in this hack is how The Drop Machine finds the drops.  I’ve tried a number of different ways to find the drops in the past – for instance, the app Where’s the Drama found the most dramatic bits of music based on changes in music dynamics. This did a pretty good job of finding the epic builds in certain kinds of music, but it wasn’t a very reliable drop detector. The Drop Machine takes a very different approach – it crowd sources the finding of the drop. And it turns out, the crowd knows exactly where the drop is.  So how do we crowd source finding the drop? Well, every time you scrub your music player to play a particular bit of music on Spotify, that scrubbing is anonymously logged. If you scrub to the chorus or the guitar solo or the epic drop, it is noted in the logs. When one person scrubs to a particular point in a song, we learn a tiny bit about how that person feels about that part of the song – perhaps they like it more than the part that they are skipping over  – or perhaps they are trying to learn the lyrics or the guitar fingering for that part of the song. Who’s to say? On an individual level, this data wouldn’t mean much. The cool part comes from the anonymous aggregate behavior of millions of listeners, from which a really detailed map of the song emerges.  People scrub to just before the best parts of the song to listen to them.  Let’s take a look at a few examples.

For starters here’s a plot that shows the most listened to part of the song In the Air Tonight by Phil Collins based upon scrubbing behavior:

The prominent peak at 3:40 is the point when the drums come in.  Based upon scrubbing behavior alone, we are able to find arguably the most interesting bit of that song.

Here’s another example – Whole Lotta Love by Led Zeppelin:

The trough at 1:40 corresponds to the psychedelic bits while the peak at 3:20 is the guitar solo. Again, by looking at scrubbing behavior we get a really good indication of what parts of a song listeners enjoy the most.

When we look at scrubbing behavior for dance music, especially dubstep and brostep, we see a very characteristic strong peak, usually at around a minute into the song. This is invariably ‘the drop’. Here are some examples:

The scrubbing behavior not only shows us where the drop is, but it also shows us how intense the drop is – drops with lots of appeal get lots of attention (and lots of scrubs) while songs with milder drops get less attention. Here’s a milder drop by Skrillex:

Compare that to the much more intense drop:

Songs with more intense drops have more prominent scrubbing and listening peaks at the drop than others.  The Drop Machine uses the prominence of the peak at the drop to find the songs with the most intense drops.

Putting it all together, the Drop Machine searches through the most popular dance, dubstep and brostep tracks and finds the ones with the most prominent listening peaks based upon scrubbing behavior. It then surfaces these tracks into a playlist, and then plays 10 seconds of each track centered around the drop. The result is non-stop drop. Add in a bit of animation synchronized to the music and that’s the Drop Machine.

Currently, the Drop Machine is an internal-use only hack, I’m working on making a public version, so hopefully the world won’t have to wait too long before you all can listen to the Drop Machine.