Archive for category playlist

The Playlist Survey

[tweetmeme source= ‘plamere’ only_single=false] Playlists have long been a big part of the music experience.  But making a good playlist is not always easy.  We can spend lots of time crafting the perfect mix, but more often than not, in this iPod age, we are likely to toss on a pre-made playlist (such as an album),  have the computer generate a playlist (with something like iTunes Genius) or (more likely) we’ll just hit the shuffle button and listen to songs at random.   I pine for the old days when Radio DJs would play well-crafted sets – mixes of old favorites and the newest, undiscovered tracks – connected in interesting ways.  These professionally created playlists magnified the listening experience.   The whole was indeed greater than the sum of its parts.

The tradition of the old-style Radio DJ continues on Internet Radio sites like Radio Paradise. RP founder/DJ Bill Goldsmith says of   Radio Paradise: “Our specialty is taking a diverse assortment of songs and making them flow together in a way that makes sense harmonically, rhythmically, and lyrically — an art that, to us, is the very essence of radio.”  Anyone who has listened to Radio Paradise will come to appreciate the immense value that a professionally curated playlist brings to the listening experience.

I wish I could put Bill Goldsmith in my iPod and have him craft personalized playlists for me  – playlists that make sense harmonically, rhythmically and lyrically, and customized to my music taste,  mood and context . That, of course, will never happen. Instead I’m going to rely on computer algorithms to generate my playlists.  But how good are computer generated playlists? Can a computer really generate playlists as good as Bill Goldsmith,  with his decades of knowledge about good music and his understanding of how to fit songs together?

To help answer this question,  I’ve created a Playlist Survey – that will collect information about the quality of playlists generated by a human expert, a computer algorithm and a random number generator.   The survey presents a set of playlists and the subject rates each playlist in terms of its quality and also tries to guess whether the playlist was created by a human expert, a computer algorithm or was generated at random.

Bill Goldsmith and Radio Paradise have graciously contributed 18 months of historical playlist data from Radio Paradise to serve as the expert playlist data. That’s nearly 50,000 playlists and a quarter million song plays spread over nearly 7,000 different tracks.

The Playlist Survey also servers as a Radio DJ Turing test.  Can a computer algorithm (or a random number generator for that matter) create playlists that people will think are created by a living and breathing music expert?  What will it mean, for instance, if we learn that people really can’t tell the difference between expert playlists and shuffle play?

Ben Fields and I will offer the results of this Playlist when we present Finding a path through the Jukebox – The Playlist Tutorial – at ISMIR 2010 in Utrecth in August. I’ll also follow up with detailed posts about the results here in this blog after the conference.  I invite all of my readers to spend 10 to 15 minutes to take The Playlist Survey.  Your efforts will help researchers better understand what makes a good playlist.

Take the Playlist Survey

, , , , , ,


The Spotified Billboard Charts – now with real playlists

Last month I Spotified the Billboard charts by using the Billboard and Spotify APIs.  However, I wasn’t actually able to create real Spotify playlists since the Spotify web API doesn’t allow creating or manipulating playlists.  But now I’m a premium Spotify user and as a premium user I can use the libspotify / despotify API to do just about anything that the official Spotify client can do.  With my new found Spotify superpower, I revamped my Billboard charts creator to create real Spotify playlists:

The Spotified Billboard Singles Charts

Instead of having to click on every song to listen to it, just click on the chart – this will open the Spotify playlist – hit play and you’ll be in Billboard chart heaven (or hell, depending on your music taste).

To interact with Spotify I used Jotify – a Java client library (based on despotify)  for Spotify.  Jotify is  well written, full featured library written by Felix Bruns (who has been extremely helpful in answering my questions).   I highly recommend Jotify.

, ,

1 Comment

Paul’s Music Wreckommender

I just posted my music hack day hack.  It is called Paul’s Music Wreckommender.  Use this Wreckommender to find anti-recommendations.  Give the wreckommender an artist that you like and it will give you a playlist of tracks from artists that are very different from the seed artist.  Some obvious use cases:

  • Your 14 year old daughter’s slumber party is getting too loud. Send the girls home by putting on the Hannah Montana Wreckommender – which yields a playlist with tracks by Glenn Gould, Dream Theater and Al Hirt.
  • It’s time to break up with your girl friend.  Give her the ‘You are the wind beneath my wings‘ wrecklist and your intentions will be clear.
  • If you like ‘everything but country’ then Garth Williams will guide the way:  Garth Williams Wreckommendations

You can try it out at

How it works:

This was a pretty easy hack.  I already had a playlister engine with some neat properties.  It maintains a complete artist graph using Echo Nest artist similarities, so I can make  make routes through the artist space for making smooth artist/song transitions. Adapting this playlister engine to create wreckommendations was really easy.  To create the recommendations,  I find the seed node in the graph and then from this node I find the set of artists that have the longest ‘shortest path’ to the seed artist.  These are the artists that are furthest away from the seed artists.  I then select songs from this set to make my ‘wrecklist’.   However, this list isn’t the best list.  There are a small set of artists that are far away from everything. These artists become frequent wrecommendations for many many artists, which is bad.  To avoid this problem I adapted the algorithm to find far away artist clusters and then draw artists from that cluster.  This gives yields a playlist with much more variety.

This hack is primarily for fun, but I think there’s something in the wreckommendations that is worth persuing.   When asked to describe their taste in music, many people will use a negative – such as “Anything but country and rap”.  If this is really the case, then using the wreckommender to literally find ‘anything but country and rap’ – whether it is J-Pop or crabcore might actually be useful.


A couple of sources of inspiration for this hack. First, the name. A word like ‘wreckommendation’ clearly deserves an application.  Second, a coffee pot conversation with Reid, and finally, the LibraryThing Unsuggester, which does a similar thing for books (but in a very different way).

I hope you like the wreckommender, let me know if you find any interesting wreckommendations.

, , , ,


Playdar polishing

Playdar got a couple of steps closer to being ready for the general public this week.  First of all , there are now installers for Windows and Mac that make it dead simple to install Playdar.  I tried the Mac OS X version (by Max Howell).  It could not have been easier to install.  I downloaded a file opened it, clicked on the pane and Playdar was added to my Prefs panel.  It scanned my music and I was ready to go.  Quite the contrast to a couple weeks ago when I hurt myself when trying  to compile Erlang from source.

Now that Playdar is easy to install, we need some apps.  Luckily Toby has been working hard on Playgrub.  Playgrub is a an app that you install in your browser tool bar. When you visit a web page that mentions music, click on the ‘grub’ link and Playgrub will scour the page for all the artists and tracks, resolve them using Playdar and give you a playlist for the page.  Here’s Playgrub in action.  I visited the page for the Weezer Blue album.  I clicked on Playgrub and in a few seconds, Playgrub showed me this:

Playgrub via Playdar was able to find the audio for the music on the page (I happen to own this album so Playdar didn’t have to go far) and give me a playlist that I could play right now, or share with others.   It is really cool.   Toby writes more about Playgrub and even has a video showing it in action in this post: Playgrub new and improved.

Two big steps in one week for Playdar – an installer, and a compelling app.  Playdar is getting closer to being ready for the mainstream.  And I think we’ll be seeing even more progress. This Friday the Echo Nest will be hosting the first face-2-face Playdar summit, where about a dozen hard core Playdar folks will gather in one room and map out the next steps for Playdar.  I’m really looking forward to it.

, ,

Leave a comment

MOG makes playlists easy

MOG has posted a video demonstrating their new playlist editor for the soon to be released MOG all access.

It looks pretty nifty – lightweight, tag-able, shareable playlists.  It’s nice to see MOG paying attention to playlists.   With millions of songs to chose from, music discovery gets to be a big problem.   Playlists can help with that.   Of course, playlists bring their own discovery problems. How can I discovery new playlists that contain music that I like?   Currently, most sites that support playlists and playlist sharing only provide limited ways for people to discover new playlists. However, as playlists become more ubiquitous, sites like MOG will need to expand the tools for helping people find new and interesting playlists.  Some options for playlist discovery:

  • Search for playlists by tag.  Example: “Find playlists tagged ’emo’ and ‘christmas'”
  • Search for playlist by artist / track.  Example: “Find playlists that have songs by Deerhoof”
  • Query by example.  Example: “Find playlists that are similar to this playlist”
  • Popularity. Example:  “Play me the most popular playlists” or “Play me the Billboard hot 100”
  • Social discovery.  Example: “What playlists are my friends listening to now?”
  • Expert curated. Example: “Give me the Pitchfork 100 playlist”
  • Machine made.  Example: “Build me a playlist that is similar to this playlist” or “Build me a playlist for tags: ’emo’, ‘female’, ’90s'”
  • Recommended playlists. Example: “Find me playlists that I will like based upon my music taste and my context (e.g. the time of day).

It’s good to see MOG working hard to make the creation of playlists easy. Next step is to make finding new and interesting playlists easy.

, ,

Leave a comment

Playing with Playdar

On Saturday morning I opened my web browser, built a playlist of a few songs and started to listen to them while I went about my morning computer tasks. Some of the songs in the playlist were on my laptop, while some were on the mac mini in the family room, and some were on a laptop of a friend that was on the other side of the Atlantic ocean. And if my friend in London had closed his laptop before I listened to ‘his’ song on my playlist it could have been replaced by a copy of the song that was on the computer of a friend in Seattle. I had a seamless music listening experience despite the fact that the music was scattered across a handful of computers on two continents. Such is the power of Playdar.
Playdar is a music content resolver. It is designed to solve one problem: given the name of track, find me a way to listen to it right now. You run Playdar on any computer that you own that has music and Playdar will make it easy to listen to all of that music as if it were on your local machine. The Playdar content resolver can also talk to other Playdar resolvers too, so if Playdar can’t find a track on my local network, it can ask my friend if it knows where the track is, extending my listening reach.

Playdar runs as a web service using standard web protocols for communicating with applications.  When Playdar receives a request to resolve a track it runs through a list of prioritized content resolvers looking for the track. First it checks your local machine, then your local network.  If it hasn’t found it there it could, if so configured, try your friends computers, or even a commercial song resolver (One could imagine for example, a music label offering up a portion of their catalog via a content resolver as a way to expose more listeners to their music).  Playdar will do its best to find a copy of a song that you can listen to now. Playdar enables a number of new listening modes:

  • Listen to my music anywhere – with Playdar, I don’t have to shoehorn my entire music collection onto every computer that I own just so I can listen to it no matter what computer I’m on.  I can distribute my music collection over all my computers – and no matter what computer I’m on I have all my music available.
  • Save money for music streamers – Music streaming services like, Spotify and Pandora spend  money for every song that is streamed.  Often times, the listener will already own the song that is being streamed.   Playdar-enabled music streaming services could save streaming costs by playing  a local copy of a song if one is available.
  • Share playlists and mixtapes – with Playdar a friend could give me a playlist (perhaps in a XSPF format) and I could listen to the playlist even if I don’t own all of the songs.
  • Pool the music – At the Echo Nest, everyone has lots of music in their personal collections.  When we are all in the same room it is fun to be able to sample music from each other.  iTunes lets you do this but  searching through 15 separate collections for music in iTunes is burdensome.  With Playdar, all the music on all of the computers running Playdar on your local lan can be available for you to search and play without any of the iTunes awkwardness.
  • Add Play buttons to songs on web pages –  Since Playdar uses standard web protocols, it is possible to query and control Playdar from Javascript – meaning that Playdar functionality can be embedded in any web page.  I could blog about  a song and  sprinkle in a little Javascript to add a ‘play’ button to the song that would use Playdar to find the best way to play the song.  If I write a review about the new Beatles reissue and want the reader to be able to listen to the tracks I’m writing about, I can do that without having to violate Beatles copyrights.  When the reader clicks the play button, Playdar will find the local copy that is already on the reader’s computer.

Playdar’s Marconi Moment

Playdar is the brainchild of RJ, the inventor of the audioscrobbler and one of the founders of  RJ started coding Playdar in March of this year – but a few weeks ago he threw away the 10,000 lines of C++ code and started to rewrite it from scratch in Erlang.  A few days later RJ tweeted I should be taken aside and shot for using C++ for Playdar originally. It’s criminal how much more concise Erlang is for this. Less than 3 weeks after starting from a clean sheet of paper, the new Erlang-based Playdar had its first transatlantic track resolution and streaming. The moment occurred on Friday, October 16th.  Here’s the transcript from the IRC channel (tobyp is Toby Padilla, of MusicMobs and fame) when London-based RJ first streamed a track from Toby’s Seattle computer:

[15:40:46] <tobyp>
[15:41:06] <RJ2> woo, transatlantic streaming
[15:41:19] <tobyp> hot!
[15:41:35] <RJ2> playdar’s marconi moment
[15:41:42] <tobyp> hahah

An incredible amount of progress has been made in the last two weeks,  a testament to RJ’s skills as much as Erlang’s expressiveness.  Still, Playdar is not ready for the general public.  It requires a bit of work to install and get running – (yep, the erlang runtime is required), but developer Max Howell has been working on making a user-friendly package to make it easy for anyone to install. Hopefully it won’t be too long before Playdar is ready for the masses.

Even though it is new, there’s already some compelling apps that use Playdar.  One is Playlick:


Playlick is a web application, developed by James Wheare that lets you build playlists. It uses Playdar for all music resolution.  Type in the name of an album and Playlick /  Playdar will find the music for you and let you listen to it.  It’s a great way to see/hear the power of Playdar.

Adding custom content resolvers

One of the strengths of Playdar is that it is very easy to add new resolvers.  If you are a music service provider you can create a Playdar content resolver that will serve up your content.    I wrote a content resolver that uses the Echo Nest to resolve tracks using our index of audio that we’ve found on the web.  This resolver can be used as a backstop.  If you can’t find a track on your computer or your friend’s computers the Echo Nest resolver might be able to find a version out there on some music blog.  Of course, the quality and availability of such free-range music is highly variable, so this resolver is a last resort.

Adding a new resolver to Playdar was extremely easy. It took perhaps 30 minutes to write – the hardest part was figuring out git – (thanks to RJ for walking me through the forks, pushes and ssh key settings).    You can see the code here:  Less than 150 lines of code, half of which is boilerplate.  150 lines and 30 minutes to add a whole new collection of music to the Playdar universe.   Hopefully soon we’ll see resolvers for music streaming services like Napster, Rhapsody and Spotify.

What’s Next for Playdar?

Playdar is new – and the plumbing and wiring are still be worked on – but already it is doing something pretty magical – letting me listen to any track I want to right now.  I can see how Playdar could be extended into acting as my music agent.  Over time, my Playdar servers will get to know quite a bit about my music tastes.  They’ll know what music I like to listen to, and when I like to listen to it.   Perhaps someday, instead of asking Playdar to resolve a specific track by name, I’ll just be able to ask Playdar to give me a playlist of new music that I might like.  Playdar can then use an Echo Nest, or an AMG playlister to build a playlist of interesting, relevant new music.  Playdar won’t just be a music resolver, Playdar will be my music agent helping me explore for and discover new music.

, , , , ,


Music Playlist quiz followup

I had a playlist quiz the other day. To recap, I asked, given a set of 6 songs, find the organizing principal and pick a new good song for the playlist.   A few attempted to extend the playlist, but only Adam offered a successful match.   Here are the seed songs, but this time I also include the album art – which may help you decide what songs fit and what don’t:

  • Made to measure – Umphrey’s McGeez
  • Diablo Rojo – Rodrigo Y Gabriella
  • Livin’ Thing – Electric Light Orchestra
  • Two Step – Dave Matthew’s Band
  • Vortex – Burst
  • Almost Honest – Megadeth

Adam’s suggestion of XTC’s Wake up fits well:

We’ll call this playlist, the squared circle.  There are lots more potential album covers for albums in this genre on Flickr: squaredcircle

, , ,

1 Comment

Music Playlist quiz

Here’a a quiz for you music junkies.   For the following playlist,  pick a good next song for the playlist.  Hint, there’s some aspect of each of the songs that is similar.

  • Made to measure – Umphrey’s McGee
  • Diablo Rojo – Rodrigo Y Gabriella
  • Livin’ Thing – Electric Light Orchestra
  • Two Step – Dave Matthew’s Band
  • Vortex – Burst
  • Almost Honest – Megadeth

No help from Echo Nesters please


Uh oh. The celestial jukebox has arrived

The Spotify iPhone app has been approved. With this app, I will now be able to carry 5 million songs in my pocket, and every week thousands more songs will be added to my collection automatically.  This is the proverbial celestial jukebox – the great jukebox in the cloud that lets me listen to any song I want to hear.    This is  going to change how we listen to music.  When we can listen to any song,  anywhere, any time and on any device our current ways of interacting with music will be woefully inadequate.     Shuffle play with 5 million songs just won’t work. Listener’s paralyzed by too much choice will just go back to the Eagles greatest hits album because its easier and safer than trying to find something new.    People will start to wonder  “What good are 5 million songs if I only listen to the 100 that I listened to in high school?”  The new challenge that these next generation music services face is helping their listeners find new and interesting music.  Tools for music discovery will be key to keeping listener’s coming back.   Five years from now, the most successful music sites will be the ones that have figured out how to help people find new music.

What will music discovery look like in 5 years?   I don’t know for sure, but I do know that it will go way beyond the ‘artist radio’ approach that we see now.   I suspect that at the core of music discovery will be a smart, personalized, context-aware playlist engine that will give you a continuous stream of interesting music. The engine will know kind of music you like and don’t like, the kind of music you like to listen to when you are driving vs. working vs. relaxing,  the music taste of the people you are with,  your sense of musical adventure, what your friends are listening to, what songs were played on the TV shows you watched last night, what song fits well with the last song that was played, what artists are in the news, what artists are coming to town in the next few weeks, what artists have new albums coming out. The list goes on and on.  It is hard to predict what will happen in 5 years, but I wouldn’t be too surprised if we see something that looks like this: magicipod

(Image courtesy of David Jennings)


The Stairway Detector

Last night I was watching the pilot for Glee (a snarky TV version of High school musical) with my 3 teenage daughters.  I was surprised to hear the soundtrack filled with songs by the band Journey, songs that  brought me back to my own high school years.   The thing that I like the most about Journey is that many of their songs have this slow and gradual build up over the course of the whole song  as in this song Lovin Touchin Squeezin:

A number of my favorite songs have this slow build up. The canonical example is Zep’s ‘Stairway to Heaven’ – it starts with a slow acoustic guitar and over the course of 8 minutes builds to metal frenzy.    I thought it would be fun to see if I could write a bit of software that could find the songs that have the same arc as ‘Stairway to Heaven’ or ‘Lovin, Touchin Squeezin’  – songs that have this slow build. With this ‘stairway detector’  I could build playlists filled with the songs that fire me up.

The obvious place to start with is to look how the loudness of a song changes overtime. To do this I used the Echo Nest developer API to extract the loudness as a function of time for  Journey’s Lovin, Touchin Squeezin:

louness-journey-no-avgIn this plot the light green curve is the loudness, while the blue line is a windowed average of the loudness.  This plot shows a nice rise in the volume over the course of the song.   Compared to a song like the Beatles ‘Ticket to Ride’ that doesn’t have this upward slope:

loudness-ticket-to-ridFrom these two examples, it is pretty clear that we can build our stairway-detector just by looking at the average slope of the volume. The higher the slope, the bigger the build.  Now, I suspect that there’s lots of ways to find the average slope of a bumpy line – but I like to always try the simplest thing that could possibly work first – and for me the simplest thing was to just divide the average loudness of the second half of the song by the average loudness of the first half of the song.   So for example, with the Journey song the average loudness of the second half of the song is -15.86 db and the average of the first half of the song is -24.37 db.  This gives us a ratio of 1.54, while ‘Ticket to ride’ gets a ratio of 1.06.  Here’s the Journey song with averages shown:

loudness-for-journeyHere are a few more songs that fit the ‘slow build’ profile:

stairway-to-heaven‘Stairway to Heaven’ has a score of 1.6 so it has a bigger build than Journey’s Lovin’.

loudness-for-bridge-over-troubled-waterSimon and Garfunkle’s ‘Bridge over troubled water’ has an even bigger build with a score of 1.7.

Also sprach ZarathustraAlso sprach Zarathustra has a more modest score of  1.56

With this new found metric I analyzed a few thousand of the tracks in my personal collection to find the songs with the biggest crescendos.  The biggest of all was this song by Muse with a whopping score of  3.07:

loudness-for-muse-take-a-bowAnother find is Arcade Fire’s “My Body is a Cage” with a  score of 2.32.


The metric isn’t perfect. For instance, I would have expected Postal Services ‘Natural Anthem’ to have a high score because it has such a great build up, but it only gets a score of 1.19. Looking at the plot we can see why:

loudness-for-postal-service-natural-anthemAfter the initial build up, there’s a drop an energy for that last quarter of the song, so even though the song has a sustained crescendo for 3 minutes it doesn’t get a high score due to this drop.

Of course, we can use this ratio to find tracks that go the other way, to find songs that gradually wind down. These seem to occur less frequently than the songs that build up.  One example is Neutral Milk Hotel’s Two Headed Boy:


Despite the fact that I’m using a very naive metric to find the loudness slope,  this stairway detector is pretty effective in finding songs that have that slow build.   It’s another tool that I can use for helping to build interesting playlists.  This is one of the really cool things about how the Echo Nest approaches music playlisting.   By having an understanding of what the music actually sounds like,  we can build much more interesting playlists than you get from genius-style playlists that only take into account  artists co-occurrence.

, , ,

1 Comment