[tweetmeme source= ‘plamere’ only_single=false] Playlists have long been a big part of the music experience.  But making a good playlist is not always easy.  We can spend lots of time crafting the perfect mix, but more often than not, in this iPod age, we are likely to toss on a pre-made playlist (such as an album),  have the computer generate a playlist (with something like iTunes Genius) or (more likely) we’ll just hit the shuffle button and listen to songs at random.   I pine for the old days when Radio DJs would play well-crafted sets – mixes of old favorites and the newest, undiscovered tracks – connected in interesting ways.  These professionally created playlists magnified the listening experience.   The whole was indeed greater than the sum of its parts.

The tradition of the old-style Radio DJ continues on Internet Radio sites like Radio Paradise. RP founder/DJ Bill Goldsmith says of   Radio Paradise: “Our specialty is taking a diverse assortment of songs and making them flow together in a way that makes sense harmonically, rhythmically, and lyrically — an art that, to us, is the very essence of radio.”  Anyone who has listened to Radio Paradise will come to appreciate the immense value that a professionally curated playlist brings to the listening experience.

I wish I could put Bill Goldsmith in my iPod and have him craft personalized playlists for me  – playlists that make sense harmonically, rhythmically and lyrically, and customized to my music taste,  mood and context . That, of course, will never happen. Instead I’m going to rely on computer algorithms to generate my playlists.  But how good are computer generated playlists? Can a computer really generate playlists as good as Bill Goldsmith,  with his decades of knowledge about good music and his understanding of how to fit songs together?

To help answer this question,  I’ve created a Playlist Survey – that will collect information about the quality of playlists generated by a human expert, a computer algorithm and a random number generator.   The survey presents a set of playlists and the subject rates each playlist in terms of its quality and also tries to guess whether the playlist was created by a human expert, a computer algorithm or was generated at random.

Bill Goldsmith and Radio Paradise have graciously contributed 18 months of historical playlist data from Radio Paradise to serve as the expert playlist data. That’s nearly 50,000 playlists and a quarter million song plays spread over nearly 7,000 different tracks.

The Playlist Survey also servers as a Radio DJ Turing test.  Can a computer algorithm (or a random number generator for that matter) create playlists that people will think are created by a living and breathing music expert?  What will it mean, for instance, if we learn that people really can’t tell the difference between expert playlists and shuffle play?

Ben Fields and I will offer the results of this Playlist when we present Finding a path through the Jukebox – The Playlist Tutorial – at ISMIR 2010 in Utrecth in August. I’ll also follow up with detailed posts about the results here in this blog after the conference.  I invite all of my readers to spend 10 to 15 minutes to take The Playlist Survey.  Your efforts will help researchers better understand what makes a good playlist.

Take the Playlist Survey