Archive for category Spotify
Wicked smart playlists
Over the past few weekends I’ve been working on a little side project called the Playlist Builder Library (or PBL for short). The Playlist Builder Library is a Python library for creating and manipulating playlists. It’s sort of like remix for playlists. With PBL you can take songs from playlists, albums, artists, genres and flexibly combined them, rearrange them, filter them and sort them into new playlists.
For example, here’s a PBL program that creates radio station of today’s top hits but guarantees that every 4th song is either by Sia or Katy Perry:[gist https://gist.github.com/plamere/2fa839150815f040450d]
Here’s the resulting playlist:[spotify spotify:user:plamere:playlist:6TIeQMve7pVBLCAY8WUX3L]
That’s 5 lines a code to create a non-trivial playlist.
PBL supports all sorts of sources for tracks such as Spotify playlists, top tracks from artists, albums, genres, the extremely flexible and powerful Echo Nest playlisting API. These sources can be manipulated in all sorts of interesting ways. Here are a couple more examples:
You can filter all the songs in ‘Your favorite coffeehouse’ to get just the lowest energy songs:
coffee = PlaylistSource('coffeehouse', ucoffee_house) low_energy_coffee = AttributeRangeFilter(coffee, 'echonest.energy', max_val=.5)
You an combine your favorite playlists in a single one:
playlist_names = ['Your Favorite Coffeehouse', 'Acoustic Summer','Acoustic Covers', 'Rainy Day'] all = DeDup(Alternate([Sample(PlaylistSource(n), 10) for n in playlist_names]))
Even sophisticated tasks are really easy. For instance, imagine dad is on a roadtrip with daughter. They agree to alternate between dad’s music and daughter’s music. Dad is selfish, so he makes a playlist that alternates the longest cool jazz tracks with the shortest teen party playlists with this 3 line script:
teen_party = First(Sorter(PlaylistSource('Teen Party'), 'duration'), 10) jazz_classics = Last(Sorter(PlaylistSource('Jazz Classics'), 'duration'), 10) both = Alternate([teen_party, Reverse(jazz_classics)])
Here’s the result[spotify spotify:user:plamere:playlist:0VKGTR6eCPe55bBjezi5z3]
Note that the average duration of Teen Party songs is much less than 3 minutes, while the average duration of Jazz Classics is above 6 minutes. Selfish dad gets to listen to his music twice as long with this jazz-skewing playlist.
There’s a whole lot of nifty things that can be done with PBL. If you are a Python programmer with an itch for creating new playlists check it out. The docs are online at http://pbl.readthedocs.org/ and the source is at https://github.com/plamere/pbl.
PBL is pretty modular so it is easy to add new sources and manipulators, so if you have an idea or two for changes let me know or just send me a pull request.
The Drop Machine
I spent last weekend in Cannes, participating in the MIDEM Hack Day – an event where music hackers from around the globe gather to hack on music. My hack is called The Drop Machine. It is a toy web app that plays nothing but the drops. Here’s a video demo of it:[youtube http://youtu.be/4C6a-MqAF_A]
The interesting bit in this hack is how The Drop Machine finds the drops. I’ve tried a number of different ways to find the drops in the past – for instance, the app Where’s the Drama found the most dramatic bits of music based on changes in music dynamics. This did a pretty good job of finding the epic builds in certain kinds of music, but it wasn’t a very reliable drop detector. The Drop Machine takes a very different approach – it crowd sources the finding of the drop. And it turns out, the crowd knows exactly where the drop is. So how do we crowd source finding the drop? Well, every time you scrub your music player to play a particular bit of music on Spotify, that scrubbing is anonymously logged. If you scrub to the chorus or the guitar solo or the epic drop, it is noted in the logs. When one person scrubs to a particular point in a song, we learn a tiny bit about how that person feels about that part of the song – perhaps they like it more than the part that they are skipping over – or perhaps they are trying to learn the lyrics or the guitar fingering for that part of the song. Who’s to say? On an individual level, this data wouldn’t mean much. The cool part comes from the anonymous aggregate behavior of millions of listeners, from which a really detailed map of the song emerges. People scrub to just before the best parts of the song to listen to them. Let’s take a look at a few examples.
For starters here’s a plot that shows the most listened to part of the song In the Air Tonight by Phil Collins based upon scrubbing behavior:
The prominent peak at 3:40 is the point when the drums come in. Based upon scrubbing behavior alone, we are able to find arguably the most interesting bit of that song.
Here’s another example – Whole Lotta Love by Led Zeppelin:
The trough at 1:40 corresponds to the psychedelic bits while the peak at 3:20 is the guitar solo. Again, by looking at scrubbing behavior we get a really good indication of what parts of a song listeners enjoy the most.
When we look at scrubbing behavior for dance music, especially dubstep and brostep, we see a very characteristic strong peak, usually at around a minute into the song. This is invariably ‘the drop’. Here are some examples:
The scrubbing behavior not only shows us where the drop is, but it also shows us how intense the drop is – drops with lots of appeal get lots of attention (and lots of scrubs) while songs with milder drops get less attention. Here’s a milder drop by Skrillex:
Compare that to the much more intense drop:
Songs with more intense drops have more prominent scrubbing and listening peaks at the drop than others. The Drop Machine uses the prominence of the peak at the drop to find the songs with the most intense drops.
Putting it all together, the Drop Machine searches through the most popular dance, dubstep and brostep tracks and finds the ones with the most prominent listening peaks based upon scrubbing behavior. It then surfaces these tracks into a playlist, and then plays 10 seconds of each track centered around the drop. The result is non-stop drop. Add in a bit of animation synchronized to the music and that’s the Drop Machine.
Currently, the Drop Machine is an internal-use only hack, I’m working on making a public version, so hopefully the world won’t have to wait too long before you all can listen to the Drop Machine.
50 Years Ago in Music
There’s a strong connection between music and memory. Whenever I here the song Lovin You by Minnie Riperton, I’m instantly transported back to 1975 when I spent the summer apprenticed to Tom, my future brother-in-law, fixing electronic organs. I was 15, Tom was 22 and super cool. He had a business (New Hampshire Organ Service) and he had a van with an 8-track player and an FM radio (a rarity in 1975). As we drove between repairs across rural New Hampshire we’d pass the time by listening to the radio. Now, when I hear those radio songs from 1975 it is like I’m sitting in that van again.
Music can be like a time machine. Transporting us to different times in our lives. I was interested in exploring this a bit more. Inspired by @realtimewwii which gives a day-by-day account of World War II, I created a set of dynamically updating Spotify playlists that follow the charts week-by-week.
For example there’s the 50 Years Ago in Music playlist that contains the top 100 or songs that were on the chart 50 years ago. As I write this on April 12, 2015, this playlist is showing the top songs for the week of April 12, 1965.
The music on this playlist sends me back to when I was 5 years old listening to music on our AM radio in the kitchen in the morning while eating breakfast.
If you follow this playlist you’ll be able to re-create what it was like to listen to music 50 years ago. If the mid-sixties doesn’t speak to you musically, there are some other playlists that you can try.
There’s 40 Years Ago in Music that brings me back to 1975 on the road with Tom.
There’s 30 years Ago in Music which is currently playing music from the mid-80s like Madonna and Phil Collins.
There’s 20 Years Ago in Music currently playing music from the mid-90s:
10 Years Ago in Music plays the music that was on the radio when Spotify was just a gleam in Daniel’s eye.
5 Years Ago in Music – the playlist of @echonest in its heyday.
All of the playlists update weekly on Monday. If you’d like a reminder about when they are updated you can follow @50yearmusic. And of course, the code is on github.
How we listen to music
Yesterday I gave a talk at SXSW about what we can learn about how we listen to music by looking at all sorts of listener data that we collect at Spotify. You can see the slides for my talk here … but the slides only tell half the story, the other have are in my words, but those aren’t written down anywhere. You’ll just have to assume that they were very insightful, and a little bit humorous, but at the end told an incredible story leaving you inspired and fulfilled.
The Playlist Miner
The Spotify Web API team pushed out a new feature recently that extends the search API to include playlist search. With this new feature it is now possible to search across all of the popular public playlists created by Spotify users. To try out the new search capability I created a new web app called The Playlist Miner.
The Playlist Miner is a web app that will create a Spotify playlist for you by finding the top songs in all of the playlists that match your criteria. Say, for example, that I want to create a dinner party playlist. First, I find the top playlists that match ‘dinner party’ with The Playlist Miner:
The Playlist Miner will find up to the top 1,000 most popular playlists that match dinner party. It shows them to me, giving me a chance to refine my query to focus in on the exact type of playlist that I am interested in.
For this first try, I see lots of Christmas-oriented playlists (‘Tis the Season after all), but since I’m looking for music for a post-holiday dinner party, I’d rather not have holiday music in the playlist. So I refine my query to find non-Christmas oriented dinner party playlists like so:
The resulting playlists are suitably non-Christmasy.
I like the look of these playlists so I hit the Find Top Tracks button and The Playlist Miner will scour through all of the matching playlists (290 of them in this case) and find the most frequently appearing tracks.
Once the top 100 tracks are found, I can save them to Spotify as my own playlist.
Selecting Prefer more distinctive labor and delivery tracks adjusts the track order for popularity so that tracks that are more distinctive to the particular playlist context will rise to the top. You can also use logical operators to focus in on the exact type of playlist you want to. You can search for “work out” OR workout NOT running to find workout playlists without running in their titles/descriptions.
Under the hood – The Playlist Miner uses lots of bits of the Spotify API – user authentication, playlist search, playlist reading, playlist saving and more. The app is a an API calling beast – aggregating all the tracks from a thousand playlists requires 1,000 API calls. It’s a testament to the Spotify Web API that it doesn’t even blink under the load. You can play with the code on github.
It’s fun to use The Playlist Miner to explore the quirkier aspects of how people listen to music. There are ironing playlists and sleeping baby playlists. There are playlists for getting psyched and playlists for Labor and Delivery. With the Playlist Miner you can pull from all the playlists created for a particular purpose and build your own. Give it a try.
The Million Songs of Christmas
No other holiday dominates our listening like Christmas. During this season, we are exposed to a seemingly never ending playlist of Christmas music. So its no surprise that there’s a huge amount of Christmas music available on Spotify. How much? Let’s take a look.
How much Christmas music is there?
It is actually quite hard to pinpoint the exact number of Christmas songs. First, every week during the holiday season thousands more Christmas songs are added to the set. Second, some songs are seasonal – is Frosty The Snowman a Christmas song? Not literally, but it gets a lot of play at this time of year, even by the antipodes. Finally, there are a number of other holidays and celebrations at this time of year such as Hanukkah, Boxing Day, New Years, Kwanzaa, the Winter Solstice, and Festivus that we want to include in this category. So when I say “Christmas Music” I’m referring to western music that is played primarily during December. There’s probably a better term to describe this music, but terms like seasonal, and holiday have their own special baggage – perhaps something like music coincident with the northern hemispheric winter solstice is the most precise description, but lets stick with Christmas music just to keep things simple. So how much Christmas music is there? In early December 2014, crack music + data nerd Aaron Daubman dove into the Spotify + Echo Nest music catalog and found 914,047 Christmas tracks – that’s just under a million Christmas tracks. Let’s unwrap this dataset to see what we can find.
First, some basic stats: Those 914,047 tracks represent 180,660 unique songs and were created by 63,711 unique artists – from Aaron Neville to Zuma the King. The top 20 artists with the most Christmas tracks in the Spotify catalog are all pre-Beatles artists:
Artists with the most Christmas Tracks
|4||Nat King Cole||11613|
|5||Johann Sebastian Bach||8958|
|15||The Andrews Sisters||3567|
Yes, that’s right, Bing Crosby has 22,382 different Christmas tracks (!) in the Spotify catalog. Now, a little digression on what we consider to be a unique track. Music, especially popular music, is released in many forms. A very popular song, such as Bing Crosby’s White Christmas, may appear on a wide range of albums – from the original studio release to a plethora of Christmas Compilations and artist ‘best of’ albums. Each of these track releases may have different album art, different rights holders and regional licenses. Thus, even though the audio for White Christmas may be the same on each of the release, we consider each release as a different track.
Let’s take a closer look at Bing Crosby’s White Christmas. In our catalog of nearly a million Christmas tracks, 2,196 of them are Bing Crosby’s classic. I’ll say that again, just because it is a rather phenomenal fact – there are 2,196 different albums on Spotify that contain Bing’s White Christmas. It is hard to believe, so I created a web page that contains all 2,196 of the albums so you can see them all. Click on the image below to load them all up (warning – with 2000+ album covers it’s a bit of a browser buster).
White Christmas isn’t the only uber-track of the holidays. Here are the top 25 Christmas tracks based upon the number of times they have been released on an album:
The most released Christmas tracks
|1||Bing Crosby – White Christmas||2196|
|2||Eartha Kitt – Santa Baby||1286|
|3||Elvis Presley – Blue Christmas||1285|
|4||Frank Sinatra – Jingle Bells||1121|
|5||Harry Belafonte – Mary’s Boy Child||904|
|6||Bing Crosby – Silver Bells||881|
|7||Nat King Cole – The Christmas Song||870|
|8||Frank Sinatra – The Christmas Waltz||811|
|9||Rosemary Clooney – Suzy Snowflake||788|
|10||Bobby Helms – Jingle Bell Rock||779|
|11||Elvis Presley – White Christmas||738|
|12||Judy Garland – Have Yourself a Merry Little Christmas||735|
|13||Frank Sinatra – White Christmas||703|
|14||Frank Sinatra – Christmas Dreaming||696|
|15||Frank Sinatra – Have Yourself a Merry Little Christmas||695|
|16||Elvis Presley – Silent Night||688|
|17||Elvis Presley – I Believe||664|
|18||Frank Sinatra – Santa Claus Is Coming to Town||660|
|19||Louis Armstrong – Zat You Santa Claus||598|
|20||Dean Martin – The Christmas Blues||575|
|21||Frank Sinatra – Mistletoe and Holly||568|
|22||Louis Armstrong – Cool Yule||566|
|23||Frank Sinatra – Silent Night||563|
|24||Bing Crosby – Jingle Bells||560|
|25||Elvis Presley – Santa Claus Is Back in Town||559|
You can see all of the releases for Elvis’s Blue Christmas and Eartha Kitt’s Santa Baby here:
So there are lots of copies of Bing Crosby’s White Christmas and Eartha Kitt’s Santa Baby out there – but what are the most common Christmas songs overall? Which ones have been recorded the most by any artist? The following table shows the top 25:
Most recorded songs
|5||Joy to the World||9093|
|6||The First Noel||8731|
|7||Have Yourself a Merry Little Christmas||8511|
|8||O Holy Night||7925|
|9||Hark The Herald Angels Sing||7727|
|10||The Christmas Song||7673|
|11||Away in a Manger||7544|
|12||God Rest Ye Merry Gentlemen||7524|
|13||O Little Town of Bethlehem||7480|
|14||Santa Claus Is Coming To Town||6851|
|15||I’ll Be Home for Christmas||6844|
|16||O Come All Ye Faithful||6273|
|17||Deck The Halls||6057|
|20||What Child Is This?||5755|
|21||We Wish You A Merry Christmas||5619|
|22||It Came Upon A Midnight Clear||5019|
|25||Let It Snow! Let It Snow! Let It Snow!||4598|
Of course this data may be confounded by the uber-tracks like White Christmas that have thousands of versions by a single artist, so lets look at the most recorded songs by unique artists – that is, we only count Bing Crosby once for White Christmas instead of 2,196 times. When we do that the top 25 changes a bit:
Most recorded Christmas songs (Unique Artists)
|3||Joy to the World||3593|
|5||O Holy Night||3536|
|6||The First Noel||3181|
|7||What Child Is This?||3150|
|8||Away in a Manger||3140|
|9||God Rest Ye Merry Gentlemen||2871|
|10||Have Yourself a Merry Little Christmas||2823|
|11||O Come All Ye Faithful||2675|
|12||Hark The Herald Angels Sing||2638|
|13||Angels We Have Heard on High||2494|
|15||The Christmas Song||2398|
|16||We Wish You A Merry Christmas||2281|
|17||Deck The Halls||2274|
|18||O Little Town of Bethlehem||2197|
|19||We Three Kings||2048|
|20||Santa Claus Is Coming To Town||1837|
|21||It Came Upon A Midnight Clear||1768|
|23||Auld Lang Syne||1603|
|25||I’ll Be Home for Christmas||1577|
The songs in green are the songs that are unique to each list.
Artists with the most number of unique songs
Bing Crosby is at the top of the Most Christmasy artists mainly because of the widespread re-issuing of White Christmas. But if we look at unique songs (i.e. White Christmas only counts once for Bing Crosby), the top Christmas artists look very different – with classical composers, Karaoke ‘artists’ and music factories topping the charts:
Artists with the most number of unique songs
|1||Johann Sebastian Bach||3681|
|3||The Karaoke Channel||1098|
|4||George Frideric Handel||903|
|7||ProSound Karaoke Band||762|
|8||Pyotr Ilyich Tchaikovsky||691|
|9||SBI Audio Karaoke||641|
|10||Mega Tracks Karaoke Band||577|
|12||Ameritz Karaoke Entertainment||508|
|13||Tbilisi Symphony Orchestra||506|
|16||Karaoke – Ameritz||428|
|17||Nat King Cole||413|
|18||Ameritz Karaoke Band||397|
|19||Merry Tune Makers||385|
Current popular Christmas crooner Michael Bublé, with 31 unique Christmas songs has a way to go before he makes it on to the most-unique-songs-recorded chart.
Speaking of Karaoke – there’s lots of Christmas Karaoke – 23,472 tracks to be precise. The top 25 Karaoke songs are the classics:
Top Karaoke Christmas Songs
|9||The Christmas Song||185|
|10||Jingle Bell Rock||172|
|11||Have Yourself a Merry Little Christmas||171|
|12||Please Come Home for Christmas||163|
|13||Little Drummer Boy||163|
|15||O Come All Ye Faithful||154|
|16||Here Comes Santa Claus||150|
|18||All I Want for Christmas Is You||146|
|19||O Holy Night||144|
|20||I Saw Mommy Kissing Santa Claus||143|
|21||Rockin’ Around the Christmas Tree||135|
|22||Santa Claus Is Coming to Town||126|
|23||Frosty the Snowman||125|
|24||Rudolph the Red Nosed Reindeer||121|
|25||We Wish You a Merry Christmas||118|
We can build a good list of seasonal terms by finding the most frequently occurring words in song titles. Here are the top 75 or so, as a word cloud created by wordle (stop words are removed of course).
Longest Christmas song name
There are lots of very long song names in the set of Christmas songs – the longest is this Christmas medly.
Andrea und Manuela – Morgen kommt der Weohnachtsmann – Medley / Morgen kommt der Weihnachtsmann/Leise rieselt der Schnee/Oh du Fröhliche/Ihr Kinderlein kommet/Süßer die Glocken nie klingen/Oh Tannenbaum/Kling Glöckchen/Stille Nacht, heilige Nacht/Alle Jahre wieder – Morgen kommt der Weihnachtsmann/Leise rieselt der Schnee/Oh du Fröhliche/Ihr Kinderlein kommet/Süßer die Glocken nie klingen/Oh Tannenbaum/Kling Glöckchen/Stille Nacht, heilige Nacht/Alle Jahre wieder
A great song for testing how well your music player UI deals with unusual titles.
One would think that with a million Christmas tracks we’d already have more than enough Christmas music – but, it seems, we still like new Christmas music. Ariana Grande’s recently released Santa Tell Me is climbing the streaming charts (currently #44 at charts.spotify.com).
Plus, there’s seemingly no-end to the variety of Christmas Music. If White Christmas with Bing Crosby is not your style, then there’s Blue Christmas by Elvis.
And If that’s not your thing, maybe you’ll enjoy Red Christmas by Insane Clown Posse.
‘Tis the Season
‘Tis the season for artists to release Christmas music … and they release lots of it. In the last two weeks Spotify has added thousands of releases with ‘Christmas’ in the title. I though it would be fun to build a little web app that lets you explore through all the releases. Here it is: ‘Tis the Season.
It shows you all the Christmas albums that have been released in the last few weeks, lets you listen to them and lets you open them in Spotify.
It makes use of the Spotify Web API – there’s a nifty search feature that lets you restrict album searches to albums that have just been recently release. That’s what makes this app possible. Check out the app at ‘Tis the Season. The source is on github.
The Ultimate Thanksgiving Playlist
On the annual drive to Thanksgiving dinner I’ve tortured my family with Alice’s Restaurant too many times over the years. Arlo Guthrie’s classic is still, in my mind, the classic Thanksgiving song, but there has to be more. So this year, I set out to expand my repertoire of Thanksgiving music – to build the ultimate Thanksgiving playlist. To do so, I looked through the top 300 or so most listened to Thanksgiving playlists on Spotify and found the top 100 songs that most frequently appear in all of these playlists, after discounting for popularity. Here are the results: The Ultimate Thanksgiving Playlist:[spotify spotify:user:plamere:playlist:7EOwcqcYhCDsrvoutSVP9E]
This is six hours of Thanksgiving music. All the classics are there, from Alice’s Restaurant to We are going to be Friends by the White Stripes. It should get you through the Thanksgiving drive, the meal, dessert and maybe even an after dinner snack.
However, if you want to synchronize your cooking and your music listening, there’s no better way then to hop on over to Time For Turkey for your basting+music needs.
And since the Christmas season starts immediately after the last piece of pumpkin pie has been consumed, lets not waste time breaking out the Christmas playlist. Here are the top 100 songs appearing across the most popular 1,000 Christmas playlists: Top Christmas Songs[spotify spotify:user:plamere:playlist:26fUkb0ZQdg62IY497OdNL]
How Students Listen
The Spotify Insights team took a deep dive into some of the listening data of college students to see if there were any differences in how students at different schools listen. We looked at a wide range of data including what artists were played, what songs were played and when, what playlists played, what genres were played and so on. We focused mostly on looking for distinctive listening patterns and behaviors at the different schools. The results were a set of infographic style visualizations that summarize the distinctive listening patterns for each school.
It was a fun study to do and really shows how much we learn about listening behavior based upon music streaming behavior. Read about the study on the Spotify Insights Blog: Top 40 Musical Universities in America:How Students Listen
More on “Where’s the Drama?”
Posted by Paul in code, events, Music, music hack day, Spotify, The Echo Nest on September 8, 2014
My Music Hack Day Berlin hack was “Where’s the Drama?” – a web app that automatically identifies the most dramatic moment in any song and plays it for you. I’ve been having lots of fun playing with it … and even though (or perhaps because) I know how it works, I’m often surprised at how well it does at finding the most dramatic moments. Here are some examples:
- When will the Bass Drop – Lonely Island
- Stairway to Heaven – Led Zeppelin
- Doomsday – Nero
- November Rain – Guns N Roses
How does it work? The app grabs the detailed audio analysis for the song from The Echo Nest. This includes a detailed loudness map of the song. This is the data I use to find the drama. To do so, I look for the part of the song with the largest rise in volume over the course of a 30 second window (longer songs can have a bit of a longer dramatic window). I give extra weight to crescendos that culminate in louder peaks (so if there are two crescendos that are 20dB in range but one ends at 5dB louder, it will win). Once I identify the most dynamic part of a song, I pad it a bit (so we get to hear a bit of the drop after the build).
Playing the music – I wanted to use Spotify to play the music, which was a bit problematic since there currently isn’t a way to play full streams with the Spotify Web API, so I did a couple of hacky hacks that got me pretty far. First of all, I discovered that you can add a time offset to a Spotify URI like so:
When this URI is opened in Spotify (even when opened via a browser), Spotify will start to play the song a the 1:05 time offset.
I still needed to be able to stop playing the track – and there’s no way to do that directly – so instead, I just open the URI:
which happens to be the URI for John Cage’s 4’33. In other words, to stop playing one track, I just start playing another (that happens to be silent). The awesome side effect of this is that I’ll be slowly turning anyone who uses “Where’s the Drama?” into experimental music listeners as the Spotify recommendation system responds to all of those John Cage ‘plays’. This should win some sort of ‘hackiest hack of the year’ award.
It was a fun hack to make, and great fun to demo. And now that I have the app, I am no longer wasting time listening to song intros and outros, I can just get to the bit of the song that matters the most.