A few weeks ago, the Spotify Web API team pushed out some updates to the API that allows developers to update the tracks in a listener’s playlist. With these changes a developer can add, replace, remove and rearrange tracks in a playlist on behalf of a listener. This week I wanted to try out these new API features so I built an app called Sort Your Music.
Sort Your Music lets you sort the tracks in any of your playlists based on a number of Echo Nest parameters. You can sort a playlist by BPM, Energy, Danceability, Loudness, Acousticness, Valence and more. Once you’ve sorted a playlist you can save it back to Spotify, letting you listen to it on any of your music devices. For example, here’s a copy of the Spotify Top 50 playlist, where the tracks have been sorted from highest energy to lowest.
Using Sort Your Music is quite simple, login with your Spotify credentials and give the app permission to modify your playlists. Then, select the playlist you want to work on and, after a few seconds (while all of the song data is retrieved from The Echo Nest), your playlist will appear in table form like so:
To sort the playlist, just click on the column headings for any of the parameters. When you are happy with the changes, just click save, and your playlist will be updated.
Under the Hood
This is a pretty straightforward app, but there were a few challenging bits. The primary challenge was dealing with the large number of calls to The Echo Nest. Each song in a playlist requires a call to The Echo Nest to fetch the song attributes, so even a modest playlist of 40 songs results in 40 Echo Nest calls. Multiply that by a few dozen active users and the app will be overwhelming my Echo Nest API rate limit. To avoid this, I created my own caching server that sits between the web app and The Echo Nest. It fields bulk requests from the web app (all the IDs at once), and retrieves the song data from The Echo Nest, eliminating any unneeded data, and passing it back to the web app. The big performance win comes from keeping a cache of the song info. After a bit of usage, most popular songs will be in the cache making most playlist song resolving quite snappy. Still, if you have a long and obscure playlist it may take 10 seconds to resolve.
Having a caching server gives me a few other benefits – I have a central point to handle rate limit throttling – if the app gets busy and we start hitting the rate limit, the server can do the throttling automatically, and I can take action. Another big advantage is that I don’t have to expose my Echo Nest API key to the world like I would need to do if I made Echo Nest calls directly from the web client.
My caching server has an info endpoint that returns some json data about the server status, including the average time to process each request to resolve a playlist. The current average resolve time is about 700ms – not too bad.
A Safari glitch – when I tested the finished app on Safari, I found that authentication didn’t work. This was quite puzzling, as it had worked for me before. I went back to some of my older Spotify apps that perform authentication and it turns out that they were no longer working as well. What changed? Well, I’m running the spiffy new Yosemite with an update to Safari. Digging deeper it turned out that the new Safari doesn’t like redirect URLs without a trailing slash. Once I added a trailing slash to the redirect URL all was well.
This week I took yet another crack at putting artists on a map. This time, I created a map that shows the top artists for every state in the US for every day of 2014 up until today. You can go through the calendar year, day-by-day to see which artists managed to capture the hearts and ears of different parts of the country. It is fascinating to see if there is a Super Bowl bump for the half-time artist, or if there is any change in how people listen to music on St. Patrick’s day, or to see which artists are regional and which are national.
You can read more about the app on the insights.spotify.com blog and then take the app for a test drive. Hit the ‘GO!’ button to animate through the days of the year or hit the arrow keys to step through the calendar day by day. Click on a state to hear the most popular song by the most popular artist in that state on that day of the year.
It was a super fun app to write. I shall certainly write a bit more about some of the tech involved in the next couple of days.
The Spotify Insights team took a deep dive into some of the listening data of college students to see if there were any differences in how students at different schools listen. We looked at a wide range of data including what artists were played, what songs were played and when, what playlists played, what genres were played and so on. We focused mostly on looking for distinctive listening patterns and behaviors at the different schools. The results were a set of infographic style visualizations that summarize the distinctive listening patterns for each school.
It was a fun study to do and really shows how much we learn about listening behavior based upon music streaming behavior. Read about the study on the Spotify Insights Blog: Top 40 Musical Universities in America:How Students Listen
My Music Hack Day Berlin hack was “Where’s the Drama?” – a web app that automatically identifies the most dramatic moment in any song and plays it for you. I’ve been having lots of fun playing with it … and even though (or perhaps because) I know how it works, I’m often surprised at how well it does at finding the most dramatic moments. Here are some examples:
- When will the Bass Drop – Lonely Island
- Stairway to Heaven – Led Zeppelin
- Doomsday – Nero
- November Rain - Guns N Roses
How does it work? The app grabs the detailed audio analysis for the song from The Echo Nest. This includes a detailed loudness map of the song. This is the data I use to find the drama. To do so, I look for the part of the song with the largest rise in volume over the course of a 30 second window (longer songs can have a bit of a longer dramatic window). I give extra weight to crescendos that culminate in louder peaks (so if there are two crescendos that are 20dB in range but one ends at 5dB louder, it will win). Once I identify the most dynamic part of a song, I pad it a bit (so we get to hear a bit of the drop after the build).
Playing the music – I wanted to use Spotify to play the music, which was a bit problematic since there currently isn’t a way to play full streams with the Spotify Web API, so I did a couple of hacky hacks that got me pretty far. First of all, I discovered that you can add a time offset to a Spotify URI like so:
When this URI is opened in Spotify (even when opened via a browser), Spotify will start to play the song a the 1:05 time offset.
I still needed to be able to stop playing the track – and there’s no way to do that directly – so instead, I just open the URI:
which happens to be the URI for John Cage’s 4’33. In other words, to stop playing one track, I just start playing another (that happens to be silent). The awesome side effect of this is that I’ll be slowly turning anyone who uses “Where’s the Drama?” into experimental music listeners as the Spotify recommendation system responds to all of those John Cage ‘plays’. This should win some sort of ‘hackiest hack of the year’ award.
It was a fun hack to make, and great fun to demo. And now that I have the app, I am no longer wasting time listening to song intros and outros, I can just get to the bit of the song that matters the most.
I’m at Music Hack Day Berlin this week where I’m just finishing up my hack called Where’s the Drama?
Where’s The Drama? will automatically find the most dramatic bits in your favorite music, so you can skip right to the best part of the song. No need to listen to anything else. Here’s a video of the app in action:
So get out your cigarette lighter, stand up and start to listen more dramatically.
For my summer vacation early-morning coding for fun project I revamped my old Acrostic Playlist Maker to work with Spotify. The app, called Acrostify, will generate acrostic playlists with the first letter of each song in the playlist spelling out a secret message. With the app, you can create acrostic playlists and save them to Spotify.
Give it a try at Acrostify.
I am at Outside Hacks this weekend – A hackathon associated with the Outside Lands music festival. For this hack I thought it would be fun to try out the brand new Your Music Library endpoints in the Spotify Web API. These endpoints let you inspect and manipulate the tracks that a user has saved to their music. Since the hackathon is all about building apps for a music festival, it seems natural to create a web app that gives you festival artist recommendations based upon your Spotify saved tracks. The result is the Outside Lands Recommender:
The Recommender works by pulling in all the saved tracks from your Spotify ‘Your Music’ collection, aggregating the artists and then using the Echo Nest Artist Similar API to find festival artists that match or are similar to those artists. The Spotify API is then used to retrieve artist images and audio samples for the recommendations where they are presented in all of their bootstrap glory.
This was a pretty straight forward app, which was good since I only had about half the normal hacking time for a weekend hackathon. I spent the other half of the time building a festival dataset for hackers to use (as well as answering lots of questions about both the Spotify and Echo Nest APIs).
It has been a very fun hackathon. It is extremely well organized, the Weebly location is fantastic, and the quality of hackers is very high. I’ve already seen some fantastic looking hacks and we are still a few hours from demo time. Plus, this happened.