Archive for category Music

The Playlist Miner

The Spotify Web API team pushed out a new feature recently that extends the search API to include playlist search. With this new feature it is now possible to search across all of the popular public playlists created by Spotify users. To try out the new search capability I created a new web app called The Playlist Miner.

The Playlist Miner is a web app that will create a Spotify playlist for you by finding the top songs in all of the playlists that match your criteria. Say, for example, that I want to create a dinner party playlist.  First, I find the top playlists that match ‘dinner party’ with The Playlist Miner:

The_Playlist_Miner

 

The Playlist Miner will find up to the top 1,000 most popular playlists that match dinner party. It shows them to me, giving me a chance to refine my query to focus in on the exact type of playlist that I am interested in.

 

The_Playlist_Miner

 

For this first try, I see lots of Christmas-oriented playlists (‘Tis the Season after all), but since I’m looking for music for a post-holiday dinner party, I’d rather not have holiday music in the playlist. So I refine my query to find non-Christmas oriented dinner party playlists like so:

 

The_Playlist_Miner

 

The resulting playlists are suitably non-Christmasy.

 

The_Playlist_Miner

I like the look of these playlists so I hit the Find Top Tracks button and The Playlist Miner will scour through all of the matching playlists (290 of them in this case) and find the most frequently appearing tracks.

The_Playlist_Miner

Once the top 100 tracks are found, I can save them to Spotify as my own playlist.

Selecting Prefer more distinctive labor and delivery tracks adjusts the track order for popularity so that tracks that are more distinctive to the particular playlist context will rise to the top. You can also use logical operators to focus in on the exact type of playlist you want to. You can search for “work out” OR workout NOT running  to find workout playlists without running in their titles/descriptions.

Under the hood – The Playlist Miner uses lots of bits of the Spotify API – user authentication, playlist search, playlist reading, playlist saving and more. The app is a an API calling beast – aggregating all the tracks from a thousand playlists requires 1,000 API calls. It’s a testament to the Spotify Web API that it doesn’t even blink under the load.  You can play with the code on github.

It’s fun to use The Playlist Miner to explore the quirkier aspects of how people listen to music. There are ironing playlists and sleeping baby playlists. There are playlists for getting psyched and playlists for Labor and Delivery.  With the Playlist Miner you can pull from all the playlists created for a particular purpose and build your own.  Give it a try.

, ,

Leave a comment

The Million Songs of Christmas

No other holiday dominates our listening like Christmas. During this season, we are exposed to a seemingly never ending playlist of Christmas music. So its no surprise that there’s a huge amount of Christmas music available on Spotify.  How much? Let’s take a look.

How much Christmas music is there?
It is actually quite hard to pinpoint the exact number of Christmas songs. First, every week during the holiday season thousands more Christmas songs are added to the set.  Second, some songs are seasonal – is Frosty The Snowman a Christmas song? Not literally, but it gets a lot of play at this time of year, even by the antipodes. Finally, there are a number of other holidays and celebrations at this time of year such as Hanukkah, Boxing DayNew Years, Kwanzaa, the Winter Solstice, and Festivus that we want to include in this category.  So when I say “Christmas Music” I’m referring to western music that is played primarily during December. There’s probably a better term to describe this music, but terms like seasonal, and holiday have their own special baggage – perhaps something like music coincident with the northern hemispheric winter solstice is the most precise description, but lets stick with Christmas music just to keep things simple. So how much Christmas music is there?  In early December 2014, crack music + data nerd Aaron Daubman  dove into the Spotify + Echo Nest music catalog and found 914,047 Christmas tracks – that’s just under a million Christmas tracks. Let’s unwrap this dataset to see what we can find.

First, some basic stats: Those 914,047 tracks represent 180,660 unique songs and were created by 63,711 unique artists – from Aaron Neville to Zuma the King. The top 20 artists with the most Christmas tracks in the Spotify catalog are all pre-Beatles artists:

Artists with the most Christmas Tracks

# Name Count
1 Bing Crosby 22382
2 Frank Sinatra 17979
3 Elvis Presley 12381
4 Nat King Cole 11613
5 Johann Sebastian Bach 8958
6 Dean Martin 8000
7 Perry Como 7529
8 Ella Fitzgerald 6428
9 Mahalia Jackson 5883
10 Mario Lanza 5377
11 Johnny Mathis 5036
12 Rosemary Clooney 4538
13 Peggy Lee 4450
14 Harry Belafonte 4054
15 The Andrews Sisters 3567
16 Louis Armstrong 3481
17 Gene Autry 3411
18 Doris Day 2985
19 Pat Boone 2767
20 Connie Francis 2500

Yes, that’s right, Bing Crosby has 22,382 different Christmas tracks (!) in the Spotify catalog. Now, a little digression on what we consider to be a unique track.  Music, especially popular music, is released in many forms. A very popular song, such as Bing Crosby’s White Christmas, may appear on a wide range of albums – from the original studio release to a plethora of Christmas Compilations and artist ‘best of’ albums. Each of these track releases may have different album art, different rights holders and regional licenses. Thus, even though the audio for White Christmas may be the same on each of the release, we consider each release as a different track.

White Christmas
Let’s take a closer look at Bing Crosby’s White Christmas. In our catalog of nearly a million Christmas tracks, 2,196 of them are Bing Crosby’s classic. I’ll say that again, just because it is a rather phenomenal fact – there are 2,196 different albums on Spotify that contain Bing’s White Christmas. It is hard to believe, so I created a web page that contains all 2,196 of the albums so you can see them all.  Click on the image below to load them all up (warning – with 2000+ album covers it’s a bit of a browser buster).

static_echonest_com_insights_christmas_whitechristmas_html 

White Christmas isn’t the only uber-track of the holidays. Here are the top 25 Christmas tracks based upon the number of times they have been released on an album:

The most released Christmas tracks

# Name Count
1 Bing Crosby – White Christmas 2196
2 Eartha Kitt – Santa Baby 1286
3 Elvis Presley – Blue Christmas 1285
4 Frank Sinatra – Jingle Bells 1121
5 Harry Belafonte – Mary’s Boy Child 904
6 Bing Crosby – Silver Bells 881
7 Nat King Cole – The Christmas Song 870
8 Frank Sinatra – The Christmas Waltz 811
9 Rosemary Clooney – Suzy Snowflake 788
10 Bobby Helms – Jingle Bell Rock 779
11 Elvis Presley – White Christmas 738
12 Judy Garland – Have Yourself a Merry Little Christmas 735
13 Frank Sinatra – White Christmas 703
14 Frank Sinatra – Christmas Dreaming 696
15 Frank Sinatra – Have Yourself a Merry Little Christmas 695
16 Elvis Presley – Silent Night 688
17 Elvis Presley – I Believe 664
18 Frank Sinatra – Santa Claus Is Coming to Town 660
19 Louis Armstrong – Zat You Santa Claus 598
20 Dean Martin – The Christmas Blues 575
21 Frank Sinatra – Mistletoe and Holly 568
22 Louis Armstrong – Cool Yule 566
23 Frank Sinatra – Silent Night 563
24 Bing Crosby – Jingle Bells 560
25 Elvis Presley – Santa Claus Is Back in Town 559

You can see all of the releases for Elvis’s Blue Christmas and Eartha Kitt’s Santa Baby  here:

static_echonest_com_insights_christmas_BlueChristmas_html

static_echonest_com_insights_christmas_SantaBaby_html

So there are lots of copies of Bing Crosby’s White Christmas and Eartha Kitt’s Santa Baby out there – but what are the most common Christmas songs overall? Which ones have been recorded the most by any artist?  The following table shows the top 25:

Most recorded songs 

# Name Recordings
1 Silent Night 19041
2 White Christmas 15928
3 Jingle Bells 14521
4 Winter Wonderland 9524
5 Joy to the World 9093
6 The First Noel 8731
7 Have Yourself a Merry Little Christmas 8511
8 O Holy Night 7925
9 Hark The Herald Angels Sing 7727
10 The Christmas Song 7673
11 Away in a Manger 7544
12 God Rest Ye Merry Gentlemen 7524
13 O Little Town of Bethlehem 7480
14 Santa Claus Is Coming To Town 6851
15 I’ll Be Home for Christmas 6844
16 O Come All Ye Faithful 6273
17 Deck The Halls 6057
18 Silver Bells 6044
19 Ave Maria 5847
20 What Child Is This? 5755
21 We Wish You A Merry Christmas 5619
22 It Came Upon A Midnight Clear 5019
23 Sleigh Ride 5004
24 Blue Christmas 4688
25 Let It Snow! Let It Snow! Let It Snow! 4598

Of course this data may be confounded by the uber-tracks like White Christmas that have thousands of versions by a single artist, so lets look at the most recorded songs by unique artists – that is, we only count Bing Crosby once for White Christmas instead of 2,196 times. When we do that the top 25 changes a bit:

Most recorded Christmas songs (Unique Artists)

# Name Recordings
1 Silent Night 7406
2 Jingle Bells 4485
3 Joy to the World 3593
4 White Christmas 3592
5 O Holy Night 3536
6 The First Noel 3181
7 What Child Is This? 3150
8 Away in a Manger 3140
9 God Rest Ye Merry Gentlemen 2871
10 Have Yourself a Merry Little Christmas 2823
11 O Come All Ye Faithful 2675
12 Hark The Herald Angels Sing 2638
13 Angels We Have Heard on High 2494
14 Winter Wonderland 2489
15 The Christmas Song 2398
16 We Wish You A Merry Christmas 2281
17 Deck The Halls 2274
18 O Little Town of Bethlehem 2197
19 We Three Kings 2048
20 Santa Claus Is Coming To Town 1837
21 It Came Upon A Midnight Clear 1768
22 Ave Maria 1705
23 Auld Lang Syne 1603
24 Silver Bells 1599
25 I’ll Be Home for Christmas 1577

The songs in green are the songs that are unique to each list.

Artists with the most number of unique songs
Bing Crosby is at the top of the Most Christmasy artists mainly because of the widespread re-issuing of White Christmas. But if we look at unique songs (i.e. White Christmas only counts once for Bing Crosby), the top Christmas artists look very different – with classical composers, Karaoke ‘artists’ and music factories topping the charts:

Artists with the most number of unique songs

1 Johann Sebastian Bach 3681
2 Bing Crosby 1462
3 The Karaoke Channel 1098
4 George Frideric Handel 903
5 A-Type Player 835
6 Frank Sinatra 816
7 ProSound Karaoke Band 762
8 Pyotr Ilyich Tchaikovsky 691
9 SBI Audio Karaoke 641
10 Mega Tracks Karaoke Band 577
11 ProSource Karaoke 539
12 Ameritz Karaoke Entertainment 508
13 Tbilisi Symphony Orchestra 506
14 Elvis Presley 472
15 Perry Como 440
16 Karaoke – Ameritz 428
17 Nat King Cole 413
18 Ameritz Karaoke Band 397
19 Merry Tune Makers 385
20 Christmas Songs 370

Current popular Christmas crooner Michael Bublé, with 31 unique Christmas songs has a way to go before he makes it on to the most-unique-songs-recorded chart.

Speaking of Karaoke – there’s lots of Christmas Karaoke – 23,472 tracks to be precise.  The top 25 Karaoke songs are the classics:

Top Karaoke Christmas Songs

# Name Count
1 White Christmas 345
2 Winter Wonderland 333
3 Silent Night 312
4 Jingle Bells 309
5 Last Christmas 258
6 Silver Bells 219
7 Blue Christmas 204
8 Santa Baby 189
9 The Christmas Song 185
10 Jingle Bell Rock 172
11 Have Yourself a Merry Little Christmas 171
12 Please Come Home for Christmas 163
13 Little Drummer Boy 163
14 Sleigh Ride 156
15 O Come All Ye Faithful 154
16 Here Comes Santa Claus 150
17 Feliz Navidad 146
18 All I Want for Christmas Is You 146
19 O Holy Night 144
20 I Saw Mommy Kissing Santa Claus 143
21 Rockin’ Around the Christmas Tree 135
22 Santa Claus Is Coming to Town 126
23 Frosty the Snowman 125
24 Rudolph the Red Nosed Reindeer 121
25 We Wish You a Merry Christmas 118

Top Terms

We can build a good list of seasonal terms by finding the most frequently occurring words in song titles. Here are the top 75 or so, as a word cloud created by wordle (stop words are removed of course).

Banners_and_Alerts_and_Wordle_Applet

Longest Christmas song name
There are lots of very long song names in the set of Christmas songs – the longest is this Christmas medly.

Andrea und Manuela – Morgen kommt der Weohnachtsmann – Medley / Morgen kommt der Weihnachtsmann/Leise rieselt der Schnee/Oh du Fröhliche/Ihr Kinderlein kommet/Süßer die Glocken nie klingen/Oh Tannenbaum/Kling Glöckchen/Stille Nacht, heilige Nacht/Alle Jahre wieder – Morgen kommt der Weihnachtsmann/Leise rieselt der Schnee/Oh du Fröhliche/Ihr Kinderlein kommet/Süßer die Glocken nie klingen/Oh Tannenbaum/Kling Glöckchen/Stille Nacht, heilige Nacht/Alle Jahre wieder

A great song for testing how well your music player UI deals with unusual titles.

Conclusion

One would think that with a million Christmas tracks we’d already have more than enough Christmas music – but, it seems, we still like new Christmas music. Ariana Grande’s recently released Santa Tell Me is climbing the streaming charts (currently #44 at charts.spotify.com).

Plus, there’s seemingly no-end to the variety of Christmas Music. If White Christmas with Bing Crosby is not your style, then there’s Blue Christmas by Elvis.

And If that’s not your thing, maybe you’ll enjoy Red Christmas by Insane Clown Posse.

Leave a comment

‘Tis the Season

‘Tis the season for artists to release Christmas music … and they release lots of it. In the last two weeks Spotify has added thousands of releases with ‘Christmas’ in the title.   I though it would be fun to build a little web app that lets you explore through all the releases.  Here it is: ‘Tis the Season.

_Tis_the_season

It shows you all the Christmas albums that have been released in the last few weeks, lets you listen to them and lets you open them in Spotify.

It makes use of the Spotify Web API – there’s a nifty search feature that lets you restrict album searches to albums that have just been recently release. That’s what makes this app possible.  Check out the app at ‘Tis the Season. The source is on github.

Leave a comment

The Mobile Infinite Jukebox Survey results

Over the last six months or so The Infinite Jukebox had a link to a survey about features peopled would like to see in a mobile version of The Infinite Jukebox.  Since then, over 10,000 people have taken the survey.   Here are the results.

The survey was linked to directly from the Infinite Jukebox. The questions asked were:

Interested_in_a_mobile_Infinite_Jukebox_

 

Since the text in link to the survey was “Interested in a mobile version of the Infinite Jukebox? Then take this one minute survey” it is no surprise that 99% of all respondents are interested in a mobile version of the app.

Interested_in_a_mobile_Infinite_Jukebox__-_Google_Forms

 

 

The split between Android and iOS aligns with other iOS vs Android metrics out there on the webs.

Interested_in_a_mobile_Infinite_Jukebox__-_Google_Forms

As for how much people would be willing to pay, 64% would be willing to pay something for the app.

Interested_in_a_mobile_Infinite_Jukebox__-_Google_Forms

This was a bit surprising – 70% of folks want to play music from their own collection, and only 11% are interested in playing music from a streaming service like Spotify or Rdio.

Interested_in_a_mobile_Infinite_Jukebox__-_Google_Forms

The final question was an open-ended question asking about what other features would you like to see in the Infinite Jukebox. Many of the responses were about what features would like to see in the current web version, while many were about what features should be in a mobile version. Some of the more common results are here:

Common new feature suggestions 

  • Background playing
  • Offline playling
  • No Ads
  • Simple tuning options
  • Playlist support
  • Choose song length
  • Time limits per song
  • Infinitise multiple songs
  • Color schemes
  • Volume controls
  • Social features (voting on best tunings)

So, you may be wondering where is the mobile version of the Infinite Jukebox?  It is coming along, all the hard coding bits are done, but it has been very much a spare time project.   I do hope to release it sometime in the near future. Here’s a short clip of the app in action:

Thanks to everyone who took the survey, its been quite informative.

Leave a comment

Sort Your Music

A few weeks ago, the Spotify Web API team pushed out some updates to the API that allows developers to update the tracks in a listener’s playlist. With these changes a developer can add, replace, remove and rearrange tracks in a playlist on behalf of a listener. This week I wanted to try out these new API features so I built an app called Sort Your Music.

Sort_Your_Music

Sort Your Music lets you sort the tracks in any of your playlists based on a number of Echo Nest parameters.  You can sort a playlist by BPM, Energy, Danceability, Loudness, Acousticness, Valence and more.  Once you’ve sorted a playlist you can save it back to Spotify, letting you listen to it on any of your music devices.  For example, here’s a copy of the Spotify Top 50 playlist, where the tracks have been sorted from highest energy to lowest.

Using Sort Your Music is quite simple, login with your Spotify credentials and give the app permission to modify your playlists. Then, select the playlist you want to work on and, after a few seconds (while all of the song data is retrieved from The Echo Nest), your playlist will appear in table form like so:

Sort_Your_Music

To sort the playlist, just click on the column headings for any of the parameters. When you are happy with the changes, just click save, and your playlist will be updated.

Under the Hood
This is a pretty straightforward app, but there were a few challenging bits. The primary challenge was dealing with the large number of calls to The Echo Nest. Each song in a playlist requires a call to The Echo Nest to fetch the song attributes, so even a modest playlist of 40 songs results in 40  Echo Nest calls.  Multiply that by a few dozen active users and the app will be overwhelming my Echo Nest API rate limit. To avoid this, I created my own caching server that sits between the web app and The Echo Nest. It fields bulk requests from the web app (all the IDs at once), and retrieves the song data from The Echo Nest, eliminating any unneeded data, and passing it back to the web app.  The big performance win comes from keeping a cache of the song info. After a bit of usage, most popular songs will be in the cache making most playlist song resolving quite snappy.  Still, if you have a long and obscure playlist it may take 10 seconds to resolve.

Having a caching server gives me a few other benefits – I have a central point to handle rate limit throttling – if the app gets busy and we start hitting the rate limit, the server can do the throttling automatically, and I can take action. Another big advantage is that I don’t have to expose my Echo Nest API key to the world  like I would need to do if I made Echo Nest calls directly from the web client.

My caching server has an info endpoint that returns some json data about the server status, including the average time to process each request to resolve a playlist.  The current average resolve time is about 700ms – not too bad.

A Safari glitch – when I tested the finished app on Safari, I found that authentication didn’t work.  This was quite puzzling, as it had worked for me before. I went back to some of my older Spotify apps that perform authentication and it turns out that they were no longer working as well.  What changed? Well, I’m running the spiffy new Yosemite with an update to Safari. Digging deeper it turned out that the new Safari doesn’t like redirect URLs without a trailing slash.  Once I added a trailing slash to the redirect URL all was well.

The code for the app is at github if you are interested in seeing how it all works. And be sure to give it the app a try. It is time to get your music in the right order.

6 Comments

Top US Artists on a map

This week I took yet another crack at putting artists on a map. This time, I created a map that shows the top artists for every state in the US for every day of 2014 up until today.  You can go through the calendar year, day-by-day to see which artists managed to capture the hearts and ears of different parts of the country. It is fascinating to see if there is a Super Bowl bump for the half-time artist, or if there is any change in how people listen to music on St. Patrick’s day, or to see which artists are regional and which are national.

Top_US_Artists_2014

You can read more about the app on the insights.spotify.com blog and then take the app for a test drive.   Hit the ‘GO!’ button to animate through the days of the year or hit the arrow keys to step through the calendar day by day. Click on a state to hear the most popular song by the most popular artist in that state on that day of the year.

It was a super fun app to write. I shall certainly write a bit more about some of the tech involved in the next couple of days.

1 Comment

More on “Where’s the Drama?”

My Music Hack Day Berlin hack was “Where’s the Drama?” – a web app that automatically identifies the most dramatic moment in any song and plays it for you. I’ve been having lots of fun playing with it … and even though (or perhaps because) I know how it works, I’m often surprised at how well it does at finding the most dramatic moments.  Here are some examples:

How does it work? The app grabs the detailed audio analysis for the song from The Echo Nest.  This includes a detailed loudness map of the song. This is the data I use to find the drama.  To do so, I look for the part of the song with the largest rise in volume over the course of a 30 second window (longer songs can have a bit of a longer dramatic window). I give extra weight to crescendos that culminate in louder peaks (so if there are two crescendos that are 20dB in range but one ends at 5dB louder, it will win). Once I identify the most dynamic part of a song, I pad it a bit (so we get to hear a bit of the drop after the build).

The rest is just UI – the song gets plotted as a heavily filtered loudness curve with the dramatic passage highlighted. I plot things with Highcharts, which is a pretty nifty javascript plotting and charting library. I recommend.

 Where_is_the_Drama_

Playing the music – I wanted to use Spotify to play the music, which was a bit problematic since there currently isn’t a way to play full streams with the Spotify Web API, so I did a couple of hacky hacks that got me pretty far. First of all, I discovered that you can add a time offset to a Spotify URI like so:

        spotify:track:2SHnUyZq0zwmvRIl4WY77G#1:05

When this URI is opened in Spotify (even when opened via a browser), Spotify will start to play the song a the 1:05 time offset.  

I still needed to be able to stop playing the track – and there’s no way to do that directly – so instead, I just open the URI:

      spotify:track:3nKrmorxtDSxnqvMgJudmV

which happens to be the URI for John Cage’s 4’33.  In other words, to stop playing one track, I just start playing another (that happens to be silent).  The awesome side effect of this is that I’ll be slowly turning anyone who uses “Where’s the Drama?” into experimental music listeners as the Spotify recommendation system responds to all of those John Cage ‘plays’. This should win some sort of ‘hackiest hack of the year’ award.

It was a fun hack to make, and great fun to demo. And now that I have the app, I am no longer wasting time listening to song intros and outros, I can just get to the bit of the song that matters the most. 

, , , , , ,

3 Comments

Follow

Get every new post delivered to your Inbox.

Join 1,313 other followers