Paul

I'm the Director of Developer Community at The Echo Nest, a research-focused music intelligence startup that provides music information services to developers and partners through a data mining and machine listening platform. I am especially interested in hybrid music recommenders and using visualizations to aid music discovery.

Outside Lands Recommendations

I am at Outside Hacks this weekend – A hackathon associated with the Outside Lands music festival. For this hack I thought it would be fun to try out the brand new Your Music Library endpoints in the Spotify Web API. These endpoints let you inspect and manipulate the tracks that a user has saved to their music. Since the hackathon is all about building apps for a music festival, it seems natural to create a web app that gives you festival artist recommendations based upon your Spotify saved tracks. The result is the Outside Lands Recommender:

 

Your_Outside_Lands_Festival_Recommendations

 

The Recommender works by pulling in all the saved tracks from your Spotify ‘Your Music’ collection, aggregating the artists and then using the Echo Nest Artist Similar API to find festival artists that match or are similar to those artists. The Spotify API is then used to retrieve artist images and audio samples for the recommendations where they are presented in all of their bootstrap glory.

This was a pretty straight forward app, which was good since I only had about half the normal hacking time for a weekend hackathon. I spent the other half of the time building a festival dataset for hackers to use (as well as answering lots of questions about both the Spotify and Echo Nest APIs).

It has been a very fun hackathon. It is extremely well organized, the Weebly location is fantastic, and the quality of hackers is very high. I’ve already seen some fantastic looking hacks and we are still a few hours from demo time.  Plus, this happened.

Screenshot_7_27_14__12_32_PM

Two more victims

,

Leave a comment

Outside Lands lineup

I’m on my way to Outside Hacks - a hackathon tied in with the Outside Lands music festival.  Since many hacks at the hackathon will be related to the festival it is pretty important to have a machine-readable version of the artist lineup for the festival. However, I couldn’t find any online.  Since I had an hour in the airport lounge, and the airport actually has decent WiFi, I thought I would try to be a good hacker citizen and generate an easily parseable lineup.

A little python + some BeautifulSoup and a bit of Echo Nest Rosetta Data and I have created an Outside Lands lineup JSON that includes links to artist pages, plus Echo Nest, Spotify and Rdio IDs.  The JSON is hosted online at:

http://static.echonest.com/OutsideLands/lineup_2014.json

Here’s the code:

It’s about time to get on the plane. If you can think of other interesting data to add to the lineup json let me know and I’ll try to add it before the hackathon.

Leave a comment

Fetching my saved tracks from Spotify

The @SpotifyPlatform team just pushed out an update to the Spotify Web API that lets a developer retrieve and manipulate the tracks that a user has saved in ‘Your Music’. To show off this new functionality I wrote a quick demo that shows how to fetch the saved tracks for a Spotify user via the nifty new API.  The demo will first solicit permission from the user, and if the user grants such permissions, the app will then retrieved the saved tracks and show them as a simple list.

My Saved Tracks

 

The demo is on line here: My Saved Tracks and the source is on github.

,

Leave a comment

Set Listener update – now listen to specific shows in Spotify

Last month I released the Set Listener –  a web app that lets you create a Spotify playlist for an artist’s most recent show.   A frequent request by users has been to allow the creation of a Spotify playlist for any show, not just the most recent one. However, I didn’t want to have to implement an elaborate event search and browse feature (this was supposed to be a quick hack) especially since SetList.fm already implements this browsing. So instead I did a bit of hack to implement this feature – in addition to accepting an artist name, the Set Listener will accept the url of any setlist.fm show. If you enter a setlist.fm show URL, the Set Listener will grab the songs for that show and generate the playlist. Here’s the flow:

Go to setlist.fm and browse until you find your show of interest. Grab the URL

God_Is_an_Astronaut_Concert_Setlist_at_Melnā_Piektdiena__Riga_on_May_19__2014___setlist_fm

 

Paste the URL into the Set Listener and you should be good to go:

The_Set_Listener

,

Leave a comment

1st Web Audio Conference

This January IRCAM and Mozilla will be hosting the 1st Web Audio Conference in Paris France.  From the conference page:

The WAC is the first international conference on web audio technologies and applications.

The conference welcomes web R&D developers, audio processing scientists, application designers and people involved in web standards.

The conference addresses research, development, design, and standards concerned with emerging audio-related web technologies such as Web Audio API, Web RTC, Web Sockets, and Javascript.

Contributions to the first edition of WAC are encouraged in, but not limited to, the following topics:

  • Innovative audio and music based web applications (with social and user experience aspects)
  • Client-side audio processing (real-time or non real-time)
  • Audio data and metadata formats and network delivery
  • Server-side audio processing and client access
  • Client-side audio engine and rendering
  • Frameworks for audio manipulation
  • Web Audio API design and implementation
  • Client-side audio visualization
  • Multimedia integration
  • Web standards and use of standards within audio based web projects
  • Hardware, tangible interface and use of Web Audio API

This is the first Call For Papers that I’ve seen that, in addition to a paper and poster track, also has a Gig submission track.  Yes, you can propose a 20 minute gig. Accepted gigs will be presented in the conference concert.  I’m thinking of a combined Infinite Jukebox + Ellie Goulding visualization – 20 minutes of flying cubes will show ‘em.

The submission deadline is October 10.  Time to dust off my Latex skills.

 

 

Leave a comment

Most Distinctive Fourth of July Songs

Independence day is just a few days away so I spent a bit of time over the weekend digging into the data to see what are the most listened to songs on the Fourth of July. Eliot wrote a great piece about the data for the Spotify Blog: The Most Distinctive Fourth of July Songs in the 50 U.S. States. Here I dig in a bit deeper.

From a musical perspective, The Fourth of July is a very interesting holiday. It’s a big summer outdoor party  – and the music reflects that. On the Fourth, people listen to patriotic songs,  BBQ songs, popular songs, party songs, songs about place and history. People listen to wide range of genres, from rock and pop to folk, country, marches,  and new weird america.   To see what music people listen to on the Fourth I went through about 5,000 playlists that people have created that have ‘fourth of july’ in the title.  I aggregated the songs across all of these playlists and created a playlist of the top 100 or songs.

As you can see, the playlist is a mix of US-centric music and summer party music – which is a pretty good reflection of what people actually listen to on the Fourth of July.  But I wanted to go a bit further and see what music was particularly distinctive for the Fourth of July – as compared to any other summer outdoor party playlist.  To do this, I collected the top most frequently appearing songs on the Fourth of July playlists and scored each song by calculating the ratio  of the play counts that occurred on July 4, 2013 in the U.S. vs. play counts for the song during the following weeks.  Songs that were played much more frequently on the Fourth than during the rest of July get a much higher score.  Ranking songs by this ratio yields the list of distinctive Fourth of July Songs:

I think it is a pretty good list of the music that we listen to more on the Fourth of July than on any other day. There’s a John Philip Sousa march, Woody Guthrie’s This Land is Your Land, Katy Perry’s Firework, lots and lots of country music. Some artists appear more frequently than others – Bruce Springsteen, John Mellencamp and Lee Greenwood are Fourth of July favorites.

I’m always interested in regional differences in how we listen to music, so I looked at the listening in each state in the U.S. to see which of the core Fourth of July songs was listened to more.  If you look just at the most popular Fourth of July song in each state the results are pretty boring – the most popular song in 46 out of 50 states for July 2013 was Party in the USA by Miley Cyrus – but if you look at the distinctive score (the ratio of plays on the fourth to plays during the following weeks), you get a more nuanced view of how people listen to music. Of course we represent this as a map:

fourth-map

So the real question is, are these playlists generated by data mining any good?  As actual playlists to listen to, I don’t think so. They are too incoherent, with jumps that take you from a sappy country song, to a teen pop song, to a march that will give you iPod whiplash. Compare the playlists above to a human-curated Fourth of July Playlist:

This is a much more listenable playlist. However, I think the data mined playlists do provide an excellent starting point – a  big pool of Fourth of July songs.  With a genre and popularity filter these could be turned into decent, listenable playlists. Similarly, give this pool of Fourth of July songs to a team of music curators and they can build some pretty fantastic Fourth of July playlists for every type of music fan.

 

3 Comments

Fetching My Starred Items from Spotify

I wrote a quick demo that shows how to create a web app that fetches the starred items for a Spotify user via the nifty new Spotify Web API.  The demo will first solicit permission from the user, and if the user grants such permissions, the app will then retrieved the starred list and show the tracks in a simple list.

My_Starred_Items

The online demo is here: My Starred ItemsThe source is on github.

,

Leave a comment

The Set Listener

Going to a show? Not totally familiar with an artist’s catalog? Give The Set Listener a try.  The Set Listener is a web app that will create a Spotify playlist of an artist’s most recent show.

 

The_Set_Listener

 

To use The Set Listener just type in the artist name, and hit the search button, you’ll be presented with a playlist of songs from that artist’s most recent show.  Hit the ‘Save this playlist to Spotify’ button and you’ll have a Spotify playlist that you can listen to on your desktop or on your mobile phone.

The app relies on the SetList.fm API and the brand new and super spiffy Spotify Web API. Now that the Spotify Web API supports the creation and saving of playlists creating these types of apps is quite straightforward – just a few hours of coding. This was my first time using the SetList.fm API – its a super resource for setlists from concerts by thousands of artists.

Check out the app online at The Set Listener.  The code is online on github.

2 Comments

My new superpower – creating Spotify playlists from a web app

labs_echonest_com_CityServer_callback_html_access_token_BQBcbPg0FicOcTQ2Epc5XxGkMNuQPU4LA-ou9LP0lo7qx-4FNd7QlJNXtXziRF04gPtbATfmh9Xe25vJeUVrOQLOrhpQi4La3jT6dEUc7XHD_7iB9oStWBN9PuNGoWB_WKg8goz92CQpYHVM50z_wjY_token_type_Bearer_expires_in_360The new Spotify Web API allows the developer to create and add tracks to a playlist on behalf of a listener. This is a pretty powerful feature, opening the door for a whole range of apps. For instance, this weekend, I added the ability to save a Roadtrip Mixtape playlist, so you can now actually take your mixtapes on the road.   The code is on github if  you are interested in seeing how it is done.

 

, , ,

9 Comments

The Echo Nest + Spotify Sandbox

I am wearing my International Executive Music Hacker hat today. I’m writing this blog post at 5AM somewhere over the Atlantic Ocean, on my way to the Barcelona Music Hack Day, where I’ll be representing both The Echo Nest and Spotify. I’m pretty excited about the hack event – first, because it’s in freaking Barcelona, and second, because I get to talk about what’s been going on with the Spotify and Echo Nest APIs.

The_Echo_Nest___Spotify_Developer

It has been just about 100 days since The Echo Nest and Spotify have joined forces. In that time we’ve been working hard to build the best music platform for listeners and for developers. This week we are releasing some of the very first fruits of our labors.

First up, we are releasing a new Spotify Web API.

This is a complete revamp of the Spotify Metadata API (the old version has now been deprecated). The Spotify Web API gives you access to all sorts of information about the Spotify catalog including details about artists, albums and tracks. Want to know the top tracks for an artist? There’s an API for that. Looking for high quality album art, artist images and 30 second audio previews? There are APIs for that too. Best of all, the new API includes perhaps the most requested Spotify API feature of all time With the Spotify Web API you can now create and modify playlists on behalf of authenticated users. Yes – you can now create a Spotify web app that creates playlists. (I personally requested this feature way back in 2008, here’s my begging plea for the feature in 2009).

static_echonest_com_SpotifyPopcorn_I’ve been using the beta version of this new API for a couple months now and I must say I am quite impressed. The API is fast, super easy to use, and provides all sorts of great data for building apps. In the past weeks I’ve had fun converting a number of my favorite apps to use the Spotify API. First there’s the Road Trip Mix Tape that lets you create a Spotify playlist of music by artists that are from the very towns you are driving through. Then there’s Music Popcorn, a visual interface for exploring genres. For the less visual, there’s the Genre Browser that gives you lots of details about the different music genres including playlists that help give you a gentle introduction to any of the thousands of Echo Nest genres. Next there’s Boil the Frog, an app that creates seamless playlists between any two artists. Finally there’s the 3D Music Maze, an app that lets you explore for music by wandering through a 3 dimensional music world.

Next up, a freshly minted Echo Nest + Spotify Sandbox — a new Spotify ID space.

180px-Rosetta_Stone_BW.jpegThese apps are possible because of the second thing we are releasing this week – a spiffy, shiny new Spotify Rosetta Stone catalog that ensures that the Echo Nest API has the freshest, and most up-to-date view of the Spotify universe of music. For those who might be new to The Echo Nest, Project Rosetta Stone is something we’ve been working on here at the Nest for many years. The goal of Project Rosetta Stone is to solve one of the most common problems that nearly every music app developer faces. The problem is that every music service has its own set of IDs – a music subscription service like Spotify has its own artist, album and track IDs. A lyric service has its own (and very different) IDs for those same artists, albums and tracks and a concert ticketing API has yet a third set of IDs. This is quite problematic for app developers that want to build an app that combines information from multiple services. Without a common ID system, the app developer has to resort to metadata searching and matching – which is slow and quite error prone – this results in a poor app.

Project Rosetta Stone solves this problem by providing ID mappings between as many music services as we can. With this mapping you can easily translate IDs from one ID space to another. With Rosetta Stone, if you have the Spotify track ID you can get Lyricfind and/or Musixmatch IDs making it easy to use those respective APIs to retrieve lyrics for that song. You can easily map the Spotify artist ID to a Songkick or Eventful ID to get ticket and touring information from those APIs. And of course you can use the Spotify track ID to get detailed Echo Nest information about the song such as its tempo, energy, danceability, along with detailed Echo Nest artist data such as latest artist news, blog posts and similar artists.

We have had Spotify IDs in Rosetta Stone for many years, but this particular mapping has in the past been problematic for us. Spotify has a huge catalog and keeping the mapping fresh and up to date between Spotify and The Echo Nest has always been a big challenge. There’s a huge back catalog with millions of tracks to deal with plus thousands of new tracks are being added every week. The result was that there was always a bit of a lag between when updates to the Spotify catalog were reflected in the Rosetta Stone mapping. This meant that if you built a Rosetta Stone-based app you could find that The Echo Nest wouldn’t always know about a Spotify track, especially if a track was very new. The result would be a less-than-perfect app.

This week we are releasing a new Spotify ID space. Our engineers have been working hard over the last 100 days to set up all sorts of infrastructure and plumbing to ensure that we have the most up-to-date view of the Spotify catalog. No more lag between when a new track appears in Spotify and when you can get Echo Nest data. Plus, all of our APIs that take IDs as inputs will now also take Spotify IDs as input as well. If you have a Spotify artist ID you can use it with any Echo Nest artist API method. Likewise, if you have a Spotify track ID you can use it with any Echo Nest song or track API method that takes a track ID as input. This makes it **really** easy for developers to use The Echo Nest and Spotify Apps together. For example, here’s an API call that returns detailed audio properties for a Spotify track given its ID.

http://developer.echonest.com/api/v4/track/profile?api_key=FILDTEOIK2HBORODV&format=json&id=spotify:track:3L7BcXHCG8uT92viO6Tikl&bucket=audio_summary

I’ve been having much fun using The Echo Nest API with the brand new Spotify API. I’ve already written some code that you can use. First, I wrote a Python library for Spotify called Spotipy. It’s makes it easy to write Python programs that use the new Spotify Web API, and it works well with my Echo Nest Python library called Pyen. Here’s an example of using the two libraries together:

I’ve also put together a number of Javascript example apps that use both APIs. These are simple apps intended to help new developers (or at least new to music apps) use the APIs together to do common things like create chillout playlists, browse through the web of similar artists, and more.

So yes, I’m pretty jazzed about this trip to Barcelona. I get to create a music hack, I get to spend a few days with some of the best music hackers in the world (The Barcelona Music Hack Day, as part of the Sonar Festival tends to attract the top music hackers). I get to spend a few days on the Mediterranean in one of the most beautiful cities in the world. Best of all, I get to talk about the new Spotify and Echo Nest developer platform and help music hackers build cool stuff on top of the newly combined platform.

I’ve put together a page that talks in detail about the new Spotify / Echo Nest platform. It has links to all of the API docs, libraries, examples, github repos, demos and details on how you can use The Echo Nest / Spotify Platform. Check it out here:

The_Echo_Nest___Spotify_Developer

 

http://static.echonest.com/enspex

Keep an eye on this space for I’ll be updating it as we continue to integrate our developer APIs. There’s lots more coming, so stay tuned!

5 Comments

Follow

Get every new post delivered to your Inbox.

Join 1,002 other followers