Archive for category The Echo Nest
I love writing music apps and playing with music data. How much? Every weekend I wake up at 6am and spend a long morning writing a new music app or playing with music data. That’s after a work week at The Echo Nest, where I spend much of my time writing music apps and playing with music data. The holiday break is even more fun, because I get to spend a whole week working on a longer project like Six Degrees of Black Sabbath or the 3D Music Maze.
That’s why I love working for The Echo Nest. We have built an amazing open API, which anyone can use, for free, to create music apps. With this API, you can create phenomenal music discovery apps like Boil the Frog that take you on a journey between any two artists, or Music Popcorn, which guides you through hundreds of music genres, from the arcane to the mainstream. It also powers apps that create new music, like Girl Talk in a Box, which lets you play with your favorite song in the browser, or The Swinger, which creates a Swing version of any song:
One of my roles at The Echo Nest is to be the developer evangelist for our API — to teach developers what our API can do, and get them excited about building things with it. This is the best job ever. I get to go to all parts of the world to attend events like Music Hack Day and show a new crop of developers what they can do with The Echo Nest API. In that regard, it’s also the easiest job ever, because when I show apps that developers have built with The Echo Nest API, like The Bonhamizer and The Infinite Jukebox, or show how to create a Pandora-like radio experience with just a few lines of Python, developers can’t help but get excited about our API.
At The Echo Nest we’ve built what I think is the best, most complete music API — and now our API is about to get so much better.
Today, we’ve announced that The Echo Nest has been acquired by Spotify, the award-winning digital music service. As part of Spotify, The Echo Nest will use our deep understanding of music to give Spotify listeners the best possible personalized music listening experience.
Spotify has long been committed to fostering a music app developer ecosystem. They have a number of APIs for creating apps on the web, on mobile devices, and within the Spotify application. They’ve been a sponsor and active participant in Music Hack Days for years now. Developers love Spotify, because it makes it easy to add music to an app without any licensing fuss. It has an incredibly huge music catalog that is available in countries around the world.
Spotify and The Echo Nest APIs already work well together. The Echo Nest already knows Spotify’s music catalog. All of our artist, song, and playlisting APIs can return Spotify IDs, making it easy to build a smart app that plays music from Spotify. Developers have been building Spotify / Echo Nest apps for years. If you go to a Music Hack Day, one of the most common phrases you’ll hear is, “This Spotify app uses The Echo Nest API”.
I am incredibly excited about becoming part of Spotify, especially because of what it means for The Echo Nest API. First, to be clear, The Echo Nest API isn’t going to go away. We are committed to continuing to support our open API. Second, although we haven’t sorted through all the details, you can imagine that there’s a whole lot of data that Spotify has that we can potentially use to enhance our API. Perhaps the first visible change you’ll see in The Echo Nest API as a result of our becoming part of Spotify is that we will be able to keep our view of the Spotify ID space in our Project Rosetta Stone ID mapping layer incredibly fresh. No more lag between when an item appears in Spotify and when its ID appears in The Echo Nest.
The Echo Nest and the Spotify APIs are complementary. Spotify’s API provides everything a developer needs to play music and facilitate interaction with the listener, while The Echo Nest provides deep data on the music itself — what it sounds like, what people are saying about it, similar music its fans should listen to, and too much more to mention here. These two APIs together provide everything you need to create just about any music app you can think of.
I am a longtime fan of Spotify. I’ve been following them now for over seven years. I first blogged about Spotify way back in January of 2007 while they were still in stealth mode. I blogged about the Spotify haircuts, and their serious demeanor:
Last month, on a very cold and snowy day in Boston, I sat across a conference table from of bunch of really smart folks with Swedish accents. As they described their vision of a music platform, it became clear to me that they share the same obsession that we do with trying to find the best way to bring fans and music together.
Together, we can create the best music listening experience in history.
I’m totally excited to be working for Spotify now. Still, as with any new beginning, there’s an ending as well. We’ve worked hard, for many years, to build The Echo Nest — and as anyone who’s spent time here knows, we have a very special culture of people totally obsessed with building the best music platform. Of course this will continue, but as we become part of Spotify things will necessarily change, and the time when The Echo Nest was a scrappy music startup when no one in their right mind wanted to be a music startup will be just a sweet memory. To me, this is a bit like graduation day in high school — it is exciting to be moving on to bigger things, but also sad to say goodbye to that crazy time in life.
There is a big difference: When I graduated from high school and went off to college, I had to say goodbye to all my friends, but now as I go off to the college of Spotify, all my friends and classmates from high school are coming with me. How exciting is that!
In previous posts, we’ve seen that different regions of the country can have different listening preferences. So far we’ve looked at the distinctive artists in any particular region. Perhaps equally interesting is to look at artists that get much fewer listens in a particular region than you would expect. These are the regional anti-preferences, the artists that are generally popular across the United States, but get much less love in a particular part of the country.
To find these artists, we merely look for artists that drop the furthest in rank on the top-most-played chart for a region when compared to the whole U.S. For example, we can look at the top 50 artists in the United States, and find those artists of the 50 that drop furthest in rank on the New Hampshire chart. Try it yourself. Here are the results:
Artists listened to more in United States than they are in New Hampshire
R. Kelly is ranked the 42nd most popular artist in the U.S., but in New Hampshire he’s the 720th most popular, a drop of 678 positions on the chart making him the most ignored artist in New Hampshire.
We can do this for each of the states in the United States and of course we can put them on a map. Here’s a map that shows the most ignored artist of the U.S. top-50 artist in each state.
What can we do with this information? If we know where a music listener lives, but we know nothing else about them, we can potentially improve their listening experience by giving them music based upon their local charts instead of the global or national charts. We can also improve the listening even if we don’t know where the listener is from. As we can see from the map, certain artists are polarizing artists, liked in some circles and disliked in others. If we eliminate the polarizing artists for a listener that we know nothing about, we can reduce the risk of musically offending the listener. Of course, once we know a little bit about the music taste of a listener we can greatly improve their recommendations beyond what we can do based solely on demographic info such as the listener location.
Future work There are a few more experiments that I’d like to try with regard to exploring regional preferences. In particular I think it’d be fun to generate an artist similarity metric based solely on regional listening behaviors. In this world, Juicy J, the southern rapper, and Hillsong United, the worship band would be very similar since they both get lots of listens from people in Memphis. A few readers have suggested alternate scoring algorithms to try, and of course it would be interesting to repeat these experiments for other parts of the world. So much music data, so little time! However, this may be the last map I make for a while since the Internet must be getting sick of ‘artists on a map’ by now.
The data for the map is drawn from an aggregation of data across a wide range of music services powered by The Echo Nest and is based on the listening behavior of a quarter million online music listeners.
In my recent regional listening preferences post I published a map that showed the distinctive artists by state. The map was rather popular, but unfortunately was a source of confusion for some who thought that the map was showing the favorite artist by state. A few folks have asked what the map of favorite artists per state would look like and how it would compare to the distinctive map. Here are the two maps for comparison.
Favorite Artists by State
This map shows the most played artist in each state over the last year. It is interesting to see the regional differences in favorite artists and how just a handful of artists dominates the listening of wide areas of the country.
Most Distinctive Artists by State
This is the previously published map that shows the artists that are listened to proportionally more frequently in a particular state than they are in all of the United States.
The data for both maps is drawn from an aggregation of data across a wide range of music services powered by The Echo Nest and is based on the listening behavior of a quarter million online music listeners.
It is interesting to see that even when we consider just the most popular artists, we can see regionalisms in listening preferences. I’ve highlighted the regions with color on this version of the map:
Favorite Artist Regions
Earlier this week we looked at how gender can affect music listening preferences. In this post, we continue the tour through demographic data and explore how the age of the listener tells us something about their music taste.
Where does the age data come from? As part of the enrollment process for most music services, the user is asked for a few pieces of demographic data, including gender and year-of-birth. As an example, here’s a typical user-enrollment screen from a popular music subscription service:
Is this age data any good? The first thing we need to do before we get too far with analyzing the age data is to get a feel for how accurate it is. If new users are entering random values for their date of birth then we won’t be able to use the listener’s age for anything useful. For this study, I looked at the age data submitted by several million listeners. This histogram shows the relative number of users by year of birth.
The first thing I notice is the curve has the shape one would expect. The number of listeners in each age bucket increases as the listener gets younger until around age 21 or so, at which points it drops off rapidly. The shape of the curve aligns with the data from this study by EMI in 2011 that shows the penetration of music streaming service by age demographic. This is a good indicator that our age data is an accurate representation of reality.
However, there are a few anomalies in the age data. There are unexpected peaks at each decade – likely due to people rounding their birth year to the nearest decade. A very small percentage (0.01 %) indicate that they are over 120 years old, which is quite unlikely. Despite this noise, the age data looks to be a valid and fairly accurate representation, in the aggregate, of the age of listeners. We should be able to use this data to understand how age impacts listening.
Does a 64-year-old listen to different music than a 13-year-old?
One would expect that people of different ages would have different music tastes. Let’s see if we can confirm this with our data. For starters, lets compare the average listening habits of 64-year-old listeners to that of the aggregate listening habits of the 13-year-old listener. For this experiment I selected 5,000 listeners in each age category, and aggregated their normalized artist plays to find the most-frequently-played artists. As expected, you can see that 64-year-old listeners have different tastes than 13-year-old listeners.
The top artists for the average 64-year-old listener include a mix of currently popular artists along with a number of artists from years gone by. While the top artists for the average 13-year-old includes only the most current artists. Still, there are seven artists (shown in bold) that overlap in the top 20 – an overlap rate of about 35%. This 35% overlap is consistent across all ranges of top artists for the two groups. No matter if we look at the top 100 or the top 1000 artists – there’s about a 35% overlap between the listening of 13- and 64-year-olds.
I suspect that 35% overlap is actually an overstatement of the real overlap between 13- and 64-year-olds. There are a few potential confounding effects:
- There’s a built-in popularity bias in music services. If you go to any popular music service you will see that they all feature a number of playlists filled with popular music. Playlists like The Billboard Top 100, The Viral 50, The Top Tracks, Popular New Releases etc. populate the home page or starting screen for most music services. This popularity bias inflates the apparent interest in popular music so, for instance, it may look like a 64-year-old is more interested in popular music than they really are because they are curious about what’s on all of those featured playlists.
- The age data isn’t perfect – for instance, there are certainly a number of people that we think are 64-years-old but are not. This will skew the results to artists that are more generally popular. We don’t really know how big this affect is, but it is certainly non-zero.
- People share listening accounts – this is perhaps the biggest confounding factor – that 64-year-old listener may be listening to music with their kids, their grand-kids, their neighbors and friends which means that not all of those plays should count as plays by a 64-year-old. Again, we don’t know how big this effect is, but it is certainly non-zero.
Let’s pause and have a listen to some music. First the favorite music of a typical 64-year-old listener:
And now the favorite music of a typical 13-year-old listener:
Finding the most distinctive artists
Perhaps more interesting than looking at how the two ages overlap in listening, is to look at how they differ – what are the artists that a 64-year-old will listen to that are rarely, if ever, listened to by a 13-year-old and vice versa. These are the most distinctive artists.
We can find the distinctive artists by identifying the artists in the top 100 of one group that fall the furthest in ranking in the other group. For example Skrillex is the 40th most listened to artist for the typical 13-year-old listener, but for 64-year-old listeners, Skrillex falls all the way to the 3,937 most listened to artist, making Skrillex one of the most distinguishing artist between the two groups of listeners. Likewise, Roy Orbison is the 42nd most listened to artist among 64-year-olds. He drops to position 4,673 among 13-year-olds making him one of the distinguishing artists that separate the 64-year-old from the 13-year-old.
We can use this technique to create playlists of artists that separate the 13-year-old from the 64-year-olds. Are you a 13-year-old, having a party and really wish that grandma would go to another room? Try this playlist:
Are you a 64-year-old and you want all of those 13 year-olds at the party to go home? Try this playlist:
We can also use this data to bring these two groups together. We can find the music that is liked the most among the two groups. We can do this by ordering artists by their worst ranking among the two groups. Artists like Skrillex and Roy Orbison fall to the bottom of the list since each is poorly ranked by one of the groups, while artists like Katy Perry and Bruno Mars rise to the top because they are favored by both groups.
Again, the confounding factors mentioned previously will bias the shared lists to more popular music. Nevertheless, if you are trying to make a playlist of music that will please both a 64-year-old and a 13-year-old, and you know nothing else about their music taste, this is probably your best bet.
Artists that are favored by both 64-year-old and 13-year-old listeners are: Bruno Mars, Taylor Swift, P!nk, Rihanna, Justin Timberlake, Katy Perry, Robin Thicke, Maroon 5, Lana Del Rey, Daft Punk, Beyoncé, Drake, Luke Bryan, Adele, Macklemore & Ryan Lewis, Miley Cyrus, David Guetta, Lorde, Jay-Z, Usher
We can sum up the differences between the two groups in this graphic:
Broadening our view
We’ve shown that, as expected, 13-year-olds and 64-year-olds have different listening preferences. We can apply the same techniques across the range of age demographics typically used by marketers. We can find the most distinctive artists for each demographic bucket. It is interesting to see the progression of music taste over time. For instance, it is clear that something happens to a music listener between the 25 to 34 and 35 to 44 age buckets. The typical listener goes from hipster (Lumineers, Vampire Weekend, The National), to old (Pearl Jam, U2, Bon Jovi).
It is interesting to look at the starting year for artists in each of these buckets to get a sense of how the artist’s own age relates to the age of their fans:
My take-way from this is that no matter how old you are, you don’t like the music from the 70s and the 80s so much.
Most homogenous Artists
We can also find the artists that are most acceptable across all demographics. These are the artists that are liked by more listeners in all of the groups. Like in the 13/64-year-old example, we can find these artists by ordering them by their worst ranking among all the demographic groups.
Most homogeneous artists: Bruno Mars, Rihanna, Katy Perry, Lana Del Rey, Beyoncé, P!nk, Jay-Z, Macklemore & Ryan Lewis, Daft Punk, Maroon 5, Justin Timberlake, Robin Thicke, David Guetta, Luke Bryan, Taylor Swift, Drake, Adele, Imagine Dragons, Miley Cyrus, Lorde
This is essentially the list of the most popular artists but with the most polarizing artists from any one demographic removed. If you don’t know the age of your listener, and you want to give the listener a low risk listening experience, these artists are a good place to start. And yes … this results in a somewhat bland, non-adventurous listening session – that’s the point. But as soon as you know a bit about the true listening preference of a new listener, you can pivot away from the bland and give them something much more in line with their music taste.
Rounding out the stats
There are a few more interesting bits of data we an pull out related to the age of the listener
Average number of artists in listening rotation
The typical 25- to 34-year old listener has more artists in active rotation than any other age group, while the 65+ listeners have the least.
Relative number of plays per user by age group
Likewise, the typical 25- to 34-year-old listener plays more music than any other category.
Tying it all up …
This quick tour through the ages confirms our thinking that the age of a listener plays a significant role in the type of music that they listen to. We can use this information to find music that is distinctive for a particular demographic. We can also use this information to help find artists that may be acceptable to a wide range of listeners. But we should be careful to consider how popularity bias may affect our view of the world. And perhaps most important of all, people don’t like music from the 70s or 80s so much.
One of the challenges faced by a music streaming service is to figure out what music to play for the brand-new listener. The first listening experience of a new listener can be critical to gaining that listener as a long time subscriber. However, figuring out what to play for that new listener is very difficult because often there’s absolutely no data available about what kind of music that listener likes. Some music services will interview the new listener to get an idea of their music tastes.
However, we’ve seen that for many listeners, especially the casual and indifferent listeners, this type of enrollment may be too complicated. Some listeners don’t know or care about the differences between Blues, R&B and Americana and thus won’t be able to tell you which they prefer. A listener whose only experience in starting a listening session is to turn on the radio may not be ready for a multi-screen interview about their music taste.
So what can a music service play for a listener when they have absolutely no data about that listener? A good place to start is to play music by the most popular artists. Given no other data, playing what’s popular is better than nothing. But perhaps we can do better than that. The key is in looking at the little bit of data that a new listener will give you.
For most music services, there’s a short user enrollment process that gets some basic info from the listener including their email address and some basic demographic information. Here’s the enrollment box for Spotify:
Included in this information is the date of birth and the gender of the listener. Perhaps we can use basic demographic data to generate a slightly more refined set of artists. For starters, lets consider gender. Let’s try to answer the question: If we know that a listener is male or female does that increase our understanding of what kind of music they might like? Let’s take a look.
Exploring Gender Differences in Listening
Do men listen to different music than women do? Anecdotally, we can think of lots of examples that point to yes – it seems like more of One Direction’s fans are female, while more heavy metal fans are male, but lets take a look at some data to see if this is really the case.
The Data – For this study, I looked at the recent listening of about 200 thousand randomly selected listeners that have self-identified as either male or female. From this set of listeners, I tallied up the number of male and female listeners for each artist and then simply ranked the artists in order or listeners. Here’s a quick look at the top 5 artists by gender.
Top 5 artists by gender
|2||Bruno Mars||Daft Punk||Bruno Mars|
|4||Katy Perry||Bruno Mars||Katy Perry|
Among the top 5 we see that the Male and Female listeners only share one artist in common:Bruno Mars. This trend continues as we look at the top 40 artists. Comparing lists by eye can be a bit difficult, so I created a slopegraph visualization to make it easier to compare. Click on this image to see the whole slopegraph:
Looking at the top 40 charts artists we see that more than a quarter of the artists are gender specific. Artists that top the female listener chart but are missing on the male listener chart include: Justin Bieber, Demi Lovato, Shakira, Britney Spears, One Direction, Christina Aguilera, Ke$ha, Ciara, Jennifer Lopez, Avril Lavigne and Nicki Minaj. Conversely, artists that top the male listener chart but are missing on the top 40 female listener chart include: Bob Marley, Kendrick Lamar, Wiz Khalifa, Avicii, T.I. Queen, J.Cole, Linkin Park, Kid Cudi and 50 Cent. While some artists seem to more easily cross gender lines like Rihanna, Justin Timberlake, Lana Del Rey and Robin Thicke.
No matter what size chart we look at – whether it is the top 40, top 200 or the top 1000 artists – about 30% of artists on a gender-specific chart don’t appear on the corresponding chart for the opposite gender. Similarly, about 15% of the artists that appear on a general chart of top artists will be of low relevance to a typical listener based on these gender-listening differences.
What does this all mean? If you don’t know anything about a listener except for their gender, you can reduce the listener WTFs by 15% for a typical listener by restricting plays to artists from the gender specific charts. But perhaps even more importantly, we can use this data to improve the listening experience for a listener even if we don’t know a listener’s gender at all. Looking at the data we see that there are a number of gender-polarizing artists on any chart. These are artists that are extremely popular for one gender, but not popular at all for the other. Chances are that if you play one of these polarizing artists for a listener that you know absolutely nothing about, 50% of the time you will get it wrong. Play One Direction and 50% of the time the listener won’t like it, just because 50% of the time the listener is male. This means that we can improve the listening experience for a listener, even if we don’t know their gender by eliminating the gender skewing artists and replacing them with more gender neutral artists.
Let’s see how this would affect our charts. Here are the new Top 40 artists when we account for gender differences.
|16||23||Lana Del Rey|
|19||27||The Black Eyed Peas|
|24||22||Macklemore & Ryan Lewis|
Artists promoted to the chart due to replace gender-skewed artists are in bold. Artists that were dropped from the top 40 are:
- Avicii – skews male
- Justin Bieber – skews female
- Christina Aguilera – skews female
- One Direction – skews female
- Demi Lovato – skews female
Who are the most gender skewed artists?
The Top 40 is a fairly narrow slice of music. It is much more interesting to look at how listening can skew across a much broader range of music. Here I look at the top 1,000 artists listened to by males and the top 1,000 artists listened to by females and find the artists that have the largest change in rank as they move from the male chart to the female chart. Artists that lose the most rank are artists that skew male the most, while artists that gain the most rank skew female.
Top male-skewed artists:
artists that skew towards male fans
- Iron Maiden
- Rage Against the Machine
- Van Halen
- Jimi Hendrix
- Limp Bizkit
- Wu-Tang Clan
- The Who
- Alice in Chains
- Black Sabbath
- Stone Temple Pilots
- Mobb Deep
- Queens of the Stone Age
- Ice Cube
Top female-skewed artists:
artists that skew towards female fans
- Danity Kane
- Cody Simpson
- Hannah Montana
- Emily Osment
- Playa LImbo
- Vanessa Hudgens
- Miranda Lambert
- Aly & AJ
- Christina Milian
- Noel Schajris
- Maria José
- Jesse McCartney
- Bridgit Mendler
- Luis Fonsi
- La Oreja de Van Gogh
- Michelle Williams
- Lindsay Lohan
By looking at the genres of the most gender skewed artists we can also get a sense of which genres are most gender skewed as well. Looking at the genres of the top 1000 artists listened to by male listeners and the top 1000 artists with female listeners we identify the most skewed genres:
Genres most skewed to female listeners:
- Dance Pop
- Contemporary Hit Radio
- Urban Contemporary
- Hot Adult Contemporary
- Latin Pop
- Teen Pop
- Neo soul
- Pop rock
- Contemporary country
Genres most skewed to male listeners:
- Hip Hop
- Album Rock
- Pop Rap
- Indie Rock
- Funk Rock
- Gangster Rap
- Electro house
- Classic rock
- Nu metal
This study confirms what we expected – that there are differences in gender listening. For mainstream listening about 30% of the artists in a typical male’s listening rotation won’t be found in a typical female listening rotation and vice versa. If we happen to know a listener’s gender and nothing else, we can improve their listening experience somewhat by replacing artists that skew to the opposite gender with more neutral artists. We can even improve the listening experience for a listener that we know absolutely nothing about – not even their gender – by replacing gender-polarized artists with artists that are more accepted by both genders.
Of course when we talk about gender differences in listening, we are talking about probabilities and statistics averaged over a large number of people. Yes, the typical One Direction fan is female, but that doesn’t mean that all One Direction fans are female. We can use gender to help us improve the listening experience for a brand new user, even if we don’t know the gender of that new user. But I suspect the benefits of using gender for music scheduling is limited to helping with the cold start problem. After a new user has listened to a dozen or so songs, we’ll have a much richer picture of the type of music they listen to – and we may discover that the new male listener really does like to listen to One Direction and Justin Bieber and that new female listener is a big classic rock fan that especially likes Jimi Hendrix.
update – 2/13 – commenter AW suggested that the word ‘bias’ was too loaded a term. I agree and have changed the post replacing ‘bias’ with ‘difference’
This past weekend was the fourth annual MIDEM Music Hack Day held in Cannes. During about 48 hours 2 dozen or so hackers collected in a beautiful hacking space at the top of the Palais Des Festivals to build something cool with music and technology.
The MIDEM Music Hack Day is no ordinary Music Hack Day. It has a very limited enrollment so only hackers that have demonstrated the ability to create music hacks are invited. Add to that, the fact that there is about 50% more time to build hacks and the result is a set of very high quality hacks.
Martyn Davies, master Music Hack day coordinator, kicked off the event with a talk to the MIDEM attendees about what Music Hack Day is all about. Martyn talked about the things that drive the hackers to spend their weekends hacking on code – in particular how the Music Hack Day is a chance to combined their love for music and technology, be creative and to build something new and cool during the weekend. Martyn demonstrated two representative hacks built at previous Music Hack Days. First he showed the demo given by master hacker @sydlawrence of a hack called Disco Disco Tech. The excitement in Syd’s voice is worth the price of admission alone.
Next he showed one of my favorite Music Hack Day hacks of all time, Johnny Cash is Everywhere by @iainmullen
One of the special features of the MIDEM Music Hack Day is that non-hackers get to pitch their hacking ideas to the hackers about what apps they’d really like to see created over the weekend. There were a number of pitches ranging from a proposal for an artist-centric tool for organizing a creative music production team to a whimsical request to show what the music on the Internet sounds like when it decays. (Here’s one answer). All of the idea pitches were interesting, but here’s the secret. The hackers are not going to build your idea. It’s not because they don’t like your idea, it is because they already have tons of good ideas. The hackers are a very creative bunch, each with a long list of ideas waiting to be built. What the hackers usually lack is a solid block of time to implement their own ideas and so a hackathon is the perfect time to take that best idea on the list and work for a solid 24 hour to get it done. It is rare for a hacker to get excited about building someone else’s idea, when they have so many of their own. As they say: “ideas are cheap, execution is everything“.
Once the opening talks concluded we hackers made their way up to the top of the Palais des Festivals (the heart of MIDEM) to our hacking space. It’s a great space with lots of natural light, a terrace that overlooks the French Riviera, and it is some distance away from the main conference so we were not bothered by stray walk-ons.
To kick things off, we went around the room introducing ourselves, briefly talking about our background skills, and ideas, and almost immediately got to hacking. Since all the hackers were experienced hackers there was no need for the typical API workshops or learning sessions. Everyone knew, for instance, that I was from The Echo Nest and was ready to answer any questions about the Echo Nest API that should arise.
The next 46 hours was a blur of coding, punctuated by food delivery, the sound of the espresso maker and the occasional wandering pigeon.
Hacker Self Portraits
There’s a math error lurking in there …
Early morning coding on the French Riveria
After 48 hours, we gathered in the Innovation Hall to demonstrate what we built. Each hacking team had about three minutes to show their stuff. Eighteen hacks were built. Here are some of my very favorites:
DJ Spotify – built by Yuli Levtov – This is a real hacker’s hack. Yuli had a problem. He wanted to use Spotify when he DJed, but Spotify won’t let you beat match and cross fade songs. In fact, Spotify won’t even let you play two streams at once. So Yuli got to work to make it happen. Along the way Yuli augmented his DJ playlists with BPM and key information from The Echo Nest (using a very clever growl hack). One of the highlights of my MIDEM week was listening to Yuli try to explain what OS virtualization is and how Soundflower works to a room full of Music Biz types. Yuli has a detailed blog post that describes how his hack works. Yuli’s hack was voted the best hack by the hackers. Well done Yuli!
That One Song - by Matt Ogle and Hannah Donovan – For this hack, Matt added a feature to his super popular This Is My Jam site. Type in any band and let the Jam community tell you the one song you should hear first. Plus: playback options, commentary, and an alternative “B-side” song recommendation for each artist.
Skrillex Invader 20 – by Vivien Barousse - Imagine Guitar Hero meets Space Invaders meets Skrillex meets a Piano Keyboard. Skrillex Invader 20 is a small game designed to help you improve your skills on a piano keyboard.
Scratchy Record - by Alastair Porter - Playing music from mp3s today has no soul. Scratchy Record reproduces the joy that can be had listening to music on vinyl. From the dirty needle causing extra noise, to the pops and skips that we all love, to the need to get up half way through the album and turn it over. Scratchy Record has it all.
HappyClappy - by Peck, ankit and mager – an IOS app that lets you queue up songs by clapping the rhythm. Uses The Echo Nest to search for songs by BPM.
PhatStats - Syd Lawerence – Syd tried to build a sustainable subscription business during the hack with PhatStats - A new tool to discover up and coming talent across the social web, and to monitor your videos and their engagement levels across the social web.
This is your tour - Sam Phippen - Going on tour is hard. You’ve got to find someone to tour with. You’ve got to pick cities and venues. You’ve got to book hotels, find places to eat and drink. All of this takes far too much time.
Nikantas – Sabrina Leandro – a clever app that helps you learn a new language through music lyrics. Fill in the blanks in lyrics of your favourite English, Spanish, Portuguese, French, German, or Italian artists. Can you recognise a word in a song? A word will be displayed on the screen, press the space key when the artist sings it
neoScores meets Deezer - Bob Hamblok – HTML5 sheet music score following inside Deezer.
Seevl hipster - by Alexandre Passant - Be a real hipster. Impress your friends with obscure music tastes. Do you want to impress your friend who’s into electro-folk, or that other one who only listens to avant-garde metal? Now you can! With seevl hipster, find obscure artists that match your friend tastes, and show-off on their Facebook wall.
Playlist Plus - Iain Mullan - Playlist Plus allows you to create a richer interactive version of a playlist. Add notes and comments to each track, to share with friends, or distribute an in-depth album review. Like a particular lyric? Bookmark it at the exact timestamp in the song. Think a track has a heavy Zeppelin influence? Link to the song/album it reminds you of. Cover version? Link to the original and let the reader stream it instantly from within your PlaylistPlus!
ScapeList - Mohammed Sordo - How does a landscape sound like? You take a picture of, let’s say, the Grand Canyon in Colorado, a la Instagram, but you also want to attach a song to it, a song that makes sense to you while you were taking that picture. Now imagine that other people went to the same place, took another picture of it but picked a different song. You end up with a playlist of songs related to that landscape, a ScapeList, curated by the users themselves, which you can listen to.
VideoFairy - by Suzie Blackman - A radio-style music discovery app designed for smart TV! It’s a bit like channel hopping, but for music videos. VideoFairy finds music videos from artists you’ll like with a simple interface that works with a remote control (use arrow keys and ‘enter’ on a keyboard).Designed for ‘lean back’ TV viewing with minimum interaction, you can sit back and watch new music recommended from your last.fm profile. Skip skip anything you don’t like with a simple tap of the remote.
Cannes Burn - my hack – a music visualizaton of Ellie Goulding’s Burn
I was unusually nervous and quite tired when I gave my demo, so I fell for a newbie demo mistake and had trouble getting my desktop to display properly. But when I finally did, my demo went off smoothly. I only had to say a few words and hit the play button, so despite the nerves, it was a pretty easy demo to give. Here’s my view of the audience while giving the demo:
After we presented the hacks the hackers themselves voted for the best hack which went to Yuli for his amazing DJ Spotify. Yuli is quite the gracious and humble winner, making sure everyone got a glass of his winning champagne.
After all the hacking the exhausted hackers took some time to kick back, have a good dinner, a few drinks and long conversations into the night about life as an international music hacker.
Halfway through the MIDEM Music Hack Day I paused to take stock. Here I was, on the other side of the world sitting at the top of the Palais des Festival, overlooking the French Riviera, surrounded by friends and writing code. It was a great place to be, and I felt very fortunate to be there. This was all possible because the music biz folks realize that we hackers have lots of ideas that will advance the state-of-the-art in music tech, and even more importantly we have the ability to actually turn those ideas into reality. And so, they treat us very well. It is good to be a music hacker.
My hack at the MIDEM Music Hack Day this year is what I’d call a Creative Hack. I built it, not because it answered any business use case or because it demonstrated some advanced capability of some platform or music tech ecosystem, I built it because I was feeling creative and I wanted to express my creativity in the best way that I can which is to write a computer program. The result is something I’m particularly proud of. It’s a dynamic visualization of the song Burn by Ellie Goulding. Here’s a short, low-res excerpt, but I strongly suggest that you go and watch the full version here: Cannes Burn
Unlike all of the other hacks that I’ve built, this one feels really personal to me. I wasn’t just trying to solve a technical problem. I was trying to capture the essence of the song in code, trying to tell its story and maybe even touch the viewer. The challenge wasn’t in the coding it was in the feeling.
After every hack day, I’m usually feeling a little depressed. I call it post-hacking depression. It is partially caused by being sleep deprived for 48 hours, but the biggest component is that I’ve put my all into something for 48 hours and then it is just over. The demo is done, the code is checked into github, the app is deployed online and people are visiting it (or not). The thing that just totally and completely took over my life for two days is completely gone. It is easy to reflect back on the weekend and wonder if all that time and energy was worth it.
Monday night after the MIDEM hack day was over I was in the midst of my post-hack depression sitting in a little pub called Le Crillon when a guy came up to me and said “I saw your hack. It made me feel something. Your hack moved me.”
Cannes Burn won’t be my post popular hack, nor is it my most challenging hack, but it may be my favorite hack because I was able to write some code and make somebody that I didn’t know feel something.
Today at the Echo Nest we are pushing out an update to our Genre APIs. The new APIs lets you get all sorts of information about any of over 800 genres including a description of the genre, representative artists in the genre, similar genres, and links to web resources for the genre (such as a wikipedia page, if one exists for a genre). You can also use the genres to create various types of playlists. With these APIs you build all sorts of music exploration apps like Every Noise At Once, Music Popcorn and Genre-A-Day.
The new APIs are quite simple to use. Here are a few python examples created using pyen.
List all of the available genres with a description
This outputs text like so:
We can get the top artists for any genre like so:
Here are the top artists for ‘cool jazz’
We can find similar genres to any genre with this bit of code:
% python sim_genres.py cool jazz bebop jazz hard bop contemporary post-bop soul jazz big band jazz christmas stride jazz funk jazz fusion avant-garde jazz free jazz
We can use the genres to create excellent genre playlists. To do so, create a playlist of type ‘genre-radio’ and give the genre name as a seed. We’ve also added a new parameter called ‘genre_preset’ that, if specified will control the type of songs that will be added to the playlist. You can chose from core, in_rotation, and emerging. Core genre playlists are great for introducing a new listener to the genre. Here’s a bit of code that generates a core playlist for any genre:
The core classic rock playlist looks like this:
- Simple Man by Lynyrd Skynyrd
- Born To Be Wild by Steppenwolf
- All Along The Watchtower by Jimi Hendrix
- Kashmir by Led Zeppelin
- Sunshine Of Your Love by Cream
- Let’s Work Together by Canned Heat
- Gimme Shelter by The Rolling Stones
- It’s My Life by The Animals
- 30 Days In The Hole by Humble Pie
- Midnight Rider by The Allman Brothers Band
- The Joker by Steve Miller Band
- Fortunate Son by Creedence Clearwater Revival
- Black Betty by Ram Jam
- Heart Full Of Soul by The Yardbirds
- Light My Fire by The Doors
The ‘in rotation’ classic rock playlist looks like this:
- Heaven on Earth by Boston
- Doom And Gloom by The Rolling Stones
- Little Black Submarines by The Black Keys
- I Gotsta Get Paid by ZZ Top
- Fly Like An Eagle by Steve Miller Band
- Blue On Black by Kenny Wayne Shepherd
- Driving Towards The Daylight by Joe Bonamassa
- When A Blind Man Cries by Deep Purple
- Over and Over (Live) by Joe Walsh
- The Best Is Yet To Come by Scorpions
- World Boss by Gov’t Mule
- One Way Out by The Allman Brothers Band
- Corned Beef City by Mark Knopfler
- Bleeding Heart by Jimi Hendrix
- My Sharona by The Knack
While the emerging ‘classic rock’ playlist looks like this:
- If You Were in Love by Boston
- Beggin’ by Shocking Blue
- Speak Now by The Answer
- Mystic Highway by John Fogerty
- Hell Of A Season by The Black Keys
- No Reward by Gov’t Mule
- Pretty Wasted by Tito & Tarantula
- The Battle Of Evermore by Page & Plant
- I Got All You Need by Joe Bonamassa
- What You Gonna Do About Me by Buddy Guy
- I Used To Could by Mark Knopfler
- Wrecking Ball by Joe Walsh
- The Circle by Black Country Communion
- You Could Have Been a Lady by April Wine
- 15 Lonely by Walter Trout
The new Genre APIs are really quite fun to use. I’m looking forward to seeing a whole new world of music exploration and discovery apps built around these APIs.
The Echo Nest knows about 800 genres of music (and that number is growing all the time). Among those 800 genres are ones that you already know about, like ‘jazz’,'rock’ and ‘classical’. But there are also hundreds of genres that you’ve probably never heard of. Genres like Filthstep, Dangdut or Skweee. Perhaps the best way to explore the genre space is via Every Noise at Once (built by Echo Nest genre-master Glenn McDonald). Every Noise At Once shows you the whole genre space, allowing you to explore the rich and varied universe of music. However, Every Noise at Once can be like drinking Champagne from a firehose – there’s just too much to take in all at once (it is, after all, every noise – at once). If you’d like to take a slower and more measured approach to learning about new music genres, you may be interested in Genre-A-Day.
Genre-A-Day is a web app that presents a new genre every day. Genre-A-Day tells you about the genre, shows you some representative artists for the genre, lets you explore similar genres, and lets you listen to music in the genre.
If you spend a few minutes every day reading about and listening to a new genre, after a few months you’ll be a much more well-rounded music listener, and after a few years your knowledge of genres will rival most musicologists’.
An easy way to make Genre-A-Day part of your daily routine is to follow @GenreADay on twitter. GenreADay will post a single tweet, once a day like so:
Under the hood - Genre-A-Day was built using the just released genre methods of The Echo Nest API. These methods allow you to get detailed info on the set of genres, the top artists for the genres, similar genres and so on. It also uses the super nifty genre presets in the playlist API that allow you to craft the genre-radio listener for someone who is new to the genre (core), for someone who is a long time listener of the genre (in rotation), or for someone looking for the newest music in that genre (emerging). The source code for Genre-A-Day is on github.
Ever since the release of the Sony Walkman 35 years ago, the play button has been the primary way we interact with music. Now the play button stands as the last barrier between a listener and their music. Read on to find out how we got here and where we are going next.
In the last 100 years, technology has played a major role in how we listen to and experience music. For instance, when I was coming of age musically, the new music technology was the Sony Walkman. With the Walkman, you could take your music with you anywhere. You were no longer tied to your living room record player to listen to your music. You no longer had to wait and hope that the DJ would play your favorite song when you were on the road. You could put your favorite songs on a tape and bring them with you and listen to them whenever you wanted to no matter where your were. The Sony Walkman really changed how we listened to music. It popularized the cassette format, which opened the door to casual music sharing by music fans. Music fans began creating mix tapes and sharing music with their friends. The playlist was reborn, music listening changed. All because of that one device.
We are once again in the middle of music+technology revolution. It started a dozen years ago with the first iPod and it continues now with devices like the iPhone combined with a music subscription service like Spotify, Rdio, Rhapsody or Deezer. Today, a music listener armed with an iPhone and a ten dollar-a-month music subscription is a couple of taps away from being able to listen to almost any song that has ever been recorded. All of this music choice is great for the music listener, but of course it brings its own problems. When I was listening to music on my Sony Walkman, I had 20 songs to choose from, but now I have millions of songs to choose from. What should I listen to next? The choices are overwhelming. The folks that run music subscription services realize that all of this choice for their listeners can be problematic. That’s why they are all working hard to add radio features like Rdio’s You.FM Personalized Radio. Personalized Radio simplifies the listening experience – instead of having to pick every song to play, the listener only needs to select one or two songs or artists and they will be presented with an endless mix of music that fits well with initial seeds.
Helping listeners pick music is especially important when you consider that not all music listeners are alike, and that most listeners are, at best, only casual music fans. A study conducted in 2003 and again in 2006 by Emap (A UK-based Advertising agency), summarized here by David Jennings, identified four main types of music listeners. Jennings describes these four main listening types as:
Savants – for whom everything in life is tied up with music
Enthusiasts – Music is a key part of life but is balanced with other interests
Casuals – Music plays a welcoming role, but other things are far more important
Indifferents – Would not lose much sleep if music ceased to exist.
These four listener categories are an interesting way to organize music listeners, but of course, real life isn’t so cut and dried. Listener categories change as life circumstances change (have a baby and you’ll likely become a much more casual music listener) and can even change based on context (a casual listener preparing for a long road-trip may act like a savant for a few days while she builds her perfect road-trip playlist).
In 2006, the distribution of people across these 4 categories was as follows:
This chart says a lot about the music world and why it works the way it does. For instance, it gives us a guide as to how much different segments of the listening world are willing to pay for music in a year. On the chart below, I’ve added my estimate of the amount of money each listener type will spend on music in a year.
Savants will spend a thousand dollars or more on vinyl, concerts, and music subscriptions. Enthusiasts will spend $100 a year on a music subscription or, perhaps, purchase a couple of new tracks per week. Casuals will spend $10 a year (maybe splurge and buy that new Beyoncé album), while Indifferents will spend nothing on music. This is why music services like Spotify and Rdio have been exploring the Fremium model. If they want to enroll the 72% of people who are Casual or Indifferent music listeners, they need a product that costs much less than the $100 a year Enthusiasts are willing to pay.
However, price isn’t the only challenge music services face in attracting the Casuals and the Indifferents. Different types of listeners have a different tolerance around the amount of time and effort it takes to play music that they want to listen to.
A music Savant – someone who lives, eats and breathes music – is happy spending hours a day poring through music blogs, forums and review sites to find new music, while the Indifferent music listener may not even make the simplest of efforts like turning the radio on or switching to a new station if they don’t like the current song. A simple metric for the time and effort spent is Interactions Per Listening Session. In this chart, I’ve added my estimate of the number of interactions, on average, a listener of a given type will tolerate to create a listening session.
Interactions per Listening Session is an indication of how many times the listener controls their music player for a listening session. That music Savant may carefully handpick each song going into a playlist after reading a few music blogs and reviews about an artist on The Hype Machine, checking out the artist bio and previewing a few tracks. The music Enthusiast may grab a few top songs from a handful of their favorite artists to build a Spotify playlist. The casual listener may fire up Pandora, select an artist station and click play, while the Indifferent music listener may passively listen to the music that is playing on the radio or in the background at the local Starbucks.
The above chart shows why a music service like Pandora has been so successful. With its simple interface, Pandora is able to better engage the Casual listeners who don’t want to spend time organizing their listening session. A Pandora listener need only pick a station, and Pandora does all the work from there. This is why music subscription services hoping to attract more users are working hard to add Pandora-like features. In order to make their service appeal to the Casuals, they need to make it incredibly easy to have a good listening experience.
But what about those Indifferents? If 40% of people are indifferent to music, is this a lost market for music services? Is it impossible to reach people who can’t even be bothered to queue up some music on Pandora? I don’t think so. Over the last 75 years, terrestrial radio has shown that even the most indifferent music fan can be coaxed into simple, “lean back” listening. Even with all of the media distractions in the world today, 92% of Americans age 12 or older listen to the radio at least weekly, much the same as it was back in 2003 (94%).
So what does it take to capture the ears of Indifferents? First, we have to drive the out-of-pocket costs to the listener to zero. This is already being done via the Freemium model – Ad supported Internet radio (non-on-demand) is becoming the standard entry point for music services. Next, and perhaps more difficult, we have to drive the number of interactions required to listen to music to zero.
Thus my current project – Zero UI – building a music player that minimizes the interactions necessary to get good music to play – a music player that can capture the attention of even the musically indifferent.
Implicit signals and context
Perhaps the biggest challenge in creating a Zero UI music player is how to get enough information about the listener to make good music choices. If a Casual or Indifferent listener can’t be bothered to explicitly tell us what kind of music they like, we have to try to figure it out based upon implicit signals. Luckily, a listener gives us all kinds of implicit signals that we can use to understand their music taste. Every time a listener adjusts the volume on the player, every time they skip a song, every time they search for an artist, or whenever they abandon a listening session, they are telling us a little bit about their music taste. In addition to the information we can glean from a listener’s implicit actions, there’s another source of data that we can use to help us understand a music listener. That’s the listener’s music listening device – i.e. their phone.
The mobile phone is now and will continue to be the primary way for people to interact with and experience music. My phone is connected to a music service with 25 million songs. It ‘knows’ in great detail what music I like and what I don’t like. It knows some basic info about me such as my age and sex. It knows where I am, and what I am doing – whether I’m working, driving, doing chores or just waking up. It knows my context – the time of day, the day of the week, today’s weather, and my schedule. It knows that I’m late for my upcoming lunch meeting and it even might even know the favorite music of the people I’m having lunch with.
Current music interfaces use very little of the extra context provided by the phone to aid in music exploration and discovery. In the Zero UI project, I’ll explore how all of this contextual information provided by the latest devices (and near future devices) can be incorporated into the music listening experience to help music listeners organize, explore, discover and manage their music listening. The goal is to create a music player that knows the best next song to play for you given your current context. No button pressing required.
There are lots of really interesting areas to explore:
Can we glean enough signal from the set of minimal listener inputs?
Which context types (user activity, location, time-of-day, etc.) are most important for scheduling music? Will we suffer from the curse of dimensionality with too many contexts?
What user demographic info is most useful for avoiding the cold start problem (age, sex, zip code)?
How can existing social data (Facebook likes, Twitter follows, social tags, existing playlists) be used to improve the listening experience?
How do we balance knowing enough about a listener to give them good music playlists and knowing so much about a listener that they are creeped out about their ‘stalker music player’?
Over the next few months I’ll be making regular posts about Zero-UI. I’ll share ideas, prototypes and maybe even some code. Feel free to follow along.