Posts Tagged Music
Duke Listens! returns (again)
A few months ago, we finally shutdown the final remnants of the old Echo Nest infrastructure. One of casualties of this final shutdown was the archive of my old blog Duke Listens! that I authored while I was a researcher at Sun Labs. However, I did manage to have a backup sitting on an old backblaze disk, so this morning I took a bit of time to re-host it on one of my personal servers. You can find it at:
The blog serves as a reminder of the history of music recommendation and discovery during the iPhone era. Some notable posts:
- My first MIR-related post (June 2004)
- My first hardcore MIR post (January 2005)
- A decade too early prediction about Apple (January 2005)
- I discover Radio Paradise (April 2005)
- First Google Music rumor (June 2005)
- First Amazon Music rumor (August 2005)
- First Pandora Post (September 2005)
- First mention of The Echo Nest (October 2005)
- Why there’s no Google Music search (December 2005)
- First mention of Spotify (January 2007)
- My review of Spotify (November 2007)
- The Echo Nest goes live (March 2008)
- The Echo Nest launches their API (September 2008)
- My first look at iTunes genius recommendations (September 2008)
- My last post (February 2009)
The Fresh 40
Every week, thousands of artists release albums on Spotify. Sifting through all this new music to find good stuff to listen to can be hard. Luckily, there are lots of tools from New Music Tuesday playlists to the Spotify Viral 50 to help us find the needles in the proverbial haystack of new music. However, most of these tools tend to surface up new music by artists that have been around for a while. For instance, the top artist on Spotify Viral 50 as I write this is Jeremih who has been on the charts for five years. The top of New Music Tuesday right now is Mumford & Sons who’ve been recording for at least eight years.
I’m interested in finding music by the freshest artists – artists that are at the very beginning of their recording careers. To that end, I’ve built a new chart called ‘The Fresh 40’ that shows the top albums by the freshest artists. To build The Fresh 40 I scour through all of the albums that have been released in the last two weeks on Spotify (on average that’s about 30 thousand albums), and find the albums that are the very first album release for its artist. I then rank each album by a weighted combination of the number of followers the artist has on Spotify and the popularity of the artist and album (which is related to Spotify track plays). The result is a chart of the top 40 most popular fresh artists.
The Fresh 40 updates every day and shows all the salient info including the rank, yesterday’s rank, the overall score, artist followers, artist popularity, album popularity and the number of days that the album has been on the chart. Since an album can only be on the chart for 15 days, there’s quite a bit of change from day to day.
If you are interested in finding music by the very newest artists on Spotify, you might be interested in The Fresh 40. Give the chart a look.
The Fresh 40 was built on top of the increasingly marvelous Spotify Web API. Code is on github.
Fresh Faces on Spotify
My weekend programming project this week was to explore a new feature of the Spotify Web API that allows you to find albums that have been released in the last two weeks. The result is a web app called Fresh Faces. This app goes through all of the recent releases and finds those that are the very first release for the artist. If you are looking for new music, there’s no fresher place to start than this app – it finds the newest music by the freshest artists – artists that are barely two weeks into their recording career.
Fresh Faces lets you sort the results based on artist popularity, album popularity, artist followers or release date. You can click on an album to hear a sample, find more info about the album or open it on Spotify.
How many new releases are there?
I was curious about how many releases there are in a two week period, and when releases tend to happen, so I added a chart at the bottom of the Fresh Faces app that shows the distribution of fresh and recurring releases and the dates when releases happen. You can see that the shift of releasing music from Tuesday to Friday is ongoing.
In the past two weeks about 32,000 albums have been released – about 5,200 of these are the first release for the artist. That’s a whole lot of fresh music.
Give Fresh Faces a try and let me know what you think.
50 Years Ago in Music
There’s a strong connection between music and memory. Whenever I here the song Lovin You by Minnie Riperton, I’m instantly transported back to 1975 when I spent the summer apprenticed to Tom, my future brother-in-law, fixing electronic organs. I was 15, Tom was 22 and super cool. He had a business (New Hampshire Organ Service) and he had a van with an 8-track player and an FM radio (a rarity in 1975). As we drove between repairs across rural New Hampshire we’d pass the time by listening to the radio. Now, when I hear those radio songs from 1975 it is like I’m sitting in that van again.
Music can be like a time machine. Transporting us to different times in our lives. I was interested in exploring this a bit more. Inspired by @realtimewwii which gives a day-by-day account of World War II, I created a set of dynamically updating Spotify playlists that follow the charts week-by-week.
For example there’s the 50 Years Ago in Music playlist that contains the top 100 or songs that were on the chart 50 years ago. As I write this on April 12, 2015, this playlist is showing the top songs for the week of April 12, 1965.
The music on this playlist sends me back to when I was 5 years old listening to music on our AM radio in the kitchen in the morning while eating breakfast.
If you follow this playlist you’ll be able to re-create what it was like to listen to music 50 years ago. If the mid-sixties doesn’t speak to you musically, there are some other playlists that you can try.
There’s 40 Years Ago in Music that brings me back to 1975 on the road with Tom.
There’s 30 years Ago in Music which is currently playing music from the mid-80s like Madonna and Phil Collins.
There’s 20 Years Ago in Music currently playing music from the mid-90s:
10 Years Ago in Music plays the music that was on the radio when Spotify was just a gleam in Daniel’s eye.
5 Years Ago in Music – the playlist of @echonest in its heyday.
All of the playlists update weekly on Monday. If you’d like a reminder about when they are updated you can follow @50yearmusic. And of course, the code is on github.
This weekend brought me to Cannes and the French Riviera for the MIDEM Music Hack Day where I’ve spent about 40 hours working on my music hack called Cannes Burn. Cannes Burn is a visualization that accompanies the song Burn by Ellie Goulding. Go check it out if you haven’t already seen it before reading on. It requires a modern computer and browser that supports webgl.
The Hack uses the new ENsync.js library that I created last week. ENsync uses the Echo Nest analysis to provide synchronization of a JS web app with music. With ENsync you can setup elaborate animations that are triggered by musical events (such on every bar, beat or tatum). The Hack also uses threejs – the amazing 3D library by Mr.Doob.
Creating the hack was a whole lot of fun – I spent hours building 3D shapes out of flying cubes. I probably listened to the song Burn many hundreds of times this weekend. (Thanks to my hacker neighbors who put up with my endless Ellie looping without complaint).
It has been a great weekend here in Cannes. It is so inspiring to be surrounded by a bunch of really smart folks who are passionate about music and technology and see and hear how they are building their stuff. Such a great, sharing vibe from all of the hackers. I feel really lucky to part of it all!
Check out all the hacks on hacker league: MIDEM Music Hack Day Hacks and check out my hack at: Cannes Burn
The Ultimate List of Music APIs
Posted by Paul in code, Music, The Echo Nest, web services on October 24, 2013
I’ve started to build the ultimate list of music APIs. My goal for the list is for it to be a one-stop spot to find the best music apis. Currently 65 APIs are listed across 10 categories. Check out the list here: Music APIs
The Most Replayed Songs
Posted by Paul in data, Music, The Echo Nest on August 27, 2013
I still remember the evening well. It was midnight during the summer of 1982. I was living in a thin-walled apartment, trying unsuccessfully to go to sleep while the people who lived upstairs were music bingeing on The B52’s Rock Lobster. They listened to the song continuously on repeat for hours, giving me the chance to ponder the rich world of undersea life, filled with manta rays, narwhals and dogfish.
We tend to binge on things we like – potato chips, Ben & Jerry’s, and Battlestar Galactica. Music is no exception. Sometimes we like a song so much, that as soon as it’s over, we want to hear it again. But not all songs are equally replayable. There are some songs that have some secret mysterious ingredients that makes us want to listen to the song over and over again. What are these most replayed songs? Let’s look at some data to find out.
The Data – For this experiment I used a week’s worth of song play data from the summer of 2013 that consists of user / song / play-timestamp triples. This data set has on the order of 100 million of these triples for about a half million unique users and 5 million unique songs. To find replays I looked for consecutive plays by a user of song within a time window (to ensure that the replays are in the same listening session). Songs with low numbers of plays or fans were filtered out.
For starters, I simply counted up the most replayed songs. As expected, this yields very boring results – the list of the top most replayed songs is exactly the same as the most played songs. No surprise here. The most played songs are also the most replayed songs.
Top Most Replayed Songs – (A boring result)
- Robin Thicke — Blurred Lines featuring T.I., Pharrell
- Jay-Z — Holy Grail featuring Justin Timberlake
- Miley Cyrus — We Can’t Stop
- Imagine Dragons — Radioactive
- Macklemore — Can’t Hold Us (feat. Ray Dalton)
To make this more interesting, instead of looking at the absolute number of replays, I adjusted for popularity by looking at the ratio of replays to the total number of plays for each song. This replay ratio tells us the what percentage of plays of a song are replays. If we plot the replay ratio vs. the number of fans a song has the outliers become quite clear. Some songs are replayed at a higher rate than others.
I made an interactive version of this graph, you can mouse over the songs to see what they are and click on the songs to listen to them.
Sorting the results by the replay ratio yields a much more interesting result. It surfaces up a few classes of frequently replayed songs: background noise, children’s music, soft and smooth pop and friday night party music. Here’s the color coded list of the top 20:
Top Replayed songs by percentage
- 91% replays White Noise For Baby Sleep — Ocean Waves
- 86% replays Eric West — Reckless (From Playing for Keeps)
- 86% replays Soundtracks For The Masters — Les Contes D’hoffmann: Barcarole
- 83% replays White Noise For Baby Sleep — Warm Rain
- 83% replays Rain Sounds — Relax Ocean Waves
- 82% replays Dennis Wilson — Friday Night
- 81% replays Sleep — Ocean Waves for Sleep – White Noise
- 74% replays White Noise Sleep Relaxation White Noise Relaxation: Ocean Waves 7hz
- 74% replays Ween — Ocean Man
- 73% replays Children’s Songs Music — Whole World In His Hands
- 71% replays Glee Cast — Friday (Glee Cast Version)
- 63% replays Rain Sounds — Rain On the Window
- 63% replays Rihanna — Cheers (Drink To That)
- 60% replays Group 1 Crew — He Said (feat. Chris August)
- 59% replays Karsten Glück Simone Sommerland — Schlaflied für Anne
- 56% replays Monica — With You
- 54% replays Jessie Ware — Wildest Moments
- 53% replays Tim McGraw — I Like It, I Love It
- 53% replays Rain Sounds — Morning Rain In Sedona
- 52% replays Rain Sounds — Rain Sounds
It is no surprise that the list is dominated by background noise. There’s nothing like ambient ocean waves or rain sounds to help baby go to sleep in the noisy city. A five minute track of ambient white noise may be played dozens of times during every nap. It is not uncommon to find 8 hour long stretches of the same five minute white noise audio track played on auto repeat.
The top most replayed song is Reckless by Eric West from the ‘shamelessly sentimental’ 2012 movie Playing for Keeps (4% rotten). 86% of the time this song is played it is a replay. This is the song that you can’t listen to just once. It is the Lays potato chip of music. Beware, if you listen to it, you may be caught in its web and you’ll never be able to escape. Listen at your own risk:
Luckily, most people don’t listen to this song even once. It is only part of the regular listening rotation of a couple hundred listeners. Still, it points to a pattern that we’ll see more of – overly sentimental music has high replay value.
Top Replayed Popular Songs
Perhaps even more interesting is to look at the top most replayed popular songs. We can do this by restricting the songs in the results to those that are by artists that have a significant fan base:
- 31% replays Miley Cyrus — The Climb
- 16% replays August Alsina — I Luv This sh*t featuring Trinidad James
- 15% replays Brad Paisley — Whiskey Lullaby
- 14% replays Tamar Braxton — The One
- 14% replays Chris Brown — Love More
- 14% replays Anna Kendrick — Cups (Pitch Perfect’s “When I’m Gone”)
- 13% replays Avenged Sevenfold — Hail to the King
- 13% replays Jay-Z — Big Pimpin’
- 13% replays Labrinth — Beneath Your Beautiful
- 13% replays Karmin — Acapella
- 12% replays Lana Del Rey — Summertime Sadness [Lana Del Rey vs. Cedric Gervais]
- 12% replays MGMT — Electric Feel
- 12% replays One Direction — Best Song Ever
- 12% replays Big Sean — Beware featuring Lil Wayne, Jhené Aiko
- 12% replays Chris Brown — Don’t Think They Know
- 11% replays Justin Bieber — Boyfriend
- 11% replays Avicii — Wake Me Up
- 11% replays 2 Chainz — Feds Watching featuring Pharrell
- 10% replays Paramore — Still Into You
- 10% replays Alicia Keys — Fire We Make
- 10% replays Lorde — Royals
- 10% replays Miley Cyrus — We Can’t Stop
- 10% replays Ciara — Body Party
- 9% replays Marc Anthony — Vivir Mi Vida
- 9% replays Ellie Goulding — Burn
- 9% replays Fantasia — Without Me
- 9% replays Rich Homie Quan — Type of Way
- 9% replays The Weeknd — Wicked Games (Explicit)
- 9% replays A$AP Ferg — Work REMIX
- 9% replays Jay-Z — Part II (On The Run) featuring Beyoncé
It is hard to believe, but the data doesn’t lie – More than 30% of the time after someone listens to Miley Cyrus’s The Climb they listen to it again right away – proving that there is indeed always going to be another mountain that you are going to need to climb. Miley Cyrus is well represented – her aptly named song We can’t Stop is the most replayed song of the top ten most popular songs.
Here are the top 30 most replayed popular songs in Spotify and Rdio playlists for you to enjoy, but I’m sure you’ll never get to the end of the playlist, you’ll just get stuck repeating The Best Song Ever or Boyfriend forever.
Here’s the Rdio version of the Top 30 Most Replayed popular songs:
Most Manually Replayed
More than once I’ve come back from lunch to find that I left my music player on auto repeat and it has played the last song 20 times while I was away. The song was playing, but no one was listening. It is more interesting to find songs replays in which the replay is manually initiated. These are the songs that grabbed the attention of the listener enough to make them interact with their player and actually queue the song up again. We can find manually replayed songs by looking at replay timestamps. Replays generated by autorepeat will have a very regular timestamp delta, while manual replay timestamps will have more random delta between timestamps.
Here are the top manually replayed songs:
- Body Party by Ciara
- Still Into You by Paramore
- Tapout featuring Lil Wayne, Birdman, Mack Maine, Nicki Minaj, Future by Rich Gang
- Part II (On The Run) featuring Beyoncé by Jay-Z
- Feds Watching featuring Pharrell by 2 Chainz
- Royals by Lorde
- V.S.O.P. by K. Michelle
- Just Give Me A Reason by Pink
- Don’t Think They Know by Chris Brown
- Wake Me Up by Avicii
There’s an Rdio playlist of these songs: Most Manually Replayed
Why do we care which songs are most replayed? It’s part of our never ending goal to try to better understand how people interact with music. For instance, recognizing when music is being used in a context like helping the baby go to sleep is important – without taking this context into account, the thousands of plays of Ocean Waves and Warn Rain would dominate the taste profile that we build for that new mom and dad. We want to make sure that when that mom and dad are ready to listen to music, we can recommend something besides white noise.
Looking at replays can help us identify new artists for certain audiences. For instance, parents looking for an alternative to Miley Cyrus for their pre-teen playlists after Miley’s recent VMA performance, may look to an artist like Fifth Harmony. Their song Miss Movin’ On has similar replay statistics to the classic Miley songs:
Finally, looking at replays is another tool to help us understand the music that people really like. If the neighbors play Rock Lobster 20 times in a row, you can be sure that they really, really like that song. (And despite, or perhaps because of, that night 30 years ago, I like the song too). You should give it a listen, or two…
Top SXSW Music Panels for music exploration, discovery and interaction
SXSW 2014 PanelPicker has opened up. I took a tour through the SXSW Music panel proposals to highlight the ones that are of most interest to me … typically technical panels about music discovery and interaction. Here’s the best of the bunch. You’ll notice a number of Echo Nest oriented proposals. I’m really not shilling, I genuinely think these are really interesting talks (well, maybe I’m shilling for my talk).
I’ve previously highlighted the best the bunch for SXSW Interactive.
A Genre Map for Discovering the World of Music
All the music ever made (approximately) is a click or two away. Your favorite music in the world is probably something you’ve never even heard of yet. But which click leads to it?
Most music “discovery” tools are only designed to discover the most familiar thing you don’t already know. Do you like the Dave Matthews Band? You might like O.A.R.! Want to know what your friends are listening to? They’re listening to Daft Punk, because they don’t know any more than you. Want to know what’s hot? It’s yet another Imagine Dragons song that actually came out in 2012. What we NEED are tools for discovery through exploration, not dictation.
This talk will provide a manic music-discovery demonstration-expedition, showcasing how discovery through exploration (The Echo Nest Discovery list & the genre mapping experiment, Every Noise at Once) in the new streaming world is not an opportunity to pay different people to dictate your taste, but rather a journey, unearthing new music JUST FOR YOU.
The Predictive Power of Music
Music taste is extremely personal and an important part of defining and communicating who we are.
Musical Identity, understanding who you are as a music fan and what that says about you, has always been a powerful indicator of other things about you. Broadcast radio’s formats (Urban, Hot A/C, Pop, and so on) are based on the premise that a certain type of music attracts a certain type of person. However, the broadcast version of Musical Identity is a blunt instrument, grouping millions of people into about 12 audience segments. Now that music has become a two-way conversation online, Musical Identity can become considerably more precise, powerful, and predictive.
In this talk, we’ll look at why music is one of the strongest predictors and how music preference can be used to make predictions about your taste in other forms of entertainment (books, movies, games, etc).
Your Friends Have Bad Taste: Fixing Social Music
Music is the most social form of entertainment consumption, but online music has failed to deliver truly social & connected music experiences. Social media updates telling you your aunt listened to Hall and Oates doesn’t deliver on the promise of social music. As access-based, streaming music becomes more mainstream, the current failure & huge potential of social music is becoming clearer. A variety of app developers & online music services are working to create experiences that use music to connect friends & use friends to connect you with new music you’ll love. This talk will uncover how to make social music a reality.
Anyone Can Be a DJ: New Active Listening on Mobile
The mobile phone has become the de facto device for accessing music. According to a recent report, the average person uses their phone as a music player 13 times per day. With over 30 million songs available, any time, any place, listening is shifting from a passive to a personalized and interactive experience for a highly engaged audience.
New data-powered music players on sensor-packed devices are becoming smarter, and could enable listeners to feel more like creators (e.g. Instagram) by dynamically adapting music to its context (e.g. running, commuting, partying, playing). A truly personalized pocket DJ will bring music listening, discovery, and sharing to an entirely new level.
In this talk, we’ll look at how data-enhanced content and smarter mobile players will change the consumer experience into a more active, more connected, and more engaged listening experience.
Human vs. Machine: The Music Curation Formula
Recreating human recommendations in the digital sphere at scale is a problem we’re actively solving across verticals but no one quite has the perfect formula. The vertical where this issue is especially ubiquitous is music. Where we currently stand is solving the integration of human data with machine data and algorithms to generate personalized recommendations that mirrors the nuances of human curation. This formula is the holy grail.
Algorithmic, Curated & Social Music Discover
As the Internet has made millions of tracks available for instant listening, digital music and streaming companies have focused on music recommendations and discovery. Approaches have included using algorithms to present music tailored to listeners’ tastes, using the social graph to find music, and presenting curated & editorial content. This panel will discuss the methods, successes and drawbacks of each of these approaches. We will also discuss the possibility of combining all three approaches to present listeners with a better music discovery experience, with on-the-ground stories of the lessons from building a Discover experience at Spotify.
Beyond the Play Button – The Future of Listening (This is my talk)
35 years after the first Sony Walkman shipped, today’s music player still has essentially the same set of controls as that original portable music player. Even though today’s music player might have a million times more music than the cassette player, the interface to all of that music has changed very little.
In this talk we’ll explore new ways that a music listener can interact with their music. First we will explore the near future where your music player knows so much about you, your music taste and your current context that it plays the right music for you all the time. No UI is needed.
Next, we’ll explore a future where music listening is no longer a passive experience. Instead of just pressing the play button and passively listening you will be able to jump in and interact with the music. Make your favorite song last forever, add your favorite drummer to that Adele track or unleash your inner Skrillex and take total control of your favorite track.
5 Years of Music Hack Day
Started in 2009 by Dave Haynes and James Darling, Music Hack Day has become the gold standard of music technology events. Having grown to a worldwide, monthly event that has seen over 3500 music hacks created in over 20 cities the event is still going great guns. But, what impact has this event had on the music industry and it’s connection with technology? This talk looks back at the first 5 years of Music Hack Day, from it’s origins to becoming something more important and difficult to control than it’s ‘adhocracy’ beginnings. Have these events really impacted the industry in a positive way or have the last 5 years simply seen a maturing attitude towards technologies place in the music industry? We’ll look at the successes, the hacks that blew people’s minds and what influence so many events with such as passionate audience have had on changing the relationship between music and tech.
The SXSW organizers pay attention when they see a panel that gets lots of votes, so head on over and make your opinion be known.
Music Tech Meetup in Dublin
A bunch of music tech folk will be in Dublin Ireland next week to attend ACM Recommender Systems 2012. We’ll be heading over to the Bull & Castle, beside Christ Church, Dublin City on September 13 at 18:30 to join <Pub> Standands Dublin, to hang out and chat about hacking music. Pub Standards is a post-conference drink-up without the conference. There’s no format, talks or presentations. It’s just geeks + beer. If you are in the area and are interested in hanging out, feel free to come on down and have a beer.
Austin climbing the Most Musical City chart with a bullet!
I’ve received quite a bit of feedback on my recent Most Musical City post, especially from folks from Austin that didn’t like Austin’s 14th place ranking. This reddit/austin comment thread was rather brutal, and this Austinist article Wait, What?! Austin Not Ranked In Top 10 Musical Cities List even closed with this appeal: any data analysts out there up for the challenge to get Austin closer to the top?
Well, John Rees, the Director of Community & Economic Development at Capital Area Council of Governments in Austin is just the data analyst that the Austinist was looking for. He re-ran the analysis but instead of using city populations he calculated the rankings based upon metropolitan statistical areas. In the May issue of Data Points Newsletter John reports on this analysis:
When data from The Echo Nest is adjusted to include metropolitan statistical area population data, the rankings of America’s most musical places changes significantly. Topping the list is Nashville, San Francisco and Los Angeles (which includes Beverly Hills). The Austin region jumps ten places from the original list to become America’s forth [sic] most musical region.
John goes on to point out some of the non-quantifiable aspects of the Austin music scene such as the diversity of music as well as the presence of events such as SXSW and Austin City Limits. John makes a strong argument that Austin is one of the country’s premier music destinations. Even the reaction of Austin’s residents to my post says a lot about Austin as a music city. People from Austin really care about music and don’t take it kindly when they are not at the top of the most musical city list. So congrats to Austin, not just for moving up the chart but also for demonstrating that Austin is the city that is most passionate about music