Archive for category playlist
Independence day is just a few days away so I spent a bit of time over the weekend digging into the data to see what are the most listened to songs on the Fourth of July. Eliot wrote a great piece about the data for the Spotify Blog: The Most Distinctive Fourth of July Songs in the 50 U.S. States. Here I dig in a bit deeper.
From a musical perspective, The Fourth of July is a very interesting holiday. It’s a big summer outdoor party – and the music reflects that. On the Fourth, people listen to patriotic songs, BBQ songs, popular songs, party songs, songs about place and history. People listen to wide range of genres, from rock and pop to folk, country, marches, and new weird america. To see what music people listen to on the Fourth I went through about 5,000 playlists that people have created that have ‘fourth of july’ in the title. I aggregated the songs across all of these playlists and created a playlist of the top 100 or songs.
As you can see, the playlist is a mix of US-centric music and summer party music – which is a pretty good reflection of what people actually listen to on the Fourth of July. But I wanted to go a bit further and see what music was particularly distinctive for the Fourth of July – as compared to any other summer outdoor party playlist. To do this, I collected the top most frequently appearing songs on the Fourth of July playlists and scored each song by calculating the ratio of the play counts that occurred on July 4, 2013 in the U.S. vs. play counts for the song during the following weeks. Songs that were played much more frequently on the Fourth than during the rest of July get a much higher score. Ranking songs by this ratio yields the list of distinctive Fourth of July Songs:
I think it is a pretty good list of the music that we listen to more on the Fourth of July than on any other day. There’s a John Philip Sousa march, Woody Guthrie’s This Land is Your Land, Katy Perry’s Firework, lots and lots of country music. Some artists appear more frequently than others – Bruce Springsteen, John Mellencamp and Lee Greenwood are Fourth of July favorites.
I’m always interested in regional differences in how we listen to music, so I looked at the listening in each state in the U.S. to see which of the core Fourth of July songs was listened to more. If you look just at the most popular Fourth of July song in each state the results are pretty boring – the most popular song in 46 out of 50 states for July 2013 was Party in the USA by Miley Cyrus – but if you look at the distinctive score (the ratio of plays on the fourth to plays during the following weeks), you get a more nuanced view of how people listen to music. Of course we represent this as a map:
So the real question is, are these playlists generated by data mining any good? As actual playlists to listen to, I don’t think so. They are too incoherent, with jumps that take you from a sappy country song, to a teen pop song, to a march that will give you iPod whiplash. Compare the playlists above to a human-curated Fourth of July Playlist:
This is a much more listenable playlist. However, I think the data mined playlists do provide an excellent starting point – a big pool of Fourth of July songs. With a genre and popularity filter these could be turned into decent, listenable playlists. Similarly, give this pool of Fourth of July songs to a team of music curators and they can build some pretty fantastic Fourth of July playlists for every type of music fan.
The Echo Nest knows about 800 genres of music (and that number is growing all the time). Among those 800 genres are ones that you already know about, like ‘jazz’,’rock’ and ‘classical’. But there are also hundreds of genres that you’ve probably never heard of. Genres like Filthstep, Dangdut or Skweee. Perhaps the best way to explore the genre space is via Every Noise at Once (built by Echo Nest genre-master Glenn McDonald). Every Noise At Once shows you the whole genre space, allowing you to explore the rich and varied universe of music. However, Every Noise at Once can be like drinking Champagne from a firehose – there’s just too much to take in all at once (it is, after all, every noise – at once). If you’d like to take a slower and more measured approach to learning about new music genres, you may be interested in Genre-A-Day.
Genre-A-Day is a web app that presents a new genre every day. Genre-A-Day tells you about the genre, shows you some representative artists for the genre, lets you explore similar genres, and lets you listen to music in the genre.
If you spend a few minutes every day reading about and listening to a new genre, after a few months you’ll be a much more well-rounded music listener, and after a few years your knowledge of genres will rival most musicologists’.
An easy way to make Genre-A-Day part of your daily routine is to follow @GenreADay on twitter. GenreADay will post a single tweet, once a day like so:
Under the hood – Genre-A-Day was built using the just released genre methods of The Echo Nest API. These methods allow you to get detailed info on the set of genres, the top artists for the genres, similar genres and so on. It also uses the super nifty genre presets in the playlist API that allow you to craft the genre-radio listener for someone who is new to the genre (core), for someone who is a long time listener of the genre (in rotation), or for someone looking for the newest music in that genre (emerging). The source code for Genre-A-Day is on github.
Ever since the release of the Sony Walkman 35 years ago, the play button has been the primary way we interact with music. Now the play button stands as the last barrier between a listener and their music. Read on to find out how we got here and where we are going next.
In the last 100 years, technology has played a major role in how we listen to and experience music. For instance, when I was coming of age musically, the new music technology was the Sony Walkman. With the Walkman, you could take your music with you anywhere. You were no longer tied to your living room record player to listen to your music. You no longer had to wait and hope that the DJ would play your favorite song when you were on the road. You could put your favorite songs on a tape and bring them with you and listen to them whenever you wanted to no matter where your were. The Sony Walkman really changed how we listened to music. It popularized the cassette format, which opened the door to casual music sharing by music fans. Music fans began creating mix tapes and sharing music with their friends. The playlist was reborn, music listening changed. All because of that one device.
We are once again in the middle of music+technology revolution. It started a dozen years ago with the first iPod and it continues now with devices like the iPhone combined with a music subscription service like Spotify, Rdio, Rhapsody or Deezer. Today, a music listener armed with an iPhone and a ten dollar-a-month music subscription is a couple of taps away from being able to listen to almost any song that has ever been recorded. All of this music choice is great for the music listener, but of course it brings its own problems. When I was listening to music on my Sony Walkman, I had 20 songs to choose from, but now I have millions of songs to choose from. What should I listen to next? The choices are overwhelming. The folks that run music subscription services realize that all of this choice for their listeners can be problematic. That’s why they are all working hard to add radio features like Rdio’s You.FM Personalized Radio. Personalized Radio simplifies the listening experience – instead of having to pick every song to play, the listener only needs to select one or two songs or artists and they will be presented with an endless mix of music that fits well with initial seeds.
Helping listeners pick music is especially important when you consider that not all music listeners are alike, and that most listeners are, at best, only casual music fans. A study conducted in 2003 and again in 2006 by Emap (A UK-based Advertising agency), summarized here by David Jennings, identified four main types of music listeners. Jennings describes these four main listening types as:
Savants – for whom everything in life is tied up with music
Enthusiasts – Music is a key part of life but is balanced with other interests
Casuals – Music plays a welcoming role, but other things are far more important
Indifferents – Would not lose much sleep if music ceased to exist.
These four listener categories are an interesting way to organize music listeners, but of course, real life isn’t so cut and dried. Listener categories change as life circumstances change (have a baby and you’ll likely become a much more casual music listener) and can even change based on context (a casual listener preparing for a long road-trip may act like a savant for a few days while she builds her perfect road-trip playlist).
In 2006, the distribution of people across these 4 categories was as follows:
This chart says a lot about the music world and why it works the way it does. For instance, it gives us a guide as to how much different segments of the listening world are willing to pay for music in a year. On the chart below, I’ve added my estimate of the amount of money each listener type will spend on music in a year.
Savants will spend a thousand dollars or more on vinyl, concerts, and music subscriptions. Enthusiasts will spend $100 a year on a music subscription or, perhaps, purchase a couple of new tracks per week. Casuals will spend $10 a year (maybe splurge and buy that new Beyoncé album), while Indifferents will spend nothing on music. This is why music services like Spotify and Rdio have been exploring the Fremium model. If they want to enroll the 72% of people who are Casual or Indifferent music listeners, they need a product that costs much less than the $100 a year Enthusiasts are willing to pay.
However, price isn’t the only challenge music services face in attracting the Casuals and the Indifferents. Different types of listeners have a different tolerance around the amount of time and effort it takes to play music that they want to listen to.
A music Savant – someone who lives, eats and breathes music – is happy spending hours a day poring through music blogs, forums and review sites to find new music, while the Indifferent music listener may not even make the simplest of efforts like turning the radio on or switching to a new station if they don’t like the current song. A simple metric for the time and effort spent is Interactions Per Listening Session. In this chart, I’ve added my estimate of the number of interactions, on average, a listener of a given type will tolerate to create a listening session.
Interactions per Listening Session is an indication of how many times the listener controls their music player for a listening session. That music Savant may carefully handpick each song going into a playlist after reading a few music blogs and reviews about an artist on The Hype Machine, checking out the artist bio and previewing a few tracks. The music Enthusiast may grab a few top songs from a handful of their favorite artists to build a Spotify playlist. The casual listener may fire up Pandora, select an artist station and click play, while the Indifferent music listener may passively listen to the music that is playing on the radio or in the background at the local Starbucks.
The above chart shows why a music service like Pandora has been so successful. With its simple interface, Pandora is able to better engage the Casual listeners who don’t want to spend time organizing their listening session. A Pandora listener need only pick a station, and Pandora does all the work from there. This is why music subscription services hoping to attract more users are working hard to add Pandora-like features. In order to make their service appeal to the Casuals, they need to make it incredibly easy to have a good listening experience.
But what about those Indifferents? If 40% of people are indifferent to music, is this a lost market for music services? Is it impossible to reach people who can’t even be bothered to queue up some music on Pandora? I don’t think so. Over the last 75 years, terrestrial radio has shown that even the most indifferent music fan can be coaxed into simple, “lean back” listening. Even with all of the media distractions in the world today, 92% of Americans age 12 or older listen to the radio at least weekly, much the same as it was back in 2003 (94%).
So what does it take to capture the ears of Indifferents? First, we have to drive the out-of-pocket costs to the listener to zero. This is already being done via the Freemium model – Ad supported Internet radio (non-on-demand) is becoming the standard entry point for music services. Next, and perhaps more difficult, we have to drive the number of interactions required to listen to music to zero.
Thus my current project – Zero UI – building a music player that minimizes the interactions necessary to get good music to play – a music player that can capture the attention of even the musically indifferent.
Implicit signals and context
Perhaps the biggest challenge in creating a Zero UI music player is how to get enough information about the listener to make good music choices. If a Casual or Indifferent listener can’t be bothered to explicitly tell us what kind of music they like, we have to try to figure it out based upon implicit signals. Luckily, a listener gives us all kinds of implicit signals that we can use to understand their music taste. Every time a listener adjusts the volume on the player, every time they skip a song, every time they search for an artist, or whenever they abandon a listening session, they are telling us a little bit about their music taste. In addition to the information we can glean from a listener’s implicit actions, there’s another source of data that we can use to help us understand a music listener. That’s the listener’s music listening device – i.e. their phone.
The mobile phone is now and will continue to be the primary way for people to interact with and experience music. My phone is connected to a music service with 25 million songs. It ‘knows’ in great detail what music I like and what I don’t like. It knows some basic info about me such as my age and sex. It knows where I am, and what I am doing – whether I’m working, driving, doing chores or just waking up. It knows my context – the time of day, the day of the week, today’s weather, and my schedule. It knows that I’m late for my upcoming lunch meeting and it even might even know the favorite music of the people I’m having lunch with.
Current music interfaces use very little of the extra context provided by the phone to aid in music exploration and discovery. In the Zero UI project, I’ll explore how all of this contextual information provided by the latest devices (and near future devices) can be incorporated into the music listening experience to help music listeners organize, explore, discover and manage their music listening. The goal is to create a music player that knows the best next song to play for you given your current context. No button pressing required.
There are lots of really interesting areas to explore:
Can we glean enough signal from the set of minimal listener inputs?
Which context types (user activity, location, time-of-day, etc.) are most important for scheduling music? Will we suffer from the curse of dimensionality with too many contexts?
What user demographic info is most useful for avoiding the cold start problem (age, sex, zip code)?
How can existing social data (Facebook likes, Twitter follows, social tags, existing playlists) be used to improve the listening experience?
How do we balance knowing enough about a listener to give them good music playlists and knowing so much about a listener that they are creeped out about their ‘stalker music player’?
Over the next few months I’ll be making regular posts about Zero-UI. I’ll share ideas, prototypes and maybe even some code. Feel free to follow along.
Back in 2001 when the first iPod was released, Shuffle Play was all the rage. Your iPod had your 1,000 favorite songs on it, so picking songs at random to play created a pretty good music listening experience. Today, however, we don’t have 1,000 songs in our pocket. With music services like Rdio, Rhapsody or Spotify, we are walking around with millions of songs in our pocket. I’ve often wondered what it would be like to use Shuffle Play when you have millions of songs to shuffle through. Would it be a totally horrible listening experience listening to artists that are so far down the long tail that they don’t even know that they are part of a dog? Would you suffer from terminal iPod whiplash as you are jerked between Japanese teen pop and a John Philip Sousa march?
To answer these questions, I built an app called Million Song Shuffle. This app will create a playlist by randomly selecting songs from a pool of many millions of songs. It draws from the Rdio collection and if you are an Rdio user you can hear listen to the full tracks.
The app also takes advantage of a nifty new set of data returned by the Echo Nest API. It shows you the absolute hotttnesss rank for the song and the artist, so you will always know how deep you are into the long tail (answer: almost always, very deep).
So how is listening to millions of songs at random? Surprisingly, it’s not too bad. The playlist certainly gets a high score for eclecticism and surprise, and most of the time the music is quite listenable. But give it a try, and form your own opinion.
Its fun, too, to see how long you can listen to the Million Song Shuffle before you encounter a song or even an artist that you’ve heard of before. If the artist is not in the top 5K artists, it is likely you’ve never heard of them. After listening to Million Song Shuffle for a little while you start to get an idea of how much music there is out there. There’s a lot.
For the ultimate eclectic music listening experience, try the Million Song Shuffle.
We are approaching peak Christmas music season. That means that many of us are getting really sick of hearing the same Christmas songs over and over. One can only hear Bing Crosby’s White Christmas so many times before measures must be taken. To remedy this situation, this morning I created a quick web app that let you chose from among a number of different Christmas genres (from classical to heavy metal) to let you add a little variety to your Christmas mix. If you are getting weary of the Christmas standards, but still want to listen to Christmas music, you may want to give it a try: The Christmas Playlister
For my Boston Music Hack Day hack I built Yet Another Party Playlisting App (YAPPA), because the world needed another party playlister – but really, I built it because I needed another hack, because 15 hours into the 24 hour hackathon I realized that my first hack just wasn’t going to work (more on that in another post). And so, with 9 hours left in the hack day, I thought I would try my hand at the party playlisting app.
The YAPPA is a frequently built app. In some sense one can look at the act of building a YAPPA as a hacking exercise. Just as a still life painter will practice by painting a bowl of fruit, or a pianist will practice scales, a music hacker can build their hacking muscle by creating a YAPPA.
The essential features of a YAPPA are straightforward – create a listening experience for a party based upon the tastes of the guests. Allow guests to suggest music for the party, apply some rules to select music that satisfies all the guests, and keep the music flowing.
With those features in mind, I created my party playlisting app. The interface is dead simple – guests can add music to the party via the master web interface or text the artist and song from the mobile phones to the party phone number. Once the party has started, PAPPA will keep the music flowing.
The key technology of PAPPA is how it picks the music to play next. Most YAPPAs will try to schedule music based on fairness so that everyone’s music taste is considered. Some YAPPAs also use song attributes such as song hotttnesss, song energy and danceability to make sure that the music matches the vibe of the party. PAPPA takes a very different approach to scheduling music. That’s because PAPPA takes a very different approach to parties. PAPPA doesn’t like parties. PAPPA wants everyone to go home. So PAPPA takes all of these songs that have been carefully texted to the party phone number, along with all the artist and song suggestions submitted via the web and throws them away. It doesn’t care about the music taste of the guests at the party. In fact it despises their taste (and the guests as well). Instead, PAPPA selects and plays the absolute worst music it can find. It gives the listener an endless string of the most horrible (but popular) music. Here’s a sample (the first 3 songs are bait to lure in the unwitting party guests):
- Royals by Lorde
- Levels by Avicii
- Blurred Lines by Robin Thicke
- #Twerkit featuring Nicki Minaj by Busta Rhymes
- From The Bottom Of My Broken Heart by Britney Spears
- Amigas Cheetahs by The Cheetah Girls
- Do Ya Think I’m Sexy by Paris Hilton
- Incredible by Clique Girlz
- No Ordinary Love by Jennifer Love Hewitt
- Mexican Wrestler by Emma Roberts
- I Don’t Think About It by Emily Osment
- A La Nanita Nana by The Cheetah Girls
- Don”t Let Me Be The Last To Know by Britney Spears
- Wild featuring Big Sean by Jessie J
- Heartbeat (Album Version) by Paris Hilton
- Love The Way You Love Me by The Pussycat Dolls
- When You Told Me You Loved Me by Jessica Simpson
- Jericho by Hilary Duff
- Strip by Brooke Hogan
- Pero Me Acuerdo De Tí by Christina Aguilera
- Bang Bang by Joachim Garraud
- Right Now featuring David Guetta (Sick Individuals Dub) by Rihanna
- Wilde Piraten by The Cool Kids
- Friend Lover by Electrik Red
- Betcha Can’t Do It Like Me by D4L
- Who’s That Girl by Hilary Duff
- Get In There, Frank! by Fun
- Hold It Don”t Drop It by Jennifer Lopez
- Sweet Sixteen by Hilary Duff
- Live It Up featuring Pitbull by Jennifer Lopez
- Freckles by Natasha Bedingfield
- I Want You by Paris Hilton
- Hold It Close by Fun
- Magic by The Pussycat Dolls
- How To Lose A Girl by Mitchel Musso
- Fairy Tales by JoJo
- Slow It Down featuring Fabolous (Album Version (Explicit)) by The-Dream
- Mr. Hamudah by Charles Hamilton
- Promise by Vanessa Hudgens
- Metamorphosis by Hilary Duff
How does PAPPA find the worst music in the world? It looks through all the data that The Echo Nest is collecting about how people experience music online to find the songs that have been banned frequently. When a music listener says “ban this song” they are making a pretty strong statement about the song – essentially saying, “I do not ever want to hear that song again in my life”. PAPPA finds these songs that have the highest banned-to-play ratio (i.e. the songs that have been proportionally banned the most when play count is taken into consideration) and adds them to the playlist. The result being a playlist filled with the most reviled music – with songs by Paris Hilton, Jennifer Love Hewitt and the great Emma Roberts. The perfect playlist to send your guests home.
At this moment, lets pause and listen to the song Mexican Wrestler by Emma Roberts:
What happens to all those carefully crafted text messages of songs sent by the guests? No, there’s no Twilio app catching all those messages, parsing out songs and adding them to a play queue to be scheduled. They just go to my phone. That’s so if people are not leaving the party fast enough, I can use all the phone numbers of the guests to start to text them back and tell them they should go home.
By the way, if you look at the songs that were texted to me during my two minute demo you’d realize how fruitless a YAPPA really is. There’s no possible way to make a party playlist that is going to satisfy everyone in the room. Tastes are too varied, and there’s always that guy who thinks he is clever by adding some Rick Astly to the party queue. Here’s what was texted to me during my two minute demo:
- Gregory Porter – be good
- Rebecca Black – It’s Friday
- Weird Al Yankovic – Fat
- Lady Gaga – Applause
- Weird Al Yankovic – Amish Paradise (from a different phone number from the other weird Al fan)
- boss ass bitch
- Basement Jaxx raindrops
- John Mayer your body is a wonderland
- jay z holy grail
- Underworld spikee
- wake me up
- Britney Spears – Hit Me Baby One More Time
- Slayer War Ensemble
- Bieber baby
- Ra Ra riot
- Rick Astley
- Mikey Cyrus
- Hi paul
- Stevie wonder overjoyed
Imagine trying to build a party playlist based upon those 24 input songs. Admittedly, a hackathon demo session is not a real test case for a party playlister but I still think you’d end up with a terrible mix of songs that no smart algorithm, nor any smart human, could stitch together into a playlist that would be appropriate and pleasing for a party. My guess is that if you did an A/B test for two parties, where one party played music based upon suggestions texted to a YAPPA and the other party played the top hotttest songs, the YAPPA party would always lose. I’d run this test, but that would mean I’d have to go to two parties. I hate parties, so this test will never happen. Its one of the flaws in our scientific method.
Who are the worst artists?
Looking at the PAPPA playlists I see a number of recurring artists – Britney Spears and Paris Hilton seem to be well represented. I thought it would be interesting to create a histogram of the top recurring artists in the most banned songs list. Here’s the fascinating result:
One thing I find notable about this list is the predominance of female artists. Females outnumber males by a substantial amount. Here’s some pie:
80% of the most banned artists are female. A stunning result. There’s something going on here. Someone suggested that the act of banning a song is an aggressive act that may skew male, and many of these aggressively banning males don’t like to listen to female artists. More study is needed here. It may involve parties, so I’m out.
Wrapping it all up
I enjoyed creating my PAPPA YAPPA. Demoing it was really fun and the audience seemed to enjoy the twist ending. The patterns in the data underlying the app are pretty interesting too. Why are so many banned songs by female artists?
If you are having your own party and want to use PAPPA to help enhance the party you can go to:
Just replace the phone number in the URL with your own and you are good to go.
On Friday evening at the Tufts hack I made a little Python script that makes playlists with an acrostic messages embedded in them. I enjoyed the hack so much that I spent a few hours turning it into a web app. This means that you don’t have to be a Pythonista to generate your own acrostic playlists.
The app, called Acrostic Playlist Maker, lets you select from a handful of genres and type in your ‘secret’ message.
When you hit the button it will generate a playlist where the first letter of each song in the playlist spells out the message.
You can listen to the music in the playlist by clicking on any song, and you can save the playlist back to Rdio.
Anyone who works in music tech has probably been called upon to ‘do the music’ at some social event. Now with the Acrostic Playlist Maker you can can make those playlists, while secretly expressing how you really feel.