Archive for category code
In my recent regional listening preferences post I published a map that showed the distinctive artists by state. The map was rather popular, but unfortunately was a source of confusion for some who thought that the map was showing the favorite artist by state. A few folks have asked what the map of favorite artists per state would look like and how it would compare to the distinctive map. Here are the two maps for comparison.
Favorite Artists by State
This map shows the most played artist in each state over the last year. It is interesting to see the regional differences in favorite artists and how just a handful of artists dominates the listening of wide areas of the country.
Most Distinctive Artists by State
This is the previously published map that shows the artists that are listened to proportionally more frequently in a particular state than they are in all of the United States.
The data for both maps is drawn from an aggregation of data across a wide range of music services powered by The Echo Nest and is based on the listening behavior of a quarter million online music listeners.
It is interesting to see that even when we consider just the most popular artists, we can see regionalisms in listening preferences. I’ve highlighted the regions with color on this version of the map:
Favorite Artist Regions
My hack at the MIDEM Music Hack Day this year is what I’d call a Creative Hack. I built it, not because it answered any business use case or because it demonstrated some advanced capability of some platform or music tech ecosystem, I built it because I was feeling creative and I wanted to express my creativity in the best way that I can which is to write a computer program. The result is something I’m particularly proud of. It’s a dynamic visualization of the song Burn by Ellie Goulding. Here’s a short, low-res excerpt, but I strongly suggest that you go and watch the full version here: Cannes Burn
Unlike all of the other hacks that I’ve built, this one feels really personal to me. I wasn’t just trying to solve a technical problem. I was trying to capture the essence of the song in code, trying to tell its story and maybe even touch the viewer. The challenge wasn’t in the coding it was in the feeling.
After every hack day, I’m usually feeling a little depressed. I call it post-hacking depression. It is partially caused by being sleep deprived for 48 hours, but the biggest component is that I’ve put my all into something for 48 hours and then it is just over. The demo is done, the code is checked into github, the app is deployed online and people are visiting it (or not). The thing that just totally and completely took over my life for two days is completely gone. It is easy to reflect back on the weekend and wonder if all that time and energy was worth it.
Monday night after the MIDEM hack day was over I was in the midst of my post-hack depression sitting in a little pub called Le Crillon when a guy came up to me and said “I saw your hack. It made me feel something. Your hack moved me.”
Cannes Burn won’t be my post popular hack, nor is it my most challenging hack, but it may be my favorite hack because I was able to write some code and make somebody that I didn’t know feel something.
Today at the Echo Nest we are pushing out an update to our Genre APIs. The new APIs lets you get all sorts of information about any of over 800 genres including a description of the genre, representative artists in the genre, similar genres, and links to web resources for the genre (such as a wikipedia page, if one exists for a genre). You can also use the genres to create various types of playlists. With these APIs you build all sorts of music exploration apps like Every Noise At Once, Music Popcorn and Genre-A-Day.
The new APIs are quite simple to use. Here are a few python examples created using pyen.
List all of the available genres with a description
This outputs text like so:
We can get the top artists for any genre like so:
Here are the top artists for ‘cool jazz’
We can find similar genres to any genre with this bit of code:
% python sim_genres.py cool jazz bebop jazz hard bop contemporary post-bop soul jazz big band jazz christmas stride jazz funk jazz fusion avant-garde jazz free jazz
We can use the genres to create excellent genre playlists. To do so, create a playlist of type ‘genre-radio’ and give the genre name as a seed. We’ve also added a new parameter called ‘genre_preset’ that, if specified will control the type of songs that will be added to the playlist. You can chose from core, in_rotation, and emerging. Core genre playlists are great for introducing a new listener to the genre. Here’s a bit of code that generates a core playlist for any genre:
The core classic rock playlist looks like this:
- Simple Man by Lynyrd Skynyrd
- Born To Be Wild by Steppenwolf
- All Along The Watchtower by Jimi Hendrix
- Kashmir by Led Zeppelin
- Sunshine Of Your Love by Cream
- Let’s Work Together by Canned Heat
- Gimme Shelter by The Rolling Stones
- It’s My Life by The Animals
- 30 Days In The Hole by Humble Pie
- Midnight Rider by The Allman Brothers Band
- The Joker by Steve Miller Band
- Fortunate Son by Creedence Clearwater Revival
- Black Betty by Ram Jam
- Heart Full Of Soul by The Yardbirds
- Light My Fire by The Doors
The ‘in rotation’ classic rock playlist looks like this:
- Heaven on Earth by Boston
- Doom And Gloom by The Rolling Stones
- Little Black Submarines by The Black Keys
- I Gotsta Get Paid by ZZ Top
- Fly Like An Eagle by Steve Miller Band
- Blue On Black by Kenny Wayne Shepherd
- Driving Towards The Daylight by Joe Bonamassa
- When A Blind Man Cries by Deep Purple
- Over and Over (Live) by Joe Walsh
- The Best Is Yet To Come by Scorpions
- World Boss by Gov’t Mule
- One Way Out by The Allman Brothers Band
- Corned Beef City by Mark Knopfler
- Bleeding Heart by Jimi Hendrix
- My Sharona by The Knack
While the emerging ‘classic rock’ playlist looks like this:
- If You Were in Love by Boston
- Beggin’ by Shocking Blue
- Speak Now by The Answer
- Mystic Highway by John Fogerty
- Hell Of A Season by The Black Keys
- No Reward by Gov’t Mule
- Pretty Wasted by Tito & Tarantula
- The Battle Of Evermore by Page & Plant
- I Got All You Need by Joe Bonamassa
- What You Gonna Do About Me by Buddy Guy
- I Used To Could by Mark Knopfler
- Wrecking Ball by Joe Walsh
- The Circle by Black Country Communion
- You Could Have Been a Lady by April Wine
- 15 Lonely by Walter Trout
The new Genre APIs are really quite fun to use. I’m looking forward to seeing a whole new world of music exploration and discovery apps built around these APIs.
The Echo Nest knows about 800 genres of music (and that number is growing all the time). Among those 800 genres are ones that you already know about, like ‘jazz’,'rock’ and ‘classical’. But there are also hundreds of genres that you’ve probably never heard of. Genres like Filthstep, Dangdut or Skweee. Perhaps the best way to explore the genre space is via Every Noise at Once (built by Echo Nest genre-master Glenn McDonald). Every Noise At Once shows you the whole genre space, allowing you to explore the rich and varied universe of music. However, Every Noise at Once can be like drinking Champagne from a firehose – there’s just too much to take in all at once (it is, after all, every noise – at once). If you’d like to take a slower and more measured approach to learning about new music genres, you may be interested in Genre-A-Day.
Genre-A-Day is a web app that presents a new genre every day. Genre-A-Day tells you about the genre, shows you some representative artists for the genre, lets you explore similar genres, and lets you listen to music in the genre.
If you spend a few minutes every day reading about and listening to a new genre, after a few months you’ll be a much more well-rounded music listener, and after a few years your knowledge of genres will rival most musicologists’.
An easy way to make Genre-A-Day part of your daily routine is to follow @GenreADay on twitter. GenreADay will post a single tweet, once a day like so:
Under the hood - Genre-A-Day was built using the just released genre methods of The Echo Nest API. These methods allow you to get detailed info on the set of genres, the top artists for the genres, similar genres and so on. It also uses the super nifty genre presets in the playlist API that allow you to craft the genre-radio listener for someone who is new to the genre (core), for someone who is a long time listener of the genre (in rotation), or for someone looking for the newest music in that genre (emerging). The source code for Genre-A-Day is on github.
For my Christmas vacation programming project this year, I revisited an old hack: Six Degrees of Black Sabbath. I wrote the original, way back in 2010 at the very first San Francisco Music Hack Day. That version is still up and running, and getting regular visits, but it is getting a bit long in the tooth and so I’ve given it a total rewrite from the ground up. The result is the new Six Degrees of Black Sabbath:
Six Degrees of Black Sabbath is like the Oracle of Bacon but for music. It lets you find connections to just about any two artists based upon their collaborations. Type in the name of two artists, and 6dobs will give you a pathway showing the connections that will get you from one artist to another. For instance, if you enter ‘The Beatles’ and ‘Norah Jones’ you’ll get a path like:
- We start with The Beatles
- The Beatles had member George Harrison
- George Harrison performed with Ravi Shankar on the song Bangla Dhun and 26 others.
- Ravi Shankar was parent of Norah Jones
If you don’t like a particular connection, you can bypass it generating a new path. For instance, if we bypass Ravi Shankar, it will take us eight steps to get to Norah Jones from the Beatles:
The Beatles -> Paul McCartney -> The Fireman -> Youth -> Pigface
-> Mike Dillon ->Garage A Trois -> Charlie Hunter -> Norah Jones
Not all connections are created equal. Mick Jagger and Keith Richards have been playing together for over 50 years in the Rolling Stones. That’s a much stronger connection than the one between Mick Jagger and Fergie for performing a single song together at the Rock and Roll Hall of Fame. We take these connection strengths into account when finding paths between artists. Preference is given to stronger connections, even if those stronger connections will yield a longer path.
The new version of Six Degrees of Black Sabbath has a number of new features:
Video - Each step in a path is represented by a Youtube video – often with a video by the two artists that represent that step. I’m quite pleased at how well the video works for establishing the connection between two artists. Youtube seems to have it all.
Live stats - The app tracks and reports all sorts of things such as the longest path discovered so far, the most frequently occurring artists on paths, the most connected artists, most searched for artists and so on.
Larger database of connections - the database has about a quarter million artists and 2.5 million artist-to-artist connections.
Autocomplete for artist names - no need to try to remember how to spell ‘Britney Spears‘ – just start typing the parts you know and it will sort it out.
Spiffier looking UI - It still looks like it was designed by an engineer, but at least it looks like it was designed in this decade by an engineer.
Path finding improvements - faster and better paths throughout.
Revisiting this app after 4 years was a lot of fun. I got to dive deep into a bunch of tech that was new to me including Redis, Bootstrap 3, and the YouTube video search API. I spent many hours untangling the various connections in the new Musicbrainz schema. I took a tour through a number of Pythonic network graph libraries (Networkx, igraph and graph-tool), I learned a lot about Python garbage collection when you have a 2.5gb heap.
Give the app a try and let me know what you think.
Back in 2001 when the first iPod was released, Shuffle Play was all the rage. Your iPod had your 1,000 favorite songs on it, so picking songs at random to play created a pretty good music listening experience. Today, however, we don’t have 1,000 songs in our pocket. With music services like Rdio, Rhapsody or Spotify, we are walking around with millions of songs in our pocket. I’ve often wondered what it would be like to use Shuffle Play when you have millions of songs to shuffle through. Would it be a totally horrible listening experience listening to artists that are so far down the long tail that they don’t even know that they are part of a dog? Would you suffer from terminal iPod whiplash as you are jerked between Japanese teen pop and a John Philip Sousa march?
To answer these questions, I built an app called Million Song Shuffle. This app will create a playlist by randomly selecting songs from a pool of many millions of songs. It draws from the Rdio collection and if you are an Rdio user you can hear listen to the full tracks.
The app also takes advantage of a nifty new set of data returned by the Echo Nest API. It shows you the absolute hotttnesss rank for the song and the artist, so you will always know how deep you are into the long tail (answer: almost always, very deep).
So how is listening to millions of songs at random? Surprisingly, it’s not too bad. The playlist certainly gets a high score for eclecticism and surprise, and most of the time the music is quite listenable. But give it a try, and form your own opinion.
Its fun, too, to see how long you can listen to the Million Song Shuffle before you encounter a song or even an artist that you’ve heard of before. If the artist is not in the top 5K artists, it is likely you’ve never heard of them. After listening to Million Song Shuffle for a little while you start to get an idea of how much music there is out there. There’s a lot.
For the ultimate eclectic music listening experience, try the Million Song Shuffle.
We are approaching peak Christmas music season. That means that many of us are getting really sick of hearing the same Christmas songs over and over. One can only hear Bing Crosby’s White Christmas so many times before measures must be taken. To remedy this situation, this morning I created a quick web app that let you chose from among a number of different Christmas genres (from classical to heavy metal) to let you add a little variety to your Christmas mix. If you are getting weary of the Christmas standards, but still want to listen to Christmas music, you may want to give it a try: The Christmas Playlister
This weekend, I’ve been in London, attending the London Music Hack Day. For this weekend’s hack, I was inspired by daughter’s music listening behavior – when she listens to music, she is good for the first verse or two and the chorus, but after that, she’s on to the next song. She probably has never heard a bridge. So for my daughter, and folks like her with short attention spans, I’ve built Attention Deficit Radio. ADR creates a Pandora-like radio station based upon a seed artist, but doesn’t bother you by playing whole songs. Instead, after about 30 seconds or so, it is on to the next song. The nifty bit is that ADR will try to beat-match and crossfade between the songs giving you a (hopefully) seamless listening experience as you fly through the playlist. Of course those with short attention spans need something to look at while listening, so ADR has lots of gauges that show the radio status – it shows the current beat, the status of the cross-fade, tempo and song loading status.
There may be a few rough edges, and the paint is not yet dry, but give Attention Deficit Radio a try if you have a short listening attention span.
I’ve been in Helsinki this weekend (which is not in Sweden btw) for the Helsinki Music Hack Day. I wanted to try my hand at a DJ app that will allow you to dynamically and interactively mix two songs. I started with Girl Talk in a Box, ripped out the innards and made a whole bunch of neat changes:
- You can load more than one song at a time. Each song will appear as its own block of music tiles.
- You can seamlessly play tiles from either song.
- You can setup branch points to let you jump from an point in one song to any point in another (or the same) song.
- And the killer feature – you can have two active play heads allowing you to dynamically interact with two separate audio streams. The two play heads are always beat matched (the first play head is the master that sets the tempo for everyone else). You can cross-fade between the two audio streams – letting you move different parts of the song into the foreground and the background.
All the regular features of Girl Talk in a Box are retained – bookmarks, arrow key control, w/a/s/d navigation and so on. See the help for more details on the controls.
You can try the app here: Swap the Drop
For my Boston Music Hack Day hack I built Yet Another Party Playlisting App (YAPPA), because the world needed another party playlister – but really, I built it because I needed another hack, because 15 hours into the 24 hour hackathon I realized that my first hack just wasn’t going to work (more on that in another post). And so, with 9 hours left in the hack day, I thought I would try my hand at the party playlisting app.
The YAPPA is a frequently built app. In some sense one can look at the act of building a YAPPA as a hacking exercise. Just as a still life painter will practice by painting a bowl of fruit, or a pianist will practice scales, a music hacker can build their hacking muscle by creating a YAPPA.
The essential features of a YAPPA are straightforward – create a listening experience for a party based upon the tastes of the guests. Allow guests to suggest music for the party, apply some rules to select music that satisfies all the guests, and keep the music flowing.
With those features in mind, I created my party playlisting app. The interface is dead simple – guests can add music to the party via the master web interface or text the artist and song from the mobile phones to the party phone number. Once the party has started, PAPPA will keep the music flowing.
The key technology of PAPPA is how it picks the music to play next. Most YAPPAs will try to schedule music based on fairness so that everyone’s music taste is considered. Some YAPPAs also use song attributes such as song hotttnesss, song energy and danceability to make sure that the music matches the vibe of the party. PAPPA takes a very different approach to scheduling music. That’s because PAPPA takes a very different approach to parties. PAPPA doesn’t like parties. PAPPA wants everyone to go home. So PAPPA takes all of these songs that have been carefully texted to the party phone number, along with all the artist and song suggestions submitted via the web and throws them away. It doesn’t care about the music taste of the guests at the party. In fact it despises their taste (and the guests as well). Instead, PAPPA selects and plays the absolute worst music it can find. It gives the listener an endless string of the most horrible (but popular) music. Here’s a sample (the first 3 songs are bait to lure in the unwitting party guests):
- Royals by Lorde
- Levels by Avicii
- Blurred Lines by Robin Thicke
- #Twerkit featuring Nicki Minaj by Busta Rhymes
- From The Bottom Of My Broken Heart by Britney Spears
- Amigas Cheetahs by The Cheetah Girls
- Do Ya Think I’m Sexy by Paris Hilton
- Incredible by Clique Girlz
- No Ordinary Love by Jennifer Love Hewitt
- Mexican Wrestler by Emma Roberts
- I Don’t Think About It by Emily Osment
- A La Nanita Nana by The Cheetah Girls
- Don”t Let Me Be The Last To Know by Britney Spears
- Wild featuring Big Sean by Jessie J
- Heartbeat (Album Version) by Paris Hilton
- Love The Way You Love Me by The Pussycat Dolls
- When You Told Me You Loved Me by Jessica Simpson
- Jericho by Hilary Duff
- Strip by Brooke Hogan
- Pero Me Acuerdo De Tí by Christina Aguilera
- Bang Bang by Joachim Garraud
- Right Now featuring David Guetta (Sick Individuals Dub) by Rihanna
- Wilde Piraten by The Cool Kids
- Friend Lover by Electrik Red
- Betcha Can’t Do It Like Me by D4L
- Who’s That Girl by Hilary Duff
- Get In There, Frank! by Fun
- Hold It Don”t Drop It by Jennifer Lopez
- Sweet Sixteen by Hilary Duff
- Live It Up featuring Pitbull by Jennifer Lopez
- Freckles by Natasha Bedingfield
- I Want You by Paris Hilton
- Hold It Close by Fun
- Magic by The Pussycat Dolls
- How To Lose A Girl by Mitchel Musso
- Fairy Tales by JoJo
- Slow It Down featuring Fabolous (Album Version (Explicit)) by The-Dream
- Mr. Hamudah by Charles Hamilton
- Promise by Vanessa Hudgens
- Metamorphosis by Hilary Duff
How does PAPPA find the worst music in the world? It looks through all the data that The Echo Nest is collecting about how people experience music online to find the songs that have been banned frequently. When a music listener says “ban this song” they are making a pretty strong statement about the song – essentially saying, “I do not ever want to hear that song again in my life”. PAPPA finds these songs that have the highest banned-to-play ratio (i.e. the songs that have been proportionally banned the most when play count is taken into consideration) and adds them to the playlist. The result being a playlist filled with the most reviled music – with songs by Paris Hilton, Jennifer Love Hewitt and the great Emma Roberts. The perfect playlist to send your guests home.
At this moment, lets pause and listen to the song Mexican Wrestler by Emma Roberts:
What happens to all those carefully crafted text messages of songs sent by the guests? No, there’s no Twilio app catching all those messages, parsing out songs and adding them to a play queue to be scheduled. They just go to my phone. That’s so if people are not leaving the party fast enough, I can use all the phone numbers of the guests to start to text them back and tell them they should go home.
By the way, if you look at the songs that were texted to me during my two minute demo you’d realize how fruitless a YAPPA really is. There’s no possible way to make a party playlist that is going to satisfy everyone in the room. Tastes are too varied, and there’s always that guy who thinks he is clever by adding some Rick Astly to the party queue. Here’s what was texted to me during my two minute demo:
- Gregory Porter – be good
- Rebecca Black – It’s Friday
- Weird Al Yankovic – Fat
- Lady Gaga – Applause
- Weird Al Yankovic – Amish Paradise (from a different phone number from the other weird Al fan)
- boss ass bitch
- Basement Jaxx raindrops
- John Mayer your body is a wonderland
- jay z holy grail
- Underworld spikee
- wake me up
- Britney Spears – Hit Me Baby One More Time
- Slayer War Ensemble
- Bieber baby
- Ra Ra riot
- Rick Astley
- Mikey Cyrus
- Hi paul
- Stevie wonder overjoyed
Imagine trying to build a party playlist based upon those 24 input songs. Admittedly, a hackathon demo session is not a real test case for a party playlister but I still think you’d end up with a terrible mix of songs that no smart algorithm, nor any smart human, could stitch together into a playlist that would be appropriate and pleasing for a party. My guess is that if you did an A/B test for two parties, where one party played music based upon suggestions texted to a YAPPA and the other party played the top hotttest songs, the YAPPA party would always lose. I’d run this test, but that would mean I’d have to go to two parties. I hate parties, so this test will never happen. Its one of the flaws in our scientific method.
Who are the worst artists?
Looking at the PAPPA playlists I see a number of recurring artists – Britney Spears and Paris Hilton seem to be well represented. I thought it would be interesting to create a histogram of the top recurring artists in the most banned songs list. Here’s the fascinating result:
One thing I find notable about this list is the predominance of female artists. Females outnumber males by a substantial amount. Here’s some pie:
80% of the most banned artists are female. A stunning result. There’s something going on here. Someone suggested that the act of banning a song is an aggressive act that may skew male, and many of these aggressively banning males don’t like to listen to female artists. More study is needed here. It may involve parties, so I’m out.
Wrapping it all up
I enjoyed creating my PAPPA YAPPA. Demoing it was really fun and the audience seemed to enjoy the twist ending. The patterns in the data underlying the app are pretty interesting too. Why are so many banned songs by female artists?
If you are having your own party and want to use PAPPA to help enhance the party you can go to:
Just replace the phone number in the URL with your own and you are good to go.