Archive for category Music
Creative hacking
My hack at the MIDEM Music Hack Day this year is what I’d call a Creative Hack. I built it, not because it answered any business use case or because it demonstrated some advanced capability of some platform or music tech ecosystem, I built it because I was feeling creative and I wanted to express my creativity in the best way that I can which is to write a computer program. The result is something I’m particularly proud of. It’s a dynamic visualization of the song Burn by Ellie Goulding. Here’s a short, low-res excerpt, but I strongly suggest that you go and watch the full version here: Cannes Burn
[youtube http://www.youtube.com/watch?v=Fys0RGi3kA8&feature=youtu.be]Unlike all of the other hacks that I’ve built, this one feels really personal to me. I wasn’t just trying to solve a technical problem. I was trying to capture the essence of the song in code, trying to tell its story and maybe even touch the viewer. The challenge wasn’t in the coding it was in the feeling.
After every hack day, I’m usually feeling a little depressed. I call it post-hacking depression. It is partially caused by being sleep deprived for 48 hours, but the biggest component is that I’ve put my all into something for 48 hours and then it is just over. The demo is done, the code is checked into github, the app is deployed online and people are visiting it (or not). The thing that just totally and completely took over my life for two days is completely gone. It is easy to reflect back on the weekend and wonder if all that time and energy was worth it.
Monday night after the MIDEM hack day was over I was in the midst of my post-hack depression sitting in a little pub called Le Crillon when a guy came up to me and said “I saw your hack. It made me feel something. Your hack moved me.”
Cannes Burn won’t be my post popular hack, nor is it my most challenging hack, but it may be my favorite hack because I was able to write some code and make somebody that I didn’t know feel something.
Cannes Burn
This weekend brought me to Cannes and the French Riviera for the MIDEM Music Hack Day where I’ve spent about 40 hours working on my music hack called Cannes Burn. Cannes Burn is a visualization that accompanies the song Burn by Ellie Goulding. Go check it out if you haven’t already seen it before reading on. It requires a modern computer and browser that supports webgl.
The Hack uses the new ENsync.js library that I created last week. ENsync uses the Echo Nest analysis to provide synchronization of a JS web app with music. With ENsync you can setup elaborate animations that are triggered by musical events (such on every bar, beat or tatum). The Hack also uses threejs – the amazing 3D library by Mr.Doob.
Creating the hack was a whole lot of fun – I spent hours building 3D shapes out of flying cubes. I probably listened to the song Burn many hundreds of times this weekend. (Thanks to my hacker neighbors who put up with my endless Ellie looping without complaint).
It has been a great weekend here in Cannes. It is so inspiring to be surrounded by a bunch of really smart folks who are passionate about music and technology and see and hear how they are building their stuff. Such a great, sharing vibe from all of the hackers. I feel really lucky to part of it all!
Check out all the hacks on hacker league: MIDEM Music Hack Day Hacks and check out my hack at: Cannes Burn
New Genre APIs
Posted by Paul in code, Music, The Echo Nest, web services on January 16, 2014
Today at the Echo Nest we are pushing out an update to our Genre APIs. The new APIs lets you get all sorts of information about any of over 800 genres including a description of the genre, representative artists in the genre, similar genres, and links to web resources for the genre (such as a wikipedia page, if one exists for a genre). You can also use the genres to create various types of playlists. With these APIs you build all sorts of music exploration apps like Every Noise At Once, Music Popcorn and Genre-A-Day.
The new APIs are quite simple to use. Here are a few python examples created using pyen.
List all of the available genres with a description
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import pyen | |
| en = pyen.Pyen() | |
| response = en.get('genre/list', bucket=['description']) | |
| for g in response['genres']: | |
| print g['name'], '-', g['description'] |
This outputs text like so:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| a cappella – A cappella is singing without instrumental accompaniment. From the Italian for "in the manner of the chapel," a cappella may be performed solo or by a group. | |
| abstract hip hop – | |
| acid house – From house music came acid house, developed in the mid-'80s by Chicago DJs experimenting with the Roland TB-303 synthesizer. That instrument produced the subgenre's signature squelching bass, used to create a hypnotic sound. | |
| acid jazz – Acid jazz, also called club jazz, is a style of jazz that takes cues from a number of genres, including funk, hip-hop, house, and soul. | |
| … |
We can get the top artists for any genre like so:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import pyen | |
| import sys | |
| en = pyen.Pyen() | |
| if len(sys.argv) > 1: | |
| genre = ' '.join(sys.argv[1:]) | |
| response = en.get('genre/artists', name=genre) | |
| for artist in response['artists']: | |
| print artist['name'] | |
| else: | |
| print "usage: python top_artists_for_genre.py genre name" |
Here are the top artists for ‘cool jazz’
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| % python top_artists_for_genre.py cool jazz | |
| Thelonious Monk | |
| Stan Getz | |
| Lee Konitz | |
| The Dave Brubeck Quartet | |
| Bill Evans | |
| Cannonball Adderley | |
| Art Pepper | |
| Charlie Parker | |
| John Coltrane | |
| Gil Evans | |
| Ahmad Jamal | |
| Miles Davis | |
| Horace Silver | |
| Dave Brubeck | |
| Oliver Nelson |
We can find similar genres to any genre with this bit of code:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import pyen | |
| import sys | |
| en = pyen.Pyen() | |
| if len(sys.argv) > 1: | |
| genre = ' '.join(sys.argv[1:]) | |
| response = en.get('genre/similar', name=genre) | |
| for genre in response['genres']: | |
| print genre['name'] | |
| else: | |
| print "usage: python sim_genres.py genre name" |
Sample output:
% python sim_genres.py cool jazz bebop jazz hard bop contemporary post-bop soul jazz big band jazz christmas stride jazz funk jazz fusion avant-garde jazz free jazz
We can use the genres to create excellent genre playlists. To do so, create a playlist of type ‘genre-radio’ and give the genre name as a seed. We’ve also added a new parameter called ‘genre_preset’ that, if specified will control the type of songs that will be added to the playlist. You can chose from core, in_rotation, and emerging. Core genre playlists are great for introducing a new listener to the genre. Here’s a bit of code that generates a core playlist for any genre:
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import pyen | |
| import sys | |
| en = pyen.Pyen() | |
| if len(sys.argv) < 2: | |
| print 'Usage: python genre_playlist.py seed genre name' | |
| else: | |
| genre = ' '.join(sys.argv[1:]) | |
| response = en.get('playlist/static', type='genre-radio', genre_preset='core-best', genre=genre) | |
| for i, song in enumerate(response['songs']): | |
| print "%d %s by %s" % ((i +1), song['title'], song['artist_name']) |
The core classic rock playlist looks like this:
- Simple Man by Lynyrd Skynyrd
- Born To Be Wild by Steppenwolf
- All Along The Watchtower by Jimi Hendrix
- Kashmir by Led Zeppelin
- Sunshine Of Your Love by Cream
- Let’s Work Together by Canned Heat
- Gimme Shelter by The Rolling Stones
- It’s My Life by The Animals
- 30 Days In The Hole by Humble Pie
- Midnight Rider by The Allman Brothers Band
- The Joker by Steve Miller Band
- Fortunate Son by Creedence Clearwater Revival
- Black Betty by Ram Jam
- Heart Full Of Soul by The Yardbirds
- Light My Fire by The Doors
The ‘in rotation’ classic rock playlist looks like this:
- Heaven on Earth by Boston
- Doom And Gloom by The Rolling Stones
- Little Black Submarines by The Black Keys
- I Gotsta Get Paid by ZZ Top
- Fly Like An Eagle by Steve Miller Band
- Blue On Black by Kenny Wayne Shepherd
- Driving Towards The Daylight by Joe Bonamassa
- When A Blind Man Cries by Deep Purple
- Over and Over (Live) by Joe Walsh
- The Best Is Yet To Come by Scorpions
- World Boss by Gov’t Mule
- One Way Out by The Allman Brothers Band
- Corned Beef City by Mark Knopfler
- Bleeding Heart by Jimi Hendrix
- My Sharona by The Knack
While the emerging ‘classic rock’ playlist looks like this:
- If You Were in Love by Boston
- Beggin’ by Shocking Blue
- Speak Now by The Answer
- Mystic Highway by John Fogerty
- Hell Of A Season by The Black Keys
- No Reward by Gov’t Mule
- Pretty Wasted by Tito & Tarantula
- The Battle Of Evermore by Page & Plant
- I Got All You Need by Joe Bonamassa
- What You Gonna Do About Me by Buddy Guy
- I Used To Could by Mark Knopfler
- Wrecking Ball by Joe Walsh
- The Circle by Black Country Communion
- You Could Have Been a Lady by April Wine
- 15 Lonely by Walter Trout
The new Genre APIs are really quite fun to use. I’m looking forward to seeing a whole new world of music exploration and discovery apps built around these APIs.
Must listen to podcast – Digital Music Trends
My top ‘must-listen’ podcast is now Andrea Leonelli’s Digital Music Trends. This weekly podcast is jam packed with analysis and info about what’s going on in the digital music space. The latest podcast includes Music Hack Day master Martyn Davies and Ben Graham from StrategyEye in a round table discussion on Gracenote’s recent sale, their Rhythm announcment, CES and more. Good stuff.
Six Degrees of Black Sabbath (v2)
Posted by Paul in code, fun, Music, The Echo Nest on January 6, 2014
For my Christmas vacation programming project this year, I revisited an old hack: Six Degrees of Black Sabbath. I wrote the original, way back in 2010 at the very first San Francisco Music Hack Day. That version is still up and running, and getting regular visits, but it is getting a bit long in the tooth and so I’ve given it a total rewrite from the ground up. The result is the new Six Degrees of Black Sabbath:
Six Degrees of Black Sabbath is like the Oracle of Bacon but for music. It lets you find connections to just about any two artists based upon their collaborations. Type in the name of two artists, and 6dobs will give you a pathway showing the connections that will get you from one artist to another. For instance, if you enter ‘The Beatles’ and ‘Norah Jones’ you’ll get a path like:
- We start with The Beatles
- The Beatles had member George Harrison
- George Harrison performed with Ravi Shankar on the song Bangla Dhun and 26 others.
- Ravi Shankar was parent of Norah Jones
If you don’t like a particular connection, you can bypass it generating a new path. For instance, if we bypass Ravi Shankar, it will take us eight steps to get to Norah Jones from the Beatles:
The Beatles -> Paul McCartney -> The Fireman -> Youth -> Pigface
-> Mike Dillon ->Garage A Trois -> Charlie Hunter -> Norah Jones
Not all connections are created equal. Mick Jagger and Keith Richards have been playing together for over 50 years in the Rolling Stones. That’s a much stronger connection than the one between Mick Jagger and Fergie for performing a single song together at the Rock and Roll Hall of Fame. We take these connection strengths into account when finding paths between artists. Preference is given to stronger connections, even if those stronger connections will yield a longer path.
The new version of Six Degrees of Black Sabbath has a number of new features:
Video – Each step in a path is represented by a Youtube video – often with a video by the two artists that represent that step. I’m quite pleased at how well the video works for establishing the connection between two artists. Youtube seems to have it all.
Live stats – The app tracks and reports all sorts of things such as the longest path discovered so far, the most frequently occurring artists on paths, the most connected artists, most searched for artists and so on.
Larger database of connections – the database has about a quarter million artists and 2.5 million artist-to-artist connections.
Autocomplete for artist names – no need to try to remember how to spell ‘Britney Spears‘ – just start typing the parts you know and it will sort it out.
Spiffier looking UI – It still looks like it was designed by an engineer, but at least it looks like it was designed in this decade by an engineer.
Path finding improvements – faster and better paths throughout.
Revisiting this app after 4 years was a lot of fun. I got to dive deep into a bunch of tech that was new to me including Redis, Bootstrap 3, and the YouTube video search API. I spent many hours untangling the various connections in the new Musicbrainz schema. I took a tour through a number of Pythonic network graph libraries (Networkx, igraph and graph-tool), I learned a lot about Python garbage collection when you have a 2.5gb heap.
Give the app a try and let me know what you think.
Million Song Shuffle
Back in 2001 when the first iPod was released, Shuffle Play was all the rage. Your iPod had your 1,000 favorite songs on it, so picking songs at random to play created a pretty good music listening experience. Today, however, we don’t have 1,000 songs in our pocket. With music services like Rdio, Rhapsody or Spotify, we are walking around with millions of songs in our pocket. I’ve often wondered what it would be like to use Shuffle Play when you have millions of songs to shuffle through. Would it be a totally horrible listening experience listening to artists that are so far down the long tail that they don’t even know that they are part of a dog? Would you suffer from terminal iPod whiplash as you are jerked between Japanese teen pop and a John Philip Sousa march?
To answer these questions, I built an app called Million Song Shuffle. This app will create a playlist by randomly selecting songs from a pool of many millions of songs. It draws from the Rdio collection and if you are an Rdio user you can hear listen to the full tracks.
The app also takes advantage of a nifty new set of data returned by the Echo Nest API. It shows you the absolute hotttnesss rank for the song and the artist, so you will always know how deep you are into the long tail (answer: almost always, very deep).
So how is listening to millions of songs at random? Surprisingly, it’s not too bad. The playlist certainly gets a high score for eclecticism and surprise, and most of the time the music is quite listenable. But give it a try, and form your own opinion.
Its fun, too, to see how long you can listen to the Million Song Shuffle before you encounter a song or even an artist that you’ve heard of before. If the artist is not in the top 5K artists, it is likely you’ve never heard of them. After listening to Million Song Shuffle for a little while you start to get an idea of how much music there is out there. There’s a lot.
For the ultimate eclectic music listening experience, try the Million Song Shuffle.
Have a Very Nestive Christmas
Posted by Paul in code, Music, playlist, The Echo Nest on December 19, 2013
We are approaching peak Christmas music season. That means that many of us are getting really sick of hearing the same Christmas songs over and over. One can only hear Bing Crosby’s White Christmas so many times before measures must be taken. To remedy this situation, this morning I created a quick web app that let you chose from among a number of different Christmas genres (from classical to heavy metal) to let you add a little variety to your Christmas mix. If you are getting weary of the Christmas standards, but still want to listen to Christmas music, you may want to give it a try: The Christmas Playlister
Scary and Stretched
In the last few months I’ve found myself listening to Skrillex non-stop – usually because I’m working on some sort of Skrillexed-based hack. One thing about Skrillex – his music is quite layered, there’s lots of interesting sounds packed into every second of a song. I thought I’d explore this layering a little bit by applying Paul’s Stretch to Scary Monsters and Nice Sprites. The effect is quite pleasing.
Who are the shallowest artists?
Posted by Paul in data, Music, The Echo Nest on November 25, 2013
Yesterday, I wrote about who the Deepest Artists are. So naturally, today I’ll turn that on its head and take a look at who are the Shallowest Artists. I define a shallow artist as an artist that despite having a substantial number of released songs, has most listens concentrated in their top five tracks. These are the artists that are best known for just a small number of songs.
For each artist, I’ve calculated a Shallowness Score which is merely the percentage of an artist’s plays that occurs in an artist’s top 5 songs. A Shallowness Score of 71% means that 71% of all listens occur in the top 5 songs. Thus, 71% of all listens to Survivor (of Eye of the Tiger fame) are found in their top 5 songs.
Update: This post used to reference the Pitch Perfect Treblemakers, but Glenn points to an ambiguous artist issue with the Treblemakers where multiple artists were conflated. The Pitch Pefect Treblemakers only have 4 songs so they are no longer a candidate for this list.
Here are the top 15 Shallowest Artists. Click to see the full chart:
Click to see the full chart
As you’d expect, there are plenty of new artists on the list, artists like Icona Pop, Avicii and Zedd that have had a few charting songs. Being tagged as a shallow artist isn’t necessarily bad, it just means that your music is dominated by a handful of hits. That’s why we find Adele and Jeff Buckley on the same list as Paris Hilton and Smash Mouth.
Check out the full list of the Shallowest Artists as well as the full list of the Deepest Artists.
Who are the deepest artists?
Posted by Paul in data, fun, Music, The Echo Nest on November 24, 2013
Our playlists our filled with One Hit Wonders like My Sharona, Tainted Love and Final Countdown. One Hit wonders are the non-nutritious food of the music world – they are Twinkie’s, the Ho Hos and the Yodels of our musical diet. But what should we listen to when we want a full and nutritious musical meal? We should look for music by artists that have deeper catalogs – artists where the fans spend substantial time listening to the non-hits. These are the Deep Artists, the opposite of the One Hit Wonders – the artists that you can spend months or years listening to and exploring their collection.
Unfortunately, there’s no master list of Deep Artists – but I have lots of music listener data, so I figured I could build one. Here’s what I did. First I restricted my results to somewhat familiar artists with at least 100 songs in their catalog. I then scored each artist by the percentage of song plays that occur in the deep catalog versus the total plays for the artist – where deep catalog means a song that is not in the top ten for that artist. This gives each artist a Deepness Score that I could then use to sort artists to give us a list of the Deepest Artists. Here are the top ten:
Click for the full list
Not surprising to see Johann Sebastian Bach at number two. Bach has no real ‘hits’ – and indeed has an incredibly deep catalog. 90% of all Bach plays occur in Bach’s non-top 10. The number one deep artist is Vitamin String Quartet – they have 3500 covers of songs with no clear hits among them.
http://rd.io/x/QFqwK0FVUAYLooking at the full list we see jam bands like Phish and Grateful Dead, AOR staples like Pink Floyd and David Bowie.
I’ve built a list of a little over 500 of the Deepest Artists. These are artists that have a deepness score of 50% or greater – meaning that at least 50% of all listens for the artist is in the deeper cuts. This Thanksgiving if you are looking for some more nutritious music, stay away from Alice’s Restaurant and other One Hit Wonders and listen to music by artists on this Deep Artists list.
Update: Glenn looked at these results and felt that a nutritious music meal shouldn’t include Vitamin String Quartet (it’s the ‘artificially-fortified sugar-coated cereal of music’ according to Glenn), so Glenn took a different approach with different results. Glenn calls his results boring, but I think they are quite interesting. Read his post: Good Boring results
Music Machinery 