Archive for category data
Austin climbing the Most Musical City chart with a bullet!
I’ve received quite a bit of feedback on my recent Most Musical City post, especially from folks from Austin that didn’t like Austin’s 14th place ranking. This reddit/austin comment thread was rather brutal, and this Austinist article Wait, What?! Austin Not Ranked In Top 10 Musical Cities List even closed with this appeal: any data analysts out there up for the challenge to get Austin closer to the top?
Well, John Rees, the Director of Community & Economic Development at Capital Area Council of Governments in Austin is just the data analyst that the Austinist was looking for. He re-ran the analysis but instead of using city populations he calculated the rankings based upon metropolitan statistical areas. In the May issue of Data Points Newsletter John reports on this analysis:
When data from The Echo Nest is adjusted to include metropolitan statistical area population data, the rankings of America’s most musical places changes significantly. Topping the list is Nashville, San Francisco and Los Angeles (which includes Beverly Hills). The Austin region jumps ten places from the original list to become America’s forth [sic] most musical region.
John goes on to point out some of the non-quantifiable aspects of the Austin music scene such as the diversity of music as well as the presence of events such as SXSW and Austin City Limits. John makes a strong argument that Austin is one of the country’s premier music destinations. Even the reaction of Austin’s residents to my post says a lot about Austin as a music city. People from Austin really care about music and don’t take it kindly when they are not at the top of the most musical city list. So congrats to Austin, not just for moving up the chart but also for demonstrating that Austin is the city that is most passionate about music
What is the most musical city in the United States?
Posted by Paul in code, data, fun, The Echo Nest on May 20, 2012
There are many cities in the United States that are known for their music. Cities like Nashville, Detroit, Seattle and New Orleans have played a major part in the musical history and development of this country. But what is the most musical city? Which city has spawned the most musical artists? To answer this question I used the soon-to-be-released artist location data from The Echo Nest artist API. I gathered up the top 50,000 or so U.S. artists, found their city of origin and tallied the number of artists per city. From this tally I calculated the number of artists per 1,000 inhabitants in each city. The more artists per 1000 inhabitants, the more musical the city.
Using the artists per 1k inhabitants, we can easily find the top 25 most musical cities in the United States:
| # | Artists per 1,000 inhabitants | Artists | Population | City |
|---|---|---|---|---|
| 1 | 3.14 | 111 | 35355 | Beverly Hills, CA |
| 2 | 2.26 | 1651 | 732072 | San Francisco, CA |
| 3 | 1.68 | 894 | 530852 | Nashville, TN |
| 4 | 1.64 | 936 | 571281 | Boston, MA |
| 5 | 1.54 | 651 | 422908 | Atlanta, GA |
| 6 | 1.53 | 53 | 34703 | Charlottesville, VA |
| 7 | 1.48 | 817 | 552433 | Washington, DC |
| 8 | 1.39 | 513 | 367773 | Minneapolis, MN |
| 9 | 1.37 | 740 | 540513 | Portland, OR |
| 10 | 1.32 | 51 | 38601 | Burlington, VT |
| 11 | 1.24 | 4789 | 3877129 | Los Angeles, CA |
| 12 | 1.22 | 15 | 12314 | Muscle Shoals, AL |
| 13 | 1.20 | 683 | 569369 | Seattle, WA |
| 14 | 1.11 | 755 | 678368 | Austin, TX |
| 15 | 1.05 | 75 | 71253 | Bloomington, IN |
| 16 | 1.05 | 50 | 47529 | Chapel Hill, NC |
| 17 | 1.05 | 47 | 44916 | Olympia, WA |
| 18 | 1.00 | 13 | 12945 | Princeton, NJ |
| 19 | 0.95 | 182 | 190886 | Richmond, VA |
| 20 | 0.94 | 11 | 11678 | Hendersonville, NC |
| 21 | 0.87 | 12 | 13769 | Malibu, CA |
| 22 | 0.87 | 88 | 100975 | Denton, TX |
| 23 | 0.86 | 179 | 207970 | Orlando, FL |
| 24 | 0.86 | 86 | 100158 | Berkeley, CA |
| 25 | 0.85 | 114 | 133874 | Orange, CA |
I find the results to be pretty interesting. Beverly Hills, the tiny city at the heart of the entertainment world is #1. San Francisco is the most musical of all large cities, followed closely by Nashville. Among, the most musical of small cities is Muscle Shoals AL which, according to Wikipedia, is famous for its contributions to American popular music. Less musical than expected are New Orleans (rank 36), NYC (rank 37), Detroit (rank 52).
Among the least musical cities in the U.S. are my hometown (Manchester NH), with only one artist in the top 50,000 U.S. based artist for the 100K inhabitants. The least musical large city in the U.S. is Kansas City KS, with only 7 top-50k artists for their nearly half million inhabitants. Luckily Kansas City residents can drive a few miles to Kansas city Missouri (with its 194 musicians for its 442k inhabitants) when they get tired of their own seven artists.
You can see the full list of cities with population greater than 5,000 ordered by their musicality here: The Most Musical Cities in the United States. I’d love to do this for all the cities in the world, but I can’t find a good source of city population data for world cities. If you know of one let me know.
I’m rather exited about this upcoming release of artist location data in our API. It will open the doors for a whole bunch of interesting applications, such as road trip playlisters that play music by artists local to the city you are near, contextual playlisters that will favor artists from your home town, or music exploration apps that will let you explore music from a particular region of the world. I can’t wait to see what people build with this data. Stay tuned, I’ll post when the API is released.
WordPress now supports embedding of Spotify, Rdio and Gists.
WordPress now supports direct embedding of Spotify songs:
And direct embedding of Rdio songs:
And best of all Gist embeddimg!
Life is good! Thanks WordPress. All the details are here in this blog post.
Map of Music Styles
Posted by Paul in code, data, events, fun, tags, The Echo Nest, visualization on April 22, 2012
I spent this weekend at Rethink Music Hackers’ Weekend building a music hack called Map of Music Styles (aka MOMS). This hack presents a visualization of over 1000 music styles. You can pan and zoom through the music space just like you can with Google maps. When you see an interesting style of music you can click on it to hear some samples of music of that style.
It is fun to explore all the different neighborhoods of music styles. Here’s the Asian corner:
Here’s the Hip-Hop neighborhood:
And a mega-cluster of ambient/chill-out music:
To build the app, I collected the top 2,000 or so terms via The Echo Nest API. For each term I calculated the most similar terms based upon artist overlap (for instance, the term ‘metal’ and ‘heavy metal’ are often applied to the same artists and so can be considered similar, where as ‘metal’ and ‘new age’ are rarely applied to the same artist and are, therefore, not similar). To layout the graph I used Gephi (Its like Photoshop for graphs) and exported the graph to SVG. After that it was just a bit of Javascript, HTML, and CSS to create the web page that will let you pan and zoom. When you click on a term, I fetch audio that matches the style via the Echo Nest and 7Digital APIs.
There are a few non-styles that snuck through – the occasional band name, or mood, but they don’t hurt anything so I let them hang out with the real styles. The app works best in Chrome. There’s a bug in the Firefox version that I need to work out.
Give it a try and let me know how you like it: Map of Music Styles
The Spotify Play Button – a lightening demo
Spotify just released a nifty embeddable play button. With the play button you can easily embed Spotify tracks in any web page or blog. Since there’s really tight integration between the Spotify and Echo Nest IDs, I thought I’d make a quick demo that shows how we can use the Echo Nest playlist API and the new Spotify Play button to make playlists.
The demo took about 5 minutes to write (shorter than it is taking to write the blog post). It is simple artist radio on a web page. Give it a go at: Echo Nest + Spotify Play Button Demo
Here’s what it looks like.
Update: Charlie and Samuel pointed out that there is a multi-track player too. I made a demo that uses that too:
The Spotify Play button is really easy to use, looks great. Well done Spotify.
Is music getting more profane?
This post has profanity in it. If you don’t like profanity, skip this post and instead just look at this picture of a cat. Otherwise, scroll on down to read about the rise and fall of profanity in music.
Now, on to the profanity …
It seems that every year the amount of profanity in music has increased. Today it seems that every other pop song drops the f-bomb, from P!nk’s ‘Fucking Perfect’ to Cee Lo’s ‘Fuck You’. I wondered if this apparent trend was real so I took a look at when certain obscene words started to show up in song titles to see if there are any obvious trends. Here’s the data:
The word ‘fuck’ doesn’t appear in a song title until 1977 when the band ‘The Way’ released ‘Fucking Police’ . This monumental song in music history seems to be lost to the Internet age. The only evidence that this song ever existed is this MusicBrainz entry. The second song with ‘fuck’ in the title, ‘To Fuck The Boss’ by Blowfly appeared in 1978. This sophmore effort is preserved on Youtube:
The peak in usage of the word ‘fuck’ in song titles occurs in 2006 with 650 songs. Since then, peak usage has dropped off substantially, 2011 saw about the same ‘fuck’ frequency as 1999.
Usage of the word ‘shit’ has a similar profile:
The first usage of the word ‘shit’ in a song title was in 1966 in the song ‘I feel like homemade shit’ by The Fugs, which appeared on The Fugs first album (originally titled The Village Fugs Sing Ballads of Contemporary Protest, Point of Views, and General Dissatisfaction). Again the peak year of use is 2006 with 322 ‘shit’ songs that year.
Looking at these graphs, one would get the impression that use of profanity has grown substantially since the 70s and reached its peak a few years ago. However, there’s more to the data than that. Let’s look at a similar plot for a non-profane word:
This plot shows a very similar usage profile for the word ‘cat’, with substantial growth in use from the 70s until 2006 when it starts to taper off. (Yes, ‘cat’ was found in many songs before 1976, but I am not showing those in the plot). Why do ‘fuck’ and ‘cat’ have such similar profiles? It is not because their usage frequency has increased, it is because the total number of songs released has been increasing year-over-year until 2006, after which the number of new releases per year has been dropping off. We see more ‘fuck’s and ‘cat’s in 2006 because there were more songs released in 2006 than any other year. For a more accurate view we need to look at the relative usage changes. This plot shows the usage of the word ‘fuck’ relative to the usage of other words in song titles. Even when we look at the use of the word ‘fuck’ relative to other words there is a clear increasing trend.
Is music getting more profane? The answer is yes. The data show that the likelihood of a song with the word ‘fuck’ in the title has more than doubled since the 80s. And it doesn’t look like this trend has reached its peak yet. I think we shall continue to see a rise in use of language that gets a rise out of moms like Tipper Gore.
Boil the Frog
I’m at Music Apps Hack Weekend doing my favorite thing: hacking on music. I’ve just finished my hack called Boil the Frog. Boil the Frog is a Spotify App that will create playlists that gradually take you from one music style to another. It is like the proverbial story of the frog in the pot of water. If you heat the water gradually, the frog won’t notice and will happily sit in the pot until it becomes frog stew. With Boil the Frog you can do the same thing musically. Create a playlist that gradually takes your pre-teen from Miley Cyrus to Miles Davis, or perhaps more perversely the Kenny G fan to Cannibal Corpse.
To build the app I built an artist similarity graph of 100,000 of the most popular artists. I use The Echo Nest artist similarity to connect each artist to its four nearest neighbors. To find the path between any two artists I use a bidirectional Dijkstra shortest path algorithm. Most paths can be computed in less than 100ms.
The Spotify Apps API is the perfect hacking platform. You can build a Spotify app that has full access to the vast Spotify music catalog and artwork, along with access to the listener’s catalog. Since the Spotify Apps run in an embedded browser all of your web app programming skills apply. You can use jQuery, make calls to JSON APIs, use HTML 5 canvas. It is all there. Spotify has done a really good job putting together this platform. The only downside is that, unlike the web, it is hard to actually release Spotify apps, but the Spotify team is working to make this easier. I’d love to release Boil the Frog because it is really fun to make playlists that bring you from one music style to another. It is interesting to see what musical neighborhoods you wander through on your way. For instance, I made a Kenny G to Cannibal Corpse playlist. To get there, the playlist brought me from easy listening, to movie soundtracks and then through video game soundtracks to get to the heavy metal world. Cool stuff. If you want to see a playlist between two artists let me know in the comments and I’ll create and share the playlist with you.
I made a video of Boil the Frog in action. Check it out:
Update: I’ve just pushed the client code out to github: https://github.com/plamere/boilthefrog
Paul vs. Billboard
Posted by Paul in code, data, fun, The Echo Nest on February 12, 2012
Another weekend, another Music Hack Day. This weekend I’m at Tokbox headquarters in San Francisco at the 3rd annual Music Hack Day San Francisco, where 200 music hackers are building the future of music.
For my hack, I thought I would try to predict who would win the Grammy awards (the annual music awards presented by The Recording Academy) which is being held this evening. To do this, I used the Echo Nest APIs to gather of lots of news and blog posts for each nominated artist. I then peered into the articles looking for mentions of the Grammy nominated items. I tallied up the mentions and combined this with the overall artist hotttnesss to give me a ranked order of each nominated item, which I could then use to create my prediction.
Since Billboard has also made some Grammy predictions, I thought it’d be interesting to do a post-facto comparison on how well each of us predicts the winners – thus the hack title ‘Paul vs. Billboard’.
The hack is online here: Paul vs. Billboard
Be sure to check out all of the other music hacks being created this weekend:
Building a Seatwave + Echo Nest App
Posted by Paul in code, data, Music, The Echo Nest on February 8, 2012
This weekend at Music Hack Day SF, Seatwave is launching their Ticketing and Event API. This API will make it easy for developers to add event discovery and ticket-buying functionality to their apps. At the Echo Nest we’ve incorporated Seatwave artist IDs into our Rosetta ID mapping layer making it possible to use Seatwave IDs directly with the Echo Nest API. This makes it easier for you to use the Seatwave and the Echo Nest APIs together. For instance, you can call the Seatwave API, get artist event IDs in response and use those IDs with the Echo Nest API to get more context about the artist.
For example, we can make a call to the Seatwave API to get the set of Featured Contest with an API call:
The results include blocks of events like this:
{
“CategoryId”: 12,
“Currency”: “GBP”,
“Id”: 934,
“ImageURL”: “http://cdn2.seatwave.com/filestore/season/image/thestoneroses_934_1_1_20111018165906.jpg”,
“MinPrice”: 95,
“Name”: “The Stone Roses”,
“SwURL”: “http://www.seatwave.com/the-stone-roses-tickets/season”,
“TicketCount”: 1810
},
{
“CategoryId”: 10,
“Currency”: “GBP”,
“Id”: 702,
“ImageURL”: “http://cdn2.seatwave.com/filestore/season/image/redhotchilipeppers_702_1_1_20110617124457.jpg”,
“MinPrice”: 45,
“Name”: “Red Hot Chili Peppers”,
“SwURL”: “http://www.seatwave.com/red-hot-chilli-peppers-tickets/season”,
“TicketCount”: 1134
},
We see events for the Stone Roses and for RHCP. The Seatwave ID for RHCP is 702. We can use this ID directly with in Echo Nest calls. For instance, to get lots of Echo Nest info on the RHCP using the Seatwave ID, we can make an artist/profile call like so:
To show off the integration of Seatwave and Echo Nest, I’ve built a little web app that shows a list of top Seatwave concerts (generated via the Seatwave API). For each artist, the app shows the number of tickets available, the artist’s biography, along with a play button that will let you listen to a sample of the artist (via 7Digital).
The application is live here: Listen to Top Seatwave Artists. The code is on github: plamere/SWDemo
The Seatwave API is quite easy to work with. They support JSON, JSONP, XML and SOAP(bleh). Lots of good data, very nice artist images, generous affiliate program, easy to understand TOS. Highly recommended. See the Seatwave page in The Echo Nest Developer Center for more info on the Seatwave / Echo Nest integration.
Controlling the artist distribution in playlists
Posted by Paul in code, data, Music, The Echo Nest on January 12, 2012
The Echo Nest engineering team just pushed out a new feature giving you more control over the artist makeup in playlists. There is a new parameter to the playlist/static API called distribution that can be set to wandering or focused. When the distribution is set to wandering the artists will appear with approximately equal distribution in the playlist. If the distribution is set to focused artists that are more similar to the seed artists will appear more frequently. When combined with the variety parameter, you have excellent control over the number and distribution of artists in a playlist. If you want to create a playlist suitable for music discovery, create a playlist with high variety and a wandering distribution. If you want to create a playlist that more closely mimics the radio experience choose a low variety and a focused distribution.
I’ve put together a little demo that lets you create playlists with different levels of variety and distribution settings. The demo will create a playlist given a seed artist and show you the artist distribution for the playlist. Here’s the output of the demo with distribution set to focused:
You can see from the artist histogram that the playlist draws more from artists that are very similar to the seed artist (Weezer). Compare to these results from a wandering playlist with the same seed and variety:
You can see that there is flatter distribution of artists in the playlist. You can use variety and distribution to tailor playlists to the listener. For instance, you can give the Classic Rock Radio experience to a listener by setting variety to relatively low, setting the distribution to focused and seeding with a classic rock artist like Led Zeppelin. Here’s the artist distribution for the resulting playlist:
That looks like the artist rotation for my local classic rock radio.
Give the demo a try to see how you can use variety and distribution to match playlists to your listener’s taste. Then read the playlist API docs to see how to use the API to start incorporating these attributes into your apps.
The Demo: Playlist Distribution Demo (source)
