The Labyrinth of Genre
Posted by Paul in code, data, tags, The Echo Nest, visualization on January 16, 2011
I’m fascinated with how music genres relate to each other, especially how one can use different genres as stepping stones as a guide through the vast complexities of music. There are thousands of genres, some like rock or pop represent thousands of artists, while some like Celtic Metal or Humppa may represent only a handful of artists. Building a map by hand that represents the relationships of all of these genres is a challenge. Is Thrash Metal more closely related to Speed Metal or to Power Metal? To sort this all out I’ve built a Labyrinth of Genre that lets you explore the many genres. The Labyrinth lets you wander though about a 1000 genres, listening to samples from representative artists.
Click on a genre and the labyrinth will be expanded to show similar half a dozen similar genres and you’ll hear songs in the genre.
I built the labyrinth by analyzing a large collection of last.fm tags. I used the cosine distance of tf-idf weighted tagged artists as a distance metric for tags. When you click on a node, I attach the six closest tags that haven’t already been attached to the graph. I then use the Echo Nest APIs to get all the media.
Even though it’s a pretty simple algorithm, it is quite effective in grouping similar genre. If you are interested in wandering around a maze of music, give the Labyrinth of Genre a try.
A Genre Map
Inspired by an email exchange with Samuel Richardson, creator of ‘Know your genre‘ I created a genre map that might serve as a basis for a visual music explorer (perhaps something to build at one of the upcoming music hack days). The map is big and beautiful (in a geeky way). Here’s an excerpt, click on it to see the whole thing.
Update – I’ve made an interactive exploration tool that lets you wander through the genre graph. See the Labyrinth of Genre
Update 2 – Colin asked the question “What’s the longest path between two genres?” – If I build the graph by using the 12 nearest neighbors to each genre, find the minimum spanning tree for that graph and then find the longest path, I find this 31 step wonder:
Of course there are lots of ways to skin this cat – if I build the graph with just the nearest 6 neighbors, and don’t extract the minimum spanning tree, the longest path through the graph is 10 steps:
Music Hack Day comes to New York City
Mark your calendar! Music Hack Day is coming to New York City on February 12 and 13. Registrations are live, and going fast. This event is guaranteed to sell out. Register early if you want a seat: nyc.musichackday.org
Know your genre
Not sure what the difference is between Black Metal or Death Metal? Looking for the quintessential nerdcore track? Confused about how 3rd wave ska relates to ska? Check out Know your Genre for all the answers.
Know your Genre is a website built by Sam Richardson that helps you better understand the world of music genres. Type in the name of a genre, and Know Your Genre will bring you to a page devoted to the genre where you listen to the best examples, see the most representative bands and discuss the genre with other fans. There’s an interesting crowd sourcing component to Know Your Genre. You can submit your favorite example of a song for a particular genre. For example, I submitted Tales of Topographic Oceans by Yes to the Progressive Rock page. It is now sitting at the end of a long list of examples (with my single vote), but if people decide that ToTO really is the pinnacle of the progrock movement, it will be voted up, reddit style to the top of the prog rock page. (Clearly, this needs to happen, currently there’s a song by Offspring on the top of the progrock heap.)
Know your genre is a well designed site, with lots of nice touches. Sam says he’s going to continue to build on the site, his ultimate plan being to build a timeline of music from pre history through to modern day music with examples of the evolution. Imagine a version of Ishkur’s guide but for all music, or an updated version of Reebee Garofalo’s Genealogy of Pop/Rock Music, that is hyperlinked to bios, audio and video. I’m looking forward to seeing what Sam does with Know Your Genre.
2010 in review
The stats helper monkeys at WordPress.com mulled over how this blog did in 2010, and here’s a high level summary of its overall blog health:

The Blog-Health-o-Meter™ reads Wow.
Crunchy numbers
The Louvre Museum has 8.5 million visitors per year. This blog was viewed about 920,000 times in 2010. If it were an exhibit at The Louvre Museum, it would take 40 days for that many people to see it.
In 2010, there were 131 new posts, growing the total archive of this blog to 358 posts. There were 209 pictures uploaded, taking up a total of 37mb. That’s about 4 pictures per week.
The busiest day of the year was May 26th with 86,049 views. The most popular post that day was The Swinger.
Where did they come from?
The top referring sites in 2010 were stumbleupon.com, reddit.com, gizmodo.com, buzz.bazooka.se, and boingboing.net.
Some visitors came searching, mostly for spotify hack, the swinger, loudness war, music machinery, and spotify premium hack.
Attractions in 2010
These are the posts and pages that got the most views in 2010.
The Swinger May 2010
402 comments and 18 Likes on WordPress.com
Inside the precision hack April 2009
309 comments
moot wins, Time Inc. loses April 2009
200 comments and 2 Likes on WordPress.com
The Loudness War Analyzed March 2009
72 comments and 1 Like on WordPress.com,
Hacking spotify February 2009
3 comments
The Music Maze
Posted by Paul in code, fun, Music, The Echo Nest, video, visualization, web services on December 20, 2010
I wrote an application over the weekend called Music Maze. The Music Maze lets you wander through the maze of similar artists until you find something you like. You can give it a try here: The Music Maze (be forewarned, the app plays music upon loading).
We’ve seen the idea behind the Music Maze in other apps like Musicovery and Tuneglue’s Music Map. The nifty thing about the Music Maze is that I didn’t have to write a single line of server code to make it all happen. The Music Maze web app talks directly to The Echo Nest API. There’s no middle man. The artist graph, the album art, the links to audio – everything are pulled on demand from the Echo Nest API. This is possible because the Echo Nest API now supports JSONP requests (in beta, full release coming soon!). With JSONP an AJAX app can escape the Javascript sandbox and make calls to 3rd party web services. No need for me to set up a server to proxy calls to the Echo Nest, no Apache or Tomcat, no MySQL, no worries about scaling. This makes it incredibly easy for me to host and deploy this app. I just toss my HTML, Javascript and CSS files into an Amazon S3 bucket, make them world readable, and I’m done. It really has never been easier to create Music Apps. This whole app is less than 500 lines of javascript, written in a few hours on a Sunday morning while the rest of the family are still asleep. It is great to see all of these technologies coming together to make easy to create music apps.
(Be sure to check out the JavaScript InfoVis Toolkit . It does all of the the graphical heavy lifting in this app. It’s pretty neat.)
SongCards – an untapped app
I saw this interesting video from IDEO for c60 – an RFID-based interface that ‘reintroduces physicality to music, something lost with digitization and the move to the cloud.’
This video got me excited, because it is the hardware piece of an idea that my friend Steve Green and I had called ‘SongCards’ while working at Sun Labs a few years ago. We pitched SongCards to Sun’s management (Sun was big into RFID at the time so it seemed like a good fit), but Sun didn’t bite – they decided to go buy MySql instead. And so this concept has been gathering digital dust in a text file on my laptop. The c60 video has inspired me to dust it off and post i here. I think there are some good ideas embedded in the concept. Perhaps the folks at IDEO will incorporate some into the c60, or maybe Eliot will add this idea to his Untapped Apps portfolio on Evolver.fm.
Here’s the concept in all its glory:
Observations
- Many people have a physical connection with their music. These people like to organize, display and interact with their music via the containers (album covers, cd cases).
- Music is a highly social medium. People enjoy sharing music with others. People learn about new music from others in their social circle.
- The location where music is stored will likely switch from devices managed by the listener to devices managed by a music service. In the future, a music purchaser will purchase the right to listen to a particular song, while the actual music data will remain managed by the music service.
- Digital music lacks much of the interesting metadata that previous generations of music listeners enjoyed – lyrics, photos of the performers, song credits. The experience of reading the liner notes while listening to a new album has been lost in this new generation of digital music.Music is collectable. People take pride in amassing large collections of music and like to be able to exhibit their collection to others.
The Problem
The digital music revolution and the inevitable move of our music from our CD racks, iPods and computers, to the back room at Yahoo, Apple, or Google will make it convenient for people to listen to music in all sorts of new ways, however at the same time it will eliminate many of the interactions people have had with the music. People can’t interact with the albums, read the liner notes, display their collection. They can’t trade songs with their friends. There is no way to show off a music collection beyond saying “I have 2,025 songs on my iPod”. Album art is a dying art.
weMusic collecting is not just about the music, it also is about the things that surround the music. Digital music has stripped away all of the packaging, and at the same time has stripped away a big part of the music collecting experience. We want to change that.
The Idea
Imagine if you could buy music like you buy collectable trading cards such as Magic the Gathering, or Pokemon cards. One could buy a pack of cards at the local 7-11 for a few dollars. The cards could be organized by genre. You could buy a pack of ‘boy-band’, ‘alternative-grunge’, ‘brit-pop’, ‘British invasion ‘, ‘drum and bass’ etc. Each pack would contain 5 or 10 cards draw from the genre of the pack. Each card would have all of the traditional liner note metadata: lyrics, album art, artist bios. Also associated with each card would be a unique ID that can be read by an electronic reader that would identify the unique instance of the card and the song/performance that the card represents. The new owner of the card would add the song to their personal collection by just presenting it to any of the owner’s music players (presumably they are connected to the music server run by the music service provider). Once this is done, the user can access and play the song at any time on any of their music devices.
A package of music cards can be packaged in the same way as other trading cards are packaged. Typically in each pack there are one or two ‘special’ cards that are highly desirable. For music cards these would be the highly desirable ‘hit cards’. The bulk of the cards in a deck could be made up of lesser known, or less popular bands. For instance a ‘British invasion’ card set may contain ‘hey jude’ as the special card, and a few lesser known Beatles songs, and few songs by “the who” and perhaps some by “the monkees” and other songs by bands of that era. This method of packing music would allow for serendipitous discovery of music since you would never know what songs you would get in the pack. It would also encourage music trading, since you could trade your duplicate songs with other music collectors.
Trading – since the cards represent a song by a digital id, trading a song is as simple as giving the card to someone. As soon as the new owner of the card puts the card into one of their music players the transfer of ownership would occur, the song would be added to the collection of the new owner and removed from the collection of the old owner. There would be no limit to how often a song could be traded.
Some interesting properties of music cards:
- Your music collection once again has a physical presence. You can touch it, you can browse through it, you can stack it, you can show it off.
- You can easily and legally trade or sell music with your friends (or on eBay for that matter). Supply and demand economics can take hold in the music card after market (just as we’ve seen with Beanie Babies and Magic cards).Cards can be grouped in packages for sale using a number of criteria such as genre, popularity, geography, appeal to a certain demographic.
- You can make playlists by ordering your cards.
- You can make a random playlist by shuffling your cards.
- At a social gathering, cards from many people can be combined into a single uber-playlist.
- You will be potentially exposed to new music every time you buy a new pack of cards.
- You will not need to carry your cards with you when you want to listen to music (the music service knows what music you own).
- Since the music service ‘knows’ what music you own it can monitor trades and music popularity to track trend setters within a social group and target appropriate marketing at the trend setters.
- Song cards can’t be ‘ripped’ in the traditional sense, giving music companies much more control over their intellectual property.
Some interesting variations:
- The artwork on the back of a card could be one section of the album art for a whole album. You could tack the cards up on the wall to form the album art when you have the whole album.
- Some of the cards could be power cards that act as modifiers:
- ‘More Like This‘ when inserted into a playlist, plays a similar song to the previously played card. The similar song is drawn from the entire music service collection not just the songs owned by the collector.
- Genre Wild card’ – plays a random song from the genre. The similar song is drawn from the entire music service collection not just the songs owned by the collector.
- ‘Musical Journey‘ – make a musical journey between the surrounding cards. The songs on the journey are drawn from the entire music service collection not just the songs owned by the collector.
- ‘Album Card’ – it’s not just a song, it’s a whole album.
Note that I don’t think that SongCards would replace all digital music sales. It would still be possible to purchase and download a song from iTunes as one can do today. I think that SongCards would appeal to the ‘Music Collector’, while the traditional download would appeal to the ‘Music Listener’.
That’s it – SongCards – Just imagine what the world would be like if Sun had invested $800 million on SongCards instead of that open source database.
Map Of Metal
Posted by Paul in fun, Music, visualization on December 7, 2010
Web designer Patrick Galbraith has built the Map Of Metal – an interactive tool for exploring the world of metal. It is a bit like Ishkur’s Guide to Electronic Music but for metal. The map shows how the various subgenres of metal are connected, provides descriptions of the various subgenres (culled from Wikipedia) and plays full track examples of each type of music (drawn from Youtube). The map has a funky design sense that adds to the fun. Patrick uses denim, patches of leather, chains, threads , band-aids, buttons, pins and a whole lot of skulls to knit the whole visualization together.
There’s lots of information crammed into this hand made visualization. There are about a hundred metal genres represented, (South American death Metal anyone?), era, connections, influences, and lots of music. Each genre is represented by a dozen or so representative artists and tracks. Unlike Ishkur’s, the Map Of Metal plays full tracks so the music never stops while you are exploring. It is certainly a whole lot more interesting than the map of metal I built a few years ago by analyzing Last.fm tags:
I do wish that I could see the whole map at once – there is a navigation map that shows the whole thing but it is far too small to give me a sense of the whole space.
Do you have a new, cool music app?
Do you have a new, cool music app? Consider entering it into the SXSW Accelerator at SXSW Music. This accelerator is for applications and technologies specifically designed for the use of musicians and the music business. These companies seek to advance the creation, distribution and promotion of recorded music, to facilitate licensing and payment for use of recordings, or to aid in the booking, logistics and promotion of live performances.
SXSW offers these to 10 reasons to enter your application into the SXSW Accelerator:
1. Expand Your Audience
Thousands of individuals from around the world flock to SXSW each year looking for the next big thing, and SXSW Accelerator is a major part of the excitement. Showcasing your idea at Accelerator is an incredible opportunity to get in front of individuals who can help take your concept to the next level.
2. Network with Industry Leaders
One of the greatest values of SXSW is the amazing mix of industry leaders, technology innovators, big-name companies, fresh startups, and independent talent attracted to the event. The ample socializing opportunities at SXSW Accelerator make it easy for participants to meet, greet, and establish a network of professionals to work with on current and future projects.
3. Refine Your Product
No matter how strong your pitch, to attract investors you need a strong product. Presenting your idea to an experienced panel of industry experts, and discussing your product with other entrepreneurs can help you to take it to the next level.
4. Polish Your Elevator Pitch
Pitching is the single-most important skill you need to rise above the competition. That pitch should be rock solid. At Accelerator, you get to pitch your nascent technologies to scores of innovators, content producers, media experts, and venture capitalists. Their feedback and expertise can help you sharpen that all-important pitch.
5. Learn About Funding Options
Discover the latest funding strategies adopted by startups, including seed combinators, angel investing, coworking, local investors, and more. The company of experts in the industry will prove a valuable source of ideas to finance your product.
6. Music-Related Technology
We’re looking for the newest and best ideas. That’s why we’ve dedicated an entire day of the SXSW Accelerator to focus specifically on Music-Related Technologies at the forefront of the industry.
7. Take Advantage of Media Exposure
SXSW attracts a lot of media attention, and SXSW Accelerator is particularly interesting to press outlets looking to break the next exciting technology story. Accelerator presenters can leverage that attention to place their company in the spotlight with ample press opportunities.
8. Register at the Lowest Rate
Entering your product or service in SXSW Accelerator guarantees you the lowest early registration rate of the year.
9. Experience All That SXSW Has to Offer
SXSW Music offers a wealth of exciting opportunities and events, including panel programming, showcases, and, of course, the inspirational experience that only SXSW can deliver. Be a part of it all and enter your innovative product or service to SXSW Accelerator today.
10. Welcome to Austin, Texas
Austin, Texas has long been a hub for freethinking technology movers and shakers. The Central Texas Hill Country has earned the nickname “Silicon Hills” because of the technology industry giants with major operations here and the scores of trailblazing startups and indie tech companies that call Central Texas home. The ever-changing nature of technology and the inclusiveness of its reach make Austin an ideal place to reach out to the larger social web. Did I mention the BBQ?
LastFM-ArtistTags2007
A few years back I created a data set of social tags from Last.fm. RJ at Last.fm graciously gave permission for me to distribute the dataset for research use. I hosted the dataset on the media server at Sun Labs. However, with the Oracle acquisition, the media server is no longer serving up the data, so I thought I would post the data elsewhere.
The dataset is now available for download here: Lastfm-ArtistTags2007
Here are the details as told in the README file:
The LastFM-ArtistTags2007 Data set
Version 1.0
June 2008
What is this?
This is a set of artist tag data collected from Last.fm using
the Audioscrobbler webservice during the spring of 2007.
The data consists of the raw tag counts for the 100 most
frequently occuring tags that Last.fm listeners have applied
to over 20,000 artists.
An undocumented (and deprecated) option of the audioscrobbler
web service was used to bypass the Last.fm normalization of tag
counts. This data set provides raw tag counts.
Data Format:
The data is formatted one entry per line as follows:
musicbrainz-artist-id<sep>artist-name<sep>tag-name<sep>raw-tag-count
Example:
11eabe0c-2638-4808-92f9-1dbd9c453429<sep>Deerhoof<sep>american<sep>14
11eabe0c-2638-4808-92f9-1dbd9c453429<sep>Deerhoof<sep>animals<sep>5
11eabe0c-2638-4808-92f9-1dbd9c453429<sep>Deerhoof<sep>art punk<sep>21
11eabe0c-2638-4808-92f9-1dbd9c453429<sep>Deerhoof<sep>art rock<sep>18
11eabe0c-2638-4808-92f9-1dbd9c453429<sep>Deerhoof<sep>atmospheric<sep>4
11eabe0c-2638-4808-92f9-1dbd9c453429<sep>Deerhoof<sep>avantgarde<sep>3
Data Statistics:
Total Lines: 952810
Unique Artists: 20907
Unique Tags: 100784
Total Tags: 7178442
Filtering:
Some minor filtering has been applied to the tag data. Last.fm will
report tag with counts of zero or less on occasion. These tags have
been removed.
Artists with no tags have not been included in this data set.
Of the nearly quarter million artists that were inspected, 20,907
artists had 1 or more tags.
Files:
ArtistTags.dat - the tag data
README.txt - this file
artists.txt - artists ordered by tag count
tags.txt - tags ordered by tag count
License:
The data in LastFM-ArtistTags2007 is distributed with permission of
Last.fm. The data is made available for non-commercial use only under
the Creative Commons Attribution-NonCommercial-ShareAlike UK License.
Those interested in using the data or web services in a commercial
context should contact partners at last dot fm. For more information
see http://www.audioscrobbler.net/data/
Acknowledgements:
Thanks to Last.fm for providing the access to this tag data via their
web services
Contact:
This data was collected, filtered and by Paul Lamere of The Echo Nest. Send
questions or comments to Paul.Lamere@gmail.com













