I like maps, especially maps that show music spaces – in fact I like them so much I have one framed, hanging in my kitchen. I’d like to create a map for all of music. Like any good map, this map should work at multiple levels; it should help you understand the global structure of the music space, while allowing you to dive in and see fine detailed structure as well. Just as Google maps can show you that Africa is south of Europe and moments later that Stark st. intersects with Reservoir St in Nashua NH a good music map should be able to show you at a glance how Jazz, Blues and Rock relate to each other while moments later let you find an unknown 80s hair metal band that sounds similar to Bon Jovi.
My goal is to build a map of the artist space, one the allows you to explore the music space at a global level, to understand how different music styles relate, but then also will allow you to zoom in and explore the finer structure of the artist space.
I’m going to base the music map on the artist similarity data collected from the Echo Nest artist similarity web service. This web service lets you get 15 most similar artists for any artist. Using this web service I collected the artist similarity info for about 70K artists along with each artists familiarity and hotness.
It would be silly to start trying to visualize 70K artists right away – the 250K artist-to-artist links would overwhelm just about any graph layout algorithm. The graph would look like this. So I started small, with just the near neighbors of The Beatles. (Beatles-K1) For my first experiment, I graphed the the nearest neighbors to The Beatles. This plot shows how the the 15 near neighbors to the Beatles all connect to each other.
In the graph, artist names are sized proportional to the familiarity of the artist. The Beatles are bigger than The Rutles because they are more familiar. I think the graph is pretty interesting, showing how all of the similar artists of the Beatles relate to each other, however, the graph is also really scary because it shows 64 interconnections for these 16 artists. This graph is just showing the outgoing links for the Beatles, if we include the incoming links to the Beatles (the artist similarity function is asymettric so outgoing similarities and incoming similarities are not the same), it becomes a real mess:
If you extend this graph one more level – to include the friends of the friends of The Beatles (Beatles-K2), the graph becomes unusable. Here’s a detail, click to see the whole mess. It is only 116 artists with 665 edges, but already you can see that it is not going to be usable.
Eliminating the edges
Clearly the approach of drawing all of the artist connections is not going to scale to beyond a few dozen artists. One approach is to just throw away all of the edges. Instead of showing a graph representation, use an embedding algorithm like MDS or t-SNE to position the artists in the space. These algorithms layout items by attempting to minimize the energy in the layout. It’s as if all of the similar items are connected by invisible springs which will push all of the artists into positions that minimize the overall tension in the springs. The result should show that similar artists are near each other, and dissimilar artists are far away. Here’s a detail for an example for the friends of the friends of the Beatles plot. (Click on it to see the full plot)
I find this type of visualization to be quite unsatisfying. Without any edges in the graph I find it hard to see any structure. I think I would find this graph hard to use for exploration. (Although it is fun though to see the clustering of bands like The Animals, The Turtles, The Byrds, The Kinks and the Monkeee).
Drawing some of the edges
We can’t draw all of the edges, the graph just gets too dense, but if we don’t draw any edges, the map loses too much structure making it less useful for exploration. So lets see if we can only draw some of the edges – this should bring back some of the structure, without overwhelming us with connections. The tricky question is “Which edges should I draw?”. The obvious choice would be to attach each artist to the artist that it is most similar to. When apply this to the Beatles-K2 neighborhood we get something like this:
This clearly helps quite a bit. We no longer have the bowl of spaghetti, while we can still see some structure. We can even see some clustering that make sense (Led Zeppelin is clustered with Jimi Hendrix and the Rolling Stones while Air Supply is closer to the Bee Gees). But there are some problems with this graph. First, it is not completely connected, there are a 14 separate clusters varying from a size of 1 to a size of 57. This disconnection is not really acceptable. Second, there are a number of non-intuitive flows from familiar to less familiar artists. It just seems wrong that bands like the Moody Blues, Supertramp and ELO are connected to the rest of the music world via Electric Light Orchestra II (shudder).
To deal with the ELO II problem I tried a different strategy. Instead of attaching an artist to its most similar artist, I attach it to the most similar artist that also has the closest, but greater familiarity. This should prevent us from attaching the Moody Blues to the world via ELO II, since ELO II is of much less familiarity than the Moody Blues. Here’s the plot:
Now we are getting some where. I like this graph quite a bit. It has a nice left to right flow from popular to less popular, we are not overwhelmed with edges, and ELO II is in its proper subservient place. The one problem with the graph is that it is still disjoint. We have 5 clusters of artists. There’s no way to get to ABBA from the Beatles even though we know that ABBA is a near neighbor to the Beatles. This is a direct product of how we chose the edges. Since we are only using some of the edges in the graph, there’s a chance that some subgraphs will be disjoint. When I look at the a larger neighborhood (Beatles-K3), the graph becomes even more disjoint with a hundred separate clusters. We want to be able to build a graph that is not disjoint at all, so we need a new way to select edges.
Minimum Spanning Tree
One approach to making sure that the entire graph is connected is to generate the minimum spanning tree for the graph. The minimum spanning tree of a graph minimizes the number of edges needed to connect the entire graph. If we start with a completely connected graph, the minimum spanning tree is guarantee to result in a completely connected graph. This will eliminate our disjoint clusters. For this next graph, built the minimum spanning tree of the Beatles-K2 graph.
As predicted, we no longer have separate clusters within the graph. We can find a path between any two artists in the graph. This is a big win, we should be able to scale this approach up to an even larger number of artists without ever having to worry about disjoint clusters. The whole world of music is connected in a single graph. However, there’s something a bit unsatisfying about this graph. The Beatles are connected to only two other artists: John Lennon & The Plastic Ono Band and The Swinging Blue Jeans. I’ve never heard of the Swinging Blue Jeans. I’m sure they sound a lot like the Beatles, but I’m also sure that most Beatles fans would not tie the two bands together so closely. Our graph topology needs to be sensitive to this. One approach is to weight the edges of the graph differently. Instead of weighting them by similarity, the edges can be weighted by the difference in familiarity between two artists. The Beatles and Rolling Stones have nearly identical familiarities so the weight between them would be close to zero, while The Beatles and the Swinging Blue Jeans have very different familiarities, so the weight on the edge between them would be very high. Since the minimum spanning is trying to reduce the overall weight of the edges in the graph, it will chose low weight edges before it chooses high weight edges. The result is that we will still end up with a single graph, with none of the disjoint clusters, but artists will be connected to other artists of similar familiarity when possible. Let’s try it out:
Now we see that popular bands are more likely to be connected to other popular bands, and the Beatles are no longer directly connected to “The Swinging Blue Jeans”. I’m pretty happy with this method of building the graph. We are not overwhelmed by edges, we don’t get a whole forest of disjoint clusters, and the connections between artists makes sense.
Of course we can build the graph by starting from different artists. This gives us a deep view into that particular type of music. For instance, here’s a graph that starts from Miles Davis:
Here’s a near neighbor graph starting from Metallica:
And here’s one showing the near neighbors to Johann Sebastian Bach:
This graphing technique works pretty well, so lets try an larger set of artists. Here I’m plotting the top 2,000 most popular artists. Now, unlike the Beatles neighborhood, this set of artists is not guaranteed to be connected, so we may have some disjoint cluster in the plot. That is expected and reasonable. The image of the resulting plot is rather large (about 16MB) so here’s a small detail, click on the image to see the whole thing. I’ve also created a PDF version which may be easier to browse through.
I pretty pleased with how these graphs have turned out. We’ve taken a very complex space and created a visualization that shows some of the higher level structure of the space (jazz artists are far away from the thrash artists) as well as some of the finer details – the female bubblegum pop artists are all near each other. The technique should scale up to even larger sets of artists. Memory and compute time become the limiting factors, not graph complexity. Still, the graphs aren’t perfect – seemingly inconsequential artists sometimes appear as gateways into whole sub genre. A bit more work is needed to figure out a better ordering for nodes in the graph.
Some things I’d like to try, when I have a bit of spare time:
- Create graphs with 20K artists (needs lots of memory and CPU)
- Try to use top terms or tags of nearby artists to give labels to clusters of artists – so we can find the Baroque composers or the hair metal bands
- Color the nodes in a meaningful way
- Create dynamic versions of the graph to use them for music exploration. For instance, when you click on an artist you should be able to hear the artist and read what people are saying about them.
To create these graphs I used some pretty nifty tools:
- The Echo Nest developer web services – I used these to get the artist similarity, familiarity and hotness data. The artist similarity data that you get from the Echo Nest is really nice. Since it doesn’t rely directly on collaborative filtering approaches it avoids the problems I’ve seen with data from other sources of artist similarity. In particular, the Echo Nest similarity data is not plagued by hubs (for some music services, a popular band like Coldplay may have hundreds or thousands of near neighbors due to a popularity bias inherent in CF style recommendation). Note that I work at the Echo Nest. But don’t be fooled into thinking I like the Echo Nest artist similarity data because I work there. It really is the other way around. I decided to go and work at the Echo Nest because I like their data so much.
- Graphviz – a tool for rendering graphs
- Jung – a Java library for manipulating graphs
If you have any ideas about graphing artists – or if you’d like to see a neighborhood of a particular artist. Please let me know.
#1 by Garg on May 31, 2009 - 6:50 pm
I don’t work for Echo Nest, but I really like the data from the API.
This is a brilliant post, thanks so much!
#2 by Nemo on June 1, 2009 - 8:23 am
Ok, this is a very interesting idea, but as I read through this, it felt like the idea had lost it’s way. In the opening you make the analogy to continents and streets, relating to genres and artists… but then the entire mapping is artist based. What happened to the genre maps? Is it intended that they be an emergent property of the ‘closeness’ of the artists? Seemed to me that when the maps started becoming hierarchal trees, then something was lost.
My own thoughts in this direction (for an automated playlist idea I once had: http://wiki.thorx.net/wiki/NIPL ) is that one should tag songs by genre, and that the genre tag be ‘tags’ in the modern plural sense, not a singular in the ‘traditiona’ or mp3 id3v1 sense. Thus the genre for albums and artists would be an emergent property of the genres for their music (both Pink Floyd and Genesis would span prog Rock, but Genesis would also heavily span into pop, whilst Floyd would span into intstrumental and concept. Moody Blues might be said to span between pop and concept. The Beatles mostly would be pop/rock, but The White Album would cross the board pretty widely). In this way links between artists can be found by the genre overlaps – the more congruent the overlaps between artists, the “closer” they are in the music map. I’d be interested to know thoughts on this.
#3 by plamere on June 1, 2009 - 8:31 am
Nemo – you are exactly right … these maps are giving the finer level of detail, the street view if you will. The next step is to start to add content that will help point out the global structure. The second item on my TODO list: “Try to use top terms or tags of nearby artists to give labels to clusters of artists – so we can find the Baroque composers or the hair metal bands” sounds just like what you are suggesting. I also like the idea of find artists that span genres to use them as ‘gateway artists’. For instance, you could use certain songs by the Beatles such as ‘within you, without you’ to serve as a gateway for a pop music fan to Indian music. My goal is to be able to automatically build something akin to Iskur’s guide to Electronic Music (http://techno.org/electronic-music-guide/) – which is just filled with relations, context, and global and local structure. Admittedly, these maps are just a few steps along the way. The goal being to figure out how to represent the vast interconnectedness of artists without overwhelming the visualization (or the computers creating them).
#4 by Garg on June 1, 2009 - 9:06 am
The genre set is too large and would result in even more messy graphs. All the bands associated with The Beatles, for instance, would be less than all the bands associated with pop, concept albums, Indian music, etc. It’s a great idea for exploring a single playlist but I don’t think it would scale well. Do you for instance pick the pop, Indian music and concept albums which are the most popular at the time? Then you might end up with Bollywood soundtracks, Lady Gaga and Iced Earth in the same category. This might make perfect sense on an individual playlist, but not on a global music map.
Another problem with genre tags is they’re very subjective. Most Tool fans regard Tool as a progressive. Ye gods know why. Yet, most Tool fans would associate Tool with prog and end up with Camel instead of with A Perfect Circle and NIN, which is probably more where they’d like to explore. This implies you’d have to jump into the bowl of spaghetti that is folksonomy.
As last.fm illustrates, folksonomy works very well because it helps users to define the tags. It also has drawbacks, particularly because it allows users to define the tags. You’re back with collaborative filtering.
Taxonomy has drawbacks because it doesn’t allow users to define the tags, so Tool would never ever end up in the prog genre, meaning no Tool fan would find the graphs useful because they’re barking up the wrong tree. As such, users would have to familiarise themselves with taxonomy before exploring artists, which means a steep learning curve for the majority of Tool fans.
I’d be very interested to see how it could be done on this scale.
#5 by Nemo on June 1, 2009 - 9:58 am
Garg, I think the problem with genres is that it’s either too specific (some of the id3 defined genres are insane. otoh, broad genres lead to too MUCH overlap…
but broad genres that allow multiple solve, imho.
Thus tool songs are likely to be tagged ‘progressive’ and ‘metal’, allowing movement into either non-metal progressive, or non-progressive metal.
So I don’t care to make a playlist of all bands who share *any* genre with The Beatles – yes, that would be overload and incoherent. It’s a band-centric view. I’m looking at it from a playlist-centric view however – the band-centric view is only relevant when you’re playing a song from that band, and want to work out which song to move to next. When you’re on that next song, you have a new band to center from.
So, what I envisage is a system where I can say “start with pop, move me to prog metal after an hour, then I want to be listening to classical baroque at the two hour mark, and by lunchtime at the three and a half hour mark, I want to be listening to african tribal. …and then let the system work out the songs which trace the best path (with the noted time constraints) as nescessary. And you may visit unexpected genres to make the fit (eg: alternative celtic might fit well between baroque and african tribal…
choosing the genres is the big issue of course… and the closer you are to a specific type of music, the more differences you can see. (thus I don’t know (or care) about the difference between ‘country’ and ‘western’, but I’m sure someone does. Thus I think sub-genres are the solution. I can tag something as ‘country and western’, and leave it at that as a top-level tag – whilst a c&w fan can tag Tool as “metal rock” without caring about subgenres of alternative, progressive, etc)
This may or may not be more confusing than my previous writeups about the same:
#6 by Garg on June 1, 2009 - 10:44 am
Yes, genre overlaps are great for exploring a playlist. If I understand you correctly, you are specialising into the areas where say prog and metal overlap given a certain band on your playlist’s relation to others on your playlist (and your choice of genre tags), and not exploring the bigger picture of either prog or metal or prog metal as genres.
Inevitably, for scaling up larger than a playlist, to where your domain is less specific, you will have to resort to less arbitrary abstractions. For instance, I have both JS Bach, hair metal, and neo-classical metal on my playlists. It’s great for me to go from JS Bach to Yngwie Malmsteen, but I’m not exploring either Baroque or neo-classical metal as genres or building a visual map of how they relate to each other in a way that would be meaningful to anyone who is unfamiliar with my ghastly music taste. Partially because to many, the distinction between hair metal and neo-classical metal is less pronounced (and more arbitrary) than the distinction between JS Bach and Sebastian Bach of Skid Row fame.
Take, for instance, Joe Jackson. Which genre would you place him in? He’s done pop, rock, classical, instrumental, film scores(which may or may not be a genre?), big band, jazz, ad nauseum. There is simply no meaningful way in which I could use genre as an abstraction to relate Joe Jackson to anybody else, because Mr Jackson is going to pop up in virtually all genres or their overlaps. To fans of big band, Joe Jackson is usually not part of the global big band picture just like Tool is not part of the global prog picture to prog fans. Which brings you back to finding some way to give preference to one band above another, whether their genres overlap in any way or not. Which means genre is not helpful when your domain is so large and you are more interested in exploration than specialisation.
#7 by Garg on June 2, 2009 - 10:28 am
I had a browse through my old data structures textbook and it turns out I was wrong about the genres necessarily returning ghastly bowls of spaghetti.
Since you want tagging on a track-by-track basis, each track ought to fit more readily into a specific genre than a particular artist might. Broadly speaking, artists also tend to stick to the specific genres. This means the sets are mostly disjoint, ie a reggae artist will be in the reggae genre set despite the odd dub adventure and will overwhelmingly not crop up in the black metal genre set.
It is thus possible to solve our dynamic equivalence problem with a disjoint-set data structure (for either playlists or for a souped up music map).
My hunch is it would work better for the playlist scenario than the music map scenario, because side projects etc tend to genre hop, but it means there appears to be a way to integrate genres with 2 minute noodles instead of bowls of spaghetti.
#8 by Nemo on June 1, 2009 - 11:09 am
The problem is that I think (correct me if I’m wrong) that you’re seeing genres as something applies to artists (who in turn make albums, which contain songs). I see genres as being applied directly to songs.
To use your Joe Jackson example. I woulnd’t apply any genre to him as an individual, but each track would be tagged as appropriate for that track. An album then has a genre tag cloud which is the sum of the genre tags for all the tracks. Similarly the artist then ends up with a genre cloud based on ALL tracks they ever have done.
regarding building a playlist… the playlist building algorithm would only care about the genres in an individual song. (thus one song might be Big Band and Jazz, so could usefully gateway between those in a playlist. Another track of instrumental rock would gateway between those genres. But that they are the same artist is immaterial.
I’m not wanting to explore only the genre overlap areas. But I do want to explore them as ways to generate a playlist which can move from deep in solid baroque territory and into solid thrash metal territory, with a minimum of “this track is jarringly unlike the previous one” jumps.
I wonder what you would get if one was to graph (as per the original blog) _songs_ (not artists), and position them according to genre… each genre has it’s own “gravity” of sorts, but tags which don’t match the gravity act as a type of anti-gravity. Thus a track tagged ‘jpop’ only sinks to the bottom of the jpop gravity well. But a song tagged ‘jpop’, ‘instrumental’ and ‘progressive’ would rise out of the jpop well and drift towards the instrumental and progressive gravity wells. Hmm, I think everything would end up adrift in the spaces between? If you then add an additional “same artist gravity” (so if our example before was the same artist, then the multi-tagged song would be dragged back towards the jpop well because that is where the artist is centered. (the artist centering being based on the summation of all tags for tracks by that artist, of course).
The biggest problem with anything like that is of course tagging everything in the first place. A daunting task even if you only tagged with a cddb/freedb level of simplicity!
#9 by Garg on June 1, 2009 - 1:46 pm
But the purpose of these graphs is to build music maps of artist space, not so much building playlists of individual songs.
Regarding tagging: I think you can automate track by track tagging to a large extent. Some genres have fairly distinctive features (ie reggae has syncopated rhythm, black metal has blast beats). If you aren’t too fussed about which genre artists land up next to each other, but more with building an integrated playlist, you should have no problem with Bob Marley’s Exodus, Blondie’s Tide is High and Joe Jackson’s Fools in Love playing back to back, for example (the Echo Nest Remix api has get_beats and even get_tempo and get_time_signature features, but I haven’t fiddled with it yet so I don’t know how that would aid with automating your tagging, although they could analyse tracks and give you similar tracks when you zone in to such a detailed level).
Putting playlist generation aside for a moment, these artists do not share the same artist map in terms of building a visual representation of artist space. Track by track genre categorisation is analogous to viewing a street map, grouping houses in the same suburb together. Artist space is analogous to relating different provinces or states in the same country together.
For the further abstraction, the get_similar feature does coincidentally get similar artists in terms of genre, although I doubt whether genre as such is a criteria at all (it works too well for that – I haven’t once received Metallica as a recommendation even though all the artists I entered were metal artists, meaning the reinforcement feedback of popular genre artists doesn’t affect the Echo Nest api much. Great piece of work!).
Perhaps using the ordinary api and the remix api in tandem for the different levels of abstraction would yield interesting results? You could have your track-by-track tailored and tagged playlist (the street map) and we could also have our global artist map (the country map) – which would ultimately also require a playlist, mind you.
#10 by Nemo on June 4, 2009 - 4:40 am
“But the purpose of these graphs is to build music maps of artist space, not so much building playlists of individual songs.”
True – if you only consider artist space. I think (and it’s purely instinct) that a map of songspace will provide means for playlist generation (next song = nearest song to current song which is in the “direction” you wish to travel), as well as creating maps of both artist and genrespace.
I see it that the ‘song’ is the basic atomic unit being dealt with, with artists being an abstraction in one direction, and genres being an abstraction in another. If those abstractions can be balanced, I think the result will be useable and relevant for all purposes. (afterall, tracking the congruency of a certain artist to a certain genre, and comparing that to other artists with similar congruency – is my understanding of the origin of the idea as blogged. :) I think this is what you were saying in your other comment? (damn the limited level of nested replies! :)
I’m not familiar with the APIs noted however, so I’m afraid that is going over my head, beyond what is deducable from the feature names – which looks good.
#11 by zazi on June 3, 2009 - 3:07 pm
It is nice to see a music map built on musical content, but what makes it different to applications like this one http://audiomap.tuneglue.net/?
The genre definition and categorisation is another problem. In my opinion a current state-of-the-art there are multiple weighted genres for artists, albums (track collections) and tracks are neccessary and useful and should be derived from different sources (e.g. collaborative filtering, expert categorisation, web sources and content analysis), which should also be weighted in a certain manner. Last but not least, these weights should be adjust to the user interests, which are analysed and stored in the user profile.
I think visualisations like Anita’s MusicBox and similar ones are going in the right direction to deal with huge music collections.
#12 by Varun on June 9, 2009 - 2:15 am
I’m sorry to post such a menial comment to an otherwise intelligent and provocative discussion, but when I download the general-2k.pdf file, I get nothing but white–there is no text or image whatsoever; only empty white space. I can zoom in or out, but still, only white. Does anyone else have this problem? Am I doing something wrong?
#13 by plamere on June 9, 2009 - 4:08 pm
@Varun – the pdf works for me. I’m on a mac using ‘preview’ – I think others have been able to view it as well. Maybe it is not downoading to your machine properly? The file size should be 323,357 bytes.
#14 by Ethan on September 3, 2009 - 1:39 pm
I enjoyed reading this quite a bit. I’ve been wrestling with some of the same challenges trying to visualize the use of samples by hip-hop and electronica artists. You and your readers might enjoy seeing what I’ve come up with.
#15 by plamere on September 3, 2009 - 1:50 pm
Small world – I’m preparing a talk on using visualizations for music discovery and just last weekend I stumbled upon your michael jackson sample map on your blog – I’m planning on incorporating a shot of that map in the talk (along with the wired timeline for the girl talk song that I’m sure you’ve seen. Your visualizations are cool stuff.
#16 by zazi on September 3, 2009 - 7:09 pm
Too much information is presented on the vizualisations (sample maps) @Ethan. Keep it simple, or us colors for differentiation. It not easy to navigate to such an amount of information.
In general the complexity of music is a mess in visualisation, but we cannot influence the music in its magic.
#17 by moi on April 3, 2010 - 9:46 pm
I found the same idea years ago on http://www.pandora.com The music genoma proyect, but later they banned ip’s outside United States. Still can’t guess why.
Anybody knows a way to enter the site anymay, or a similar one, not being musicovery.com ?