Archive for category music information retrieval
How music recommendation works – and doesn’t work
Posted by Paul in Music, music information retrieval, recommendation, The Echo Nest on December 12, 2012
Brian just posted ’How Music Recommendation works – and doesn’t work‘ over at his Variogr.am blog. It is a must-read for anyone interested in the state of the art in music recommendation. Here’s an excerpt:
Try any hot new artist in Pandora and you’ll get the dreaded:
Pandora not knowing about YUS
This is Pandora showing its lack of scale. They won’t have any information for YUS for some time and may never unless the artist sells well. This is bad news and should make you angry: why would you let a third party act as a filter on top of your very personal experiences with music? Why would you ever use something that “hid” things from you?
Grab a coffee, sit back and read Brian’s post. Highly recommended.
Visualizing the Structure of Pop Music
Posted by Paul in code, data, ismir, Music, music information retrieval, The Echo Nest, visualization on November 19, 2012
The Infinite Jukebox generates plots of songs in which the most similar beats are connected by arcs. I call these plots cantograms. For instance, below is a labeled cantogram for the song Rolling in the Deep by Adele. The song starts at 3:00 on the circle and proceeds clockwise, beat by beat completely around the circle. I’ve labeled the plot so you can see how it aligns with the music. There’s an intro, a first verse, a chorus, a second verse, etc. until the outro and the end of the song.
Rolling in the Deep (labelled) by Adele
One thing that’s interesting is that most of the beat similarity connections occur between the beats in the three instances of the chorus. This certainly makes intuitive sense. The verses have different lyrics, so for the most part they won’t be too similar to each other, but the choruses have the same lyrics, the same harmony, the same instrumentation. They may even be, for all we know may even be exactly the same audio, that perfect performance, cut and pasted three times by the audio engineer to make the best sounding version of the song.
Now take a look at the cantogram for another popular song. The plot below shows the beat similarities for the song Tik Tok by Ke$ha. What strikes me the most about this plot is how similar it looks to the plot for Rolling in the Deep. It has the characteristic longer intro+first verse, some minor inter-verse similarities and the very strong similarities between the three choruses.
Tik Tok by Ke$ha
As we look at more plots for modern pop music we see the same pattern over and over again. In this plot for Lady Gag’s Paparazzi a cantogram we again see the same pattern.
Lady Gaga – Paparazzi
We see it in the plot for Justin Bieber’s Baby:
Justin Bieber – Baby
Taylor Swift’s Fearless has a two verses before the first chorus, shifting it further around the circle, but other than that the pattern holds:
Taylor Swift – Fearless
Now compare and contrast the pop cantograms with those from other styles of music. First up is Led Zeppelin’s Stairway to heaven. There’s no discernable repeating chorus, or global song repetition, the only real long-arc repetition occurs during the guitar solo for the last quarter of the song.
Led Zeppelin – Stairway to Heaven
Here’s another style of music. Deadmau5′s Raise your weapon. This is electronica (and maybe some dubstep). Clearly from the cantogram we can see that is is not a traditional pop song. Very little long arc repetition, with the densest cluster being the final dubstep break.
Deadmau5 – Raise your weapon
Dave Brubeck’s Take Five has a very different pattern, with lots of short term repetition during the first half of the song, while during the second half with Joe Morello’s drum solo there’s a very different pattern.
Dave Brubeck – Take Five
Green Grass and High Tides has yet a different pattern – no three choruses and out here. (By the way, the final guitar solo is well worth listening to in the Infinite Jukebox. It is the guitar solo that never ends).
Green Grass And High Tides by The Outlaws
The progressive rock anthem Roundabout doesn’t have the Pop Pattern
Yes – Roundabout
Nor does Yo-Yo Ma’s performance of the Cello suite No. 1.
01 Cello Suite No.1, 1. Prelude by Yo-Yo Ma
Looking at the pop plots one begins to understand that pop music really could be made in a factory. Each song is cut from the same mold. In fact, one of the most successful pop songs in recent years, was produced by a label with factory in its name. Looking at Rebecca Black’s Friday we can tell right away that it is a pop song:
Friday by Rebecca Black
Compare that plot to this years Youtube breakout, Thanksgiving by Nicole Westbrook, (another Ark Music Factory assembly):
Nicole Westbrook – It’s Thanksgiving (Official Video)
The plot has all the makings of the standard pop song for the 2010s.
In the music information retrieval research community there has been quite a bit of research into algorithmically extracting song structure, and visualizations are often part of this work. If you are interested in learning more about this research, I suggest looking at some of the publications by Meinard Müller and Craig Sapp.
Of course, not every pop song will follow the pattern that I’ve shown here. Nevertheless, I find it interesting that this very simple visualization is able to show us something about the structure of the modern pop song, and how similar this structure is across many of the top pop songs.
update: since publishing this post I’ve updated the layout algorithm in the Infinite Jukebox so that songs start and end at 12 Noon and not 3PM, so the plots you see in this post are rotated 90degrees clockwise from what you would see in the jukebox.
Data Mining Music – a SXSW 2012 Panel Proposal
Posted by Paul in data, events, Music, music information retrieval, The Echo Nest on August 15, 2011
I’ve submitted a proposal for a SXSW 2012 panel called Data Mining Music. The PanelPicker page for the talk is here: Data Mining Music. If you feel so inclined feel free to comment and/or vote for the talk. I promise to fill the talk with all sorts of fun info that you can extract from datasets like the Million Song Dataset.
Here’s the abstract:
Data mining is the process of extracting patterns and knowledge from large data sets. It has already helped revolutionized fields as diverse as advertising and medicine. In this talk we dive into mega-scale music data such as the Million Song Dataset (a recently released, freely-available collection of detailed audio features and metadata for a million contemporary popular music tracks) to help us get a better understanding of the music and the artists that perform the music.
We explore how we can use music data mining for tasks such as automatic genre detection, song similarity for music recommendation, and data visualization for music exploration and discovery. We use these techniques to try to answers questions about music such as: Which drummers use click tracks to help set the tempo? or Is music really faster and louder than it used to be? Finally, we look at techniques and challenges in processing these extremely large datasets.
Questions answered:
- What large music datasets are available for data mining?
- What insights about music can we gain from mining acoustic music data?
- What can we learn from mining music listener behavior data?
- Who is a better drummer: Buddy Rich or Neil Peart?
- What are some of the challenges in processing these extremely large datasets?
Flickr photo CC by tristanf
How do you spell ‘Britney Spears’?
Posted by Paul in code, data, Music, music information retrieval, research, The Echo Nest on July 28, 2011
I’ve been under the weather for the last couple of weeks, which has prevented me from doing most things, including blogging. Luckily, I had a blog post sitting in my drafts folder almost ready to go. I spent a bit of time today finishing it up, and so here it is. A look at the fascinating world of spelling correction for artist names.
In today’s digital music world, you will often look for music by typing an artist name into a search box of your favorite music app. However this becomes a problem if you don’t know how to spell the name of the artist you are looking for. This is probably not much of a problem if you are looking for U2, but it most definitely is a problem if you are looking for Röyksopp, Jamiroquai or Britney Spears. To help solve this problem, we can try to identify common misspellings for artists and use these misspellings to help steer you to the artists that you are looking for.
A spelling corrector in 21 lines of code
A good place for us to start is a post by Peter Norvig (Director of Research at Google) called ’How to write a spelling corrector‘ which presents a fully operational spelling corrector in 21 lines of Python. (It is a phenomenal bit of code, worth the time studying it). At the core of Peter’s algorithm is the concept of the edit distance which is a way to represent the similarity of two strings by calculating the number of operations (inserts, deletes, replacements and transpositions) needed to transform one string into the other. Peter cites literature that suggests that 80 to 95% of spelling errors are within an edit distance of 1 (meaning that most misspellings are just one insert, delete, replacement or transposition away from the correct word). Not being satisfied with that accuracy, Peter’s algorithm considers all words that are within an edit distance of 2 as candidates for his spelling corrector. For Peter’s small test case (he wrote his system on a plane so he didn’t have lots of data nearby), his corrector covered 98.9% of his test cases.
Spell checking Britney
A few years ago, the smart folks at Google posted a list of Britney Spears spelling corrections that shows nearly 600 variants on Ms. Spears name collected in three months of Google searches. Perusing the list, you’ll find all sorts of interesting variations such as ‘birtheny spears’ , ‘brinsley spears’ and ‘britain spears’. I suspect that some these queries (like ‘Brandi Spears’) may actually not be for the pop artist. One curiosity in the list is that although there are 600 variations on the spelling of ‘Britney’ there is exactly one way that ‘spears’ is spelled. There’s no ‘speers’ or ‘spheres’, or ‘britany’s beers’ on this list.
One thing I did notice about Google’s list of Britneys is that there are many variations that seem to be further away from the correct spelling than an edit distance of two at the core of Peter’s algorithm. This means that if you give these variants to Peter’s spelling corrector, it won’t find the proper spelling. Being an empiricist I tried it and found that of the 593 variants of ‘Britney Spears’, 200 were not within an edit distance of two of the proper spelling and would not be correctable. This is not too surprising. Names are traditionally hard to spell, there are many alternative spellings for the name ‘Britney’ that are real names, and many people searching for music artists for the first time may have only heard the name pronounced and have never seen it in its written form.
Making it better with an artist-oriented spell checker
A 33% miss rate for a popular artist’s name seems a bit high, so I thought I’d see if I could improve on this. I have one big advantage that Peter didn’t. I work for a music data company so I can be pretty confident that all the search queries that I see are going to be related to music. Restricting the possible vocabulary to just artist names makes things a whole lot easier. The algorithm couldn’t be simpler. Collect the names of the top 100K most popular artists. For each artist name query, find the artist name with the smallest edit distance to the query and return that name as the best candidate match. This algorithm will let us find the closest matching artist even if it is has an edit distance of more than 2 as we see in Peter’s algorithm. When I run this against the 593 Britney Spears misspellings, I only get one mismatch – ‘brandi spears’ is closer to the artist ‘burning spear’ than it is to ‘Britney Spears’. Considering the naive implementation, the algorithm is fairly fast (40 ms per query on my 2.5 year old laptop, in python).
Looking at spelling variations
With this artist-oriented spelling checker in hand, I decided to take a look at some real artist queries to see what interesting things I could find buried within. I gathered some artist name search queries from the Echo Nest API logs and looked for some interesting patterns (since I’m doing this at home over the weekend, I only looked at the most recent logs which consists of only about 2 million artist name queries).
Artists with most spelling variations
Not surprisingly, very popular artists are the most frequently misspelled. It seems that just about every permutation has been made in an attempt to spell these artists.
- Michael Jackson - Variations: michael jackson, micheal jackson, michel jackson, mickael jackson, mickal jackson, michael jacson, mihceal jackson, mickeljackson, michel jakson, micheal jaskcon, michal jackson, michael jackson by pbtone, mical jachson, micahle jackson, machael jackson, muickael jackson, mikael jackson, miechle jackson, mickel jackson, mickeal jackson, michkeal jackson, michele jakson, micheal jaskson, micheal jasckson, micheal jakson, micheal jackston, micheal jackson just beat, micheal jackson, michal jakson, michaeljackson, michael joseph jackson, michael jayston, michael jakson, michael jackson mania!, michael jackson and friends, michael jackaon, micael jackson, machel jackson, jichael mackson
- Justin Bieber – Variations: justin bieber, justin beiber, i just got bieber’ed by, justin biber, justin bieber baby, justin beber, justin bebbier, justin beaber, justien beiber, sjustin beiber, justinbieber, justin_bieber, justin. bieber, justin bierber, justin bieber<3 4 ever<3, justin bieber x mstrkrft, justin bieber x, justin bieber and selens gomaz, justin bieber and rascal flats, justin bibar, justin bever, justin beiber baby, justin beeber, justin bebber, justin bebar, justien berbier, justen bever, justebibar, jsustin bieber, jastin bieber, jastin beiber, jasten biber, jasten beber songs, gestin bieber, eiine mainie justin bieber, baby justin bieber,
- Red Hot Chili Peppers – Variations: red hot chilli peppers, the red hot chili peppers, red hot chilli pipers, red hot chilli pepers, red hot chili, red hot chilly peppers, red hot chili pepers, hot red chili pepers, red hot chilli peppears, redhotchillipeppers, redhotchilipeppers, redhotchilipepers, redhot chili peppers, redhot chili pepers, red not chili peppers, red hot chily papers, red hot chilli peppers greatest hits, red hot chilli pepper, red hot chilli peepers, red hot chilli pappers, red hot chili pepper, red hot chile peppers
- Mumford and Sons – Variations: mumford and sons, mumford and sons cave, mumford and son, munford and sons, mummford and sons, mumford son, momford and sons, modfod and sons, munfordandsons, munford and son, mumfrund and sons, mumfors and sons, mumford sons, mumford ans sons, mumford and sonns, mumford and songs, mumford and sona, mumford and, mumford &sons, mumfird and sons, mumfadeleord and sons
- Katy Perry - Even an artist with a seemingly very simple name like Katy Perry has numerous variations: katy perry, katie perry, kate perry, kathy perry, katy perry ft.kanye west, katty perry, katy perry i kissed a girl, peacock katy perry, katyperry, katey parey, kety perry, kety peliy, katy pwrry, katy perry-firework, katy perry x, katy perry, katy perris, katy parry, kati perry, kathy pery, katey perry, katey perey, katey peliy, kata perry, kaity perry
Some other most frequently misspelled artists:
- Britney Spears
- Linkin Park
- Arctic Monkeys
- Katy Perry
- Guns N’ Roses
- Nicki Minaj
- Muse
- Weezer
- U2
- Oasis
- Moby
- Flyleaf
- Seether
- byran adams - ryan adams
- Underworld – Uverworld
Do you do Music Information Retrieval?
Posted by Paul in code, ismir, Music, music information retrieval, The Echo Nest on September 10, 2010
We’re ramping up hiring at the Echo Nest. We’re looking for good MIR people at different experience levels to help us realize the company’s vision of knowing everything about all music automatically. I would guess that we are the closest analog to ISMIR in the industry– we only do music (audio and text), the base technology is straight out of our dissertations (brian, tristan) and we’re active in conferences and universities. We work with an amazing amount of music data on a daily basis and we sell it to some great people and companies that are changing the face of music.
MIR-background candidates are especially encouraged to apply as long as you have relevant experience and want to work on implementation at a very fast growing startup. These are almost all full time positions in our offices near Boston, MA USA. Even if you’re not graduating for a while let us know if you’re interested now.
More info at:
http://the.echonest.com/company/jobs/
Group coding session at The Echo Nest
Upbeat and Quirky, With a Bit of a Build: Interpretive Repertoires in Creative Music Search
Posted by Paul in events, Music, music information retrieval, research on August 13, 2010
Upbeat and Quirky, With a Bit of a Build: Interpretive Repertoires in Creative Music Search
Charlie Inskip, Andy MacFarlane and Pauline Rafferty
ABSTRACT Pre-existing commercial music is widely used to accompany moving images in films, TV commercials and computer games. This process is known as music synchronisation. Professionals are employed by rights holders and film makers to perform creative music searches on large catalogues to find appropriate pieces of music for syn- chronisation. This paper discusses a Discourse Analysis of thirty interview texts related to the process. Coded examples are presented and discussed. Four interpretive re- pertoires are identified: the Musical Repertoire, the Soundtrack Repertoire, the Business Repertoire and the Cultural Repertoire. These ways of talking about music are adopted by all of the community regardless of their interest as Music Owner or Music User.
Music is shown to have multi-variate and sometimes conflicting meanings within this community which are dynamic and negotiated. This is related to a theoretical feedback model of communication and meaning making which proposes that Owners and Users employ their own and shared ways of talking and thinking about music and its context to determine musical meaning. The value to the music information retrieval community is to inform system design from a user information needs perspective.
What Makes Beat Tracking Difficult? A Case Study on Chopin Mazurkas
Posted by Paul in events, ismir, music information retrieval, research on August 13, 2010
What Makes Beat Tracking Difficult? A Case Study on Chopin Mazurkas
Peter Grosche, Meinard Müller and Craig Stuart Sapp
ABSTRACT – The automated extraction of tempo and beat information from music recordings is a challenging task. Especially in the case of expressive performances, current beat tracking approaches still have significant problems to accurately capture local tempo deviations and beat positions. In this paper, we introduce a novel evaluation framework for detecting critical passages in a piece of music that are prone to tracking errors. Our idea is to look for consistencies in the beat tracking results over multiple performances of the same underlying piece. As another contribution, we further classify the critical passages by specifying musical properties of certain beats that frequently evoke trac ing errors. Finally, considering three conceptually different beat tracking procedures, we conduct a case study on the basis of a challenging test set that consists of a variety of piano performances of Chopin Mazurkas. Our experimental results not only make the limitations of state-of-the-art beat trackers explicit but also deepens the understanding of the underlying music material.
An Audio Processing Library for MIR Application Development in Flash
Posted by Paul in events, ismir, music information retrieval, research on August 13, 2010
An Audio Processing Library for MIR Application Development in Flash
Jeffrey Scott, Raymond Migneco, Brandon Morton, Christian M. Hahn, Paul Diefenbach and Youngmoo E. Kim
The Audio processing Library for Flash affords music-IR researchers the opportunity to generate rich, interactive, real-time music-IR driven applications. The various lev-els of complexity and control as well as the capability to execute analysis and synthesis simultaneously provide a means to generate unique programs that integrate content based retrieval of audio features. We have demonstrated the versatility and usefulness of ALF through the variety of applications described in this paper. As interest in mu sic driven applications intensifies, it is our goal to enable the community of developers and researchers in music-IR and related fields to generate interactive web-based media.
Music21: A Toolkit for Computer-Aided Musicology and Symbolic Music Data
Music21: A Toolkit for Computer-Aided Musicology and Symbolic Music Data
Michael Scott Cuthbert and Christopher Ariza
ABSTRACT – Music21 is an object-oriented toolkit for analyzing, searching, and transforming music in symbolic (score- based) forms. The modular approach of the project allows musicians and researchers to write simple scripts rapidly and reuse them in other projects. The toolkit aims to pro- vide powerful software tools integrated with sophisticated musical knowledge to both musicians with little pro- gramming experience (especially musicologists) and to programmers with only modest music theory skills.
Music21 looks to be a pretty neat toolkit for analyzing and manipulating symbolic music. It’s like Echo Nest Remix for MIDI. The blog has lots more info: music21 blog. You can get the toolkit here: music21
State of the Art Report: Audio-Based Music Structure Analysis
Posted by Paul in events, ismir, music information retrieval, research on August 13, 2010
State of the Art Report: Audio-Based Music Structure Analysis
Jouni Paulus, Meinard Müller and Anssi Klapuri
ABSTRACT - Humans tend to organize perceived information into hierarchies and structures, a principle that also applies to music. Even musically untrained listeners unconsciously analyze and segment music with regard to various musical aspects, for example, identifying recurrent themes or detecting temporal boundaries between contrasting musical parts. This paper gives an overview of state-of-the- art methods for computational music structure analysis, where the general goal is to divide an audio recording into temporal segments corresponding to musical parts and to group these segments into musically meaningful categories. There are many different criteria for segmenting and structuring music audio. In particular, one can identify three conceptually different approaches, which we refer to as repetition-based, novelty-based, and homogeneity- based approaches. Furthermore, one has to account for different musical dimensions such as melody, harmony, rhythm, and timbre. In our state-of-the-art report, we address these different issues in the context of music structure analysis, while discussing and categorizing the most relevant and recent articles in this field.
This presentation is an overview of the music structure analysis problem, and the methods proposed for solving it. The methods have been divided into three categories: novelty-based approaches, homogeneity-based approaches, and repetition-based approaches. The comparison of different methods has been problematic because of the differring goals, but current evaluations suggest that none of the approaches is clearly superior at this time, and that there is still room for considerable improvements.
