Archive for category recommendation
I’m excited! Next week I travel to Austin for a week long computer+music geek-fest at SXSW. A big part of SXSW is the music – there are nearly 2,000 different artists playing at SXSW this year. But that presents a problem – there are so many bands going to SXSW (many I’ve never heard of) that I find it very hard to figure out which bands I should go and see. I need a tool to help me find sift through all of the artists – a tool that will help me decide which artists I should add to my schedule and which ones I should skip. I’m not the only one who was daunted by the large artist list. Taylor McKnight, founder of SCHED*, was thinking the same thing. He wanted to give his users a better way to plan their time at SXSW. And so over a couple of weekends Taylor built (with a little backend support from us) The Unofficial Artist Discovery Guide to SXSW.
The Unofficial Artist Discovery Guide to SXSW is a tool that allows you to explore the many artists attending this year’s SXSW. It lets you search for artists, browse popularity, music style, ‘buzzworthiness’, or similarity to your favorite artists – and it will make recommendations for you based on your music taste (using your Last.fm, Sched* or Hype Machine accounts) . The Artist Guide supplies enough context (bios, images, music, tag clouds, links) to help you decide if you might like an artist.
Here’s the guide:
Here’s a quick tour of some of the things you can do with the guide. First off, you can Search for artists by name, genre/tag or location. This helps you find music when you know what you are looking for.
However, you may not always be sure what you are looking for – that’s where you use Discover. This gives you recommendations based on the music you already like. Type in the name of a few artists (even artists that are not playing at SXSW) or your SCHED*, Hype Machine or Last.fm user name, and ‘Discover’ will give you a set of recommendations for SXSW artists based on your music taste. For example, I’ve been listening to Charlotte Gainsbourg lately so I can use the artist guide to help me find SXSW artists that I might like:
If I see an artist that looks interesting I can drill down and get more info about the artist:
I use Last.fm quite a bit, so I can enter my Last.fm name and get SXSW recommendations based upon my Last.fm top artists. The artist guide tries to mix things up a little bit so if I don’t like the recommendations I see, I can just ask again and I can get a different set. Here are some recommendations based on my recent listening at Last.fm:
If you’ve been using the wonderful SCHED* to keep track of your SXSW calendar you can use the guide to get recommendations based on artists that you’ve already added to your SXSW calendar.
In addition to search and discovery, the guide gives you a number of different ways to browse the SXSW Artist space. You can browse by ‘buzzworthy’ artists – these are artists that are getting the most buzz on the web:
Or the most well-known artists:
You can browse by the style of music via a tag cloud:
And by venue:
Building the guide was pretty straightforward. Taylor used the Echo Nest APIs to get the detailed artist data such as familiarity, popularity, artist bios, links, images, tags and audio. The only data that was not available at the Echo Nest was the venue and schedule info which was provided by Arkadiy (one of Taylor’s colleagues). Even though SXSW artists can be extremely long tail (some don’t even have Myspace pages), the Echo Nest was able to provide really good coverage for these sets (There was coverage for over 95% of the artists). Still there are a few gaps and I suspect there may be a few errors in the data (my favorite wrong image is for the band Abe Vigoda). If you are in a band that is going to SXSW and you see that we have some of your info wrong, send me an email (email@example.com) and I’ll make it right.
We are excited to see the this Artist Discovery guide built on top of the Echo Nest. It’s a great showcase for the Echo Nest developer platform and working with Taylor was great. He’s one of these hyper-creative, energetic types – smart, gets things done and full of new ideas. Taylor may be adding a few more features to the guide before SXSW, so stay tuned and we’ll keep you posted on new developments.
When I test-drive a new music recommender I usually start by getting recommendations based upon ‘The Beatles’ (If you like the Beatles, you make like XX). Most recommenders give results that include artists like John Lennon, Paul McCartney, George Harrison, The Who, The Rolling Stones, Queen, Pink Floyd, Bob Dylan, Wings, The Kinks and Beach Boys. These recommendations are reasonable, but they probably won’t help you find any new music. The problem is that these recommenders rely on the wisdom of the crowds and so an extremely popular artist like The Beatles tends to get paired up with other popular artists – the results being that the recommender doesn’t tell you anything that you don’t already know. If you are trying to use a recommender to discover music that sounds like The Beatles, these recommenders won’t really help you – Queen may be an OK recommendation, but chances are good that you already know about them (and The Rolling Stones and Bob Dylan, etc.) so you are not finding any new music.
At The Echo Nest we don’t base our artist recommendations solely on the wisdom of crowds, instead we draw upon a number of different sources (including a broad and deep crawl of the music web). This helps us avoid the popularity biases that lead to ineffectual recommendations. For example, looking at some of the Echo Nest recommendations based upon the Beatles we find some artists that you may not see with a wisdom of the crowds recommender – artists that actually sound like the Beatles – not just artists that happened to be popular at the same time as the Beatles. Echo Nest recommendations include artists such as The Beau Brummels , The Dukes of Stratosphear, Flamin’ Groovies and an artist named Emitt Rhodes. I had never ever seen Emitt Rhodes occur in any recommendation based on the Beatles, so I was a bit skeptical, but I took a listen and this is what I found:
Update: Don Tillman points to this Beatle-esque track:
Emitt could be the sixth Beatles. I think it’s a pretty cool recommendation
Last week NPR’s On the Media had a special show called ‘The Future of Music’ – all about the current state of the music industry and where it is all going. The hour is broken into a number of sections:
- Facing the (Free) music – about what has happened in the 10 years since Napster – Yep Spotify gets a mention. Choice quote by Hilary Rosen – “Napster was a missed opportunity’
- They Say That I stole this – about the legalities of sampling (with interviews with Girl Talk among others)
- Played Out – interview with John Scher about the state of live music
- Teens on Tunes – interviews with teens about where they get their music. Answer: Limewire
- Charting the Charts – interesting piece about the charts – the history of billboard, and the next generation of tracking including an interview with Bandmetrics founder Duncan Freeman (way to go Duncan!)
- Why I’m not afraid to take your money – interesting interview with Amanda Palmer about how artists make money in today’s music world
One thing that they didn’t talk about at all was music discovery – no mention of the role of the critic, music blogs, hype machine, no discussion of the role social sites like last.fm play in music discovery, no mention of automated tools for music discovery like recommenders and playlisters. Maybe next year, when everyone has access to infinite music, we’ll see more emphasis on discovery tools.
It was a great show. Highly recommended: NPR’s On the Media Special Edition: The Future of the Music Industry
This year ISMIR concludes with the 1st Workshop on the Future of MIR. The workshop is organized by students who are indeed the future of MIR.
MIR, where we are, where we are going
Session Chair: Amélie Anglade Program Chair of f(MIR)
Meaningful Music Retrieval
Frans Wiering – [pdf]
- Some unfortunate tendencies: anatomical view of music – a dead body that we do autopsies, time is the loser Traditional production-oriented/
- Measure of similarity: relevance, surprise
- Few interesting applications for end-users
- bad fit to present-day musicological themes
- We are in the world of ‘pure applied research’ – no truth interdisciplinary between music domain knowledge and computer science.
- Music is meaningful (and the underlying personal motivation of most MIR researchers).
- Meaning in musicology – traditionally a taboo suject
- Subjectivity: an indivds. disposition to engage in social and cultural interactions
- Meaning generation process – we have a long-term memory for music –
- Can musical meaning provide the ‘big story line’ for MIR?
The Discipline Formerly Known As MIR
Perfecto Herrera, Joan Serrà, Cyril Laurier, Enric Guaus, Emilia Gómez and Xavier Serra
Intro: Our exploration is not a science-fiction essay. We do not try to imagine how music will be conceptualized, experienced and mediated by our yet-to-come research, technological achievements and music gizmos. Alternatively, we reflect on how the discipline should evolve to become consolidated as such, in order it may get an effective future instead of becoming, after a promising start, just a “would-be” discipline.Our vision addresses different aspects: the discipline’s object of study, the employed methodologies, social and cultural impacts (which are out of this long abstract because of space restrictions), and we finish with some (maybe) disturbing issues that could be taken as partial and biased guidelines for future research.
Notes: One motivation for advancing MIR – more banquets!
- MIR is no more about retrieval than computer science is about computers
- Music Information Retrieval – it’s too narrow
- Music Information or Information about Music?
- Interested in the interaction with music information
- We should be asking more profound questions
- content tresasures in short musical exceprts, tracks performances etc.
- music understanding systems
- Most metadata will be generated in the creation / production phase (hmm.. don’t agree necessarily, all the good metadata (tags, who likes what) is based on context and use which is post-hoc)
- Instead of automatic analysis – build systems to help humans help humans
- Music like water? or Music as dog!!! – a friend – companion –
- Personalization, Findability
- Music turing test
Good, provocative talk
Oral Session 2: Potential future MIR applications
Session Chair: Jason Hockman (McGill University), Program Chair of f(MIR)
Machine Listening to Percussion: Current Approaches and Future Directions – [pdf]
Abstract: approaches have been taken to detect and classify percussive events within music signals for a variety of purposes with differing and converging aims. In this paper an overview of those technologies is presented and a discussion of the issues still to overcome and future possibilities in the field are presented. Finally a system capable of monitoring a student drummer is envisaged which draws together current approaches and future work in the field.
- Challengs: Onset detection of isolated drum strokes
- Onset detection and classification of overlapping drum sounds
- Onset detection and classification in the presence of other instruments
- Variability in Percussive sounds . Dozens of criteria effect the sounds produced (strike velocity, angle, position etc.)
- Future Research Areas
- Extension of recognition to include the wide variety of strokes. (open hh, half-open hh, hh foot splash etc)
MIR When All Recordings Are Gone: Recommending Live Music in Real-Time – [pdf]
Marco Lüthy and Jean-Julien Aucouturier
Recommending live and short lived events. Bandsintown, Songkick, gigulate … pay attention to this paper.
- Recommendation for live music in real-time
- Coldplay -> free album when you get a ticket to a coldplay concert – give away the music
- NIN -> USB keys in the toilet – which had strange recording on the file – strange sounds – an FFT of the sounds showed phone number and GPS coordinates – turned into a treasure hunt to a NIN nails concert.
- Komuso Tokugawa – an avatar for a musiciaon in second life. Plays in second life, twitters concert announcements (playing wake for Les Paul in 3 minutes)
- ‘How do we get there in time?’
- JJ walked through how to implement a recommender system in second life
- Implicit preference inferred from how long your avatar listens to a concert (Nicole Yankelovich at Sun Labs should look at this stuff)
- Great talk by JJ – full of energy – neat ideas. Good work.
- Global Access to Ethnic Music: The Next Big Challenge?
Olmo Cornelis, Dirk Moelants and Marc Leman
- The Future of Music IR: How Do You Know When a Problem Is Solved?
Eric Nichols and Donald Byrd
Sten has created a mobile music discovery application that runs on a mobile device. The application shows similar artists using Echo Nest data. You can read about the app and give it a try (it runs on a desktop too), on Sten’s Blog: Music Explorer FX Mobile Edition
On Saturday morning I opened my web browser, built a playlist of a few songs and started to listen to them while I went about my morning computer tasks. Some of the songs in the playlist were on my laptop, while some were on the mac mini in the family room, and some were on a laptop of a friend that was on the other side of the Atlantic ocean. And if my friend in London had closed his laptop before I listened to ‘his’ song on my playlist it could have been replaced by a copy of the song that was on the computer of a friend in Seattle. I had a seamless music listening experience despite the fact that the music was scattered across a handful of computers on two continents. Such is the power of Playdar.
Playdar is a music content resolver. It is designed to solve one problem: given the name of track, find me a way to listen to it right now. You run Playdar on any computer that you own that has music and Playdar will make it easy to listen to all of that music as if it were on your local machine. The Playdar content resolver can also talk to other Playdar resolvers too, so if Playdar can’t find a track on my local network, it can ask my friend if it knows where the track is, extending my listening reach.
Playdar runs as a web service using standard web protocols for communicating with applications. When Playdar receives a request to resolve a track it runs through a list of prioritized content resolvers looking for the track. First it checks your local machine, then your local network. If it hasn’t found it there it could, if so configured, try your friends computers, or even a commercial song resolver (One could imagine for example, a music label offering up a portion of their catalog via a content resolver as a way to expose more listeners to their music). Playdar will do its best to find a copy of a song that you can listen to now. Playdar enables a number of new listening modes:
- Listen to my music anywhere – with Playdar, I don’t have to shoehorn my entire music collection onto every computer that I own just so I can listen to it no matter what computer I’m on. I can distribute my music collection over all my computers – and no matter what computer I’m on I have all my music available.
- Save money for music streamers – Music streaming services like Last.fm, Spotify and Pandora spend money for every song that is streamed. Often times, the listener will already own the song that is being streamed. Playdar-enabled music streaming services could save streaming costs by playing a local copy of a song if one is available.
- Share playlists and mixtapes – with Playdar a friend could give me a playlist (perhaps in a XSPF format) and I could listen to the playlist even if I don’t own all of the songs.
- Pool the music – At the Echo Nest, everyone has lots of music in their personal collections. When we are all in the same room it is fun to be able to sample music from each other. iTunes lets you do this but searching through 15 separate collections for music in iTunes is burdensome. With Playdar, all the music on all of the computers running Playdar on your local lan can be available for you to search and play without any of the iTunes awkwardness.
Playdar’s Marconi Moment
Playdar is the brainchild of RJ, the inventor of the audioscrobbler and one of the founders of Last.fm. RJ started coding Playdar in March of this year – but a few weeks ago he threw away the 10,000 lines of C++ code and started to rewrite it from scratch in Erlang. A few days later RJ tweeted: I should be taken aside and shot for using C++ for Playdar originally. It’s criminal how much more concise Erlang is for this. Less than 3 weeks after starting from a clean sheet of paper, the new Erlang-based Playdar had its first transatlantic track resolution and streaming. The moment occurred on Friday, October 16th. Here’s the transcript from the IRC channel (tobyp is Toby Padilla, of MusicMobs and Last.fm fame) when London-based RJ first streamed a track from Toby’s Seattle computer:
[15:40:46] <tobyp> http://www.playdar.org/demos/search.html#artist=pantera&album=&track=burnnn
[15:41:06] <RJ2> woo, transatlantic streaming
[15:41:19] <tobyp> hot!
[15:41:35] <RJ2> playdar’s marconi moment
[15:41:42] <tobyp> hahah
An incredible amount of progress has been made in the last two weeks, a testament to RJ’s skills as much as Erlang’s expressiveness. Still, Playdar is not ready for the general public. It requires a bit of work to install and get running – (yep, the erlang runtime is required), but developer Max Howell has been working on making a user-friendly package to make it easy for anyone to install. Hopefully it won’t be too long before Playdar is ready for the masses.
Even though it is new, there’s already some compelling apps that use Playdar. One is Playlick:
Playlick is a web application, developed by James Wheare that lets you build playlists. It uses Playdar for all music resolution. Type in the name of an album and Playlick / Playdar will find the music for you and let you listen to it. It’s a great way to see/hear the power of Playdar.
Adding custom content resolvers
One of the strengths of Playdar is that it is very easy to add new resolvers. If you are a music service provider you can create a Playdar content resolver that will serve up your content. I wrote a content resolver that uses the Echo Nest to resolve tracks using our index of audio that we’ve found on the web. This resolver can be used as a backstop. If you can’t find a track on your computer or your friend’s computers the Echo Nest resolver might be able to find a version out there on some music blog. Of course, the quality and availability of such free-range music is highly variable, so this resolver is a last resort.
Adding a new resolver to Playdar was extremely easy. It took perhaps 30 minutes to write – the hardest part was figuring out git – (thanks to RJ for walking me through the forks, pushes and ssh key settings). You can see the code here: echonest-resolver.py. Less than 150 lines of code, half of which is boilerplate. 150 lines and 30 minutes to add a whole new collection of music to the Playdar universe. Hopefully soon we’ll see resolvers for music streaming services like Napster, Rhapsody and Spotify.
What’s Next for Playdar?
Playdar is new – and the plumbing and wiring are still be worked on – but already it is doing something pretty magical – letting me listen to any track I want to right now. I can see how Playdar could be extended into acting as my music agent. Over time, my Playdar servers will get to know quite a bit about my music tastes. They’ll know what music I like to listen to, and when I like to listen to it. Perhaps someday, instead of asking Playdar to resolve a specific track by name, I’ll just be able to ask Playdar to give me a playlist of new music that I might like. Playdar can then use an Echo Nest, Last.fm or an AMG playlister to build a playlist of interesting, relevant new music. Playdar won’t just be a music resolver, Playdar will be my music agent helping me explore for and discover new music.
There has been quite a bit of rumor in the last couple of days that iLike is about to be acquired by MySpace. iLike is one of the biggest music apps in the Facebook world so it seems that this acquisition could set up an interesting dynamic between MySpace and Facebook. I’ve never been a big fan of iLike. It never has really worked for me as a music discovery site, instead it always seemed to me to be just another social web site that just happened to use music taste as a way to find new friends.
Back in October 2005, on the day when iLike first launched, I took the site for a spin and wrote about the rather poor iLike music recommendations. Six months later I checked again and their music recommendations were still really crappy. With iLike in the news, I decided to take one more look to see how there music recommendations have improved since 2005. Here’s what I found.
For my first test, I created an iLike radio station with a seed artist of Miles Davis, iLike happily added The Pogues, Christina Aguileira and the Dixie Chicks to the mix. That left me feeling kind of blue.
Next up, a little bit of James Brown – iLike filled out the playlist with the Pretenders and the electronic artist A.M. (and who is Carl Hatmaker? – this feels like a shill recommendation for an iLike/Garageband artist). Again, a playlist that left my neck hurting from the iPod whiplash as I was jerked from genre to genre.
Another try, some Aphex Twin. This leads to some PJ Harvey, The Buzzcocks and the Mars Volta. (ouch!)
Listening to Bob Marley – iLike gave me some Clapton, Moby and Queen.
It looks like today’s iLike music recommendations are not much better than they were back in October of 2005. A good fraction of the recommended artists are clunkers that don’t match the seed artist – sometimes feeling like anti-recommendations – (Christina may be just about as far away from Miles as one can get). They also like to sprinkle in their own Garageband artists which seems to me more like an artist promotion rather than an honest recommendation. After four years, I’m still not impressed with iLike’s music recommendations. When I’m looking for new music, I’ll continue to go somewhere else. But I’m open minded, I’ll be sure to check in again in four years to see if they’ve got it right.
Last week, on the Hype machine blog, Anthony indicated his increasing frustration in how easily charts could be manipulated – Anthony wanted a better way, one that was transparent, and gave more influence to the influential. Anthony’s solution was to create a twitter chart that is based on the twittering activity of Hype Machine songs. In this new chart Twitterers with more followers have more influence than those with few.
A number of commenters on Anthony’s blog pointed out how it would be easy for a single very popular twitter user to influence the charts. And that is exactly what Erick Schonfeld of TechCrunch did. Erick used the power of TechCrunch for evil.
With one tweet from the TechCrunch twitter account (with its nearly 1 million-person reach) he was able to put Rick Astley’s Never Gonna Give you Up at the top of the Hype Machine Twitter chart. Erick writes “The Hype Machine’s formula is flawed. No single person should be able to affect the rankings so easily“.
It’s arguable whether or not this is a dishonest manipulation of the charts. TechCrunch really does have a reach of 1 million people – and so by tweeting Rick Astley they are potentially exposing those millions to this song. However, in reality, people don’t read TechCrunch for music recommendations – TechCruch is just not a music tastemaker (sorry Erick). A tweet by TechCrunch counts much less than a tweet by Indie music guide Pitchfork.
Update – Note that the spammers are now starting to recognize the twitterverse as a place that they can target. If you have $27 you can get the twittertrafficmachine to get you 20K followers in a month:
Anthony should adjust how he scores a tweet to not only include the reach of the tweet but to also include the music reputation of the source. It is not as easy to determine the music reputation as the number of followers for a source, but it is much more important. Some indicators that a tweet has real influences are whether people actually click on the link and listen to the song and whether the poster actually listens to music, especially new music, before it gets popular.
I suspect Anthony will be tweaking his scoring algorithms soon to make the charts better reflect what real music listeners are listening to, not just what popular people are listening to.
Update: Anthony has responded in he comments.
The very popular blog aggregator The Hype Machine has a ‘Popular Page‘ that shows the tracks that have been most favorited in the last 3 days. This is a great way to find out what the music zeitgeist is. However, Anthony (Mr. Hype Machine) recently discovered that a number of highly favorited artists seemed to have reached the popular page by nefarious means. According to Anthony, it appears that a number of artists became popular when many presumably fake accounts, created from the same IP address in a very short period of time all favorited a single artist in an apparent effort to get the artist to appear on the popular page. This type of hacking is not too surprising – whenever you have a chart or poll that relies on ‘the wisdom of crowds’ you are susceptible to the shill who will try to manipulate the chart in order to promote their interests. We see this in online polls, social news sites and popular music sites.
When Anthony became aware of how the Hype Machine was being manipulated, he and the rest of the Hype machine team fought back, instituting a Captcha mechanism to prevent automated account creation, ignoring favoriting activity for new accounts, and keeping a much closer eye on new account activity.
But Anthony didn’t stop there, he went one step further. He named names. He posted on his blog a list of all the artists that, according to Anthony have “attempted to manipulate the charts on the Hype Machine”. Anthony says he published the list to “let everyone make their own judgments about quality, integrity and marketing strategies:”. But really, I suspect that Anthony’s real motivation was to shame those that would attempt to try to enlist the Hype Machine to promote their band.
A commenter on that blog post that claims membership in one of the outed shilling bands protests that they absolutely did not create fake accounts and they had been unfairly defamed (literally) by the Hype Machine. But Anthony responds with a list 4 tracks by the band that had each been favorited from a single IP address by over 40 separate, newly created accounts. Anthony says “Given that this is a time-consuming activity that primarily benefits you, you can see how it appears likely that you or your team may have been involved”.
Should Anthony have outed these artists? Surely the excessive favoriting could have been an overzealous fan that decided to try out a new way to hype their favorite band (to put the ‘hype’ in Hype Machine, if you will), and the band is blameless. But from Anthony’s point of view it doesn’t really matter. Anthony is going to protect the integrity of the Hype Machine and he’s going to do it by pointing to any band that has benefited from ‘unnatural’ enthusiasm. Even if it means public humiliation for the blameless.
I suspect Anthony’s next problem will occur when some pranksters realize that they can get any band blacklisted at the Hype Machine by a bit of nefarious activity. By simply creating a set of sham accounts and favoriting tracks by the vicitim band from those sham acounts, the Hype Machine can be manipulated into blacklisting and humilating the band. Is your ex-girlfriend’s new boyfriend in a band? Get your dorm floor to create 50 Hype Machine accounts, favorite his tracks and watch the fun as he gets outed and shamed as a shill.
The lesson here is that charts that show popularity are hard to get right – they can be easily manipulated for fun or for profit. Anthony should be prepared to fight an escalating war against those that want to manipulate his charts. And the more popular the Hype Machine becomes, the bigger the target it will be for the hackers and the shills.