[tweetmeme source= ‘plamere’ only_single=false]
TL;DR; I built a game called Name Dropper that tests your knowledge of music artists.
One bit of data that we provide via our web APIs is Artist Familiarity. This is a number between 0 and 1 that indicates how likely it is that someone has heard of that artists. There’s no absolute right answer of course – who can really tell if Lady Gaga is more well known than Barbara Streisand or whether Elvis is more well known than Madonna. But we can certainly say that The Beatles are more well known, in general, than Justin Bieber.
To make sure our familiarity scores are good, we have a Q/A process where a person knowledgeable in music ranks our familiarity score by scanning through a list of artists ordered in descending familiarity until they start finding artists that they don’t recognize. The further they get into the list, the better the list is. We can use this scoring technique to rank multiple different familiarity algorithms quickly and accurately.
One thing I noticed, is that not only could we tell how good our familiarity score was with this technique, this also gives a good indication of how well the tester knows music. The further a tester gets into a list before they can’t recognize artists, the more they tend to know about music. This insight led me to create a new game: The Name Dropper.
The Name Dropper is a simple game. You are presented with a list of dozen artist names. One name is a fake, the rest are real.
If you find the fake, you go onto the next round, but if you get fooled, the game is over. At first, it is pretty easy to spot the fakes, but each round gets a little harder, and sooner or later you’ll reach the point where you are not sure, and you’ll have to guess. I think a person’s score is fairly representative of how broad their knowledge of music artists are.
The biggest technical challenge in building the application was coming up with a credible fake artist name generator. I could have used Brian’s list of fake names – but it was more fun trying to build one myself. I think it works pretty well. I really can’t share how it works since that could give folks a hint as to what a fake name might look like and skew scores (I’m sure it helps boost my own scores by a few points). The really nifty thing about this game is it is a game-with-a-purpose. With this game I can collect all sorts of data about artist familiarity and use the data to help improve our algorithms.
So go ahead, give the Name Dropper a try and see if you can push me out of the top spot on the leaderboard:
Play the Name Dropper
Music Machinery 
#1 by alsotop on July 10, 2010 - 2:53 pm
I think it was obvious your name generator never creates 1 word artist names, which meant ruling out some possible options when I got to the point where I had to guess…
#2 by Dan Geiser on July 10, 2010 - 4:20 pm
I was playing this in between activities at work and I noticed when I answered one question right my score went down. It did take me a particularly long time to answer. Is the score you get for a correct answer based on some sort of time factor as well?
#3 by Paul on July 10, 2010 - 4:28 pm
Dan – yes – there’s a penalty if you take too long to answer. That’s to prevent people from relying on Wikipedia to boost their scores.
#4 by Logan on July 20, 2010 - 11:56 am
“…we have a Q/A process where a person knowledgeable in music ranks our familiarity score by scanning through a list of artists ordered in descending familiarity until they start finding artists that they don’t recognize. The further they get into the list, the better the list is.”
Sounds a lot like a ROC score, but not quite as accurate. Instead of just stopping the first time they see an artist they don’t know, wouldn’t it better (and not that much more time consuming) to have them keep going while marking the ones they don’t know. Once they go say 10 or 20 unknown artists in a row they can stop with the assumption that the rest are unknown, and you can calculate a fairly accurate ROC score.
#5 by zazi on July 20, 2010 - 5:58 pm
Hi,
what about using this game for spelling correction? That means including wrong and right spellings of music artists and score these ones.
Furthermore, I don’t really think that the classic composer are that difficult (people often know these names from the music class in school). My problem was artists from music genres I’m not familar with. That means, it was in general to difficult for me.
Cheers,
zazi