Posts Tagged cal
UCSD Researcher Gert Lanckriet announced today that “Herd It” – the game-with-a-purpose for collecting audio annotations has been officially launched on Facebook. Following in the footsteps of other gwaps such as Major Miner, Tag-a-tune and the Listen Game.
On the music-ir mailing list Gert explains ‘Herd it’: “The scientific goal of this experiment is to investigate whether a human computation game integrated with the Facebook social network will allow the collection of high quality tags for audio clips. The quality of the tags will be tested by using them to train an automatic music tagging system (based on statistical models). Its predictive accuracy will be compared to a system trained on high quality tags collected through controlled human surveys (such as, e.g., the CAL500 data set). The central question we want to answer is whether the “game tags” can train an auto-tagging system as (or more) accurately than “survey tags” and, if yes, under what conditions (amount of tags needed, etc.). The results will be reported once enough data has been collected.”
I’ve played a few rounds of the game and enjoyed myself. I recognized all of the music that they played (it seemed to be drawn from top 100 artists like Nirvana, Led Zeppelin, Maria Carey and John Lennon). The timed rounds made the game move quickly. Overall, the game was fun. But I did miss the feeling of close collaboration that I would get from some other Gwaps where I would have to try to guess how my partner would try to describe a song. Despite this, I found the games to be fun and I could easily see spending a few hours trying to get a top score. The team at UCSD clearly has put lots of time into making the games highly interactive and fun. Animations, sound and transparent game play all add to the gaming experience. Once glitch, even though I was logged into Facebook, the Herd It game didn’t seem to know who I was, it just called me ‘Herd It’. So my awesome highscore is anonymous.
Here are some screen shots from the game. For this round, I had to chose the most prominent sound (this was for the song ‘Heart of Gold’), I chose slide guitar, but most people chose acoustic guitar (what do they know!).
For this round, I had to chose the genre for a song. easy enough.
For this round I had to position a song on a Thayer mood model scale.
Here’s the game kick off screen … as you can see, I’m “Herd it” and not Paul
I hope the Herd It game attracts lots of attention. It could be a great source of music metadata.
If you are interested in helping out, take the survey.
Here are the details from Luke:
With music similarity sites like Pandora.com or iTunes’ Genius feature that recommends playlists, based on a song that we like, our MIR domain of music similarity and recommendation is finding a mass audience. But are these systems any good? Could we make something better?
This is what I’m trying to figure out and I would like to include your opinion in my analysis.
We are conducting an experiment where you can listen to playlists that are recommended, based on a “seed song”, and evaluate these recommendations. We are comparing different recommendation systems, including Genius, artist similarity and tag-based similarity. Most importantly, we’re are trying to discover the important factors that go into creating and evalutating a playlist.
If you’d like to participate in the experiment by listening to and evaluating some playlists, please go to:
As an incentive, we’re offering a $20 iTunes gift card to whoever rates the most playlists (but it’s about quality, not quantity!)
To learn more, ask questions or make suggestions, feel free to drop me a line.
Thanks for your help,
Thanks for your help,
Computer Audition Laboratory
U.C. San Diego