Stephen Downie presents the MIREX session

Statistics for 2009:

• 26 tasks
• 138 participants
• 289 evaluation runs

Results are now published: http://music-ir.org/r/09results

This year, new datasets:

• Mazurkas
• MIR 1K
• Back Chorales
• Chord and Segmentation datasets
• Mood dataset
• Tag-a-Tune

Evalutron 6K – Human evaluations – this year, 50 graders / 7500 possible grading events.

What’s Next?

• NEMA
• End-to-End systems and tasks
• Qualitative assessments
• Possible Journal ‘Special Issue’
• MIREX 2010 early start: http://www.music-ir.org/mirex/2010/index.php/Main_Page
• Suggestions to model after the ACM MM Grand challenge

Issues about MIREX

• Rein in the parameter explosion
• Not rigorously tested algorithms
• Hard-coded parameters, path-separators, etc
• Poorly specified data inputs/outputs
• Dynamically linked libraries
• Windows submissions
• Pre-compiled Matlab/MEX Submissions
• The ‘graduation’ problem – Andreas and Cameron will be gone in summer.

Long discussion with people opining about tests, data.  Ben Fields had a particularly good point about trying to make MIREX  better reflect real systems that draw upon web resources.

Music Recommendation and playlisting

Session Chair:  Douglas Turnbull

CONTINUOUS PLSI AND SMOOTHING TECHNIQUES FOR HYBRID MUSIC RECOMMENDATION

by Kazuyoshi Yoshii and Masataka Goto

• Unexpected encounters with unknown songs is increasingly important.
• Want accurate and diversifed recommendations
• Use a probabilistic approach suitable to deal with uncertainty of rating histories
• Compares CF vs. content-based and his Hybrid filtering system

Approach: Use PLSI to create a  3-way aspect model: user-song-feature – the unobservable category regading genre, tempo, vocal age, popularity etc.  – pLSI typical patterns are given by relationships between users, songs and a limited number of topics.  Some drawbacks: PLSI needs discrete features, multinomial distributions are assumed.  To deal with this formulate continuous pLSI, use gaussian mixture models and can assume continuous distributions.  A drawback of continuous pLSI – local minimum problem and the hub problem.  Popular songs are recommended often because of the hubs.  How to deal with this: Gaussian parameter tying – this reduces the number of free parameters. Only the mixture weights vary. Artist-based song clustering: Train an artist-based model and update it to a song-based model by an incremental training method (from 2007).

Here’s the system model:

Evaluation: They found that using the  techniques to adjust model complexity significantly improved the accuracy of recommendations and that the second technique could also reduce hubness.

STEERABLE PLAYLIST GENERATION BY LEARNING SONG SIMILARITY FROM RADIO STATION PLAYLISTS

François Maillet, Douglas Eck, Guillaume Desjardins, Paul Lamere

This paper presents an approach to generating steerable playlists. They first demonstrate a method for learning song transition probabilities from audio features extracted from songs played in professional radio station playlists and then show that by using this learnt similarity function as a prior, they are able to generate steerable playlists by choosing the next song to play not simply based on that prior, but on a tag cloud that the user is able to manipulate to express the high-level characteristics of the music he wishes to listen to.

• Learn a similarity space from commercial radion staion playlists
• generate steerable playlists

Francois defines a playlist. Data sources: Radio Paradise and Yes.com’s API.  7million tracks,

Problem:  They had positive examples but didn’t have an explicit set of negative examples.   Chose them at random.

Learning the song space: Trained a binary classifier to determine if a song sequence is real.

Features: Timbre, Rhythmic/dancability, loudness

Eval:

EVALUATING AND ANALYSING DYNAMIC PLAYLIST GENERATION HEURISTICS USING RADIO LOGS AND FUZZY SET THEORY

Klaas Bosteels, Elias Pampalk, Etienne Kerr

Abstract: In this paper, we analyse and evaluate several heuristics for adding songs to a dynamically generated playlist. We explain how radio logs can be used for evaluating such heuristics, and show that formalizing the heuristics using fuzzy set theory simplifies the analysis. More concretely, we verify previous results by means of a large scale evaluation based on 1.26 million listening patterns extracted from radio logs, and explain why some heuristics perform better than others by analysing their formal definitions and conducting additional evaluations.

Notes:

• Dynamic playlist generation
• Formalization using fuzzy sets. Sets of accepted songs and sets of rejected songs
• Why last two songs not accepted? To make sure the listener is still paying attention?
• Interesting observation that the thing that matters most is membership in the fuzzy set of rejected songs. Why? Inconsistent skipping behavior.

SMARTER THAN GENIUS? HUMAN EVALUATION OF MUSIC RECOMMENDER SYSTEMS.

Luke Barrington, Reid Oda, Gert Lanckriet

Abstract: Genius is a popular commercial music recommender sys- tem that is based on collaborative filtering of huge amounts of user data. To understand the aspects of music similarity that collaborative filtering can capture, we compare Genius to two canonical music recommender systems: one based purely on artist similarity, the other purely on similarity of acoustic content. We evaluate this comparison with a user study of 185 subjects. Overall, Genius produces the best recommendations. We demonstrate that collaborative filter- ing can actually capture similarities between the acoustic content of songs. However, when evaluators can see the names of the recommended songs and artists, we find that artist similarity can account for the performance of Genius. A system that combines these musical cues could generate music recommendations that are as good as Genius, even when collaborative filtering data is unavailable.

Great talk, lots of things to think about.

Poster madness! Version 2 – even faster this time. I can’t keep up

1. Singing Pitch Extraction – Taiwan
2. Usability Evaluation of Visualization interfaces for content-based music retrieval – looks really cool! 3D
3. Music Paste – concatenating music clipbs based on chroma and rhythm features
4. Musical bass-line pattern clustering and its application aduio gener classification
5. Detecting cover sets – looks nice – visualization – MTG
6. Using Musical Structure to enhance automatic chord transcription –
7. Visualizing Musical Structure from performance gesture – motion
8. From low-level to song-level percussion descriptors of polyphonic music
9. MTG – Query by symbolic example – use a DNA/Blast type approach
10. sten – web-based approach to determine the origin of an artist – visualizations
11. XML-format for any kind of time related symbolic data
12. Erik Schmidt – FPGA feature extraction. MIR for devices
13. Accelerating QBH – another hardware solution – 160 times faster
14. Learning to control a reverberator using subjective perceptual descriptors –  more boomy
15. Interactive GTTM Analyzer –
16. Estimating the error distribution of a tap sequence without ground Truth – Roger Dannenburg
17. Cory McKay – ACE XML – Standard formats for features, metadata, labels  and class ontologies
18. An efficient multi-resolution spectral transform for music analysis
19. Evaluation of multiple F0 estimation and tracking systems

BTW – Oscar informs me that this is not the first ever poster madness – there was one in Barcelona