ISMIR – MIREX Panel Discussion

Stephen Downie presents the MIREX session

Statistics for 2009:

26 tasks
138 participants
289 evaluation runs

Results are now published: http://music-ir.org/r/09results

This year, new datasets:

Mazurkas
MIR 1K
Back Chorales
Chord and Segmentation datasets
Mood dataset
Tag-a-Tune

Evalutron 6K – Human evaluations – this year, 50 graders / 7500 possible grading events.

What’s Next?

NEMA
End-to-End systems and tasks
Qualitative assessments
Possible Journal ‘Special Issue’
MIREX 2010 early start: http://www.music-ir.org/mirex/2010/index.php/Main_Page
Suggestions to model after the ACM MM Grand challenge

Issues about MIREX

Rein in the parameter explosion
Not rigorously tested algorithms
Hard-coded parameters, path-separators, etc
Poorly specified data inputs/outputs
Dynamically linked libraries
Windows submissions
Pre-compiled Matlab/MEX Submissions
The ‘graduation’ problem – Andreas and Cameron will be gone in summer.

Long discussion with people opining about tests, data. Ben Fields had a particularly good point about trying to make MIREX better reflect real systems that draw upon web resources.

ismir, ismir2009

This entry was posted on October 28, 2009, 2:41 am and is filed under events, ismir. You can follow any responses to this entry through RSS 2.0. Both comments and pings are currently closed.

Music Machinery