Toward the Scientific Evaluation of Music Information Retrieval Systems

Show full item record

Title: Toward the Scientific Evaluation of Music Information Retrieval Systems
Author: J. Stephen Downie
Abstract: This paper outlines the findings-to-date of a project to assist in the efforts being made to establish a TREC-like evaluation paradigm within the Music Information Retrieval (MIR) research community. The findings and recommendations are based upon expert opinion garnered from members of the Information Retrieval (IR), Music Digital Library (MDL) and MIR communities with regard to the construction and implementation of scientifically valid evaluation frameworks. Proposed recommendations include the creation of data-rich query records that are both grounded in real-world requirements and neutral with respect to retrieval technique(s) being examined; adoption, and subsequent validation, of a “reasonable person” approach to “relevance” assessment; and, the development of a secure, yet accessible, research environment that allows researchers to remotely access the large-scale testbed collection.
URI: http://jhir.library.jhu.edu/handle/1774.2/7
Date: 2003-10-26
Subject: IR Systems and Algorithms
Digital Libraries

Files in this item

Files Size Format Download
paper.pdf 221.7Kb application/pdf Download

This item appears in the following Collection(s)

Show full item record