Perceptual analysis of the f-measure for evaluating section boundaries in music: Proceedings of the 15th International Society for Music Information Retrieval Conference (ISMIR 2014)

Oriol Nieto, Morwaread M. Farbood, Tristan Jehan, Juan Pablo Bello

Research output: Contribution to conferencePaperpeer-review

Abstract

In this paper, we aim to raise awareness of the limitations of the F-measure when evaluating the quality of the boundaries found in the automatic segmentation of music. We present and discuss the results of various experiments where subjects listened to different musical excerpts containing boundary indications and had to rate the quality of the boundaries. These boundaries were carefully generated from state-of-the-art segmentation algorithms as well as human-annotated data. The results show that humans tend to give more relevance to the precision component of the F-measure rather than the recall component, therefore making the classical F-measure not as perceptually informative as currently assumed. Based on the results of the experiments, we discuss the potential of an alternative evaluation based on the F-measure that emphasizes precision over recall, making the section boundary evaluation more expressive and reliable.

Original languageEnglish (US)
Pages265-270
Number of pages6
StatePublished - 2014
Event15th International Society for Music Information Retrieval Conference, ISMIR 2014 - Taipei, Taiwan, Province of China
Duration: Oct 27 2014Oct 31 2014

Conference

Conference15th International Society for Music Information Retrieval Conference, ISMIR 2014
CountryTaiwan, Province of China
CityTaipei
Period10/27/1410/31/14

ASJC Scopus subject areas

  • Music
  • Information Systems

Fingerprint Dive into the research topics of 'Perceptual analysis of the f-measure for evaluating section boundaries in music: Proceedings of the 15th International Society for Music Information Retrieval Conference (ISMIR 2014)'. Together they form a unique fingerprint.

Cite this