TY - GEN
T1 - Investigating the Effect of Sound-Event Loudness on Crowdsourced Audio Annotations
AU - Cartwright, Mark
AU - Salamon, Justin
AU - Seals, Ayanna
AU - Nov, Oded
AU - Bello, Juan Pablo
N1 - Funding Information:
This work was partially supported by National Science Foundation award 1544753.
Publisher Copyright:
© 2018 IEEE.
PY - 2018/9/10
Y1 - 2018/9/10
N2 - Audio annotation is an important step in developing machine-listening systems. It is also a time consuming process, which has motivated investigators to crowdsource audio annotations. However, there are many factors that affect annotations, many of which have not been adequately investigated. In previous work, we investigated the effects of visualization aids and sound scene complexity on the quality of crowdsourced sound-event annotations. In this paper, we extend that work by investigating the effect of sound-event loudness on both sound-event source annotations and sound-event proximity annotations. We find that the sound class, loudness, and annotator bias affect how listeners annotate proximity. We also find that loudness affects recall more than precision and that the strengths of these effects are strongly influenced by the sound class. These findings are not only important for designing effective audio annotation processes, but also for effectively training and evaluating machine-listening systems.
AB - Audio annotation is an important step in developing machine-listening systems. It is also a time consuming process, which has motivated investigators to crowdsource audio annotations. However, there are many factors that affect annotations, many of which have not been adequately investigated. In previous work, we investigated the effects of visualization aids and sound scene complexity on the quality of crowdsourced sound-event annotations. In this paper, we extend that work by investigating the effect of sound-event loudness on both sound-event source annotations and sound-event proximity annotations. We find that the sound class, loudness, and annotator bias affect how listeners annotate proximity. We also find that loudness affects recall more than precision and that the strengths of these effects are strongly influenced by the sound class. These findings are not only important for designing effective audio annotation processes, but also for effectively training and evaluating machine-listening systems.
KW - Audio annotations
KW - Crowdsourcing
KW - Machine listening
KW - Sound event detection
UR - http://www.scopus.com/inward/record.url?scp=85054244462&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85054244462&partnerID=8YFLogxK
U2 - 10.1109/ICASSP.2018.8461833
DO - 10.1109/ICASSP.2018.8461833
M3 - Conference contribution
AN - SCOPUS:85054244462
SN - 9781538646588
T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
SP - 341
EP - 345
BT - 2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018 - Proceedings
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2018 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2018
Y2 - 15 April 2018 through 20 April 2018
ER -