Voice assignment in vocal quartets using deep learning models based on pitch salience
This paper deals with the automatic transcription of four-part, a cappella singing, audio performances. In particular, we exploit an existing, deep-learning based, multiple F0 estimation method and complement it with two neural network architectures for voice assignment (VA) in order to create a mus...
| Autores: | , |
|---|---|
| Tipo de recurso: | artículo |
| Estado: | Versión publicada |
| Fecha de publicación: | 2022 |
| País: | España |
| Institución: | Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya) |
| Repositorio: | Recercat. Dipósit de la Recerca de Catalunya |
| OAI Identifier: | oai:recercat.cat:10230/55358 |
| Acceso en línea: | http://hdl.handle.net/10230/55358 http://dx.doi.org/10.5334/tismir.121 |
| Access Level: | acceso abierto |
| Palabra clave: | voice assignment multi-pitch estimation music information retrieval vocal quartets polyphonic vocal music deep learning |
| id |
ES_7d83e89bde28cdda88b485abe6a29fdd |
|---|---|
| oai_identifier_str |
oai:recercat.cat:10230/55358 |
| network_acronym_str |
ES |
| network_name_str |
España |
| repository_id_str |
|
| spelling |
Voice assignment in vocal quartets using deep learning models based on pitch salienceCuesta, HelenaGómez Gutiérrez, Emilia, 1975-voice assignmentmulti-pitch estimationmusic information retrievalvocal quartetspolyphonic vocal musicdeep learningThis paper deals with the automatic transcription of four-part, a cappella singing, audio performances. In particular, we exploit an existing, deep-learning based, multiple F0 estimation method and complement it with two neural network architectures for voice assignment (VA) in order to create a music transcription system that converts an input audio mixture into four pitch contours. To train our VA models, we create a novel synthetic dataset by collecting 5381 choral music scores from public-domain music archives, which we make publicly available for further research. We compare the performance of the proposed VA models on different types of input data, as well as to a hidden Markov model-based baseline system. In addition, we assess the generalization capabilities of these models on audio recordings with differing pitch distributions and vocal music styles. Our experiments show that the two proposed models, a CNN and a ConvLSTM, have very similar performance, and both of them outperform the baseline HMM-based system. We also observe a high confusion rate between the alto and tenor voice parts, which commonly have overlapping pitch ranges, while the bass voice has the highest scores in all evaluated scenarios.This work is partially supported by the European Commission under the TROMPA project (H2020 770376), the Spanish Ministry of Science and Innovation under the Musical AI project (PID2019-111403GB-I00), and by AGAUR (Generalitat de Catalunya) through an FI Predoctoral Grant (2018FI-B01015).Ubiquity Press202320232022info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfapplication/pdfhttp://hdl.handle.net/10230/55358http://dx.doi.org/10.5334/tismir.121reponame:Recercat. Dipósit de la Recerca de Catalunyainstname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)InglésTransactions of the International Society for Music Information Retrieval. 2022;5(1):99-112.https://github.com/helenacuesta/voas-vocal-quartetshttps://doi.org/10.5334/tismir.121.s1info:eu-repo/grantAgreement/EC/H2020/770376info:eu-repo/grantAgreement/ES/2PE/PID2019-111403GB-I00© 2022 The Author(s). This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC-BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. See http://creativecommons.org/licenses/by/4.0/.http://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccessoai:recercat.cat:10230/553582026-05-29T05:05:01Z |
| dc.title.none.fl_str_mv |
Voice assignment in vocal quartets using deep learning models based on pitch salience |
| title |
Voice assignment in vocal quartets using deep learning models based on pitch salience |
| spellingShingle |
Voice assignment in vocal quartets using deep learning models based on pitch salience Cuesta, Helena voice assignment multi-pitch estimation music information retrieval vocal quartets polyphonic vocal music deep learning |
| title_short |
Voice assignment in vocal quartets using deep learning models based on pitch salience |
| title_full |
Voice assignment in vocal quartets using deep learning models based on pitch salience |
| title_fullStr |
Voice assignment in vocal quartets using deep learning models based on pitch salience |
| title_full_unstemmed |
Voice assignment in vocal quartets using deep learning models based on pitch salience |
| title_sort |
Voice assignment in vocal quartets using deep learning models based on pitch salience |
| dc.creator.none.fl_str_mv |
Cuesta, Helena Gómez Gutiérrez, Emilia, 1975- |
| author |
Cuesta, Helena |
| author_facet |
Cuesta, Helena Gómez Gutiérrez, Emilia, 1975- |
| author_role |
author |
| author2 |
Gómez Gutiérrez, Emilia, 1975- |
| author2_role |
author |
| dc.subject.none.fl_str_mv |
voice assignment multi-pitch estimation music information retrieval vocal quartets polyphonic vocal music deep learning |
| topic |
voice assignment multi-pitch estimation music information retrieval vocal quartets polyphonic vocal music deep learning |
| description |
This paper deals with the automatic transcription of four-part, a cappella singing, audio performances. In particular, we exploit an existing, deep-learning based, multiple F0 estimation method and complement it with two neural network architectures for voice assignment (VA) in order to create a music transcription system that converts an input audio mixture into four pitch contours. To train our VA models, we create a novel synthetic dataset by collecting 5381 choral music scores from public-domain music archives, which we make publicly available for further research. We compare the performance of the proposed VA models on different types of input data, as well as to a hidden Markov model-based baseline system. In addition, we assess the generalization capabilities of these models on audio recordings with differing pitch distributions and vocal music styles. Our experiments show that the two proposed models, a CNN and a ConvLSTM, have very similar performance, and both of them outperform the baseline HMM-based system. We also observe a high confusion rate between the alto and tenor voice parts, which commonly have overlapping pitch ranges, while the bass voice has the highest scores in all evaluated scenarios. |
| publishDate |
2022 |
| dc.date.none.fl_str_mv |
2022 2023 2023 |
| dc.type.none.fl_str_mv |
info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion |
| format |
article |
| status_str |
publishedVersion |
| dc.identifier.none.fl_str_mv |
http://hdl.handle.net/10230/55358 http://dx.doi.org/10.5334/tismir.121 |
| url |
http://hdl.handle.net/10230/55358 http://dx.doi.org/10.5334/tismir.121 |
| dc.language.none.fl_str_mv |
Inglés |
| language_invalid_str_mv |
Inglés |
| dc.relation.none.fl_str_mv |
Transactions of the International Society for Music Information Retrieval. 2022;5(1):99-112. https://github.com/helenacuesta/voas-vocal-quartets https://doi.org/10.5334/tismir.121.s1 info:eu-repo/grantAgreement/EC/H2020/770376 info:eu-repo/grantAgreement/ES/2PE/PID2019-111403GB-I00 |
| dc.rights.none.fl_str_mv |
http://creativecommons.org/licenses/by/4.0/ info:eu-repo/semantics/openAccess |
| rights_invalid_str_mv |
http://creativecommons.org/licenses/by/4.0/ |
| eu_rights_str_mv |
openAccess |
| dc.format.none.fl_str_mv |
application/pdf application/pdf |
| dc.publisher.none.fl_str_mv |
Ubiquity Press |
| publisher.none.fl_str_mv |
Ubiquity Press |
| dc.source.none.fl_str_mv |
reponame:Recercat. Dipósit de la Recerca de Catalunya instname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya) |
| instname_str |
Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya) |
| reponame_str |
Recercat. Dipósit de la Recerca de Catalunya |
| collection |
Recercat. Dipósit de la Recerca de Catalunya |
| repository.name.fl_str_mv |
|
| repository.mail.fl_str_mv |
|
| _version_ |
1869411669622390784 |
| score |
15,811543 |