Analysis of inter-transcriber consistency in the Cat_ToBI prosodic labeling system
A set of tools to analyze inconsistencies observed in a Cat_ToBI labeling experiment are presented. We formalize and use the metrics that are commonly used in inconsistency tests. The metrics are systematically applied to analyze the robustness of every symbol and every pair of transcribers. The res...
| Autores: | , , , |
|---|---|
| Tipo de recurso: | artículo |
| Estado: | Versión aceptada para publicación |
| Fecha de publicación: | 2012 |
| País: | España |
| Institución: | Universitat Pompeu Fabra |
| Repositorio: | Repositorio Digital de la UPF |
| OAI Identifier: | oai:repositori.upf.edu:10230/27918 |
| Acceso en línea: | http://hdl.handle.net/10230/27918 http://dx.doi.org/10.1016/j.specom.2011.12.002 |
| Access Level: | acceso abierto |
| Palabra clave: | Prosody Prosodic labeling Inter-transcriber consistency ToBI |
| Sumario: | A set of tools to analyze inconsistencies observed in a Cat_ToBI labeling experiment are presented. We formalize and use the metrics that are commonly used in inconsistency tests. The metrics are systematically applied to analyze the robustness of every symbol and every pair of transcribers. The results reveal agreement rates for this study that are comparable to previous ToBI inter-reliability tests. The inter-transcriber confusion rates are transformed into distance matrices to use multidimensional scaling for visualizing the confusion between the different ToBI symbols and the disagreement between the raters. Potential different labeling criteria are identified and subsets of symbols that are candidates to be fused are proposed. |
|---|