Statistical Depth for Text Data: An Application to the Classification of Healthcare Data

This manuscript introduces a new concept of statistical depth function: the compositional D-depth. It is the first data depth developed exclusively for text data, in particular, for those data vectorized according to a frequency-based criterion, such as the tf-idf (term frequency?inverse document fr...

Descripción completa

Detalles Bibliográficos
Autores: Bolívar Gómez, Sergio, Nieto Reyes, Alicia|||0000-0002-0268-3322, Rogers, Heather L.
Tipo de recurso: artículo
Fecha de publicación:2023
País:España
Institución:Universidad de Cantabria (UC)
Repositorio:UCrea Repositorio Abierto de la Universidad de Cantabria
Idioma:inglés
OAI Identifier:oai:repositorio.unican.es:10902/27736
Acceso en línea:https://hdl.handle.net/10902/27736
Access Level:acceso abierto
Palabra clave:Compositional Depth
Multivariate Data
Natural Language Processing
Qualitative Data
Statistical Depth
Supervised Classification
Text Mining
id ES_2ebfc3df7c2e97eed9ece59c2c1512b0
oai_identifier_str oai:repositorio.unican.es:10902/27736
network_acronym_str ES
network_name_str España
repository_id_str
spelling Statistical Depth for Text Data: An Application to the Classification of Healthcare DataBolívar Gómez, SergioNieto Reyes, Alicia|||0000-0002-0268-3322Rogers, Heather L.Compositional DepthMultivariate DataNatural Language ProcessingQualitative DataStatistical DepthSupervised ClassificationText MiningThis manuscript introduces a new concept of statistical depth function: the compositional D-depth. It is the first data depth developed exclusively for text data, in particular, for those data vectorized according to a frequency-based criterion, such as the tf-idf (term frequency?inverse document frequency) statistic, which results in most vector entries taking a value of zero. The proposed data depth consists of considering the inverse discrete Fourier transform of the vectorized text fragments and then applying a statistical depth for functional data, D. This depth is intended to address the problem of sparsity of numerical features resulting from the transformation of qualitative text data into quantitative data, which is a common procedure in most natural language processing frameworks. Indeed, this sparsity hinders the use of traditional statistical depths and machine learning techniques for classification purposes. In order to demonstrate the potential value of this new proposal, it is applied to a real-world case study which involves mapping Consolidated Framework for Implementation and Research (CFIR) constructs to qualitative healthcare data. It is shown that the DDG -classifier yields competitive results and outperforms all studied traditional machine learning techniques (logistic regression with LASSO regularization, artificial neural networks, decision trees, and support vector machines) when used in combination with the newly defined compositional D-depth.Funding: A.N.-R. is supported by Grant 21.VP67.64662 funded by “Proyectos Puente 2022” from the Spanish Government of Cantabria. For H.L.R., the qualitative data used in study were funded by Instituto de Salud Carlos III through the project “PI17/02070” (co-funded by the European Regional Development Fund/European Social Fund “A way to make Europe”/“Investing in your future”) and the Basque Government Department of Health project “2017111086”. The funding bodies had no role in the design of the study, collection, analysis, interpretation of data nor the writing of the manuscript.MDPIUniversidad de Cantabria20232023-01-01journal articlehttp://purl.org/coar/resource_type/c_6501NAhttp://purl.org/coar/version/c_be7fb7dd8ff6fe43info:eu-repo/semantics/articlehttps://hdl.handle.net/10902/27736Mathematics, 2023, 11(1), 228reponame:UCrea Repositorio Abierto de la Universidad de Cantabriainstname:Universidad de Cantabria (UC)Inglésengopen accesshttp://purl.org/coar/access_right/c_abf2Attribution 4.0 Internationalhttp://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccessoai:repositorio.unican.es:10902/277362026-06-02T12:39:31Z
dc.title.none.fl_str_mv Statistical Depth for Text Data: An Application to the Classification of Healthcare Data
title Statistical Depth for Text Data: An Application to the Classification of Healthcare Data
spellingShingle Statistical Depth for Text Data: An Application to the Classification of Healthcare Data
Bolívar Gómez, Sergio
Compositional Depth
Multivariate Data
Natural Language Processing
Qualitative Data
Statistical Depth
Supervised Classification
Text Mining
title_short Statistical Depth for Text Data: An Application to the Classification of Healthcare Data
title_full Statistical Depth for Text Data: An Application to the Classification of Healthcare Data
title_fullStr Statistical Depth for Text Data: An Application to the Classification of Healthcare Data
title_full_unstemmed Statistical Depth for Text Data: An Application to the Classification of Healthcare Data
title_sort Statistical Depth for Text Data: An Application to the Classification of Healthcare Data
dc.creator.none.fl_str_mv Bolívar Gómez, Sergio
Nieto Reyes, Alicia|||0000-0002-0268-3322
Rogers, Heather L.
author Bolívar Gómez, Sergio
author_facet Bolívar Gómez, Sergio
Nieto Reyes, Alicia|||0000-0002-0268-3322
Rogers, Heather L.
author_role author
author2 Nieto Reyes, Alicia|||0000-0002-0268-3322
Rogers, Heather L.
author2_role author
author
dc.contributor.none.fl_str_mv Universidad de Cantabria
dc.subject.none.fl_str_mv Compositional Depth
Multivariate Data
Natural Language Processing
Qualitative Data
Statistical Depth
Supervised Classification
Text Mining
topic Compositional Depth
Multivariate Data
Natural Language Processing
Qualitative Data
Statistical Depth
Supervised Classification
Text Mining
description This manuscript introduces a new concept of statistical depth function: the compositional D-depth. It is the first data depth developed exclusively for text data, in particular, for those data vectorized according to a frequency-based criterion, such as the tf-idf (term frequency?inverse document frequency) statistic, which results in most vector entries taking a value of zero. The proposed data depth consists of considering the inverse discrete Fourier transform of the vectorized text fragments and then applying a statistical depth for functional data, D. This depth is intended to address the problem of sparsity of numerical features resulting from the transformation of qualitative text data into quantitative data, which is a common procedure in most natural language processing frameworks. Indeed, this sparsity hinders the use of traditional statistical depths and machine learning techniques for classification purposes. In order to demonstrate the potential value of this new proposal, it is applied to a real-world case study which involves mapping Consolidated Framework for Implementation and Research (CFIR) constructs to qualitative healthcare data. It is shown that the DDG -classifier yields competitive results and outperforms all studied traditional machine learning techniques (logistic regression with LASSO regularization, artificial neural networks, decision trees, and support vector machines) when used in combination with the newly defined compositional D-depth.
publishDate 2023
dc.date.none.fl_str_mv 2023
2023-01-01
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
NA
http://purl.org/coar/version/c_be7fb7dd8ff6fe43
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/10902/27736
url https://hdl.handle.net/10902/27736
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution 4.0 International
http://creativecommons.org/licenses/by/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution 4.0 International
http://creativecommons.org/licenses/by/4.0/
eu_rights_str_mv openAccess
dc.publisher.none.fl_str_mv MDPI
publisher.none.fl_str_mv MDPI
dc.source.none.fl_str_mv Mathematics, 2023, 11(1), 228
reponame:UCrea Repositorio Abierto de la Universidad de Cantabria
instname:Universidad de Cantabria (UC)
instname_str Universidad de Cantabria (UC)
reponame_str UCrea Repositorio Abierto de la Universidad de Cantabria
collection UCrea Repositorio Abierto de la Universidad de Cantabria
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869405431678369792
score 15,300719

Similares en LA Referencia