Integer constraints for enhancing interpretability in linear regression

One of the main challenges researchers face is to identify the most relevant features in a prediction model. As a consequence, many regularized methods seeking sparsity have flourished. Although sparse, their solutions may not be interpretable in the presence of spurious coefficients and correlated...

Descripción completa

Detalles Bibliográficos
Autores: Carrizosa Priego, Emilio José, Olivares Nadal, Alba Victoria, Ramírez Cobo, Josefa
Tipo de recurso: artículo
Estado:Versión publicada
Fecha de publicación:2020
País:España
Institución:Universidad de Sevilla (US)
Repositorio:idUS. Depósito de Investigación de la Universidad de Sevilla
OAI Identifier:oai:idus.us.es:11441/104390
Acceso en línea:https://hdl.handle.net/11441/104390
https://doi.org/10.2436/20.8080.02.95
Access Level:acceso abierto
Palabra clave:Linear regression
Multicollinearity
Sparsity
Cardinality constraint
Mixed Integer Non Linear Programming
id ES_bbe55e76ab613e6d820b63e2a56338bb
oai_identifier_str oai:idus.us.es:11441/104390
network_acronym_str ES
network_name_str España
repository_id_str
spelling Integer constraints for enhancing interpretability in linear regressionCarrizosa Priego, Emilio JoséOlivares Nadal, Alba VictoriaRamírez Cobo, JosefaLinear regressionMulticollinearitySparsityCardinality constraintMixed Integer Non Linear ProgrammingOne of the main challenges researchers face is to identify the most relevant features in a prediction model. As a consequence, many regularized methods seeking sparsity have flourished. Although sparse, their solutions may not be interpretable in the presence of spurious coefficients and correlated features. In this paper we aim to enhance interpretability in linear regression in presence of multicollinearity by: (i) forcing the sign of the estimated coefficients to be consistent with the sign of the correlations between predictors, and (ii) avoiding spurious coefficients so that only significant features are represented in the model. This will be addressed by modelling constraints and adding them to an optimization problem expressing some estimation procedure such as ordinary least squares or the lasso. The so-obtained constrained regression models will become Mixed Integer Quadratic Problems. The numerical experiments carried out on real and simulated datasets show that tightening the search space of some standard linear regression models by adding the constraints modelling (i) and/or (ii) help to improve the sparsity and interpretability of the solutions with competitive predictive quality.Institut d´Estadística de Catalunya (Idescat)Estadística e Investigación Operativa2020info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfapplication/pdfhttps://hdl.handle.net/11441/104390https://doi.org/10.2436/20.8080.02.95reponame:idUS. Depósito de Investigación de la Universidad de Sevillainstname:Universidad de Sevilla (US)InglésSORT. Statistics and Operations Research Transactions, 44 (1), 67-98.https://doi.org/10.2436/20.8080.02.95info:eu-repo/semantics/openAccessoai:idus.us.es:11441/1043902026-06-17T12:51:07Z
dc.title.none.fl_str_mv Integer constraints for enhancing interpretability in linear regression
title Integer constraints for enhancing interpretability in linear regression
spellingShingle Integer constraints for enhancing interpretability in linear regression
Carrizosa Priego, Emilio José
Linear regression
Multicollinearity
Sparsity
Cardinality constraint
Mixed Integer Non Linear Programming
title_short Integer constraints for enhancing interpretability in linear regression
title_full Integer constraints for enhancing interpretability in linear regression
title_fullStr Integer constraints for enhancing interpretability in linear regression
title_full_unstemmed Integer constraints for enhancing interpretability in linear regression
title_sort Integer constraints for enhancing interpretability in linear regression
dc.creator.none.fl_str_mv Carrizosa Priego, Emilio José
Olivares Nadal, Alba Victoria
Ramírez Cobo, Josefa
author Carrizosa Priego, Emilio José
author_facet Carrizosa Priego, Emilio José
Olivares Nadal, Alba Victoria
Ramírez Cobo, Josefa
author_role author
author2 Olivares Nadal, Alba Victoria
Ramírez Cobo, Josefa
author2_role author
author
dc.contributor.none.fl_str_mv Estadística e Investigación Operativa
dc.subject.none.fl_str_mv Linear regression
Multicollinearity
Sparsity
Cardinality constraint
Mixed Integer Non Linear Programming
topic Linear regression
Multicollinearity
Sparsity
Cardinality constraint
Mixed Integer Non Linear Programming
description One of the main challenges researchers face is to identify the most relevant features in a prediction model. As a consequence, many regularized methods seeking sparsity have flourished. Although sparse, their solutions may not be interpretable in the presence of spurious coefficients and correlated features. In this paper we aim to enhance interpretability in linear regression in presence of multicollinearity by: (i) forcing the sign of the estimated coefficients to be consistent with the sign of the correlations between predictors, and (ii) avoiding spurious coefficients so that only significant features are represented in the model. This will be addressed by modelling constraints and adding them to an optimization problem expressing some estimation procedure such as ordinary least squares or the lasso. The so-obtained constrained regression models will become Mixed Integer Quadratic Problems. The numerical experiments carried out on real and simulated datasets show that tightening the search space of some standard linear regression models by adding the constraints modelling (i) and/or (ii) help to improve the sparsity and interpretability of the solutions with competitive predictive quality.
publishDate 2020
dc.date.none.fl_str_mv 2020
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv https://hdl.handle.net/11441/104390
https://doi.org/10.2436/20.8080.02.95
url https://hdl.handle.net/11441/104390
https://doi.org/10.2436/20.8080.02.95
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv SORT. Statistics and Operations Research Transactions, 44 (1), 67-98.
https://doi.org/10.2436/20.8080.02.95
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Institut d´Estadística de Catalunya (Idescat)
publisher.none.fl_str_mv Institut d´Estadística de Catalunya (Idescat)
dc.source.none.fl_str_mv reponame:idUS. Depósito de Investigación de la Universidad de Sevilla
instname:Universidad de Sevilla (US)
instname_str Universidad de Sevilla (US)
reponame_str idUS. Depósito de Investigación de la Universidad de Sevilla
collection idUS. Depósito de Investigación de la Universidad de Sevilla
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869418066072305664
score 15,300719

Similares en LA Referencia