Functional form estimation using oblique projection matrices for ls-SVM regression models
Kernel regression models have been used as non-parametric methods for fitting experimental data. However, due to their non-parametric nature, they belong to the so-called 'black box' models, indicating that the relation between the input variables and the output, depending on the kernel se...
| Autores: | , , , |
|---|---|
| Tipo de recurso: | artículo |
| Estado: | Versión publicada |
| Fecha de publicación: | 2019 |
| País: | Colombia |
| Institución: | Universidad del Rosario |
| Repositorio: | Repositorio EdocUR - U. Rosario |
| Idioma: | inglés |
| OAI Identifier: | oai:repository.urosario.edu.co:10336/22835 |
| Acceso en línea: | https://doi.org/10.1371/journal.pone.0217967 https://repository.urosario.edu.co/handle/10336/22835 |
| Access Level: | acceso abierto |
| Palabra clave: | Analysis of variance Article Decomposition Human Least square analysis Support vector machine Algorithm Artificial intelligence Machine learning Statistical model Algorithms Least-squares analysis Statistical Models |
| id |
CO_cee21df7a2c28e8a08ed6421faffe3d5 |
|---|---|
| oai_identifier_str |
oai:repository.urosario.edu.co:10336/22835 |
| network_acronym_str |
CO |
| network_name_str |
Colombia |
| repository_id_str |
|
| spelling |
Functional form estimation using oblique projection matrices for ls-SVM regression modelsCaicedo Dorado, AlexanderVaron, CarolinaVan Huffel, SabineSuykens, Johan A. K.Analysis of varianceArticleDecompositionHumanLeast square analysisSupport vector machineAlgorithmArtificial intelligenceLeast square analysisMachine learningStatistical modelSupport vector machineAlgorithmsArtificial intelligenceLeast-squares analysisMachine learningSupport vector machineStatisticalModelsKernel regression models have been used as non-parametric methods for fitting experimental data. However, due to their non-parametric nature, they belong to the so-called 'black box' models, indicating that the relation between the input variables and the output, depending on the kernel selection, is unknown. In this paper we propose a new methodology to retrieve the relation between each input regressor variable and the output in a least squares support vector machine (LS-SVM) regression model. The method is based on oblique subspace projectors (ObSP), which allows to decouple the influence of input regressors on the output by including the undesired variables in the null space of the projection matrix. Such functional relations are represented by the nonlinear transformation of the input regressors, and their subspaces are estimated using appropriate kernel evaluations. We exploit the properties of ObSP in order to decompose the output of the obtained regression model as a sum of the partial nonlinear contributions and interaction effects of the input variables, we called this methodology Nonlinear ObSP (NObSP). We compare the performance of the proposed algorithm with the component selection and smooth operator (COSSO) for smoothing spline ANOVA models. We use as benchmark 2 toy examples and a real life regression model using the concrete strength dataset from the UCI machine learning repository. We showed that NObSP is able to outperform COSSO, producing stable estimations of the functional relations between the input regressors and the output, without the use of prior-knowledge. This methodology can be used in order to understand the functional relations between the inputs and the output in a regression model, retrieving the physical interpretation of the regression models. © 2019 Caicedo et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.Public Library of Science20192020-05-25T23:58:17Zinfo:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfhttps://doi.org/10.1371/journal.pone.02179671932-6203https://repository.urosario.edu.co/handle/10336/22835reponame:Repositorio EdocUR - U. Rosarioinstname:Universidad del Rosarioinstacron:Universidad del Rosarioenghttps://www.scopus.com/inward/record.uri?eid=2-s2.0-85067381270&doi=10.1371%2fjournal.pone.0217967&partnerID=40&md5=a0bbb2bd48d46bdbdc95d15b265f2fedinfo:eu-repo/semantics/openAccess2022-08-27T13:02:10Z |
| dc.title.none.fl_str_mv |
Functional form estimation using oblique projection matrices for ls-SVM regression models |
| title |
Functional form estimation using oblique projection matrices for ls-SVM regression models |
| spellingShingle |
Functional form estimation using oblique projection matrices for ls-SVM regression models Caicedo Dorado, Alexander Analysis of variance Article Decomposition Human Least square analysis Support vector machine Algorithm Artificial intelligence Least square analysis Machine learning Statistical model Support vector machine Algorithms Artificial intelligence Least-squares analysis Machine learning Support vector machine Statistical Models |
| title_short |
Functional form estimation using oblique projection matrices for ls-SVM regression models |
| title_full |
Functional form estimation using oblique projection matrices for ls-SVM regression models |
| title_fullStr |
Functional form estimation using oblique projection matrices for ls-SVM regression models |
| title_full_unstemmed |
Functional form estimation using oblique projection matrices for ls-SVM regression models |
| title_sort |
Functional form estimation using oblique projection matrices for ls-SVM regression models |
| dc.creator.none.fl_str_mv |
Caicedo Dorado, Alexander Varon, Carolina Van Huffel, Sabine Suykens, Johan A. K. |
| author |
Caicedo Dorado, Alexander |
| author_facet |
Caicedo Dorado, Alexander Varon, Carolina Van Huffel, Sabine Suykens, Johan A. K. |
| author_role |
author |
| author2 |
Varon, Carolina Van Huffel, Sabine Suykens, Johan A. K. |
| author2_role |
author author author |
| dc.subject.none.fl_str_mv |
Analysis of variance Article Decomposition Human Least square analysis Support vector machine Algorithm Artificial intelligence Least square analysis Machine learning Statistical model Support vector machine Algorithms Artificial intelligence Least-squares analysis Machine learning Support vector machine Statistical Models |
| topic |
Analysis of variance Article Decomposition Human Least square analysis Support vector machine Algorithm Artificial intelligence Least square analysis Machine learning Statistical model Support vector machine Algorithms Artificial intelligence Least-squares analysis Machine learning Support vector machine Statistical Models |
| description |
Kernel regression models have been used as non-parametric methods for fitting experimental data. However, due to their non-parametric nature, they belong to the so-called 'black box' models, indicating that the relation between the input variables and the output, depending on the kernel selection, is unknown. In this paper we propose a new methodology to retrieve the relation between each input regressor variable and the output in a least squares support vector machine (LS-SVM) regression model. The method is based on oblique subspace projectors (ObSP), which allows to decouple the influence of input regressors on the output by including the undesired variables in the null space of the projection matrix. Such functional relations are represented by the nonlinear transformation of the input regressors, and their subspaces are estimated using appropriate kernel evaluations. We exploit the properties of ObSP in order to decompose the output of the obtained regression model as a sum of the partial nonlinear contributions and interaction effects of the input variables, we called this methodology Nonlinear ObSP (NObSP). We compare the performance of the proposed algorithm with the component selection and smooth operator (COSSO) for smoothing spline ANOVA models. We use as benchmark 2 toy examples and a real life regression model using the concrete strength dataset from the UCI machine learning repository. We showed that NObSP is able to outperform COSSO, producing stable estimations of the functional relations between the input regressors and the output, without the use of prior-knowledge. This methodology can be used in order to understand the functional relations between the inputs and the output in a regression model, retrieving the physical interpretation of the regression models. © 2019 Caicedo et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
| publishDate |
2019 |
| dc.date.none.fl_str_mv |
2019 2020-05-25T23:58:17Z |
| dc.type.none.fl_str_mv |
info:eu-repo/semantics/article info:eu-repo/semantics/publishedVersion |
| format |
article |
| status_str |
publishedVersion |
| dc.identifier.none.fl_str_mv |
https://doi.org/10.1371/journal.pone.0217967 1932-6203 https://repository.urosario.edu.co/handle/10336/22835 |
| url |
https://doi.org/10.1371/journal.pone.0217967 https://repository.urosario.edu.co/handle/10336/22835 |
| identifier_str_mv |
1932-6203 |
| dc.language.none.fl_str_mv |
eng |
| language |
eng |
| dc.relation.none.fl_str_mv |
https://www.scopus.com/inward/record.uri?eid=2-s2.0-85067381270&doi=10.1371%2fjournal.pone.0217967&partnerID=40&md5=a0bbb2bd48d46bdbdc95d15b265f2fed |
| dc.rights.none.fl_str_mv |
info:eu-repo/semantics/openAccess |
| eu_rights_str_mv |
openAccess |
| dc.format.none.fl_str_mv |
application/pdf |
| dc.publisher.none.fl_str_mv |
Public Library of Science |
| publisher.none.fl_str_mv |
Public Library of Science |
| dc.source.none.fl_str_mv |
reponame:Repositorio EdocUR - U. Rosario instname:Universidad del Rosario instacron:Universidad del Rosario |
| instname_str |
Universidad del Rosario |
| instacron_str |
Universidad del Rosario |
| institution |
Universidad del Rosario |
| reponame_str |
Repositorio EdocUR - U. Rosario |
| collection |
Repositorio EdocUR - U. Rosario |
| _version_ |
1825051686619578368 |
| score |
15.811543 |