Analysis of kernel matrices via the von Neumann entropy and its relation to RVM performances

Kernel methods have played a major role in the last two decades in the modeling and visualization of complex problems in data science. The choice of kernel function remains an open research area and the reasons why some kernels perform better than others are not yet understood. Moreover, the high co...

Descripción completa

Detalles Bibliográficos
Autores: Belanche Muñoz, Luis Antonio|||0000-0002-7577-1964, Wiejacha, Malgorzata
Tipo de recurso: artículo
Fecha de publicación:2023
País:España
Institución:Universitat Politècnica de Catalunya (UPC)
Repositorio:UPCommons. Portal del coneixement obert de la UPC
Idioma:inglés
OAI Identifier:oai:upcommons.upc.edu:2117/386026
Acceso en línea:https://hdl.handle.net/2117/386026
https://dx.doi.org/10.3390/e25010154
Access Level:acceso abierto
Palabra clave:Machine learning
Entropy (Information theory)
von Neumann entropy
Relevance vector machines
Generalization error
Aprenentatge automàtic
Entropia (Teoria de la informació)
Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàtic
id ES_7e5822bc65cb4d8febaa88efc2ded8eb
oai_identifier_str oai:upcommons.upc.edu:2117/386026
network_acronym_str ES
network_name_str España
repository_id_str
spelling Analysis of kernel matrices via the von Neumann entropy and its relation to RVM performancesBelanche Muñoz, Luis Antonio|||0000-0002-7577-1964Wiejacha, MalgorzataMachine learningEntropy (Information theory)von Neumann entropyRelevance vector machinesGeneralization errorAprenentatge automàticEntropia (Teoria de la informació)Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàticKernel methods have played a major role in the last two decades in the modeling and visualization of complex problems in data science. The choice of kernel function remains an open research area and the reasons why some kernels perform better than others are not yet understood. Moreover, the high computational costs of kernel-based methods make it extremely inefficient to use standard model selection methods, such as cross-validation, creating a need for careful kernel design and parameter choice. These reasons justify the prior analyses of kernel matrices, i.e., mathematical objects generated by the kernel functions. This paper explores these topics from an entropic standpoint for the case of kernelized relevance vector machines (RVMs), pinpointing desirable properties of kernel matrices that increase the likelihood of obtaining good model performances in terms of generalization power, as well as relate these properties to the model’s fitting ability. We also derive a heuristic for achieving close-to-optimal modeling results while keeping the computational costs low, thus providing a recipe for efficient analysis when processing resources are limited.Peer Reviewed20232023-01-1220232023-04-06journal articlehttp://purl.org/coar/resource_type/c_6501VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/2117/386026https://dx.doi.org/10.3390/e25010154reponame:UPCommons. Portal del coneixement obert de la UPCinstname:Universitat Politècnica de Catalunya (UPC)Inglésengopen accesshttp://purl.org/coar/access_right/c_abf2Attribution 4.0 Internationalhttp://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccessoai:upcommons.upc.edu:2117/3860262026-05-27T15:37:01Z
dc.title.none.fl_str_mv Analysis of kernel matrices via the von Neumann entropy and its relation to RVM performances
title Analysis of kernel matrices via the von Neumann entropy and its relation to RVM performances
spellingShingle Analysis of kernel matrices via the von Neumann entropy and its relation to RVM performances
Belanche Muñoz, Luis Antonio|||0000-0002-7577-1964
Machine learning
Entropy (Information theory)
von Neumann entropy
Relevance vector machines
Generalization error
Aprenentatge automàtic
Entropia (Teoria de la informació)
Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàtic
title_short Analysis of kernel matrices via the von Neumann entropy and its relation to RVM performances
title_full Analysis of kernel matrices via the von Neumann entropy and its relation to RVM performances
title_fullStr Analysis of kernel matrices via the von Neumann entropy and its relation to RVM performances
title_full_unstemmed Analysis of kernel matrices via the von Neumann entropy and its relation to RVM performances
title_sort Analysis of kernel matrices via the von Neumann entropy and its relation to RVM performances
dc.creator.none.fl_str_mv Belanche Muñoz, Luis Antonio|||0000-0002-7577-1964
Wiejacha, Malgorzata
author Belanche Muñoz, Luis Antonio|||0000-0002-7577-1964
author_facet Belanche Muñoz, Luis Antonio|||0000-0002-7577-1964
Wiejacha, Malgorzata
author_role author
author2 Wiejacha, Malgorzata
author2_role author
dc.subject.none.fl_str_mv Machine learning
Entropy (Information theory)
von Neumann entropy
Relevance vector machines
Generalization error
Aprenentatge automàtic
Entropia (Teoria de la informació)
Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàtic
topic Machine learning
Entropy (Information theory)
von Neumann entropy
Relevance vector machines
Generalization error
Aprenentatge automàtic
Entropia (Teoria de la informació)
Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàtic
description Kernel methods have played a major role in the last two decades in the modeling and visualization of complex problems in data science. The choice of kernel function remains an open research area and the reasons why some kernels perform better than others are not yet understood. Moreover, the high computational costs of kernel-based methods make it extremely inefficient to use standard model selection methods, such as cross-validation, creating a need for careful kernel design and parameter choice. These reasons justify the prior analyses of kernel matrices, i.e., mathematical objects generated by the kernel functions. This paper explores these topics from an entropic standpoint for the case of kernelized relevance vector machines (RVMs), pinpointing desirable properties of kernel matrices that increase the likelihood of obtaining good model performances in terms of generalization power, as well as relate these properties to the model’s fitting ability. We also derive a heuristic for achieving close-to-optimal modeling results while keeping the computational costs low, thus providing a recipe for efficient analysis when processing resources are limited.
publishDate 2023
dc.date.none.fl_str_mv 2023
2023-01-12
2023
2023-04-06
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/2117/386026
https://dx.doi.org/10.3390/e25010154
url https://hdl.handle.net/2117/386026
https://dx.doi.org/10.3390/e25010154
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution 4.0 International
http://creativecommons.org/licenses/by/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution 4.0 International
http://creativecommons.org/licenses/by/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.source.none.fl_str_mv reponame:UPCommons. Portal del coneixement obert de la UPC
instname:Universitat Politècnica de Catalunya (UPC)
instname_str Universitat Politècnica de Catalunya (UPC)
reponame_str UPCommons. Portal del coneixement obert de la UPC
collection UPCommons. Portal del coneixement obert de la UPC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869411728875323392
score 15,300719

Similares en LA Referencia