Comparative analysis of kernel approximation methods and their ensemble architectures
Kernel methods offer strong performance in supervised learning, but their scalability remains a key challenge. As a result, kernel approximation methods have emerged as promising alternatives, but their comparison and ensemble performance is still a field of study. Some open questions include how in...
| Authors: | , , |
|---|---|
| Format: | article |
| Publication Date: | 2026 |
| Country: | España |
| Institution: | Universidad Autónoma de Madrid |
| Repository: | Biblos-e Archivo. Repositorio Institucional de la UAM |
| Language: | English |
| OAI Identifier: | oai:repositorio.uam.es:10486/754340 |
| Online Access: | https://hdl.handle.net/10486/754340 https://dx.doi.org/10.1016/j.neucom.2026.133202 |
| Access Level: | Open access |
| Keyword: | Kernel approximation Nyström Random features Kernel thinning Ensemble Informática |
| Summary: | Kernel methods offer strong performance in supervised learning, but their scalability remains a key challenge. As a result, kernel approximation methods have emerged as promising alternatives, but their comparison and ensemble performance is still a field of study. Some open questions include how individual methods compare against each other, and whether ensembles could benefit from certain combinations of approximations. In this paper, we evaluate four methods: Nyström, Random Fourier Features, Kernel Thinning and Neural Orthogonal Random Features (NORF). NORF is presented as a new contribution. We evaluate each method in terms of balanced accuracy, training time, and prediction diversity based on the prediction correlations. In addition, we investigate their performance in voting ensemble architectures. As a result, we observe that the best model is Nyström, not only individually but also because of its potential improvement in ensemble settings. These ensembles do not differ significantly from Kernel Support Vector Machine in performance, but they reduce training time. Moreover, NORF stands out as an alternative that does not rely on a predefined kernel and increases prediction diversity |
|---|