Extensions of infinite width neural network Kernels
This thesis explores the diverse amount of known closed form expressions of the infinite width neural network kernels. These are kernels whose map to the feature space mimics the function defined by a single hidden layer neural network when the width of this layer goes to infinity, which are defined...
| Autor: | |
|---|---|
| Tipo de recurso: | tesis de maestría |
| Fecha de publicación: | 2024 |
| País: | España |
| Institución: | Universitat Politècnica de Catalunya (UPC) |
| Repositorio: | UPCommons. Portal del coneixement obert de la UPC |
| Idioma: | inglés |
| OAI Identifier: | oai:upcommons.upc.edu:2117/420590 |
| Acceso en línea: | https://hdl.handle.net/2117/420590 |
| Access Level: | acceso abierto |
| Palabra clave: | Kernel functions Neural networks (Computer science) svm kernel methods infinite width neural network Kernel, Funcions de Xarxes neuronals (Informàtica) Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial |
| Sumario: | This thesis explores the diverse amount of known closed form expressions of the infinite width neural network kernels. These are kernels whose map to the feature space mimics the function defined by a single hidden layer neural network when the width of this layer goes to infinity, which are defined in integral form. A possible interpretation of the parameters of the implicit neural network is given, which motivates making the distribution of the data and the distribution of these parameters equal. This allows the estimation of the covariance matrix of the kernel by using the data. Since this approach is usually impractical due to the limited amount of closed form expressions of the kernel, a methodology based on kernel-SVMs is proposed in a way that 1) exploits the identification of the distributions of the network parameters with the distribution of the data and 2) uses the available closed form expressions without resorting to numerical approximations of the integral. In the experiments in binary classification it has been proved that estimating the covariance matrix of the neural network parameters provides improvements with respect to setting it as the identity, as the models can reach state-of-the-art results. |
|---|