BAM-SLDK: biologically inspired attention mechanism with spiking learnable delayed kernel synapses

Spiking neural networks are emerging as an alternative neural network model due to their biological plausibility, energy efficiency, and built-in ability to learn from temporal dynamics. However, in order to effectively process data with rich spatial and temporal dependencies, the usual static proje...

ver descrição completa

Detalhes bibliográficos
Autores: Chacón Falcón, Mario, Patiño Saucedo, Alberto, Camuñas Mesa, Luis Alejandro, Serrano Gotarredona, María Teresa, Linares Barranco, Bernabé
Formato: artículo
Estado:Versión publicada
Fecha de publicación:2025
País:España
Recursos:Universidad de Sevilla (US)
Repositorio:idUS. Depósito de Investigación de la Universidad de Sevilla
OAI Identifier:oai:idus.us.es:11441/183173
Acesso em linha:https://hdl.handle.net/11441/183173
https://doi.org/10.1088/2634-4386/addb6c
Access Level:acceso abierto
Palavra-chave:Spiking neural networks (SNNs)
Delays
Convolutions
Gesture recognition
id ES_bc7c08abd441ec03c2f69151dbd75777
oai_identifier_str oai:idus.us.es:11441/183173
network_acronym_str ES
network_name_str España
repository_id_str
spelling BAM-SLDK: biologically inspired attention mechanism with spiking learnable delayed kernel synapsesChacón Falcón, MarioPatiño Saucedo, AlbertoCamuñas Mesa, Luis AlejandroSerrano Gotarredona, María TeresaLinares Barranco, BernabéSpiking neural networks (SNNs)DelaysConvolutionsGesture recognitionSpiking neural networks are emerging as an alternative neural network model due to their biological plausibility, energy efficiency, and built-in ability to learn from temporal dynamics. However, in order to effectively process data with rich spatial and temporal dependencies, the usual static projections (feedforward and recurrent) among layers of spiking neurons fail to represent all the information needed. Inspired by how synaptic delays affect the learning process in biological neurons, in this paper, we propose a biologically inspired attention mechanism based on spiking convolutions with learnable delayed kernel synapses. The proposed model increases temporal learning ability, attending simultaneously to spatial and temporal dynamics with few parameters required. More precisely, our main technical contributions are: (1) we add kernels to the temporal dimension to enlarge the receptive field of the convolution; (2) we time kernels activations to mimic multiple delayed times; and (3) we introduce three different pruning techniques to optimize the number of delays and parameters used. Experiments show that our method surpasses conventional spiking convolutional modules and achieves state-of-the-art results. When pruning, we show that, for some datasets or pruning techniques, removing up to 80% of the initially trained delays results in minimal performance loss, effectively reducing memory consumption and parameters required. To the best of our knowledge, this is the first time that learnable delayed synapses have been included in spiking convolutional layers for neuromorphic datasets classification, unlocking a new biologically inspired attention mechanism and achieving superior performance on high temporal demanding tasks.IOP PublishingArquitectura y Tecnología de ComputadoresInstituto de Microelectrónica de Sevilla (IMSE-CNM)Ministerio para la Transformación Digital y de la Función PúblicaEuropean Commission (EC)Ministerio de Ciencia, Innovación y Universidades (MICIU). España2025info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfapplication/pdfhttps://hdl.handle.net/11441/183173https://doi.org/10.1088/2634-4386/addb6creponame:idUS. Depósito de Investigación de la Universidad de Sevillainstname:Universidad de Sevilla (US)InglésNeuromorphic Computing and Engineering, 5 (2), 024017.Grant USECHIP TSI-069100-2023-001Grant SAMURAI PDC2023-145841-C31Grant BIOVEO PROYEXCEL_00060https://iopscience.iop.org/article/10.1088/2634-4386/addb6cinfo:eu-repo/semantics/openAccessoai:idus.us.es:11441/1831732026-06-17T12:51:07Z
dc.title.none.fl_str_mv BAM-SLDK: biologically inspired attention mechanism with spiking learnable delayed kernel synapses
title BAM-SLDK: biologically inspired attention mechanism with spiking learnable delayed kernel synapses
spellingShingle BAM-SLDK: biologically inspired attention mechanism with spiking learnable delayed kernel synapses
Chacón Falcón, Mario
Spiking neural networks (SNNs)
Delays
Convolutions
Gesture recognition
title_short BAM-SLDK: biologically inspired attention mechanism with spiking learnable delayed kernel synapses
title_full BAM-SLDK: biologically inspired attention mechanism with spiking learnable delayed kernel synapses
title_fullStr BAM-SLDK: biologically inspired attention mechanism with spiking learnable delayed kernel synapses
title_full_unstemmed BAM-SLDK: biologically inspired attention mechanism with spiking learnable delayed kernel synapses
title_sort BAM-SLDK: biologically inspired attention mechanism with spiking learnable delayed kernel synapses
dc.creator.none.fl_str_mv Chacón Falcón, Mario
Patiño Saucedo, Alberto
Camuñas Mesa, Luis Alejandro
Serrano Gotarredona, María Teresa
Linares Barranco, Bernabé
author Chacón Falcón, Mario
author_facet Chacón Falcón, Mario
Patiño Saucedo, Alberto
Camuñas Mesa, Luis Alejandro
Serrano Gotarredona, María Teresa
Linares Barranco, Bernabé
author_role author
author2 Patiño Saucedo, Alberto
Camuñas Mesa, Luis Alejandro
Serrano Gotarredona, María Teresa
Linares Barranco, Bernabé
author2_role author
author
author
author
dc.contributor.none.fl_str_mv Arquitectura y Tecnología de Computadores
Instituto de Microelectrónica de Sevilla (IMSE-CNM)
Ministerio para la Transformación Digital y de la Función Pública
European Commission (EC)
Ministerio de Ciencia, Innovación y Universidades (MICIU). España
dc.subject.none.fl_str_mv Spiking neural networks (SNNs)
Delays
Convolutions
Gesture recognition
topic Spiking neural networks (SNNs)
Delays
Convolutions
Gesture recognition
description Spiking neural networks are emerging as an alternative neural network model due to their biological plausibility, energy efficiency, and built-in ability to learn from temporal dynamics. However, in order to effectively process data with rich spatial and temporal dependencies, the usual static projections (feedforward and recurrent) among layers of spiking neurons fail to represent all the information needed. Inspired by how synaptic delays affect the learning process in biological neurons, in this paper, we propose a biologically inspired attention mechanism based on spiking convolutions with learnable delayed kernel synapses. The proposed model increases temporal learning ability, attending simultaneously to spatial and temporal dynamics with few parameters required. More precisely, our main technical contributions are: (1) we add kernels to the temporal dimension to enlarge the receptive field of the convolution; (2) we time kernels activations to mimic multiple delayed times; and (3) we introduce three different pruning techniques to optimize the number of delays and parameters used. Experiments show that our method surpasses conventional spiking convolutional modules and achieves state-of-the-art results. When pruning, we show that, for some datasets or pruning techniques, removing up to 80% of the initially trained delays results in minimal performance loss, effectively reducing memory consumption and parameters required. To the best of our knowledge, this is the first time that learnable delayed synapses have been included in spiking convolutional layers for neuromorphic datasets classification, unlocking a new biologically inspired attention mechanism and achieving superior performance on high temporal demanding tasks.
publishDate 2025
dc.date.none.fl_str_mv 2025
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv https://hdl.handle.net/11441/183173
https://doi.org/10.1088/2634-4386/addb6c
url https://hdl.handle.net/11441/183173
https://doi.org/10.1088/2634-4386/addb6c
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv Neuromorphic Computing and Engineering, 5 (2), 024017.
Grant USECHIP TSI-069100-2023-001
Grant SAMURAI PDC2023-145841-C31
Grant BIOVEO PROYEXCEL_00060
https://iopscience.iop.org/article/10.1088/2634-4386/addb6c
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv IOP Publishing
publisher.none.fl_str_mv IOP Publishing
dc.source.none.fl_str_mv reponame:idUS. Depósito de Investigación de la Universidad de Sevilla
instname:Universidad de Sevilla (US)
instname_str Universidad de Sevilla (US)
reponame_str idUS. Depósito de Investigación de la Universidad de Sevilla
collection idUS. Depósito de Investigación de la Universidad de Sevilla
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869418117859377152
score 15,811543

Registos relacionados