A slimmable framework for practical neural video compression

Deep learning is being increasingly applied to image and video compression in a new paradigm known as neural video compression. While achieving impressive rate–distortion (RD) performance, neural video codecs (NVC) require heavy neural networks, which in turn have large memory and computational cost...

Descripción completa

Detalles Bibliográficos
Autores: Liu, Zhaocheng, Yang, Fei, Wang, Defa, Górriz Blanch, Marc, Murn, Luka, Wan, Shuai, Zhang, Saiping, Mrak, Marta, Herranz Arribas, Luis
Tipo de recurso: artículo
Fecha de publicación:2024
País:España
Institución:Universidad Autónoma de Madrid
Repositorio:Biblos-e Archivo. Repositorio Institucional de la UAM
Idioma:inglés
OAI Identifier:oai:repositorio.uam.es:10486/714888
Acceso en línea:http://hdl.handle.net/10486/714888
https://dx.doi.org/10.1016/j.neucom.2024.128525
Access Level:acceso abierto
Palabra clave:Neural Video Compression
Slimmable Network
Deep Learning
Variable Rate
Feature Modulation
Slimmable Codec
Informática
id ES_293dbfcf79830005c5c077f4ee701c2a
oai_identifier_str oai:repositorio.uam.es:10486/714888
network_acronym_str ES
network_name_str España
repository_id_str
spelling A slimmable framework for practical neural video compressionLiu, ZhaochengYang, FeiWang, DefaGórriz Blanch, MarcMurn, LukaWan, ShuaiZhang, SaipingMrak, MartaHerranz Arribas, LuisNeural Video CompressionSlimmable NetworkDeep LearningVariable RateFeature ModulationSlimmable CodecInformáticaDeep learning is being increasingly applied to image and video compression in a new paradigm known as neural video compression. While achieving impressive rate–distortion (RD) performance, neural video codecs (NVC) require heavy neural networks, which in turn have large memory and computational costs and often lack important functionalities such as variable rate. These are significant limitations to their practical application. Addressing these problems, recent slimmable image codecs can dynamically adjust their model capacity to elegantly reduce the memory and computation requirements, without harming RD performance. However, the extension to video is not straightforward due to the non-trivial interplay with complex motion estimation and compensation modules in most NVC architectures. In this paper we propose the slimmable video codec framework (SlimVC) that integrates an slimmable autoencoder and a motion-free conditional entropy model. We show that the slimming mechanism is also applicable to the more complex case of video architectures, providing SlimVC with simultaneous control of the computational cost, memory and rate, which are all important requirements in practice. We further provide detailed experimental analysis, and describe application scenarios that can benefit from slimmable video codecsThis work was partially supported by Grant PID2021-128178OB-I00 funded by MCIN/AEI/10.13039/501100011033 and by ERDF ‘‘A way of making Europe’’, and by the Spanish government under the Ramón Cajal program, grant number RYC2019-027020-IElsevierDepartamento de Ingeniería InformáticaEscuela Politécnica Superior20242024-09-05research articlehttp://purl.org/coar/resource_type/c_2df8fbb1VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10486/714888https://dx.doi.org/10.1016/j.neucom.2024.128525reponame:Biblos-e Archivo. Repositorio Institucional de la UAMinstname:Universidad Autónoma de MadridInglésengopen accesshttp://purl.org/coar/access_right/c_abf2Attribution 4.0 Internationalhttp://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccessoai:repositorio.uam.es:10486/7148882026-06-23T12:46:27Z
dc.title.none.fl_str_mv A slimmable framework for practical neural video compression
title A slimmable framework for practical neural video compression
spellingShingle A slimmable framework for practical neural video compression
Liu, Zhaocheng
Neural Video Compression
Slimmable Network
Deep Learning
Variable Rate
Feature Modulation
Slimmable Codec
Informática
title_short A slimmable framework for practical neural video compression
title_full A slimmable framework for practical neural video compression
title_fullStr A slimmable framework for practical neural video compression
title_full_unstemmed A slimmable framework for practical neural video compression
title_sort A slimmable framework for practical neural video compression
dc.creator.none.fl_str_mv Liu, Zhaocheng
Yang, Fei
Wang, Defa
Górriz Blanch, Marc
Murn, Luka
Wan, Shuai
Zhang, Saiping
Mrak, Marta
Herranz Arribas, Luis
author Liu, Zhaocheng
author_facet Liu, Zhaocheng
Yang, Fei
Wang, Defa
Górriz Blanch, Marc
Murn, Luka
Wan, Shuai
Zhang, Saiping
Mrak, Marta
Herranz Arribas, Luis
author_role author
author2 Yang, Fei
Wang, Defa
Górriz Blanch, Marc
Murn, Luka
Wan, Shuai
Zhang, Saiping
Mrak, Marta
Herranz Arribas, Luis
author2_role author
author
author
author
author
author
author
author
dc.contributor.none.fl_str_mv Departamento de Ingeniería Informática
Escuela Politécnica Superior
dc.subject.none.fl_str_mv Neural Video Compression
Slimmable Network
Deep Learning
Variable Rate
Feature Modulation
Slimmable Codec
Informática
topic Neural Video Compression
Slimmable Network
Deep Learning
Variable Rate
Feature Modulation
Slimmable Codec
Informática
description Deep learning is being increasingly applied to image and video compression in a new paradigm known as neural video compression. While achieving impressive rate–distortion (RD) performance, neural video codecs (NVC) require heavy neural networks, which in turn have large memory and computational costs and often lack important functionalities such as variable rate. These are significant limitations to their practical application. Addressing these problems, recent slimmable image codecs can dynamically adjust their model capacity to elegantly reduce the memory and computation requirements, without harming RD performance. However, the extension to video is not straightforward due to the non-trivial interplay with complex motion estimation and compensation modules in most NVC architectures. In this paper we propose the slimmable video codec framework (SlimVC) that integrates an slimmable autoencoder and a motion-free conditional entropy model. We show that the slimming mechanism is also applicable to the more complex case of video architectures, providing SlimVC with simultaneous control of the computational cost, memory and rate, which are all important requirements in practice. We further provide detailed experimental analysis, and describe application scenarios that can benefit from slimmable video codecs
publishDate 2024
dc.date.none.fl_str_mv 2024
2024-09-05
dc.type.none.fl_str_mv research article
http://purl.org/coar/resource_type/c_2df8fbb1
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv http://hdl.handle.net/10486/714888
https://dx.doi.org/10.1016/j.neucom.2024.128525
url http://hdl.handle.net/10486/714888
https://dx.doi.org/10.1016/j.neucom.2024.128525
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution 4.0 International
http://creativecommons.org/licenses/by/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution 4.0 International
http://creativecommons.org/licenses/by/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Elsevier
publisher.none.fl_str_mv Elsevier
dc.source.none.fl_str_mv reponame:Biblos-e Archivo. Repositorio Institucional de la UAM
instname:Universidad Autónoma de Madrid
instname_str Universidad Autónoma de Madrid
reponame_str Biblos-e Archivo. Repositorio Institucional de la UAM
collection Biblos-e Archivo. Repositorio Institucional de la UAM
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869404992425689088
score 15,811543

Similares en LA Referencia