Audio and music analysis on the web using Essentia.js

Open-source software libraries have a significant impact on the development of Audio Signal Processing and Music Information Retrieval (MIR) systems. Despite the abundance of such tools, there is a lack of an extensive and easy-to-use reference library for audio feature extraction on Web clients. In...

Descripción completa

Detalles Bibliográficos
Autores: Correya, Albin Andrew, Marcos Fernández, Jorge, Joglar-Ongay, Luis, Alonso Jiménez, Pablo, Serra, Xavier, Bogdanov, Dmitry
Tipo de recurso: artículo
Estado:Versión publicada
Fecha de publicación:2021
País:España
Institución:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
Repositorio:Recercat. Dipósit de la Recerca de Catalunya
OAI Identifier:oai:recercat.cat:10230/49060
Acceso en línea:http://hdl.handle.net/10230/49060
http://dx.doi.org/10.5334/tismir.111
Access Level:acceso abierto
Palabra clave:Software
Web audio
Audio analysis
Music signal processing
Music audio classification
Deep learning
id ES_73241a4dde2e6da5fda8b26ff4a2ff9e
oai_identifier_str oai:recercat.cat:10230/49060
network_acronym_str ES
network_name_str España
repository_id_str
spelling Audio and music analysis on the web using Essentia.jsCorreya, Albin AndrewMarcos Fernández, JorgeJoglar-Ongay, LuisAlonso Jiménez, PabloSerra, XavierBogdanov, DmitrySoftwareWeb audioAudio analysisMusic signal processingMusic audio classificationDeep learningOpen-source software libraries have a significant impact on the development of Audio Signal Processing and Music Information Retrieval (MIR) systems. Despite the abundance of such tools, there is a lack of an extensive and easy-to-use reference library for audio feature extraction on Web clients. In this article, we present Essentia.js, an open-source JavaScript (JS) library for audio and music analysis on both web clients and JS engines. Along with the Web Audio API, it can be used for both offline and real-time audio feature extraction on web browsers. Essentia.js is modular, lightweight, and easy-to-use, deploy, maintain, and integrate into the existing plethora of JS libraries and web technologies. It is powered by a WebAssembly back end cross-compiled from the Essentia C++ library, which facilitates a JS interface to a wide range of low-level and high-level audio features, including signal processing MIR algorithms as well as pre-trained TensorFlow.js machine learning models. It also provides a higher-level JS API and add-on MIR utility modules along with extensive documentation, usage examples, and tutorials. We benchmark the proposed library on two popular web browsers and the Node.js engine, and four devices, including mobile Android and iOS, comparing it to the native performance of Essentia and the Meyda JS library.The work on Essentia.js has been partially funded by the Ministry of Science and Innovation of the Spanish Government under the grant agreement PID2019-111403GB-I00 (Musical AI).Ubiquity Press202120212021info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersionapplication/pdfapplication/pdfhttp://hdl.handle.net/10230/49060http://dx.doi.org/10.5334/tismir.111reponame:Recercat. Dipósit de la Recerca de Catalunyainstname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)InglésTransactions of the International Society for Music Information Retrieval. 2021;4(1):167-81.info:eu-repo/grantAgreement/ES/2PE/PID2019-111403GB-I00© 2021 The Author(s). This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC-BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. See http://creativecommons.org/licenses/by/4.0/.https://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccessoai:recercat.cat:10230/490602026-05-29T05:05:01Z
dc.title.none.fl_str_mv Audio and music analysis on the web using Essentia.js
title Audio and music analysis on the web using Essentia.js
spellingShingle Audio and music analysis on the web using Essentia.js
Correya, Albin Andrew
Software
Web audio
Audio analysis
Music signal processing
Music audio classification
Deep learning
title_short Audio and music analysis on the web using Essentia.js
title_full Audio and music analysis on the web using Essentia.js
title_fullStr Audio and music analysis on the web using Essentia.js
title_full_unstemmed Audio and music analysis on the web using Essentia.js
title_sort Audio and music analysis on the web using Essentia.js
dc.creator.none.fl_str_mv Correya, Albin Andrew
Marcos Fernández, Jorge
Joglar-Ongay, Luis
Alonso Jiménez, Pablo
Serra, Xavier
Bogdanov, Dmitry
author Correya, Albin Andrew
author_facet Correya, Albin Andrew
Marcos Fernández, Jorge
Joglar-Ongay, Luis
Alonso Jiménez, Pablo
Serra, Xavier
Bogdanov, Dmitry
author_role author
author2 Marcos Fernández, Jorge
Joglar-Ongay, Luis
Alonso Jiménez, Pablo
Serra, Xavier
Bogdanov, Dmitry
author2_role author
author
author
author
author
dc.subject.none.fl_str_mv Software
Web audio
Audio analysis
Music signal processing
Music audio classification
Deep learning
topic Software
Web audio
Audio analysis
Music signal processing
Music audio classification
Deep learning
description Open-source software libraries have a significant impact on the development of Audio Signal Processing and Music Information Retrieval (MIR) systems. Despite the abundance of such tools, there is a lack of an extensive and easy-to-use reference library for audio feature extraction on Web clients. In this article, we present Essentia.js, an open-source JavaScript (JS) library for audio and music analysis on both web clients and JS engines. Along with the Web Audio API, it can be used for both offline and real-time audio feature extraction on web browsers. Essentia.js is modular, lightweight, and easy-to-use, deploy, maintain, and integrate into the existing plethora of JS libraries and web technologies. It is powered by a WebAssembly back end cross-compiled from the Essentia C++ library, which facilitates a JS interface to a wide range of low-level and high-level audio features, including signal processing MIR algorithms as well as pre-trained TensorFlow.js machine learning models. It also provides a higher-level JS API and add-on MIR utility modules along with extensive documentation, usage examples, and tutorials. We benchmark the proposed library on two popular web browsers and the Node.js engine, and four devices, including mobile Android and iOS, comparing it to the native performance of Essentia and the Meyda JS library.
publishDate 2021
dc.date.none.fl_str_mv 2021
2021
2021
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/10230/49060
http://dx.doi.org/10.5334/tismir.111
url http://hdl.handle.net/10230/49060
http://dx.doi.org/10.5334/tismir.111
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv Transactions of the International Society for Music Information Retrieval. 2021;4(1):167-81.
info:eu-repo/grantAgreement/ES/2PE/PID2019-111403GB-I00
dc.rights.none.fl_str_mv https://creativecommons.org/licenses/by/4.0/
info:eu-repo/semantics/openAccess
rights_invalid_str_mv https://creativecommons.org/licenses/by/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Ubiquity Press
publisher.none.fl_str_mv Ubiquity Press
dc.source.none.fl_str_mv reponame:Recercat. Dipósit de la Recerca de Catalunya
instname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
instname_str Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
reponame_str Recercat. Dipósit de la Recerca de Catalunya
collection Recercat. Dipósit de la Recerca de Catalunya
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869410787375710208
score 15,81155

Similares en LA Referencia