Information Theory–based Compositional Distributional Semantics

In the context of text representation, Compositional Distributional Semantics models aim to fuse the Distributional Hypothesis and the Principle of Compositionality. Text embedding is based on co-ocurrence distributions and the representations are in turn combined by compositional functions taking i...

Descripción completa

Detalles Bibliográficos
Autores: Amigo Cabrera, Enrique, Ariza Casabona, Alejandro, Fresno Fernández, Víctor Diego, Martí, M. Antònia
Tipo de recurso: artículo
Fecha de publicación:2022
País:España
Institución:Universidad Nacional de Educación a Distancia
Repositorio:e-spacio. Repositorio Institucional de la UNED
Idioma:inglés
OAI Identifier:oai:e-spacio.uned.es:20.500.14468/30978
Acceso en línea:https://hdl.handle.net/20.500.14468/30978
Access Level:acceso abierto
Palabra clave:1203.04 Inteligencia artificial
5705.08 Semántica
1203.11 Logicales de ordenadores
id ES_734dc659ca0ecb60ba6dad4a3c693df2
oai_identifier_str oai:e-spacio.uned.es:20.500.14468/30978
network_acronym_str ES
network_name_str España
repository_id_str
spelling Information Theory–based Compositional Distributional SemanticsAmigo Cabrera, EnriqueAriza Casabona, AlejandroFresno Fernández, Víctor DiegoMartí, M. Antònia1203.04 Inteligencia artificial5705.08 Semántica1203.11 Logicales de ordenadoresIn the context of text representation, Compositional Distributional Semantics models aim to fuse the Distributional Hypothesis and the Principle of Compositionality. Text embedding is based on co-ocurrence distributions and the representations are in turn combined by compositional functions taking into account the text structure. However, the theoretical basis of compositional functions is still an open issue. In this article we define and study the notion of Information Theory–based Compositional Distributional Semantics (ICDS): (i) We first establish formal properties for embedding, composition, and similarity functions based on Shannon’s Information Theory; (ii) we analyze the existing approaches under this prism, checking whether or not they comply with the established desirable properties; (iii) we propose two parameterizable composition and similarity functions that generalize traditional approaches while fulfilling the formal properties; and finally (iv) we perform an empirical study on several textual similarity datasets that include sentences with a high and low lexical overlap, and on the similarity between words and their description. Our theoretical analysis and empirical results show that fulfilling formal properties affects positively the accuracy of text representation models in terms of correspondence (isometry) between the embedding and meaning spaces.Massachusetts Institute of Technology PressAgencia Estatal de Investigación (España)European Commissione-Spacio UNED20252025-12-0220222022-12-0120222022-12-01journal articlehttp://purl.org/coar/resource_type/c_6501info:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/20.500.14468/30978reponame:e-spacio. Repositorio Institucional de la UNEDinstname:Universidad Nacional de Educación a DistanciaInglésengopen accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccesshttp://creativecommons.org/licenses/by-nc-nd/4.0/deed.esoai:e-spacio.uned.es:20.500.14468/309782026-06-06T12:38:31Z
dc.title.none.fl_str_mv Information Theory–based Compositional Distributional Semantics
title Information Theory–based Compositional Distributional Semantics
spellingShingle Information Theory–based Compositional Distributional Semantics
Amigo Cabrera, Enrique
1203.04 Inteligencia artificial
5705.08 Semántica
1203.11 Logicales de ordenadores
title_short Information Theory–based Compositional Distributional Semantics
title_full Information Theory–based Compositional Distributional Semantics
title_fullStr Information Theory–based Compositional Distributional Semantics
title_full_unstemmed Information Theory–based Compositional Distributional Semantics
title_sort Information Theory–based Compositional Distributional Semantics
dc.creator.none.fl_str_mv Amigo Cabrera, Enrique
Ariza Casabona, Alejandro
Fresno Fernández, Víctor Diego
Martí, M. Antònia
author Amigo Cabrera, Enrique
author_facet Amigo Cabrera, Enrique
Ariza Casabona, Alejandro
Fresno Fernández, Víctor Diego
Martí, M. Antònia
author_role author
author2 Ariza Casabona, Alejandro
Fresno Fernández, Víctor Diego
Martí, M. Antònia
author2_role author
author
author
dc.contributor.none.fl_str_mv Agencia Estatal de Investigación (España)
European Commission
e-Spacio UNED
dc.subject.none.fl_str_mv 1203.04 Inteligencia artificial
5705.08 Semántica
1203.11 Logicales de ordenadores
topic 1203.04 Inteligencia artificial
5705.08 Semántica
1203.11 Logicales de ordenadores
description In the context of text representation, Compositional Distributional Semantics models aim to fuse the Distributional Hypothesis and the Principle of Compositionality. Text embedding is based on co-ocurrence distributions and the representations are in turn combined by compositional functions taking into account the text structure. However, the theoretical basis of compositional functions is still an open issue. In this article we define and study the notion of Information Theory–based Compositional Distributional Semantics (ICDS): (i) We first establish formal properties for embedding, composition, and similarity functions based on Shannon’s Information Theory; (ii) we analyze the existing approaches under this prism, checking whether or not they comply with the established desirable properties; (iii) we propose two parameterizable composition and similarity functions that generalize traditional approaches while fulfilling the formal properties; and finally (iv) we perform an empirical study on several textual similarity datasets that include sentences with a high and low lexical overlap, and on the similarity between words and their description. Our theoretical analysis and empirical results show that fulfilling formal properties affects positively the accuracy of text representation models in terms of correspondence (isometry) between the embedding and meaning spaces.
publishDate 2022
dc.date.none.fl_str_mv 2022
2022-12-01
2022
2022-12-01
2025
2025-12-02
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/20.500.14468/30978
url https://hdl.handle.net/20.500.14468/30978
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
info:eu-repo/semantics/openAccess
http://creativecommons.org/licenses/by-nc-nd/4.0/deed.es
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
http://creativecommons.org/licenses/by-nc-nd/4.0/deed.es
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Massachusetts Institute of Technology Press
publisher.none.fl_str_mv Massachusetts Institute of Technology Press
dc.source.none.fl_str_mv reponame:e-spacio. Repositorio Institucional de la UNED
instname:Universidad Nacional de Educación a Distancia
instname_str Universidad Nacional de Educación a Distancia
reponame_str e-spacio. Repositorio Institucional de la UNED
collection e-spacio. Repositorio Institucional de la UNED
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869410799610494976
score 15,811543

Similares en LA Referencia