Lexical semantics on word embeddings through deep metric learning.
In the last time, remarkable advances in natural language processing, such as translation and dialog systems, have been obtained. The technologies used include neural network models and distributed representations of word semantics, known as word embeddings. This thesis deepens in the field of word...
| Autor: | |
|---|---|
| Tipo de recurso: | tesis doctoral |
| Estado: | Versión aceptada para publicación |
| Fecha de publicación: | 2024 |
| País: | Uruguay |
| Institución: | Universidad de la República |
| Repositorio: | COLIBRI |
| Idioma: | inglés |
| OAI Identifier: | oai:colibri.udelar.edu.uy:20.500.12008/46940 |
| Acceso en línea: | https://hdl.handle.net/20.500.12008/46940 |
| Access Level: | acceso abierto |
| Palabra clave: | Semántica léxica Aprendizaje métrico profundo Sinonimia Antonimia Hiperonimia Semántica vectorial Morfismo de orden Red siamesa Red parasiamesa lexical semantics Deep metric learning Synonymy Antonymy Hypernymy Word embedding Order embedding Siamese network Parasiamese network |
| Sumario: | In the last time, remarkable advances in natural language processing, such as translation and dialog systems, have been obtained. The technologies used include neural network models and distributed representations of word semantics, known as word embeddings. This thesis deepens in the field of word embeddings and the use of deep metric learning approaches for lexical semantics. Three main lexical relations are considered: synonymy, hypernymy, and antonymy. These relations, which are particularly related with paraphrasing and textual entailment, are aligned with three mathematical binary relations: equivalence, partial orders, and antitransitive relations, respectively. Equivalence and partial order relations are well-known in mathematics, differing in the relation symmetry property. Regarding antitransitive relations, they overcome the inadequacy of the two previous ones to model contrariness or opposition, which is present in antonymic terms. The approach of this thesis is to address the aforementioned semantic relations using techniques inspired by deep metric learning. Particularly, taking into consideration properties like relation symmetry and transitivity, in order to formulate well-suited approaches for each relation. The approach is to reencode a set of pretrained word embeddings on a supervised learning setup to generalize the relation to unseen cases. For synonymy, siamese neural networks are suitable, since equivalence relations fit well to be modeled by terms of distance functions. For hypernymy detection, order embeddings were used to learn ordered representations with successful results. For antonymy, the parasiamese and repelling parasiamese network models were developed, allowing us to distinguish antonyms from synonyms in the embedding space, through weakening the tendency of siamese and triplet networks to learn transitive relations. The main contributions from this thesis are: (1) the use of order embeddings to encode word embeddings for hypernymy detection, (2) the development of the parasiamese and repelling-parasiamese models to learn antitransitive relations, particularly antonymy, (3) experiments with benchmark and specially tailored datasets obtaining state-of-the-art results, and (4) two datasets in Spanish developed for hypernymy detection and antonymysynonymy discrimination tasks. |
|---|