Towards zero-shot cross-lingual named entity disambiguation

[EN]In cross-Lingual Named Entity Disambiguation (XNED) the task is to link Named Entity mentions in text in some native language to English entities in a knowledge graph. XNED systems usually require training data for each native language, limiting their application for low resource languages with...

ver descrição completa

Detalhes bibliográficos
Autores: Barrena Madinabeitia, Ander, Soroa Echave, Aitor, Agirre Bengoa, Eneko
Formato: artículo
Fecha de publicación:2021
País:España
Recursos:Universidad del País Vasco
Repositorio:Addi. Archivo Digital para la Docencia y la Investigación
OAI Identifier:oai:addi.ehu.eus:10810/54482
Acesso em linha:http://hdl.handle.net/10810/54482
Access Level:acceso abierto
Palavra-chave:cross-lingual named entity disambiguation
cross-lingual entity linking
zero-shot learning
transfer learning
pre-trained language models
low-resource languages
id ES_4c0afe0739f736c66f7bd638908d8040
oai_identifier_str oai:addi.ehu.eus:10810/54482
network_acronym_str ES
network_name_str España
repository_id_str
spelling Towards zero-shot cross-lingual named entity disambiguationBarrena Madinabeitia, AnderSoroa Echave, AitorAgirre Bengoa, Enekocross-lingual named entity disambiguationcross-lingual entity linkingzero-shot learningtransfer learningpre-trained language modelslow-resource languages[EN]In cross-Lingual Named Entity Disambiguation (XNED) the task is to link Named Entity mentions in text in some native language to English entities in a knowledge graph. XNED systems usually require training data for each native language, limiting their application for low resource languages with small amounts of training data. Prior work have proposed so-called zero-shot transfer systems which are only trained in English training data, but required native prior probabilities of entities with respect to mentions, which had to be estimated from native training examples, limiting their practical interest. In this work we present a zero-shot XNED architecture where, instead of a single disambiguation model, we have a model for each possible mention string, thus eliminating the need for native prior probabilities. Our system improves over prior work in XNED datasets in Spanish and Chinese by 32 and 27 points, and matches the systems which do require native prior information. We experiment with different multilingual transfer strategies, showing that better results are obtained with a purpose-built multilingual pre-training method compared to state-of-the-art generic multilingual models such as XLM-R. We also discovered, surprisingly, that English is not necessarily the most effective zero-shot training language for XNED into English. For instance, Spanish is more effective when training a zero-shot XNED system that dis-ambiguates Basque mentions with respect to an English knowledge graph.This work has been partially funded by the Basque Government (IXA excellence research group (IT1343-19) and DeepText project), Project BigKnowledge (Ayudas Fundacion BBVA a equipos de investigacion cientifica 2018) and via the IARPA BETTER Program contract 2019-19051600006 (ODNI, IARPA activity). Ander Barrena enjoys a post-doctoral grant ESPDOC18/101 from the UPV/EHU and also acknowledges the support of the NVIDIA Corporation with the donation of a Titan V GPU used for this research. The author thankfully acknowledges the computer resources at CTE-Power9 + V100 and technical support provided by Barcelona Supercomputing Center (RES-IM-2020-1-0020).Elsevier202120212021info:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10810/54482reponame:Addi. Archivo Digital para la Docencia y la Investigacióninstname:Universidad del País VascoIngléshttps://www.sciencedirect.com/science/article/pii/S0957417421009490?via%3Dihubinfo:eu-repo/semantics/openAccesshttp://creativecommons.org/licenses/by-nc-nd/3.0/es/© 2021 The Author(s). This is an open access article under the CC BY-NC-ND licensAtribución-NoComercial-SinDerivadas 3.0 Españaoai:addi.ehu.eus:10810/544822026-06-18T09:23:17Z
dc.title.none.fl_str_mv Towards zero-shot cross-lingual named entity disambiguation
title Towards zero-shot cross-lingual named entity disambiguation
spellingShingle Towards zero-shot cross-lingual named entity disambiguation
Barrena Madinabeitia, Ander
cross-lingual named entity disambiguation
cross-lingual entity linking
zero-shot learning
transfer learning
pre-trained language models
low-resource languages
title_short Towards zero-shot cross-lingual named entity disambiguation
title_full Towards zero-shot cross-lingual named entity disambiguation
title_fullStr Towards zero-shot cross-lingual named entity disambiguation
title_full_unstemmed Towards zero-shot cross-lingual named entity disambiguation
title_sort Towards zero-shot cross-lingual named entity disambiguation
dc.creator.none.fl_str_mv Barrena Madinabeitia, Ander
Soroa Echave, Aitor
Agirre Bengoa, Eneko
author Barrena Madinabeitia, Ander
author_facet Barrena Madinabeitia, Ander
Soroa Echave, Aitor
Agirre Bengoa, Eneko
author_role author
author2 Soroa Echave, Aitor
Agirre Bengoa, Eneko
author2_role author
author
dc.subject.none.fl_str_mv cross-lingual named entity disambiguation
cross-lingual entity linking
zero-shot learning
transfer learning
pre-trained language models
low-resource languages
topic cross-lingual named entity disambiguation
cross-lingual entity linking
zero-shot learning
transfer learning
pre-trained language models
low-resource languages
description [EN]In cross-Lingual Named Entity Disambiguation (XNED) the task is to link Named Entity mentions in text in some native language to English entities in a knowledge graph. XNED systems usually require training data for each native language, limiting their application for low resource languages with small amounts of training data. Prior work have proposed so-called zero-shot transfer systems which are only trained in English training data, but required native prior probabilities of entities with respect to mentions, which had to be estimated from native training examples, limiting their practical interest. In this work we present a zero-shot XNED architecture where, instead of a single disambiguation model, we have a model for each possible mention string, thus eliminating the need for native prior probabilities. Our system improves over prior work in XNED datasets in Spanish and Chinese by 32 and 27 points, and matches the systems which do require native prior information. We experiment with different multilingual transfer strategies, showing that better results are obtained with a purpose-built multilingual pre-training method compared to state-of-the-art generic multilingual models such as XLM-R. We also discovered, surprisingly, that English is not necessarily the most effective zero-shot training language for XNED into English. For instance, Spanish is more effective when training a zero-shot XNED system that dis-ambiguates Basque mentions with respect to an English knowledge graph.
publishDate 2021
dc.date.none.fl_str_mv 2021
2021
2021
dc.type.none.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv http://hdl.handle.net/10810/54482
url http://hdl.handle.net/10810/54482
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv https://www.sciencedirect.com/science/article/pii/S0957417421009490?via%3Dihub
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
http://creativecommons.org/licenses/by-nc-nd/3.0/es/
© 2021 The Author(s). This is an open access article under the CC BY-NC-ND licens
Atribución-NoComercial-SinDerivadas 3.0 España
eu_rights_str_mv openAccess
rights_invalid_str_mv http://creativecommons.org/licenses/by-nc-nd/3.0/es/
© 2021 The Author(s). This is an open access article under the CC BY-NC-ND licens
Atribución-NoComercial-SinDerivadas 3.0 España
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Elsevier
publisher.none.fl_str_mv Elsevier
dc.source.none.fl_str_mv reponame:Addi. Archivo Digital para la Docencia y la Investigación
instname:Universidad del País Vasco
instname_str Universidad del País Vasco
reponame_str Addi. Archivo Digital para la Docencia y la Investigación
collection Addi. Archivo Digital para la Docencia y la Investigación
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869407601617272832
score 15,301603

Registos relacionados