Mitigating social biases in machine translation using domain adaptation techniques

Misrepresentation of certain communities in current datasets is causing serious disruptions in artificial intelligence applications. Examples of this can be found from lower performance of speech recognizers for women than for men to lower accuracy in face recognition for Asian faces compared to Ame...

ver descrição completa

Detalhes bibliográficos
Autor: Jorge Sánchez, Adrián de
Formato: tesis de maestría
Fecha de publicación:2020
País:España
Recursos:Universitat Politècnica de Catalunya (UPC)
Repositorio:UPCommons. Portal del coneixement obert de la UPC
Idioma:inglés
OAI Identifier:oai:upcommons.upc.edu:2117/334921
Acesso em linha:https://hdl.handle.net/2117/334921
Access Level:acceso abierto
Palavra-chave:Neural networks (Computer science)
Machine translating
Neural Machine Translation
Word Embeddings
Gender Bias
Domain Adaptation
Xarxes neuronals (Informàtica)
Traducció automàtica
Àrees temàtiques de la UPC::Enginyeria de la telecomunicació
id ES_bfeeead8b70684a3cd93a2a34de2ba5b
oai_identifier_str oai:upcommons.upc.edu:2117/334921
network_acronym_str ES
network_name_str España
repository_id_str
spelling Mitigating social biases in machine translation using domain adaptation techniquesJorge Sánchez, Adrián deNeural networks (Computer science)Machine translatingNeural Machine TranslationWord EmbeddingsGender BiasDomain AdaptationXarxes neuronals (Informàtica)Traducció automàticaÀrees temàtiques de la UPC::Enginyeria de la telecomunicacióMisrepresentation of certain communities in current datasets is causing serious disruptions in artificial intelligence applications. Examples of this can be found from lower performance of speech recognizers for women than for men to lower accuracy in face recognition for Asian faces compared to American or European ones. It also amplifies stereotypes in Machine Translation. These challenges are at the core of natural language processing applications and, in particular, there are many works focusing on trying to solve gender biases. Previous research in the area of Machine Translation (MT) has proposed to either mitigate biases by means of using debiased word embeddings and using contextual information or evaluating and measuring the amount of bias present in the translation. The closest work to ours is the one by were authors generate a very small gender-balanced dataset and use techniques of Elastic Weight Consolidation to perform transfer learning and mitigate the consequences of training with unbalanced datasets. Differently from this one, we use a larger non-synthetic balanced dataset to perform fine-tunning on an unbalanced-dataset and evaluate the reduction of presence of gender bias in the final translation. We also evaluate the gender bias in word embedding models like in, and conclude that they can be successfully applied to downstream systems in the case of the gender-balanced dataset. The results are not exactly what we expected, since our hypothesis was that the model which would eliminate the gender bias to a greater degree would be the model that was fine-tuned with only the balanced dataset. This has not been the case, given some known difficulties that translation models have when adapting to a new and totally different distribution of data, i.e. catastrophic forgetting, which means that the model fits the new distribution but forgets the one which was trained on before. Some regularization techniques like dropout or adaptive learning rate have been applied, without having a significant improvement. Nevertheless, results show that even if the balanced dataset is from a different domain than the training and the test of the NMT system, it does improve the translation quality (up to 2 BLEU points) and it is able to mitigate the gender bias in a significant amount, up to a 12.5\% accuracy.Universitat Politècnica de CatalunyaRuiz Costa-Jussà, Marta20202020-09-0120202020-12-24master thesishttp://purl.org/coar/resource_type/c_bdccNAhttp://purl.org/coar/version/c_be7fb7dd8ff6fe43info:eu-repo/semantics/masterThesisapplication/pdfhttps://hdl.handle.net/2117/334921reponame:UPCommons. Portal del coneixement obert de la UPCinstname:Universitat Politècnica de Catalunya (UPC)Inglésengopen accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccessoai:upcommons.upc.edu:2117/3349212026-05-27T15:37:01Z
dc.title.none.fl_str_mv Mitigating social biases in machine translation using domain adaptation techniques
title Mitigating social biases in machine translation using domain adaptation techniques
spellingShingle Mitigating social biases in machine translation using domain adaptation techniques
Jorge Sánchez, Adrián de
Neural networks (Computer science)
Machine translating
Neural Machine Translation
Word Embeddings
Gender Bias
Domain Adaptation
Xarxes neuronals (Informàtica)
Traducció automàtica
Àrees temàtiques de la UPC::Enginyeria de la telecomunicació
title_short Mitigating social biases in machine translation using domain adaptation techniques
title_full Mitigating social biases in machine translation using domain adaptation techniques
title_fullStr Mitigating social biases in machine translation using domain adaptation techniques
title_full_unstemmed Mitigating social biases in machine translation using domain adaptation techniques
title_sort Mitigating social biases in machine translation using domain adaptation techniques
dc.creator.none.fl_str_mv Jorge Sánchez, Adrián de
author Jorge Sánchez, Adrián de
author_facet Jorge Sánchez, Adrián de
author_role author
dc.contributor.none.fl_str_mv Ruiz Costa-Jussà, Marta
dc.subject.none.fl_str_mv Neural networks (Computer science)
Machine translating
Neural Machine Translation
Word Embeddings
Gender Bias
Domain Adaptation
Xarxes neuronals (Informàtica)
Traducció automàtica
Àrees temàtiques de la UPC::Enginyeria de la telecomunicació
topic Neural networks (Computer science)
Machine translating
Neural Machine Translation
Word Embeddings
Gender Bias
Domain Adaptation
Xarxes neuronals (Informàtica)
Traducció automàtica
Àrees temàtiques de la UPC::Enginyeria de la telecomunicació
description Misrepresentation of certain communities in current datasets is causing serious disruptions in artificial intelligence applications. Examples of this can be found from lower performance of speech recognizers for women than for men to lower accuracy in face recognition for Asian faces compared to American or European ones. It also amplifies stereotypes in Machine Translation. These challenges are at the core of natural language processing applications and, in particular, there are many works focusing on trying to solve gender biases. Previous research in the area of Machine Translation (MT) has proposed to either mitigate biases by means of using debiased word embeddings and using contextual information or evaluating and measuring the amount of bias present in the translation. The closest work to ours is the one by were authors generate a very small gender-balanced dataset and use techniques of Elastic Weight Consolidation to perform transfer learning and mitigate the consequences of training with unbalanced datasets. Differently from this one, we use a larger non-synthetic balanced dataset to perform fine-tunning on an unbalanced-dataset and evaluate the reduction of presence of gender bias in the final translation. We also evaluate the gender bias in word embedding models like in, and conclude that they can be successfully applied to downstream systems in the case of the gender-balanced dataset. The results are not exactly what we expected, since our hypothesis was that the model which would eliminate the gender bias to a greater degree would be the model that was fine-tuned with only the balanced dataset. This has not been the case, given some known difficulties that translation models have when adapting to a new and totally different distribution of data, i.e. catastrophic forgetting, which means that the model fits the new distribution but forgets the one which was trained on before. Some regularization techniques like dropout or adaptive learning rate have been applied, without having a significant improvement. Nevertheless, results show that even if the balanced dataset is from a different domain than the training and the test of the NMT system, it does improve the translation quality (up to 2 BLEU points) and it is able to mitigate the gender bias in a significant amount, up to a 12.5\% accuracy.
publishDate 2020
dc.date.none.fl_str_mv 2020
2020-09-01
2020
2020-12-24
dc.type.none.fl_str_mv master thesis
http://purl.org/coar/resource_type/c_bdcc
NA
http://purl.org/coar/version/c_be7fb7dd8ff6fe43
dc.type.openaire.fl_str_mv info:eu-repo/semantics/masterThesis
format masterThesis
dc.identifier.none.fl_str_mv https://hdl.handle.net/2117/334921
url https://hdl.handle.net/2117/334921
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Universitat Politècnica de Catalunya
publisher.none.fl_str_mv Universitat Politècnica de Catalunya
dc.source.none.fl_str_mv reponame:UPCommons. Portal del coneixement obert de la UPC
instname:Universitat Politècnica de Catalunya (UPC)
instname_str Universitat Politècnica de Catalunya (UPC)
reponame_str UPCommons. Portal del coneixement obert de la UPC
collection UPCommons. Portal del coneixement obert de la UPC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869418434183299072
score 15.300719

Registos relacionados