Recognizing misogynous memes: Biased models and tricky archetypes

[EN] Misogyny is a form of hate against women and has been spreading exponentially through the Web, especially on social media platforms. Hateful content towards women can be conveyed not only by text but also using visual and/or audio sources or their combination, highlighting the necessity to addr...

Descripción completa

Detalles Bibliográficos
Autores: Rizzi, Giulia, Gasparini, Francesca, Saibene, Aurora, Fersini, Elisabetta, Rosso, Paolo
Tipo de recurso: artículo
Fecha de publicación:2023
País:España
Institución:Universitat Politècnica de València (UPV)
Repositorio:RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
Idioma:inglés
OAI Identifier:oai:riunet.upv.es:10251/214065
Acceso en línea:https://riunet.upv.es/handle/10251/214065
Access Level:acceso abierto
Palabra clave:Misogyny identification
Meme
Bias estimation
Bias mitigation
LENGUAJES Y SISTEMAS INFORMATICOS
id ES_3b4ffdeeaa645bbb6114983388202d64
oai_identifier_str oai:riunet.upv.es:10251/214065
network_acronym_str ES
network_name_str España
repository_id_str
spelling Recognizing misogynous memes: Biased models and tricky archetypesRizzi, GiuliaGasparini, FrancescaSaibene, AuroraFersini, ElisabettaRosso, PaoloMisogyny identificationMemeBias estimationBias mitigationLENGUAJES Y SISTEMAS INFORMATICOS[EN] Misogyny is a form of hate against women and has been spreading exponentially through the Web, especially on social media platforms. Hateful content towards women can be conveyed not only by text but also using visual and/or audio sources or their combination, highlighting the necessity to address it from a multimodal perspective. One of the predominant forms of multimodal content against women is represented by memes, which are images characterized by pictorial content with an overlaying text introduced a posteriori. Its main aim is originally to be funny and/or ironic, making misogyny recognition in memes even more challenging. In this paper, we investigated 4 unimodal and 3 multimodal approaches to determine which source of information contributes more to the detection of misogynous memes. Moreover, a bias estimation technique is proposed to identify specific elements that compose a meme that could lead to unfair models, together with a bias mitigation strategy based on Bayesian Optimization. The proposed method is able to push the prediction probabilities towards the correct class for up to 61.43% of the cases. Finally, we identified the most challenging archetypes of memes that are still far to be properly recognized, highlighting the most relevant open research directions.The work of Paolo Rosso was done in the framework of the FairTransNLP-Stereotypes research project on Fairness and Transparency for equitable NLP applications in social media: Identifying stereotypes and prejudices and developing equitable systems (PID2021-124361OB-C31) funded by MCIN/AEI/10.13039/501100011033 and by ERDF, EU A way of making Europe. The work of Elisabetta Fersini and Francesca Gasparini has been partially funded by the European Union - NextGenerationEU under the National Research Centre For HPC, Big Data and Quantum Computing-Spoke 9-Digital Society and Smart Cities (PNRR-MUR), and by REGAINS - Excellence Department Research Project.ElsevierDepartamento de Sistemas Informáticos y ComputaciónEscuela Técnica Superior de Ingeniería InformáticaCentro de Investigación Pattern Recognition and Human Language TechnologyEuropean CommissionAGENCIA ESTATAL DE INVESTIGACIONEuropean Regional Development FundRepositorio Institucional de la Universitat Politècnica de València Riunet20232023-09-01journal articlehttp://purl.org/coar/resource_type/c_6501VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttps://riunet.upv.es/handle/10251/214065reponame:RiuNet. Repositorio Institucional de la Universitat Politécnica de Valénciainstname:Universitat Politècnica de València (UPV)InglésengAgencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023 PID2021-124361OB-C31 FAIRTRANSNLP-STEREOTYPES: IDENTIFICACION DE ESTEREOTIPOS Y PREJUICIOS Y DESARROLLO DE SISTEMAS EQUITATIVOSopen accesshttp://purl.org/coar/access_right/c_abf2Reconocimiento - No comercial - Sin obra derivada (by-nc-nd) http://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccessoai:riunet.upv.es:10251/2140652026-06-13T07:49:27Z
dc.title.none.fl_str_mv Recognizing misogynous memes: Biased models and tricky archetypes
title Recognizing misogynous memes: Biased models and tricky archetypes
spellingShingle Recognizing misogynous memes: Biased models and tricky archetypes
Rizzi, Giulia
Misogyny identification
Meme
Bias estimation
Bias mitigation
LENGUAJES Y SISTEMAS INFORMATICOS
title_short Recognizing misogynous memes: Biased models and tricky archetypes
title_full Recognizing misogynous memes: Biased models and tricky archetypes
title_fullStr Recognizing misogynous memes: Biased models and tricky archetypes
title_full_unstemmed Recognizing misogynous memes: Biased models and tricky archetypes
title_sort Recognizing misogynous memes: Biased models and tricky archetypes
dc.creator.none.fl_str_mv Rizzi, Giulia
Gasparini, Francesca
Saibene, Aurora
Fersini, Elisabetta
Rosso, Paolo
author Rizzi, Giulia
author_facet Rizzi, Giulia
Gasparini, Francesca
Saibene, Aurora
Fersini, Elisabetta
Rosso, Paolo
author_role author
author2 Gasparini, Francesca
Saibene, Aurora
Fersini, Elisabetta
Rosso, Paolo
author2_role author
author
author
author
dc.contributor.none.fl_str_mv Departamento de Sistemas Informáticos y Computación
Escuela Técnica Superior de Ingeniería Informática
Centro de Investigación Pattern Recognition and Human Language Technology
European Commission
AGENCIA ESTATAL DE INVESTIGACION
European Regional Development Fund
Repositorio Institucional de la Universitat Politècnica de València Riunet
dc.subject.none.fl_str_mv Misogyny identification
Meme
Bias estimation
Bias mitigation
LENGUAJES Y SISTEMAS INFORMATICOS
topic Misogyny identification
Meme
Bias estimation
Bias mitigation
LENGUAJES Y SISTEMAS INFORMATICOS
description [EN] Misogyny is a form of hate against women and has been spreading exponentially through the Web, especially on social media platforms. Hateful content towards women can be conveyed not only by text but also using visual and/or audio sources or their combination, highlighting the necessity to address it from a multimodal perspective. One of the predominant forms of multimodal content against women is represented by memes, which are images characterized by pictorial content with an overlaying text introduced a posteriori. Its main aim is originally to be funny and/or ironic, making misogyny recognition in memes even more challenging. In this paper, we investigated 4 unimodal and 3 multimodal approaches to determine which source of information contributes more to the detection of misogynous memes. Moreover, a bias estimation technique is proposed to identify specific elements that compose a meme that could lead to unfair models, together with a bias mitigation strategy based on Bayesian Optimization. The proposed method is able to push the prediction probabilities towards the correct class for up to 61.43% of the cases. Finally, we identified the most challenging archetypes of memes that are still far to be properly recognized, highlighting the most relevant open research directions.
publishDate 2023
dc.date.none.fl_str_mv 2023
2023-09-01
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://riunet.upv.es/handle/10251/214065
url https://riunet.upv.es/handle/10251/214065
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv Agencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2021-2023 PID2021-124361OB-C31 FAIRTRANSNLP-STEREOTYPES: IDENTIFICACION DE ESTEREOTIPOS Y PREJUICIOS Y DESARROLLO DE SISTEMAS EQUITATIVOS
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Reconocimiento - No comercial - Sin obra derivada (by-nc-nd)
http://creativecommons.org/licenses/by-nc-nd/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Reconocimiento - No comercial - Sin obra derivada (by-nc-nd)
http://creativecommons.org/licenses/by-nc-nd/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Elsevier
publisher.none.fl_str_mv Elsevier
dc.source.none.fl_str_mv reponame:RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
instname:Universitat Politècnica de València (UPV)
instname_str Universitat Politècnica de València (UPV)
reponame_str RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
collection RiuNet. Repositorio Institucional de la Universitat Politécnica de Valéncia
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869406298157613056
score 15,811543

Similares en LA Referencia