Transformer based Model for Coherence Evaluation of Scientific Abstracts: Second Fine-tuned BERT

Coherence evaluation is a problem related to the area of natural language processing whose complexity lies mainly in the analysis of the semantics and context of the words in the text. Fortunately, the Bidirectional Encoder Representation from Transformers (BERT) architecture can capture the aforeme...

ver descrição completa

Detalhes bibliográficos
Autores: Gutiérrez Choque, Anyelo Carlos, Medina Mamani, Vivian, Castro Gutiérrez, Eveling, Núñez Pacheco, Rosa, Aguaded, José Ignacio
Formato: artículo
Fecha de publicación:2022
País:España
Recursos:Universidad de Huelva (UHU)
Repositorio:Arias Montano. Repositorio Institucional de la Universidad de Huelva
Idioma:inglés
OAI Identifier:oai:ariasmontano.uhu.es:10272/22012
Acesso em linha:https://hdl.handle.net/10272/22012
Access Level:acceso abierto
Palavra-chave:Coherence evaluation
Inconsistent sentences detection
BERT
Second fine-tuned
5701 Lingüística Aplicada
id ES_6495df9afbb0d2c5b4bb533e9f3110bd
oai_identifier_str oai:ariasmontano.uhu.es:10272/22012
network_acronym_str ES
network_name_str España
repository_id_str
spelling Transformer based Model for Coherence Evaluation of Scientific Abstracts: Second Fine-tuned BERTGutiérrez Choque, Anyelo CarlosMedina Mamani, VivianCastro Gutiérrez, EvelingNúñez Pacheco, RosaAguaded, José IgnacioCoherence evaluationInconsistent sentences detectionBERTSecond fine-tuned5701 Lingüística AplicadaCoherence evaluation is a problem related to the area of natural language processing whose complexity lies mainly in the analysis of the semantics and context of the words in the text. Fortunately, the Bidirectional Encoder Representation from Transformers (BERT) architecture can capture the aforementioned variables and represent them as embeddings to perform Fine-tunings. The present study proposes a Second Fine-Tuned model based on BERT to detect inconsistent sentences (coherence evaluation) in scientific abstracts written in English/Spanish. For this purpose, 2 formal methods for the generation of inconsistent abstracts have been proposed: Random Manipulation (RM) and K-means Random Manipulation (KRM). Six experiments were performed; showing that performing Second Fine-Tuned improves the detection of inconsistent sentences with an accuracy of 71%. This happens even if the new retraining data are of different language or different domain. It was also shown that using several methods for generating inconsistent abstracts and mixing them when performing Second Fine-Tuned does not provide better results than using a single technique.The Science and Information Organization20222022-01-0120222022-01-01journal articlehttp://purl.org/coar/resource_type/c_6501VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/10272/22012reponame:Arias Montano. Repositorio Institucional de la Universidad de Huelvainstname:Universidad de Huelva (UHU)Inglésengopen accesshttp://purl.org/coar/access_right/c_abf2Atribución-NoComercial-SinDerivadas 3.0 Españahttp://creativecommons.org/licenses/by-nc-nd/3.0/es/info:eu-repo/semantics/openAccessoai:ariasmontano.uhu.es:10272/220122026-06-02T14:58:11Z
dc.title.none.fl_str_mv Transformer based Model for Coherence Evaluation of Scientific Abstracts: Second Fine-tuned BERT
title Transformer based Model for Coherence Evaluation of Scientific Abstracts: Second Fine-tuned BERT
spellingShingle Transformer based Model for Coherence Evaluation of Scientific Abstracts: Second Fine-tuned BERT
Gutiérrez Choque, Anyelo Carlos
Coherence evaluation
Inconsistent sentences detection
BERT
Second fine-tuned
5701 Lingüística Aplicada
title_short Transformer based Model for Coherence Evaluation of Scientific Abstracts: Second Fine-tuned BERT
title_full Transformer based Model for Coherence Evaluation of Scientific Abstracts: Second Fine-tuned BERT
title_fullStr Transformer based Model for Coherence Evaluation of Scientific Abstracts: Second Fine-tuned BERT
title_full_unstemmed Transformer based Model for Coherence Evaluation of Scientific Abstracts: Second Fine-tuned BERT
title_sort Transformer based Model for Coherence Evaluation of Scientific Abstracts: Second Fine-tuned BERT
dc.creator.none.fl_str_mv Gutiérrez Choque, Anyelo Carlos
Medina Mamani, Vivian
Castro Gutiérrez, Eveling
Núñez Pacheco, Rosa
Aguaded, José Ignacio
author Gutiérrez Choque, Anyelo Carlos
author_facet Gutiérrez Choque, Anyelo Carlos
Medina Mamani, Vivian
Castro Gutiérrez, Eveling
Núñez Pacheco, Rosa
Aguaded, José Ignacio
author_role author
author2 Medina Mamani, Vivian
Castro Gutiérrez, Eveling
Núñez Pacheco, Rosa
Aguaded, José Ignacio
author2_role author
author
author
author
dc.contributor.none.fl_str_mv
dc.subject.none.fl_str_mv Coherence evaluation
Inconsistent sentences detection
BERT
Second fine-tuned
5701 Lingüística Aplicada
topic Coherence evaluation
Inconsistent sentences detection
BERT
Second fine-tuned
5701 Lingüística Aplicada
description Coherence evaluation is a problem related to the area of natural language processing whose complexity lies mainly in the analysis of the semantics and context of the words in the text. Fortunately, the Bidirectional Encoder Representation from Transformers (BERT) architecture can capture the aforementioned variables and represent them as embeddings to perform Fine-tunings. The present study proposes a Second Fine-Tuned model based on BERT to detect inconsistent sentences (coherence evaluation) in scientific abstracts written in English/Spanish. For this purpose, 2 formal methods for the generation of inconsistent abstracts have been proposed: Random Manipulation (RM) and K-means Random Manipulation (KRM). Six experiments were performed; showing that performing Second Fine-Tuned improves the detection of inconsistent sentences with an accuracy of 71%. This happens even if the new retraining data are of different language or different domain. It was also shown that using several methods for generating inconsistent abstracts and mixing them when performing Second Fine-Tuned does not provide better results than using a single technique.
publishDate 2022
dc.date.none.fl_str_mv 2022
2022-01-01
2022
2022-01-01
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/10272/22012
url https://hdl.handle.net/10272/22012
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Atribución-NoComercial-SinDerivadas 3.0 España
http://creativecommons.org/licenses/by-nc-nd/3.0/es/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Atribución-NoComercial-SinDerivadas 3.0 España
http://creativecommons.org/licenses/by-nc-nd/3.0/es/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv The Science and Information Organization
publisher.none.fl_str_mv The Science and Information Organization
dc.source.none.fl_str_mv reponame:Arias Montano. Repositorio Institucional de la Universidad de Huelva
instname:Universidad de Huelva (UHU)
instname_str Universidad de Huelva (UHU)
reponame_str Arias Montano. Repositorio Institucional de la Universidad de Huelva
collection Arias Montano. Repositorio Institucional de la Universidad de Huelva
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869409656783241216
score 15,811543

Registos relacionados