Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-review

Background: From 2005 to 2010, we conducted 2 randomized studies on a journal (Medicina Clínica), where we took manuscripts received for publication and randomly assigned them to either the standard editorial process or to additional processes. Both studies were based on the use of methodological re...

Descripción completa

Detalles Bibliográficos
Autores: Vilaró, Marta, Cortés, Jordi, Selva O'Callaghan, Albert, Urrutia, Agustín, Ribera, Josep Maria, Cardellach, Francesc, Basagaña, Xavier, Elmore, Matthew, Vilardell, Miquel, Altman, Douglas G., González, José Antonio, Cobo, Erik
Tipo de recurso: artículo
Estado:Versión publicada
Fecha de publicación:2019
País:España
Institución:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
Repositorio:Recercat. Dipósit de la Recerca de Catalunya
OAI Identifier:oai:recercat.cat:2445/177039
Acceso en línea:https://hdl.handle.net/2445/177039
Access Level:acceso abierto
Palabra clave:Manuscrits
Avaluació d'experts
Bibliometria
Manuscripts
Peer review
Bibliometrics
id ES_b404053b6612af3d4ebe2b707f543f8a
oai_identifier_str oai:recercat.cat:2445/177039
network_acronym_str ES
network_name_str España
repository_id_str
spelling Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-reviewVilaró, MartaCortés, JordiSelva O'Callaghan, AlbertUrrutia, AgustínRibera, Josep MariaCardellach, FrancescBasagaña, XavierElmore, MatthewVilardell, MiquelAltman, Douglas G.González, José AntonioCobo, ErikManuscritsAvaluació d'expertsBibliometriaManuscriptsPeer reviewBibliometricsBackground: From 2005 to 2010, we conducted 2 randomized studies on a journal (Medicina Clínica), where we took manuscripts received for publication and randomly assigned them to either the standard editorial process or to additional processes. Both studies were based on the use of methodological reviewers and reporting guidelines (RG). Those interventions slightly improved the items reported on the Manuscript Quality Assessment Instrument (MQAI), which assesses the quality of the research report. However, masked evaluators were able to guess the allocated group in 62% (56/90) of the papers, thus presenting a risk of detection bias. In this post-hoc study, we analyse whether those interventions that were originally designed for improving the completeness of manuscript reporting may have had an effect on the number of citations, which is the measured outcome that we used. Methods: Masked to the intervention group, one of us used the Web of Science (WoS) to quantify the number of citations that the participating manuscripts received up December 2016. We calculated the mean citation ratio between intervention arms and then quantified the uncertainty of it by means of the Jackknife method, which avoids assumptions about the distribution shape. Results: Our study included 191 articles (99 and 92, respectively) from the two previous studies, which all together received 1336 citations. In both studies, the groups subjected to additional processes showed higher averages, standard deviations and annual rates. The intervention effect was similar in both studies, with a combined estimate of a 43% (95% CI: 3 to 98%) increase in the number of citations. Conclusions: We interpret that those effects are driven mainly by introducing into the editorial process a senior methodologist to find missing RG items. Those results are promising, but not definitive due to the exploratory nature of the study and some important caveats such as: the limitations of using the number of citations as a measure of scientific impact; and the fact that our study is based on a single journal. We invite journals to perform their own studies to ascertain whether or not scientific repercussion is increased by adhering to reporting guidelines and further involving statisticians in the editorial process.BioMed Central2021202120192021info:eu-repo/semantics/articleinfo:eu-repo/semantics/publishedVersion7 p.application/pdfhttps://hdl.handle.net/2445/177039Articles publicats en revistes (Medicina)reponame:Recercat. Dipósit de la Recerca de Catalunyainstname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)InglésReproducció del document publicat a: https://doi.org/10.1186/s12874-019-0746-4BMC Medical Research Methodology, 2019, vol. 19, num. 1, p. 112https://doi.org/10.1186/s12874-019-0746-4info:eu-repo/grantAgreement/EC/H2020/676207cc-by (c) Vilaró, Marta et al., 2019http://creativecommons.org/licenses/by/3.0/esinfo:eu-repo/semantics/openAccessoai:recercat.cat:2445/1770392026-05-29T05:05:01Z
dc.title.none.fl_str_mv Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-review
title Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-review
spellingShingle Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-review
Vilaró, Marta
Manuscrits
Avaluació d'experts
Bibliometria
Manuscripts
Peer review
Bibliometrics
title_short Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-review
title_full Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-review
title_fullStr Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-review
title_full_unstemmed Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-review
title_sort Adherence to reporting guidelines increases the number of citations: the argument for including a methodologist in the editorial process and peer-review
dc.creator.none.fl_str_mv Vilaró, Marta
Cortés, Jordi
Selva O'Callaghan, Albert
Urrutia, Agustín
Ribera, Josep Maria
Cardellach, Francesc
Basagaña, Xavier
Elmore, Matthew
Vilardell, Miquel
Altman, Douglas G.
González, José Antonio
Cobo, Erik
author Vilaró, Marta
author_facet Vilaró, Marta
Cortés, Jordi
Selva O'Callaghan, Albert
Urrutia, Agustín
Ribera, Josep Maria
Cardellach, Francesc
Basagaña, Xavier
Elmore, Matthew
Vilardell, Miquel
Altman, Douglas G.
González, José Antonio
Cobo, Erik
author_role author
author2 Cortés, Jordi
Selva O'Callaghan, Albert
Urrutia, Agustín
Ribera, Josep Maria
Cardellach, Francesc
Basagaña, Xavier
Elmore, Matthew
Vilardell, Miquel
Altman, Douglas G.
González, José Antonio
Cobo, Erik
author2_role author
author
author
author
author
author
author
author
author
author
author
dc.subject.none.fl_str_mv Manuscrits
Avaluació d'experts
Bibliometria
Manuscripts
Peer review
Bibliometrics
topic Manuscrits
Avaluació d'experts
Bibliometria
Manuscripts
Peer review
Bibliometrics
description Background: From 2005 to 2010, we conducted 2 randomized studies on a journal (Medicina Clínica), where we took manuscripts received for publication and randomly assigned them to either the standard editorial process or to additional processes. Both studies were based on the use of methodological reviewers and reporting guidelines (RG). Those interventions slightly improved the items reported on the Manuscript Quality Assessment Instrument (MQAI), which assesses the quality of the research report. However, masked evaluators were able to guess the allocated group in 62% (56/90) of the papers, thus presenting a risk of detection bias. In this post-hoc study, we analyse whether those interventions that were originally designed for improving the completeness of manuscript reporting may have had an effect on the number of citations, which is the measured outcome that we used. Methods: Masked to the intervention group, one of us used the Web of Science (WoS) to quantify the number of citations that the participating manuscripts received up December 2016. We calculated the mean citation ratio between intervention arms and then quantified the uncertainty of it by means of the Jackknife method, which avoids assumptions about the distribution shape. Results: Our study included 191 articles (99 and 92, respectively) from the two previous studies, which all together received 1336 citations. In both studies, the groups subjected to additional processes showed higher averages, standard deviations and annual rates. The intervention effect was similar in both studies, with a combined estimate of a 43% (95% CI: 3 to 98%) increase in the number of citations. Conclusions: We interpret that those effects are driven mainly by introducing into the editorial process a senior methodologist to find missing RG items. Those results are promising, but not definitive due to the exploratory nature of the study and some important caveats such as: the limitations of using the number of citations as a measure of scientific impact; and the fact that our study is based on a single journal. We invite journals to perform their own studies to ascertain whether or not scientific repercussion is increased by adhering to reporting guidelines and further involving statisticians in the editorial process.
publishDate 2019
dc.date.none.fl_str_mv 2019
2021
2021
2021
dc.type.none.fl_str_mv info:eu-repo/semantics/article
info:eu-repo/semantics/publishedVersion
format article
status_str publishedVersion
dc.identifier.none.fl_str_mv https://hdl.handle.net/2445/177039
url https://hdl.handle.net/2445/177039
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.relation.none.fl_str_mv Reproducció del document publicat a: https://doi.org/10.1186/s12874-019-0746-4
BMC Medical Research Methodology, 2019, vol. 19, num. 1, p. 112
https://doi.org/10.1186/s12874-019-0746-4
info:eu-repo/grantAgreement/EC/H2020/676207
dc.rights.none.fl_str_mv cc-by (c) Vilaró, Marta et al., 2019
http://creativecommons.org/licenses/by/3.0/es
info:eu-repo/semantics/openAccess
rights_invalid_str_mv cc-by (c) Vilaró, Marta et al., 2019
http://creativecommons.org/licenses/by/3.0/es
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv 7 p.
application/pdf
dc.publisher.none.fl_str_mv BioMed Central
publisher.none.fl_str_mv BioMed Central
dc.source.none.fl_str_mv Articles publicats en revistes (Medicina)
reponame:Recercat. Dipósit de la Recerca de Catalunya
instname:Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
instname_str Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya)
reponame_str Recercat. Dipósit de la Recerca de Catalunya
collection Recercat. Dipósit de la Recerca de Catalunya
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869417225861988352
score 15,81155

Similares en LA Referencia