Evaluating research and researchers by the journal impact factor: Is it better than coin flipping?

The journal impact factor (JIF) is the average of the number of citations of the papers published in a journal, calculated according to a specific formula; it is extensively used for the evaluation of research and researchers. The method assumes that all papers in a journal have the same scientific...

Descripción completa

Detalles Bibliográficos
Autores: Brito López, Ricardo, Rodríguez Navarro, Alonso
Tipo de recurso: artículo
Fecha de publicación:2019
País:España
Institución:Universidad Complutense de Madrid (UCM)
Repositorio:Docta Complutense
Idioma:inglés
OAI Identifier:oai:docta.ucm.es:20.500.14352/115459
Acceso en línea:https://hdl.handle.net/20.500.14352/115459
Access Level:acceso abierto
Palabra clave:001.8
004
Research evaluation
Impact factor
Failure probability
Biblioteconomía y Documentación
Bibliometría
5399 Otras Especialidades económicas
id ES_df72dc7491c2430b68fd8eb3e3ee4a0d
oai_identifier_str oai:docta.ucm.es:20.500.14352/115459
network_acronym_str ES
network_name_str España
repository_id_str
spelling Evaluating research and researchers by the journal impact factor: Is it better than coin flipping?Brito López, RicardoRodríguez Navarro, Alonso001.8004Research evaluationImpact factorFailure probabilityBiblioteconomía y DocumentaciónBibliometría5399 Otras Especialidades económicasThe journal impact factor (JIF) is the average of the number of citations of the papers published in a journal, calculated according to a specific formula; it is extensively used for the evaluation of research and researchers. The method assumes that all papers in a journal have the same scientific merit, which is measured by the JIF of the publishing journal. This implies that the number of citations measures scientific merits but the JIF does not evaluate each individual paper by its own number of citations. Therefore, in the comparative evaluation of two papers, the use of the JIF implies a risk of failure, which occurs when a paper in the journal with the lower JIF is compared to another with fewer citations in the journal with the higher JIF. To quantify this risk of failure, this study calculates the failure probabilities, taking advantage of the lognormal distribution of citations. In two journals whose JIFs are ten-fold different, the failure probability is low. However, in most cases when two papers are compared, the JIFs of the journals are not so different. Then, the failure probability can be close to 0.5, which is equivalent to evaluating by coin flipping.ElsevierUniversidad Complutense de Madrid20192019-01-0120192019-01-01journal articlehttp://purl.org/coar/resource_type/c_6501AOhttp://purl.org/coar/version/c_b1a7d7d4d402bcceinfo:eu-repo/semantics/articleapplication/pdfapplication/pdfhttps://hdl.handle.net/20.500.14352/115459reponame:Docta Complutenseinstname:Universidad Complutense de Madrid (UCM)InglésengMinisterio de Economía y Competitividad http://dx.doi.org/10.13039/501100003329 Not available FIS2014-52486-R TERMODINAMICA EN LA MICRO-ESCALAAgencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016 FIS2017-83709-R CONVERSION DE ENERGIA, TRANSPORTE Y PROCESOS ACTIVOS EN SISTEMAS CON FLUCTUACIONESopen accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccessoai:docta.ucm.es:20.500.14352/1154592026-06-02T12:44:21Z
dc.title.none.fl_str_mv Evaluating research and researchers by the journal impact factor: Is it better than coin flipping?
title Evaluating research and researchers by the journal impact factor: Is it better than coin flipping?
spellingShingle Evaluating research and researchers by the journal impact factor: Is it better than coin flipping?
Brito López, Ricardo
001.8
004
Research evaluation
Impact factor
Failure probability
Biblioteconomía y Documentación
Bibliometría
5399 Otras Especialidades económicas
title_short Evaluating research and researchers by the journal impact factor: Is it better than coin flipping?
title_full Evaluating research and researchers by the journal impact factor: Is it better than coin flipping?
title_fullStr Evaluating research and researchers by the journal impact factor: Is it better than coin flipping?
title_full_unstemmed Evaluating research and researchers by the journal impact factor: Is it better than coin flipping?
title_sort Evaluating research and researchers by the journal impact factor: Is it better than coin flipping?
dc.creator.none.fl_str_mv Brito López, Ricardo
Rodríguez Navarro, Alonso
author Brito López, Ricardo
author_facet Brito López, Ricardo
Rodríguez Navarro, Alonso
author_role author
author2 Rodríguez Navarro, Alonso
author2_role author
dc.contributor.none.fl_str_mv Universidad Complutense de Madrid
dc.subject.none.fl_str_mv 001.8
004
Research evaluation
Impact factor
Failure probability
Biblioteconomía y Documentación
Bibliometría
5399 Otras Especialidades económicas
topic 001.8
004
Research evaluation
Impact factor
Failure probability
Biblioteconomía y Documentación
Bibliometría
5399 Otras Especialidades económicas
description The journal impact factor (JIF) is the average of the number of citations of the papers published in a journal, calculated according to a specific formula; it is extensively used for the evaluation of research and researchers. The method assumes that all papers in a journal have the same scientific merit, which is measured by the JIF of the publishing journal. This implies that the number of citations measures scientific merits but the JIF does not evaluate each individual paper by its own number of citations. Therefore, in the comparative evaluation of two papers, the use of the JIF implies a risk of failure, which occurs when a paper in the journal with the lower JIF is compared to another with fewer citations in the journal with the higher JIF. To quantify this risk of failure, this study calculates the failure probabilities, taking advantage of the lognormal distribution of citations. In two journals whose JIFs are ten-fold different, the failure probability is low. However, in most cases when two papers are compared, the JIFs of the journals are not so different. Then, the failure probability can be close to 0.5, which is equivalent to evaluating by coin flipping.
publishDate 2019
dc.date.none.fl_str_mv 2019
2019-01-01
2019
2019-01-01
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
AO
http://purl.org/coar/version/c_b1a7d7d4d402bcce
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/20.500.14352/115459
url https://hdl.handle.net/20.500.14352/115459
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv Ministerio de Economía y Competitividad http://dx.doi.org/10.13039/501100003329 Not available FIS2014-52486-R TERMODINAMICA EN LA MICRO-ESCALA
Agencia Estatal de Investigación http://dx.doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016 FIS2017-83709-R CONVERSION DE ENERGIA, TRANSPORTE Y PROCESOS ACTIVOS EN SISTEMAS CON FLUCTUACIONES
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
application/pdf
dc.publisher.none.fl_str_mv Elsevier
publisher.none.fl_str_mv Elsevier
dc.source.none.fl_str_mv reponame:Docta Complutense
instname:Universidad Complutense de Madrid (UCM)
instname_str Universidad Complutense de Madrid (UCM)
reponame_str Docta Complutense
collection Docta Complutense
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869422065041276928
score 15.812455

Similares en LA Referencia