Like-for-like bibliometric substitutes for peer review: Advantages and limits of indicators calculated from the e(p) index

The use of bibliometric indicators would simplify research assessments. The 2014 Research Excellence Framework (REF) is a peer review assessment of UK universities, whose results can be taken as benchmarks for bibliometric indicators. In this study, we use the REF results to investigate whether the...

Descripción completa

Detalles Bibliográficos
Autores: Rodríguez Navarro, Alonso, Brito López, Ricardo
Tipo de recurso: artículo
Fecha de publicación:2020
País:España
Institución:Universidad Complutense de Madrid (UCM)
Repositorio:Docta Complutense
Idioma:inglés
OAI Identifier:oai:docta.ucm.es:20.500.14352/6363
Acceso en línea:https://hdl.handle.net/20.500.14352/6363
Access Level:acceso abierto
Palabra clave:536
Impact indicators
Excellence
Efficient
Collaboration
Criteria
Science
Choice
Values
Author
Termodinámica
2213 Termodinámica
id ES_0e9f9a9ffe2b2f358b6ceece851efd72
oai_identifier_str oai:docta.ucm.es:20.500.14352/6363
network_acronym_str ES
network_name_str España
repository_id_str
spelling Like-for-like bibliometric substitutes for peer review: Advantages and limits of indicators calculated from the e(p) indexRodríguez Navarro, AlonsoBrito López, Ricardo536Impact indicatorsExcellenceEfficientCollaborationCriteriaScienceChoiceValuesAuthorTermodinámica2213 TermodinámicaThe use of bibliometric indicators would simplify research assessments. The 2014 Research Excellence Framework (REF) is a peer review assessment of UK universities, whose results can be taken as benchmarks for bibliometric indicators. In this study, we use the REF results to investigate whether the e(p) index and a top percentile of most cited papers could substitute for peer review. The probability that a random university's paper reaches a certain top percentile in the global distribution of papers is a power of the e(p) index, which can be calculated from the citation-based distribution of university's papers in global top percentiles. Making use of the e(p) index in each university and research area, we calculated the ratios between the percentage of 4-star-rated outputs in REF and the percentages of papers in global top percentiles. Then, we fixed the assessment percentile so that the mean ratio between these two indicators across universities is 1.0. This method was applied to four units of assessment in REF: Chemistry, Economics and Econometrics joined to Business and Management Studies, and Physics. Some relevant deviations from the 1.0 ratio could be explained by the evaluation procedure in REF or by the characteristics of the research field; other deviations need specific studies by experts in the research area. These results indicate that in many research areas the substitution of a top percentile indicator for peer review is possible. However, this substitution cannot be made straightforwardly; more research is needed to establish the conditions of the bibliometric assessment.Oxford University PressUniversidad Complutense de Madrid20202020-04-0120202020-04-01journal articlehttp://purl.org/coar/resource_type/c_6501info:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/20.500.14352/6363reponame:Docta Complutenseinstname:Universidad Complutense de Madrid (UCM)Inglésengopen accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccessoai:docta.ucm.es:20.500.14352/63632026-06-02T12:44:21Z
dc.title.none.fl_str_mv Like-for-like bibliometric substitutes for peer review: Advantages and limits of indicators calculated from the e(p) index
title Like-for-like bibliometric substitutes for peer review: Advantages and limits of indicators calculated from the e(p) index
spellingShingle Like-for-like bibliometric substitutes for peer review: Advantages and limits of indicators calculated from the e(p) index
Rodríguez Navarro, Alonso
536
Impact indicators
Excellence
Efficient
Collaboration
Criteria
Science
Choice
Values
Author
Termodinámica
2213 Termodinámica
title_short Like-for-like bibliometric substitutes for peer review: Advantages and limits of indicators calculated from the e(p) index
title_full Like-for-like bibliometric substitutes for peer review: Advantages and limits of indicators calculated from the e(p) index
title_fullStr Like-for-like bibliometric substitutes for peer review: Advantages and limits of indicators calculated from the e(p) index
title_full_unstemmed Like-for-like bibliometric substitutes for peer review: Advantages and limits of indicators calculated from the e(p) index
title_sort Like-for-like bibliometric substitutes for peer review: Advantages and limits of indicators calculated from the e(p) index
dc.creator.none.fl_str_mv Rodríguez Navarro, Alonso
Brito López, Ricardo
author Rodríguez Navarro, Alonso
author_facet Rodríguez Navarro, Alonso
Brito López, Ricardo
author_role author
author2 Brito López, Ricardo
author2_role author
dc.contributor.none.fl_str_mv Universidad Complutense de Madrid
dc.subject.none.fl_str_mv 536
Impact indicators
Excellence
Efficient
Collaboration
Criteria
Science
Choice
Values
Author
Termodinámica
2213 Termodinámica
topic 536
Impact indicators
Excellence
Efficient
Collaboration
Criteria
Science
Choice
Values
Author
Termodinámica
2213 Termodinámica
description The use of bibliometric indicators would simplify research assessments. The 2014 Research Excellence Framework (REF) is a peer review assessment of UK universities, whose results can be taken as benchmarks for bibliometric indicators. In this study, we use the REF results to investigate whether the e(p) index and a top percentile of most cited papers could substitute for peer review. The probability that a random university's paper reaches a certain top percentile in the global distribution of papers is a power of the e(p) index, which can be calculated from the citation-based distribution of university's papers in global top percentiles. Making use of the e(p) index in each university and research area, we calculated the ratios between the percentage of 4-star-rated outputs in REF and the percentages of papers in global top percentiles. Then, we fixed the assessment percentile so that the mean ratio between these two indicators across universities is 1.0. This method was applied to four units of assessment in REF: Chemistry, Economics and Econometrics joined to Business and Management Studies, and Physics. Some relevant deviations from the 1.0 ratio could be explained by the evaluation procedure in REF or by the characteristics of the research field; other deviations need specific studies by experts in the research area. These results indicate that in many research areas the substitution of a top percentile indicator for peer review is possible. However, this substitution cannot be made straightforwardly; more research is needed to establish the conditions of the bibliometric assessment.
publishDate 2020
dc.date.none.fl_str_mv 2020
2020-04-01
2020
2020-04-01
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/20.500.14352/6363
url https://hdl.handle.net/20.500.14352/6363
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Oxford University Press
publisher.none.fl_str_mv Oxford University Press
dc.source.none.fl_str_mv reponame:Docta Complutense
instname:Universidad Complutense de Madrid (UCM)
instname_str Universidad Complutense de Madrid (UCM)
reponame_str Docta Complutense
collection Docta Complutense
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869403407930884096
score 15,301629

Similares en LA Referencia