Knocking down some assumptions about the effectos of evaluation systems on publications

In 1989 the Spanish Government established an individual retrospective research evaluation system (RES) for public researchers. This system was voluntary but involved the incentive of a permanent salary bonus. Policy makers have associated the establishment of this evaluation system with the signifi...

Descripción completa

Detalles Bibliográficos
Autores: Osuna, Carmen, Cruz Castro, Laura, Sanz Menéndez, Luis
Tipo de recurso: artículo
Fecha de publicación:2010
País:España
Institución:Consejo Superior de Investigaciones Científicas (CSIC)
Repositorio:DIGITAL.CSIC. Repositorio Institucional del CSIC
OAI Identifier:oai:digital.csic.es:10261/27730
Acceso en línea:http://hdl.handle.net/10261/27730
Access Level:acceso abierto
Palabra clave:Research evaluation systems
Scientific production
Effects of evaluations
Quasi-experimental design
Spain
CNEAI
id ES_4ae3a87bc829b2cccd4f7b62e0f2a901
oai_identifier_str oai:digital.csic.es:10261/27730
network_acronym_str ES
network_name_str España
repository_id_str
spelling Knocking down some assumptions about the effectos of evaluation systems on publicationsOsuna, CarmenCruz Castro, LauraSanz Menéndez, LuisResearch evaluation systemsScientific productionEffects of evaluationsQuasi-experimental designSpainCNEAIIn 1989 the Spanish Government established an individual retrospective research evaluation system (RES) for public researchers. This system was voluntary but involved the incentive of a permanent salary bonus. Policy makers have associated the establishment of this evaluation system with the signifi cant increase in the volume of scientifi c publications attributed to Spain over the last decades. In a similar vein to the analyses of other country cases, some scholars have also claimed that the growth of Spain’s international scientifi c publications is a result of the establishment of the new evaluation system. In this paper, we provide a methodological revision of the validity threats in previous research, including some interrupted time series analyses and control groups to investigate the effects of this policy instrument on the number of papers produced by Spanish authors. In the years following the establishment of the evaluation system, the results indicate a considerable increase in the number of papers attributed to Spanish authors among those eligible for evaluation (the “treated” group), but also in the control publications cannot be attributed indisputably to the effect of the establishment of the RES, but rather to the increase of expenditure and number of researchers in the Spanish R&D system along with some maturation effects. We take this case as an example of the need to improve and refi ne methodologies and to be more cautious when attributing effects to research evaluation mechanisms at the national level.Peer reviewedConsejo Superior de Investigaciones Científicas (España)Sanz Menéndez, Luis [0000-0002-6869-8105]Cruz Castro, Laura [0000-0002-9786-6251]201020102010info:eu-repo/semantics/articlehttp://purl.org/coar/resource_type/c_6501241326 bytesapplication/pdfhttp://hdl.handle.net/10261/27730reponame:DIGITAL.CSIC. Repositorio Institucional del CSICinstname:Consejo Superior de Investigaciones Científicas (CSIC)EspañolDocumento de Trabajo IPP, 10-2010http://ipp.csic.es/es/workpaper/knocking-down-some-assumptions-about-effects-evaluation-systems-publicationsinfo:eu-repo/semantics/openAccessoai:digital.csic.es:10261/277302026-05-22T06:33:51Z
dc.title.none.fl_str_mv Knocking down some assumptions about the effectos of evaluation systems on publications
title Knocking down some assumptions about the effectos of evaluation systems on publications
spellingShingle Knocking down some assumptions about the effectos of evaluation systems on publications
Osuna, Carmen
Research evaluation systems
Scientific production
Effects of evaluations
Quasi-experimental design
Spain
CNEAI
title_short Knocking down some assumptions about the effectos of evaluation systems on publications
title_full Knocking down some assumptions about the effectos of evaluation systems on publications
title_fullStr Knocking down some assumptions about the effectos of evaluation systems on publications
title_full_unstemmed Knocking down some assumptions about the effectos of evaluation systems on publications
title_sort Knocking down some assumptions about the effectos of evaluation systems on publications
dc.creator.none.fl_str_mv Osuna, Carmen
Cruz Castro, Laura
Sanz Menéndez, Luis
author Osuna, Carmen
author_facet Osuna, Carmen
Cruz Castro, Laura
Sanz Menéndez, Luis
author_role author
author2 Cruz Castro, Laura
Sanz Menéndez, Luis
author2_role author
author
dc.contributor.none.fl_str_mv Sanz Menéndez, Luis [0000-0002-6869-8105]
Cruz Castro, Laura [0000-0002-9786-6251]
dc.subject.none.fl_str_mv Research evaluation systems
Scientific production
Effects of evaluations
Quasi-experimental design
Spain
CNEAI
topic Research evaluation systems
Scientific production
Effects of evaluations
Quasi-experimental design
Spain
CNEAI
description In 1989 the Spanish Government established an individual retrospective research evaluation system (RES) for public researchers. This system was voluntary but involved the incentive of a permanent salary bonus. Policy makers have associated the establishment of this evaluation system with the signifi cant increase in the volume of scientifi c publications attributed to Spain over the last decades. In a similar vein to the analyses of other country cases, some scholars have also claimed that the growth of Spain’s international scientifi c publications is a result of the establishment of the new evaluation system. In this paper, we provide a methodological revision of the validity threats in previous research, including some interrupted time series analyses and control groups to investigate the effects of this policy instrument on the number of papers produced by Spanish authors. In the years following the establishment of the evaluation system, the results indicate a considerable increase in the number of papers attributed to Spanish authors among those eligible for evaluation (the “treated” group), but also in the control publications cannot be attributed indisputably to the effect of the establishment of the RES, but rather to the increase of expenditure and number of researchers in the Spanish R&D system along with some maturation effects. We take this case as an example of the need to improve and refi ne methodologies and to be more cautious when attributing effects to research evaluation mechanisms at the national level.
publishDate 2010
dc.date.none.fl_str_mv 2010
2010
2010
dc.type.none.fl_str_mv info:eu-repo/semantics/article
http://purl.org/coar/resource_type/c_6501
format article
dc.identifier.none.fl_str_mv http://hdl.handle.net/10261/27730
url http://hdl.handle.net/10261/27730
dc.language.none.fl_str_mv Español
language_invalid_str_mv Español
dc.relation.none.fl_str_mv Documento de Trabajo IPP, 10-2010
http://ipp.csic.es/es/workpaper/knocking-down-some-assumptions-about-effects-evaluation-systems-publications
dc.rights.none.fl_str_mv info:eu-repo/semantics/openAccess
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv 241326 bytes
application/pdf
dc.publisher.none.fl_str_mv Consejo Superior de Investigaciones Científicas (España)
publisher.none.fl_str_mv Consejo Superior de Investigaciones Científicas (España)
dc.source.none.fl_str_mv reponame:DIGITAL.CSIC. Repositorio Institucional del CSIC
instname:Consejo Superior de Investigaciones Científicas (CSIC)
instname_str Consejo Superior de Investigaciones Científicas (CSIC)
reponame_str DIGITAL.CSIC. Repositorio Institucional del CSIC
collection DIGITAL.CSIC. Repositorio Institucional del CSIC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869407523772039168
score 15.811543

Similares en LA Referencia