Rubric vs. numeric rating scale: agreement among evaluators on endodontic treatments performed by dental students

Background Students´ assessment should be carried out in an effective and objective manner, which reduces the possibility of different evaluators giving different scores, thus influencing the qualification obtained and the consistency of education. The aim of the present study was to determine the a...

Descripción completa

Detalles Bibliográficos
Autores: ESCRIBANO, NURIA, BELLIARD, VIRGINIA, BARACCO, BRUNO, DA SILVA, DAYANA, CEBALLOS, LAURA, FUENTES, M VICTORIA
Tipo de recurso: artículo
Fecha de publicación:2023
País:España
Institución:Universidad Rey Juan Carlos
Repositorio:BURJC-Digital. Repositorio Institucional de la Universidad Rey Juan Carlos
OAI Identifier:oai:burjcdigital.urjc.es:10115/28207
Acceso en línea:https://hdl.handle.net/10115/28207
Access Level:acceso abierto
Palabra clave:Dental education
Rubric
Endodontics training
Portfolio evaluation
Evaluators´ agreement
id ES_ab2ce36e2f42fd114ceb2fa7d04f551a
oai_identifier_str oai:burjcdigital.urjc.es:10115/28207
network_acronym_str ES
network_name_str España
repository_id_str
spelling Rubric vs. numeric rating scale: agreement among evaluators on endodontic treatments performed by dental studentsESCRIBANO, NURIABELLIARD, VIRGINIABARACCO, BRUNODA SILVA, DAYANACEBALLOS, LAURAFUENTES, M VICTORIADental educationRubricEndodontics trainingPortfolio evaluationEvaluators´ agreementBackground Students´ assessment should be carried out in an effective and objective manner, which reduces the possibility of different evaluators giving different scores, thus influencing the qualification obtained and the consistency of education. The aim of the present study was to determine the agreement among four evaluators and compare the overall scores awarded when assessing portfolios of endodontic preclinical treatments performed by dental students by using an analytic rubric and a numeric rating scale. Methods A random sample of 42 portfolios performed by fourth-year dental students at preclinical endodontic practices were blindly assessed by four evaluators using two different evaluation methods: an analytic rubric specifically designed and a numeric rating scale. Six categories were analyzed: radiographic assessment, access preparation, shaping procedure, obturation, content of the portfolio, and presentation of the portfolio. The maximum global score was 10 points. The overall scores obtained with both methods from each evaluator were compared by Student’s t, while agreement among evaluators was measured by Intraclass correlation coefficients (ICC). The influence of the difficulty of the endodontic treatment on the evaluators´ scores was analyzed by one-way ANOVA. Statistical tests were performed at a pre-set alpha of 0.05 using Stata 16. Results Difficulty of canal treatment did not influence the scores of evaluators, irrespective of the evaluation method used. When the analytic rubric was used, inter-evaluator agreement was substantial for radiographic assessment, access preparation, shaping procedure, obturation, and overall scores. Inter-evaluator agreement ranged from moderate to fair with the numeric rating scale. Mean higher overall scores were achieved when numeric rating scale was used. Presentation and content of the portfolio showed slight and fair agreement, respectively, among evaluators, regardless the evaluation method applied. Conclusions Assessment guided by an analytic rubric allowed evaluators to reach higher levels of agreement than those obtained when using a numeric rating scale. However, the rubric negatively affected overall scores.BMC202420242023info:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/10115/28207reponame:BURJC-Digital. Repositorio Institucional de la Universidad Rey Juan Carlosinstname:Universidad Rey Juan CarlosInglésAtribución 4.0 Internacionalhttp://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccessoai:burjcdigital.urjc.es:10115/282072026-06-24T12:48:17Z
dc.title.none.fl_str_mv Rubric vs. numeric rating scale: agreement among evaluators on endodontic treatments performed by dental students
title Rubric vs. numeric rating scale: agreement among evaluators on endodontic treatments performed by dental students
spellingShingle Rubric vs. numeric rating scale: agreement among evaluators on endodontic treatments performed by dental students
ESCRIBANO, NURIA
Dental education
Rubric
Endodontics training
Portfolio evaluation
Evaluators´ agreement
title_short Rubric vs. numeric rating scale: agreement among evaluators on endodontic treatments performed by dental students
title_full Rubric vs. numeric rating scale: agreement among evaluators on endodontic treatments performed by dental students
title_fullStr Rubric vs. numeric rating scale: agreement among evaluators on endodontic treatments performed by dental students
title_full_unstemmed Rubric vs. numeric rating scale: agreement among evaluators on endodontic treatments performed by dental students
title_sort Rubric vs. numeric rating scale: agreement among evaluators on endodontic treatments performed by dental students
dc.creator.none.fl_str_mv ESCRIBANO, NURIA
BELLIARD, VIRGINIA
BARACCO, BRUNO
DA SILVA, DAYANA
CEBALLOS, LAURA
FUENTES, M VICTORIA
author ESCRIBANO, NURIA
author_facet ESCRIBANO, NURIA
BELLIARD, VIRGINIA
BARACCO, BRUNO
DA SILVA, DAYANA
CEBALLOS, LAURA
FUENTES, M VICTORIA
author_role author
author2 BELLIARD, VIRGINIA
BARACCO, BRUNO
DA SILVA, DAYANA
CEBALLOS, LAURA
FUENTES, M VICTORIA
author2_role author
author
author
author
author
dc.subject.none.fl_str_mv Dental education
Rubric
Endodontics training
Portfolio evaluation
Evaluators´ agreement
topic Dental education
Rubric
Endodontics training
Portfolio evaluation
Evaluators´ agreement
description Background Students´ assessment should be carried out in an effective and objective manner, which reduces the possibility of different evaluators giving different scores, thus influencing the qualification obtained and the consistency of education. The aim of the present study was to determine the agreement among four evaluators and compare the overall scores awarded when assessing portfolios of endodontic preclinical treatments performed by dental students by using an analytic rubric and a numeric rating scale. Methods A random sample of 42 portfolios performed by fourth-year dental students at preclinical endodontic practices were blindly assessed by four evaluators using two different evaluation methods: an analytic rubric specifically designed and a numeric rating scale. Six categories were analyzed: radiographic assessment, access preparation, shaping procedure, obturation, content of the portfolio, and presentation of the portfolio. The maximum global score was 10 points. The overall scores obtained with both methods from each evaluator were compared by Student’s t, while agreement among evaluators was measured by Intraclass correlation coefficients (ICC). The influence of the difficulty of the endodontic treatment on the evaluators´ scores was analyzed by one-way ANOVA. Statistical tests were performed at a pre-set alpha of 0.05 using Stata 16. Results Difficulty of canal treatment did not influence the scores of evaluators, irrespective of the evaluation method used. When the analytic rubric was used, inter-evaluator agreement was substantial for radiographic assessment, access preparation, shaping procedure, obturation, and overall scores. Inter-evaluator agreement ranged from moderate to fair with the numeric rating scale. Mean higher overall scores were achieved when numeric rating scale was used. Presentation and content of the portfolio showed slight and fair agreement, respectively, among evaluators, regardless the evaluation method applied. Conclusions Assessment guided by an analytic rubric allowed evaluators to reach higher levels of agreement than those obtained when using a numeric rating scale. However, the rubric negatively affected overall scores.
publishDate 2023
dc.date.none.fl_str_mv 2023
2024
2024
dc.type.none.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/10115/28207
url https://hdl.handle.net/10115/28207
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.rights.none.fl_str_mv Atribución 4.0 Internacional
http://creativecommons.org/licenses/by/4.0/
info:eu-repo/semantics/openAccess
rights_invalid_str_mv Atribución 4.0 Internacional
http://creativecommons.org/licenses/by/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv BMC
publisher.none.fl_str_mv BMC
dc.source.none.fl_str_mv reponame:BURJC-Digital. Repositorio Institucional de la Universidad Rey Juan Carlos
instname:Universidad Rey Juan Carlos
instname_str Universidad Rey Juan Carlos
reponame_str BURJC-Digital. Repositorio Institucional de la Universidad Rey Juan Carlos
collection BURJC-Digital. Repositorio Institucional de la Universidad Rey Juan Carlos
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869416245987639296
score 15.811543

Similares en LA Referencia