Beyond grammar: can AI understand the nuances of academic writing?

On July 4, 2024, the free versions of ChatGPT, Gemini, and Claude were evaluated for their effectiveness in grading technical English assignments in an academic context. The study sought to determine if these AI platforms could serve as reliable grading tools by analyzing their responses to two sets...

Descripción completa

Detalles Bibliográficos
Autores: Montalbán Martínez, Nicolás, Montaner Villalba, Salvador
Tipo de recurso: artículo
Fecha de publicación:2026
País:España
Institución:Universidad de Alcalá (UAH)
Repositorio:e_Buah Biblioteca Digital Universidad de Alcalá
Idioma:español
OAI Identifier:oai:ebuah.uah.es:10017/68377
Acceso en línea:http://hdl.handle.net/10017/68377
Access Level:acceso abierto
Palabra clave:Instructional design
ESP
ICT
PBL
AI
Diseño didáctico
TIC
ABP
IA
Humanidades
Filología hispánica
Humanities
Spanish philology
id ES_eebf4aa5290891513b45b341af55d7d1
oai_identifier_str oai:ebuah.uah.es:10017/68377
network_acronym_str ES
network_name_str España
repository_id_str
spelling Beyond grammar: can AI understand the nuances of academic writing?Más allá de la gramática: ¿puede la IA comprender los matices de la escritura académica?Montalbán Martínez, NicolásMontaner Villalba, SalvadorInstructional designESPICTPBLAIDiseño didácticoTICABPIAHumanidadesFilología hispánicaHumanitiesSpanish philologyOn July 4, 2024, the free versions of ChatGPT, Gemini, and Claude were evaluated for their effectiveness in grading technical English assignments in an academic context. The study sought to determine if these AI platforms could serve as reliable grading tools by analyzing their responses to two sets of papers. The first set contained original student submissions written in technical English, while the second consisted of the same papers rewritten in grammatically correct but academically incoherent language. The AI models provided three grading suggestions for each assignment and were also asked to assign a final grade. These AI-generated grades were then compared with the grades given through traditional manual grading by educators. The analysis focused on the consistency of the AI's grading, its alignment with human assessments, and its ability to distinguish between technical precision and overall linguistic coherence. The findings contribute to ongoing discussions about the role of AI in education, particularly in the assessment of academic writing. This study highlights the potential benefits and limitations of using AI as a grading tool, emphasizing the need for further refinement before such systems can be reliably implemented in academic settings.El 4 de julio de 2024, las versiones gratuitas de ChatGPT, Gemini y Claude fueron evaluadas por su efectividad en la calificación de trabajos de inglés técnico en un contexto académico. El estudio buscó determinar si estas plataformas de IA podrían servir como herramientas de calificación fiables analizando sus respuestas a dos conjuntos de trabajos. El primer conjunto contenía entregas originales de estudiantes escritas en inglés técnico, mientras que el segundo consistía en los mismos trabajos reescritos en un lenguaje gramaticalmente correcto pero académicamente incoherente. Los modelos de IA proporcionaron tres sugerencias de calificación para cada trabajo y también se les pidió que asignaran una calificación final. Estas calificaciones generadas por IA fueron entonces comparadas con las calificaciones otorgadas mediante la calificación manual tradicional por educadores. El análisis se centró en la consistencia de la calificación de la IA, su alineación con las evaluaciones humanas y su capacidad para distinguir entre la precisión técnica y la coherencia lingüística global. Los hallazgos contribuyen a las discusiones en curso sobre el papel de la IA en la educación, particularmente en la evaluación de la escritura académica. Este estudio destaca los beneficios y limitaciones potenciales del uso de la IA como herramienta de calificación, enfatizando la necesidad de un refinamiento ulterior antes de que tales sistemas puedan ser implementados de manera fiable en entornos académicos.Servicio de Publicaciones de la Universidad de Alcalá20262026-01-30journal articlehttp://purl.org/coar/resource_type/c_6501NAhttp://purl.org/coar/version/c_be7fb7dd8ff6fe43info:eu-repo/semantics/articleapplication/pdfhttp://hdl.handle.net/10017/68377reponame:e_Buah Biblioteca Digital Universidad de Alcaláinstname:Universidad de Alcalá (UAH)Españolspaopen accesshttp://purl.org/coar/access_right/c_abf2Attribution-NonCommercial-NoDerivatives 4.0 Internationalhttp://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccessoai:ebuah.uah.es:10017/683772026-06-18T11:13:07Z
dc.title.none.fl_str_mv Beyond grammar: can AI understand the nuances of academic writing?
Más allá de la gramática: ¿puede la IA comprender los matices de la escritura académica?
title Beyond grammar: can AI understand the nuances of academic writing?
spellingShingle Beyond grammar: can AI understand the nuances of academic writing?
Montalbán Martínez, Nicolás
Instructional design
ESP
ICT
PBL
AI
Diseño didáctico
TIC
ABP
IA
Humanidades
Filología hispánica
Humanities
Spanish philology
title_short Beyond grammar: can AI understand the nuances of academic writing?
title_full Beyond grammar: can AI understand the nuances of academic writing?
title_fullStr Beyond grammar: can AI understand the nuances of academic writing?
title_full_unstemmed Beyond grammar: can AI understand the nuances of academic writing?
title_sort Beyond grammar: can AI understand the nuances of academic writing?
dc.creator.none.fl_str_mv Montalbán Martínez, Nicolás
Montaner Villalba, Salvador
author Montalbán Martínez, Nicolás
author_facet Montalbán Martínez, Nicolás
Montaner Villalba, Salvador
author_role author
author2 Montaner Villalba, Salvador
author2_role author
dc.subject.none.fl_str_mv Instructional design
ESP
ICT
PBL
AI
Diseño didáctico
TIC
ABP
IA
Humanidades
Filología hispánica
Humanities
Spanish philology
topic Instructional design
ESP
ICT
PBL
AI
Diseño didáctico
TIC
ABP
IA
Humanidades
Filología hispánica
Humanities
Spanish philology
description On July 4, 2024, the free versions of ChatGPT, Gemini, and Claude were evaluated for their effectiveness in grading technical English assignments in an academic context. The study sought to determine if these AI platforms could serve as reliable grading tools by analyzing their responses to two sets of papers. The first set contained original student submissions written in technical English, while the second consisted of the same papers rewritten in grammatically correct but academically incoherent language. The AI models provided three grading suggestions for each assignment and were also asked to assign a final grade. These AI-generated grades were then compared with the grades given through traditional manual grading by educators. The analysis focused on the consistency of the AI's grading, its alignment with human assessments, and its ability to distinguish between technical precision and overall linguistic coherence. The findings contribute to ongoing discussions about the role of AI in education, particularly in the assessment of academic writing. This study highlights the potential benefits and limitations of using AI as a grading tool, emphasizing the need for further refinement before such systems can be reliably implemented in academic settings.
publishDate 2026
dc.date.none.fl_str_mv 2026
2026-01-30
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
NA
http://purl.org/coar/version/c_be7fb7dd8ff6fe43
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv http://hdl.handle.net/10017/68377
url http://hdl.handle.net/10017/68377
dc.language.none.fl_str_mv Español
spa
language_invalid_str_mv Español
language spa
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Servicio de Publicaciones de la Universidad de Alcalá
publisher.none.fl_str_mv Servicio de Publicaciones de la Universidad de Alcalá
dc.source.none.fl_str_mv reponame:e_Buah Biblioteca Digital Universidad de Alcalá
instname:Universidad de Alcalá (UAH)
instname_str Universidad de Alcalá (UAH)
reponame_str e_Buah Biblioteca Digital Universidad de Alcalá
collection e_Buah Biblioteca Digital Universidad de Alcalá
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869423774510612480
score 15,81155

Similares en LA Referencia