Application of large language models in clinical record correction : a comprehensive study on various retraining methods
Objectives: We evaluate the effectiveness of large language models (LLMs), specifically GPT-based (GPT-3.5 and GPT-4) and Llama-2 models (13B and 7B architectures), in autonomously assessing clinical records (CRs) to enhance medical education and diagnostic skills. Materials and Methods: Various tec...
| Autores: | , , , , , , , , , , , , , |
|---|---|
| Tipo de recurso: | artículo |
| Fecha de publicación: | 2025 |
| País: | España |
| Institución: | Universidad Francisco de Vitoria |
| Repositorio: | DDFV. Repositorio Institucional de la Universidad Francisco de Vitoria |
| Idioma: | inglés |
| OAI Identifier: | oai:ddfv.ufv.es:10641/7515 |
| Acceso en línea: | https://hdl.handle.net/10641/7515 |
| Access Level: | acceso abierto |
| Palabra clave: | LLMs artificial intelligence clinical records retraining Health Informatics Journal Article Research Support, Non-U.S. Gov't Yes yes |
| id |
ES_be12647d5fd2de84b09f7b1f8a8fd88a |
|---|---|
| oai_identifier_str |
oai:ddfv.ufv.es:10641/7515 |
| network_acronym_str |
ES |
| network_name_str |
España |
| repository_id_str |
|
| spelling |
Application of large language models in clinical record correction : a comprehensive study on various retraining methodsMaitin, Ana M.Maitín, Ana MaríaNogales, AlbertoFernández-Rincón, SergioNogales Moyano, AlbertoAranguren, EnriqueCervera-Barba, EmilioDenizon-Arranz, SophiaMateos-Rodríguez, AlonsoCervera Barba, Emilio JuanGarcía-Tejedor, Álvaro J.Denizon Arranz, SophiaMateos-Rodríguez, Alonso A.García Tejedor, Álvaro JoséLLMsartificial intelligenceclinical recordsretrainingHealth InformaticsJournal ArticleResearch Support, Non-U.S. Gov'tYesyesObjectives: We evaluate the effectiveness of large language models (LLMs), specifically GPT-based (GPT-3.5 and GPT-4) and Llama-2 models (13B and 7B architectures), in autonomously assessing clinical records (CRs) to enhance medical education and diagnostic skills. Materials and Methods: Various techniques, including prompt engineering, fine-tuning (FT), and low-rank adaptation (LoRA), were implemented and compared on Llama-2 7B. These methods were assessed using prompts in both English and Spanish to determine their adaptability to different languages. Performance was benchmarked against GPT-3.5, GPT-4, and Llama-2 13B. Results: GPT-based models, particularly GPT-4, demonstrated promising performance closely aligned with specialist evaluations. Application of FT on Llama-2 7B improved text comprehension in Spanish, equating its performance to that of Llama-2 13B with English prompts. Low-rank adaptation significantly enhanced performance, surpassing GPT-3.5 results when combined with FT. This indicates LoRA’s effectiveness in adapting open-source models for specific tasks. Discussion. While GPT-4 showed superior performance, FT and LoRA on Llama-2 7B proved crucial in improving language comprehension and task-specific accuracy. Identified limitations highlight the need for further research. Conclusion: This study underscores the potential of LLMs in medical education, providing an innovative, effective approach to CR correction. Low-rank adaptation emerged as the most effective technique, enabling open-source models to perform on par with proprietary models. Future research should focus on overcoming current limitations to further improve model performance.Centro de Innovación Experimental del Conocimiento (CEIEC)Universidad Francisco de VitoriaFacultad de Medicina20252025-02-0120252025-02-01journal articlehttp://purl.org/coar/resource_type/c_6501info:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/10641/7515reponame:DDFV. Repositorio Institucional de la Universidad Francisco de Vitoriainstname:Universidad Francisco de VitoriaInglésengopen accesshttp://purl.org/coar/access_right/c_abf2http://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccessoai:ddfv.ufv.es:10641/75152026-06-11T12:44:57Z |
| dc.title.none.fl_str_mv |
Application of large language models in clinical record correction : a comprehensive study on various retraining methods |
| title |
Application of large language models in clinical record correction : a comprehensive study on various retraining methods |
| spellingShingle |
Application of large language models in clinical record correction : a comprehensive study on various retraining methods Maitin, Ana M. LLMs artificial intelligence clinical records retraining Health Informatics Journal Article Research Support, Non-U.S. Gov't Yes yes |
| title_short |
Application of large language models in clinical record correction : a comprehensive study on various retraining methods |
| title_full |
Application of large language models in clinical record correction : a comprehensive study on various retraining methods |
| title_fullStr |
Application of large language models in clinical record correction : a comprehensive study on various retraining methods |
| title_full_unstemmed |
Application of large language models in clinical record correction : a comprehensive study on various retraining methods |
| title_sort |
Application of large language models in clinical record correction : a comprehensive study on various retraining methods |
| dc.creator.none.fl_str_mv |
Maitin, Ana M. Maitín, Ana María Nogales, Alberto Fernández-Rincón, Sergio Nogales Moyano, Alberto Aranguren, Enrique Cervera-Barba, Emilio Denizon-Arranz, Sophia Mateos-Rodríguez, Alonso Cervera Barba, Emilio Juan García-Tejedor, Álvaro J. Denizon Arranz, Sophia Mateos-Rodríguez, Alonso A. García Tejedor, Álvaro José |
| author |
Maitin, Ana M. |
| author_facet |
Maitin, Ana M. Maitín, Ana María Nogales, Alberto Fernández-Rincón, Sergio Nogales Moyano, Alberto Aranguren, Enrique Cervera-Barba, Emilio Denizon-Arranz, Sophia Mateos-Rodríguez, Alonso Cervera Barba, Emilio Juan García-Tejedor, Álvaro J. Denizon Arranz, Sophia Mateos-Rodríguez, Alonso A. García Tejedor, Álvaro José |
| author_role |
author |
| author2 |
Maitín, Ana María Nogales, Alberto Fernández-Rincón, Sergio Nogales Moyano, Alberto Aranguren, Enrique Cervera-Barba, Emilio Denizon-Arranz, Sophia Mateos-Rodríguez, Alonso Cervera Barba, Emilio Juan García-Tejedor, Álvaro J. Denizon Arranz, Sophia Mateos-Rodríguez, Alonso A. García Tejedor, Álvaro José |
| author2_role |
author author author author author author author author author author author author author |
| dc.contributor.none.fl_str_mv |
Centro de Innovación Experimental del Conocimiento (CEIEC) Universidad Francisco de Vitoria Facultad de Medicina |
| dc.subject.none.fl_str_mv |
LLMs artificial intelligence clinical records retraining Health Informatics Journal Article Research Support, Non-U.S. Gov't Yes yes |
| topic |
LLMs artificial intelligence clinical records retraining Health Informatics Journal Article Research Support, Non-U.S. Gov't Yes yes |
| description |
Objectives: We evaluate the effectiveness of large language models (LLMs), specifically GPT-based (GPT-3.5 and GPT-4) and Llama-2 models (13B and 7B architectures), in autonomously assessing clinical records (CRs) to enhance medical education and diagnostic skills. Materials and Methods: Various techniques, including prompt engineering, fine-tuning (FT), and low-rank adaptation (LoRA), were implemented and compared on Llama-2 7B. These methods were assessed using prompts in both English and Spanish to determine their adaptability to different languages. Performance was benchmarked against GPT-3.5, GPT-4, and Llama-2 13B. Results: GPT-based models, particularly GPT-4, demonstrated promising performance closely aligned with specialist evaluations. Application of FT on Llama-2 7B improved text comprehension in Spanish, equating its performance to that of Llama-2 13B with English prompts. Low-rank adaptation significantly enhanced performance, surpassing GPT-3.5 results when combined with FT. This indicates LoRA’s effectiveness in adapting open-source models for specific tasks. Discussion. While GPT-4 showed superior performance, FT and LoRA on Llama-2 7B proved crucial in improving language comprehension and task-specific accuracy. Identified limitations highlight the need for further research. Conclusion: This study underscores the potential of LLMs in medical education, providing an innovative, effective approach to CR correction. Low-rank adaptation emerged as the most effective technique, enabling open-source models to perform on par with proprietary models. Future research should focus on overcoming current limitations to further improve model performance. |
| publishDate |
2025 |
| dc.date.none.fl_str_mv |
2025 2025-02-01 2025 2025-02-01 |
| dc.type.none.fl_str_mv |
journal article http://purl.org/coar/resource_type/c_6501 |
| dc.type.openaire.fl_str_mv |
info:eu-repo/semantics/article |
| format |
article |
| dc.identifier.none.fl_str_mv |
https://hdl.handle.net/10641/7515 |
| url |
https://hdl.handle.net/10641/7515 |
| dc.language.none.fl_str_mv |
Inglés eng |
| language_invalid_str_mv |
Inglés |
| language |
eng |
| dc.rights.none.fl_str_mv |
open access http://purl.org/coar/access_right/c_abf2 http://creativecommons.org/licenses/by-nc-nd/4.0/ |
| dc.rights.openaire.fl_str_mv |
info:eu-repo/semantics/openAccess |
| rights_invalid_str_mv |
open access http://purl.org/coar/access_right/c_abf2 http://creativecommons.org/licenses/by-nc-nd/4.0/ |
| eu_rights_str_mv |
openAccess |
| dc.format.none.fl_str_mv |
application/pdf |
| dc.source.none.fl_str_mv |
reponame:DDFV. Repositorio Institucional de la Universidad Francisco de Vitoria instname:Universidad Francisco de Vitoria |
| instname_str |
Universidad Francisco de Vitoria |
| reponame_str |
DDFV. Repositorio Institucional de la Universidad Francisco de Vitoria |
| collection |
DDFV. Repositorio Institucional de la Universidad Francisco de Vitoria |
| repository.name.fl_str_mv |
|
| repository.mail.fl_str_mv |
|
| _version_ |
1869418255077081088 |
| score |
15,811543 |