LLMs outperform outsourced human coders on complex textual analysis
This paper evaluates the effectiveness of large language models (LLMs) in extracting complex information from text data. Using a corpus of Spanish news articles, we compare how accurately various LLMs and outsourced human coders reproduce expert annotations on five natural language processing tasks,...
| Autores: | , , , |
|---|---|
| Tipo de documento: | artigo |
| Data de publicação: | 2025 |
| País: | España |
| Recursos: | Universitat Ramon Llull (URL) |
| Repositório: | DAU Arxiu Digital de la Universitat Ramon Llull |
| OAI Identifier: | oai:dau.url.edu:20.500.14342/6013 |
| Acesso em linha: | http://hdl.handle.net/20.500.14342/6013 https://doi.org/10.1038/s41598-025-23798-y |
| Access Level: | Acceso aberto |
| Palavra-chave: | Data Mining Natural Language Processing |
| Resumo: | This paper evaluates the effectiveness of large language models (LLMs) in extracting complex information from text data. Using a corpus of Spanish news articles, we compare how accurately various LLMs and outsourced human coders reproduce expert annotations on five natural language processing tasks, ranging from named entity recognition to identifying nuanced political criticism in news articles. We find that LLMs consistently outperform outsourced human coders, particularly in tasks requiring deep contextual understanding. These findings suggest that current LLM technology offers researchers without programming expertise a cost-effective alternative for sophisticated text analysis. |
|---|