Classifying illicit dark web content through zero-shot prompting: An empirical study with GPT models
This study evaluates the classification performance of four GPT-based models (GPT-4.1, GPT-4.1-mini, GPT-4.1-nano, and o4-mini) under zero-shot prompting conditions on the complete, multilingual CoDA dataset of Dark Web content, comprising 10 illicit activity categories. The models GPT-4.1, GPT-4.1-...
| Autores: | , , , , |
|---|---|
| Tipo de recurso: | artículo |
| Fecha de publicación: | 2025 |
| País: | España |
| Institución: | Universidad de Alcalá (UAH) |
| Repositorio: | e_Buah Biblioteca Digital Universidad de Alcalá |
| Idioma: | inglés |
| OAI Identifier: | oai:ebuah.uah.es:10017/67901 |
| Acceso en línea: | http://hdl.handle.net/10017/67901 https://dx.doi.org/10.1016/j.ipm.2025.104476 |
| Access Level: | acceso abierto |
| Palabra clave: | Zero-shot GPT Text classification Illicit activities Informática Computer science |
| Sumario: | This study evaluates the classification performance of four GPT-based models (GPT-4.1, GPT-4.1-mini, GPT-4.1-nano, and o4-mini) under zero-shot prompting conditions on the complete, multilingual CoDA dataset of Dark Web content, comprising 10 illicit activity categories. The models GPT-4.1, GPT-4.1-mini, and o4-mini achieve a weighted F1 score of 0.885, surpassing prior zero-shot baselines on this dataset. Stability analysis using TARa@10 demonstrates high output consistency for GPT-4.1 (0.964) and GPT-4.1-mini (0.970), indicating their reliability for operational use. Multilingual evaluation reveals only a modest English vs. non-English performance gap for GPT-4.1 (0.031), while other models perform comparably across languages. The strongest results appear in Drugs, Gambling, and Porn (F1 > 0.9), whereas lower scores are observed in ambiguous or overlapping categories like Violence (F1 <= 0.76) or Crypto (F1 <= 0.84). A qualitative review of misclassifications suggests that some model predictions align with reasonable semantic interpretations, potentially highlighting annotation inconsistencies. This work establishes a performance baseline for GPT-based models in zero-shot classification of multilingual Dark Web content and underscores the importance of clear category definitions for effective deployment. |
|---|