Attacking misinformation detection using adversarial examples generated by language models
Large language models have many beneficial applications, but can they also be used to attack content-filtering algorithms in social media platforms? We investigate the challenge of generating adversarial examples to test the robustness of text classification algorithms detecting low-credibility cont...
| Autores: | , , |
|---|---|
| Tipo de recurso: | capítulo de libro |
| Estado: | Versión publicada |
| Fecha de publicación: | 2025 |
| País: | España |
| Institución: | Universitat Pompeu Fabra |
| Repositorio: | Repositorio Digital de la UPF |
| OAI Identifier: | oai:repositori.upf.edu:10230/72718 |
| Acceso en línea: | https://hdl.handle.net/10230/72718 http://dx.doi.org/10.18653/v1/2025.emnlp-main.1405 |
| Access Level: | acceso abierto |
| Palabra clave: | Misinformation detection Adversarial examples Language models |
| Sumario: | Large language models have many beneficial applications, but can they also be used to attack content-filtering algorithms in social media platforms? We investigate the challenge of generating adversarial examples to test the robustness of text classification algorithms detecting low-credibility content, including propaganda, false claims, rumours and hyperpartisan news. We focus on simulation of content moderation by setting realistic limits on the number of queries an attacker is allowed to attempt. Within our solution (TREPAT), initial rephrasings are generated by large language models with prompts inspired by meaning-preserving NLP tasks, such as text simplification and style transfer. Subsequently, these modifications are decomposed into small changes, applied through beam search procedure, until the victim classifier changes its decision. We perform quantitative evaluation using various prompts, models and query limits, targeted manual assessment of the generated text and qualitative linguistic analysis. The results confirm the superiority of our approach in the constrained scenario, especially in case of long input text (news articles), where exhaustive search is not feasible. |
|---|