Exploring morphology-aware tokenization: a case study on Spanish language modeling
This paper investigates to what extent the integration of morphological information can improve subword tokenization and thus also language modeling performance. We focus on Spanish, a language with fusional morphology, where subword segmentation can benefit from linguistic structure. Instead of rel...
| Autores: | , , |
|---|---|
| Tipo de recurso: | capítulo de libro |
| Estado: | Versión publicada |
| Fecha de publicación: | 2025 |
| País: | España |
| Institución: | Varias* (Consorci de Biblioteques Universitáries de Catalunya, Centre de Serveis Científics i Acadèmics de Catalunya) |
| Repositorio: | Recercat. Dipósit de la Recerca de Catalunya |
| OAI Identifier: | oai:recercat.cat:10230/72720 |
| Acceso en línea: | https://hdl.handle.net/10230/72720 http://dx.doi.org/10.18653/v1/2025.emnlp-main.1552 |
| Access Level: | acceso abierto |
| Palabra clave: | Morphology-aware tokenization Spanish language modeling |
| Sumario: | This paper investigates to what extent the integration of morphological information can improve subword tokenization and thus also language modeling performance. We focus on Spanish, a language with fusional morphology, where subword segmentation can benefit from linguistic structure. Instead of relying on purely data-driven strategies like Byte Pair Encoding (BPE), we explore a linguistically grounded approach: training a tokenizer on morphologically segmented data. To do so, we develop a semi-supervised segmentation model for Spanish, building gold-standard datasets to guide and evaluate it. We then use this tokenizer to pre-train a masked language model and assess its performance on several downstream tasks. Our results show improvements over a baseline with a standard tokenizer, supporting our hypothesis that morphology-aware tokenization offers a viable and principled alternative for improving language modeling. |
|---|