Promoting generalized cross-lingual question answering in few-resource scenarios via self-knowledge distillation
We address the challenge of Generalized Cross-Lingual Transfer (G-XLT) in extractive Question Answering, where question and context languages differ, a problem particularly difficult for low-resource languages. Working with only a thousand parallel QA samples, we combine cross-lingual sampling with...
| Autores: | , , , |
|---|---|
| Tipo de recurso: | artículo |
| Fecha de publicación: | 2025 |
| País: | España |
| Institución: | Universitat Politècnica de Catalunya (UPC) |
| Repositorio: | UPCommons. Portal del coneixement obert de la UPC |
| Idioma: | inglés |
| OAI Identifier: | oai:upcommons.upc.edu:2117/449061 |
| Acceso en línea: | https://hdl.handle.net/2117/449061 https://dx.doi.org/10.26342/2025-75-5 |
| Access Level: | acceso abierto |
| Palabra clave: | Extractive question answering Cross-lingual transfer Knowledge distillation Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Llenguatge natural |
| Sumario: | We address the challenge of Generalized Cross-Lingual Transfer (G-XLT) in extractive Question Answering, where question and context languages differ, a problem particularly difficult for low-resource languages. Working with only a thousand parallel QA samples, we combine cross-lingual sampling with self-knowledge distillation to regularize cross-lingual fine-tuning. We introduce the novel mean Average Precision at k (mAP@k) coefficient, which mitigates the negative impact of incorrect predictions during training and serves as a diagnostic tool providing early training guidance and reliable indicators of model learning. Evaluations on MLQA, XQuAD, and TyDiQA-GoldP datasets demonstrate that our approach consistently outperforms standard cross-entropy fine-tuning of the mBERT multilingual model. Our method represents a promising alternative to machine translation-based approaches, particularly valuable for low-resource languages where translation quality is poor, offering an efficient solution for cross-lingual transfer in data-scarce settings. |
|---|