Optimising retrieval performance in RAG systems: A new growing window semantic chunking strategy to address weak semantic boundaries
The release of ChatGPT in November 2022 signified a pivotal shift within the domain of Natural Language Processing, particularly in the context of chatbot technology. Since then, chatbots powered by Large Language Models have been widely adopted, demonstrating remarkable capabilities across a wide r...
| Authors: | , , , |
|---|---|
| Format: | article |
| Publication Date: | 2026 |
| Country: | España |
| Institution: | Universidad de Alcalá (UAH) |
| Repository: | e_Buah Biblioteca Digital Universidad de Alcalá |
| Language: | English |
| OAI Identifier: | oai:dnet:ebuahbibliot::aff6b05122c18c741ee8bcb821e52b73 |
| Online Access: | http://hdl.handle.net/10017/69046 https://dx.doi.org/10.1016/j.knosys.2025.114896 |
| Access Level: | Open access |
| Keyword: | Artificial intelligence NLP RAG Semantic splitting Sentence textual similarity Chunking Informática Computer science |
| Summary: | The release of ChatGPT in November 2022 signified a pivotal shift within the domain of Natural Language Processing, particularly in the context of chatbot technology. Since then, chatbots powered by Large Language Models have been widely adopted, demonstrating remarkable capabilities across a wide range of tasks. In light of the substantial impact of this technology, the study of its application in specific domains has emerged as a crucial area of research. However, given that these chatbots are based on Large Language Models, they are subject to the well-known limitations of this technology when handling with knowledge-intensive tasks, and are prone to generate hallucinated answers. The RAG (Retrieval-Augmented Generation) architecture, which comprises an indexing phase, was developed to address this issue. During the indexing phase, text should be split into different chunks using a chunking technique. We thoroughly analyse state-of-the-art text chunking techniques used in RAG pipelines and propose a novel semantic text chunking technique. To evaluate the effectiveness of our technique within RAG pipelines, an exhaustive evaluation framework has been defined and applied. This evaluation also enables the comparison with previous techniques, resulting in a noticeable improvement in respect to state-of-the-art strategies. |
|---|