Data stream treatment using sliding windows with MapReduce
Knowledge Discovery in Databases (KDD) techniques present limitations when the volume of data to process is very large. Any KDD algorithm needs to do several iterations on the complete set of data in order to carry out its work. For continuous data stream processing it is necessary to store part of...
| Autores: | , , |
|---|---|
| Tipo de recurso: | artículo |
| Estado: | Versión publicada |
| Fecha de publicación: | 2016 |
| País: | Argentina |
| Institución: | Consejo Nacional de Investigaciones Científicas y Técnicas |
| Repositorio: | CONICET Digital (CONICET) |
| Idioma: | español |
| OAI Identifier: | oai:ri.conicet.gov.ar:11336/115823 |
| Acceso en línea: | http://hdl.handle.net/11336/115823 |
| Access Level: | acceso abierto |
| Palabra clave: | BIG DATA MAPREDUCE STREAM PROCESSING https://purl.org/becyt/ford/1.2 https://purl.org/becyt/ford/1 |
| Sumario: | Knowledge Discovery in Databases (KDD) techniques present limitations when the volume of data to process is very large. Any KDD algorithm needs to do several iterations on the complete set of data in order to carry out its work. For continuous data stream processing it is necessary to store part of it in a temporal window.In this paper, we present a technique that uses the size of the temporal window in a dynamic way, based on the frequency of the data arrival and the response time of the KDD task. The obtained results show that this technique reaches a great size window where each example of the stream is used in more than one iteration of the KDD task. |
|---|