BigOPERA: An OPportunistic and Elastic Resource Allocation for Big Data Frameworks
The rapid growth of data-intensive applications has driven the need for more scalable, flexible, and sustainable resource management in Big Data (BD) frameworks. Traditional computing infrastructures often rely exclusively on dedicated resources, which can lead to inefficient utilization and increas...
| Autor: | |
|---|---|
| Formato: | tesis doctoral |
| Fecha de publicación: | 2025 |
| País: | España |
| Recursos: | Universidad de Santiago de Compostela (USC) |
| Repositorio: | Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela |
| Idioma: | inglés |
| OAI Identifier: | oai:minerva.usc.gal:10347/42898 |
| Acesso em linha: | https://hdl.handle.net/10347/42898 |
| Access Level: | acceso abierto |
| Palavra-chave: | Big Data Apache Spark Opportunistic Computing Green Computing 120311 Logicales de ordenadores 120313 Calculo digital |
| Resumo: | The rapid growth of data-intensive applications has driven the need for more scalable, flexible, and sustainable resource management in Big Data (BD) frameworks. Traditional computing infrastructures often rely exclusively on dedicated resources, which can lead to inefficient utilization and increased operational costs. To address this challenge, this thesis explores the integration of opportunistic computing into Apache Spark through a hybrid resource allocation framework named BigOPERA. BigOPERA combines the elasticity of opportunistic nodes, machines not primarily dedicated to the cluster, with the stability of dedicated infrastructure, achieving cost-aware scalability without compromising performance. The system architecture integrates Apache Spark in standalone mode for primary data processing, Docker for containerized task isolation, and HTCondor as the orchestrator for opportunistic resource provisioning. |
|---|