BigOPERA: An OPportunistic and Elastic Resource Allocation for big data frameworks
Efficient asset management is essential for optimizing the performance and scalability of modern Big Data (BD) frameworks. However, traditional resource allocation methods often suffer from static partitioning, inefficient resource utilization, and high operational costs, limiting their ability to a...
| Autores: | , , , |
|---|---|
| Tipo de recurso: | artículo |
| Fecha de publicación: | 2025 |
| País: | España |
| Institución: | Universidad de Santiago de Compostela (USC) |
| Repositorio: | Minerva. Repositorio Institucional de la Universidad de Santiago de Compostela |
| Idioma: | inglés |
| OAI Identifier: | oai:minerva.usc.gal:10347/42844 |
| Acceso en línea: | https://hdl.handle.net/10347/42844 |
| Access Level: | acceso abierto |
| Palabra clave: | DB Apache Spark Opportunistic scheduling Dynamic resource provisioning Resource allocation Elastic computing Green computing |
| Sumario: | Efficient asset management is essential for optimizing the performance and scalability of modern Big Data (BD) frameworks. However, traditional resource allocation methods often suffer from static partitioning, inefficient resource utilization, and high operational costs, limiting their ability to adapt to fluctuating workloads dynamically. This paper introduces BigOPERA, an opportunistic and elastic resource allocation framework designed to enhance BD processing environments by integrating dedicated and non-dedicated computing assets. Leveraging containerization and a two-tiered scheduling mechanism, BigOPERA dynamically manages available resources to improve workload execution efficiency. Experimental results demonstrate that BigOPERA achieves up to 35% performance improvement over native Apache Spark configurations, significantly enhancing computational throughput while optimizing resource consumption. Our findings highlight the potential of BigOPERA in scalable, cost-effective, and sustainable BD processing. |
|---|