Support for managing dynamically Hadoop clusters
Final master project developed in Barcelona Supercomputing Centre. This project focuses on the virtualization of Hadoop environments and the design of a piece of software for automatizing the configuration of the shared resources for Hadoop environments. In addition, this project uses a modified int...
| Autor: | |
|---|---|
| Tipo de recurso: | tesis de maestría |
| Fecha de publicación: | 2010 |
| País: | España |
| Institución: | Universitat Politècnica de Catalunya (UPC) |
| Repositorio: | UPCommons. Portal del coneixement obert de la UPC |
| Idioma: | inglés |
| OAI Identifier: | oai:upcommons.upc.edu:2099.1/9920 |
| Acceso en línea: | https://hdl.handle.net/2099.1/9920 |
| Access Level: | acceso abierto |
| Palabra clave: | Cluster analysis Hadoop EMOTIVE MapReduce Virtualizatio Anàlisi de conglomerats Àrees temàtiques de la UPC::Informàtica::Programació |
| Sumario: | Final master project developed in Barcelona Supercomputing Centre. This project focuses on the virtualization of Hadoop environments and the design of a piece of software for automatizing the configuration of the shared resources for Hadoop environments. In addition, this project uses a modified internal Hadoop scheduler that adapts the available resources in the cluster for running jobs according to time restrictions. With the adapted internal scheduler and the virtualization capabilities the software developed in this project provides a flexible and self-adaptive service for Hadoop enviroments. In this report I introduce the context of the technology, the specifications of the system, the design of the solution integrated with EMOTIVE Cloud platform and the testing for showing the performance of the product. |
|---|