Distributed scheduling and data sharing in late-binding overlays
Pull-based late-binding overlays are used in some of today’s largest computational grids. Job agents are submitted to resources with the duty of retrieving real workload from a central queue at runtime. This helps overcome the problems of these very complex environments, namely, heterogeneity, impre...
| Autores: | , , |
|---|---|
| Tipo de recurso: | artículo |
| Fecha de publicación: | 2014 |
| País: | España |
| Institución: | Universidad Complutense de Madrid (UCM) |
| Repositorio: | Docta Complutense |
| Idioma: | inglés |
| OAI Identifier: | oai:docta.ucm.es:20.500.14352/35470 |
| Acceso en línea: | https://hdl.handle.net/20.500.14352/35470 |
| Access Level: | acceso abierto |
| Palabra clave: | 004.42 004.7 Grid and Cluster Computing Scalable Computing Peer-to-Peer Architectures and Networks Reliable Parallel and Distributed Algorithms Programación de ordenadores (Informática) Redes 1203.23 Lenguajes de Programación |
| Sumario: | Pull-based late-binding overlays are used in some of today’s largest computational grids. Job agents are submitted to resources with the duty of retrieving real workload from a central queue at runtime. This helps overcome the problems of these very complex environments, namely, heterogeneity, imprecise status information and relatively high failure rates. In addition, the late job assignment allows dynamic adaptation to changes in the grid conditions or user priorities. However, as the scale grows, the central assignment queue may become a bottleneck for the whole system. This article presents a distributed scheduling architecture for late-binding overlays, which addresses these scalability issues. Our system lets execution nodes build a distributed hash table and delegates job matching and assignment to them. This reduces the load on the central server and makes the system much more scalable and robust. Moreover, scalability makes fine-grained scheduling possible, and enables new functionalities like the implementation of a distributed data cache on the execution nodes, which helps alleviate the commonly congested grid storage services. |
|---|