DMRlib: Easy-coding and efficient resource management for job malleability

Process malleability has proved to have a highly positive impact on the resource utilization and global productivity in data centers compared with the conventional static resource allocation policy. However, the non-negligible additional development effort this solution imposes has constrained its a...

Descripción completa

Detalles Bibliográficos
Autores: Iserte, Sergio, Mayo, Rafael, Quintana Ortí, Enrique Salvador, Peña, Antonio|||0000-0002-3575-4617
Tipo de recurso: artículo
Fecha de publicación:2020
País:España
Institución:Universitat Politècnica de Catalunya (UPC)
Repositorio:UPCommons. Portal del coneixement obert de la UPC
Idioma:inglés
OAI Identifier:oai:upcommons.upc.edu:2117/329704
Acceso en línea:https://hdl.handle.net/2117/329704
https://dx.doi.org/10.1109/TC.2020.3022933
Access Level:acceso abierto
Palabra clave:High performance computing
Data centers
Processes Reconfiguration
MPI malleability
Job Elastic Resize
Dynamic Reallocation of Resources
Productivity-Aware Computation
Càlcul intensiu (Informàtica)
Àrees temàtiques de la UPC::Informàtica::Enginyeria del software
id ES_8d874d3e83aef49b57235bb6e42e40eb
oai_identifier_str oai:upcommons.upc.edu:2117/329704
network_acronym_str ES
network_name_str España
repository_id_str
spelling DMRlib: Easy-coding and efficient resource management for job malleabilityIserte, SergioMayo, RafaelQuintana Ortí, Enrique SalvadorPeña, Antonio|||0000-0002-3575-4617High performance computingData centersProcesses ReconfigurationMPI malleabilityJob Elastic ResizeDynamic Reallocation of ResourcesProductivity-Aware ComputationCàlcul intensiu (Informàtica)Àrees temàtiques de la UPC::Informàtica::Enginyeria del softwareProcess malleability has proved to have a highly positive impact on the resource utilization and global productivity in data centers compared with the conventional static resource allocation policy. However, the non-negligible additional development effort this solution imposes has constrained its adoption by the scientific programming community. In this work, we present DMRlib, a library designed to offer the global advantages of process malleability while providing a minimalist MPI-like syntax. The library includes a series of predefined communication patterns that greatly ease the development of malleable applications. In addition, we deploy several scenarios to demonstrate the positive impact of process malleability featuring different scalability patterns. Concretely, we study two job submission modes (rigid and moldable) in order to identify the best-case scenarios for malleability using metrics such as resource allocation rate, completed jobs per second, and energy consumption. The experiments prove that our elastic approach may improve global throughput by a factor higher than 3x compared to the traditional workloads of non-malleable jobs.This work was supported by projects TIN2014-53495-R, TIN2015-65316-P, and TIN2017-82972-R from MINECO and FEDER. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska Curie grant agreement No. 749516. Sergio Iserte was supported by a postdoctoral fellowship from Generalitat Valenciana and European Social Fund APOSTD/2020/026. Finally, the authors want to thank the anonymous reviewers whose suggestions significantly improved the quality of this manuscript.Peer ReviewedInstitute of Electrical and Electronics Engineers (IEEE)20202020-01-0120202020-10-02journal articlehttp://purl.org/coar/resource_type/c_6501AMhttp://purl.org/coar/version/c_ab4af688f83e57aainfo:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/2117/329704https://dx.doi.org/10.1109/TC.2020.3022933reponame:UPCommons. Portal del coneixement obert de la UPCinstname:Universitat Politècnica de Catalunya (UPC)InglésengEuropean Commission http://doi.org/10.13039/100010661 Horizon 2020 Framework Programme 749516 Advanced Ecosystem for Broad Heterogeneous Memory Usageopen accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccessoai:upcommons.upc.edu:2117/3297042026-05-27T15:37:01Z
dc.title.none.fl_str_mv DMRlib: Easy-coding and efficient resource management for job malleability
title DMRlib: Easy-coding and efficient resource management for job malleability
spellingShingle DMRlib: Easy-coding and efficient resource management for job malleability
Iserte, Sergio
High performance computing
Data centers
Processes Reconfiguration
MPI malleability
Job Elastic Resize
Dynamic Reallocation of Resources
Productivity-Aware Computation
Càlcul intensiu (Informàtica)
Àrees temàtiques de la UPC::Informàtica::Enginyeria del software
title_short DMRlib: Easy-coding and efficient resource management for job malleability
title_full DMRlib: Easy-coding and efficient resource management for job malleability
title_fullStr DMRlib: Easy-coding and efficient resource management for job malleability
title_full_unstemmed DMRlib: Easy-coding and efficient resource management for job malleability
title_sort DMRlib: Easy-coding and efficient resource management for job malleability
dc.creator.none.fl_str_mv Iserte, Sergio
Mayo, Rafael
Quintana Ortí, Enrique Salvador
Peña, Antonio|||0000-0002-3575-4617
author Iserte, Sergio
author_facet Iserte, Sergio
Mayo, Rafael
Quintana Ortí, Enrique Salvador
Peña, Antonio|||0000-0002-3575-4617
author_role author
author2 Mayo, Rafael
Quintana Ortí, Enrique Salvador
Peña, Antonio|||0000-0002-3575-4617
author2_role author
author
author
dc.subject.none.fl_str_mv High performance computing
Data centers
Processes Reconfiguration
MPI malleability
Job Elastic Resize
Dynamic Reallocation of Resources
Productivity-Aware Computation
Càlcul intensiu (Informàtica)
Àrees temàtiques de la UPC::Informàtica::Enginyeria del software
topic High performance computing
Data centers
Processes Reconfiguration
MPI malleability
Job Elastic Resize
Dynamic Reallocation of Resources
Productivity-Aware Computation
Càlcul intensiu (Informàtica)
Àrees temàtiques de la UPC::Informàtica::Enginyeria del software
description Process malleability has proved to have a highly positive impact on the resource utilization and global productivity in data centers compared with the conventional static resource allocation policy. However, the non-negligible additional development effort this solution imposes has constrained its adoption by the scientific programming community. In this work, we present DMRlib, a library designed to offer the global advantages of process malleability while providing a minimalist MPI-like syntax. The library includes a series of predefined communication patterns that greatly ease the development of malleable applications. In addition, we deploy several scenarios to demonstrate the positive impact of process malleability featuring different scalability patterns. Concretely, we study two job submission modes (rigid and moldable) in order to identify the best-case scenarios for malleability using metrics such as resource allocation rate, completed jobs per second, and energy consumption. The experiments prove that our elastic approach may improve global throughput by a factor higher than 3x compared to the traditional workloads of non-malleable jobs.
publishDate 2020
dc.date.none.fl_str_mv 2020
2020-01-01
2020
2020-10-02
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
AM
http://purl.org/coar/version/c_ab4af688f83e57aa
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/2117/329704
https://dx.doi.org/10.1109/TC.2020.3022933
url https://hdl.handle.net/2117/329704
https://dx.doi.org/10.1109/TC.2020.3022933
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv European Commission http://doi.org/10.13039/100010661 Horizon 2020 Framework Programme 749516 Advanced Ecosystem for Broad Heterogeneous Memory Usage
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Institute of Electrical and Electronics Engineers (IEEE)
publisher.none.fl_str_mv Institute of Electrical and Electronics Engineers (IEEE)
dc.source.none.fl_str_mv reponame:UPCommons. Portal del coneixement obert de la UPC
instname:Universitat Politècnica de Catalunya (UPC)
instname_str Universitat Politècnica de Catalunya (UPC)
reponame_str UPCommons. Portal del coneixement obert de la UPC
collection UPCommons. Portal del coneixement obert de la UPC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869413053851762688
score 15.301603

Similares en LA Referencia