Accelerated execution via eager-release of dependencies in task-based workflows

Task-based programming models offer a flexible way to express the unstructured parallelism patterns of nowadays complex applications. This expressive capability is required to achieve maximum possible performance for applications that are executed in distributed execution platforms. In current task-...

Descripción completa

Detalles Bibliográficos
Autores: Elshazly, Hatem Mohamed Abdelfattah Eid|||0000-0002-8591-1502, Lordan Gomis, Francesc|||0000-0002-9845-8890, Ejarque Artigas, Jorge, Badia Sala, Rosa Maria|||0000-0003-2941-5499
Tipo de recurso: artículo
Fecha de publicación:2021
País:España
Institución:Universitat Politècnica de Catalunya (UPC)
Repositorio:UPCommons. Portal del coneixement obert de la UPC
Idioma:inglés
OAI Identifier:oai:upcommons.upc.edu:2117/341363
Acceso en línea:https://hdl.handle.net/2117/341363
https://dx.doi.org/10.1177/1094342021997558
Access Level:acceso abierto
Palabra clave:High performance computing
Parallel programming (Computer science)
Task-based workflows
Partial dependencies
Lazy dependency release
Eager dependency release
Distributed execution
Càlcul intensiu (Informàtica)
Programació en paral·lel (Informàtica)
Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
id ES_fca44a1ff5aebb0bfe1698e9d56668fc
oai_identifier_str oai:upcommons.upc.edu:2117/341363
network_acronym_str ES
network_name_str España
repository_id_str
spelling Accelerated execution via eager-release of dependencies in task-based workflowsElshazly, Hatem Mohamed Abdelfattah Eid|||0000-0002-8591-1502Lordan Gomis, Francesc|||0000-0002-9845-8890Ejarque Artigas, JorgeBadia Sala, Rosa Maria|||0000-0003-2941-5499High performance computingParallel programming (Computer science)Task-based workflowsPartial dependenciesLazy dependency releaseEager dependency releaseDistributed executionCàlcul intensiu (Informàtica)Programació en paral·lel (Informàtica)Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadorsTask-based programming models offer a flexible way to express the unstructured parallelism patterns of nowadays complex applications. This expressive capability is required to achieve maximum possible performance for applications that are executed in distributed execution platforms. In current task-based workflows, tasks are launched for execution when their data dependencies are satisfied. However, even though the data dependencies of a certain task might have been already produced, the execution of this task will be delayed until its predecessor tasks completely finish their execution. As a consequence of this approach of releasing dependencies, the amount of parallelism inherent in applications is limited and performance improvement opportunities are wasted. To mitigate this limitation, we propose an eager approach for releasing data dependencies. Following this approach, the execution of tasks will not be delayed until their predecessor tasks completely finish their execution, instead, tasks will be launched for execution as soon as their data requirements are available. Hence, more parallelism is exposed and applications can achieve higher levels of performance by overlapping the execution of tasks. Towards achieving this goal, in this paper we propose applying two changes to task-based workflow systems. First, modifying the dependency relationships of tasks to be specified not only in terms of predecessor and successor tasks but also in terms of the data that caused these dependencies. Second, triggering the release of dependencies as soon as a predecessor task generates the output data instead of having to wait until the end of the predecessor execution to release all of its dependencies. We realize this proposal using PyCOMPSs: a task-based programming model for parallelizing Python applications. Our experiments show that using an eager approach for releasing dependencies achieves more than 50% performance improvement in the total execution time as compared to the default approach of releasing dependencies.This work is partially supported by the European Union through the Horizon 2020 research and innovation programme under contracts 721865 (EXPERTISE Project) by the Spanish Government (SEV2015-0493,TIN2015-65316-P) and the Generalitat de Catalunya (contract 2014-SGR1051).Peer ReviewedSage20212021-07-0120212021-03-10journal articlehttp://purl.org/coar/resource_type/c_6501AMhttp://purl.org/coar/version/c_ab4af688f83e57aainfo:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/2117/341363https://dx.doi.org/10.1177/1094342021997558reponame:UPCommons. Portal del coneixement obert de la UPCinstname:Universitat Politècnica de Catalunya (UPC)InglésengMinisterio de Economía y Competitividad http://doi.org/10.13039/501100003329 TIN2015-65316-P COMPUTACION DE ALTAS PRESTACIONES VIIEuropean Commission http://doi.org/10.13039/100010661 Horizon 2020 Framework Programme 721865 models, EXperiments and high PERformance computing for Turbine mechanical Integrity and Structural dynamics in Europeopen accesshttp://purl.org/coar/access_right/c_abf2info:eu-repo/semantics/openAccessoai:upcommons.upc.edu:2117/3413632026-05-27T15:37:01Z
dc.title.none.fl_str_mv Accelerated execution via eager-release of dependencies in task-based workflows
title Accelerated execution via eager-release of dependencies in task-based workflows
spellingShingle Accelerated execution via eager-release of dependencies in task-based workflows
Elshazly, Hatem Mohamed Abdelfattah Eid|||0000-0002-8591-1502
High performance computing
Parallel programming (Computer science)
Task-based workflows
Partial dependencies
Lazy dependency release
Eager dependency release
Distributed execution
Càlcul intensiu (Informàtica)
Programació en paral·lel (Informàtica)
Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
title_short Accelerated execution via eager-release of dependencies in task-based workflows
title_full Accelerated execution via eager-release of dependencies in task-based workflows
title_fullStr Accelerated execution via eager-release of dependencies in task-based workflows
title_full_unstemmed Accelerated execution via eager-release of dependencies in task-based workflows
title_sort Accelerated execution via eager-release of dependencies in task-based workflows
dc.creator.none.fl_str_mv Elshazly, Hatem Mohamed Abdelfattah Eid|||0000-0002-8591-1502
Lordan Gomis, Francesc|||0000-0002-9845-8890
Ejarque Artigas, Jorge
Badia Sala, Rosa Maria|||0000-0003-2941-5499
author Elshazly, Hatem Mohamed Abdelfattah Eid|||0000-0002-8591-1502
author_facet Elshazly, Hatem Mohamed Abdelfattah Eid|||0000-0002-8591-1502
Lordan Gomis, Francesc|||0000-0002-9845-8890
Ejarque Artigas, Jorge
Badia Sala, Rosa Maria|||0000-0003-2941-5499
author_role author
author2 Lordan Gomis, Francesc|||0000-0002-9845-8890
Ejarque Artigas, Jorge
Badia Sala, Rosa Maria|||0000-0003-2941-5499
author2_role author
author
author
dc.subject.none.fl_str_mv High performance computing
Parallel programming (Computer science)
Task-based workflows
Partial dependencies
Lazy dependency release
Eager dependency release
Distributed execution
Càlcul intensiu (Informàtica)
Programació en paral·lel (Informàtica)
Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
topic High performance computing
Parallel programming (Computer science)
Task-based workflows
Partial dependencies
Lazy dependency release
Eager dependency release
Distributed execution
Càlcul intensiu (Informàtica)
Programació en paral·lel (Informàtica)
Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
description Task-based programming models offer a flexible way to express the unstructured parallelism patterns of nowadays complex applications. This expressive capability is required to achieve maximum possible performance for applications that are executed in distributed execution platforms. In current task-based workflows, tasks are launched for execution when their data dependencies are satisfied. However, even though the data dependencies of a certain task might have been already produced, the execution of this task will be delayed until its predecessor tasks completely finish their execution. As a consequence of this approach of releasing dependencies, the amount of parallelism inherent in applications is limited and performance improvement opportunities are wasted. To mitigate this limitation, we propose an eager approach for releasing data dependencies. Following this approach, the execution of tasks will not be delayed until their predecessor tasks completely finish their execution, instead, tasks will be launched for execution as soon as their data requirements are available. Hence, more parallelism is exposed and applications can achieve higher levels of performance by overlapping the execution of tasks. Towards achieving this goal, in this paper we propose applying two changes to task-based workflow systems. First, modifying the dependency relationships of tasks to be specified not only in terms of predecessor and successor tasks but also in terms of the data that caused these dependencies. Second, triggering the release of dependencies as soon as a predecessor task generates the output data instead of having to wait until the end of the predecessor execution to release all of its dependencies. We realize this proposal using PyCOMPSs: a task-based programming model for parallelizing Python applications. Our experiments show that using an eager approach for releasing dependencies achieves more than 50% performance improvement in the total execution time as compared to the default approach of releasing dependencies.
publishDate 2021
dc.date.none.fl_str_mv 2021
2021-07-01
2021
2021-03-10
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
AM
http://purl.org/coar/version/c_ab4af688f83e57aa
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/2117/341363
https://dx.doi.org/10.1177/1094342021997558
url https://hdl.handle.net/2117/341363
https://dx.doi.org/10.1177/1094342021997558
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv Ministerio de Economía y Competitividad http://doi.org/10.13039/501100003329 TIN2015-65316-P COMPUTACION DE ALTAS PRESTACIONES VII
European Commission http://doi.org/10.13039/100010661 Horizon 2020 Framework Programme 721865 models, EXperiments and high PERformance computing for Turbine mechanical Integrity and Structural dynamics in Europe
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Sage
publisher.none.fl_str_mv Sage
dc.source.none.fl_str_mv reponame:UPCommons. Portal del coneixement obert de la UPC
instname:Universitat Politècnica de Catalunya (UPC)
instname_str Universitat Politècnica de Catalunya (UPC)
reponame_str UPCommons. Portal del coneixement obert de la UPC
collection UPCommons. Portal del coneixement obert de la UPC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869425441051246592
score 15,300724

Similares en LA Referencia