Extending science gateway frameworks to support Big Data applications in the cloud

Cloud computing offers massive scalability and elasticity required by many scientific and commercial applications. Combining the computational and data handling capabilities of clouds with parallel processing also has the potential to tackle Big Data problems efficiently. Science gateway frameworks...

Descripción completa

Detalles Bibliográficos
Autores: Gugnani, Shashank, Blanco Real, José Carlos, Kiss, Tamas, Terstyanszky, Gabor
Tipo de recurso: artículo
Fecha de publicación:2016
País:España
Institución:Universidad de Cantabria (UC)
Repositorio:UCrea Repositorio Abierto de la Universidad de Cantabria
Idioma:inglés
OAI Identifier:oai:repositorio.unican.es:10902/19311
Acceso en línea:http://hdl.handle.net/10902/19311
Access Level:acceso abierto
Palabra clave:Big data
Hadoop
MapReduce
Science gateway
WS-PGRADE
Workflow
id ES_a0ff189def22ff8ecc015f93f9955241
oai_identifier_str oai:repositorio.unican.es:10902/19311
network_acronym_str ES
network_name_str España
repository_id_str
spelling Extending science gateway frameworks to support Big Data applications in the cloudGugnani, ShashankBlanco Real, José CarlosKiss, TamasTerstyanszky, GaborBig dataHadoopMapReduceScience gatewayWS-PGRADEWorkflowCloud computing offers massive scalability and elasticity required by many scientific and commercial applications. Combining the computational and data handling capabilities of clouds with parallel processing also has the potential to tackle Big Data problems efficiently. Science gateway frameworks and workflow systems enable application developers to implement complex applications and make these available for end-users via simple graphical user interfaces. The integration of such frameworks with Big Data processing tools on the cloud opens new opportunities for application developers. This paper investigates how workflow systems and science gateways can be extended with Big Data processing capabilities. A generic approach based on infrastructure aware workflows is suggested and a proof of concept is implemented based on the WS-PGRADE/gUSE science gateway framework and its integration with the Hadoop parallel data processing solution based on the MapReduce paradigm in the cloud. The provided analysis demonstrates that the methods described to integrate Big Data processing with workflows and science gateways work well in different cloud infrastructures and application scenarios, and can be used to create massively parallel applications for scientific analysis of Big Data.This work is partially funded by the CloudSME Cloud-Based Simulation platform for Manufacturing and Engineering Project No. 608886 (FP7-2013-NMPICT-FOF). Financial support from Programa de Personal Investigador en Formacion Predoctoral from Universidad de ´ Cantabria, co-funded by the regional government of Cantabria, has also been utilized.Springer NatureUniversidad de Cantabria20162016-12-01journal articlehttp://purl.org/coar/resource_type/c_6501NAhttp://purl.org/coar/version/c_be7fb7dd8ff6fe43info:eu-repo/semantics/articlehttp://hdl.handle.net/10902/19311Journal of Grid Computing, 2016, 14(4), 589-601reponame:UCrea Repositorio Abierto de la Universidad de Cantabriainstname:Universidad de Cantabria (UC)Inglésengopen accesshttp://purl.org/coar/access_right/c_abf2Attribution 4.0 Internationalhttp://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccessoai:repositorio.unican.es:10902/193112026-06-02T12:39:31Z
dc.title.none.fl_str_mv Extending science gateway frameworks to support Big Data applications in the cloud
title Extending science gateway frameworks to support Big Data applications in the cloud
spellingShingle Extending science gateway frameworks to support Big Data applications in the cloud
Gugnani, Shashank
Big data
Hadoop
MapReduce
Science gateway
WS-PGRADE
Workflow
title_short Extending science gateway frameworks to support Big Data applications in the cloud
title_full Extending science gateway frameworks to support Big Data applications in the cloud
title_fullStr Extending science gateway frameworks to support Big Data applications in the cloud
title_full_unstemmed Extending science gateway frameworks to support Big Data applications in the cloud
title_sort Extending science gateway frameworks to support Big Data applications in the cloud
dc.creator.none.fl_str_mv Gugnani, Shashank
Blanco Real, José Carlos
Kiss, Tamas
Terstyanszky, Gabor
author Gugnani, Shashank
author_facet Gugnani, Shashank
Blanco Real, José Carlos
Kiss, Tamas
Terstyanszky, Gabor
author_role author
author2 Blanco Real, José Carlos
Kiss, Tamas
Terstyanszky, Gabor
author2_role author
author
author
dc.contributor.none.fl_str_mv Universidad de Cantabria
dc.subject.none.fl_str_mv Big data
Hadoop
MapReduce
Science gateway
WS-PGRADE
Workflow
topic Big data
Hadoop
MapReduce
Science gateway
WS-PGRADE
Workflow
description Cloud computing offers massive scalability and elasticity required by many scientific and commercial applications. Combining the computational and data handling capabilities of clouds with parallel processing also has the potential to tackle Big Data problems efficiently. Science gateway frameworks and workflow systems enable application developers to implement complex applications and make these available for end-users via simple graphical user interfaces. The integration of such frameworks with Big Data processing tools on the cloud opens new opportunities for application developers. This paper investigates how workflow systems and science gateways can be extended with Big Data processing capabilities. A generic approach based on infrastructure aware workflows is suggested and a proof of concept is implemented based on the WS-PGRADE/gUSE science gateway framework and its integration with the Hadoop parallel data processing solution based on the MapReduce paradigm in the cloud. The provided analysis demonstrates that the methods described to integrate Big Data processing with workflows and science gateways work well in different cloud infrastructures and application scenarios, and can be used to create massively parallel applications for scientific analysis of Big Data.
publishDate 2016
dc.date.none.fl_str_mv 2016
2016-12-01
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
NA
http://purl.org/coar/version/c_be7fb7dd8ff6fe43
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv http://hdl.handle.net/10902/19311
url http://hdl.handle.net/10902/19311
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution 4.0 International
http://creativecommons.org/licenses/by/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution 4.0 International
http://creativecommons.org/licenses/by/4.0/
eu_rights_str_mv openAccess
dc.publisher.none.fl_str_mv Springer Nature
publisher.none.fl_str_mv Springer Nature
dc.source.none.fl_str_mv Journal of Grid Computing, 2016, 14(4), 589-601
reponame:UCrea Repositorio Abierto de la Universidad de Cantabria
instname:Universidad de Cantabria (UC)
instname_str Universidad de Cantabria (UC)
reponame_str UCrea Repositorio Abierto de la Universidad de Cantabria
collection UCrea Repositorio Abierto de la Universidad de Cantabria
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869415091220250624
score 15,300724

Similares en LA Referencia