Adaptive sliding windows for improved estimation of data center resource utilization

Accurate prediction of data center resource utilization is required for capacity planning, job scheduling, energy saving, workload placement, and load balancing to utilize the resources efficiently. However, accurately predicting those resources is challenging due to dynamic workloads, heterogeneous...

Descripción completa

Detalles Bibliográficos
Autores: Baig, Shuja-ur-Rehman, Iqbal, Waheed, Berral García, Josep Lluís|||0000-0003-3037-3580, Carrera Pérez, David|||0000-0003-4898-3424
Tipo de recurso: artículo
Fecha de publicación:2020
País:España
Institución:Universitat Politècnica de Catalunya (UPC)
Repositorio:UPCommons. Portal del coneixement obert de la UPC
Idioma:inglés
OAI Identifier:oai:upcommons.upc.edu:2117/186459
Acceso en línea:https://hdl.handle.net/2117/186459
https://dx.doi.org/10.1016/j.future.2019.10.026
Access Level:acceso abierto
Palabra clave:Cloud computing
Data processing service centers
Machine learning
Resource allocation
Sliding windows
Adaptive observation window
Time series
Resource estimation
Data center
Computació en núvol
Centres informàtics
Aprenentatge automàtic
Assignació de recursos
Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
id ES_f975cfbac146c003eebc408c2faf74cb
oai_identifier_str oai:upcommons.upc.edu:2117/186459
network_acronym_str ES
network_name_str España
repository_id_str
spelling Adaptive sliding windows for improved estimation of data center resource utilizationBaig, Shuja-ur-RehmanIqbal, WaheedBerral García, Josep Lluís|||0000-0003-3037-3580Carrera Pérez, David|||0000-0003-4898-3424Cloud computingData processing service centersMachine learningResource allocationSliding windowsAdaptive observation windowTime seriesResource estimationData centerComputació en núvolCentres informàticsAprenentatge automàticAssignació de recursosÀrees temàtiques de la UPC::Informàtica::Arquitectura de computadorsAccurate prediction of data center resource utilization is required for capacity planning, job scheduling, energy saving, workload placement, and load balancing to utilize the resources efficiently. However, accurately predicting those resources is challenging due to dynamic workloads, heterogeneous infrastructures, and multi-tenant co-hosted applications. Existing prediction methods use fixed size observation windows which cannot produce accurate results because of not being adaptively adjusted to capture local trends in the most recent data. Therefore, those methods train on large fixed sliding windows using an irrelevant large number of observations yielding to inaccurate estimations or fall for inaccuracy due to degradation of estimations with short windows on quick changing trends. In this paper we propose a deep learning-based adaptive window size selection method, dynamically limiting the sliding window size to capture the trend for the latest resource utilization, then build an estimation model for each trend period. We evaluate the proposed method against multiple baseline and state-of-the-art methods, using real data-center workload data sets. The experimental evaluation shows that the proposed solution outperforms those state-of-the-art approaches and yields 16 to 54% improved prediction accuracy compared to the baseline methods.This work is partially supported by the European ResearchCouncil (ERC) under the EU Horizon 2020 programme(GA 639595), the Spanish Ministry of Economy, Industry andCompetitiveness (TIN2015-65316-P and IJCI2016-27485), theGeneralitat de Catalunya, Spain (2014-SGR-1051) and Universityof the Punjab, Pakistan. The statements made herein are solelythe responsibility of the authors.Peer ReviewedElsevier20202020-03-0120202020-05-06journal articlehttp://purl.org/coar/resource_type/c_6501VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/2117/186459https://dx.doi.org/10.1016/j.future.2019.10.026reponame:UPCommons. Portal del coneixement obert de la UPCinstname:Universitat Politècnica de Catalunya (UPC)InglésengEuropean Commission http://doi.org/10.13039/100010661 Horizon 2020 Framework Programme 639595 Holistic Integration of Emerging Supercomputing TechnologiesMinisterio de Economía y Competitividad http://doi.org/10.13039/501100003329 TIN2015-65316-P COMPUTACION DE ALTAS PRESTACIONES VIIopen accesshttp://purl.org/coar/access_right/c_abf2Attribution-NonCommercial-NoDerivatives 4.0 Internationalhttp://creativecommons.org/licenses/by-nc-nd/4.0/info:eu-repo/semantics/openAccessoai:upcommons.upc.edu:2117/1864592026-05-27T15:37:01Z
dc.title.none.fl_str_mv Adaptive sliding windows for improved estimation of data center resource utilization
title Adaptive sliding windows for improved estimation of data center resource utilization
spellingShingle Adaptive sliding windows for improved estimation of data center resource utilization
Baig, Shuja-ur-Rehman
Cloud computing
Data processing service centers
Machine learning
Resource allocation
Sliding windows
Adaptive observation window
Time series
Resource estimation
Data center
Computació en núvol
Centres informàtics
Aprenentatge automàtic
Assignació de recursos
Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
title_short Adaptive sliding windows for improved estimation of data center resource utilization
title_full Adaptive sliding windows for improved estimation of data center resource utilization
title_fullStr Adaptive sliding windows for improved estimation of data center resource utilization
title_full_unstemmed Adaptive sliding windows for improved estimation of data center resource utilization
title_sort Adaptive sliding windows for improved estimation of data center resource utilization
dc.creator.none.fl_str_mv Baig, Shuja-ur-Rehman
Iqbal, Waheed
Berral García, Josep Lluís|||0000-0003-3037-3580
Carrera Pérez, David|||0000-0003-4898-3424
author Baig, Shuja-ur-Rehman
author_facet Baig, Shuja-ur-Rehman
Iqbal, Waheed
Berral García, Josep Lluís|||0000-0003-3037-3580
Carrera Pérez, David|||0000-0003-4898-3424
author_role author
author2 Iqbal, Waheed
Berral García, Josep Lluís|||0000-0003-3037-3580
Carrera Pérez, David|||0000-0003-4898-3424
author2_role author
author
author
dc.subject.none.fl_str_mv Cloud computing
Data processing service centers
Machine learning
Resource allocation
Sliding windows
Adaptive observation window
Time series
Resource estimation
Data center
Computació en núvol
Centres informàtics
Aprenentatge automàtic
Assignació de recursos
Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
topic Cloud computing
Data processing service centers
Machine learning
Resource allocation
Sliding windows
Adaptive observation window
Time series
Resource estimation
Data center
Computació en núvol
Centres informàtics
Aprenentatge automàtic
Assignació de recursos
Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
description Accurate prediction of data center resource utilization is required for capacity planning, job scheduling, energy saving, workload placement, and load balancing to utilize the resources efficiently. However, accurately predicting those resources is challenging due to dynamic workloads, heterogeneous infrastructures, and multi-tenant co-hosted applications. Existing prediction methods use fixed size observation windows which cannot produce accurate results because of not being adaptively adjusted to capture local trends in the most recent data. Therefore, those methods train on large fixed sliding windows using an irrelevant large number of observations yielding to inaccurate estimations or fall for inaccuracy due to degradation of estimations with short windows on quick changing trends. In this paper we propose a deep learning-based adaptive window size selection method, dynamically limiting the sliding window size to capture the trend for the latest resource utilization, then build an estimation model for each trend period. We evaluate the proposed method against multiple baseline and state-of-the-art methods, using real data-center workload data sets. The experimental evaluation shows that the proposed solution outperforms those state-of-the-art approaches and yields 16 to 54% improved prediction accuracy compared to the baseline methods.
publishDate 2020
dc.date.none.fl_str_mv 2020
2020-03-01
2020
2020-05-06
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/2117/186459
https://dx.doi.org/10.1016/j.future.2019.10.026
url https://hdl.handle.net/2117/186459
https://dx.doi.org/10.1016/j.future.2019.10.026
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv European Commission http://doi.org/10.13039/100010661 Horizon 2020 Framework Programme 639595 Holistic Integration of Emerging Supercomputing Technologies
Ministerio de Economía y Competitividad http://doi.org/10.13039/501100003329 TIN2015-65316-P COMPUTACION DE ALTAS PRESTACIONES VII
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution-NonCommercial-NoDerivatives 4.0 International
http://creativecommons.org/licenses/by-nc-nd/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv Elsevier
publisher.none.fl_str_mv Elsevier
dc.source.none.fl_str_mv reponame:UPCommons. Portal del coneixement obert de la UPC
instname:Universitat Politècnica de Catalunya (UPC)
instname_str Universitat Politècnica de Catalunya (UPC)
reponame_str UPCommons. Portal del coneixement obert de la UPC
collection UPCommons. Portal del coneixement obert de la UPC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869425096584593408
score 15,300719

Similares en LA Referencia