Automatic generation of workload profiles using unsupervised learning pipelines
The complexity of resource usage and power consumption on cloud-based applications makes the understanding of application behavior through expert examination difficult. The difficulty increases when applications are seen as “black boxes”, where only external monitoring can be retrieved. Furthermore,...
| Authors: | , , |
|---|---|
| Format: | article |
| Publication Date: | 2017 |
| Country: | España |
| Institution: | Universitat Politècnica de Catalunya (UPC) |
| Repository: | UPCommons. Portal del coneixement obert de la UPC |
| Language: | English |
| OAI Identifier: | oai:upcommons.upc.edu:2117/113596 |
| Online Access: | https://hdl.handle.net/2117/113596 https://dx.doi.org/10.1109/TNSM.2017.2786047 |
| Access Level: | Open access |
| Keyword: | Memory management (Computer science) Hidden Markov models Machine learning CRBM Deep learning MapReduce Measurement Monitoring Phase detection Telemetry Unsupervised learning Workload modeling Gestió de memòria (Informàtica) Aprenentatge automàtic Àrees temàtiques de la UPC::Informàtica::Intel·ligència artificial::Aprenentatge automàtic Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors |
| Summary: | The complexity of resource usage and power consumption on cloud-based applications makes the understanding of application behavior through expert examination difficult. The difficulty increases when applications are seen as “black boxes”, where only external monitoring can be retrieved. Furthermore, given the different amount of scenarios and applications, automation is required. Here we examine and model application behavior by finding behavior phases. We use Conditional Restricted Boltzmann Machines (CRBM) to model time-series containing resources traces measurements like CPU, Memory and IO. CRBMs can be used to map a given given historic window of trace behaviour into a single vector. This low dimensional and time-aware vector can be passed through clustering methods, from simplistic ones like k-means to more complex ones like those based on Hidden Markov Models (HMM). We use these methods to find phases of similar behaviour in the workloads. Our experimental evaluation shows that the proposed method is able to identify different phases of resource consumption across different workloads. We show that the distinct phases contain specific resource patterns that distinguish them. |
|---|