Performance characterization and optimization of in-memory data analytics on a scale-up server
The sheer increase in volume of data over the last decade has triggered research in cluster computing frameworks that enable web enterprises to extract big insights from big data. While Apache Spark defines the state of the art in big data analytics platforms for (i) exploiting data-flow and in-memo...
| Autor: | |
|---|---|
| Formato: | tesis doctoral |
| Estado: | Versión publicada |
| Fecha de publicación: | 2017 |
| País: | España |
| Recursos: | CBUC, CESCA |
| Repositorio: | TDR. Tesis Doctorales en Red |
| OAI Identifier: | oai:www.tdx.cat:10803/461767 |
| Acesso em linha: | http://hdl.handle.net/10803/461767 https://dx.doi.org/10.5821/dissertation-2117-114440 |
| Access Level: | acceso abierto |
| Palavra-chave: | Àrees temàtiques de la UPC::Informàtica 004 |
| Resumo: | The sheer increase in volume of data over the last decade has triggered research in cluster computing frameworks that enable web enterprises to extract big insights from big data. While Apache Spark defines the state of the art in big data analytics platforms for (i) exploiting data-flow and in-memory computing and (ii) for exhibiting superior scale-out performance on the commodity machines, little effort has been devoted at understanding the performance of in-memory data analytics with Spark on modern scale-up servers. This thesis characterizes the performance of in-memory data analytics with Spark on scale-up servers. Through empirical evaluation of representative benchmark workloads on a dual socket server, we have found that in-memory data analytics with Spark exhibit poor multi-core scalability beyond 12 cores due to thread level load imbalance and work-time inflation. We have also found that workloads are bound by the latency of frequent data accesses to DRAM. By enlarging input data size, application performance degrades significantly due to substantial increase in wait time during I/O operations and garbage collection, despite 10% better instruction retirement rate (due to lower L1 cache misses and higher core utilization). For data accesses we have found that simultaneous multi-threading is effective in hiding the data latencies. We have also observed that (i) data locality on NUMA nodes can improve the performance by 10% on average, (ii) disabling next-line L1-D prefetchers can reduce the execution time by up-to 14%. For GC impact, we match memory behaviour with the garbage collector to improve performance of applications between 1.6x to 3x. and recommend to use multiple small executors that can provide up-to 36% speedup over single large executor. Based on the characteristics of workloads, it envisions nearmemory and near storage hardware acceleration to improve the single-node performance of scale-out frameworks like Apache Spark. Using modelling approaches, it estimates the speed-up of 5x for Apache Spark using near data hardware acceleration. |
|---|