ParaView + Alya + D8tree: Integrating high performance computing and high performance data analytics

Large scale time-dependent particle simulations can generate massive amounts of data, making it so that storing the results is often the slowest phase and the primary time bottleneck of the simulation. Furthermore, analysing this amount of data with traditional tools has become increasingly challeng...

Full description

Bibliographic Details
Authors: Artigues, Antoni, Cugnasco, Cesare, Becerra Fontal, Yolanda|||0000-0003-2357-7796, Cucchietti, Fernando|||0000-0002-9027-1263, Houzeaux, Guillaume|||0000-0002-2592-1426, Vázquez, Mariano|||0000-0002-2526-6708, Torres Viñals, Jordi|||0000-0003-1963-7418, Ayguadé Parra, Eduard|||0000-0002-5146-103X, Labarta Mancho, Jesús José|||0000-0002-7489-4727
Format: article
Publication Date:2017
Country:España
Institution:Universitat Politècnica de Catalunya (UPC)
Repository:UPCommons. Portal del coneixement obert de la UPC
Language:English
OAI Identifier:oai:upcommons.upc.edu:2117/106697
Online Access:https://hdl.handle.net/2117/106697
https://dx.doi.org/10.1016/j.procs.2017.05.170
Access Level:Open access
Keyword:Information visualization
High performance computing
Big data
HPC
HPDA
Key-Value data stores
Visualització de la informació
Càlcul intensiu (Informàtica)
Macrodades
Àrees temàtiques de la UPC::Informàtica::Arquitectura de computadors
Description
Summary:Large scale time-dependent particle simulations can generate massive amounts of data, making it so that storing the results is often the slowest phase and the primary time bottleneck of the simulation. Furthermore, analysing this amount of data with traditional tools has become increasingly challenging, and it is often virtually impossible to have a visual representation of the full set. We propose a novel architecture that integrates an HPC-based multi-physics simulation code, a NoSQL database, and a data analysis and visualisation application. The goals are two: On the one hand, we aim to speed up the simulations taking advantage of the scalability of key-value data stores, while at the same time enabling real-time approximated data visualisation and interactive exploration. On the other hand, we want to make it efficient to explore and analyse the large data base of results produced. Therefore, this work represents a clear example of integrating High Performance Computing with High Performance Data Analytics. Our prototype proves the validity of our approach and shows great performance improvements. Indeed, we reduced by 67.5% the time to store the simulation while we made real-time queries run 52 times faster than alternative solutions.

Similar Items