The computational and energy cost of simulation and storage for climate science: lessons from CMIP6

The Coupled Model Intercomparison Project (CMIP) is one of the biggest international efforts aimed at better understanding the past, present, and future of climate changes in a multi-model context. A total of 21 model intercomparison projects (MIPs) were endorsed in its sixth phase (CMIP6), which in...

Descripción completa

Detalles Bibliográficos
Autores: Acosta Cobos, Mario César, Palomas Martínez, Sergi, Paronuzzi Ticco, Stella Valentina, Utrera Iglesias, Gladys Miriam|||0000-0002-0637-1297, Biercamp, Joachim, Bretonnière, Pierre-Antoine|||0000-0002-3066-6685, Budich, Reinhard, Castrillo Melguizo, Miguel, Caubel, Arnaud, Doblas Reyes, Francisco
Tipo de recurso: artículo
Fecha de publicación:2024
País:España
Institución:Universitat Politècnica de Catalunya (UPC)
Repositorio:UPCommons. Portal del coneixement obert de la UPC
Idioma:inglés
OAI Identifier:oai:upcommons.upc.edu:2117/409333
Acceso en línea:https://hdl.handle.net/2117/409333
https://dx.doi.org/10.5194/gmd-17-3081-2024
Access Level:acceso abierto
Palabra clave:Climatic changes -- Mathematical models
Coupled Model Intercomparison Project
CMIP
Climate model performance
Canvis climàtics -- Models matemàtics
Àrees temàtiques de la UPC::Informàtica::Aplicacions de la informàtica::Aplicacions informàtiques a la física i l‘enginyeria
Àrees temàtiques de la UPC::Desenvolupament humà i sostenible::Degradació ambiental::Canvi climàtic
id ES_c968d8f00a0bace86e12eebbef47e0ce
oai_identifier_str oai:upcommons.upc.edu:2117/409333
network_acronym_str ES
network_name_str España
repository_id_str
dc.title.none.fl_str_mv The computational and energy cost of simulation and storage for climate science: lessons from CMIP6
title The computational and energy cost of simulation and storage for climate science: lessons from CMIP6
spellingShingle The computational and energy cost of simulation and storage for climate science: lessons from CMIP6
Acosta Cobos, Mario César
Climatic changes -- Mathematical models
Coupled Model Intercomparison Project
CMIP
Climate model performance
Canvis climàtics -- Models matemàtics
Àrees temàtiques de la UPC::Informàtica::Aplicacions de la informàtica::Aplicacions informàtiques a la física i l‘enginyeria
Àrees temàtiques de la UPC::Desenvolupament humà i sostenible::Degradació ambiental::Canvi climàtic
title_short The computational and energy cost of simulation and storage for climate science: lessons from CMIP6
title_full The computational and energy cost of simulation and storage for climate science: lessons from CMIP6
title_fullStr The computational and energy cost of simulation and storage for climate science: lessons from CMIP6
title_full_unstemmed The computational and energy cost of simulation and storage for climate science: lessons from CMIP6
title_sort The computational and energy cost of simulation and storage for climate science: lessons from CMIP6
dc.creator.none.fl_str_mv Acosta Cobos, Mario César
Palomas Martínez, Sergi
Paronuzzi Ticco, Stella Valentina
Utrera Iglesias, Gladys Miriam|||0000-0002-0637-1297
Biercamp, Joachim
Bretonnière, Pierre-Antoine|||0000-0002-3066-6685
Budich, Reinhard
Castrillo Melguizo, Miguel
Caubel, Arnaud
Doblas Reyes, Francisco
author Acosta Cobos, Mario César
author_facet Acosta Cobos, Mario César
Palomas Martínez, Sergi
Paronuzzi Ticco, Stella Valentina
Utrera Iglesias, Gladys Miriam|||0000-0002-0637-1297
Biercamp, Joachim
Bretonnière, Pierre-Antoine|||0000-0002-3066-6685
Budich, Reinhard
Castrillo Melguizo, Miguel
Caubel, Arnaud
Doblas Reyes, Francisco
author_role author
author2 Palomas Martínez, Sergi
Paronuzzi Ticco, Stella Valentina
Utrera Iglesias, Gladys Miriam|||0000-0002-0637-1297
Biercamp, Joachim
Bretonnière, Pierre-Antoine|||0000-0002-3066-6685
Budich, Reinhard
Castrillo Melguizo, Miguel
Caubel, Arnaud
Doblas Reyes, Francisco
author2_role author
author
author
author
author
author
author
author
author
dc.subject.none.fl_str_mv Climatic changes -- Mathematical models
Coupled Model Intercomparison Project
CMIP
Climate model performance
Canvis climàtics -- Models matemàtics
Àrees temàtiques de la UPC::Informàtica::Aplicacions de la informàtica::Aplicacions informàtiques a la física i l‘enginyeria
Àrees temàtiques de la UPC::Desenvolupament humà i sostenible::Degradació ambiental::Canvi climàtic
topic Climatic changes -- Mathematical models
Coupled Model Intercomparison Project
CMIP
Climate model performance
Canvis climàtics -- Models matemàtics
Àrees temàtiques de la UPC::Informàtica::Aplicacions de la informàtica::Aplicacions informàtiques a la física i l‘enginyeria
Àrees temàtiques de la UPC::Desenvolupament humà i sostenible::Degradació ambiental::Canvi climàtic
description The Coupled Model Intercomparison Project (CMIP) is one of the biggest international efforts aimed at better understanding the past, present, and future of climate changes in a multi-model context. A total of 21 model intercomparison projects (MIPs) were endorsed in its sixth phase (CMIP6), which included 190 different experiments that were used to simulate 40 000 years and produced around 40 PB of data in total. This paper presents the main findings obtained from the CPMIP (the Computational Performance Model Intercomparison Project), a collection of a common set of metrics, specifically designed for assessing climate model performance. These metrics were exclusively collected from the production runs of experiments used in CMIP6 and primarily from institutions within the IS-ENES3 consortium. The document presents the full set of CPMIP metrics per institution and experiment, including a detailed analysis and discussion of each of the measurements. During the analysis, we found a positive correlation between the core hours needed, the complexity of the models, and the resolution used. Likewise, we show that between 5 %–15 % of the execution cost is spent in the coupling between independent components, and it only gets worse by increasing the number of resources. From the data, it is clear that queue times have a great impact on the actual speed achieved and have a huge variability across different institutions, ranging from none to up to 78 % execution overhead. Furthermore, our evaluation shows that the estimated carbon footprint of running such big simulations within the IS-ENES3 consortium is 1692 t of CO2 equivalent. As a result of the collection, we contribute to the creation of a comprehensive database for future community reference, establishing a benchmark for evaluation and facilitating the multi-model, multi-platform comparisons crucial for understanding climate modelling performance. Given the diverse range of applications, configurations, and hardware utilised, further work is required for the standardisation and formulation of general rules. The paper concludes with recommendations for future exercises aimed at addressing the encountered challenges which will facilitate more collections of a similar nature.
publishDate 2024
dc.date.none.fl_str_mv 2024
2024-04-19
2024
2024-06-06
dc.type.none.fl_str_mv journal article
http://purl.org/coar/resource_type/c_6501
VoR
http://purl.org/coar/version/c_970fb48d4fbd8a85
dc.type.openaire.fl_str_mv info:eu-repo/semantics/article
format article
dc.identifier.none.fl_str_mv https://hdl.handle.net/2117/409333
https://dx.doi.org/10.5194/gmd-17-3081-2024
url https://hdl.handle.net/2117/409333
https://dx.doi.org/10.5194/gmd-17-3081-2024
dc.language.none.fl_str_mv Inglés
eng
language_invalid_str_mv Inglés
language eng
dc.relation.none.fl_str_mv European Commission http://doi.org/10.13039/100010661 Horizon 2020 Framework Programme 824084 Infrastructure for the European Network for Earth System modelling
Agencia Estatal de Investigación http://doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 PID2020-116324RA-I00 OPTIMIZACION DE MODELOS DEL SISTEMA TERRESTRE EN EL CAMINO A LA NUEVA GENERACION DE SISTEMAS "EXASCALE" DE ALTO RENDIMIENTO
dc.rights.none.fl_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution 4.0 International
http://creativecommons.org/licenses/by/4.0/
dc.rights.openaire.fl_str_mv info:eu-repo/semantics/openAccess
rights_invalid_str_mv open access
http://purl.org/coar/access_right/c_abf2
Attribution 4.0 International
http://creativecommons.org/licenses/by/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv application/pdf
dc.publisher.none.fl_str_mv European Geosciences Union (EGU)
publisher.none.fl_str_mv European Geosciences Union (EGU)
dc.source.none.fl_str_mv reponame:UPCommons. Portal del coneixement obert de la UPC
instname:Universitat Politècnica de Catalunya (UPC)
instname_str Universitat Politècnica de Catalunya (UPC)
reponame_str UPCommons. Portal del coneixement obert de la UPC
collection UPCommons. Portal del coneixement obert de la UPC
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869419368639627264
spelling The computational and energy cost of simulation and storage for climate science: lessons from CMIP6Acosta Cobos, Mario CésarPalomas Martínez, SergiParonuzzi Ticco, Stella ValentinaUtrera Iglesias, Gladys Miriam|||0000-0002-0637-1297Biercamp, JoachimBretonnière, Pierre-Antoine|||0000-0002-3066-6685Budich, ReinhardCastrillo Melguizo, MiguelCaubel, ArnaudDoblas Reyes, FranciscoClimatic changes -- Mathematical modelsCoupled Model Intercomparison ProjectCMIPClimate model performanceCanvis climàtics -- Models matemàticsÀrees temàtiques de la UPC::Informàtica::Aplicacions de la informàtica::Aplicacions informàtiques a la física i l‘enginyeriaÀrees temàtiques de la UPC::Desenvolupament humà i sostenible::Degradació ambiental::Canvi climàticThe Coupled Model Intercomparison Project (CMIP) is one of the biggest international efforts aimed at better understanding the past, present, and future of climate changes in a multi-model context. A total of 21 model intercomparison projects (MIPs) were endorsed in its sixth phase (CMIP6), which included 190 different experiments that were used to simulate 40 000 years and produced around 40 PB of data in total. This paper presents the main findings obtained from the CPMIP (the Computational Performance Model Intercomparison Project), a collection of a common set of metrics, specifically designed for assessing climate model performance. These metrics were exclusively collected from the production runs of experiments used in CMIP6 and primarily from institutions within the IS-ENES3 consortium. The document presents the full set of CPMIP metrics per institution and experiment, including a detailed analysis and discussion of each of the measurements. During the analysis, we found a positive correlation between the core hours needed, the complexity of the models, and the resolution used. Likewise, we show that between 5 %–15 % of the execution cost is spent in the coupling between independent components, and it only gets worse by increasing the number of resources. From the data, it is clear that queue times have a great impact on the actual speed achieved and have a huge variability across different institutions, ranging from none to up to 78 % execution overhead. Furthermore, our evaluation shows that the estimated carbon footprint of running such big simulations within the IS-ENES3 consortium is 1692 t of CO2 equivalent. As a result of the collection, we contribute to the creation of a comprehensive database for future community reference, establishing a benchmark for evaluation and facilitating the multi-model, multi-platform comparisons crucial for understanding climate modelling performance. Given the diverse range of applications, configurations, and hardware utilised, further work is required for the standardisation and formulation of general rules. The paper concludes with recommendations for future exercises aimed at addressing the encountered challenges which will facilitate more collections of a similar nature.This research has been supported by the Barcelona Supercomputing Center (grant nos. PID2020116324RA-I00 (OEMES) and H2020-GA-824084 (ISENES)).Peer ReviewedArticle signat per 22 autors/es: Mario C. Acosta 1, Sergi Palomas 1, Stella V. Paronuzzi Ticco 1, Gladys Utrera 1, Joachim Biercamp 2, Pierre-Antoine Bretonniere 1, Reinhard Budich 3, Miguel Castrillo 1, Arnaud Caubel 4, Francisco Doblas-Reyes 1,14, Italo Epicoco 5, Uwe Fladrich 6, Sylvie Joussaume 4, Alok Kumar Gupta 7, Bryan Lawrence 8, Philippe Le Sager 9, Grenville Lister 8, Marie-Pierre Moine 10, Jean-Christophe Rioual 11, Sophie Valcke 10, Niki Zadeh 12, and Venkatramani Balaji 13 / 1- Barcelona Supercomputing Center, Plaça d’Eusebi Güell, 1–3, 08034 Barcelona, Spain; 2- German Climate Computing Centre, Bundesstraße 45a, 20146 Hamburg, Germany; 3- Max Planck Institute, Hofgartenstr. 8, 80539 Munich, Germany; 4- Institut Pierre-Simon Laplace, 11 Bd d’Alembert, 78280 Guyancourt, France; 5- Euro-Mediterranean Center on Climate Change, Via della Libertà, 12, 30121 Venice, Italy; 6- Swedish Meteorological and Hydrological Institute, 601 76 Norrköping, Sweden; 7- Norwegian Research Centre, Nygårdsgaten 112, 5008 Bergen, Norway; 8- National Centre for Atmospheric Science, Fairbairn House, 71–75 Clarendon Rd, Woodhouse, Leeds LS2 9PH, United Kingdom; 9- Royal Netherlands Meteorological Institute, Utrechtseweg 297, 3731 GA De Bilt, the Netherlands; 10- European Center for Advanced Research and Training in Scientific Computing, 42 Av. Gaspard Coriolis, 31100 Toulouse, France; 11- Meteorological Office, Fitzroy Road, Exeter, Devon, EX1 3PB, United Kingdom; 12- National Oceanic and Atmospheric Administration, 1401 Constitution Avenue NW, Room 5128, Washington, DC 20230, USA; 13- High Meadows Environmental Institute, Princeton University, Guyot Hall, Room 129, Princeton, NJ 08544-1003, USA; 14- Catalan Institution for Research and Advanced Studies, Passeig Lluís Companys 23, 08010 Barcelona, SpainEuropean Geosciences Union (EGU)20242024-04-1920242024-06-06journal articlehttp://purl.org/coar/resource_type/c_6501VoRhttp://purl.org/coar/version/c_970fb48d4fbd8a85info:eu-repo/semantics/articleapplication/pdfhttps://hdl.handle.net/2117/409333https://dx.doi.org/10.5194/gmd-17-3081-2024reponame:UPCommons. Portal del coneixement obert de la UPCinstname:Universitat Politècnica de Catalunya (UPC)InglésengEuropean Commission http://doi.org/10.13039/100010661 Horizon 2020 Framework Programme 824084 Infrastructure for the European Network for Earth System modellingAgencia Estatal de Investigación http://doi.org/10.13039/501100011033 Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 PID2020-116324RA-I00 OPTIMIZACION DE MODELOS DEL SISTEMA TERRESTRE EN EL CAMINO A LA NUEVA GENERACION DE SISTEMAS "EXASCALE" DE ALTO RENDIMIENTOopen accesshttp://purl.org/coar/access_right/c_abf2Attribution 4.0 Internationalhttp://creativecommons.org/licenses/by/4.0/info:eu-repo/semantics/openAccessoai:upcommons.upc.edu:2117/4093332026-05-27T15:37:01Z
score 15,811543

Similares en LA Referencia