AQUA (the “Application for QUality Assessment”) is an open-source software package, developed within the Climate Change Adaptation Digital Twin (Climate DT) of the European Commission’s Destination Earth (DestinE) initiative. The framework supports efficient processing and automated evaluation of high-resolution climate simulations. A new paper published in the Geoscientific Model Development introduces version 1.0.0 of AQUA and describes its architecture, core function and role within the operational workflow of the Climate DT.
The Climate DT provides globally consistent high-resolution climate and impact-sector information at spatial scales where climate change and extreme events can be observed. A single multi-decadal simulation can generate data on the scale of petabytes, with tens of terabytes produced every day. This creates significant challenges for processing and analysing the data, particularly when different data sources use formats, structures and metadata conventions.
The paper focuses on how AQUA is solving these challenges by providing a unified framework for preprocessing and analysing the data produced by high-resolution Climate DT simulations, and for comparing them with a range of observational and reanalysis datasets. It was developed by scientists from the Politecnico di Torino, Institute of atmospheric sciences and climate (ISAC), Barcelona Supercomputing Center (BSC) and Alfred Wegener Institute (AWI), who are part of the Climate DT consortium, led by CSC IT Center for Science and working in close collaboration with ECMWF.
AQUA in the Climate DT workflow
An important feature of the AQUA framework is its ability to harmonise data and metadata from different sources. This allows observations, reanalysis and outputs from different climate models to be accessed and processed in a consistent way, simplifying the analysis of otherwise heterogeneous datasets.
The AQUA framework is designed around four key principles: homogeneity, efficiency, modularity and flexibility and it is already an essential component of the operational Climate DT workflow. The Climate DT runs three coupled climate models – ICON, IFS-NEMO and IFS-FESOM – on the LUMI and MareNostrum 5 supercomputers, through strategic access provided by a Special Access call to the EuroHPC supercomputers. Within this workflow, AQUA is responsible for the online monitoring of the simulations.
By building on widely used Python tools including Xarray, Dask, CDO and Intake, AQUA can process datasets that are larger than the memory available on an individual machine and make use of parallel computing resources. At the same time, it provides a common interface for datasets stored in different formats and following different metadata conventions. This enables climate data from different models and sources to be processed in a consistent way, while allowing to select only the components required for a particular analysis.
When a new month of simulation data becomes available, AQUA can reduce the original hourly or daily high-resolution data to produce a monthly Low Resolution Archive (LRA). The LRA provides the basis for much of the subsequent analysis and allows the simulations to continue to be monitored even when the original high-resolution data have been moved from the supercomputer to the DestinE Data Lake.
AQUA diagnostics are then executed in parallel, with the resulting data and figures providing the backend for the Climate DT monitoring dashboard. This automated process provides an overview of the status of the climate simulations close to real time.
AQUA can also access Climate DT data through different parts of the Destination Earth data infrastructure, including local storage on the high-performance computing systems and remote access through the DestinE Data Lake.
An open-source framework for climate data analysis
Overall, AQUA provides an efficient, modular and flexible framework for the comprehensive and near-real-time evaluation of Climate DT simulations, supporting the continuous assessment and improvement of their quality, robustness and trustworthiness.
The tool is designed as an open-source and extensible platform that can evolve with the needs of the Climate DT and the wider climate modelling community. Next to the Climate DT, the open-source framework can potentially support the wider climate modelling community in processing and evaluating increasingly large and complex climate datasets.
Watch our latest recording to learn more about the Climate DT and its high-resolution climate simulations, their evaluation and how to access the data.
Discover the Climate DT.
The Climate DT, procured by ECMWF in the framework of Destination Earth is developed through a contract led by CSC-IT Center for Science and includes Alfred Wegener Institute Helmholtz Centre for Polar and Marine Research (AWI), Barcelona Supercomputing Center (BSC), Max Planck Institute for Meteorology (MPI-M), Institute of Atmospheric Sciences and Climate (CNR-ISAC), German Climate Computing Centre (DKRZ), National Meteorological Service of Germany (DWD), Finnish Meteorological Institute (FMI), Hewlett Packard Enterprise (HPE), Polytechnic University of Turin (POLITO), Helmholtz Centre for Environmental Research (UFZ) and University of Helsinki (UH), in close collaboration with ECMWF.
Destination Earth is a European Union funded initiative launched in 2022, with the aim to build a digital replica of the Earth system by 2030. The initiative is being jointly implemented under the leadership of DG CNECT, by three entrusted entities: the European Centre for Medium-Range Weather Forecasts (ECMWF) responsible for the implementation of the two ‘digital twins’, the ‘Digital Twin Engine’ and the AI Earth system model, the European Space Agency (ESA) responsible for implementation the DestinE Platform, and the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT), responsible for the implementation of the ‘Data Lake’.
We acknowledge the EuroHPC Joint Undertaking for awarding DestinE strategic access through a EuroHPC Special Access call to the EuroHPC supercomputers LUMI, hosted by CSC (Finland) and the LUMI consortium, Marenostrum5, hosted by BSC (Spain), which were used to produce the Climate DT simulations.
More information about Destination Earth is on the Destination Earth website and the EU Commission website.
