EuroHPC JU, a European success story
Destination Earth (DestinE) is an example of European cooperation to make the most of Europe’s high-performance computing (HPC) capabilities. This initiative of the European Union was quoted by many of the speakers at the second EuroHPC Summit 2024, organised by the EuroHPC Joint Undertaking (EuroHPC JU) in Antwerp, Belgium, on 18-21 March. The European Centre for Medium-Range Weather Forecasts (ECMWF) sent a significant delegation to explain how we apply cutting edge physical and machine learning / artificial intelligence models to deliver the first digital twins of DestinE in partnership with HPC centres across Europe, and many other partners.
For four days, Antwerp was the European HPC capital. The EuroHPC Summit 2024 gathered nearly 800 participants for its second edition, a fast growth that reflects the increasing interest in and excitement about European HPC and the EuroHPC JU. The Joint Undertaking has set a unique network gathering some of the most powerful supercomputers in the world only six years after its foundation, a flagship of European leadership in HPC and, now, artificial intelligence.
ECMWF took the opportunity to highlight the strategic partnership with EuroHPC within DestinE. Considered a strategic initiative for the European Union, DestinE has been granted access to four EuroHPC JU supercomputers: LUMI (CSC-IT Center for Science, Finland), Leonardo (Cineca, Italy), MareNostrum5 (Barcelona Supercomputing Center, Spain and MeluXina (LuxProvide, Luxemburg). These machines are used to deliver the two high-priority digital twins being developed by ECMWF, on Weather-induced Extremes and on Climate Change Adaptation with the software environment provided by the Digital Twin Engine.
ECMWF created a video for the EuroHPC Summit 2024 explaining the key elements of the DestinE system and how the digital twins can create a very-high-definition view of our planet, from days to decades ahead.
An example of successful collaboration
Chair of the EuroHPC JU Governing Board Rafał Duczmal said during the opening that proof that the Joint Undertaking has reached its maturity is its capacity to set and keep priorities while at the same time keeping flexibility to adapt to emerging challenges and opportunities and better serve the community.
“Achieving this requires an unwavering will from all of us involved: participating states, European Commission, private partners of the Joint Undertaking, and all of the communities that the JU serves. It is through our collective effort that we can navigate the complexities of the HPC landscape and drive meaningful progress.”
Duczmal added that after the successful inauguration of three pre-exascale machines (Leonardo, Marenostrum5 and LUMI, all of them involved in DestinE digital twins) EuroHPC JU is preparing for the exascale and what will come next, including quantum computing and the development of AI, and reminded that none of this progress would be possible without educated and talented people that are the greatest European asset. “Our partnership relations, or what I like to call the European way of solving challenges we face, that without neglect of excellence looks for an inclusive solution,” he said.
Anders Jensen, Executive Director of EuroHPC, welcomed the participants of a summit with a strong focus on skills as “they are the backbone of our sovereign European supercomputing ecosystem” and hoped for a successful week of exchanges to “reflect further on how to best develop Europe’s research agenda in applications and technologies to support our common ambition to strengthen the HPC and quantum ecosystem in Europe and thus lead innovation and support research globally.” Jensen also celebrated the good health of the EuroHPC JU with some of the most powerful supercomputers in the world, but also some of the greenest.
Attendees included members of the Governing board of the EuroHPC JU, EuroHPC hosting entities and the main HPC centres in Europe, policymakers, industry and entrepreneurship, experts, researchers and young talents to exchange on the fast-evolving European HPC landscape and the disruptive developments of artificial intelligence and machine learning.
The ECMWF delegation in Antwerp included Deputy Director-General Florian Pappenberger, Director for DestinE Irina Sandu, DestinE Digital Technology Lead Nils Wedi and DestinE Technology Partnership Lead Thomas Geenen, which participated in different panels and sessions.
The summit was also an occasion to exchange with ECMWF’s partners at the EuroHPC JU and the officials from the centres involved in DestinE-related activities on the efforts to deploy and run DestinE digital twins. These efforts have already allowed the teams involved to set up the complex digital twin workflows, and to successfully adapt, deploy and run the cutting-edge earth system models underpinning the digital twins on the brand-new Lumi and Leonardo machines with heterogenous architectures.
ECMWF is thus contributing ensuring EuroHPC systems can be exploited for running complex Earth system applications, transforming European investments in HPC into benefits for the society.
Artificial intelligence and supercomputers, the revolution for Earth system simulations
Director of Forecasts and Deputy Director-General at ECMWF Florian Pappenberg was invited to the session “Fostering European Leadership in AI” in which he highlighted that a system such as Destination Earth, aiming to simulate the behaviour not only of the weather but the whole Earth system at kilometre scales, is only made possible thanks to supercomputers. “Without EuroHPC JU, without machines like LUMI, Destination Earth and the digital twins wouldn’t exist,” he said.
Regarding the disruptive arrival of new artificial intelligence technologies, Florian Pappenberger explained that machine learning is taking weather and climate prediction to a new level: “We are doing better than 10 years ago in weather forecasts, some might laugh, but we do. And with AI we are going to a new revolution, towards predictability levels that we’ve never seen before, and this in the midst of a climate crisis that is bringing more and more extremes.
“Machine learning is allowing us to do things we didn’t imagine we could do before and achieving things we couldn’t, allowing us to make an unprecedented impact in informing users. That is how radical machine learning will be for Destination Earth, but all this cannot be done without the EuroHPC machines.”
Florian Pappenberger also told the audience how creating high-fidelity digital twins of the entire Earth system involves complex simulations and computation of earth system models that require immense computational resources to represent the dynamics of the physical system. “Supercomputers provide the necessary processing power, memory, and storage capabilities to simulate real-world phenomena with unprecedented detail and accuracy,” he said.
The panel, including Lucilla Sioli, Director for Artificial Intelligence and Digitial Industry at the EU Commission’s DG COMM, and Kimmo Koski, CEO of the CSC-IT Center for Science and Eugenio Monteleone from Igenius, as a representative of the private sector, discussed a number of pressing issues related to AI, including legislation, in which the EU is taking a global lead, ethics, infrastructure and R&D.
Kimmo Koski highlighted that HPC and AI ecosystems are also a matter of data, be it to train AI models or as a result of the simulations of the digital twins, raised the issue of connectivity, and agreed with the other guests about the need for competent people and skills.
AI and ML algorithms will enable users to analyse the vast amounts of data resulting from the simulations, detect patterns and predict behaviours as well as optimise the performance the simulations autonomously.
“AI-driven digital twins facilitate proactive decision-making, predictive maintenance, resource optimization, and scenario planning, thereby maximizing productivity and minimizing risks,” said Florian Pappenberger, also pointing out the importance of developing competences “we need brains, brains and brains. The EU needs to develop comprehensive educational curricula and training programs tailored to AI, HPC, and digital technologies, this is essential.” And insisted on the importance of inclusion. “Promoting diversity and inclusion in the AI, HPC, and digital workforce is essential for fostering innovation and addressing skill gaps. Encouraging participation from underrepresented groups, including women, minorities, and individuals from diverse socio-economic backgrounds, can bring new perspectives, talents, and ideas.”
Florian Pappenberger also pointed out that Europe has a great tradition in collaboration, of which DestinE is an example with around 100 public, private and academia centres involved alongside ECMWF, the European Space Agency (ESA) and EUMETSAT, and highlighting the European success story that is the EuroHPC JU. The recording is available on the EuroHPC Summit website.
Meeting the future HPC and AI talents
With the summit highly focused on education and talent, EuroHPC JU organised its first Career Fair. During the session, early career scientists could ‘speed-date’ with senior scientists from various organisations. Many of the aspirants are following the master programme sponsored by EuroHPC.
“It was really interesting to discuss with all these talented early career scientists about career perspectives,” said Irina Sandu, the Director of DestinE at ECMWF. “Many of them wondered about the convenience of following a PhD or go to the private sector after their master’s course. Of course, I told them why it is so exciting to work in an organisation such as ECMWF, where scientists with varied backgrounds come together for a common goal: delivering the best possible predictions and products to our Member and Cooperating States,” she said.
Demonstrating the power of DestinE digital twins
Digital Technology Lead for DestinE at ECMWF Nils Wedi participated at a plenary session devoted to the real-life uses of EuroHPC JU. “EuroHPC Users: How Are They Exploiting the Current EuroHPC Systems & Will Exploit Future Exascale Capabilities?” gathered experts from different disciplines that are applying HPC capabilities to language, astrophysics, nano-biotechnology or computer sciences.
Nils Wedi showcased DestinE results and explained how, in less than one year the ECMWF teams and their partners have set up two digital twin workflows on the LUMI supercomputer, reaching the maturity levels required to move into production at scale.
“Our partners from the Climate Change digital twin consortium are now running the first multi-decadal projections at a resolution of 5km with two models, IFS-NEMO and ICON, in the Climate DT on LUMI,” he said, before showing the audience the world’s first simulated satellite view animation of one possible future climate… in 2028!
The weather on 1 June 2027. Simulated infrared and visible spectrum imagery with the Climate Change Adaptation Digital Twin at 4.4km resolution on LUMI supercomputer. Credit: ECMWF/ Andreas Müller/ LUMI
The visualisation, including the visible and the infrared spectrum, was produced from the IFS-NEMO runs for 2028 at 5 km resolution. “These simulations will be able to provide information to users at the scales where the impacts of climate change and extreme weather are felt,” said Nils Wedi, citing the example of the renewable energy sector or the energy supply systems, that the consortium led by CSC who is developing the Climate DT is including since the early stages of the development to ensure the Climate DT responds to the needs of its future users.
The DestinE Digital Technology Lead also explained that such simulations are part of the first attempt to create an operational capability for multi-decadal climate projections. Illustrating why DestinE needs EuroHPC JU supercomputers, Nils Wedi explained that these simulations have a high computing cost of about 1.5 million GPU hours or 34 million CPU core hours despite the continuous efforts to optimise the models.
The presentations triggered a lively session of questions from the audience. The recording is available on the EuroHPC Summit website.
Nils Wedi also participated in an interview to be published soon by EuroHPC JU.
Technology challenges and opportunities in DestinE
ECMWF presence at the summit closed with a double participation of Thomas Geenen, Technology Partnership Lead for DestinE, in two parallel sessions, “Interconnecting EuroHPC Supercomputers for Scientific and Industrial Advancement” and “Destination Earth: Challenges of Implementing Digital Twins on EuroHPC HPC Systems”, organised by ETP4HPC. This industry-led association is partnering with ECMWF to make suggestions regarding technology challenges and needs. The event was an occasion to present the first results of the white papers with the main proposals, explained by their authors.
The session covered many angles of DestinE deployment such as federation of data and computing resources, data streaming on EuroHPC systems, cyber security or the most challenging mathematical methods and algorithms needed to run the digital twins on EuroHPC systems at scale. The white papers made recommendations on actions in the short, medium and long run.
Thomas Geenen also participated in the panel discussion of the session about the connectivity requirements between EuroHPC systems, a key topic for DestinE since the system needs to operate seamlessly between different systems with different protocols and distributed across Europe. Both sessions were held at the same time in different rooms. “I had to stretch a bit to be in both sessions but there was somehow a parallel between both, because the projects operating in EuropHPC systems share many of the challenges around operating services across distributed systems. It was very enlightening to hear the problems and the solutions of other systems and initiatives. The takeaway is that these challenges require a holistic approach,” he said.
The presentations are available on the EuroHPC Summit website.
The second EuroHPC summit closed with an AI Bootcamp on Leonardo HPC and a workshop for national competence centres and (NCCs) and EuroHPC Centres of Excellence (CoEs), showcasing the determination of EuroHPC to make high-performance computing flourish in Europe.
Destination Earth is a European Union funded initiative launched in 2022, with the aim to build a digital replica of the Earth system by 2030. The initiative is being jointly implemented by three entrusted entities: the European Centre for Medium-Range Weather Forecasts (ECMWF) responsible for the creation of the first two ‘digital twins’ and the ‘Digital Twin Engine’, the European Space Agency (ESA) responsible for building the ‘Core Service Platform’, and the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT), responsible for the creation of the ‘Data Lake’.
We acknowledge the EuroHPC Joint Undertaking for awarding this project strategic access to the EuroHPC supercomputers LUMI, hosted by CSC (Finland) and the LUMI consortium, Marenostrum5, hosted by BSC (Spain) Leonardo, hosted by Cineca (Italy) and MeluXina, hosted by LuxProvide (Luxembourg) through a EuroHPC Special Access call.
More information about Destination Earth is on the Destination Earth website and the EU Commission website.
For more information about ECMWF’s role visit ecmwf.int/DestinE
For any questions related to the role of ECMWF in Destination Earth, please use the following email links: