A day in the life of an ECMWF machine learning scientist 

Machine learning (ML) and artificial intelligence (AI) are becoming omnipresent also in weather and climate prediction. ECMWF is a leading centre implementing these new tools and techniques in Europe, and our scientists are increasingly required to explain their work with ML/ AI in workshops and seminars across Europe and beyond. Mariana Clare, one of the scientists representing ECMWF, gives us some insights into her work. 

On any given day, Mariana Clare will work alongside the growing ML team at ECMWF to improve the performance of the newly created AIFS, the ML-based version of its Integrated Forecasting System used to predict the weather and to exploring new techniques to apply to the ECMWF operational predictions or in the framework of Destination Earth (DestinE). Indeed, the use of artificial intelligence is also key in several areas of the Destination Earth initiative of the European Commission and will be ramped up in the second phase of the initiative which has just started. 

With a background in mathematics, Mariana specialises in the use of ML to quantify the degree of uncertainty of the forecast and simulations performed by ECMWF.  

But recently, fewer and fewer days are “normal” for Mariana. In addition to her work enhancing the use of these new techniques at ECMWF, she is increasingly being requested by organisations around Europe to explain the work of ECMWF with ML, in the middle of an ever-growing interest.  

“There is a certain hype about ML and AI. Of course, these new technologies are revolutionary and will change forever weather and climate prediction, but in the meantime, people tend to forget that these new ML-based models are largely based on the work carried out by many organisations like ECMWF over decades, with the output of traditional physics-based models being used to train the ML-based models. So, I am always happy to participate in the seminars and workshops we are being invited to, because it’s a great opportunity to separate the reality from the myth and explain the reach of ML in our daily work, that is really pushing the boundaries.”  

During Destination Earth’s system launch at the EuroHPC Joint Undertaking LUMI supercomputing center in Kaajani, Finland, Mariana Clare was invited to participate in a panel where the European Commission’s Executive Vice-President for a Europe Fit for the Digital Age Margrethe Vestager moderated a discussion with early career scientists from different horizons and backgrounds about European Leadership in Artificial intelligence.  

Vestager led a lively discussion looking at some of the hottest topics of the development of European ML and AI in Europe. The Executive Vice-President opened the discussion to summarise that multilateral cooperation, catalysed by the EuroHPC JU through initiatives like DestinE and the human factor, the highly skilled professionals working in this domain, are the key assets of Europe in the development of a strong ML ecosystem.

Simon Pickard, moderator of the launch ceremony and Network Director at Science | Business, Niclas Joswig from the University of Helsinki, Paulina Tedesco (Met Norway), Georgie Mercier (ESA Copernicus Team) and Piotr Wyrwinski, from the Poznan University of Technology shared the stage with Mariana during the closing panel of the DestinE launch event.  

With a background in mathematics and statistics, Mariana introduced her job helping train the ECMWF data-driven model AIFS. The objective of AIFS is to generate weather forecasts with the same or better accuracy as the current physics and observations-based model, but at an exponentially lower computing cost.  

Later during the panel Clare discussed the use of ML for uncertainty quantification, explaining that operational centres like ECMWF need to quantify as accurately as possible the level of uncertainty of weather forecasts by running the model with different scenarios. With traditional models this is costly and time-consuming, whereas with the efficiency and accuracy provided by AI models means that “we can afford to run potentially thousands of forecast scenarios, which is invaluable when it comes to the forecasting of extreme events and understanding the risk likelihoods. If you have an ensemble of scenarios for a full earth system this means you can understand the whole trajectory from the initial storm event all the way to the flooding and destruction impact at people’s doors.”  Using ML-based models for uncertainty quantification is also very important in DestinE, where the km-scale scale simulations are even more computationally expensive than today’s weather predictions and climate projections performed at lower resolutions. 

Mariana also explained that this approach can also be powerful when it comes to multi-decadal climate projections, but that ML-driven climate models are still in the early days of development. One of the activities carried out by ECMWF in Destination Earth is to develop an AI-based climate emulator via an invitation to tender. 

Another application of ML in DestinE (also via an invitation to tender) is the development of a weather and climate web-based Chatbot that will ease the use of DestinE data by a growing number of users. 

“This enhanced interactivity will make our data and our services much more accessible to not only academic researchers but also policymakers and local authorities. This in turn will increase the use of systems like DestinE and enable more feedback from real users, hopefully creating an open line of communications between developers and users, and a positive feedback cycle.” 

Mariana concluded her contribution to the panel by advocating for an ethical and trustworthy development of AI to ensure that society benefits from these revolutionary technologies. “In addition to this race for forecast accuracy and new functionality, we also need to make sure from the beginning that these developments are ethical, with open models, open data and equal access which is key for transparency and so trustworthiness.” 

The high-level event organised by the European Commission’s DG CNECT at the LUMI supercomputer center gathered more than a hundred policymakers, stakeholders, scientists and users to mark the start of the second phase of Destination Earth (2024-2026) after the initial implementation phase (2022-2024). 

 Watch again the panel discussion on YouTube.  

Shortly after the launch in Finland, Mariana Clare was invited to participate in a discussion at the Supercomputing Day in Luxembourg. The conference organised by Supercomputing Luxembourg and Luxembourg’s National Competence Centre delved into the perspectives of high-performance computing both for Academia and the industry.  

Mariana Clare participated in the session “AI Acceleration with HP. From Theory to Practice,” moderated by Valentin Plugaru Chief Technology Officer at LuxProvide, that hosts MeLuxina, one of the EuroHPC JU supercomputers, a valuable resource in Destination Earth. 

Asked about the use of AI at ECMWF, Clare explained how AI is revolutionising the organisation with over 20 people working on the AIFS and more members joining in the future, also in the framework of Destination Earth. “I don’t think anyone can yet see where the limits are. Our AIFS model is already comparable if not better in some metrics to our traditional physics-based atmospheric weather model.” But she also reminded the audience that providing reliable weather forecasts also needs Earth system information. “We are also developing ML-based sea ice, ocean and land components within the framework of Destination Earth.” said Mariana.  

The scientist also commented that a great advantage of AI-based models is the flexibility to be easily deployed in different HPC systems, compared to traditional physics-based models where there’s a complex adaptation and development process in order to be adapted to a new HPC infrastructure. 

Panellists discussed some of the challenges of the AI deployment in HPC systems like the development of codes for specific GPUs or some limitations to cloud-based operations, as well as dark data (data on HPCs taking up space and not being used), data openness and the importance of training people to the new skills demanded by the development of AI on HPC machines.  

“Both events were independent and totally unrelated, but they were somehow complementary, it was very enriching to participate in both because I could engage with very different types of audiences. In Kaajani we met scientists, policymakers and users and the Supercomputing Day in Luxembourg gathered specialists from the HPC ecosystem, and of course the cooperation between all these fields is what makes an initiative like Destination Earth possible. Every time I participate in these events I am facing a huge interest in AI, and often some misconceptions, so it is very important to go out there and explain what we do and what AI can and cannot solve.”  

Destination Earth is a European Union-funded initiative launched in 2022, with the aim to build a digital replica of the Earth system by 2030. The initiative is being jointly implemented under the leadership of DG CNECT by three entrusted entities: the European Centre for Medium-Range Weather Forecasts (ECMWF), responsible for the creation of the first two ‘digital twins’ and the ‘Digital Twin Engine’, the European Space Agency (ESA) responsible for building the ‘Core Service Platform’, and the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT), responsible for the creation of the ‘Data Lake’.  

We acknowledge the EuroHPC Joint Undertaking for awarding this project strategic access to the EuroHPC supercomputers LUMI, hosted by CSC (Finland), and the LUMI consortium, Marenostrum5, hosted by BSC (Spain) Leonardo, hosted by Cineca (Italy) and MeluXina, hosted by LuxProvide (Luxembourg) through a EuroHPC Special Access call.   

More information about Destination Earth is on the Destination Earth website and the EU Commission website.    

For more information about ECMWF’s role visit ecmwf.int/DestinE     

For any questions related to the role of ECMWF in Destination Earth, please use the following email links:     

General enquiries  

Press and Communications enquiries