Why AI Weather Models Fall Short for Extreme Events: A New Study

A recent study published in Science Advances delivers a cautionary message for the rapid adoption of artificial intelligence (AI) in weather forecasting: when it comes to record-breaking extreme weather events, traditional physics-based models still outperform their AI counterparts. While AI has made impressive strides in routine forecasting, the analysis shows that these models significantly underestimate both the frequency and intensity of rare, extreme phenomena like heatwaves, cold snaps, and severe storms.

Extreme weather events cause hundreds of billions of dollars in damages annually, destroying crops, damaging infrastructure, and claiming lives. Governments around the world rely on early warning systems to prepare communities and deploy disaster response teams—systems that have proven effective at minimizing harm. For decades, these systems have been built on numerical weather prediction models, which use complex equations based on fundamental physics to simulate atmospheric and oceanic processes. These are known as physics-based models.

However, in recent years, AI-based climate models have emerged as a faster, less computationally expensive alternative. Instead of solving physics equations, they learn patterns from vast datasets of historical weather observations. This approach has achieved remarkable success in many forecasting tasks, sometimes even surpassing traditional models. Yet, as the new research highlights, AI models have a critical weakness: they struggle to predict events that lie outside the range of their training data—exactly the kind of rare, record-breaking extremes that matter most.

AI Weather Forecasting: The Rise and Limitations

AI weather models leverage machine learning to analyze historical weather data. They are trained on decades of records, learning statistical relationships that allow them to make forecasts quickly and with relatively low computing power. This efficiency is a major advantage over physics-based models, which must run thousands of equations for each forecast. Moreover, many AI models have been shown to outperform traditional ones in tasks like predicting temperature, precipitation, and wind patterns for common weather scenarios.

But this strength is also a weakness. AI models are fundamentally constrained by the data they are trained on. As study author Prof. Sebastian Engelke, a professor at the University of Geneva’s research institute for statistics and information science, told Carbon Brief, AI models “depend strongly on the training data” and are “relatively constrained to the range of this dataset.” In other words, they can only forecast weather patterns similar to those they have seen before. When a truly unprecedented event occurs—one that breaks historical records—the AI has no analogous example to draw from, leading to inaccurate predictions.

Traditional Physics-Based Models: Still the Gold Standard for Extremes

Traditional physics-based weather models, in contrast, are built on universal physical laws. They simulate the behavior of the atmosphere and ocean using equations that govern fluid dynamics, thermodynamics, and radiation. These models do not rely on historical data to make predictions; instead, they calculate outcomes from first principles. This makes them inherently more capable of handling novel conditions, including record-breaking extremes. While they require significant computational resources, their underlying physics provides a robust foundation for forecasting events that have never been observed before.

The new study directly compared the performance of both types of models in forecasting thousands of record-breaking hot, cold, and windy events recorded in 2018 and 2020. The results were clear: physics-based models consistently outperformed AI models in both the frequency and intensity of these extreme events. The AI models systematically underestimated how often such records occurred and how severe they were.

Study Reveals Gaps in Predicting Record-Breaking Weather

Methodology and Key Findings

The researchers tested how well both AI and traditional weather models could simulate extreme events from two specific years. They examined thousands of records of temperature and wind extremes. The key findings include:

• Underestimation of frequency: AI models predicted fewer record-breaking events than actually occurred, missing many of the most extreme cases.
• Underestimation of intensity: Even when AI models did predict an extreme event, they often forecasted a milder version than what was observed.
• Better performance of physics-based models: Traditional models showed much higher accuracy for these record-breaking events, correctly capturing both the occurrence and severity.

These limitations stem from a fundamental issue known as “distributional shift”—the training data for AI models represents the climate of the past, but as the climate changes, new extremes emerge that fall outside that historical distribution. Traditional physics models, by simulating the actual physical processes, are less affected by this shift.

Implications for Weather Prediction

The study’s authors caution against replacing physics-based models with AI models too quickly. As Prof. Engelke put it, the analysis is a “warning shot” for the meteorological community. While AI offers speed and efficiency for routine forecasts, it should not be relied upon for extreme event prediction without careful consideration.

This does not mean AI has no role in weather forecasting. Hybrid approaches that combine the strengths of both methods are already being explored. For instance, AI can be used to post-process physics-based model outputs, improving resolution or correcting biases. But for now, when a record-breaking heatwave or storm is on the horizon, traditional models remain the more reliable tool.

The findings also have broader implications for climate adaptation and disaster preparedness. As extreme weather becomes more frequent and intense due to climate change, accurate early warnings are increasingly critical. Relying solely on AI models could lead to underprepared communities and greater damage. Policymakers and forecasters should ensure that their early warning systems are built on the most robust science available—including, but not limited to, the power of AI.

In conclusion, while AI weather models have revolutionized many aspects of forecasting, they still have a significant blind spot when it comes to record-breaking extremes. Traditional physics-based models, though more computationally demanding, offer a more reliable foundation for predicting the rare but devastating events that matter most. As the study makes clear, the road to replacing traditional models with AI is longer than some may think, and caution is warranted.