Google and ECMWF release WeatherBench 2, a benchmark for developing AI models for weather forecasting.
As in many other fields, artificial intelligence is transforming weather forecasting, in many cases already enabling much more accurate predictions. To accelerate progress, Google has partnered with ECMWF (European Centre for Medium-Range Weather Forecasts) to release WeatherBench 2, an open benchmark for evaluating AI models for weather forecasting.
WeatherBench 2 aims to support AI development in weather forecasting
In the 70 years since the first computerized weather forecasts, traditional models have gained about one day of predictability per decade, according to Google. Machine learning promises faster progress by learning patterns from weather data rather than relying solely on human-developed, physics-based models.
However, evaluating and comparing AI models used for weather forecasting has been difficult due to the complexity of weather, with many variables such as temperature, wind, and precipitation, the company said. WeatherBench 2 aims to provide a standardized way to evaluate a new generation of AI-based forecasting models. According to Google, the benchmark includes a common framework with metrics suitable for different forecasting purposes and fair comparisons across many different methods.
These include a neural graph network, Google DeepMind's GraphCast, and Huawei's Pangu-Weather, a transform-based AI model, as well as forecasts from ECMWF's high-resolution and ensemble forecasting systems, which Google says are among the best traditional weather forecasting models.
WeatherBench 2 provides a framework for distributed computing
WeatherBench 2 is an update to the first version of the benchmark, released in 2020, which was based on the first low-resolution AI models.
The key feature of WeatherBench 2 is an open source framework for evaluating weather forecasts. Due to the large amount of high-resolution weather data, the evaluation can be extremely computationally intensive. Therefore, the evaluation tool is built on Apache Beam, which enables distributed computation by breaking tasks into manageable parts.
More information is available on the official WeatherBench 2 website and on GitHub.