Much like a doctor tries to diagnose the cause of a patient’s symptoms, a team of NOAA-funded researchers has been helping to diagnose the causes behind poor weather and climate model forecasts. Now, thanks to the team’s new model evaluation software, scientists can more quickly and easily identify the source of model errors to accelerate improvements and help Americans better plan and respond.
Developed by the NOAA Climate Program Office’s Model Diagnostics Task Force (MDTF), the software package was recently transferred to NOAA’s Geophysical Fluid Dynamics Laboratory (GFDL) where it has already led to significant advancements in model performance, including forecasts of regional precipitation and extreme events like monsoons.
“Many long-standing model biases have been cut roughly by half,” said David Neelin, UCLA Professor of Atmospheric and Oceanic Sciences and MDTF Lead, about the new software. “These improvements are critical for advancing a number of NOAA priorities.”
Diving deeper into weather and climate model errors
Americans are increasingly seeking accurate weather and climate forecasts from NOAA. But to improve model errors and provide more reliable forecasts, scientists can’t just fix temperature or precipitation—there’s no single knob or equation in the model controlling each variable. Model developers need to go deeper and address issues with underlying physical and chemical processes in the models that drive these errors.
With this in mind, the NOAA Climate Program Office’s Modeling, Analysis, Predictions, and Projections (MAPP) program, in partnership with GFDL, created the Model Diagnostic Task Force (MDTF). The Task Force brings together researchers and scientists from universities, national modeling centers, and research laboratories to focus on “diagnostics” that evaluate or diagnose the error source in different atmospheric, land, and ocean processes in models.
Creating the MDTF was motivated by the 2012 Working Group on Numerical Experimentation (WGNE) Task Force, which advocated for a diagnostic framework focused on the Madden Julian Oscillation—an eastward moving disturbance of clouds and rain that circles the tropics every 30-60 days. Since 2015, the MDTF has expanded on that concept and has been developing software that collects diagnostics for many key weather and climate model processes—called Process Oriented Diagnostics (PODs)—produced by researchers at universities and modeling centers, into a central framework that can be readily used as a model evaluation tool.
The image illustrates multiple weather climate and model process areas included in the new Model Diagnostic Task Force software package.
“The idea is to work closely with modeling centers to come up with a way to enable the modeling community outside of the centers […] to easily contribute their diagnostics,” said Eric Maloney, Colorado State University Professor and MDTF Co-lead, during a 2017 interview with NOAA. “Then, modeling centers will be able to apply the diagnostics to their simulations to hopefully lead to fantastic modeling improvements in an accelerated fashion.”
Now at GFDL, the package is also moving to an open development framework on GitHub to encourage diagnostic contributions from the public, facilitate the exchange of ideas between modeling centers and the scientific community, and make it easier for researchers to integrate their work. By engaging the modeling community through a collaborative approach, the package could help quicken the transition of research innovations into advances in life-saving forecasts and projections.
“Ultimately, 10 years down the line, I think that model diagnosis work […] may be some of my most important contributions,” Maloney said.
An evolving evaluation software for more accurate forecasts and projections
The MDTF’s evolving software package has eight different diagnostics implemented, 11 under development to be included by 2022, and 28 proposed which may be implemented by 2024. Current diagnostics help tackle things like atmospheric convection, clouds, and radiative processes, while near-term diagnostics will cover tropical and extratropical cyclones and phenomena like the El Niño-Southern Oscillation. In the longer-term, future diagnostics will address key processes that drive our climate’s sensitivity to greenhouse gas emissions and other processes critical to modeling marine ecosystem variability and change.
“A lot of the diagnostics are aligned with key NOAA priorities—high impact water events, extreme weather events, weather and climate events that have high societal and economic impacts,” said John Krasting, who now oversees the development of the package at GFDL. “This package allows users of our models and model developers to explore NOAA mission-relevant processes and identify how we’re doing and ways we can improve.”
Development of the software framework for process-oriented diagnostics was most recently supported by the National Oceanic and Atmospheric Administration (NOAA) Climate Program Office Modeling, Analysis, Predictions and Projections (MAPP) Program (grant # NA18OAR4310280). Additional support was provided by University of California Los Angeles, the Geophysical Fluid Dynamics Laboratory, the National Center for Atmospheric Research, Colorado State University, Lawrence Livermore National Laboratory and the US Department of Energy.
All of the process-oriented diagnostics modules (PODs) were contributed by members of the NOAA Model Diagnostics Task Force with MAPP support.