An Open Framework for Process-Oriented Diagnostics of Earth System Models (Type II proposal)

Problem addressed and rationale: Process-oriented diagnostics (PODs) aim to characterize physical processes in a manner that relates directly to mechanisms essential to their simulation, providing guidance for improvement of a climate/weather model or assessment of its ability to address a specific research question. The predecessor Team project (PIs of which form part of the current team) advanced an initial bare-bones framework into a community-based software framework that brings process-oriented diagnostics into the diagnostic suite for modeling centers at the Geophysical Fluid Dynamics Laboratory (GFDL) and the National Center for Atmospheric Research (NCAR). Experience with the recent development suggests multiple areas where refinement and expansion would be beneficial for both modeling centers and POD developers.

Work Summary: The proposed work will build on the previous Model Diagnostics Task Force (MDTF) framework and coordinate with the Type I individual proposals to expand the open frame-
work to entrain PODs developed by multiple research teams into the development stream of the modeling centers. The framework developed over the previous two phases specifies POD protocols
for the target model version and the comparison to observations and permits diagnostics to be placed in a multi-model context using results from the CMIP6 archive. The Type II framework team main-
tains consistency with the previous lead team while expanding the coordination with the diagnostic streams at GFDL and NCAR, and formalizing common standards with the Department of Energy
(DOE) Coordinated Model Evaluation Capabilities (CMEC) effort to further coordinate US-based model evaluation efforts. The proposed work will include the following elements. (1) The current
Team project has established GitHub-based documentation, setup and configuration protocols. With the successful incorporation of a substantial number of PODs, developments on the software side
proposed for the next phase expand on this process, emphasizing maintainability, interoperability portability, provenance and usability. (2) PODs targeting related phenomena and on similar time-
scales will be identified and grouped to coordinate development teams and assist navigation of the results. This organization will also help model developers assess the output frequency requirements
for PODs targeting phenomena on different climate timescales. (3) A task force will be led by the Type II team, modeled on the current MDTF with regular teleconferences, facilitated scientific con-
ference sessions, and coordinated publications. New community-building activities planned by the Type II team include “Developer days” to facilitate communication between climate model and POD
developers and tutorials to familiarize diagnostic developers with coding best practices in the context of the framework. (4) The Type II team will explore ways to include mean-state and variability
diagnostics as context for PODs. Both GFDL and NCAR have expressed the need to modernize their legacy diagnostic suites and the MDTF Framework can help prevent redundancy. Enhancements to
the framework will be implemented to handle common functions, such as atmospheric pressure-level sub-setting and ocean depth range integrals. (5) Similar to the previous Team Proposal, the team will
develop tools and additional prototype PODs in key areas.

Relevance to competition: This proposal addresses the call for the “Modeling, Analysis, Predictions, and Projections (MAPP) Competition 2: Process-Oriented Diagnostics for NOAA Climate Model
Improvement and Applications” for a Type II proposal that advances the model diagnostics software package led by the MDTF and a synergetic process for integrating results of individual projects on process-oriented diagnostics. It proposes infrastructure for code and data sharing that engages researchers in model evaluation and facilitates integration of their research products into the diagnostics packages used by modeling centers, as well as dissemination of this information. It addresses NOAA’s long-term climate goals by strengthening foundational capabilities, combining observations with modeling and prediction, and communication of scientific understanding.