TY - JOUR
T1 - Data Imbalance, Uncertainty Quantification, and Transfer Learning in Data-Driven Parameterizations
T2 - Lessons From the Emulation of Gravity Wave Momentum Transport in WACCM
AU - Sun, Y. Qiang
AU - Pahlavan, Hamid A.
AU - Chattopadhyay, Ashesh
AU - Hassanzadeh, Pedram
AU - Lubis, Sandro W.
AU - Alexander, M. Joan
AU - Gerber, Edwin P.
AU - Sheshadri, Aditi
AU - Guan, Yifei
N1 - Publisher Copyright:
© 2024 The Author(s). Journal of Advances in Modeling Earth Systems published by Wiley Periodicals LLC on behalf of American Geophysical Union.
PY - 2024/7
Y1 - 2024/7
N2 - Neural networks (NNs) are increasingly used for data-driven subgrid-scale parameterizations in weather and climate models. While NNs are powerful tools for learning complex non-linear relationships from data, there are several challenges in using them for parameterizations. Three of these challenges are (a) data imbalance related to learning rare, often large-amplitude, samples; (b) uncertainty quantification (UQ) of the predictions to provide an accuracy indicator; and (c) generalization to other climates, for example, those with different radiative forcings. Here, we examine the performance of methods for addressing these challenges using NN-based emulators of the Whole Atmosphere Community Climate Model (WACCM) physics-based gravity wave (GW) parameterizations as a test case. WACCM has complex, state-of-the-art parameterizations for orography-, convection-, and front-driven GWs. Convection- and orography-driven GWs have significant data imbalance due to the absence of convection or orography in most grid points. We address data imbalance using resampling and/or weighted loss functions, enabling the successful emulation of parameterizations for all three sources. We demonstrate that three UQ methods (Bayesian NNs, variational auto-encoders, and dropouts) provide ensemble spreads that correspond to accuracy during testing, offering criteria for identifying when an NN gives inaccurate predictions. Finally, we show that the accuracy of these NNs decreases for a warmer climate (4 × CO2). However, their performance is significantly improved by applying transfer learning, for example, re-training only one layer using ∼1% new data from the warmer climate. The findings of this study offer insights for developing reliable and generalizable data-driven parameterizations for various processes, including (but not limited to) GWs.
AB - Neural networks (NNs) are increasingly used for data-driven subgrid-scale parameterizations in weather and climate models. While NNs are powerful tools for learning complex non-linear relationships from data, there are several challenges in using them for parameterizations. Three of these challenges are (a) data imbalance related to learning rare, often large-amplitude, samples; (b) uncertainty quantification (UQ) of the predictions to provide an accuracy indicator; and (c) generalization to other climates, for example, those with different radiative forcings. Here, we examine the performance of methods for addressing these challenges using NN-based emulators of the Whole Atmosphere Community Climate Model (WACCM) physics-based gravity wave (GW) parameterizations as a test case. WACCM has complex, state-of-the-art parameterizations for orography-, convection-, and front-driven GWs. Convection- and orography-driven GWs have significant data imbalance due to the absence of convection or orography in most grid points. We address data imbalance using resampling and/or weighted loss functions, enabling the successful emulation of parameterizations for all three sources. We demonstrate that three UQ methods (Bayesian NNs, variational auto-encoders, and dropouts) provide ensemble spreads that correspond to accuracy during testing, offering criteria for identifying when an NN gives inaccurate predictions. Finally, we show that the accuracy of these NNs decreases for a warmer climate (4 × CO2). However, their performance is significantly improved by applying transfer learning, for example, re-training only one layer using ∼1% new data from the warmer climate. The findings of this study offer insights for developing reliable and generalizable data-driven parameterizations for various processes, including (but not limited to) GWs.
KW - data imbalance
KW - data-driven parameterizations
KW - generalization via transfer learning
KW - gravity wave momentum transport
KW - uncertainty quantification
UR - http://www.scopus.com/inward/record.url?scp=85199864659&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85199864659&partnerID=8YFLogxK
U2 - 10.1029/2023MS004145
DO - 10.1029/2023MS004145
M3 - Article
AN - SCOPUS:85199864659
SN - 1942-2466
VL - 16
JO - Journal of Advances in Modeling Earth Systems
JF - Journal of Advances in Modeling Earth Systems
IS - 7
M1 - e2023MS004145
ER -