Assessment of Model Performance and Predictive Skill in High-Resolution Ocean and Sea Ice Forecasting Systems

Dmitry Dukhovskoy

NOAA Physical Sciences Laboratory

Tuesday, Mar 11, 2025, 2:00 pm MT
DSRC Room GC402

JOIN THE WEBINAR:

Abstract

Significant efforts have been made in recent years to assess the prediction skills and limits of predictability in high-resolution ocean and sea ice forecasting systems. Various skill assessment metrics provide quantitative measures of model accuracy and forecasting performance. The choice of metric depends on the goals of the assessment and the phenomena being forecasted. In many cases, quantitative validation methods use integrated (or aggregated) scalar fields, such as spatially averaged sea surface temperature or sea ice extent. These methods offer a quick and convenient way to quantify model performance. However, they have limited applicability when the spatial structure of the ocean (e.g., ocean fronts) or sea ice fields (e.g., ice edge location and shape) is crucial.

This presentation will explore several quantitative methods for evaluating and assessing the skill of ocean and sea ice models based on spatial patterns of forecasted characteristics, using examples from high-resolution data-assimilative and free-running modeling systems. The application of these techniques for assessing model predictability and forecasting skills will be demonstrated using results from observing system simulation experiments in the Gulf of America (Gulf of Mexico). This study was conducted under the Understanding Gulf Ocean Systems (UGOS) research initiative Phase 1 (UGOS-1), which focuses on improving the understanding of physical processes that control Loop Current dynamics through advanced data-assimilative modeling and analysis. One key objective of the study was to examine the limits of predictability of the Loop Current (LC) system using multi-model forward simulations. The presentation will highlight results from numerical experiments using a high-resolution (2.5 km) regional data-assimilative system to assess the impact of various types of observations on forecast skill in the Gulf of America (Gulf of America). Model predictability is assessed using a saturated error growth model. The estimated predictability of the LC system ranges from 2 to 3 months, with the predictability limit depending on the LC's activity state; shorter predictability limits are observed during active LC configurations. Assimilation of subsurface temperature and salinity profiles in the LC region significantly impacts medium-range forecasts (2–3 months), while the effect on shorter-term forecasts is less prominent. Forecast errors are primarily influenced by the uncertainty in the initial state, and thus the accuracy of the initial conditions. Forecasts with smaller initial errors exhibit the best predictive skill, with reliable predictions extending beyond 2 months. This suggests that model error has less influence than initial error during this time period. Therefore, substantial improvements in forecast accuracy, up to 3 months, can be achieved by increasing the accuracy of initialization.

Bio: Dmitry Dukhovskoy is a Research Physical Scientist at NOAA's Physical Sciences Laboratory in the Atmosphere-Ocean Processes and Predictability Division with expertise in numerical modeling for geophysical applications and data analysis. Throughout his career, Dmitry has worked with both coupled and stand-alone models for ocean, sea ice, and waves, applying them to a variety of regional and global contexts. His research interests include model skill assessment, and he has co-led several projects focused on evaluating ocean, sea ice, and oil spill forecasts, such as OSE/OSSE, with an emphasis on short- and medium-range prediction accuracy. Before joining the PSL, Dmitry worked at NOAA EMC, where he contributed to improving the data assimilation component of the NOAA global operational ocean-ice forecasting system (RTOFS). At PSL, his research continues to focus on numerical simulations and predictions, particularly in the Northeast Pacific. Dmitry holds a B.S., M.S., and Ph.D. in Physical Oceanography, and an M.S. in Applied and Computational Mathematics.