OBJECTIVES: External validation of health economic models involves comparing various outcomes across multiple sources while accounting for differences in populations and standards of care across study settings. These inherent challenges are exacerbated for models in complex chronic diseases, especially when data used to populate the model span a period with significant advances in care. This research describes the methods used and lessons learned during validation of a model developed to conduct population-level analyses for diabetic retinopathy (DR) screening.
METHODS: The external validation process consisted of four steps. First, in anticipation of validation scenarios, relationships between key patient characteristics (e.g., glycemic control, blood pressure) and DR progression were identified and included in model programming. Second, targeted cohort-level outcomes (e.g., incidence of DR, progression to blindness) were compared using model cohorts matched to cohorts described in published studies. Third, stability in population-level outcomes (e.g., population health state distribution) was assessed accounting for current population characteristics and the current standard of care. Fourth, informed by the results of this population-level comparison, selected parameters and assumptions were revisited prior to model finalization.
RESULTS: After adjusting for baseline characteristics, cohort-level model outcomes for DR incidence and progression over 4- and 10-year horizons were well-aligned with published outcomes. Initial population-level outcomes revealed an unexpected shift in the health state distribution over time which was traced to an imbalance in the baseline data for the health state distribution and DR progression rates. Ultimately, baseline patient characteristics were updated with data for a contemporary cohort, baseline progression rates were adjusted accordingly, and the baseline health state distribution was calibrated to generate the expected stability in population-level trends.
CONCLUSIONS: External validation of models for complex chronic diseases requires a systematic approach that integrates validation planning with model development and first targets narrow well-defined outcomes before considering broad interconnected outcomes.
