OBJECTIVES: The amount of missing data can impact the reliability and measurement error of clinical outcome assessment (COA) scores and ultimately reduce or inflate the statistical power in determining treatment efficacy. This simulation study examines the feasibility of using the COA scores’ measurement error or agreement coefficient to support a missing data scoring rule for developing clinical trial endpoints.
METHODS: One hundred datasets (each n = 200) with complete data were generated using 7 items with a 0-10 ordinal rating scale (a mixture of 4 normal and 3 skewed distributions). Partially complete datasets were generated under multiple missing-at-random (MAR) and missing-not-at-random (MNAR) patterns for 50% and 75% of subjects. For the 7-item mean two statistics were computed: the standard error of measurement (SEM; based on internal consistency reliability) of every complete dataset (SEM-complete) and of every partial dataset (SEM-partial); and the agreement (intraclass correlation coefficient [ICC]) between each pair of complete and partial datasets.
RESULTS: Overall, when subjects missed more items, the percentage of SEM-partial in the range of 0.9*SEM-complete to 1.1*SEM-complete consistently declined. For example, in one MNAR condition, this in-range percentage declined from 67% to 33% when the proportion of subjects missing all high item-level scores of 8 to 10 increased from 50% to 75%; in one MAR condition, this in-range percentage declined from 100% to 3.6% when the number of randomly missed items by 50% of subjects increased from 1 to 5. A declining trend was also observed for the percentage of ICC ≥ 0.81 but was not as pronounced as in SEM.
CONCLUSIONS: Conducting MAR and MNAR simulations based on complete data (e.g., baseline) and evaluating the impact on SEM and ICC in comparison to complete data can help establish a missing data rule for multiple-item or multiple-day COA composite scores.
