OBJECTIVE: Network meta-analysis (NMA) is a valuable tool for synthesis of results from different studies, allowing an understanding of the relative efficacy of multiple interventions. However, between-study heterogeneity and reporting bias can limit the validity of results obtained. This research extends recent proposed methods of addressing the difficulties presented by heterogeneity in meta-analysis, to assess robustness of findings in NMA.
METHODS: Data came from recently published Cochrane systematic reviews investigating the effect of atypical antipsychotics in schizophrenia patients. Positive and Negative Syndrome Scale (PANSS) total score endpoint data for trials included in seven direct treatment comparisons (aripiprazole vs. clozapine, quetiapine, risperidone, olanzapine, and ziprasidone; risperidone vs. quetiapine, and olanzapine) were extracted and synthesized using Bayesian random-effects NMA. Relative efficacy was measured using mean difference (MD) in average PANSS total score. NMA results of all trials were compared to NMA results when (1) the single most precise trial was used for each direct comparison, (2) analyses were restricted to the largest trials, (3) a meta-regression model was used to allow effect size to depend on its standard error, and (4) analyses were restricted to trials at a low risk of attrition bias.
RESULTS: Pair-wise effect sizes between antipsychotics, probabilities of being the best antipsychotic, and ranking of antipsychotics differed between analysis strategies. Differences in results for comparisons with strategies (1) and (2) favored clozapine; however, MDs were not statistically significant. Furthermore, differences were small, with all differences in MD (∆MD) less than 4, which is unlikely to correspond to a clinically significant difference. No meaningful differences in effect sizes were observed in the comparison with strategies (3) and (4) (|∆MD|<1).
CONCLUSIONS: Relative efficacy estimates differ depending on the strategy used; however, in this example such differences were small. The study illustrates how confirmatory sensitivity techniques can be used to help validate NMA conclusions.
