Restricted accessResearch articleFirst published online 2020-5
Properties and pitfalls of weighting as an alternative to multilevel multiple imputation in cluster randomized trials with missing binary outcomes under covariate-dependent missingness
The generalized estimating equation (GEE) approach can be used to analyze cluster randomized trial data to obtain population-averaged intervention effects. However, most cluster randomized trials have some missing outcome data and a GEE analysis of available data may be biased when outcome data are not missing completely at random. Although multilevel multiple imputation for GEE (MMI-GEE) has been widely used, alternative approaches such as weighted GEE are less common in practice. Using both simulations and a real data example, we evaluate the performance of inverse probability weighted GEE vs. MMI-GEE for binary outcomes. Simulated data are generated assuming a covariate-dependent missing data pattern across a range of missingness clustering (from none to high), where all covariates are measured at baseline and are fully observed (i.e. a type of missing-at-random mechanism). Two types of weights are estimated and used in the weighted GEE: (1) assuming no clustering of missingness (W-GEE) and (2) accounting for such clustering (CW-GEE). Results show that, even in settings with high missingness clustering, CW-GEE can lead to more bias and lower coverage than W-GEE, whereas W-GEE and MMI-GEE provide comparable results. W-GEE should be considered a viable strategy to account for missing outcomes in cluster randomized trials.
BrookerSOkelloGNjagiK, et al.Improving educational achievement and anaemia of school children: design of a cluster randomised trial of school-based malaria prevention and enhanced literacy instruction in Kenya. Trials2010; 11: 93.
4.
HudgensMGHalloranME. Toward causal inference with interference. J Am Stat Assoc2008; 103: 832–842.
5.
CampbellMKPiaggioGElbourneDR, et al.Consort 2010 statement: extension to cluster randomised trials. BMJ2012; 345: e5661.
6.
FieroMHHuangSOrenE, et al.Statistical analysis and handling of missing data in cluster randomized trials: a systematic review. Trials2016; 17: 72.
7.
Turner EL, Prague MP, Gallis JA, et al. Review of recent methodological developments in group-randomized trials: part 2-analysis. Am J Public Health 2017; 107: 1078–1086.
8.
PreisserJSYoungMLZaccaroDJ, et al.An integrated population-averaged approach to the design, analysis and sample size determination of cluster-unit trials. Stat Med2003; 22: 1235–1254.
9.
TurnerELLiFGallisJA, et al.Review of recent methodological developments in group-randomized trials: part 1-design. Am J Public Health2017; 107: 907–915.
10.
Diaz-OrdazKKenwardMGCohenA, et al.Are missing data adequately handled in cluster randomised trials? A systematic review and guidelines. Clin Trials2014; 11: 590–600.
RobinsJMRotnitzkyAZhaoLP. Analysis of semiparametric regression-models for repeated outcomes in the presence of missing data. J Am Stat Assoc1995; 90: 106–121.
13.
PaikMC. The generalized estimating equation approach when data are not missing completely at random. J Am Stat Assoc1997; 92: 1320–1329.
14.
SchaferJLYucelRM. Computational strategies for multivariate linear mixed-effects models with missing values. J Comput Graph Stat2002; 11: 437–457.
15.
SeamanSRWhiteIRCopasAJ, et al.Combining multiple imputation and inverse-probability weighting. Biometrics2012; 68: 129–137.
16.
HunsbergerSMurrayDDavisCE, et al.Imputation strategies for missing data in a school-based multi-centre study: the Pathways study. Stat Med2001; 20: 305–316.
17.
TaljaardMDonnerAKlarN. Imputation strategies for missing continuous outcomes in cluster randomized trials. Biometrical J2008; 50: 329–345.
18.
AndridgeRR. Quantifying the impact of fixed effects modeling of clusters in multiple imputation for cluster randomized trials. Biometrical J2011; 53: 57–74.
19.
HossainADiaz-OrdazKBartlettJW. Missing continuous outcomes under covariate dependent missingness in cluster randomised trials. Stat Methods Med Res2016; 26: 1543–1562.
20.
CailleALeyratCGiraudeauB. A comparison of imputation strategies in cluster randomized trials with missing binary outcomes. Stat Methods Med Res2016; 25: 2650–2669.
21.
MaJAkhtar-DaneshNDolovichL, et al.Imputation strategies for missing binary outcomes in cluster randomized trials. BMC Med Res Methodol2011; 11: 18.
22.
MaJRainaPBeyeneJ, et al.Comparison of population-averaged and cluster-specific models for the analysis of cluster randomized trials with missing binary outcomes: a simulation study. BMC Med Res Methodol2013; 13: 9.
23.
MaJRainaPBeyeneJ, et al.Comparing the performance of different multiple imputation strategies for missing binary outcomes in cluster randomized trials: a simulation study. J Open Access Med Stat2012; 2: 93–103.
24.
HossainADiazOrdazKBartlettJW. Missing binary outcomes under covariate-dependent missingness in cluster randomised trials. Stat Med2017; 36: 3092–3109.
25.
Molenberghs G and Kenward M. Missing data in clinical studies. Chichester: John Wiley & Sons, 2007.
26.
Van Buuren S. Flexible imputation of missing data. Boca Raton: Chapman and Hall/CRC, 2018.
27.
VansteelandtSCarpenterJKenwardMG. Analysis of incomplete data using inverse probability weighting and doubly robust estimators. Methodology2010; 6: 37–48.
28.
SalazarAOjedaBDuenasM, et al.Simple generalized estimating equations (GEEs) and weighted generalized estimating equations (WGEEs) in longitudinal studies with dropouts: guidelines and implementation in R. Stat Med2016; 35: 3424–3448.
29.
PreisserJSLohmanKKRathouzPJ. Performance of weighted estimating equations for longitudinal binary data with drop-outs missing at random. Stat Med2002; 21: 3035–3054.
30.
InanGYucelR. Joint GEEs for multivariate correlated data with incomplete binary outcomes. J Appl Stat2017; 44: 1920–1937.
31.
SkinnerCJD'ArrigoJ. Inverse probability weighting for clustered nonresponse. Biometrika2011; 98: 953–966.
32.
JukesMCHTurnerELDubeckMM, et al.Improving literacy instruction in Kenya through teacher professional development and text messages support: a cluster randomized trial. J Res Educ Eff2016; 10: 449–481.
33.
HanleyJANegassaAEdwardesMD, et al.Statistical analysis of correlated data using generalized estimating equations: an orientation. Am J Epidemiol2003; 157: 364–375.
34.
ZegerSLLiangKY. Longitudinal data analysis for discrete and continuous outcomes. Biometrics1986; 42: 121–130.
35.
NeuhausJMKalbfleischJDHauckWW. A comparison of cluster-specific and population-averaged approaches for analyzing correlated binary data. Int Stat Rev1991; 59: 25–35.
36.
LiangKYZegerSL. Longitudinal data-analysis using generalized linear-models. Biometrika1986; 73: 13–22.
37.
MengX-L. Multiple-imputation inferences with uncongenial sources of input. Stat Sci1994; 9: 538–558.
38.
RubinDB. Multiple imputation for nonresponse in surveys, Hoboken: Wiley-Interscience, 2004.
39.
MolenberghsGFitzmauriceGKenwardMG, et al.Handbook of missing data methodology, Hoboken: Chapman and Hall/CRC, 2014.
40.
BelitserSVMartensEPPestmanWR, et al.Measuring balance and model selection in propensity score methods. Pharmacoepidemiol Drug Saf2011; 20: 1115–1129.
41.
PragueMWangRStephensA, et al.Accounting for interactions and complex inter-subject dependency in estimating treatment effect in cluster-randomized trials with missing outcomes. Biometrics2016; 72: 1066–1077.
42.
WangCLPaikMC. A weighting approach for GEE analysis with missing data. Commun Stat Theory Methods2011; 40: 2397–23411.
43.
SeamanSRWhiteIR. Review of inverse probability weighting for dealing with missing data. Stat Methods Med Res2013; 22: 278–295.
44.
McDanielLSHendersonNCRathouzPJ. Fast pure R implementation of GEE: application of the Matrix package. R J2013; 5: 181–187.
45.
BarnardJRubinDB. Miscellanea. Small-sample degrees of freedom with multiple imputation. Biometrika1999; 86: 948–955.
46.
Quartagno M and Carpenter J. jomo: a package for multilevel joint modelling multiple imputation. R package version 2.6-7, http://CRAN.R-project.org/package=jomo, 2017.
47.
EldridgeSMUkoumunneOCCarlinJB. The intra-cluster correlation coefficient in cluster randomized trials: a review of definitions. Int Stat Rev2009; 77: 378–394.
48.
ManclLADeRouenTA. A covariance estimator for GEE with improved small-sample properties. Biometrics2001; 57: 126–134.
49.
KauermannGCarrollRJ. A note on the efficiency of sandwich covariance matrix estimation. J Am Stat Assoc2001; 96: 1387–1396.
50.
FayMPGraubardBI. Small-sample adjustments for Wald-type tests using sandwich estimators. Biometrics2001; 57: 1198–1206.
51.
LiFForbesABTurnerEL, et al.Power and sample size requirements for GEE analyses of cluster randomized crossover trials. Stat Med2018; 38: 636–649.
52.
LiFTurnerELHeagertyPJ, et al.An evaluation of constrained randomization for the design and analysis of group-randomized trials with binary outcomes. Stat Med2017; 36: 3791–3806.
53.
LiFTurnerELPreisserJS. Sample size determination for GEE analyses of stepped wedge cluster randomized trials. Biometrics2018; 74: 1450–1458.
54.
SantosCASFiacconeRLOliveiraNF, et al.Estimating adjusted prevalence ratio in clustered cross-sectional epidemiological data. BMC Med Res Methodol2008; 8: 80.
55.
WilliamsonEJForbesAWhiteIR. Variance reduction in randomised trials by inverse probability weighting using the propensity score. Stat Med2014; 33: 721–737.
56.
CarpenterJRKenwardMGVansteelandtS. A comparison of multiple imputation and doubly robust estimation for analyses with missing data. J Roy Stat Soc Sta2006; 169: 571–584.
57.
PragueMWangRDe GruttolaV. CRTgeeDR: an R package for doubly robust generalized estimating equations estimations in cluster randomized trials with missing data, Cambridge: Harvard University Working Paper, 2016.
58.
StephensAJTchetgenEJTDe GruttolaV. Augmented generalized estimating equations for improving efficiency and validity of estimation in cluster randomized trials by leveraging cluster-level and individual-level covariates. Stat Med2012; 31: 915–930.
59.
BirhanuTMolenberghsGSottoC, et al.Doubly robust and multiple-imputation-based generalized estimating equations. J Biopharm Stat2011; 21: 202–225.
60.
LiFThomasLELiF. Addressing extreme propensity scores via the overlap weights. Am J Epidemiol2018; 188: 250–27.
61.
SunBTchetgen TchetgenEJ. On inverse probability weighting for nonmonotone missing at random data. J Am Stat Assoc2018; 113: 369–379.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
