Multiple imputation (MI) can be used to address missing data at Level 2 in multilevel research. In this article, we compare joint modeling (JM) and the fully conditional specification (FCS) of MI as well as different strategies for including auxiliary variables at Level 1 using either their manifest or their latent cluster means. We show with theoretical arguments and computer simulations that (a) an FCS approach that uses latent cluster means is comparable to JM and (b) using manifest cluster means provides similar results except in relatively extreme cases with unbalanced data. We outline a computational procedure for including latent cluster means in an FCS approach using plausible values and provide an example using data from the Programme for International Student Assessment 2012 study.
AllisonP. D. (2001). Missing data. Thousand Oaks, CA: Sage.
2.
AndridgeR. R. (2011). Quantifying the impact of fixed effects modeling of clusters in multiple imputation for cluster randomized trials. Biometrical Journal, 53, 57–74. doi:10.1002/bimj.201000140
3.
AndridgeR. R.ThompsonK. J. (2015). Using the fraction of missing information to identify auxiliary variables for imputation procedures via proxy pattern-mixture models. International Statistical Review, 83, 472–492. doi:10.1111/insr.12091
4.
AsparouhovT.MuthénB. O. (2006). Constructing covariates in multilevel regression(Mplus Web Notes No. 11). Retrieved fromhttp://www.statmodel.com/
5.
AsparouhovT.MuthénB. O. (2010). Multiple imputation with Mplus(Technical Appendix). Retrieved fromhttp://statmodel.com/
6.
BarnardJ.McCullochR.MengX.-L. (2000). Modeling covariance matrices in terms of standard deviations and correlations, with application to shrinkage. Statistica Sinica, 10, 1281–1312. Retrieved fromhttp://www.stat.sinica.edu.tw/statistica/
7.
BlackA. C.HarelO.McCoachD. B. (2011). Missing data techniques for multilevel data: Implications of model misspecification. Journal of Applied Statistics, 38, 1845–1865. doi:10.1080/02664763.2010.529882
8.
BlackwellM.HonakerJ.KingG. (2017). A unified approach to measurement error and missing data: Overview and applications. Sociological Methods & Research, 46, 303–341. doi:10.1177/0049124115585360
9.
BodnerT. E. (2008). What improves with increased missing data imputations?Structural Equation Modeling: A Multidisciplinary Journal, 15, 651–675. doi:10.1080/10705510802339072
10.
CarpenterJ. R.KenwardM. G. (2013). Multiple imputation and its application. Hoboken, NJ: Wiley.
11.
CheungM. W.-L. (2007). Comparison of methods of handling missing time-invariant covariates in latent growth models under the assumption of missing completely at random. Organizational Research Methods, 10, 609–634. doi:10.1177/1094428106295499
12.
CronbachL. J. (1976). Research on classrooms and schools: Formulation of questions, design and analysis. Stanford, CA: Stanford Evaluation Consortium. Retrieved fromhttp://eric.ed.gov/
13.
CroonM. A.van VeldhovenM. J. P. M. (2007). Predicting group-level outcome variables from variables measured at the individual level: A latent variable multilevel model. Psychological Methods, 12, 45–57. doi:10.1037/1082-989X.12.1.45
14.
de LeeuwJ.KreftI. G. G. (1995). Questioning multilevel models. Journal of Educational and Behavioral Statistics, 20, 171–189. doi:10.3102/10769986020002171
15.
DemirtasH.FreelsS. A.YucelR. M. (2008). Plausibility of multivariate normality assumption when multiply imputing non-Gaussian continuous outcomes: A simulation assessment. Journal of Statistical Computation and Simulation, 78, 69–84. doi:10.1080/10629360600903866
16.
DrechslerJ. (2015). Multiple imputation of multilevel missing data—Rigor versus simplicity. Journal of Educational and Behavioral Statistics, 40, 69–95. doi:10.3102/1076998614563393
17.
EndersC. K. (2010). Applied missing data analysis. New York, NY: Guilford Press.
18.
EndersC. K.MistlerS. A.KellerB. T. (2016). Multilevel multiple imputation: A review and evaluation of joint modeling and chained equations imputation. Psychological Methods, 21, 222–240. doi:10.1037/met0000063
19.
GelmanA. (2006). Prior distributions for variance parameters in hierarchical models. Bayesian Analysis, 1, 515–534. Retrieved fromhttp://projecteuclid.org
20.
GelmanA.CarlinJ. B.SternH. S.DunsonD.VehtariA.RubinD. B. (2014). Bayesian data analysis (3rd ed.). Boca Raton, FL: CRC press.
21.
GibsonN. M.OlejnikS. (2003). Treatment of missing data at the second level of hierarchical linear models. Educational and Psychological Measurement, 63, 204–238. doi:10.1177/0013164402250987
22.
GoldsteinH.CarpenterJ. R.KenwardM. G.LevinK. A. (2009). Multilevel models with multivariate mixed response types. Statistical Modelling, 9, 173–197. doi:10.1177/1471082X0800900301
23.
GrahamJ. W.OlchowskiA. E.GilreathT. D. (2007). How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prevention Science, 8, 206–213. doi:10.1007/s11121-007-0070-9
24.
GrilliL.RampichiniC. (2011). The role of sample cluster means in multilevel models: A view on endogeneity and measurement error issues. Methodology, 7, 121–133. doi:10.1027/1614-2241/a000030
25.
GrundS.LüdtkeO.RobitzschA. (2016). Multiple imputation of missing covariate values in multilevel models with random slopes: A cautionary note. Behavior Research Methods, 48, 640–649. doi:10.3758/s13428-015-0590-3
26.
GrundS.LüdtkeO.RobitzschA.(in press). Multiple imputation of missing data for multilevel models: Simulations and recommendations. Organizational Research Methods. doi:10.1177/1094428117703686
27.
HeY.RaghunathanT. E. (2006). Tukey’s gh distribution for multiple imputation. The American Statistician, 60, 251–256. doi:10.1198/000313006X126819
28.
HoxJ. J.van BuurenS.JolaniS. (2016). Incomplete multilevel data. In HarringJ.StapletonL. M.BeretvasS. N. (Eds.), Advances in multilevel modeling for educational research: Addressing practical issues found in real-world applications (pp. 39–61). Charlotte, NC: Information Age Publishing.
29.
HughesR. A.WhiteI. R.SeamanS. R.CarpenterJ. R.TillingK.SterneJ. A. (2014). Joint modelling rationale for chained equations. BMC Medical Research Methodology, 14, 1–10. doi:10.1186/1471-2288-14-28
30.
LairdN. M.WareJ. H. (1982). Random-effects models for longitudinal data. Biometrics, 38, 963–974.
31.
LittleR. J. A.RubinD. B. (2002). Statistical analysis with missing data (2nd ed.). Hoboken, NJ: Wiley.
32.
LiuH.ZhangZ.GrimmK. J. (2015). Comparison of inverse Wishart and separation-strategy priors for Bayesian estimation of covariance parameter matrix in growth curve analysis. Structural Equation Modeling: A Multidisciplinary Journal, 1–14. doi:10.1080/10705511.2015.1057285
33.
LüdtkeO.MarshH. W.RobitzschA.TrautweinU.AsparouhovT.MuthénB. O. (2008). The multilevel latent covariate model: A new, more reliable approach to group-level effects in contextual studies. Psychological Methods, 13, 203–229. doi:10.1037/a0012869
34.
LüdtkeO.RobitzschA.GrundS. (2017). Multiple imputation of missing data in multilevel designs: A comparison of different strategies. Psychological Methods, 22, 141–165. doi:10.1037/met0000096
35.
McNeishD. M. (2016). Using data-dependent priors to mitigate small sample bias in latent growth models: A discussion and illustration using Mplus. Journal of Educational and Behavioral Statistics, 41, 27–56. doi:10.3102/1076998615621299
36.
MengX.-L. (1994). Multiple-imputation inferences with uncongenial sources of input. Statistical Science, 9, 538–558. doi:10.1214/ss/1177010269
37.
MislevyR. J. (1991). Randomization-based inference about latent variables from complex samples. Psychometrika, 56, 177–196. doi:10.1007/BF02294457
38.
MistlerS. A. (2015). Multilevel multiple imputation: An examination of competing methods(Doctoral dissertation). Retrieved fromhttp://repository.asu.edu/
39.
MorrisC. N. (1983). Parametric empirical Bayes inference: Theory and applications. Journal of the American Statistical Association, 78, 47. doi:10.2307/2287098
40.
MuthénB. O. (1990). Mean and covariance structure analysis of hierarchical data. Paper presented at the meeting of the Psychometric Society, Princeton, NJ. Retrieved fromhttps://www.statmodel.com
41.
MuthénL. K.MuthénB. O. (2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Muthén & Muthén.
42.
Organisation for Economic Co-operation and Development. (2014). PISA 2012 technical report. Paris, France: Author.
43.
PreacherK. J.ZyphurM. J.ZhangZ. (2010). A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods, 15, 209–233. doi:10.1037/a0020141
44.
QuartagnoM.CarpenterJ. R. (2016). jomo: A package for multilevel joint modelling multiple imputation[Version 2.3-1]. Retrieved fromhttp://CRAN.R-project.org/package=jomo
45.
RaghunathanT. E.LepkowskiJ. M.van HoewykJ.SolenbergerP. (2001). A multivariate technique for multiply imputing missing values using a sequence of regression models. Survey Methodology, 27, 85–96. Retrieved fromhttp://www.statcan.gc.ca/
46.
RaudenbushS. W.BrykA. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.
47.
Resche-RigonM.WhiteI. R.(in press). Multiple imputation by chained equations for systematically and sporadically missing multilevel data. Statistical Methods in Medical Research. doi:10.1177/0962280216666564
48.
RobitzschA.GrundS.HenkeT. (2017). miceadds: Some additional multiple imputation functions, especially for mice[Version 2.3-0]. Retrieved fromhttp://CRAN.R-project.org/package=miceadds
49.
RoystonP. (2005). Multiple imputation of missing values: Update. The Stata Journal, 5, 188–201. Retrieved fromhttp://www.stata-journal.com
50.
RoystonP.WhiteI. R. (2011). Multiple imputation by chained equations (MICE): Implementation in Stata. Journal of Statistical Software, 45, 1–20.
51.
RubinD. B. (1987). Multiple imputation for nonresponse in surveys. Hoboken, NJ: Wiley.
52.
RubinD. B. (2003). Nested multiple imputation of NMES via partially incompatible MCMC. Statistica Neerlandica, 57, 3–18. doi:10.1111/1467-9574.00217
53.
RustK. (2013). Sampling, weighting, and variance estimation in international large-scale assessments. In RutkowskiL.von DavierM.RutkowskiD. (Eds.), Handbook of international large-scale assessment: Background, technical issues, and methods of data analysis (pp. 117–153). Boca Raton, FL: CRC Press.
54.
RutkowskiL.GonzalezE.JoncasM.von DavierM. (2010). International large-scale assessment data: Issues in secondary analysis and reporting. Educational Researcher, 39, 142–151. doi:10.3102/0013189X10363170
55.
SchaferJ. L. (1997). Analysis of incomplete multivariate data. Boca Raton, FL: CRC Press.
56.
SchaferJ. L.GrahamJ. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7, 147–177. doi:10.1037//1082-989X.7.2.147
57.
SchaferJ. L.OlsenM. K. (1998). Multiple imputation for multivariate missing-data problems: A data analyst’s perspective. Multivariate Behavioral Research, 33, 545–571. doi:10.1207/s15327906mbr3304_5
58.
SchaferJ. L.YucelR. M. (2002). Computational strategies for multivariate linear mixed-effects models with missing values. Journal of Computational and Graphical Statistics, 11, 437–457. doi:10.1198/106186002760180608
59.
SchuurmanN. K.GrasmanR. P. P. P.HamakerE. L. (2016). A comparison of inverse-Wishart prior specifications for covariance matrices in multilevel autoregressive models. Multivariate Behavioral Research, 51, 185–206. doi:10.1080/00273171.2015.1065398
60.
ShinY.RaudenbushS. W. (2010). A latent cluster-mean approach to the contextual effects model with missing data. Journal of Educational and Behavioral Statistics, 35, 26–53. doi:10.3102/1076998609345252
SnijdersT. A. B.BoskerR. J. (2012). Multilevel analysis: An introduction to basic and advanced multilevel modeling. Thousand Oaks, CA: Sage.
63.
TaljaardM.DonnerA.KlarN. (2008). Imputation strategies for missing continuous outcomes in cluster randomized trials. Biometrical Journal, 50, 329–345. doi:10.1002/bimj.200710423
64.
van BuurenS. (2011). Multiple imputation of multilevel data. In HoxJ. J. (Ed.), Handbook of advanced multilevel analysis (pp. 173–196). New York, NY: Routledge.
65.
van BuurenS. (2012). Flexible imputation of missing data. Boca Raton, FL: CRC Press.
66.
van BuurenS.BrandJ. P. L.Groothuis-OudshoornC. G. M.RubinD. B. (2006). Fully conditional specification in multivariate imputation. Journal of Statistical Computation and Simulation, 76, 1049–1064. doi:10.1080/10629360600810434
67.
van BuurenS.Groothuis-OudshoornK. (2011). MICE: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45, 1–67. doi:10.18637/jss.v045.i03
68.
von HippelP. T. (2013). Should a normal imputation model be modified to impute skewed variables?Sociological Methods & Research, 42, 105–138. doi:10.1177/0049124112464866
69.
WeirichS.HaagN.HechtM.BöhmeK.SiegleT.LüdtkeO. (2014). Nested multiple imputation in large-scale assessments. Large-scale Assessments in Education, 2, 1–18. doi:10.1186/s40536-014-0009-0
70.
YangJ. S.SeltzerM. (2016). Handling measurement error in predictors using a multilevel latent variable plausible values approach. In HarringJ.StapletonL. M.BeretvasS. N. (Eds.), Advances in multilevel modeling for educational research: Addressing practical issues found in real-world applications. Charlotte, NC: Information Age Publishing.
71.
YuanK.-H.HayashiK. (2005). On Muthén’s maximum likelihood for two-level covariance structure models. Psychometrika, 70, 147–167. doi:10.1007/s11336-003-1070-8
72.
YucelR. M.SchenkerN.RaghunathanT. E. (2007). Sequential hierarchical regression imputation (SHRIMP). Unpublished Manuscript, University of Massachusetts, Amherst, MA.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
