Sage Journals: Discover world-class research

Abstract

When a multisite randomized trial reveals between-site variation in program impact, methods are needed for further investigating heterogeneous mediation mechanisms across the sites. We conceptualize and identify a joint distribution of site-specific direct and indirect effects under the potential outcomes framework. A method-of-moments procedure incorporating ratio-of-mediator-probability weighting (RMPW) consistently estimates the causal parameters. This strategy conveniently relaxes the assumption of no Treatment × Mediator interaction while greatly simplifying the outcome model specification without invoking strong distributional assumptions. We derive asymptotic standard errors that reflect the sampling variability of the estimated weight. We also offer an easy-to-use R package, MultisiteMediation , that implements the proposed method. It is freely available at the Comprehensive R Archive Network (http://cran.r-project.org/web/packages/MultisiteMediation).

Keywords

asymptotic variance direct effect indirect effect multilevel modeling multisite randomized trials method of moments RMPW

Get full access to this article

View all access options for this article.

References

Albert

J. M.

Nelson

(2011). Generalized causal mediation analysis. Biometrics, 67, 1028–1038.

Avin

Shpitser

Pearl

(2005). Identifiability of path-specific effects. Los Angeles: Department of Statistics, UCLA.

Bates

Maechler

Bolker

Walker

(2014). lme4: Linear mixed-effects models using Eigen and s4 (R package version 1.1-7). Retrievable from https://cran.r-project.org/web/packages/lme4/index.html.

Bauer

D. J.

Preacher

K. J.

Gil

K. M.

(2006). Conceptualizing and testing random indirect effects and moderated mediation in multilevel models: New procedures and recommendations. Psychological Methods, 11, 142.

Bein

Deutsch

Porter

Qin

Yang

Hong

(2015). Technical report on two-step estimation in RMPW analysis. Oakland, CA: MDRC.

Bind

M.-A.

Vanderweele

Coull

Schwartz

(2016). Causal mediation analysis for longitudinal data with exogenous exposure. Biostatistics, 17, 122–134.

Bloom

Hill

C. J.

Riccio

(2005). Modeling cross-site experimental differences to find out why program effectiveness varies. In Bloom

H. S.

(Ed.), Learning more from social experiments: Evolving analytic approaches (pp. 37–74). New York, NY: Russell Sage Foundation.

Diggle

Heagerty

Liang

K.-Y.

Zeger

(2002). Analysis of longitudinal data. Oxford, England: Oxford University Press.

Fitzmaurice

G. M.

Lipsitz

S. R.

Ibrahim

J. G.

(2007). A note on permutation tests for variance components in multilevel generalized linear mixed models. Biometrics, 63, 942–946.

10.

Flores

C. A.

Flores-Lagunes

(2013). Partial identification of local average treatment effects with an invalid instrument. Journal of Business & Economic Statistics, 31, 534–545.

11.

Goldstein

(2011). Multilevel statistical models (Vol. 922). Chichester, England: John Wiley.

12.

Hansen

L. P.

(1982). Large sample properties of generalized method of moments estimators. Econometrica: Journal of the Econometric Society, 50, 1029–1054.

13.

Hedeker

Gibbons

R. D.

(2006). Longitudinal data analysis (Vol. 451). Hoboken, NJ: John Wiley.

14.

Hirano

Imbens

G. W.

(2001). Estimation of causal effects using propensity score weighting: An application to data on right heart catheterization. Health Services and Outcomes Research Methodology, 2, 259–278.

15.

Hong

(2010). Ratio of mediator probability weighting for estimating natural direct and indirect effects. Proceedings of the American Statistical Association, Biometrics Section (pp. 2401–2415). Alexandria, VA: American Statistical Association.

16.

Hong

(2015). Causality in a social world: Moderation, mediation and spill-over. West Sussex, England: John Wiley.

17.

Hong

Deutsch

Hill

H. D.

(2011). Parametric and non-parametric weighting methods for estimating mediation effects: An application to the national evaluation of welfare-to-work strategies. In Proceedings of the American Statistical Association, Social Statistics Section (pp. 3215–3229). Alexandria, VA: American Statistical Association.

18.

Hong

Deutsch

Hill

H. D.

(2015). Ratio-of-mediator-probability weighting for causal mediation analysis in the presence of treatment-by-mediator interaction. Journal of Educational and Behavioral Statistics, 40, 307–340.

19.

Hong

Nomi

(2012). Weighting methods for assessing policy effects mediated by peer change. Journal of Research on Educational Effectiveness, 5, 261–289.

20.

Hong

Raudenbush

S. W.

(2006). Evaluating kindergarten retention policy: A case study of causal inference for multilevel observational data. Journal of the American Statistical Association, 101, 901–910.

21.

Horvitz

D. G.

Thompson

D. J.

(1952). A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association, 47, 663–685.

22.

Huber

(2014). Identifying causal mechanisms (primarily) based on inverse probability weighting. Journal of Applied Econometrics, 29, 920–943.

23.

Hudgens

M. G.

Halloran

M. E.

(2008). Toward causal inference with interference. Journal of the American Statistical Association, 103, 832–842.

24.

Imai

Keele

Tingley

(2010). A general approach to causal mediation analysis. Psychological Methods, 15, 309.

25.

Imai

Keele

Yamamoto

(2010). Identification, inference and sensitivity analysis for causal mediation effects. Statistical Science, 25, 51–71.

26.

Imai

Yamamoto

(2013). Identification and sensitivity analysis for multiple causal mechanisms: Revisiting evidence from framing experiments. Political Analysis, 21, 141–171.

27.

Johnson

Gritz

Jackson

Burghardt

Boussy

Leonard

Orians

(1999). National job corps study: Report on the process analysis (Research and Evaluation Report Series 8140-510). Princeton, NJ: Mathematica Policy Research.

28.

Judd

C. M.

Kenny

D. A.

(1981). Process analysis estimating mediation in treatment evaluations. Evaluation Review, 5, 602–619.

29.

Kenny

D. A.

Korchmaros

J. D.

Bolger

(2003). Lower level mediation in multilevel models. Psychological Methods, 8, 115.

30.

Kling

J. R.

Liebman

J. B.

Katz

L. F.

(2007). Experimental analysis of neighborhood effects. Econometrica, 75, 83–119.

31.

Krull

J. L.

MacKinnon

D. P.

(2001). Multilevel modeling of individual and group level mediated effects. Multivariate Behavioral Research, 36, 249–277.

32.

Lange

Rasmussen

Thygesen

(2014). Assessing natural direct and indirect effects through multiple pathways. American Journal of Epidemiology, 179, 513.

33.

Lange

Vansteelandt

Bekaert

(2012). A simple unified approach for estimating natural direct and indirect effects. American Journal of Epidemiology, 176, 190–195.

34.

Leite

W. L.

Jimenez

Kaya

Stapleton

L. M.

MacInnes

J. W.

Sandbach

(2015). An evaluation of weighting methods based on propensity scores to reduce selection bias in multilevel observational studies. Multivariate Behavioral Research, 50, 265–284.

35.

MacKinnon

D. P.

Dwyer

J. H.

(1993). Estimating mediated effects in prevention studies. Evaluation Review, 17, 144–158.

36.

Newey

W. K.

(1984). A method of moments interpretation of sequential estimators. Economics Letters, 14, 201–206.

37.

Neyman

Iwaszkiewicz

(1935). Statistical problems in agricultural experimentation. Supplement to the Journal of the Royal Statistical Society, 2, 107–180.

38.

Pearl

(2001). Direct and indirect effects. In Breese

Koller

(Eds.), Proceedings of the seventeenth conference on uncertainty in artificial intelligence (pp. 411–420). San Francisco, CA: Morgan Kaufmann.

39.

Preacher

K. J.

Zyphur

M. J.

Zhang

(2010). A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods, 15, 209.

40.

Raudenbush

S. W.

Bloom

(2015). Using multi-site randomized trials to learn about and from a distribution of program impacts. American Journal of Evaluation, 36, 475–499.

41.

Raudenbush

S. W.

Reardon

S. F.

Nomi

(2012). Statistical analysis for multisite trials using instrumental variables with random coefficients. Journal of Research on Educational Effectiveness, 5, 303–332.

42.

Reardon

S. F.

Raudenbush

S. W.

(2013). Under what assumptions do site-by-treatment instruments identify average causal effects? Sociological Methods & Research. doi:10.1177/0049124113494575

43.

Robins

J. M.

(2000). Marginal structural models versus structural nested models as tools for causal inference. In Halloran

M. E.

Berry

(Eds.), Statistical models in epidemiology, the environment, and clinical trials (pp. 95–133). New York, NY: Springer.

44.

Robins

J. M.

Greenland

(1992). Identifiability and exchangeability for direct and indirect effects. Epidemiology, 3, 143–155.

45.

Rosenbaum

P. R.

(1987). Model-based direct adjustment. Journal of the American Statistical Association, 82, 387–394.

46.

Rubin

D. B.

(1978). Bayesian inference for causal effects: The role of randomization. The Annals of Statistics, 6, 34–58.

47.

Rubin

D. B.

(1980). Randomization analysis of experimental data: The fisher randomization test comment. Journal of the American Statistical Association, 75, 591–593.

48.

Rubin

D. B.

(1986). Statistics and causal inference: Comment: Which ifs have causal answers. Journal of the American Statistical Association, 81, 961–962.

49.

Rubin

D. B.

(1990). Formal mode of statistical inference for causal effects. Journal of Statistical Planning and Inference, 25, 279–292.

50.

Seltzer

J. A.

(1994). Consequences of marital dissolution for children. Annual Review of Sociology, 20, 235–266.

51.

Spybrook

Raudenbush

S. W.

(2009). An examination of the precision and technical accuracy of the first wave of group-randomized trials funded by the institute of education sciences. Educational Evaluation and Policy Analysis, 31, 298–318.

52.

Tchetgen Tchetgen

E. J.

(2013). Inverse odds ratio-weighted estimation for causal mediation analysis. Statistics in Medicine, 32, 4567–4580.

53.

Tchetgen Tchetgen

E. J.

Shpitser

(2012). Semiparametric theory for causal mediation analysis: Efficiency bounds, multiple robustness, and sensitivity analysis. Annals of Statistics, 40, 1816.

54.

VanderWeele

T. J.

(2010a). Bias formulas for sensitivity analysis for direct and indirect effects. Epidemiology (Cambridge, Mass.), 21, 540.

55.

VanderWeele

T. J.

(2010b). Direct and indirect effects for neighborhood-based clustered and longitudinal data. Sociological Methods & Research, 38, 515–544.

56.

VanderWeele

T. J.

Chiba

(2014). Sensitivity analysis for direct and indirect effects in the presence of exposure-induced mediator–outcome confounders. Epidemiology, Biostatistics, and Public Health, 11, 1–16.

57.

Vanderweele

T. J.

Hong

Jones

S. M.

Brown

J. L.

(2013). Mediation and spillover effects in group-randomized trials: A case study of the 4rs educational intervention. Journal of the American Statistical Association, 108, 469–482.

58.

VanderWeele

T. J.

Vansteelandt

(2010). Odds ratios for mediation analysis for a dichotomous outcome. American Journal of Epidemiology, 172, 1339–1348.

59.

Weiss

M. J.

Bloom

H. S.

Brock

(2014). A conceptual framework for studying the sources of variation in program effects. Journal of Policy Analysis and Management, 33, 778–808.

60.

Zhang

Zyphur

M. J.

Preacher

K. J.

(2009). Testing multilevel mediation using hierarchical linear models problems and solutions. Organizational Research Methods, 12, 695–719.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.23 MB

0.00 MB