The precision of estimates of treatment effects in multilevel experiments depends on the sample sizes chosen at each level. It is often desirable to choose sample sizes at each level to obtain the smallest variance for a fixed total cost, that is, to obtain optimal sample allocation. This article extends previous results on optimal allocation to four-level cluster randomized designs and randomized block designs. It also introduces the idea of constrained optimal allocation, where the sample size at one or more levels is fixed by considerations other than cost or sampling variation. Explicit formulas are given for constrained optimal allocation in three- and four-level designs.
BergerM. P. F.WongW. K. (2009). An introduction to optimal designs for social and biomedical research. Chichester, England: John Wiley.
2.
BloomH. S.Richburg-HayesL.BlackA. R. (2007). Using covariates to improve precision: Empirical guidelines for studies that randomize schools to measure the impacts of educational interventions. Educational Evaluation and Policy Analysis, 29, 30–59.
3.
GlazermanS.IsenbergE.DolfinS.BleekerM.JohnsonA.GriderM.JacobusM. (2010). Impacts of comprehensive teacher induction: Final results from a randomized controlled study (NCEE 2010-4027). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.
4.
HedgesL. V.HedbergE. C. (2007). Intraclass correlation values for planning group randomized trials in education. Educational Evaluation and Policy Analysis, 29, 60–87.
5.
HedgesL. V.HedbergE. C. (2014). Intraclass correlations and covariate outcome correlations for planning 2 and 3 level cluster randomized experiments in education. Evaluation Review, 38, 1–45.
6.
JacobR.ZhuP.BloomH. (2010). New empirical evidence for the design of group randomized trials in education. Journal of Research on Educational Effectiveness, 3, 157–198.
7.
KonstantopoulosS. (2009). Incorporating cost in power analysis for three-level cluster randomized designs. Evaluation Review, 33, 335–357.
8.
KonstantopoulosS. (2011). Optimal sampling of units in three-level cluster randomized designs: An ANCOVA framework. Educational and Psychological Measurement, 71, 798–813.
9.
KorendijkE. J. H.MoerbeekM.MaasC. J. M. (2010). The robustness of designs for trials with nested data against incorrect initial intracluster correlation coefficient estimates. Journal of Educational and Behavioral Statistics, 35, 566–585.
10.
LiuX. (2003). Statistical power and optimum sample allocation ratio for treatment and control having unequal costs per unit of randomization. Journal of Educational and Behavioral Statistics, 28, 231–248.
11.
MoerbeekM.van BreukelenG.BergerM. (2000). Design issues for experiments in multilevel populations. Journal of Educational and Behavioral Statistics, 25, 271–284.
12.
MurrayD. M.HannanP. J.BakerW. L. (1996). A Monte Carlo study of alternative responses to intraclass correlation in community trials. Evaluation Review, 20, 313–337.
13.
RaudenbushS. W. (1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2, 173–185.
14.
RaudenbushS. W.LiuX. (2000). Statistical power and optimal design for multisite randomized trials. Psychological Methods, 5, 199–213.
15.
SchochetP. Z. (2008). Statistical power for random assignment evaluations of educational programs. Journal of Educational and Behavioral Statistics, 33, 62–87.
16.
SpybrookJ.RaudenbushS. W. (2009). An examination of the precision and technical accuracy of the first wave of group randomized trials funded by the Institute of Education Sciences. Educational Evaluation and Policy Analysis,31, 298–318.
17.
WeissM. J.BloomH. S.BrockT. (2013). A conceptual framework for studying the sources of variation in program effects. New York, NY: MDRC.
