Reporting standards and critical appraisal tools serve as beacons for researchers, reviewers, and research consumers. Parallel to existing guidelines for researchers to report and evaluate group-comparison studies, single-case experimental (SCE) researchers are in need of guidelines for reporting and evaluating SCE studies. A systematic search was conducted for quality standards (QSs) for reporting and/or evaluating SCE studies. In total, 11 unique QSs were retrieved. In this article, we discuss the extent to which there is agreement with regard to randomization and data analysis of SCE studies among the 11 proposed sets of QSs. We provide recommendations regarding the inclusion of a randomization and data-analysis standard.
AllisonD. B.GormanB. S. (1993). Calculating effect sizes for meta-analysis: The case of the single case. Behaviour Research and Therapy, 31, 621–631. doi:10.1016/0005-7967(93)90115-B
2.
American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author.
3.
BarlowD. H.NockM. K.HersenM. (2009). Single case experimental designs: Strategies for studying behavior change (3rd ed.). Boston, MA: Allyn & Bacon.
4.
BultéI.OnghenaP. (2008). An R package for single-case randomization tests. Behavior Research Methods, 40, 467–478. doi:10.3758/BRM.40.2.467
5.
BultéI.OnghenaP. (2009). Randomization tests for multiple baseline designs: An extension of the SCRT-R package. Behavior Research Methods, 41, 477–485. doi:10.3758/BRM.41.2.477
6.
BuskP. L.SerlinR. C. (1992). Meta-analysis for single-case research. In KratochwillT. R.LevinJ. R. (Eds.), Single-case research design and analysis: New directions for psychology and education (pp. 187–212). Hillsdale, NJ: Lawrence Erlbaum.
7.
BusseR. T.KratochwillT. R.ElliottS. N. (1995). Meta-analysis for single-case consultation outcomes: Applications to research and practice. Journal of School Psychology, 33, 269–285. doi:10.1016/0022-4405(95)00014-D
8.
CenterB. A.SkibaR. J.CaseyA. (1985–1986). A methodology for the quantitative synthesis of intra-subject design research. The Journal of Special Education, 19, 387–400. doi:10.1177/002246698501900404
9.
CrosbieJ. (1993). Interrupted time-series analysis with brief single-subject data. Journal of Consulting and Clinical Psychology, 61, 966–974. doi:10.1037/0022-006X.61.6.966
10.
Des JarlaisD. C.LylesC.CrepazN. (2004). Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: The TREND statement. American Journal of Public Health, 94, 361–366. doi:10.2105/AJPH.94.3.361
11.
EdgingtonE. S. (1980). Overcoming obstacles to single-subject experimentation. Journal of Educational Statistics, 5, 261–267. doi:10.3102/10769986005003261
12.
EdgingtonE. S. (1996). Randomized single-subject experimental designs. Behaviour Research and Therapy, 34, 567–574. doi:10.1016/0005-7967(96)00012-5
FerronJ.JonesP. K. (2006). Tests for the visual analysis of response-guided multiple-baseline data. Journal of Experimental Education, 75, 66–81. doi:10.3200/JEXE.75.1.66-81
16.
GastD. L. (2010). Single subject research methodology in behavioral sciences. New York, NY: Routledge.
17.
HedgesL. V.PustejovskyJ. E.ShadishW. R. (2012). A standardized mean difference effect size for single case designs. Research Synthesis Methods, 3, 224–239. doi:10.1002/jrsm.1052
18.
HeyvaertM.MaesB.Van den NoortgateW.KuppensS.OnghenaP. (2012). A multilevel meta-analysis of single-case and small-n research on interventions for reducing challenging behavior in persons with intellectual disabilities. Research in Developmental Disabilities, 33, 766–780. doi:10.1016/j.ridd.2011.10.010
19.
HeyvaertM.OnghenaP. (2014). Analysis of single-case data: Randomization tests for measures of effect size. Neuropsychological Rehabilitation, 24, 507–527.
20.
HeyvaertM.SaenenL.MaesB.OnghenaP. (2015). Systematic review of restraint interventions for challenging behaviour among persons with intellectual disabilities: Focus on effectiveness in single-case experiments. Journal of Applied Research in Intellectual Disabilities, 28, 61–80.
21.
*HornerR. H.CarrE. G.HalleJ.McGeeG.OdomS.WoleryM. (2005). The use of single subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179.
22.
HornerR. H.SwaminathanH.SugaiG.SmolkowskiK. (2012). Considerations for the systematic analysis and use of single-case research. Education & Treatment of Children, 35, 269–290. doi:10.1353/etc.2012.0011
23.
HouleT. T. (2009). Statistical analyses for single-case experimental designs. In BarlowD. H.NockM. K.HersenM. (Eds.), Single case experimental designs: Strategies for studying behavior change (3rd ed., pp. 271–305). Boston, MA: Allyn & Bacon.
JonesR. R.VaughtR. S.WeinrottM. (1977). Time series analysis in operant research. Journal of Applied Behavior Analysis, 10, 151–166. doi:10.1901/jaba.1977.10-151
26.
KazdinA. E. (1999). The meanings and measurement of clinical significance. Journal of Consulting and Clinical Psychology, 67, 332–339. doi:10.1037/0022-006X.67.3.332
27.
KazdinA. E. (2011). Single-case research designs: Methods for clinical and applied settings. (2nd ed.). New York, NY: Oxford University Press.
28.
KoehlerM. J.LevinJ. R. (1998). Regulated randomization: A potentially sharper analytical tool for the multiple-baseline design. Psychological Methods, 3, 206–217. doi:10.1037/1082-989X.3.2.206
29.
KoehlerM. J.LevinJ. R. (2000). RegRand: Statistical software for the multiple-baseline design. Behavior Research Methods, Instruments, & Computers, 32, 367–371. doi:10.3758/BF03207807
30.
*KratochwillT. R.HitchcockJ.HornerR. H.LevinJ. R.OdomS. L.RindskopfD. M.ShadishW. R. (2010). Single-case designs technical documentation. Retrieved from http://ies.ed.gov/ncee/wwc/documentsum.aspx?sid=229
31.
KratochwillT. R.LevinJ. R. (2010). Enhancing the scientific credibility of single-case intervention research: Randomization to the rescue. Psychological Methods, 15, 124–144. doi:10.1037/a0017736
32.
LevinJ. R.FerronJ.KratochwillT. R. (2012). Nonparametric statistical tests for single-case systematic and randomized ABAB.AB and alternating treatment intervention designs: New developments, new directions. Journal of School Psychology, 50, 599–624. doi:10.1016/j.jsp.2012.05.001
33.
LevinJ. R.WampoldB. E. (1999). Generalized single-case randomization tests: Flexible analyses for a variety of situations. School Psychology Quarterly, 14, 59–93. doi:10.1037/h0088998
34.
MagginD. M.SwaminathanH.RogersH. J.O’KeeffeB. V.SugaiG.HornerR. H. (2011). A generalized least squares regression approach for computing effect sizes in single-case research: Application examples. Journal of School Psychology, 49, 301–321. doi:10.1016/j.jsp.2011.03.004
35.
MaherC. G.SherringtonC.HerbertR. D.MoseleyA. M.ElkinsM. (2003). Reliability of the PEDro scale for rating quality of randomized controlled trials. Physical Therapy, 83, 713–721.
36.
ManolovR.SolanasA. (2008). Comparing N = 1 effect size indices in presence of autocorrelation. Behavior Modification, 32, 860–875. doi:10.1177/0145445508318866
37.
ManolovR.SolanasA.SierraV.EvansJ. J. (2011). Choosing among techniques for quantifying single-case intervention effectiveness. Behavior Therapy, 42, 533–545. doi:10.1016/j.beth.2010.12.003
38.
OnghenaP.EdgingtonE. S. (2005). Customization of pain treatments: Single-case design and analysis. Clinical Journal of Pain, 21, 56–68. doi:10.1097/00002508-200501000-00007
39.
ParkerR. I.VannestK. J.DavisJ. L. (2011). Effect size in single-case research: A review of nine nonoverlap techniques. Behavior Modification, 35, 303–322. doi:10.1177/0145445511399147
40.
ParsonsonB.BaerD. (1978). The analysis and presentation of graphic data. In KratchowillT. (Ed.), Single subject research (pp. 101–166). New York, NY: Academic Press.
41.
*ReichowB.VolkmarF.CicchettiD. (2008). Development of the evaluative method for evaluating and determining evidence-based practices in autism. Journal of Autism and Developmental Disorders, 38, 1311–1319. doi:10.1007/s10803-007-0517-7
42.
*Romeiser-LoganL.HickmanR. R.HarrisS. R.HerizaC. B. (2008). Single-subject research design: Recommendations for levels of evidence and quality rating. Developmental Medicine & Child Neurology, 50, 99–103. doi:10.1111/j.1469-8749.2007.02005.x
43.
SchlosserR. W. (2009). The role of single-subject experimental designs in evidence-based practice times (FOCUS: Technical Brief 22). National Center for the Dissemination of Disability Research (NCDDR). Retrieved from www.ncddr.org/kt/products/focus/focus22/
ShadishW. R.CookT. D.CampbellD. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
48.
ShadishW. R.KyseE. N.RindskopfD. M. (2013). Analyzing data from single-case designs using multilevel models: New applications and some agenda items for future research. Psychological Methods, 18, 385–405. doi:10.1037/a0032964
49.
ShadishW. R.RindskopfD. M. (2007). Methods for evidence-based practice: Quantitative synthesis of single-subject designs. New Directions for Evaluation, 113, 95–109. doi:10.1002/ev.217
50.
ShadishW. R.RindskopfD. M.HedgesL. V. (2008). The state of the science in the meta-analysis of single-case experimental designs. Evidence-Based Communication Assessment and Intervention, 2, 188–196. doi:10.1080/17489530802581603
51.
ShadishW. R.SullivanK. J. (2011). Characteristics of single-case designs used to assess intervention effects in 2008. Behavior Research Methods, 43, 971–980. doi:10.3758/s13428-011-0111-y
52.
SimeraI.AltmanD. G.MoherD.SchulzK. F.HoeyJ. (2008). Guidelines for reporting health research: The EQUATOR network’s survey of guideline authors. PLoS Medicine, 5, 869–874. doi:10.1371/journal.pmed.0050139
53.
SmithJ. D. (2012). Single-case experimental designs: A systematic review of published research and current standards. Psychological Methods, 17, 510–550. doi:10.1037/a0029312
54.
*SmithT.ScahillL.DawsonG.GuthrieD.LordC.OdomS.. . .WagnerA. (2007). Designing research studies on psychosocial interventions in autism. Journal of Autism and Developmental Disorders, 37, 354–366.
55.
*SmithV.JelenM.PattersonS. (2009). Video modeling to improve play skills in a child with autism: A procedure to examine single-subject experimental research. Evidence-Based Practice Briefs, 4, 1–13.
*TateR. L.McDonaldS.PerdicesM.TogherL.SchultzR.SavageS. (2008). Rating the methodological quality of single-subject designs and n-of-1 trials: Introducing the Single-Case Experimental Design (SCED) Scale. Neuropsychological Rehabilitation, 18, 385–401. doi:10.1080/09602010802009201
58.
TodmanJ. B.DugardP. (2001). Single-case and small-n experimental designs: A practical guide to randomization tests. Mahwah, NJ: Lawrence Erlbaum.
59.
Van den NoortgateW.OnghenaP. (2003). Hierarchical linear models for the quantitative integration of effects sizes in single-case research. Behavior Research Methods, Instruments, & Computers, 35, 1–10. doi:10.3758/BF03195492
60.
WampoldB. E.WorshamN. L. (1986). Randomization tests for multiple-baseline designs. Behavioral Assessment, 8, 135–143.
61.
*WangS.-Y.ParrilaR. (2008). Quality indicators for single-case research on social skill interventions for children with autistic spectrum disorder. Developmental Disabilities Bulletin, 36, 81–105.
62.
WendtO.MillerB. (2012). Quality appraisal of single-subject experimental designs: An overview and comparison of different appraisal tools. Education & Treatment of Children, 35, 235–268. doi:10.1353/etc.2012.0010
63.
WhiteD. M.RuschF. R.KazdinA. E.HartmannD. P. (1989). Applications of meta analysis in individual-subject research. Behavioral Assessment, 11, 281–296.
64.
WoleryM.BusickM.ReichowB.BartonE. E. (2010). Comparison of overlap methods for quantitatively synthesizing single-subject data. The Journal of Special Education, 44, 18–29. doi:10.1177/0022466908328009
