In a comment on our paper, Bradley and Brand (2013) argue that effect sizes are exaggerated owing to low power and publication bias. They propose to correct these exaggerations by application of a specific formula leading to a better estimate of the “true” effect size. In a simulation we test the effect of this formula and find this “corrective” approach unsatisfactory. We agree with Bradley and Brand on the points that effect sizes are important in primary and secondary research, and that exaggerated effect sizes are a serious problem in research. However, we disagree on the appropriate reaction: A diagnostic approach may be more appropriate than a corrective approach.
BradleyM. T.BrandA. (2013). Sweeping recommendations regarding effect size and sample size can miss important nuances: A comment on “A comprehensive review of reporting practices in psychological journals”. Theory & Psychology, 23, 797–800.
2.
BradleyM. T.StoicaG. (2004). Diagnosing estimate distortion due to significance testing in literature on detection of deception. Perceptual and Motor Skills, 98, 827–839. doi:10.2466/pms.98.3.827-839
3.
DechartresA.TrinquartL.BoutronI.RavaudP. (2013). Influence of trial sample size on treatment effect estimates: Meta-epidemiological study. British Medical Journal, 346, 2304–2309. doi:10.1136/bmj.f2304
4.
FergusonC. J.HeeneM. (2012). A vast graveyard of undead theories publication bias and psychological science’s aversion to the null. Perspectives on Psychological Science, 7(6), 555–561. doi:10.1177/1745691612459059
5.
FritzA.ScherndlT.KühbergerA. (2013a). A comprehensive review of reporting practices in psychological journals: Are effect sizes really enough?Theory & Psychology, 23, 98–122. doi:10.1177/0959354312436870
6.
FritzA.ScherndlT.KühbergerA. (2013b). Publication bias and the correlation between effect size and sample size in psychological research: Sources, consequences, and remedies. Manuscript submitted for publication.
7.
LauJ.IoannidisJ. P. A.TerrinN.SchmidC. H.OlkinI. (2006). The case of the misleading funnel plot. British Medical Journal, 333, 597–600.
8.
SedlmeierP.GigerenzerG. (1989). Do studies of statistical power have an effect on the power of studies?Psychological Bulletin, 105, 309–316. doi:10.1037/0033–2909.105.2.309
9.
SterneJ. A. C.EggerM.SmithG. D. (2001). Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. British Medical Journal, 323, 101–105. doi:10.1136/bmj.323.7304.101
