Reproducibility is the cornerstone of science. If an effect is reliable, any competent researcher should be able to obtain it when using the same procedures with adequate statistical power. Two of the articles in this special section question the value of direct replication by other laboratories. In this commentary, I discuss the problematic implications of some of their assumptions and argue that direct replication by multiple laboratories is the only way to verify the reliability of an effect.
AsendorpfJ. B.ConnerM.de FruytF.de HouwerJ.DenissenJ. J. A.FiedlerK.. . . WichertsJ. M. (2013). Recommendations for increasing replicability in psychology. European Journal of Personality, 27, 108–119. doi:10.1002/per.1919
2.
BrandtM. J.IJzermanH.DijksterhuisA.FarachF. J.GellerJ.Giner-SorollaR.. . . Van ’t VeerA. (in press). The replication recipe: What makes for a convincing replication?Journal of Experimental Social Psychology. Retrieved from http://dx.doi.org/10.1016/j.jesp.2013.10.005
3.
CesarioJ. (2014). Priming, replication, and the hardest science. Perspectives on Psychological Science, 9, 40–48.
4.
CrumpM. J. C.McDonnellJ. V.GureckisT. M. (2013). Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral research. PLoS ONE, 8(3), e57410. doi:10.1371/journal.pone.0057410
5.
FanelliD. (2010). “Positive” results increase down the hierarchy of the sciences. PLoS ONE, 5(4), e10068. doi:10.1371/journal.pone.0010068
6.
GermineL.NakayamaK.DuchaineB. C.ChabrisC. F.ChatterjeeG.WilmerJ. B. (2012). Is the web as good as the lab? Comparable performance from Web and lab in cognitive/perceptual experiments. Psychonomic Bulletin & Review, 19, 847–857.
7.
Giner-SorollaR. (2012). Science or art? How aesthetic standards grease the way through the publication bottleneck but undermine science. Perspectives on Psychological Science, 7, 562–571. doi:10.1177/1745691612457576
JohnL. K.LoewensteinG.PrelecD. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23, 524–532.
10.
KlatzkyR. L.CreswellD. (2014). An intersensory interaction account of priming effects—and their absence.Perspectives on Psychological Science, 9, 49–58.
11.
PashlerH.HarrisC. R. (2012). Is the replicability crisis overblown? Three arguments examined. Perspectives on Psychological Science, 7, 531–536. doi:10.1177/1745691612463401
12.
PashlerH.WagenmakersE.-J. (2012). Editors’ introduction to the special section on replicability in psychological science: A crisis of confidence?Perspectives on Psychological Science, 7, 528–530. doi:10.1177/1745691612465253
13.
RoedigerH. L. (2012). Psychology’s woes and a partial cure: The value of replication. Observer, 25(2), 9, 27–29.
14.
SimmonsJ. P.NelsonL. D.SimonsohnU. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359–1366.
15.
SimmonsJ. P.NelsonL. D.SimonsohnU. (2013, January). Life after P-Hacking (2013). Paper presented at the Society for Personality and Social Psychology annual meeting, New Orleans, LA. Retrieved from http://ssrn.com/abstract=2205186
16.
SmartR. G. (1964). The importance of negative results in psychological research. Canadian Psychologist, 5a, 225–232. doi:10.1037/h0083036
17.
StroebeW.StrackF. (2014). The alleged crisis and the illusion of exact replication. Perspectives on Psychological Science, 9, 59–71.
18.
TverskyA.KahnemanD. (1971). Belief in the law of small numbers. Psychological Bulletin, 76, 105–110.
