During the past 30 years, meta-analysis has been an indispensable tool for revealing the hidden meaning of our research literatures. The four articles in this special section on meta-analysis illustrate some of the complexities entailed in meta-analysis methods. Although meta-analysis is a powerful tool for advancing cumulative knowledge, researchers can be confused by the complicated issues involved in the methodology. Each of these four articles contributes both to advancing this methodology and to the increasing complexities that can befuddle researchers. In these comments, the author attempts to clarify both of these aspects and provide a perspective on the methodological issues examined in these articles.
Aguinis, H., & Pierce, C.A. (1998). Testing moderator variable hypotheses meta-analytically . Journal of Management, 24, 577-592.
2.
Aguinis, H., Sturman, M.C., & Pierce, C.A. (in press). Comparison of three meta-analytic procedures for estimating moderating effects of categorical variables. Organizational Research Methods.
3.
Bechtel, W. (1988). Philosophy of science: An overview for cognitive science. Hillsdale, NJ: Lawrence Erlbaum.
4.
Field, A.P. (2001). Meta-analysis of correlation coefficients: A Monte Carlo comparison of fixed- and random-effects methods. Psychological Methods, 6, 161-180.
5.
Field, A.P. (2003). The problem in using fixed-effects models of meta-analysis on real world data. Understanding Statistics, 2, 77-96.
6.
Field, A.P. (2005). Is the meta-analysis of correlation coefficients accurate when population correlations vary?Psychological Methods, 10, 444-467.
7.
Hall, S.M., & Brannick, M.T. (2002). Comparison of two random effects methods of meta-analysis. Journal of Applied Psychology , 74, 469-477.
8.
Hedges, L.V. (1994). Statistical considerations. In H. Cooper & L. V. Hedges (Eds.),The handbook of research synthesis (pp. 29-38). New York: Russell Sage.
9.
Hedges, L.V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
10.
Hedges, L.V., & Vevea, J.L. (1998). Fixed- and random-effects models in meta-analysis . Psychological Methods, 3, 486-504.
11.
Hunter, J.E., & Schmidt, F.L. (1990). Methods of meta-analysis: Correcting error and bias in research findings. Newbury Park, CA: Sage.
12.
Hunter, J.E., & Schmidt, F.L. (1994). The estimation of sampling error variance in meta-analysis of correlations: The homogeneous case. Journal of Applied Psychology, 79, 171-177.
13.
Hunter, J.E., & Schmidt, F.L. (2000). Fixed effects vs. random effects meta-analysis models: Implications for cumulative research knowledge. International Journal of Selection and Assessment, 8, 275-292.
14.
Hunter, J.E., & Schmidt, F.L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Thousand Oaks, CA: Sage.
15.
Hunter, J.E., Schmidt, F.L., & Jackson, G.B. (1982).Meta-analysis: Cumulating research findings across studies. Beverly Hills, CA: Sage .
16.
Hunter, J.E., Schmidt, F.L., & Le, H. (2006). Implications of direct and indirect range restriction for meta-analysis methods and findings. Journal of Applied Psychology, 91, 594-612.
17.
Kierein, N.M., & Gold, M.A. (2000). Pygmalion in work organizations: A meta-analysis . Journal of Organizational Behavior, 21, 913-928.
18.
Kisamore, J.L., & Brannick, M.T. (in press). An illustration of the consequences of meta-analysis model choice. Organizational Research Methods .
19.
Law, K.S., Schmidt, F.L., & Hunter, J.E. (1994). A test of two refinements in meta-analysis procedures . Journal of Applied Psychology, 79, 978-986.
20.
Le, H., & Schmidt, F.L. (2006). Correcting for indirect range restriction in meta-analysis: Testing a new meta-analytic method. Psychological Methods , 11, 416-438.
21.
Lipsey, M.W., & Wilson, D.B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48, 1181-1209.
22.
McNatt, D.B. (2000). Ancient Pygmalion joins contemporary management: A meta-analysis of the result. Journal of Applied Psychology , 85, 314-322.
23.
National Research Council. (1992). Combining information: Statistical issues and opportunities for research. Washington, DC: National Academy of Sciences Press .
24.
Overton, R.C. (1998). A comparison of fixed effects and mixed (random effects) models for meta-analysis tests of moderator variable effects. Psychological Methods, 3, 354-379.
25.
Phillips, D.C. (1987). Philosophy, science, and social inquiry. Oxford, UK: Pergamon.
26.
Raju, N.S., & Drasgow, F. (2003). Maximum likelihood estimation in validity generalization . In K. R. Murphy (Ed.), Validity generalization: A critical review (pp. 263-285). Mahwah, NJ : Lawrence Erlbaum.
27.
Raudenbush, S.W. (1994). Random effects models. In H. Cooper & L. V. Hedges (Eds.), The handbook of research synthesis (pp. 301-322). New York: Russell Sage.
28.
Rosenthal, R. (1991). Meta-analytic procedures for social research (2nd ed.). Newbury Park, CA: Sage.
29.
Rothstein, H.F., Sutton, A.J., & Borenstein, M. (2005). Publication bias in meta-analysis: Prevention, assessment and adjustment. London: Wiley.
30.
Sackett, P.R., Harris, M.M., & Orr, J.M. (1986). On seeking moderator variables in the meta-analysis of data: A Monte Carlo investigation statistical power and resistance to Type I error. Journal of Applied Psychology, 71, 302-310.
31.
Sanchez-Meca, J., & Marin-Martinez, F. (1998). Weighting by inverse variance or by sample size in meta-analysis: A simulation study. Educational and Psychological Measurement, 58, 211-220.
32.
Schmidt, F.L. (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers. Psychological Methods, 1, 115-129.
33.
Schmidt, F.L., & Hunter, J.E. (1977). Development of a general solution to the problem of validity generalization. Journal of Applied Psychology , 62, 529-540.
34.
Schmidt, F.L., Hunter, J.E., Pearlman, K., & Hirsh, H.R. (1985). Forty questions about validity generalization and meta-analysis. Personnel Psychology, 38, 697-798.
35.
Schmidt, F.L., Hunter, J.E., & Urry, V. (1976). Statistical power in criterion-related validation studies. Journal of Applied Psychology, 61, 473-485.
36.
Schmidt, F.L., Law, K., Hunter, J.E., Rothstein, H.R., Pearlman, K., & McDaniel, M. (1993). Refinements in validity generalization methods: Implications for the situational specificity hypothesis. Journal of Applied Psychology, 78, 3-13.
37.
Schmidt, F.L., & Le, H. (2004). Software for the Hunter—Schmidt meta-analysis methods. Iowa City: University of Iowa, Department of Management & Organizations.
38.
Schmidt, F.L., Oh, I.-S., & Hayes, T. (2006). Fixed vs. random effects models in meta-analysis: An empirical comparison of differences in results in the psychological literature . Manuscript submitted for publication.
39.
Schmidt, F.L., Oh, I.-S., & Le, H. (2006). Increasing the accuracy of corrections for range restriction: Implications for selection procedure validities and other research results. Personnel Psychology, 59, 281-305.
40.
Schmidt, F.L., & Raju, N.S. (2007). Updating meta-analytic research findings: Bayesian approaches versus the medical model. Journal of Applied Psychology , 92, 297-308.
41.
Schulze, R. (2004). Meta-analysis: A comparison of approaches. Cambridge, MA: Hogrefe & Huber.
42.
Steel, P.D.G., & Kammeyer-Mueller, J. (in press). Bayesian variance estimation for meta-analysis: Quantifying our uncertainty. Organizational Research Methods.
43.
Stewart, W.H., & Roth, P.L. (2001). Risk propensity differences between entrepreneurs and managers: A meta-analytic review. Journal of Applied Psychology , 86, 145-153.
44.
Thorndike, R.L. (1949). Personnel selection. New York : John Wiley.
45.
Toulmin, S.S. (1961). Foresight and understanding: An enquiry into the aims of science. New York: Harper .
46.
Wood, J. (in press). Methodology for dealing with duplicate study effects in a meta-analysis. Organizational Research Methods.
