For meta-analyses of observational epidemiology studies, unadjusted and adjusted study estimates are often extracted. However, there is evidence of selective reporting of adjusted study estimates. We investigate adjustment reporting bias, examining the reasons why some studies do not contribute an adjusted estimate to a meta-analysis. Ten published meta-analyses were re-analysed to assess evidence of adjustment reporting bias and over 100 primary studies were read to investigate why they did not contribute an adjusted estimate to a meta-analysis. Selective reporting of adjusted estimates may lead to a bias in some meta-analyses when adjusted study estimates are not reported because univariate analyses indicated a non-significant effect. We recommend that unadjusted and adjusted study estimates be extracted for a meta-analysis. If adjusted estimates cannot be obtained, the reasons for this should be investigated and sensitivity analyses could be used to assess the impact of this on the meta-analysis.
Ali, S., Stone, M.A., Peters, J.L., Davies, M.J., & Khunti, K. (2006). The prevalence of co-morbid depression in adults with type 2 diabetes: a systematic review and meta-analysis. Diabetic Medicine, 23, 1165-1173.
2.
Anda, R., Williamson, D., Jones, D., Macera, C., Eaker, E., Glassman, A., & Marks, J. (1993). Depressed affect, hopelessness, and the risk of ischaemic heart disease in a cohort of US adults. Epidemiology , 4, 285-294.
3.
Begg, C.B., & Mazumdar, M. (1994). Operating characteristics of a rank correlation test for publication bias. Biometrics, 50, 1088-1101.
4.
Beral, V. (1995). `The practice of meta-analysis`: discussion. Meta-analysis of observational studies: a case study of work in progress. Journal of Clinical Epidemiology, 48, 165-166.
5.
Bhatia, R., Lopipero, P., & Smith, A.H. (1998). Diesel exhaust exposure and lung cancer. Epidemiology, 9, 84-91.
6.
Blair, A., Burg, J., Foran, J., Gibb, H., Greenland, S., Morris, R., Raabe, G., Savitz, D., Teta, J., Wartenberg, D., Wong, O., & Zimmerman, R. (1995). Guidelines for application of meta-analysis in environmental epidemiology. Regulatory Toxicology and Pharmacology , 22, 189-197.
7.
Blettner, M., Sauerbrei, W., Shlehofer, B., Scheuchenpflug, T., & Friedenreich, C. (1999). Traditional reviews, meta-analyses and pooled analyses in epidemiology. International Journal of Epidemiology, 28, 1-9.
8.
Bowden, J., Thompson, J., & Burton, P. (2006). Using pseudo-data to correct for publication bias in meta-analysis. Statistics in Medicine, 25, 3798-3813.
9.
Chan, A.W., Hrobjartsson, A., Haahr, M.T., Gotzsche, P.C., & Altman, D.G. (2004). Empirical evidence for selective reporting of outcomes in randomized trials - Comparison of Protocols to published articles . Journal of the American Medical Association, 291, 2457-2465.
10.
Clouse, R., Lustman, P., Freedland, K., Griffith, L., McGill, J., & Carney, R. (2003). Depression and coronary heart disease in women with diabetes. Psychosomatic Medicine, 65, 376-383.
11.
Cohen, H., Madhavan, S., & Alderman, M. (2001). History of treatment for depression: risk factor for myocardial infarction in hypertensive patients. Psychosomatic Medicine, 63, 203-209.
12.
Cosgrove, S.E., Sakoulas, G., Perencevich, E.N., Schwaber, M.J., Karchmer, A.W., & Carmeli, Y. (2003). Comparison of mortality associated with methicillin-resistant and methicillin-suscepible staphylococcus aureus bacteremia: a meta-analysis . Clinical Infectious Diseases, 36, 53-59.
13.
DerSimonian, R., & Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7, 177-188.
14.
Duval, S., & Tweedie, R.L. (2000). A nonparametric ``Trim and Fill'' method of accounting for publication bias in meta-analysis. Journal of the American Statistical Society, 95, 89-98.
15.
Egger, M., Davey Smith, G., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315, 629-634.
16.
Egger, M., Schneider, M., & Smith, G.D. (1998). Meta-analysis: Spurious precision? Meta-analysis of observational studies. BMJ, 316, 140-144.
17.
El-Serag, H.B., Hampel, H., & Javadi, F. (2006). The association between diabetes and hepatocellular carcinoma: a systematic review of epidemiologic evidence. Clinical Gastroenterology and Hepatology, 4, 369-380.
18.
Fleiss, J.L., & Gross, A.J. (1991). Meta-analysis in epidemiology, with special reference to studies of the association between exposure to environmental tobacco smoke and lung cancer: a critique. Journal of Clinical Epidemiology , 44, 127-139.
19.
Ford, D.E., Mead, L.A., Chang, P.P., Cooper-Patrick, L., Wang, N.Y., & Klag, M.J. (1998). Depression is a risk factor for coronary artery disease in men: the precursors study. Archives of Internal Medicine, 158, 1422-1426.
20.
Friedenriech, C.M. (1993). Methods for pooled analyses of epidemiologic studies . Epidemiology, 4, 295-302.
21.
Hahn, S., Williamson, P.R., & Hutton, J.L. (2002). Investigation of within-study selective reporting in clinical research: follow-up of applications submitted to a local research ethics committee. Journal of Evaluation in Clinical Practice , 8, 353-359.
22.
Hampel, H., Abraham, N.S., & El-Serag, H.B. (2005). Meta-analysis: obesity and the risk for gastroesophageal reflux disease and its complications. Annals of Internal Medicine, 143, 199-211.
23.
Harbord, R.M., Egger, M., & Sterne, J.A.C. (2006). A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Statistics in Medicine, 25, 3443-3457.
24.
Henmi, M., & Copas, J.B. (2007). Confidence intervals and p-values for meta-analysis with publication bias. Biometrics, 63, 475-782.
Hutton, J.L., & Williamson, P.R. (2000). Bias in meta-analysis due to outcome variable selection within studies. Journal of the Royal Statistical Society, Series C ( Applied Statistics, 49, 359-370.
27.
Lee, P.N., & Sanders, E. (2004). Does increased cigarette consumption nullify any reduction in lung cancer risk associated with low-tar filter cigarettes? Inhalation Toxicology, 16, 817-833.
28.
Light, R.J., & Pillemar, D.B. (1984). Summing up: the science of reviewing research. Cambridge, MA: Harvard University Press.
29.
Luukinen, H., Laippala, P., & Huikuri, H. (2003). Depressive symptoms and the risk of sudden cardiac death among the elderly. European Heart Journal, 24, 2021-2026.
30.
Minelli, C., Thompson, J.R., Tobin, M.D., & Abrams, K.R. (2004). An integrated approach to the meta-analysis of genetic association studies using mendelian randomization. American Journal of Epidemiology, 160, 445-452.
31.
Nicholson, A., Kuper, H., & Hemingway, H. (2006). Depression as an aetiologic and prognostic factor in coronary heart disease: a meta-analysis of 6362 events among 146 538 participants in 54 observational studies. European Heart Journal, 27, 2763-2774.
32.
Penttinen, J., & Valonen, P. (1996). Use of psychtropic drugs and risk of myocardial infarction: a case-control study in Finnish farmers. International Journal of Epidemiology, 25, 760-762.
33.
Peters, J.L., Sutton, A.J., Jones, D.R., Abrams, K.R., & Rushton, L. (2006). Comparison of two methods to detect publication bias in meta-analysis. Journal of the American Medical Association , 295, 676-680.
34.
Peters, J.L., Sutton, A.J., Jones, D.R., Abrams, K.R., & Rushton, L. (2007). Performance of the trim and fill method in the presence of publication bias and between-study heterogeneity. Statistics in Medicine, 26, 4544-4562.
35.
Pratt, L.A., Ford, D.E., Crum, R.M., Armenian, H.K., Gallo, J.J., & Eaton, W.W. (1996). Depression, psychotropic medication, and risk of myocardial infarction. Prospective data from the Baltimore ECA follow-up. Circulation, 94, 3123-3129.
36.
Rothstein, H.R., Sutton, A.J., & Borenstein, M. (2005). Publication Bias in Meta-Analysis . Chichester, England: Wiley.
37.
Rucker, G., Schwarzer, G., & Carpenter, J. (2008). Arcsine test for publication test in meta-analyses with binary outcomes. Statistics in Medicine, 27, 746-763.
38.
Rushton, L., & Jones, D.R. (1992). Oral contraceptive use and breast cancer risk: a meta-analysis of variations with age at diagnosis, parity and total duration of oral contraceptive use. British Journal of Obstetrics and Gynaecology, 99, 239-246.
39.
Schmid, C.H., Stark, P.C., Berlin, J.A., Landais, P., & Lau, J. (2004). Meta-regression detected associations between heterogeneous treatment effects and study-level, but not patient-level, factors . Journal of Clinical Epidemiology, 57, 683-697.
40.
Schwarzer, G., Antes, G., & Schumacher, M. (2002). Inflation of type I error rate in two statistical tests for the detection of publication bias in meta-analyses with binary outcomes. Statistics in Medicine, 21, 2465-2477.
41.
Shapiro, S. (1994). Meta-analysis Shmeta-analysis. American Journal of Epidemiology, 140, 771-778.
42.
Siegfried, N., Muller, M., Deeks, J., Volmink, J., Egger, M., Low, N., Walker, S., & Williamson, P. (2005). HIV and male circumcision - a systematic review with assessment of the quality of studies. Lancet Infectious Disease, 5, 165-173.
43.
Simmonds, M.C., Higgins, J.P.T., Stewart, L.A., Tierney, J.F., Clarke, M.J., & Thompson, S.G. (2005). Meta-analysis of individual patient data from randomized trials: a review of methods used in practice . Clinical Trials, 2, 209-217.
44.
Sterne, J.A.C., Gavaghan, D., & Egger, M. (2000). Publication and related bias in meta-analysis: Power of statistical tests and prevalence in the literature. Journal of Clinical Epidemiology, 53, 1119-1129.
45.
Stewart, L.A., & Parmar, M.K.B. (1993). Meta-analysis of the literature or of individual patient data: is there a difference? The Lancet, 341, 418-422.
46.
Stroup, D.F., Berlin, J.A., Morton, S.C., Olkin, I., Williamson, G.D., Rennie, D., Moher, D., Becker, B.J., Sipe, T.A., & Thacker, S.B. (2000). Meta-analysis of observational studies in epidemiology: a proposal for reporting. Journal of the American Medical Association, 283, 2008-2012.
47.
Sutton, A.J., Abrams, K.R., Jones, D.R., Sheldon, T.A., & Song, F. (2000). Methods for Meta-Analysis in Medical Research . Chichester: Wiley.
48.
Terrin, N., Schmid, C.H., Lau, J., & Olkin, I. (2003). Adjusting for publication bias in the presence of heterogeneity. Statistics in Medicine, 22, 2113-2126.
49.
Tudur, C., Williamson, P.R., Khan, S., & Best, L.Y. (2001). The value of the aggregate data approach in meta-analysis with time-to-event outcomes. Journal of the Royal Statistical Society, Series A Statistics in Society, 164, 357-370.
50.
Weiss, H.A., Quigley, M.A., & Hayes, R.J. (2000). Male circumcision and risk of HIV infection in sub-Saharan Africa: a systematic review and meta-analysis. AIDS, 14, 2361-2370.
51.
Williamson, P.R., & Gamble, C. (2005). Identification and impact of outcome selection bias in meta-analysis. Statistics in Medicine, 24, 1547-1561.
52.
Williamson, P.R., Gamble, C., Altman, D.G., & Hutton, J.L. (2005). Outcome selection bias in meta-analysis. Statistical Methods in Medical Research, 14, 515-524.
53.
Yasuda, N., Mino, Y., Koda, S., & Ohara, H. (2002). The differential influence of distinct clusters of psychyatric symptoms, as assessed by the general health questionnaire, on cause of death in older persons living in a rural community of Japan. Journal of the American Geriatric Society, 50, 313-320.
54.
Zhou, B., Teramukai, S., & Fukushima, M. (2007). Prevention and treatment of dementia or Alzheimer's disease by statins: a meta-analysis. Dementia and Geriatric Cognitive Disorders, 23, 194-201.
