Sage Journals: Discover world-class research

Abstract

Evidence from the household survey literature shows a declining response rate trend in recent decades, but whether a similar trend exists for voluntary establishment surveys is an understudied issue. This article examines trends in nonresponse rates and nonresponse bias over a period of 17 years in the annual cross-sectional refreshment samples of the IAB Establishment Panel in Germany. In addition, rich administrative data about the establishment and employee composition are used to examine changes in nonresponse bias and its two main components, refusal and noncontact, over time. Our findings show that response rates dropped by nearly a third: from 50.2% in 2001 to 34.5% in 2017. Simultaneously, nonresponse bias increased over this period, which was mainly driven by increasing refusal bias whereas noncontact bias fluctuated relatively evenly over the same period. Nonresponse biases for individual establishment and employee characteristics did not show a distinct pattern over time with few exceptions. Notably, larger establishments participated less frequently than smaller establishments over the entire period. This implies that survey organizations may need to put more effort into recruiting larger establishments to counteract nonresponse bias.

Keywords

Survey participation establishment characteristics administrative data unit nonresponse

Get full access to this article

View all access options for this article.

References

AAPOR (American Association for Public Opinion research). 2016. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. AAPOR.

Bavdaz, M., D. Giesen, D.L. Moore, P.A. Smith, and J. Jones. 2019. “Qualitative Testing for Official Establishment Survey Questionnaires”. Survey Research Methods 13(3): 267–288. DOI: https://doi.org/10.18148/srm/2019.v13i3.7366.

Beckler, D.G., and K. Ott. 2006. “Indirect Monetary Incentives with a Complex Agricultural Establishment Survey”. In Proceedings of the American Statistical Association, Survey Research Methodology Section, 2741–2748. Alexandria, VA, USA. American Statistical Association. Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.511.9782&rep=rep1&type=pdf (accessed November 2021).

Beullens, K., G. Loosveldt, C. Vandenplas, and I. Stoop. 2018. “Response Rates in the European Social Survey: Increasing, Decreasing, or a Matter of Fieldwork Efforts?” Survey Methods: Insights from the Field: 1–12. DOI: https://doi.org/10.13094/SMIF-2018-00003.

Brick, J.M., and R. Tourangeau. 2017. “Responsive Survey Designs for Reducing Nonresponse Bias.” Journal of Official Statistics 33(3): 735–752. DOI: http://dx.doi.org/10.1515/JOS-2017-0034.

Brick, J.M., and D. Williams. 2013. “Explaining Rising Nonresponse Rates in Cross-sectional Surveys.” In The Annals of the American Academy of Political and Social Science 645: 36–59. DOI: https://doi.org/10.1177/0002716212456834.

Brixy, U., S. Kohaut, and C. Schnabel. 2007. “Do Newly Founded Firms Pay Lower Wages? First Evidence from Germany.” Small Business Economics 29(1-2): 161–171. DOI: https://doi.org/10.1007/s11187-006-0015-x.

Christianson, A., and R.D. Tortora. 1995. “Issues in Surveying Businesses: An International Survey.” In Business Survey Methods, 235–256. New York: John Wiley & Sons. DOI: https://doi.org/10.1002/9781118150504.ch14.

D’Aurizio, L., and G. Papadia. 2019. “Using Administrative Data to Evaluate Sampling Bias in a Business Panel Survey.” Journal of Official Statistics 35(1): 67–92. DOI: https://doi.org/10.2478/jos-2019-0004.

10.

Davis, W.R., and N. Pihama. 2009. “Survey Response as Organisational Behaviour: An Analysis of the Annual Enterprise Survey 2003-2007.” New Zealand Association of Economists Conference 2009: 1–16. Wellington, New Zealand: New Zeland Association of Economists. Available at: https://ro.uow.edu.au/eispapers/826/ (accessed November 2021).

11.

De Leeuw, E., and W. de Heer. 2002. “Trends in Household Survey Nonresponse: A Longitudinal and International Comparison.” In Survey Nonresponse, edited by R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little, 41–54. New York: Wiley.

12.

Dillman, D.A., J.D. Smyth, and L.M. Christian. 2014. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. New York: John Wiley & Sons.

13.

Earp, M., D. Toth, P. Phipps, and C. Oslund. 2018. “Assessing Nonresponse in a Longitudinal Establishment Survey Using Regression Trees.” Journal of Official Statistics 34(2): 463–481. DOI: https://doi.org/10.2478/jos-2018-0021.

14.

Ellguth, P., S. Kohaut, and I. Möller. 2013. “The IAB Establishment Panel – Methodological Essentials and Data Quality.” Journal for Labour Market Research 47: 27–41. DOI: https://doi.org/10.1007/s12651-013-0151-0.

15.

Ellguth, P., and S. Kohaut. 2019. “A Note on the Decline of Collective Bargaining Coverage: The Role of Structural Change.” Jahrbücher für Nationalökonomie und Statistik 239(1): 39–66. DOI: https://doi.org/10.1515/jbnst-2017-0163.

16.

Eurofound. 2015. Third European Company Survey – Overview Report: Workplace Practices – Patterns, Performance and Well-Being. Luxembourg: Publications Office of the European Union. Available at: https://www.eurofound.europa.eu/sites/default/files/ef_publication/field_ef_document/ef1502en_0.pdf (accessed November 2021).

17.

Fischer, G., F. Janik, D. Müller, and A. Schmucker. 2008. “The IAB Establishment Panel – from Sample to Survey to Projection.” FDZ Methodenreport 01/2008: Nuremberg. Available at: https://core.ac.uk/reader/6561380 (accessed November 2021).

18.

Fisher, S., J. Bosley, K. Goldenberg, W. Mockovak, and C. Tucker. 2003. “A Qualitative Study of Nonresponse Factors Affecting BLS Establishment Surveys: Results.” Presented at the Annual Joint Statistical Meetings, San Francisco, CA. Available at: https://www.bls.gov/osmr/research-papers/2003/pdf/st030230.pdf (accessed November 1st 2021).

19.

Groves, R.M. 2006. “Nonresponse Rates and Nonresponse Bias in Household Surveys.” Public Opinion Quarterly 70(5): 646–675. DOI: https://doi.org/10.1093/poq/nfl033.

20.

Groves, R.M., and M.P. Couper. 1998. Nonresponse in Household Interview Surveys. New York: John Wiley & Sons.

21.

Groves, R.M., and E. Peytcheva. 2008. “The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis.” Public Opinion Quarterly 72(2): 167–189. DOI: https://doi.org/10.1093/poq/nfn011.

22.

Henze, p. 2014. “Structural Change and Wage Inequality: Evidence from German Micro Data.” Center for European, Governance and Economic Development Research Working Paper 204. DOI: http://dx.doi.org/10.2139/ssrn.2422471.

23.

Janik, F. 2011. “Unit Non-response in Establishments Surveyed for the First Time in the IAB Establishment Panel.” FDZ Methodenreport 04/2011: Nuremberg. Available at: https://www.researchgate.net/publication/228956272_Unit_non-response_in_establishments_surveyed_for_the_first_time_in_the_IAB_Establishment_Panel (accessed November 2021).

24.

Janik, F., and S. Kohaut. 2012 “Why Don’t They Answer? Unit Non-response in the IAB Establishment Panel.” Quality & Quantity 46: 917–934. DOI: https://doi.org/10.1007/s11135-011-9436-y.

25.

Jobber, D., H. Mirza, and K.H. Wee. 1991. “Incentives and Response Rates to Cross-national Business Surveys: A Logit Model Analysis.” Journal of International Business Studies 22: 711–721. DOI: https://doi.org/10.1057/palgrave.jibs.8490852.

26.

Kohler, U., and F. Kreuter. 2012. Datenanalyse mit Stata: Allgemeine Konzepte der Datenanalyse und ihre praktische Anwendung. München: Oldenbourg.

27.

Luiten, A., J Hox, and E. de Leeuw. 2020. “Survey Nonresponse Trends and Fieldwork Effort in the 21st Century: Results of an International Study across Countries and Surveys.” Journal of Official Statistics 36(3): 469–487. DOI: http://dx.doi.org/10.2478/JOS-2020-0025.

28.

Luo, A., and G.D. White. 2005. “Exploring a New Establishment Survey Incentive to Improve Response Rates.” In Proceedings of the American Statistical Association, Survey Research Methodology Section, May 12-15, 2005. 3915–3918. Miami Beach, Florida. American Statistical Association. Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.528.9845&rep=rep1&type=pdf (accessed November 2021).

29.

Mullins, J.P. 2016. “One Hundred Years of Current Employment Statistics – an Overview of Survey Advancements.” Monthly Labor Review. DOI: https://doi.org/10.21916/mlr.2016.39.

30.

Paxson, M.C., D.A. Dillman, and J. Tarnai. 1995. “Improving Response to Business Mail Surveys.” In Business Survey Methods, edited by B.G. Cox, D.A. Binder, B.N. Chinnappa, A. Christianson, M.J. Colledge, and P.S. Kott, 303-316. New York: Wiley.

31.

Petroni, R., R. Sigman, D. Willimack, S. Cohen, and C. Tucker. 2004 “Response Rates and Nonresponse in Establishment Surveys–BLS and Census Bureau.” Presented to the Federal Economic Statistics Advisory Committee, 1–50. Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.485.5983&rep=rep1&type=pdf (accessed November 2021).

32.

Phipps, P., and D. Toth. 2012. “Analyzing Establishment Nonresponse Using an Interpretable Regression Tree Model with Linked Administrative Data.” The Annals of Applied Statistics 6(2): 772–794. DOI: https://doi.org/10.1214/11-AOAS521.

33.

Sakshaug, J.W., and M. Huber. 2016. “An Evaluation of Panel Nonresponse and Linkage Consent Bias in a Survey of Employees in Germany.” Journal of Survey Statistics and Methodology 4(1): 71–93. DOI: https://doi.org/10.1093/jssam/smv034.

34.

Sakshaug, J.W., S. Hülle, A. Schmucker, and S. Liebig. 2019a. “Panel Survey Recruitment With or Without Interviewers? – Implications for Nonresponse Bias, Panel Consent Bias, and Total Recruitment Bias.” Journal of Survey Statistics and Methodology 8(3): 540–565. DOI: https://doi.org/10.1093/jssam/smz012.

35.

Sakshaug, J.W., B. Vicari, and M.P. Couper. 2019b. “Paper, E-mail, or Both? Effects of Contact Mode on Participation in a Web Survey of Establishments.” Social Science Computer Review 37(6): 750–765. DOI: https://doi.org/10.1177/0894439318805160.

36.

Schmucker, A., A. Ganzer, J. Stegmaier, and S. Wolter. 2018. “Establishment History Panel 1975-2017.” FDZ Datenreport 09/2018: Nuremberg. DOI: https://doi.org/10.5164/IAB.FDZD.1809.en.v1.

37.

Seiler, C. 2010. “Dynamic Modelling of Nonresponse in Business Surveys.” Ifo Working Paper No. 93. Munich: ifo Institute – Leibniz Institute for Economic Research at the University of Munich. Available at: https://www.econstor.eu/bitstream/10419/73706/1/IfoWorkingPaper-93.pdf (accessed November 2021).

38.

StataCorp. 2017. Stata Statistical Software: Release 15. College Station, TX: StataCorp LLC.

39.

Sudman, S., D.K. Willimack, E. Nichols, and T.L. Mesenbourg. 2000. “Exploratory Research at the U.S. Census Bureau on the Survey Response Process in Large Companies.” In Proceedings of the Second International Conference on Establishment Surveys, June 17-21, 2000: 327–337. Buffalo, New York. American Statistical Association.

40.

Tomaskovic-Devey, D., J. Leiter, and S. Thompson. 1995. “Item Nonresponse in Organizational Surveys.” Sociological Methodology 25: 77–110. DOI: https://doi.org/10.2307/271062.

41.

Wagner, J. 2012. “Average Wage, Qualification of the Workforce and Export Performance in German Enterprises: Evidence from KombiFiD Data.” Journal for Labour Market Research 45: 161–170. DOI: https://doi.org/10.1007/s12651-012-0106-x.

42.

Willimack, D.K., E. Nichols, and S. Sudman. 2002. “Understanding Unit and Item Nonresponse in Business Surveys.” In Survey Nonresponse, edited by R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little, 213–227. New York: Wiley.

43.

Willimack, D.K., and E. Nichols. 2010. “A Hybrid Response Process Model for Business Surveys.” Journal of Official Statistics 26 (1): 3–24. Available at: https://www.scb.se/-contentassets/f6bcee6f397c4fd68db6452fc9643e68/a-hybrid-response-process-model-for-business-surveys.pdf%20 (accessed November 2021).

44.

Willimack, D.K., and G. Snijkers. 2013. “The Business Context and Its Implications for the Survey Response Process.” In Designing and Conducting Business Surveys, edited by G. Snijkers, G. Haraldsen, J. Jones, and D.K. Willimack, 39–82. Hoboken, NJ: John Wiley & Sons.