In questionnaires, items can be presented in a grouped format (same-scale items are presented in the same block) or in a randomized format (items from one scale are mixed with items from other scales). Some researchers have advocated the grouped format because it enhances discriminant validity. The current study demonstrates that positioning items in separate blocks of a questionnaire may indeed lead to increased discriminant validity, but this can happen even in instances where discriminant validity should not be present. In particular, the authors show that splitting an established unidimensional scale into two arbitrary blocks of items separated by unrelated buffer items results in the emergence of two clearly identifiable but artificial factors that show discriminant validity.
ArtinoA. R.Jr.GehlbachH.DurningS. J. (2011). AM last page: Avoiding five common pitfalls of survey design. Academic Medicine: Journal of the Association of American Medical Colleges, 86, 1327-1327.
2.
BillietJ.DavidovE. (2008). Testing the stability of an acquiescence style factor behind two interrelated substantive variables in a panel design. Sociological Methods & Research, 36, 542-562.
3.
ConteJ. M.DeanM. A.RingenbachK. L.MoranS. K.LandyF. J. (2005). The relationship between work attitudes and job analysis ratings: Do rating scale type and task discretion matter?Human Performance, 18, 1-21.
4.
ConverseJ. M.PresserS. (1986). Survey questions: Handcrafting the standardized questionnaire. Beverly Hills, CA: Sage.
5.
EmonsW. H. M. (2009). Detection and diagnosis of person misfit from patterns of summed polytomous item scores. Applied Psychological Measurement, 33, 599-619.
6.
FornellCLarckerD. F. (1981). Evaluating structural equation models with unobserved variables and measurement errors. Journal of Marketing Research, 18, 39-50.
7.
FowlerF. J. (2009). Survey research methods (4th ed.). Thousand Oaks, CA: Sage.
8.
FrankeG. H. (1997). “The whole is more than the sum of its parts”: The effects of grouping and randomizing items on the reliability and validity of questionnaires. European Journal of Psychological Assessment, 13, 67-74.
9.
GehlbachH.BargeS. (2012). Anchoring and adjusting in questionnaire responses. Basic and Applied Social Psychology, 34, 417-433.
10.
HarrisonD. A.McLaughlinM. E. (1996). Structural properties and psychometric qualities of organizational self-reports: Field tests of connections predicted by cognitive theory. Journal of Management, 22, 313-338.
11.
HuL.-T.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1-55.
12.
LastovickaJ. L.BettencourtL. A.HughnerR. S.KuntzeR. J. (1999). Lifestyle of the tight and frugal: Theory and measurement. Journal of Consumer Research, 26, 85-98.
13.
LehmannD. R. (1988). An alternative procedure for assessing convergent and discriminant validity. Applied Psychological Measurement, 12, 411-423.
14.
McFarlandL. A.RyanA. M.EllisA. (2002). Item placement on a personality measure: Effects on faking behavior and test measurement properties. Journal of Personality Assessment, 78, 348-369.
15.
RickS. I.CryderC. E.LoewensteinG. (2008). Tightwads and spendthrifts. Journal of Consumer Research, 34, 767-782.
16.
SchriesheimC. A. (1981a). The effect of grouping or randomizing items on leniency response bias. Educational and Psychological Measurement, 41, 401-411.
17.
SchriesheimC. A. (1981b). Leniency effects on convergent and discriminant validity for grouped questionnaire items: A further investigations. Educational and Psychological Measurement, 41, 1093-1099.
18.
SchriesheimC. A.DeNisiA. S. (1980). Item presentation as an influence on questionnaire validity: A field experiment. Educational and Psychological Measurement, 40, 175-182.
19.
SchriesheimC. A.SolomonE.KopelmanR. E. (1989a). The effect of grouped versus randomized questionnaire format on scale reliability and validity: A three-study investigation. Educational and Psychological Measurement, 49, 487-508.
20.
SchriesheimC. A.SolomonE.KopelmanR. E. (1989b). Grouped versus randomized format: An investigation of scale convergent and discriminant validity using LISREL confirmatory factor analysis. Applied Psychological Measurement, 13, 19-31.
21.
ShippA. J.EdwardsJ. R.LambertL. S. (2009). Conceptualization and measurement of temporal focus: The subjective experience of the past, present, and future. Organizational Behavior and Human Decision Processes, 110, 1-22.
22.
TourangeauR.CouperM. P.ConradF. (2004). Spacing, position, and order: Interpretive heuristics for visual features of survey questions. Public Opinion Quarterly, 68, 368-393.
23.
TourangeauR.RipsL. J.RasinskiK. A. (2000). The psychology of survey response. Cambridge, England: Cambridge University Press.
24.
WeijtersB.CabooterE.SchillewaertN. (2010). The effect of rating scale format on response styles: The number of response categories and response category labels. International Journal of Research in Marketing, 27, 236-247.
25.
WeijtersB.GeuensM.SchillewaertN. (2009). The proximity effect: The role of interitem distance on reverse-item bias. International Journal of Research in Marketing, 26, 2-12.
26.
WeijtersB.GeuensM.SchillewaertN. (2010a). The individual consistency of acquiescence and extreme response style in self-report questionnaires. Applied Psychological Measurement, 34, 105-121.
27.
WeijtersB.GeuensM.SchillewaertN. (2010b). The stability of individual response styles. Psychological Methods, 15, 96-110.
