As scale development researchers, we have followed Mallinckrodt and colleagues’ work over recent years with significant interest. The scale development process can be challenging and typically demands a large investment of time. However, the end of the process is rewarding when the resulting scale is psychometrically sound. Mallinckrodt, Miles, and Recabarren’s contribution in this volume of The Counseling Psychologist attempted to help integrate the use of item response theory in the scale development process. Although they are to be commended for this endeavor, there are some limitations inherent in their contribution. This reaction provides a critical evaluation of their work to augment their recommendations.
American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author.
2.
AndrichD. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561-573.
3.
CookeD. J.MichieC. (1997). An item response theory analysis of the Hare Psychopathy Checklist–Revised. Psychological Assessment, 9, 3-14.
4.
de AyalaR. J. (2009). The theory and practice of item response theory. New York, NY: The Guilford Press.
5.
DeMarsC. (2010). Item response theory. Oxford, UK: Oxford University Press.
6.
EllisB. B.BeckerP.KimmelH. D. (1993). An item response theory evaluation of an English version of the Trier Personality Inventory. Journal of Cross-Cultural Psychology, 2, 133-148.
7.
EmbretsonS. E.ReiseS. P. (2000). Item response theory for psychologists. Mahwah, NJ: Erlbaum.
8.
ForeroC. G.Maydeu-OlivaresA. (2009). Estimation of IRT graded response models: Limited versus full information methods. Psychological Methods, 14, 275-299.
9.
FraleyR. C.WallerN. G.BrennanK. A. (2000). An item response theory analysis of self-report measures of adult attachment. Journal of Personality and Social Psychology, 78, 350-365.
10.
HarveyR. J.MurryW. D. (1994). Scoring the Myers–Briggs Type Indicator: Empirical comparison of preference score versus latent-trait analyses. Journal of Personality Assessment, 62, 116-129.
11.
KamataA.BauerD. J. (2008). A note on the relation between factor analytic and item response theory models. Structural Equation Modeling, 15, 136-153.
12.
LordF. M. (1968). An analysis of the Verbal Scholastic Aptitude Test using Birnbaum’s three-parameter logistic model. Educational and Psychological Measurement, 28, 989-1020.
13.
LordF. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
14.
LordF. M.NovickM. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
15.
MallinckrodtB.MilesJ. R.BhaskarT.CheryN.ChoiG.SungM.-R. (2014). Development and psychometric evaluation of a comprehensive scale to assess college diversity programming. Journal of Counseling Psychology, 61, 133-145.
16.
MallinckrodtB.MilesJ. R.RecabarrenD. A. (2016). Using focus groups and Rasch item response theory to improve instrument development. The Counseling Psychologist, 44(2), 146-194. doi:10.1177/0011000015596437
17.
MarshH. W.HauK.-T.WenZ. (2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11, 320-341.
18.
MatsunagaM. (2008). Item parceling in structural equation modeling: A primer. Communication Methods and Measures, 2, 260-293.
19.
MeadeA. W.KroustalisC. M. (2005, April). Problems with item parceling for confirmatory factor analysis tests of measurement invariance of factor loadings. Paper presented at the 20th Annual Conference of the Society for Industrial and Organizational Psychology, Los Angeles, CA.
20.
MellenberghG. J. (1989). Item bias and item response theory. International Journal of Educational Research, 13, 127-143.
21.
MeredithW.MillsapR. E. (1992). On the misuse of manifest variables in the detection of measurement bias. Psychometrika, 57, 289-311.
MuthénB.LehmanJ. (1985). Multiple group IRT modeling: Applications to item bias analysis. Journal of Educational Statistics, 10, 133-142.
24.
MuthénL. K.MuthénB. O. (1998-2015). Mplus user’s guide (7th ed.). Los Angeles, CA: Author.
25.
PeltonT. W. (2002). The accuracy of unidimensional measurement models in the presence of deviations from the underlying assumptions (Unpublished doctoral dissertation). Brigham Young University, Provo, UT.
26.
ReiseS. P.HensonJ. M. (2003). A discussion of modern versus traditional psychometrics as applied to personality assessment scales. Journal of Personality Assessment, 81, 93-103.
27.
ReiseS. P.WallerN. G. (1990). Fitting the two-parameter model to personality data. Applied Psychological Measurement, 14, 45-58.
28.
RouseS. V.FingerM. S.ButcherJ. N. (1999). Advances in clinical personality measurement: An item response theory analysis of the MMPI-2 PSY-5 scales. Journal of Personality Assessment, 72, 282-307.
29.
SamejimaF. (1969). Estimation of latent trait ability using a response pattern of graded scores. Psychometrika Monograph Supplement, 34, No. 17.
30.
StarkS.ChernyshenkoO. S.DrasgowF. (2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy. Journal of Applied Psychology, 91, 1292-1306.
31.
TakaneY.de LeeuwJ. (1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52, 393-408.
32.
ThissenD.OrlandoM. (2001). Item response theory for items scored in two categories. In ThissenD.WainerH. (Eds.), Test scoring (pp. 73-140). Mahwah, NJ: Lawrence Erlbaum.
33.
WorthingtonR. W.WhittakerT. A. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34, 806-838.
34.
YenW. M. (1981). Using simulation results to choose a latent trait model. Applied Psychological Measurement, 5, 245-262.
