Dimensionality assessment using the full-information item bifactor model for graded response data is provided. The model applies to data in which each item relates to a general factor and one group factor. Specifically, alternative model specification within item response theory (IRT) is shown to test a scale's factor structure. For illustrative purposes, the bifactor model and competing IRT models were fit to the data of separate cohorts of incoming college students (Cohort 1, n = 1,490; Cohort 2, n = 1,533) to test the dimensionality of an adapted version of the State Metacognitive Inventory. Overall, the bifactor analysis did not strongly support distinct group factors after accounting for the general factor. Instead, results suggested conceptualizing the scale as unidimensional, indicating that scores should be based on the total scale, not subscales. Considerations related to the use of the bifactor IRT model are discussed.
Ackerman, T.A. (1987, April). The robustness of LOGIST and BILOG IRT estimation programs to violations of local independence. Paper presented at the annual meeting of the American Educational Research Association, Washington, DC. (ERIC Documentation Reproduction Service No. ED284902)
2.
Ackerman, T.A. (2005). Multidimensional item response theory modeling . In A. Maydeu-Olivares & J. J. McArdle (Eds.), Contemporary psychometrics (pp. 3-26). Mahwah, NJ: Lawrence Erlbaum.
3.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing . Washington, DC: American Educational Research Association.
4.
Ansley, T.N., & Forsyth, R.A. (1985). An examination of the characteristics of unidimensional IRT parameter estimates derived from two-dimensional data. Applied Psychological Measurement, 9, 37-48. Bandura, A. (1997). Self-efficacy: The exercise of control. New York : W. H. Freeman.
5.
Bartholomew, D.J. (1980). Factor analysis for categorical data. Journal of the Royal Statistical Society, Series B, 42, 293-321.
6.
Beck, C.T., & Gable, R.K. (2001). Item response theory in affective instrument development: An illustration. Journal of Nursing Measurement, 9, 5-22.
7.
Bock, R.D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika , 46, 443-459.
8.
Bock, R.D., & Gibbons, R.D. (in press). Factor analysis of categorical item responses. In R. Ostini & M. Nering (Eds.), Handbook of polytomous item response theory models: Development and applications. Thousand Oaks, CA : Sage.
Bock, R.D., & Mislevy, R.J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444.
12.
Bollen, K.A. (1989). Structural equations with latent variables. New York: John Wiley.
13.
Borsboom, D. (2006). The attack of the psychometricians. Psychometrika, 71, 425-440.
14.
Byrne, B.M., & Shavelson, R.J. (1987). Adolescent self-concept: Testing the assumption of equivalent structure across gender. American Educational Research Journal, 24, 365-385.
15.
Chen, F.F., West, S.G., & Sousa, K.H. (2006). A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research, 41, 189-225.
16.
Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory . Fort Worth, TX: Harcourt Brace .
17.
Demars, C.E. (2006). Application of the bifactor multidimensional item response theory model to testlet-based tests. Journal of Educational Measurement, 43, 145-168.
18.
Embretson, S.E. (2004). The second century of ability testing: Some predictions and speculations. Measurement, 2, 1-32.
19.
Floyd, F.J., & Widaman, K.F. (1995). Factor analysis in the development and refinement of clinical assessment instruments. Psychological Assessment , 7, 286-299.
20.
Gibbons, R.D., Bock, R.D., Hedeker, D., Weiss, D.J., Segawa, E., Bhaumik, D., et al. (2007). Full-information item bi-factor analysis of graded response data. Applied Psychological Measurement , 31, 4-19.
Gibbons, R.D., & Hedeker, D. (n.d.). POLYBIF [Computer software]. Chicago: Center for Health Statistics, University of Illinois. Retrieved August 22, 2007, from http://www.uic.edu/labs/biostat/projects.html
23.
Gibbons, R.D., Immekus, J.C., & Bock, R.D. (2007). Didactic workbook: The added value of multidimensional IRT models (Final Report for National Cancer Institute Contract No. 2005-05828-00-00) . Retrieved January 3, 2008, from http://www.uic.edu/labs/biostat/papers.html
24.
Hambleton, R.K., & Swaminathan, H. (1985). Item response theory: Principle and applications. Boston: Kluwer .
25.
Henson, R.K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurement and Evaluation in Counseling and Development, 34, 177-189.
26.
Holzinger, K.J., & Swineford, R. (1937). The bifactor method. Psychometrika , 2, 41-54.
27.
Jöreskog, K.G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183-202.
28.
Jöreskog, K.G. (1990). New developments in LISREL: Analysis of ordinal variables using polychoric correlations and weighted least squares. Quality and Quantity, 24, 387-404.
29.
Leonelli, B.T., Chang, C., Bock, R.D., & Schilling, S.G. (2000). Interpretation of a full-information item-level factor analysis of the MMPI-2: Normative sampling and nonpathognomonic descriptors. Journal of Personality Assessment, 74, 400-422.
30.
Li, Y., Bolt, D.M., & Fu, J. (2006). A comparison of alternative methods for testlets . Applied Psychological Measurement, 30, 3-20.
31.
Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
32.
Lord, F.M., & Novick, M. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749.
35.
Muraki, E., & Carlson, J.E. (1995). Full-information factor analysis for polytomous item responses. Applied Psychological Measurement, 19, 73-90.
36.
Múthen, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49, 115-132.
37.
Nandakumar, R., & Stout, W. (1993). Refinements of Stout's procedure for assessing latent trait unidimensionality. Journal of Educational Statistics , 18, 41-68.
38.
O'Neil, H.F., & Abedi, J. (1996). Reliability and validity of a state metacognitive inventory: Potential for alternative assessment. Journal of Educational Research, 89, 234-245.
39.
Pintrich, P.R., & DeGroot, E.V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82, 33-40.
40.
Pintrich, P.R., & Schunk, D.H. (1999). Motivation in education: Theory, research, and applications. Englewood Cliffs, NJ: Prentice Hall.
41.
Pressley, M., & Afflerbach, P. (1995). What readers can do when they read: A summary of the results from the on-line self-report studies of reading. In M. Pressley & P. Afflerbach, Verbal protocols of reading: The nature of constructively responsive reading (pp. 31-82). Hillsdale, NJ: Lawrence Erlbaum.
42.
Reckase, M.D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9, 401-412.
43.
Reise, S.P., Morizot, J., & Hays, R.D. (2007). The role of the bi-factor model in resolving dimensionality issues in health outcomes measures. Medical Care, 16, 19-31.
44.
Samejima, F. (1969). Estimation of ability using a response pattern of graded scores. Psychometrika Monograph, 17.
45.
Samejima, F. (1997). Graded response model. In W. van der Linden & R. Hambleton (Eds.), Handbook of modern item response theory (pp. 85-100). New York: Springer.
46.
Schmeiser, C.B., & Welch, C.J. (2006). Test development. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 307-353). Westport, CT: American Council on Education and Praeger.
47.
Schunk, D.H., & Ertmer, P.A. (2000). Self-regulation and academic learning: Self-efficacy enhancing interventions. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 631-649). San Diego, CA: Academic Press.
48.
Sireci, S.D., Thissen, D., & Wainer, H. (1991). On the reliability of testlet-based tests. Journal of Educational Measurement, 28, 237-247.
49.
Stout, W. (1987). A nonparametric approach for assessing latent trait dimensionality. Psychometrika, 57, 423-436.
50.
Wainer, H., & Wang, C. (2000). Using a new statistical model for testlets to score TOEFL. Journal of Educational Measurement, 37, 203-220.
51.
Way, W.D., & Forsyth, R.A. (1988). The comparative effects of compensatory and noncompensatory two-dimensional data on unidimensional estimates. Applied Psychological Measurement, 12, 239-252.
52.
Woods, C.M. (2002). Factor analysis of scales composed of binary items: Illustration with the Maudsley Obsessional Compulsive Inventory. Journal of Psychopathology and Behavioral Assessment, 24, 215-223.
53.
Yen, W.M., & Fitzpatrick, A.R. (2006). Item response theory. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 111-153). Westport, CT: American Council on Education and Praeger .
54.
Zhang, J., & Stout, W. (1999). The theoretical DETECT index of dimensionality and its application to approximate simple structure. Psychometrika , 64, 213-249.
55.
Zimmerman, B.J. (1989). A social cognitive view of self-regulated academic learning. Journal of Educational Psychology, 81, 329-339.
