The Kaufman Adolescent and Adult Intelligence Test (KAIT™) is an individually administered test of intelligence for individuals ranging in age from 11 to 85+ years. The item response theory—likelihood ratio procedure, based on the two-parameter logistic model, was used to detect differential item functioning (DIF) in the KAIT across males and females in the standardization sample. Root mean squared differences and item parameter differences were used to indicate the magnitude of DIF and identify which group the item parameter favored. A z test of proportion differences was conducted to determine if the number of parameters exhibiting gender DIF exceeded the number expected by chance, estimated by randomly dividing the sample in half and repeating the analyses. Of the 176 item parameters examined, 42 (24%) lacked invariance, with most items reporting uniform DIF. Implications for test score interpretation and future research are discussed.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. ( 1999). Standards for educational and psychological testing . Washington, DC: Author.
2.
Anastasi, A. ( 1958). Differential psychology (3rd ed.). New York: Macmillan.
3.
Anderson, M. ( 2004). Sex differences in general intelligence. In R. L. Gregory (Ed.), The Oxford companion to the mind (pp. 828-829). Oxford, UK: Oxford University Press.
4.
Angoff, W.H. ( 1972, September). A technique for the investigation of cultural differences. Paper presented at the annual meeting of the American Psychological Association, Honolulu . (ERIC Document Reproduction Service No. ED 069686)
5.
Angoff, W.H. ( 1982). Use of difficulty and discrimination indices for detecting item bias. In R. A. Berk (Ed.), Handbook of methods for detecting test bias (pp. 96-116). Baltimore : Johns Hopkins University Press.
6.
Angoff, W.H. ( 1993). Perspectives on differential item functioning methodology . In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 3-24). Hillsdale, NJ: Lawrence Erlbaum.
7.
Berk, R. A. (Ed.). (1982). Handbook of methods for detecting test bias. Baltimore : Johns Hopkins University Press.
8.
Bock, R.D., & Aitkin, M. ( 1981). Marginal maximum likelihood estimation of item parameters: An application of the EM algorithm. Psychometrika , 46, 443-449.
9.
Borsboom, D. ( 2006). The attack of the psychometricians. Psychometrika, 71, 425-440.
10.
Camilli, G. ( 1994). Origin of the scaling constant "d" = 1.7 in item response theory. Journal of Educational and Behavioral Statistics , 19, 293-295.
11.
Cohen, A.S., Kim, S., & Wollack, J.A. ( 1996). An investigation of the likelihood ratio test for detection of differential item functioning. Applied Psychological Measurement , 20, 15-26.
12.
Dai, X., & Lynn, R. ( 1994). Gender differences in intelligence among Chinese children . Journal of Social Psychology, 134, 123-125.
13.
Diana v California State Board of Education. (1970). No. C-70-37, United States District Court of Northern California.
14.
Engelhard, G., Hansche, L., & Rutledge, K.E. (1990). Accuracy of bias review judges in identifying differential item functioning on teacher certification tests. Applied Measurement in Education, 3, 347-360.
15.
Feingold, A. ( 1992). Sex differences in variability in intellectual abilities: A new look at an old controversy. Review of Educational Research , 62, 61-84.
16.
Feingold, A. ( 1993). Cognitive gender differences: A developmental perspective . Sex Roles, 29, 91-112.
17.
Flowers, C.P., Oshima, T.C., & Raju, N.S. ( 1999). A description and demonstration of the polytomous-DFIIT framework. Applied Psychological Measurement, 23, 309-326.
18.
Golden Rule Insurance Company et al. v. Washburn et al. (1984). 419-76 (stipulation for dismissal and order dismissing case, filed in the Circuit Court of the Seventh Judicial Circuit , Sangamon County, IL, 1984).
19.
Herrnstein, R.J., & Murray, C. ( 1994). The bell curve. New York: Free Press.
20.
Hildalgo, M.D., & Lόpez-Pina, J.A. (2004). Differential item functioning detection and effect size: A comparison between logistic regression and Mantel-Haenszel procedures. Educational and Psychological Measurement, 64, 903-915.
21.
Holland, P.W., & Thayer, D.T. ( 1986). Differential item functioning and the Mantel-Haenszel procedure (Technical Rep. No. 86-69). Princeton, NJ: Educational Testing Service .
22.
Holland, P.W., & Thayer, D.T. ( 1988). Differential item performance and the Mantel-Haentzel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum.
23.
Holland, P.W., & Wainer, H. ( 1993). Differential item functioning. Hillsdale, NJ: Lawrence Erlbaum.
24.
Hong, S., & Rosnowski, M. ( 2001). An investigation of the influence of internal test bias on regression slope. Applied Measurement in Education , 14, 351-368.
25.
Horn, J.L., & Cattell, R.B. ( 1966). Refinement and test of the theory of fluid and crystallized intelligence. Journal of Educational Psychology , 57, 253-270.
26.
Hutton, J.B., Dubes, R., & Muir, S. ( 1992). Assessment practices of school psychologists: Ten years later. School Psychology Review, 21, 271-284.
27.
Hyde, J.S., & Linn, M.C. ( 1988). Gender differences in verbal ability: A meta-analysis . Psychological Bulletin, 104, 53-69.
28.
Irwing, P., & Lynn, R. ( 2005). Sex differences in means and variability on the progressive matrices in university students: A meta-analysis. British Journal of Psychology, 96, 505-524.
29.
Jenson, A.R. ( 1980). Bias in mental testing. New York: Free Press.
30.
Kaufman, A.S., & Kaufman, N.L. ( 1993). Kaufman Adolescent & Adult Intelligence Test . Circle Pines, MN: American Guidance Service.
31.
Kaufman, A.S., Kaufman, J.C., & MacLean, J.E. ( 1994). Factor structure of the Kaufman Adolescent and Adult Intelligence Test (KAIT) for Caucasian, African Americans, and Hispanics . Educational and Psychological Measurement, 55, 365-376.
32.
Kaufman, A.S., MacLean, J.E., & Reynolds, C.R. (1988). Sex, race, residence, region, and education differences on the 11 WAIS-R subtests. Journal of Clinical Psychology, 44, 231-248.
33.
Keith, T.Z. ( 1997). Using confirmatory factor analysis to aid in understanding the constructs measured by intelligence tests. In D. P. Flanagan, J. L. Genshaft, & P. L. Harrison (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (pp. 373-403). New York: Guilford Press.
34.
Larry P. et al. v. Wilson Riles. (1979). Superintendent of Public Instruction for the State of California et al. Case No. C-71-2270 (N.D. Cal. 1979).
35.
Linn, R., & Drasgow, F. ( 1987). Implications of the Golden Rule settlement for test construction . Educational Measurement: Issues and Practice , 6, 13-17.
36.
Lord, F.M. ( 1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
37.
Lord, F.M., & Novick, M. ( 1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
38.
Maccoby, E.E. ( 1966). Sex differences in intellectual functioning. In E. E. Maccoby (Ed.), The development of sex differences (pp. 25-55). Stanford, CA: Stanford University Press.
39.
Maccoby, E.E., & Jacklin, C.N. ( 1974). The psychology of sex differences. Stanford, CA: Stanford University Press.
40.
Maller, S.J. ( 1997). Deafness and WISC-III item difficulty: Invariance and fit. Journal of School Psychology, 35, 299-314.
41.
Maller, S.J. ( 2000). Item invariance in four subtests of the Universal Nonverbal Intelligence Test across groups of deaf and hearing children. Journal of Psychoeducational Assessment, 18, 240-254.
42.
Maller, S.J. ( 2001). Differential item functioning in the WISC-III: Item parameters for boys and girls in the national standardization sample . Educational and Psychological Measurement, 61, 793-817.
43.
Maller, S.J., (2003). Best practices in detecting bias in nonverbal tests. In S. McCallum (Ed.), Handbook of nonverbal assessment of intelligence and related abilities (pp. 23-48). Need-ham Heights, MA: Allyn & Bacon.
44.
Mazor, K.M., Clauser, B.E., & Hambleton, R.K. (1994). Identification of nonuniform differential item functioning using a variation of the Mantel-Haenszel procedure . Educational and Psychological Measurement, 54, 284-291.
45.
Messick, S. ( 1989). Meaning and values in test validation: The science and ethics of assessment. Educational Researcher, 18, 5-11.
46.
Messick, S. ( 1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749.
47.
O’Neill, K.A., & McPeek, W.M. (1993). Item and test characteristics that are associated with differential item functioning. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 67-114). Hillsdale, NJ: Lawrence Erlbaum.
48.
Oshima, T.C., & Morris, S.B. ( 2008). Raju’s differential functioning of items and tests (DFIT). Educational Measurement: Issues and Practice , 27, 43-50.
49.
Penfield, R.D., & Algina, J. ( 2006). A generalized DIF effect variance estimator for measuring unsigned differential test functioning in mixed format tests. Journal of Educational Measurement, 43, 295-312.
50.
Plake, B.S. ( 1981). An ANOVA methodology to identify biased test items that takes instructional level into account. Educational and Psychological Measurement, 41, 365-368.
51.
Raju, N.S., Laffitte, L.J., & Byrne, B.M. ( 2002). Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory. Journal of Applied Psychology, 87, 517-529.
52.
Raju, N.S., van der Linden, W.J., & Fleer, P.F. ( 1995). An IRT-based internal measure of test bias with applications for differential item functioning. Applied Psychological Measurement , 19, 353-368.
53.
Reise, S.P., Widaman, K.F., & Pugh, R.H. ( 1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114, 552-566.
54.
Rosnowski, M., & Reith, J. ( 1999). Examining the measurement quality of tests containing differentially functioning items: Do biased items result in poor measurement . Educational and Psychological Measurement, 59, 248-269.
55.
Ryan, K.E., & Fan, M. ( 1996). Examining gender DIF on a multiple-choice test of mathematics: A confirmatory approach. Educational Measurement: Issues and Practice, 15, 15-20.
56.
Sattler, J.M. ( 2001). Assessment of children (4th ed.). San Diego, CA: Author.
57.
Shealy, R.T., & Stout, W.F. ( 1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/ DIF. Psychometrika, 58, 159-194.
58.
Stinnett, T.A., Havey, J.M., & Oehler-Stinnett, J. (1994). Current test usage by practicing psychologists: A national survey. Journal of Psychoeducational Assessment, 12, 331-350.
59.
Stout, W., Groelich, A.G., & Gao, F. ( 2001). Using resampling methods to produce an improved DIMTEST procedure. In A. Boomsma, M. A. J. Duijn, T. A. B. Snijders (Eds.), Essay on item response theory (pp. 357-376). New York: Spring-Verlag.
Thissen, D., Cheng, W., & Bock, D. ( 2003). MULTILOG for Windows (Version 7.0) [Computer software] . Chicago: Scientific Software International .
62.
Thissen, D., Steinberg, L., & Wainer, H. ( 1988). Use of item response theory in the study of group differences in trace lines. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 149-169). Hillsdale, NJ: Lawrence Erlbaum.
63.
Thissen, D., Steinberg, L., & Wainer, H. ( 1993). Detection of differential item functioning using the parameters of item response theory. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 67-114). Hillsdale, NJ : Lawrence Erlbaum.
U.S. Census Bureau. (1988). Projections of the population of states, by age, sex, and race: 1988 to 2010 (Current population Reports, Series P-20, No. 451, Table 3). Washington, DC: Government Printing Office.
66.
Wechsler, D. ( 1981). Wechsler Adult Intelligence Scale-Revised. San Antonio, TX: Psychological Corporation .
67.
Wechsler, D. ( 1991). Wechsler Intelligence Scale for Children (3rd ed.). New York: Psychological Corporation.
68.
Wechsler, D. ( 2004). Wechsler Intelligence Scale for Children (4th ed.). New York: Psychological Corporation.
