Mantel-Haenszel methods comprise a highly flexible methodology for assessing the degree of association between two categorical variables, whether they are nominal or ordinal, while controlling for other variables. The versatility of Mantel-Haenszel analytical approaches has made them very popular in the assessment of the differential functioning of both dichotomous and polytomous items. Up to now, researchers have limited the use of Mantel-Haenszel statistics to analyzing contingency tables of dimensions 2 × 2 (by means of the Mantel-Haenszel chi-square statistic) and of dimensions of 2 × C (by means of either the generalized Mantel-Haenszel test or Mantel's test). The main objective of this article is to illustrate a unified framework for the analysis of differential item functioning using the Mantel-Haenszel methods. This is done by means of the generalized Mantel-Haenszel statistic for the analysis of the general case of Q contingency tables with dimensions R × C. Moreover, with the new formulation in consideration, this article reviews the most recent research on differential item functioning and suggests new applications and research lines in relation to the statistics proposed.
Agresti, A. (1990). Categorical data analysis. New York: Wiley.
2.
Allen, N.L., & Donoghue, J.R. (1996). Applying the Mantel-Haenszel procedure to complex samples of items. Journal of Educational Measurement, 33, 231-251.
3.
Ankenmann, R.D., Witt, E.A., & Dunbar, S.B. (1999). An investigation of the power of the likelihood ratio goodness-of-fit statistic in detecting differential item functioning . Journal of Educational Measurement, 36(4), 277-300.
4.
Birch, M.W. (1964). The detection of partial association, I: The 2 × 2 case. Journal of the Royal Statistical Society, Series B, 26, 313-324.
5.
Birch, M.W. (1965). The detection of partial association, II: The general case. Journal of the Royal Statistical Society, Series B, 27, 111-124.
6.
Camilli, G., & Congdon, P. (1999). Application of a method of estimating DIF for polytomous test items. Journal of Educational and Behavioural Statistics, 4, 323-341.
7.
Camilli, G., & Penfield, D.A. (1997). Variance estimation for differential test functioning based on Mantel-Haenszel statistics. Journal of Educational Measurement, 34, 123-139.
8.
Camilli, G., & Shepard, L.A. (1994). Methods for identifying biased test items. Newbury Park, CA: Sage.
9.
Camilli, G., & Smith, J.K. (1990). Comparison of the Mantel-Haenszel test with a randomized and a jackknife test for detecting biased items. Journal of Educational Statistics, 15, 53-67.
10.
Chang, H.H., Mazzeo, J., & Roussos, J. (1996). Detecting DIF for polytomously scored items: An adaptation of the SIBTEST procedure. Journal of Educational Measurement, 33, 333-353.
11.
Clauser, B.E., & Mazor, K.M. (1998). Using statistical procedures to identify differentially functioning test items. Educational Measurement: Issues and Practice, 17, 31-44.
12.
Clauser, B., Mazor, K.M., & Hambleton, R.K. (1993). The effects of purification of the matching criterion on the identification of DIF using the Mantel-Haenszel procedure. Applied Measurement in Education, 6, 269-279.
13.
Cochran, W.G. (1954). Some methods for strengthening the common χ2 test. Biometrics, 10, 417-451.
14.
Cox, D.R. (1958). The regression analysis of binary sequences. Journal of the Royal Statistical Society, Series B, 20, 215-232.
15.
Donoghue, J.R., Holland, P.W., & Thayer, D.T. (1993). A Monte Carlo study of factors that affect the Mantel-Haenszel and standardization measures of differential item functioning . In W. P. Holland & H. Wainer (Eds.), Differential item functioning (pp. 137-166). Hillsdale, NJ: Lawrence Erlbaum.
16.
Fidalgo, A.M. (2005). Mantel-Haenszel methods. In B. S. Everitt & D. C. Howell (Eds.), Encyclopedia of statistics in behavioral science (Vol. 3, pp. 1120-1126). Chichester, UK: Wiley.
17.
Fidalgo, A.M., & Bartram, D. (2008). DIF detection in polytomous items using the generalized ordinal Mantel-Haenszel statistic. What type of scores should be selected ? Manuscript submitted for publication.
18.
Fidalgo, A.M., Ferreres, D., & Muñiz, J. (2004). Utility of the Mantel-Haenszel procedure for detecting differential item functioning with small samples. Educational and Psychological Measurement, 64, 925-936.
19.
Fidalgo, A.M., Hashimoto, K., Bartram, D., & Muñiz, J. (2007). Empirical Bayes versus standard Mantel-Haenszel statistics for detecting differential item functioning under small sample conditions. Journal of Experimental Education, 75, 293-314.
20.
Fidalgo, A.M., Mellenbergh, G.J., & Muñiz, J. (2000). Effects of amount of DIF, test length, and purification type on robustness and power of Mantel-Haenszel procedures. Methods of Psychological Research, 5, 43-53.
21.
Fischer, G. (1995). Some neglected problems in IRT. Psychometrika, 60, 459-487.
22.
Graubard, B.I., & Korn, E.L. (1987). Choice of column scores for testing independence in ordered 2 × K contingency tables. Biometrics, 43, 471-476.
23.
Greenland, S. (1989). Generalized Mantel-Haenszel estimators for K 2 × J tables. Biometrics, 45, 183-191.
24.
Hidalgo, M.D., & López-Pina J. A. (2004). Differential item functioning detection and effect size: A comparison between logistic regression and Mantel-Haenszel procedures. Educational and Psychological Measurement, 64, 903-915.
25.
Holland, W.P., & Thayer, D.T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum.
26.
Koch, G.G., Imrey, P.B., Singer, J.M., Atkinson, S.S., & Stokes, M.E. (1985). Analysis of categorial data. Montreal, Canada: Presses de l'Université de Montreal .
27.
Koch, G.G., Sen, P.K., & Amara, I.A. (1985). Log-rank scores, statistics and test. In N. L. Johnson & S. Kotz (Eds.), Encyclopedia of statistical sciences (Vol. 5, pp. 136-142). New York: John Wiley.
28.
Kristjansson, E., Aylesworth, R., McDowell, I., & Zumbo, B.D. (2005). A comparison of four methods for detecting differential item functioning in ordered response items. Educational and Psychological Measurement, 65, 935-953.
29.
Kuritz, S.J., Landis, J.R., & Koch, G.G. (1988). A general overview of Mantel-Haenszel methods: Applications and recent developments. Annual Review of Public Health, 9, 123-160.
30.
Landis, J.R., Heyman, E.R., & Koch, G.G. (1978). Average partial association in three-way contingency tables: A review and discussion of alternative tests. International Statistical Review, 46, 237-254.
31.
Liu, I.-M., & Agresti, A. (1996). Mantel-Haenszel-type inference for cumulative odds ratios with a stratified ordinal response. Biometrics , 52, 1223-1234.
32.
Mantel, N. (1963). Chi-square tests with one degree of freedom: Extensions of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58, 690-700.
33.
Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719-748.
34.
Mazor, K.M., Clauser, B.E., & Hambleton, R.K. (1992). The effect of sample size on the functioning of the Mantel-Haenszel statistic. Educational and Psychological Measurement, 52, 443-452.
35.
Mazor, K.M., Clauser, B.E., & Hambleton, R.K. (1994). Identification of nonuniform differential item functioning using a variation of the Mantel-Haenszel procedure . Educational and Psychological Measurement, 54, 284-291.
36.
Meyer, J.P., Huynh, H., & Seaman, M.A. (2004). Exact small-sample differential item functioning for polytomous items with illustration based on an attitude survey. Journal of Educational Measurement, 41, 331-344.
37.
Mickey, R.M., & Elashoff, R.M. (1985). A generalization of the Mantel-Haenszel estimator of partial association for 2 × J × K tables. Biometrics, 41, 623-635.
Muñiz, J., Hambleton, R.K., & Xing, D. (2001). Small sample studies to detect flaws in item translations . International Journal of Testing, 1, 115-135.
40.
Narayanan, P., & Swaminathan, H. (1994). Performance of the Mantel-Haenszel and simultaneous item bias procedures for detecting differential item functioning . Applied Psychological Measurement, 18, 315-328.
41.
Narayanan, P., & Swaminathan, H. (1996). Identification of items that show nonuniform DIF. Applied Psychological Measurement , 20, 257-274.
42.
Parshall, C.G., & Miller, T.R. (1995). Exact versus asymptotic Mantel-Haenszel DIF statistics: A comparison of performance under small-sample conditions. Journal of Educational Measurement, 32, 302-316.
43.
Penfield, R.D. (2001). Assessing differential item functioning among multiple groups: A comparison of three Mantel-Haenszel procedures. Applied Measurement in Education, 14, 235-259.
44.
Penfield, R.D., & Algina, J. (2003). Applying the Liu-Agresti estimator of the cumulative common odds ratio to DIF detection in polytomous items. Journal of Educational Measurement, 40(4), 353-370.
45.
Penfield, R.D., & Lam, T.C.M. (2000). Assessing differential item functioning in performance assessment: Review and recommendations. Educational Measurement: Issues and Practice, 19, 5-15.
46.
Penny, J., & Johnson, R.L. (1999). How group differences in matching criterion distribution and IRT item difficulty can influence the magnitude of the Mantel-Haenszel chi-square DIF index. Journal of Experimental Education, 67, 343-366.
47.
Potenza, M.T., & Dorans, N.J. (1995). DIF assessment for polytomously scored items: A framework for classification and evaluation. Applied Psychological Measurement, 19, 23-37.
48.
Robins, J., Breslow, N., & Greenland, S. (1986). Estimators of the Mantel-Haenszel variance consistent in both sparse data and large-strata limiting models. Biometrics , 42, 311-324.
49.
Rogers, H.J., & Swaminathan, H. (1993). A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17, 105-116.
50.
Roussos, L.A., Schnipke, D.L., & Pashley, P.J. (1999). A generalized formula for the Mantel-Haenszel differential item functioning parameter. Journal of Educational and Behavioral Statistics, 24(3), 293-322.
51.
Roussos, L.A., & Stout, W.F. (1996). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel Type I error performance. Journal of Educational Measurement, 33, 215-230.
52.
Ryan, K.E., & Chiu, S. (2001). An examination of item context effects, DIF, and gender DIF. Applied Measurement in Education, 14, 73-90.
53.
Shealy, R., & Stout, W. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58, 159-194.
54.
Stokes, M.E., Davis, C.S., & Koch, G.G. (2000). Categorical data analysis using the SAS system (2nd ed.). Cary, NC: SAS Institute.
55.
Su, Y.-H., & Wang, W.-C. (2005). Efficiency of the Mantel, generalized Mantel-Haenszel, and logistic discriminant function analysis methods in detecting differential item functioning in polytomous items. Applied Measurement in Education, 18, 313-350.
56.
Uttaro, T., & Millsap, R.E. (1994). Factors influencing the Mantel-Haenszel procedure in the detection of differential item functioning. Applied Psychological Measurement, 18, 15-25.
57.
Walker, C.M., & Beretvas, S.N. (2001). An empirical investigation demonstrating the multidimensional DIF paradigm: A cognitive explanation for DIF. Journal of Educational Measurement, 38, 147-163.
58.
Wang, W.-C., & Su, Y.-H. (2004a). Effect of average signed area between two item characteristic curves and test purification procedures on the DIF detection via the Mantel-Haenszel method. Applied Measurement in Education , 17, 113-144.
59.
Wang, W.-C., & Su, Y.-H. (2004b). Factors influencing the Mantel and generalized Mantel-Haenszel methods for the assessment of differential item functioning in polytomous items. Applied Psychological Measurement, 28, 450-480.
60.
Welch, C., & Hoover, H.D. (1993). Procedures for extending item bias detection techniques to polytomously scored items. Applied Measurement in Education , 6, 1-19.
61.
Yanagawa, T., & Fujii, Y. (1995). Projection-method Mantel-Haenszel estimator for K 2 × J tables. Journal of the American Statistical Association , 90, 649-656.
62.
Zieky, M. (1993). Practical questions in the use of DIF statistics in test development. In W.P. Holland & H. Wainer (Eds.), Differential item functioning (pp. 337-347). Hillsdale, NJ: Lawrence Erlbaum.
63.
Zwick, R. (1990). When do item response function and Mantel-Haenszel definitions of differential item functioning coincide? Journal of Educational Statistics, 15, 185-197.
64.
Zwick, R., Donoghue, J.R., & Grima, A. (1993a). Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30, 233-251.
65.
Zwick, R., Donoghue, J.R., & Grima, A. (1993b). Assessing differential item functioning in performance tests (ETS Research Rep. No. 93-14). Princeton, NJ: Educational testing Service .
66.
Zwick, R., & Thayer, D.T. (1996). Evaluating the magnitude of differential item functioning in polytomous items. Journal of Educational and Behavioural Statistics, 21, 187-201.
67.
Zwick, R., Thayer, D.T., & Mazzeo, J. (1997). Descriptive and inferential procedures for assessing differential item functioning in polytomous items. Applied Measurement in Education, 10, 321-344.
