Omitted variable bias (OVB) in differential item functioning (DIF) assessment can lead to biased estimates of DIF effects, increasing type I error rates and reducing statistical power. Methodologies for addressing OVB, particularly under exploratory conditions where DIF is assessed without prior information, remain underdeveloped. This study addresses these challenges by applying the propensity score (PS) method to enhance the parsimony of the DIF model. Furthermore, this study incorporated the DIF-free-then-DIF (DFTD) strategy into the PS method, establishing the PS.DFTD method for exploratory DIF assessment. The simulation results reveal that the PS.DFTD method appears promising for mitigating OVB in DIF assessment, demonstrating its potential for practical applications.
ArikanS.van de VijverF.YagmurK. (2018). Propensity score matching helps to understand sources of DIF and mathematics performance differences of Indonesian, Turkish, Australian, and Dutch students in PISA. International Journal of Research in Education and Science, 4(1), 69–81. https://doi.org/10.21890/ijres.382936
3.
BirnbaumA. (1968). Some latent trait models and their use in inferring an examinee’s ability. In LordF. M.NovickM. R. (Eds.), Statistical theories of mental test scores (pp. 397–424). Addison Wesley.
4.
BoltD. M. (2002). A Monte Carlo comparison of parametric and nonparametric polytomous DIF detection methods. Applied Measurement in Education, 15(2), 113–141. https://doi.org/10.1207/S15324818AME1502_01
5.
CaliendoM.KopeinigS. (2008). Some practical guidance for the implementation of propensity score matching. Journal of Economic Surveys, 22(1), 31–72. https://doi.org/10.1111/j.1467-6419.2007.00527.x
6.
ChaoH. Y.ChenC. C.ChengC. P.ChenJ. H. (2018). Omitted variable bias in differential item functioning assessment. Chinese Journal of Psychology, 60(4), 233–250. https://doi.org/10.6129/CJP.201812_60(4).0001
7.
ChenJ. H.ChenC. T.ShihC. L. (2014). Improving the control of type I error rate in assessing differential item functioning for hierarchical generalized linear model when impact is presented. Applied Psychological Measurement, 38(1), 18–36. https://doi.org/10.1177/0146621613488643
8.
ChenC. T.HwuB. S. (2018). Improving the assessment of differential item functioning in large-scale programs with dual-scale purification of Rasch models: The PISA example. Applied Psychological Measurement, 42(3), 206–220. https://doi.org/10.1177/0146621617726786
9.
ChenM. Y.LiuY.ZumboB. D. (2020). A propensity score method for investigating differential item functioning in performance assessment. Educational and Psychological Measurement, 80(3), 476–498. https://doi.org/10.1177/0013164419878861
10.
ChengY.ShaoC.LathropQ. N. (2016). The mediated MIMIC model for understanding the underlying mechanism of DIF. Educational and Psychological Measurement, 76(1), 43–63. https://doi.org/10.1177/0013164415576187
11.
ChunS.StarkS.KimE. S.ChernyshenkoO. S. (2016). MIMIC methods for detecting DIF among multiple groups: Exploring a new sequential-free baseline procedure. Applied Psychological Measurement, 40(7), 486–499.
12.
ClarkeK. A. (2005). The phantom menace: Omitted variable bias in econometric research. Conflict Management and Peace Science, 22(4), 341–352. https://doi.org/10.1080/07388940500339183
13.
ClarkeK. A. (2009). Return of the phantom menace: Omitted variable bias in political research. Conflict Management and Peace Science, 26(1), 46–66. https://doi.org/10.1177/0738894208097666
DeMarsC. E. (2010). Type I error inflation for detecting DIF in the presence of impact. Educational and Psychological Measurement, 70(6), 961–972. https://doi.org/10.1177/0013164410366691
16.
DouglasJ. A.RoussosL. A.StoutW. (1996). Item-bundle DIF hypothesis testing: Identifying suspect bundles and assessing their differential functioning. Journal of Educational Measurement, 33(4), 465–484. https://doi.org/10.1111/j.1745-3984.1996.tb00502.x
17.
FinchW. H. (2005). The MIMIC model as a method for detecting DIF: Comparison with Mantel-Haenszel, SIBTEST, and the IRT likelihood ratio. Applied Psychological Measurement, 29(4), 278–295. https://doi.org/10.1177/0146621605275728
18.
FinchW. H. (2016). Detection of differential item functioning for more than two groups: A Monte Carlo comparison of methods. Applied Measurement in Education, 29(1), 30–45. https://doi.org/10.1080/08957347.2015.1102916
19.
FrenchB. F.FinchW. H. (2013). Extensions of Mantel-Haenszel for multilevel DIF detection. Educational and Psychological Measurement, 73(4), 648–671. https://doi.org/10.1177/0013164412472341
20.
FrenchB. F.FinchW. H.ImmekusJ. C. (2019). Multilevel generalized Mantel-Haenszel for differential item functioning detection. Frontiers in Quantitative Psychology, 4, 47–57. https://doi.org/10.3389/feduc.2019.00047
21.
FrenchB. F.MallerS. J. (2007). Iterative purification and effect size use with logistic regression for differential item functioning detection. Educational and Psychological Measurement, 67(3), 373–393. https://doi.org/10.1177/0013164406294781
22.
GreeneW. H. (2003). Econometric analysis. Pearson Education.
23.
HauserR. M.GoldbergerA. S. (1971). The treatment of unobservable variables in path analysis. Sociological Methodology, 3, 81–117. https://www.jstor.org/stable/270819
24.
HollandP. W.ThayerD. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In WainerH.BraunH. I. (Eds.), Test validity (pp. 129–145). Erlbaum.
25.
JodoinM. G.GierlM. J. (2001). Evaluating type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education, 14(4), 329–349. https://doi.org/10.1207/S15324818AME1404_2
26.
KopfJ.ZeileisA.StroblC. (2015a). Anchor selection strategies for DIF analysis: Review, assessment, and new approaches. Educational and Psychological Measurement, 75(1), 22–56. https://doi.org/10.1177/0013164414529792
27.
KopfJ.ZeileisA.StroblC. (2015b). A framework for anchor methods and an iterative forward approach for DIF detection. Applied Psychological Measurement, 39(2), 83–103. https://doi.org/10.1177/0146621614544195
28.
LeeH.GeisingerK. F. (2014). The effect of propensity scores on DIF analysis: Inference on the potential cause of DIF. International Journal of Testing, 14(4), 313–338. https://doi.org/10.1080/15305058.2014.922567
29.
LiuY.KimC.WuA. D.GustafsonP.KrocE. (2019). Investigating the performance of propensity score approaches for differential item functioning analysis. Journal of Modern Applied Statistical Methods, 18(1), eP2744. https://doi.org/10.22237/jmasm/1556669280
30.
LiuY.ZumboB. D.GustafsonP.HuangY.KrocE.WuA. D. (2016). Investigating causal DIF via propensity score methods. Practical Assessment, Research, and Evaluation, 21(1), 13. https://doi.org/10.7275/ewqz-n963
31.
LordF. M. (1980). Applications of item response theory to practical testing problems. Lawrence Erlbaum.
32.
MacKinnonD. P.KrullJ. L.LockwoodC. M. (2000). Equivalence of the mediation, confounding and suppression effect. Prevention Science, 1, 173–181. https://doi.org/10.1023/A:1026595011371
33.
McCaffreyD. F.GriffinB. A.AlmirallD.SlaughterM. E.RamchandR.BurgetteL. F. (2013). A tutorial on propensity score estimation for multiple treatments using generalized boosted models. Statistics in Medicine, 32(19), 3388–3414. https://doi.org/10.1002/sim.5753
34.
MelguizoT.KienzlG. S.AlfonsoM. (2011). Comparing the educational attainment of community college transfer students and four-year college rising juniors using propensity score matching methods. The Journal of Higher Education, 82(3), 265–291. https://doi.org/10.1080/00221546.2011.11777202
35.
OnealJ. R.RussettB. (2005). Rule of three, let it be? When more really is better. Conflict Management and Peace Science, 22(4), 293–310. https://doi.org/10.1080/07388940500339209
36.
OortF. J. (1998). Simulation study of item bias detection with restricted factor analysis. Structural Equation Modeling, 5(2), 107–124. https://doi.org/10.1080/10705519809540095
37.
PineS. M. (1977). Applications of item characteristic curve theory to the problem of test bias. In WeissD. J. (Ed.), Applications of computerized adaptive testing: Proceedings of a symposium presented at the 18th annual convention of the Military Testing Association (Research Report No. 77-1, pp. 37–43). University of Minnesota, Department of Psychology, Psychometric Methods Program.
38.
RaschG. (1960). Probabilistic models for some intelligence and attainment test. University of Chicago Press.
39.
R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
40.
RosenbaumP. R.RubinD. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55. https://doi.org/10.1093/biomet/70.1.41
41.
RosseelY. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36.
42.
ShihC. L.LiuT. H.WangW. C. (2014). Controlling type I error rates in assessing DIF for logistic regression method combined with SIBTEST regression correction procedure and DIF-free-then-DIF strategy. Educational and Psychological Measurement, 74(6), 1018–1048. https://doi.org/10.1177/0013164413520545
43.
ShihC. L.WangW. C. (2009). Differential item functioning detection using the multiple indicators, multiple causes method with a pure short anchor. Applied Psychological Measurement, 33(3), 184–199. https://doi.org/10.1177/0146621608321758
44.
ThibodeauM. A.WelchP. G.SareenJ.AsmundsonG. J. (2013). Anxiety disorders are independently associated with suicide ideation and attempts: Propensity score matching in two epidemiological samples. Depression and Anxiety, 30(10), 947–954. https://doi.org/10.1002/da.22203
45.
TsutaniY.MiyataY.NakayamaH.OkumuraS.AdachiS.YoshimuraM.OkadaM. (2013). Oncologic outcomes of segmentectomy compared with lobectomy for clinical stage IA lung adenocarcinoma: Propensity score–matched analysis in a multicenter study. The Journal of Thoracic and Cardiovascular Surgery, 146(2), 358–364. https://doi.org/10.1016/j.jtcvs.2013.02.008
46.
WangW. C. (2004). Effects of anchor item methods on the detection of differential item functioning within the family of Rasch models. Journal of Experimental Education, 72(3), 221–261. https://doi.org/10.3200/JEXE.72.3.221-261
47.
WangW. C. (2008). Assessment of differential item functioning. Journal of Applied Measurement, 9(4), 387–408.
48.
WangW. C.ShihC. L. (2010). MIMIC methods for assessing differential item functioning in polytomous items. Applied Psychological Measurement, 34(3), 166–180. https://doi.org/10.1177/0146621609355279
49.
WangW. C.ShihC. L.SunG. W. (2012). The DIF-Free-Then-DIF strategy for the assessment of differential item functioning. Educational and Psychological Measurement, 72(4), 687–708. https://doi.org/10.1177/0013164411426157
50.
WangW. C.YehY. L. (2003). Effects of anchor item methods on differential item functioning detection with the likelihood ratio test. Applied Psychological Measurement, 27(6), 479–498. https://doi.org/10.1177/0146621603259902
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
