A local equating method of cognitively diagnostic modeled observed score was introduced to address the problem of equating without anchor test as well as the satisfaction of equating criteria. The criterion of test construct was redefined on attribute level. Then, a true family of equating transformations given attribute mastery pattern was derived from the criterion of equity, which is the main concern of local equating. The results of simulation studies showed a substantial bias for the traditional equipercentile equating under the equivalent groups design but much smaller bias for the local method. The proposed method also showed satisfactory performances under the nonequivalent groups design even without anchor test. The local method appeared to be quite robust with respect to changes in the sample size, but its biases became larger with increasing model misfit.
BrennanR. L. (1992). The context of context effects. Applied Measurement in Education, 5, 225-264.
2.
BrennanR. L. (2001). Some problems, pitfalls, and paradoxes in educational measurement. Educational Measurement: Issues and Practice, 20, 6-18.
3.
de la TorreJ.DouglasJ. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333-353.
4.
de la TorreJ.LeeY. S. (2010). A note on the invariance of the DINA model parameters. Journal of Educational Measurement, 47, 115-127.
5.
DiBelloL. V.RoussosL. A.StoutW. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In RaoC. R.SinharayS. (Eds.), Handbook of statistics 26, psychometrics (pp. 979-1030). Amsterdam, the Netherlands: Elsevier.
6.
DiBelloL. V.StoutW. F.RoussosL. A. (1995). Unified cognitive psychometric diagnostic assessment likelihood-based classification techniques. In NicholsP. D.ChipmanS. F.BrennanR. L. (Eds.), Cognitively diagnostic assessment (pp. 361-390). Mahwah, NJ: Erlbaum.
7.
DingS.LuoF.CaiY.LinH.WangX. (2008). Complement to Tatsuoka’s Q matrix theory. In ShigemasuK.OkadaA.ImaizumiT.HoshinoT. (Eds.), New trends in psychometrics (pp. 417-424). Tokyo, Japan: Universal Academy Press.
8.
DoransN. J.HollandP. W. (2000). Population invariance and the equatability of tests: Basic theory and the linear case. Journal of Educational Measurement, 37, 281-306.
9.
EmbretsonS. E. (1983). Construct validity: Construct representation versus nomothetic span. Psychological Bulletin, 93, 179-197.
10.
HaertelE. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 301-321.
11.
HaertelE. H. (1990). Continuous and discrete latent structure models for item response data. Psychometrika, 55, 477-494.
12.
HaertelE. H. (2006). Reliability. In BrennanR. L. (Ed.), Educational measurement (4th ed., pp. 65-110). Westport, CT: Praeger.
13.
HansonB. A. (1991). A note on Levine’s formula for equating unequally reliable tests using data from the common item nonequivalent groups design. Journal of Educational Statistics, 16, 93-100.
14.
HarrisD. J.GaoX. (2003, April). A conceptual synthesis of context effect. In Context effects: Implications for pretesting and CBT—American Educational Research Association Annual Meeting, Chicago, IL.
15.
HartzS. M. (2002). A Bayesian framework for the Unified Model for assessing cognitive abilities: Blending theory with practicality (Unpublished doctoral dissertation). University of Illinois at Urbana–Champaign.
16.
HollandP. W.DoransN. J. (2006). Linking and equating. In BrennanR. L. (Ed.), Educational measurement (4th ed., pp. 187-220). Westport, CT: Praeger.
17.
JunkerB.SijtsmaK. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258-272.
18.
KolenM. J.BrennanR. L. (2004). Test equating, scaling, and linking: Methods and practices. New York, NY: Springer.
19.
LeightonJ. P.GierlM. J.HunkaS. M. (2004). The attribute hierarchy method for cognitive assessment: A variation on Tatsuoka’s rule-space approach. Journal of Educational Measurement, 41, 205-237.
20.
LevineR. S. (1955). Equating the score scales of alternate forms administered to samples of different ability (ETS RB-55-23). Princeton, NJ: Education Testing Service.
21.
LiF. (2009). Linking between two tests without common items—By rule space model (Unpublished doctoral dissertation). Beijing Normal University, China.
22.
LiuJ.WalkerM. E. (2007). Score linking issues related to test content changes. In DoransN. J.PommerichM.HollandP. (Eds.), Linking and aligning scores and scales (pp. 109-134). New York, NY: Springer.
23.
LordF. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
24.
LordF. M.WingerskyM. S. (1984). Comparison of IRT true-score and equipercentile observed-score “equating.”Applied Psychological Measurement, 8, 452-461.
25.
MacreadyG. B.DaytonC. M. (1977). The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics, 2, 99-120.
26.
R Development Core Team. (2011). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Available from http://www.R-project.org/
27.
RoussosL. A.TemplinJ. L.HensonR. A. (2007). Skills diagnosis using IRT-Based latent class models. Journal of Educational Measurement, 44, 293-311.
28.
TatsuokaK. K. (1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345-354.
29.
TatsuokaK. K. (1995). Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. In NicholsP. D.ChipmanS. F.BrennanR. L. (Eds.), Cognitively diagnostic assessment (pp. 327-359). Hillsdale, NJ: Erlbaum.
30.
TatsuokaK. K. (2009). Cognitive assessment: An introduction to the rule space method. New York, NY: Routledge.
31.
van der LindenW. J. (2000). A test-theoretic approach to observed-score equating. Psychometrika, 65, 437-456.
32.
van der LindenW. J. (2006a). Equating an adaptive test to a linear test. Applied Psychological Measurement, 30, 493-508.
33.
van der LindenW. J. (2006b). Equating error in observed-score equating. Applied Psychological Measurement, 30, 355-378.
34.
van der LindenW. J. (2010). Local observed-score equating. In von DavierA. A. (Ed.), Statistical models for equating, scaling, and linking (pp. 201-223). New York, NY: Springer.
35.
van der LindenW. J.WibergM. (2010). Local observed-score equating with anchor-test designs. Applied Psychological Measurement, 34, 620-640.
