Sage Journals: Discover world-class research

Abstract

The higher-order structure and attribute hierarchical structure are two popular approaches to defining the latent attribute space in cognitive diagnosis models. However, to our knowledge, it is still impossible to integrate them to accommodate the higher-order latent trait and hierarchical attributes simultaneously. To address this issue, this article proposed a sequential higher-order latent structural model (LSM) by incorporating various hierarchical structures into a higher-order latent structure. The feasibility of the proposed higher-order LSM was examined using simulated data. Results indicated that, in conjunction with the deterministic-inputs, noisy “and” gate model, the sequential higher-order LSM produced considerable improvement in person classification accuracy compared with the conventional higher-order LSM, when a certain attribute hierarchy existed. An empirical example was presented as well to illustrate the application of the proposed LSM.

Keywords

cognitive diagnosis higher-order latent structure attribute hierarchy sequential tree cognitive diagnosis models DINA model

Get full access to this article

View all access options for this article.

References

Akbay

(2016). Identification, estimation, and Q-matrix validation of hierarchically structured attributes in cognitive diagnosis (Unpublished doctoral dissertation). Rutgers, The State University of New Jersey, New Brunswick.

Ayers

Rabe-Hesketh

Nugent

(2013). Incorporating student covariates in cognitive diagnosis models. Journal of Classification, 30, 195-224.

Brooks

S. P.

Gelman

(1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7, 434-455.

Cai

(2017). flexMIRT® version 3.51: Flexible multilevel multidimensional item analysis and test scoring [Computer software]. Chapel Hill, NC: Vector Psychometric Group.

De Boeck

Partchev

. (2012). IRTrees: Tree-based item response models of the GLMM family. Journal of Statistical Software, 48, 1-28.

de la Torre

(2011). The generalized DINA model framework. Psychometrika, 76, 179-199.

de la Torre

Douglas

(2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333-353.

de la Torre

Hong

Deng

(2010). Factors affecting the item parameter estimation and classification accuracy of the DINA model. Journal of Educational Measurement, 47, 227-249.

Ding

Yang

Wang

(2010). The importance of reachability matrix in constructing cognitively diagnostic testing. Journal of Jiangxi Normal University (Natural Sciences Edition), 34, 490-494.

10.

Gelfand

A. E.

Smith

A. F. M.

(1990). Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 85, 398-409.

11.

George

A. C.

Robitzsch

Kiefer

Gross

Uenlue

(2016). The R package CDM for cognitive diagnosis models. Journal of Statistical Software, 74, 1-24.

12.

Gierl

M. J.

Leighton

J. P.

Hunka

S. M.

(2007). Using the attribute hierarchy method to make diagnostic inferences about examinees’ cognitive skills. In Leighton

J. P.

Girtl

M. J.

(Eds.), Cognitive diagnostic assessment for education (pp. 242-274). Cambridge, UK: Cambridge University Press.

13.

Hansen

Cai

Monroe

(2016). Limited-information goodness-of-fit testing of diagnostic classification item response models. British Journal of Mathematical and Statistical Psychology, 69, 225-252.

14.

Hsu

C.-L.

Wang

W.-C.

(2015). Variable-length computerized adaptive testing using the higher-order DINA model. Journal of Educational Measurement, 52, 125-143.

15.

Junker

B. W.

Sijtsma

(2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258-272.

16.

Köhn

H.-F.

Chiu

C.-Y.

(2017). A procedure for assessing the completeness of the Q-matrices of cognitively diagnostic tests. Psychometrika, 82, 112-132.

17.

Kuhn

(2001). Why development does (and does not) occur: Evidence from the domain of inductive reasoning. In McClelland

J. L.

Siegler

(Eds.), Mechanisms of cognitive development: Behavioral and neural perspectives (pp. 221-249). Hillsdale, NJ: Lawrence Erlbaum.

18.

Leighton

J. P.

Gierl

M. J.

Hunka

S. M.

(2004). The attribute hierarchy method for cognitive assessment: A variation on Tatsuoka’s rule-space approach. Journal of Educational Measurement, 41, 205-237.

19.

Liu

(2018). Misspecification of attribute structure in diagnostic measurement. Educational and Psychological Measurement, 78, 605-634. doi:10.1177/0013164417702458

20.

(2019). A diagnostic tree model for polytomous responses with multiple strategies. British Journal of Mathematical and Statistical Psychology, 72, 61-82. doi:10.1111/bmsp.12137

21.

de la Torre

(2016). A sequential cognitive diagnosis model for polytomous responses. British Journal of Mathematical and Statistical Psychology, 69, 253-275.

22.

de la Torre

(2018). GDINA: The generalized DINA model framework [R package version 2.2.0]. Retrieved from http://CRAN.R-project.org/package=GDINA

23.

R Core Team. (2016). R: A language and environment for statistical computing. Vienna, Austria: The R Foundation for Statistical Computing.

24.

Rupp

Templin

Henson

(2010). Diagnostic measurement: Theory, methods, and applications. New York, NY: Guilford Press.

25.

Spiegelhalter

D. J.

Best

N. G.

Carlin

B. P.

van der Linde

(2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B, 64, 583-639. doi:10.1111/1467-9868.00353

26.

Y. L.

Choi

K. M.

Lee

W. C.

Choi

McAninch

(2013). Hierarchical cognitive diagnostic analysis for TIMSS 2003 mathematics. Centre for Advanced Studies in Measurement and Assessment, 35, 1-71.

27.

Tatsuoka

K. K.

(1983). Rule space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345-354.

28.

Templin

Bradshaw

(2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies. Psychometrika, 79, 317-339.

29.

Templin

Hoffman

(2013). Obtaining diagnostic classification model estimates using Mplus. Educational Measurement: Issues and Practice, 32, 37-50.

30.

Wang

Cai

Douglas

Chang

H.-H.

(2018). Cognitive diagnostic models with attribute hierarchies: Model estimation with a restricted Q-matrix design. Applied Psychological Measurement. Advanced online publication. doi:10.1177/0146621618765721

31.

von Davier

Haberrman

S. J

. (2014). Hierarchical diagnostic classification models morphing into unidimensional “diagnostic” classification models—A commentary. Psychometrika, 79, 340-346.

32.

Ding

Qin

(2011). Application of Bayesian networks to identify hierarchical relation among attributes in cognitive diagnosis. Acta Psychologica Sinica, 3, 338-346.

33.

Zhan

Jiao

Liao

(2019). A longitudinal higher-order diagnostic classification model. Journal of Educational and Behavioral Statistics. Advanced Online Publication. Retrieved from https://doi.org/10.3102/1076998619827593

34.

Zhan

Jiao

Liao

(2018). Cognitive diagnosis modelling incorporating item response times. British Journal of Mathematical and Statistical Psychology, 71, 262-286.

35.

Zhan

Jiao

Man

Wang

(2019). Using JAGS for Bayesian cognitive diagnosis modeling: A tutorial. Journal of Educational and Behavioral Statistics. Advanced online publication. Retrieved from https://doi.org/10.3102/107699861982604

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

1.71 MB

0.00 MB