A family of learning models that integrates a cognitive diagnostic model and a higher-order, hidden Markov model in one framework is proposed. This new framework includes covariates to model skill transition in the learning environment. A Bayesian formulation is adopted to estimate parameters from a learning model. The developed methods are applied to a computer-based assessment with a learning intervention. The results show the potential application of the proposed model to track the change of students’ skills directly and provide immediate remediation as well as to evaluate the efficacy of different interventions by investigating how different types of learning interventions impact the transitions from nonmastery to mastery.
AndersenE. B. (1985). Estimating latent correlations between repeated testings. Psychometrika, 50, 3–16.
2.
BradshawL.MadisonM. (2016). Invariance properties for general diagnostic classification models. International Journal of Testing, 16, 99–118
3.
BrooksS. P.GelmanA. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7, 434–455.
4.
CappéO.MoulinesE.RydénT. (2005). An information-theoretic perspective on order estimation. In CappéO.MoulinesE.RydénT. (Eds.), Inference in hidden Markov models (pp. 565–601). New York, NY: Springer.
5.
ChenY.CulpepperS. A.WangS.DouglasJ. A. (in press). A hidden Markov model for learning trajectories in cognitive diagnosis with application to spatial rotation skills. Applied Psychological Measurement.
6.
ChenY.LiuJ.XuG.YingZ. (2015). Statistical analysis of Q-matrix based diagnostic classification models. Journal of the American Statistical Association, 110, 850–866.
7.
ChiuC.-Y.KöhnH.-F. (2015). The reduced RUM as a logit model: Parameterization and constraints. Psychometrika, 81, 350–370.
8.
ChungH.WallsT. A.ParkY. (2007). A latent transition model with logistic regression. Psychometrika, 72, 413–435.
9.
CorbettA. T.AndersonJ. R. (1994). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-adapted Interaction, 4, 253–278.
10.
CulpepperS. A. (2015). Bayesian estimation of the DINA model with Gibbs sampling. Journal of Educational and Behavioral Statistics, 40, 454–476.
11.
DavierM. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61, 287–307.
12.
de la TorreJ. (2011). The generalized DINA model framework. Psychometrika, 76, 179–199.
13.
de la TorreJ.DouglasJ. A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333–353.
14.
de la TorreJ.LeeY. (2010) A note on the invariance of the DINA model parameters. Journal of Educational Measurement, 47, 115–127
15.
EddelbuettelD. (2013). Seamless R and C++ integration with Rcpp. New York, NY: Springer.
16.
EmbretsonS. E. (1991). Implications of a multidimensional latent trait model for measuring change. In CollinsL.HornJ. (Eds.), Best methods for the analysis of change: Recent advances, unanswered questions, future directions (pp. 184–197). Washington, DC: American Psychological Association Books.
17.
González-BrenesJ.HuangY.BrusilovskyP. (2014). General features in knowledge tracing: Applications to multiple subskills, temporal item response theory, and expert knowledge. In MavrikisM.McLarenB. M. (Eds.), Proceedings of the 7th International Conference on Educational Data Mining (pp. 84–91). Pittsburgh, PA: University of Pittsburgh.
18.
González-BrenesJ. P.MostowJ. (2013). What and when do students learn? Fully data-driven joint estimation of cognitive and student models. In D’MelloS. K.CalvoR. A.OlneyA. (Eds.), Proceedings of the 6th International Conference on Educational Data Mining (pp. 236–240). Memphis, TN: Educational Data Mining Society.
19.
HartzS.RoussosL. (2008). The fusion model for skills diagnosis: Blending theory with practicality. ETS Research Report Series, 2008, i–57.
20.
HensonR. A.TemplinJ. L.WillseJ. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.
21.
JunkerB. W.SijtsmaK. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.
22.
KayaY.LeiteW. L. (2017). Assessing change in latent skills across time with longitudinal cognitive diagnosis modeling an evaluation of model performance. Educational and Psychological Measurement, 77, 369–388. doi:0013164416659314.
23.
KhajahM. M.HuangY.González-BrenesJ. P.MozerM. C.BrusilovskyP. (2014). Integrating knowledge tracing and item response theory: A tale of two frameworks. In KravcikM.SantosO. C.BoticarioJ. G. (Eds.), Proceedings of Workshop on Personalization Approaches in Learning Environments (PALE 2014) at the 22th International Conference on User Modeling, Adaptation, and Personalization (pp. 7–12). Pittsburgh, PA: University of Pittsburgh.
24.
LiF.CohenA.BottgeB.TemplinJ. (2016). A latent transition analysis model for assessing change in cognitive skills. Educational and Psychological Measurement, 76, 181–204.
25.
LiH.HunterC. V.LeiP.-W. (2015). The selection of cognitive diagnostic models for a reading comprehension test. Language Testing, 33, 391–409.
PardosZ. A.HeffernanN. T. (2010). Modeling individualization in a Bayesian networks implementation of knowledge tracing. In De BraP.KobsaA.ChinD. (Eds.), User Modeling, Adaptation, and Personalization: 18th International Conference, UMAP 2010, Big Island, HI, USA, June 20–24, 2010, (Vol. 6075, p. 255). Berlin, Germany: Springer Science & Business Media.
28.
RavandH. (2015). Application of a cognitive diagnostic model to a high-stakes reading comprehension test. Journal of Psychoeducational Assessment, 1, 18.
29.
RubinD. B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. The Annals of Statistics, 12, 1151–1172.
30.
SinharayS. (2006). Model diagnostics for Bayesian networks. Journal of Educational and Behavioral Statistics, 31, 1–33.
31.
SinharayS.AlmondR. G. (2007). Assessing fit of cognitive diagnostic models a case study. Educational and Psychological Measurement, 67, 239–257.
32.
SinharayS.JohnsonM. S.SternH. S. (2006). Posterior predictive assessment of item response theory models. Applied Psychological Measurement, 30, 298–321.
33.
StuderC. (2012). Incorporating learning over time into the cognitive assessment framework (PhD thesis). Carnegie Mellon University, Pittsburgh, PA.
34.
TatsuokaK. K. (1985). A probabilistic model for diagnosing misconceptions by the pattern classification approach. Journal of Educational and Behavioral Statistics, 10, 55–73.
35.
TemplinJ.BradshawL. (2013). Measuring the reliability of diagnostic classification model examinee estimates. Journal of Classification, 30, 251–275.
36.
TemplinJ. L.HensonR. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287.
37.
TemplinJ. L.HoffmanL. (2013). Obtaining diagnostic classification model estimates using Mplus. Educational Measurement: Issues and Practice, 32, 37–50.
38.
WangS.DouglasJ. (2015). Consistency of nonparametric classification in cognitive diagnosis. Psychometrika, 80, 85–100.
39.
XuG.ZhangS. (2015). Identifiability of diagnostic classification models. Psychometrika, 81, 625–649.
40.
XuY.MostowJ. (2012). Comparison of methods to trace multiple subskills: Is LRDBN best? In YacefK.ZaïaneO.HershkovitzH.YudelsonM.StamperJ. (Eds.), Proceedings of the 5th International Conference on Educational Data Mining, (pp. 41–48). Chania, Greece.
41.
YeS.FellourisG.CulpepperS.DouglasJ. (2016). Sequential detection of learning in cognitive diagnosis. British Journal of Mathematical and Statistical Psychology, 69, 139158.
42.
YoonS. Y. (2011). Psychometric properties of the revised Purdue Spatial Visualization Tests: Visualization of rotations (The revised PSVT-R) (Unpublished doctoral dissertation). Purdue University, West Lafayette.
