Item response theory (IRT) models typically rely on a normality assumption for subject-specific latent traits, which is often unrealistic in practice. Semiparametric extensions based on Dirichlet process mixtures (DPMs) offer a more flexible representation of the unknown distribution of the latent trait. However, the use of such models in the IRT literature has been extremely limited, in good part because of the lack of comprehensive studies and accessible software tools. This article provides guidance for practitioners on semiparametric IRT models and their implementation. In particular, we rely on NIMBLE, a flexible software system for hierarchical models that enables the use of DPMs. We highlight efficient sampling strategies for model estimation and compare inferential results under parametric and semiparametric models.
AldousD. J. (1985). Exchangeability and related topics. In HennequinP. L. (Ed.), École d’été de probabilités de saint-flour xiii—1983 (pp. 1–198). Springer.
2.
AntoniakC. E. (1974). Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. Annals of Statistics, 2(6), 1152–1174. https://doi.org/10.1214/aos/1176342871
3.
AzevedoC. L.BolfarineH.AndradeD. F. (2011). Bayesian inference for a skew-normal IRT model under the centred parameterization. Computational Statistics & Data Analysis, 55 (1), 353–365.
4.
AzzaliniA. (1985). A class of distributions which includes the normal ones. Scandinavian Journal of Statistics, 12, 171–178.
5.
BafumiJ.GelmanA.ParkD. K.KaplanN. (2005). Practical issues in implementing and understanding Bayesian ideal point estimation. Political Analysis, 13(2), 171–187.
6.
Bambirra GonçalvesF.da Costa Campos DiasB.Machado SoaresT. (2018). Bayesian item response model: A generalized approach for the abilities’ distribution using mixtures. Journal of Statistical Computation and Simulation, 88(5), 967–981.
7.
BedrickE. J.ChristensenR.JohnsonW. (1996). A new perspective on priors for generalized linear models. Journal of the American Statistical Association, 91(436), 1450–1460.
8.
BergerJ. O.BernardoJ. M.SunD. (2009). The formal definition of reference priors. The Annals of Statistics, 37(2), 905–938. http://www.jstor.org/stable/30243652
9.
BergerJ. O.PericchiL. R. (1996). The intrinsic Bayes factor for model selection and prediction. Journal of the American Statistical Association, 91(433), 109–122.
10.
BernardoJ. M. (1979). Reference posterior distributions for Bayesian inference. Journal of the Royal Statistical Society: Series B (Methodological), 41(2), 113–128.
11.
BetancourtM.ByrneS.LivingstoneS.GirolamiM. (2017). The geometric foundations of Hamiltonian Monte Carlo. Bernoulli, 23(4A), 2257–2298.
12.
BlackwellD.MacQueenJ. B. (1973). Ferguson distributions via Pólya urn schemes. The Annals of Statistics, 1(2), 353–355.
13.
BoltD. M.CohenA. S.WollackJ. A. (2001). A mixture item response model for multiple-choice data. Journal of Educational and Behavioral Statistics, 26(4), 381–409. http://www.jstor.org/stable/3648167
CarpenterB.GelmanA.HoffmanM.LeeD.GoodrichB.BetancourtM.BrubakerM.GuoJ.LiP.RiddellA. (2017). Stan: A probabilistic programming language. Journal of Statistical Software, Articles, 76(1), 1–32. https://www.jstatsoft.org/v076/i01
16.
ClintonJ.JackmanS.RiversD. (2004). The statistical analysis of roll call data. American Political Science Review, 98, 355–370.
17.
de ValpineP.PaciorekC.TurekD.MichaudN.Anderson-BergmanC.ObermeyerF.WehrhahnCortes, C.RodríguezA.TempleLang, D.PaganinS.HugJ. (2022). Nimble user manual [Computer software manual]. doi:10.5281/zenodo.1211190
18.
de ValpineP.PaciorekC. J.TurekD.MichaudN.Anderson-BergmanC.ObermeyerF.WehrhahnCortes, C.RodríguezA.TempleLang, D.PaganinS.HugJ. (2020). NIMBLE: MCMC, particle filtering, and programmable hierarchical modeling (R package version 0.9.1). https://cran.r-project.org/package=nimble
19.
de ValpineP.TurekD.PaciorekC. J.Anderson-BergmanC.LangD. T.BodikR. (2017). Programming with models: Writing statistical algorithms for general model structures with nimble. Journal of Computational and Graphical Statistics, 26(2), 403–413. https://doi.org/10.1080/10618600.2016.1172487
20.
DuncanK. A.MacEachernS. N. (2008). Nonparametric Bayesian modelling for item response. Statistical Modelling, 8(1), 41–66.
21.
EscobarM. D.WestM. (1995). Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association, 90(430), 577–588.
22.
FergusonT. S. (1973). A Bayesian analysis of some nonparametric problems. The Annals of Statistics, 1(2), 209–230. https://doi.org/10.1214/aos/1176342360
23.
FinchH.EdwardsJ. M. (2016). Rasch model parameter estimation in the presence of a nonnormal latent trait using a nonparametric Bayesian approach. Educational and Psychological Measurement, 76(4), 662–684. https://doi.org/10.1177/0013164415608418
24.
FlegalJ. M.HughesJ.VatsD.DaiN.GuptaK.MajiU. (2021). mcmcse: Monte Carlo standard errors for MCMC[Computer software manual]. Riverside, CA, and Kanpur, India. (R package version 1.5-0).
25.
FoxJ.-P. (2010). Bayesian item response modeling: Theory and applications. Springer Science & Business Media.
GelfandA. E.KottasA. (2002). A computational approach for full nonparametric Bayesian inference under Dirichlet Process mixture models. Journal of Computational and Graphical Statistics, 11(2), 289–305. https://doi.org/10.1198/106186002760180518
28.
GelmanA. (2004). Parameterization and Bayesian modeling. Journal of the American Statistical Association, 99(466), 537–545.
29.
GewekeJ. F.SingletonK. J. (1981). Maximum likelihood “confirmatory” factor analysis of economic time series. International Economic Review, 22, 37–54.
30.
GuhaniyogiR.RodriguezA. (2020). Joint modeling of longitudinal relational data and exogenous variables. Bayesian Analysis, 15(2), 477–503.
31.
HaysR. D.MoralesL. S.ReiseS. P. (2000). Item response theory and health outcomes measurement in the 21st century. Medical Care, 38(9 Suppl), II28.
32.
HoffmanM. D.GelmanA. (2014). The No-U-Turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research, 15(1), 1593–1623.
JaraA.HansonT. E.QuintanaF. A.MüllerP.RosnerG. L. (2011). DPpackage: Bayesian semi-and nonparametric modeling in R. Journal of Statistical Software, 40(5), 1.
35.
JohnsonM. S. (2007). Modeling dichotomous item responses with free-knot splines. Computational Statistics & Data Analysis, 51(9), 4178–4192.
36.
Joint Health Surveys Unit of Social and Community Planning Research and University College London. (2017). Health survey for England, 1996. [data collection]. 5th Edition. UK Data Service. SN: 3886. http://doi.org/10.5255/UKDA-SN-3886-2
KirisciL.chi HsuT.YuL. (2001). Robustness of item parameter estimation programs to assumptions of unidimensionality and normality. Applied Psychological Measurement, 25(2), 146–162. https://doi.org/10.1177/01466210122031975
39.
LairdN. M. (1978). Nonparametric maximum likelihood estimation of a mixing distribution. Journal of the American Statistical Association, 73, 805–807.
40.
LiY.MüllerP.LinX. (2011). Center-adjusted inference for a nonparametric Bayesian random effect distribution. Statistica Sinica, 21(3), 1201–1223.
41.
LiuC.RubinD. B.WuY. N. (1998). Parameter expansion to accelerate EM: The PX-EM algorithm. Biometrika, 85(4), 755–770.
42.
LiuJ. S.WuY. N. (1999). Parameter expansion for data augmentation. Journal of the American Statistical Association, 94(448), 1264–1274.
43.
LiuS. J. (1996). Nonparametric hierarchical Bayes via sequential imputations. The Annals of Statistics, 24(3), 911–930. https://doi.org/10.1214/aos/1032526949
44.
LoA. Y. (1984). On a class of Bayesian nonparametric estimates: I. density estimates. The Annals of Statistics, 12(1), 351–357.
45.
McHorneyC. A.HaleyS. M.WareJ. E.Jr (1997). Evaluation of the MOS SF-36 physical functioning scale (PF-40): II. comparison of relative precision using Likert and Rasch scoring methods. Journal of Clinical Epidemiology, 50(4), 451–461.
46.
MicceriT. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105(1), 156.
MiyazakiK.HoshinoT. (2009). A Bayesian semiparametric item response model with Dirichlet Process priors. Psychometrika, 74(3), 375–393.
49.
NatesanP.NandakumarR.MinkaT.RubrightJ. D. (2016). Bayesian prior choice in IRT estimation using MCMC and variational Bayes. Frontiers in Psychology, 7, 1422.
50.
NguyenD.de ValpineP.AtchadeY.TurekD.MichaudN.PaciorekC. J. (2020). Nested adaptation of mcmc algorithms. Bayesian Analysis, 15(4), 1323–1343. https://doi.org/10.1214/19-BA1190
51.
PaulonG.De IorioM.GuglielmiA.IevaF. (2018, 07). Joint modeling of recurrent events and survival: A Bayesian non-parametric approach. Biostatistics, 21(1), 1–14. https://doi.org/10.1093/biostatistics/kxy026
52.
PitmanJ. (1996). Some developments of the Blackwell-Macqueen urn scheme. In FergusonT. S.ShapleyL. S.MacQueenJ. B. (Eds.), Statistics, probability and game theory (Vol. 30, pp. 245–267). Institute of Mathematical Statistics. https://doi.org/10.1214/lnms/1215453576
53.
QinL. (1998). Nonparametric Bayesian models for item response data(Unpublished doctoral dissertation). The Ohio State University.
54.
RaschG. (1990). Probabilistic models for some intelligence and attainment tests. Danish Institute for Educational Research.
55.
ReiseS.RodríguezA. (2016). Item response theory and the measurement of psychiatric constructs: Some empirical and conceptual issues and challenges. Psychological Medicine, 46(10), 2025–2039.
56.
RutkowskiL.GonzalezE.JoncasM.von DavierM. (2010). International large-scale assessment data: Issues in secondary analysis and reporting. Educational Researcher, 39(2), 142–151.
57.
SamejimaF. (1997). Departure from normal assumptions: A promise for future psychometrics with substantive mathematical modeling. Psychometrika, 62(4), 471–493.
58.
San MartínE.JaraA.RolinJ.-M.MouchartM. (2011, Jul01). On the Bayesian nonparametric generalization of IRT-type models. Psychometrika, 76(3), 385–409. https://doi.org/10.1007/s11336-011-9213-9
59.
SchmittJ. E.MehtaP. D.AggenS. H.KubarychT. S.NealeM. C. (2006). Seminonparametric methods for detecting latent non-normality: A fusion of latent trait and ordered latent class modeling. Multivariate Behavioral Research, 41(4), 427–443.
60.
SeongT. (1990). Sensitivity of marginal maximum likelihood estimation of item and ability parameters to the characteristics of the prior ability distributions. Applied Psychological Measurement, 14(3), 299–311.
61.
SethuramanJ. (1994). A constructive definition of Dirichlet priors. Statistica Sinica, 4(2), 639–650.
62.
ShengY. (2010). A sensitivity analysis of Gibbs sampling for 3PNO IRT models: Effects of prior specifications on parameter estimates. Behaviormetrika, 37(2), 87–110.
63.
SmitsN.OğredenO.Garnier-VillarrealM.TerweeC. B.ChalmersR. P. (2020). A study of alternative approaches to non-normal latent trait distributions in item response theory models used for health outcome measurement. Statistical Methods in Medical Research, 29(4), 1030–1048.
64.
SosaJ.RodríguezA. (2021). A latent space model for cognitive social structures data. Social Networks, 65, 85–97.
65.
Stan Development Team. (2018). Stan modeling language users guide and reference manual, version 2.18.0. http://mc-stan.org
66.
VatsD.FlegalJ. M.JonesG. L. (2019). Multivariate output analysis for Markov chain Monte Carlo. Biometrika, 106(2), 321–337.
WatanabeS.OpperM. (2010). Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory. Journal of Machine Learning Research, 11(12), 3571–3594.
69.
WoodsC. M. (2007). Empirical histograms in item response theory with ordinal data. Educational and Psychological Measurement, 67(1), 73–87.
70.
WoodsC. M.ThissenD. (2006). Item response theory with estimation of the latent population distribution using spline-based densities. Psychometrika, 71(2), 281.
71.
YangM.DunsonD. B. (2010). Bayesian semiparametric structural equation models with latent variables. Psychometrika, 75(4), 675–693.
72.
YangM.DunsonD. B.BairdD. (2010). Semiparametric Bayes hierarchical models with mean and variance constraints. Computational Statistics & Data Analysis, 54(9), 2172–2186.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
