Bayesian methods to infer model dimensionality in factor analysis generally assume a lower triangular structure for the factor loadings matrix. Consequently, the ordering of the outcomes influences the results. Therefore, we propose a method to infer model dimensionality without imposing any prior restriction on the loadings matrix. Our approach considers a relatively large number of factors and includes auxiliary multiplicative parameters, which may render null the unnecessary columns in the loadings matrix. The underlying dimensionality is then inferred based on the number of nonnull columns in the factor loadings matrix, and the model parameters are estimated with a postprocessing scheme. The advantages of the method in selecting the correct dimensionality are illustrated via simulations and using real data sets.
AbramowitzM.StegunI. A. (1964). Handbook of mathematical functions: With formulas, graphs, and mathematical tables (Vol. 55). Courier Corporation.
2.
AndersonT.RubinH. (1956). Statistical inference in factor analysis. In Proceedings of the third berkeley symposium on mathematical statistics and probability, University of California, December, 1954, July and August, 1955 (Vol. 1, p. 111). Held at the statistical laboratory. https://digitalassets.lib.berkeley.edu/math/ucb/text/math_s3_v5_article-08.pdf
3.
AßmannC.Boysen-HogrefeJ.PapeM. (2016). Bayesian analysis of static and dynamic factor models: An ex-post approach towards the rotation problem. Journal of Econometrics, 192(1), 190–206.
4.
BrooksS. P.GelmanA. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7(4), 434–455.
5.
CarvalhoC. M.ChangJ.LucasJ. E.NevinsJ. R.WangQ.WestM. (2008). High-dimensional sparse factor modeling: Applications in gene expression genomics. Journal of the American Statistical Association, 103(484), 1438–1456.
6.
ChenJ. (2021). A Bayesian regularized approach to exploratory factor analysis in one step. Structural Equation Modeling: A Multidisciplinary Journal, 28(4), 518–528.
DuttaR.GhoshJ. K. (2013). Bayes model selection with path sampling: Factor models and other examples. Statistical Science, 28(1), 95–115.
9.
EngzellP. (2021). What do books in the home proxy for? A cautionary tale. Sociological Methods & Research, 50(4), 1487–1514.
10.
EroshevaE. A.CurtisS. M. (2017). Dealing with reflection invariance in Bayesian factor analysis. Psychometrika, 82, 295–307.
11.
Frühwirth-SchnatterS.LopesH. F. (2018). Sparse Bayesian factor analysis when the number of factors is unknown. https://arxiv.org/pdf/1804.04231.pdf
12.
GeorgeE. I.McCullochR. E. (1993). Variable selection via Gibbs sampling. Journal of the American Statistical Association, 88(423), 881–889.
13.
GewekeJ. F.SingletonK. J. (1980). Interpreting the likelihood ratio statistic in facto models when sample size is small. Journal of the American Statistical Association, 75(369), 133–137.
GhoshJ.DunsonD. B. (2009). Default prior distributions and efficient posterior computation in Bayesian factor analysis. Journal of Computational and Graphical Statistics, 18(2), 306–320.
16.
GuoZ. Y. (2017). Heavy-tailed distributions and risk management of equity market tail events. Journal of Risk and Control, 4(1), 31–41.
17.
HayashiK.BentlerP. M.YuanK. H. (2007). On the likelihood ratio test for the number of factors in exploratory factor analysis. Structural Equation Modeling, 14(3), 505–526.
18.
HolzingerK. J.SwinefordF. (1939). A study in factor analysis: The stability of a bi-factor solution. Supplementary Educational Monographs, 48, xi–91.
19.
KaufmannS.SchumacherC. (2017). Identifying relevant and irrelevant variables in sparse factor models. Journal of Applied Econometrics, 32(6), 1123–1144.
20.
KuoL.MallickB. (1998). Variable selection for regression models. Sankhyā: The Indian Journal of Statistics, Series B, 60(1), 65–81.
21.
LeeS. Y.SongX. Y. (2002). Bayesian selection on the number of factors in a factor analysis model. Behaviormetrika, 29(1), 23–39.
22.
LesaffreE.LawsonA. B. (2012). Bayesian biostatistics (statistics in practice) (1st ed.). Wiley.
23.
LopesH. F.WestM. (2004). Bayesian model assessment in factor analysis. Statistica Sinica, 14, 41–67.
24.
LucasJ.CarvalhoC.WangQ.BildA.NevinsJ. R.WestM. (2006). Sparse statistical modelling in gene expression genomics. Bayesian Inference for Gene Expression and Proteomics, 1, 0–1.
25.
ManA. X.CulpepperS. A. (2022). A mode-jumping algorithm for Bayesian factor analysis. Journal of the American Statistical Association, 117(537), 277–290.
26.
MavridisD.NtzoufrasI. (2014). Stochastic search item selection for factor analytic models. British Journal of Mathematical and Statistical Psychology, 67(2), 284–303.
27.
MitchellT. J.BeauchampJ. J. (1988). Bayesian variable selection in linear regression. Journal of the American Statistical Association, 83(404), 1023–1032.
28.
PanD.WeiY.SongX. (2021). Joint analysis of mixed types of outcomes with latent variables. Statistics in Medicine, 40(5), 1272–1284.
29.
PapastamoulisP.NtzoufrasI. (2022). On the identifiability of Bayesian factor analytic models. Statistics and Computing, 32(2), 1–29.
30.
PeetersC. F. W. (2012). Bayesian exploratory and confirmatory factor analysis: Perspectives on constrained-model selection[Unpublished doctoral dissertation]. Utrecht University.
31.
RočkováV.GeorgeE. I. (2016). Fast Bayesian factor analysis via automatic rotations to sparsity. Journal of the American Statistical Association, 111(516), 1608–1622.
32.
RosseelY. (2012). Lavaan: An r package for structural equation modeling. Journal of Statistical Software, 48, 1–36.
33.
RoweD. B.PressS. J. (1994). Bayesian factor analysis by Gibbs sampling and iterated conditional modes.
34.
SimonM. K. (2007). Probability distributions involving Gaussian random variables: A handbook for engineers and scientists (1st ed.). Springer-Verlag.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
