When data come from an unobserved heterogeneous population, common factor analysis is not appropriate to estimate the underlying constructs of interest. By replacing the traditional assumption of Gaussian distributed factors by a finite mixture of multivariate Gaussians, the unobserved heterogeneity can be modelled by latent classes. In so doing, we obtain a particular factor mixture analysis with heteroscedastic components. In this paper, the model is presented and a maximum likelihood estimation procedure via the expectation–maximization algorithm is developed. We also show that the approach well performs as a dimensionally reduced model-based clustering. Two real applications are illustrated and performances are compared to standard model-based clustering methods.
Atchley WR and Martin J (1971) A Morphometric analysis of differential sexual dimorphism in larvae of Chironomus . Canadian Entomologist, 108, 819–27 .
Baek J and McLachlan GJ (2008) Mixtures of factor analyzers with common factor loadings for the clustering and visualisation of high-dimensional data. Technical Report NI08018-SCH, Preprint Series of the Isaac Newton Institute for Mathematical Sciences, Cambridge .
4.
Banfield JD and Raftery AE (1993) Model-based Gaussian and non-Gaussian clustering . Biometrics, 49, 803–21 .
5.
Bartholomew DJ (1988) The sensitivity of latent trait analysis to choice of prior distribution . British Journal of Mathematical and Statistical Psychology, 41, 101–7 .
6.
Bock HH (1996) Probabilistic models in cluster analysis . Computational Statistics and Data Analysis, 23, 5–28 .
7.
Bouveyron C , Girard S and Schmid C (2007) High-dimensional data clustering . Computational Statistics and Data Analysis, 52, 502–19 .
8.
Calò DG , Montanari A and Viroli C (2006) Model-based density estimation by independent factor analysis. In From data and information analysis to knowledge engineering, Spiliopoulou M , Kruse R , Borgelt C , Nrnberger A , Gaul W eds.Studies in classification, data analysis, and knowledge organization. Berlin: Springer-Verlag , 166–73.
9.
Dempster NM , Laird AP and Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm (with discussion) . Journal of the Royal Statistical Society, Series B, 39, 1–38 .
10.
Forgy EW (1965) Cluster analysis of multivariate data: efficiency vs interpretability of classifications . Biometrics, 21, 768–69 .
11.
Fraley C (1998) Algorithms for model-based Gaussian hierarchical clustering . SIAM Journal on Scientific Computing, 20, 270–81 .
12.
Fraley C and Raftery AE (2002a) Model-based clustering, discriminant analysis, and density estimation . Journal of the American Statistical Association, 97, 611–31 .
13.
Fraley C and Raftery AE (2002b) MCLUST: software for model-based clustering, discriminant analysis, and density estimation. Technical Report No. 415, Department of Statistics, University of Washington .
14.
Fraley C and Raftery AE (2003) Enhanced software for model-based clustering, discriminant analysis, and density estimation: MCLUST . Journal of Classification, 20, 263–86 .
15.
Ghahramani Z and Hilton GE (1997) The EM algorithm for mixture of factor analyzers. Technical Report CRG-TR-96-1, Department of Computer Science, University of Toronto, Canada .
16.
Gordon AD (1999) Classification. London: Chapman & Hall, CRC .
17.
Hartigan JA and Wong MA (1979) A K-means clustering algorithm . Applied Statistics, 28, 100–08 .
18.
Hubert L and Arabie P (1985) Comparing partitions . Journal of Classification, 2, 193–218 .
19.
Hyvarinen A , Karhunen J and Oja E (2001) Independent component analysis. New York: John Wiley & Sons Inc .
20.
Jöreskog KG (1967) Some contributions to maximum likelihood factor analysis . Psychometrika, 32, 443–82 .
21.
Jöreskog KG (1971) Simultaneous factor analysis in several populations . Psychometrika, 36, 409–26 .
22.
Khan J , Wei J and Ringner M (2001) Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks . Nature Medicine, 7, 673–79 .
23.
Lawley DN (1940) The estimation of factor loadings by the method of maximum likelihood . Proceedings of the Royal Society of Edinburgh, 60, 64–82 .
24.
Lee T , Lewicki M and Sejnowski T (2000) ICA mixture models for unsupervised classification of non-Gaussian sources and automatic context switching in blind signal separation . IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 1078–89 .
25.
Lo Y , Mendell NR and Rubin DB (2001) Testing the number of components in a normal mixture . Biometrika, 88, 767–78 .
26.
Lubke GH and Muthén B (2005) Investigating population heterogeneity with factor mixture models . Psychological Methods, 10, 21–39 .
27.
Mardia KV , Kent JT and Bibby JM (1976) Multivariate analysis. Oxford: Academic Press .
28.
McLachlan GJ and Peel D (2000) Finite mixture models. New York: John Wiley & Sons Inc .
29.
McLachlan GJ , Peel D and Bean RW (2003) Modelling high-dimensional data by mixtures of factor analyzers . Computational Statistics and Data Analysis, 41, 379–88 .
30.
Meredith W (1993) Measurement invariance, factor analysis, and factorial invariance . Psychometrika, 58, 525–43 .
31.
Muthén B (1989) Multiple-group factor analysis with non-normal continuous variables . British Journal of Mathematical and Statistical Psychology, 42, 55–62 .
32.
Schwarz G (1978) Estimating the dimension of a model . Annals of Statistics, 6, 461–64 .
33.
Titterington DM , Smith AFM and Makov UE (1985) Statistical analysis of finite mixture distributions. New York: Wiley .
34.
Vermunt JK and Magidson J (2003) Latent class models for classification . Computational Statistics and Data Analysis, 41, 531–37 .
35.
Viroli C (2007) Fitting the independent factor analysis model using the MCMC algorithm . Journal of Statistical Computation and Simulation, 77, 725–37 .
36.
Yeoh E , Ross ME , et al. (2002) Classification, subtype discovery, and prediction of outcome in pediatric acute lymphoblastic leukemia by gene expression profiling . Cancer Cell, 1, 133–43 .
37.
Yoshida R , Higuchi T and Imoto S (2004) A mixed factors model for dimension reduction and extraction of a group structure in gene expression data . Proceedings of the 2004 IEEE Computational Systems Bioinformatics Conference, Stanford, CA.
38.
Yoshida R , Higuchi T , Imoto S and Miyano S (2006) ArrayCluster: an analytic tool for clustering, data visualization and model finder on gene expression profiles . Bioinformatics, 22, 1538–39 .
