The conceptual and methodological framework for measurement equivalence procedures has been well established and widely used. Although multilevel theories and methods have been widely used in organizational research, there is no comparable framework for measurement equivalence of multilevel constructs, or psychometric isomorphism. In this article, we present a conceptual and methodological framework for understanding and testing various forms of isomorphism. Within this framework, we explicate (a) the different types of psychometric isomorphism, (b) the conditions where psychometric isomorphism is appropriate and necessary, (c) how psychometric isomorphism corresponds with different composition models and estimation methods, and (d) the analytic procedures that can be used. Using simulated data, we also illustrate how the proposed procedures may be applied via two analytic methods—item response theory and factor analysis. We conclude with a discussion of theoretical and methodological implications provided by the proposed framework of psychometric isomorphism.
AdamsR. J.WilsonM.WuM. (1997). Multilevel item response models: An approach to errors in variables regression. Journal of Educational and Behavioral Statistics, 22, 47–76. doi:10.2307/1165238
2.
AguinesH.PierceC. A.BoscoF. A.MuslinI. S. (2009). First decade of Organizational Research Methods: Trends in design, measurement, and data-analysis topics. Organizational Research Methods, 12, 69–112. doi:10.1177/1094428108322641
3.
American Psychological Association. (1985). Standards for educational and psychological testing. Washington, DC: Author.
4.
American Psychological Association, American Educational Research Association, & National Council of Measurement Use in Education. (1999). Standards for educational and psychological testing. Washington, DC: Author.
5.
American Psychological Association, American Educational Research Association, & National Council of Measurement Use in Education (joint committee). (1954). Technical recommendations for psychological tests and diagnostic techniques. Psychological Bulletin, 51, 201–238.
6.
BarryB.StewartG. (1997). Composition, process, and performance in self-managed groups: The role of personality. Journal of Applied Psychology, 82, 62–78. doi:10.1037/0021-9010.82.1.62
7.
BentlerP. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238–246.
8.
BentlerP. M.BonnettD. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–606.
9.
BlieseP. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In KleinK. J.KozlowskiS. W. J. (Eds.), Multilevel theory, research, and methods in organizations (pp. 349–381). San Francisco, CA: Jossey-Bass.
10.
BlieseP. D.ChanD.PloyhartR. E. (2007). Multilevel methods: Future directions in measurement, longitudinal analyses and nonnormal outcomes. Organizational Research Methods, 4, 551–563. doi:10.1177/1094428107301102
11.
BorsboomD.MellenberghG. J.van HeerdenJ. (2003). The theoretical status of latent variables. Psychological Review, 110, 203–219.
12.
BozdoganH. (1987). Model selection for Akaike’s information criteria. Psychometrika, 53, 345–370.
13.
ChanD. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234–246. doi:10.1037/0021-9010.83.2.234
14.
ChenG.BlieseP. D.MathieuJ. E. (2005). Conceptual framework and statistical procedures for delineating and testing multilevel theories of homology. Organizational Research Methods, 8, 375–409. doi:10.1177/1094428105280056
15.
ChenG.MathieuJ. E.BlieseP. D. (2004). A framework for conducting multi-level construct validation. Research in Multi Level Issues, 3, 273–303.
16.
CheungG. W.RensvoldR. W. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233–255.
17.
CheungM. W.-L.LeungK.AuK. (2006). Evaluating multilevel models in cross-cultural research: An illustration with social axioms. Journal of Cross-Cultural Psychology, 37, 522–541. doi:10.1177/0022022106290476
18.
CronbachL. J.MeehlP. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302.
19.
DedrickR. F.GreenbaumP. E. (2011). Multilevel confirmatory factor analysis of a scale measuring interagency collaboration of children’s mental health agencies. Journal of Emotional and Behavioral Disorders, 19, 27–40.
20.
DiMaggioP. J.PowellW. W. (1983). The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological Review, 48, 147–160.
21.
DrasgowF. (1984). Scrutinizing psychological tests: Measurement equivalence and equivalent relations with external variables. Psychological Bulletin, 95, 134–135. doi:10.1037/0033-2909.95.1.134
22.
DrasgowF.HulinC. L. (1990). Item response theory. In DunnetteM. D.HoughL. M. (Eds.), Handbook of industrial and organizational psychology (pp. 577–636). Palo Alto, CA: Consulting Psychologist Press.
23.
DrasgowF.LevineM. V.TsienS.WilliamsB. A.MeadA. D. (1995). Fitting polytomous item response models to multiple-choice tests. Applied Psychological Measurement, 19, 145–165. doi:10.1177/014662169501900203
24.
DyerN. G.HangesP. J.HallR. J. (2005). Applying multilevel confirmatory factor analysis techniques to the study of leadership. Leadership Quarterly, 16, 149–167. doi:10.1016/j.leaqua.2004.09.009
25.
EndersC. K.TofighiD. (2007). Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological Methods, 12, 121–138.
26.
FelpsW.MitchellT. R.HekmanD. R.LeeT. W.HoltomB. C.HarmanW. S. (2009). Turnover contagion: How coworkers’ job embeddedness and job search behaviors influence quitting. Academy of Management Journal, 52, 545–561.
27.
FischerR.VauclairC.-M.FontaineJ. R. J.SchwartzS. H. (2010). Are individual-level and country-level value structures different? Testing Hofstede’s legacy with the Schwartz value survey. Journal of Cross-Cultural Psychology, 41, 135–151. doi:10.1177/0022022109354377
28.
FontaineJ. R. J.FischerR. (2011). Data analytic approaches for investigating isomorphism. In MatsumotoD.Van de VijverF. J. R. (Eds.), Cross-cultural research methods in psychology (pp. 273–298). Cambridge, UK: Cambridge University Press.
29.
FoxJ.-P.GlasC. A. W. (2001). Bayesian estimation of a multilevel IRT model using Gibbs sampling. Psychometrika, 66, 269–286. doi:10.1007/BF02294839
30.
GelfandM. J.NishiiL. H.RaverJ. L. (2006). On the nature and importance of cultural tightness-looseness. Journal of Applied Psychology, 91, 1225–1244. doi:10.1037/0021-9010.91.6.1225
31.
GottfredsonN. C.PanterA. T.DayeC. E.AllenW. F.WightmanL. F. (2009). The effects of educational diversity in a national sample of law students: Fitting multilevel latent variable models in data with categorical indicators. Multivariate Behavioral Research, 44, 305–331.
32.
HannerzU. (1992). Cultural complexity: Studies in the social organization of meaning. New York, NY: Columbia University Press.
33.
HendersonD. J.WayneS. J.ShoreL. M.BommerW. H.TetrickL. E. (2008). Leader-member exchange, differentiation, and psychological contract fulfillment: A multilevel examination. Journal of Applied Psychology, 6, 1208–1219. doi:10.1037/a0012678
34.
HofmannD. A.GavinM. B. (1998). Centering decisions in hierarchical linear models: Implications for research in organizations. Journal of Management, 24, 623–641.
35.
HofmannD. A.JonesL. M. (2005). Leadership, collective personality, and performance. Journal of Applied Psychology, 90, 509–522. doi:10.1037/0021-9010.90.3.509
36.
IdaszakJ. R.BottomW. P.DrasgowF. (1988). A test of measurement equivalence of the revised Job Diagnostic Survey: Past problems and current solutions. Journal of Applied Psychology, 73, 647–656.
37.
KozlowskiS. W. J.KleinK. J. (2000). A multilevel approach to theory and research in organizations. In KleinK. J.KozlowskiS. W. J. (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 3–90). San Francisco, CA: Jossey-Bass.
38.
LeBretonJ. M.SenterJ. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11, 815–852. doi:10.1177/1094428106296642
39.
LudtkeO.MarshH. W.RobtizschA.TrautweinU. (2011). A 2 × 2 taxonomy of multilevel latent contextual models: Accuracy-bias trade-offs in full and partial error correction models. Psychological Methods, 16, 444–467. doi:10.1037/a0024376
40.
LudtkeO.MarshH. W.RobtizschA.TrautweinU.AsparouhovT.MuthenB. (2008). The multilevel latent covariate model: A new, more reliable approach to group-level effects in contextual studies. Psychological Methods, 13, 203–229. doi:10.1037/a0012869
41.
McCraeR. R.TerraccianoA. (2008). The five-factor model and its correlates in individuals and cultures. In Van de VijverF. J. R.Van HemertD. A.PoortingaY. H. (Eds.), Multilevel analysis of individuals and cultures (pp. 249–283). Hillsdale, NJ: Lawrence Erlbaum.
42.
MeyerJ. W.RowanB. (1977). Institutionalized organizations: Formal structure as myth and ceremony. American Journal of Sociology, 83, 340–363.
43.
MorgesonF. P.HofmannD. A. (1999). The structure and function of collective constructs: Implications for multilevel research and theory development. Academy of Management Journal, 24, 249–265. doi:10.2307/259081
44.
MorrisonE. W.Wheeler-SmithS. L.KamdarD. (2011). Speaking up in groups: A cross-level study of group voice climate and voice. Journal of Applied Psychology, 96, 183–191. doi:0.1037/a0020744
45.
MuthénB. (1991). Multilevel factor analysis of class and student achievement components. Journal of Educational Measurement, 28, 338–354. doi:10.1111/j.1745-3984.1991.tb00363.x
MuthénB.MuthénL. K. (2007). Mplus version 5.2[Computer software]. Los Angeles, CA: Muthén & Muthén.
48.
MuthénL. K.MuthénB. (2007). Multilevel modeling with latent variables using Mplus. Unpublished manuscript.
49.
OstroffC.KinickiA. J.ClarkM. A. (2002). Substantive and operational issues of response bias across levels of analysis: An example of climate-satisfaction relationships. Journal of Applied Psychology, 87, 355–368. doi:10.1037//0021-9010.87.2.355
50.
ParkG.DeShonR. P. (2010). A multilevel model of minority opinion expression and team decision-making effectiveness. Journal of Applied Psychology, 95, 824–833. doi:0.1037/a0019939
51.
PelledL. H.EisenhardtK. M.XinK. R. (1999). Exploring the black box: An analysis of work group diversity, conflict, and performance. Administrative Science Quarterly, 44, 1–28. doi:10.2307/2667029
52.
RaudenbushS. W.SampsonR. J. (1999). Ecometrics: Toward a science of assessing ecological settings, with application to the systematic social observation of neighborhoods. Sociological Methodology, 29, 1–41. doi:10.1111/0081-1750.00059
53.
ReiseS. P.VenturaJ.NuechterleinK. H.KimK. H. (2005). An illustration of multilevel factor analysis. Journal of Personality Assessment, 84, 126–136.
54.
RobertC.LeeW. C.ChanK.-Y. (2006). An empirical analysis of measurement equivalence with the INDCOL measure of individualism and collectivism: Implications for valid cross-cultural inference. Personnel Psychology, 59, 65–99.
55.
RousseauD. M. (1985). Issues of level in organizational research: Multi-level and cross-level perspectives. Research in Organizational Behavior, 7, 1–37.
56.
SaccoJ. M.SchmittN. (2005). A dynamic multilevel model of demographic diversity and misfit effects. Journal of Applied Psychology, 90, 203–231. doi:10.1037/0021-9010.90.2.203
57.
SchwartzS. H. (1992). Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries. Advances in Experimental Social Psychology, 25, 1–65. doi:10.1016/S0065-2601(08)60281-6
58.
SchwartzS. H. (2006). A theory of cultural value orientations: Explication and applications. Comparative Sociology, 5, 136–182.
59.
SchwarzG. E. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464.
SteelP.OnesD. S. (2002). Personality and happiness: A national-level analysis. Journal of Personality and Social Psychology, 83, 767–781. doi:10.1037/0022-3514.83.3.767
62.
StarkS.ChernyshenkoO. S.DrasgowF. (2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy. Journal of Applied Psychology, 91, 1292–1306. doi:10.1037/0021-9010.91.6.1292
63.
TajfelH. (1982). Social psychology of intergroup relations. Annual Review of Psychology, 33, 1–39. doi:10.1146/annurev.ps.33.020182.000245
64.
TarasV.KirkmanB. L.SteelP. (2010). Examining the impact of culture’s consequences: A three-decade, multilevel, meta-analytic review of Hofstede’s cultural value dimensions. Journal of Applied Psychology, 95, 405–439.
65.
TayL.DienerE. (2011). Needs and subjective well-being. Journal of Personality and Social Psychology, 101, 354–365. doi:10.1037/a0023779
66.
TayL.DienerE.DrasgowF.VermuntJ. K. (2011). Multilevel mixed-measurement IRT analysis: An explication and application to self-reported emotions around the world. Organizational Research Methods, 14, 177–207. doi:10.1177/1094428110372674
67.
TayL.KuykendallL. (2013). Promoting happiness: Malleability of individual and societal-level happiness. International Journal of Psychology, 48, 159–176. doi:10.1080/00207594.2013.779379
68.
TayL.NewmanD. A.VermuntJ. K. (2011). Using mixed-measurement item response theory with covariates (MM-IRT-C) to ascertain observed and unobserved measurement equivalence. Organizational Research Methods, 14, 147–176. doi:10.1177/1094428110366037
69.
TayL.VermuntJ. K.WangC. (2013). Assessing the item response theory with covariate (IRT-C) procedure for ascertaining DIF. International Journal of Testing, 13, 201–222.
70.
TuckerL. R.LewisC. (1973). The reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1–10.
71.
van de VijverF. J. R.LeungK. (2001). Personality in cultural context: Methodological issues. Journal of Personality, 69, 1007–1031. doi:10.1111/1467-6494.696173
72.
van de VijverF. J. R.PoortingaY. H. (2002). Structural equivalence in multilevel research. Journal of Cross-Cultural Psychology, 33, 141–156.
73.
van de VijverF. J. R.WatkinsD. (2006). Assessing similarity of meaning at the individual and country level. European Journal of Psychological Assessment, 22, 69–77. doi:10.1027/1015-5759.22.2.69
74.
van MierloH.VermuntJ. K.RutteC. G. (2009). Composing group-level constructs from individual-level survey data. Organizational Research Methods, 12, 368–392.
75.
VandenbergR. J.LanceC. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–70. doi:10.1177/109442810031002
ZyphurM. J.KaplanS. A.ChristianM. S. (2008). Assumptions of cross-level measurement and structural invariance in the analysis of multilevel data: Problems and solutions. Group Dynamics: Theory, Research, and Practice, 12, 127–140. doi:10.1037/1089-2699.12.2.127
