Abstract
Introduction
In this
In this article we use the term “multilevel research,” and not the more common expression, “multilevel analysis,” in order to offer a broader vision of this methodological approach. When authors refer to “multilevel analysis” or “multilevel models” (some books use these expressions in their titles, such as Goldstein (1995), Hox (2002), Kreft and de Leeuw (2002), Snijders and Bosker (2004)), they focus mainly on statistical techniques for conducting analysis on several levels (e.g., hierarchical linear models, as noted below). By using the term “multilevel research,” we want to emphasize that this approach goes beyond statistical techniques (Hitt et al., 2007; Klein & Kozlowski, 2000), and consider other important elements. Multilevel research includes the development of multilevel theory (e.g., combining different theoretical approaches at different levels and establishing relationships between constructs at different levels), as well as the main elements of methods for empirical studies (sampling, data collection, variables and their measures, and analysis techniques, which may include quantitative and qualitative techniques).
The main purpose of this
The main contribution of this
The focus and intent of this work is to introduce the main ideas of multilevel research. In management studies, more researchers use a single level of analysis than use multilevel research, and we consider that many scholars in our field do not know the foundations of this type of research. Our approach will be to see the “forest” rather than some particular “trees.” We examine the big picture, indicating the main elements of multilevel research. An exhaustive analysis of all the elements of multilevel research goes beyond the purpose of this
Why is multilevel research interesting for management studies?
In this section we highlight opportunities and potential offered by multilevel research. Specifically, we indicate its role in bridging two important gaps in management: the micro–macro gap and the research–practice gap. We also examine the benefits of integrating theories at different levels to advance management research. Finally, we suggest that avoiding the fallacy of the wrong level also justifies the application of multilevel research.
Bridging the micro–macro gap in management and integration of other levels
One characteristic of the development of management is the growing diversity and specialization. Several areas have emerged and consolidated in the field of management, for example, strategy, human resources, operations, organizational behavior, and environmental management. These specialized areas are reflected both in teaching (in specialized subjects in undergraduate and postgraduate degrees) and research (with specialized journals and conferences). One aspect of this growing specialization is the widely accepted distinction between macro and micro areas (Aguinis et al., 2011). Macro areas (e.g., strategic management and organization theory) mainly focus research questions and analysis at the organizational level, while micro areas (organizational behavior, organizational psychology) are concerned with research questions at levels of analysis within the organization, mainly at the individual and group level. The existence of macro and micro areas in the management field is reflected not only in the topics and levels studied but also in the creation of specific divisions within management associations (even specific associations linked to macro or micro areas), separate and independent sessions for each area within management conferences (even specific conferences for each macro or micro area), and specialized journals. This specialization has benefits that arise from the opportunities that occur when deepening specific topics. However, it also entails less positive aspects linked to a growing separation and fragmentation (Durand et al., 2017). The micro–macro gap is an important indication of this fragmentation.
The relevance of multilevel research is that it can help to bridge this gap by integrating disciplines and levels of analysis. Multilevel research emphasizes the joint analysis of variables located at different levels, examining relationships between them. Many multilevel works examine in the same study variables at the micro (individual or group) level and variables at the macro (organization) level, and then integrate these levels bridging the micro–macro gap. 1
Multilevel research can also integrate other levels. For example, a multilevel study can integrate the organizational level and higher levels. In strategic management, several studies have examined the importance of determinants of firm performance, especially the firm, corporate, and industry effects, but also other determinants such as the strategic group and the location effects (Hough, 2006; Misangyi et al., 2006; Molina-Azorín et al., 2010; Pereira-Moliner et al., 2011a; Short et al., 2007). 2 In addition, levels below the organizational level can also be integrated, for example, individuals and work groups, as has been studied in the area of organizational behavior. 3 As examined in the next section, the integration of several levels in the same study can help to address and bridge another traditional gap in management: the science–practice gap.
Bridging the science–practice gap
The increasing specialization can also lead to another important gap in management: the science–practice gap (research–practice gap, or academic rigor–practical relevance gap) (Bansal et al., 2012; Kieser et al., 2015). Specialization and fragmentation in the management field can lead to separate, independent and very specific studies that are very different from real and integrated business problems. Academic studies tend to be remote from issues and solutions, and from research questions and responses that may be of interest to companies and managers in their daily practice. In other words, the fragmentation and disintegration of the field of management into various disciplines and areas means that studies carried out in each area suppose an oversimplification of the complexity of managers’ tasks and practices (Pettigrew & Starkey, 2016).
The real problems faced by companies and managers usually involve a joint analysis of various disciplines. For example, the formulation and implementation of strategies at the corporate and business levels must take into account actions and decisions in the operations and human resources departments. These problems presuppose relationships between aspects located at different levels of analysis. For example, a human resources management system or the firm culture (variables at the organizational level) can influence employees’ behavior at the individual level (e.g., their motivation and satisfaction), which in turn can also influence both firm and individual performance. As indicated by Hitt et al. (2007), most problems that managers face imply phenomena at multiple levels, but most research is conducted on a single level.
In activities carried out in business practice (analysis of information, decision making by managers, implementation of decisions, etc.), elements at different levels are involved (e.g., a management system at the organizational level, characteristics of managers and their actions, and interactions between managers and employees). The joint analysis of several levels through multilevel studies can close this science–practice gap. As indicated above, the problems that managers must solve involve actions and variables at different levels, and from several disciplines within management. Therefore, multilevel research brings us closer to the reality of business practice.
This usefulness and practical relevance of research in management is one of the basic pillars of responsible research, along with credibility. Therefore, multilevel research can also contribute to promoting responsible research and its basic principles (the basic principles of responsible research can be found at https://www.rrbm.network/).
Integration of theories and improvement of theoretical development in management
Related to the previous aspects, multilevel research promotes the integration of theories from different disciplines. This methodological approach facilitates theoretical development to improve knowledge of business phenomena, which can help the advance of management (Shaw et al., 2018). If a certain business phenomenon is multilevel in nature, theory and techniques of analysis should also be multilevel (Hitt et al., 2007; Luke, 2004; Mathieu & Chen, 2011). Therefore, multilevel research can promote the improvement of theoretical development by facilitating the understanding of business phenomena with antecedents and/or consequences in different contexts and at different levels. For example, we could analyze the influences that variables located at two or more levels have on a dependent variable located at one of these levels (direct effects). We could also analyze a cross-level interaction, examining how the linkage between two variables at the same level is influenced or moderated by a variable at a different level (Aguinis et al., 2013).
In addition, regarding the distinction between macro and micro areas in management, there are movements and initiatives in different areas to integrate the parts. In macro areas, where research questions and issues are examined at the organizational level, the role of individuals is being encouraged. For example, in strategic management, the microfoundations movement (Felin et al., 2015; Felin & Foss, 2005; Molina-Azorín, 2014) emphasizes the key role of individual actions and interactions to explain strategic phenomena, as well as the reciprocal influences between the individual and the organizational levels in the study of strategic issues. In micro areas, such as organizational behavior and organizational psychology, the integration of macro variables and relationships with macro approaches is also being encouraged (Ployhart, 2015; Wright & Nishii, 2007). Another example is the area of human resources management. Although this area usually includes micro-level work analyzing the effects of individual characteristics on individual variables, macro work has also been carried out, for example, in the specific area of strategic human resources management, examining, among other aspects, the influence of systems and organizational practices of human resources on organizational performance. Ostroff and Bowen (2000) defend the integration of micro (individual and group) and macro (organizational) aspects in research on human resources management, being this integration a key aspect for theory development in this area.
The key point is that multilevel research can play an important role in these initiatives where micro areas seek to integrate macro aspects and macro areas try to integrate micro issues. Multilevel research may also be relevant to the conduct of multidisciplinary studies that integrate theories, relationships, processes, and variables from different micro and macro areas, and from different levels of analysis.
The fallacy of the wrong level
As indicated by Gaviria and Castro (2005), most data in social science research come from phenomena where subjects form nested hierarchies. For example, in the field of education, students are grouped or nested in classrooms, which in turn are grouped into schools. In management, employees are nested in work groups or departments, which are grouped into firms, and these are nested in industries.
Before the use of multilevel research to analyze relationships between variables at different levels, there were two ways to study nested data: aggregation and disaggregation. But these two strategies may produce errors when conclusions are drawn at the wrong level, that is, when inferences and interpretations are made about relationships between variables at a certain level but the analysis has been carried out at a different level, or when an effect, variable, or relationship are attributed to a level of analysis when they really refer to a different level (Dansereau et al., 2006; Hitt et al., 2007; Hox, 2002; Mathieu & Chen, 2011; Rousseau, 1985; Snijders & Bosker, 2004).
Aggregation consists of obtaining data at a lower level, and combining the values of those variables to the higher level. For example, if we analyze employees who work in different organizations and we have data from several employees for each organization, the average for each company may be calculated from their employees. 4 The analysis may then be carried out at that organizational level, where we could also have and use other variables at this level. If we are interested in the relationships between variables at this organizational level (because this is the theoretical level of hypotheses), then there is not any problem. But if the theoretical level of interest is the individual level (employees), it would be a mistake to interpret the results of the organizational relationships and generalize them at the level of employees. In other words, we cannot apply the results that refer to the companies to the individuals. This error is known as the ecological fallacy. The main issue is that we have eliminated variance within companies, and the relationships may be strong at the organizational level, but can be very different at the individual level. In short, we would have a statistical problem and a conceptual/theoretical problem (Hox, 2002). The statistical problem stems from the fact that the employees’ values have been combined to form a smaller number of company values, losing much information and power in the statistical analysis. We would have a conceptual problem as we interpret results that are valid at the company level as if they were also valid at the individual level. Given the importance of aggregation in multilevel research, we will discuss some aspects later.
In disaggregation, data from higher level units are disaggregated into data on a larger number of lower level units (Hox, 2002; Snijders & Bosker, 2004). Using the example of companies and employees, the values of the variables at the company level would be assigned to their corresponding employees. Each employee would also have individual values for other variables. If an analysis is carried out at the individual level, there would be a statistical problem derived from the lack of independence: the employees of two different companies (and their observations) are independent, but two employees working in the same company (and their observations) are not independent, as they receive common influences (which, possibly, will not be measured). Later we will emphasize this problem of lack of independence. Another problem is that we could not interpret or make inferences at the company level based on the analysis developed at the individual level. This is known as the atomistic fallacy (Hox, 2002).
In summary, the problem with aggregation and disaggregation is that we would reach irrelevant conclusions regarding a certain level when using data measured and analyzed at a different level. Multilevel research helps to deal with these problems or fallacies that result from ignoring the hierarchical and nested structure of the data. Thus, this methodological approach resolves the dilemma between aggregation and disaggregation, working with several levels simultaneously.
Fundamentals of multilevel research
As well as knowing why multilevel research should be conducted, it is also important to know how to carry out a multilevel study. Describing in depth all aspects and steps to conduct multilevel research goes beyond this
Pioneering and basic works on multilevel research in management
The main principles, methodological foundations, and techniques linked to multilevel research have been developed in other fields, mainly in education and psychology. For example, in education an important issue is to analyze the causes of students’ performance (individual level), which could be determined by aspects and characteristics of students (individual level) as well as contextual elements of their classrooms and schools (higher levels). In the management field, authors from micro areas, such as organizational behavior and organizational psychology (e.g., Bliese, Dansereau, Hofmann, Klein, Kozlowski, Mathieu, Rousseau, Yammarino), have mainly carried out the pioneering works on multilevel research (conceptual, theoretical, methodological, and empirical studies).
Rousseau (1985) can be considered one of the pioneering works that introduces key aspects of multilevel research in the field of management in general, and in organizational behavior in particular. This author refers to an important book (Roberts et al., 1978) which defends the need to conduct multidisciplinary research, highlighting the excessive specialization in the management field. Rousseau (1985) points out possibilities of collaboration by establishing some basic principles of multilevel research. Later, this author, together with other colleagues (House et al., 1995), would emphasize the need to develop a “meso” paradigm, that is, the simultaneous study of at least two levels of analysis (micro and macro) and their reciprocal relationships, indicating the need for this perspective and some basic concepts and principles of multilevel research.
In the three decades of multilevel research in management since the publication of Rousseau (1985), a key publication is the book, published in 2000, titled
Other important works in the development and consolidation of multilevel research in management include a book series edited by Yammarino and Dansereau (
In addition, some literature reviews, articles, and chapters have also been published highlighting the potential of multilevel research in specific management areas, such as human resources (K. Jiang et al., 2013; Peccei & Van de Voorde, 2019; Shen, 2016), organizational psychology and organizational behavior (González-Romá & Hernández, 2017), international business (Arregle et al., 2006; Peterson et al., 2012), and strategic management (Moliterno & Ployhart, 2016). These methodological works and literature reviews examine empirical studies published in these areas, emphasizing best practice recommendations for designing and conducting multilevel studies. Empirical works can also be of value, illustrating how other colleagues design and implement multilevel studies. In the next sections of this
Basic principles of multilevel research
Nested/hierarchical structure, main models and relationships, and applications
As noted above, a key idea of multilevel research is the existence of a hierarchical system of several levels, where some entities reside in nested structures (Hitt et al., 2007). For example, in management studies, employees are nested in groups or work teams, which in turn are nested in other larger organizational units (e.g., departments), which in turn are nested in firms. Moreover, firms are members of industries, and industries are located in certain environments or territories.
Another important aspect is that variables at each level may influence variables at other levels. For example, in organizational behavior, the dependent variable that is usually analyzed is some outcome variable of employees (e.g., productivity). This variable at the individual/employee level can depend on both the characteristics of the employee (lower level or level 1) (e.g., job satisfaction and/or motivation) and variables of the group or team where each employee works (higher level or level 2) (e.g., climate or cohesion). In strategic management, a key research question is why some firms are more profitable than others. We have a structure or hierarchical pattern of nested relationships: firm profitability depends on firm resources and capabilities (firms as the lower level) and on the industries membership (industries as the higher level).
Regarding relationships between levels, influences may be reciprocal between the levels considered. Thus, productivity of an employee is influenced by the characteristics of her or his work team, and the team productivity is also influenced by employees. Performance of a firm is influenced by the characteristics of its industry, but the actions by firms also impact on industry performance and its characteristics. Therefore, a phenomenon at any level may have antecedents and consequences at other levels, and it is of special interest to analyze the immediate levels, both at higher and lower levels (Hackman, 2003).
Figure 1 represents multilevel antecedents and consequences for some business aspect. For example, a firm capability can have antecedents and consequences at the organizational level, as well as at a higher level (industry) and lower level (individuals). In multilevel research, we refer to the different levels with a number; the higher level is indicated with the greater number (level 1 =
Multilevel antecedents and outcomes of a firm capability (three levels).
Most multilevel studies examine two levels, and there are two main types of direct relationships and models (Figure 2) (Aguinis & Molina-Azorin, 2015). In top-down models, the outcome variable is located at the lower level, and we analyze some antecedent at this lower level and other antecedents at a higher level. The traditional multilevel analysis techniques, such as hierarchical linear models (HLMs), were developed for these top-down models. An example in the field of human resources would be to consider as a dependent variable the individual performance of each employee (level 1), using as explanatory antecedents variables at the individual (level 1) (such as motivation) and variables at the organizational level (level 2) (such as the human resources management system or organizational practices). Another example in the field of strategy would be to analyze the profitability of firms as the dependent variable (level 1), using as explanatory variables some characteristics of these firms (level 1) and some characteristics of their industries (level 2). In bottom-up models, the outcome variable is located at the higher level and the antecedents are located at that level and some lower level. For example, we can examine how firm profitability depends on some firm characteristics and some individual features of employees.
Multilevel top-down and bottom-up models.
In the previous examples there is a pure and strict hierarchical relationship. For example, when examining employees and firms (two levels), each employee works in a specific firm. In other words, each unit of the micro level (employee) is nested in one (and only one) unit of the macro level (firm). This pure hierarchical relationship can also occur when we analyze more than two levels. For example, employees (level 1, micro) are nested in firms (level 2, macro), and these firms are nested in industries (level 3, macro). Each employee works in a specific firm and each firm belongs to a specific industry. Apart from these pure hierarchical relationships, there are other possible applications of multilevel research.
An extension is multilevel cross-classification. In this case, several macro levels are used and the units of the micro level are nested simultaneously in these macro levels, but there is not a pure and strict hierarchy between these macro levels. Pereira-Moliner et al. (2011a) used this cross-classification structure in their multilevel study of the influence of firms, strategic groups, and location (regions) on firm profitability. The firms (micro level) are nested in strategic groups (macro level) and regions (macro level), that is, a firm belongs to a strategic group and a region. But there is not a pure hierarchical relationship between strategic groups and regions: a region can have firms from different strategic groups, and firms from the same strategic group can be located in different regions. Guo (2017) also uses a cross-classification scheme in a longitudinal multilevel study on the determinants of profitability of business units, considering as the higher-level corporations and industries.
Another application is multiple membership. Multilevel models usually refer to single membership: each unit of the micro level belongs to one (and only one) of the units of the macro level/levels considered. Thus, in the previous example of firms, strategic groups, and regions, each firm belongs to only one strategic group and only one region. But multilevel analysis also allows multiple membership. Mollick (2012) conducted a multilevel study with multiple membership and cross-classification and analyzed the role of two types of employees (designers and managers) on the development of games. Each of the designers and managers (micro level) performs their work for the development of more than one game (macro level), making them members of multiple groups.
Multilevel research can be also used to analyze longitudinal data. In previous examples, the macro levels referred to context variables that could influence variables at the micro levels. For example, industry variables may influence the profitability of firms. However, the higher/macro level does not have to be a context for the lower/micro level.
Longitudinal data (repeated measurements) can be seen as multilevel data, as those repeated measures correspond to the lower level (e.g., data on firm profitability for several years) and firms would be the higher level. These repeated measures are nested in the firms and are correlated within each firm. In the longitudinal multilevel study by Guo (2017), level 1 is repeated measures of profitability for several years of business units (level 2).
Existence of dependence
Another key aspect of multilevel research is the existence of dependence (or lack of independence) among the observations/measurements considered, as a consequence of nested structures (students and classrooms, workers and work groups, companies and industries, repeated measurements in longitudinal data). For example, in the field of education, in studies about performance of students (dependent variable, level 1) considering as independent variables both individual characteristics of students (level 1) and classroom aspects (level 2), the fact that different students are grouped within the same classroom implies that they share the same context variables of the classroom (e.g., teachers). In addition, people in the same group interact and this can also imply similarities in terms of performance obtained by students. In the case of longitudinal analysis, the repeated measures will be correlated (e.g., profitability for several years by each company considered). In the field of strategy, when analyzing, for example, the determinants of firm profitability (level 1) considering both firm factors (level 1) and characteristics of their industries (level 2), it is important to consider dependence, as firms in the same industry are influenced by the same structural variables of that industry.
As the independence assumption of observations of the classical linear model is not fulfilled, there are important statistical implications. If a traditional regression analysis (which assumes independence) is applied with nested data, type I error will increase (greater probability of rejecting the null hypothesis when it is true) as the existence of dependence will lead to obtaining a
Levels of theory, data source, measurement, and analysis
An important issue in multilevel research is the need to consider the levels of theory, data source, measurement, and analysis (Hitt et al., 2007; Hofmann, 2004; Klein et al., 1994; Mathieu & Chen, 2011; Moliterno & Ployhart, 2016; Rousseau, 1985). The level of theory refers to the entity or focal unit on which it is intended to make generalizations. In other words, it is the level at which a particular construct of effect is predicted to exist. The focal unit determines a specific level (organization, group, individuals, etc.) but in multilevel research different levels are considered. Scholars must clearly determine these levels to study theoretical relationships between them. Some authors prefer to consider the level of construct (Hofmann, 2004; Kozlowski & Klein, 2000) as a specific theory may include constructs that reside at different levels. Therefore, researchers must determine the level of each construct.
There may be a mismatch between the level of some construct and the level of the data source. For example, in research with primary data it is usual that managers (individual level as data source) provide information about a construct at the organizational level (e.g., firm competitive strategy). In this example, the level of measurement (the unit to which the data are directly attached) is the organization. The important point is that the level of construct is aligned with the level of measurement. If we are working with a construct at the organizational level and managers provide information about this construct, questions or items should refer to that organizational level (level of measurement).
The level of analysis is the unit to which data are assigned for hypothesis testing and statistical analysis. It is important that the levels of theory and measurement are aligned with the level of analysis. When these levels are misaligned, problems related to fallacies of the wrong level may occur (see above ideas about these fallacies). However, the level of measurement may differ from the level of analysis when an adequate process of aggregation is implemented. For example, several employees of a firm indicate their individual perceptions about the organizational culture (measurement at the individual level). If we want to analyze the influence of organizational culture on another organizational variable (e.g., firm performance) and, therefore, the analysis is at the organizational level, we must aggregate the individual perceptions of culture to create a variable at the organizational level. But, as indicated above, this aggregation must be justified both theoretically (processes that relate the two levels) and statistically (examining whether there is some agreement in individual assessments).
Justification for the application of multilevel research
The application of a multilevel design and analysis must be justified based on additional aspects over and above the existence of nested structures and dependence. A key requirement is to have sufficient data at the levels analyzed. Hitt et al. (2007) indicate that a multilevel theory may not necessitate a multilevel design, and they point out an interesting example. Upper echelons theory is multilevel in nature in which it incorporates features on individuals (mainly CEO), groups (top management team), and organizations. However, empirical research into this theory has been carried out at a single level (the organizational level) because there is only one CEO and one top management team per organization. Therefore, a multilevel design is not used as multiple lower-level entities must be nested within a higher-level entity (several top management teams and several CEOs would be needed in each firm).
Another requirement is to use sufficient data at the higher level. An example would be a study on behavior of companies in three different countries (several companies in each country). There is a nested structure and companies in each country could show similarities in their behavior due, for example, to the regulatory framework of each country. However, the fact that only three units (three countries) are available at the higher level (three countries) would violate one of the requirements of multilevel modeling as there are insufficient units at the micro level and the macro level. 5 Later, in the section on multilevel sampling, we examine the issue of sample sizes.
In addition, even if there are enough units at the micro and macro level, a statistical justification is required to conduct a multilevel analysis. Specifically, there should be significant differences in the lower-level variables based on the higher level. In other words, contextual variables at the macro level should exert an influence on the micro-level variables. This statistical justification can be based on various indices, such as the intraclass correlation coefficient 1-ICC(1)-, which determines which part of the variance of the dependent variable at the lower level is due to the higher-level variability (Bliese, 1998, 2000). ICC(1) values different from zero are desirable for considering the higher level. Bliese (2000) indicated that ICC(1) values typically range from .05 to .20. A high intergroup variability would justify the search for higher-level predictors. When ICC(1) is zero or very low, it would not make sense to include a higher-level predictor. In this case, the results of a multilevel model will be similar to a classical linear model, and the use of a multilevel analysis would not produce benefits in terms of bias and efficiency of the estimated parameters, even if the data have a nested hierarchical structure.
Theoretical, methodological, and analytical aspects in multilevel research
Theory and models in multilevel research
As will be indicated later, methodological and analytical issues are important in multilevel research. However, the first and key steps in multilevel research are those related to theoretical and conceptual elements, establishing theoretical arguments that serve to specify the model to analyze, justifying the relationships between variables as well as the processes and mechanisms that connect the variables at different levels. It will also be important to establish the constructs and their definition, justifying their levels.
With regard to multilevel models and relationships, we previously referred to top-down and bottom-up models (Figure 2). These models incorporate within-level direct effects and cross-level direct effects. We can extend the effects that can be examined, including cross-level moderation effects and aggregation/emergence processes (Moliterno & Ployhart, 2016). Cross-level mediation effects can also be considered and analyzed. Figure 3 includes all these effects.
Multilevel effects.
As noted above, theoretical arguments that justify the multilevel relationships among variables are important. Several theories that examine relationships at different levels may be integrated (e.g., macro theories at the organizational level and micro theories at the individual level). Theories that link constructs at different levels can also be used.
Sampling and data collection in multilevel research
Sampling in multilevel research is carried out in several stages, taking into account the levels that are considered. For example, in a multilevel study with data at the organizational level (level 2) and at the individual (level 1), there must be enough organizations (first stage of sampling) and enough individuals in each organization (second stage) to reach appropriate statistical power. Therefore, there are requirements of sample sizes needed at each level to conduct multilevel analysis.
The 30/30 rule of thumb (at least 30 organizations and at least 30 individuals in each organization) is usually indicated as the minimum to reach enough power for cross-level direct effects and cross-level interactions (Hox, 2002; Kreft & de Leeuw, 2002) to ensure sufficient variance between organizations and within organizations. However, this rule has been considered excessively demanding. In practice, studies consider fewer units in the lower level (in our example, the number of individuals within each organization). In fact, it is usually thought that it is more important to have a large number of elements in the higher level than in the lower level (Shen, 2016). Many published works include five or even fewer individuals in each organization, but try to have a large number of organizations. As there are several multilevel analysis techniques and multilevel effects (interactions, direct effects, . . .), different and specific requirements of sample sizes may apply (see González-Romá, 2019; González-Romá & Hernández, 2017; Maas & Hox, 2005).
In terms of data collected, some multilevel works use primary data sources (Liao & Chuang, 2004; Linuesa-Langreo, 2017; Pereira-Moliner et al., 2011b) and other studies use secondary data (Guo, 2017; Short et al., 2007). A positive aspect of multilevel studies (especially works that use primary data through questionnaires and the levels considered are individuals and organizations) is the use of several informants for each organization. As indicated by Bou-Llusar et al. (2016), empirical studies with a single informant present several problems. Multilevel studies require multiple informants in each organization, and this positive characteristic of multilevel studies could be strengthened in two ways: first, it is possible to use informants from different hierarchical levels within the organization (e.g., managers and employees); second, employees could provide information about variables related to their managers and managers could evaluate certain aspects of their employees. This is an important point as researchers must determine the appropriate informants for specific constructs. To avoid social desirability bias, in a study about leadership in teams, leaders/managers/supervisors in these teams could provide biased information about their own leadership style, and then it would be important that employees provide information about leadership style of their managers (Bavik et al., 2017; Y. Jiang & Chen, 2018; Linuesa-Langreo et al., 2018). As noted above, managers may also provide information about their employees. For example, in Zhu et al. (2018), managers assessed their subordinates’ creativity at work.
There are other important aspects to consider on data collection in multilevel studies. For example, in studies with primary data through questionnaires, a relevant issue will be the decision about the specific individuals in each organization who would provide information, depending on the objectives of the work. It is not only important to determine whether informants are managers and/or employees but also to determine the specific hierarchical levels, jobs, specific workers, or core members in specific teams. Another important aspect is how to manage and collect that information from individuals (managers and employees): through a visit to the organization where the researcher interviews all the informants; or the researcher only interviews a manager in each organization, and then this manager distributes questionnaires to other individuals in this organization; or even the use of a telephone or online survey.
Aggregation and collective constructs
For bottom-up relationships, an important theoretical and methodological aspect of multilevel research is the distinction between influences between variables at different levels and aggregation or emergence of collective constructs. We indicate some aspects of aggregation and collective constructs in this section. Aggregation refers to the emergence of a variable at a higher level as a consequence of the aggregation of characteristics or perceptions of some units at a lower level. Examples would be organizational culture and organizational climate, which emerge as aggregation of the perceived values of individuals on these aspects.
As will be indicated in the next section, this aggregation must be justified from a statistical point of view. However, first, a theoretical justification is needed. González-Romá (2019) points out that this theoretical explanation is frequently disregarded in research manuscripts but understanding the nature of higher-level constructs and the processes involved in their emergence from lower-level properties is theoretically crucial. In this regard, two basic aggregation or emergence principles can be identified (Chan, 1998; Hitt et al., 2007; Kozlowski & Klein, 2000; Mathieu & Chen, 2011): composition and compilation.
In the composition process, each unit at the lower level (e.g., each employee) contributes equally to the index that represents the variable at the higher level. Chan (1998) indicates different forms of composition models (e.g., direct consensus, referent-shift). Each model implies different theoretical assumptions about the nature of the higher-level construct (Paruchuri et al., 2018). In the composition process, descriptive statistics (such as the sum of individual scores or mean) adequately represents the processes in which the lower-level data (e.g., individual perceptions of several employees about the values of organizational culture) are associated with a higher-level collective construct (organizational culture). Another example is the use of the average of individual knowledge and skills of employees in an organization to represent organizational human capital.
In the compilation process, the higher-level phenomenon is a complex combination of the contributions of the lower-level units. Here measures of the units at the lower level (e.g., individuals) are combined in a complex way, with some individuals being able to contribute more than others to the higher-level variable. Therefore, simple descriptive statistics could not be used as in the composition process. An example of compilation process is the concept of climate uniformity (González-Romá & Hernández, 2014).
As noted above, composition and compilation processes are linked to collective constructs. These collective constructs are characteristics or properties of some group at some higher level that includes units at a lower level. For example, work teams, organizational departments, and firms are groups of individuals (employees and managers). There are three main types of collective constructs: global, shared, and configural constructs (Hofmann, 2004; Kozlowski & Klein, 2000).
Global constructs are descriptive characteristics of the group (e.g., the size of a firm considering the number of employees). The main characteristic is that these constructs are objective attributes that do not depend on individual perceptions and behavior. Therefore, there is not any aggregation or emergence process. A global construct does not cross levels, and it is assumed only to operate at the group level. However, these global constructs can influence the characteristics of members working in the group.
Unlike global constructs, shared and configural constructs cross levels as these two collective constructs have their origin at the individual level. Shared constructs only come into existence and have validity when members of the specific group share similar perceptions. An example would be the organizational climate. Homogeneity is considered in the sense that all individuals are equally important. These shared constructs are linked to composition processes of emergence, and a structural equivalence between levels is usually established (isomorphism).
Configural constructs are similar to shared constructs in the sense that they arise from individual-level attributes and perceptions. However, in configural constructs, the individual actions, characteristics, or perceptions combine in some complex and nonlinear way. There is not homogeneity as some individuals may contribute more than other individuals. Therefore, configural constructs are associated with compilation processes. For example, regarding efficacy and performance of teams (e.g., a football team or an orchestra), the actions and interactions of individuals may compile in a complex way, with each player or musician performing different interdependent roles to produce the overall group performance. In an orchestra, each musician can play their instrument very well, but this does not mean that the orchestra as a whole plays well.
An important aspect of collective constructs is their validity and reliability. Chen et al. (2004) provide recommendations regarding indices of internal consistency, agreement/consensus among members, and aggregate reliability for each specific type of construct. Another relevant issue is the wording of survey items used to measure group-level constructs (collective constructs) through individual-level data. Chan (1998) and Klein et al. (2001) provide important ideas and suggestions.
Analysis in multilevel research
In the “Introduction” section, we indicated that we use the term “multilevel research,” and not “multilevel analysis,” to offer a broader vision of this methodological approach, as examined above. However, multilevel analysis, through the use of multilevel statistical techniques, is an important part of multilevel research. Over the years, great advances have been made in multilevel statistical analysis. Progress has focused not only on statistical justification of aggregation of lower-level variables creating higher-level, collective variables, but also on multilevel statistical techniques to analyze relationships between variables at different levels (cross-level direct, cross-level moderation, and cross-level mediation effects). As noted below, HLMs and multilevel structural equation modeling (MSEM) are some of the main advances. From a quantitative and statistical point of view, we can distinguish two groups of techniques: justification of aggregation and analysis of influences among variables.
With regard to aggregation, once the process of aggregation has been theoretically justified, statistical aggregation must be also supported through the use of several indices and statistical coefficients. There are two main types of indices: indices used to estimate inter-rater agreement/consensus and indices used to estimate inter-rater reliability (LeBreton & Senter, 2008). For some composition models and collective constructs (e.g., shared constructs based on direct consensus models and referent-shift consensus models), to justify aggregating lower-level data to the emergence of a higher-level construct, it is necessary to demonstrate that the lower-level data are in agreement with one another (e.g., organizational climate). Within-group agreement and between-group variability are needed to justify data aggregation. Some indices to estimate inter-rater agreement are rWG indices and average deviation (AD) indices. It is also important to consider and estimate reliability. Reliability can be considered a measure of the consistency of responses among raters (Bliese, 2000), and it may be assessed through intraclass correlation coefficients. Bliese (2000) and LeBreton and Senter (2008) offer a conceptual and methodological description of some of these indices. Some empirical studies examine aggregation without analyzing influences between variables at different levels (cross-level effects) and therefore the statistical analyses are carried out at a single level (González-Romá & Hernández, 2016; von Bonsdorff et al., 2018). Other empirical studies use aggregation to create some higher-level construct and next implement a multilevel analysis of cross-level effects between variables (Zhu et al., 2018).
Regarding the analysis of influences between variables at different levels in multilevel models (cross-level effects), several techniques and models can be used. These models can be classified in two main groups: conventional multilevel modeling (such as HLMs) and more recent and advanced techniques (such as MSEM). González-Romá and Hernández (2017) study these multilevel models, examining their main characteristics, their performance, and how MSEM overcomes the limitations of conventional multilevel models. Next, we indicate some brief ideas about these two models. Conventional multilevel models were mainly developed for top-down relationships. For example, through HLMs (Hofmann, 1997), we can examine the impact of a lower-level variable (level 1) and a higher-level variable (level 2) on a dependent variable at the lower level (level 1). Pioneering areas that developed multilevel analysis were mainly interested in these top-down relationships. For example, in education, a key dependent variable is students’ performance (level 1) that is influenced not only by individual characteristics of students (level 1) but also by contextual variables, for example, characteristics of schools (level 2). In organizational behavior, important research questions examine which variables (individual and group variables) influence employees’ performance, productivity, satisfaction, and other characteristics of individuals. This top-down approach that characterizes conventional models is also based on the logic that context (higher-level variables) exerts greater influence on variables at lower levels than the influence of lower-level variables on context variables (Hitt et al., 2007). Some empirical works that have applied HLMs are Liao and Chuang (2004) and Jansen et al. (2012).
Conventional multilevel modeling has several limitations. One important limitation is that it cannot model bottom-up effects (effect of a lower-level variable on a higher-level variable). Moreover, there are problems with cross-level mediation models. In recent years, new advanced multilevel analysis techniques are being developed to examine upward influences and improve the analysis of cross-level mediation. This progress is being made through MSEM (Preacher et al., 2010, 2011). For example, 1-2-2 models can be analyzed (influence of a lower-level variable on a higher-level variable through a mediating variable at the higher level). Several empirical studies that use MSEM are Yao and Chang (2017), Beltrán-Martín et al. (2017), Beltrán-Martín and Bou-Llusar (2018), and Distel (2019).
To carry out multilevel analysis, researchers can use generic software that includes a multilevel research module (e.g., SPSS, MPlus, EQS) and specific multilevel software (e.g., MLwiN and HLMs).
Finally, it is important to note that in addition to quantitative/statistical analysis, multilevel research can also be conducted using qualitative methodologies. Some examples of qualitative studies are Salvato (2009) and Huy (2011). Qualitative methods can be useful and appropriate to study the specific mechanisms of emergence processes that help build higher-level concepts (e.g., human capital) from lower-level units (e.g., individual employees’ knowledge, skills, abilities, and other characteristics). Moreover, quantitative and qualitative methods can be combined and integrated in the same multilevel study, using a mixed methods approach (Bapuji et al., 2012).
Conclusion
In this
As examined in this article, multilevel research offers opportunities to advance the management field, but this methodological approach is not a panacea. Multilevel research should not be considered superior to other research approaches. Research questions determine the appropriate methodological approach, and research at a single level will continue to be conducted and will be relevant to many research questions. Nonetheless, an important aspect of the relationship between research questions and methods is that research questions influence the methods we use, but methods may also influence the research questions we ask. Research questions shape and are shaped by methods. By extending our methodological skills and our repertoire of methods, we can improve the question-asking process, increase the rigor of our conceptual thinking, see new ways to answer research questions, and even identify questions that would not have occurred to us otherwise (Edwards, 2008). And we consider that knowledge of multilevel research can provide opportunities for identifying relevant research questions and answering these questions with theoretical and methodological rigor. Multilevel research may facilitate the identification and analysis of questions and problems that are relevant to practitioners as this methodological approach considers several levels of analysis. Moreover, multilevel research may promote collaboration between researchers from different management areas, both macro (e.g., strategy, organizational theory) and micro areas (organizational behavior, human resource management), conducting multidisciplinary studies closer to real problems of organizations.
Together with these advantages and opportunities, researchers must also consider that multilevel research has important challenges. Multilevel studies are usually more complicated and complex (e.g., combination and integration of micro and macro theories) and require more effort and resources (e.g., data collection from multiple informants; research skills at the theoretical, methodological and analytical level) than studies at a single level. An important aspect for addressing these challenges is training. Some doctoral training of novice researchers does not promote a broad understanding of micro and macro aspects, and knowledge of several methodologies. An obstacle for conducting multilevel research is the specialization in theoretical approaches (either macro or micro) and in research methodologies focused on a single level.
We encourage researchers in management to consider the advantages and opportunities of multilevel research, assessing the possibilities of this methodological approach for studying their research topics. In this