Sage Journals: Discover world-class research

Abstract

Backed by decades of empirical research, there has been increasing acknowledgment in policy, practice and research of the importance of neighborhood opportunity in shaping well-being. This has led to the proliferation of opportunity maps in cities throughout the United States with the purpose of identifying low opportunity neighborhoods in need of investment and intervention and high opportunity neighborhoods that can offer access to resources and amenities to disadvantaged population groups. By explicitly linking investment to the identification of neighborhoods that are high or low in opportunity, opportunity indices have the potential to help transform local and regional landscapes of spatial inequality. Despite this common goal, indices rely on varying theoretical conceptualizations, data, variables, and statistical approaches. How much these opportunity definitions overlap has yet to be fully examined. In this study, we compared five approaches to measuring neighborhood opportunity in California. We found low to moderate overlap across the indices, with disagreement higher for low opportunity designations. As with any quantitative analysis, opportunity mapping is not a purely technical exercise and requires a series of subjective decisions. The only way to validate these decisions is for opportunity measures to be constructed transparently and vetted by the research community. This study is a first step in this process.

Keywords

Neighborhood Opportunity Mapping Ecometrics Community Development Spatial

Introduction

Growing evidence from a wide set of disciplines has demonstrated that neighborhoods are associated with the health and well-being of their residents (Chetty et al., 2016; Diez Roux and Mair, 2010; Oakes et al., 2015). Because the neighborhoods of low-income, black and Hispanic populations tend to be considerably more disadvantaged than those of comparable higher-income and white populations, exposure to adverse neighborhood conditions helps explain socioeconomic and racial inequalities across a diverse set of outcomes, including teenage pregnancy, high school graduation, and life expectancy (Arcaya et al., 2016; Sharkey and Faber, 2014). As a consequence, policymakers and practitioners are coordinating efforts to implement cross-cutting interventions to increase spatial opportunity for disadvantaged population groups. The goal is to enhance the well-being of historically disadvantaged households by improving the places around them, whether through strategic neighborhood investments, residential mobility programs, or both (Brazil, 2016).

The increasing inclusion of neighborhood opportunity in policy intervention efforts calls for a need to quantify it. Several approaches to measuring neighborhood opportunity have emerged in recent years. The tradition in policy is to rely on standard measures of socioeconomic status such as the poverty rate or median household income (Brazil and Portier, 2022). In contrast, opportunity mapping consolidates multiple variables derived from publicly available data sources into single composite indices, where the upper and lower bounds of an index correspond to the highest and lowest opportunity levels in a region (Jennings, 2012; Knaap, 2017). Although sharing in this multivariate approach, applications of opportunity mapping vary widely across a number of factors, including their intended use, geographic scale, and the number and types of variables included in the model. These widely varying approaches to measuring opportunity coupled with the increasing recognition of a neighborhood’s role in shaping access to resources has led to the proliferation of publicly available data-driven web applications and equity atlases that plot and identify opportunity on a map (Finio et al., 2020). However, there have not been sufficient efforts to critically evaluate the construction of these indices and quantitatively examine whether and how much they overlap.

In order to fill this gap, we compare the following five neighborhood opportunity indices in California: CalEnviroScreen 3.0 (CES), Child Opportunity Index 2.0 (COI), Low Income Housing Tax Credit Opportunity Index (LIHTC), Opportunity Atlas, and the Regional Opportunity Index (ROI). These indices were chosen because of their use in either federal and state large-scale policies or smaller-scale programs administered at the local level by municipal governments and community-based organizations to identify low and high opportunity neighborhoods eligible for intervention. Our purpose is not to identify which of these indices is the best measure of opportunity. We are not adjudicating which index has the greatest theoretical consistency, statistical robustness, practical interpretability, or policy relevance, or which scores high on any metric that may be used to judge the value of a composite index (Spielman et al., 2020). Instead, we seek to accomplish a more descriptive objective: to examine how much overlap exists between various indices that purport to capture the same latent construct. A low correlation between opportunity indices suggests either that some neighborhoods are being inappropriately labeled as low or high opportunity, that opportunity is multidimensional and neighborhoods may be high in one domain but not in another, or both. In all of these cases, opportunity mapping as an off-the-shelf “one-size-fits-all” tool is inappropriate, and thus it is important to be clear in specifying the goal of the mapping application to ensure the identification strategy is targeted effectively and does not have unintended negative consequences.

Background

Opportunity, loosely defined, can be thought of as all of the pathways to better lives, including through health, education, and employment. Neighborhood opportunity is focused on how these pathways are shaped by the social, institutional and economic conditions within one’s residential neighborhood (Jencks and Mayer, 1990; Sharkey and Faber, 2014). That is, neighborhoods, as unique packages of institutions and resources, can exert influence on well-being above and beyond individual and household-level circumstances.

Neighborhood opportunity is an example of a “latent” variable, something inherent to a place but not directly observable. Viewing neighborhood opportunity as a latent variable implies that from a quantitative perspective, it can only be measured indirectly through statistical procedures. However, precisely what constitutes neighborhood opportunity is still up for debate, and thus there is little consensus regarding what constitutes or characterizes high and low opportunity neighborhoods (Lung-Amam et al., 2018). A typical approach to measuring neighborhood opportunity is to use a single socioeconomic variable such as median household income or the poverty rate (Brazil and Portier, 2022). A major drawback of the single variable approach is that it fails to capture other important dimensions of neighborhood opportunity, such as a neighborhood’s physical (e.g. proximity to greenspace), social (e.g. social cohesion and collective efficacy) and environmental (e.g. air pollution) conditions (Arcaya et al., 2016; Sampson et al., 2002; Galster, 2019).

The reliance on a single variable is partly due to the difficulty of synthetizing the wide set of results from the neighborhood effects literature in a way that is inclusive yet compact, and accurately reflects the spatial opportunity structure within an area (Knaap, 2017). Opportunity mapping, which was originally designed as a tool for facilitating community engagement and planning to address equity challenges, has emerged as one solution to this problem. The history of opportunity mapping and its methodological details have been covered (Knaap, 2017), but, in brief, the distinguishing feature of opportunity mapping is the combining of various social, economic and environmental variables into domain specific indices (Kirwan Institute, 2013). The subdomains are then combined to create an overall opportunity index whereby higher index values indicate higher opportunity. Beyond this feature, there exists no standardized approach or an agreed upon set of best practices (Knaap, 2017). This inconsistency may be due to differences in the theoretical, programmatic and methodological decisions underlying the construction of an index.

Variations are often driven by differences in the types of opportunity being measured. Although most include an index of general opportunity, often as an aggregation of opportunity subdimensions, the specific domains often differ. For example, California’s Low Income Housing Tax Credit (LIHTC) opportunity index, which is used to site affordable housing units in high opportunity areas, includes domains capturing economic, environmental, and educational opportunity, whereas the CalEnviroScreen (CES), which was developed by the California Environmental Protection Agency (CEPA) to direct funding towards disadvantaged neighborhoods experiencing environmental burdens, includes domains measuring pollution burden and sensitivity to environmental effects in addition to socioeconomic disadvantage. The Child Opportunity Index (COI) has three broad domains (Education, Health and Environment, and Social and Economic), with each domain containing several subdomains. The Regional Opportunity Index (ROI) separates opportunity into people and place dimensions, with each containing domains measuring opportunity in Education, Economic, Housing, Transportation, Health and Environment, and Civic Life. Even in cases when indices capture the same opportunity construct, they may differ in the number and types of characteristics included in the model. For example, while the LIHTC and CES indices include the same measures of educational attainment, employment and poverty in their socioeconomic subdomains, the LIHTC index also includes a measure of job proximity and median housing values, whereas the CES index includes a measure of linguistic isolation and housing burden. Some indices focus on specific population groups across age, racial/ethnic, and other demographic strata. For example, the COI includes a range of measures enumerating relative opportunity in domains that are related to healthy child development.

Indices also vary in their temporal conceptualizations of opportunity, viewing it as either a point-in-time or longitudinal construct. For example, the LIHTC index incorporates “rapid changes” in the neighborhood, defined as significant increases in median home value, poverty rate, percent with a bachelor’s degree, and employment rate. In contrast, the ROI, COI and CES do not incorporate change over time in their indices. The Opportunity Atlas also conceptualizes opportunity longitudinally by considering the economic outcomes in adulthood of children growing up in neighborhoods across the United States. It uses anonymized longitudinal income tax data covering nearly the entire U.S. population to track children’s outcomes into adulthood. For each neighborhood, they estimate children’s earning distributions, incarceration rates, and other outcomes in adulthood by parental income, race, and gender. Under this approach, the index may be capturing neighborhood conditions approximately 20 years ago, when today’s young adults were children, as opposed to current conditions.

Geographic factors may also explain differences. Indices typically measure opportunity relative to only neighborhoods located within an appropriate geographic region, such as the state, metropolitan area, or county, but comparison groups vary across indices based on the policy, programmatic and geographic scope of the index. For example, the COI offers its opportunity data normed at the metropolitan, state and national levels, allowing users to choose the geographic comparison most appropriate to their particular need. Most other indices do not provide multiple geographic comparisons and instead make comparisons in alignment with the index’s intended purpose. For example, as the LIHTC is used to site affordable housing across all regions of California, the index is based on regional comparisons so that high opportunity neighborhoods are distributed across all regions as opposed to being more concentrated in certain areas of the state. As a consequence of varying comparison groups, a neighborhood may be identified as high opportunity in one index but not in another because it is higher on the opportunity spectrum in comparison to neighborhoods in a region that is generally more disadvantaged relative to the rest of the state, but lower when compared to a region that is relatively more advantaged. Furthermore, some indices but not all separate comparison groups by urban and non-urban, and those that acknowledge the urban/non-urban difference often vary in how they treat rural areas. For example, the LIHTC shifted from census tracts to block groups for opportunity measurement in rural areas beginning in 2020, as rural tracts are often much larger than in suburban and urban areas and may mask variation across rural communities as a result. Other methodological factors that may cause variations include differences in how missing data are handled, data sources, the statistical approaches to combining multiple variables into a single domain, and multiple domains into a single overall index, and whether and how indices are binned into high/low opportunity categories.

Non-methodological factors such as the programmatic and policy initiatives motivating the construction of an index influence the statistical decisions described above. For some indices, their purpose is explicitly tied to federal or state policies and funding objectives. For example, the CES was developed to direct funding towards disadvantaged neighborhoods experiencing environmental burdens. Here, the focus is on identifying neighborhoods in the bottom 25% of opportunity. In contrast, the LIHTC index was created to identify neighborhoods at the opposite end of the opportunity spectrum—highest and high resource areas—in order to increase access to these areas for low-income families with children. Furthermore, the index has the underlying goal of providing a more balanced set of locational choices for families living in state-subsidized affordable rental housing across regions, which means that high opportunity in one region might be low in another. Some indices are presented as general resources for any stakeholder interested in capturing the opportunity structure within their community. This means they are free to conceptualize opportunity more broadly, without the constraints of policy, which leads to their use across a wider set of applications. For example, recent policies have adopted the Opportunity Atlas to expand affordable housing into higher opportunity areas, change zoning restrictions, and invest in community redevelopment (Bergman et al., 2020).

By either explicitly or indirectly linking funding, project development, and investment to the identification of neighborhoods that are high or low in opportunity, opportunity indices have the potential to help transform local and regional landscapes of spatial inequality. Despite this common goal, these indices rely on varying theoretical conceptualizations, data, variables, and statistical approaches. How much these opportunity definitions overlap has yet to be fully examined. The current study explores the importance of the strategy used to quantify neighborhood opportunity by comparing five opportunity indices for California neighborhoods.

Data

We collected census tract-level data for the versions of the opportunity indices examined in the study that were publicly available as of June 2020. Census tracts are small-area geographic regions defined by the U.S. Census Bureau for analyzing populations that approximate neighborhoods containing between 2500 and 8000 persons. As relatively permanent subdivisions of a county and designed to be homogeneous with respect to general population characteristics, census tracts are the most common proxy for neighborhoods in policy, practice and research. Census tracts missing data on any of the indices were excluded from the sample, yielding a final analytic sample of 7667 (out of 8057 total possible) California census tracts. We provide a description of each opportunity measure below. Additionally, Table 1 summarizes key features of the opportunity measures, and Table S1 in the Supplementary Material shows a summary of all indicators included by each index with variables organized under opportunity subdomains.

Table 1.

Summary of neighborhood opportunity measures.

Index	Measure	Domains	Purpose	Geographic scale of ranking	Temporal approach	Years of data (range)
CalEnviroScreen 3.0	Composite index	Environmental exposure, environmental effects, sensitive population, socioeconomic factors	SB 535 funding for disadvantaged communities	State	Cross-sectional	2005–2016
Child Opportunity Index	Composite index	Education, health and environment, social and economic	General opportunity index (all 50 states) to compare the level of opportunity that neighborhoods provide for children across the U.S. in a single metric	Metropolitan, state, and national (we use state)	Cross-sectional	2008–2019
LIHTC Opportunity Index	Composite index	Economic, environment, education, high segregation, and poverty	Siting of 9% new construction, large-family LIHTC developments in high opportunity neighborhoods	Region	Cross-sectional, but incorporates rapid changes in neighborhoods	2006–2018
Regional Opportunity Index—people	Composite index	Education, economic, housing, mobility, and transportation, health/environment and civic life	General CA specific opportunity index	State	Cross-sectional	2010–2014
Regional Opportunity Index—place	Composite index	Education, economic, housing, health/environment and civic life	General CA specific opportunity index	State	Cross-sectional	2009–2015
Opportunity Atlas	Adult mean household income rank for children whose parents were at the 25th percentile of the national income distribution	None	Trace the roots of outcomes such as poverty and incarceration back to the neighborhoods in which children grew up	National	Longitudinal	1989–2015

Note: We provide the full range of years covered by the variables included in each index; however, each individual variable has a unique date range depending on availability at the time the index was constructed.

CalEnviroScreen

Using data from federal and state sources, the CEPA developed the CES to identify communities disadvantaged by a cluster of social and environmental injustices to inform governmental agencies on where to allocate resources toward improving the environment and the health of the people (Truong, 2014). A full description of the CES and the other indices examined in this study are provided in the Supplementary Material. We downloaded CES 3.0 data from the California Office of Environmental Health Hazard Assessment website (OEHHA, 2017). The underlying data reflect multiple years, mostly the period 2010–2016. The index ranges from 0.98 to 94.09 with larger values reflecting greater disadvantage. In order to align with the other indices such that higher values signify greater opportunity, we inverted the CES score for our analysis.

Low Income Housing Tax Credit Opportunity Index

The LIHTC is the primary subsidy available for developing and preserving affordable rental housing in the United States. In 2015, the U.S. Supreme Court ruled that Texas allocated too many LIHTC credits in predominantly Black and poor urban neighborhoods, arguing that the program should instead increase the access of poor, minority households to higher resourced areas (Walter et al., 2018). Consequently, several states including California established new criteria to reward developers locating projects in high opportunity neighborhoods (Reid, 2019). Drawing from publicly available data sources, California created the LIHTC opportunity index to determine which neighborhoods were high in opportunity. We downloaded the 2019 version of the LIHTC opportunity index data from the California State Treasurer website (California Tax Credit Allocation Committee, 2020a). A more recent version of the LIHTC opportunity index provides more granular estimates for rural areas using block groups; however, we use the 2019 version for geographic consistency across opportunity measures. The index covers the years between 2010 and 2018 and ranges from −1.51 to 1.56 with higher values reflecting greater advantage.

Regional Opportunity Index

The ROI was created by the Center for Regional Change at the University of California, Davis (Benner et al., 2014). Unlike the CES and LIHTC indices, the ROI was not created for the specific purpose of identifying eligible neighborhoods for a state policy. Instead, it stands as a general opportunity index that has been used by practitioners and community-based organizations across the state to descriptively evaluate spatial opportunity within a given region. The ROI is not a single index, but rather is composed of two indices which assess the relative well-being of people and places at the neighborhood level. The People index is a relative measure of people’s assets (e.g. employment rate) whereas the Place index is a relative measure of an area’s assets (e.g. number of jobs in high-paying industries). We collected ROI data from the CRC website (Benner et al., 2014). The ROI relies on numerous data sources that span the years 2009–2014. The People and Place indices range from 15.79 to 90.42 and 17.14 to 58.67, respectively, with higher values reflecting greater advantage.

Child Opportunity Index

The COI is a general opportunity index developed by the Heller School for Social Policy and Management at Brandeis University. The COI measures available neighborhood resources and conditions that impact development over the life course and is intended to be used by researchers, practitioners, and community-based organizations across the U.S. to highlight inequities between neighborhoods and groups and inform equity-centered conversations and efforts (Acevedo-Garcia et al., 2014). We obtained the state-normed COI 2.0 data from the Diversity Data Kids.org website. Like the other indices included in this study, the COI’s individual indicators represent several data sources and multiple years (2010–2017). Index values range from 1 to 100 with higher values reflecting greater advantage.

Opportunity Atlas

The Opportunity Atlas measures upward socioeconomic mobility as the average household income rank in 2015 at age 30–35 for children who grew up in the 1980–1985 birth cohorts. For each tract, we have the mean household income rank for children whose parents were at the 25th percentile of the national income distribution. Incomes for children were measured as mean earnings in 2014–2015 when they were between the ages 31–37. Neighborhoods with children whose parents were at the 25^th percentile that ascended to a much higher rank as adults are considered to be high opportunity. We collected income mobility data from the Opportunity Atlas website (Chetty et al., 2020). The index ranges from 25.4 to 77.8 with higher values reflecting greater opportunity.

Methods

Our first set of analyses examined the correlations between the opportunity measures. We used Pearson correlations r to measure the degree to which values of an index track with values of another index. Rather than measuring opportunity on a continuous scale, programs typically group neighborhoods into low and high opportunity categories. The second set of analyses examined the overlap in these categories across the opportunity indices. We categorized neighborhoods by quintile whereby tracts located in the top 20% and bottom 20% of each measure’s distribution are identified as high and low opportunity, respectively. Because LIHTC scores are assigned within each California region, we constructed LIHTC quintiles by region. We present matrices showingthe proportion of high (low) opportunity neighborhoods on one index that are also high (low) on the other indices. In addition to analyzing overlap in quintiles across all indices, we also examined the overlap between indices that are binned according to policy defined categories. Rather than quintile breaks, California uses different thresholds to categorize opportunity for the CES and LIHTC. The state designates tracts in the top 25% of the CES as most disadvantaged but does not define highly advantaged areas. For its LIHTC program, California first identified tracts as high poverty or racially segregated. It then categorized the remaining tracts in the top and bottom 20% as the “Highest Resource” and “Low Resource.” The ROI, COI and Opportunity Atlas do not have specific statewide designations of high/low opportunity. We examine the overlap between the top and bottom 20% of these indices and the policy defined categories for the LIHTC and CES.

To measure the level of overall agreement across the opportunity measures, we calculated the average absolute deviation D. This measure calculates the absolute deviation of a neighborhood’s set of quintile scores (1 = bottom 20%; 2 = 20^th–40^th percentile; 3 = 40^th–60^th percentile; 4 = 60^th–80^th percentile; 5 = top 20%) from the neighborhood’s mean quintile score. D is defined by the following formula $D = \frac{\sum_{i = 1}^{N} \sum_{m}^{M} \frac{| q_{i}^{m} - \bar{q_{i}} |}{M}}{N}$ (1)where $q_{i}^{m}$ is the quintile score for neighborhood i and opportunity measure m, $\bar{q_{i}}$ is the average quintile score for neighborhood i, M is the total number of opportunity measures (6), and N is the total number of neighborhoods. If the quintile scores for all six indices are in agreement for every neighborhood in the state, D will equal 0. Therefore, the higher the value of D, the greater the disagreement.

Equation (1) measures overall disagreement across the indices, but does not identify which opportunity measures contribute more or less to the disagreement. To address this, we calculated average absolute deviation scores for each opportunity measure $D^{m}$ using the following formula $D^{m} = \frac{\sum_{i = 1}^{N} | q_{i}^{m} - {\bar{q_{i}}}^{(- m)} |}{N}$ (2)where ${\bar{q_{i}}}^{(- m)}$ is neighborhood i’s average quintile score excluding opportunity measure m.

The last set of analyses examined the sociodemographic predictors of high and low opportunity designations. Specifically, we ran logistic models of high (top quintile) and low (bottom quintile) opportunity, regressing an indicator of whether a neighborhood is high (low) opportunity or not on neighborhood sociodemographic characteristics. Here, we are interested in whether there are differences in the sociodemographic characteristics predicting higher opportunity across the indices. We focused on ethnoracial composition (% non-Hispanic Black, % non-Hispanic Asian, and % Hispanic), age composition (% below 18 and % above 65), educational attainment status (% of residents with a bachelor’s degree), median household income, and log population size. These characteristics are not meant to be completely inclusive, but instead capture the set of sociodemographic characteristics often associated with neighborhood racial/ethnic and socioeconomic stratification and inequality in the United States. We standardized the variables in order to compare their coefficients using the same scale. We also included county fixed effects to control for regional differences and an indicator of whether the tract is urban or not. A tract is defined as urban if either their centroid is located in or 50% of its area is within the first principal city listed in the title of the metropolitan area or a principal city with a total population greater than 100,000. All sociodemographic data were downloaded from the 2013–2017 American Community Survey (ACS).

Results

Correlational analysis

Figure 1 presents a correlation heatmap of the opportunity indices. Although all the pairwise correlations are statistically significant (p-value < 0.05) and positive, there is variation in their magnitude, ranging from a low of 0.28 to a high of 0.86, with most values indicating moderate correlation. The ROI Place index exhibits the lowest correlations, with no values greater than 0.5. CES and the Opportunity Atlas are moderately correlated with most indices, with values generally between 0.3 and 0.6. The ROI People and COI exhibit the strongest correlations, with no values lower than 0.45 and several correlations above 0.7.

Figure 1.

Correlation heatmap of opportunity measures.

One explanation for the range of correlations among the five indices may be the varying overlap in the types of variables they include or themes they are intended to capture (see Table S1). The LIHTC and CES are only moderately correlated (0.52) despite the LIHTC incorporating all of the CES’ environmental measures. There are 14 variables that are shared across the CES and LIHTC, most of which capture environmental conditions, representing 70% and 67% of the total number of variables included in each index, respectively. The lower degree of agreement between the two indices is partly due to the LIHTC’s inclusion of school quality and performance measures, which the CES does not include and are not highly correlated with environmental conditions. Other methodological decisions, such as the statistical method for combining the variables into a single index, may also contribute to the lack of agreement. Even in the case of COI and ROI People, which exhibit the highest correlation (0.86), the number of overlapping variables represents only 26.7% and 15.4% of the total number of variables included in each index, respectively. In some cases, indices contain variables in the same subdomain, but the variables may not necessarily be measuring the same conditions. For example, the LIHTC and COI contain several variables in the educational quality and performance subdomain, but while the LIHTC uses fourth grade proficiency, the COI measures third grade proficiency in addition to including variables that capture school quality outside of academic performance. The CES and ROI People contain three variables each in the Health outcomes subdomain, but the CES includes measures of hospitalization rates whereas the ROI People includes measures of life expectancy and teen births. The ROI Place stands out in that it shares the least overlap with the other indices in terms of both variables included and subdomains captured. Of the 15 variables included in the index, only five are shared by any other index, which represents the lowest share among the five indices (33.3%).

Overlap in opportunity categories

Rather than measured on a continuous scale, opportunity is typically binned in order to place neighborhoods in high and low opportunity categories. We next examine the overlap in opportunity designations based on quintile breaks. The first column in Table 2 presents the average absolute deviation score for all measures combined (D) and separated by opportunity measure (

D^{m}

). The average absolute deviation in quintile scores is 0.628. However, the deviation varies by opportunity measure. The COI and ROI People show the highest agreement with

D^{m}

values of 0.511 and 0.550, respectively. The ROI Place and Opportunity Atlas exhibit the lowest agreement (or highest disagreement) with

D^{m}

values of 1.035 and 0.828, respectively. That is, a neighborhood’s quintile ranking in the ROI Place and Opportunity Atlas may be on average a full quintile above or below the rankings designated by the other indices. These results corroborate the high correlations for the COI and ROI People and the low correlations for the ROI Place and Opportunity Atlas presented in the previous section.

Table 2.

Average absolute deviation of quintile scores.

	Total (N = 7667)	Highest (N = 3614)	Lowest N = 3884)
Overall	0.628	0.685	1.027
CES	0.736	0.748	0.803
COI	0.511	0.526	0.531
LIHTC	0.778	0.799	0.830
Opportunity Atlas	0.828	0.944	0.895
ROI people	0.550	0.593	0.569
ROI place	1.035	1.217	1.557

CES: CalEnviroScreen; COI: Child Opportunity Index; LIHTC: Low Income Housing Tax Credit; ROI: Regional Opportunity Index.

Many interventions are focused on identifying just the highest and lowest opportunity neighborhoods. We next examined whether $D^{m}$ differs by neighborhoods defined as high (top quintile) and low (bottom quintile) opportunity. That is, what is the level of agreement across the indices if at least one index identifies the neighborhood as high (low) opportunity? Of the 3614 neighborhoods that had at least one index identifying them as high opportunity, 4.5% had all measures in agreement (i.e. all measures designated the neighborhood as high opportunity). The percentage is smaller (2.0%) for the 3884 neighborhoods that had at least one index identifying them as low opportunity. Columns 2 and 3 in Table 2 show the average absolute deviation in neighborhood quintile scores conditioned on the neighborhood being identified as high or low opportunity by at least one opportunity measure. The quintile scores for high opportunity neighborhoods differ by 0.685, which is only slightly higher than the total D. The $D^{m}$ high opportunity values show similar results as those for the total, with greater agreement for the COI (0.526) and ROI People (0.593) and the greatest disagreement for the ROI Place (1.217) and Opportunity Atlas (0.944). However, the ROI Place and Opportunity Atlas $D^{m}$ values are higher in high opportunity neighborhoods compared to all neighborhoods, whereas the other measures, in particular the ROI People and COI, show similar values.

For neighborhoods identified as low opportunity by at least one index, their quintile scores differed by 1.027, which is a near half quintile higher than the disagreement for high opportunity neighborhoods. We also found that the level of disagreement increased for the ROI Place. Here, if a neighborhood is defined as low opportunity by the ROI Place index, other indices may rank the neighborhood on average 1.6 quintiles higher. To a lesser degree, there was also an increase in $D^{m}$ for the LIHTC and CES. In contrast, the COI, Opportunity Atlas, and ROI People indices exhibit similar levels of disagreement across total, high and low opportunity designations.

To identify which pairs of indices disagree in their designations of high and low opportunity neighborhoods, the matrices shown in Tables 3 and 4 present the percentage overlap in low (bottom 20%) and high (top 20%) opportunity neighborhoods between each opportunity measure. The cell values of each matrix are dependent on the row index. For example, the row 1 and column 2 cell value in Table 3 indicates that 59% of CES low opportunity neighborhoods were identified also as low opportunity by the COI. The greatest overlap occurs between the ROI People and COI. Of the neighborhoods identified as low opportunity by the COI, 71% are also identified as low opportunity by the ROI People. The next highest overlap is between the ROI People and CES, with a percentage overlap nearly 10 percentage points lower than the overlap between the ROI People and COI.

Table 3.

Overlap in low opportunity neighborhoods (bottom 20%) between opportunity measures.

	CES	COI	LIHTC	Opportunity Atlas	ROI people	ROI place	Row mean, %
CES		59%	47%	47%	63%	35%	50
COI	58%		55%	55%	71%	39%	56
LIHTC	47%	56%		46%	54%	32%	47
Opportunity Atlas	46%	56%	46%		53%	32%	47
ROI people	63%	72%	54%	53%		37%	56
ROI place	35%	40%	32%	32%	37%		35

CES: CalEnviroScreen; COI: Child Opportunity Index; LIHTC: Low Income Housing Tax Credit; ROI: Regional Opportunity Index.

Table 4.

Overlap in high opportunity neighborhoods (top 20%) between opportunity measures.

	CES	COI	LIHTC	Opportunity Atlas	ROI people	ROI place	Row mean, %
CES		63%	55%	46%	66%	46%	55
COI	66%		61%	57%	74%	50%	62
LIHTC	56%	58%		48%	61%	44%	53
Opportunity Atlas	46%	55%	48%		53%	36%	47
ROI people	66%	70%	61%	53%		47%	59
ROI place	46%	48%	44%	36%	47%		44

CES: CalEnviroScreen; COI: Child Opportunity Index; LIHTC: Low Income Housing Tax Credit; ROI: Regional Opportunity Index.

We find similar patterns in the overlap between high opportunity designations. The ROI People and COI show the greatest overlap with the other indices (mean overlap values of 59% and 62%, respectively) whereas the ROI Place shows the lowest overlap (44%). However, the overlap in high opportunity neighborhood designations is generally greater than in low opportunity designations. The mean overlap is higher for high opportunity than low opportunity across all indices except for the Opportunity Atlas, which exhibited the same overlap in high and low opportunity neighborhoods (47%). For example, 46% of high opportunity ROI Place neighborhoods are also identified as high opportunity by CES, which is 11 percentage points higher than the overlap in low opportunity neighborhoods between these two measures. We emphasize that although high opportunity neighborhoods exhibit greater overlap, the mean overlap is less than two-thirds across all indices, with many of the pairwise percentages exhibiting approximately or less than 50% overlap.

The state programs that rely on the LIHTC and CES indices designate high and low opportunity neighborhoods using different demarcations than quintiles. Table 5 shows the cross-tabulation between the official LIHTC and CES categories, and the high and low opportunity categories based on quintile breaks for the other four measures. Similar to the results from the correlation and overlap analyses, ROI People and COI exhibit the greatest consistency in opportunity designations whereas the Opportunity Atlas and ROI Place show the least consistency. While more than 60% of high opportunity ROI People and COI neighborhoods are designated Highest Resource according to the LIHTC, the overlap is less than 50% for the Opportunity Atlas and the ROI Place. Similarly, whereas 67.8% and 72.0% of COI and ROI People low opportunity neighborhoods, respectively, are designated as disadvantaged by the CES, 53.4% and 40.8% of the low opportunity neighborhoods according to the Opportunity Atlas and the ROI Place measures are CES disadvantaged.

Table 5.

Overlap in LIHTC and CES neighborhood opportunity categories and COI, Opportunity Atlas, and ROI People and Place high and low opportunity designations based on quintiles.

	COI		Opportunity atlas		ROI people		ROI place
	Highest, %	Lowest, %	Highest, %	Lowest, %	Highest, %	Lowest, %	Highest, %	Lowest, %
LIHTC^a
Highest resource	61.7	0.4	49.1	4.7	61.6	0.5	45.0	12.0
High resource	28.6	3.1	28.6	8.6	26.7	2.5	28.4	15.0
Moderate resource	9.4	11.2	17.6	15.1	10.6	12.0	18.0	20.7
Low resource	0.3	38.6	4.0	37.2	1.0	37.7	7.7	29.7
High segregation and poverty	0.0	46.8	0.7	34.4	0.0	47.3	0.9	22.6
Total	100	100	100	100	100	100	100	100
CalEnviroScreen^b
Disadvantaged	0.1	67.8	2.5	53.4	0.1	72.0	3.1	40.8

COI: Child Opportunity Index; LIHTC: Low Income Housing Tax Credit; ROI: Regional Opportunity Index.

^aCalifornia identifies tracts as high poverty, racially segregated. It then categorizes the remaining tracts in the top as “Highest resource,” the next 20% as “High resource” and the remaining equally divided as “Moderate resource” and “Low resource.”

^bDesignates tracts in the top 25% of the CES as most disadvantaged.

The results in this section indicate that there is inconsistency in the labeling of high and low opportunity neighborhoods across the measures. What does this inconsistency look like geographically? Figure 2 provides an example of a city exhibiting strong disagreement of where high and low opportunity neighborhoods are located. The maps show high and low opportunity neighborhoods in Sacramento, CA, which has average absolute deviation scores of 0.750, 0.861 and 1.202 for all, high and low opportunity neighborhoods, respectively. Most of the maps exhibit broad patterns of where high and low opportunity neighborhoods are located. Low opportunity neighborhoods are generally located in the southern and northeastern portions of the city, and high opportunity neighborhoods are generally located in the central and northwestern portions of the city. However, the specific neighborhoods and the scale differ from measure to measure. For example, the LIHTC categorizes contiguous geographic clusters of low opportunity neighborhoods in the south and northeast, whereas the other measures identify some but not all the same neighborhoods as low opportunity. The LIHTC also identifies a few high opportunity neighborhoods in the northwest corner and a few in the central city whereas the ROI People identifies no high opportunity neighborhoods in the northwest and more high opportunity areas east of downtown and south of the city. The most extreme divergence is the ROI Place, which designates many of the low opportunity neighborhoods defined by the other measures as high.

Figure 2.

High and low opportunity neighborhoods by index, Sacramento, CA.

The sociodemographic characteristics of high and low opportunity neighborhoods

The final set of analyses compares the sociodemographic characteristics associated with high and low opportunity designations across the indices. Figures 3 and 4 present results from logistic regression models predicting high and low opportunity, respectively. For interpretability, we converted all coefficients into odds ratios with their 95% confidence intervals. The interpretation of the odds ratio is a value greater than 1 (dashed line) represents a positive association between a sociodemographic variable and the probability of a neighborhood being designated as high opportunity. Full results are provided in Tables S2 and S3 in the Supplementary Material.

Figure 3.

Odds ratios and 95% confidence intervals for models predicting high opportunity.

Figure 4.

Odds ratios and 95% confidence intervals for models predicting low opportunity.

A few general patterns emerge from the results. First, log median household income, percent black, percent Hispanic and percent of residents with a college degree generally show consistent results across the opportunity measures. With few exceptions, greater household income and percent with a college degree, and lower percent black and percent Hispanic are associated with lower odds of being designated low opportunity and a greater odds of being designated high opportunity. The other variables included in the model do not exhibit consistent patterns.

Second, we find that the odds ratios in the high opportunity models generally exhibit consistency across the indices in their size and direction, with narrow confidence intervals indicating high precision. In general, neighborhoods with lower percent non-Hispanic black, percent Hispanic, and percent non-Hispanic Asian, and higher income, percent of residents with a bachelor’s degree, proportion of residents who are less than 18 years old, and proportion of residents who are greater than 65 years old have a higher odds of being designated high opportunity. Urbanicity and log population size show some inconsistency in direction; however, odds ratios are near one for all indices.

Third, we find there is less agreement in the direction and size of the odds ratios across the indices in the low opportunity models. For example, while percent 18 years old and under is positively associated with high opportunity for all indices, it is positively associated with low opportunity for the COI, ROI Place, and Opportunity Atlas, negatively associated for the CES, and has no association with the ROI People and LIHTC. Percent 65 years and above also exhibits similar inconsistencies. Furthermore, the magnitude of the low opportunity odds ratios varies widely, and their confidence intervals are generally wider compared to the high opportunity odds ratios. This is particularly pronounced for percent Hispanic. Although percent Hispanic in all cases except the COI is positively associated with low opportunity, the odds ratios are well above 3 for the CES and ROI people, between 2 and 3 for the Opportunity Atlas, and below 2 for the ROI Place and LIHTC. In the case of ROI People, the 95% confidence intervals suggest an odds ratio as high as 4.6 and as low as 2.6. In comparison, the 95% confidence intervals in the high opportunity models suggest an odds ratio as high as 1.01 and as low as 0.98.

Fourth, consistent with the results from the prior sections, the ROI Place and Opportunity Atlas stand out from the rest of the indices. In comparison to the other indices, their odds ratios are either generally larger or smaller, and show no or the opposite association for many variables. For example, log household income is associated with opportunity across all the indices, but is not associated with ROI Place in either low or high opportunity models. Log population size, urbanicity and percent of residents above 65 years old in low opportunity neighborhoods also show opposite associations with the ROI Place compared to the other indices. The Opportunity Atlas shows no or an opposite association for percent non-Hispanic Asian in high opportunity and log population size and percent with a bachelor’s degree in low opportunity. The index has the smallest odds ratios for percent with a bachelor’s degree, percent black, and percent Hispanic and the highest odds ratio for percent non-Hispanic Asian in the high opportunity models. For low opportunity, the index has the smallest odds ratio for percent non-Hispanic Asian and the highest odds ratios for percent with a bachelor’s degree and percent non-Hispanic black.

Discussion

In this study, we compared five approaches to measuring and mapping neighborhood opportunity in California. We found low to moderate overlap across the indices, with correlations generally falling between the range of 0.4 and 0.6. However, the level of disagreement varied across the five measures. On one end of the spectrum, the COI and ROI People indices show relatively strong agreement with the other measures. The correlations were generally above 0.5, with some well above 0.7. Both measures also showed agreement for both low and high opportunity designations, and generally agreed with the LIHTC and CES program categories of high and low opportunity. A potential explanation for this greater overlap is that both measures incorporate more variables that cut across multiple domains, and thus are more inclusive in how they measure opportunity.

On the other end of the spectrum, the ROI Place and Opportunity Atlas show the greatest levels of disagreement with the other indices across all the metrics examined in this study. In the case of the ROI Place index, it incorporates measures of opportunity that focus less on the resident characteristics of a neighborhood (e.g. household income) and more on the characteristics of the built, institutional, environmental, and civic environments. As such, the ROI Place is unique in that it exclusively focuses on measuring opportunity based on access to resources and amenities from the supply side (e.g. proximity to good schools, banks, health care) rather than the demand side (e.g. income, educational attainment, English proficiency). The disagreement suggests that by mixing the two types together, indices that combine place-based with people-based measures may be counterbalancing the negative associations of one type with the positive associations of the other. Disagreement with the Opportunity Atlas may be due to its focused approach to measuring opportunity. The measure captures how adult outcomes are shaped by childhood neighborhood environments. Therefore, it is likely capturing neighborhood opportunity during childhood, which was more than 20 years ago for the data they used in their current measure.

When examining the overlap in the extreme ends of the distribution, disagreement was much higher for low opportunity neighborhoods. Furthermore, whereas we found general agreement in the demographic and socioeconomic characteristics that predict high opportunity across all measures, we found disagreement in the characteristics that predict low opportunity. These results align with prior work characterizing high opportunity neighborhoods as high income, highly educated, and low racial minority, with lower agreement on the characteristics of low opportunity areas (Dawkins, 2017).

A few limitations of our study warrant comment. First, our findings are specific to California. The spatial opportunity structure in other areas of the country likely differs. We focused on California because of the prevalence of California specific opportunity maps and the state’s reliance on these maps to directly link public funding to expanding neighborhood opportunity. Future work should consider comparisons of opportunity measures in other parts of the country. Second, we did not examine differences between urban and non-urban neighborhoods. The degree to which opportunity measures overlap may vary across the urban-rural spectrum. Third, we focused on five opportunity measures, but others exist, and continue to be developed, even within California. Most of these measures are city, region or state specific (Jennings, 2012; Walter et al., 2018), and thus were not included in this study because of its California statewide focus. Future work examining other opportunity measures, and more generally, comparing the statistical methods to measuring opportunity, is needed.

Despite these limitations, the study offers several implications for policy, practice and research. A methodological implication is that a one-size-fits-all approach to measuring opportunity should be reconsidered. Many of the indices examined in this study are often framed as capturing the general neighborhood opportunity structure within a city or region. Some of these indices are more transparent in their narrow conceptualization of opportunity, but they are still often presented as a general measure in their public-facing mapping applications. Nearly all of these measures have sub-indices that capture various opportunity domains, but the overall index is the primary measure used in policy and practice. Furthermore, agencies often discard the continuous measure of opportunity by designating neighborhoods with the highest and lowest values as high and low opportunity. We do not suggest that identifying the intersection of all methods will lead a practitioner to the “true” high or low opportunity neighborhoods. Instead, opportunity measures should be dissected based on the type of opportunity being measured and the outcomes predicted to be associated with those opportunity domains. The variables included in or excluded from the different methods relate at the most fundamental level to the creator’s decisions about the most salient causes and indicators of spatial opportunity.

A policy implication is that given the lower level of agreement between the measures in their designation of low opportunity neighborhoods, disadvantaged neighborhoods targeted for investment may not be the most in need and neighborhoods in greater need of intervention may be ignored. Moreover, programs targeting low opportunity neighborhoods should align the type of opportunity being measured with the types of projects that are intended to be developed in the neighborhood. Prior qualitative work has demonstrated that without careful forethought about their intended audience and use, opportunity measures are often less effective than expected (Finio et al., 2020).

A social implication is that a neighborhood mislabeled as low opportunity may carry a stigma that can lead to negative outcomes for the neighborhood and its residents. The consequences of neighborhood stigma arise when negative perceptions of a place are attached to the individuals who live there, leading to suspicion and mistrust in interactions with strangers when the neighborhood of residence is revealed, and systematic disapproval, discrimination, and/or exclusion (Anderson, 2013; Besbris et al., 2015; Link and Phelan, 2001). These negative consequences may mitigate the benefits that a neighborhood receives from an intervention. Furthermore, because we find general disagreement in the designation of low opportunity, some neighborhoods will be labeled as low opportunity under one set of criteria when they have overlooked assets that actually make them higher in opportunity using another set of criteria. This is particularly problematic for low income, communities of color, which tend to be painted by opportunity maps through a deficit-based lens (Lung-Amam, et al., 2018). More work is needed to either construct indices that can better capture the opportunity structure within a region, emphasize the multidimensional nature of opportunity by creating appropriate subdomains and favoring them over an overall index, or move away from the binning of neighborhoods into low and high opportunity categories and towards a continuous and non-linear representation of opportunity.

Supplemental Material

Supplemental Material - Measuring and mapping neighborhood opportunity: A comparison of opportunity indices in California

Supplemental Material for Measuring and mapping neighborhood opportunity: A comparison of opportunity indices in California by Noli Brazil, Jenny Wagner, Raziel Ramil in Environment and Planning B: Urban Analytics and City Science

Footnotes

Acknowledgments

We are grateful to Corey Rodda and Grace Kumetat for their excellent research assistance,and for feedback provided by participants of the Population Association of America conference.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research,authorship,and/or publication of this article.

Funding

The author(s) received no financial support for the research,authorship,and/or publication of this article.

ORCID iDs

Noli Brazil

Jenny Wagner

Supplemental Material

Supplemental material for this article is available online.

Author biograpies

Noli Brazil is an Assistant Professor in the Department of Human Ecology at the University of California,Davis. His research interests include understanding the causes and consequences of neighborhood inequality,residential mobility,and spatial demography.

Jenny Wagner,MPH is a doctoral student in the Department of Public Health Sciences at the University of California,Davis. She studies the mechanisms by which structural inequities produce health disparities and is particularly interested the ways residential contexts shape health over the life course.

Raziel Ramil is a Program Coordinator and Executive Assistant at the California Center for Civic Participation. She earned a Bachelor's Degree in Community and Regional Development at the University of California,Davis.

References

Acevedo-Garcia

McArdle

Hardy

, et al. (2014) The child opportunity index: improving collaboration between community development and public health. Health affairs 33(11): 1948–1957.

Anderson

(2013) Streetwise: Race, Class, and Change in an Urban Community. Chicago: University of Chicago Press.

Arcaya

Tucker-Seeley

Kim

, et al. (2016) Research on neighborhood effects on health in the United States: a systematic review of study characteristics. Social Science & Medicine 168: 16–29.

Benner

Erbstein

London

, et al. (2014) Regional Opportunity Index (ROI). UC Davis Center for Regional Change. Available at: https://interact.regionalchange.ucdavis.edu/roi/ (accessed 20 September 2020).

Bergman

Chetty

DeLuca

, et al. (2020) Creating Moves to Opportunity: Experimental Evidence on Barriers to Neighborhood Choice (No. W26164). National Bureau of Economic Research.

Besbris

Faber

Rich

, et al. (2015) Effect of neighborhood stigma on economic transactions. Proceedings of the National Academy of Sciences 112(16): 4994–4998.

Brazil

(2016) Using Place-and Person-Based Interventions to Measure Neighborhood Effects. In Recapturing Space: New Middle-Range Theory in Spatial Demography (pp. 57–75). Springer, Cham.

Brazil

Portier

(2022). Investing in gentrification: The eligibility of gentrifying neighborhoods for federal place-based economic investment in US cities. Urban Affairs Review, 58(5), 1234–1276.

California Tax Credit Allocation Committee (2020a) TCAC/HCD Opportunity Area Maps. Available at: https://www.treasurer.ca.gov/ctcac/opportunity.asp (accessed 20 September 2020).

10.

Chetty

Hendren

Katz

(2016) The effects of exposure to better neighborhoods on children: New evidence from the Moving to Opportunity experiment. American Economic Review 106(4): 855–902.

11.

Chetty

Friedman

Hendren

, et al. (2020) The Opportunity Atlas: Mapping the Childhood Roots of Social Mobility (No. W25147). National Bureau of Economic Research.

12.

Dawkins

(2017) Putting equality in place: The normative foundations of geographic equality of opportunity. Housing Policy Debate 27(6): 897–912.

13.

Diez Roux

Mair

(2010) Neighborhoods and health. Annals of the New York Academic of Sciences 1186(1): 125–145.

14.

Finio

Lung-Amam

Knaap

, et al. (2020) Equity, Opportunity, Community Engagement, and the Regional Planning Process: Data and Mapping in Five US Metropolitan Areas. Journal of Planning Education and Research 1-12. Epub ahead of print 29 July 2020. DOI: 10.1177/0739456X20945385.

15.

Galster

(2019) Making Our Neighborhoods, Making Our Selves. Chicago: University of Chicago Press.

16.

Jencks

Mayer

(1990) The social consequences of growing up in a poor neighborhood. In: Lynn

McGeary

MFH

(eds) Inner-city poverty in the United States. Washington DC: National Academic Press, pp. 111–186.

17.

Jennings

(2012) Measuring neighborhood distress: a tool for place-based urban revitalization strategies. Community Development 43(4): 464–475.

18.

Kirwan Institute for the Study of Race and Ethnicity (2013) Opportunity Metrics Resource of Data and Research. Report, Columbus, OH. Available at: http://kirwaninstitute.osu.edu/docs/FINAL{\_}2013OppMetrics.pdf

19.

Knaap

(2017) The cartography of opportunity: Spatial data science for equitable urban policy. Housing Policy Debate 27(6): 913–940.

20.

Link

Phelan

(2001) Conceptualizing stigma. Annual Review of Sociology 27(1): 363–385.

21.

Lung–Amam

Knaap

Dawkins

, et al. (2018) Opportunity for Whom? The Diverse Definitions of Neighborhood Opportunity in Baltimore. City & Community 17(3): 636–657.

22.

California Environmental Protection Agency Office of Environmental Health Hazard Assessment (OEHHA) (2017) CalEnviroScreen Version 3.0. Available at: https://oehha.ca.gov/calenviroscreen/report/calenviroscreen-30 (accessed 20 September 2020).

23.

Oakes

Andrade

Biyoow

, et al. (2015) Twenty years of neighborhood effect research: an assessment. Current epidemiology reports 2(1): 80–87.

24.

Reid

(2019) Rethinking “opportunity” in the siting of affordable housing in California: Resident perspectives on the low-income housing tax credit. Housing Policy Debate 29(4): 645–669.

25.

Sampson

Morenoff

Gannon-Rowley

(2002) Assessing “neighborhood effects”: Social processes and new directions in research. Annual Review of Sociology 28(1): 443–478.

26.

Sharkey

Faber

(2014) Where, when, why, and for whom do residential contexts matter? Moving away from the dichotomous understanding of neighborhood effects. Annual Review of Sociology 40: 559–579.

27.

Spielman

Tuccillo

Folch

, et al. (2020) Evaluating social vulnerability indicators: criteria and their application to the Social Vulnerability Index. Natural Hazards 100(1): 417–436.

28.

Truong

(2014) Addressing poverty and pollution: California's SB 535 greenhouse gas reduction fund. Harv. CR-CLL Rev 49: 493.

29.

Walter

Wang

Jones

(2018) Comparing opportunity metrics and locational outcomes in the Low-Income Housing Tax Credit program. Journal of Planning Education and Research 38(4): 449–462.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.28 MB

0.00 MB