Abstract
Introduction
On 13 February 2016, news broke that Supreme Court Associate Justice Antonin Scalia had passed away following an afternoon of quail hunting at the Cibolo Creek Ranch in Texas. On that very same day, even before flags nationwide could be lowered to half-staff, the Senate Majority Leader, Mitch McConnell of Kentucky, released a written statement announcing that the Senate would not act on any nomination by President Barack Obama. “This vacancy should not be filled until we have a new president,” the Republican senator declared.
McConnell’s resolve is being tested by Obama’s nomination of Merrick Garland, but McConnell’s threatened exercise of negative agenda power aims to preserve Republican Party cohesion. Nominating an extremely qualified and ideologically moderate nominee might drive a wedge between the conservative and moderate clusters of Republican senators. Senators facing difficult re-election battles might be tempted to defect from the party position if forced to vote, and so obstructing the confirmation process shields these senators from a controversial choice.
A strategic explanation for McConnell’s actions, such as the one in the last paragraph, depends on an implicit theory of how senators decide whether to support or oppose a Supreme Court nominee. This paper uses one such theory to forecast current senators’ votes on Merrick Garland’s nomination to the U.S. Supreme Court, in the unlikely case that a vote actually takes place. The forecasts are necessarily conditional, awaiting measurement of the nominee’s characteristics, particularly his ideological reputation and perceived qualifications. Nonetheless, a model that combines parameters estimated from existing data with values of some measurable characteristics of senators—particularly their party affiliations, party loyalty levels, and ideological “ideal points”—is sufficient to identify potential swing voters in the Senate.
Models of senators’ confirmation voting behavior
Empirical studies of Supreme Court confirmation votes have focused on explaining both confirmation outcomes (cf. Segal, 1987; Segal and Spaeth, 2002) and senators’ votes on nominees (cf. Cameron et al., 1990; Epstein et al., 2006; Segal et al., 1992). Let us consider two simplistic models of senators’ confirmation voting behavior and one more complicated model. We begin by describing each model, then show how it performs on explaining senators’ voting behavior between the administrations of Franklin D. Roosevelt and George W. Bush. Comparing the three models’ performance at predicting senators’ voting behavior for Barack Obama’s first two nominees, Sonia Sotomayor and Elena Kagan, reveals how the nominating process has become more partisan, so that models that pay insufficient attention to partisanship fit the current political environment poorly.
First, the “deferential” model would emphasize the unanimity that historically characterized the constitutional Advice and Consent process. From Hugo Black in 1937 to Thurgood Marshal in 1967, 22 consecutive nominees were confirmed by the Senate. 1 Senators cast 1906 yea votes in total, versus just 83 nay votes; the average nominee received 96% of senators’ support. Over the long run the deferential model was highly successful. From 1937 through 2007, a period covering 12 presidents and 41 nominations, the deferential model correctly predicted 87.2% of votes.
Second, the “partisan” model would emphasize whether an individual senator shares the president’s party affiliation. Over the 70-year span from 1937 to 2007, the partisan model would have been successful on fewer than 63.9% of senators’ votes.
Third, the “Segal” model has four explanatory variables: the president’s political strength, meaning that his party holds a majority of seats in the Senate
So far, we have described the relative accuracy of these three models in terms of in-sample validation, i.e. referring back to the votes that were used to generate the model’s parameters. A recent perspective holds that we should use out-of-sample validation 5 to judge models. King and Zeng state, “We must regard models that make causal inferences as also capable of forecasting. … Scholars would do well to judge all models in terms of their forecasting prowess, regardless of the purpose for which they were originally developed” (2001: 634). An example of out-of-sample validation would be assessing how well each model would fare at predicting senators’ votes on future nominees.
The deferential model and Segal model fared surprisingly poorly in forecasting senators’ votes during the confirmations of Sonia Sotomayor and Elena Kagan. Both nominees were portrayed in the media as highly qualified, and Obama was a “strong” president, as Democrats held the Senate’s majority. If one used all nominees from Black through Alito to estimate the parameters of the Segal model, and then used those parameters to generate forecasts of votes on Sotomayor and Kagan, the prediction would have been near unanimous support among Republicans. In reality, 31 of 99 senators opposed Sotomayor’s confirmation and 37 of 100 senators opposed Kagan’s confirmation. Forecasting strict party-line voting would have correctly predicted 92.5% of senators’ votes on Obama’s nominees, compared to just 65.5% for the deferential model and 67.8% for the Segal model. 6
When examining political phenomena that take place over long spans of time, models must account for dynamics: “The only way that forecasts can remain accurate far into the future is if the causal structure giving rise to the data remain stable” (King and Zeng, 2001: 634). A comparison of in-sample and out-of-sample predictions suggests that consideration of Supreme Court nominees has switched from a regime in which the president’s nominee is presumed worthy of confirmation unless his or her reputation or ideological position undermines that presumption, to a regime in which voting is largely based on party considerations. Our challenges are to explain the transition over time and to use that knowledge to predict behavior on current and future nominees.
The partisan dynamic
Two recent articles on Supreme Court confirmations recognized the increased role of partisanship on senators’ votes. Shipan (2008) noted that while senators in the president’s party have always been more likely to support nominees than senators in the opposing party, the gap widened from less than 10% between Presidents Kennedy and Ford, to nearly 20% when Reagan took office, and then continued to grow. Shipan used a counter variable to estimate the changing importance of party considerations, so his model can be extrapolated indefinitely, but ultimately his analysis does not identify a theory of precisely
Basinger and Mak (2012) argued that party cohesion provides both a theoretical link missing from Shipan’s approach and an empirical measure of how important partisan considerations are to senators. 7 Legislative parties’ cohesion and partisanship in the electorate tend to be in equilibrium: as parties’ brand names become more meaningful cues for voters, reinforcing those brand names becomes more imperative to legislators. As party cohesion rises, senators who belong to the president’s party should be more likely to rally around the president’s nominees, while senators who belong to the opposing party should be increasingly likely to oppose nominees. Thus, rising levels of average party cohesion explain the transition over time from a deferential regime to a partisan regime.
Basinger and Mak (2012) also theorized that variations within each party should be influenced by individual senators’ levels of party loyalty. Members of each party who are more loyal than average will be more likely to adhere to their party’s position, while “maverick” or independent-minded senators will be more likely to defect. The most loyal senators in the president’s party are most likely to support the president’s nominee, and the most loyal senators in the opposing party are least likely to support the nominee, all else being held equal.
Average party loyalty in a Congress and relative party loyalty (measured as the deviation from the average in that Congress) can be incorporated into empirical models as a multiplicative interaction. We include each term separately and in interaction with a dummy variable indicating whether the senator shares the president’s party affiliation or belongs to the opposition party.
To estimate the parameters of the party loyalty model, we used all nominations in the dataset, from Hugo Black through Elena Kagan, and utilized a probit regression model. The party loyalty model’s estimated coefficients and standard errors are shown in the second column of Table 1; for comparison purposes, we also provide the Segal model estimates in the first column of Table 1. A positive coefficient indicates that an increase in a variable’s value makes a yea vote more likely, while a negative coefficient indicates the opposing effect. Using either model, a senator was significantly more likely to vote for the nominee when the president was in a strong political position and as the nominee’s qualifications rose, and a senator was significantly less likely to vote for the nominee when the president belonged to the other political party and as the nominee was reputed to be further away in ideological space.
Competing models of Supreme Court confirmation votes.
AIC: Akaike Information Criterion; BIC: Bayesian Information Criterion.
Note: Cell entries are probit coefficient (standard error).
Every coefficient is statistically significant at
The party loyalty model adds two variables plus two interaction terms: average party loyalty 8 and relative party loyalty 9 are included by themselves and as interactions with the opposing-party variable. The positive coefficient for average party loyalty indicates that the likelihood of the same-party senators supporting the president’s nominee increases as the level of party unity rises. When we take into account the interaction term (average party unity × opposing party), the likelihood of opposition party members voting for the nominee decreases as the level of party unity rises. Thus, increasing cohesion leads to a widening gap between “typical” members of two parties. Similar calculations can be carried out for relative party loyalty: more loyal members of the president’s party are more likely to support nominees while more loyal members of the opposing party are less likely to support nominees, all else equal.
The party loyalty model out-performs the Segal model according to several measures of in-sample performance. The party loyalty model predicts more senators’ votes correctly, by a margin of nearly 93% versus 89%. Since the null model classifies 86.2% correctly, the party loyalty model eliminates more than twice as many erroneous classifications.
In the section titled “Models of senators’ confirmation voting behavior,” we performed an out-of-sample analysis of model fit to augment the in-sample analysis. We can duplicate that analysis by using just the votes for Black through Alito to estimate model parameters. The Segal model’s in-sample correct classification statistic is 90.2%, compared to 92.6% for the party loyalty model. When these model parameters were applied to predict votes on Sotomayor and Kagan, the Segal model’s out-of-sample correct classification statistic is 67.8% of votes, compared to 94.0% for the party loyalty model.
To substantiate the predictive superiority of a model which incorporates the changing importance of partisan considerations in Senate voting, we generated out-of-sample forecasts for every nominee in the dataset using both the Segal model and the party loyalty model. Figure 1 plots the percent of incorrect predictions for each model. The two track quite closely from Black through Breyer. After Breyer’s confirmation vote in August 1994 (by an 87–9 margin), there was an 11-year gap until Justice Roberts was confirmed in September 2005 (by a 78–22 margin). One can clearly see that the Segal model begins to deviate from the party loyalty model, with a far higher error rate for the former, for the four most recent nominees.

Comparison of out-of-sample forecast errors.
In summary, incorporating the average level of Senate parties’ cohesion and the relative loyalty of individual senators allows for more accurate predictions of senators’ votes in-sample as well as out-of-sample. In what follows, we will apply the party loyalty model to forecasting senators’ votes on the current nominee, Merrick Garland.
The known and the unknown
When generating forecasts, scholars rely on a few known elements, and then make educated guesses to fill in unknown elements. Senators’ party affiliations are known when the election is final, and rarely change mid-session; James Jeffords of Vermont and Arlen Spector of Pennsylvania provide notable recent exceptions. Because senators’ party affiliations are known and rarely change, aggregate party affiliations can be computed, allowing the president’s strength to be filled in; James Jeffords again provides the notable exception.
The characteristics of individual senators, other than their party affiliations, are also unknown but also can be estimated. Ideological positions can be estimated using the past voting behavior of individual senators (see Poole and Rosenthal, 1997, 2007). We follow Basinger and Mak (2012), who used the Common Space DW-NOMINATE scores, which are estimates of a single ideal point for each member of Congress for his or her entire record of service in Congress, and which are updated weekly. The model’s population parameters are unknown but can be estimated using existing data.
Historically, individual senators’ party loyalty levels were not calculated until the end of a Congress, but
Although Obama’s nominee is known, Merrick Garland’s salient characteristics have not yet been measured. Segal and Cover (1989) pioneered the practice of measuring nominees’ qualifications and ideological position using newspaper editorials. Faced with incomplete data, we can still make educated guesses and consider various alternative scenarios.
Conditional forecasts
Our forecast of senators’ votes are conditional statements, of the form, “Suppose the nominee has qualifications
As an illustration, consider two senators, the most conservative Democrat, Joe Manchin of West Virginia, and the most liberal Republican, Susan Collins of Maine. These senators’ Common Space DW-NOMINATE scores are –.07 and +.10, respectively, and their party loyalty levels are both 63%, far below the average in the first term of the 114th Congress.
Suppose Merrick Garland’s qualifications are perceived as being as high as those of John Roberts (
Perhaps it is unrealistic to anticipate that Obama would select an extremely liberal nominee when the opposing party controls the Senate. Moraski and Shipan (1999) theorized that the president and Senate would perceive that any new appointment to the Supreme Court would move the median justice; with eight sitting Justices, the feasible range of the new median is between the 4th nominee (presently Breyer, at
For a wider set of senators, Table 2 shows the names, Common Space scores, and party loyalty levels of the five most conservative Democrats and the 17 most liberal Republicans in the Senate. An asterisk after the name indicates that the senator is up for re-election in 2016, a condition which applies to 10 Republican senators shown in the Table. Table 2 then shows the predicted probabilities of supporting a hypothetical nominee with impeccable credentials and three different ideological positions: at President Obama’s ideal point, at Justice Kennedy’s ideal point, and at the median senator’s ideal point. Notice the absence of probabilities above 50% among Republican senators in any of the three conditions. Interestingly, our model’s predictions shown in the middle column directly contradict the expectation stated by Cameron and Kastellec: “Obama should nominate the best confirmable nominee — a Kennedy clone… — whom the Senate would approve.”
Selected senators’ ideological positions, party loyalty levels, and predicted votes.
denotes Republican senator whose seat is up for election in 2016.
Figure 2 amplifies the seeming hopelessness of the nomination by showing the predicted probabilities of all current Republican senators voting for clones of two sitting Justices – Sonia Sotomayor and Ruth Bader Ginsburg – if they, instead of Merrick Garland, had been nominated by President Obama during the year 2016. Senators are ordered by their ideological position, from liberal (left) to conservative (right). The graphs of predicted probabilities are not monotonically decreasing because the predictions also take into account relative party loyalty, which is only modestly correlated to ideological extremity. The figure also includes 95% confidence intervals around the predicted probabilities.

Republican senators’ predicted probabilities of voting for hypothetical nominees.
Sonia Sotomayor would not gain much Republican support in the 114th Congress; just nine Republican senators voted for her when she was confirmed in 2009, although that includes Susan Collins, Lamar Alexander, and Lindsey Graham. Perhaps more surprisingly, Ruth Bader Ginsburg would not gain much Republican support in the 114th Congress either, despite the fact that in 1993 she earned the support of 40 out of 43 Republicans, including Charles Grassley, Orren Hatch, John McCain, and Mitch McConnell. Figure 2 shows that a Ginsburg clone would be more likely to receive support from moderate Republican senators than a Sotomayor clone, due to her higher qualifications and less liberal reputation, yet the president’s partisan label is so important in the present regime that no Republican senator is predicted to vote for any Obama nominee. Indeed, if we extrapolated the analysis to imagine Obama hypothetically nominating a clone of Stephen Breyer, Anthony Kennedy, or even Chief Justice John Roberts, our model still predicts zero Republican votes in favor of the nominee during the 114th Congress. 12
Discussion and conclusions
With Antonin Scalia’s passing, President Obama was presented with an opportunity to name a third Associate Justice of the Supreme Court. The Segal model theorizes that ideological distance and qualifications are the primary determinants of senators’ votes on Supreme Court nominees, and so by naming a moderate and exceptionally well qualified nominee, Obama could attract sufficient Republican votes to guarantee confirmation. This expectation is naïve given the current politics. With the Senate’s present composition, a successful confirmation requires that there be zero Democratic defections, and at least four Republicans plus two Independents must support the nominee. With partisan cohesion in the Senate at unprecedentedly high levels, Obama faces little risk of losing any Democratic votes, but gaining Republican votes seems unlikely, no matter how much political capital 13 the President spends on the nominee.
By accounting for a more nuanced and refined understanding of the confirmation process, the party loyalty model reveals that any candidate that President Obama could select, whether liberal, conservative, or moderate, would be rejected if a vote was allowed to take place. So why nominate anyone at all?
Our model of senators’ political calculus incorporates the nominee’s qualifications and ideology, the president’s political strength, the Senate’s partisan environment, and each senator’s party loyalty and ideological position. What is missing from the political calculus is the senator’s beliefs about the election, including their own prospects for re-election, their party’s prospects of holding the Senate majority after the election, and their presidential candidate’s prospects for victory. So few nominations take place during the election year that it would be infeasible to incorporate electoral effects into the empirical model. For now, these factors must remain part of the disturbance term – i.e. factors that affect the probability of voting for the nominee that are unmeasured and/or idiosyncratic. 14 The results we computed in Table 2 and Figure 2 capture the systematic component of the data; Obama’s hope for a successful confirmation must come from the stochastic component, that is, from outside the traditional decision-making calculus. As the election approaches, un-modeled factors might affect Republican senators’ choices. Swing-state senators may sense potential electoral advantage from casting a vote against partisan gridlock; or, all senators might concede that the post-election environment will be worse than the status quo, if Hillary Clinton is the likely victor and if she is likely to have long coattails. The fact that 10 of the 17 Republican senators listed in Table 2 are up for re-election in 2016 might significantly reshape their voting calculus in a predictable yet unprecedented way.
