Sage Journals: Discover world-class research

Abstract

Pre-election opinion polls are an increasingly prominent aspect of political campaigns, yet they often vary in terms of their results, sources, and where they are published. Citizens are therefore increasingly confronted with the proposition of which polls to give more credence to than others in shaping their voting behaviour. This study investigates the relationship between subjective determinations of poll credibility and voting behaviour in the polarised context of Turkish mayoral elections. The theoretical perspective of motivated reasoning is employed to consider how individuals determine and act upon credible opinion polls in mixed information environments. Using an original two-step experimental approach conducted in 2020, this paper establishes that while the effects of opinion poll credibility on party choice are limited to the strategic considerations of the supporters of smaller parties, opinion polls can have considerable demobilising effects when polling environments are conflicting or deemed not credible. The findings produced in this study are more suggestive of accuracy-seeking than directional motivations, and they have considerable implications for how we think about the relationship between polls, politicians, and voters.

Keywords

opinion polls credibility partisanship motivated reasoning voting behaviour conjoint analysis survey experiment

Introduction

Opinion polls are abundant in the run-up to elections, yet often do not share a unified voice. Pre-election polls can – and often do – vary considerably in terms of their estimations, the vendors who conduct them, and where the results are published. This variation is noteworthy given how polls can affect the behaviour of political actors. For officials and parties, polls contribute significantly to political decision-making procedures, and serve as cues for the development of campaign strategies (Mutz 1995; Lau and Redlawsk 2006). For voters, however, while much research has considered the potential consequences of singular or coherent polling figures, it is less apparent how voters make sense of mixed-poll environments to inform their vote choice. As polls and election projections become an ever-greater part of pre-election discourse, the public is confronted with a growing range of polling information in increasingly polarised media environments, forcing them to consider which information to give more credence to.

Politicians are concerning themselves with the potential impact of opinion polls with increasing regularity. In the aftermath of the declaration of Wisconsin for Joe Biden in the 2020 US presidential election, Donald Trump's campaign manager, Bill Stepien, released a statement claiming that “ridiculous” public polling had been used as a voter suppression tactic, seemingly suggesting that polls indicating a substantial lead for Biden in the run-up to the election had disincentivised would-be Trump-voters from turning out. Similar accusations were made during the Venezuelan election of 2012, in which pollsters were attacked from both sides for publishing what were contended to be erroneous and misleading figures (Reuters 2012). There is evidence to suggest that polling numbers have been manipulated to reflect favourably on the perpetrator, as evidenced by Recep Tayyip Erdogan's attempts ahead of settlement talks with a pro-Kurdish party prior to their publication in a newspaper in 2013 (Nordic Monitor 2020). Similarly, Donald Trump's former lawyer Michael Cohen reportedly hired a polling firm to attempt to rig online polls ahead of the 2016 presidential election. (The Hill 2019). Whether it is intentional or otherwise, it is clear that officials care about polling estimates and their potential consequences. One explanation for this is the ability to claim fraud in the event of electoral defeat. Another explanation – which this paper will explore – is that they believe polls can have consequences for voting behaviour.

While theoretical work on the potential consequences of pre-election polls is plentiful and long-standing, empirical evidence for such effects is largely mixed. Traditional conceptions of bandwagon or underdog effects fundamentally suggest that individuals consider the likely outcome of the election when making their choice at the polling station (Simon 1954). In this light, polls can serve as an information shortcut to indicate the viability of candidates or parties (Bartels 1996). However, such effects have proven difficult to consistently replicate and studies report results ranging from negligible (Sonck and Loosveldt 2010) to conditional poll effects (Donovan and Bowler 2016). This is in line with a developing vein of research which has found that polling information is not perceived as equally credible by all, and that partisanship is a key factor in determining poll credibility due to a pre-disposition toward preference-congruent information (e.g., Kuru, Pasek, and Traugott 2017; Madson and Hillygus 2020).

Against this background, this paper applies the theory of motivated reasoning to the effects of opinion polls on voting behaviour. By taking this approach, this paper is able to address two notable gaps in the field: perceptions of poll credibility and mixed-poll environments. When observing a poll, individuals are likely to use subjective criteria to establish its credibility, and in polarised contexts, these criteria are primarily expected to be the function of partisanship. When polls conflict and contradict one another, motivated reasoners should be more willing to accept subjectively credible vis-à-vis non-credible polls and use this information to inform their voting behaviour.

The impact of the credibility of pre-election polls on voting behaviour is tested through an original two-step experimental design conducted in Turkish municipalities, where polls vary considerably in their estimations and sources, and where political and media discourse is significantly polarised. A conjoint analysis conducted in June 2020 establishes considerable divergence in the perceived credibility of polls across partisan lines. These results are used to inform the construction of subjective credibility treatments for a survey experiment conducted in September 2020 that tests the impact of subjectively credible polls on voting behaviour. The survey experiment finds that the effect of polls on party choice is limited to the tactical considerations of the supporters of smaller parties. For the majority of voters, however, a potentially suppressive effect of non-credible and contradictory polls on turnout is identified. The findings of this study are more indicative of accuracy-seeking than directional motivations with regard to the effect of polls on voting behaviour.

The Effects of Opinion Polls

Existing research on the effects of polls on voting behaviour concerns itself with the proposition that pre-election polls can lead to self-fulfilling or self-falsifying results (Simon 1954). The most prominent theoretical assertions – which refer primarily but not exclusively to vote choice – are the purported bandwagon and underdog effects, in which voters are thought to be more likely to mobilise or cast their ballot for a candidate or party based on their relative popularity (Hardmeier 2008; Morton et al. 2015; Barnfield 2020). However, while theories detailing these potential effects of polls on voting behaviour are abundant, robust empirical evidence to support these ideas are scarce (for a comprehensive review, see Hardmeier 2008). The effects of polls on vote choice, for example, may be limited to certain subgroups of the electorate (Blais, Gidengil, and Nevitte 2006; Fredén 2014; Donovan and Bowler 2016), contingent on media coverage (Stolwijk, Schuck, and de Vreese 2017), or in some cases negligible (Sonck and Loosveldt 2010). With regard to turnout, however, there is considerably less empirical evidence to support such theories and experimental and quasi-experimental work chiefly concerns the effect of close elections in mobilising voters (e.g., Ansolabehere and Iyengar 1994; Großer and Schram 2010; Morton et al. 2015).

What complicates the matter further is that contrary to how polls are frequently presented in experimental studies, prior to elections citizens are likely to encounter several polls that may differ in their estimates and sources. How, then, do people respond when there is competing polling information? There are three possible answers to this question. First, and given the inconsistency of findings surrounding solitary polling effects, one possibility is that conflicting polls would illicit no response. A second answer would be that both polls are perceived by individuals even-handedly and both sets of information are used to update beliefs in a Bayesian manner. A third possibility is that individuals perceive one poll to be in some way superior to another, and thus give more credence to that poll when deciding whether and how to vote.

Perceptions of Poll Credibility

One fundamental way in which polling information can be perceived differently is its credibility. In the field of social psychology, the credibility or expertise associated to information has long been associated with greater abilities to persuade and change attitudes (e.g., Chaiken and Maheswaran 1994). However, with regard to its effects on individual level behaviour, it is important to note that there can be both objective and subjective guises of credibility. Previous objective operationalisations refer to the incentives of the producers and distributors of information to lie and mislead (Chiang and Knight 2011). Such interpretations more closely resemble the accuracy and quality of information than its perceived believability. This has important implications for the study of opinion polls as although what constitutes an objectively high quality or “credible” poll may be clear to experts and practitioners, voters may have a different set of criteria.

For experts, the credibility of a poll is based on considerations of sampling and methodology, which – if lacking in quality – can misrepresent the target population or the validity of responses. For voters, however, these aspects are likely to matter less (Panagopoulos et al. 2009). While there is evidence to suggest that voters do consider poll methodology when confronted with objectively high- and low-quality polls (Kuru, Pasek, and Traugott 2020), they may be less likely to discriminate on this basis when the objective difference in quality is less apparent or pronounced. For example, the difference in quality between two polls conducted by competing professional vendors is likely much less clear than that between one such poll and another conducted on social media using convenience sampling. When objective quality is less clear, the public are more likely to consider the credibility of a poll based on its source or whether it is congruent with their prior beliefs (Kuru, Pasek, and Traugott 2017; Searles, Smith, and Sui 2018; Madson and Hillygus 2020). This paper therefore considers the credibility of a poll to be a subjective evaluation of its believability. While this may imply considerations of perceived accuracy, source quality or reliability, the favouring of one of these dimensions over another is not presumed.

A growing literature on the perceived credibility of opinion polls argues that the cognitive process lying behind these perceptions is motivated reasoning, which contends that individuals deal with new information in a way which helps them reach correct (accuracy-seeking) or particular (directional) conclusions (Kunda 1990; Taber and Lodge 2006; Nir 2011; Kahan 2016). The latter has been found to be of particular significance in partisan-relevant issues (Jerit and Barabas 2012; Klar 2013), and such directional motivations have been used to explain why individuals are less willing to update already formed opinions in the face of new information (e.g., Redlawsk 2002; Taber and Lodge 2006; Bisgaard 2019). And while a significant share of the research on partisan motivated reasoning has considered issue-based polls on gun control or climate change, for example (e.g., Kuru, Pasek, and Traugott 2017; Druckman and McGrath 2019), there is less evidence of whether the same mechanisms apply to pre-election polls which are less normative and more circumstantial. Yet the case for the impact of partisanship remains strong in this regard, as partisans have been shown to consistently seek out and stand by information that reflects favourably on their party even when this information is inaccurate (Peterson and Iyengar 2021).

Poll Credibility and Voting Behaviour

From the perspective of motivated reasoning, different groups of people (such as partisans of competing parties) are likely to view and process the same information in different ways, and while a slender lead for a certain candidate could be viewed as credible to one group, it may not for the other. The potential effects of this information on voting behaviour are unclear, however. For example, while loyal partisans may be the most liable to this kind of perceptual bias, they are also the least likely to switch their vote from one party to another. This is particularly applicable to partisans of larger parties who lack a viable and ideologically similar alternative to their favoured party.

However, this is not necessarily the case for the supporters of smaller parties in multi-party contexts. Rather, these individuals may be more likely to vote tactically in response to polls they consider credible. Tactical voting has been shown to be prominent in both majoritarian and proportional electoral systems (Cox 1997; Blais, Gidengil, and Nevitte 2006; Fredén 2014). One primary reason for voting tactically in both systems is to avoid a wasted vote, i.e., not voting for a party with little possibility of affecting seat allocations (Gallagher 1991). Voters may therefore try to maximise the utility of their vote by opting for the party with the best chance of defeating their least-favoured party. Supporters of smaller parties within electoral blocs – given that their perception of a credible poll is akin to that of supporters of the major party in the bloc – may therefore attempt to avoid wasting their vote by switching to the largest in-bloc party. Hypothesis 1 therefore expects that the exposure to subjectively credible polls will increase in-bloc tactical voting.

When polling information is considered not to be credible, however, voters may become apathetic to, or cynical of, the electoral process, and choose rather to spurn the ballot box. Previous research has demonstrated that negative information environments and low-credibility sources can lead to a disillusionment with the democratic process and demobilise voters (Ansolabehere and Iyengar 1995). Pre-election opinion polls are also part of a wider environment of strategy-framed metacoverage that can have negative consequences for how electoral campaigns are perceived (de Vreese and Semetko 2002; de Vreese and Elenbaas 2008) and potentially demobilise cynical voters (Capella and Jamieson 1997; Krupnikov 2011). In the face only of polling information that is perceived to lack credibility, cynical citizens or those who lack confidence in the effectiveness or sufficiency of their vote may withdraw from the electoral process (Chong et al. 2015). Hypothesis 2 therefore expects that the exposure to subjectively non-credible polls will decrease the likelihood of an individual turning out to vote.

In mixed information environments, individuals are confronted with a decision about how to perceive and process different pieces of information that can contradict one another. With respect to opinion polls, the risk of perceptual bias may be particularly high as such information inevitably has partisan relevance (Jerit and Barabas 2012). From this perspective, polls in conflict with an individual's perceptions of credibility would be rejected in favour of a poll perceived to be more credible through the mechanism of motivated reasoning. This could be the function of accuracy or directional motivations, subject to the criteria used to determine credibility. In other words, partisans are likely to uncritically accept subjectively credible polls while disregarding a conflicting poll perceived not to be credible. Thus, Hypothesis 3 contends that in the face of conflicting polling information, poll effects are expected to be driven by polls that are perceived as credible. An implication of this is that mixed polling environments should produce a similar effect to that referred to in Hypothesis 1. In addition to party choice, this also has implications for the expected effects on turnout as the rejection of non-credible polls should dampen the suppressive effects anticipated by Hypothesis 2.

The Turkish Case

These hypotheses are empirically tested in the context of local elections in Turkish metropolitan municipalities. Mayoral elections in Turkey are high stakes competitive affairs where political discourse is increasingly polarised between pro- and anti-government party alliances: the People's Alliance which is composed of the national incumbent Justice and Development Party (AKP) and the Nationalist Movement Party (MHP), and the opposition Nation Alliance which is fronted by the People's Republican Party (CHP) and Good Party (IYI).¹ These alliances are largely representative of the primary political cleavage in Turkish society, characterised by religious conservativism on the one hand and secular centrism on the other (Aytac and Çarkoglu 2018; Laebens and Öztürk 2020).

Turkey constitutes a crucial case to test whether the subjective perceptions of opinion polls can affect voting behaviour. Importantly for the validity of testing the above hypotheses, pre-election polls in Turkey also tend to vary substantially with respect to their estimations between pollsters and are distributed through a media environment that emulates the central political divide. For example, one month prior to the municipal elections in March 2019, two polls conducted in the capital Ankara showed opposing candidates as each having a lead of at least five percentage points (Euronews 2019; Gazete Duvar 2019).² Inconsistent polling information is therefore something that Turkish voters are frequently exposed to prior to elections and, as such, are forced to make the decision of which polls to give more credence to when deciding their vote choice. It should be noted, however, that this phenomenon is not limited to Turkey, and variations in polling estimates (and relatedly “outlier” polls) are a concern in many nations around the world. Theories of polling effects and partisan bias are predominantly focused on the US and Western European contexts. The study of Turkey therefore adds to the few studies that consider such issues beyond these contexts,³ and the frequent variability of polls provides a fertile environment in which to test their consequences.

Despite the considerable lack of competition on the national stage in recent years as Recep Tayyip Erdogan has sought to tighten his grip on political power, the local elections of 2019 were the most hotly contested since Erdogan's AKP came to power in 2002 (Esen and Gumuscu 2019). The major opposition CHP made considerable gains in the election, including victories in Turkey's capital (Ankara) and largest city (Istanbul). A total of 30 metropolitan mayors were elected across the major population centres of Turkey, of which 15 were won by the AKP, 11 by the CHP, 1 by the MHP, and 3 by the pro-Kurdish HDP.

Research Design

The impact of the perceived credibility of polls on voting behaviour is assessed through a pre-registered original two-step experimental design. Initially, a conjoint analysis (Study I) conducted in June 2020 is used to determine the subjective determinants of poll credibility for pro- and anti-government partisans along three dimensions: the results of the poll, the polling vendor, and the distributing newspaper.⁴ The most preferred level of each dimension (or attribute) for both groups are then used to construct two subjective poll treatments that are subsequently implemented in a survey experiment (Study II), conducted between September-October 2020. Study II therefore tests the above hypotheses. In the survey experiment, respondents are presented with either one (credible or non-credible) or both (mixed) poll treatments and asked for their turnout likelihood and vote choice in the context of a hypothetical municipal election.⁵ Both studies were embedded in online surveys that were conducted by two separate private polling companies (Dynata in Study 1, NG Teknoloji in Study 2) with sampling representative of the metropolitan areas of Turkey.

Study I: The Determinants of Poll Credibility

Data collection for the conjoint analysis was conducted in June 2020, wherein 806 subjects were presented with six sets of two polls consecutively, thus providing a total of 4836 poll sets and 9672 polls evaluated in total.⁶⁷ The polls were presented graphically in the form of a bar chart including the five largest political parties.⁸ Along with this graphic, each poll also contained written information regarding the marginal lead of the AKP or CHP, as well as which company conducted the poll and the newspaper in which the poll was published (see Figure 1). While the polling numbers were included on the basis that they were a somewhat realistic representation of mayoral races in metropolitan areas, the newspapers were selected to represent an ideologically broad but representative range of mainstream publications. Similarly, polling organisations were included that are international and domestic, and range in their reputation from neutral to alleged biases or affiliations with government and opposition actors.⁹ The levels of these attributes were randomised for each poll. Following exposure to each set of two polls, subjects were asked which poll they would be more likely to “take into consideration” for a hypothetical local election.¹⁰

Figure 1.

Conjoint analysis poll set example.

The terminology in this question is left ambiguous for two reasons. First – and of particular relevance to how this study conceives of credibility – this approach does not signal one particular aspect or dimension of credibility over another. For example, if respondents were asked to identify the most credible poll it may encourage some respondents to pay more attention to the pollster or newspaper. By the same token, asking respondents to identify the more accurate poll would likely trigger focus on the content in relation to conditions outside of experimental conditions. As such, Study I does not assume either directional or accuracy motivations. The second reason for this choice of terminology is more practical and is simply that the above considerations become much more complex when translating such concepts and cognitive triggers between languages and cultural contexts.

For the analysis, the marginal means were obtained for each attribute (variable) value using a simple linear regression model of the selected poll on all attribute values as dummy variables, with standard errors clustered by respondent. The marginal mean therefore corresponds to the likelihood of selecting a given poll when an attribute (e.g., lead) takes a specific value (e.g. AKP + 4).¹¹

Figure 2 displays the effects of poll attributes by partisanship, with the effects for government supporters on the left, and opposition supporters on the right.¹²¹³ There is a clear and demonstrable difference between the two groups, suggesting a substantial mediating effect of partisanship on the perceived credibility of polls.¹⁴ The results of the conjoint analysis suggest that government supporters seem to base their perceptions of credibility almost exclusively on how the poll describes the relative position of parties (i.e. who's winning). In contrast, although opposition supporters place some weight on polls that indicate a lead for the main opposition party, the most influential factor is the distributor of the information (publishing newspaper). Study I therefore not only provides further evidence that polls are perceived differently by partisan rivals, but also how the criteria of credibility can deviate between government- and opposition-supporters.

Figure 2.

The effect of poll attributes on credibility by partisanship. Respondents were coded as government supporters if their preferred party is the AKP or MHP, and opposition supporters if they preferred the CHP, IYI, or DP. Horizontal bars represent 95% confidence intervals for marginal means. For full regression output see Supplementary Information file, A2.

Study II: Poll Credibility and Voting Behaviour

From the results of Study I, two polls are constructed and implemented as treatments in a survey experiment which concerns the effect of subjective poll credibility on voting behaviour. See Table 1. These polls are respectively (a) credible for government partisans but not for opposition partisans and (b) credible for opposition partisans but not for government partisans. The conflicting poll treatment consists of both of these polls presented simultaneously.¹⁵ Due to the fact that the source (pollster) in each case produced only minimal effects, this attribute will be held constant across treatments at a value that is statistically indistinguishable from other values across government and opposition supporters.

Table 1.

Perceived credibility poll formulations.

Treatment Attribute	Poll (a)	Poll (b)
Content (leader, margin)	+9 AKP	+4 CHP
Source (pollster)	Metropoll	Metropoll
Sender (newspaper)	Sabah	Sözcü
Credibility	Government (+)	Government (-)
	Opposition (-)	Opposition (+)

Study II was conducted between September-October 2020 in the same metropolitan regions of Turkey and was designed to mimic a mayoral election like those to which the opinion polls referred in Study ||I. 1,877 respondents were allocated either to a control group or one of three treatment conditions in which they were presented with a credible poll, a non-credible poll, or a combination of both. The allocation of treatment was randomised and what constitutes a “credible” treatment for any given respondent was based on a pre-treatment question that asks which party the respondent feels closest to.¹⁶ Those in the control group received no polling information. Prior to treatment, each respondent was presented with a text that read “imagine that there are local elections to elect a mayor tomorrow.”¹⁷ Those in the treatment groups were then informed that on the following page they would be presented with the results of a hypothetical pre-election poll (or polls) published one week before the election, as well as the company that conducted the survey and in which newspaper it was published. Those in the control condition received only the introductory sentence. Following the presentation of the poll(s), respondents were asked how likely they would be to vote under such conditions (on a 7-point scale from very unlikely to very likely), and for which party's candidate they would cast their vote.

For the full sample across treatments, 32.8% reported a willingness to vote for the AKP and 29.5% stated the CHP as their preference. The mean turnout level was 5.7 on a scale from 1–7, largely reflective of turnout during the 2019 local elections (84.7%).¹⁸ The results for an OLS regression analysis on party choice are illustrated with a coefficient plot in Figures 3 and 4. Two dependent variables are considered in these figures: the likelihood of voting for the AKP and CHP respectively. In each case the reference category is the control group, so coefficients indicate the conditional average treatment effect for each group: government partisans, opposition partisans, and non-partisans. Each model also includes a series of pre-treatment covariates that could influence voting behaviour as well as how they perceive pre-election polls. These include age, sex, education level, political knowledge, party identification (the blocking variable), and party attachment. In addition to demographic considerations, political knowledge is included as political sophisticates may be more resistant to polling information whilst also being more pre-disposed to participate (Donovan and Bowler 2016).

Figure 3.

This coefficient plot shows the effect of poll credibility on vote choice. Partisan groups consist of party blocs. The dependent variable for the plot on the left-hand side shows the change in likelihood of voting for the AKP relative to the control (vertical dotted line), and the plot on the right-hand side shows the same for voting for the CHP. Horizontal lines represent 95% confidence intervals. Results are shown for each of the three experimental conditions. For full OLS results see Supplementary Information file, B5.

Figure 4.

This coefficient plot shows the effect of poll credibility on party choice for supporters of the major and minor parties in the government and opposition alliances. The left-hand side of the plot shows the change in likelihood of voting for the AKP for supporters of the AKP (major party) and MHP (minor party) under three experimental conditions versus the control (vertical dotted line). The right hand side shows the likelihood for voting for the CHP for supporters of the CHP (major party) and the IYI (minor party). Horizontal lines represent 95% confidence intervals. For full OLS results see Supplementary Information file, B6.

Results: Vote Choice

Figure 3 suggests limited evidence that opinion polls affect the party choice of most citizens. None of the experimental conditions presented in the figure have a significant effect on an individual's likelihood of voting for the national incumbent AKP. As illustrated on the right-hand side of the figure, this is also broadly the case when the dependent variable is the likelihood of voting for the major opposition party.¹⁹ This appears to be as true for pro- and anti-government partisans as it is for non-partisans, which runs contrary to research which anticipates poll effects to be strongest among those with weaker preferences and allegiances (e.g. Mutz 1997; Donovan and Bowler 2016). ²⁰ This result casts yet more empirical doubt on the purported effect of polls on vote-switching, at least for supporters of larger parties.

However, in line with Hypothesis 1, Figure 4 demonstrates a notable effect of opinion polls on the party choice of supporters of smaller parties. Across all three experimental conditions, minor party partisans of the opposition electoral alliance show significant willingness to switch their vote to the senior opposition party. The strongest of these effects occurs in instances when the polling information is most favourable to the opposition, leading to a 31% (t = 3.7) increase in the likelihood of voting for the bloc's major party. When the polling information is mixed, smaller opposition party supporters are similarly likely to switch their vote to the CHP (21.1%, t = 2.76). However, given that the treatment favourable to the AKP also has a positive effect on voting CHP for minor party supporters, it may be that these voters consider the contest to be close enough to cause them to vote for the CHP, or simply that they will vote for whichever opposition party has the best chance to challenge the AKP.²¹

Results: Turnout

Figure 5 illustrates the effects of opinion polls on the likelihood of turning out to vote for partisans. Results are presented for government and opposition partisans separately, as well as a pooled sample which takes the subjectivity of the treatments into account. The first notable finding to point out is that – in line with the expectations of Hypothesis 2 – partisan respondents in the pooled analysis who were exposed to a non-credible poll were significantly less willing to turnout relative to the control group who received no polling information. This finding can be interpreted in one of two ways. First, experimenter bias may be a possibility in this case, as respondents who were presented only with information that they perceive to be biased against their beliefs could have developed a mistrust for the experiment itself. If this result is not a function of experimental conditions, however, the implication of this finding is that when the polling environment is consistent and perceived not to be credible, there is a risk that prospective voters will opt out of the process. These results suggest there is at least some support for the notion that pre-election polls – through their perceived credibility – can demobilise voters.²²

Figure 5.

This coefficient plot shows the effect of poll credibility on likely turnout, which ranges from 1 (very unlikely to vote) to 7 (very likely to vote). The horizontal axis therefore represents a change in turnout likelihood on this scale versus the control (vertical dotted line). Results are shown for subsamples of party blocs and the full pooled sample of partisans. Horizontal lines represent 95% confidence intervals. Respondents are coded such that credibility corresponds to the treatment and the respondent's party preference. For full OLS results see Supplementary Information file, B7.

Turning to the mixed poll treatments, however, there is perhaps a surprising similarity with the potentially suppressive effects of contradictory polls. To reiterate, Hypothesis 3 expects respondents to disregard non-credible polls in favour of those perceived to be more in line with their preferences. Thus, we should expect a greater resemblance between the effects of mixed and credible polls than mixed and non-credible polls. In contrast to this expectation, respondents treated with contradictory information reported a decrease in turnout likelihood of. The implication of this finding is that partisans seem to behave at least in part based on the consideration of the non-credible poll rather than disregarding this contrary information. Interestingly, this demobilising effect appears to be slightly more pronounced for government supporters than opposition supporters who, if anything, are more resolute in the face of non-credible polling information. However, the difference between these groups is not significant across experimental conditions.

These findings have implications for theories of perceptual bias and motivated reasoning, as although people may perceive this information differently (i.e., as credible or not credible), this does not appear to necessarily translate directly into political behaviour. Rather, when polls contradict one another, partisans on both sides of the political divide appear to be less willing to vote. This may either be the result of a loss in faith in the process due to the considerable uncertainty produced by the polls, or a replication of the suppressive effect of non-credible polls. The similarity of the magnitude in decreased turnout likelihood for non-credible and conflicting polls is more suggestive of accuracy-motivated reasoning – whereby both polls have been considered – or Bayesian updating than directional motivations (Druckman and McGrath 2019). This seemingly applies equally to government and opposition supporters. This may be a feature of election-related polls vis-à-vis issue-based polls, or it could also indicate a missing step in the literature regarding motivated reasoning: does bias in perceptions of political information necessarily lead to bias in subsequent behaviour? This is a potentially fruitful question for future research.

Discussion and Conclusion

Building upon work that has demonstrated how polls are perceived differently by different groups of society (e.g., Kuru, Pasek, and Traugott 2017; Madson and Hillygus 2020), this paper assesses the role of the subjective credibility of polls in voting behaviour. Two studies conducted in Turkey have produced evidence that the criteria used to determine the credibility of polls can differ across partisan lines, and these perceptions have greater implications for turnout than party choice. Using an original two-step experimental design in which a conjoint analysis informed the construction of poll treatments in a subsequent survey experiment, this paper finds evidence that the perceived credibility of pre-election polls is rooted in partisan bias, and that contradictory and non-credible polling information can negatively affect turnout. With regard to party choice, the impact of polls appears to be limited to the supporters of smaller parties who may tactically switch their vote to the largest in-bloc party. For most, the effect of polls therefore seems to pertain more to the willingness to vote than an individual's favoured party or candidate.

This study has attempted to consider the effect of a mixed information environment, yet there is only so far experimental designs can go to reflect real-world settings. And while the context of polls for hypothetical mayoral elections in Turkey was used to reflect some degree of validity, people are likely to seek out affirmatory information – such as polls – in election campaigns that support or validate their own preferences (e.g., Peterson and Iyengar 2021). In the same vein, the mixed poll treatments in this paper are limited to only two polls, whereas people can expect to encounter several in the weeks preceding elections, which offers the chance to cross-reference conflicting polls and identify outliers. Relatedly, the four- and nine-point margins used in this design may be too low to yield a significant bandwagon effect, which may be more prominent in cases where polling leads are larger. There are also other ways in which polls could influence political participation. While it seems that partisans are unlikely to alter their vote choice in light of polling information, they may be more or less willing to contribute to campaigns, or become party members, for example (Strömbäck 2012).

Nonetheless, these findings have important implications for the literature on the effects of opinion polls. First, the results presented here provide further evidence to suggest that the effect of polls on party choice is limited, and specifically to intra-coalition tactical voting. The primary way in which opinion polls can affect elections appears to be through encouraging or discouraging participation of certain groups of voters. Second, while this study has provided substantial evidence that pre-election polls are perceived subjectively along partisan lines, it failed to observe the corresponding behaviour that would be expected by an individual driven by directional motivated reasoning. Taking these two findings together, it is important to note that the demobilising effect of polls is asymmetrical only when this information is uniform. When information environments are mixed and contain both credible and non-credible polls, voters are demobilised relatively equally across partisan lines. Additionally, as the effects on turnout were similar across non-credible and mixed polling environments, the findings are more suggestive of accuracy than directional motivations in this case. Indeed, it is also possible that Study I is indicative of accuracy motivations, as opposition supporters tended to favour polls that suggested a more reasonable lead over a larger lead.

The results of this study also have important normative and practical implications. There is a considerable debate about whether opinion polls should or do influence elections. Polls act as information shortcuts for the uninformed, and some suggest that they fall under the umbrella of information that can aid voters to make a correct and informed voting decision. However, many states restrict the publishing of polls in the days or weeks prior to polling day on the basis that this kind of information can be misconstrued and potentially harmful to the integrity of the process. This study has shown that the risks associated with polling information are particularly acute when media environments are conducive to perceptual bias and when the electorate is polarised along partisan lines. Given that the results indicate that both non-credible and mixed polling environments can demobilise voters, there is also a risk that some political actors could instrumentalise or fabricate polls to affect the supporters of other parties. While the context of the compromised independence of much of the media in Turkey may augment the perceptions of rival media institutions, similar patterns of polarisation are not unfamiliar to many electoral contexts around the world – with varying degrees of democratic freedom. The independence and integrity of polling and media institutions therefore appear to be of great importance to ensure the integrity of voting.

Supplemental Material

sj-docx-1-ijpp-10.1177_19401612221087181 - Supplemental material for Poll Wars: Perceptions of Poll Credibility and Voting Behaviour

Supplemental material, sj-docx-1-ijpp-10.1177_19401612221087181 for Poll Wars: Perceptions of Poll Credibility and Voting Behaviour by Stephen Dawson in The International Journal of Press/Politics

Footnotes

Acknowledgments

I am appreciative of the support given for this paper by the Knowledge Resistance Project and the Quality of Government Institute. I would also like to express my gratitude to Mattias Agerberg,Nicholas Charron,Ezgi Irgil,Berker Kavasoglu,Melis Laebens,Aykut Öztürk,Jacob Sohlberg,Jesper Strömbäck,Georgios Xezonakis,and three anonymous reviewers for their feedback throughout this process.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research,authorship,and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research,authorship,and/or publication of this article: This work was supported by the Riksbankens Jubileumsfond-funded Knowledge Resistance Project and Stiftelsen Lars Hiertas Minne,(grant number M18-0310:1).

ORCID iD

Stephen Dawson

Supplemental Material

Supplemental material for this article is available online.

Author Biography

Stephen Dawson is a PhD Candidate at the University of Gothenburg,Sweden. He is affiliated to the Quality of Government (QoG) Institute and the Knowledge Resistance Project. His research interests include political institutions,the integrity and conduct of elections,corruption,and voting behaviour.

References

Ansolabehere

Iyengar

. 1994. “Of Horseshoes and Horse Races: Experimental Studies of the Impact of Poll Results on Electoral Behavior.” Political Communication 11(4):413–30. doi:10.1080/10584609.1994.9963048.

Ansolabehere

Iyengar

. 1995. Going Negative: How Political Advertisements Shrink and Polarize the Electorate. Free Press: New York.

Aytac

S. E.

Çarkoglu

. 2018. “Presidents Shaping Public Opinion in Parliamentary Democracies: A Survey Experiment in Turkey.” Political Behaviour 40:371–93. doi: 10.1007/s11109-017-9404-x.

Barnfield

2020. “Think Twice Before Jumping on the Bandwagon: Clarifying Concepts in Research on the Bandwagon Effect.” Political Studies Review 18(4):553–74. doi:10.1177/1478929919870691.

Bartels

L. M.

1996. “Uninformed Votes: Information Effect in Presidential Elections.” American Journal of Political Science 40(1):194–230. doi:10.2307/2111700.

Bisgaard

2019. “How Getting the Facts Right can Fuel Partisan Motivated Reasoning.” American Journal of Political Science 63(4):824–39. doi:10.1111/ajps.12432.

Blais

Gidengil

Nevitte

. 2006. ““Do Polls Influence the Vote?”.” In Capturing Campaign Effects, 263–79. Ann Arbor: University of Michigan Press.

Capella

J. N.

Jamieson

K. H.

. 1997. Spiral of Cynicism: The Press and the Public Good. New York: Oxford University Press.

Chaiken

Maheswaran

. 1994. “Heuristic Processing can Bias Systematic Processing: Effects of Source Credibility, Argument, Ambiguity, and Task Importance on Attitude Judgment.” Journal of Personality and Social Psychology 66:460–73. doi:10.1037/0022-3514.66.3.460.

10.

Chia

S. C.

Chang

. 2017. “Not my Horse: Voter Preferences, Media Sources, and Hostile Poll Reports in Election Campaigns.” International Journal of Public Opinion Research 29(1):23–45. doi:10.1093/ijpor/edv046.

11.

Chiang

Knight

. 2011. “Media Bias and Influence: Evidence from Newspaper Endorsements.” Review of Economic Studies 78(3):795–820. doi:10.1093/restud/rdq037.

12.

Chong

De La O

Karlan

Wantchekon

. 2015. “Does Corruption Information Inspire the Fight or Quash the Hope? A Field Experiment in Mexico on Voter Turnout, Choice, and Party Identification.” The Journal of Politics 77(1):55–71. doi:10.1086/678766.

13.

Cox

1997. Making Votes Count: Strategic Coordination in the World’s Electoral Systems. Cambridge: Cambridge University Press.

14.

de Vreese

C. H.

Elenbaas

. 2008. “Media in the Game of Politics: Effects of Strategic Metacoverage on Political Cynicism.” International Journal of Press/Politics 13(3):285–309. doi:10.1177/1940161208319650.

15.

de Vreese

C. H.

Semetko

H. A.

. 2002. “Cynical and Engaged: Strategic Campaign Coverage, Public Opinion, and Mobilization in a Referendum.” Communication Research 29(6):615–41. doi:10.1177/009365002237829.

16.

Donovan

Bowler

. 2016. “Experiments on the Effects of Opinion Polls and Implications for Laws Banning pre-Election Polling.” In Voting Experiments, ed. Blais

Laslier

Van der Straeten

. Switzerland: Springer.

17.

Druckman

J. N.

McGrath

M. C.

. 2019. “The Evidence for Motivated Reasoning in Climate Change Preference Formation.” Nature: Climate Change 9:111–119. doi:10.1038/s41558-018-0360-1.

18.

Esen

Gumuscu

. 2019. “Killing Competitive Authoritarianism Softly: The 2019 Local Elections in Turkey.” South European Society and Politics 24(3):317–42. doi:10.1080/13608746.2019.1691318.

19.

Euronews. 2019, March 4. 31 Mart Yerel Seçimleri: Gezici'ye göre İstanbul ve Ankara'da AK Parti önde. https://tr.euronews.com/2019/03/04/31-mart-yerel-secimleri-gezici-ye-gore-istanbul-ve-ankara-da-ak-parti-onde.

20.

Fredén

2014. “Threshold Insurance Voting in PR Systems: A Study of Voters’ Strategic Behavior in the 2010 Swedish General Election.” Journal of Elections, Public Opinion and Parties 24:473–92. doi:10.1080/17457289.2013.872118.

21.

Gallagher

1991. “Proportionality, Disproportionality and Electoral Systems.” Electoral Studies 10:33–51. doi:10.1016/0261-3794(91)90004-C.

22.

Gazete Duvar. 2019, March 2. AREA anketine göre Ankara'da Yavaş farklı önde!. https://www.gazeteduvar.com.tr/galeri/2019/03/02/area-anketine-gore-ankarada-yavasfarkli- onde.

23.

Großer

Schram

. 2010. “Public Opinion Polls, Voter Turnout, and Welfare: An Experimental Study.” American Journal of Political Science 54(3):700–17. doi:10.1111/j.1540-5907.2010.00455.x.

24.

Hainmueller

Hopkins

D. J.

Yamamoto

. 2014. “Causal Inference in Conjoint Analysis: Understanding Multidimensional Choices via Stated Preference Experiments.” Political Analysis 22(1):1–30. doi:10.1093/pan/mpt024.

25.

Hardmeier

2008. “The Effects of Published Polls on Citizens.” In The Sage Handbook of Public Opinion Research, ed. Donbach

Traugott

, 504–14. Thousand Oaks: Sage.

26.

The Hill. January 17^th, 2019. ‘Cohen hired firm to rig polls, only paid part of bill with bag of cash: report’, https://thehill.com/homenews/news/425785-cohen-hired-firm-to-try-to-rig-online-polls-for-trump-only-paid-part-of-bill

27.

Jerit

Barabas

. 2012. “Partisan Perceptual Bias and the Information Environment.” The Journal of Politics 74(3):672–84. doi:10.1017/S0022381612000187.

28.

Kahan

2016. “The Politically Motivated Reasoning Paradigm, Part 1: What Politically Motivated Reasoning is and how to Measure it.” Emerging Trends in Social & Behavioral Sciences. doi:10.1002/9781118900772.etrds0417.

29.

Klar

2013. “The Influence of Competing Identity Primes on Political Preferences.” The Journal of Politics 75(4):1108–24. doi:10.1017/S0022381613000698.

30.

Krupnikov

2011. “When Does Negativity Demobilize? Tracing the Conditional Effect of Negative Campaigning on Voter Turnout.” American Journal of Political Science 55(4):797–813. doi:10.1111/j.1540-5907.2011.00522.x.

31.

Kunda

1990. “The Case for Motivated Reasoning.” Psychological Bulletin 108(3):480. doi:10.1037/0033-2909.108.3.480.

32.

Kuru

Pasek

Traugott

M. W.

. 2017. “Motivated Reasoning in the Perceived Credibility of Public Opinion Polls.” Public Opinion Quarterly 81(2):422–46. doi:10.1093/poq/nfx018.

33.

Kuru

Pasek

Traugott

M. W.

. 2020. “When Polls Disagree: How Competitive Results and Methodological Quality Shape Partisan Perceptions of Polls and Electoral Predictions.” International Journal of Public Opinion Research 32(3):586–603. doi:10.1093/ijpor/edz035.

34.

Laebens

Öztürk

. 2020. “Partisanship and Autocratization: Polarization, Power Asymmetry, and Partisan Social Identities in Turkey.” Comparative Political Studies. doi:10.1177/0010414020926199.

35.

Lau

R. R.

Redlawsk

D. P.

. 2006. How Voters Decide: Information Processing in Election Campaigns. Cambridge: Cambridge University Press.

36.

Leeper

Hobolt

Tilley

. 2019. “Measuring Subgroup Preferences in Conjoint Experiments.” Political Analysis 28(2):207–21. doi:10.1017/pan.2019.30.

37.

Madson

G. J.

Hillygus

D. S.

. 2020. “All the Best Polls Agree with me: Bias in Evaluations of Political Polling.” Political Behavior 42:1055–72. doi:10.1007/s11109-019-09532-1.

38.

Morton

R. B.

Muller

Page

Torgler

. 2015. “Exit Polls, Turnout, and Bandwagon Voting: Evidence from a Natural Experiment.” European Economic Review 77:65–81. doi:10.1016/j.euroecorev.2015.03.012.

39.

Mutz

D. C.

1995. “Effects of Horse-Race Coverage on Campaign Coffers: Strategic Contributing in Presidential Primaries.” The Journal of Politics 57(4):1015–42. doi:10.2307/2960400.

40.

Mutz

D. C.

1997. “Mechanism of Momentum: Does Thinking Make it so?” Journal of Politics 59(1):104–25. doi:10.2307/2998217.

41.

Nir

2011. “Motivated Reasoning and Public Opinion Perception.” Public Opinion Quarterly 75(3):504–32. doi:10.1093/poq/nfq076.

42.

Nordic Monitor. March 25^th, 2020. ‘Secret wiretaps expose how Erdoğan had poll numbers manipulated before publication in a Turkish daily’. https://www.nordicmonitor.com/2020/03/secret-wiretaps-exposed-how-erdogan-had-poll-numbers-manipulated-before-publishing-in-a-turkish-daily/

43.

Panagopoulos

Gueorguieva

Slotnick

Gulati

Williams

. 2009. Politicking Online: The Transformation of Election Campaign Communications. New Brunswick: Rutgers University Press.

44.

Peterson

Iyengar

. 2021. “Partisan Gaps in Political Information and Information-Seeking Behavior: Motivated Reasoning or Cheerleading?” American Journal of Political Science 65(1):133–47. doi:10.1111/ajps.12535.

45.

Redlawsk

D. P.

2002. “Hot Cognition or Cool Consideration? Testing the Effects of Motivated Reasoning on Political Decision Making.” The Journal of Politics 64(4):1021–44. doi:10.1111/1468-2508.00161.

46.

Reuters (June 6th, 2012), ‘Venezuela poll wars rage as presidential race heats up’, https://www.reuters.com/article/us-venezuela-election-polls/venezuela-poll-wars-rage-as-presidential-race-heats-up-idUSBRE85512920120606

47.

Searles

Smith

Sui

. 2018. “Partisan media, Electoral Predictions, and Wishful Thinking.” Public Opinion Quarterly 82(S1):302–24. doi:10.1093/poq/nfy006.

48.

Simon

H. A.

1954. “Bandwagon and Underdog Effects and the Possibility of Election Predictions.” Public Opinion Quarterly 18(3):245–53. doi:10.1086/266513.

49.

Sonck

Loosveldt

. 2010. “Impact of Poll Results on Personal Opinions and Perceptions of Collective Opinion.” International Journal of Public Opinion Research 22(2):230–55. doi:10.1093/ijpor/edp045.

50.

Stolwijk

Schuck

de Vreese

. 2017. “How Anxiety and Enthusiasm Help Explain the Bandwagon Effect.” International Journal of Public Opinion Research 29(4):554–74. doi:10.1093/ijpor/edw018.

51.

Strömbäck

2012. “Published Opinion Polls, Strategic Party Behaviour and News Management.” In Opinion Polls and the media: Reflecting and Shaping Public Opinion, ed. Holtz-Bacha

Strömbäck

, 246–66. Basingstoke: Palgrave Macmillan.

52.

Taber

C. S.

Lodge

. 2006. “Motivated Skepticism in the Evaluation of Political Beliefs.” American Journal of Political Science 50(3):755–69. doi:10.1111/j.1540-5907.2006.00214.x.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.21 MB