The myth of ‘evidence-based policymaking’ in a decentred state

Abstract

I describe a policy theory story in which a decentred state results from choice and necessity. Governments often choose not to centralise policymaking but they would not succeed if they tried. Many policy scholars take this story for granted, but it is often ignored in other academic disciplines and wider political debate. Instead, commentators call for more centralisation to deliver more accountable, ‘rational,’ and ‘evidence-based’ policymaking. Such contradictory arguments, about the feasibility and value of government centralisation, raise an ever-present dilemma for governments to accept or challenge decentring. They also accentuate a modern dilemma about how to seek ‘evidence-based policymaking’ in a decentred state. I identify three ideal-type ways in which governments can address both dilemmas consistently. I then identify their ad hoc use by UK and Scottish governments. Although each government has a reputation for more or less centralist approaches, both face similar dilemmas and address them in similar ways. Their choices reflect their need to appear to be in control while dealing with the fact that they are not.

Introduction: Policy theory and the decentred state

I use policy theory to tell a story about the decentred state. Decentred policymaking may result from choice, when a political system contains a division of powers or a central government chooses to engage in cooperation with many other bodies rather than assert central control. However, it is largely borne of necessity: governments would fail if they tried to centralise policymaking. I base this argument on two abstract propositions: all policymakers face ‘bounded rationality’ and engage in a policymaking environment over which they have limited knowledge and minimal control. The former suggests that they must ignore most information most of the time, since they can only pay attention to a tiny proportion of their responsibilities at any time (Baumgartner, 2017). The latter suggests that there are many policymakers and influencers spread across many types of government, producing a myriad of rules, networks, locally dominant beliefs, and ways to respond to socioeconomic conditions and events outside of the control of any single ‘centre’ (Cairney and Weible, 2017; Heikkila and Cairney, 2018). Some policymakers are more powerful than other actors, but not to the extent that we should conclude that there is singular central control. Rather, policymaking is multi-centric (Cairney et al., 2019).

Variants of this story represent the conventional wisdom in key branches of US and European policy studies (Cairney, 2019; Weible and Sabatier, 2018). Its main messages are present in UK-focused comparisons between the simplified Westminster model story of central government power versus more realistic accounts of governance (Bache and Flinders, 2004; Kerr and Kettell, 2006; Rhodes, 1997). However, it does not have the same impact in wider political debate (Jordan and Cairney, 2013b: 235) or other academic disciplines (Oliver et al., 2014a, 2014b). Its lack of impact in political debate often prompts central governments to try, in vain, to project an image of governing competence by regaining central control of policymaking (Bevir and Rhodes, 2003). Its lack of wider academic impact allows governments to tell stories of ‘evidence-based policymaking’ (EBPM) (Sullivan, 2011), and commentators to make misguided calls for more centralisation to deliver more ‘rational’ policymaking (Cairney, 2016).

Indeed, the idea of EBPM sums up a large collection of longstanding myths, hopes and political slogans about the use of information in central government (Cairney, 2016). For some governments, like New Labour (from 1997) in the UK, EBPM phraseology accompanied reforms to ‘modernise’ and take control of policymaking (Richards and Smith, 2004). Further, ‘what counts is what works’ symbolised the – sometimes sincere, sometimes strategic – assertion that evidence should trump ideology when addressing policy problems (Davies et al., 1999). EBPM became a ‘useful myth’ to depoliticise issues (Boswell, 2018). For many critics of government, EBPM is an ideal to compare with actual practices, to identify ‘policy-based evidence,’ in which politicians make a choice then cherry-pick evidence to support it (Cairney, 2019b). More generally, scientists bemoan the gap between the supply and use of evidence, often blaming it on ill-defined causes – such as low scientific literacy in government, low political will or ‘post truth politics’ – without reference to the necessity of decentred policymaking or the contested nature of high quality or policy-relevant evidence (Cairney and Oliver, 2018; Jasanoff and Simmet, 2017; Oliver et al., 2014a, 2014b).

Such arguments raise an ever-present dilemma for governments, facing contradictory pressures to accept or challenge decentring, and accentuate a modern dilemma about how to describe ambitions for centralised EBPM in a decentred state. Governments could project a mix of noble intent and policymaking order, to use the language of EBPM to reinforce a story of a clearly defined single centre of government and central government control. Their story could be: a core group of policymakers and policy analysts can, and should, privilege high-quality scientific evidence to identify problems and produce and evaluate solutions via a series of stages in a policy cycle (Cairney, 2016). If so, they need to resolve two problems: how to project central control in its absence; and how to make political choices, regarding what counts as policy-relevant evidence, while projecting the sense that such choices are technical or objective (Cairney, 2019d).

To show how these dual dilemmas play out in practice, I first identify three ideal-type ways in which governments can address them consistently, to connect choices about how to (a) deal with decentred policymaking, and (b) gather and use evidence. There are several possible models of EBPM, from the centralist top-down rollout of uniform interventions that privilege a ‘hierarchy’ of scientific evidence, to the localist bottom-up spread of diverse programmes that privilege storytelling-based knowledge exchange between practitioners and service users.

I then identify their – often ad hoc and inconsistent – use by UK and Scottish central governments. Each government has a reputation for top down (UK) and bottom up (Scottish) ‘policy styles’ (Cairney, 2019c), but both face similar pressures to let go and hold on to control (Matthews, 2016). Their choices reflect their need to appear to be in control while dealing with the fact that they are not. Pragmatism may prompt them to acknowledge their limits somewhat, but Westminster-style democratic accountability reminds them to portray an image of central control, in which you know who is responsible and to blame in elections. Although the Scottish and UK governments project a preference for different models, in practice they adopt similar mix-and-match approach to governance and evidence choices.

The policy theory story of decentring: Bounded rationality and complex policymaking environments preclude EBPM

This special issue contains several ways to engage with ‘decentring,’ prompting some need for disambiguation. Bevir (2013: 1, 152–153) provides a useful distinction between:

His preferred (a) description of governance as ‘as a set of diverse practices that people are constantly creating and recreating’ rather than real and fixed formal laws and institutions, and (b) emphasis on theoretical analysis that shows why we should not assume that a powerful central government would exist.

Accounts that are too keen to ‘reify’ such practices, to (a) describe a real and identifiable source of central authority (such as ‘the centre,’ ‘central government’ or ‘core executive’), and (b) present empirical analysis in which the loss or gain of central government power is measurable.

My story largely takes the former approach to use (a) theoretical reasons to expect ever-present decentred policymaking, to inform (b) old and new concepts that describe too-high expectations for central government control over policy processes and outcomes. I use some of the language of central control to engage with an audience familiar with those terms, rather than to suggest that there is a well-defined actor that we can simply call ‘central government.’ A policy theory story, that policymakers have to ignore almost all evidence and decisions taken in their name, and do not understand far less control the policy process, provides a lens through which to analyse claims to its contrary.

EBPM and comprehensive rationality as an ideal-type or ideal state

It is not straightforward to analyse new discussions of EBPM, because many seem to conflate two propositions that should be separated analytically: an ideal state to which we should aspire (an empirical proposition), and an ideal-type to compare with what is possible in the real world (a theoretical proposition). Many scientists and policymakers articulate the idea that policymakers and researchers should cooperate to use the best evidence to determine ‘what works’ in policy. At times, this language is reminiscent of 1950s discussions of the ideal-type ‘comprehensive’ rationality in policymaking (Cairney, 2019d). At other times, EBPM appears to be an ideal state to compare with the more disappointing ‘policy-based evidence’ processes they find when they engage in politics (Cairney and Oliver, 2017).

Still, these modern discussions clearly try to re-establish the value of classic theoretical discussions of the ideal-type, which involves a core group of elected policymakers at the ‘centre’ (or ‘top’), using their values to identify problems to solve, and translating their policies into action to maximise benefits to society, aided by neutral organisations gathering all the facts necessary to produce solutions (John, 1998: 33). In practice, central actors are unable to: separate values from facts in a meaningful way; rank policy aims in a logical and consistent manner; gather information comprehensively, or possess the cognitive ability to process it (Cairney, 2012a: 96–97; Lindblom, 1959; Simon, 1976). Instead, Simon (1976: xxviii) described policymakers addressing ‘bounded rationality’ by using ‘rules of thumb’ to limit their analysis and produce ‘good enough’ decisions. Further, Baumgartner and Jones (1993) and Jones and Baumgartner (2005) show that policymakers can only pay attention to a tiny proportion of their responsibilities, which always limits their potential control of the many decisions made in their name.

In that context, modern references to a comprehensively evidence-driven policy process are misleading if they describe the sense that we are getting closer to an ideal type. As Botterill and Hindmoor (2012: 367) describe, advances in scientific research methods and information technology have given ‘a new breath of life’ to hopes for comprehensive rationality, but the accumulation of new evidence does not diminish the problem of bounded rationality (Cairney, 2016: 14a). Instead, modern discussions of policymaker psychology focus on the ‘rational’ short cuts that policymakers use to identify good enough sources of information, combined with the ‘irrational’ ways in which they use their beliefs, emotions, habits and familiarity with issues to identify policy problems and solutions (Cairney and Kwiatkowski, 2017; Haidt, 2001; Kahneman, 2012; Lewis, 2013; Lodge and Wegrich, 2012).

This scarcity of attention helps explain, for example, why policymakers ignore most issues in the absence of a major event (Shaffer, 2017), policymaking organisations make searches for information, which miss key elements routinely (Workman et al., 2017), and organisations fail to respond to changing circumstances proportionately (Epp, 2017). Actors engage in ‘trial-and-error strategies,’ use their ‘social tribal instincts’ to rely on ‘different decision heuristics to deal with uncertain and dynamic environments,’ tell stories to persuade their audience to see a policy problem and its solution in a particular way, or make emotional judgements based on stereotypes to propose benefits to some populations and punishments for others (Cairney, 2013a: 279; Cairney, 2020; Jones et al., 2014; Lubell, 2013: 544; Schneider and Ingram, 1997). In each case, ‘most members of the system are not paying attention to most issues most of the time’ (Baumgartner, 2017: 72). Bounded rationality precludes the possibility of a small group of people at the centre with the ability to gather all evidence and coordinate policymaking (and studies such as Maybin, 2016 describe this limitation in action).

Policymakers do not control the policy process

Advocates of EBPM often seem drawn to the idea of a linear and orderly policy cycle with discrete stages – agenda setting, policy formulation, legitimation, implementation, evaluation, policy maintenance/succession/termination – because it offers a simple and appealing model with clear advice on how to engage (Oliver et al., 2014b). Indeed, the stages approach offers an idea of how policy should be made: elected policymakers in central government, aided by expert policy analysts, make and legitimise choices; skillful public servants carry them out; and, analysts assess the results with the aid of scientific evidence (Colebatch, 1998: 102; Jann and Wegrich, 2007: 44; compare with Lasswell, 1956). It exists as a story for policymakers to tell about their work, partly because it is consistent with the idea of elected policymakers being in charge and accountable (Cairney, 2015: 25–26; Everett, 2003: 66–68; Rhodes, 2013: 486).

Some scholars question its normative appropriateness, since it does not allow for the diffusion of power in multi-level systems or ‘localism’ to tailor policy to local needs and desires (Barrett and Fudge, 1981; Durose and Richardson, 2015, Durose et al., 2017; Cairney et al., 2019; Hill and Hupe, 2009; Hooghe and Marks, 2003). Further, the cycle provides misleading practical advice: you will generally not find an orderly process with a clearly defined debate on problem definition, a single moment of authoritative choice, and a clear chance to use scientific evidence to evaluate policy before deciding whether to continue (Cairney, 2012a, 2018; Sabatier and Weible, 2014; Weible and Cairney, 2018). In the place of cycles, policy theories conceptualise the relationship between five main factors that constitute the policymaking environment (Cairney, 2016: 27–30; Heikkila and Cairney, 2018; John, 1998: 173; John, 2003: 488):

Many actors making and influencing choices at many levels of government. Scientists are competing with many actors to present evidence and secure a policymaker audience, and there are many ‘venues’ in which authoritative decisions can take place.

A proliferation of ‘institutions,’ or the rules and norms maintained in many ‘venues.’ Rules can be formal and well understood, or (more in keeping with Bevir’s analysis) ‘exist in the minds of the participants and sometimes are shared as implicit knowledge rather than in an explicit and written form’ (Ostrom, 2007: 23). They include rules of evidence gathering; support for ‘evidence-based’ solutions varies markedly according to which organisation takes the lead, how its actors understand problems, and the practices they encourage.

The pervasiveness of policy networks, or relationships between policymakers and influencers, many of which develop in ‘subsystems’ and contain small groups of specialists. Some networks are close-knit and difficult to access because bureaucracies have operating procedures that favour particular sources of evidence and participants. Exclusivity can help scientific advocates, as privileged insiders in some venues, and hinder them, as outsiders in others (Boswell, 2009: 16; Cairney et al., 2012: 43).

A tendency for entrenched ‘ideas’ – the ‘core beliefs’ of policymakers or ‘paradigms’ in which they operate – to dominate discussion (Hall, 1993; Cairney and Weible, 2015). Beliefs influence levels of receptivity to new policy solutions (Kingdon, 1984). New evidence on the effectiveness of a solution must be accompanied by persuasion to prompt a shift of attention to a problem and a willingness to understand it in a new way.

Policy conditions and events that can reinforce stability or prompt policymaker attention to shift. Social or economic ‘crises’ or ‘focusing events’ prompt lurches of attention from one issue to another, and evidence can be used to encourage that shift, but this outcome is not inevitable (Birkland, 1997).

On that basis, many theories present variations on the theme of limited central control. For example, the study of ‘policy communities’ begins with the identification of a state, so large and crowded, that it is in danger of becoming unmanageable. Consequently, the state’s component parts are broken down into sectors and sub-sectors, with power spread across government. Elected policymakers can only pay attention to a tiny proportion of issues for which they are responsible. They pay attention to a small number and ignore the rest. They delegate responsibility to other actors such as bureaucrats, often at low levels of government. Bureaucrats rely on specialist organisations for information and advice. Organisations trade such resources for access to, and influence within, government. Therefore, most public policy is conducted through small and specialist communities at a level of government not particularly visible to the public, and with minimal senior policymaker involvement (Jordan and Maloney, 1997; Jordan and Cairney, 2013; Richardson and Jordan, 1979).

Similarly, punctuated equilibrium theory suggests that policymakers at the ‘macro’ level can only pay attention to a tiny proportion of all possible issues, and that attention lurches from issue to issue. This dynamic produces ‘hyperincremental’ change in most fields and profound change in a small number (Baumgartner and Jones, 2009). Multi-level governance (MLG) and polycentric governance suggest that there is no single source of authority in political systems. Central governments cooperate with other actors to produce the rules to which many policymaking organisations might agree (Aligica and Tarko, 2012; Bache and Flinders, 2004; Heikkila and Andersson, 2018; Rhodes, 1997).

Studies of policy succession, inheritance and historical institutionalism suggest that most ‘new’ policies are revisions of old policy (Hogwood and Peters, 1983), new governments begin by accepting the commitments of their predecessors (Rose, 1990), and choices in the past contributed to the formation of the rules of organisations operating in the present (Pierson, 2000). Studies of implementation suggest that policy is delivered at the ‘bottom’ by actors subject to too many rules to follow; they use their discretion, judgement, and training to fulfil a proportion of central government aims (Lipsky, 1980). Complexity theorists identify a tendency for policy to ‘emerge’ from complex policymaking systems, often at a local level and despite attempts by central governments to control outcomes (Cairney, 2012b: 349; Geyer and Cairney, 2015; Sanderson, 2006, 2009; Teisman and Klijn, 2008: 288).

In relatively few cases, policy theories provide a greater sense of central government or policymaker control. Social construction and policy design (SCPD) suggests that policymakers make quick moral choices based on social stereotypes, and these choices have a profound effect on policy design (Schneider and Ingram, 1997; Schneider et al., 2014). In other approaches, there is more emphasis on strategic control of policymaking, including ‘blame game,’ ‘statecraft’ and ‘depoliticisation’ strategies used by central governments to appear authoritative, or ‘metagovernance’ strategies to control policy networks (Cairney et al., 2019: 25–26; Dommett and Flinders, 2015; Hood, 2010; Jessop, 2003; Sørensen and Torfing, 2005).

However, a story of multi-centric policymaking seems to represent the conventional wisdom in policy studies (Cairney et al., 2019). There are many variants of this story, and it is often told with more subtlety than in this article, but each variant suggests that highly centralised government is unlikely because boundedly rational policymakers must ignore most evidence and most decisions taken in their name, and only partially understand the policymaking environment in which they operate (Jones and Thomas, 2017).

Further, this story accentuates a large literature referring to ‘evidence informed’ rather than ‘based’ policy. For many scholars, ‘informed’ symbolises a way to manage expectations for EBPM (Boaz et al., 2019; Head, 2008). Indeed, some practitioners describe a discursive shift from ‘based’ to ‘informed’ to highlight their experience of a ‘messy’ policy process in which the connection between evidence and policy is unclear (Gluckman, 2016; Topp et al., 2018). In comparison, my story is designed to prompt some readers to wonder why they thought that EBPM driven by ‘the centre’ could ever be a realistic proposition.

The Westminster Model’s story of central government control

Stories of decentred policymaking are pervasive in policy studies, but not Westminster politics. They compete badly with variants of the Westminster model’s story (Bevir and Rhodes, 1999; Gains and Stoker, 2009; Marsh et al., 2001; Rhodes, 1997; Richards and Smith, 2002: 3). At its heart is a simple normative message about democratic accountability, based on concepts such as:

Representative democracy, in which most public participation is via elections rather than direct engagement, and most government accountability is to the public through elections and parliamentary scrutiny.

Parliamentary sovereignty, unitary state and a fusion of executive and legislature, in which power is retained in UK central government (then shared somewhat with EU and devolved governments).

Majority party control, in which a plurality electoral system exaggerates a party’s majority to command a majority in Parliament.

Cabinet government and individual ministerial responsibility, in which the Prime Minister appoints the Cabinet, and Secretaries of State are responsible for all that happens in their departments.

There are many ways to tell this story, but the moral remains the same: power and responsibility go hand in hand since, if you know who is in charge, you know who to reward or punish. It remains important even if fictional (Duggett, 2009). Its rhetorical power endures when people maintain a simple argument during general election and referendum debates: we know who should be in charge and should hold people to account on that basis. This perspective has a profound effect on the ways in which policymakers defend their actions, and political actors compete for votes or policy outcomes, even when it is ridiculously misleading and ministers know it (Bevir and Rhodes, 2006; Bevir, 2013; Cairney, 2019a; Rhodes, 2013). Consequently, policymakers respond to two contradictory stories, stressing:

Their need to let go, and share power with many actors, to reflect their limited control.

Their need to hold on, to live up to an electoral imperative, and respond to parliamentary scrutiny by telling a story of their success. Or, some might believe that they could control government if they tried (Hay, 2009; Matthews, 2016).

To do so, they play a game of shared and sole responsibility. Ministers try to construct an image of governing competence by making ‘hard choices’ and dominating the legislative process without necessarily expecting long term results (McConnell, 2010). Or, when frustrated with their lack of progress, they seek to reform government to reassert power by: forming quangos to bypass the role of local authorities (Greenwood et al., 2001: 157; Stoker, 2004: 32); transferring industries to the market; obliging local authorities to ‘contract out’ the delivery of public services to other bodies; and, constructing regulatory and performance management regimes designed to control public bodies from the centre (Hood, 2007).

In that context, a common argument is that the UK Government has exacerbated its ‘governance problem’ (the gap between an appearance of central control and what central governments can actually do) while trying to solve it (Cairney, 2009). ‘New public management’ reforms, perhaps designed to reassert central government power, often reinforced a fragmented public landscape. This outcome prompted academic debate about the extent to which the UK state was: (a) ‘hollowing out’ and increasingly unable to influence policy outcomes without the aid of a range of other bodies, or (b) ‘rejuvenated’ and ‘lean’, with ministers able to focus on core tasks, make strategic decisions, and create rules to ensure that their aims are carried out by others (Bevir and Rhodes, 2003: 6; Cairney, 2012a: 160; Hogwood, 1997; Holliday, 2000; Marinetto, 2003; Marsh et al., 2003: 308; Rhodes, 1994; 1997; Richards and Smith, 2006: 182).

We should understand EBPM in that context: policymakers are unable to hoard power at the centre, but struggle to find a defendable way to let go. They have gone to incredible lengths to juggle their need to appear to be in control, to defend their record in elections and to Parliament, with their need to pragmatic and share policymaking responsibility to get things done (or to pass on responsibility for outcomes to others). This juggling act reflects the difficulty of reconciling forms of accountability based on the centralisation versus diffusion of power. Governments rely on quasi-non-governmental, private and third sector bodies for the success of policy delivery and periodically seek to reform the public sector to re-inject clarity of responsibility (Flinders and Skelcher, 2012; Richards and Smith, 2004). Ministers devolve decisions to public bodies but also intervene, in an ad hoc way, to deal with organisational crises (Gains and Stoker, 2009: 11). Therefore, we should we should be wary of governments when they use EBPM phraseology to suggest that they can coordinate the use of evidence from the centre in a straightforward and predictable manner.

The Scottish approach to EBPM: Territorial responses to universal pressures

The election of new parliaments in Scotland, Wales and Northern Ireland in 1999 marked a new era of devolved government in which a series of constitutional choices fostered deliberate decentring (Birrell, 2012). However, it also prompted each devolved central government to deal with their own versions of these practical limits to central control. For example, compare the ways in which UK and Scottish governments have addressed issues of decentring, given three main ways in which their contexts differ:

Electoral dynamics. The Scottish Parliament is elected using a mixed-member proportional system, producing more need for coalition or minority government, potentially contributing to a general spirit of ‘inclusiveness, bargaining and compromise’ (Lijphart, 1999: 2). However, it also operates largely as a mini-Westminster, in which the government governs and Parliament scrutinises (Cairney and McGarvey, 2013).

Responsibilities. The Scottish Government’s responsibilities do not include areas, such as economic policy, most associated with ideological contestation (Cairney, 2015). It largely took control of social policy functions including health, education, social work, criminal justice, housing and local government, while the UK retained control of areas including international affairs, economic policy and social security. Scotland enjoyed further periods of devolution after 1999, including key aspects of taxation and social security in 2016.

Policymaking scale. Scotland’s population of five million is relatively small, and the Scottish Government’s capacity is limited. The scale of policymaking produces the potential for relatively close personal relationships to develop between senior policymakers in central government and leaders of public bodies and key stakeholders. It also prompts civil servants to rely more on external experts and the organisations with experience of policy implementation.

In that context, the Scottish Government developed a reputation for making policy in a different way than the UK Government, partly to deal with decentred policymaking in a ‘bottom up’ way. A ‘Scottish policy style’ describes:

The Scottish Government’s reputation for making policy through networks, pursuing a consultative and cooperative style with ‘pressure participants’ (Cairney, 2009, 2011b, 2013b; Cairney and McGarvey, 2013; Jordan et al., 2004; Keating, 2005, 2010).

A distinctive ‘governance’ style: a relative willingness to devolve the delivery of policy to public bodies (Cairney, 2009).

Scotland’s reputation owes much to the views of pressure participants, who describe the UK policy process as less consultative, more top-down, less reliant on professional or policy networks, and with more willingness to generate competition between participants, and senior Scottish Government ministers and civil servants, who have accentuated this image (Cairney, 2011a: 130; Cairney, 2014: 9; Cairney and McGarvey, 2013: 142). Most participants identify hopes for a new culture of policymaking, rather than systematic evidence (Cairney, 2008: 358; Cairney, 2019c; Cairney and McGarvey, 2013: 13). Most notably, former Permanent Secretaries of the Scottish Government describe how it developed several models of policymaking to:

join up government by abolishing policy-specific departments,

develop long term outcome-based measures of success, not short-term targets driven from the centre,

form a concordat with the Convention of Scottish Local Authorities (COSLA) to reduce central ‘micromanagement,’

use the ‘improvement method’ to combine evidence and governance, with a strong emphasis on practitioner experience, the ‘assets’ of local communities, and the co-production of public services with service users (Elvidge, 2011; Housden, 2014).

Further, Housden (2014: 69–70) argues that this approach contrasts with most periods of UK policymaking, marked by centrally driven targets and a performance management regime associated with New Public Management.

Yet, there are three important qualifications. First, Housden’s (2014: 71–72) justification of the Scottish Approach, in relation to the limits to central government control and the non-linear nature of policymaking systems, bears a strong resemblance to my story regarding all political systems. Second, in practice, the Scottish Government plays ‘the game of democratic accountability’ which requires it to present an image of central government competence (Cairney, 2016: 289–290). Third, key local government actors describe the central-local relationship as centralist (Cairney, 2011a: 130; O’Neill, 2014).

The UK and Scottish governments face a continuous trade-off between central and local control, and the unintended consequences of the balance they attempt to strike. When they pursue centralism, they encounter limits to joined-up government at the centre, and emergent outcomes beyond their control at local levels. When they encourage local discretion, and the involvement of users and communities in service delivery, they encounter problems of accountability when there does not seem to be a meaningful nationally-driven strategy and there is high potential to identify a ‘postcode lottery’ in which people receive a different level of service according to where they live (Cairney et al., 2016: 2). These discussions suggest that the UK’s top-down image is often exaggerated by a focus on electorally salient issues and adversarial party competition, while Scotland’s bottom-up image is exaggerated by misleading comparisons with the UK. If we focus on the bigger picture, we find similar practices that occur when governments seek ways to emphasise central government authority while adapting to their limits.

Three consistent EBPM models and their inconsistent use

When scientists promote the use of the best evidence to determine ‘what works’ in policy, they often side-step the fact that evidence quality is highly contested (Oliver and Cairney, 2019; Parkhurst, 2017). For some researchers, there is a hierarchy of evidence: at the top is the randomised control trial (RCT) and the systematic review of RCTs, with expertise much further down, followed by practitioner experience and service user feedback near the bottom (Oliver et al., 2014a, 2014b; Oliver and Pearce, 2017). For others, this hierarchy should be flipped on its head to accentuate the role of experience and judgement to make choices based on many claims to knowledge (see Boaz et al., 2019; Cairney, 2019b; Fleming and Rhodes, 2018; Nutley et al., 2013). Most policymakers prefer a wider range of sources of information, combining their own experience with information ranging from peer reviewed scientific evidence and the ‘grey’ literature, to public opinion and feedback from consultation (Cairney and Oliver, 2017; Davidson, 2017; Lomas and Brown, 2009; Nilsson et al., 2008). Further, while it may be possible to persuade some central government departments or agencies to privilege specific forms of scientific evidence, they also pursue other principles, such as to foster consensus-driven policymaking or a shift from centralist to localist practices (Cairney, 2016). The choice of good evidence is political, and linked inextricably to other choices about good policymaking.

Consequently, EBPM involves the combination of two simultaneous political choices, to resolve (a) contradictory – decentring and centralising – pressures in governance and (b) debates on how to identify what evidence counts. Table 1 identifies three ideal-type models to describe the ways in which governments could make consistent choices when selecting sources of evidence and deciding how to centralise or localise policymaking when they seek to spread policy interventions across a whole country:

Table 1.

Three consistent approaches to ‘scaling up’ evidence-informed policy.

	Model 1 Implementation science	Model 2 Story telling	Model 3 Improvement method
The main story	Interventions are highly regarded when backed by empirical data from international randomised control trials (RCTs). The approach has relatively high status in health departments, often while addressing issues of health, social care, and social work.	Practitioners tell stories of policy experiences, and invite other people to learn from them. Policy is driven by governance principles based on co-producing policy with service users.	Central governments identify promising evidence, train practitioners to use the improvement method, and experiment with local interventions. Discussion about how to ‘scale up’ policy combines personal reflection and empirical evidence of success.
How should you gather evidence of effectiveness and best practice?	With reference to a hierarchy of evidence and evidence gathering, generally with systematic reviews and RCTs at the top.	With reference to principles of good practice, and practitioner and service user testimony. No hierarchy of evidence.	Identify promising interventions, based on a mix of evidence. Encourage trained practitioners to adapt interventions to their area, and gather comparable data on their experience.
How should you ‘scale up’ from evidence of best practice?	Introduce the same model in each area. Require fidelity, to administer the correct dosage, and allow you to measure its effectiveness using RCTs.	Tell stories based on your experience, and invite other people to learn from them.	A simple message to practitioners: if your practice is working, keep doing it; if it is working better elsewhere, consider learning from their experience.
What aim should you prioritise?	To ensure the correct administration of the same active ingredient.	To foster key principles, such as localism and respect for service user experiences.	To train then allow local practitioners to experiment and decide how best to turn evidence into practice.

Source: Adapted from Cairney (2017) and Cairney and Oliver (2017). Note that definitions of ‘implementation science’ and ‘improvement method’ are contested. The ideal-types could simply be called models 1–3.

For example, the choice to (try to) centralise policymaking is consistent with the choice to roll out uniform policy interventions driven by evidence from RCTs (Cairney and Oliver, 2017). Or, the choice to delegate policy routinely to local communities, service users and practitioners is consistent with the choice to value experiential knowledge shared personally via storytelling. Within these two extremes are many ways to combine evidence and governance, including compromise models to combine pragmatic forms of decentred governance with centrally-provided training. For example, the improvement method – advertised most highly in Scotland, but also encouraged by parts of the UK government – developed partly with reference to the limitations of a uniform approach built primarily on RCT evidence (Cairney and Oliver, 2017).

In a small number of cases, we can find empirical examples of projects that come close to each of the three models (see Cairney, 2017). The Family Nurse Partnership gained its reputation from US-based RCTs and wider international experience, before the UK and Scottish governments cooperated to roll out a uniform model (Model 1). My Home Life, introduced in England and then Scotland, combines governance principles (such as respectful conversations with service users) and storytelling to share evidence of good practice in care home settings (Model 2). The Early Years Collaborative (in Scotland) is an ambitious project to adapt the improvement method (used primarily in healthcare) to cross-cutting policy aims (Model 3).

However, more generally, policymakers do not make these choices consistently. Rather, we can see a large collection of ad hoc decisions to exercise power and use evidence in patchy ways. For example, UK and Scottish governments have used the phrase ‘let a thousand flowers bloom’ to symbolise a desire to entertain many models of policymaking, and to pursue an often contradictory commitment to local discretion when pursuing evidential approaches that seem to inhibit localism. The Scottish Government advertises improvement methods but entertains a mix of all three (Cairney, 2017). The UK Government style is harder to piece together, but key experiences like early years intervention are instructive (Cairney and St Denny, 2020). New Labour used international RCT evidence from the US’ Head Start to establish its Sure Start policy, but then encouraged local policymakers to experiment and redesign these approaches. The result was the inability to oblige ‘fidelity’ to a specific intervention. For proponents of Model 1, it removed any inability to evaluate UK government policy success because they no longer knew what its policy was (Rutter, 2006). The hierarchy of evidence informed initial problem definition but not policy delivery. More recently, the Conservative-led government’s ‘Troubled Families’ programme became the ultimate example of the projection of centralised government to provide political cover for decentralised policymaking, combined with a rather inconsistent and often-cynical use of research evidence (Cairney, 2019b). Overall, we can see that contradictory pressures, to make governance and evidence choices, contribute to a confusing mixture of strategies, in the context of a process that can never be ‘evidence based’ in the ways described by its strongest proponents.

Conclusion

This story of decentred policymaking warns against simplistic models invoking central government control over a linear policy process. There is no policy cycle used by a core group of comprehensively rational policymakers and analysts at the heart of government. Instead, there are many policymakers, spread across many levels and types of government, and pursuing their own ideas about how to combine good evidence and good policymaking. Policymakers act without knowledge of most information available to them, and they do so by employing cognitive and organisational shortcuts. No one is in control of this process: ministers inherit choices from their predecessors, delegate new choices to civil servants and pressure participants, delegate policy delivery to a vast landscape of organisations, and have to ignore almost all decisions taken in their name. In turn, the organisations and individuals that deliver policy can only fulfil a small proportion of central government demands, and often develop their own standard operating procedures and make policy as they deliver. In such complex policymaking systems, outcomes often seem to ‘emerge’ despite many central government attempts to control them.

Central government policymakers deal with such constraints in contradictory ways. They often delegate choices and foster new forms of accountability to reflect the limits to central control. They also try to hoard power or reassert central control, partly to respond to an electoral imperative associated with Westminster-style democratic accountability. The overall result is messy, with ministers letting go and intervening in policy in an ad hoc fashion. It is not described well by accounts that focus only on aspect of this dynamic (attempts to centralise some aspects of policymaking), and ignore the logical consequence (the need to ignore all other aspect of policymaking).

Although the UK and Scottish governments have reputations for more or less centralist approaches, both face similar dilemmas and address them in similar ways. Their choices reflect their need to appear to be in control while dealing with the fact that they are not. The UK Government often accentuates its top-down approach while actually encouraging a range of governance practices. The Scottish Government experience suggests that ministers face the same need to hold on and let go, even if they declare support for a bottom-up style of governance that contrasts with the caricature of the UK government’s top-down style.

These conclusions help us understand the ways in which UK and Scottish Governments try to encourage EBPM and make trade-offs between centralism and letting go. Both governments entertain three competing models of EBPM, as well as many variants combining these models in often-contradictory ways. The end-result is a ‘let a million flowers bloom’ mantra, which implies that governments choose to entertain methodological pluralism rather than engaging unwittingly in, or being powerless to halt, contradictory practices.

Overall, the idea of a powerful central state is perhaps attractive but its pursuit is counterproductive, since it prompts us to build hope for EBPM on false assumptions and simplistic solutions. The idea of the decentred state is less attractive but more useful. It prompts us to think, about the practices we seek to understand and influence, rather than assume that they are bad because they do not live up to an artificial ideal.