Sage Journals: Discover world-class research

Abstract

Policy failures are often assumed to be unintentional and anomalous events about which well-intentioned governments can learn why they occurred and how they can be corrected. These assumptions color many of the results from contemporary studies of policy learning which remain optimistic that ongoing policy problems can be resolved through technical learning and lesson drawing from comparative case studies. Government intentions may not be solely oriented toward the creation of public value and publics may not abide by government wishes, however, and studies of policy learning need to take these “darksides” of policy-making more seriously if the risks of policy failure are to be mitigated.

Keywords

policy learning policy failure policy volatility policy risk lesson-drawing

Introduction: Problems in the Study of Policy Learning as Subject and Process

As the Introduction to this special issue has highlighted, learning is a complex metaphor for the reflective and reflexive activity carried out by policy-makers and policy actors in interlinked policy subsystems and evolving governance contexts, one which also serves as a framework for analyzing this very same activity (Goyal & Howlett, 2019). As the Introduction underscores, there are many processual aspects to this kind of policy-related activity, as different actors evaluate policy quality through comparative synchronic and diachronic activity and vie to define “best practices” which governments can emulate and this activity is by no means automatic or unproblematic.

Much current work on policy learning, however, is underlain by the a priori assumption that policy-making activity results from well-intentioned efforts on the part of governments to address and resolve social, economic, environmental, and other kinds of problems in the pursuit of public value (Moore, 1994, 1995). This assumes that much policy-making, and its evaluation, is undertaken in a dispassionate, technical way in order to serve the public interest and, with respect to policy learning, that its aim is to learn the best way to improve policy and program delivery and avoid policy failures (Arestis & Kitromilides, 2010). Even those studies which highlight the political and power-based nature of policy narratives and target constructions typically still hold out the hope that policy learning can contribute to the adoption of more evidence-based policy solutions and improved outcomes emerging from policy processes (Feldman, 2018; Oliver & Pearce, 2017).

Historically, studies of policy learning have addressed three key questions: “who learns,” “learns what,” and “with what effect” (Bennett & Howlett, 1992). And one common way in which learning has been conceived, for example, has been to assume that “who learns” are government decision-makers, that “what they learn” is scientific or social scientific knowledge relevant to the solution to policy problems, and that “the effect of this learning” is better policies, that is, ones which are better able to solve the social problems they set out to resolve (Bennett, 1991; Rose, 1993; Schneider & Ingram, 1988; Tenbensel, 2004). This is, for example, the “rationalist” logic found or promoted in many studies of policy analysis and learning and of the behavior of the many professional analysts in government ostensibly dedicated to this goal (Meltsner, 1976). Most recently, for example, just such an approach can be found in many studies urging governments to utilize more “evidence-based” forms of policy-making in order to better design policy content and enhance policy outcomes (Nutley et al., 2007).

Many discussions of learning, however, by focussing exclusively on these processual aspects of policy-making, tend to ignore or take for granted what is the subject of learning that is, how policy success and failure are constituted and recognized. And these discussions also generally avoid the equally important question of “why learn,” often assuming, again a priori, that this is self-evident in a model of policy-making based on the ideas of problem-solving and continual improvement by elimination of common causes of failure (Lasswell, 1951).

Assuming well-intentioned policy actors and the goal of policy improvement, however, fails to address the possibilities that policy-makers are often driven by malicious or venal motivations rather than socially beneficial or disinterested ones (Feldman, 2018; Hoppe, 2018; Taylor, 2021) and that the goals of learning may extend far beyond consideration and identification of “best practices” (Legrand, 2018; Legrand & Jarvis, 2014). Most works on learning, for example, do not take into account the possibility that policy “success” in the eyes of some policy-makers or members of the public may involve promotion of partisan or clientelistic interests (Howlett & Rayner, 2021) or other more nefarious ends, from the marginalization of ethnic groups to corruption and the suppression of dissent. In this sense, many studies of policy learning can be seen to join those others in the policy orientation which focus mainly upon the “good” or ‘brightside” of policy-making, that is, studying cases of policy successes and failures in order to better inform and design more effective, efficient, and benevolent policies in any given problem context (Douglas et al., 2021), rather than the “darkside” of more self-interested or malicious ends or goals (Howlett, 2021a).

Working under the assumption of well-intentioned governments desiring to “do better” in addressing ongoing social and other problems is a noble ambition but presents a benevolent picture of policy-making that not only ignores issues around policy content, but also says little about other kinds of common policy-making behavior such as, for example, “sweeping problems under the rug,” doing nothing or addressing serious issues in a purely symbolic or performative way (McConnell, 2018, 2020; McConnell & t‘Hart, 2019). Studies of policy learning also generally ignore the lessons of the large literature on knowledge utilization in government which generally downplays the impact of scientific and social science research on government decision-makers and decision-making, making the assumption of the simple adoption of “best practices” highly problematic (Landry et al., 2003; Oh & Rich, 1996; Radaelli, 1995; Rich, 1981, 1997; Shulock, 1999; Weiss, 1977, 1992; Weiss & Bucuvalas, 1980).

This is not to say that studies of policy learning only study policy successes, since many studies do address failures, and perhaps overly so (Compton et al., 2019). However, it is the case that these studies tend to address failures in a way which still often underestimates the potential for malevolent intentions, suggesting that whatever learning is being identified in such works is of a very limited type (Caplan, 1976; Feldman, 1989; Whiteman, 1985). That is, despite the plethora of work on policy learning in the policy studies field over the last several decades, both the independent variables—what factors drive or contribute to learning and non-learning processes—and the dependent variable itself—what constitutes learning and how it can be discerned—remain unclear and often mis- or under-specified in many studies on the subject (Goyal & Howlett, 2019; Radaelli, 2009).

As a result, despite a large literature on the subject, why learning is undertaken, what is learned and what kinds of learning contribute to enhanced policy outcomes or “policy success” and under what circumstances remain murky. This has led to some confusion and especially some difficulties in compiling the results of various studies of policy learning into a coherent and consistent set of principles that can be applied to specific problem areas in order to develop better policies or at least better understand the risks involved with certain policy designs over others (Goyal & Howlett, 2019; Moyson et al., 2017).

Re-evaluating the existing literature on these two subjects—policy learning and policy success and failure—thus promises to help move the discussion of both subjects forward by re-examining and clarifying the logic and rationales behind each and in this way better aid the process of risk mitigation in policy-making and policy designs.

(Re)Thinking the Role(s) of Policy Success and Failure in Policy Learning

Policy success and failure are subjects which have pre-occupied both policy scholars and practitioners for decades (Kerr, 1976; McConnell, 2020). These studies, unfortunately, have suffered from a wealth of incompatible terms and concepts used to ascertain success or failure, from attainment of original government objectives to normative arguments, to a focus on results or an emphasis on continued support by key policy actors and stakeholders, which has greatly diminished their cumulative impact.

McConnell (2010b), for example, usefully listed nine commonly-used criteria for defining and assessing success and failure found in the policy literature (see Table 1).

Table 1.

Criteria for Policy Success and Failure (after McConnell).

Basis of claim	Claim of success	Claim of failure
Original objectives	Achieved	Not achieved
Target group Impact	Positive impact	Negative impact
Results	Problem improvement	Problem worsening
Significance	Important to act	Failing to Act
Source of support/opposition	Key groups support	Key groups oppose
Jurisdictional comparisons	Best practice or superior performance	Someone is doing this better elsewhere
Balance sheet	High benefits	High costs
Level of innovation	New changes	Old response
Normative stance	Right thing to do	Wrong thing to do

Source. Adapted from McConnell (2010b): 106, 108).

Although disparate, in each of the nine approaches success and failure can be seen to be viewed as mutually exclusive and zero sum. That is, for example, to the extent that the original objectives of a policy are achieved, a policy may be termed a success. However, to the extent that they are not, it is a failure. Such approaches, then, suggest that the lessons of policy success and those of failure are reciprocals, and that learning about either activity can contribute to the common goal of improving a policy, either by enhancing success or limiting failure.

These approaches, however, are not very helpful when it comes to helping determine what should be done in order to bolster the odds of success, or shed any light at all on why a government would want to do so or if, in the larger scheme of things, from an ethical or philosophical position a success is desirable. And the notion of mutual exclusivity and the reciprocal nature of success and failure, while logical, is only one way these relationships can be conceived.

Re-thinking the nature of policy success

Success in this latter sense, for example, is a very problematic concept and judgment if it involves malign intent. Can a well-executed policy of side-lining or eliminating political dissent, for example, be considered a “success”? Or a well-run genocidal campaign? What lessons are to be derived from such experiences? Would deriving lessons from failed genocidal campaigns which improve future ones be considered “learning”?

And the general problem of poor or self-interested behavior interfering or undermining efforts to promote the public good (or “maliciousness”), of course, is one of the oldest in political study (Saxonhouse, 2015). It has several aspects which are relevant to policy learning but often ignored or downplayed in thinking and writing about policy success and failure (McConnell et al., 2020). Examples of malicious policies are legion and range from the use of public authority to promote the interests of ethnic, religious, and other favored groups or specific sets of “clients” (Gans-Morse et al., 2014; Marie Goetz, 2007) or penalize or punish others (Howlett et al., 2017), its (mis)use to enrich or otherwise benefit policy-makers and administrators (Uribe, 2014), and its use to manipulate a variety of activities of target groups through, for example, vote-buying or other forms of electoral pandering (Brancati, 2014; Manor, 2013).

Although omnipresent in popular accounts and traditional and social media visions of policy-making, these activities are generally absent from standard textbooks and other works on policy learning (Anderson, 1975; Howlett et al., 2020; Weimer & Vining, 1989). The idea that policy-makers are often driven by these kind of malicious or venal motivations rather than socially beneficial or disinterested ones and that policy targets have proclivities toward activities such as gaming, free-ridership, and rent-seeking rather than simply and obediently complying with government intentions is well established in the field of policy studies and in other disciplines, but not in those around learning (Cappella & Jamieson, 1996; Dahlström et al., 2013; Howlett, 2020; Legrand & Jarvis, 2014; Taylor, 2021).

But eliminating or managing such activities should be a subject of learning even if this is only defined narrowly to mean learning how to advance public welfare, as it is currently. Studies of learning should not ignore questions around these kinds of behaviors and how to manage or mitigate them, but should address them head-on.

And there is an ample literature in other fields, such as public management and risk analysis, which can be used for this purpose. Some of the better-known policy tools that can help offset or manage risk and volatility discussed in this literature are set out in Table 2 below. Better management of policy processes along these lines to avoid or restrain malevolent behaviors and results is key if these problems are to be avoided. As such, learning studies should focus on them just as much as on the kinds of “second-degree” lessons about tweaking policy instruments and programs in order to learn from “mistakes” which are often derived by the learning literature from studies of program failures.

Table 2.

Sources of Policy Malignancy and Design Alternatives.

Tool type	Flaw	Examples	Procedural Tools Solutions
Organization	Agency failure or co-optation	CorruptionContracting and procurement IssuesPerverse co-production/clientelism	Accountability/transparency rulesAnti-corruption bureausBlockchain bidding and procurement/contracting recordsFraud squads
Authority	Corruption, favoritism, and gaming	Capture and venue-shoppingBribes and kick-backsExtortion	Whistleblower lawsSunset lawsMerit appointmentsConflict of interest laws
Treasure	Fraud and gaming	MisrepresentationFalse claimsSwindlesCheating	Complete contracts,Careful subsidy/tax designMonitoringverification/enforcement/inspections
Nodality	Biased information	PropagandaSlanderImproper secrecy	More carefully targeted messagesTruth in advertising lawsLibel and slander lawsAccess to information laws

That is, better appreciation of the entire range of diverse possible policy goals is needed here, to accompany the precise analysis of program attributes or outcomes. In other words, improving policies requires the accumulation of “deeper” policy learning linked to a more profound understanding of the institutional and policy processes which lead to self-interested policies and their mitigation strategies, rather than only “shallow” program-oriented learning focusing on the technical program characteristics of a policy, per se, which may have contributed to its success or failure (Banks, 1995).

Re-thinking the nature of policy failure

As pointed out above, programmatic failures, like successes, are also common sources of studies of policy learning and lesson -drawing. While attention has been paid in the literature to such failures, however, it has not been uniform and, as with policy success, requires more nuance in their study and in the kinds of lessons which can be drawn from failures than is typically the case.

Of course, to begin with, program failures are only one kind of policy failure which, as McConnell (2010a and 2015) has noted, may also fail on the political front or simply fail to resolve a problem. An exclusive focus on program elements or components further reinforces the tendency in the literature to highlight technical issue and “shallow” learning.

Further, like success, failures also differ significantly in extent and magnitude. They can be large or small, long-term or short-term, for example, and some may be highly visible while others may be discernible only to experts. The kinds of lessons which might be drawn from one type of failure may not be the same as from another kind and the learning literature needs to take this into account.

The central characteristics of any such failures are set out in Table 3 (Howlett, 2012). As the table shows, failures vary along several key dimensions, from extent to visibility, duration, and intensity but also in terms of intentionality or avoidability.

Table 3.

Six Dimensions of Policy Failures.

Attribute	Range
Extent (size)	From large (regime) to small (event)
Avoidability	From low to high
Visibility	From low to high
Intentionality	From low to high
Duration	From long to short
Intensity	From low to high

Source. Howlett (2012).

Combining several of these elements together, and focussing on their magnitude and salience, it is possible to arrive at concise taxonomy of failure types (see Table 4) which allows distinctions to be made between major and minor failures, and focussed and diffuse ones, clarifying the dependent variable in studies of both failures and considerations of the kinds of learning which might emerge from or around them.

Table 4.

Principal Types of Policy Failures by Magnitude and Salience.

		Magnitude (extent and duration)
		High	Low
Salience (intensity and visibility)	High	Type I: Major failure e.g., Climate change (international treaty) policy failure	Type II: Focused failure e.g., Sports crowd control (riots) policy failure
Salience (intensity and visibility)	Low	Type III: Diffuse failure e.g. Anti-poverty policy failure	Type IV: Minor failure e.g., Charities (fraud) policy

Source. Howlett (2012).

This is important because examining and learning from high magnitude/high salience failures, for example, is likely to involve different lessons from ones which have limited extent, duration, intensity and visibility.

Similarly, these kinds of failures may originate in different kinds of government activity, as McConnell (2010a and 2010b) noted. Searching for the causes of these different kinds of failures should note not just their politics, process and problem orientation but also that these kinds of problems may originate at different points in the policy process (see Table 5). That is some failures originate immediately in the agenda-setting process while others emerge later due to problems encountered in formulation or decision-making, while others are linked to implementation or evaluation (Howlett et al., 2020) (See Table 5).

Table 5.

Policy Failures by Stage of the Policy Cycle.

Policy stage	Example of Source of Policy Failure
Agenda-setting	Over-reaching governments establishing or agreeing to establish over-burdened or unattainable policy agendas.
Policy formulation	Attempting to deal with problems without investigating or researching problem causes and identifying the probable effects of policy alternatives.
Decision-making	Failing to decide on a policy within a reasonable period of time or distorting its intent through bargaining and log-rolling.
Policy implementation	Failing to deal with implementation problems including lack of resources, principle-agent problems, oversight failures, and others.
Policy evaluation	Lack of learning due to lack of, ineffective or inappropriate policy monitoring and/or feedback processes and structures.

As Table 5 shows, this means some policies can fail “early on” in the policy process if and when governments “bite off more than they can chew” or take on too many issues beyond their capacity to deal with (Howlett et al., 2017). Others emerge later, including the failure to draw any or appropriate lessons from policy evaluations. More clearly distinguishing different levels or gradations of failure and locating their likely origins is crucial to understanding the role learning can play in helping avoid or mitigating many common sources of policy failure, again something which is typically not done in contemporary studies of policy learning.

The Need to Learn More about Policy Volatility: Dealing With the Inherent Vices of Public Policies

Both these problematic aspects of the depictions of policy success and failures used in the contemporary literature on policy learning point to the need to re-conceive the fundamental nature of the problems governments encounter in formulating and implementing policies, and of observers to evaluate them, in order that simplistic conceptions of the lessons which can be drawn about how to improve policies can be avoided and a deeper level of knowledge and insight developed in this field.

Learning in this sense is better conceived as focussing on what Ching and Howlett (2021b) have defined as “policy volatility” and the factors which condition the propensity of policies to go astray either right at the start in terms of the orientation of governments toward themselves and their publics, or in terms of their capability and capacity to (Wu et al., 2015) to grasp and incorporate into their policy-making deliberations and designs the “inherent vices” of policy-making which can affect even the most well-intentioned government effort (Howlett & Leong, 2022).

These “vices” are aspects of the fundamental nature of policy-making which are “inherent” or built into policies, programs and processes which governments must, but often do not, adequately address in policy designs and deliberations. They include “unpreparedness,” a policy vice or common problem which events such as the 2020 to 2022 coronavirus has underscored (Capano et al., 2020). That is, as t’Hart et al. (2001) noted, variations in the timing and nature of governments interventions in crises vary according to the nature of the crisis: rapid and severe crises demand more from leaders and systems than their “slow onset” counterparts and highlight the need for pre-crisis planning and preparations in the way the latter do not (Kehinde, 2014; Staupe-Delgado, 2019).

Preparedness, of course, is often linked to previous experience with a problem but unpreparedness can also occur even when this previous experience exists. In the case of the responses to the COVID-19 crisis, for example, the key determinants of government’ initial responses to the pandemic typically related to their existing capabilities, which in turn were related to their preparation and planning for such pandemics and the managerial and organizational resources they had at their disposal when first encountering the virus (Capano et al., 2020; see also McConnell & Drennan, 2006). But even some countries with previous experience with diseases like MERS or SARS were caught unprepared for the Coronavirus.

That is, while there is no doubt that learning from past experiences can play a key role here in overcoming this source of failure, it does not guarantee success. Governments which were prepared for pandemics and had recent similar past experience, such as those in Asia which had dealt with SARS-CoV-1, H1N1, and MERS, for example, were quite likely to have been prudent and have a realistic level of confidence in the relative capability of their existing public health and financial systems to handle new communicable diseases. They were well-informed about their actual capabilities and also of the potentially very dangerous nature of the disease. This led them to be wary of the disease and, ceteris paribus, pushed them toward relatively early, slow but steady and strong responses in terms of rolling out elements of the standard portfolio of policy tools listed above. Countries which were both unprepared and had no recent relevant past experience were often complacent initially and then shocked as the true nature of the pandemic and their lack of preparedness was revealed. But important exceptions to this general situation in countries such as the US or Brazil highlight the important role played by other factors than experience and capacity in these responses, from poor leadership to recalcitrant and vaxxine averse populations, which also affected state responses and outcomes (Howlett, 2021b).

A second inherent vice, which manifests itself most clearly at the formulation stage of policy-making is “uncertainty.” Policy-makers face many different kinds of uncertainties (Nair & Howlett, 2017). Uncertainties surround the choice of policy options, their consequences, confidence in available information and the uncertain values of multiple stakeholders including decision-makers. When these are not well understood this leads to a great deal of ambiguity concerning what might be the correct action to follow in many cases, allowing plentiful opportunities for self-interested interventions and poorly designed interventions and making the derivation of what lessons to follow or can be drawn from such circumstances highly problematic (Knight, 1921; Ove Hansson, 1996; van der Sluijs, 2005). The types of uncertainty common in policy-making include:

Substantive uncertainty that relates to lack of relevant information related to the nature of the complex problem, and the different interpretations of information arising from different “frames of reference” of the social actors;

Strategic uncertainty that arises due to unpredictability of strategies deployed by different actors based on their perception of the problem and strategies likely to be deployed by other actors, and

Institutional uncertainty that arises owing to the complexity of interaction of different actors guided by institutional frameworks i.e. rules and procedures of the organizations they represent (Koppenjan & Klijn, 2004).

Uncertainty has been widely studied in diverse disciplines from psychology to organization theory and studies in these fields have had some impact on policy analysis and policy-making but very little on policy learning (Manski, 2011; Morgan & Henrion, 1990). In the policy world, much of this discussion has centered on the nature of what Simon (1973) termed “ill-structured problems” or ones in which the nature of policy problems and solutions are unknown, poorly defined or contested. The concepts of “wicked” and “tame” problems, which for decades have dominated thinking around uncertainty in the policy sciences are examples of this distinction (Alford & Head, 2017; Churchman, 1967; Levin et al., 2012; Rittel & Webber, 1973) and the lessons that can be drawn about these kinds of problems are not clear, nor have they been a major subject of interest in the field.

“Maliciousness” has already been described above and is a vice which affects decision-making when the criteria undergirding policy choice are not benevolent and public-spirited. Studies of policy learning have not yet dealt with it in any serious kind of way.

“Non-Compliance” is a fourth ‘vice’ which affects implementation in particular. Implementation is by no means straightforward and involves a plethora of problems related to administrative behavior and issues linked to problems such as the complex principle-agent chains policies often involved in translating policy intentions into action (Ellig & Lavoie, 1995). These compliance problems are fundamental in implementation and need to be incorporated into learning studies but are not yet a major subject of concern. Policy “takers” or targets, for example, often fail to comply with government wishes, even when these are expressed or enacted with the best of intentions. While such behavior is an essential component of studies in cognate fields such as law and accounting (Doig et al., 2001; Howlett, 2020 and 2021a, 2021b; Kuhn & Siciliani, 2013), it is glossed over almost completely in studies of public policy. Yet this is a key question related to better understanding the conditions of policy success and failure and the kinds of designs and activities more likely to attain success with minimal effort and expenditure (Feeley, 1970; Mulford, 1978; Schneider & Ingram, 1990a, 1990b)

The fifth vice, “Non-Learning” is the subject of this essay. It is an outcome of policy appraisals, studies, and evaluations which either do not accurately derive appropriate lessons or which “learn the wrong lessons” (Dunlop, 2017a, 2017b; Dunlop & Radaelli, 2018a, 2018b). It too needs to be more seriously addressed by studies of learning.

Such vices—“unpreparedness,” “uncertainty,” “maliciousness,” “non-compliance,” and “non-learning”—can also usefully be organized according to the stage of policy-making they most affect (Howlett, 2000; Lang, 2019) (see Table 6). Each is an potentially important source of policy failure which injects volatility into policy-making and, as such, each should properly be well within the purview of studies of policy learning (Howlett & Mukherjee, 2019 and 2017).

Table 6.

The Inherent Vices of Public Policy.

	Policy stage/task	Central policy risk	Cause/source
Sources of policy volatility	Agenda-setting	Unpreparedness (surprise)	Lack of attention
	Policy formulation	Uncertainty (wickedness)	Lack of knowledge
	Decision-making	Maliciousness (poor decisions)	Self-interestedness
	Policy implementation	Non-compliance (misaligned target behavior)	Unknown behavioral mechanisms
	Policy evaluation	Non-learning (poor learning)	Unknown intervention effects

As noted above, other fields have also encountered these problems and developed ideas and mechanisms expected to be able to identfy these risks and mitigate or otherwise deal with them. This work has shown that each of these risks can be mitigated or managed through a variety of means—from institutionalizing foresight agencies to deal with the risk of surprises affecting government agendas, to evaluation and measurement activities to reduce the risk of poor or non-learning in policy evaluation—if not always completely eliminated (see Table 7)

Table 7.

Potential Policy Risk Management Strategies About Which More Learning is Needed.

	Policy stage/task	Central policy risk	Cause/source	Management strategy (procedural tools)	Result
Sources of policy volatility	Agenda-setting	Unpreparedness (surprise)	Lack of attention	Institutionalized foresight/issue Management	Reduced surprise
	Policy formulation	Uncertainty (wickedness)	Lack of knowledge	Institutionalized policy analysis/risk management/modeling	Reduced ignorance
	Decision-making	Maliciousness (poor decisions)	Self-interestedness	Institutionalized evidence-based policy-making	Reduced opportunity for political or personal gain from poor decisions
	Policy implementation	Non-compliance (misaligned target behavior)	Unknown behavioral mechanisms	Institutionalized policy design/ policy labs	Reduced non-compliance
	Policy evaluation	Non-learning (poor learning)	Unknown intervention effects	Institutionalized evaluation and measurement	Separates signal from noise

Strategies for better policy-making need to better understand all of these kinds risks, and policy designs need to be prepared to deal with them. And a key role in ensuring this happens should, but does not yet, fall to studies of policy learning.

Thus, for example, accounts of the actions of bureaucrats and other implementers in dealing with non-compliance are often noted in the policy literature, but the lessons drawn from much research in this area, for example, is often just to “suggest that the only real issue in policy compliance is one of correctly calibrating incentives and disincentives” (Howlett, 2020). However, as Howlett (2020) has argued, “this not only ignores aspects involved in the social and political construction of targets (Schneider & Ingram, 1990a, 1990b), but also minimizes the complex behaviors which go into compliance – from levels of trust to other social and individual behavioral characteristics including the operation of a wide variety of descriptive and injunctive social norms (Bamberg & Möser, 2007; Howlett, 2019; Thomas et al., 2016).” Even the most basic activities of governing such as collecting taxes and ensuring laws and rules are obeyed involves considerations on the part of targets and the public of issues such as the legality and normative “appropriateness” of government’s levying and collecting such taxes or passing and enforcing such rules and learning studies should address these issues around “appropriateness” at least as much as they address those related to calculations of ‘consequence’ (March & Olsen, 2004).

Similarly, policy-takers are often viewed as targets who do not attempt, or at least do not attempt very hard, to evade policies or, in some cases, try to profit from them (Braithwaite, 2003; Howlett, 2019; Marion & Muehlegger, 2007). Such activities on the part of policy takers, however, as Howlett (2020) has argued “are key in determining the success of various government initiatives ranging from tobacco control to bus fare evasion” (Delbosc & Currie, 2016; Kulick et al., 2016). Such behaviors need to be “designed for” in the sense that determined non-compliance and gaming should be taken into account in designing policies, along with many other such behaviors, such as free-ridership, fraud, and misrepresentation (Harring, 2016), and studies of policy learning need to address this gap and help in the design of risk mitigation strategies towards them or at least be able to better specify when such risks are higher or lower.

Conclusion: Learning about the Better Management of Policy Risks

As this discussion has noted, learning to properly diagnose policy risks and the volatility they bring to policy-making and policy outcomes is crucial to understanding policy success and faillure and drawing meaningful lessons about effective policy and program risks, design and operation.

However, as the discussion above has also shown, there are many problems in policy-making that affect policy outcomes which should serve as subjects of studies of policy learning but currently do not (McConnell, 2010b). Studies of policy-learning to date have focused almost exclusively on activities which take place under assumptions of the “right” design conditions in which policy makers and policy takers are expected to act in good faith in the pursuit of the public interest. Or, put another way, most studies try to draw lessons for policy-making about only the programmatic aspects of policies under the assumption that “positive” policy spaces allow policy processes to be driven by knowledge and good intentions, and where high levels of government legitimacy mean a high likelihood that the resulting policies will be obeyed by their targets among the population (Howlett, 2021a; Howlett & Mukherjee, 2017). In such situations of “optimal” design spaces (Chindarkar et al., 2017), policy learning is indeed all about ensuring best knowledge or present-day evidence is marshaled toward developing programs with a high level of confidence that these will “work”: that is, that they will be effective in altering target behavior in a manner which complies with government wishes and that these expectations will be reached in the manner anticipated (Peters et al., 2018). This kind of technical learning and programs and policy solutions is useful but needs to be supplemented with higher level learning about the politics of policy-making and the nature of the problems with which policy-making is faced if policy success and failure are to be fully understood along with their policy consequences (McConnell, 2010a and 2010b).

All of the policy-making vices listed above need to be studies more intently and not ignored or assumed away. They can be thought of as “policy risks” affecting the program, political and problem dimensions of policy-making, ranging from unpreparedness in agenda-setting to non-learning in policy evaluation, and include poor decision-making, policy implementation as well as problems in policy formulation (Howlett, 2012) and all are properly subjects of studies of policy learning. Better understanding these risks, which are the sources of failure, and also often serve as barriers to policy success, is important to learning useful lessons from policy studies and evaluations. Incorporating studies of these phenomena into policy learning is crucial to the development of measures to correct, offset, or at least mitigate, such risks. Here it is important to note that policy success and failure are fundamentally different and not simply the reciprocals of each other. Thus the conditions of policy success and failure should be categories of interest in learning scholarship in themselces and not subsumed under eachother.

This is not to say that studies of policy learning have not dealt with aspects of this issue in the past. Dunlop, for example, has very usefully distinguished between non-learning—the failure to learn anything—or negative learning—learning the wrong lessons (Dunlop, 2017a, 2017b), which can impact on the likely success or failure of policy-making, be it programmatic, political, or problem-related. But most studies of learning have not done so and, in general, have failed to take into account the nature of the many ‘inherent vices” of policy-making which introduce volatility into policy-making. This makes understanding and deriving lessons from policy success inherently more difficult than often considered and calls for more nuanced and detailed understandings of it and its relationship to the sources of policy failure and the means to overcome them (Howlett & Leong, 2022).

As this article has argue, moving in this direction requires re-visiting the existent literature on policy failures and successes and their relationship together. Students of policy learning need to take observations and concerns about the “darksides” of policy-making more seriously if they are to help develop thinking and practices which help offset or control it (Howlett, 2021a; McConnell, 2018 and 2020). And recent efforts to more systematically describe and analyze the self-interested and strategic use of knowledge and its deployment in the pursuit of other ends than public value (Moore, 1994, 1995), for example, provide insight into how this can be done.

The development of a new terminology around such concepts as “malign” or self-interested public policy (Legrand & Jarvis, 2014), to give only one example, has gone some distance in this regard. Borrowing concepts from other industries and social activities, such as the insurance industry, in conceiving of activities such as policy-taker non-compliance and policy-maker maliciousness as the “inherent vices” of policy-making (Ching & Howlett, 2021), such studies argue that those aspects of policy-making which lead to policy failures should be the subject of closer and better analysis and inquiry, just as much as instances where government and policy actor motivations may be more benign (Ching & Howlett, 2021; Howlett, 2020, 2021a).

Incorporating such considerations into policy-making, policy management and policy designs promises to advance the study of policy-making, and the role of policy learning within it, by better clarifying how lessons from both failures and successes can be made more relevant and practical to policy-makers.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research,authorship,and/or publication of this article.

Funding

The author(s) received no financial support for the research,authorship,and/or publication of this article.

ORCID iD

Michael Howlett

Author Biographies

Ching Leong is Vice-Provost and Associate Professor in the Lee Kuan Yew School of Public Policy at the National University of Singapore. Her work lies in making sense of apparently irrational environmental behavior,whether in refusal to use recycled water,underinvesting in water utilities,or decision making in building dams and managing rivers. Her field research is focused on water institutions and governance in Asia.

Michael Howlett,FRSC is Burnaby Mountain Professor and Canada Research Chair (Tier 1) in the Department of Political Science at Simon Fraser University in Vancouver,BC,Canada. He specializes in public policy analysis,political economy,and resource and environmental policy. His most recent books are Dictionary of Public Policy;(2022),Policy Consultancy in Comparative Perspective;(2020) and Designing for Policy Effectiveness: Defining and Understanding a Concept (2018).

References

Alford

Head

B. W.

(2017). Wicked and less wicked problems: A typology and a contingency framework. Policy and Society, 36(3), 397–413.

Anderson

J. E.

(1975). Public policymaking. Praeger.

Arestis

Kitromilides

(2010). What economists should know about public policymaking? International Journal of Public Policy, 6(1/2), 136–153.

Bamberg

Möser

(2007). Twenty Years after Hines, Hungerford, and Tomera: A new meta-analysis of psycho-social determinants of pro-environmental behaviour. Journal of Environmental Psychology, 27(1), 14–25.

Banks

J. S.

(1995). The design of institutions. In Weimer

D. L.

(Ed.), Institutional design (pp. 17–36). Kluwer.

Bennett

C. J.

(1991). How states utilize foreign evidence. Journal of Public Policy, 11(1), 31–54.

Bennett

C. J.

Howlett

(1992). The lessons of learning: Reconciling theories of policy learning and policy change. Policy Sciences, 25(3), 275–294.

Braithwaite

V. A.

(Ed.). (2003). Taxing democracy: Understanding Tax avoidance and evasion. Ashgate Pub Ltd.

Brancati

(2014). Democratic authoritarianism: Origins and effects. Annual Review of Political Science, 17(1), 313–326.

10.

Capano

Howlett

Jarvis

D. S. L.

Ramesh

Goyal

(2020). Mobilizing policy (in)capacity to fight COVID-19: Understanding variations in state responses. Policy and Society, 39(3), 285–308.

11.

Caplan

(1976). Factors associated with knowledge use among federal executives. Policy Studies Journal, 4(3), 229–234.

12.

Cappella

J. N.

Jamieson

K. H.

(1996). News frames, political cynicism, and media cynicism. The Annals of the American Academy of Political and Social Science, 546, 71–84.

13.

Chindarkar

Howlett

Ramesh

(2017). Conceptualizing effective social policy design: Design spaces and capacity challenges. Public Administration and Development, 37(1), 3–14.

14.

Churchman

C. W.

(1967). Wicked problems. Management Science, 14(4), B141–B142.

15.

Compton

M. E.

Luetjens

Hart

(2019). Designing for policy success. International Review of Public Policy, 1(2), 119–146.

16.

Dahlström

Lindvall

Rothstein

(2013). Corruption, bureaucratic failure and social policy priorities. Political Studies, 61, 523–542.

17.

Delbosc

Currie

(2016). Four types of fare evasion: A qualitative study from Melbourne, Australia. Transportation Research Part F: Traffic Psychology and Behaviour, 43, 254–264.

18.

Doig

Johnson

Levi

(2001). New public management, old populism and the policing of fraud. Public Policy and Administration, 16(1), 91–113.

19.

Douglas

Schillemans

t’ Hart

Ansell

Bøgh Andersen

Flinders

Head

Moynihan

Nabatchi

O’Flynn

Peters

B. G.

Raadschelders

Sancino

Sørensen

Torfing

(2021). Rising to Ostrom’s challenge: An invitation to walk on the bright side of public governance and public service. Policy Design and Practice, 4, 441–451.

20.

Dunlop

C. A.

(2017a). Pathologies of policy learning: What are they and how do they contribute to policy failure? Policy and Politics, 45(1), 19–37.

21.

Dunlop

C. A.

(2017b). Policy learning and policy failure: Definitions, dimensions and intersections. Policy and Politics, 45(1), 3–18.

22.

Dunlop

C. A.

Radaelli

C. M.

(Eds.). (2018a). Learning in public policy: Analysis, modes and outcomes. Claire A. Dunlop, Palgrave Macmillan.

23.

Dunlop

C. A.

Radaelli

C. M.

(2018b). The lessons of policy learning: Types, triggers, hindrances and pathologies. Policy and Politics, 46(2), 255–272.

24.

Ellig

Lavoie

(1995). The principle-agent relationship in organizations. In Foss

(Ed.), Economic approaches to organizations and institutions : An introduction (pp. 267–295). Dartmouth.

25.

Feeley

(1970). Coercion and compliance: A new look at an old problem. Law & Society Review, 4(4), 505–519.

26.

Feldman

(1989). Order without design: Information production and policy making (1st ed.). Stanford University Press.

27.

Feldman

(2018). Law good people challenging states ability regulate human behavior. Cambridge University Press.

28.

Gans-Morse

Mazzuca

Nichter

(2014). Varieties of clientelism: Machine politics during elections. American Journal of Political Science, 58(2), 415–432.

29.

Goyal

Howlett

(2019). Framework or metaphor? Analysing the status of policy learning in the policy sciences. Journal of Asian Public Policy, 12(3), 257–273.

30.

Harring

(2016). Reward or punish? Understanding preferences toward economic or regulatory instruments in a cross-national perspective. Political Studies, 64, 573–592.

31.

Hoppe

(2018). Heuristics for practitioners of policy design: Rules-of-thumb for structuring unstructured problems. Public Policy and Administration, 33(4), 384–408.

32.

Howlett

(2000). Managing the ‘Hollow State’: Procedural policy instruments and modern governance. Canadian Public Administration, 43(4), 412–431.

33.

Howlett

(2012). The lessons of failure: Learning and blame avoidance in public policy-making. International Political Science Review, 33(5), 539–555.

34.

Howlett

(2019). Designing public policies: Principles and instruments (2nd ed.). Routledge.

35.

Howlett

(2020). Dealing with the dark side of policy-making: Managing behavioural risk and volatility in policy designs. Journal of Comparative Policy Analysis Research and Practice, 22(6), 612–625.

36.

Howlett

(2021a). Avoiding a panglossian policy science: The need to deal with the darkside of policy-maker and policy-taker behaviour. Public Integrity, forthcoming, 1–13.

37.

Howlett

(2021b). From national lockdowns to Herd immunity: Understanding the spectrum of government responses to COVID-19 (2019–2021). Bestuurskunde, 30(3), 8–23.

38.

Howlett

Ramesh

X. U. N.

Fritzen

(2017). The public policy primer. Taylor & Francis.

39.

Howlett

Leong

(2022). The ‘Inherent Vices’ of policy design: Uncertainty, maliciousness and non-compliance. Risk Analysis. forthcoming.

40.

Howlett

Mukherjee

(2017). Handbook of policy formulation. Edward Elgar.

41.

Howlett

Mukherjee

(2019). Handbook of policy design. Routledge.

42.

Howlett

Ramesh

Perl

(2020). Studying public policy: Principles and processes (4th ed.). Oxford University Press.

43.

Howlett

Rayner

(2021). Administrative clientelism and policy reform failure: The Western Canada integrated Land management experience 1990–2015. Acta Politica, 56, 622–638.

44.

Kehinde

(2014, January 1). Applicability of risk transfer tools to manage loss and damage from slow-onset climatic risks [Conference session]. Procedia Economics and Finance, 4th International Conference on Building Resilience, Incorporating the 3rd Annual Conference of the ANDROID Disaster Resilience Network, 8–11 September 2014, Salford Quays, United Kingdom (pp. 710–717).

45.

Kerr

D. H.

(1976). The logic of ?policy? And successful policies. Policy Sciences, 7(3), 351–363.

46.

Knight

(1921). Risk uncertainty and profit (illustrat ed.). Martino Fine Books.

47.

Koppenjan

Klijn

E. H.

(2004). Managing uncertainties in networks: A network approach to problem solving and decision making. Routledge.

48.

Kuhn

Siciliani

(2013). Manipulation and auditing of public sector contracts. European Journal of Political Economy, 32, 251–267.

49.

Kulick

Prieger

Kleiman

M. A. R.

(2016). Unintended consequences of cigarette prohibition, regulation, and taxation. International Journal of Law Crime and Justice, 46, 69–85.

50.

Landry

Lamari

Amara

(2003). The extent and determinants of the utilization of university research in government agencies. Public Administration Review, 63(2), 192–205.

51.

Lang

(2019). Collaborative governance in health and technology policy: The use and effects of procedural policy instruments. Administration & Society, 51(2), 272–298.

52.

Lasswell

H. D.

(1951). The policy orientation. In Lerner

Lasswell

H. D.

(Eds.), The policy sciences: Recent developments in scope and method (pp. 3–15). Stanford University Press.

53.

Legrand

(2018). Structure, agency and policy learning: Australia’s multinational corporations dilemma. In Dunlop

C. A.

Radaelli

C. M.

Trein

(Eds.), Learning in public policy: Analysis, modes and outcomes (pp. 215–241). Springer International Publishing.

54.

Legrand

Jarvis

(2014). Enemies of the state: Proscription powers and their use in the United Kingdom. British Politics, 9(4), 450–471.

55.

Levin

Cashore

Bernstein

Auld

(2012). Overcoming the tragedy of super wicked problems: Constraining our future selves to ameliorate global climate change. Policy Sciences, 45, 123–152.

56.

Manor

(2013). Post-Clientelist initiatives. In Stokke

Törnquist

(Eds.), Democratization in the global south (pp. 243–253). Palgrave Macmillan.

57.

Manski

C. F.

(2011). Policy analysis with incredible certitude. The Economic Journal, 121(554), F261–F289.

58.

March

J. G.

Olsen

J. P.

(2004). The logic of appropriateness (ARENA Working Paper).

59.

Marie Goetz

. (2007). Manoeuvring past clientelism: Institutions and incentives to generate constituencies in support of governance reforms. Commonwealth and Comparative Politics, 45(4), 403–424.

60.

Marion

Muehlegger

(2007). Measuring illegal activity and the effects of regulatory innovation: A study of diesel fuel tax evasion (Faculty Research Working Paper Series RWP07-026). John F. Kennedy School of Government.

61.

McConnell

(2010a). Policy success, policy failure and grey areas in-between. Journal of Public Policy, 30(3), 345–362.

62.

McConnell

(2010b). Understanding Policy Success: Rethinking Public Policy. Palgrave Macmillan.

63.

McConnell

(2015). What Is policy failure? A primer to help navigate the maze. Public Policy And Administration, 30, 221–242.

64.

McConnell

(2018). Hidden agendas: Shining a light on the dark side of public policy. Journal of European Public Policy, 25, 1739–1758.

65.

McConnell

(2020). The use of placebo policies to escape from policy traps. Journal of European Public Policy, 27(7), 957–976.

66.

McConnell

Drennan

(2006). Mission Impossible? Planning and preparing for crisis1. Journal of Contingencies and Crisis Management, 14(2), 59–70.

67.

McConnell

Grealy

Lea

(2020). Policy success for whom? A framework for analysis. Policy Sciences, 53, 589–608.

68.

McConnell

T’Hart

(2019). Inaction and public policy: Understanding why policymakers ‘Do Nothing.’. Policy Sciences, 52(4), 645–661.

69.

Meltsner

A. J.

(1976). Policy analysts in the bureaucracy. University of California Press.

70.

Moore

(1994). Public value as the focus of strategy. Australian Journal of Public Administration, 53(3), 296–303.

71.

Moore

M. H.

(1995). Creating public value: Strategic management in government. Harvard University Press.

72.

Morgan

M. G.

Henrion

(1990). Uncertainty: A guide to dealing with uncertainty in quantitative risk and policy analysis. Cambridge University Press.

73.

Moyson

Scholten

Weible

C. M.

(2017). Policy learning and policy change: Theorizing their relations from different perspectives. Policy and Society, 36(2), 161–177.

74.

Mulford

C. L.

(1978). Why they don’t even when they ought to: Implications of compliance theory for policymakers. In Etzioni

(Ed.), Policy research (pp. 47–62). E.J. Brill.

75.

Nair

Howlett

(2017). Policy myopia as a source of policy failure: Adaptation and policy learning under deep uncertainty. Policy and Politics, 45(1), 103–118.

76.

Nutley

S. M.

Walter

I. T. O.

Davies

(2007). Using evidence: How research Can inform public services. Policy Press.

77.

C. H.

Rich

R. F.

(1996). Explaining use of information in public policymaking. Knowledge and Policy, 9(1), 3–35.

78.

Oliver

Pearce

(2017). Three lessons from evidence-based medicine and policy: Increase transparency, balance inputs and understand power. Palgrave Communications, 3(1), 43.

79.

Ove Hansson

. (1996). Decision making under great uncertainty. Philosophy of the Social Sciences, 26(3), 369–386.

80.

Peters

B. G.

Ravinet

Howlett

Capano

Mukherjee

Chou

M. H.

(2018). Designing for policy effectiveness: Defining and understanding a concept. Elements Series. Cambridge University Press.

81.

Radaelli

C. M.

(1995). The role of knowledge in the policy process. Journal of European Public Policy, 2(2), 159–183.

82.

Radaelli

C. M.

(2009). Measuring policy learning: Regulatory impact assessment in Europe. Journal of European Public Policy, 16(8), 1145–1164.

83.

Rich

R. F.

(1981). Social science information and public policy making. Jossey-Bass Social and Behavioral Science Series, The NORC Series in Social Research. Jossey-Bass.

84.

Rich

R. F.

(1997). Measuring knowledge utilization: Processes and outcomes. Knowledge and Policy, 10(3), 11–24.

85.

Rittel

H. W. J.

Webber

M. M.

(1973). Dilemmas in a general theory of planning. Policy Sciences, 4, 155–169.

86.

Rose

(1993). Lesson-Drawing in public policy: A guide to learning across time and space. Chatham House Publishing.

87.

Saxonhouse

A. W.

(2015). Aristotle on the corruption of regimes: Resentment and justice. In Thornton

Thanassis

(Eds.), Aristotle’s politics: A critical guide (pp. 184–203). Cambridge University Press.

88.

Schneider

Ingram

(1988). Systematically pinching ideas: A comparative approach to policy design. Journal of Public Policy, 8(1), 61–80.

89.

Schneider

A. L.

Ingram

(1990a). Behavioral assumptions of policy tools. The Journal of Politics, 52(2), 510–529.

90.

Schneider

A. L.

Ingram

(1990b). Policy design: Elements, premises and strategies. In Nagel

S. S.

(Ed.), Policy theory and policy evaluation: Concepts, knowledge, causes and Norms (pp. 77–102). Greenwood.

91.

Shulock

(1999). The paradox of policy analysis: If it Is not used, why do we produce so much of it? Journal of Policy Analysis and Management, 18(2), 226–244.

92.

Simon

H. A.

(1973). The structure of ill structured problems. Artificial Intelligence, 4(3–4), 181–201.

93.

Staupe-Delgado

(2019). Progress, traditions and future directions in research on disasters involving slow-onset hazards. Disaster Prevention and Management An International Journal, 28(5), 623–635.

94.

Taylor

(2021). Public officials’ gaming of performance measures and targets: The nexus between motivation and opportunity. Public Performance & Management Review, 44(2), 272–293.

95.

Tenbensel

(2004). Does more evidence lead to better policy? The implications of explicit priority-setting in New Zealand’s health policy for evidence-based policy. Policy Studies, 25(3), 190–207.

96.

Thomas

A. S.

Milfont

T. L.

Gavin

M. C.

(2016). A new approach to identifying the drivers of regulation compliance using multivariate behavioural models. PLoS One, 11(10), e0163868.

97.

t’Hart

Heyse

Boin

(2001). New trends in crisis management practice and crisis management research: Setting the agenda. Journal of Contingencies and Crisis Management, 9(4), 181–188.

98.

Uribe

C. A.

(2014). TheDark side of social capital re-examined from a policy analysis perspective: Networks of trust and corruption. Journal of Comparative Policy Analysis Research and Practice, 16(2), 175–189.

99.

van der Sluijs

. (2005). Uncertainty as a monster in the science–policy interface: Four coping strategies. Water Science & Technology, 52(6), 87–92.

100.

Weimer

D. L.

Vining

A. R.

(1989). Policy analysis: Concepts and practice. Prentice-Hall.

101.

Weiss

C. H.

(1977). Research for policy’s sake: The enlightenment function of social science research. Policy Analysis, 3(4), 531–545.

102.

Weiss

C. H.

(1992). Helping government think: Functions and consequences of policy analysis organizations. In Weiss

C. H.

(Ed.), Organizations for policy analysis: Helping government think (pp. 1–18). Sage Publications.

103.

Weiss

C. H.

Bucuvalas

M. J.

(1980). Social science research and decision-making. Columbia University Press.

104.

Whiteman

(1985). The fate of policy analysis in congressional decision making: Three types of use in committees. The Western Political Quarterly, 38(2), 294–311.

105.

Ramesh

Howlett

(2015). Policy capacity: A conceptual framework for understanding policy competences and capabilities. Policy and Society, 34(3/4), 165–171.