Sage Journals: Discover world-class research

Abstract

Methodological innovation is needed that actively engages a range of policy makers in policy learning to address the climate and biodiversity crises. We developed Walkable Floor Map Policy Workshops (WFMPWs) as a way of engaging policy makers in policy learning towards NBS innovation in their local context. This paper examines WFMPW methodology for NBS through an analysis of three WFMPWs in three European urban case study sites. We find that implementation of WFMPWs facilitated policy learning through group discussion and experience sharing amongst participating policy makers. The WFMPWs added greater spatial contextualisation and the futures workshop elements facilitated discussions of future opportunities and barriers, leading to questioning of common approaches and assumptions and ‘thinking outside of the box’. The data provided in-depth, comparative, nuanced, and locally contextualised qualitative insights of interest across multiple disciplines. The findings provide a different way of understanding political and decision-making processes around NBS, including problem framing and shifting policy debates. The WFMPW approach enabled participating policy makers to describe and critique the current situation; imagine/visualise a preferable future situation; and explore ways of moving from the actual situation to the preferred one.

Keywords

policy workshop methodology policy innovation nature-based solutions policy learning walkable floor maps

Introduction

Innovative methods are needed to foster policy making that is capable of resolving the grand challenges of the climate crisis and loss of biodiversity. In the past, conventional methods of policy making have often led to siloing and policy fragmentation, with little progress in addressing these issues (Russel, 2022). Nature-based Solutions (NBS), potentially offer a road out of the silos, through holistic, multi-functional, cross-sectoral solutions to climate-related and environmental problems, such as flooding and heat effects. However, political and institutional barriers and policy silos still need to be overcome to initiate effective NBS implementation.

Extreme flooding events and prolonged heat effects are pushing climate adaptation and natural environmental solutions up the policy agenda. At the same time, urban planners and environmental policy makers have had to contend with housing and urban development pressures exacerbated by economic conditions and in some cases, natural disasters (such as the earthquakes in Croatia in 2020). Conventional policy making, exacerbated by these pressures and tensions, has led to policy silos (Peters, 2018), involving fragmented sectoral implementation and lack of strategic and systemic action. In contrast, Nature-based Solutions (NBS) offer potential holistic and multi-functional solutions to climate-related and environmental problems such as flooding and heat effects (see Pauleit et al., 2017). However, conventional methods for NBS policy making have in general not been able to overcome existing socio-economic, institutional and political barriers. New methods need to engage with, and understand, the social and locally contextualised nature of policy learning, and to actively involve policy makers, rather than relying on the dissemination of knowledge alone (see Dunlop et al., 2018; Petersen et al., 2023). Newig et al. (2023, p. 6) in a meta-study of 305 case studies found that if there is more intensive and dialogical communication among stakeholders, there is in general a positive impact on environmental governance outcomes:

“… exchange of knowledge, values, and ideas, and the production of shared perspectives and innovative solutions in participatory settings benefit strong environmental provisions”.

We argue, therefore, that to increase dialogical communication, methodological innovation is needed that actively engages a range of policy makers in policy learning to produce innovative and effective NBS policies and implementation, and overcome these barriers. We have therefore developed the Walkable Floor Map Policy Workshop (WFMPW) as a methodology for engaging policy makers and intensifying dialogue across sectors in policy learning towards NBS innovation in their local urban context.

This paper contributes to the literature through reporting on the development and use of an innovative WFMPW methodology for engagement with policy makers, and bringing the resulting insights to new audiences (political scientists, qualitative researchers, as well as environmental and climate researchers, along with policy makers, stakeholders and planners). Whilst there are numerous examples of participatory mapping involving stakeholders in the relevant literature, including relating to NBS (e.g. Kiss et al., 2022; Maurer et al., 2023; see also Giuffrida et al., 2019 on transport; Pillai, 2015 on cultural mapping), the predominant focus has been on using Geographic Information Systems (GIS) software, or on participatory governance or stakeholder engagement involving citizens (e.g. Kiss et al., 2022; Ferreira et al., 2020). The majority of these studies are quantitative, theoretical, digital, geospatial and/or involve systematic literature reviews, and therefore lack either an empirical or in-depth qualitative analysis (with some exceptions, e.g. Maurer et al., 2023 on meaningful places). Above all, in this literature there has been a gap in attention to processes of policy learning.

Our novel contribution is the focus on the policy learning process and the use of large walkable floor maps to engage directly with policy makers in a workshop setting, in combination with the ability to bring in-depth, qualitative insights using concepts from political science to inform the analysis. The use of the floor maps provides opportunities for observing stakeholder dialogues and understanding of the NBS policy context, in a manner which adds a kinaesthetic element where stakeholders and researchers alike can embed a spatial element to the understanding of policy processes through walking around and engaging with the floor map. This research aims to make a contribution to understanding shifting policy debates and problem framing in NBS, as well as to knowledge about how to actually achieve NBS policy change (process). Specifically, this study seeks to shed light on the links between policy learning workshop processes and wider institutional learning and policy change—including through reframing of policy questions, shifting of policy debates and questioning/influencing of policy approaches.

The workshop methodology involves engagement with policy makers in three WFMPWs in three European cities with differing governance characteristics. The large walkable floor maps were chosen as a visualisation tool for the workshop format suitable for promoting localised knowledge exchange amongst NBS policy-makers.

Conceptual Framing

In order for policy change and innovative NBS policy to be implemented, policy learning by stakeholders (both governmental and wider networks) is frequently argued to be required at some level (see e.g. Dunlop et al., 2018). We use the basic definition of policy learning as involving the updating of knowledge¹ and beliefs about public policy (Dunlop et al., 2018; Dunlop & Radaelli, 2012). This definition foregrounds the knowledge and belief aspects of policy learning, i.e., it focuses on processes, rather than changes in policy outcomes, highlighting the importance of methodological concerns. When considering outcomes, policy learning links to policy change at different levels—from slight modifications to an existing policy to fundamental changes in strategy and approach—although policy change outcomes from policy learning may be hard to determine. Because of the uncertainties and risks inherent in trying out new approaches, including NBS, policy change in general tends to be incremental (Lindblom, 1979), unless there is a powerful societal, environmental or political stimulus (see Petersen et al., 2023). Due to limitations of space, we focus here on methodological aspects of policy learning, rather than on wider research on policy learning, change and innovation (for this see e.g. Bennett & Howlett, 1992; Goyal & Howlett, 2024; Grin & Loeber, 2006; Dunlop & Radaelli, 2012, 2021; Dunlop et al., 2018; Petersen et al., 2023).

Policy change involves learning about new ideas, techniques and ways of working, and novel strategic policy formation. Effecting policy change may also entail policy learning around modifications of institutional practices and governance. Policy learning in response to policy failure (as well as success) is also more than a technical process; it requires processes of social interaction (see Strother, 2018). However, suitable methodologies for facilitating NBS policy learning have not been well documented or articulated in the literature to date. There is little empirical evidence on how NBS policy learning processes actually operate, since much of the policy learning literature is theoretical or consists of reviews of the literature (see e.g. Bennett & Howlett, 1992; Dunlop et al., 2018; Dunlop & Radaelli, 2012, 2021; Grin & Loeber, 2006). Another gap which is likely to be harder to fill is in terms of outcomes—which policy learning methods and processes are most likely to result in either the institutional changes that might be necessary (e.g. increasing policy integration and coherence across policy sectors), or in real world NBS policy transformation. Thirdly, there is little work around spatial scale—the importance of providing spatial information at a scale that is suitable for urban decision making around NBS (e.g. local as opposed to regional/national/EU-level).

The research draws on data from three WFMPWs held in 2022, one in each case study: Paris Region, France, in Western Europe; Aarhus, Denmark in Northern Europe; and Velika Gorica, Croatia, in Central Europe (see Figure 1). We use a case study design, collecting in-depth, qualitative data aimed at analytical rather than statistical generalisation (see Yin, 2009). In terms of case selection, we employ a most different cases approach (Bozonelos et al., 2022), incorporating different sized urban municipalities, with different topographies and histories as well as variations in cultures and governance. Using this approach, we are able to ascertain elements around policy learning methods (and outcomes) that are consistent across the cases and therefore may be more related to the challenges posed by NBS, as well as those elements that differ by case and may be more related to governance systems and cultures specific to the contexts studied. The city case studies allow us to reflect on comparative insights across the participating cities (see conceptual framework in Jensen et al., 2020; Petersen et al., 2023 for more information about differences in governance). These case studies were chosen for data collection across the whole project (European sites).

Figure 1.

Map of the three case study sites. Source: https://www.regreen-project.eu/.

The WFMPW research reported on here uses an embedded case study design (see Yin, 1994, 2012). This enables us to look at the characteristics of NBS policy at city scale as well as the implementation of local NBS projects/initiatives. In particular, an embedded case study area (Aulnay-sous-Bois, a town [commune] in the Paris Region) was selected for the WFMPW in consultation with the project partners in the Paris Region because of the difficulties associated with understanding NBS implementation for and facilitating useful discussions with stakeholders about an area of the size and complexity of the Paris Region. Using these embedded case studies we examine how the WFMPWs enabled an exploration of NBS policy and implementation and of its future potential, creating policy learning amongst participants (policy makers—including a range of governmental and non-governmental stakeholders).

To analyse the content of the WFMPW data we use a conceptual framing based on the policy learning literature, particularly Bennett and Howlett’s (1992, p.289) work, to differentiate between different types of policy learning operating in the workshops. We distinguish between three types of policy learning as follows (drawing on additional insights from Grin & Loeber, 2006; Dunlop & Radaelli, 2012, 2021; Dunlop et al., 2018; Petersen et al., 2023; see also Goyal & Howlett, 2024):

(1) Incremental policy learning – (small) changes to existing policy processes;

(2) Lesson-drawing – changes in policy instruments e.g., transfer of instruments from other locations/policy sectors; and

(3) Dialogical learning – changes in policy ideas, approaches, assumptions and values (often linked to policy dialogue between stakeholders).

However, in contrast with Bennett and Howlett’s (1992) original framework and in line with Goyal and Howlett (2024), we acknowledge that all these types of policy learning may be enacted by and involve different types of stakeholders (e.g., government staff; NBS experts of different disciplines; non-governmental staff; citizens and residents). By extension, particular types of stakeholders, e.g., government officials, may engage in all three types of policy learning. In addition, policy learning may lead to different effects (e.g., changes in implementation, organisational or institutional processes, programme changes; or wider changes in policy approaches, or indeed a retrenchment of approaches). We have also altered the original terms used in Bennett and Howlett’s (1992) framework in order to bring greater clarity (and avoiding pitfalls inherent in usage of the term ‘social learning’ with its multiple and sometimes conflicting uses).

Methods for WFMPWs: Futures-Based Policy Learning Workshops using Walkable Floor Maps

The WFMPW methodology is based on futures methods, utilising walkable floor maps to understand current policy-making dynamics and to think about future possibilities. The data has also been triangulated with interview data in each case study site collected as part of the project (see Petersen et al., 2023). The methods chosen for the WFMPWs foreground policy learning processes and experiential aspects (i.e. learning from experience) with a specific recognition of the spatial nature of NBS. We situate the methodological discussion below within the general WFMPW methodology literature and provide some more detailed information about walkable floor maps.

Review of Workshop Literature

We drew on the literature and on prior experience of project workshops to develop the WFMPW design. The relevant policy literature² cites a range of types of methodology used in conducting workshops with policy-makers and stakeholders. This includes: futures workshops (Lauttamäki, 2016); workshops incorporating scenarios (see below), design workshops (e.g. real-life design options presented and discussed by students) (De Waegemaeker et al., 2017); risk perception workshops using Problem Structuring Methods to build consensus (Santoro et al., 2019); ‘sharing circles’ – similar to focus groups (Menzies et al., 2022); a mix of academic workshop format and world café methods³ (Kabisch et al., 2016) and narrative workshops⁴ (Shaw & Corner, 2017). We have insufficient space to cover here other tangentially-related literature on, for example, the use of participatory mapping in workshop contexts (see e.g. Pérez-Ramírez et al., 2019 on participatory mapping of ecosystem services); or the long history of using participatory methods (including mapping) in workshops in a range of contexts (see for example Chambers, 2008). Existing literature indicates the potential for, as well as examples of, actual use of maps (either conventional or GIS) in current policy-making contexts (see e.g. Carton, 2007; Noyons, 2004).

Futures or explorative scenarios featured in workshop methodology in several articles e.g. future scenario methods (Foran et al., 2016)⁵; citizen-participatory scenario design methodology (based on a combination of scenario design and future design approaches) (Uwasu et al., 2020)⁶; exploring alternative scenarios using Q methodology (Ligtvoet et al., 2016); and a co-evolutionary scenario approach (future scenarios – assessing risks and benefits) (Mann, 2015). Other articles mentioned use of scenarios in policy-making contexts (e.g. Hughes, 2013) and as part of workshop discussions (e.g. De Waegemaeker et al., 2017; Shaw & Corner, 2017). From the literature, we can distinguish between three types of scenarios that could be used for policy learning workshops: predictive (what will happen), explorative (what could happen) and normative scenarios (what should happen). Explorative scenarios incorporate and anticipate unexpected outcomes/futures rather than just the expected futures, and so may be more useful in enabling questioning where an ‘official’ or expected future tends to dominate the policy discourse (Ligtvoet et al., 2016, pp. 19,21). Futures workshop methods incorporate this exploratory aspect, but enable a more flexible approach to exploring possible futures in a workshop format, allowing creative and novel ideas to emerge (see e.g. Lauttamäki, 2014; 2016; Vidal, 2006; Foran et al., 2016). The futures workshop approach has three main components: describing and critiquing the current situation; imagining/visualising a preferable future situation; and exploring ways of moving from the actual situation to the preferred one (see Vidal, 2006).

Other methodological elements found in the literature that are relevant, but for our purposes more peripheral, included prioritisation exercises and/or consensus-building exercises. Prioritisation featured in a number of the articles in some form as part of the workshop methodology—prioritising design/implementation options or policy instruments (e.g. Santoro et al., 2019 (using scoring and aggregated ranking); Menzies et al., 2022; Foran et al., 2016; Uwasu et al., 2020); with two more based on the Delphi model (through voting, discussion and scoring) (Grace et al., 2021; Sutherland et al., 2018). The literature seems to be mixed on whether reaching consensus should be regarded as a/the goal of the workshop(s) (e.g. Mann, 2015; Santoro et al., 2019; Shaw & Corner, 2017)—depending on the aims and specific context of the workshops (e.g. stakeholder policy learning may not require consensus (Mann, 2015)).

In addition, many of the relevant articles emphasised the importance of participatory methodologies or collaborative governance in engaging with stakeholders (e.g. Foran et al., 2016; Uwasu et al., 2020). More specific participatory methodologies were also cited, e.g. Constructive Conflict Methodology (a participatory problem-structuring approach) in combination with the use of Q methodology for (qualitative and quantitative) analysis (Ligtvoet et al., 2016). Others have highlighted the usefulness of participatory creative arts/theatre workshop methodologies in engaging stakeholders on policy issues and alternatives (see e.g. McEachern et al., 2020).

The workshop design was developed informed by the literature above and by previous usage of the walkable floor maps by project staff (see below).

Walkable Floor Maps

Large walkable floor maps (see Figure 2) are a relatively new methodological tool for policy workshops and therefore few examples have been cited in the literature to date outside of this project (see Banzhaf et al., 2021; Banzhaf et al., 2021). Use of walkable floor maps, however, do show some similarities with other types of visualization techniques that feature in the literature on policy workshops, including landscape visualization, urban climate maps and graphics, e.g., on the operation of local water systems and vulnerability to future climate impacts (see De Waegemaeker et al., 2017). Other types of visual research methods (e.g. picture elicitation) are also documented in the literature and have some commonalities with how we have used walkable floor maps (see e.g. Marais-Potgieter & Thatcher, 2021) but lack the combination of kinaesthetic and physical elements of collective movement around the map.

Figure 2.

Walkable floor map with additional post-it notes during the WFMPW in Aarhus, November 2022. Source: Pedersen, Nov 2022.

The walkable floor maps consist of large (up to 8 x 5 m) maps of each of the case study municipalities dependent on the area of investigation and the best-fitting scale. The floor maps consist of very large aerial photos or very high-resolution satellite images (with a ground resolution of 20–30 cm per pixel; which corresponds to a scale of approximately 1:1500 to 1:2000), printed onto a flexible and durable background (either fixed to the floor or rollable for easy storage). These were produced with most recent remote sensing data and show individual features such as houses, streets and trees⁷. This tangible, visual tool has been found through various examples of previous usage with different stakeholders within the project to facilitate a deepening of the optical sensory perception of urban objects, as well as of connectivities and grounding of the social interaction and discussions in the local context. Walkable floor maps have, since 2021, been used to engage with policy makers in multiple European cities (see Banzhaf et al., 2021; Banzhaf et al., 2021). At first glance, the maps present a static view of a study area at a certain point in time. However, these maps can be made dynamic through the use of overlays. These may be QR codes linking to information on a website, transparencies of planning processes, photos, drawings and sketches and other ways of illustrating information (e.g. sticky notes; photo elicitation) that tell stories about present and planned activities relating to NBS.

WFMPW Methodology: Developing a Design and Implementing the Workshops

Drawing on the literature as highlighted above, we integrated an explorative scenario element (scenarios incorporating/anticipating unexpected outcomes/futures) into the workshop design, enabling questioning where an ‘official’ or expected future could be dominating the policy discourse (Ligtvoet et al., 2016, pp. 19,21) (see WFMPW semi-structured outline in Appendix 1). Use of futures workshop methods also enabled a more flexible approach, allowing creative/innovative ideas to emerge (see e.g. Lauttamäki, 2016; 2014; Vidal, 2006; Foran et al., 2016). We incorporated the three main components of futures workshops into the WFMPWs as detailed below (describing and critiquing the current situation; imagining/visualising a preferable future situation; and exploring ways of moving from the actual situation to the preferred one (Vidal, 2006)).

Within the constraints of the project scope, timescale and deadlines, we aimed to integrate interactive and participatory elements into our methodology using the walkable floor maps and futures methods. With participatory principles in mind, we guided participants using questions and other prompts while they walked around on the walkable floor maps. In groups, participants were asked to: 1) describe and critique the current situation of NBS implementation, including barriers and governance aspects; 2) imagine what their preferred future NBS implementation could look like without the current constraints—e.g., participants were asked, ‘If your local area won the lottery and you had the necessary political support, which NBS initiatives would you implement?’; and 3) explore the obstacles and constraints involved as well as ways of getting to the preferred future situation. Participants were asked to reflect and write down suggestions on sticky notes, to place these on the map where appropriate and finally to discuss their thoughts and ideas with the other participants and facilitators. The prompts were designed to elicit reflections about governance and policy learning aspects of NBS in their area.

We carried out three WFMPWs with local policy makers/stakeholders (a total of 22 participants plus researchers) carried out in 2022—one in each case study. This comprised: Aulnay-sous-Bois, Paris Region, France (6 stakeholders plus 2 researchers); Aarhus, Denmark (5 stakeholders plus 2 researchers⁸); and Velika Gorica, Croatia (11 stakeholders plus 3 researchers) (see map Figure 1). The workshops were recorded, the recordings transcribed, and the transcriptions checked against the original recording and edited by a project researcher⁹.

Drawing on their networks and local knowledge, our local project partners recruited participants for the WFMPWs, based on participants’ involvement in and/or knowledge of NBS initiatives. We requested different types of stakeholders involved in NBS policy and implementation—from local government/city administration, businesses, non-governmental organisations (NGOs), small civil society organisations, consultants, etc.—in order to give a range of views around NBS policy, governance and implementation. However, recruitment was necessarily constrained by project timescales made more difficult because of the COVID-19 pandemic, by who was able and willing to attend, and who was known to and active in the project partner’s NBS networks. In Aulnay-sous-Bois, Paris Region and in Aarhus, participants comprised local government staff/policy makers from the municipality administrations (and in the case of Aulnay-sous-Bois, Paris Region, also a public organisation/association of communes [local towns]). All were selected according to their involvement in working with urban nature. In Velika Gorica, there was the greatest range of participants from all the types of stakeholders indicated above. Further research could engage a wider range of stakeholders in policy learning workshops, and potentially include online workshops as well as in-person workshops in order to maximise participation and inclusivity (see e.g. Bolin et al., 2023).

Data Management and Methods for the Data Analysis

The transcripts of the WFMPWs were first checked by a participating researcher and minor edits made to ensure consistency with the original recordings of the workshops. Institutional ethics approval, including informed consent, was obtained in line with the relevant institutional procedures. The resulting data was anonymised and kept securely in password-protected files/folders and shared only with the project team. The data analysis was guided by the methodological framing outlined above and by a common analytical coding matrix (see Appendix 2) compiled by the team as part of the conceptual background work for use in this project component. The coding matrix identified common themes and categories from the literature. The data analysis was carried out using a thematic analysis approach (Braun & Clarke, 2006, 2021), combining both deductive (based on the coding matrix and the literature) and inductive analysis. The results are presented in the following section. The use of the coding matrix ensured compatibility and comparability of findings across the case studies.

Results of the Case Studies

The following section first outlines the composition of workshops, and then provides results of qualitative analysis of the WFMPWs based on the methodology and framework on policy learning highlighted above, and illustrated with examples of policy learning reported or observed in the WFMPWs. The analysis covers instances where participants reflected in the workshop on their own learning about NBS policies, including specific shifts in NBS policy or thinking.

Composition of the Workshop Participants

A table of the workshop participants and their roles and genders is included as Appendix 3.

The Aarhus WFMPW participants were all female staff from Aarhus municipality (local government) that are involved in NBS-related work—four from the environment department and one from the health department.

In Velika Gorica, the participants consisted of three staff from the city of Velika Gorica municipality administration (one from planning; one from EU funds; one from tourism); two staff from an urban planning institute (one architect, one construction engineer); a representative of a development agency; a staff member from the University of Velika Gorica (professional services); a head teacher from a local school; and a representative from a local community NGO. Velika Gorica therefore had the biggest range of types of stakeholders of the three workshops, as well as a mix of male (3) and female (8) participants.

In Aulnay-sous-Bois, Paris Region, the workshop participants consisted of three staff from the local municipality administration of the town of Aulnay-sous-Bois (one each from environment/biodiversity; green spaces; transport/environment/sustainable development); and two staff from the public organisation that groups together the eight local municipalities including Aulnay-sous-Bois (this included one staff member working on sustainable development) and one participant from the partner organisation in Paris. The gender breakdown of participants for this workshop was 2 female, 4 male.

Results: Comparative Findings Across the Three Case Studies

This section details and illustrates the findings from the three case studies (see Figures 3 and 4).

Figure 3.

Participant discussions using the walkable floor map during the WFMPW in Velika Gorica, September 2022. Source: Petersen, Sep 2022.

Figure 4.

Participants interacting with the walkable floor map at the WFMPW in Aulnay-sous-Bois, Paris Region, September 2022. Source: Petersen, Sep 2022.

We analysed and categorised the data according to the three types of policy learning using our adapted analytical framework (government learning, lesson drawing and dialogical learning). Substantive examples are provided in Table 1 below to illustrate the methods used. We acknowledge that in some instances there may be more than one of these types operating concurrently but have categorised them according to the predominant type observed.

Table 1.

Summary Table Comparing NBS policy Learning Across the Three European Walkable Floor Map Policy Workshops (WFMPWs).

Key to symbols used:

Findings

Although the researchers were mainly involved in actively facilitating the workshops rather than playing a purely observational role, the workshop participants were observed to concentrate on certain areas of interest as the discussions progressed. These tended to be areas of green/blue space which were particularly enjoyed (e.g. existing park areas in Aulnay-sous-Bois), areas where implementation was either being carried out or planned (e.g. along watercourses in Paris Region), as well as areas where it would be most helpful/desirable to develop green/blue space and where connectivity gains would be greatest (e.g. industrial areas near the airport in Velika Gorica).

A range of policy learning was evident on a number of themes across all the case studies, with other examples shared across two case studies, as illustrated in the table above, indicating that this methodology is capable of producing insights that are both broadly comparative and more locally nuanced results. Most of the examples in the findings fell under the dialogical policy learning category (see Table 1), e.g. political policy learning—how to appeal to and negotiate around political and strategic priorities of decision-makers. Other policy learning examples that featured across all the case studies included approaches to planning regulations, greenspace targets and enforcement (including how these are working in practice and how they might be improved); and how ‘nature’ is defined by different groups. Other instances of policy learning were shared across two case studies, such as the need for re-routing of transport infrastructure in both Aulnay-sous-Bois and Velika Gorica; and issues of multi-functionality, access, inclusion and socio-economic impact, which were emphasised by participants in Aulnay-sous-Bois and Aarhus but not to the same extent in Velika Gorica.

As indicated above, the future vision component was integral to the workshop design, and policy learning was evident in all the case studies on this theme. The policy learning examples exhibited commonalities across all three case studies, such as on green corridors and green/blue connectivity. The workshop methodology also enabled a discussion of barriers and constraints to NBS policy learning and implementation, including funding and capacity constraints.

The workshop data reflect the nature of policy learning as often involving some kind of social interaction, whether direct or indirect (see Bennett & Howlett, 1992; Grin & Loeber, 2006). As highlighted above, many more of the examples observed in the workshops fall into the dialogical policy learning category than the incremental policy learning or lesson-drawing categories, indicating that the workshops facilitated dialogue about and questioning of policy approaches, ideas and / or values. We take this as partially arising from the workshop design and methodology used, as the prompts and elicitation guided participants towards questioning current approaches. However, it also points towards efficacy of the methods used—since from the literature it is clear that innovation in NBS that is capable of mitigating climate change effects is likely to require different ways of thinking that go beyond incremental learning or lesson drawing in order to be effective.

The workshop process and data highlighted the political nature of policy learning. The policy learning by stakeholders in all the workshops was significantly shaped by power relationships amongst the stakeholder participants (c.f. policy learning in the shadow of hierarchy—see Dunlop & Radaelli, 2012). In the Aarhus workshop, the participants consisted of a relatively homogeneous group of government officials at a similar level, all female, and mostly known to each other. Observations of the workshop processes suggests that they appeared relatively less affected by hierarchical elements. In Velika Gorica, the group involved more varied stakeholders, both governmental and non-governmental, male and female. In this workshop, participants seemed to be more influenced by local hierarchies, with participants appearing to hold back or moderate what they were willing to contribute within the workshop (e.g. when triangulated with data obtained from other workshops and/or from individual interviews—see Petersen et al., 2023). In the Paris (Aulnay-sous-Bois) workshop there was more homogeneity of participants than in Velika Gorica, although less than in Aarhus, and political hierarchies were also evident here but to a lesser extent than in Velika Gorica.

Discussion – Reflections on Policy Learning and WFMPWs

How did the use of Walkable Floor Maps Impact on the Policy Learning Process?

The data analysis indicates that the walkable floor maps were helpful in creating a group learning experience—where participants could move around the map in groups and discuss while focusing on different sites (a kind of virtual tour; expressing individual associations of places in group interactions). Use of the walkable floor maps also enabled transfer of learning about specific locations from participants to each other and to facilitators through enhanced visualisation and thus greater identification of local contexts and landscape features (e.g. local parks, green spaces and watercourses). The maps enabled the dialogue and learning to be spatially contextualised in the local setting, leading to further discussions about, and questioning of, broader approaches to NBS policy and implementation (dialogical learning). The analysis highlights that the futures workshop design using the floor maps was effective in facilitating dialogue and policy learning about possible visions for the future, and what the constraints are to achieving these (e.g. of increasing green/blue space connectivity across currently industrialised areas). The futures discussions, along with the floor map, created opportunities for thinking about incremental and programme level policy change as well as dialogical policy learning with regard to specific locations and NBS initiatives. The WFMPW model used therefore shows potential as a tool to foster significant shifts in policy especially where a mix of policy-makers (and stakeholder input) is involved.

Reflections on what People Learned and who did the Policy Learning

The workshop data shows that participants engaged actively with the participatory workshop process, with evidence of policy learning as well as reflecting on existing institutional practices observed within the workshops. Additional examples were also reported that had happened previously external to the workshop. The policy and institutional learning reported here was by government staff and other NBS stakeholders, consisting of a range of NBS aspects (including design, implementation, maintenance, liveability, co-benefits, constraints and enabling factors).

Policy learning by the participants was observed as arising both from listening to each other’s perspectives, and from reflections based on prompts and examples highlighted by facilitators. There was, of course, also extensive policy learning by the facilitators as part of the research process, predominantly about the local context, including the governance architecture and its implications—which is likely to have influenced the participants’ policy learning processes as well. This includes via workshop prompts, and through participants making local conditions more explicit for those less familiar with local policies and practices.

The inclusion of future visions, opportunities and constraints in the WFMPW design produced interesting results on policy learning across all the case studies. The futures discussions created opportunities for thinking about incremental and programme level policy change as well as dialogical policy learning—including questioning of assumptions, practices and values relating to NBS. In Velika Gorica in particular, it took time and further prompts for participants to offer insights into what a future vision without the current financial and political limitations could look like, at least in part influenced by power relationships and institutional/governance characteristics. In all the workshops the futures discussions also involved deliberations of current barriers. Despite the different contexts, several common ideas for future NBS implementation emerged across all three case studies, e.g. increasing green space connectivity, and greater policy integration with, for example, transport policy. Based on the experience in our workshops, although including more varied stakeholders as participants is likely to yield greater and more in-depth insights (see Petersen et al., 2023), it may also bring power and gendered relationship elements further to the fore in the workshop setting, which in turn would need to be carefully managed.

How do we Establish when and to what Extent Policy Learning Occurs—what Evidence do we Need?

The data collected for this paper indicates that the WFMPW methodology outlined was effective in promoting policy learning within the workshops and led to questioning of policy approaches and values (dialogical learning) as well as policy learning around the more practical side of NBS implementation (incremental policy learning and lesson learning). It also allowed us as researchers to collect data on that learning to better understand the governance processes for NBS in the selected case studies. However, follow-up work is needed to track how this translates into policy change in real world settings, or what type(s) of policy learning might most effectively result in changes in institutional practices or governance. There is also a need for further work around what can/should be counted as evidence of policy learning, including when it occurs in a workshop setting, as well as evidence of impacts on policy change.

Conclusions

We have highlighted that methodological innovation is needed for NBS policy learning to be effective, entailing in some cases implementing the institutional changes required, including to create greater policy integration, and to overcome siloed thinking in a more tangible way. The search for new and innovative methods for NBS policy making in urban planning and managing green/blue spaces has become pressing in the face of the climate crisis and the widespread reduction in biodiversity. Addressing these challenges requires innovative methods to foster policy learning in order to effect policy change, especially given the need to overcome political and institutional barriers to change.

We investigated the contribution to policy learning on NBS of this research based on three workshops conducted using a futures workshop design and innovative walkable floor maps. We found that implementation of WFMPWs using walkable floor maps and a futures workshop methodology facilitated policy learning and reflections on institutional practices through group discussion and experience sharing amongst the participating policy makers. The walkable floor maps were found to add greater spatial contextualisation and locally grounded policy learning, which are likely to be important for tackling environmental problems as experienced by communities at the local level. The futures workshop elements facilitated discussions of future opportunities and barriers. These methods together led to the questioning of common approaches and assumptions and ‘thinking outside of the box’ and produced both comparative and nuanced results. The WFMPW approach enabled participating policy makers to describe and critique the current situation; imagine/visualise a preferable future situation; and explore ways of moving from the actual situation to the preferred one.

The data about the workshop processes demonstrates the value of providing opportunities for dialogue and social interaction (involving experts, stakeholders and local citizens) in which all three types of policy learning may occur, but especially where the dialogue involves potential futures and thinking outside of current limitations (e.g. talking about future visions). The WFMPW model used therefore shows potential as a tool to foster significant shifts in policy and institutional practices especially where a mix of policy-makers and stakeholder input is involved. However, attention to power relationships and how these impact on policy learning processes, including within workshop processes, is required.

Limitations and Further Research Ideas

Due to constraints on recruitment and time associated with the funding cycle exacerbated by the COVID-19 pandemic, the workshops were limited in the numbers and composition of participants, such as the spread across types of stakeholders. In addition, we were not able to ensure an even gender balance in all the case studies. Therefore, further workshops addressing these aspects would deepen the findings. Additional research is also needed to make the process of determining what counts as evidence of policy learning more robust and transferable in this context, as well as how this methodology could result in actual policy changes.

During the course of compiling this paper and as a result of helpful feedback from reviewers and colleagues we have identified several areas for future research. These include follow-up research to investigate further 1) the relationship between policy learning and policy change, and how policy learning is translated into policy change; 2) developing robust and transferable ways of identifying when policy learning occurs and what counts as evidence of policy learning; 3) exploring walkable floor maps and other visualisation techniques to promote policy learning and discussion amongst policy makers, comparing with other methods and approaches; and 4) investigating the influence of power and gender relationship aspects in policy learning and workshop processes. Future visualisation ideas by project partners include developing techniques to create a three-dimensional representation of walking through a neighbourhood or green space.

Supplemental Material

Supplemental Material - Walkable Maps and Policy Innovation for Nature: A Novel Methodology for Understanding Policy Learning

Supplemental Material for Walkable Maps and Policy Innovation for Nature: A Novel Methodology for Understanding Policy Learning by Carolyn J. Petersen, Duncan J. Russel, Anne Jensen, Anders Branth Pedersen, Ellen Banzhaf and Ingrid Kaltenegger in International Journal of Qualitative Methods

Footnotes

Acknowledgements

Thank you to the REGREEN project funders (see funding section) and those who contributed to project funding acquisition and the management of project activities. In addition,we would very much like to thank the journal reviewers and editors and Sally Anderson,who reviewed the paper,as well as those who gave feedback at the relevant session during a project workshop in Alnarp,Sweden in April 2023,for all their helpful and constructive comments.

We would also like to thank the various staff of the project partner organisations in the city case studies who assisted with various essential tasks,particularly the organisation of the WFMPWs and recruitment of participants for the workshops,without which this publication would not have been possible. Last but not least,we would like to thank all of the workshop participants for spending time in providing very valuable information for this research.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research,authorship,and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research,authorship,and/or publication of this article: This work was supported by the REGREEN project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement no. 821016. This document reflects only the authors’ views and the Commission is not responsible for any use that may be made of the information it contains.

Ethical Statement

Open Access Statement

For the purpose of open access,the author has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising from this submission.

ORCID iDs

Carolyn J. Petersen

Duncan J. Russel

Anders Branth Pedersen

Ellen Banzhaf

Ingrid Kaltenegger

Supplemental Material

Supplemental material for this article is available online.

Notes

References

Banzhaf

Anderson

Læssøe

Jensen

Iversen

(2021). REGREEN 3rd Newsletter (Dec 2021). REGREEN. https://www.regreen-project.eu/wp-content/uploads/10_Walkable-floor-maps2.pdf

Banzhaf

Luo

Knopp

(2021). Integrated mapping of spatial urban dynamics—a European-Chinese exploration. Part 1—methodology for automatic land cover classification tailored towards spatial allocation of ecosystem services features. Remote Sensing, 13(9), 1744. https://doi.org/10.3390/rs13091744

Bennett

C. J.

Howlett

(1992). The lessons of learning: Reconciling theories of policy learning and policy change. Policy Sciences, 25(3), 275–294. JSTOR. https://doi.org/10.1007/bf00138786 https://www.jstor.org/stable/4532260

Bolin

Kalmus

Figueiras

(2023). Conducting online focus group interviews with two generations: Methodological experiences and reflections from the pandemic context. International Journal of Qualitative Methods, 22(2), Article 160940692311820. https://doi.org/10.1177/16094069231182029

Bozonelos

Wendt

Lee

Scarffe

Omae

Franco

Martin

Veldhuis

(2022). Case selection (or, how to Use cases in your comparative analysis). ASCCC Open Educational Resources Initiative (OERI). https://socialsci.libretexts.org/Bookshelves/Political_Science_and_Civics/Introduction_to_Comparative_Government_and_Politics_(Bozonelos_et_al.)/02%3A_How_to_Study_Comparative_Politics-_Using_Comparative_Methods/2.03%3A_Case_Selection_(Or_How_to_Use_Cases_in_Your_Comparative_Analysis

Braun

Clarke

(2021). One size fits all? What counts as quality practice in (reflexive) thematic analysis? Qualitative Research in Psychology, 18(3), 328–352. https://doi.org/10.1080/14780887.2020.1769238

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Carton

L. J.

(2007). Map making and map use in a multi-actor context: Spatial visualizations and frame conflicts in regional policymaking in The Netherlands. [Sl]:[Sn]. Available on: https://repository.ubn.ru.nl/bitstream/handle/2066/121859/121859.pdf

Chambers

(2008). Revolutions in development inquiry (1st ed.). Routledge. https://doi.org/10.4324/9781849772426

10.

De Waegemaeker

Kerselaers

Van Acker

Rogge

(2017). Design workshops in the age of climate change: Analysis of a design workshop on drought in Flanders. Smart and Sustainable Built Environment, 6(1), 48–63. https://doi.org/10.1108/sasbe-05-2017-0020

11.

Dunlop

C. A.

Radaelli

C. M.

(2012). Systematising policy learning: From Monolith to dimensions. Political Studies, 61(3), 599–619. https://doi.org/10.1111/j.1467-9248.2012.00982.x

12.

Dunlop

C. A.

Radaelli

C. M.

(2021). The lessons of policy learning: Types, triggers, hindrances and pathologies. In Weible

Cairney

(Eds.), Practical lessons from policy theories (pp. 83–104). Policy Press. ORE. https://hdl.handle.net/1814/71838

13.

Dunlop

C. A.

Radaelli

C. M.

Trein

(2018). Introduction: The family tree of policy learning. In Dunlop

Radaelli

Trein

(Eds.), Learning in public policy. International Series on public policy. Palgrave Macmillan. https://doi.org/10.1007/978-3-319-76210-4_1

14.

Ferreira

Barreira

A. P.

Loures

Antunes

Panagopoulos

(2020). Stakeholders’ engagement on nature-based solutions: A systematic literature review. Sustainability, 12(2), 640. https://doi.org/10.3390/su12020640

15.

Foran

Fleming

Spandonide

Williams

Race

(2016). Understanding energy-related regimes: A participatory approach from central Australia. Energy Policy, 91(2), 315–324. https://doi.org/10.1016/j.enpol.2016.01.014

16.

Giuffrida

Le Pira

Inturri

Ignaccolo

(2019). Mapping with stakeholders: An overview of public participatory GIS and VGI in transport decision-making. ISPRS International Journal of Geo-Information, 8(4), 198. https://doi.org/10.3390/ijgi8040198

17.

Goyal

Howlett

(2024). Types of learning and varieties of innovation: How does policy learning enable policy innovation? Policy & Politics, 6(4), 1–22. https://doi.org/10.1332/030557321X16841388707452

18.

Grace

Balzan

Collier

Geneletti

Tomaskinova

Abela

Borg

Buhagiar

Camilleri

Cardona

Cassar

Cattafi

Cauchi

Galea

La Rosa

Malekkidou

Masini

Portelli

Dicks

L. V.

(2021). Priority knowledge needs for implementing nature-based solutions in the Mediterranean islands. Environmental Science & Policy, 116(1), 56–68. https://doi.org/10.1016/j.envsci.2020.10.003

19.

Grin

Loeber

(2006). Theories of policy learning: Agency, structure, and change. In Fischer

Miller 2006

G. J.

(Eds.), Handbook of public policy analysis: Theory, politics, and methods (pp. 201–219). CRC Press. https://www.taylorfrancis.com/chapters/edit/10.4324/9781315093192-25/theories-policy-learning-agency-structure-change-john-grin-anne-loeber

20.

Hughes

(2013). Towards improving the relevance of scenarios for public policy questions: A proposed methodological framework for policy relevant low carbon scenarios. Technological Forecasting and Social Change, 80(4), 687–698. https://doi.org/10.1016/j.techfore.2012.07.009

21.

Jensen

Russel

Grandin

Kaltenegger

Morère

Barra

Hardiman

Zhilin

(2020). Deliverable N°6.1. Research protocol for the analysis of governance systems in European Urban Living Labs (ULLs). REGREEN. https://www.regreen-project.eu/resources/

22.

Kabisch

Frantzeskaki

Pauleit

Naumann

Davis

Artmann

Haase

Knapp

Korn

Stadler

Zaunberger

Bonn

(2016). Nature-based solutions to climate change mitigation and adaptation in urban areas: Perspectives on indicators, knowledge gaps, barriers, and opportunities for action. Ecology and Society, 21(2), 39. https://doi.org/10.5751/ES-08373-210239

23.

Kiss

Bernadett

Sekulova

Filka

Hörschelmann

Kathrin

Salk

Carl

Takahashi

Wakana

Wamsler

Christine

, et al. (2022). Citizen participation in the governance of nature‐based solutions. Environmental Policy and Governance, 32(3), 247–272. https://doi.org/10.1002/eet.1987

24.

Lauttamäki

(2014). Practical guide for facilitating a futures workshop. Finland futures research Centre papers. Finland Futures Research Centre, 2–11. https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwjvr7re4oaBAxXRW0EAHV30DfUQFnoECBMQAQ&url=https%3A%2F%2Fwww.utu.fi%2Fsites%2Fdefault%2Ffiles%2Fpublic%3A%2Fmedia%2Ffile%2FVille-Lauttamaki_futures-workshops.pdf&usg=AOvVaw3FY5-fF77AwKYvMGvMHYJ1&opi=89978449

25.

Lauttamäki

(2016). ACTVOD-futures workshop – a generic structure for a one-day futures workshop. Foresight, 18(2), 156–171. https://doi.org/10.1108/FS-01-2015-0003

26.

Ligtvoet

Cuppen

Di Ruggero

Hemmes

Pesch

Quist

Mehos

(2016). New future perspectives through constructive conflict: Exploring the future of gas in The Netherlands. Futures, 78(9), 19–33. https://doi.org/10.1016/j.futures.2016.03.008

27.

Lindblom

C. E.

(1979). Still Muddling, not yet through. Public Administration Review, 39(6), 517–526. https://doi.org/10.2307/976178

28.

Mann

(2015). Strategies for sustainable policy design: Constructive assessment of biodiversity offsets and banking. Ecosystem Services, 16(7), 266–274. https://doi.org/10.1016/j.ecoser.2015.07.001

29.

Marais-Potgieter

Thatcher

(2021). Using a deductive approach to develop a picture elicitation technique to explore the Human-nature Nexus. International Journal of Qualitative Methods, 20(2), Article 160940692110374. https://doi.org/10.1177/16094069211037492

30.

Maurer

Chang

Olafsson

A. S.

Møller

M. S.

Gulsrud

N. M.

(2023). A social-ecological-technological system approach to just nature-based solutions: A case of digital participatory mapping of meaningful places in a marginalized neighborhood in Copenhagen, Denmark. Urban Forestry and Urban Greening, 89(56), 128120. https://doi.org/10.1016/j.ufug.2023.128120

31.

McEachern

M. G.

Middleton

Cassidy

(2020). Encouraging sustainable behaviour change via a social practice approach: A focus on apparel consumption practices. Journal of Consumer Policy, 43(2), 397–418. https://doi.org/10.1007/s10603-020-09454-0

32.

Menzies

A. K.

Bowles

Gallant

Patterson

Kozmik

Chiblow

McGregor

Ford

Popp

J. N.

(2022). “I see my culture starting to disappear”: Anishinaabe perspectives on the socioecological impacts of climate change and future research needs. FACETS, 7(1), 509–527. https://doi.org/10.1139/facets-2021-0066

33.

Newig

Jager

N. W.

Challies

Kochskämper

(2023). Does stakeholder participation improve environmental governance? Evidence from a meta-analysis of 305 case studies. Global Environmental Change: Human and Policy Dimensions, 82(2), 102705. https://doi.org/10.1016/j.gloenvcha.2023.102705

34.

Noyons

E. C.

(2004). Science maps within a science policy context. In Handbook of quantitative science and technology research (pp. 237–255). Springer Dordrecht. https://link.springer.com/chapter/10.1007/1-4020-2755-9_11.

35.

Pauleit

Zölch

Hansen

Randrup

T. B.

Konijnendijk van den Bosch

(2017). Nature-based solutions and climate change – four Shades of green. In Kabisch

Korn

Stadler

Bonn

(Eds.), Nature-based solutions to climate change adaptation in urban areas. Theory and practice of urban Sustainability Transitions: Springer. https://doi.org/10.1007/978-3-319-56091-5_3

36.

Pérez-Ramírez

García-Llorente

Benito

Castro

A. J.

(2019). Exploring sense of place across cultivated lands through public participatory mapping. Landscape Ecology, 34(7), 1675–1692. https://doi.org/10.1007/s10980-019-00816-9

37.

Peters

B. G.

(2018). The challenge of policy coordination. Policy Design and Practice, 1(1), 1–11. https://doi.org/10.1080/25741292.2018.1437946

38.

Petersen

C. J.

Russel

Jensen

Branth Pedersen

Kirsop-Taylor

Hardiman

Dai

Kaltenegger

Grandin

Barra

Banzhaf

(2023). Co-creation and evaluation of innovative policy for Nature-based Solutions (NBS); and experiential learning in policy for NBS. REGREEN Deliverable D6.3.

39.

Pillai

(2015). Engaging public, professionals, and policy-makers in the mapping process. Cultural mapping as cultural inquiry, 1(1), 153–169. https://www.taylorfrancis.com/chapters/edit/10.4324/9781315743066-10/engaging-public-professionals-policy-makers-mapping-process-janet-pillai

40.

Russel

(2022). Policy integration for sustainable development governance. In Russel D

Kirsop-Taylor

(Eds.), Handbook on the governance of sustainable development (pp. 132–144). Edward Elgar Publishing. https://doi.org/10.4337/9781789904321.00017

41.

Santoro

Pluchinotta

Pagano

Pengal

Cokan

Giordano

(2019). Assessing stakeholders’ risk perception to promote nature based solutions as flood protection strategies: The case of the Glinščica river (Slovenia). The Science of the total environment, 655(1), 188–201. https://doi.org/10.1016/j.scitotenv.2018.11.116

42.

Shaw

Corner

(2017). Using narrative workshops to socialise the climate debate: Lessons from two case studies – centre-right audiences and the Scottish public. Energy Research & Social Science, 31(2), 273–283. https://doi.org/10.1016/j.erss.2017.06.029

43.

Strother

(2018). The national flood Insurance Program: A case study in policy failure, reform, and retrenchment. Policy Studies Journal, 46(2), 452–480. https://doi.org/10.1111/psj.12189

44.

Sutherland

W. J.

Butchart

S. H.

Connor

Culshaw

Dicks

L. V.

Dinsdale

Doran

Entwistle

A. C.

Fleishman

Gibbons

D. W.

Jiang

Keim

Roux

X. L.

Lickorish

F. A.

Markillie

Monk

K. A.

Mortimer

Pearce-Higgins

J. W.

Peck

L. S.

Gleave

R. A.

(2018). A 2018 horizon scan of emerging issues for global conservation and biological diversity. Trends in Ecology & Evolution, 33(1), 47–58. https://doi.org/10.1016/j.tree.2017.11.006

45.

Uwasu

Kishita

Hara

Nomaguchi

(2020). Citizen-participatory scenario design methodology with future design approach: A case study of visioning of a low-carbon society in Suita city, Japan. Sustainability, 12(11), 4746. https://doi.org/10.3390/su12114746

46.

Vidal

R. V. V.

(2006). The future workshop: Democratic problem solving. Economic Analysis Working Papers, 5(4), 21. https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=438d8ef47cfe2f1db544687b9a64a100bad4188f

47.

Yin

R. K.

(1994). Designing Single-and multiple-case. In Bennett

Glatter

Levacic

(Eds.), Improving educational management: Through research and consultancy (pp. 135–155). Sage. https://books.google.com/books?hl=en&lr=&id=mcUH125X6_8C&oi=fnd&pg=PP7&dq=Yin,+R.+K.+1994.+Designing+Single-and+Multiple-Case.+In+N.+Bennett,+R.+Glatter+%26+R.+Levacic+Improving+educational+management:+Through+research+and+consultancy+&ots=XkVcHZzMJ6&sig=rdBoVJCFM84PJcj9DhDZcdQPu_w

48.

Yin

R. K.

(2009). Case study research: design and methods (5). Sage. https://books.google.com/books?hl=en&lr=&id=FzawIAdilHkC&oi=fnd&pg=PR1&dq=Yin,+R.K.+2009.+Case+study+research:+design+and+methods+(Vol.+5).+Sage&ots=l-_R6ciSZt&sig=91cIlqyUJTk9Stzxw1cUvb024Vk

49.

Yin

R. K.

(2012). Case study methods. In Cooper

Camic

P. M.

Long

D. L.

Panter

A. T.

Rindskopf

Sher

K. J.

(Eds.), APA handbook of research methods in psychology vol 2Research designs: Quantitative, qualitative, neuropsychological, and biological (pp. 141–155). https://doi.org/10.1037/13620-009

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.72 MB