Sage Journals: Discover world-class research

Abstract

Practice-based case studies that describe learning from implementation are a useful source of evidence for policy makers, practitioners and researchers. Despite the value of narratives developed in context as a form of experiential or applied evidence, little is known about appropriate qualitative review and synthesis methods to deal with multiple practice-based case studies. This paper reports on a methodological study to develop and pilot a synthesis method using a sample of community wellbeing case studies focused on interventions to improve community infrastructure. The study was conducted in four interlinked phases: (i) literature review to scope synthesis methods (ii) piloting search and selection methods to identify a sample of relevant practice-based case studies that reported experiential learning (iii) undertaking cross case analysis and qualitative synthesis based on framework analysis methods (iv) review and reflection to produce a summative account of the method and agreed definition. The main output from the study was a staged approach to qualitative synthesis of practice-based case studies based on seven steps, which move from identification of a conceptual framework through to producing a narrative report. The potential transferability of this approach and its application in research and policy are critically discussed. Synthesis of case studies derived from community-based interventions could address knowledge gaps in the formal evidence base. While further methodological development is warranted, it is argued that study results form a credible qualitative framework for synthesising practice-based evidence.

Keywords

evidence synthesis case study practice-based evidence community wellbeing framework analysis

Introduction

Case studies that report from the perspective of practice are widely used as a means of communicating key features of programme implementation and illustrating broader learning around intervention models. Practice-based case studies report experiential knowledge from professional and lay stakeholders involved in project or programme development and delivery (Ng & de Colombani, 2015; Simpson et al., 2013). Typically, this involves telling a story about activities and change occurring within a specific context or setting (De Leeuw et al., 2015; Zwald et al., 2013), accompanied by some reflections on learning. Such case studies can be distinguished from the more familiar research-based case studies, which investigate social phenomena in context through a systematic process of enquiry (Van Wynsberghe & Khan, 2007; Yin, 2018). Research-based case studies can also encompass applied research conducted by and with practitioners to provide in-depth analysis of a programme and its impacts (Boblin et al., 2013; Lee & Chavis, 2012). In contrast, practice-based case studies, which are the focus of this paper, are primarily built on experiential or tacit knowledge of implementation rather than a systematic process of enquiry (Simpson et al., 2013). They are typically written by practitioners and other stakeholders engaged in implementation. Such case studies contain descriptive information on a specific programme (derived from monitoring data, informal feedback or small scale evaluation), and use a narrative structure that highlights emergent learning (Zwald et al., 2013). Published collections of practice-based case studies, also called practice examples, can be found in many fields. Examples of themed collections from the UK, which illustrate how experiential learning from practice is reported, include: public health (UK Health Security Agency Knowledge & Library Services, undated), social care (Social Care Institute for Excellence, undated), local government innovation (Local Government Association, 2024), community wellbeing - places and spaces (What Works Centre for Wellbeing, undated a), and coproduction (Scottish Co-production Network, 2024).

Exploring the use of case study evidence to inform health systems, Simpson, Kelly et al. (2013) argue that case studies which gather tacit knowledge and learning from practice are often overlooked as a source of evidence, in part due to a lack of understanding about how best to process this type of evidence. Where multiple case studies are available, this offers opportunities to understand common mechanisms, outcomes and factors shaping implementation (Lee & Chavis, 2012; Morestin et al., 2010; Simos et al., 2015). However, there is scant methodological literature specific to the review and synthesis of multiple practice-based case studies (Denyer & Tranfield, 2006; Shankardass et al., 2014), particularly in relation to secondary analysis of published case studies or practice examples. Assumptions that synthesis research methods developed either for primary qualitative research or secondary systematic reviews can be simply applied to practice-based evidence need exploring. A major challenge is dealing with heterogeneity where practice-based case studies have been produced for different purposes and in different conditions. This is a pertinent issue when seeking to understand wider lessons from community-based interventions that evolve in response to community needs and contextual factors (George et al., 2018).

This paper explores the potential to synthesise practice-based evidence, reporting on a methodological study to develop and pilot a synthesis method with a sample of community wellbeing case studies. These published case studies all reported on the development and implementation of community-based wellbeing interventions in context. The study was undertaken as part of an evidence programme commissioned by the What Works Centre for Wellbeing, UK. As a ‘what works’ centre aimed at improving evidence on wellbeing measurement and interventions (What Works Centre for Wellbeing, undated b), this centre had an interest in what can be learnt from practice-based case studies that potentially provided context-rich information on implementation and impacts. Two public hearings held in 2017-18 had exposed a pool of experiential learning about promoting wellbeing that was generated through community-based organisations and settings (Gamsu et al., 2019). However, there appeared to be no accepted method for gathering, analysing and synthesising such evidence, in contrast to the extensive guidance on reviewing wellbeing research literature (Snape et al., 2019), using established methods such as narrative qualitaive synthesis (Popay et al., 2006). This study was initiated to begin to address this gap in knowledge and identify a relevant synthesis method that could be used to process and review published practice-based case studies reporting experiential learning.

Prior to presenting the pilot study design and results, which is the main focus of this paper, we provide a brief overview of some common methodological debates that have framed this study and set out a rationale for the potential value of synthesising practice-based evidence. The methodological contribution of the pilot study is positioned within broader debates about what types of evidence are validated and valued. Acknowledging the positioning of this study is important because if practice-based case studies are regarded as a weak form of evidence, within a hierarchy of evidence, this undermines arguments for the benefits of synthesis.

Study Context

The notion of an evidence hierarchy, which was the original foundation for the evidence movement in health and privileges certain study designs such as randomised controlled designs, has been widely critiqued (Hansen, 2014; Thomson et al., 2004). A more common position recognises the value of different types of evidence, generated within both qualitative and quantitative traditions, and therefore focuses on which study designs are appropriate to answer different types of questions (Parkhurst & Abeysinghe, 2016; Petticrew & Roberts, 2003). This has led to the notion of ‘best available evidence’, which has been adapted for rapid reviews undertaken to inform policy makers (Thomson et al., 2004). The implications are that the knowledge base for a subject is likely to comprise a range of research of variable quality and relevance, and additionally may include information that is not derived from research, such as expert opinion (Kelly et al., 2010), lay perspectives (Springett et al., 2007) or practice-based learning (Ammerman et al., 2014).

In general, case studies are regarded as important sources of evidence that can build in depth understanding of social processes (Hansen, 2014; Morestin et al., 2010). The term ‘case study’ encompasses diverse research strategies and study designs (Van Wynsberghe & Khan, 2007; Yin, 2018) as well as practice-based outputs such as a ‘success story’ or illustrative example (Lewis et al., 2004; Zwald et al., 2013). While the conceptualisation of case studies is subject to much methodological debate (Crowe et al., 2011; Ebneyamini & Sadeghi Moghadam, 2018; Gerring, 2004; Van Wynsberghe & Khan, 2007), Simons (2009) sees a commitment to studying the complexity inherent in ‘real-world’ situations as a common thread. This means that practice-based case studies, derived primarily from experiential evidence, sit alongside of other types of case study where research methods are used to provide in-depth exploration of a setting, organisation or programme (Yin, 2018). We recognise that practitioners can become actively involved in conducting, and leading, case study research in practice settings, although this is not the focus of this paper.

Case studies are used often in policy to provide insights into questions of ‘how’ and ‘why’ interventions work over time in diverse settings (Goodrick, 2014; HM Treasury, 2011). In public health, practice-based case studies, also called practice examples, exist as ‘stories from the field’ which use the narrative form to explain context, implementation and results of specific programmes (Zwald et al., 2013). Dissemination of such case studies can support effective knowledge translation and inform decision making (Korjonen et al., 2016; Ng & de Colombani, 2015).

This study focuses on accounts of promoting community wellbeing in community settings, where interventions are not always supported by formal evaluative studies (Preston et al., 2010). If community-based interventions are viewed, as George et al. (2018) suggest, as social processes which ‘dynamically evolve’ within a given social context, then both practice-based and research-based case studies will be important sources of evidence to illuminate complex development pathways in communities. McLean and McNeice (2012), for example, show how learning can be gathered across diverse asset-based projects in different communities. There is also a long tradition of using methods based on community stories where marginalised populations generate collective narratives of change (Scott & Proescholdbell, 2009).

One challenge for synthesis of practice-based case studies is that much of the experiential learning from community projects is underreported and not easily accessible through academic or grey literature databases. Previous work by Public Health England (UK Health Security Agency Knowledge & Library Services, undated), which informed this study, shows the potential to gather and curate a library of community-centred practice-examples (South et al., 2023). Being able to identify, access, or in some cases collate, collections of practice-based case studies is an important precondition to synthesis.

Having considered the potential value of practice-based evidence, particularly for community-based interventions, it is useful to explore what a synthesis of practice-based case studies might offer, drawing on discussions in methodological and policy literature around the synthesis of research-based case studies. Firstly, a synthesis of practice-based case studies could distil information on key features and learning about a particular approach, setting or outcome, that may enhance transferability for others working in the field (Morestin et al., 2010; Simpson et al., 2013). Secondly, a case study synthesis builds understanding of factors that influence uptake and adaption of a programme into real life settings (Chatterji, 2008; Lee & Chavis, 2012; Shankardass et al., 2014). Goodrick (2014) argues that comparative analysis can help to identify patterns between case studies and understand how context influences the success or failure of a programme, all of which can be useful for funders and policy makers. For community-based interventions, a case study synthesis could help unpack how the components of an intervention interrelate within complex community systems (George et al., 2018). Thirdly, synthesis of practice-based case studies has potential to complement the evidence derived from systematic and rapid reviews. Reviews are deemed useful for decision makers because the review process filters, appraises and summarises existing knowledge on a topic (Mulrow, 1994). A case study synthesis could mirror this process of review and abstraction, what Hansen (2014) describes as moving from individual studies to ‘knowledge streams’ (p. 12). Narratives on history, culture and context are often missing from traditional intervention studies on community participation (Rifkin, 2014), and therefore a practice-based synthesis could help address knowledge gaps. In summary, there is significant potential for synthesis of practice-based case studies as a valid form of experiential evidence; however, consideration is needed about appropriate review methods that can identify, process and analyse heterogenous narratives generated within practice. This paper now reports on how we developed and piloted a method to collate, review and synthesise practice-based case studies.

Design and Pilot

The primary research question for the study was: ‘what are the best methods of identifying, reviewing, synthesising and reporting methods and approaches seen in community-based practice?‘. The focus was on practice-based case studies that reported on the development and implementation of community wellbeing projects. In the first instance, a study protocol with interim definitions was developed (South, Bagnall, Southby, et al., 2019), informed by scoping review methodologies (Arksey & O'Malley, 2005; The Joanna Briggs Institute, 2015). The final definition of practice-based case studies was based on understandings of the nature of practice-based evidence and how it is generated through narratives:

Practice-based case studies report on the evidence generated from the implementation of an intervention in a real-life practice setting and include the learning from those involved in the development and delivery of that intervention. Such case studies typically provide a narrative explaining how the intervention developed in that context and what happened. They are most often developed by practitioners involved in an intervention, but can also be developed in collaboration with funders, third sector organisations or researchers aiming to capture practice-based knowledge.

Throughout the pilot study, an inductive approach was taken that allowed emergent learning to refine design choices at each stage. An advisory group, with membership including academics, policy makers, What Works Centre for Wellbeing staff and the civil society research partner, Locality, was critical to this process. The advisory group agreed the study protocol and definition of practice-based case studies, and our focus on published ‘practice stories’ reflecting experiential learning rather than research-based case studies. This was seen to fill the gap in review methods for the What Works Centre for Wellbeing (Snape et al., 2019). The study was designed and delivered in four interconnected phases, which are described in turn. We explain how each phase informed the study and led to the final synthesis method presented in this paper. Ethical approval was obtained through Research Ethics Procedures of Leeds Beckett University (Ref: 61643).

Phase 1- Scoping Methods

The first phase was to scope and select appropriate methods for case study collection and synthesis. This was undertaken through a rapid narrative review (Gough et al., 2012) to identify the most relevant methodological papers, with a primary focus on collection and synthesis of practice-based case studies in health and social sciences. An iterative search strategy was used, starting with an initial list of 21 key studies identified by the research team and advisory group and then supplemented by a further 19 publications identified through citation searching.

Results in this phase highlighted the potential value of synthesis from the perspective of various social science disciplines with regard to qualitative and mixed method case studies (Chatterji, 2008; Gilson, 2014; Goodrick, 2014; Hoon, 2013; Lee & Chavis, 2012; Rousseau et al., 2008; Yin, 2018). Some papers described the application of qualitative analysis methods to research generated in practice (Boblin et al., 2013; McLean & McNeice, 2012; Neale et al., 2006). Notwithstanding this established literature, we found scant discussion of synthesis in relation to practice-based case studies derived from tacit or experiential knowledge of implementation (Simpson et al., 2013). A key theme from literature was the balance that needed to be achieved between rich contextual detail presented in a single case and potential benefits from drawing learning across multiple case studies (Hoon, 2013; Khan & VanWynsberghe, 2008). We considered that this theme was also relevant to practice-based case studies as accounts rooted in context.

We identified four broad comparative approaches to synthesis that had all been used to analyse research-based case study evidence: narrative synthesis (Denyer & Tranfield, 2006; Popay et al., 2006); meta-ethnography (Simpson et al., 2013); realist synthesis (Chatterji, 2008; Shankardass et al., 2014) and cross-case analysis (Khan & VanWynsberghe, 2008; Lee & Chavis, 2012; Morestin et al., 2010). The first three of these approaches have also been used for secondary analysis as part of qualitative systematic and non-systematic reviews (Barnett-Page & Thomas, 2009; Popay, 2006; Popay et al., 2006). Assumptions that existing evidence derived from community practice would be heterogenous suggested the utility of combining a case-oriented approach with some analysis across variables (Khan & VanWynsberghe, 2008; Lee & Chavis, 2012), and this influenced methods in Phase 3.

A further consideration was around purposive sampling to select case studies suitable for synthesis (Goodrick, 2014; McLean & McNeice, 2012). Potential sampling criteria identified in this literature review phase were later applied in the selection criteria and data extraction in Phases 2 &3. These included:

• a common focus or shared characteristics (Stewart, 2012)

• having sufficient information on core areas of interest (Goodrick, 2014; Shankardass et al., 2014)

• reported outcomes and impact including counterfactual information (Goodrick, 2014; Ng & de Colombani, 2015)

• involvement of community members or other stakeholders in case study production (Ng & de Colombani, 2015)

• description of the data collection method used to allow others to judge credibility of evidence (HM Treasury, 2012; Stewart, 2012).

Phase 2- Gathering Practice-Based Case Studies

Case study synthesis methods outlined in the protocol, and then refined through the scoping review and advisory group discussions, were piloted in phases 2 & 3. The agreed topic was community-based interventions that aim to improve community wellbeing and social relations through improving community infrastructure (places and spaces). This topic built an earlier systematic review, published by the What Works Centre for Wellbeing, which had identified a loose typology of eight intervention types (Bagnall et al., 2018). Although these were not assumed to be representative of all community wellbeing interventions, the topic illustrated some common features in terms of implementation in existing community settings, diverse contexts shaping local activity, and the involvement of communities in design and delivery.

We undertook a preliminary scoping exercise to identify suitable collections of practice-based case studies. Compared to existing methods of searching academic and grey literature (Adams et al., 2017; Finfgeld-Connett & Johnson, 2013), there was no specific guidance on searching for practice-based case studies, which are often published in diverse formats through web-based platforms and other media. Using a Public Health England list (UK Health Security Agency Knowledge & Library Services, undated) and suggestions from the advisory group, potential case study collections were identified and their main information fields were mapped. This led to 17 online collections and 11 reports being searched using key words on community wellbeing and social relations.

Inclusion criteria were drawn up based on learning from Phase 1. The primary inclusion criterion was the intervention approach as this avoided a highly heterogenous sample that would be unsuitable for synthesis (Hoon, 2013). A further criterion was having sufficient information reporting outcomes and/or learning to allow for meaningful analysis to take place (Goodrick, 2014; Shankardass et al., 2014). Brief practice-based case studies with minimal description of processes or outcomes, for example one or two sentences listing outcomes without detail, were excluded.

Screening generated a list of 61 published case studies, which were mapped to the eight intervention types identified in the review (Bagnall et al., 2018). Further purposive sampling to achieve a manageable sample size for the pilot, led to two contrasting intervention types being chosen: community hub interventions (n = 14) and green & blue space (n = 7) interventions. Although the pilot was focused on secondary analysis of published evidence, a parallel process took place with Locality that resulted in three new practice-based case studies on community hubs, completed by practitioners, which were included in the synthesis.

Phase 3 - Analysis and Synthesis

Informed by the scoping of literature in Phase 1, we chose an analytical approach based on cross-case analysis in order to identify patterns between cases without losing the essential contextual details in a heterogenous sample (Khan & VanWynsberghe, 2008). Coding and theme development was informed by framework analysis (Gale et al., 2013; Ritchie et al., 2003), as this method was originally developed for applied research and fitted with the structured reports of many practice-based case studies. The process was supported by use of matrices to organise data within cases and across variables or themes (Miles et al., 2014). A preliminary stage involved developing and piloting a data extraction template (Table 1) based on common descriptive fields identified through the scoping review (Phase 1) and mapping of websites (Phase 2).

Table 1.

Data Extraction Fields.

Data extraction fields	Subfields
Setting
Purpose	Identified problem/need
Purpose	Aim/goals
Project description: When? What? Who?	Funding
Project description: When? What? Who?	Approach taken
Participants
Data collection methods	Availability of additional reports
Reported outcomes/impact	Intervention reach
	Individual-level outcomes
	Community-level outcomes
	Organisational-level outcomes
	Unintended consequences
Influencing factors	Enablers
	Barriers
	What works - project approach
Key learning (as reported in case study)
Sustainability
Further information

Cross case analysis of the sample of practice-based case studies was undertaken iteratively with increasing levels of abstraction. The first round involved applying the data extraction template to chart all case study data for each intervention (community hubs and green spaces), displayed in an Excel sheet with a line for each case study and columns for each field. Data from the case study reports were summarised in each cell, highlighting key phrases and potential themes (Gale et al., 2013). In effect, this process aligned the data extraction process typically used in systematic reviews of qualitative and mixed methods studies (Baxter et al., 2010) with the first stages of producing a thematic framework (Gale et al., 2013; Ritchie et al., 2003).

The second round of analysis involved developing a series of thematic matrices, initially with the green space case studies and then further developed with the community hub sample. Using the first stage charts, summary statements were grouped, and then themes and sub-themes were identified and labelled with numeric codes. Three new matrices were formed that charted themes on: Purpose & approach (project rationale, features and activities); Outcomes (reported outcomes at individual, community and organisational levels) and Learning & what works (process factors and interpretive themes on mechanisms). Individual case studies continued to be displayed in these matrices and a final column was created with interpretive memos.

This was followed by the creation of the overall thematic framework, where we left the case-by-case analysis in order to merge and reorganise higher order themes and sub themes in an analytic hierarchy (Ritchie et al., 2003). The three overarching thematic categories (purpose & approach; outcomes; learning & what works) were retained with major themes to explain the data. Descriptive information on individual projects was not included in this framework. We presented the emergent analysis to the advisory group and then further refined it. To ensure reliability, all stages of coding and analysis were checked by three researchers (CF, KS, JS).

For the final stage of synthesis, we produced a narrative report of results for each intervention, structured according to the overarching thematic framework and including: an overview of the case study attributes; description of major themes; contextual detail on the individual cases; illustrative quotations; and summary tables. This combination of abstraction in a thematic account and including specific detail about individual case studies, particularly where that illustrates divergent themes or contextual information, is common to reporting of research-based case studies (Boblin et al., 2013; Lee & Chavis, 2012).

Phase 4 - Application and Recommendations

The final phase involved producing a summative account of the case study synthesis approach and application. To aid transparency, we documented significant points of discussion with the advisory group. As a research team, we also documented our reflections on emergent issues and agreed a reflexive statement about practice-based evidence. Discussion in this phase confirmed the value of practice-base evidence for policy makers, practitioners and researchers. Consideration was given to the merits of structured case studies, where data were collected using systematic processes, and valuing the creativity seen where participants told their own story.

In reviewing phases 1-3, we documented significant issues including searching and availability of case studies, use of templates and choice of analytical methods. Websites often had poor functionality for searches and that led to recommendations to improve the curation of practice-based collections. A further recommendation was on use of templates both for developing new case studies and as a means to chart data in the initial stages of analysis. Advisory group discussions highlighted the importance of several fields that ideally should be covered in a synthesis including unanticipated or negative outcomes.

Results – a Staged Approach to Practice-Based Case Study Synthesis

The four phases of the study resulted in a method for synthesis of practice-based case studies, distilled into a series of stages, presented in Table 2. Details of findings and summary tables specific to green space and community hub interventions are reported elsewhere (South et al., 2021).

Table 2.

A Staged Approach to the Synthesis of Practice-Based Case Studies.

Stages	Notes
(i) Identify or develop a conceptual framework that helps define, categorise and select interventions of interest	A conceptual framework helps define, categorise and select interventions or outcomes of interest. In some instances, an existing logic model or theory of change could be used to map concepts of interest including mechanisms and expected outcomes
(ii) Identify websites and case study collections	Where available, online databases, repositories and other collections offer a straightforward means to access case studies using key word searches; however, many practice-based collections are not curated in a systematic way. Practice-based case studies tend not to be located in academic data bases, but it may be possible to extend searches of grey literature or issue a call for evidence
(iii) Search and select case studies that group round a topic or intervention approach	A search strategy and selection process will result in a sample of relevance to the topic. Practice-based case studies vary in terms of content, format, and quality of data/reporting. Well-defined inclusion and exclusion criteria, linked to a conceptual framework (i), should be applied to select a group suitable for cross case analysis. Case studies can be excluded where there is insufficient information to support data analysis
(iv) Organise the case study data using a template with common fields/domains	Structured templates help to organise and display the data extracted from practice-based case studies. Common domains include aims, setting, approach, participants, activities, outcomes, influencing factors and learning. Structured practice-based case studies are easier to code than those based on looser narrative accounts
(v) Use cross case analysis with matrices to develop the analysis and synthesis	Cross-case analysis seeks to identify the patterns and differences between case studies, retaining links to the contextual information within individual cases. Matrices support that process, as data are summarised, coded and displayed by theme and case. The pilot used framework analysis as a recognised qualitative analysis method appropriate for applied research
(vi) Develop an overarching framework that explains the data and can be adapted as more case studies are analysed	Synthesis involves producing an explanatory framework that fits with the data in the sample. This means reordering and clustering themes and looking at the interrelationships to build interpretations. The final output from this stage will be a coherent thematic framework, which could be presented in a table, logic model or as a conceptually ordered figure
(vii) Write a narrative report of themes, with illustrative quotations alongside contextual information	The final stage is producing a narrative textual account of results, grouped around higher order themes or categories. Given the importance of context for practice-based case studies, relevant contextual information and illustrative quotations should be included. To aid transparency of reporting, all published case studies included in the analysis should be listed

Discussion

The premise of this study was that practice-based case studies are a valued source of evidence on implementation in community settings, but the potential benefits of synthesising this type of evidence can only be realised by the application of methods suited to narrative accounts from practice. Other qualitative research methods are available for analysis of applied primary research in practice settings (Miles et al., 2014; Yin, 2018), and also secondary analysis of research-based case studies (Barnett-Page & Thomas, 2009). As illustrated by this study, practice-based case studies are likely to include descriptions of interventions in context (McLean & McNeice, 2012), detail on implementation (Zwald et al., 2013) and reports of experiential learning from practitioners or community members (Simpson et al., 2013). The findings of the literature review (Phase 1) highlighted knowledge gaps on how this type of evidence can be processed and a range of possible analytical approaches, although most related to case studies as a research design. A lack of specific guidance on synthesis of practice-based case studies, which took account of the nature of those accounts, led to us selecting methods that were deemed feasible and then refining these methods as we worked through the stages. In doing so, we drew heavily on qualitative traditions in the application of cross-case analysis (Van Wynsberghe & Khan, 2007; Yin, 2018) and use of matrices (Miles et al., 2014). The choice of framework analysis (Gale et al., 2013) fitted with the structured templates recommended for writing practice-based case studies (De Leeuw et al., 2015; Zwald et al., 2013), and also aligned to framework synthesis approaches used in qualitative systematic reviews (Baxter et al., 2010; Dixon-Woods, 2011). Realist synthesis (Denyer & Tranfield, 2006) might offer an alternative approach to heterogenous practice-based evidence that is rooted in specific contexts.

Practice-based case studies can include descriptive quantitative data, often derived from programme monitoring data or user surveys. Appropriate quantitative methods could be used display and analyse these data across case studies; for example, numbers of participants or reported project costs. There was very limited quantitative data reported in the community-based case studies from our sample. There would need to be caution in interpreting quantitative data as transparent accounts of methods, which help judge validity and reliability, may be missing in practice-based case studies. Overall, we recognise that a range of established qualitative and quantitative analysis methods that are used in primary case study research could be drawn on. However, these methods would need applying within a review framework appropriate for processing secondary sources of experiential evidence.

One piece of learning from the pilot was the importance of identifying a conceptual framework to underpin selection and analysis. Conceptual frameworks support categorisation of interventions (Helitzer et al., 2014), and this is helpful given the inevitable heterogeneity of cases in a secondary analysis of practice-based evidence.

This pilot was informed by prior understandings of community wellbeing as a multidimensional, relational concept (Atkinson et al., 2019) and the contested nature of evidence (Hansen, 2014). A typology from a prior systematic review was used, which identified eight intervention types focused on developing community wellbeing through improving social relations in community infrastructure (Bagnall et al., 2018). Having a typology helped deal with the lack of shared terminology for community-centred approaches and the variation in community practice (McLean, J., & McNeice, 2012). In developing additional selection criteria, the advisory group and research team agreed that applying exclusion criteria based on the quality of study design, as used in systematic reviews of effectiveness, was inappropriate. Instead, one inclusion criterion was having sufficient information on outcomes or learning reported in the case study (Goodrick, 2014; Ng & de Colombani, 2015; Shankardass et al., 2014). This was not tightly defined and therefore open to interpretation; however, it served as a heuristic aid to identify those cases that contained explicit reports of relevance to the review and to exclude superficial or short case studies which simply illustrated or promoted a project.

Practice-based evidence, which is not necessarily derived through robust data collection processes, may lack the objectivity associated with research studies. The risks of a positive bias to reporting successes in practice-based case studies has been noted elsewhere (Centers for Disease Control and Prevention, 2008). Further research on what quality might mean and how it would be assessed in a practice-based synthesis is merited (Ng & de Colombani, 2015). As in systematic and non-systematic reviews, the sampling or selection strategy should be reported for transparency (Higgins et al., 2019).

In terms of study limitations, searching and retrieval of practice-based evidence was challenging. The pilot synthesis was limited in its focus on published community wellbeing case studies presented in narrative form. Compared to access to academic data bases, which provide a huge potential pool of research-based studies, there appear to be few curated repositories for practice-based evidence. Prior work by Public Health England, involving one of the authors [JS], had resulted in a library of community-centred practice examples and lists of alternative collections (UK Health Security Agency Knowledge & Library Services, undated). This list was used to scope collections, which meant there was a bias towards evidence in the health field. Further development of robust search techniques is needed to aid more systematic retrieval of practice-based case studies. Organisations routinely publishing practice-based evidence or hosting repositories should consider improving the curation of case studies to facilitate searching and retrieval.

Overall, analysis of practice-based evidence is an underdeveloped area, as demonstrated by the literature review. We were unable to identify definitive descriptions of this type of evidence and there was scant methodological literature directly related to the topic. In line with rapid review methodology (Gough et al., 2012), we were only able to look at key literature and there is scope for a more comprehensive scoping review on the topic. While the proposed definition presented in this paper contributes some clarity, the distinction between practice-based and research-based evidence can be viewed on more of a continuum, particularly where research-based studies have been undertaken on community practice (McLean & McNeice, 2012). There is also a potential overlap with some of the issues around managing and reviewing grey literature (Adams et al., 2017). Overall, there is scope for development of a theoretical or conceptual framework covering the full range of practice-based evidence to underpin categorisation and analysis of practice-based case studies. This should explore the differences and similarities to research-based studies conducted in practice settings.

Application

The staged approach to synthesis of practice-based case studies presented in Table 2 has potential application to other topics where there are existing sources of practice-based case studies, including in public health, social care, local government, arts and culture, housing and neighbourhood development. Our focus was on community wellbeing interventions and reported methods will have high relevance for those wishing to review other community-based interventions. Collaborative work with communities is typically built in a developmental and iterative way (George et al., 2018). Synthesis could help to distil information from diverse community-led and grassroots projects, an area where there are notable evidence gaps (Rifkin, 2014). The innovative case study synthesis method outlined here has already been used to structure an evidence synthesis of practice-based case studies of volunteering and wellbeing in Wales during the pandemic (Taylor-Collins et al., 2021). The UK ‘Mobilising community assets to tackle health inequalities programme’, funded through the Arts and Humanities Research Council (UK Research and Innovation, 2022), has also used this approach to document and synthesise across diverse arts and health research projects (Mughal et al., 2024). There is further scope for assessing transferability in other contexts.

The study aimed to complete a full cycle of methods identification, testing and review. We make no claims to this being a definitive approach, as more testing and development would be required. Nonetheless, this transparent account of methodological choices offers a starting point for others wanting to synthesise practice-based case studies. Search and selection strategies described in this paper, alongside an improved understanding of the collections that host practice-based case studies, will be useful for researchers wanting to incorporate grey literature in their reviews. There is also potential to apply the approach in policy analysis across multiple case studies (De Leeuw et al., 2015; Gilson, 2014). The data extraction template (Table 1) could be applied and adapted for other topics, particularly when a rapid review of practice is needed, as occurred during the pandemic (Taylor-Collins et al., 2021).

What Works Centre for Wellbeing has since incorporated the approach into guidance for researchers undertaking synthesis of practice-based wellbeing evidence (Hardoon et al., 2021). Having guidance helps improve the rigour of a review process and this complements existing guidance on systematic reviews (Snape et al., 2019). There is also guidance for practitioners, funders and policy makers covering the features of practice-based case studies and pointers as to production and dissemination (What Works Wellbeing, 2021).

Conclusions

Our enquiry was founded on an understanding of the value of practice-based evidence as a source of knowledge, one that is rich with the insights that emerge from learning by doing. In responding to a gap in methodological literature and guidance around how to undertake synthesis of stories from the field, we have set out an innovative staged approach to qualitative review and synthesis of practice-based case studies. This approach accounts for the likely heterogeneity of source material, the lack of databases for easy searching, the deep significance of context and the need for rigorous qualitative analysis to build explanations of intervention implementation and adaption. The logic is that by adopting this approach, a researcher will move in a series of steps from gathering and selecting data sources to eventually producing higher order themes of relevance to the original review questions.

Overall, the opportunities and challenges set out in this paper have broad relevance for social science and health research and practice, particularly in relation to synthesising learning from community-based interventions developed in different contexts and settings. Areas for future research include assessing transferability of the approach, quality assessment, and stakeholder engagement to evaluate the utility of the method. There remains considerable scope for methodological development and we hope that this paper stimulates interest in practice-based case studies and what synthesis can offer researchers, policy makers and practitioners.

Footnotes

Acknowledgements

The study was undertaken for the What Works Centre for Wellbeing,UK,as part of their ‘Communities of Place’ evidence stream. The authors would like to thank the chair of the advisory group,Ingrid Abreu Scherer,What Works Centre for Wellbeing,and all advisory group members for their support and contributions throughout the study. Locality,UK,a national network of local community organisations,were the civil society research partner for the study. The authors would like to thank Locality staff and members who provided case studies about their community-based organisations.

Author Contributions

Charlotte Freeman contributed to the study whilst working as a researcher in the School of Health,Leeds Beckett University.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research,authorship,and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research,authorship,and/or publication of this article: The study was commissioned by the What Works Centre for Wellbeing,UK and supported by a grant from the Economic and Social Research Council. Grant ref: ES/N003756/1.

ORCID iD

Jane South

References

Adams

R. J.

Smart

Huff

A. S.

(2017). Shades of grey: Guidelines for working with the grey literature in systematic reviews for management and organizational studies. International Journal of Management Reviews, 19(4), 432–454. https://doi.org/10.1111/ijmr.12102

Ammerman

Smith

Calancie

(2014). Practice-based evidence in public health: Improving reach, relevance, and results. Annual Review of Public Health, 35, 47–63. https://doi.org/10.1146/annurev-publhealth-032013-182458

Arksey

O'Malley

(2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19–32. https://doi.org/10.1080/1364557032000119616

Atkinson

Bagnall

Corcoran

South

Curtis

(2019). Being well together: Individual subjective and community wellbeing. Journal of Happiness Studies, 21(5), 1903–1921. https://doi.org/10.1007/s10902-019-00146-2

Bagnall

South

Di Martino

Southby

Pilkington

Mitchell

Pennington

Corcoran

(2018). A systematic review of interventions to boost social relations through improvements in community infrastructure (places and spaces). What Works Centre for Wellbeing.

Barnett-Page

Thomas

(2009). Methods for the synthesis of qualitative research: A critical review. BMC Medical Research Methodology, 9(1), 59–111. https://doi.org/10.1186/1471-2288-9-59

Baxter

Killoran

Kelly

M. P.

Goyder

(2010). Synthesizing diverse evidence: The use of primary qualitative data analysis methods and logic models in public health reviews. Public Health, 124(2), 99–106. https://doi.org/10.1016/j.puhe.2010.01.002

Boblin

S. L.

Ireland

Kirkpatrick

Robertson

(2013). Using Stake’s qualitative case study approach to explore implementation of evidence-based practice. Qualitative Health Research, 23(9), 1267–1275. https://doi.org/10.1177/1049732313502128

Centers for Disease Control and Prevention . (2008). How to develop a success story. U.S. Department of Health and Human Services.

10.

Chatterji

(2008). Comments on slavin: Synthesizing evidence from impact evaluations in education to inform action. Educational Researcher, 37(1), 23–26. https://doi.org/10.3102/0013189x08314287

11.

Crowe

Cresswell

Robertson

Huby

Avery

Sheikh

(2011). The case study approach. BMC Medical Research Methodology, 11(1), 100. https://doi.org/10.1186/1471-2288-11-100

12.

De Leeuw

Green

Dyakova

Spanswick

Palmer

(2015). European Healthy Cities evaluation: Conceptual framework and methodology. Health Promotion International, 30(Suppl 1), i8–i17. https://doi.org/10.1093/heapro/dav036

13.

Denyer

Tranfield

(2006). Using qualitative research synthesis to build an actionable knowledge base. Management Decision, 44(2), 213–227. https://doi.org/10.1108/00251740610650201

14.

Dixon-Woods

(2011). Using framework-based synthesis for conducting reviews of qualitative studies. BMC Medicine, 9(1), 39. https://doi.org/10.1186/1741-7015-9-39

15.

Ebneyamini

Sadeghi Moghadam

M. R.

(2018). Toward developing a framework for conducting case study research. International Journal of Qualitative Methods, 17(1), 160940691881795. https://doi.org/10.1177/1609406918817954

16.

Finfgeld-Connett

Johnson

E. D.

(2013). Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing, 69(1), 194–204. https://doi.org/10.1111/j.1365-2648.2012.06037.x

17.

Gale

N. K.

Heath

Cameron

Rashid

Redwood

(2013). Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Medical Research Methodology, 13(1), 117. https://doi.org/10.1186/1471-2288-13-117

18.

Gamsu

Southby

South

Newton

Di Martino

Bagnall

(2019). Debating what works for Community Wellbeing: Findings from two national public hearings on community wellbeing, health inequalities and local system. What Works Centre for Wellbeing. Available from: https://whatworkswellbeing.org/product/public-debate-wellbeing/

19.

George

A. S.

LeFevre

A. E.

Schleiff

Mancuso

Sacks

Sarriot

(2018). Hubris, humility and humanity: Expanding evidence approaches for improving and sustaining community health programmes. BMJ Global Health, 3(3), Article e000811. https://doi.org/10.1136/bmjgh-2018-000811

20.

Gerring

(2004). What is a case study and what is it good for? American Political Science Review, 98(2), 341–354. https://doi.org/10.1017/s0003055404001182

21.

Gilson

(2014). Qualitative research synthesis for health policy analysis: What does it entail and what does it offer? Health Policy and Planning, 29(Suppl 3), iii1–iii5. https://doi.org/10.1093/heapol/czu121

22.

Goodrick

(2014). Comparative case studies. UNICEF Office of Research.

23.

Gough

Thomas

Oliver

(2012). Clarifying differences between review designs and methods. Systematic Reviews, 1, 28. https://doi.org/10.1186/2046-4053-1-28, Available from: https://www.systematicreviewsjournal.com/content/1/1/28

24.

Hansen

H. F.

(2014). Organisation of evidence-based knowledge production: Evidence hierarchies and evidence typologies. Scandinavian Journal of Public Health, 42(13 Suppl), 11–17. https://doi.org/10.1177/1403494813516715

25.

Hardoon

South

Southby

Freeman

Bagnall

Pennington

Corcoran

(2021). A guide to synthesising case studies. What Works Centre for Wellbeing. Available from: https://whatworkswellbeing.org/resources/case-study-synthesis-centre-guide/

26.

Helitzer

D. L.

Sussman

A. L.

Hoffman

R. M.

Getrich

C. M.

Warner

T. D.

Rhyne

R. L.

(2014). Along the way to developing a theory of the program: A re-examination of the conceptual framework as an organizing strategy. Eval Program Plann, 45, 157–163. https://doi.org/10.1016/j.evalprogplan.2014.04.005

27.

Higgins

J. P. T.

Thomas

Chandler

Cumpston

Page

M. J.

Welch

V. A.

(Eds.). (2019).Cochrane handbook for systematic reviews of interventions. Cochrane. Available from: https://www.training.cochrane.org/handbook

28.

HM Treasury . (2011). The magenta book: Guidance for evaluation. HM Treasury.

29.

HM Treasury . (2012). Quality in qualitative evaluation: A framework for assessing research evidence (supplementary Magenta book guidance. HM Treasury.

30.

Hoon

(2013). Meta-synthesis of qualitative case studies: An approach to theory building. Organizational Research Methods, 16(4), 522–556. https://doi.org/10.1177/1094428113484969

31.

Kelly

Morgan

Ellis

Younger

Huntley

Swann

(2010). Evidence based public health: A review of the experience of the National Institute of Health and Clinical Excellence (NICE) of developing public health guidance in England. Social Science and Medicine, 71(6), 1056–1062. https://doi.org/10.1016/j.socscimed.2010.06.032

32.

Khan

VanWynsberghe

(2008). Cultivating the under-mined: Cross-case analysis as knowledge mobilization. Forum for Qualitative Social Research, 9(1), 334.

33.

Korjonen

Hughes

Ford

Keswani

(2016). The role of case studies as evidence in public health. UK Health Forum.

34.

Lee

K. S.

Chavis

D. M.

(2012). Cross-case methodology: Bringing rigour to community and systems change research and evaluation. Journal of Community & Applied Social Psychology, 22(5), 428–438. https://doi.org/10.1002/casp.1131

35.

Lewis

S. D.

Johnson

V. R.

Farris

R. P.

Will

J. C.

(2004). Using success stories to share knowledge and lessons learned in health promotion. Journal of Women's Health, 13(5), 616–624. https://doi.org/10.1089/1540999041280954

36.

Local Government Association . (2024). Case studies. Local Government Association. Available from: https://www.local.gov.uk/case-studies

37.

McLean

McNeice

(2012). Assets in action: Illustrating asset based approaches for health improvement. Glasgow Centre for Population Health.

38.

Miles

M. B.

Huberman

A. M.

Saldaña

(2014). Qualitative data analysis. A methods sourcebook (3rd ed.). Sage.

39.

Morestin

Gauvin

Hogue

Benoit

(2010). Method for synthesizing knowledge about public policies. National Collaborating Centre for Healthy Public Policy.

40.

Mughal

Schrerer

I. A.

Smithson

Bagnall

A. M.

South

Chatterjee

H. J.

(2024). Mobilising community assets to tackle health inequalities: A case studies synthesis and review. University College London. Available at: https://ncch.org.uk/uploads/MCA-Case-Study-Synthesis.pdf

41.

Mulrow

C. D.

(1994). Rationale for systematic reviews. British Medical Journal, 309(6954), 597–599. https://doi.org/10.1136/bmj.309.6954.597

42.

Neale

Thapa

Boyce

(2006). Preparing a case study: A guide for designing and conducting a case study for evaluation input. Pathfinder International.

43.

de Colombani

(2015). Framework for selecting best practices in public health: A systematic literature review. Journal of Public Health Research, 4(3), 577. https://doi.org/10.4081/jphr.2015.577

44.

Parkhurst

J. O.

Abeysinghe

(2016). What constitutes “good” evidence for public health and social policy-making? From hierarchies to appropriateness. Social Epistemology, 30(5-6), 665–679. https://doi.org/10.1080/02691728.2016.1172365

45.

Petticrew

Roberts

(2003). Evidence, hierarchies, and typologies: Horses for courses. Journal of Epidemiology and Community Health, 57(7), 527–529. https://doi.org/10.1136/jech.57.7.527

46.

Popay

(Ed.), (2006). Moving beyond effectiveness in evidence synthesis: Methodological issues in the synthesis of diverse sources of evidence. National Institute for Health and Clinical Excellence.

47.

Popay

Roberts

Sowden

Petticrew

Arai

Rodgers

Britten

Roen

Duffy

(2006). Guidance on the conduct of narrative synthesis in systematic reviews. Institute for Health Research, Lancaster University.

48.

Preston

Waugh

Larkins

Taylor

(2010). Community participation in rural primary health care: Intervention or approach? Australian Journal of Primary Health, 16(1), 4–16. https://doi.org/10.1071/py09053

49.

Rifkin

S. B.

(2014). Examining the links between community participation and health outcomes: A review of the literature. Health Policy and Planning, 29(Suppl 2), ii98–ii106. https://doi.org/10.1093/heapol/czu076

50.

Ritchie

Spencer

O'Connor

(2003). Carrying out qualitative analysis. In Lewis

Ritichie

(Eds.), Qualitative research practice. A guide for social science students and researchers (pp. 219–262). Sage.

51.

Rousseau

Manning

Denyer

(2008). Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. Advanced Institute of Management Research Paper No.67. Available from: https://doi.org/10.2139/ssrn.1309606

52.

Scott

S. A.

Proescholdbell

(2009). Informing best practice with community practice: The community change chronicle method for program documentation and evaluation. Health Promotion Practice, 10(1), 102–110. https://doi.org/10.1177/1524839907307677

53.

Scottish Co-production Network . (2024). Co-Production case studies . Scottish Community Development Centre (retrieved 15th August 2024). Available from: https://www.coproductionscotland.org.uk/case-studies

54.

Shankardass

Renahy

Muntaner

O’Campo

(2014). Strengthening the implementation of health in all policies: A methodology for realist explanatory case studies. Health Policy and Planning, 30(4), 462–473. https://doi.org/10.1093/heapol/czu021

55.

Simons

(2009). Case study research in practice. Sage.

56.

Simos

Spanswick

Palmer

Christie

(2015). The role of health impact assessment in Phase V of the Healthy Cities European Network. Health Promotion International, 30(Suppl 1), i71–i85. https://doi.org/10.1093/heapro/dav032

57.

Simpson

Kelly

M. P.

Morgan

(2013). Defining principles for good practice: Using case studies to inform health systems action on health inequalities. Evaluation and Program Planning, 36(1), 191–197. https://doi.org/10.1016/j.evalprogplan.2012.03.007

58.

Snape

Meads

Bagnall

Tregaskis

Mansfield

(2019). What works wellbeing: A guide to our evidence review methods. What Works Centre for Wellbeing.

59.

Social Care Institute for Excellence (undated) . Examples of co-production in social care. Social Care Institute for Excellence. Available from. https://www.scie.org.uk/co-production/examples/

60.

South

Bagnall

Southby

Freeman

Pennington

Corcoran

(2019). Community wellbeing case study synthesis: Study protocol. Leeds Beckett University. Available from: https://eprints.leedsbeckett.ac.uk/6769/

61.

South

Mapplethorpe

Gledhill

Marsh

Stansfield

Evans

Mancini

Outhwaite

(2023). Learning from public health practice: The development of a library of community-centered practice examples. Journal of Public Health, 45(2), 414–422. https://doi.org/10.1093/pubmed/fdac065

62.

South

Southby

Freeman

Bagnall

Pennington

Corcoran

(2021). Community wellbeing case study synthesis. Technical report. What Works Centre for Wellbeing. Available from: https://whatworkswellbeing.org/wp-content/uploads/2018/05/Community-wellbeing-case-study-synthesis-Technical-Report-2021v2.pdf

63.

Springett

Owens

Callaghan

(2007). The challenge of combining ‘lay’ knowledge with ‘evidence-based’ practice in health promotion: Fag Ends Smoking Cessation Service. Critical Public Health, 17(3), 243–256. https://doi.org/10.1080/09581590701225854

64.

Stewart

(2012). Multiple-case study methods in governance-related research. Public Management Review, 14(1), 67–82. https://doi.org/10.1080/14719037.2011.589618

65.

Taylor-Collins

Havers

Durrant

Passey

Bagnall

A.-M.

South

(2021). Volunteering and wellbeing in the pandemic. Part I: Learning from practice. Available from: https://www.wcpp.org.uk/wp-content/uploads/2021/06/Volunteering-and-wellbeing-in-the-pandemic.-Part-1-Learning-from-practice.pdf

66.

The Joanna Briggs Institute . (2015). Joanna Briggs Institute reviewers’ manual: 2015 edition/supplement. Methodology for JBI scoping reviews. The Joanna Briggs Institute.

67.

Thomson

Hoskins

Petticrew

Ogilvie

Craig

Quinn

Lindsay

(2004). Evaluating the health effects of social interventions. British Medical Journal, 328(7434), 282–285. https://doi.org/10.1136/bmj.328.7434.282

68.

UK Health Security Agency Knowledge & Library Services . Practice examples. Retrieved 10^th April 2024. Available from: https://ukhsalibrary.koha-ptfs.co.uk/practice-examples/

69.

UK Research and Innovation . (2022). Investment in culture and nature to boost the nation’s health. From: https://www.ukri.org/news/investment-in-culture-and-nature-to-boost-the-nations-health/

70.

Van Wynsberghe

Khan

(2007). Redefining case study. International Journal of Qualitative Methods, 6(2), 80–94. https://doi.org/10.1177/160940690700600208

71.

What Works Centre for Wellbeing. (undated a) . Practice examples. Available from: https://whatworkswellbeing.org/practice-examples/

72.

What Works Centre for Wellbeing. (undated b) . What works wellbeing. Available from: https://whatworkswellbeing.org

73.

What Works Wellbeing . (2021). A guide to effective case studies. What Works Centre for Wellbeing. Available from: https://whatworkswellbeing.org/resources/guide-to-effective-case-studies/

74.

Yin

R. K.

(2018). Case study research and applications: Design and methods (6th ed.). Sage Publications.

75.

Zwald

Jernigan

Payne

Farris

(2013). Developing stories from the field to highlight policy, systems, and environmental approaches in obesity prevention. Preventing Chronic Disease, 10, Article 120141. https://doi.org/10.5888/pcd10.120141

Synthesising Practice-Based Case Study Evidence From Community Interventions: Development of a Method

Abstract

Keywords

Introduction

Study Context

Design and Pilot

Phase 1- Scoping Methods

Phase 2- Gathering Practice-Based Case Studies

Phase 3 - Analysis and Synthesis

Phase 4 - Application and Recommendations

Results – a Staged Approach to Practice-Based Case Study Synthesis

Discussion

Application

Conclusions

Footnotes

Acknowledgements

Author Contributions

Declaration of Conflicting Interests

Funding

ORCID iD

References