How to Teach Evidence-Based Practice in Social Work: A Systematic Review

Abstract

Purpose:

This article presents a systematic review of research regarding how best to educate social work students and practitioners concerning of the process of evidence-based practice and/or the application of empirically supported treatments (ESTs).

Method:

We conducted a systematic review with a narrative synthesis, largely following the Cochrane Handbook of Systematic Reviews for Interventions and PRISMA reporting guidelines for systematic reviews and meta-analyses.

Results:

Twenty-seven studies met our eligibility criteria. These consisted mostly of uncontrolled designs and their measures relied mainly on learners’ self-perception regarding acquisition of declarative and procedural knowledge, motivation, and satisfaction. Reports were mostly positive (88.7%).

Conclusions:

Research regarding the education of social work students and practitioners about the process of evidence-based practice as well as ESTs is limited. Further investigation is needed concerning the effectiveness of specific teaching methods using controlled designs and more rigorous outcome measures including observation of practice in real-life situations and/or in role-plays.

It has been argued that the process of evidence-based practice (EBP) will contribute to making informed decisions that help clients attain valued outcomes (Emparanza, Cabello, & Burls, 2015; Gambrill, 2006; Sackett, Richardson, Rosenberg, & Haynes, 1997; Sackett, Rosenberg, Gray, Haynes, & Richardson, 1996). In EBP, compared to authority-based approaches (Gambrill, 1999), currently available research related to particular clients is sought as well as information about client circumstances and characteristics including their preferences and values, and clinical expertise is drawn on to integrate all information. Uncertainty and ignorance as well as knowledge is shared among professionals and clients. In authority-based approaches, criteria such as consensus and tradition are relied on in making decisions. Ever since EBP was promoted in social work (Gambrill, 1999), it has sparked interest. Two different approaches emerged: (1) the process of EBP and (2) empirically supported treatments (ESTs; promotion of specific interventions) also referred to as evidence-based interventions (EBIs) or evidence-based practices (EBPs). In the following, we will refer to all of these terms as ESTs for easier readability. Since there are different views of (Rubin & Parrish, 2007) and misconceptions about EBP (Gibbs & Gambrill, 2002), both approaches are addressed in this review. To date, little is known about how ESTs and/or the process of EBP are typically taught in social work education (or if they are). Thus, the aim of this article is to systematically describe the state of research on how to best teach the process of EBP and/or ESTs to social work students and practitioners as well as with regard to its quality.

Evidence-Based Practice: Two Different Approaches

There are two main different understandings of EBP. One is the process of EBP as described in original sources in medicine designed to help practitioners make informed decisions (Haynes, Devereaux, & Guyatt, 2002; Sackett et al., 1996; Straus, Richardson, Glasziou, & Haynes, 2011). A second (ESTs) refers to interventions claimed to be effective by some individual or organization. Both approaches are briefly discussed next (see Thyer & Myers, 2011, for an elaborated distinction).

The process of EBP

The term “evidence-based medicine (EBM)” was coined by Guyatt (1991; see Sur & Dahm, 2011, for a description of the history of EBM). In the process of EBP, clinical expertise is drawn on to integrate relevant research findings, and information about the clients’ unique circumstances and characteristics including their values and preferences, and hoped-for outcomes in order to arrive at informed decisions. This process involves “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual [clients]” (Sackett et al., 1997, p. 2; see also Sackett et al., 1996). Clinical expertise includes basic skills of clinical practice, including relationship skills and the practitioner’s individual experience (Haynes et al., 2002). The process of EBP includes five steps as described in original sources (Sackett et al., 1996; Straus et al., 2011).

Converting information needs related to practice decisions into well-structured questions.

Tracking down, with maximum efficiency, the best evidence with which to answer those questions.

Critically appraising that evidence for its validity, impact (size of effect), and applicability (usefulness in practice).

Integrating this critical appraisal with clinical expertise and with a client’s unique circumstances and characteristics including their values and preferences and making a decision together with the client.

Evaluating the effectiveness and efficiency in carrying out Steps 1-4 and seeking ways to improve them in the future.

This approach requires a search for knowledge as well as for ignorance such as lack of relevant research concerning a problem (Gambrill, 2019). Results are shared with clients to enable informed decisions that are most likely to result in hoped-for-outcomes for clients.

ESTs

The term “empirically supported treatments” (other terms include EBPs, empirically tested interventions, and EBIs) refers to manualized interventions (e.g., cognitive behavioral therapy, motivational interviewing) deemed to be “empirically supported” based on related research (Thyer & Myers, 2011). For example, the American Psychological Association 2005 Presidential Task Force on Evidence-Based Practice suggested criteria for categories such as “well-established” (at least two good group design studies or a large series of single case design studies, study conducted with treatment manual, clearly specified sample characteristics) and “probably efficacious” (e.g., two studies showing that a treatment is more effective than a waiting-list control group, Task Force, 1993).

Implementation of EBP in Social Work

Even though EBP has become an intensively discussed topic within social work, its implementation in social work practice still lacks behind. With regard to EBP as a process, Pope, Rollins, Chaumba, & Risker (2011) found in a survey of social workers (n = 200) in the United States that, although 83% agreed or somewhat agreed to be familiar with social work databases, only 56% agreed or somewhat agreed that they used relevant research to answer clinical questions (range: agreed, somewhat agreed, undecided, somewhat disagree, disagree). In a survey by Parrish and Rubin (2012) of 688 social workers carried out in Texas (a 21% response rate), it was found that few social workers indicated on a 5-point Likert-type scale that they “often” or “very often” use the Internet to search for the best evidence to “guide practice decisions” (32.8%), “read about research evidence to guide practice decisions” (37.8%), “inform clients of the degree of research evidence supporting alternative intervention options” (25.6%), and “engage in all steps of the EBP process” (15.1%). With regard to ESTs, Morago (2010) reported that 42.6% of 155 social workers and social care professionals indicated the level of implementation of ESTs in their respective agency as “very poor” and 40% as “modest” in a survey conducted in the UK (range: very poor, modest, good, excellent).

Research in a variety of professions has shown that implementation of EBP is difficult due to numerous barriers (e.g., Gray, Joy, Plath, & Webb, 2012; Scurlock-Evans & Upton, 2015). Skill and knowledge may be lacking. There may be insufficient preparation to use EBP (Teater & Chonody, 2018), unsound training (Bellamy, Bledsoe, & Traube, 2006), negative attitudes toward EBP (Murphy & McDonald, 2004), and diverse views of EBP (Rubin & Parrish, 2007). Therefore, social workers may be ill-prepared to use either ESTs and/or the process of EBP. It is important to identify effective educational interventions (EIs) to help students and practitioners to acquire and use related knowledge and skills if these enhance success in helping clients.

Systematic Reviews on EBP Education in Other Areas

The production of systematic reviews has greatly increased over the past decades. Yet, many reviews have been criticized as flawed (Ioannidis, 2016). There are a number of systematic reviews concerning the process of EBP in medicine. Aglen (2016) conducted a systematic review with 39 articles to provide an overview of strategies used to teach the process of EBP to nursing students at the bachelor level. Most studies (n = 31) used a qualitative, descriptive design and a formative evaluation; the focus was on students’ satisfaction with the EIs. Aglen (2016) concluded that a key issue in teaching the process of EBP was that nursing students do not see how research findings can contribute to their practice. She argued that teaching critical thinking and an emphasis on clinical problems are important aspects in teaching the process of EBP. Dizon, Grimmer-Somers, and Kumar (2012) reviewed six studies (four randomized controlled trials and two studies that used a single group pre–posttest design) regarding the effectiveness of training programs for the process of EBP to improve the knowledge, skills, attitudes, and behavior of allied health professionals (e.g., physiotherapists, speech pathologists). They concluded that training had significantly positive effects on all of the aforementioned constructs. Kyriakoulis et al. (2016) reviewed 20 studies (4 controlled and 16 uncontrolled trials) to find the best strategies and methods to teach the process of EBP to undergraduate health students. They concluded that a multifaceted approach that entails a combination of methods like lectures, computer sessions, small group discussions, journal clubs, and assignments was more likely to improve learners’ EBP knowledge and motivation than interventions offering only one of these methods or no intervention. Patelarou et al. (2017) reviewed 20 studies (3 controlled trials and 17 uncontrolled) to find the best teaching strategies and methods used to teach the process of EBP to health professionals and found that an increase in EBP competencies and attitudes was reported in nine of them. These authors also recommend a multifaceted teaching approach. In addition, they reported that online EIs are effective in enhancing practitioners’ claimed motivation to use the process of EBP. They also reported that online interventions were not effective in achieving changes in self-reports of behavior (however, the review also offered some data that support the effectiveness of online interventions regarding EBP behavior).

All of these reviews were conducted in fields different from social work. We could not find a review regarding teaching EBP in social work—neither with respect to the process of EBP nor the application of ESTs. Thus, a systematic review of research on how to teach EBP in social work is lacking.

How to Teach EBP: Instructional Approaches and Knowledge Application

The question how to best teach the process of EBP and/or ESTs can be tackled from different perspectives. One research community that is particularly concerned with the teaching of complex skills is the Learning Sciences community (e.g., Fischer, Hmelo-Silver, Goldman, & Reimann, 2018; Sawyer, 2009; see Hoadley & van Heneghan, 2012, for a brief history of the Learning Sciences and their implications for instructional designs). To categorize different teaching approaches, Learning Sciences research has repeatedly differentiated between “teacher-centered” approaches on the one hand (approaches that view the teacher as the main instance regarding what and how to learn in the classroom) and more “learner- or student-centered” approaches on the other hand (approaches that provide learners with more freedom regarding how to structure their learning process). It is argued that these concepts provide a useful analytical segregation for empirical research on EIs and its potential implications (see Hmelo-Silver, Duncan, & Chinn, 2007; Kirschner, Sweller, & Clark, 2006; Sweller, Kirschner, & Clark, 2007, for a critical discussion of this dichotomy). Direct instruction (DI, e.g., Slavin, 2018) is an example for the teacher-centered approach. Problem-based learning (e.g., Hmelo-Silver, 2004) is an example of the student-centered approach. In the following, we describe the two approaches and their respective examples in more detail.

Teacher-centered instructional approaches

The basic idea of teacher-centered approaches is to have a teacher to support student learning by providing information that explains concepts and procedures (Kirschner et al., 2006) optimally in a way that “fits” the human cognitive architecture (especially not to overstrain leaners’ working memory capacity; Sweller, Ayres, & Kalyuga, 2011). DI is an example of this approach. Slavin (2018) suggests seven steps to apply this approach in an ideal way: (1) define learning goals and provide a syllabus, (2) activate prior knowledge, (3) present new subject matter in a structured and efficient way, (4) use comprehension checks like questions, (5) let learners apply previously presented knowledge, (6) induce further elaboration (e.g., homework), and (7) assess performance and give feedback. In a meta-meta-analysis, Hattie (2009) reported an average effect size of d = .59 for DI in comparison with other traditional teaching approaches. If delivered correctly, DI helps to avoid exposing too much load on the leaners’ working memory (Sweller et al., 2011). However, knowledge acquired through teacher-centered approaches often remains inert, that is, it is often difficult for learners to use this knowledge for problem-solving. One possible explanation is that the acquisition of knowledge is context-bound. Transferring that knowledge to a situation that is very different from the situation in which it was acquired can be very difficult (Barnett & Ceci, 2002). Situated (e.g., student-centered) approaches have been developed to tackle this issue (Renkl, Mandl, & Gruber, 1996).

Student-centered instructional approaches

In student-centered instructional approaches, learners are granted a more active role. This is achieved by presenting learners more complex and practical problems that they are supposed to solve either alone or in groups but optimally guided by a teacher or tutor. One example is problem-based learning (PBL; Barrows & Tamblyn, 1980; Hmelo-Silver, 2004). In PBL, after the presentation of a problem, students discuss possible explanations for it. Discussing the problem before receiving any further instructions is important to activate and evaluate prior knowledge and discover knowledge gaps that should trigger interest and motivation to find out more about the problem (Loyens & Rikers, 2011). In PBL, students learn by solving complex real-world problems and reflect on their experiences guided by a teacher or a tutor (Hmelo-Silver et al., 2007). In Hattie’s (2009) meta-meta-analysis, the average effect of PBL on student achievement compared to more traditional approaches was rather small (d = .15). Yet other research shows that PBL has particular advantages in comparison with other instructional approaches regarding the acquisition of skills and application-oriented knowledge (Dochy, Segers, van den Bossche, & Gijbels, 2003; Gijbels, Dochy, van den Bossche, & Segers, 2005; Schmidt, van der Molen, Te Winkel, & Wijnen, 2009). Since EBP is supposed to be applied in real-world settings (it is application-oriented knowledge), student-centered approaches like PBL might be more effective in teaching EBP than strongly teacher-centered approaches like DI. Indeed, Tian, Liu, Yang, and Shen (2013) found PBL to be more effective in teaching the process of EBP compared to a lecture-based approach in a randomized controlled trial with medical postgraduates (n = 103).

Knowledge application

Since EBP can be considered application-oriented knowledge, it is important to explore how knowledge is applied within learning processes, for example, working with a fictional case or with real clients (or if knowledge is applied at all). The concept of “situated cognition” tackles the importance of knowledge application during the learning process. The basic idea of situated cognition is not to focus only on isolated aspects like cognition, but take into account the individuals and their actions as well as the situation in which practice takes place (Wilson & Myers, 2000). Proponents of situated cognition such as Lave (1988) assume that during the learning process knowledge cannot be decontextualized, transmitted, and then applied in another context (see Gruber, Law, Mandl, & Renkl, 1996, for an overview of situated learning models). She found that skills learned in informal environments are rarely generalized but remain connected to the contexts and the circumstances in which they are acquired. She emphasized the importance of everyday practice and the necessity to embed learners in social communities that support participation and increasingly independent application of skills in relevant settings (see more recent research concerning the importance of deliberate practice in enhancing expertise such as Rousmaniere, Goodyear, Miller, & Wampold, 2017).

Effects of EIs

Much research is interested in studying the effects of certain EIs on desired outcomes. An effect is the difference between what happened when people received an intervention and what would have happened if they had not received it (Shadish, Cook, & Campbell, 2002). One important outcome is knowledge acquisition that may be declarative and/or procedural (Anderson, 1996). Declarative knowledge (knowing what) refers to knowledge about facts, concepts, and principles. Procedural knowledge (knowing how) refers to skills and actions. Researchers are also interested in effects of EIs on other variables such as learner’s motivation to engage with the subject matter (e.g., Ruzafa-Martínez, López-Iborra, Armero Barranco, & Ramos-Morcillo, 2016). The development of standardized instruments to measure social workers’ attitudes toward and intentions to use EBP suggest that motivation toward EBP is an important construct related to EIs in social work (Aarons, 2004; Aarons et al., 2010; Rubin & Parrish, 2010). Finally, the learner’s satisfaction with an EI is also an outcome that is often measured in EI studies.

Quality of Empirical Intervention Studies

To determine the effectiveness of an EI on relevant dependent variables, it is important to consider the methodological quality of related empirical studies. Study quality can be operationalized at different levels including rigor in design and reliability and validity measures. Both concerns are affected by risk of bias which we also discuss.

Rigor in design

Studies that lack a controlled design can be problematic in identifying effects and do not support strong causal inferences (Shadish et al., 2002). This does not mean that discovery of important aspects of learning is restricted to well-controlled experimental research (Hoadley & van Heneghan, 2012). Yet the inclusion of a control group is a sign of quality with regard to claimed effects, especially for quantitative methods. Nevertheless, Yaffe (2013) notes that most evaluation studies in social work education do not apply a controlled design. Qualitative research usually has other goals than detecting a causal relationship such as reconstructing interpretative patterns or exploring learners’ individual adaptations of knowledge. Qualitatively oriented researchers may speculate about what would have happened if a causal factor was missing (Johnson & Christensen, 2013).

Reliability and validity

Reliability refers to how consistently a construct is measured. One way to estimate the reliability of a measure is to examine its internal consistency, how closely items on a measure are related by calculating the Cronbach’s α. An alternative is examining stability of a measure by administering this at different times and examining scores. Validity refers to whether a measure actually reflects the construct of interest. Different kinds of validity include content validity (do items accurately reflect the domain of interest?), construct validity including convergent (are two constructs that should theoretically be related in fact related?) and divergent/discriminant validity (are two constructs that should theoretically be not related in fact not related?), criterion validity that includes concurrent validity (relationship between test scores and criterion scores obtained at the same time), and predictive validity (relationship between test scores obtained at one point in time and criterion scores obtained at a later time). Self-report measures may not reflect behavior in real-life settings. Relying solely on learners’ perceived learning is problematic since we tend to overestimate our knowledge (Kruger & Dunning, 1999; Snibsøer et al., 2018). Instead, when assessing knowledge and its use, observation of performance in real life or simulated work settings using valid measures is preferable (Johnson & Christensen, 2013). Thus, the “measurement strategy” (performance tests vs. self-report) of a study is a particularly important aspect of validity in our review.

Risk of bias

Bias refers to systemic error in one direction. Factors that may contribute to such bias are, for example, incomplete outcome data (attrition bias) or selective outcome reporting (reporting bias; Higgins & Green, 2011). Risk of bias assessment is closely connected to the type of empirical data used, the theoretical rationale drawn on and the unique circumstances of a study. Different methods to assess risk of bias exist and the method used in a particular review should be selected with reference to the methodological features relevant to the included studies (Liberati et al., 2009).

Objectives and Research Questions

The aim of this study is to describe and review research on EIs used to teach the process of EBP and/or ESTs to social work students and practitioners and their effects on various dependent variables (such as knowledge, motivation, and satisfaction), considering the quality of the studies. We investigated the following research questions:

Research Question 1: What EIs are applied in research on EBP education in social work? Given the potentials of more student-centered approaches regarding the acquisition of application-oriented knowledge, we were particularly interested in the extent to which such approaches have been used in related research. And, was knowledge applied during the learning processes and if so, how?

Research Question 2: What are the effects of these EIs? We are interested in effects regarding the acquisition of both declarative and procedural knowledge, motivation toward EBP, and satisfaction with the EIs.

Research Question 3: What is the methodological quality of the studies? To what extent did studies use controlled designs and reliable and valid measures? We are especially interested in the studies’ measurement strategies (performance tests vs. self-report) and their risk of bias.

We carried out a systematic review to answer these questions. Due to the varied means of data collection and analysis in research reports (qualitative, quantitative, and mixed methods) as well as the heterogeneity of designs, samples, and interventions, we did not conduct a meta-analysis. Instead, we provide a narrative synthesis.

Method

We largely followed the Cochrane Handbook of Systematic Reviews for Interventions (Higgins & Green, 2011) in conducting our review. However, we used different criteria to estimate risk of bias since the criteria suggested in the handbook focus mainly on well-designed randomized controlled trials that were rare in our sample (see later discussion of risk of bias). We followed the preferred reporting items for systematic reviews and meta-analyses (PRISMA) reporting guidelines regarding pertinent categories (Liberati et al., 2009).

Eligibility Criteria

We included studies that met the following criteria. First, the studies had to be empirical. Second, the studies had to include one or several interventions designed to help participants develop relevant declarative and/or procedural knowledge and/or motivation regarding ESTs and/or the process of EBP (studies that address both approaches are labeled as “Both”). Studies solely interested in learners’ satisfaction with a particular EI were not included. Third, the sample used had to consist at least partially of social workers or social work students. Fourth, only studies in English or German language were included (see Online Appendix Table A1 for a detailed description of the eligibility criteria).

Literature Search

We carried out a bibliographic search to locate relevant articles using search terms grouped into the following categories: field, EBP, and educational concepts (see Table 1). We used a filter of “peer-reviewed” to identify publications subject to some kind of quality control. We searched in the following databases using the combined terms (see Table 1): Social Services Abstracts, Sociological Abstracts, Applied Social Sciences Index and Abstracts, SocINDEX, PsychINFO, ERIC, Social Service Citation Index, and Social Care Online. The search was carried out in December 2017. In addition, we handsearched the Journal of Evidence-Based Social Work since it is particularly concerned with research on EBP in social work. Furthermore, we handsearched the special issue of 2015 of Research on Social Work Practice regarding the Houston Bridging the Research-Practice Gap Symposium, and we also performed a snowball search for relevant articles in the references of already retrieved articles.

Table 1.

Search Terms Used to Find Relevant Studies for a Systematic Review of Educational Intervention Studies to Teach the Process of EBP and/or ESTs in Social Work.

Construct	Search Term(s)
Field	Social Work*
EBP	Evidence N1 basedEvidence N1 supportEvidence N1 informed2 OR 3 OR 4
Educational concepts	Teach*Train*WorkshopEducat*Curricul*Apprais*Implement*Attitud*Learn*Instruct*CourseFosterFacilitat*Appl*6 OR 7 OR 8 OR 9 OR 10 OR 11 OR 12 OR 13 OR 14 OR 15 OR 16 OR 17 OR 18 OR 19
Combined Terms	1 AND 5 AND 20

Note. EBP = process of evidence-based practice and empirically supported interventions; N1 = near/n, “near” searches for instances of the search term in any order, “n” specifies number of terms between the search terms (e.g., “based on evidence” would be a possible result). Syntax (e.g., N/1, near/1) was adjusted due to the requirements of the respective database.

Study Selection

Two independent coders used the described eligibility criteria to review abstracts of >10% of all potentially relevant articles using a binary code (study to be included vs. study not to be included), until a sufficient interrater reliability (IR; Cohen’s Kappa coefficient = κ) was reached. When a screening failed to attain a sufficient IR, conflicting cases were resolved through discussion, the coding scheme was adjusted accordingly and a new screening conducted with a new set of >10% of all articles in an interactive process. Table 2 provides an overview of the eligibility criteria and their respective κ values as well as the screening rounds needed to attain these values.

Table 2.

Interrater Reliability for Eligibility Criteria.

Interrater Reliability/Screening Rounds	Eligibility Criteria
	Empirical	Intervention	Sample	EBP
κ	0.78	0.82	0.62 (1.0)a	1.0
Screenings	3	8	4	4

Note. EBP = process of evidence-based practice and empirically supported interventions, κ = Cohen’s Kappa.

^aThe low κ value for “sample” occurred with only one conflict (interrater agreement was 87.5%) due to binary coding (social work vs. no social work). After achieving consensus for this one case, κ for sample was 1.0.

After a sufficient κ was attained for all eligibility criteria, the remaining articles were coded by the first author. The 52 articles that remained after abstract screening were subject to a full text screening by the first author (see Figure 1 for a visualized description of the inclusion and exclusion process).

Figure 1.

Flow diagram of the inclusion and exclusion process of studies for the review. Two of the 28 articles refer to one study, thus we analyzed 27 studies.

Data Extraction

We defined a set of variables (see Table 3) to answer our research questions and extracted respective data from the articles. The procedure of data extraction differed with respect to different variables.

Table 3.

Variables.

Variable	Description/Subcodes	IR
Preliminary analysis
Location	The location where the study was conducted	N/A
Sampling method	Procedure for selecting sampling members	N/A
Sample	The sample must contain at least one social work student (BSW, MSW, PhD) or social worker (e.g., case manager, clinical supervisor, field instructor). It may also contain other professionals (e.g., nurses, psychologists). Subcodes: “BSW,” “MSW,” “PhD,” “social worker,” “mixed social work” (social work students and social work professionals), “mixed students” (social work students as well as students from other profession/s, e.g., BSW and psychology students), “mixed professionals” (social workers as well as professionals from other profession/s, e.g., social workers and nurses), and “mixed” (students and professionals from social work and other profession/s)	consensus
Sample size	The sample size of a studies’ sample	N/A
Gender	Subcodes: “exclusively female” (the sample consisted entirely of females), “predominantly female” (75% or more were female), “mixed” (females and males were both below 75%), “predominantly male” (75% or more were male), and “exclusively male” (the sample consisted entirely of males)	κ = 1.00
Age	The variable “age” refers to the mean age of the sample. If the mean was not provided, the median was extracted. Subcodes: “under 30,” “between 30 and 40,” “between 40 and 50,” and “between 50 and 60”	κ = 0.75
Ethnic diversity	Refers to the question whether a study was based on an ethnically diverse or homogenous sample	N/A
Intervention	Refers to the question whether the educational intervention was a workshop, a lecture, a university course, and so on	N/A
Duration	The duration of a particular educational intervention	N/A
Steps in EBP process	The steps of the EBP process which were addressed by a particular educational intervention	N/A
EST	The EST (e.g., cognitive behavioral therapy, motivational interviewing) addressed by a particular educational intervention	N/A
Research Question 1—EIs
Instructional approach	Subcodes: “teacher” (e.g., a teacher-centered lecture in which an instructor is attempting to transmit knowledge), “student–teacher” (e.g., a workshop in which instructional parts alternate with problem-based small group work), or “student” (e.g., a student-centered educational intervention that offers a lot a freedom for the learners to apply new knowledge, e.g., with the combination of self-dependent work with real clients)	consensus
Knowledge application	Knowledge application refers to the realness of the EI’s case-based content. Subcodes: “real-world practice” (e.g., the learner applies content from the EI with actual clients), “case-based” (e.g., the learner applies content from the EI in a fictional social work case), or “not case-based” (the EI does not require to apply knowledge with social work cases)	consensus
Research Question 2—Effects
Tested declarative knowledge	Declarative knowledge (knowing what) of EBP, measured and subsequently assessed by a third person Subcodes: “positive effect” (e.g., increase of the mean from pre- to postmeasurement, citations from participants, or conclusions of researchers that imply a positive effect, e.g., “Through the workshop I realized the importance of empirical research for daily practice”), “no effect” (e.g., conclusions of researchers that no differences between pre- and postmeasurement occurred), and “negative effect” (e.g., decrease of the mean from pre- to postmeasurement)	consensus
Tested procedural knowledge	Procedural knowledge (knowing how) of EBP, measured and subsequently assessed by a third person Subcodes: See “tested declarative knowledge”	κ = 1.00
Perceived declarative knowledge	Declarative knowledge (knowing what) of EBP, measured with a self-report instrument (e.g., Questionnaire) Subcodes: See “tested declarative knowledge”	κ = 0.62
Perceived procedural knowledge	Declarative knowledge (knowing what) of EBP, measured with a self-report instrument (e.g., Questionnaire) Subcodes: See “Tested Declarative Knowledge”	κ = 0.67
Motivation	Attitude toward EBP, feasibility, intentions to use, interest in research, ability to make connections between research and practice, and so on Subcodes: See “tested declarative knowledge”	consensus
Satisfaction	Satisfaction, perceived usefulness or helpfulness of the participants with the intervention. Subcodes: “positive effect” (e.g., a score above 75% in a postmeasurement, e.g., 8 out of 10 in a 10-point Likert-type questionnaire, citations from participants, or conclusions of researchers that imply satisfaction, e.g., “overall, participants found the training very useful”), “no effect” (e.g., a mediocre score in a postmeasurement, e.g., 5 out of 10 in a 10-point Likert-type questionnaire), and “negative effect” (e.g., a score below 25% in a postmeasurement, e.g., 2 out of 10 in a 10-point Likert-type questionnaire)	consensus
Research Question 3—Quality
Design	The design was determined by coding the points of measurement (PoM) of the variables of Research Question 2—effects. These variables are depicted in hierarchical order with “tested declarative knowledge” being of the highest and “Satisfaction” of the lowest interest. Subcodes for the PoM of each variable: pre, post, follow-up, post follow-up, pre-post, pre follow-up, and pre-post follow-up. These codes are depicted in hierarchical order with “pre” being the least and “pre-post follow-up” being the most desirable subcode. The subcode “prerepeated” was added later on for one particular study. The design was determined through (1) the outcome of highest interest and (2) the most desirable PoM (pre-post over post-only, pre-post follow-up over pre-post)	consensus (motivation) κ = 0.62–1.00
Measure instrument strategy	Refers to whether the measurement instrument implied a subsequent performance assessment by a third person or not. Subcodes: “test” (e.g., observation, knowledge test, vignette), “perception” (e.g., focus groups, interview, Likert-type scale questionnaire) or a combination of “both”	consensus
Analysis paradigm	Refers to the methods of analysis. Subcodes: “qualitative” (e.g., content analysis), “quantitative” (e.g., inferential statistics), or “mixed methods” (qualitative and quantitative methods)	κ = 0.82
Reliability	Measures of reliability (e.g., Cronbach’s α) of an applied measurement instrument	N/A
Validity	Refers to the validity of an applied measurement instrument. If a measure was claimed to be valid, we only included information on validity if we could locate other studies in support of these claims	N/A
Risk of bias	The Mixed Methods Appraisal Tool was used to assess the studies’ risk of bias (Pluye et al., 2011)	75–100%

Note. EBP = process of evidence-based practice and empirically supported treatments; IR = interrater reliability; κ = Cohen’s Kappa; % = interrater agreement; N/A = not applicable (descriptive variable); EI = educational intervention.

Descriptive variables

Variables that are rather descriptive in nature such as location where a study was conducted or the duration of an EI were not coded but directly extracted.

Coded variables

To code variables that were not descriptive in nature such as instructional approaches or knowledge application, we developed and iteratively refined a standardized data abstraction form. A number was allocated to each subcode and studies were coded numerically. All studies were coded by two independent coders using >20% of the relevant articles until a sufficient IR (κ > 0.60) was attained. The remaining articles were coded by the first author. We encountered a great deal of vague or missing descriptions (we contacted eight authors to ask for request additional information and three answered). Thus, all ratings for variables with which we experienced problems to attain a sufficient IR were double coded by two coders based on consensus. Table 3 provides an overview of all coded variables, their operationalization, and their IR.

Risk of bias assessment

The Mixed Methods Appraisal Tool (MMAT) was used to assess the studies’ risk of bias (Pluye et al., 2011). The MMAT was developed for use with systematic reviews that include quantitative, qualitative, and mixed method studies. It has been validated with regard to content validity (Pace et al., 2012; Pluye, Gagnon, Griffiths, & Johnson-Lafleur, 2009, 2011). Its reliability ranges from κ = 0.21 to 1.00. This tool needs improvement, especially regarding qualitative studies (Souto et al., 2015). The MMAT consists of various questions that differ with respect to the data collection and analysis used in a study. It consists of four questions each for qualitative studies, randomized controlled trials, nonrandomized trials, and quantitative (descriptive) studies and three questions for mixed methods studies. Mixed methods studies are rated using these questions as well as qualitative questions and the respective quantitative questions. An exemplary question for mixed method studies would be “Is the integration of qualitative and quantitative data relevant to address the research question (objective)?” Response options are yes, no, or don’t know (not enough information available). An overall score could be allocated to the studies, ranging from 0 (all questions answered with no) to 4 (all questions answered with yes). We used binary coding, simply using yes or no, since don’t know results in the same rating as no. One (>10%) study of each paradigm (n = 6; quantitative studies could be quantitative, quasi-experimental or experimental) was rated by two independent coders. Paradigms were determined by the studies “analysis paradigm” and “design.” After two screenings, a sufficient interrater agreement was achieved for each paradigm (range: 75–100%). Remaining studies were coded by the first author. Results based on the MMAT should be treated with caution given the many judgments involved, some based on unclear criteria. Thus, overall scores for the specific studies are not given in this review. However, a summary will be provided to indicate the overall risk of bias of evidence generated with this review.

The variables “sample size,” “reliability,” and “effects” may have been simplified for evidence aggregation and/or easier readability in the following sense.

Sample size

To determine a study’s sample size, we extracted the number of participants who completed the posttest. If a study involved a pretest, we extracted the number of the participants who completed both pre- and posttests. If the study reported more than one outcome of interest, we extracted the smallest of the provided numbers. For example, if a study measured “motivation” of 34 participants and also “satisfaction” of 31 participants, 31 was extracted as the sample size. The same was done for follow-up sample sizes.

Reliability

If multiple values for measurements of internal consistency were reported for subscales relevant to a single dependent variable (DV), the range was reported (e.g., for feasibility, α = .76; attitude, α = .89; and intentions to use, α = .63, we reported α = .63–.89 for motivation). If multiple internal consistency values were reported for various points of measurement, we computed the mean (e.g., pretest, α = .90; posttest, α = .89; follow-up, α = .93; then we report, α = .91).

Effects

With respect to “effects” derived from quantitative results, we extracted the reported means, the standard deviations, and p values (see Online Appendix Table A2). However, note that significance testing is increasingly criticized because of misleading implications (Wasserstein, Schirm, & Lazar, 2019). If studies reported several items (e.g., from a survey) that referred to the same DV, we calculated the mean of the items. If n was reported for each item, we computed a weighted pooled mean. Regarding qualitative results, we extracted quotes from participants or conclusions of researchers that imply an effect on a DV (e.g., “Through the workshop I realized the importance of empirical research for daily practice”). We summarized all extracted data as positive (+), no (o), and negative (−) effect (see Table 5). Regarding the variable satisfaction, (+) was interpreted as a high, (o) as a medium, and (−) as a low level of satisfaction. Some studies had additional objectives that went beyond the examination of the EIs’ effects on social workers. For instance, some studies were interested in clients’ perceptions of an EST delivered after providers received training using a particular EST. However, only objectives, measures, and effects regarding the facilitation of EBP knowledge, motivation, and/ or satisfaction with the EI were extracted for this review.

Variables

To answer our research questions, the articles included in this analysis were coded with respect to a broad range of variables. Regarding Research Question 1, we coded them with respect to the instructional approach of the respective EI and how (or if) learners had to apply EBP knowledge (knowledge application). With respect to Research Question 2, we coded descriptive as well as procedural EBP knowledge, motivation toward EBP, and the learners’ satisfaction with the EI. Concerning Research Question 3, we coded the studies’ design, their measure instrument strategy (whether data collected by a measure instrument referred to a performance test or the learners’ self-assessment of their knowledge) as well as the studies’ analysis paradigm (the methods of analysis). Furthermore, we coded the articles with respect to a number of background variables, to characterize the studies’ broader circumstances in a preliminary analysis (see Table 3). All variables are described in more detail in Table 3.

Results

Study Selection

Our search across the different databases revealed 2,085 hits. Handsearching the Journal of Evidence-Based Social Work revealed another three potentially relevant articles. Furthermore, hand-searching the special issue of 2015 of Research on Social Work Practice regarding the Houston Bridging the Research-Practice Gap Symposium revealed no potentially relevant articles and 10 additional articles were identified with a snowball search. We ended up with 2,098 potentially relevant articles. Figure 1 provides a detailed overview of the inclusion and exclusion process.

One reason for the large discrepancy between the initially identified articles and the final sample lies in the large number of duplicates (n = 982), resulting in a search in eight different databases. Another reason might be our broad inclusion criteria yielding many conceptual articles. One article was only available in Spanish. The first author of this article was contacted via e-mail and she confirmed that no English version was available. In the case of one article that was not accessible to us, we contacted all three authors by e-mail. No one replied. Thus, we ultimately analyzed 28 articles that refer to 27 studies (two different articles refer to the same study).

Preliminary Analysis

Fifteen (55.5%) studies were conducted solely with social workers and/or social work students. Eleven studies (40.7%) did not provide any information on the age of the participants and six (22.2%) did not provide information on gender. Table 4 provides an overview of the samples including information on age and gender for studies that represent the two different approaches (the process of EBP and ESTs).

Table 4.

Sample Characteristics.

Sample Characteristics	Total, n (%)	EBP Approaches
		EBP, n (%)	ESTs, n (%)	Both, n (%)
All studies	27 (100)	13 (48.1)	12 (44.4)	2 (7.4)
Sample
Social work
MSW	6 (22.2)	4 (14.8)	2 (7.4)	0
Social worker	5 (18.5)	4 (14.8)	1 (3.4)	0
BSW	3 (11.1)	1 (3.7)	2 (7.4)	0
Mixed social work	1 (3.7)	0	0	1 (3.7)
Mixed professions
Mixed professionals	9 (33.3)	4 (14.8)	5 (18.5)	0
Mixed	1 (3.7)	0	1 (3.7)	0
Mixed students	1 (3.7)	0	1 (3.7)	0
Unknown	1 (3.7)	0	0	1 (3.7)
Age
Under 30	5 (18.5)	2 (7.4)	3 (11.1)	0
Between 30 and 40	5 (24.1)	1 (3.7)	3 (11.1)	1 (3.7)
Between 40 and 50	6 (22.2)	4 (14.8)	2 (7.4)	0
Unknown	11 (40.7)	6 (22.2)	4 (14.8)	1 (3.7)
Gender
Exclusively female	1 (3.7)	0	1 (3.7)	0
Predominantly female	14 (51.9)	6 (22.2)	7 (25.9)	1 (3.7)
Mixed	6 (22.2)	2 (7.4)	4 (14.8)	0
Unknown	6 (22.2)	5 (18.5)	0	1 (3.7)

Note. EBP = process of evidence-based practice; ESTs = empirically supported treatments.

Twenty-three studies were conducted in the United States, three in the UK, and one in Israel (see Table 5). All studies used nonprobability sampling. Sixteen studies used convenience samples, 10 studies used purposive samples (participants fulfilled certain eligibility criteria), and one study did not provide information on its sampling method. Nineteen studies provided information on the ethnicity of participants. All used diverse samples. The majority of the studies evaluated a university course (n = 10). Intervention time ranged from 5.5 hr to 9 months for self-regulated trainings and from 7 hr up to one semester for courses and workshops. Five studies focused on enhancing skills in the process of EBP, addressing all five steps in the process. Another five studies, two of which attempted to facilitate both the process of EBP and ESTs, addressed steps one to four in the process. Two studies addressed only Step 3 (critical appraisal) and another two studies addressed Step 4 to some extent (applying research evidence in practice).

Table 5.

Main Characteristics.

Author (Year)	EBP	Main Characteristics (Objective, Location, Sample Size [Follow-Up])	Design	Educational Intervention (Duration)	Instructional Approach, Knowledge Application	Coded Effect: Measure Instrument (Reliability/Validity)
Ager et al. (2005)	ESTs	Foster declarative and procedural motivational enhancement therapy (MET) knowledge, the United States, n = 175 (118)	Three-groupa pre-post follow-up	Workshop (2 days + 4-hr follow-up)	Student–teacher, real world	(+) TDK: MCT (α = .87, r = .61)(+) TPK: Vignettes (ICC = .68)(+2) PPK: LSQ (α = .65, r = .48–.54)(+) M: LSQ (α = .82, r = .69)
Bellamy et al. (2013) b	EBP	Foster EBP attitudes, understanding, knowledge and skills (article: report participants perceptions of training outcomes, EBP barriers and promoters), the United States, n = 15	One-group pre-post	Training in university–agency partnership context (2 days to 3 months)	Student–teacher, case based	(+) M: Field notes, Focus Groups
Bledsoe-Mansori et al. (2013) b		Foster EBP attitudes, understanding, knowledge and skills (article: report acceptability of the EI), the United States, n = 15				(+) S: Field Notes, Focus Groups
Bender, Altschul, Yoder, Parrish, and Nickels (2014)	EBP	Foster EBP attitudes, familiarity, self-efficacy, current/intended use, and belief that EBP is feasible in the real world, the United States, n = 152	Two-group pre-post	Integrating EBP process material in a program evaluation course (1 semester)	Student–teacher, real world	(+) PDK: modified EBPPAS (α = .89)(+) PPK: modified EBPPAS (α = .90)(−/o/+) M: modified EBPPAS (α = .67 to .84)EBPPAS: Content, criterion, and factorial validity
Daniel, Torres, and Reeser (2016)	ESTs	Teach hospice social workers and chaplains an evidence-based music protocol, the United States, n = 10	One-group pre-post	Unknown	Unknown, unknown	(+) TDK: MCT (r = .84, content validity)(+) PPK: Journal entries(+) M: Focus Group
Dauenhauer, Glose, and Watt (2015)	ESTs	Demonstrate knowledge and skills related to the matter of balance evidence-based falls management program, the United States, n = 16	One-group pre-post	University course (one semester)	Student, real world	(+) TDK: MCT (3 subscales, α = 0–.533)
Ducharme, Rober, and Wharff (2015)	ESTs	Develop interns’ ability to critically evaluate research and to teach specific manualized interventions (cognitive behavioral therapy [CBT], psychodynamic treatment, interpersonal therapy [IPT]), the United States, n = 12	One-group pre-post	University course (one semester)	Student–teacher, case based	(+3) PDK: OEQ(+) PPK: LSQ (α = .92)(+) S: OEQ
Gromoske and Berger (2017)	EBP	Foster social workers’ knowledge of, attitudes toward, views of feasibility, intentions to use, and current use of the EBP process, the United States, n = 45 (45)	One-group pre-post follow-up	Workshop (replication of Rubin and Parrish without opinion leader) (7 hr)	Student–teacher, case based	(+) PDK: EBPPAS-short (α = .91c)(+) PPK: EBPPAS-short (α = .87c)(o/+2) M: EBPPAS-short (3 subscales, α = .79c–.87c)EBPPAS-short: Content, criterion, and factorial validity
Hagell and Spencer (2004)	EBP	Keeping social care staff informed about research findings, UK, n = 20 (20)	One-group post-follow-up	Audio tapes with summarized research knowledge (self-regulated, up to 3 months)	Teacher, not case based	(o) PPK: Focus Group(o) M: Focus Group
Hohman, Pierce, and Barnett (2015)	ESTs	Decrease students’ use of dysfunctional communication skills and increase motivational interviewing skills, the United States, n = 137	One-group pre-post	University course (one semester)	Student–teacher, case based	(+) TPK: Vignettes (α = .76c, ICC = .62–.91)(+) TPK: Vignettes (α = .87, ICC = .62–.91)
Holmes (2008)	EBP	Gain a deeper understanding of the relevance and potential application of secondary research within participants’ social work practice, as well as becoming familiar and interested in accessing and reading research articles, UK, n = 6	One-group post only	Teaching course (3 months)	Student–teacher, not case based	(+) M: OEQ
Kobak, Mundt, and Kennard (2015)	ESTs	Teach cognitive behavior therapy, the United States, n = 8	One-group pre-post	Online CBT tutorial (self-regulated, about 5.5 hr)	Teacher, real world	(+) TDK: MCT (α = .82)(+) PDK: LSQ(+) PPK: LSQ(+2) S: RS (α = .86)LSQ: Construct validity
Leathers and Strand (2013)	ESTs	Increase participants’ knowledge and attitude about EBPs, the United States, n = 18 (18)	rand. Two-group pre-post-follow up	Both groups: Web training (30 min) + access to universities online journals (3 months)EG: Access to practisewise	Student, real world	(+) TDK: MCT (r = .56, discriminant validity)(−/+)M: EBPAS (α = .75)(+) S: LSQ (α = .75)
Leathers, Spielfogel, Blakey, Christian, and Atkins, (2016)	ESTs	Promote the use of an EBP to address child behavior problems, the United States, n = 57 (57)	rand. Two-group prerepeated	Both groups: Training of a parent management EBP (16 hr)EG: Change agent (9 months)	Student–teacher, real world	(+2) PPK: CQ
Lopez, Osterberg, Jensen-Doss, and Rae (2011)	ESTs	Foster attitudes and use of an EBP (behavioral parent training, BPT) in the context of a system level mandate, the United States, n = 21 (26)	One-group pre-post follow-up	Workshop (2 days)	Teacher, unknown	(+4) PPK: LSQ (4 subscales, all α > .89)(o) M: EBPAS (α = .43–.95)(+) M: LSQ (α = .62–.81)(+) S: LSQ (α = .62–.81)
Martin, Waites, Hopp, Sobeck, and Agius (2013)	Both	Teach service providers about the benefits of EBHP planning and implementation strategies for older adults, the United States, n = 18	One-group pre-post	Training session (2 days)	Student–teacher, not case based	(+) TDK: MCT(+) S: OEQ
Matthieu, Carter, Casner, and Edmond (2016)	EBP	Align practicum instructors’ EBP knowledge with content taught in MSW curriculums and enhance abilities to assist students in applying the EBP process in their field placements, the United States, n = 186	One-group pre-post	Workshop (1 day)	Student–teacher, case based	(+) PDK: EBPPAS(+2) M: EBPPASEBPPAS: Content, criterion, and factorial validity
Parrish and Rubin (2011) a	EBP	Improve practitioner self-efficacy with and knowledge, attitudes, beliefs regarding the feasibility and intentions and behaviors regarding their adoption and implementation of/ about EBP, the United States, n = 69 (61)	One-group pre-post	Workshop with opinion leader (7 hr)	Student–teacher, case based	(+) TDK: MCT(+) PDK: EBPPAS (α = .91)(+) PPK: EBPPAS (α = .86)(+3) M: EBPPAS (α = .63–.86)EBPPAS: Content, criterion, and factorial validity
Peterson, Phillips, Bacon, and Machunda (2011)	EBP	EBP, develop students’ familiarity with EBP concepts and improve their abilities to search for and use interventions from the professional literature, the United States, n = 81	One-group pre-post	BSW course (12 hr)	Student, real world	(+) PDK: LSQ(+2) PPK: LSQ(+) M: OEQ
Ronen (2005)	ESTs	To train students in cognitive-behavioral intervention, Israel, n = 30	One-group post-only	University course (two semesters) + leading a group intervention in practice (12 weekly sessions + two follow-ups, 75 min each)	Student, real world	(+) PDK: OEQ(+) PPK: OEQ(+) M: OEQ(+) S: LSQ (α = .82), OEQ
Sacco et al. (2017)	ESTs	Foster MSW students “screening, brief intervention, and referral to treatment” (SBIRT) knowledge, confidence, skills, and use, the United States, n = 58 (33)	One-group pre-post follow-up	Training with standardized clients (15 hr)	Student–teacher, case based	(+) TDK: MCT (α = .48c knowledge questions and .96c case examples)(+) TPK: Observation in role-play (ICC = .82–.96)(+3) PPK: LSQ (3 subscales, α = .95–.96)(+) PPK: LSQ (α = .91c)
Salcido (2008)	EBP	Teach EBP and enable students to translate theoretical and empirical content into assessment and intervention strategies using management and planning methods, the United States, n = 38	One-group pre-post	Workshop (2 days) + macro practice course (MPC) + research course (15 weeks)	Student–teacher, real world	(+) PDK: EBPAS(+) PPK: LSQ(+) S: LSQ
Salloum and Smyth (2013)	ESTs	To assist with implementing a manualized treatment (grief and trauma intervention), the United States, n = 12	One-group post-only	Podcasts (self-regulated, 1 day to 3 weeks)	Teacher, real world	(+) S: Interview (κ = .86)
Smith, Cohen-Callow, Harnek-Hall, and Hayward (2007)	EBP	Foster student attitudes toward and aptitude in using research evidence, the United States, n = 77	One-group pre-post	Research methods course (one semester)	Student–teacher, case based	(o) TPK: MCT(+) PPK: LSQ(−) M: LSQd
Straussner et al. (2006)	EBP	Foster the competencies to critically evaluate and apply information from research articles to clinical supervision practice, the United States, n = 29	One-group pre-post	Online course (8 weeks)	Teacher, real world	(+) PDK: LSQ(+) PPK: LSQ
Tennille, Solomon, Brusilovskiy, and Mandell (2016)	Both	To impart EBP skills while enhancing attitudes toward and intentions to practice both the process of EBP and designated EBPs, the United States, n = 72 (72)	rand. Two-group pre-post follow-up	Educational project with dyads (student + field instructor, 14 weeks)	Student, real world	(+) PDK: EBPPAS-short (α = .89)(+) PPK: EBPPAS-short (α = .89)(−/+2)M: EBPAS (α = .76), EBPPAS-short (α = .89)EBPPAS-short: Content, criterion, and factorial validity
Webber, Currin, Groves, Hay, and Fernando (2010)	EBP	Delivering research methods and critical appraisal skills training to social workers, UK, n = 15	Two-group pre-post	E-learning research methods course (12 weeks)	Teacher, not case-based	(o) TDK: Concept Mapping(+2) PPK: LSQ, Focus Group(+) S: Focus Group
Wong (2017)	EBP	To prepare MSW students for appropriate decision-making strategies in working with multicultural client populations, the United States, n = 80	One-group post-only	EBP module at university (9 hr)	Student–teacher, real world	(+) TPK: Assignment(+) PDK: OEQ (κ = .87)(+) PPK: OEQ (κ = .87)(+) M: OEQ (κ = .87)(+) S: LSQ

Note. + ⁿ Signifies number of coded effects. In these cases, effects for more than one construct that refer to one of the DVs (e.g., “attitude” and “intentions to use” subsumed to “motivation”) were measured and reported in the respective study. See Table A2 in Online Appendix for a detailed description. EBP = process of evidence-based practice; ESTs = empirically supported treatments; Both = process of evidence-based practice and empirically supported treatments; (+) = positive effect; (o) no effect; (−) negative effect; EBPAS = Evidence-based Practice Assessment Scale; EBPPAS = Evidence-based Practice Process Assessment Scale; TDK = tested declarative knowledge; TPK = tested procedural knowledge; PDK = perceived declarative knowledge; PPK = perceived procedural knowledge; M = motivation; S = satisfaction; MCT = multiple choice test; LSQ = Likert-type Scale Questionnaire; RS = rating scale; OEQ = open-ended questionnaire; CQ = closed questionnaire; α = internal consistency; r = test–retest reliability; ICC = intraclass correlation coefficient; κ = Cohen’s Kappa; EG = experimental group; CG = control group.

^aOne group received a delayed intervention (after T2). ^bArticles refer to the same study. ^cMean value, computed from n values (n = points of measurement with reference to the study design). ^dThe study reports a positive effect on “motivation,” however, due to our coding scheme, we allocated 7 of 11 items of the respective survey to the construct “perceived procedural knowledge.” An example of an effected item would be “evaluating sample size adequacy.” Afterward, three of the remaining four items were allocated to the construct “motivation.” An example of an affected item would be “original research is confusing.” Means were reported for each item and we summarized the three items with a pooled weighted mean.

Main Characteristics

In line with the PRISMA guidelines (Liberati et al., 2009), Table 5 provides an overview of the study characteristics.

Research Question 1: EIs

Our first research question concerned characteristics of EIs focused on, including instructional approaches and knowledge application. As for instructional approaches, a “student–teacher” approach was used in 15 (55.6%) of the EIs and a “student” approach in 5 (18.5%) meaning that 74.1% of the EIs entailed, at least to some extent, student-centered elements. Two thirds of these studies concerned EBP as a process, focusing on certain steps in the process (see Figure 2).

Figure 2.

Line graph of instructional approaches.

With respect to knowledge application such as working with a fictional case in classroom, working with simulated or real clients and so on, 13 (48.2%) studies asked learners to apply knowledge in real-world practice. Five (18.5%) studies did not use any case-based application (see Figure 3).

Figure 3.

Line graph of knowledge application.

Research Question 2: Effects of EIs

Studies that addressed EBP as a process primarily measured motivation (12 effects, 26.4%) and perceived procedural knowledge (11 effects, 24.2%; see Table 6). Two negative effects were reported, both for motivation and both occurred after a semester-long research course, one with real-world knowledge application (Bender, Altschul, Yoder, Parrish, & Nickels, 2014) and one with case-based knowledge application (Smith, Cohen-Callow, Harnek-Hall, & Hayward, 2007).

Table 6.

Coded Effects.

All Effects	EBP Approach
	EBP, n (%)			ESTs, n (%)			Both, n (%)
	45 (100)			45 (100)			7 (100)
Effect	+	o	−	+	o	−	+	o	−
TDK	1 (2.2)	1 (2.2)	0	6 (13.2)	0	0	1 (14.3)	0	0
TPK	1 (2.2)	1 (2.2)	0	2 (4.4)	0	0	0	0	0
PDK	8 (17.6)	0	0	5 (11.0)	0	0	1 (14.3)	0	0
PPK	11 (24.2)	1 (2.2)	0	16 (35.2)	0	0	1 (14.3)	0	0
M	12 (26.4)	3 (6.6)	2 (4.4)	7 (15.4)	1 (2.2)	1 (2.2)	2 (28.6)	0	1 (14.3)
S	4 (8.8)	0	0	7 (15.4)	0	0	1 (14.3)	0	0
Total	37 (81.4)	6 (13.2)	2 (4.4)	43 (94.6)	1 (2.2)	1 (2.2)	6 (85.5)	0	1 (14.3)

Note. EBP = process of evidence-based practice; ESTs = empirically supported treatments; TDK = tested declarative knowledge; TPK = tested procedural knowledge; PDK = perceived declarative knowledge; PPK = perceived procedural knowledge; M = motivation; S = satisfaction; + = positive effect; o = no effect; − = negative effect.

For studies that addressed ESTs, 43 (95.6%) of the 45 coded effects were positive. Table 6 provides an overview of the coded effects with respect to the different EBP approaches.

Research Question 3: Study Quality

The third research question concerned quality of studies, specifically their designs, reliability, and validity of measures and risk of bias. Three (11.1%) studies were “qualitative,” six (22.2%) “mixed methods” and 18 (66.7%) “quantitative.” Twenty-one (77.8%) studies used a one-group design, 4 of which (two concerned the process of EBP and two EBPs) used only postmeasurements and 8 included follow-ups. About half of the studies applied a one-group pre-post design (48.1%) followed by one-group post-only (14.8%) and one-group pre-post follow-up (11.1%). The designs were evenly distributed among the two EBP approaches. Only six studies (22.2%) used a controlled design. Figure 4 provides an overview of the studies’ designs.

Figure 4.

Line graph of study designs. One-group designs were summarized (pre-post, post-only, and with follow-up measurements) to provide a more accessible overview. The same is true for two two-group pre-post and a three-group pre-post follow-up (quasi-experimental) and two randomized two-group pre-post follow-up and a randomized two-group prerepeated (experimental).

Regarding the reliability and validity of measurement instruments, 38 (67.9%) of 56 measurement instruments were quantitative such as use of a Likert-type scale and 18 (32.1%) were qualitative such as an interview. Of the 38 quantitative instruments, 21 (55.3%) provided data regarding internal consistency and 5 (13.2%) provided data concerning test–retest reliability. For eight (21.1%) quantitative instruments, some sort of validity was mentioned. Two (11.1%) of the 18 qualitative instruments provided a value for internal consistency and 6 (33.3%) provided data regarding interrater reliability. Others provided no such information. With regard to measurement, only one (1.8%) measure was a performance test that was based on observation (in role-play; Sacco et al., 2017). Twenty-five (92.6%) studies based their measures solely (51.9%) or partly (40.7%) on self-report data. Figure 5 provides an overview of the studies’ measurement strategies, that is, whether the participants’ knowledge was actually tested (e.g., multiple choice test) or if they were asked to provide a self-assessment of their knowledge, motivation, and satisfaction (e.g., a Likert-type scale questionnaire).

Figure 5.

Line graph of the measurement strategies.

As for the risk of bias assessment, 1 study scored 0, 4 studies scored 1, 12 studies scored 2, and 12 studies scored 3. No study received an optimal rating of 4. Overall, 15 (55.5%) out of 27 studies scored 0, 1, or 2 (range 0–4).

Discussion

The aim of this article was to provide a comprehensive overview of empirical studies concerned with supporting social work students and/or professionals in their development and application of EBP. We distinguished between two EBP approaches, namely the process of EBP and ESTs. Our main goals were to find out (1) what kinds of interventions have been used so far to foster EBP in social work, (2) what the effects of these interventions are, and (3) to assess the methodological quality of those studies.

EIs and Their Effects

Research Questions 1 and 2 concerned the conceptualization of EIs and their effects in order to find out how to teach the process of EBP and/or ESTs in social work in an effective way. Studies predominantly applied a guided student-centered instructional approach. This approach is viewed favorably for education in the process of EBP (Straus et al., 2011; Tian, Liu, Yang, & Shen, 2013). Based on a meta-analysis that supports the effectiveness of PBL regarding facilitation of application-oriented knowledge and skills (e.g., Dochy et al., 2003), this focus on student-centered teaching seems warranted. Most studies requested participants to apply EBP knowledge in real-world settings. Learning effects reported as a result of using guided student-centered instructional approaches were mostly positive, especially for studies attempting to foster ESTs (94.6%). Other instructional approaches were also reported to be successful. This may suggest to the uncritical reviewer that any kind of intervention may be effective (Dizon, Grimmer-Somers, & Kumar, 2012). However, reliance on self-report data and variable study quality makes it difficult to determine. Notably, there were no measures of actual use of the process of EBP or ESTs in real-life settings or of the fidelity with which ESTs were implemented with one exception. Sacco et al. (2017) assessed fidelity of an EST used with standardized clients. Clearly, more research that includes the use of relevant declarative and procedural knowledge in real-world settings is needed to discover guidelines for teaching both the process of EBP and ESTs.

Assessment of Study Quality

Our third research question addresses study quality. We approached this question in three ways. First, we looked at the designs that were used in the studies we investigated. Only about one fifth of the studies used a controlled design that allowed for comparison of the effects of different types of instruction. The majority of the studies used a one-group pre-post design, followed by a one-group post-only design. Eleven percent of the studies were qualitative and none of which uses a controlled design. Both controlled designs as well as qualitative research studies are important in educational research and both are underrepresented in our sample. As previously noted, studies without a controlled design do not support causal inferences (Shadish et al., 2002). In summary, to date, studies investigating the effects of EIs on EBP in social work leave unanswered questions regarding the best teaching approach, for example, whether the teaching approach they used is superior to alternative approaches.

Second, we looked at the reliability and validity of measures used. Only about 13% of quantitative measures provided data concerning test–retest reliability, about 20% concerning validity, and only one third of qualitative instruments were checked for reliability. To assess declarative and procedural knowledge, self-reports were much more prevalent than performance measures such as multiple choice tests or observation of performance during practice scenarios or in real-life settings. This is problematic for at least two reasons. First, as mentioned earlier, individuals tend to overestimate their knowledge (Kruger & Dunning, 1999; Snibsøer et al., 2018). And second, because the goal of EIs related to EBP is (or at least should be) to help learners become more proficient in the use of the process of EBP and/or ESTs in practice, asking them for whether they feel to be equipped with respect to EBP says little about whether they actually are.

Third, we assessed the risk of bias of the investigated studies. More than half of the studies scored 0-2 (range 0-4). Thus, the very positive results need to be treated with caution. In fact, only one study (Smith et al., 2007) included a “test” as well as a “perception” measure regarding the same dependent variable (procedural knowledge). Even though students reported that they knew more about how to critically analyze research, tests of their knowledge showed no improvement for these skills. This result casts further doubts on relying solely on self-report measures for the assessment of declarative and procedural knowledge, which, as we have seen, seems to be the approach taken in most research on the effects of EIs on EBP in social work. Additional research is needed using reliable, valid performance measures of EBP knowledge and skills.

Recommendations

Given the findings of this review, it is difficult to offer recommendations for teaching the process of EBP and/or ESTs in social work. We should draw on related studies in other areas to inform practices in social work.

Even though most studies in the social work context used EIs based on student-centered teaching approaches, we do not know whether these approaches are actually more effective than other approaches, particularly, more teacher-centered approaches. Perhaps certain kinds of learners (e.g., novices) benefit more from teacher-centered instruction, while others such as more advanced students and practitioners would learn more from student-centered instruction. Evidence from other research areas supports this hypothesis (Kalyuga, 2007). Thus, more research is needed in the social work context to discover what kind of teaching methods under what circumstances are most effective in facilitating the use of EBP by students and practitioners.

Nevertheless, social work educators of course cannot wait for this research to be carried out. In planning courses or other kinds of interventions, we therefore recommend them to carefully review and critically appraise the research evidence they want to base their teaching on and also to consider research from other areas. Based on the review of Aglen (2016), it might be valuable to include aspects of critical thinking (e.g., Gambrill, 2013; Gambrill & Gibbs, 2017) in EBP education. Multifaceted approaches (those using a combination of methods like lectures, computer sessions, small group discussions, journal clubs, and assignments) might be more promising than interventions that offer only one of these methods or no intervention (Kyriakoulis et al., 2016; Patelarou et al., 2017).

Limitations and Conclusions

First, even though all coding was based on reliability checks through double coding, coding still remains a subjective endeavor. Coding was based on published descriptions and some reports failed to provide detailed information, for example, regarding the EI and sample. This might have contributed to moderate interrater reliability values for several variables. Another consequence of lack of detail was that it was not possible to carry out a more specific investigation of the EIs. Second, in order to include qualitative, quantitative, and mixed methods studies, we applied broad operationalizations for the effect variables. This might have contributed to subjectivity in ratings, especially for qualitative results. Also, the classification of effects in “positive,” “no,” and “negative” is coarse-grained. Third, the broad inclusion criteria used in our review resulted in a study sample including a wide variety of EIs and designs making comparison a challenge. Fourth, the almost exclusively positive results reported make it difficult to discover the most effective training methods for EBP in social work. Fifth, more than half of the studies achieved low scores (0, 1, or 2 out of 4) on risk of bias. Sixth, most studies relied on self-reports. Thus, results of research on EBP education in social work need to be treated with caution. We need more studies using controlled designs with measures that focus on performance rather than self-report.

To conclude, much remains to be done to make informed decisions regarding the design of EIs and measurement of their effects. We hope that our study stimulates additional related empirical research.

Supplemental Material

Supplemental Material, 03_EBP_Review_Spens_et_al._-_Appendix - How to Teach Evidence-Based Practice in Social Work: A Systematic Review

Supplemental Material, 03_EBP_Review_Spens_et_al._-_Appendix for How to Teach Evidence-Based Practice in Social Work: A Systematic Review by Florian Spensberger, Ingo Kollar, Eileen Gambrill, Christian Ghanem and Sabine Pankofer in Research on Social Work Practice