Abstract
Since its introduction into the social work profession by Gambrill (1999) and Macdonald (1998), evidence-based practice (EBP) has become increasingly influential in social work, particularly in English-speaking countries such as the United Kingdom, the United States, Canada, and Australia. Also, in many northern European countries, including the Netherlands, social workers are now increasingly being urged by policy makers to engage in EBP. For the last decade or so, the Dutch government, local authorities, and funding bodies are demanding more accountability and effectiveness in social work, which has led to increasing attention for EBP as a means of professionalization in social work (Steyaert, Van Den Biggelaar, & Peels, 2010). In 2008, the Dutch Ministry of Health, Welfare, and Sport commissioned the 5-year project effective social interventions to improve the use of research knowledge in social work. This national project was launched by Movisie (the Netherlands Centre for Social Development) to encourage and facilitate the use of research knowledge in social work practice by making available at no cost information about the effectiveness of frequently used social interventions in an online database. Also in 2008, the Dutch Ministry of Education, Culture and Science decided to fund a new master of social work (MSW) program for two universities of applied sciences which aims to create new professionals who focus on the effectiveness of interventions and accountability of the profession (HBO-raad/Vereniging Hogescholen, 2006; Van Pelt, 2011).
Despite these initiatives of policy makers and educators to improve and facilitate EBP in social work, the extent of social work practitioner acceptance of and engagement in EBP is still cause for skepticism, based on the criticisms of social workers and scholars regarding the feasibility and usefulness of EBP in social work. For example, the
In a previous literature study, we found that one of the major issues in the acceptance and the implementation of EBP in the Netherlands is the suspicious attitude (or sometimes even aversion) of social workers themselves (Van der Zwet, Beneken genaamd Kolmer, & Schalk, 2011). Another barrier we found was the inconsistent way EBP is used in Dutch literature. Descriptions are often not clear or transparent, which possibly leads to misperceptions of EBP (and possibly to misplaced criticism or distrust regarding EBP) and obstructs its adoption by social workers and organizations. Also a lack of research skills appeared to be an important barrier to the implementation of EBP in social work.
Recognizing that the EBP process can be successfully implemented in social work only if social workers believe it is both important and feasible, the current study assesses social workers’ views about the EBP process and implementation of the EBP process (Parrish & Rubin, 2012). It focuses on the extent to which Dutch social workers are familiar with the EBP process and feel capable of engaging in the EBP process, accept the EBP process, view that process as feasible, intend to engage in the EBP process, and actually engage in that process. Social workers’ views about the EBP process and use of the EBP process have never been investigated in the Netherlands. The present study adds to the existing literature by assessing Dutch social workers’ orientation toward the EBP process. Furthermore, this study explores whether specific variables (level of education, age, prior courses in EBP as a student and prior continuing education in EBP as a practitioner) are associated with the level of orientation toward the EBP process. Identifying such variables can be helpful in suggesting ways to improve practitioner acceptance and implementation of the EBP process (Rubin & Parrish, 2010). The rationale for investigating the differences in orientation toward the EBP process between the levels of education is the notion that social workers with a master’s degree are perhaps more oriented toward the EBP process than those with only a bachelor’s degree, as they are likely to have had (more) courses about research methods and EBP. The rationale for investigating the differences in orientation toward the EBP process between the different age-groups is the notion that younger social workers, as they are more likely to have had courses in EBP as a student, are perhaps more oriented toward the EBP process than older social workers who were educated in the pre-EBP era. The notion that social workers with prior education in EBP are perhaps more oriented toward the EBP process than those without was the rationale for investigating this association. In the following two sections, we will discuss which definition of EBP was adopted in this study and provide more information on the context of social work in the Netherlands.
Definition of EBP
This study does not focus on “evidence-based practices”; instead, it focuses on the EBP process. Social work literature often fails to distinguish between EBP as a decision-making process and EBPs. Descriptions of EBP in social work literature differ greatly, ranging from the broad definition as envisioned by its originators to narrow, fragmented views (Gambrill, 2011; Gray, Joy, Plath, & Webb, 2013). Recently, several authors have therefore argued that it is important to distinguish the singular term, the EBP “process,” from the plural term, evidence-based practices (or more correctly, empirically supported treatments or empirically supported interventions [ESIs]; Parrish & Rubin, 2012; Thyer & Myers, 2011). The latter refers to interventions for which there is consistent scientific evidence showing that they improve client outcomes (Drake et al., 2001). In contrast, the EBP process has been defined by its founders as a process that involves “the integration of best research evidence with clinical expertise and patient values” (Sackett, Straus, Richardson, Rosenberg, & Haynes, 2000, p. 1). This process involves the following five steps: (1) formulating an answerable practice question; (2) searching for the best research evidence; (3) critically appraising the research evidence; (4) selecting the best intervention after integrating the research evidence with clinical expertise and client characteristics, preferences, and values; and (5) evaluating practice decisions (Straus, Richardson, Glasziou, & Haynes, 2005). The EBP process acknowledges the importance of both clinical expertise and client characteristics/values, along with the consideration of the best available evidence when making practice decisions (Rubin & Parrish, 2011).
Social Work in the Netherlands
In the Netherlands, social workers are professionals who are active in social and community work in a broad sense. Professionals employed in social welfare and social services organizations offer community work, social work, youth work, debt counselling, welfare assistance, shelter for the homeless, social work with the elderly, day care, and support for refugees and asylum seekers. The National Association of Social Workers (NVMW) has a professional register, a professional code, and disciplinary rules; however, social workers are not required to register. Furthermore, as the Dutch government is cutting down on the funding of the social welfare and social services organizations, which are dependent on this funding, organizations are confronted with reorganizations, reductions, and budget cuts. Because of this, the flexible shell of young professionals decreases and the workforce in organizations is aging.
Before 2009, social work education in the Netherlands had only Intermediate Vocational Education and Higher Vocational Education (bachelor-level programs). Twelve of the 14 bachelor-level programs have specific educational programs for
Method
Sample
A convenience sample was obtained (
Our main strategy to find respondents was through the
This resulted in 22 organizations agreeing to participate in the study (of which 16 were recruited via the
Data Collection
The survey was conducted online. There were two online follow-ups. The original EBP Process Assessment Scale (EBPPAS; Rubin & Parrish, 2011), which includes five subscales and 45 items, was used to measure social workers’ views about the EBP process and implementation of the EBP process. This scale is the first validated instrument to assess practitioners’ views on the EBP process and was developed with the intention of measuring indicators of practitioner orientation toward the EBP process in both surveys and evaluative efforts (Parrish & Rubin, 2011). It was developed and validated in the United States by Rubin and Parrish (2011) to specifically tap into practitioner (and student) views regarding the EBP
As reported by Rubin and Parrish (2011), the EBPPAS has an excellent internal consistency, with a reported Cronbach’s α coefficient of .94. The original EBPPAS includes the following five separate subscale constructs: (1) familiarity/self-efficacy with the EBP process (10 items), (2) attitudes toward the EBP process (14 items), (3) perceived feasibility of the EBP process (5 items), (4) intentions to engage in the EBP process (8 items), and (5) actual self-reported EBP behaviors (8 items; Rubin & Parrish, 2011). All 45 items are followed by a 5-point Likert-type scale and 10 of the items convey negative responses about EBP and are reverse scored. The 45 items can be added up to get a composite score assessing the extent to which social workers are oriented to the EBP process. Higher scores indicate a more favorable response in each section and for the overall scale. In the current study, however, we removed 1 item from the attitudes subscale because it had a negative corrected item total correlation.
The questionnaire included the EBPPAS and 13 background/demographic questions. The EBPPAS (see Rubin & Parrish, 2011) was translated into Dutch separately by the researcher (RvdZ) and a translator (with a master of arts [MA] in English language and literature and an MA in translation studies). They compared both translations and agreed upon the best translation. Because the researcher and translator did not find many differences between their translations, they decided it was unnecessary to translate the questionnaire back to English to confirm a correct translation. As EBP is a loanword also used in the Dutch language, there were no problems translating it into Dutch. The background/demographic questions asked about self-reported familiarity with the EBP process, prior courses as a student in the EBP process, prior continuing education as a practitioner in the EBP process, age, gender, years of practice in social work, the number of employees working in their organization, field of practice, current position, years in current position, highest level of education, and years since earning that degree. The online survey was tested with a convenience sample of social workers in order to identify and address any problems. This study was not subject to an institutional review board. In the Netherlands, only studies that are medical–scientific studies, and studies in which persons are subject to procedures and/or are imposed to a way of behaving need to be approved by the Dutch Medical Research Involving Human Subjects Act.
Data Analysis
The Statistical Package for Social Sciences (SPSS) 19.0 was used to run descriptive statistics and to assess internal consistency. Internal consistency was assessed using Cronbach’s α. To assess whether the sample was representative for the entire Dutch population of social workers with regard to age and gender, we used one-sample χ2 tests. Furthermore, as the self-reported familiarity was skewed positively to a large extent, we used a Mann–Whitney
One-way between-group analyses of variance (ANOVAs) were conducted to examine the impact of the following four variables on the overall EBPPAS score: level of education, age, prior courses in EBP as a student, and prior continuing education in EBP as a practitioner. We used one-way between-group ANOVAs to investigate the impact of age, prior courses in EBP as a student, and prior continuing education in EBP as a practitioner on each of the five subscale scores. However, in some cases we needed to run a Welch’s test because the homogeneity of variance assumption was violated. To provide an indication of the magnitude of the effect, an effect size was calculated using eta square (η2; .01 = small; .06 = moderate; .14 = large). To minimize the chance of a Type 1 error, we conducted post hoc analyses using the Tukey’s honest significant difference (HSD) test and the Games–Howell post hoc procedure (in those cases where the homogeneity of variance assumption was not met).
Results
Response Rate and Sample Characteristics
Overall 992 social workers were invited to participate in this study, 565 social workers started the questionnaire and 373 social workers completed the questionnaire. Since we wanted to examine practitioners’ attitudes, we excluded the respondents who reported working in management or policy and research departments. This resulted in a sample of 341 social workers, providing a 34.4% response rate. The 341 social workers who completed the questionnaire also completed all scale items as it was not possible to skip questions.
Table 1 displays the background characteristics of the final sample used for the data analyses. The sample was primarily female (75.7%) and the mean age of the sample was 43 years. The largest proportion of respondents was 50 years of age or older (38.4%), 22.9% was 40–49 years old, 22.3% was 30–39 years old, and only 16.4% was 29 years or younger. The majority (86.8%) of the sample reported no familiarity or very little familiarity with EBP. Only 10.9% of the social workers in the sample reported having taken courses in EBP as a student and an even smaller percentage of social workers (8.8%) reported having taken continuing education in EBP as a practitioner. Social workers in the survey sample reported a mean of 10 years in their current position and a mean of 14 years of practice in social work. The largest proportion of respondents worked with adults (55%,
Sample Characteristics and Background Variables (
Internal Consistency
In the current study, the internal consistency for the entire 45-item scale was excellent, with a Cronbach’s α coefficient of
Coefficient α, Mean Score, Standard Deviation, and Per-Item Mean for Entire Scale and Subscales.
Sample Representativeness
As there is no national data available on all social welfare and social services organizations, we used data of the organizations that are affiliated to the
Profile of Participating Organizations and
With regard to age and gender, the sample was representative of the entire Dutch social worker population. The one-sample χ2 test showed that there was no statistically significant difference (χ2 = 5.1,
Furthermore, because the respondents are perhaps more familiar with the EBP process than the nonrespondents, we also compared the self-reported familiarity mean scores. The Mann–Whitney
Social Workers’ Orientation Toward the EBP Process
The mean score for the entire scale was 128.11, which suggests a slightly lower per-item mean (2.9) than the midpoint of 3 (on a Likert-type scale of 1–5, higher scores reflect a more favorable response; see Table 2). The standard deviation was 17.08, suggesting that the scale allows for detection of variability. The five subscales had per-item means ranging between 2.3 and 3.2, with the behavior subscale lowest (per-item mean = 2.3) and the attitudes subscale highest (per-item mean = 3.2). The attitudes subscale (per-item mean = 3.2) and the familiarity/self-efficacy subscale (per-item mean = 3.1) have higher per-item means compared to the intentions subscale (per-item mean = 2.8) and the behavior subscale (per-item mean= 2.3; see Table 2). These results indicate that although the respondents are slightly familiar with the EBP process and have slightly positive attitudes about it, their intentions to engage in the EBP process and their actual engagement in the EBP process are relatively low. To benchmark Dutch social workers’ orientation toward the EBP process, we compared our findings to a US study. Rubin and Parrish (2011) found a higher per-item (3.3) mean for the entire scale. Furthermore, in the U.S. study, the subscales had per-item means ranging between 2.9 and 3.5, with the behavior subscale lowest (per-item mean = 2.9) and the familiarity/self-efficacy subscale highest (per-item mean = 3.5).
Approximately 5% of the respondents (strongly) agreed with the question, “I know how to skillfully apply the steps of the EBP process,” which is included in the familiarity/self-efficacy subscale. Approximately 50% of the respondents (strongly) agreed with the question, “I understand how to evaluate the outcomes of my practice decisions” (familiarity/self-efficacy subscale). However, it is conceivable that this item was interpreted by some respondents to mean
Education
We found no statistically significant difference in the overall EBPPAS scores between the five levels of education (see Table 4). We found a statistically significant difference in the familiarity/self-efficacy subscale scores for the five different levels of education,
Mean Differences Between Different Levels of Education on Scale and Subscale Scores.
*
Age
Respondents were divided into four groups according to their age (Group 1: <29; Group 2: 30–39; Group 3: 40–49; and Group 4: 50 and above). There was a statistically significant difference in EBPPAS scores for the four age-groups,
Mean Differences Between Age-Groups on Scale and Subscale Scores.
*
We found a statistically significant difference in the familiarity/self-efficacy subscale scores for the four different age-groups,
Courses on the EBP Process as a Student
We also investigated whether there was a difference in orientation toward the EBP process between social workers who reported having followed a course on the EBP process as a student (Group 1), social workers who reported that they did not know whether they did (Group 2), and social workers who reported not having followed a course on the EBP process as a student (Group 3; see Table 6). We found a statistically significant difference in the overall EBPPAS for the three groups,
Mean Differences on Scale and Subscale Scores for “Prior Course in the EBP Process as a Student.”
***
Subscale analyses showed a statistically significant difference in the familiarity/self-efficacy subscale scores for the three groups, Welch’s
Given the finding that social workers under 29 (Group 1) appeared to be more familiar with the EBP process than social workers over 40 (Groups 3 and 4) and the finding that social workers who reported having followed a course on the EBP process as a student are more oriented toward the EBP process than social workers who did not, we wondered whether young social workers were perhaps more familiar with the EBP process because they have had more prior exposure to EBP during their education than older social workers who have been educated in the pre-EBP era. We found that 28.5% of the social workers under 29 (Group 1) in our sample reported having followed a course on the EBP process as a student, while this was only 10.5% of the social workers between 30 and 39 (Group 2), 10.3% of the social workers between 40 and 49 (Group 3) and even 3.8% of the social workers of 50 and above (Group 4). These findings indicate that young social workers have indeed had more prior exposure to the EBP process during their education than older social workers.
Continuing Education as a Practitioner
Finally, we investigated whether there was a difference in orientation toward the EBP process between social workers who reported having followed prior continuing education on the EBP process as a practitioner (Group 1), social workers who reported that they did not know whether they did (Group 2), and social workers who reported not having followed prior continuing education on the EBP process (Group 3; see Table 7). We found a statistically significant difference on the overall EBPPAS for the three groups,
Mean Differences in Scale and Subscale Scores for “Prior Continuing Education in the EBP Process as a Practitioner.”
*
Subscale analyses showed a statistically significant difference in the attitudes subscale scores for the three groups,
Discussion and Applications to Social Work
This is the first study in the Netherlands to measure the levels of social workers’ orientation toward the EBP process and implementation of the EBP process. The primary finding of this survey is that, despite initiatives by policy makers and educators to improve EBP, Dutch social workers in this sample have a relatively low overall orientation toward the EBP process. They are slightly familiar with the EBP process and have slightly positive attitudes about it, but their intentions to engage in the EBP process and their actual engagement are relatively low. When comparing the results of this Dutch study to a US study, Dutch social workers have a lower overall orientation toward the EBP process than respondents from the United States. In both studies the behavior subscale had the lowest score of the five subscales, but the U.S. study had a higher per-item mean (2.9) on the behavior subscale than the Dutch study (2.3). These results are not surprising, as the U.S. sample was obtained from both social workers and students from four areas that were selected because they had prestigious graduate social work education programs that integrated EBP into the curriculum. Social work educators in the United States have been optimistic for a longer time than those in the Netherlands about the usefulness of EBP. This optimism has resulted in the initiation of innovative approaches in various places in the United States to educate students in the EBP process and provide continuing education on the EBP process to social workers (Rubin & Parrish, 2010). Also, a number of American graduate social work programs have adopted EBP as their unifying conceptual framework (Thyer & Myers, 2011). To our knowledge, there have been no similar initiatives to educate students in the EBP process in the Netherlands. Moreover, because the MSW program in the Netherlands is new and thus there are only a few social workers with a master’s degree yet, the Dutch sample contained only 16 social workers with a master’s degree.
This study also explored whether specific variables were associated with the level of orientation toward the EBP process, as these variables can be helpful in suggesting ways to improve practitioner acceptance and implementation of the EBP process. The strongest effect size (the effect size was medium) was identified in a comparison of social workers who reported having followed a course on the EBP process as a student with those who reported not having followed a course on the EBP process as a student: the former are more positively oriented toward the EBP process than the latter. Furthermore, their actual engagement in the EBP process is higher (the effect size was small to medium). They are also more familiar with the EBP process, have more positive attitudes about it, and their intentions to engage in the EBP process are higher; however, these differences were not large. We found no difference with regard to their perceived feasibility of implementing EBP in the real world.
Furthermore, we found that social workers under 29 were more familiar with the EBP process than social workers over 40, although this difference was not large. Given this finding, and the finding that the largest percentage of social workers that followed a course on the EBP process as a student was found in the youngest age-group, we believe it is likely that young social workers are more familiar with the EBP process because they have had more prior exposure to EBP during their education than older social workers. Furthermore, social workers who reported having prior continuing education on the EBP process as a practitioner have more positive attitudes about the EBP process than social workers who reported not having followed prior continuing education on the EBP process, although this difference was not large. We found no statistically significant difference in the overall orientation toward the EBP process between the five levels of education. However, in four of the five groups, the number of respondents was low since most social workers in our sample had a bachelor’s degree (higher vocational education). This may have affected the result. As mentioned before, the low number of social workers with an MSW degree in the sample is not surprising, because only a few social workers have an MSW degree. An unexpected and interesting finding was that social workers with intermediate vocational education were more familiar with the EBP process than social workers with higher vocational education, although this difference was small. It is difficult to explain why this is the case. It might be that there is a difference in the extent to which EBP is taught in intermediate vocational education and higher vocational education. Another explanation could be the occurrence of self-selection bias, as social workers with intermediate vocational education may have been more inclined to not respond to the survey, if they were not familiar with EBP, than social workers with higher vocational education. However, since it is not clear why this finding emerged, additional research is needed to explain and interpret this effect.
There are certain limitations to be considered in interpreting our findings. We were able to obtain a relatively large total number of respondents (
However, despite the aforementioned limitations this study provides preliminary evidence that Dutch social workers are not much oriented toward the EBP process and suggests implications for social work education, practice, and research. In discussing these implications, it is important that we acknowledge our caution in generalizing our findings to the entire population of Dutch social workers. Our sample was probably more familiar with the EBP process than the entire population of Dutch social workers. Furthermore, in our sample large organizations were overrepresented and small organizations were underrepresented, therefore the findings are most applicable to large organizations.
As only 11% of the respondents reported “reading about research evidence to guide my practice decisions” often or very often and approximately 1% of the respondents reported “engaging in all steps of the EBP process” often or very often, there is a need for improvement. Although it might be considered encouraging that approximately 32% of the respondents (strongly) agreed that “EBP helps to improve clients’ outcomes,” only approximately 5% of the respondents (strongly) agreed with the question, “I know how to skillfully apply the steps of the EBP process.” Our results show that having followed courses in EBP is associated with a higher overall level of orientation toward the EBP process. Social workers who followed courses in EBP were more familiar with EBP, had more positive attitudes about it, had more intentions to engage with the EBP process, and were more engaged with the process. Therefore, we recommend universities of applied sciences (and universities) to take a more active role in teaching students (both bachelor and master) and social workers the principles of the EBP process. Educators should be aware however that there was no difference in the perceived feasibility of implementing EBP in the real world. The results from this survey show that social workers see insufficient time and lack of access to research literature as barriers to EBP implementation in practice. Educators should address these feasibility issues, for example, by teaching methods for finding evidence efficiently. In addition, respondents’ skepticism about the feasibility of implementing EBP in practice may be an accurate assessment of the barriers that hamper EBP implementation. This is underscored by findings from a review of empirical studies examining the implementation of EBP in the human services literature that identified several barriers to EBP implementation, such as inadequate agency resources dedicated to EBP, lack of time and access to research evidence (Gray et al., 2013). Addressing these barriers in practice might improve the perceived feasibility to engage in the EBP process and also improve EBP implementation.
In the meantime, current efforts to improve EBP implementation may want to focus especially on social workers who are already familiar with the EBP process. Our study suggests that social workers who followed courses in EBP as a student and social workers under 29 tend to be more familiar with the EBP process. Familiarity, as measured by the EBPPAS, is an indication of social workers’ self-efficacy in using the EBP process. Bender, Altschul, Yoder, Parrish, and Nickels (2014) have argued that it is possible that higher self-efficacy in the ability to use the EBP process will lead to successful implementations of the EBP process. In other words, social workers who feel more confident in their knowledge of the EBP process may be better equipped to apply it in practice. Therefore, it is encouraging that both social workers who followed courses in EBP as a student and social workers under 29 have greater familiarity in the EBP process. However, as mentioned before, the flexible shell of young professionals decreases and the workforce of organizations is aging. Therefore, it might be useful for social work organizations’ human resource departments to take this into consideration. For example, when they are hiring new staff, they might want to select social workers who are familiar with the EBP process.
In order to evaluate the impact of these efforts, it would be interesting to repeat this study in the future and use the findings of the current study for comparison. Future studies should aim to enhance methodological rigor through random sampling and monitoring of reasons for nonresponse to overcome the threats to external validity inherent in this initial study. Furthermore, social work research should assess Dutch MSW students’ level of orientation toward the EBP process. Given the aim of the MSW program to create new professionals that focus on the effectiveness of interventions and accountability of the profession, it is conceivable that MSW students are more oriented toward EBP than, for example, social workers with only a bachelor’s degree. Therefore, we recommend that future research be directed at universities of applied sciences to investigate to what extent MSW students are actually being educated in the EBP process and to evaluate the impact of the MSW programs (preferably with a pretest–posttest design) on MSW students’ orientation toward the EBP process.
