Abstract
Introduction
Competency-Based Medical Education (CBME) is being increasingly implemented in post-graduate medical education (PGME) programs, including diagnostic radiology. 1 Among key components of CBME is the application of a framework of competencies, which are outcomes-based and are organized to delineate progressive achievement of competence.2,3 Workplace-based assessments, which foster increased opportunities for feedback and coaching, formulate the foundation of assessment in CBME. 4 Several benefits of CBME in PGME have been described in the literature, including the delineation of standardized specialty-specific competencies,5,6 increased opportunities for the provision of feedback and coaching,6,7 facilitation of modified teaching and learning opportunities for individual residents,5,6 and early identification of residents who may benefit from additional teaching and learning supports. 5 Despite the benefits of CBME, there remains limited knowledge as to how to effectively implement CBME models in diagnostic radiology PGME training.
One of the most established CBME models is Competence by Design (CBD), which was developed by the Royal College of Physicians and Surgeons of Canada (RCPSC). 8 Competence by Design integrates CBME through the application of a framework of competencies with traditional time-based PGME training.8-10 Since 2017, the RCPSC has progressively transitioned the PGME training of residents in various specialties in Canada to CBD. 11 Previously, the PGME training of residents in diagnostic radiology at Queen’s University, Kingston, Ontario was transitioned to CBD as part of an institutional initiative in 2017.12,13 Recently, the PGME training of residents in diagnostic radiology in Canada nationally transitioned to CBD in July 2022 for the incoming resident cohort, and thus diagnostic radiology PGME programs nationally are actively preparing.9,14,15
Implementation of CBME models, such as CBD, requires active involvement of multiple stakeholders not only to ensure successful implementation but also to assure that the intended educational outcomes of CBME are manifested.6,16 In other specialties, provision of professional development activities (PDAs) pertaining to CBD has been identified as being among those strategies integral to achieving these goals.6,17-22 The objectives of this study were to evaluate diagnostic radiology faculty and residents’ knowledge and perceptions of CBD, an established CBME model, and to identify evidence-informed priorities for associated PDAs. Based on our observations and review of the literature, we subsequently propose a framework for PDAs, which may be used by diagnostic radiology PGME programs in preparation for the implementation of CBD or other CBME models. Such frameworks will become of increasing importance as diagnostic radiology and radiology subspecialty PGME programs throughout the world increasingly implement CBME models.
Methods
This was a cross-sectional, survey-based study. Approval was received from the institutional review board (REB H21-02493). Diagnostic radiology teaching faculty and residents at a single, large diagnostic radiology PGME program were eligible to participate. Data collection was by means of the Qualtrics survey tool (Provo, Utah). 23 An anonymized link to the survey was electronically distributed to eligible participants through the department’s electronic mailing list. Eligible participants were provided the opportunity to complete the survey during a 6-week period. The survey was divided into 4 sections: (1) academic demographics, (2) knowledge of CBD, (3) perceptions of CBD, and (4) identification of priorities for PDAs.
Academic Demographics
Academic demographic information was collected, including academic level, academic rank, years in practice, and area of specialization.
Knowledge of CBD
Knowledge of CBD was evaluated through multiple choice questions (MCQs) (Table S1) adapted from Stefan et al (2019), 22 which assessed understanding of major principles and terms utilized in CBD. For each MCQ, participants were asked to select the single correct response out of 5 possible responses. The number of MCQs administered was chosen to balance the evaluation of depth of knowledge with survey length. 24 An overall score attained by each participant on these MCQs was calculated, which served as an indicator of knowledge of CBD.
Perceptions of CBD
Participants’ perceptions of the anticipated impact of CBD on resident education and patient care were evaluated through comparison with the current time-based PGME training paradigm. Participants were asked whether they perceived that CBD would provide better educational experiences than the current time-based PGME training paradigm and were asked to rank their response on a 5-point Likert scale. Participants were also asked whether they perceived that CBD would impact patient care in the radiology department as compared to the current time-based PGME training paradigm and were asked to select a single response from among 5 categorical responses: “yes, patient care will be negatively affected”, “patient care may or may not be negatively affected”, “no, patient care will not be affected”, “patient care may or may not be positively affected”, or “yes, patient care will be positively affected”.
Professional Development Activities
Finally, participants were asked to select priority topics for PDAs from among the following topics previously identified in the literature in other specialties: overview of general concepts of CBD, delivery of high-quality feedback, receiving feedback on teaching, completion of learner assessments following a clinical encounter, and principles of resident promotion and advancement. 22 Participants were also able to provide additional priority topics for PDAs through the provision of free text responses.
Statistical Analyses
Data were summarized by descriptive statistics. Bivariate analyses were conducted as appropriate. Where data points were not provided by participants, the proportion of participants was calculated using the number of participants for which data was available as the denominator.
Results
Participants
Knowledge of CBD
The mean ± standard deviation (SD) overall score on MCQs to evaluate CBD knowledge was 39.0% ± 20.4%, with a range of .0% to 83.3%. Mean performance on each individual MCQ varied from 4.0% to 60.0% (Table S1). The mean ± SD overall score of faculty (38.5% ± 17.9%) was comparable to that of residents (39.7% ± 38.5%,
Perceptions of CBD
Participants’ perceptions of the anticipated impact of CBD on resident education are summarized in Figure 1. Overall, 50.0% of participants perceived that CBD may or may not provide better educational experiences for residents than the current time-based PGME training paradigm. 34.0% of participants perceived that CBD would probably provide better educational experiences for residents while 14.0% of participants perceived that CBD would probably not provide better educational experiences for residents than the current time-based PGME training paradigm. No participants perceived that CBD would not definitely or would definitely provide better educational experiences for residents than the current time-based PGME training paradigm. Likewise, both faculty (44.8%) and residents (57.1%) most often perceived that CBD may or may not provide better educational experiences for residents than the current time-based PGME training paradigm. Perceptions of the anticipated impact of CBD on residents’ educational experiences as compared to the current time-based PGME training paradigm were not statistically significantly variable between faculty and residents (
Participants’ perceptions of the anticipated impact of CBD on patient care are summarized in Figure 2. Only one participant (2.0%) perceived that the implementation of CBD would positively impact patient care in the radiology department as compared to the current time-based PGME training paradigm. Overall, 40.0% of participants perceived that the implementation of CBD would not impact patient care in the radiology department as compared to the current time-based PGME training paradigm. Perceptions of the anticipated impact of CBD on patient care in the radiology department as compared to the current time-based PGME training paradigm were significantly variable between faculty and residents (
Participants’ overall performance on MCQs to evaluate CBD knowledge was not observed to be statistically significantly associated with participants’ perceptions of the impact of CBD on either resident education or patient care in the radiology department (
Priorities for PDAs
Participants identified information on the delivery of high-quality feedback (52.0%), principles of resident promotion and advancement (50.0%), and general concepts of CBD (50.0%) as priorities for PDAs pertaining to CBD (Figure 3). Identified priorities for PDAs pertaining to CBD were not statistically significantly variable between faculty and residents (
Discussion
Despite the increasing emphasis on CBME in PGME, little information remains available to guide diagnostic radiology PGME programs in the implementation of CBME models. Literature from other specialties may be limitedly applicable to diagnostic radiology PGME given the unique nature of diagnostic radiology PGME training thus further exacerbating the impact of the paucity of literature on the implementation of CBME models, such as CBD. This is the first study which identified evidence-informed priorities for associated PDAs in diagnostic radiology PGME.
Overall, our observations suggest that knowledge of CBD is poor. At the time of survey distribution, only a few PDAs, such as grand rounds presentations pertaining to CBD, had been previously provided at our institution. These observations are comparable to that of Stefan
Despite the growing body of literature demonstrating the benefits of CBME in PGME,5-7 our study found that diagnostic radiology faculty and residents are not aware of the advantages of CBD. Most participants in our study perceived the anticipated impact of CBD on residents’ educational experiences as compared to the current time-based PGME training paradigm to be equivocal, which may suggest limited confidence that the implementation of CBD will manifest intended educational outcomes. The RCPSC Pulse Check 6 describes 2 measures pertaining to the evaluation of the implementation of CBD: fidelity and integrity. Fidelity is described as the extent to which “key elements” of CBD have been implemented while integrity is described as “[embracement] of the underlying principles [of CBD] and [appreciation of the rationale] for change”. 6 Our observations suggest that PDAs be targeted to assure the integrity of the implementation of CBD in diagnostic radiology PGME. We recommend that diagnostic radiology PGME programs planning to implement CBD emphasize the educational advantages of CBME over the current time-based PGME training paradigm and include associated evidence where applicable.
The assertion that improved educational experiences of residents shall translate to improved patient care underlies the rationale for the development of CBME.5,26 However, most participants in our study likewise did not perceive the anticipated impact of CBD to affect patient care in the radiology department. Failure of stakeholders to perceive meaningful impacts of CBD poses a significant challenge to its successful implementation. Poor “buy in” to CBD has been recurrently cited in the medical education literature across multiple specialties.6,15,27 While our study did not demonstrate that knowledge of CBD was associated with perceptions of CBD, whether poor knowledge of CBD is contributory to perceptions of CBD must nonetheless be considered. 22 Investment in acquiring diagnostic radiology-specific data to establish an evidence-base to demonstrate the benefits of CBD to all stakeholders shall be integral to the successful implementation of CBD in diagnostic radiology PGME.7,15
Identified priorities for PDAs pertaining to CBD are denoted in
Various domains of faculty development have been previously described. However, there remains a paucity of literature documenting the use of these domains in diagnostic radiology PGME. Steinert (2014) previously described 5 domains of faculty development in the health professions: “teaching improvement”, “leadership and management”, “research capacity building”, “academic and career development”, and “organizational change”.
28
Applying Steinert’s (2014) model of faculty development,
28
Lupi
To inform PDAs, we suggest application of Steinert’s 5 domains of faculty development 28 in keeping with the approach utilized by Lupi et al (2018) 29 to delineate the requisite skills required of each identified stakeholder in the implementation of CBD in diagnostic radiology PGME. Suggested requisite skills required of each identified stakeholder in CBD implementation in diagnostic radiology PGME are summarized in Table 2. Subsequent mapping of our identified priorities for PDAs pertaining to CBD to these aforementioned domains revealed their consistent delineation to the teaching improvement domain (Table 2). Although it is important for all of these domains to be addressed, we recommend PGME programs implementing CBD to focus their PDAs within the teaching improvement domain as a priority. Nonetheless, systematic implementation of PDAs will be required in diagnostic radiology PGME to ensure that all stakeholders possess the requisite skills associated with not only the teaching improvement domain but also all domains. 30
Preferred formats for PDAs pertaining to CBD in diagnostic radiology PGME have only been limitedly assessed. A recent needs assessment identified half-day academic retreats and educational lecture sessions or workshops as preferred formats for development of knowledge of CBD among diagnostic radiology residents. 31 Preference for such immersive formats has been suggested to be attributed to their interactive nature and the opportunity to foster discussion regarding the transition to CBD. 31
There are 2 main limitations to our study. First, our study is limited by its cross-sectional, survey-based design. Only faculty involved in the teaching of diagnostic radiology residents and diagnostic radiology residents were eligible to participate in our study and information was not collected from other stakeholders. Our study is also limited by inherent sampling bias as those with a pre-determined opinion of CBD, whether positive or negative, may have increasingly chosen to participate in this study. Second, our study is limited by a low faculty response rate. This low response rate may demonstrate disinterest in CBD and is worrisome as it may portend poor “buy in” from faculty, which may be a challenge to the successful implementation of CBD in diagnostic radiology PGME. This limitation highlights the importance of evidence-based PDAs. Given this result, it may be wise to emphasize faculty PDAs in the implementation of CBD in diagnostic radiology PGME.
Conclusion
We evaluated faculty and residents’ knowledge and perceptions of an established CBME model, CBD, in diagnostic radiology PGME and identified evidence-informed priorities for associated PDAs. We identified gaps in diagnostic radiology faculty and residents’ knowledge and perceptions of CBD. Assessment of diagnostic radiology faculty and residents’ knowledge and perceptions of CBD may help to inform the provision of associated PDAs. Despite the merits of CBME, CBME models may only be successfully implemented in diagnostic radiology PGME programs if stakeholders understand the principles and educational advantages of CBME over the current time-based PGME training paradigm. Our results and proposed CBD PDAs framework may help to guide diagnostic radiology PGME programs in designing evidence-informed PDAs to strengthen stakeholders’ understanding of the principles and educational advantages of CBD, which may meaningfully contribute to the successful implementation of CBD or other CBME models in diagnostic radiology PGME.
Supplemental Material
Supplemental Material - Knowledge and Perceptions of Competency-Based Medical Education in Diagnostic Radiology Post-Graduate Medical Education: Identifying Priorities and Developing a Framework for Professional Development Activities
Supplemental Material for Knowledge and Perceptions of Competency-Based Medical Education in Diagnostic Radiology Post-Graduate Medical Education: Identifying Priorities and Developing a Framework for Professional Development Activities by Helena Bentley, Kathryn E. DarrasBruce B. Forster, Linda Probyn, Anto Sedlic, and Cameron J. Hague in Canadian Association of Radiologists Journal
Footnotes
Declaration of Conflicting Interests
Funding
Supplemental Material
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
