Abstract
Introduction
Teachers are the cornerstone of any education system, serving as the primary architects of students’ intellectual, emotional, and social development (Darling-Hammond, 2017; Darling-Hammond et al., 2020). Effective teachers foster inclusive learning environments, adapt to diverse student needs, and integrate innovative pedagogies to equip students with 21st-century skills such as digital literacy and problem-solving (Stronge, 2018). In primary education, teachers are particularly vital because they lay the foundation for lifelong learning (Bakx et al., 2015). The quality of teacher preparation directly influences educational outcomes, making the evaluation of teacher education programs a critical priority to ensure teachers are equipped to meet these multifaceted demands (Cochran-Smith et al., 2018).
The Turkish education system, centrally administered by the Ministry of National Education (MoNE), oversees a structured framework encompassing primary, secondary, and higher education, with primary education serving as the critical foundation for developing students’ cognitive, affective, and social competencies (Asia-Pacific Network of National Information Centres, 2025). Primary education, spanning Grades 1 to 4, is pivotal in shaping lifelong learning skills and preparing students for subsequent educational stages. Classroom teachers, as the primary facilitators of this foundational stage, bear significant responsibility for fostering critical thinking, emotional resilience, and social adaptability among young learners (Darling-Hammond, 2017). In Türkiye, preservice classroom-teacher training is delivered through 4-year undergraduate programs within faculties of education. The training is designed to equip prospective teachers with the pedagogical knowledge, practical skills, and professional dispositions necessary for effective teaching (Akıncı & Köse, 2022). These programs aim to prepare teachers to meet the diverse needs of primary students while aligning with national educational goals and global standards (KılıçÖzmen, 2019).
However, rapid societal changes, technological advancements, and globalization have intensified the pressure on teacher-education programs. The 21st-century classroom demands teachers who are proficient in digital literacy, inclusive education, and innovative pedagogies; yet studies indicate that Turkish teacher education often struggles with outdated curricula, limited practical training, and misalignment with these contemporary expectations (KılıçÖzmen, 2019; Zeichner et al., 2015). Nevertheless, challenges persist, including insufficient integration of practical experiences, resource constraints, and a lack of systematic evaluation to ensure program effectiveness (Akıncı & Köse, 2022). These gaps underscore the need to rigorously evaluate teacher education programs to ensure they prepare teachers capable of navigating modern educational complexities.
The CIPP Model in Educational Program Evaluation
Program evaluation, defined as the systematic assessment of a program’s merit based on established criteria (Fitzpatrick et al., 2022), is essential for optimizing teacher education. The Context, Input, Process, and Product (CIPP) model, developed by Stufflebeam and Coryn (2014), offers a robust and systematic framework for such program evaluations. The core purpose of the CIPP model is to provide decision-makers with the necessary information to judge the merit of and improve a program (Al-Shanawani, 2019; Dizon, 2023; Stufflebeam, 1983; Stufflebeam & Zhang, 2017). A key theoretical strength of the CIPP model is its dual-purpose utility for both formative and summative evaluations (Al-Shanawani, 2019; Ornstein & Hunkins, 2016). The model’s flexibility and comprehensive nature make it a fundamental and integrated framework for evaluating educational programs (G. Zhang et al., 2011).
The CIPP model assesses four dimensions: context (needs and environmental factors), input (resources and strategies), process (implementation), and product (outcomes). The context component of program evaluation examines the environment in which a program operates, identifying needs, problems, assets, and opportunities that the program is designed to address (Stufflebeam & Coryn, 2014). The input phase examines the resources and strategies available, from curriculum content and faculty expertise to technological tools, ensuring the program has the necessary foundation to prepare preservice classroom teachers for diverse classrooms (Fitzpatrick et al., 2022). The process component scrutinizes how the program is implemented, assessing whether teaching methods, practice opportunities, and assessments are effectively delivered or hindered by issues like large class sizes (Mertens & Wilson, 2025). Finally, the product phase evaluates the outcomes, measuring the program’s success in equipping graduates with competencies like classroom management and inclusivity while also considering their long-term impact on the quality of primary education (Mertens & Wilson, 2025). Its structured yet adaptable approach makes it ideal for evaluating complex educational programs, providing both formative feedback for ongoing improvement and summative insights for accountability (Barrett, 2016; Cochran-Smith et al., 2018; Nkwake, 2023).
The CIPP Model in Higher Education Program Evaluation
The CIPP model has been widely applied in higher education to assess a variety of programs and curricula. Sankaran and Saad (2022) evaluated a Bachelor of Education program in Malaysia using the CIPP model, finding that although the program successfully aligned with national educational goals, it faced challenges in curriculum implementation due to outdated lesson plans and insufficient infrastructure. Similarly, Karimi et al. (2020) applied the CIPP model to assess English language school curriculum in Afghanistan, revealing strengths in knowledge dissemination but weaknesses in resource allocation and follow-up monitoring.
Neyazi et al. (2016) examined undergraduate programs across four faculties at Tehran University of Medical Sciences using the CIPP model. Results of this study showed that most participants rated context factors (goals, management, organization) as undesirable whereas input showed mixed results for adequate field interest and job market prospects but poor faculty quality and inadequate research/educational facilities. Process evaluation indicated deficiencies in student research activities, educational delivery, and assessment methods. Graduate outcomes were also deemed unsatisfactory by most respondents regarding knowledge enhancement and job performance.
Umam and Saripah (2018) evaluated a training program using CIPP. Those authors found that the context dimension showed adequate alignment between training objectives and identified needs although participant communication required improvement. Input evaluation revealed satisfactory preparedness with strong instructor qualifications and facilities but identified minor deficiencies in instructional media suitability. Process assessment indicated effective discussion and demonstration methods while highlighting limitations in field visits and practical training opportunities. The product dimension achieved good results in knowledge, attitudes, and skills development, demonstrating successful competence enhancement among participants. Irene (2023) examined diverse educational programs using the CIPP model, determining that context evaluations showed adequate alignment with program objectives and needs assessments but gaps in participant communication, developmental standards, and curricular-professional examination congruence. Input assessments found sufficient basic resources (instructors, curricula, facilities) but deficiencies in media suitability, teaching competencies, and interactive learning strategies. Process evaluations highlighted effective discussion and demonstration methods while identifying limitations in practical applications, field experiences, assessment procedures, and implementation fidelity. Product outcomes demonstrated successful knowledge and skills enhancement but revealed constraints in socioemotional development, comprehensive evaluation, and professional competency alignment.
S. Zhang et al. (2025) examined blended teaching quality in an organizational behavior course at a public university using the CIPP model for evaluation. The context dimension demonstrated strong foundational alignment among course positioning, objectives, content, and design although requiring clearer conceptual guidance. Input evaluation found adequate online and offline resources, teaching platforms, environments, and teams necessary for implementation, with needs for enhanced resource precision. The process dimension emerged as the primary quality determinant, effectively assessing student learning, teacher instruction, and assessments while identifying optimizational needs in teacher–student interactions. Product evaluation showed positive long-term outcomes regarding influence, sustainability, and generalization potential.
The literature review reveals that the CIPP model has established itself as a robust, versatile, and systematically comprehensive framework for educational program evaluation across diverse contexts, geographical regions, and educational levels. The reviewed studies also collectively highlight several key strengths of the CIPP model. First, its systematic approach enables comprehensive evaluation from program conception through outcome measurement, providing stakeholders with a complete picture of program effectiveness. Second, the model demonstrates remarkable adaptability across various educational contexts. Third, the framework consistently identifies critical gaps and misalignments among program goals and implementation, resource inadequacies, and curricular delivery challenges, thereby facilitating evidence-based decision-making for program improvement.
However, the literature also reveals persistent limitations that warrant careful consideration. The most frequently cited challenge is the model’s tendency toward quantitative data collection methods, which can limit the depth of qualitative insights and stakeholder perspectives. Multiple studies identified consistent weaknesses in the process dimension, particularly regarding practical application opportunities, field experiences, and implementation fidelity. Additionally, although the model’s comprehensiveness is advantageous for systematic evaluation, it may present implementation challenges in resource-constrained environments due to its complexity and extensive data requirements.
CIPP Model Evaluation Using the Mixed-Methods Approach
The use of a mixed-methods approach in CIPP model evaluation is particularly significant because it leverages the strengths of both quantitative and qualitative methodologies to provide a comprehensive understanding of the program’s effectiveness (Ornstein & Hunkins, 2016). Mixed-methods research, as outlined by Creswell and Creswell (2022), balances the limitations of quantitative and qualitative methods by integrating numerical data and in-depth narrative insights. In this study, quantitative data from preservice classroom teachers’ responses to the Classroom Teaching Undergraduate Program Evaluation Scale and their academic performance (GPAs) offer measurable indicators of program outcomes, such as academic achievement and perceived program quality. Qualitative data, gathered through semistructured interviews with lecturers, provide rich, contextual perspectives on the program’s strengths and challenges, capturing nuances that quantitative data alone may overlook (Mertens, 2024).
Incorporating the perspectives of both preservice classroom teachers and lecturers is critical to achieving a balanced and stakeholder-informed evaluation. Their evaluations, particularly from fourth-year students who have completed the full program, reflect the program’s impact on their academic and professional development (Cochran-Smith et al., 2018). Lecturers, as program implementers and experts in curriculum and instruction, provide a complementary perspective on the program’s design, implementation challenges, and alignment with educational goals. Their qualitative insights, drawn from interviews, can identify structural issues such as resource inadequacies or curriculum misalignment and suggest practical improvements (Fitzpatrick et al., 2022). This dual perspective aligns with the CIPP model’s stakeholder-oriented approach, ensuring that the evaluation captures both recipient and provider viewpoints, thus increasing its validity and applicability (Stufflebeam, 2000; Stufflebeam & Coryn, 2014).
The quantitative operationalization of input and product dimensions using first-year and fourth-year GPAs, respectively, reflects established program evaluation practices in which baseline academic capabilities serve as input measures and culminating performance indicators represent program outcomes (McDavid et al., 2018; Stufflebeam & Coryn, 2014). This targeted quantitative focus provides specific, measurable evidence for two critical aspects of program effectiveness, including resource adequacy (input) and learning achievement (product). Integrating these findings allows for a nuanced understanding of how program input translates into measurable products whereas the qualitative data from all four CIPP dimensions provide a contextual framework for interpreting these relationships (McDavid et al., 2018).
The mixed-methods approach with stakeholder perspectives is particularly relevant in the Turkish context, where teacher education faces unique challenges. Studies indicate that Turkish classroom-teaching programs often prioritize theoretical knowledge over practical application, leading to gaps in preservice classroom teachers’ preparedness for real-world classrooms (KılıçÖzmen, 2019). A mixed-methods evaluation using the CIPP model can address these complexities by quantifying outcomes such as academic improvement while qualitatively exploring contextual barriers such as insufficient practice hours or outdated teaching materials. By integrating these perspectives, the evaluation provides a nuanced understanding of the program’s strengths and weaknesses, informing targeted reforms. This stakeholder-driven approach not only enhances the evaluation’s credibility but also ensures that recommendations are practical and contextually relevant, addressing both immediate program needs and long-term educational goals (McDavid et al., 2018).
The CIPP evaluation is further underscored by the broader context of teacher education reform in Türkiye. Recent studies have criticized Turkish teacher education for its lack of systematic evaluation, with over half of existing assessments failing to meet minimal standards of rigor (Akıncı & Köse, 2022; Ayvaz-Tuncel & Çobanoğlu, 2018). The CIPP model offers a solution by providing a structured framework to assess and refine teacher education programs (Stufflebeam & Coryn, 2014). By combining quantitative metrics such as GPA comparisons with qualitative insights from stakeholders, this mixed-methods study ensures a comprehensive evaluation that can guide curriculum redesign, resource allocation, and pedagogical innovation (Fitzpatrick et al., 2022).
The Present Study
In Türkiye, classroom-teaching programs are particularly critical because they form the bedrock of primary education, influencing students’ foundational skills and subsequent academic success. Despite their importance, evaluations of these programs are scarce and often lack the methodological rigor needed to provide comprehensive insights (Akıncı & Köse, 2022). The present study addresses this gap by applying the CIPP model to evaluate the classroom-teaching undergraduate program at Amasya University. The choice to evaluate this specific program using the CIPP model serves multiple purposes. First, it provides a comprehensive assessment of all program dimensions within a systematic framework. Second, it demonstrates the practical application of CIPP evaluation methodology in the Turkish higher education context. Last, it generates evidence-based insights that can inform program improvement not only at Amasya University but potentially at other institutions offering similar programs.
The connection between the CIPP model and preservice teacher education programs is particularly relevant because these programs represent complex educational interventions that require multidimensional evaluation. The context dimension examines whether the program addresses the actual needs of future teachers and the schools they will serve. The input dimension assesses whether adequate resources, curricula, and faculty expertise are available to achieve program goals. The process dimension evaluates how effectively the program is implemented, including teaching methods, student engagement, and practical training opportunities. Finally, the product dimension measures whether graduates possess the competencies needed for effective classroom teaching. This comprehensive evaluation approach is essential for understanding program effectiveness and identifying areas for improvement in teacher preparation.
This study represents the first systematic application of the CIPP model to evaluate a Turkish classroom-teaching undergraduate program, specifically examining the program at Amasya University. The evaluation serves to identify strengths and areas for improvement within the existing program structure, providing evidence-based recommendations for future program development. The aim of this study is to conduct a comprehensive mixed-methods evaluation of the undergraduate classroom-teaching program at Amasya University using the CIPP model, integrating stakeholder perceptions with quantitative performance measures to ensure both depth and empirical validation. Consistent with this general purpose, we sought answers to the following logically connected research questions:
What are the views of preservice classroom teachers and instructors on the context, input, process, and product dimensions of the undergraduate classroom-teaching program?
Is there a significant difference between the GPAs of preservice classroom teachers in the first year and the GPAs of preservice classroom teachers in the fourth year regarding the evaluation of the input and product dimensions of the undergraduate classroom-teaching program?
Methods
Research Design
This study employed a convergent parallel mixed-methods design to evaluate the undergraduate classroom-teaching program using the CIPP model framework (Creswell & Creswell, 2022). Quantitative data provided measurable indicators of program effectiveness and academic performance changes whereas qualitative data captured contextual nuances and stakeholder perspectives that numerical data alone cannot reveal (Mertens, 2024). The convergent parallel mixed-methods design for CIPP program evaluation is presented in Figure 1.
Research design.
Participants
All student participants in this study were enrolled in the 4-year undergraduate classroom-teaching program at Amasya University’s Faculty of Education. I utilized two distinct samples for different quantitative measures. Thirty-eight volunteer preservice classroom teachers completed the program evaluation scale, assessing the context, input, process, and product dimensions of the program. Although the initial invitation was extended to approximately 80 fourth-year students, 38 volunteered to participate, representing a 47.5% response rate. Additionally, 67 students provided consent for the use of their grade point averages from both first and fourth years for academic performance analysis. Participants were selected using purposive sampling (Creswell & Creswell, 2022).
For the qualitative strand, I conducted semistructured interviews with five lecturers teaching within the classroom-teaching program. Initially, 20 lecturers were invited to participate, but only five volunteered, representing a response rate of 25%. Although this represents a smaller number, these five lecturers collectively covered the core areas of the classroom-teaching program, including educational foundations, pedagogical content knowledge, teaching methods, and practicum supervision. This purposive selection ensured representation across the program’s key components rather than comprehensive coverage of all courses, which aligns with qualitative research principles that prioritize depth and expertise over breadth (Patton, 2014). The participating lecturers possessed extensive experience in teacher education, with teaching experience of over 10 years in the program. Lecturers were also selected using the purposive sampling method, with selection criteria focusing on active involvement in the undergraduate classroom-teaching program. All participating lecturers held doctoral degrees and were faculty members within the classroom-teaching department.
Data Collection Tools
To evaluate the undergraduate classroom-teaching program according to CIPP dimensions, data were collected through a multimethod approach. The quantitative data collection involved two sources. First, the Classroom Teaching Undergraduate Program Evaluation Scale, developed by Akdoğdu and Uşun (2017), was administered to 38 fourth-year preservice classroom teachers. This scale comprises 25 items across four sub-dimensions corresponding to the CIPP model: Context (5 items, α = .74), Input (4 items, α = .79), Process (5 items, α = .81), and Product (11 items, α = .93). Responses were recorded on a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree. Second, grade point average (GPA) scores of preservice classroom teachers were utilized. First-year GPA scores were used to inform the input dimension, and fourth-year GPA scores contributed to the evaluation of the product dimension. The use of first-year and fourth-year GPAs to inform the input and product dimensions, respectively, was grounded in the theoretical framework of the CIPP model and established practices in educational program evaluation.
According to Stufflebeam and Coryn (2014), the input dimension assesses the baseline resources, capabilities, and preparedness that students bring to a program, which directly influences program implementation and outcomes. First-year GPA serves as a proxy measure for students’ foundational academic capabilities and readiness upon program entry, representing their preparedness to engage with program content and activities. This baseline academic performance indicator helps evaluate whether students possess adequate foundational skills to benefit from the program’s educational interventions (Mertens & Wilson, 2025). The product dimension, conversely, measures program outcomes and the extent to which educational objectives have been achieved (Stufflebeam & Coryn, 2014). Fourth-year GPA represents students’ academic achievement after completing the majority of program coursework and experiences, serving as an outcome indicator of their cumulative learning and skills development. The comparison between these 2 time points (first-year as input, fourth-year as product) follows established longitudinal evaluation practices in higher education that assess program impact through pre- and post-academic performance measures (Babadi et al., 2024; Derakhshanfard et al., 2022; Gunung & Darma, 2019; Pugh & Tschannen-Moran, 2016).
For the qualitative dimension, I developed a semistructured interview form to gather in-depth perspectives from 15 lecturers in the classroom-teaching program. The form covered the program’s context, input, process, and product dimensions. The development of the interview form followed a systematic process. Initially, I had established the objectives for evaluating each CIPP dimension through a review of the relevant literature. Subsequently, I drafted preliminary interview questions aligned with these objectives, resulting in an initial pool of 32 questions. To ensure the quality and appropriateness of the questions, three experts in the field of curriculum and instruction reviewed the form. Based on expert feedback and recommended revisions, I finalized the interview form, which comprised 20 questions distributed across the CIPP dimensions.
The semistructured interview questions were based on Stufflebeam and Coryn’s (2014) CIPP evaluation framework objectives, addressing each dimension through targeted questions. Context evaluation examined program-needs alignment, resource adequacy, course distribution, and environmental conditions (seven questions). Input evaluation assessed teaching strategies’ appropriateness, material adequacy, implementation compatibility, and budget feasibility (four questions). Process evaluation investigated program implementation fidelity, strategy execution, and implementation challenges (five questions). Product evaluation explored goal achievement, competency development, knowledge foundation provision, and program sustainability (four questions). Complete interview protocols aligned with specific CIPP objectives are provided in the Supplemental Materials (Table S1). Table 1 illustrates the specific mapping of quantitative and qualitative data sources to each evaluation dimension.
CIPP Model Operationalization in Data Collection.
Introduction of Amasya University Classroom-Teaching Undergraduate Program
The classroom-teaching undergraduate program is a 4-year program comprising a total of eight semesters. Students are admitted to the classroom-teaching undergraduate program through Türkiye’s centralized university entrance system. Admission requirements include completion of high school education with a diploma, minimum score requirements on the Higher Education Institutions Examination in the equal weight score type, and meeting the base score threshold established by the Council of Higher Education. No additional special aptitude tests or interviews are required for admission. The program admits students annually through the national placement system, with admitted students representing diverse geographical backgrounds from across Turkey and varying levels of academic preparation in foundational subjects relevant to primary education.
The program, which has been implemented since the 2018 to 2019 academic year, includes 60 courses comprising field knowledge, professional knowledge, and general culture courses and consists of 240 European Credit Transfer System (ECTS). The program operates within the framework established by the Council of Higher Education and follows national teacher competency standards set by the Ministry of National Education. It aims to prepare graduates who can teach all subjects in primary education (Grades 1–4), demonstrate competency in child development and learning psychology, integrate technology effectively in classroom instruction, and serve diverse student populations, including those with special educational needs.
The program is delivered by faculty members generally holding doctoral degrees in education-related fields, including curriculum and instruction, educational psychology, educational foundations, and subject-matter specializations. Key facilities supporting the program include specialized classrooms equipped for mathematics and science education methods courses, drama and arts studios for creative pedagogy development, computer laboratories with educational software and digital learning tools, a well-equipped library with educational resources, and established school–university partnership agreements for practicum placements in various primary schools across the region.
The program includes compulsory field knowledge (25 courses), professional knowledge (20 courses), general culture knowledge (11 courses), and practical knowledge (four courses). Courses employ diverse pedagogical approaches designed to model effective teaching practices while developing preservice classroom teachers’ competencies. Instructional methods include traditional lectures for establishing theoretical foundations in education, psychology, and subject-matter knowledge; hands-on laboratory sessions for science and mathematics teaching methods; microteaching sessions in which students practice teaching skills in controlled environments; drama-based activities and creative arts integration; school-based practicum experiences in partner institutions providing authentic classroom contexts; collaborative group projects and peer learning activities; technology-enhanced instruction utilizing digital tools and platforms; portfolio-based assessment requiring reflection on professional development; and case study analysis connecting theory to real classroom situations. Students who successfully complete the program by fulfilling all the requirements receive a Bachelor’s Degree in Classroom Education.
The program employs multiple assessment strategies to evaluate preservice classroom teacher development across cognitive, affective, and practical dimensions. These include traditional examinations for knowledge assessment, performance-based evaluations during practicum experiences, portfolio submissions demonstrating professional growth, peer and self-assessment activities, and capstone projects integrating program learning outcomes. Professional development components include mentorship relationships with experienced educators, participation in educational conferences and workshops, and opportunities for research engagement in education-related topics. Table 2 shows the structure of the program.
Classroom-Teaching Undergraduate Program.
Procedures
Prior to data collection, I obtained ethical approval from the Amasya University Ethics Committee. All participants were informed about the study’s purpose and procedures, and I obtained written consent before participation. Quantitative data were collected from 38 fourth-year preservice classroom teachers who participated voluntarily. Additionally, I obtained academic records of 67 preservice classroom teachers from the university’s Student Affairs Department to compare first-year and fourth-year GPAs after receiving permission from students. For qualitative data, I conducted semistructured interviews with five instructors from the classroom-teaching department. Interviews averaged 50 min, were audio-recorded, and were transcribed verbatim.
Data Analyses
Quantitative data were analyzed using SPSS 26. Descriptive statistics including frequencies, percentages, and means were calculated for scale responses. Program evaluation ratings were interpreted using established thresholds: scores of 1.00–2.99 indicated insufficient evaluation, 3.00–4.99 indicated moderately sufficient, and 5.00–7.00 indicated sufficient (Akdoğdu & Uşun, 2017). For the comparison of preservice classroom teachers’ first- and fourth-year GPAs, I conducted a dependent samples
Qualitative data underwent content analysis following a systematic coding process (Lahman, 2024; Mertens, 2024). Initial open coding was followed by axial coding to identify subthemes; then selective coding was used to establish main themes, consistent with established qualitative analysis procedures (Cohen et al., 2018; Creswell & Creswell, 2022; Lahman, 2024; Mertens, 2024). Instructor interview findings are presented in a condensed format, with complete analysis available in the Supplemental Materials (Table S2–Table S5).
I established the validity and reliability of this mixed method study in several manners. The classroom-teaching program evaluation scale developed by Akdoğdu and Uşun (2017) demonstrated strong reliability with Cronbach’s alpha internal consistency coefficients of .75 for context evaluation, .79 for input evaluation, .81 for process evaluation, .83 for product evaluation, and .93 for the overall scale. These values exceed the recommended threshold of 0.70, confirming instrument reliability (George & Mallery, 2024). For qualitative data collection, I developed a semistructured interview protocol based on the context, input, process, and product dimensions of the CIPP assessment model. I established content and face validity through expert review by three faculty members from the Curriculum and Instruction program, with revisions made according to their recommendations (Fitzpatrick et al., 2022).
Consistent with contemporary qualitative research standards, I employed trustworthiness criteria rather than traditional validity and reliability measures (Lahman, 2024; Mertens, 2024). To enhance credibility, I implemented several strategies: (1) expert review throughout the research process, (2) member checking by sharing interview transcripts with participants for verification, and (3) triangulation of data sources by comparing instructor interviews with quantitative findings (Creswell & Creswell, 2022; Lahman, 2024; Mertens, 2024; Patton, 2014). Detailed documentation of research procedures and analytical decisions established an audit trail, supporting both dependability and confirmability (Korstjens & Moser, 2018).
Results
Results of Context Evaluation
Table 3 presents the frequency, percentage, and mean values from the “classroom-teaching undergraduate program evaluation scale” administered to fourth-grade preservice classroom teachers regarding the context dimension of the program.
Pre-service Classroom Teachers’ Evaluation of the Context Dimension of the Classroom-Teaching Undergraduate Program.
As seen in Table 3, analysis of the preservice classroom teachers’ responses reveals mean scores ranging from 4.13 to 4.95 across all items in the context dimension, with an overall mean of 4.44, indicating that participants rated this dimension as moderately adequate. Within this dimension, the item “The number of field knowledge courses is sufficient” received the highest mean score, whereas “The program is qualified to provide classroom-teaching special field competencies” and “The program is theoretically functional” received the lowest mean scores. These findings suggest that preservice classroom teachers particularly value the acquisition of specialized field competencies and theoretical functionality within the program context.
Table 4 presents the qualitative analysis results, derived from instructor interviews regarding the context evaluation dimension.
Condensed Context Evaluation From Instructor Interviews.
As seen in Table 4, the qualitative findings triangulate with quantitative results, revealing deeper insights into the moderate satisfaction scores. Although students rated field-knowledge courses positively, instructors identified significant alignment issues. Lecturer 4 highlighted deficiencies in “achievements in the program, the teaching methods, techniques, tools, and projects,” and Lecturer 3 emphasized contextualization needs: “The profile of the incoming students and their needs can change … there should be general consistency … but these should be determined … to be more realistic.” Regarding resources, mixed perceptions emerged. Lecturer 1 emphasized physical constraints: “A faculty is not just a classroom … the classroom atmosphere is not suitable, the equipment is not enough, there is no laboratory.” However, Lecturer 3 noted specialized strengths: “There are elective courses on games, sportive games … I see that they are also more effective.”
The lower competency-development scores align with instructor concerns about the program’s effectiveness in developing higher-order thinking skills. Lecturer 5 suggested improvements: “There should be more courses suitable for the field of classroom teaching … more such courses with games should be added.” However, Lecturer 2 expressed skepticism about the feasibility of implementing such changes, responding bluntly, “It does not, it will not, because which one of them will.” This pessimistic view suggests institutional barriers or resource constraints that may prevent meaningful curriculum reform. The tension between recognizing the need for improvement (Lecturer 5) and doubting the capacity for change (Lecturer 2) reveals not only systematic gaps between program design and implementation effectiveness but also a lack of institutional commitment or infrastructure to address identified deficiencies.
Results of Input Evaluation
Table 5 presents the frequency, percentage, and mean values from the Classroom Teaching Undergraduate Program Evaluation Scale administered to fourth-grade preservice classroom teachers regarding the input dimension of the classroom-teaching undergraduate program.
Preservice Classroom Teachers’ Evaluations of the Input Dimension of the Classroom-Teaching Undergraduate Program.
Table 5’s analysis of the preservice classroom teachers’ responses reveals mean scores ranging from 3.66 to 4.16 across all items in the input dimension, with an overall mean of 3.92, indicating that participants rated this dimension as moderately adequate. Within this dimension, the item “The physical environment where classes are held positively affects success” received the highest mean score (4.16) whereas “The visual and audio materials used are engaging” received the lowest mean score (3.66). These findings suggest that preservice classroom teachers particularly value adequate physical learning spaces and technological infrastructure yet express concerns about the program’s practical implementation functionality and the quality of instructional materials. The results indicate that preservice teachers expect enhanced implementation effectiveness and more engaging educational resources within the program’s input dimension.
Table 6 presents the qualitative analysis results from instructor interviews regarding the input evaluation dimension.
Condensed Input Evaluation From Instructor Interviews.
As seen in Table 6, the qualitative findings help explain the moderate quantitative ratings. Although students appreciated physical environments, instructors reported implementation challenges. Lecturer 5 noted, “Especially in some courses, the insufficient class hours don’t give us time to have students practice, no matter how appropriately we select strategies, methods, and techniques according to the lesson objectives.” Conversely, Lecturer 4 emphasized positive outcomes: “I think it’s extremely appropriate because we frequently benefit from enriched and diversified methods and techniques in our courses.” Material adequacy received mixed reviews, with Lecturer 3 explaining contextual constraints: “What these materials can be is not only determined by possibilities and conditions, but also by the lesson itself and its nature.” Implementation compatibility varied significantly based on external factors. Lecturer 1 highlighted multiple challenges: “Physical space creates an advantage or disadvantage, student profile creates an advantage or disadvantage, the faculty’s own facilities create advantages and disadvantages.” Fixed seating arrangements emerged as a particular obstacle, with one instructor noting, “Students had difficulty performing activities due to the fixed seating arrangement in the amphitheater.”
Budget concerns were universal. Lecturer 5 questioned technology currency: “Smart boards were installed 7–8 years ago—when were they last revised? When were they updated? I don’t think it’s very likely.” Resource limitations forced creative solutions, with one instructor noting, “We don’t have poster boards, they don’t have environments to display their products, we create and form them with our own means,” requiring faculty to seek external support from local businesses.
Results of Process Evaluation
Table 7 presents the frequency, percentage, and mean values from the classroom-teaching undergraduate program evaluation scale administered to fourth-grade preservice classroom teachers regarding the process dimension of the classroom-teaching undergraduate program.
Preservice Classroom Teachers’ Evaluations of the Process Dimension of the Classroom-Teaching Undergraduate Program.
As seen in Table 7, analysis of the preservice classroom teachers’ responses reveals mean scores ranging from 3.82 to 4.74 across all items in the process dimension, with an overall mean of 4.32, indicating moderate satisfaction with this dimension. The highest mean score was observed for “The faculty members communicate with the students in the department outside the class” whereas the lowest scores were recorded for “The program is qualified to provide the special field competencies of classroom teaching” and “The methods and techniques of the faculty members provide easier learning.” These findings suggest that preservice classroom teachers desire more diverse and effective teaching methodologies within the program’s process dimension. Table 8 presents the qualitative analysis results from instructor interviews regarding the process-evaluation dimension.
Condensed Process Evaluation From Instructor Interviews.
The qualitative findings complement quantitative results showing moderate process satisfaction. Implementation challenges were evident in Lecturer 1′s description: “I have 10 hours of class … teaching sociology of education to five groups … approximately 50 [students each] … I do not have the chance to give activities, homework, projects to 250 students every week.” Lecturer 5 emphasized structural issues: “Many courses are still taught theoretically … but by its very nature, classroom teaching should be … practice-oriented.”
However, adaptation strategies emerged. Lecturer 3 described innovative approaches: “I look at my children’s faces, they are asleep … I put on some nice music … everyone starts to wake up; they start to focus their attention on me.” Lecturer 2 noted proactive material preparation: “I use teaching materials sufficiently … I prepare some activities myself … but I also ask students to prepare materials and activities.” Mixed implementation success was reported, with Lecturer 4 stating, “I do the practices according to this plan, and I think it is effective,” whereas Lecturer 2 acknowledged limitations: “The conditions for implementation are not available.” The strong faculty–student communication ratings align with these adaptive efforts although concerns about teaching effectiveness correspond with the structural challenges identified by instructors.
Results of Product Evaluation
Table 9 presents the frequency, percentage, and mean values from the classroom-teaching program evaluation scale administered to fourth-grade preservice classroom teachers for evaluating the product dimension of the classroom-teaching undergraduate program.
Preservice Classroom Teachers’ Evaluations of the Product Dimension of the Classroom-Teaching Undergraduate Program.
As seen in Table 9, analysis of the preservice classroom teachers’ responses reveals mean scores ranging from 3.26 to 4.37 across all items in the product dimension, with an overall mean of 3.91, indicating that participants rated this dimension as moderately adequate. The highest mean score was observed for “The courses in the program complemented one another” whereas the lowest scores were recorded for “The program provides classroom teachers with the competence to become administrators in the schools where they will work” and “The program provides the necessary skills for correspondence and preparation of official documents during the execution of the profession.” These findings suggest preservice classroom teachers desire stronger preparation in administrative competencies and official documentation skills within the program’s product dimension. Table 10 presents the qualitative analysis results from instructor interviews regarding the product evaluation dimension.
Condensed Product Evaluation From Instructor Interviews.
The triangulated data reveal mixed achievement outcomes. Although Lecturer 1 reported 80% to 85% goal achievement, Lecturer 3 noted limitations: “Such diligent and meticulous work in only one lesson and in a limited time is not enough; it is important to support it with other lessons in terms of its continuity.” Competency development concerns were prominent. Lecturer 2 highlighted structural imbalances: “Teacher training in Türkiye today is based only on a cognitive structure. Affective gains should always be at the forefront.” Lecturer 3 emphasized missing skills: “Teamwork, collaborative work, enabling children to think critically and analytically, and gaining skills such as questioning.” External factors also influenced outcomes; Lecturer 2 observed declining motivation among students who “think that I will not be appointed as a teacher. I will become a police officer.” Lecturer 5 noted shared responsibility: “The prospective teacher must also take responsibility and fulfill his/her duties in this process.” The moderate product evaluation scores reflect these mixed findings, with high course complementarity ratings contrasting with concerns about administrative skills and practical competency development.
Results of GPA Analysis
Table 11 reports a paired-samples
Comparison of Preservice Classroom Teachers’ GPA Scores Between First and Fourth Years.
As seen in Table 11, analysis of preservice classroom teachers’ academic performance revealed statistically significant differences between first-year and fourth-year grade point averages (
Discussion
In this convergent parallel mixed-method study, I employed the CIPP model to evaluate the undergraduate classroom-teaching program at Amasya University, revealing moderate effectiveness among preservice teachers and lecturers across the context, input, process, and product dimensions of the classroom-teaching program, alongside significant academic improvement among preservice classroom teachers. The evaluation revealed that preservice teachers and lecturers perceived the classroom-teaching program to be moderately sufficient. This finding aligns with Khaksar et al. (2023), who reported generally positive yet qualified stakeholder perspectives in a private-sector teacher-training context. Similarly, Sankaran and Saad (2022) observed moderate satisfaction across CIPP dimensions when evaluating a Bachelor of Education program, noting that formative and summative insights often converge to indicate areas of strength alongside opportunities for enhancement. These findings are also consistent with recommendations of researchers emphasizing the importance of systematic program evaluation in teacher education (Darling-Hammond & Oakes, 2019; Mertens & Wilson, 2025).
The context evaluations in this study revealed that preservice classroom teachers rated the program as moderately sufficient, particularly in the adequacy of field knowledge courses, yet expressed concerns about its capacity to develop specialized teaching competencies and theoretical functionality. This finding aligns with Sankaran and Saad (2022), who observed that context evaluations often uncover gaps between curriculum design and evolving teaching environments in Bachelor of Education programs. However, our study extends this understanding by revealing a critical stakeholder perspective divergence. Although students showed moderate satisfaction, instructors were substantially more critical, noting incompatibilities with digital-age needs, societal conditions, and student profiles.
These findings are consistent with international and national studies that identified similar problems in teacher-education programs. In a study examining teacher-education programs, Mhishi et al. (2012) found that teacher-education programs in developing countries often lag behind technological and pedagogical innovations. KılıçÖzmen (2019) also indicated that classroom-teaching education programs in Türkiye must adapt to globalization and technological advancements to remain relevant. The instructors’ emphasis on the need for a flexible, contextually responsive curriculum aligns with recent calls for dynamic teacher-education frameworks that address 21st-century skills (Darling-Hammond et al., 2020). The discrepancy between preservice classroom teachers’ moderate satisfaction and instructors’ critical perspectives may reflect differing expectations, with students focusing on immediate academic needs and instructors prioritizing long-term professional preparedness (Cochran-Smith et al., 2018).
The discrepancy between preservice classroom teachers’ moderate satisfaction and instructors’ critical perspectives reveals a possible evaluation challenge that has implications beyond this specific program. Students focused on immediate academic success and credential attainment. Thus, students may lack the professional expertise to recognize gaps in their preparation that only become apparent through teaching experience. This aligns with situated learning theory, which suggests that authentic learning contexts are essential for meaningful skills development but may not be apparent to novice learners (Korthagen, 2010). The instructors’ concerns about age/digital mismatch and skills gaps in higher-level thinking development suggest that the program’s contextual foundation requires substantial updating to address contemporary educational demands.
The input evaluation highlighted mixed perceptions about resource adequacy and course distribution. Preservice classroom teachers’ moderate satisfaction with course allocations contrasts with instructors’ concerns about the appropriateness of teaching strategies, material adequacy, implementation compatibility, and budget feasibility. These findings are consistent with previous studies that indicate that resource adequacy problems impede delivery of core competencies among students (Lippe & Carter, 2018). According to Stufflebeam and Coryn (2014), input evaluations should guide resource allocation and strategic planning to meet program objectives. Recent studies suggest that teacher education programs often struggle to balance theoretical and practical components, particularly in resource-constrained settings (Zeichner et al., 2015). The instructors’ call for increased practice hours and quality-focused course design supports McDavid et al.’s (2018) advocacy for aligning input strategies with implementation environments to enhance program feasibility and effectiveness. Dwikurnaningsih et al. (2022) recommended targeted resource investment and flexible implementation plans to overcome such barriers identified among instructors.
However, our study reveals a more complex dynamic than traditional input–output models suggest. Despite reporting universal inadequacy in budget and technology, instructors demonstrated remarkable adaptive innovation, utilizing drama facilities, creating outdoor learning experiences, and developing student-generated materials. This finding show how faculty agency can partially compensate for institutional constraints. The adaptive strategies represent forms of pedagogical resilience that merit theoretical consideration within complexity-theory approaches to educational change, suggesting that programs can maintain functionality through distributed adaptation rather than centralized resource provision (Mason, 2008). The sustainability concerns raised by instructors, however, highlight the limitations of individual adaptation strategies for addressing systemic deficiencies. Thus, creative solutions may not provide long-term program stability.
The process evaluation indicated moderate satisfaction among preservice classroom teachers regarding teaching methods and instructor–student interactions, but instructors reported significant implementation challenges, including excessive course hours, large class sizes, and systemic barriers. These issues align with Fitzpatrick et al.’s (2022) observation that process evaluations are critical for identifying implementation gaps and guiding mid-course corrections. The instructors’ adaptive strategies—such as leveraging technology and extracurricular activities—reflect innovative responses to resource limitations, consistent with recent research on resilient teaching practices in higher education (Brammer, 2020). However, the lack of practical course hours underscores a broader issue in teacher education: the prioritization of theoretical instruction over experiential learning (Cochran-Smith et al., 2018).
The qualitative findings help explain the quantitative ratings by revealing how structural constraints limit pedagogical effectiveness. Instructors’ reports regarding teaching demonstrate implementation barriers that prevent effective skills transfer despite faculty commitment to diverse teaching methodologies. However, the instructors’ adaptive strategies reflect innovative responses that partially overcome structural limitations. The finding that students rated faculty–student communication highest also suggests that despite implementation challenges, interpersonal relationships remain strong, indicating faculty dedication to student success. The persistent concern about insufficient practical course hours underscores a broader issue in teacher education, the prioritization of theoretical instruction over experiential learning (Cochran-Smith et al., 2018). The instructors’ observations highlight a fundamental misalignment between program design and professional preparation requirements.
The product evaluation demonstrated moderate satisfaction among preservice classroom teachers, with particular weaknesses in administrative and special education competencies. Instructors similarly noted deficiencies in affective and higher-order cognitive skills development, attributing these to an overly cognitively focused curriculum. These findings are consistent with Canrinus et al. (2019), who found that teacher education programs across different countries often emphasize knowledge acquisition over dispositional development. These findings also corroborate Darling-Hammond’s (2017) argument that teacher education must foster holistic competencies, including emotional and social skills, to prepare teachers for diverse classroom demands. The significant GPA improvement from first to fourth year suggests that the program contributes to academic development, aligning with Nkwake’s (2023) emphasis on measurable outcomes in program evaluation.
The triangulated qualitative data reveals concerning disconnects between academic achievement and practical competency development. Moreover, research suggests that external factors such as limited career prospects in teaching significantly affect student motivation and program outcomes. Concerns about inadequate competency development are alarming, given the program’s core mission. Instructors note deficiencies in emotional and social skills as well as limited preparation for teamwork and the fostering of critical thinking in students, indicating that graduates may be inadequately prepared for the complex demands of contemporary classrooms. Moreover, the instructors’ concerns about employment challenges and outdated course content highlight the need for curricula that prepare teachers for real-world professional contexts (Zeichner et al., 2015).
The interpretation of GPA improvement as evidence of program effectiveness requires careful consideration of alternative explanations. Academic performance gains may reflect natural developmental processes, increased familiarity with university assessment practices, or changes in grading standards rather than enhanced teaching competencies (Carter & Lara, 2016; Morris et al., 2021; Pascarella & Terenzini, 2005). This limitation aligns with concerns raised in educational evaluation literature about the inadequacy of single metrics to assess complex educational outcomes (Cochran-Smith et al., 2018). Future research should incorporate multiple assessment methods, including portfolio assessments of teaching artifacts, classroom observation scores during practicum experiences, employer evaluations of graduates, and longitudinal tracking of teaching effectiveness in professional practice to provide more comprehensive evidence of program impact.
The CIPP evaluation reveals that the moderate effectiveness ratings mask interconnected systemic challenges that span all program dimensions. The theory–practice divide identified in the context dimension manifests in input resource allocation (insufficient practical courses), process implementation (limited hands-on learning opportunities), and product outcomes (weak practical competencies). This convergent evidence suggests that addressing the program’s limitations requires comprehensive, systematic reform rather than incremental adjustments within individual dimensions.
The stakeholder perspective divergence observed across all dimensions has profound implications for program evaluation methodology. Students’ moderate satisfaction contrasts systematically with instructors’ critical assessments, suggesting that traditional evaluation approaches relying primarily on student feedback may underestimate program deficiencies. This finding supports recent calls for methodological sophistication in teacher education evaluation research and emphasizes the importance of triangulated data sources. The resource constraint–innovation dynamic identified in input and process dimensions demonstrates both the resilience of educational professionals and the limitations of individual adaptations for addressing structural problems. Although instructor creativity partially compensates for institutional inadequacies, the sustainability concerns suggest that long-term program effectiveness requires institutional commitment to comprehensive resource allocation and pedagogical innovation.
Practical Implications
The findings suggest several critical directions for program redesign that address the systematic weaknesses identified across CIPP dimensions. First, the persistent theory–practice divide requires structural curriculum revision that integrates practical experiences throughout the 4-year program rather than concentrating them in final-year practicum courses. Curriculum designers should expand teaching practicum opportunities and incorporate simulation-based training to provide sustained experiential learning (Darling-Hammond & Oakes, 2019).
Second, faculty development initiatives should focus on pedagogical innovation that combines theoretical knowledge with practical application. Institutions must invest in enhancing instructors’ capacity to employ diverse teaching methodologies, particularly those fostering critical thinking and affective competencies that instructors identified as lacking (Day & Gu, 2013). This professional development should equip faculty with strategies to develop students’ emotional intelligence, ethical reasoning, and the dispositional qualities essential for effective teaching.
Third, resource allocation must prioritize infrastructure development that supports experiential learning. Physical and technological inadequacies, including insufficient laboratory facilities, outdated equipment, and limited technology integration, require immediate attention to create an enabling environment for learning (McDavid et al., 2018). Strategic investments should also support community partnerships that facilitate authentic teaching experiences.
Fourth, stakeholder engagement processes should be strengthened to incorporate external perspectives from practicing teachers and school administrators. This collaborative approach ensures program–field alignment and addresses the evaluation methodology limitations identified in this study, helping to bridge the gap between program outcomes and actual classroom demands. Finally, regular program evaluations using comprehensive models like CIPP should be institutionalized to ensure continuous improvement and alignment with evolving societal and educational needs (Mertens & Wilson, 2025).
Limitations
This study has several limitations that should be acknowledged when interpreting the findings. The first limitation of this evaluation study is its inability to establish causal relationships between program components and student outcomes. The pre–post GPA comparison, although showing statistically significant improvement, cannot be attributed directly to program effectiveness due to the absence of a control group. The second limitation of the study is the methodological design choice to collect only quantitative data from preservice classroom teachers rather than incorporating their qualitative perspectives through interviews or focus groups. Although the study employed a convergent parallel mixed-methods approach in which quantitative data from students were triangulated with qualitative data from lecturers, the absence of in-depth student voices represents a constraint on the comprehensiveness of the evaluation.
Additionally, this study acknowledges methodological limitations in measuring program effectiveness through GPA comparisons alone. Although the significant improvement in academic performance from first year to fourth year provides some evidence of program impact, GPA increases may be influenced by multiple confounding variables beyond program effectiveness. These include natural academic maturation, grade inflation over time, changes in assessment standards across academic years, selection effects (as lower-performing students may drop out), instructor variability in grading practices, and students’ increased familiarity with university expectations. Moreover, the operationalization of input and product dimensions through GPA measures, although theoretically grounded in CIPP evaluation methodology (Stufflebeam & Coryn, 2014), represents a simplified approach to complex constructs.
The third limitation of the study concerns sample size and representativeness. Although initial invitations were extended to 80 fourth-year students and 20 lecturers, only 38 preservice classroom teachers (47.5% response rate) and five lecturers (25% response rate) participated. The fourth limitation of the study concerns contextual and temporal constraints. The study was conducted within a single institution (Amasya University) in the Turkish educational context, which may limit the transferability of findings to other universities or educational systems. The cross-sectional design captured perspectives at specific time points but lacks comprehensive tracking of individual student progress throughout the 4-year program or follow-up into professional practice to assess long-term program effectiveness.
Another limitation involves the theoretical framework employed for program evaluation. This study utilized the CIPP model to evaluate program effectiveness, which focuses on systematic evaluation across these four dimensions. However, other program evaluation approaches such as goal-free evaluation, responsive evaluation, or utilization-focused evaluation might yield different insights and conclusions about program effectiveness. The choice of evaluation model inherently shapes which aspects of the program are examined and how success is defined, potentially limiting the comprehensiveness of the evaluation findings. The last limitation of the study involves instrumentation and stakeholder perspectives. The study relied on self-report measures that may be subject to social desirability bias and focused exclusively on internal stakeholders (students and lecturers) without including external perspectives from school administrators, practicing teachers, or education ministry representatives who could provide valuable insights into graduate preparedness and field expectations.
Future research should address these methodological limitations by incorporating multiple data sources to evaluate program effectiveness more comprehensively. Recommended approaches include longitudinal tracking of graduates into their teaching careers with measures of classroom effectiveness and job performance alongside portfolio-based assessments of teaching competencies throughout the program. Additionally, comparative analysis with similar programs using standardized teaching competency measures would strengthen the evaluation framework. The inclusion of external stakeholder perspectives from school administrators and mentor teachers would provide valuable insights, and examination of program impact on student learning outcomes in graduates’ classrooms would offer direct evidence of effectiveness. Furthermore, mixed-methods data collection from both student and lecturer populations should be incorporated to capture more nuanced understanding of program experiences and outcomes.
Conclusion
This study reveals that the Amasya University classroom-teacher education program is moderately effective across context, input, process, and product dimensions. Although academic achievement improves significantly over the 4-year program, stakeholders identify various structural and implementation challenges. The findings suggest a need for comprehensive program redesign that addresses the balance between theoretical knowledge and practical application, between cognitive and affective development, and between traditional instruction and contemporary educational needs. Such reform would require substantial institutional commitment to resource allocation, pedagogical innovation, and sustained stakeholder engagement. Future research should expand on these findings by exploring longitudinal outcomes and incorporating diverse evaluation models to ensure robust, adaptable teacher education programs.
Supplemental Material
sj-docx-1-sgo-10.1177_21582440261416973 – Supplemental material for Evaluating the Turkish Classroom-Teaching Undergraduate Program: A CIPP Model Approach
Supplemental material, sj-docx-1-sgo-10.1177_21582440261416973 for Evaluating the Turkish Classroom-Teaching Undergraduate Program: A CIPP Model Approach by Özgü Yalçın Çer in SAGE Open
Footnotes
Ethical Considerations
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Consent to Participate
Informed consent was obtained from all individual participants included in this study.
Author Contributions
Ö.Ç.: Conceptualization, Investigation, Methodology, Data curation, Formal analysis, Writing—original draft, Writing—review & editing, Validation.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.*
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
