Sage Journals: Discover world-class research

Abstract

Responding to calls for more future-oriented teaching and learning, an increasing number of institutions of higher education have recently implemented challenge-based learning (CBL). Through work on open-ended and real-life challenges, CBL aims to stimulate students to take the lead in their own learning, acquire and apply knowledge relevant for responding to the challenge and developing disciplinary and transdisciplinary skills. Prior research on student learning has suggested multiple advantages of such active involvement of students in their learning, which calls for integrating these insights into emerging CBL research. This paper therefore presents a framework for capturing the conditions, process and outcomes of student learning in challenge-based learning, paying specific attention to the distinct levels (microlevel or course context, mesolevel or institutional context and macrolevel or university, societal, local, national or international context) and content (social, physical, and formal) of the context in which CBL takes place. Use of the proposed framework is investigated with data from interviews with eight teachers experimenting with CBL in their courses. The analyses show that the framework is useful for capturing all aspects teachers consider and mention as relevant in implementing CBL courses. The data furthermore suggest interactions between some parts of the framework that need to be examined in future research.

Keywords

CBL challenge-based learning context higher education student learning

Introduction

Today more than ever, for many students the future represents a source of uncertainty and concern (e.g. Morselli, 2013). This leads to a decrease in learners’ motivation and lack of focus in learning goals. The knowledge and skills students acquire today may no longer have actual or perceived use value in the future. As a pedagogy responding to this, challenge-based learning (CBL) has become increasingly popular in higher education. Through work on open-ended and real-life challenges, CBL aims to stimulate students to take the lead in their own learning, acquire and apply knowledge relevant for responding to the challenge presented in the learning task, and develop disciplinary and transdisciplinary skills.

Prior research on student learning in other settings has revealed numerous insights into how different educational concepts affect student learning, suggesting, for example, that students’ active involvement in their learning has positive effects (e.g. Hadgraft & Kolmos, 2020), with students’ perceived usefulness of the learning content, sense of responsibility and self-regulation of learning representing some of the relevant factors. While these are often used to argue for the advantages of CBL, models of student learning have not yet been related to student learning in CBL and we know little about the interplay between CBL course characteristics and student learning patterns and outcomes in CBL.

This paper therefore presents a first attempt to integrate these two lines of research. We will take a close look at CBL as an educational concept and at prior ways in which student learning has been described. Based on this, we will develop a framework for capturing the conditions, context, process and outcomes of student learning in challenge-based learning. The proposed framework will then be tested with data collected from CBL experiments to determine whether it is useful for capturing the proposed aspects and dimensions of student learning in CBL.

Background for this research

Numerous educators, teachers and researchers have argued that today’s students need to be prepared for an increasingly volatile, uncertain, complex and ambiguous world. Many of today’s grand challenges, that is, ‘issues that critically need to be addressed by society to improve for humankind during the coming century’ (Kohn Rådberg et al., 2020, p. 22), are overly complex and ill-defined. How to educate learners and workers who will be able to tackle such complex issues has been the subject of academics’ and practitioners’ discussions for roughly 30 years now. One focus has been the design of inductive teaching and learning settings, allowing for flexible instructional support and a newly balanced ratio of self-discovery to direct instruction (De Corte, 1995). This should lead to more integrated study of different knowledge domains that makes the acquired knowledge more accessible for students (Vermunt & Verloop, 1999). Such modern teaching and learning environments should employ ‘representative authentic, real-life contexts that have personal meaning for the learners, and offer opportunities for distributed and co-operative learning through social interaction’ (Dochy et al., 2003, p. 534), and thereby respond to conceptions of student learning not as an externally directed passive consumption of knowledge, but as an active, constructive and self-directed process (Bednar et al., 1991).

Many educational institutions have turned to different experiential learning initiatives. Experiential learning represents an inductive, integrated, and holistic approach to learning that combines experience, cognition, and behaviour (Akella, 2010). These initiatives aim to offer students engaging open-ended learning experiences, where they can use what they have learned in authentic settings, facing real problems that require them to identify information, debate with other learners, and try out different solutions (Moore, 2013). Consequently, students will apply a deep approach to learning, discover meaning and acquire competence by critically examining and relating new information to their existing knowledge and experience (Tang & Chow, 2020). Thus, they will be more motivated to learn because they clearly perceive a need to know, as when analysing data or a scenario to solve a complex real-world problem (Prince & Felder, 2006). One educational approach fostering experiential learning is challenge-based learning.

Challenge-based learning

Challenge-based learning (CBL) shares many characteristics with related inductive teaching approaches such as ‘inquiry learning, problem-based learning, project-based learning, case-based teaching, discovery learning, and just-in-time teaching’ (Prince & Felder, 2006, p. 123). For example, these precursors present students with a design or problem situation, and learning takes place through working out a suitable solution (Kohn Rådberg et al., 2020). CBL, however, raises the level of difficulty of the problem the students are working on by the ‘absence of predefined study, content or challenge’ (Gallagher & Savage, 2020, p. 3). Students’ working goal in CBL is ‘to learn to define and address the problem and to learn what it takes to work towards a solution, rather than to solve the problem itself’ (van den Beemt et al., 2023, p. 24). This not only requires dialogues with core stakeholders, but also combines addressing societal goals such as sustainability with the need for an entrepreneurial mindset and working method (Kohn Rådberg et al., 2020; Malmqvist et al., 2015).

The various CBL implementations thus far have yielded several attempts at definitions, all emphasizing distinct characteristics of the concept. Furthermore, CBL practice has yielded various approaches all labelled with the same term, which has produced ‘a confusing array of what CBL is and how to implement it’ (Gallagher & Savage, 2020, p. 2). For CBL in higher education, van den Beemt et al. (2020) stated:

CBL is an interdisciplinary experience where learning takes place through identification, analysis, and collaborative design of a sustainable and responsive solution to a sociotechnical problem of which both the problem and outcomes are open. CBL at least involves (1) open ended problems from real world practice that require working in interdisciplinary teams, (2) entrepreneurial acting and design thinking, (3) combining disciplines, and (4) linking curricular and extracurricular activities. CBL both deepens disciplinary knowledge and stimulates 21st century skills such as self-awareness, self-leadership, teamwork, and an entrepreneurial mindset. (p. 62)

This definition shows that CBL is well-embedded in a landscape of current higher education developments that all aim to help students to cross disciplinary boundaries and develop competencies for solving complex real-world problems. Learning in CBL can be described as knowledge creation, which is more powerful than knowledge acquisition or participation (Paavola & Hakkarainen 2005), as students consciously advance towards new knowledge and are epistemic agents who, together with other students and teachers, co-create shareable knowledge objects or artefacts (Damşa et al., 2010). Thus, strong conceptual links exist between CBL and other approaches to creative and interdisciplinary learning that also have one or more of the four CBL characteristics in the above definition.

CBL has been implemented in numerous educational contexts but has become specifically popular in higher education (for a review, see Gallagher & Savage, 2020) and even more so in higher engineering education (for a review, see Doulougeri et al., 2021). Here, the CBL context allows students to be confronted with the full technical difficulty of the challenge and its combination of social and technological problems (Malmqvist et al., 2015), as CBL embraces authentic, active learning, offering choice in problem-solving and learning practices and enabling training in multidisciplinary teamwork and decision-making, as well as harnessing many students’ desire for a sense of meaning in their education (Graham, 2017).

Doulougeri et al. (2021) conducted a review of publications on CBL curriculum implementation in higher engineering education to understand how CBL is currently being implemented in that setting and the challenges and lessons learned. For their analysis of the 48 identified publications, the authors used an adaptation of van den Akker’s (2003) spider-web framework to find out why students are learning, by looking at the aims and objectives, content, learning activities, teachers’ role, materials and resources, grouping, location, timing and assessment in CBL. This approach revealed a vast variety of CBL implementations and objectives, which ‘highlights the flexibility of CBL to be adapted and shaped to the different contexts, needs, and learning objectives of Engineering education’ (Doulougeri et al., 2021).

To capture and describe this variety of possible CBL implementations, van den Beemt et al. (2023) developed a tool called the CBL Compass. By examining CBL course designs on the basis of their vision (i.e. the extent to which CBL experiments employ real-life open-ended challenges, refer to global themes and involve stakeholders), teaching and learning (i.e. educating T-shaped engineers, self-directed learning, assessment, teaching, collaborative learning, interdisciplinarity and learning technology), as well as support (i.e. facilities and teacher support), the compass produces a graphical overview of the selected CBL characteristics. This allows easy comparison of different CBL courses and identification of between-course variance (which was found to be greatest on the dimensions ‘self-directed learning’, ‘teaching’ and ‘interdisciplinarity’).

The ultimate reason for researching characteristics of different CBL experiments is to eventually understand whether and how CBL influences student learning. As already mentioned above, students’ active involvement in their learning has been suggested to have positive effects on student learning (e.g. Hadgraft & Kolmos, 2020). Its interplay with CBL course characteristics, however, is far from clear. Therefore, our aim is to put student learning at the centre of our exploration of CBL implementation in higher (engineering) education, and take a first step by exploring:

(1) How to apply models of student learning to challenged-based learning and

(2) Whether and how these aspects are addressed when implementing CBL experiments in higher education courses.

A framework for capturing student learning in CBL

Prior models of student learning

Prior research on student learning has seen multiple approaches to modelling learning processes and characteristics within and outside the classroom that affect students’ learning outcomes. In the following, we present some central developments that constitute the basis for our framework for student learning in CBL.

One of the most prominent and influential models of student learning is Biggs’s 3P model (Biggs & Tang, 2011), in which he interrelated presage (i.e. characteristics that existed prior to learning engagement such as those connected to students’ personal environment and attitudes, as well as the environment), process (i.e. students’ learning activities) and product variables (i.e. students’ learning outcomes). This model was evaluated and extended in prior research (e.g. Hamilton & Tee, 2009) but was repeatedly criticized for not including learners’ metacognitive processes and self-regulated learning.

Consequently, Vermunt and colleagues (e.g. Vermunt & Vermetten, 2004) put forward a more unified framework to bring these research traditions together and developed a learning patterns perspective on student learning. Their framework (Vermunt & Donche, 2017) is comprised of students’ cognitive processing strategies (i.e., cognitive learning activities employed to process subject matter and leading to knowledge and skill), metacognitive regulation strategies (i.e. activities to plan, monitor, steer and evaluate cognitive learning processes), (metacognitive) conceptions of learning (i.e. views and beliefs about learning and teaching) and learning motivations or orientations (i.e. aims, goals, motives and worries). The interplay of these four types of strategy results in four qualitatively different learning patterns, namely, reproduction-directed learning, meaning-directed learning, application-directed learning and undirected learning (e.g. Lonka et al., 2004; Vermunt & Donche, 2017). These have been shown in prior research to be interrelated with students’ personal factors (such as age, personal background or educational experience), the contextual factors in the learning environment and learning outcomes (for a review, see Vermunt & Donche, 2017).

In applying the learning patterns model of student learning to CBL, it is obvious that contextual factors in student learning patterns were included in most of the prior learning research. However, CBL differs from other educational practices in many ways, and touches upon more contextual factors than traditional teaching practices in higher education. Many CBL experiments seem to share the assumption that students would use more preferable meaning- and application-directed learning patterns, if only the contextual characteristics in the CBL learning experiences were implemented in the ‘right’ way. Apart from the variety of CBL implementations described so far (for a review, see Doulougeri et al., 2021), this may also be one reason why most CBL research has stayed on a case-based, descriptive level (Leijon et al., 2022).

Our framework for student learning in CBL

Prior research has shown that several factors present in CBL are not covered by existing models of student learning, so we suggest a framework for student learning in CBL that accommodates these factors of interest when teaching or conducting research in CBL settings (see Figure 1).

Figure 1.

Framework for student learning patterns in CBL.

In the following, we describe our framework as depicted in Figure 1, starting at its core, students’ learning patterns in CBL, before turning to different layers of contextual factors that affect these learning patterns, and finally focussing on the outcomes of student learning in CBL. It should be noted that the framework as depicted in Figure 1 applies to any learning in context, but by focussing on the content of the contexts of student learning in CBL, we find several aspects that distinguish CBL from other educational concepts.

Naturally, student learning lies at the core of our model. Drawing on Vermunt and Donche’s (2017) learning patterns model, we suggest that students’ learning patterns in CBL derive of an interplay of students’ conceptions of learning, learning motivations and orientations, metacognitive regulation strategies and cognitive processing strategies. These learning patterns are affected by students’ personal factors and contextual factors. Personal factors comprise student characteristics such as personal background, prior knowledge, educational experience, age, gender and so on and can include aspects such as experience with CBL and with interdisciplinary, collaborative work, as well as students’ readiness to take responsibility for their learning or deal with uncertainty.

Much attention should be paid to the contextual factors of student learning in CBL. Prior research in CBL implementation and course design (e.g. van den Beemt et al., 2023) suggested a broad number of contextual factors referring to the vision, teaching and learning and support related to CBL implementation, which again influence students’ learning and their learning experience. In this framework, we suggest structuring these contextual factors based on Wosnitza and Beltman’s (2012) model, which describes higher education learning contexts from different perspectives (subjective and objective), comprising different types of content (physical, social and formal, based on Lewin, 1936) and conceptualized as multilevel (micro, meso-, exo- and macrolevel, based on Bronfenbrenner, 1979).

Accordingly, in our framework, we describe the microlevel CBL context as the course-level context. The mesolevel represents the institutional context (e.g. departmental vision of CBL, curriculum level and stakeholders), and the macrolevel refers to the wider (university, societal, local, national and international) context. Each of these levels can be described with regard to its physical, social and formal content. So, for example, on the microlevel, the context may include aspects of the physical environment (e.g. learning resources and available work spaces), the formal content (referring to many aspects of CBL implementation, such as the nature of the challenge as far as being real-life and open-ended, teaching/coaching strategies, interdisciplinarity, assessment/feedback/reflection elements) and the social content (i.e. frequency and nature of meetings with tutor or collaboration with peers). The social content of the microcontext is crucial, as student collaboration (with other students or teachers as co-learners) plays a significant role in CBL. Prior research on group work has suggested multiple ways in which collaboration among learners, as a complex and socially demanding learning experience, affects learning motivation and emotions and may thus affect learning patterns (e.g. Kimmel & Volet, 2010; Linnenbrink et al., 2011; Schultz & Wosnitza, 2018). Collaboration may lead to improved understanding of learning content and higher-order problem solving, as well as improved group, time and task management (Summers & Volet, 2010; Wilkinson & Fung, 2002), and can thus be perceived to represent one of the main contextual influences on student learning in CBL.

To capture student learning outcomes in our framework, we draw on a conceptual framework of learning gains in higher education by Vermunt et al. (2018), who understand learning gains as ‘students’ change in knowledge, skills, attitudes, and values that may occur during higher education across disciplines’ (Vermunt et al., 2018, p. 272). Learning gain is conceptualized as consisting of cognitive, metacognitive, affective and socio-communicative components and three dimensions (view of knowledge and learning, research attitude and moral reasoning). In particular, these three dimensions reveal how strongly student learning gains may influence future learning patterns via students’ personal factors. In the CBL literature, we find strong references to the socio-communicative component of student learning, with emphasis being placed on how CBL can help students to move from novice to expert and develop a professional identity, innovation and creativity, as well as entrepreneurial thinking and skills (e.g. Colombelli et al., 2022). Students’ learning gains with regard to collaborative skills can also be subsumed under this component.

Furthermore, a framework for learning in CBL also must accommodate those outcomes of the completed projects that do not concern student learning, but focus either on product development or a solution with a societal impact, as these may again spark motivation and a sense of the usefulness of CBL-based learning processes (e.g. Bohm et al., 2020; Kohn Rådberg et al., 2020).

Another aspect that needs to be included in our framework is that of the teacher as a coach and co-learner in CBL. Vermunt et al. (2017) have put forward models of student and teacher learning that are also able to capture more contemporary teaching and learning approaches, in which a teacher is viewed as a learning process expert who fosters active, self-regulated and collaborative learning in students. The interplay between teacher and student learning is highly dynamic:

When teachers use these changed knowledge, skills, beliefs or identity to change their teaching practices, they become part of the student learning environment (. . .). The changed student learning environment may then initiate student learning processes, leading to student learning outcomes, which can be conceptualized as changed knowledge, beliefs, identity, motives, attitudes, skills, and so on. (Vermunt et al., 2017, p. 151)

Accordingly, although our model focusses on individual factors or one student in the CBL process, it is crucial to understand that the indicated processes exist for each of the student’s co-learners (be they other students or teachers), and thus represent the social microcontext again, which cannot be graphically depicted.

Empirical exploration of the framework

The development of the above framework was guided by prior research on student learning as well as conceptual and review work on CBL implementation. This is a first attempt to bring together these hitherto separate lines of research, with the aim of proposing a framework that allows for analysing instructional designs intended to support student learning through CBL. Therefore, it is crucial to explore the usefulness and exhaustiveness of this framework when it comes to real-life CBL being put into practice in higher education courses.

In this preliminary study, the aim is to explore whether and how the aspects and dimensions built into our framework are being addressed when CBL is implemented in higher education courses. Furthermore, we want to investigate whether there are aspects that call for further adjustment of the framework.

In newer teaching approaches such as CBL, teachers are viewed as facilitators of learning (Vermunt et al., 2017, p. 143). Furthermore, teachers are also knowledgeable about the macrolevel contexts in which their CBL experiments are being embedded. They may therefore be able to share broader insights than the specific students participating in the courses can, whose perspectives may naturally be limited. In this first step, we thus decided to draw on interviews with teachers regarding their plans and experiences, aims and hurdles regarding their CBL implementation, to determine whether our framework is adequate for capturing student learning in CBL.

Methodology

To identify courses at the front line of CBL development and implementation in higher (engineering) education, we drew on a corpus of innovative CBL projects at Eindhoven University of Technology (TU/e). TU/e aims at enabling educational innovation and improvement by providing resources to ‘academic staff with an innovative mindset’ to ‘make teaching more efficient and enjoyable for both students and teachers’. The call for proposals to do this had recently been extended with an explicit line for funding of CBL innovations. We started out by perusing all accepted submissions from 2020 and 2021 to identify projects that aimed at developing, improving and/or experimenting with one or more aspects of CBL. We recruited eight teachers associated with the identified projects, who volunteered to share their in-depth insights into the educational innovations and how various aspects of implementation were expected to facilitate effective student learning patterns and gains. Informed consent of the teachers was obtained before the interview.

At the time of the interview, all the teachers had already implemented the CBL project for which they had submitted a proposal, or elements thereof, in their courses, had taught the full course at least once and were already considering changes for upcoming courses. This was an appropriate time for interviews on their experiences with CBL implementation and teaching; the interview transcripts were then analysed for dominant themes from our framework.

Interviewees came from the departments of Mechanical Engineering, Mathematics and Computer Science, Electrical Engineering, Industrial Engineering and Innovation Science, Applied Physics and the School of Education, with the majority (n = 6) being assistant professors (one full professor, one doctoral candidate), and reported varying degrees of experience with CBL (from this project being their first CBL experiment to working in CBL settings during their masters and PhD degree programmes).

The semi-structured interviews included open-ended questions on the interviewees’ CBL experiment, as well as asking them to describe their department, students, teachers, supporting staff and other stakeholders involved. Then, the teachers were asked what the main goals and drawbacks/challenges were in implementing the project and whether (and if so, which) aspects of the project developed differently from what they were planning or had anticipated prior to implementation. The questions were phrased this openly so as not to push teachers to focus on one or a few specific parts of the proposed framework. Rather, we were interested to see which parts of the framework teachers would address in their reports about their CBL experiments, in order to evaluate the usefulness of the framework.

Furthermore, the interviewees were questioned on how satisfied they were with the outcomes of the CBL experiment and how they thought the project was perceived by other stakeholders such as students, other teachers, support staff and/or external partners. At the end of the interview, all the interviewees were asked to reflect on their personal key take-aways from this CBL experiment.

This interview schedule was designed to trigger teachers’ reports on their hopes and worries for their students’ learning regarding their implementation of CBL. Due to this openness to further aspects interviewees wanted to mention, interviews were very flexible regarding timing and lasted between 30 and 60 minutes. Interviews took place in English, were conducted remotely, recorded and transcribed via MS Teams and then analysed. The coding scheme was derived deductively from the proposed framework; thus, the analysis sought to identify statements that could be mapped to the framework. ID numbers were used for preserving the anonymity of the teachers. Anonymized data may be shared with other researchers upon reasonable request.

Results

We present the results of this analysis starting with the innermost circle in the framework (see Figure 1), that is, the microcontext in which student learning takes place, followed by students’ learning patterns and outcomes, before turning to teachers’ references to the meso and macrocontexts.

Microcontext

In the interviews, we found that the teachers predominantly talked about how they could design the formal content of the microcontext for students, for example, the nature of the challenge and strategies for teaching/coaching and assessment in the classroom, that would allow the most advantageous learning patterns and learning outcomes. This mostly referred to the freedom the students have in CBL classes, which on the one hand refers to allowing students’ exploration and self-regulation, by choosing truly open-ended challenges and giving students the freedom to explore the problem in the way they like, but also involves setting guidelines for students’ work and not falling back into teaching patterns that would restrict students’ options for self-regulation:

The task of a teacher is mainly to be sure that they don't go off the rail when they start doing their project. (. . .) and that situation is difficult too, because I see them struggling, so I might jump in with what I think, but that should not be, and I should avoid doing it. (Teacher #1)

Naturally, the open-endedness of what students learn in CBL was described by several teachers as sometimes conflicting with the assessment of student outcomes – obviously depending on what learning outcomes the assessment is focussing on:

It depends a bit, I think. You can say ‘OK, I want to them to learn these concepts’ and then you need to measure exactly these concepts or ‘I want them to learn to have the general idea of how things work and I want them to be able to pick up a similar challenge as the one they encountered in the course and be able then to look up the knowledge that they need by themselves’. So, if you only focus on the concepts of the course, then you can have a test at the end. But if your goal is to give them a general way of dealing with challenges, apart from a certain domain that is very difficult to measure, I think. (Teacher #1)

The difficulty in aligning learning goals with assessment in general described in this quote and with assessment requirements from their department were referred to in several interviews. The teachers were diverse in how they assessed student learning in their courses and what types of assessment they considered adequate (e.g. reports, presentations and videos).

Apart from the clarity of learning objectives, scaffolding by coaches, which is part of the social content of the microcontext, was described as crucial for allowing students to develop learning motivation and self-regulation. The teachers referred to promoting students’ responsibility in their classes, which diminishes the presence of hierarchy over time:

I support them, and they are allowed to make mistakes, so to speak, and to develop a way of thinking based on their responsibility. (Teacher #2)

It’s really their problem and you're just helping them. But you also don't know how to solve it completely. (Teacher #8)

A few teachers talked about how team characteristics and dynamics affect all team members’ learning. One teacher strongly advised smaller student groups (‘Because the benefits in coordination and accountability outweigh the increased workload,’ Teacher #4) while another one shared his insights on the interdisciplinarity of the group:

Of course, they have different prior knowledge so that's important, but I think it's also bit more about knowing your strengths as a student in a team and knowing your weaknesses or what you want to learn - more than you really need different departments. It's a bit less important and more about teamwork and learning how to deal with that in these kinds of projects. (Teacher #8)

We found that the social content of the microcontext is interrelated with the aspects of students’ learning patterns and assessment, as the teachers expressed their concern about ‘separating the individual component from the team component’ (Teacher #4).

Further aspects that were mentioned were aspects of how the physical content of the microcontext can influence students’ learning. One teacher strongly emphasized the need for learning technology to support the student teams’ learning process by making it visible (including the failures):

The thing is you can have a student group that is going very well and then after 4 weeks they get stuck. You want to spot that as soon as possible. (. . .) So, they are obligated to report weekly on how the student group is going and so on (. . .). If there is something that is not working well. I mean that is nothing bad - of course it happens all the time - but we want to be able to find it as soon as possible. (Teacher #3)

Two teachers also reported that they were trying to get rooms that were more adequate for hosting CBL courses than the standard lecture rooms they have been using up to now.

Personal factors

Personal student factors were only rarely mentioned in the interviews, but teachers did refer to students’ prior (subject) knowledge that would affect their learning process and learning gains:

It would not make sense to make that challenge-based because they don't have any prior knowledge. And so yes, it would be interesting for them to do a project, but didactically speaking it will not bring them to a higher level. (Teacher #5)

Student learning patterns

The social content of the microcontext, specifically the change in teacher roles as described above, positively affected students’ learning patterns, that is, their conception of learning, learning motivation/orientation, regulatory strategies and processing strategies. In the interviews, the teachers described how this change motivates students, ‘triggers their passion and as a consequence their intrinsic motivation’ (Teacher #1).

CBL is our possibility to give students the freedom to define their own project and then come to make a first step into the solution and so they have their own challenge and then they should articulate another guiding question, then guiding activities, guiding resources and know what they need to learn and study. (Teacher #3)

Although the teachers described CBL as triggering the most advantageous learning patterns for students, some also raised concerns regarding the interplay with other educational concepts at the university:

Students are like ‘OK. I really like this idea. I think I could work in it,’ but it's really difficult if they are taking three or four courses in one quartile and one of those courses is asking a lot of self-regulation and the other courses are as they are used to do, then [CBL is] always the odd one out. No matter how well it's designed, it will be evaluated badly. Students will not know how to study for such a course because the other ones asked them to do what they always do. (Teacher #7)

Student learning gains

With regard to what students take away from CBL, some teachers strongly focussed on the cognitive component of learning gains and linked it back to the formal content of the microcontext by emphasizing that students could not take away new knowledge from challenge-based courses without specific prior knowledge to build on:

I was trying to argue that I cannot expect students to run this analysis if I don't tell them what it is, why it matters and how we can use it. Even a very small introduction. Because if they start looking for it on their own, it can be super messy. (Teacher #4)

Other teachers did not share this view, and emphasized the affective component of students’ learning:

We need some knowledge, but the knowledge is not that interesting because, in general, it’s in the textbook or on the Internet. So, what you try to do is to really target the skills of the students that come with reading, studying, understanding, analysing, having specific perspectives on problems, possible solutions or scientific fields and that also comes with motivation. Motivation has to do with enthusiasm and vision and explaining the importance of the field. (Teacher #5)

Furthermore, teachers could identify the metacognitive and socio-communicative components of students’ learning outcomes:

Reality later on is many companies don’t want them to be the specialist on this thing and this thing alone. Being able to work in a team and being able to look wider and then narrower is what gets the job done. (Teacher #6)

Knowing how to deal with problems and knowing that they can really solve everything if they just know how to use certain strategies or whatever. I think that’s really important teaching. (Teacher #8)

This links to one teacher’s emphasis that traditional universities ‘teach the students in one direction, acquire knowledge and applying it. And in my experience, I have the feeling that creativity is missing’ (Teacher #2) as well as interviewees stressing how CBL can help students develop a sense of interdisciplinary work: ‘you cannot survive as an engineer if you do not understand the other domains’ (Teacher #2). The research dimension was present in the comment of one teacher, who stated the need ‘that the students wake up as a researcher during such a project. And use that as something that they can also feed into how they tackle other courses’ (Teacher #6).

Mesocontext and macrocontext

In the interviews, we observed that some teachers mentioned the formal content of mesocontext, that is, the departmental or institution environment, quite frequently. They talked about how the university’s policy and vision and departmental requirements influence the set-up of the learning environment, specifically regarding the assessment of student learning gains in interdisciplinary CBL.

But it also has to fit in the assessment requirements, so that's also different for different departments. (Teacher #8)

Social and physical content of the mesocontext was evident when teachers referred to colleagues they were collaborating with when developing courses or to facilities, such as teacher support, that were provided.

With respect to the macrocontext, we observed that some teachers described collaborations with stakeholders, part of the social content of the macrocontext of student learning, as also limiting students’ freedom in developing guiding questions, as well as problems with stakeholders disappearing:

[we had the] situation that the contact person was very willing but left the company very close to the beginning of the project and there was nobody to substitute for him in in the company. (Teacher #6)

The physical and formal content of the macrocontext were not addressed in the interviews.

Implications of findings for the framework for student learning in CBL

The analysis of the interviews with teachers revealed that the proposed framework is useful for describing student learning in CBL. We could identify most of the suggested elements of our framework, although not to the same degree across different interviews. No references could be identified to the formal content of the macrocontext (i.e. referring to global themes that guide the development of challenges). Regarding student learning patterns, teachers talked about students’ learning motivation and regulatory strategies, but did not mention students’ conception of learning. While all the components of student learning gains were mentioned in the interviews, teachers shared different foci on student learning gains due to the varying intended learning outcomes of the courses. Student creativity and creative thinking was one aspect mentioned in the interviews that should be added to the outcomes of student learning in CBL. Both the moral dimension and students’ view of knowledge and learning were not focussed on. The societal impact of student learning and students’ personal factors were also not mentioned in the interviews.

Regarding the aspects that were mentioned, we could identify that the teachers often talked about aspects of the microcontext of student learning, that is, the nature of the challenges, assessment, collaboration, teaching/coaching and learning technology. Zooming in on the microcontext and the teachers’ descriptions of student learning patterns and learning outcomes, several links and conflicts became obvious that will have to be addressed by future research on student learning in CBL, as we discuss next.

Change in teacher and student roles and student motivation

Having teachers step away from instruction in CBL and leave room for students to identify learning goals and guiding questions affects student learning patterns, specifically their motivation. In the interviews, teachers vividly described how some students’ make the most of the openness of the CBL approach to learning while others ‘go off the rail’ (Teachers #1, #3) due to the lack of directions, but how to make students take responsibility for their own (and their team members’) learning in CBL was not fully clear. It should be noted that most CBL experiments still attract students who are already motivated to take the lead in their learning process and seek out such opportunities. Future research will need to focus on this interplay between students’ personal factors and their constructive participation in CBL courses, especially if students must participate in a CBL course as part of their curriculum. Scaffolding may have to start even before the course, preparing students for this kind of work and interaction.

Furthermore, it became apparent in the interviews that teachers’ understanding of their own role as coaches/tutors was quite diverse. These varying degrees of being able to let go and granting autonomy to students naturally shape the microcontext of student learning in CBL. As already described in section 4, teachers apply ‘changed knowledge, skills, beliefs, or identity to change their teaching practices, (. . .) become part of the student learning environment (. . .). The changed student learning environment may then initiate student learning processes’ (Vermunt et al., 2017, p. 151). Accordingly, future research on student learning in CBL should always take teachers into account as co-learners and therefore crucial factors in the social content of the microcontext.

Teamwork, interdisciplinarity and student learning

The social content of the microcontext, specifically students’ collaboration in their teams, showed strong links with student learning patterns and learning outcomes. One of the main learning goals in CBL courses is for students to collaborate with peers with various levels of prior knowledge and coming from different disciplines. This can pose considerable difficulties for some teachers. The teachers in this study were remarkably divergent regarding whether they also attached cognitive learning goals to the challenges or not. When they did, some even described a large student team as a hindrance to the acquisition of content knowledge, and called for a preparatory ‘class zero’ at the beginning of the course that brings students to the same level of content knowledge before the start of the course.

Assessment of student learning

The formal content of the mesocontext (i.e. curricula and institutional visions) was described as often conflicting with the formal content of the microcontext (i.e. design of the challenge, openness, interdisciplinarity), specifically with regard to assessment. This is not surprising, given that assessment in CBL has been highlighted as an important, yet underdeveloped topic (van den Beemt et al., 2023), insofar as the open-endedness of CBL raises important challenges for the assessment of students (Membrillo-Hernández et al., 2021): Prior research has highlighted dilemmas in assessment of the end-product (the solution) vs. the process in CBL, the assessment of individual students vs. student teams and the use of formative vs. summative forms of assessment (Doulougeri et al., 2022; van den Beemt et al., 2023). Future research will have to explore the adequacy of different types of assessment such as self- and formative assessment to promote student learning in this situation.

Conclusions, limitations and future research

The aim of this paper was to bring together two hitherto separate lines of research and put forward a framework based on the existing literature, in order to capture the various aspects of student learning in CBL that are relevant in CBL implementation and that could guide further development of CBL. For future research, this framework can help indicate necessary variables to be studied and their relationships, while for practice, it can serve as a tool for teachers to plan, orchestrate and analyse their CBL implementations. We illustrated the use of this framework by analysing interviews with teachers who had implemented CBL in their courses. We found the suggested framework to be adequate and useful for describing student learning in CBL in general, with the data pointing to unexplored elements that will need to be addressed in future research.

More generally, the new ways of teaching in CBL require deeper investigations to define the characteristics that separate them from other educational approaches, as well as to confirm that these new practices really do work and do improve all or some aspects of students’ learning. Some research has emerged already focussing on various aspects of CBL, and some reviews have been done to identify and structure the existing characteristics of CBL, as well as which aspects of CBL have been or are being covered by the research so far. Still, ‘there is not enough evidence about how CBL processes influence student learning. (. . .) Researchers and practitioners should develop CBL experiences and articulate the principles underpinning the effects of those interventions on students' learning processes and gains’ (Doulougeri et al., in press). Apart from these structural aspects, we believe that students’ personal factors, referring to prior knowledge and students’ ability to engage in such open-ended learning settings when starting CBL courses, need closer attention. Future studies will need to further address the opinion that CBL is a pedagogy that is only appropriate for specific types of students who are able to handle the openness of the learning process and adopt advantageous learning patterns.

One limitation of this study is that it drew only on interviews with teachers who were experimenting with CBL and volunteered to report on that. Naturally, as these were teachers at the front line of educational innovation, they can be assumed to be more attentive to the many factors that may foster or hinder student learning. Nevertheless, teacher data were useful, as teachers have the most comprehensive overview of the several levels of context the CBL courses are embedded in.

However, ‘research has shown that it is not so much the learning environment in itself that influences the way students go about learning, but rather it is the learning environment as perceived and experienced by students’ (Vermunt et al., 2017, p. 150), which is why in our future research we will address students’ perspectives on the implementation of CBL courses.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research,authorship,and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research,authorship,and/or publication of this article: This study was supported by the grant “Fostering Challenge-Based Learning through TU/e innovation Space,” awarded through the NRO Comenius Leadership Fellow program.

ORCID iD

Kerstin Helker

Author biographies

Kerstin Helker is a Postdoctoral Educational Researcher at Eindhoven University of Technology,the Netherlands. After finishing her PhD on responsibility ascriptions in the school context and the effects on motivation and achievement,Kerstin worked as Interim Professor at the University of Bonn and Senior Lecturer at RWTH Aachen University,Germany. Kerstin aims to improve educational practices through empirical research and to understand how real-life and authentic challenges can be used as effective and motivating learning opportunities. In the Comenius research project at TU/e innovation Space,Kerstin is currently focusing on students’ learning gains in challenge-based learning settings.

Miguel Bruns is Associate Professor in the Interactive Matters Cluster of the Department of Industrial Design. He has an MSc and PhD in Industrial Design Engineering from TU Delft and was visiting researcher at the Center for Design Research,Stanford University,and the Design Research and Ubiquitous Computing Groups,Aarhus University. He was program director (Bachelor and Master) of Industrial Design,an innovative educational program founded in 2001 based on self-directed and competence-centered learning,largely composed of design challenges with industry. He led the redesign of its curriculum and cofounded TU/e innovation Space.

Isabelle Reymen is Full Professor Design of Innovation Ecosystems at Eindhoven University of Technology (TU/e). Furthermore,she is Scientific Director of TU/e innovation Space,center of expertise for Challenge-Based Learning and student entrepreneurship at TU/e,a learning hub for education innovation and an open community where students,researchers,industry,and societal organizations can exchange knowledge and develop responsible solutions to real-world challenges. Her research focusses on decision-making under uncertainty,design,and governance of entrepreneurial innovation ecosystems,the university as a strategic partner in innovation ecosystems,and education innovation in higher education.

Jan D Vermunt is an Emeritus Full Professor of Learning Sciences and Educational Innovation at Eindhoven University of Technology,Eindhoven School of Education. His expertise area are the learning sciences,with a focus on teaching and student learning in higher education,and teachers’ learning and professional development. Current research interests include effective ways to support personal learning pathways,using student learning data to improve teaching and learning,developing innovative teachers,and promoting deep and self-regulated learning in students and teachers and new pedagogies for challenge-based learning.

References

Akella

(2010) Learning together: Kolb’s experiential theory and its application. Journal of Management and Organisation, 16, 100–112.

Bednar

A. K.

Cunningham

Duffy

T. M.

Perry

J. D.

(1991). Theory into practice: How do we link? In Anglin

G. J.

(Ed.), Instructional Technology: Past, Present and Future (pp. 88–101). Libraries Unlimited.

Biggs

Tang

(2011). Teaching for quality learning at University. McGraw Hill.

Bohm

Klaassen

den Brok

van Bueren

(2020). Choosing challenges in challenge-based courses. In van der Veen

J. T.

van Hattum-Janssen

Jarvinen

de Laat

ten Dam

(Eds.), Engaging engineering education: SEFI 48th conference proceedings, 20-24 September 2020 (pp. 98–109). European Society for Engineering Education.

Bronfenbrenner

(1979). The ecology of human development: Experiments by nature and design. Harvard University Press.

Colombelli

Loccisano

Panelli

Pennisi

O. A. M.

Serraino

(2022). Entrepreneurship education: The effects of challenge-based learning on the entrepreneurial mindset of university students. Administrative Sciences, 12(1), 10.

Damşa

C. I.

Kirschner

P. A.

Andriessen

J. E. B.

Erkens

Sins

P. H. M.

(2010). Shared epistemic agency: An empirical study of an emergent construct. Journal of the Learning Sciences, 19(2), 143–186.

De Corte

. (1995). Fostering cognitive growth: A perspective from research on mathematics learning and instruction. Educational Psychologist, 30, 37–46.

Dochy

Segers

van den Bossche

Gijbels

(2003). Effects of problem-based learning: A meta-analysis. Learning and Instruction, 13, 533–568.

10.

Doulougeri

van den Beemt

A. A. J.

Vermunt

J. D.

Bots

Bombaerts

(2022). Challenge-based learning in engineering education: Toward mapping the landscape and guiding educational practice. In Vilalta-Perdomo

Membrillo-Hernández

Michel-Villarreal

Lakshmi

Martínez-Acosta

(Eds.), The Emerald handbook of challenge based learning (pp. 35–68). Emerald Group Publishing Ltd. https://doi.org/10.1108/9781801174909

11.

Doulougeri

Vermunt

J. D.

Bombaerts

Bots

. (in press). Challenge-based learning implementation in engineering education: A systematic literature review. Journal of Engineering Education.

12.

Doulougeri

Vermunt

J. D.

Bots

Bombaerts

(2021). Defining key components of challenge-based learning in engineering education: A review study [Paper presentation]. 19th Biennial EARLI conference, online/Gothenburg, Sweden.

13.

Gallagher

S. E.

Savage

(2020). Challenge-based learning in higher education: An exploratory literature review. Teaching in Higher Education, 28, 1135–1157.

14.

Graham

(2017). The global state-of-the-art in engineering education. Outcomes of Phase 1 benchmarking study. https://rhgraham.org/resources/Phase-1-engineering-education-benchmarking-study-2017.pdf, last accessed on February 6, 2024.

15.

Hadgraft

R. G.

Kolmos

(2020). Emerging learning environments in engineering education. Australasian Journal of Engineering Education, 25(1), 3–16.

16.

Hamilton

Tee

(2009). Extending the Biggs 3P teaching and learning model: A structural equation modeling approach. In Proceedings of the Ninth International Conference on Electronic Business, 30 November-4 December 2009, Macau, China (pp. 1037-1042). The Chinese University of Hong Kong.

17.

Kimmel

Volet

(2010). Significance of context in university students’ (meta)cognitions related to group work: A multi-layered, multi-dimensional and cultural approach. Learning and Instruction, 20(6), 449–464.

18.

Kohn Rådberg

Lundqvist

Malmqvist

Svensson

O. H

. (2020). From CDIO to challenge-based learning experiences – Expanding student learning as well as societal impact? European Journal of Engineering Education, 45(1), 22–37.

19.

Leijon

Gudmundsson

Staaf

Christersson

(2022). Challenge based learning in higher education: A systematic literature review. Innovations in Education and Teaching International, 59(5), 609–618.

20.

Lewin

(1936). Principles of topological psychology. McGraw-Hill.

21.

Linnenbrink

Rogat

T. K.

Koskey

K. L. K.

(2011). Affect and engagement during small group instruction. Contemporary Educational Psychology, 36(1), 13–24.

22.

Lonka

Olkinuora

Makinen

(2004). Aspects and prospects of measuring studying and learning in higher education. Educational Psychology Review, 16, 301–331.

23.

Malmqvist

Kohn Rådberg

Lundqvist

(2015). Comparative analysis of challenge-based learning experiences. In Proceedings of the 11th International CDIO Conference, 8-11 June 2015. Chengdu University of Information Technology.

24.

Membrillo-Hernández

de Jesús Ramírez-Cadena

Ramírez-Medrano

García-Castelán

R. M. G.

García-García

(2021). Implementation of the challenge-based learning approach in Academic Engineering Programs. International Journal on Interactive Design and Manufacturing, 15(2), 287–298. https://doi.org/10.1007/s12008-021-00755-3

25.

Moore

(2013). For interns, experience isn’t always the best teacher. The Chronicle of Higher Education. https://www.chronicle.com/article/for-interns-experience-isnt-always-the-best-teacher/, last accessed on February 6, 2024.

26.

Morselli

(2013). The olive tree effect: Future time perspective when the future is uncertain. Culture and Psychology, 19(3), 305–322. https://doi.org/10.1177/1354067X13489319

27.

Paavola

Hakkarainen

(2005). The knowledge creation metaphor: An emergent epistemological approach to learning. Science & Education, 14(6), 535–557.

28.

Prince

M. J.

Felder

R. M.

(2006). Inductive teaching and learning methods: Definitions, comparisons, and research bases. Journal of Engineering Education, 95, 123-138.

29.

Rådberg

K. K.

Lundqvist

Malmqvist

Svensson

O. H.

(2020). From CDIO to challenge-based learning experiences – expanding student learning as well as societal impact? European Journal of Engineering Education, 45(1), 22–37.

30.

Schultz

Wosnitza

(2018). Emotionen von Studierenden in einem computerbasierten kollaborativen Setting. In Hagenauer

Hascher

(Eds.), Emotionen und Emotionsregulation in Schule und Hochschule (pp. 137–150). Waxmann.

31.

Summers

Volet

(2010). Group work does not necessarily equal collaborative learning: Evidence from observations and self-report. European Journal of Psychology of Education, 25, 473–492.

32.

Tang

A. C.

Chow

M. C.

(2020). To evaluate the effect of challenge-based learning on the approaches to learning of Chinese nursing students: A quasi-experimental study. Nurse Education Today, 85, 104293.

33.

Van den Akker

J. J. H

. (2003). Curriculum perspectives: An introduction. In van den Akker

J. J. H.

Kuiper

W. A. J. M.

Hameyer

(Eds.), Curriculum landscapes and trends (pp. 1–10). Kluwer Academic Publishers.

34.

Van den Beemt

MacLeod

Van der Veen

. (2020). Interdisciplinarity in tomorrow’s engineering education. In van der Veen

J. T.

van Hattum-Janssen

Jarvinen

de Laat

ten Dam

(Eds.), Engaging engineering education: SEFI European Society for Engineering Education 48th conference proceedings, 20-24 September 2020 (pp. 61–74). Twente: University of Twente.

35.

van den Beemt

van de Watering

Bots

. (2023). Conceptualising variety in challenge-based learning in higher education: The CBL-compass. European Journal of Engineering Education, 48(1), 24–41. https://doi.org/10.1080/03043797.2022.2078181

36.

Vermunt

J. D.

Donche

(2017). A learning patterns perspective on student learning in higher education: State of the art and moving forward. Educational Psychology Review, 29(2), 269–299.

37.

Vermunt

J. D.

Illie

Vignoles

(2018). Building the foundations for measuring learning gain in higher education: A conceptual framework and measurement instrument. Higher Education Pedagogies, 3(1), 266–301.

38.

Vermunt

J. D.

Verloop

(1999). Congruence and friction between learning and teaching. Learning and Instruction, 9, 257–280.

39.

Vermunt

J. D.

Vermetten

Y. J.

(2004). Patterns in student learning: Relationships between learning strategies, conceptions of learning, and learning orientations. Educational Psychology Review, 16(4), 359–384.

40.

Vermunt

J. D.

Vrikki

Warwick

Mercer

(2017). Connecting teacher identity formation to patterns in teacher learning. In Clandinin

D. J.

Husu

(Eds.), The SAGE Handbook of research on teacher education (pp. 143–159). Sage.

41.

Wilkinson

I. A. G.

Fung

I. Y. Y.

(2002). Small-group composition and peer effects. International Journal of Educational Research, 37(5), 425–447.

42.

Wosnitza

Beltman

(2012). Learning and motivation in multiple contexts: The development of a heuristic framework. European Journal of Psychology of Education, 26(2), 177–193.

A framework for capturing student learning in challenge-based learning

Abstract

Keywords

Introduction

Background for this research

Challenge-based learning

A framework for capturing student learning in CBL

Prior models of student learning

Our framework for student learning in CBL

Empirical exploration of the framework

Methodology

Results

Microcontext

Personal factors

Student learning patterns

Student learning gains

Mesocontext and macrocontext

Implications of findings for the framework for student learning in CBL

Change in teacher and student roles and student motivation

Teamwork, interdisciplinarity and student learning

Assessment of student learning

Conclusions, limitations and future research

Footnotes

Declaration of conflicting interests

Funding

ORCID iD

Author biographies

References