Moodle analytics are often touted as adequate measure of engagement, however what does it mean to engage – and whose engagement is it anyway?
Imagine looking through the activity logs for a week of your Moodle module. It may show that many students have accessed the lecture video, a higher number have completed the multiple-choice quiz, and a smaller group have contributed to the discussion forum. This will give you a useful picture of participation with content, but how much does it tell you about students’ engagement with the learning?
A student may have opened a reading without getting beyond the first paragraph. Another may have downloaded the lecture slides and used them to organise their thinking alongside the reading; all completed offline and at home. They then return to class with questions and other contributions delivered verbally. Understanding the impact of learning experiences on engagement requires a qualitatively different perspective.
We often talk about making teaching more engaging. We want students to engage with the reading, engage and contribute to discussions and engage with feedback to help their learning. Yet what we mean by engagement is not always clear, particularly when Moodle analytics for engagement do not extend passed View or Complete. For the DABL design team, this is a key question that forms our conversations, because the meaning we give engagement shapes the learning we design.
What does engagement look like?
Moodle analytics feed the perspective that engagement refers to the contribution students make towards their learning: the time, commitment and resources invested in the learning journey. This provides a useful starting point for measuring something, but it can condense a complex learning experience into a set of expected and measurable behaviours.
A student who attends every lecture and contributes regularly may be labelled as ‘highly engaged’, and this could be from significant amounts of their own self-study away from Moodle. Another student may say little while internally analysing and evaluating an argument, making connections with previous learning, or preparing a considered research question that is developed in an offline context with their peers. They may not show a frequency of viewing or ‘completing’ activities on Moodle in the general sense. In a digital environment, opening a resource establishes that it has been accessed; it does not establish what the student has understood or done with it.
This matters when we consider what we want an activity to achieve. If the purpose is to help students evaluate an argument, knowing they have opened the resource tells us less than seeing how they explain their judgement through an explicit production activity. Contributing one post or several may not provide the distinction we need to ascertain engagement with the subject. Making the intended learning explicit in the storyboarding and preparation of learning helps us decide what evidence would be useful demonstrations of knowledge and understanding, however this can’t simply equate to an arbitrary number of clicks. Digitally-mediated engagement is not a straightforward phenomenon to quantify.
Three ways to think about engagement
A common categorisation is Fredricks, Blumenfeld and Paris (2004) dimensions of engagement. Although developed in a non-tertiary education context, the distinction offers a useful dimensional split for thinking about engagement in university learning. It is a regular citation in literature on engagement in Higher Education. It classifies engagement along three dimensions, the behavioural, the cognitive, and the emotional (or affective).
- Behavioural engagement concerns students’ participation and effort: attending, preparing, contributing and carrying out learning activities. These are often the most visible aspects of engagement and the easiest to record. Behavioural indicators, however shallow, are the most common measures of engagement in Higher Education.
- Cognitive engagement concerns the psychological investment and mental energy students bring to learning. This is the kind of engagement we often idealise for our students. This dimension directs our attention towards how students approach a task: whether they examine an explanation, persist with a difficult problem set, connect ideas across contexts or unpack and reconsider their understanding or think about their learning in interesting ways.
- Emotional engagement concerns students’ feelings in relation to learning, including interest, enthusiasm and their responses to the learning environment. It prompts us to consider whether students find meaning in a task and how they experience their participation. Understanding the affective dimension is critical as emotional engagement is the gateway into other forms of engagement. A student’s willingness to contribute may depend on whether they feel their contribution will be valued. Persistence with a difficult task may depend on its perceived relevance to a students’ life. Equally, completing the procedure of a task does not necessarily mean that the student has connected with it, investing only minimal cognitive resources to complete it.
Kahu (2013) makes further useful distinction between the dimensions of engagement, the factors that can influence them and their outcomes. For example, clear instructions that help students prepare may help with initial emotional engagement with learning, while a well-reasoned assignment may provide proxy evidence of sustained emotional engagement.
Understanding engagement also means considering what happens between those points: how the student participates, thinks and feels as they work through their learning sequences and tasks. For instance, an appropriately deep investigative task can positively affect motivation, confidence and enthusiasm to interact with other tasks that we could potentially measure.
Whose engagement is it anyway?
The student body of contemporary higher education is wonderfully diverse. This makes any expectation that engagement will manifest in the same behavioural way for everyone instantly problematic.
Our students bring different educational experiences, cultural expectations, academic resources and capital into the learning context. Their familiarity with tertiary education’s hidden-curriculum and practices, confidence in public contributions, and the opportunities to participate with learning, will vary. A requirement to speak spontaneously in a large group, for example, creates a particular route into observable participation with learning. It may also make some students’ thinking more visible than others’. It is an ostensive display of learning that we often make a judgement on. The question is whether spontaneous contribution is a necessary condition of an inclusive-task that demonstrates the dimensions of engagement and depth of learning that we want to see in a university context.
This raises a question about whose behaviours are the ones we recognise as indicators of engagement. Zepke (2014) encourages a more critical examination of this assumptions that sits behind student engagement research. In our teaching, we can ask whether our expectations give students appropriate opportunities to demonstrate their involvement. Would a short written response before a discussion help more students prepare for a Socratic challenge? Could time to compare ideas in pairs support a more considered whole-class debate or exchange?
We can retain appropriate academic challenge while making the routes into that challenge clearer and accessible. This is why it is essential to be clear about what we recognise as engagement for our students. This may require reflection on our own assumptions.
What does this mean for active blended learning?
Design for Active Blended Learning (DABL) is a lens that helps us consider how learning connects across physical and digital environments and between synchronous and asynchronous sequences. In the digital context, before selecting a tool or adding an interaction, it helps to identify what we want students to think about, contribute or develop, and how the activity fits into the wider sequence of their modules and learning. It is not just an activity that provides a morsel of data.
Consider a seminar built around evaluating competing explanations. Before the session, students might read a short case study and record an initial judgement on an anonymous post it board. During the session, an anonymous poll could make the range of judgements available for discussion, encouraging emotional investment. Students could then reflect on the reasoning behind their responses and submit a short paragraph for peer-feedback to examine meta-thinking skills.
The value lies in the relationship between these stages. The initial response gives students time to prepare. The poll provides a shared student-led resource for discussion. Comparing reasons creates an opportunity to examine an argument, while the later reflection allows students to articulate how their thinking has developed.
Each stage offers different evidence of engagement. A submitted judgement is an indication of participation but may increase positive emotional engagement (such as enthusiasm). An exploration of anonymous judgements in class gives us more insight into reasoning processes and other cognitive processes. Asking students to reflect can encourage more emotional investment and understanding of their learning experience, and can help us to understand what supported or constrained their involvement.
Providing opportunity for interaction is not an exclusive domain of technology. In active blended learning, our task is to connect physical and digital learning environments so that each has a clear purpose in a scaffolded sequence of learning. Adding a digital activity does not, by itself, establish that engagement has improved.
A useful starting point for your next activity
For our design conversations, a useful working definition of engagement is:
students’ behavioural, cognitive and emotional involvement in learning, experienced within a particular social and educational context.
This brings together the ideas discussed here as a lens to view the learning design context, rather than being a strict definition shared across the literature.
Consider selecting one activity in an upcoming week of your module. What will your students need to do? What thinking will it require? What might help them feel able and willing to take part?
Then consider how you and the students will recognise that involvement, and what you could ask them about that experience.
A small change might be enough to provide a clearer route into the learning: a guiding question alongside a reading, time to prepare a response, or an opportunity to revisit an initial idea after discussion. Asking students what helped them get started or what made the task difficult can help us understand what the activity records cannot show.
Learning is blended by design and active through intention. Being clearer about engagement helps us make that intention explicit and keeps the student at the centre of the experience.
If you would like support with designing active blended learning opportunities in your module, book a consultation with a Learning Designer.
References (All available through the library at Kent)
Fredricks, J. A., Blumenfeld, P. C. and Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59–109.
Kahu, E. R. (2013). Framing student engagement in higher education. Studies in Higher Education, 38(5), 758–773.
Zepke, N. (2014). Student engagement research in higher education: Questioning an academic orthodoxy. Teaching in Higher Education, 19(6), 697–708.