Ms Jennifer Whitfield, Ms Nadia Kempfe
University of NSW
UNSW Library obtains large amounts of quantitative completion data through a university-wide online induction quiz. But this data is not as useful as might be thought. Due to the large-scale size of the cohort undertaking the quiz, the platform analytics are high-level and only broadly define where students get stuck. To achieve a more granular understanding of the student experience, the Library applied a threshold concepts approach – a lens to provide insight into the “why” of “troublesome knowledge” (Blackmore, 2010).
On completion of the induction quiz, students are invited to complete an anonymous feedback survey. A surprisingly large amount of qualitative feedback is returned. The team analyses this feedback using the UNSW Library threshold concepts: Academic Rules, Pattern Perception, Time/Outcome Ratio, Play & Exploration and Systemic Thinking. These were originally developed in 2010 and have continued to underpin innovations in service delivery. A peer-review process is used to match each piece of feedback to relevant concept/s. Through categorisation, patterns are identi ed around where students are struggling or engaging with fundamental understandings of the quiz content. This analysis is then combined with the high-level learning analytics to drive iterative change.
The combined analysis has given UNSW Library authentic and actionable insights into the student experience. Findings include students struggling with core concepts such as peer reviewed literature, academic writing and referencing and citations. In response, new interactive learning activities have been integrated into the learning content, areas have been streamlined and activities have been redesigned to improve understandability and usability. What is always kept in mind is the Time/Outcome Ratio.
The ongoing project has demonstrated the importance of viewing the student experience from di erent angles and data sources helping UNSW to ensure relevance and true adaptivity. Building on the actionable insights garnered from this combined approach, the team is now interested in drawing more qualitative data from the quiz itself via open-ended question design as technology develops.