Early warning systems often rely on a combination of student background data and outcomes from exams and quizzes. But it is possible that waiting until after a student does poorly on an exam is too late. This research focuses on exploring whether student behavior data gleaned from the LMS and other classroom digital technology can be used to identify students at risk before the first exam or quiz.
Student behavior and survey data from the LMS, Instructure Canvas, and Echo360 Active Learning Platform as well as the Student Information System (SIS) at the University of Michigan were acquired and merged for five courses from the College of Engineering and the School of Kinesiology. The in-class student behavior system provided class-by-class data on student attendance, participation in class activities, correctness to answers, volume of note taking, submission of questions and viewing of lecture captures. Surveys solicited students’ opinions on technology, commitment to external responsibilities and interest in the course content. The courses were offered both face-to-face and streamed for synchronous participation.
Results suggest that use of the quantity and quality of student participation during the class session can provide predictive skill of student performance even by the second week of the course. The results indicate that it is possible to identify those students who will receive average exam grades below 70% with about 77% accuracy by week two of the semester. The accuracy is about 70% based solely on student background data including incoming Grade Point Average. Because it is controversial whether instructors should know a student’s background data (as it might produce biases in the instructor’s grading) we also explored just using student behaviors in class to predict student success. These data (LMS, Echo360, etc.) produced accuracies near 75% independent of student background data.
This research represents a next step in the development of “earlier” warning systems wherein students at risk can be identified well before they fail their first exam. The University of Michigan is expanding its advisor dashboard, the Student Explorer, to make student behavior data available to students’ advisors.