Learning Analytics for Educational Design and Student Predictions: Beyond the Hype with Real-Life Examples

Title:  Learning Analytics for Educational Design and Student Predictions:  Beyond the Hype with Real-Life Examples

Presenters:

• Nynke Kruiderink, Teamleader, Educational Technology Social Sciences, University of Amsterdam
• Perry J. Samson, Professor, Department of Atmospheric, Oceanic & Space Sciences, University of Michigan-Ann Arbor
• Nynke Bos, Program Manager, Educational Technology, University of Amsterdam

 

SLIDE:  Lessons Learned:  February 2012 – Present

Our Proof of Concept is Two-Tiered

1. Interviews with lecturers, professors, managers
2. Gather and store data in central place for easy access

General things we learned

• Emotional response to “Big Brother” aspect of accessing data
• Data from LMS not detailed enough (folder based, not file based)
• 50% of learning data available
• Piwki, not secure enough

 

Next Steps

• Focus group:  learning analytics
• Professor Erik Duval – KU Leuven  (his advice: undertake one project involving all that proves learning analytics is useful)

What is the Problem?

Recorded lectures

Recording of F2F lectures

No policy at the University of Amsterdam

Different deployment throughout the curriculum

Not at all (fears/emotional)

Week after the lecture

Week before the assessment

 

Student vs. Policy

Students demanded policy

QA department wanted insight into academic achievement before doing so

Development of didactic framework

Research:  learning analytics

 

 Design

• Two courses on psychology
• Courses run simultaneously
• Intervention in one condition, but not the other

 

Data Collection

• Viewing of recorded lecture
• Lecture attendance per lecture
• Final grade on the course (with more segmented view)
• Some other data (grades on previous courses, distance to the lecture hall, gender, age, etc.)

 

Lessons Learned

• Let people know what you are doing
• Data preparation:  fuzzy, messy
• Choose the data
• Simplify the data
• Keep an eye on the prize

 

LectureTools:  Analytics

Females in class were much more likely to ask questions using a clicker (lovingly referred to as the “WTF” button).

90% of students at University of Michigan in the meteorology class cited felt they would have gotten the same grade if they had never opened the textbook