I attended this webinar today because I have a great interest in learning analytics, specifically with the integration of co-curricular data with respect to intervention strategies. It was great, glad that I attended! These folks will be at the EDUCAUSE conference in October, I’ll be attending some of their presentations 🙂
Presenters
- Lou Harrison from ncsu.edu
- Josh Baron from marist.edu
- Kate Valenti from unicon.net
Historical Context: OAAI Overview (Open Academic Analytics Initiative)
- EDUCAUSE Next Generation Learning Challenges (aka NGLC)
- Funded by Bill and Melinda Gates Foundation
- $250K over 15 months
- Goal: leverage big data to create an open-source academic early alert system and research “scaling factors”
Basic Flow
- Feed in Student Aptitudes: (i.e. SATs, current GPA, etc.), demographic data (i.e. age, gender, etc.)
- Feed in LMS data: Sakai Event Log and Gradebook data
- Fed into predictive scoring model
- Fed into Academic Alert system (AAR)
- Intervention deployed (to students & instructors)
Review of research design (I didn’t capture all this)
- Deployed to 2,200 students across 4 institutions
Conclusions
- Predicitve models are more “portable” than anticipated
- It’s possible to create generic models that are then “tuned” to use at specific types of institutions
- It’s possible to create a library of open predictive models that could be shared globally
Findings on Intervention Effectiveness
- Final course grades had a statistically significant positive impact on final corse grades
Apereo Learning Analytics Initiative Update
- Like the Apache foundation
- Serves higher education
- Other projects: Learning Analytics Processor (LAP), OpenDashboard, Larrisa, Student Success Plan (SSP)
- Modular System: Collection > Storage > Analysis > Communication > Action
- Just got started with JISC National Learning Analytics Project (UK org)
Moving Toward Enterprise Learning Analytics at NC State
How we’re getting there
- Lunch and learn sessions on LA space
- Bring people up to speed on what questions to ask
- Start thinking about who can generate answers
Details
- Many products vendors try to sell us are NOT predictive!
- We built a plan to build us a model, and then we validated it
Predictive Power
- Gradebook
- Cumulative GPA
- Academic Standing
- Then: course logins, content access, online flag…
Model Results
- Overall Accuracy: 75-77%
- Recall rates 88-90%
- False positives were a little high at 25-26%
Proof to Production
- Initial steps: small sample sizes
- Predictions at 1/4, 1/2, 3/4 points in course
- Multi-step, manual process
Goal 1: More Enterprise-y
- Large sample sizes (all student enrollments)
- Frequent early runs (maybe daily)
- Automatic, no more than 1 click
Currently in Progress
- Rebuild infrastructure for scale
- Daily snapshots of fall semester data
- After fall semester ends, look for sweet spot
Future Goals
- Refine model more
- Segment model by pops
- Balance models and accuracy
- Refine & improve models over time
- Explore ways to track efficacy over time
- Once we intervene, can never go back to virgin state
