Categories
Student Affairs Technology

UNICON Learning Analytics Webinar

I attended this webinar today because I have a great interest in learning analytics, specifically with the integration of co-curricular data with respect to intervention strategies.  It was great, glad that I attended!  These folks will be at the EDUCAUSE conference in October, I’ll be attending some of their presentations 🙂

Presenters

  • Lou Harrison from ncsu.edu
  • Josh Baron from marist.edu
  • Kate Valenti from unicon.net

Historical Context:  OAAI Overview (Open Academic Analytics Initiative)

  • EDUCAUSE Next Generation Learning Challenges (aka NGLC)
  • Funded by Bill and Melinda Gates Foundation
  • $250K over 15 months
  • Goal:  leverage big data to create an open-source academic early alert system and research “scaling factors”

Basic Flow

  1. Feed in Student Aptitudes: (i.e. SATs, current GPA, etc.), demographic data (i.e. age, gender, etc.)
  2. Feed in LMS data:  Sakai Event Log and Gradebook data
  3. Fed into predictive scoring model
  4. Fed into Academic Alert system (AAR)
  5. Intervention deployed (to students & instructors)

Review of research design (I didn’t capture all this)

  • Deployed to 2,200 students across 4 institutions

Conclusions

  • Predicitve models are more “portable” than anticipated
  • It’s possible to create generic models that are then “tuned” to use at specific types of institutions
  • It’s possible to create a library of open predictive models that could be shared globally

Findings on Intervention Effectiveness

  • Final course grades had a statistically significant positive impact on final corse grades

Apereo Learning Analytics Initiative Update

  • Like the Apache foundation
  • Serves higher education
  • Other projects:  Learning Analytics Processor (LAP), OpenDashboard, Larrisa, Student Success Plan (SSP)
  • Modular System:  Collection > Storage > Analysis > Communication > Action
  • Just got started with JISC National Learning Analytics Project (UK org)

Moving Toward Enterprise Learning Analytics at NC State

How we’re getting there
  • Lunch and learn sessions on LA space
  • Bring people up to speed on what questions to ask
  • Start thinking about who can generate answers

Details

  • Many products vendors try to sell us are NOT predictive!
  • We built a plan to build us a model, and then we validated it

Predictive Power

  • Gradebook
  • Cumulative GPA
  • Academic Standing
  • Then:  course logins, content access, online flag…

Model Results

  • Overall Accuracy:  75-77%
  • Recall rates 88-90%
  • False positives were a little high at 25-26%

Proof to Production

  • Initial steps:  small sample sizes
  • Predictions at 1/4, 1/2, 3/4 points in course
  • Multi-step, manual process

Goal 1:  More Enterprise-y

  • Large sample sizes (all student enrollments)
  • Frequent early runs (maybe daily)
  • Automatic, no more than 1 click

Currently in Progress

  • Rebuild infrastructure for scale
  • Daily snapshots of fall semester data
  • After fall semester ends, look for sweet spot

Future Goals

  • Refine model more
  • Segment model by pops
  • Balance models and accuracy
  • Refine & improve models over time
  • Explore ways to track efficacy over time
  • Once we intervene, can never go back to virgin state

By Paul Schantz

CSUN Director of Web & Technology Services, Student Affairs. husband, father, gamer, part time aviator, fitness enthusiast, Apple fan, and iguana wrangler.

%d bloggers like this: