Predictive Learning Analytics: Fueling Actionable Intelligence


  • Josh Baron, Assistant Vice President, Information Technology for Digital Education, Marist College
  • Shady Shehata, Principal Data Scientist, D2L
  • John Whitmer, Director for Analytics and Research, Blackboard Inc.

This is all about an ECAR report that was released a few weeks ago.  Report was compiled over about a year and had a large number of contributors.  It’s a great example of collaboration across organizations for profit and non-profit, scientific, etc.

39% is the Key Number

  • In the US, only this percentage completes a 4-year program.
  • This is a national challenge, because the US has slipped from #1 or #2 completion in the world to #12
  • How Can Predictive Learning Analytics Help?  It provides the ability to predict the future with a reasonable level of accuracy give you the ability to intervene on behalf of the student

What is Predictive Learning Analytics?

  • The statistical analysis of historical and current data derived from the learning process to create models that allow for predictions that improve learning outcomes
  • Subset of larger learning analytics field
  • Uses sophisticated mathematical models rather than user-defined rules.  Example:  Academic Early Alert Systems
  • OAAI:  the Open Academic Analytics Initiative
  • Apereo Learning Analytics Initiative

Data Sources, Relevance & Diversity

  • LMS:  academic technology’s first killer app
  • What’s been successful is the penetration and usage of LMS
  • What data from conventional data sources are systematic, significant predictors of course success?  High school GPA?  Race/Ethnicity?  First in Family to Attend College?  NONE OF THEM!
  • Academic Technology data is a systematic predictor of course success – caveat is that the academic material is connected in a deep and meaningful way.  Having a number of triggers help to track actionable details.
  • Embedding Predictive Analytics
  • Strategic Importance other Data Points:  underrepresented student groups.
  • Conclusion:  Learning Data Comes in Many Flavors and Relevance, i.e. Activity (behavioral) data and Static Data (survey data, student aptitude, extra-curricular activities, demographics and prior educational experience).
  • There are very few institutions that employe full-time Predictive Learning Analytics professionals.

How it Works, What is the Data Impact?

  • Historical data and predictive analytics are used to generate a predictive model
  • We want to tease out and surface those patterns that result in successful outcomes
  • After first month, predictive model can provide predictions of the final grades based on the # of content views of the students in the current course offering
  • Examples of what can go wrong:  what if students are viewing the content from mobile application (data incomplete); what if one of the historical course has hundreds of course topics, where other courses have tens of course topics?  (data is inaccurate)
  • Garbage In, Garbage Out
  • Data quality:  accurate, complete, unique, timely, consistent, valid, reliable, integrity

Brightspace Student Success System

  • Created a predictive model for a course
  • A number of screen shots showed how it was implemented with a group of students, with drilldowns on where students were having difficulty.
  • Good visualizations are critical to easily decoding information and making it useful

Data Strategy

  • ETL/Data Integration > Data Warehouse & MDM > BI & Data Visualization > Predictive Analytics

Strategic Implementation Considerations

  • Institutional Stages of analytics usage
  • Organizational leadership, culture & skills
  • gaining access to learning data
  • Ethics & privacy

Institutional Stages of Analytics Usage

  • Basic:  past trends & data observations
  • Automated:  automatically perform analytics & provide results directly to end-users
  • Predictive:  large amounts of data is crunched

Silos are antithetical to successful implementation; investing in new skill sets is imperative.

Gaining Access to Learning Data

  • Activity, clickstream data
  • It’s the fuel on which LA runs
  • Extracting sample data sets is often a good start

Ethics & Privacy

  • Ethics:  using LA for good and not evil
  • Privacy:  balance the need to protect confidentail records while maximizing the benefits of LA
  • Often requires new policies and procedures
  • LA “task force” to address ethics and privacy issues
  • JISC code of practice
  • SURF Learning Analytics SIG


  • How do you best approach the introduction of PA to a group of people who don’t even know what it does?  EDUCAUSE has some great white papers on this (“Penetrating the Fog”).  Give examples of products, strategies and solutions.
  • Have you done anything to look at the performance of blended/flipped classrooms?  We get asked this a lot!  We’ve looked at some of the open course offerings of MiT, Blackboard usage patterns.  Many folks are interested in using it for academic course design.

By Paul Schantz

CSUN Director of Web & Technology Services, Student Affairs. husband, father, gamer, part time aviator, fitness enthusiast, Apple fan, and iguana wrangler.

%d bloggers like this: