Predictive Learning Analytics: Fueling Actionable Intelligence

Presenters:

  • Josh Baron, Assistant Vice President, Information Technology for Digital Education, Marist College
  • Shady Shehata, Principal Data Scientist, D2L
  • John Whitmer, Director for Analytics and Research, Blackboard Inc.

This is all about an ECAR report that was released a few weeks ago.  Report was compiled over about a year and had a large number of contributors.  It’s a great example of collaboration across organizations for profit and non-profit, scientific, etc.

39% is the Key Number

  • In the US, only this percentage completes a 4-year program.
  • This is a national challenge, because the US has slipped from #1 or #2 completion in the world to #12
  • How Can Predictive Learning Analytics Help?  It provides the ability to predict the future with a reasonable level of accuracy give you the ability to intervene on behalf of the student

What is Predictive Learning Analytics?

  • The statistical analysis of historical and current data derived from the learning process to create models that allow for predictions that improve learning outcomes
  • Subset of larger learning analytics field
  • Uses sophisticated mathematical models rather than user-defined rules.  Example:  Academic Early Alert Systems
  • OAAI:  the Open Academic Analytics Initiative
  • Apereo Learning Analytics Initiative

Data Sources, Relevance & Diversity

  • LMS:  academic technology’s first killer app
  • What’s been successful is the penetration and usage of LMS
  • What data from conventional data sources are systematic, significant predictors of course success?  High school GPA?  Race/Ethnicity?  First in Family to Attend College?  NONE OF THEM!
  • Academic Technology data is a systematic predictor of course success – caveat is that the academic material is connected in a deep and meaningful way.  Having a number of triggers help to track actionable details.
  • Embedding Predictive Analytics
  • Strategic Importance other Data Points:  underrepresented student groups.
  • Conclusion:  Learning Data Comes in Many Flavors and Relevance, i.e. Activity (behavioral) data and Static Data (survey data, student aptitude, extra-curricular activities, demographics and prior educational experience).
  • There are very few institutions that employe full-time Predictive Learning Analytics professionals.

How it Works, What is the Data Impact?

  • Historical data and predictive analytics are used to generate a predictive model
  • We want to tease out and surface those patterns that result in successful outcomes
  • After first month, predictive model can provide predictions of the final grades based on the # of content views of the students in the current course offering
  • Examples of what can go wrong:  what if students are viewing the content from mobile application (data incomplete); what if one of the historical course has hundreds of course topics, where other courses have tens of course topics?  (data is inaccurate)
  • Garbage In, Garbage Out
  • Data quality:  accurate, complete, unique, timely, consistent, valid, reliable, integrity

Brightspace Student Success System

  • Created a predictive model for a course
  • A number of screen shots showed how it was implemented with a group of students, with drilldowns on where students were having difficulty.
  • Good visualizations are critical to easily decoding information and making it useful

Data Strategy

  • ETL/Data Integration > Data Warehouse & MDM > BI & Data Visualization > Predictive Analytics

Strategic Implementation Considerations

  • Institutional Stages of analytics usage
  • Organizational leadership, culture & skills
  • gaining access to learning data
  • Ethics & privacy

Institutional Stages of Analytics Usage

  • Basic:  past trends & data observations
  • Automated:  automatically perform analytics & provide results directly to end-users
  • Predictive:  large amounts of data is crunched

Silos are antithetical to successful implementation; investing in new skill sets is imperative.

Gaining Access to Learning Data

  • Activity, clickstream data
  • It’s the fuel on which LA runs
  • Extracting sample data sets is often a good start

Ethics & Privacy

  • Ethics:  using LA for good and not evil
  • Privacy:  balance the need to protect confidentail records while maximizing the benefits of LA
  • Often requires new policies and procedures
  • LA “task force” to address ethics and privacy issues
  • JISC code of practice
  • SURF Learning Analytics SIG

Q&A

  • How do you best approach the introduction of PA to a group of people who don’t even know what it does?  EDUCAUSE has some great white papers on this (“Penetrating the Fog”).  Give examples of products, strategies and solutions.
  • Have you done anything to look at the performance of blended/flipped classrooms?  We get asked this a lot!  We’ve looked at some of the open course offerings of MiT, Blackboard usage patterns.  Many folks are interested in using it for academic course design.

One thought on “Predictive Learning Analytics: Fueling Actionable Intelligence”

Comments are closed.