Presenters:
This is the last concurrent session of the conference. Most of this presentation will be about specific implementations of adaptive learning at a couple institutions.
Adaptive Learning Market Acceleration Grant Program (ALMAP)
- 14 grants
- 17 colleges
- 9 adaptive learning platforms
- 22 courses
- 44% average % of Pell eligible students at grantees
- 21,644 total students enrolled across 3 terms
- 699 instructors
Adaptive Tech personalizes instruction and learning. Courseware provides customized feedback to student on learning gaps. Courseware tracks progress for instructor support.
ALMAP vision and goals
Expand and build understanding of how US higher ed programs are using adaptive courseware to support student success and completion.
ALMAP Evaluation Portfolio
- 14 grantees conducted QED student impact evaluations. Collected instructor / student survey data and cost data. Collected over 3 academic terms (summer 2013 – Winter 2015.
- Grantee studies featured 3 different types of comparisons: lecture vs. blended adaptive; online vs online adaptive; blended vs blended adaptive
- Evaluator checked rigor of local designs, extracted insights across portfolio.
What Did You Do and Why?
Essex County, 12,000 students. Math sequence is the biggest barrier to success.
- How did the adaptive courseware meet your expectations? In the adaptive classes, students use labs with adaptive courseware, and we ask the students to set goals for the things they want to master. Invariably, the goals student set for themselves are higher than what they actually achieve. This is something that we then work with them on.
- The software worked perfectly for us, did exactly what we expected of us. However, the adaptive software took our instructors about 2 semesters to get fluent with.
Rio Salado, with 60,000 students.
- Our courses were fully online, using Pearson’s product. We looked at student learning outcomes, faculty/staff feedback, and cost analysis. What we’ve seen in the past is that our students tend to drop out if they were less than successful with their coursework, or if the class was “too slow” for them.
- We were mostly satisfied with our experiment with adaptive learning. We had a fluid working relationship with Pearson, and they were amenable to working out difficulties we had with our pilot. Our writing assessments needed more content for our students’ needs. While we could pick content from what Pearson had to offer, we could not develop our own. We had to take our material for the writing assessments to beef up the product. We videotaped sessions and embedded writing into each lesson to help ensure completion.
Aggregate Evaluation Research Questions
- What student impacts are noted and in what HE contexts/disciplines?
- How does using adaptive courseware affect the costs of instruction?
- How are students and instructors experiencing adaptive courseware?
ALMAP Evidence of Impacts
Significant positive course grade gains were noted when adaptivity was: part of course redesign (lecture to blended) OR added to online courses BUT NOT when replacing another blended technology.
Product features linked with learning gains: progress dashboards, regular quizzes/feedback; referrals to remedial content and study tips; spaced memorization practice; vendor content (but 1 supported memorization of faculty content)
Course disciplines showing more learning gains: 50% of psychology courses; 42% of math courses; 25% of biology courses; 16% of English courses
Instructor Experience: 78% of instructors reported satisfaction; 57% devoted 1-9 hours to courseware training
Student Experience: most students reported positive learning gains; students reported different levels of engagement.
Courseware Cost Drivers & ROI
- Courseware based on instructor content had 8% to 19% higher development and training costs.
- Most cost reductions occurred when adding adaptivity during course redesign, so cannot attributed to courseware.
How did you change your use of adaptive learning products over time and why? What’s next for you?
- In Essex County, we’re not changing our approach at all. Students going through the adaptive developmental classes are showing greater signs of success in traditional COLLEGE LEVEL math courses later on, which for us are ONLY delivered in a traditional way.
- At Rio Salado, we didn’t see much difference, but we’re still following the students who went through online versus adaptive classes. We’re now doing “student learning cafes,” group sessions where students and faculty can share their experiences with using the adaptive learning material. Students like the ability to move at their own pace, but faculty want improvements in navigation and assessments. We have 3 grant opportunities that we’re pursuing to do more.
