Hello, friends!

As part of my normal conference coverage, I publish a post of posts, which I call a “MEGA POST.”  It’s my attempt to capture all the different sessions I attended.  The 2015 EDUCAUSE annual conference in Indianapolis had so many great sessions, I often found it difficult to pick one over another.  The increased use of data to drive campus decision-making was a hot topic at the conference this year.

I do my best to capture the content of ever session, but I am human…any errata, misstatements or omissions are totally mine.  I hope you find some benefit from my conference experience.  Enjoy!

Tuesday, October 27

  1. EDUCAUSE 2015!
  2. Building an Emerging Technology and Futures Capacity in Your Organization
  3. Cloud 101:  Tools and Strategies for Evaluating Cloud Services

Wednesday, October 28

  1. KEYNOTE:  The Cascade Effect:  How Small Wins Can Transform Your Organization
  2. A View from the Top: Taking the Mobile Experience to New Heights
  3. The Science of Predictive Analytics in Education
  4. Opening Up Learning Analytics:  Addressing a Strategic Imperative

Thursday, October 29

  1. The 2015 Campus Computing Survey
  2. Web Portals
  3. KEYNOTE:  The Second Machine Age:  Work, Progress, and Prosperity in a Time of Brilliant Technologies
  4. Optimizing Business Intelligence at Lehman College/CUNY:  A Road to Change
  5. Predictive Learning Analytics:  Fueling Actionable Intelligence
  6. Unifying Data Systems to Turn Insights into Student Success Interventions

Friday, October 30

  1. How to Use the EDUCAUSE CDS to support Student Success
  2. Progress on Using Adaptive Learning Technology for Student College Success
  3. KEYNOTE:  If You Build It:  The Power of Design to Change the World

If You Build It: The Power of Design to Change the World


Emily Pilloton, Founder and Executive Director, Project H Design


Project H Design was created out of a sense of frustration.  Emily began as an architect and after 3 years, was “totally over it.”  She was designing doorknobs and lighting fixtures and found it extremely dull.  It was disconnected from what the things that made her fall in love with architecture in the first place:  getting dirty and solving interesting projects with other people.  Project H is in it’s 8th year.  MacGyver was Emily’s first crush, not just because he’s cute, but because he solves problems in unconventional ways.  Her two grandmothers were very strong and creative women (and librarians!), who invested a lot into their professional practices.

Experience More Important Than Content

We have a responsibility to create learning experiences for our young people that are meaningful.

My partner and I were invited to a failing school district to use design “by any means” to help it succeed.  The results of this was Studio H, a class that takes place during the school day for which students earn credit.  Students create something that is architectural that has a public benefit.  The first project was to build a farmer’s market, which in Windsor, North Carolina was a revolutionary idea.  Now, a big part of architecture is to sell your idea to stakeholders.  This brought people together that would normally not speak with each other…a big win.


  • $50,000 construction budget
  • Construction crew of teenagers
  • Hurricane zone
  • Flood zone


Construction began on the first day of summer.  When everyone else was going out on Jet Skis, my students showed up every day in extreme heat and humidity.  Labor law in South Carolina says that children under 17 cannot operate power machinery on a construction site.  As a result, we had only one student who could operate the chop saw.

RESULT:  this project created 4 new businesses and 17 full-time positions!

STUDENT QUOTE:  “I want to come back here with my kids someday and tell them that I built this”

Seeking is More Important Than Knowing

A constant state of inquiry is important to moving forward.  Our next project was done in Berkeley, California…pretty much at the opposite end of the spectrum from our first project.  Students here come from pretty much everywhere and span the range of experience.  These children wanted to build a library as their class gift back to their school.

The students wanted their library to be a space for discovery, not reference.  How to build a library that is meaningful to all 108 students in the 8th grade?  We use building blocks built by a CNC Router to build modular shelving, tables, etc.  While the project feels unfinished and uncomfortable to Emily, because it doesn’t feel finished.  However, it’s exactly what the students wanted.

Student quote to Emily about her unease:  “In algebra, X is the unknown.  The X-Space is where we go tot discover the things we don’t know.”

We is Greater Than I

I’m super-introverted and don’t like to work with other people.  However, I know that collaboration is important to creativity.  I’ve found that collaboration is less about democracy and more about trust.  The next project we worked on was to build two “tiny homes.”

I had my students build about 100 different models.  We did precedent studies that made students think through all the different reasons why they should include different design elements into a project, like space, lighting, flow through, etc.

We purchased trailers and reinforced them.  The day that we raised the walls was my favorite day.  25 teenagers had to work together to get it set up and squared together.  The students really wanted to use pallets, which I can tell you was NOT easy to work with.  The homes turned out beautiful!  One was auctioned, and one was given to a an organization that supports the homeless.

STUDENT QUOTE:  “I gave someone a place to live.  Oh, and I got an A in this class and know how to build a house.”

Curiosity is More Important Than Passion

Passion is big, often difficult to quantify, and hard to access for some.  Curiosity, on the other hand, is incremental, approachable and generates more creativity.

  • Learn to use a set of tools
  • How do you use this to express your own ideas
  • How do you use this to apply yourself to build something bigger with other people

STUDENT QUOTE:  “I’m a 10 year old girl and I know how to weld.  What can’t I do?”

Progress on Using Adaptive Learning Technology for Student College Success


  • Yvonne M. Belanger, Senior Program Officer, Bill and Melinda Gates Foundation
  • Jo Jorgenson, Dean of Instruction & Community Development, Rio Salado College; ALMAP Grantee
  • Douglas Walcerz, VP Planning Research and Assessment, Essex County College; ALMAP Grantee
  • Louise Yarnall, Senior Researcher, SRI International; ALMAP Portfolio Evaluator

This is the last concurrent session of the conference.  Most of this presentation will be about specific implementations of adaptive learning at a couple institutions.

Adaptive Learning Market Acceleration Grant Program (ALMAP)

  • 14 grants
  • 17 colleges
  • 9 adaptive learning platforms
  • 22 courses
  • 44% average % of Pell eligible students at grantees
  • 21,644 total students enrolled across 3 terms
  • 699 instructors

Adaptive Tech personalizes instruction and learning.  Courseware provides customized feedback to student on learning gaps.  Courseware tracks progress for instructor support.

ALMAP vision and goals

Expand and build understanding of how US higher ed programs are using adaptive courseware to support student success and completion.

ALMAP Evaluation Portfolio

  • 14 grantees conducted QED student impact evaluations.  Collected instructor / student survey data and cost data.  Collected over 3 academic terms (summer 2013 – Winter 2015.
  • Grantee studies featured 3 different types of comparisons:  lecture vs. blended adaptive; online vs online adaptive; blended vs blended adaptive
  • Evaluator checked rigor of local designs, extracted insights across portfolio.

What Did You Do and Why?

Essex County, 12,000 students.  Math sequence is the biggest barrier to success.

  • How did the adaptive courseware meet your expectations?  In the adaptive classes, students use labs with adaptive courseware, and we ask the students to set goals for the things they want to master.  Invariably, the goals student set for themselves are higher than what they actually achieve.  This is something that we then work with them on.
  • The software worked perfectly for us, did exactly what we expected of us.  However, the adaptive software took our instructors about 2 semesters to get fluent with.

Rio Salado, with 60,000 students.

  • Our courses were fully online, using Pearson’s product.  We looked at student learning outcomes, faculty/staff feedback, and cost analysis.  What we’ve seen in the past is that our students tend to drop out if they were less than successful with their coursework, or if the class was “too slow” for them.
  • We were mostly satisfied with our experiment with adaptive learning.  We had a fluid working relationship with Pearson, and they were amenable to working out difficulties we had with our pilot.  Our writing assessments needed more content for our students’ needs.  While we could pick content from what Pearson had to offer, we could not develop our own.  We had to take our material for the writing assessments to beef up the product.  We videotaped sessions and embedded writing into each lesson to help ensure completion.

Aggregate Evaluation Research Questions

  • What student impacts are noted and in what HE contexts/disciplines?
  • How does using adaptive courseware affect the costs of instruction?
  • How are students and instructors experiencing adaptive courseware?

ALMAP Evidence of Impacts

Significant positive course grade gains were noted when adaptivity was:  part of course redesign (lecture to blended) OR added to online courses BUT NOT when replacing another blended technology.

Product features linked with learning gains: progress dashboards, regular quizzes/feedback; referrals to remedial content and study tips; spaced memorization practice; vendor content (but 1 supported memorization of faculty content)

Course disciplines showing more learning gains:  50% of psychology courses; 42% of math courses; 25% of biology courses; 16% of English courses

Instructor Experience:  78% of instructors reported satisfaction; 57% devoted 1-9 hours to courseware training

Student Experience:  most students reported positive learning gains; students reported different levels of engagement.

Courseware Cost Drivers & ROI

  • Courseware based on instructor content had 8% to 19% higher development and training costs.
  • Most cost reductions occurred when adding adaptivity during course redesign, so cannot attributed to courseware.

How did you change your use of adaptive learning products over time and why?  What’s next for you?

  • In Essex County, we’re not changing our approach at all.  Students going through the adaptive developmental classes are showing greater signs of success in traditional COLLEGE LEVEL math courses later on, which for us are ONLY delivered in a traditional way.
  • At Rio Salado, we didn’t see much difference, but we’re still following the students who went through online versus adaptive classes.  We’re now doing “student learning cafes,” group sessions where students and faculty can share their experiences with using the adaptive learning material.  Students like the ability to move at their own pace, but faculty want improvements in navigation and assessments.  We have 3 grant opportunities that we’re pursuing to do more.

How to Use the EDUCAUSE CDS to Support Student Success


  • Susan Grajek, Vice President, Data, Research, and Analytics, EDUCAUSE
  • Laurie Heacock, National Director of Data, Technology and Analytics, Achieving the Dream, Inc.
  • Louis Kompare, Director, Information Systems and Services, Lorain County Community College
  • Celeste M. Schwartz, VP for IT & IR, Montgomery County Community College

Susan kicked off this session by describing what the CDS is.  It’s been around for over 10 years, includes data from over 800 institutions and allows members to use it to:

  • Study their IT org
  • To benchmark against past performance
  • To look at trends over time
  • To start gathering and using metrics
  • To have data available “just in case”


Improve Student Outcomes Through an Institutional Approach that Strategically Leverages Technology. Data shared today come from module 3 of the CDS

Student Success Technologies Maturity Index

These 6 measurements are set by subject matter experts, and are measured against a 5 point radar scale

  1. Leadership and governance
  2. Collaboration and involvement
  3. Advising and student support
  4. Process and policy
  5. Information systems
  6. Student Success analytics

Maturity Index

  1. Weak
  2. Emerging
  3. Developing
  4. Strong
  5. Excellent

Deployment Index

  1. No deployment
  2. Expected deployment
  3. Initial deployment
  4. targeted deployment
  5. institution-wide deployment


Provide higher ed institutions with a reliable, affordable, and useful set of tools to benchmark and improve the cost and quality of IT services, improving the value and efficiency of IT’s contribution to higher education.


Complete Core Data > order and configure reports > receive and use reports.  It takes between 40 and 70 hours to complete, but data is saved for auto-filling the following year.  This speeds the re-entry process considerably.

You can also use the reports for benchmarking against other institutions.  You can create your own, and some peer groups are pre-provided for you.

Achieving the Dream’s Institutional Capacity Framework

Montgomery County Community College (near Philadelphia), about 13,000 students, participating in CDS for about 13 years.  Celeste then went on…In the past, we used CDS more on the justification of new staff.  We used to look at numbers of computers for students, but we tend to look at those numbers less today.  What’s really helped us recently are in how we ask questions about technology.  While you only HAVE to complete module 1, I recommend you dip your toes in some of the other modules.  I’ve used SurveyMonkey to extend my reach and gather additional information from other folks, and then moved it into CDS.  The CDS is really helping to drive our own IT strategic plan.

Lorain Community College (near Cleveland), about 12,000 students, participating in CDS for 2 years.  Our enrollment is highly tied to local industry; local business cycles make make our completion rates look terrible!  CDS is the most valuable way I have to find out the various elements of IT in the higher ed world.  It really helps to discover the things that change from year-to-year.

Top 10 IT Issues Sneak Peek

Coming out in January in EDUCAUSE Review.  IT security is the #1 issue.  Three dimensions that will be discussed in the upcoming report:

  • Divest:  change the way you design, deliver and manage IT services.  Eliminate old processes and silos!
  • Reinvest:  to run state-of-the-art technology services, you need to double down on some things, like information security.  Hiring and retaining good talent, along with restructuring that talent to meet the changing needs of delivering IT services.  The ability to change funding models to meet those needs is also important.
  • Differentiate: institutions are now able to apply technology to strategically meet their goals and differentiate themselves from other institutions.  Ability to apply analytics against strategic objectives is hugely valuable to help provide feedback on where we are and what we need to do to improve.

Unifying Data Systems to Turn Insights into Student Success Interventions


  • Angela Baldasare, Asst.Provost, Institutional Research, The University of Arizona
  • Phil Ice, Vice President, Research & Development, American Public University System
  • Matthew Milliron, Senior Director, Solutions Engineering, Civitas Learning Inc.


Unifying the data; connect disparate data systems, data and initiatives to gain insight into what’s working & what’s not & for whom

The area that’s of most interest to Phil is the LMS, because that’s where we have the most interaction.  Unfortunately, it’s mostly log file information.  Scroll and click information is not captured.  LTI integration does not help much because it’s based on an iFrame and we lose context.  Instead, they’re using Adobe Analytics (formerly Omniture).  We’re also using social sharing.

Institution-Specific Platform for Innovation

Unified Data Layer (Student Data Footprint – historic and incoming disparate systems) is connected to:

  • Institution-Specific Deep Predictive Flow Models
  • Frontline Apps & Initiatives
  • Robust Testing and Measurement

Matthew then talked about the Civitas Learning Platform components (not exactly a sales pitch, but not too far off).


  • UA Historical Overall Fall to Fall Retention Rate = 87%
  • FTIC FT Freshman Historical Average First Year REtention Rate = 80%
  • Prediction for Fall 2015 Cohort = 80%
  • n=6,970 students

Data set used for modeling:

  • Train:  Fall 2012 to Fall 2013
  • Test:  Fall 2013 to Fall 2014

Model accuracy

  • AUC .844
  • 90% accuracy


For FTFT Freshman in their First Term, students with SAT Math >550 persist at a rate +1- percentage points higher than SAT Math <550

For FTFT Freshman in their First Term, an LMS Course Grade on day 14 that is lower than 75% is associated with lower persistence than students with grades over 75%.

For transfer students overall, following course pathways traveled by students who graduated is beneficial for persistence.

For FTIC students who deviate significantly from the course pathway, the effect can be very bad.

Angela mentioned how useful the toolset is for the ability to see a list of students that make up any active filter segment within the too, and dig deeper on the their activity to extract additional actionable insights.

Continuing Adventures in Higher Ed & Technology