Tag: edu15

Categories
Technology Uncategorized

If You Build It: The Power of Design to Change the World

Presenter

Emily Pilloton, Founder and Executive Director, Project H Design

 

Project H Design was created out of a sense of frustration.  Emily began as an architect and after 3 years, was “totally over it.”  She was designing doorknobs and lighting fixtures and found it extremely dull.  It was disconnected from what the things that made her fall in love with architecture in the first place:  getting dirty and solving interesting projects with other people.  Project H is in it’s 8th year.  MacGyver was Emily’s first crush, not just because he’s cute, but because he solves problems in unconventional ways.  Her two grandmothers were very strong and creative women (and librarians!), who invested a lot into their professional practices.

Experience More Important Than Content

We have a responsibility to create learning experiences for our young people that are meaningful.

My partner and I were invited to a failing school district to use design “by any means” to help it succeed.  The results of this was Studio H, a class that takes place during the school day for which students earn credit.  Students create something that is architectural that has a public benefit.  The first project was to build a farmer’s market, which in Windsor, North Carolina was a revolutionary idea.  Now, a big part of architecture is to sell your idea to stakeholders.  This brought people together that would normally not speak with each other…a big win.

Constraints

  • $50,000 construction budget
  • Construction crew of teenagers
  • Hurricane zone
  • Flood zone

Construction

Construction began on the first day of summer.  When everyone else was going out on Jet Skis, my students showed up every day in extreme heat and humidity.  Labor law in South Carolina says that children under 17 cannot operate power machinery on a construction site.  As a result, we had only one student who could operate the chop saw.

RESULT:  this project created 4 new businesses and 17 full-time positions!

STUDENT QUOTE:  “I want to come back here with my kids someday and tell them that I built this”

Seeking is More Important Than Knowing

A constant state of inquiry is important to moving forward.  Our next project was done in Berkeley, California…pretty much at the opposite end of the spectrum from our first project.  Students here come from pretty much everywhere and span the range of experience.  These children wanted to build a library as their class gift back to their school.

The students wanted their library to be a space for discovery, not reference.  How to build a library that is meaningful to all 108 students in the 8th grade?  We use building blocks built by a CNC Router to build modular shelving, tables, etc.  While the project feels unfinished and uncomfortable to Emily, because it doesn’t feel finished.  However, it’s exactly what the students wanted.

Student quote to Emily about her unease:  “In algebra, X is the unknown.  The X-Space is where we go tot discover the things we don’t know.”

We is Greater Than I

I’m super-introverted and don’t like to work with other people.  However, I know that collaboration is important to creativity.  I’ve found that collaboration is less about democracy and more about trust.  The next project we worked on was to build two “tiny homes.”

I had my students build about 100 different models.  We did precedent studies that made students think through all the different reasons why they should include different design elements into a project, like space, lighting, flow through, etc.

We purchased trailers and reinforced them.  The day that we raised the walls was my favorite day.  25 teenagers had to work together to get it set up and squared together.  The students really wanted to use pallets, which I can tell you was NOT easy to work with.  The homes turned out beautiful!  One was auctioned, and one was given to a an organization that supports the homeless.

STUDENT QUOTE:  “I gave someone a place to live.  Oh, and I got an A in this class and know how to build a house.”

Curiosity is More Important Than Passion

Passion is big, often difficult to quantify, and hard to access for some.  Curiosity, on the other hand, is incremental, approachable and generates more creativity.

  • Learn to use a set of tools
  • How do you use this to express your own ideas
  • How do you use this to apply yourself to build something bigger with other people

STUDENT QUOTE:  “I’m a 10 year old girl and I know how to weld.  What can’t I do?”

Categories
Student Affairs Technology

Progress on Using Adaptive Learning Technology for Student College Success

Presenters:

  • Yvonne M. Belanger, Senior Program Officer, Bill and Melinda Gates Foundation
  • Jo Jorgenson, Dean of Instruction & Community Development, Rio Salado College; ALMAP Grantee
  • Douglas Walcerz, VP Planning Research and Assessment, Essex County College; ALMAP Grantee
  • Louise Yarnall, Senior Researcher, SRI International; ALMAP Portfolio Evaluator

This is the last concurrent session of the conference.  Most of this presentation will be about specific implementations of adaptive learning at a couple institutions.

Adaptive Learning Market Acceleration Grant Program (ALMAP)

  • 14 grants
  • 17 colleges
  • 9 adaptive learning platforms
  • 22 courses
  • 44% average % of Pell eligible students at grantees
  • 21,644 total students enrolled across 3 terms
  • 699 instructors

Adaptive Tech personalizes instruction and learning.  Courseware provides customized feedback to student on learning gaps.  Courseware tracks progress for instructor support.

ALMAP vision and goals

Expand and build understanding of how US higher ed programs are using adaptive courseware to support student success and completion.

ALMAP Evaluation Portfolio

  • 14 grantees conducted QED student impact evaluations.  Collected instructor / student survey data and cost data.  Collected over 3 academic terms (summer 2013 – Winter 2015.
  • Grantee studies featured 3 different types of comparisons:  lecture vs. blended adaptive; online vs online adaptive; blended vs blended adaptive
  • Evaluator checked rigor of local designs, extracted insights across portfolio.

What Did You Do and Why?

Essex County, 12,000 students.  Math sequence is the biggest barrier to success.

  • How did the adaptive courseware meet your expectations?  In the adaptive classes, students use labs with adaptive courseware, and we ask the students to set goals for the things they want to master.  Invariably, the goals student set for themselves are higher than what they actually achieve.  This is something that we then work with them on.
  • The software worked perfectly for us, did exactly what we expected of us.  However, the adaptive software took our instructors about 2 semesters to get fluent with.

Rio Salado, with 60,000 students.

  • Our courses were fully online, using Pearson’s product.  We looked at student learning outcomes, faculty/staff feedback, and cost analysis.  What we’ve seen in the past is that our students tend to drop out if they were less than successful with their coursework, or if the class was “too slow” for them.
  • We were mostly satisfied with our experiment with adaptive learning.  We had a fluid working relationship with Pearson, and they were amenable to working out difficulties we had with our pilot.  Our writing assessments needed more content for our students’ needs.  While we could pick content from what Pearson had to offer, we could not develop our own.  We had to take our material for the writing assessments to beef up the product.  We videotaped sessions and embedded writing into each lesson to help ensure completion.

Aggregate Evaluation Research Questions

  • What student impacts are noted and in what HE contexts/disciplines?
  • How does using adaptive courseware affect the costs of instruction?
  • How are students and instructors experiencing adaptive courseware?

ALMAP Evidence of Impacts

Significant positive course grade gains were noted when adaptivity was:  part of course redesign (lecture to blended) OR added to online courses BUT NOT when replacing another blended technology.

Product features linked with learning gains: progress dashboards, regular quizzes/feedback; referrals to remedial content and study tips; spaced memorization practice; vendor content (but 1 supported memorization of faculty content)

Course disciplines showing more learning gains:  50% of psychology courses; 42% of math courses; 25% of biology courses; 16% of English courses

Instructor Experience:  78% of instructors reported satisfaction; 57% devoted 1-9 hours to courseware training

Student Experience:  most students reported positive learning gains; students reported different levels of engagement.

Courseware Cost Drivers & ROI

  • Courseware based on instructor content had 8% to 19% higher development and training costs.
  • Most cost reductions occurred when adding adaptivity during course redesign, so cannot attributed to courseware.

How did you change your use of adaptive learning products over time and why?  What’s next for you?

  • In Essex County, we’re not changing our approach at all.  Students going through the adaptive developmental classes are showing greater signs of success in traditional COLLEGE LEVEL math courses later on, which for us are ONLY delivered in a traditional way.
  • At Rio Salado, we didn’t see much difference, but we’re still following the students who went through online versus adaptive classes.  We’re now doing “student learning cafes,” group sessions where students and faculty can share their experiences with using the adaptive learning material.  Students like the ability to move at their own pace, but faculty want improvements in navigation and assessments.  We have 3 grant opportunities that we’re pursuing to do more.
Categories
Technology Uncategorized

How to Use the EDUCAUSE CDS to Support Student Success

Presenters

  • Susan Grajek, Vice President, Data, Research, and Analytics, EDUCAUSE
  • Laurie Heacock, National Director of Data, Technology and Analytics, Achieving the Dream, Inc.
  • Louis Kompare, Director, Information Systems and Services, Lorain County Community College
  • Celeste M. Schwartz, VP for IT & IR, Montgomery County Community College

Susan kicked off this session by describing what the CDS is.  It’s been around for over 10 years, includes data from over 800 institutions and allows members to use it to:

  • Study their IT org
  • To benchmark against past performance
  • To look at trends over time
  • To start gathering and using metrics
  • To have data available “just in case”

TOP IT ISSUE #4

Improve Student Outcomes Through an Institutional Approach that Strategically Leverages Technology. Data shared today come from module 3 of the CDS

Student Success Technologies Maturity Index

These 6 measurements are set by subject matter experts, and are measured against a 5 point radar scale

  1. Leadership and governance
  2. Collaboration and involvement
  3. Advising and student support
  4. Process and policy
  5. Information systems
  6. Student Success analytics

Maturity Index

  1. Weak
  2. Emerging
  3. Developing
  4. Strong
  5. Excellent

Deployment Index

  1. No deployment
  2. Expected deployment
  3. Initial deployment
  4. targeted deployment
  5. institution-wide deployment

Goal

Provide higher ed institutions with a reliable, affordable, and useful set of tools to benchmark and improve the cost and quality of IT services, improving the value and efficiency of IT’s contribution to higher education.

Process

Complete Core Data > order and configure reports > receive and use reports.  It takes between 40 and 70 hours to complete, but data is saved for auto-filling the following year.  This speeds the re-entry process considerably.

You can also use the reports for benchmarking against other institutions.  You can create your own, and some peer groups are pre-provided for you.

Achieving the Dream’s Institutional Capacity Framework

Montgomery County Community College (near Philadelphia), about 13,000 students, participating in CDS for about 13 years.  Celeste then went on…In the past, we used CDS more on the justification of new staff.  We used to look at numbers of computers for students, but we tend to look at those numbers less today.  What’s really helped us recently are in how we ask questions about technology.  While you only HAVE to complete module 1, I recommend you dip your toes in some of the other modules.  I’ve used SurveyMonkey to extend my reach and gather additional information from other folks, and then moved it into CDS.  The CDS is really helping to drive our own IT strategic plan.

Lorain Community College (near Cleveland), about 12,000 students, participating in CDS for 2 years.  Our enrollment is highly tied to local industry; local business cycles make make our completion rates look terrible!  CDS is the most valuable way I have to find out the various elements of IT in the higher ed world.  It really helps to discover the things that change from year-to-year.

Top 10 IT Issues Sneak Peek

Coming out in January in EDUCAUSE Review.  IT security is the #1 issue.  Three dimensions that will be discussed in the upcoming report:

  • Divest:  change the way you design, deliver and manage IT services.  Eliminate old processes and silos!
  • Reinvest:  to run state-of-the-art technology services, you need to double down on some things, like information security.  Hiring and retaining good talent, along with restructuring that talent to meet the changing needs of delivering IT services.  The ability to change funding models to meet those needs is also important.
  • Differentiate: institutions are now able to apply technology to strategically meet their goals and differentiate themselves from other institutions.  Ability to apply analytics against strategic objectives is hugely valuable to help provide feedback on where we are and what we need to do to improve.
Categories
Technology

Unifying Data Systems to Turn Insights into Student Success Interventions

Presenters:

  • Angela Baldasare, Asst.Provost, Institutional Research, The University of Arizona
  • Phil Ice, Vice President, Research & Development, American Public University System
  • Matthew Milliron, Senior Director, Solutions Engineering, Civitas Learning Inc.

Hypothesis

Unifying the data; connect disparate data systems, data and initiatives to gain insight into what’s working & what’s not & for whom

The area that’s of most interest to Phil is the LMS, because that’s where we have the most interaction.  Unfortunately, it’s mostly log file information.  Scroll and click information is not captured.  LTI integration does not help much because it’s based on an iFrame and we lose context.  Instead, they’re using Adobe Analytics (formerly Omniture).  We’re also using social sharing.

Institution-Specific Platform for Innovation

Unified Data Layer (Student Data Footprint – historic and incoming disparate systems) is connected to:

  • Institution-Specific Deep Predictive Flow Models
  • Frontline Apps & Initiatives
  • Robust Testing and Measurement

Matthew then talked about the Civitas Learning Platform components (not exactly a sales pitch, but not too far off).

Prediction

  • UA Historical Overall Fall to Fall Retention Rate = 87%
  • FTIC FT Freshman Historical Average First Year REtention Rate = 80%
  • Prediction for Fall 2015 Cohort = 80%
  • n=6,970 students

Data set used for modeling:

  • Train:  Fall 2012 to Fall 2013
  • Test:  Fall 2013 to Fall 2014

Model accuracy

  • AUC .844
  • 90% accuracy

Discoveries

For FTFT Freshman in their First Term, students with SAT Math >550 persist at a rate +1- percentage points higher than SAT Math <550

For FTFT Freshman in their First Term, an LMS Course Grade on day 14 that is lower than 75% is associated with lower persistence than students with grades over 75%.

For transfer students overall, following course pathways traveled by students who graduated is beneficial for persistence.

For FTIC students who deviate significantly from the course pathway, the effect can be very bad.

Angela mentioned how useful the toolset is for the ability to see a list of students that make up any active filter segment within the too, and dig deeper on the their activity to extract additional actionable insights.

Categories
Technology

Predictive Learning Analytics: Fueling Actionable Intelligence

Presenters:

  • Josh Baron, Assistant Vice President, Information Technology for Digital Education, Marist College
  • Shady Shehata, Principal Data Scientist, D2L
  • John Whitmer, Director for Analytics and Research, Blackboard Inc.

This is all about an ECAR report that was released a few weeks ago.  Report was compiled over about a year and had a large number of contributors.  It’s a great example of collaboration across organizations for profit and non-profit, scientific, etc.

39% is the Key Number

  • In the US, only this percentage completes a 4-year program.
  • This is a national challenge, because the US has slipped from #1 or #2 completion in the world to #12
  • How Can Predictive Learning Analytics Help?  It provides the ability to predict the future with a reasonable level of accuracy give you the ability to intervene on behalf of the student

What is Predictive Learning Analytics?

  • The statistical analysis of historical and current data derived from the learning process to create models that allow for predictions that improve learning outcomes
  • Subset of larger learning analytics field
  • Uses sophisticated mathematical models rather than user-defined rules.  Example:  Academic Early Alert Systems
  • OAAI:  the Open Academic Analytics Initiative
  • Apereo Learning Analytics Initiative

Data Sources, Relevance & Diversity

  • LMS:  academic technology’s first killer app
  • What’s been successful is the penetration and usage of LMS
  • What data from conventional data sources are systematic, significant predictors of course success?  High school GPA?  Race/Ethnicity?  First in Family to Attend College?  NONE OF THEM!
  • Academic Technology data is a systematic predictor of course success – caveat is that the academic material is connected in a deep and meaningful way.  Having a number of triggers help to track actionable details.
  • Embedding Predictive Analytics
  • Strategic Importance other Data Points:  underrepresented student groups.
  • Conclusion:  Learning Data Comes in Many Flavors and Relevance, i.e. Activity (behavioral) data and Static Data (survey data, student aptitude, extra-curricular activities, demographics and prior educational experience).
  • There are very few institutions that employe full-time Predictive Learning Analytics professionals.

How it Works, What is the Data Impact?

  • Historical data and predictive analytics are used to generate a predictive model
  • We want to tease out and surface those patterns that result in successful outcomes
  • After first month, predictive model can provide predictions of the final grades based on the # of content views of the students in the current course offering
  • Examples of what can go wrong:  what if students are viewing the content from mobile application (data incomplete); what if one of the historical course has hundreds of course topics, where other courses have tens of course topics?  (data is inaccurate)
  • Garbage In, Garbage Out
  • Data quality:  accurate, complete, unique, timely, consistent, valid, reliable, integrity

Brightspace Student Success System

  • Created a predictive model for a course
  • A number of screen shots showed how it was implemented with a group of students, with drilldowns on where students were having difficulty.
  • Good visualizations are critical to easily decoding information and making it useful

Data Strategy

  • ETL/Data Integration > Data Warehouse & MDM > BI & Data Visualization > Predictive Analytics

Strategic Implementation Considerations

  • Institutional Stages of analytics usage
  • Organizational leadership, culture & skills
  • gaining access to learning data
  • Ethics & privacy

Institutional Stages of Analytics Usage

  • Basic:  past trends & data observations
  • Automated:  automatically perform analytics & provide results directly to end-users
  • Predictive:  large amounts of data is crunched

Silos are antithetical to successful implementation; investing in new skill sets is imperative.

Gaining Access to Learning Data

  • Activity, clickstream data
  • It’s the fuel on which LA runs
  • Extracting sample data sets is often a good start

Ethics & Privacy

  • Ethics:  using LA for good and not evil
  • Privacy:  balance the need to protect confidentail records while maximizing the benefits of LA
  • Often requires new policies and procedures
  • LA “task force” to address ethics and privacy issues
  • JISC code of practice
  • SURF Learning Analytics SIG

Q&A

  • How do you best approach the introduction of PA to a group of people who don’t even know what it does?  EDUCAUSE has some great white papers on this (“Penetrating the Fog”).  Give examples of products, strategies and solutions.
  • Have you done anything to look at the performance of blended/flipped classrooms?  We get asked this a lot!  We’ve looked at some of the open course offerings of MiT, Blackboard usage patterns.  Many folks are interested in using it for academic course design.
%d bloggers like this: