Using Splunk Data Analytics to Protect Students, Faculty, and the University


  • Chris Kurtz, System Architect, Arizona State University

About Us

  • First Google Apps for Education customer
  • Multiple campuses with a diverse IT infrastructure
  • Large # of governing reqs: FERPA, HIPAA, DARPA, DoJ, NASA, JPL
  • Splunk is an Enterprise-level product, with easy access to all departments inside the University Technology Office (ISO/InfoSec, Ops, Dev, BA/BI, Accounting, Netcom, etc.). We wanted everyone to have equal access

The Power of Splunk

  • Is ASU’s universal aggregator of all machine generated logs
  • Typical response time to incident without Splunk: multiple days.
  • With Splunk, we have direct, immediate access…minutes!

Splunk and ASU

  • Had it for 4 years now.
  • It needs a lot of power to run properly
  • Use enterprise search head clustering and security
  • Licensing 1TB/day
  • Growth slowing down because we’re learning to better filter data
  • Admissions and payroll are beginning to use it

We Didn’t Know!

“It was like the invention of the microscope: we didn’t know what we couldn’t see” – Martin Idaszak, Security Architect, ASU

Use Case: Protecting Direct Deposit

  • Changing EE info online is great, but a target for hackers
  • ASU has international students, faculty and staff, just blocking other countries isn’ accessible
  • Before Splunk: whenever an EE was missing a direct deposit check, the investigation would take days, during which time it would sit between HR and Payroll systems. We were hand-protecting only a handful of people’s paychecks.
  • With Splunk, we check geo tag info, do an affiliate lookup, and put it into an unusual changes report which payroll checks.
  • Payroll will not run the payroll job WITHOUT this report now.
  • This is the most valuable data I have in Splunk, by far.
  • Where do you change your direct deposit from? Home and work. We take advantage of the “user’s center of gravity” to make a determination if the request is unusual.
  • False positives? YES. False responses? NO.

Use Case: Phishing as a Teaching Tool

  • We have 100K users. In 2015, we received 1 billion email messages, more than 750 million were spam and phishing.
  • We have students from all across the world, transient by nature, can’t assume traffic from Nigeria, China or Malaysia are hacking attempts. In fact, it’s probably legitimate!
  • Some Indian students were forced by their parents to give them their login credentials, which resulted in some interesting traffic and double-logins from completely different areas! We ended up setting up special limited accounts for these parents.
  • Do NOT store user emails in Splunk, only the headers that transit our system.
  • “This is the best tool we’ve seen in 10 years” – Jay Steed, AVP for UTO Operations, ASU

Leveraging Your Custom Data

  • It’s limited if you’re only reading logs.
  • If you don’t understand context of your data sources, you won’t get as much as you can get out of the product.
  • No schemas! No types! Eval is your friend.
  • Combine all data types in any way you want, on the fly.
  • “Think of it like a database where time is the primary key”
  • Don’t limit the power of Splunk!
  • Start using the Common Information Model now!
  • Not formatting data limits its value. Pull in secondary/ancillary data that makes sense of data in your logs. Makes the field extractions more valuable.
  • For ASU, the master datasource is the Data Warehouse. Affiliate ID is the unique ID.
  • Isolated Splunk server running Splunk DB Connect (DBXv2) runs SQL queries on several databases, and writes a series of lookup tables (with the Affiliate ID) every 4 hours. Linux ionotify monitors the lookup tables, and on write-close copies data to production systems (sanity checking applies).


  • Heavily invested in Splunk because it solves many of our outstanding problems.
  • 1st round of data onboarding concentrated on needs of ISO office
  • 2nd round focused on operations needs, with some interesting use cases thrown in as they appear
  • 3rd round is expanding Splunk usage and bringing it to the enterprise
  • Splunk’s savings in man hours, extreme flexibility, use to validate other systems, and goals to replace antiquated systems has very much paid off
  • Get your data into Splunk!
  • Modify it later.
  • Use the people who “get it” as evangelists
  • Don’t get caught up on “use cases.” Once you have the data in Splunk, use cases present themselves repeatedly. Think of it as use case on demand.

The EDUCAUSE 2017 Top 10 IT Issues


  • Rebecca Davis, Director of Instructional and Emerging Technology, St. Edward’s University
  • Susan Grajek, VP of Data, Research, and Analytics, EDUCAUSE
  • Marden Paul, Director, Planning, Governance, Assessment & Comms, University of Toronto
  • Gerard Au, AVP, IT Services, CSU San Bernardino
  • John Landers, PMO Leader, Case Western Reserve University
  • Michele Norin, SVP and CIO, Rutgers, The State University of New Jersey

Top 10 IT issues

We have a large slate of professionals who weigh in on these issues each year. They come from pretty much every role in higher education IT. They meet four times a year, and we ask “what is the most important IT issue facing your institution?” We do this each quarter, and then again in the Summer before the annual conference…and that’s the list we’re presenting today.

  1. Information Security
  2. Student Success & completion
  3. Data-informed decision making
  4. Strategic leadership
  5. Sustainable funding
  6. Data mgmt and governance
  7. Higher ed affordability
  8. Sustainable staffing
  9. Next-gen enterprise IT
  10. Digital transformation of learning

* Bold items are new to the list this year.

Discussion of the Issues

  • Next Generation Enterprise IT: developing and implementing enterprise IT applications, architectures, and sourcing strategies to achieve agility, scalability, cost-effectiveness, and effective analytics.
  • Digital Transformation of Learning: collaborating with faculty and academic leadership to apply technology to teaching and learning in ways that reflect innovations in pedagogy and the institutional mission. Do you limit what tech people can use or do you have a limited set of tools you offer (limited set won the day). What’s the most misunderstood aspect of this issue? X solution = the panacea for digital learning. New networks are horizontal and personalized, not just content delivery online. It’s bigger than the implementation of a tech stack. Roles for people in this ecosystem will be different.
  • Strategic Uses of Data
    • Student Success and completion – effectively applying data and predictive analytics to improve student success and completion. What are the implications? The tech implementation does NOT necessarily solve the problem. Changing the culture is the hard part, especially when the technology adds another burden, i.e. just another chore. Aggregation of data from systems of record continues to be troublesome.
    • Data-informed decision making – ensuring that BI, reporting and analytics are relevant, convenient, and used by administrators, faculty and students. Engaging with all stakeholders is important to ensure we have the right data to make decisions…we all need to refer to the same dataset. IT needs to be the glue that holds all these folks together. Consider hiring a Chief Data Officer, to ask all these questions. Some faculty are getting student involvement by giving FitBits and recording what they eat as a means of educating them about owning their own data.
    • Data management and governance – improving the management of institutional data through data standards, integration, protection, and governance. How many people think that their data is always accurate? << laughter >> Are there outsized expectations of Big Data? Yes! It’s not about the tools, it’s about how people use the tools and how that affects the data downstream (which can be very bad). Use the data for what it is supposed to do…don’t adapt it for your own immediate purposes; rely on authoritative sources. Data is not a project, it’s a process.



How does this list make you feel? Do you feel hopeful? Cautious? Something else?

  • I think we’re collecting more data right now than we’re able to use effectively. Eventually, our ability to manage and process this data will become doable.
  • Cautiously optimistic.
  • As a PM, I’m nervous! What gives me hope is the fact that we’re not alone…we have this organization and our colleagues on other campuses.
  • IT is more relevant than ever…everybody is now a part of IT!

What was issue 11?

  • Building a sustainable workforce (but next-gen IT workforce was)

Did we slice the data by different university types?

  • Yes! It will be in the January issue.

Where can you get a copy of these slides?

  • Somewhere on the EDUCAUSE web site 🙂

What skills are necessary for next-gen IT?

  • How to read contracts << laughter >>
  • Click-throughs are not a good idea. Legal counsel will have a problem with 60 page documents with embedded links to other documents.
  • Business Analyst mentality will become more important.
  • You don’t know how much you know until you know how much you care.
  • Soft skills are really important – distributed leadership and the support to do that.


Machine Learning 101


  • Greg Corrado, Senior Research Scientist, Google
  • Vincent Nestler, Professor & Assistant Director of Cybersecurity, CSU San Bernardino
  • David Vasilia, Enterprise Network Administrator & Faculty, CSU, San Bernardino
  • Internet2 & GCP:
  • CS edu grants:

Machine Learning 101

  • Already in everyday products: photos, inbox, maps
  • 2 disciplines: AI and machine learning
  • Traditional AI systems are programmed to be clever
  • ML-based AI systems are designed to learn to be clever
  • Classic AI works on rules and contingencies; ML AI learns from examples and data.
  • Machines learn by example: models (which have parameters) feed predictions, which feeds a learner, which in turn feeds the parameters. This is surprisingly simple and generic.
  • Need 4 things: computational resources, good tools & algorithms, training examples, creativity and ingenuity of people.
  • Effective, but very gradual process that takes millions or billions of examples for it to work. It needs to cycle many many times.
  • ML coming of age in this decade because the computational power is exists now and it’s cheap and plentiful enough, i.e. CPU, GPU, Google TPU.
  • a toolkit for machine learning
    • Open standard
    • Next gen deep learning tools built in
    • One system flexible enough for ML research
    • Robust enough for use in real products
    • Same software Google researchers use
  • Deep learning not one function, but a set of composable subfunctions for model building.
  • Distributing ML Tech Globally
    • Shared Tools: TensorFlow + CloudML
    • Ready-made ML systems (Cloud Vision API, Cloud Speech API, Cloud Translate API, etc.)
    • Use our tools to build your own system!
    • Example: TensorFlow cucumber sorting tool (really!)
    • Shared knowledge: open research publication at intl conferences; global direct community education; funding academic research and education.
  • Google published 90+ papers in the last 4 years
  • Takeaways:
    • Differentiation between AI vs. ML vs. Robotics
    • It isn’t magic, just a tool
    • Machines learn best from examples
    • Why now? fast computation
    • Make ML work requires creativity/ingenuity, cheap/fast computation/examples to learn from (data), tools & algorithms, TensorFlow makes ML software available for free.
    • Google Cloud makes hardware available.

Cloud for Higher Ed

  • Programming a campus rover: students are given a sensor, a raspberry pi, and Python. Then, they need to figure out how to integrate it.
  • Hacking now means hacking things together. You don’t have to be an engineer and you don’t need to know everything.
  • How can I level the playing field for my students? Be able to connect to Chrome and a Google compute engine. Everyone can look at and work with this environment, and they can explore from there.
  • A project we worked on in class: Android mapping for WiFi signal strength on campus. War driving took signal strength and using mapping API to literally map it to a real topographical map. Now we can “see our WiFi.”
  • We used Intermapper software to map the Internet, specifically the CENIC network from Los Angeles. The students loved this.


  • What is the difference between deep learning and machine learning? ML is the larger field of making machines that learn. DL is a small subset of this.
  • How far is Google taking cultural sensitivity into account with ML? Take translate as an example: you can dig into what the algorithm did to come up with its response.
  • If we use a Google tool, does this tool report what it learns back to Google? NO. What is pricing model for Google Cloud for Google Apps customers? It is independent of G-Suite.

Next Steps

  • Google is now a member of Internet2.
  • Will work with universities across the US to explore how Google Cloud Platform can better serve higher education
  • Help students build what’s next!
  • GCP Education Grants are available to: faculty in US, teaching university courses in CS or related fields in 2016-17 academic year. Examples: general CS, Cybersecurity, systems administration, networking.

Your Legacy: An Organization That Delivers Strategic Value, Again and Again


  • Dean Meyer, President, NDMA
  • Julie Little, VP, EDUCAUSE

There’s lots going on in education technology right now! Tons of enabling tech that changes business, education, and business models. Make learning engaging, contextual and visual.

Types of Strategic Value

Growing human intellect is absolutely strategic!

  1. Keep business running (deliver existing services)
  2. Reduce costs of IT
  3. Reduce costs of business (productivity)
  4. Improve human effectiveness (thinking, collaboration)
  5. Improve customer relationships, loyalty (engagement)
  6. Enhance product value

As a senior leader, what can you do to drive your organization up this ladder?

The Classic Definition of the Role of a CIO portrays us superman / superwoman, but the reality is that the CIO becomes a cog in the machine…often a bottleneck. Wouldn’t it be better to be the driver of the machine. To get there, you need to first be the designer of the machine. Our systems send signals that guide people. For CIOs, these signals are often about building an empire.

How you define leadership changes as you advance in your career, describable through different lenses.

  1. Project management
  2. Supervision
  3. Business strategies
  4. Organizational designer

Leaving the legacy of an organization that can prosper, with or without you. Program the organizational system.

The Machine

“The programming language of leadership”

  1. Structure
  2. Metrics & rewards
  3. Internal Economy
  4. Culture
  5. Methods and Tools


The easiest thing to change. Mixture of values and behaviors. You critique the behavior, not the people.


  • Org chart
  • Workflows

Internal Economy

  • Planning
  • Dynamic governance

Methods & Tools

  • Individual competencies of individual groups
  • Fine tuning

Metrics & Rewards

  • Dashboards, consequences
  • Fine tuning

Value Chain

  • Expertise in linkage: business-IT. This is a kind of “bridging knowledge.” [Structure: “sales”]
  • Collaborative discovery: help others find the things they need. [Methods: discovery]
  • Broad, innovative catalog. You need a quiver full of different arrows to apply to any given problem. [Internal Economy: business planning]
  • Time to develop proposals. [Internal Economy: unbillable time]
  • Project funding. [Internal Economy: demand management]
  • Delivery: capability, teamwork. Deliver on-time and on-budget. [Structure: walk-throughs]
  • Benefits realization. Make sure things get used! [Culture: business within a business; Methods: benefits measurement]

3 Parallel Leadership Strategies

  • Business Value
  • Capabilities: tech, operations
  • Organization (often neglected by leadership because it’s so foundational to the success of the first two strategies)

Big three are Culture, Structure, and Internal Economy. After this, methods & tools, metrics & rewards.


  • What does world class IT team mean to you?
  • Measure the gaps. These are the symptoms of something deeper in your organization. Keep asking WHY until you get to one of the fundamentals.
  • Sequence the Root Causes into your strategy.
  • Publicize your strategy among your staff and the peers in your institution.

Free yourself from the tyranny of urgency.

Initiative Impact Analysis to Prioritize Action and Resource Allocation


  • Virginia Fraire, VP of Student Success, Austin Community College District
  • Laura Malcom, VP of Product, Civitas Learning Inc.
  • Angela Baldasare, Asst. Provost, Institutional Research, The University of Arizona

University of Arizona

  • Goal: improve 1st year retention rate from 81% to 91% by 2025
  • How do we find and integrate good data to make good decisions that help our students?
  • When I came on board, I found out that we never had a centralized student academic support office
  • SALT office (Strategic Alternative Learning Techniques) – used to support students with learning disabilities. How can we adopt and adapt some of the techniques that worked there?
  • We were using siloed participant data that was not very helpful. It was not transformative and it didn’t tell us much.
  • We came to Civitas for help.
  • In 2009, U of A opened doors to the “Think Tank” to streamline and centralize a number of academic support services offered by nationally certified tutors; mission is to empower UA students by providing a positive environment where they can master the skills needed to become successful lifelong learners.
  • In one year, nearly 11,000 students make more than 70,000 visits and spend 85,000+ hours with support staff.

Think Tank Impact

  • Illume Impact used PPSM to measure 2.7%(pp) overall life in persistence for students using the writing center
  • 3.4% (pp) increase for 1st year students
  • Less than 10% of 1st year students taking advantage of this service!
  • These results will inform strategic campaigns to offer Think Tank services to students as part of first-year experience.
  • 8.2% persistence increase for students who were most at risk

Taking Initiative With Confidence

  • Sharing impact findings with academic colleges to discuss the need for increased referrals to Think Tank.
  • PPSM has changed the conversation with faculty who want rigorous data.
  • Bolstering credibility and validity to Think Tank services.

Austin Community College

Highland Campus is home to the ACCelerator, one of the largest high-tech learning environments in the country.

“The Accelerator”

  • Provides access to 600+ desktop computer stations spread over 32,000 square feet, surrounded by classrooms and study rooms.
  • Offers redesigned DevEd math courses powered by learning software with an onsite faculty members, tutors and academic coaches to increase personalization and engagement
  • Additional support services are offered, including non-math tutoring, advising, financial aid, supplemental instruction, and peer tutoring.
  • During the 2015-16 year, the ACCelerator served over 13,000 unique students in well over 170,000 interactions.

Accelerator Impact

  • Students who visit the lab at least once each term persist at a higher rate.
  • 4x persistence impact found for DevEd students.
  • Part-time DevEd students and DevEd students with the lowest persistence predictions had even better outcomes.
  • 6.15% increase in persistence for students visiting the learning lab.
  • Results are informing strategic decisions about creating similar learning spaces at other campuses.
  • Impact results have helped validate ACC data and in-house analyses
  • Discussions with math faculty continue to strengthen the developmental math redesign
  • Persistence results leading to further investigation of other metrics related to accelerated learning, particularly for DevEd students.
  • For this kind of approach to work, silos need to be broken down.




Continuing Adventures in Higher Ed & Technology