Categories
Technology

Using Splunk Data Analytics to Protect Students, Faculty, and the University

Presenters

  • Chris Kurtz, System Architect, Arizona State University

About Us

  • First Google Apps for Education customer
  • Multiple campuses with a diverse IT infrastructure
  • Large # of governing reqs: FERPA, HIPAA, DARPA, DoJ, NASA, JPL
  • Splunk is an Enterprise-level product, with easy access to all departments inside the University Technology Office (ISO/InfoSec, Ops, Dev, BA/BI, Accounting, Netcom, etc.). We wanted everyone to have equal access

The Power of Splunk

  • Is ASU’s universal aggregator of all machine generated logs
  • Typical response time to incident without Splunk: multiple days.
  • With Splunk, we have direct, immediate access…minutes!

Splunk and ASU

  • Had it for 4 years now.
  • It needs a lot of power to run properly
  • Use enterprise search head clustering and security
  • Licensing 1TB/day
  • Growth slowing down because we’re learning to better filter data
  • Admissions and payroll are beginning to use it

We Didn’t Know!

“It was like the invention of the microscope: we didn’t know what we couldn’t see” – Martin Idaszak, Security Architect, ASU

Use Case: Protecting Direct Deposit

  • Changing EE info online is great, but a target for hackers
  • ASU has international students, faculty and staff, just blocking other countries isn’ accessible
  • Before Splunk: whenever an EE was missing a direct deposit check, the investigation would take days, during which time it would sit between HR and Payroll systems. We were hand-protecting only a handful of people’s paychecks.
  • With Splunk, we check geo tag info, do an affiliate lookup, and put it into an unusual changes report which payroll checks.
  • Payroll will not run the payroll job WITHOUT this report now.
  • This is the most valuable data I have in Splunk, by far.
  • Where do you change your direct deposit from? Home and work. We take advantage of the “user’s center of gravity” to make a determination if the request is unusual.
  • False positives? YES. False responses? NO.

Use Case: Phishing as a Teaching Tool

  • We have 100K users. In 2015, we received 1 billion email messages, more than 750 million were spam and phishing.
  • We have students from all across the world, transient by nature, can’t assume traffic from Nigeria, China or Malaysia are hacking attempts. In fact, it’s probably legitimate!
  • Some Indian students were forced by their parents to give them their login credentials, which resulted in some interesting traffic and double-logins from completely different areas! We ended up setting up special limited accounts for these parents.
  • Do NOT store user emails in Splunk, only the headers that transit our system.
  • “This is the best tool we’ve seen in 10 years” – Jay Steed, AVP for UTO Operations, ASU

Leveraging Your Custom Data

  • It’s limited if you’re only reading logs.
  • If you don’t understand context of your data sources, you won’t get as much as you can get out of the product.
  • No schemas! No types! Eval is your friend.
  • Combine all data types in any way you want, on the fly.
  • “Think of it like a database where time is the primary key”
  • Don’t limit the power of Splunk!
  • Start using the Common Information Model now!
  • Not formatting data limits its value. Pull in secondary/ancillary data that makes sense of data in your logs. Makes the field extractions more valuable.
  • For ASU, the master datasource is the Data Warehouse. Affiliate ID is the unique ID.
  • Isolated Splunk server running Splunk DB Connect (DBXv2) runs SQL queries on several databases, and writes a series of lookup tables (with the Affiliate ID) every 4 hours. Linux ionotify monitors the lookup tables, and on write-close copies data to production systems (sanity checking applies).

Conclusion

  • Heavily invested in Splunk because it solves many of our outstanding problems.
  • 1st round of data onboarding concentrated on needs of ISO office
  • 2nd round focused on operations needs, with some interesting use cases thrown in as they appear
  • 3rd round is expanding Splunk usage and bringing it to the enterprise
  • Splunk’s savings in man hours, extreme flexibility, use to validate other systems, and goals to replace antiquated systems has very much paid off
  • Get your data into Splunk!
  • Modify it later.
  • Use the people who “get it” as evangelists
  • Don’t get caught up on “use cases.” Once you have the data in Splunk, use cases present themselves repeatedly. Think of it as use case on demand.

By Paul Schantz

CSUN Director of Web & Technology Services, Student Affairs. husband, father, gamer, part time aviator, fitness enthusiast, Apple fan, and iguana wrangler.

%d bloggers like this: