Using Splunk Data Analytics to Protect Students, Faculty, and the University

Presenters

First Google Apps for Education customer
Multiple campuses with a diverse IT infrastructure
Large # of governing reqs: FERPA, HIPAA, DARPA, DoJ, NASA, JPL
Splunk is an Enterprise-level product, with easy access to all departments inside the University Technology Office (ISO/InfoSec, Ops, Dev, BA/BI, Accounting, Netcom, etc.). We wanted everyone to have equal access

“It was like the invention of the microscope: we didn’t know what we couldn’t see” – Martin Idaszak, Security Architect, ASU

Changing EE info online is great, but a target for hackers
ASU has international students, faculty and staff, just blocking other countries isn’ accessible
Before Splunk: whenever an EE was missing a direct deposit check, the investigation would take days, during which time it would sit between HR and Payroll systems. We were hand-protecting only a handful of people’s paychecks.
With Splunk, we check geo tag info, do an affiliate lookup, and put it into an unusual changes report which payroll checks.
Payroll will not run the payroll job WITHOUT this report now.
This is the most valuable data I have in Splunk, by far.
Where do you change your direct deposit from? Home and work. We take advantage of the “user’s center of gravity” to make a determination if the request is unusual.
False positives? YES. False responses? NO.

We have 100K users. In 2015, we received 1 billion email messages, more than 750 million were spam and phishing.
We have students from all across the world, transient by nature, can’t assume traffic from Nigeria, China or Malaysia are hacking attempts. In fact, it’s probably legitimate!
Some Indian students were forced by their parents to give them their login credentials, which resulted in some interesting traffic and double-logins from completely different areas! We ended up setting up special limited accounts for these parents.
Do NOT store user emails in Splunk, only the headers that transit our system.
“This is the best tool we’ve seen in 10 years” – Jay Steed, AVP for UTO Operations, ASU

It’s limited if you’re only reading logs.
If you don’t understand context of your data sources, you won’t get as much as you can get out of the product.
No schemas! No types! Eval is your friend.
Combine all data types in any way you want, on the fly.
“Think of it like a database where time is the primary key”
Don’t limit the power of Splunk!
Start using the Common Information Model now!
Not formatting data limits its value. Pull in secondary/ancillary data that makes sense of data in your logs. Makes the field extractions more valuable.
For ASU, the master datasource is the Data Warehouse. Affiliate ID is the unique ID.
Isolated Splunk server running Splunk DB Connect (DBXv2) runs SQL queries on several databases, and writes a series of lookup tables (with the Affiliate ID) every 4 hours. Linux ionotify monitors the lookup tables, and on write-close copies data to production systems (sanity checking applies).

Heavily invested in Splunk because it solves many of our outstanding problems.
1st round of data onboarding concentrated on needs of ISO office
2nd round focused on operations needs, with some interesting use cases thrown in as they appear
3rd round is expanding Splunk usage and bringing it to the enterprise
Splunk’s savings in man hours, extreme flexibility, use to validate other systems, and goals to replace antiquated systems has very much paid off
Get your data into Splunk!
Modify it later.
Use the people who “get it” as evangelists
Don’t get caught up on “use cases.” Once you have the data in Splunk, use cases present themselves repeatedly. Think of it as use case on demand.