Title: How Good are your IT Services?
Presenter: Timothy M. Chester, VP for IT, University of Georgia (@accidentalcio)
Tim brings a passion around CIO and organizational performance. “Credibility ultimately comes from how well IT services are perceived by students, faculty and staff.” Tim will talk about his TechQual tool. Came into position at University of Georgia as AVP, was promoted to VP. Day-to-day relationships are what drive success, not the position.
Web site for this project: TechQual.org
Goals and Outcomes
- Review the context for creation of the TechQual + Survey and Tools
- Demonstrate the linkage between customer satisfaction and IT organization credibility
- Understand how assessment of IT outcomes can drive continuous improvement processes
- How to design and crae a TechQual+ survey to assess IT service quality
- …etc (slides went too fast)
TechQual grew out of LibQual (a library assessment tool). Tim was also involved in an accreditation evaluation of a research university, where a number of focus groups and surveys were done regarding delivery of IT services. People who responded ran the gamut:
- Good perception
- Meh attitude
- Angry with services (led by one furious person)
Do you want your leadership’s perception of IT services to be driven by random individuals in each of the above groups? No, you want to own this narrative. Despite Tim’s engineering background (which is why he got hired into his current position), his conversations tend to be around aspirations, dreams, and gaps.
The Power of Analytics
Showed us radar graphs of questions to visualize IT strengths and weaknesses. The radar graphs specifically address the following areas:
- Connectivity & Access
- Technology & Technology Services
- The End User Experience
A series of questions identify a) user expectations, and b) user perceptions. Colors indicate differences between the two. Items closer to the “hub” indicate lower priorities. This method can be broken down by respondent constituent groups. Each group brings their own set of priorities. Faculty are always a standard deviation below every other respondent group.
Tableau Visualization
Tim gave a demonstration of the power of visualization of data using a tool named Tableau.
IT Services that deliver value
IT these days is often simply administering services, not actually running it.
The Credibility Cycle
Ellen Kizsis book – “The New CIO Leader”
Initial Credibility > Resources & Expectations > Outcomes > Results > Back to Credibility
Alternate path: Poor results > Reduced Credibility > (Cycle of Overcommitment and Underperformance) > Diminished Authority
The IT Delivery Ecosystem
- Strategic Leadership
- assesment & Planning
- Operational Best practices
- Foundations
SWOT Analysis
Group was divided into 4 groups and went around the room identifying their own organization’s Strengths, Weaknesses, Opportunities, and Threats. Top 5 from each was highlighted. This was a pretty powerful exercise.
TechQual+ Project Outcomes
Measures that conceptualize the effective delivery and use of technology, or effective IT service outcomes, from the perspective of those outside the IT organization
A set of easy-to-use Web-based tools that allows institutions to create surveys based on the TechQual+ core instrument, communicate with respondents, and analyze survey results.
A peer database that allows institutions to make comparisons of IT service outcomes on the campus against the performance of other institutions, aggregated by Carnegie basic classification.
3 Core Commitments
- Connectivity and Access
- Technology and Collaboration Services
- Support and Training
Naturalistic Inquiry
What is a Positive Outcome?
People tend to make a positive evaluation when the IT service is:
- Delivered Consistently
- Communication is timely, relevant, and in an easy-to-understand form
- Increases collaboration opportunities with others
Navigating a TechQual+ Survey
Tim showed us what a respondent sees.
Higher Education TechQual+ Major Influences
SERVQUAL, an approach that conceives of service quality as a range of expectations that can be assessed by measuring three different dimensions of service
- Minimum Expectations
- Desired Expectations
- Perceived Performance
Survey Design & Setup
Tim reviewed the layout of the TechQual+ web site and tools available to a University TechQual+ survey administrator. This included Options, Core items, Custom Items, Other Questions, Instructions, Preview. The system has the ability to tailor the way you communicate the message that your survey is available to respondents by a) generic link that can be used anywhere or b) a tool that lets you upload tailored respondent lists (students, faculty, staff).
Sampling Concepts
- N = the entire population under study
- n = the sample of respondents that is representative of N
- Random Sampling = method for choosing n to ensure that n is truly representative of N
UGA always selects 25% of population each year to help avoid “survey fatigue.” The TechQual+ web site has a downloadable tool that will do a random sampling for you based on data you provide (lname, fname, email, field1, field2, field3, etc.).
To Get Good Response Rates
Tim reviewed some of the e-mail communication tools built into the TechQual+
- 4 e-mail message notifications is the sweet spot (pre-survey, survey, plus two reminders)
- Should be personalized (salutation and signature)
- ReplyTo e-mail should be a high-ranking person in the organization
- Link needs to be “above the fold”
Peer Comparison Functions
Tim demonstrated the ability of the system to compare institutional surveys. Need 50 valid responses in a survey in order for it to be added to the peer comparison data.
ETS Planning and Continuous Improvement Cycle
Data drives decision-making; this data goes into presentation “road show” that helps tell the IT story. Monthly Status and Activity Reports are extremely important!