An interesting email dropped on my digital doormat this morning asking for my input on the “key KPI’s for a Fortune 2000 IT department.” What made this more interesting is that I have just left Singapore where I have been working with a client on this very topic.
So, in the way that atoms randomly smash together sometimes into what looks like a pattern, it seems like a good time to explore this subject in the blog.
Let me start by saying that my view of KPI’s (Key Performance Indicators) is that they serve the purpose of helping managers see the effect of their decisions and to give them early warning of when they need to intervene. I fully support the trend towards compensating managers based on their affect on KPI’s. And I also see a real distinction between KPI’s and SLA’s (Service Level Agreements) that often gets blurred these days. KPI’s help you manage what you are responsible for delivering: SLA’s help you manage your relationship with other parts of the business and what they deliver to you or you deliver to them.
In this morning’s email the author posited that these 5 KPI’s were a good starting point.
- IT Cost/Revenue
- Rate of Application Change
- Employee Satisfaction
I like the simplicity of these. They have the same emotional charm as the “flat tax”: I can work them out in my head, I know what they mean and I know what good and bad are.
But after a little thought it occurred to me that if we dig deeper they are flawed too:
- IT Cost/Revenue – It is simple to calculate but what does it actually tell you? If the KPI is trending up or down is that a good or a bad thing? How this number compares to industry norms is interesting for investors but this ratio depends a lot on the effectiveness of all parts of the business that are outside the control of IT. One might suggest that when revenue is down increased investment in IT should be the strategy. My counter suggestion would be Cost/Revenue:Industry norm so that the business can see it is investing at a rate 85% of of similar insurance companies or 110% the rate of the world as a whole? This would then reflect trends in the IT industry as well as vertical sector the business is in.
- Availability – Also easy and straightforward to calculate as total time available/total systems are up with planned and unplanned downtime factored in or reported as part of the KPI. But for this to be a meaningful measure I would want the word availability to also measure the reach of the systems. Today much of the delivery of the system is through hand held devices. A company that just deploys to the iPhone because it is easy is missing out on the fast growing ‘Droid market and may be missing the BlackBerry market entirely. So built into availability has to be reach too. This is a key performance indicator of the appeal of the application and by extension the business as a whole.
- Rate of Application Change – Is for me, an ALM Guru, the most interesting as it rolls up many aspects of change. It acknowledges the rate at which the business is placing demand on IT, IT’s ability to respond to the demand and the agility of the application development system to deliver on quality, on budget, on time change into production. On Monday this week I just completed a study on release management for a large telco and the problem came down to “less haste, more speed”, in other words the more they rushed to change things the more the business processes clogged up, error rates soared, customer satisfaction declined and changes missed market windows. So rate of change is inextricably tied to quality of change, timeliness of change and customer satisfaction (internal and external). So the recommendation I would make about rate of change is to measure the satisfaction of the business with IT’s performance across a broad range of vectors rolled up into one rating. Those vectors would include, value for money, collaboration, engagement, communication, timeliness, quality, fit for purpose, strategy alignment, business alignment, innovation, delighting the customers (internal and external) and would be measured across all departmental executives on a quarterly basis.
- Risk – this is a fascinating measurement. Of course there is the quantification of any risk associated with Security, Disaster, or Compliance but let’s not forget the risk associated with software changes. Too much lip service is paid to this kind of risk and its evil twin “complexity”. Serious tracking of risk and complexity of releases is essential and their trending needs to be carefully monitored. Incremental increases in risk and complexity are significant multipliers on development, testing and deployment time frames and failure to acknowledge this leads to overruns of time and budget and compromising to address those impacts quality. Software change risk is a clear indicator of poor early stage planning and lack of communication and collaboration between IT and the business.
- Employee Satisfaction – Don’t underestimate the stress on employees. When there is stress there is staff turnover, loss of institutional knowledge and resulting drop in productivity and quality.
And if I could only have two it would be number 3 and number 5. Customer satisfaction and employee satisfaction are the best litmus tests on what we do. Ultimately they are all that matters.