For many organizations, measuring the performance of the information systems divisions is a frustrating exercise. The problem centers around confusion over what should be measured and how, as well as uncertainty over how to define value. That value concerns senior managers because IT is consuming greater shares of the overall budget. In some cases, growth in IT expenses is exceeding revenue growth.
In the COMPASS IT Strategy Census 1996, conducted by the London School of Economics, participating organizations reported average IT expenditures of 7 percent of total costs. This proportion is 17 percent higher than in 1993, and represents a 250 percent increase in spending since the 1970s and 1980s, when IT expenditures were only about 2 percent of total costs.
What constitutes ‘value’ in IT?
IT is valuable when it provides something useful or important to the organization at a fair price. COMPASS believes that IT demonstrates its value to the organization in the following four ways:
- By delivering operational efficiency to the business units. This has been IT’s traditional role. Few would disagree that without information systems, labor costs would skyrocket at many organizations.
- By enabling new ways to accomplish business goals. Without information systems, for example, banks simply could not offer 24-hour banking and overnight shipping providers could not allow customers to track their own packages.
- By ensuring competitiveness. Handheld billing terminals, for instance, became such a convenience for one car rental firm’s customers that competing firms were forced to adopt the technology. IT also enabled discount retailers to achieve the inventory reductions that make their competitive pricing possible.
- By creating wealth. While this is the most elusive form of IT value, it is achievable. Examples include SABRE, the airline reservations system, and Ryder Truck’s logistics system — technology that not only generates revenue, but ultimately is as valuable as the businesses it serve.
Traditionally, IT’s efforts were aimed at increasing operational efficiency for the business units. Later, IT units also began to be judged by the effectiveness of what they do. Both effectiveness and efficiency are measurable forms of value in IT. Efficiency is measured through cost and productivity analysis. IT effectiveness is measured through objective analysis of areas such as business efficiency cycle time, product quality and customer satisfaction.
The newest development is to assess IT’s contribution to bottom-line revenue. Unfortunately, such measures have proved elusive. For example, no one yet knows the dollar value of a web site or sales force automation or data warehousing and mining. Neither is it clear what return to expect from systems that connect businesses with clients. Whether enterprise resource planning systems are worth the expense remains to be seen.
Of the three forms of value now attributed to IT — efficiency, effectiveness, and revenue impact — COMPASS believes only one, efficiency, can rest wholly on IT. Effectiveness and revenue impact are really issues of core business units such as sales, production, and customer service. While IT remains responsible for the value it gives to the business units that are its customers, it is these business units that are ultimately responsible for the effectiveness and revenue impact of the initiatives they drive.
The Right Metrics
As noted above, no proven models exist for gauging IT’s revenue impact. However, models do exist for efficiency and effectiveness.
To measure efficiency, you can look at the IT unit alone. IT efficiency is best measured using comparative analysis. This method examines unit costs, personnel productivity and service levels against best-of-class reference groups to arrive at a baseline and identify action items for improvement.
To measure effectiveness, you need to look at how the business unit uses IT. Therefore, effectiveness is best quantitifed using techniques upon which IT and the appropriate functional units agree. Two possible models are the economic value add (EVA) and balance scorecard systems. Others are specifically designed for IT-based projects, such as the applied information econimics, information productivity and options models systems. Some models measure both. Comparative analysis based models have been developed to analyze both business efficiency and the role of IT in achieving productivity and quality in business processes– in other words, IT effectiveness.
Value and service levels are intertwined. IT creates value by providing useful or important services at a fair price. So an essential element in tracking the value is identifying and measuring the service levels provided.
Lessons from the Outsourcing Primer:
- Efficiency is the only form of value that rests wholly on IT.
- Efficiency is best measured using comparative analysis.
- Effectiveness can be quantified using existing models.