Technology staff use the scorecard’s lower layers to track the status and performance of each service offering’s technology components, such as e-mail applications, directory services, WANs, LANs, servers, storage systems, and so on. Technology component performance is what IT shops traditionally are concerned with and what their internal systems traditionally report. Mixing the two can lead to confusion and mistrust. Tell a user who could not get into e-mail Tuesday that e-mail was 100 percent available last week, and you will know what I mean. He sees e-mail as being down on Tuesday; the IT manager sees e-mail as up and running. Why? Because the network, which is supported by a different IT group, was down. The result: irreconcilable differences.
High-level summaries with data that “drills down” into certain details are common in IT scorecards. What is unusual about our scorecard is the nature of the layered approach. Items at each level have been carefully selected for a particular audience. Indeed, creating a scorecard for multiple audiences with such diverse information needs meant we had to do detailed customer segmentation analysis to understand what key performance indicators users of the scorecard wanted to see monitored. Once the key performance indicators were understood, it was relatively easy to identify the performance measures. With a clear understanding of our audience, we were well positioned to identify the data various constituencies needed and develop the right detailed data definitions for our scorecard.
At our company, we knew that if a multiyear IT scorecard project was to succeed, we needed to:
- Keep the communication program about the scorecard project in high gear, and keep pressure on the participants to produce.
- Ensure that all senior IT managers were publicly enthusiastic about the project (even if they were skeptical in private).
- Introduce (almost monthly) new functionality/features in the scorecard. Each change can be small; the advantage of gradual improvements is that mistakes will be smaller and easier to handle, so there’s less risk of a grandstand collapse. Steady improvements can also reduce the project’s dark days when enthusiasm and support wane.
- Maintain users’ support though formal communication and demonstrations, informal dog-and-pony shows, and hands-on interaction. We try to emphasize the value of the scorecard and manage expectations. As is always the case, it is better to underpromise and overdeliver.
To date, we are pleased with the results. Our IT scorecard provides on a single page top-level information about our basic service offerings plus financial, customer satisfaction, and human resources information using a stoplight metaphor. (Green is good, yellow means caution, and red indicates there’s a problem.) The colors are all backed up with detailed data (numeric where possible).
Level Two reports on the same 14 areas, but in greater detail — approximately 140 data points — using the same stoplight format. Together the top two levels comprise the organization’s summary scorecard, which is sent to company senior management each month.
Starting with Level Three and down through the lowest levels of the scorecard, the focus shifts to technology components. The data is numerical, and it is linked to numerical targets. At the lowest levels, the scorecard is organized for use by specific IT managers, and can involve thousands of data points. All information is available to all IT staff. Critical to all levels and all data points is a set of definitions that include the information about what is being measured and what the service level targets are.
For us, the next steps are quite clear. We are tightly integrating our monthly scorecard with numerous daily and near-real-time scorecards, monitors, and event management systems to prevent dueling data. We plan to enhance the scorecard’s ability to predict rather than just report on status. More sophisticated trend analysis means our databases will have to better capture changes over time and draw the proper predictive conclusions.