In November 2012, the US Air Force finally decided to cancel its Expeditionary Combat Support System (ECSS) modernization project after spending US $1 billion on it. ECSS was intended to replace more than 240 outdated Air Force logistics computer systems, some over 40 years old, with a single, integrated system. The Air Force deemed the effort critical to the successful modernization of its antiquated and operationally costly logistics infrastructure. However, in April 2012 the Air Force’s comptroller told the US Senate Armed Services Committee, “We’re now approaching seven years since funds were first expended on this system…. I’m personally appalled at the limited capabilities that project has produced relative to that amount of investment.” The Air Force’s ECSS project leadership offered various excuses for the project’s failure, but the reasons all boiled down to the claim that they did not have adequate insight into the project’s implementation.

The Air Force’s claim of ignorance is curious given that there was a government mandate to provide detailed project status information to the US Department of Defense’s CIO, who in turn was required to review it and then post said information on a government-wide, publicly available IT dashboard along with her personal assessment of the ECSS project’s risk status on a quarterly basis. A quick look at the dashboard shows that the project was rated as only a moderate risk at its demise and had been “closely monitored” by the CIO for the past two years.

A better example of an ineffective IT dashboard for making project decisions would be hard to find.

The ECCS debacle leads to the interesting question of why a reasonably planned IT project using a dashboard would fail. (A “reasonably planned IT project” is defined as one that is technically feasible and has adequate resources of time, money, and people.)

Setting aside project suicides as a possible explanation, there really seem to be only three possibilities. The first is that project management doesn’t want to hear bad news; therefore, bad news doesn’t show up on the dashboard, or if it does, it is ignored.

The second is that project managers don’t understand what the information on the dashboard is telling them. This is always a possibility, especially if the dashboard provides too much information or presents it in a way that is confusing. Still, that’s unlikely for anyone with a minimal amount of professional project management training and a bit of project management experience.

The third is that the IT project dashboard doesn’t provide meaningful information to the manager responsible for the project. (Note that when I say “project manager,” I am including anyone on the project who has decision authority.) I further discuss what I mean by “meaningful information” in the Cutter IT Journal article “A Case for Decision-Focused Dashboards“, but we should remember that a project manager’s sine qua non is to monitor project status and, when required, to make decisions regarding changes or deviations to the project plan. If a project plan were perfect, you wouldn’t need a project manager.

But what is a project plan? A plan is merely a proposed sequence of activities to be carried out over a period of time to achieve some set of feasible objectives, given specific constraints and assumptions (which we call the project’s context). When the project’s activities or context deviate from what is expected — say, the activities were harder to complete than expected, the context changed, or what have you — then it is management’s job to intervene and try to adjust the project plan to achieve the original (or recalibrated) objectives given the changed set of circumstances. (Deviations may also be caused by opportunities to gain increased benefits from the project in less time and/or with less cost.)

avatar

Robert Charette

Robert N. Charette is a Fellow with Cutter's Business Technology Strategies practice. He is also President of ITABHI Corporation, a business and technology risk management consultancy. With 35 years’ experience in a wide variety of international technology and management positions, Dr. Charette is recognized as an international authority and pioneer regarding IS, IT, and telecommunications risk management.

Discussion

  4 Responses to “IT Dashboards: Part of the Solution or Part of the Problem?”

  1. Large organizations like the DoD run many multiple year complex IT projects. There are all sorts of quality standards for project management used. This seems to be a clear example where the focus was more on the ceremony of the process than the intent of that process.

    An effective IT dashboard should be risk-oriented and have good measurements for % completion. Internal organizational incentives can lead many to report status of something as “complete” when it is not yet complete. You combine the wrong risk measures with ineffective project management and senior management is blindsided when the first major milestone whose failure cannot be glossed over is hit.

    You would think that after so many budget overages and lack of meeting project timetables for ERP in government, particularly in DoD, that there would be a set of established early-warning metrics!

    • avatar

      Doug,

      Thanks for your insightful comment. I fully agree- however, in government and in DoD in particular, the focus is mainly on maintaining program cost and schedule “in the green.” A program goes “yellow” if it hits a 10% over plan in either category; take a look at the number of IT programs on the government IT dashboard where cost or schedule are 9.8% or 9.9% over plan.

      The important thing to remember even with dashboards is novelist Upton Sinclair’s observation that, “It is difficult to get a man to understand something when his salary depends upon his not understanding it.”

      • Bob,

        I guess my point is that tracking cost and schedule as scorecards is not an effective strategy in the case of DoD. They should be monitoring whether any of a set of risk factors are becoming apparent regardless of % complete. That’s the kind of early warning that one needs and the proper risk management approach to monitoring projects.

        • avatar

          Doug,

          We’re in violent agreement. The sad fact is that, even after 35+ years of DoD programs being told (by law) to explicitly manage their spectrum of risks, few do so in more than a tick-in-the-box manner. Until there is an actual penalty for not managing risk, as opposed to the all too common situation where highlighting risk is more often than not a career killer, nothing is likely to change.

          Bob

 Leave a Reply

(required)

(required)

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>