For years now, I have made a good living by exploiting Geary Rummler and Alan Brache’s famous subtitle, “How to Manage the White Space in the Organization Chart” (Improving Performance: How to Manage the White Space in the Organization Chart, Jossey-Bass, 1995). What Rummler and Brache meant by white space in this case was all the activities for which no one was explicitly responsible. My experience, which mirrors theirs, is that those things that fall between the cracks often get overlooked and undermanaged. Much like “white space” is the physics notion of “dark matter.” Dark matter is that unobservable “stuff” that physicists have computed exists in the universe that causes known celestial bodies to behave the way they behave with respect to our most fundamental physical laws.

In both white space and dark matter, there is an understanding that something important is being overlooked that is only known indirectly through some observed effect on some other body. The same is true, I have concluded, in the way we understand the software systems and data that exist in the average large enterprise. CIOs and CTOs constantly refer to the major applications and databases known to them. However, if one examines the difficulties in replacing or even making major improvements or modifications to those systems, one has to reason that there are dozens of small (and sometimes not-so-small) undocumented systems (“dark systems,” if you will) that exist in the white space between the major systems.

Any knowledgeable systems manager, especially after a drink or two, will acknowledge that there are indeed large numbers of small applications that orbit around the acknowledged large systems as well as hundreds — sometimes thousands — of undocumented databases and interfaces that exist as well. In my EA classes, when I refer to some of these dark systems as “mission-critical spreadsheets” or “phantom ACCESS databases applications,” my students nod knowingly. Increasingly, we hear the term “spreadmart.”

A great many of these dark systems have been developed without the knowledge, much less the assistance, of the central IT organization. A fair number of the major dark systems in most large organizations include the business intelligence applications that exist to support marketing and/or financial functions and which have their own developers, employees, or contractors who are often not classified as programmers or developers but as management analysts or general consultants.

My experience also shows that large organizations rarely face up to the number of dark systems that exist in their systems’ white spaces (or see how critical they are) until there is: (1) some major failure, (2) an urgent desire to replace some core system, or (3) some key person leaves (or dies), and it is discovered that some heretofore unknown system exists that has little or no documentation. This is not surprising. Central IT usually has enough on its plate without looking for more work and more organizational fights. However, failing to adequately understand just how central many of these “off-book” systems are creates a biased picture of the organization’s true systems and database landscape. In a future Enterprise Architecture Advisor, I’ll address what needs to be done to bring all these dark systems into the light.

avatar

Ken Orr

Ken Orr is a Fellow of the Cutter Business Technology Council and Government & Public Sector practice and a Senior Consultant with Cutter Consortium's Data Insight & Social BI, Business Technology Strategies, and Business & Enterprise Architecture Practices.

Discussion

  4 Responses to “White Space, Dark Matter, and Enterprise Architecture”

  1. Great article. I run into this all the time with our customers and prospects looking to buy an enterprise pricing system.

    I believe this is caused by some or all of the following:

    - I.T. systems and groups lacking the agility to keep up with the business team
    - The world is changing quickly, at an ever increasing pace
    - Competitive landscapes change rapidly
    - Excel is such a ubiquituous tool and is easy to use, but yet still very powerful

    To address this problem, IT will need to become more nimble. One example of a system that I frequently run into is SAP. If you ask an IT owner of SAP how long it takes to make a change, you will get an answer that makes a business owner cringe.

    Once again, thanks for the insightful and well-written article!

  2. You’re right on track. I think that it is a bit broader and I hope to focus it a little later, namely that we need to provide intelligent (and not so intelligent) users a “safe way to develop their own applications”. Today, the two top choices are Excel as you said and ACCESS, which are not tracked by IT. (This is going to get worse, just look at Power Pivot that MS has just released for manipulating multi-million row tables).

  3. avatar

    Ken,

    Ironic that you mention ‘Dark Matter’ in your note above. I have been writing about the ‘Dark Matter of Enterprise Architecture’ for years… and Cutter Consortium just published my Executive Report on “Realizing the Value of your Business Driven EA”. The link is here for your reference: http://www.cutter.com/offers/busdrivenEA.html. PLease feel free to contact me if you have any questions.

    Thank you.
    Skip

  4. avatar

    Hello,

    Good post. It’s right that many times we run into such “dark system” as you call it. I would say, at the first sight, that as an EA myself, this generally piss me off. But, then we have to consider the flexibility needed by business that central IT is usually not able to bring. So, if we look at it with pragmatism. There will be always such “dark system”, what is important to take care off is that they are not on the “critical path”. What I mean with “critical path” is that they are not directly used to critical processes that deliver the product to the customer. For instance, it is OK that R&D has such tools when it comes to verify calculations or list CAD models to build their digital mock-up… But if these “dark system” are involved in the production flow. If they are jeopardizing the delivery to the end-customer, then it is (too me) not acceptable.

 Leave a Reply

(required)

(required)

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>