Sep 242013

The desire to be agile has long impacted human behavior. Consider the elite athlete, the army general, the opera singer, the belly dancer, the professional golfer, the heavyweight boxer, the high seas sailor, the commercial pilot, the top-end banker, and even the federal politician — they all love agility, and so do we. Why? Put simply, agility provides the basis for adaptability and change which, in turn, are integral to our survival and growth. The same agility that enables a springbok to outrun a lion or an ant to carry a load more than 20 times its size allows a small start-up in Southern California to prevail against the might of a large, well-established brick-and-mortar organization (based on my arguments in the preface to my book, The Art of Agile Practice: A Composite Approach for Projects and Organizations). No wonder mainstream business is increasingly fascinated by “Agile.”

In a special issue of Communications of the ACM commemorating the first 50 years of computing, virtual reality pioneer Jaron Lanier wrote: “The biggest surprise from the first 50 years of computers is that computation turns out to be a cultural object in its own right, with all its warts and semicolons.” This phenomenal importance of “human issues” in IT project management has also found its way in discussions by Cutter Fellows Tom DeMarco and Timothy Lister in their book Peopleware: Productive Projects and Teams; Gerald M. Weinberg’s The Psychology of Computer Programming, and Larry Constantine’s Constantine on Peopleware and Soft Issues and Other Hard Problems in Software Development. Indeed, Constantine claimed: “Good software does not come from CASE tools, visual programming, rapid prototyping, or object technology. Good software comes from people. So does bad software.” I have also discussed the importance of people in software projects and the destructive nature of “game playing” — together with suggested antidotes. These discussions led to an inescapable conclusion: the dire need to address the social and cultural factors in project management. Contemporary Agile emerged out of the exploration of such issues in software development projects. Agile helped the software development community climb out of its cellars of up-front planning, analysis paralysis, and siloed (primarily driven by the waterfall lifecycle) approaches to the users and business.

Unlike construction and manufacturing, which are “linear” in nature, software projects tend to be “fuzzy” and continuously “changing.” I consider their nature as “amorphous,” with the output not as clearly visible and measurable as that of building a house or manufacturing a car. Yet, the project management frameworks (e.g., PMBOK and Prince2 ), techniques (e.g., WBS, Pert/CPM), and corresponding tools (e.g., MS Project) are all based on a “linear” engineering paradigm. Applying engineering principles to software projects helped in early information systems days but, with growing size and complexity, the same principles led to many challenges (such as those very well documented by Robert Glass in Software Runaways: Monumental Software Disasters). Experience suggests that the nature of software projects is far less organized than construction and manufacturing. This could be one of the main reasons why the field of software development picked up on Agile — because the many subdisciplines of software are closer to art than engineering (see The Art of Agile Practice), and Agile is much better suited to this type of work. While we do have many right-brained subdisciplines of computer science and wireless networking, when it comes to developing software there is very little engineering in it. Besides, the “success” criteria of the software output is heavily dependent on the subjective viewpoints of the user. Furthermore, the needs and the wants of the users of software systems are continuously changing and demand continuous attention.

Figure 1 depicts a simple mindmap of Agile, which is based around the Agile Manifesto. The Agile Manifesto encompasses key values of Agile: trust, simplicity, honesty, and courage, amongst others. A cursory glance at these values tells us that these are universal human values, and that they do not pertain to anything specific to software. In fact, it will be hard to find a plan-driven hierarchical project manager who would not want these values in his or her project. The Agile movement deserves our gratitude for extracting these human values and applying them nicely to the field of software development. We can credit the successes of Agile-based projects to these characteristics of individuals, their ability to collaborate, and their capacity to take on and absorb change. However, the Agile practices and methods (mindmapped in Figure 1) are not entirely right-brained; they do need certain planning and discipline (left-brained activities) for Agile to succeed in practice. The need to understand and apply Agile as a balanced psychosociological paradigm could not be greater.

Figure 1 -- A mindmap of Agile.

Figure 1 — A mindmap of Agile.

While the roots of Agile lie unequivocally in the IT domain, the popularity of Agile transcends the software development space. This places responsibility on us, the IT community, to understand the psychosociological basis of Agile. In what ways do you think Agile would be well served by exploring the connections between knowledge, culture, psychology, and technology?


Bhuvan Unhelkar

Bhuvan Unhelkar is a Senior Consultant with Cutter Consortium's Business Technology & Digital Transformation Strategies practice. He has more than two decades of strategic as well as hands-on professional experience in the information and communication technologies (ICT) industry


  6 Responses to “Sociocultural Aspects of Software Projects”

  1. If Glenn Vanderburg, Real Software Engineering is correct, the “engineering principles” often associated with the Waterfall process don’t even work for several (other) branches of engineering!

  2. Thank you for a concise discussion that once again highlights aspects about the special nature of software development. But let us not forget that the circumstances under which teams operate when developing software are different – depending on the type of result required.

    The following statement is made in the article: “Furthermore, the needs and the wants of the users of software systems are continuously changing and demand continuous attention”. This is often the case – and when this is so, a well executed, Agile response is certainly called for.

    But not all software is written to satisfy the changeable needs of users. Think about software that interacts with an electronic highway when receiving and transmitting data to and from other electronic devices. In these situations, the “user” is an electronic device and the level of change is often dramatically lower. For this type of software development, it is quite feasible to follow the plan-driven approach and deliver software that can be verified and validated against baselined requirements.

    I prefer to speak about “Situational Software Engineering” where different situations require different approaches: from Agile, Crafted Quality on the one hand, to Plan-Driven, Controlled Quality on the other as depicted in the Complex-Adaptive Situational Model (CASM).

  3. @Barry, I couldn’t find much about CASM or Situational Software Engineering, so I will ask my questions here.

    1. What are the differences between Crafted Quality and Controlled Quality?

    2. How does a CASM-compliant Plan-Driven process respond to deviations from plan, e.g. a task was missed in initial planning, or took twice as long as estimated?

    • @Robert. In response to your questions:

      1) One of the crucial differences:
      Crafted Quality can be likened to being set a task of building a puzzle – but without having access to the picture on the box. So our understanding of the picture being built emerges as we manage to assemble pieces of the puzzle. (And we run the risk of having to re-work the puzzle when we realize that earlier assumptions about what goes where are wrong).
      Controlled Quality situations are like building the puzzle while being able to organize and track progress against the up-front understanding provided by the complete picture.
      The picture on the box is a metaphor for the vision and architectural understanding of the desired result.
      So in Agile situations (Crafted Quality), we usually have less well developed understanding of the ultimate result.
      In Engineering situations (Controlled Quality), it is possible to work from well developed and understood descriptions of the final result.

      2) Responding to deviations to plan:
      By implementing efficient and effective progress monitoring and also change control. And when required, by changing baseline plans to accommodate the effect of the change.

      • Thank you, Barry.

        Here’s a follow-up question regarding Situational Software Engineering.

        What if the deviation from plan is happening because the estimate on which the plan was based is in error? From the sponsor / buyer / customer perspective, there’s no change to be managed or accommodated. If there are already strong expectations about scope, schedule, and budget (whether contractual or associated with a corporate authority structure), what happens to the plan?

        I ask because this is at the root of many troubled projects.

        • I agree that many troubled projects have to deal with the dilemma of initial estimates not being accurate enough. But I suggest that most of the projects to which you refer are not Controlled Quality projects.

          The CASM model defines 4 domains based on 2 dimensions: Management Governance (MG) and Production Governance (PG). The MG scale ranges from High Ceremony to Low Ceremony, whereas PG ranges from Engineering to Organic. So Crafted Quality (Agile) situations come about when Low Ceremony MG and Organic PG apply. Controlled Quality results from High Ceremony MG and Engineering PG.

          In Controlled Quality situations, the project budget would typically include an element of Management Reserve. And this would serve to absorb most of the consequences of inaccurate estimation. But sometimes the need for extra budget will be because of additional customer requirements (scope creep). And then the mature approach to change control will result either in the customer being willing to pay for the increased scope, or the proposed changes being set aside.

          When High Ceremony MG and Organic PG apply, we experience a situation that CASM describes as “Managed Costs”. This name was chosen based on my experience of situations where management focuses on the budget while technical teams focus on delivering results as fast as possible. Management is in fact, quite out of touch with technical work actually being performed – but as long as expenditure is within budget, they feel that the project is on track. In these (somewhat dysfunctional) situations, management expect estimates to be highly accurate (an oxymoron) and they resist attempts to increase the budget.

          Steve Pieczko described the Managed Costs domain as “WetAgile” – suggesting a transition from the Waterfall of true Controlled Quality to true Agile. (Still dripping from the Waterfall, trying to be Agile).

 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>