Read more]]> Big Agile is “agile as far as the eye can see.” It is not “one Big Agile organization.” The distinction becomes clear when you consider the context of size: team versus whole organization. While it is certainly possible, and desirable, for a company as a whole to be influenced by agile practices and principles, that doesn’t mean that agile alone can make a whole company move more quickly and easily.

Agile, as both a theory of management and of software engineering, is tuned very well for one to several teams of individuals working closely together. Beyond that scale, agile alone won’t address all organizational challenges — no agile transformation initiative alone could do so. Solving larger-scale problems will require other management and engineering methods, some of which I discuss below.

When companies grow beyond a handful of individuals or teams, many changes will affect the organization of work and people. One of the first results of large scale is decreased information sharing, both across teams and for individuals joining existing teams. At small scale, and for a limited period of time, a team can organically “remember” its own history. Beyond that small scale, though, organizations need to promote mechanisms that encourage learning and sharing across the organization.

Mentoring

Today’s organizations are heavily focused on knowledge work — the creation and application of knowledge. It is truly the people that are the most valuable resources. Learning can’t be rushed or accomplished in a quick orientation. The 10,000-Hour Rule says that it takes approximately 10,000 hours of deliberate practice to master a skill. That is about five years of working full time at something in an environment where feedback and progress can occur.

Mentoring is a mechanism for long-term leadership and growth. It encourages learning and transfer of knowledge within and between parts of an organization. Mentoring is used extensively in Japanese management, and it is the foundation of craft and artisan learning. In such organizations, everyone has a mentor who assists and actively guides them.

Value Stream

Beyond the challenges of sharing information effectively, organizations at scale frequently structure into functional silos. This is often the result of focusing on efficiencies of scale in the attempt to keep individuals fully utilized and working on the most appropriate tasks. This functional efficiency comes at the cost of cycle time and responsiveness, though, and it doesn’t account for the cost-of-delay economic tradeoff.

Value stream-based organizations will optimize for the flow, and reduced cycle time, of valuable product features. Instead of trying to maximize efficiency of any one function, this approach measures the whole system while trying to reduce total time to delivery. In this method of managing work, deep queues of work don’t exist to cause significant delay.

Aligning toward the customer and stream of value promotes another benefit for individuals and teams: it is easier to define a vision and more motivating for people to work toward that vision when they can connect themselves with delivering actual value to real customers.

Risk Management

Planning work at scale involves committing resources with energy and momentum behind them, often with a fairly detailed budget for a fiscal year or more. This commitment creates an inertia that is a particular challenge for agile teams, which by definition assume the future is going to change, yet are pressured by a static plan.

Risk management can be applied during yearly planning cycles to projects and technical investments alike. Assuming “economics is an exchange of risk and opportunity,” we can conclude that budgeting and financial planning activities should account for the temporal nature of both risk and opportunity.

At smaller scales, risk management can be as simple as creating what-if scenarios, but as investment and commitment scale, so too should risk management. Such topics as technical debt, new or old technologies, competitive landscape, and so on, should all be represented in the planning cycles to mitigate the risks posed by the unknown. It may be very appropriate to take a risk, but it is never appropriate to forget about the facts — your economic outlook depends upon it.

Real Options Theory

In addition to avoiding or mitigating risks, deferring the planning or execution of work until the appropriate time is a planning option that can be used effectively at any scale. Real options theory holds that options have value, and options expire. Never commit early unless you know why.

Options thinking encourages us to delay commitment instead of trying to lock things down as soon as possible. This delay improves our position by giving us time to acquire more information and, when it’s time, make a better decision. This style of planning blends well with agile teams: both the decision making process and execution evolve together over time.

Evidence-Based Management

Finally, in a large-scale environment where it’s all too easy to be less connected with customers, colleagues, and other teams, it is even more important to pursue empirical data of what is truly needed and actually happening.

Evidence-based management is the “conscientious, explicit, and judicious use of current best evidence in making decisions.” There is a long pedigree of methods based on this idea:

• W. Edwards Deming taught using data to make decisions in TQM.
• The Toyota/Lean process has both “go to the source” (the Japanese word gemba means “source,” or where the work is actually being done) and data-driven continuous improvement toward target conditions.
• OODA/maneuver warfare teaches both gathering information and making decisions with imperfect information. (John Boyd defined the OODA [Observe, Orient, Decide, Act] loop for fighter pilots while a member of the US Air Force. It is now a theoretical basis used in maneuver warfare.)
• The Lean Startup method is strongly focused on growing knowledge through experiments.

This is the scientific method applied to business and management: have an idea, experiment to prove it out, then either alter the idea or invest in it. Agile teams are very good at doing exactly this sort of experimentation and changing direction based on the results. Overly lengthy and detailed planning horizons or “hope-based” management (which skips right from idea to investment, skipping over any real validation) often leads to death march projects.

Read more]]> We believe leadership is just as much a definable science as management. What has made this notion difficult for most people to grasp is that leadership is seen as being something someone is born with (or not). In addition, management appears more “concrete” to people than leadership. Management is something that they can get their hands around because it is largely about following a set of defined processes.

We would argue that most C-suite executives are selected on their management skills, not their leadership skills, which is why there is a dearth of leadership across both corporations and government. This has occurred in large part due to the fundamental reengineering of organizational structures, operations, and finance that began in the early 1990s and continues today. Organizations are now increasingly organized around short-term projects. Instead of being hired to work for possibly a lifetime in a company, people are hired to work on a project that has a definite lifespan. Once the project is over, individuals are likely to be let go unless they can find work on a new project. According to US government statistics, up until the recent economic troubles, the years a person worked for the same employer has, for more than a decade, been steadily dropping for all age groups except those over 65.

The growth of the project-oriented organization has meant that the acquisition of management skills is seen as the key to career advancement. The other career path is to become a specialist in marketing or sales, or in the financial/human resource management aspects of a company. In earlier organizational structures, where managers could learn leadership skills as they advanced through their careers, there was more opportunity to acquire the experience needed to develop robust spoke behaviors and generalized rim skills. In today’s working environment, specialization is rewarded, whereas being a generalist is undervalued.

The opportunities to learn to become a leader have dwindled precipitously, which means that the deliberate nurturing and selection of individuals for leadership positions is more important than ever. A recent Chief Executive article stated that the best companies for leaders, which generate dramatically greater market value over time than the companies weakest in leadership development, make leadership development a very high-priority commitment despite the current economic situation. Leadership development pays.

Developing the IT Leaders of Tomorrow

A recent article in the Wall Street Journal asked, “Do Techies Make Good Leaders?” The answer was yes, but there are a number of unique challenges the IT industry faces in comparison to others. For instance, the IT industry is full of young people with strong backgrounds in science and technology. They often like working on technical problems more than they like working with people, yet leadership is, by definition, working with people. Surely this makes the development of both leaders and managers more difficult. A 2005 Cutter survey on IT leadership supported this view: the lack of empathy (a spoke behavior), lack of emotional ability (a hub trait), and lack of interpersonal communication (a spoke behavior) were rated as the top three failings in IT managers.

In addition, the IT industry tends to hire (and promote) the person with the best technical skills for a given project. Little thought is given to what is required to develop future corporate leaders. As in other industries and in the HP example, those promoted to the IT C-suite are more likely to be from marketing or finance than engineering, since these individuals are perceived to be more capable of leading (and growing) an organization than someone from engineering, computing, or another technical discipline.

Indeed, few IT organizations have internal programs that can objectively evaluate and enhance leadership — as opposed to project management — skills. Although a previous Cutter study has shown that virtually all attributes significant to the profile of a leader are observable and quantifiable, in the absence of a formal evaluation process, simply knowing what to look for may in itself yield large benefits in terms of organizational performance.

Read more]]> Quite a few clients report that agile is anti-innovation. The developers have a vested interest in developing whatever they can produce within the allowable time. They are rewarded for maintaining the velocity of the project, not for their innovative solutions. Note that innovation, as we use the term here, means fresh thinking. We do not mean that innovation is the same as invention — it’s not. Innovation is thinking differently about the business problem with the intention of finding more beneficial things for the business to do.

User stories that are not based on real business stories will struggle to be innovative. The user story describes what happens at the interface and is mostly what the product owner thinks the user wants to do with the software. But without some innovative thinking, it is all too easy to provide just some incremental improvement and let it go at that. Let’s look at an example of a user story that was written without any real concern for innovation or the real need that the story is meant to satisfy: “As a bank account holder, I want to check my balance online.”

At first, this might seem a reasonable and obvious story. However, it can be made a lot better. Some authors suggest that the “so that” part can be omitted. We suggest very strongly that you always include the reason in your stories. While the Agile Manifesto favors working code over documentation — and there is a lot to be said for that — there is nevertheless a need for the development team to leave behind a guideline of its thinking. Without justifying the requirement — that means including the “so that” — future maintenance teams are deprived of a valuable clue as to why a particular requirement was included in the software product and, hence, what would be the effect of changing it.

Our first question is, “Why does the account owner want to check the balance?” Let’s revisit the story and this time look at the reason given for the requirement: “As a bank account holder, I want to check my balance online so that I can access my daily balance 24 hours a day.”

This is not exactly a good reason for checking the balance. The “24 hours a day” is slightly more enlightening but doesn’t really do more than tell us that the account owner might have nocturnal habits. Why does the account owner wish to check the balance? It is not something we do for fun, so there probably is some business reason behind it. We just don’t yet know what that is. Let’s make some conjectures.

Suppose the reason for the frequent checking is that the account owner is on a tight budget and is concerned about becoming overdrawn. If so, the owner of the product, presumably a bank, has the opportunity here to be more effective and at the same time provide a better service. Instead of the account owner having to repeatedly check his or her balance to see that it’s not going into the red, it would be better to build a feature that notifies the account owner if the normal monthly payments such as rent, electricity, school fees, and so on, will reduce the account balance to zero or beyond: “As a bank account holder I want to be informed if my monthly balance is projected to go to zero or below so that I can arrange for an overdraft.”

Furthermore, we suggest that it is far more useful for the account owner to be periodically informed of the amount of discretionary money in the account: “As a bank account holder I want to be informed of my projected balance after all regular monthly payments have been deducted so that I know how much I can safely spend.”

Having a feature that lets the account owner check the balance online is the simplest feature to have. However, we suspect that without any kind of business thinking, or innovative thinking, it would probably turn out to be a feature that did not solve the real problem. If other banks solve the real problem (i.e., they understand the real needs of their customers and offer this service to attract more customers), then the original story could hardly be said to providing real business value.

When a business analyst investigates a business story, he or she is consciously trying to encourage innovation by first making abstractions that uncover the real business problem. Figure 7 illustrates the skill of looking at the same business story from several different points of view and thereby discovering undreamed of innovations that could make a significant difference to the business.

Figure 1 — The Brown Cow model illustrates four points of view that help to uncover the real business problem and identify useful innovations.

In the bottom-left quadrant of the Brown Cow model 6 the viewpoint focuses on how things are done now. This is commonly what business people ask for — a little bit more of what we already have. In the bottom-right quadrant the viewpoint focuses on how things could work in the future. Once again, it is likely that someone who has a requirement will express it in these terms. In other words, like our nurses in the previous example, rather than asking for what they really need, they ask for a solution.

As part of his or her analytical skills, the business analyst also learns to explore the problem in a solution-neutral fashion. We refer to this as the ability to “think above the line.” The line in this case is the horizontal line that separates the top and bottom quadrants of the model — the abstract thinking about the real problem from the technological view of the solutions.

The top-left quadrant focuses on what we do now, independent from how it is done now or how it might be done in the future. This view uncovers the essence of the problem: the business rules and the business data that has to be there independent of any solution. Above the line, the business analyst exposes the real business requirements by stripping away all solution-oriented aspects and thereby coming up with a policy-only statement of what the business is really doing. The top-right quadrant of the model is where the business innovation happens. It is here that the business analyst can make suggestions about improving business rules or using existing business data to be able to make better business decisions.

Innovative business value discovered by this sort of innovative thinking are then included in the business stories. These are innovations that could never be discovered by focusing on a software interface.

Read more]]> A resilient organization aligns its strategy, operations, management systems, governance structure, and decision support capabilities so that it can adjust to continually changing risks, rebound from disruptions of any type including those that involve primary earnings drivers, and create advantages both through ability to respond and through beneficial changes brought about by absorbing new learning. It should be able to withstand the widest range of threats, including natural disaster, hazardous economic and market conditions, fraudulent employee behavior, IT infrastructure failure, disruptions of independent supply chains, disruption of customer channels, intellectual property theft, inability to respond to emerging conditions, and a host of other factors. Resilience should accomplish several objectives that are only achievable by combining a unified view of risk management with flexible and rapid response. Among these objectives are:

• Protecting the business from change and mitigating or benefitting from any change that might occur
• Mitigating risk that arises in any area of business and technology
• Identifying and decreasing reliance on nonresilient programs, processes, and procedures
• Enabling the business to adaptively respond to internal and external pressures
• Strengthening and enhancing business processes by improving governance and flexibility
• Using disruptions to improve efficiency and develop more effective responses, rather than merely focusing upon survival
• Reducing insurance costs and exposure to uninsured losses by providing enhanced oversight and a better understanding of response

The goal of resilience is not just recovery or continuity, but transformation from reactive to proactive and then to adaptive. Recovery efforts have demonstrated the value of several principles that constitute the core of business resilience, including:

• Mission focus. All resilience measures need to focus on the core business processes and values of the company at all times, ensuring the confidence of customers, partners, and suppliers. This is often critical to recovery, particularly from situations that might impact the reputation of the firm.
• Risk management. A thorough risk management program is essential, including a business impact analysis and development of mitigation strategies for key threats. Risk needs to be understood and evaluated in a unified way across the enterprise.
• Agile local empowerment. Empowerment of managers and workers who are close to the crisis, either geographically or by specific area of the threat (IT security, for example), ensure that lines of authority are known and response can begin immediately.
• Agile structural flexibility. The capability to realign organizational structure to meet new needs of the changed business or technological situation is vital. This realignment may require immediate and radical changes to existing hierarchies.
• Agile collaborative capability. The capability to cooperate with other businesses, outside organizations, and others, as needed, to meet recovery demands is also important. This is often critical to success, particularly in catastrophes that involve an entire region or business sector.
• Practice and rehearsal. Practice, rehearsal, and simulation of recovery locates weaknesses in resilience and strengthens the ability to recover. Rehearsals and simulations demonstrate the current recovery capability and prepare employees to respond to the real thing.
• Absorption of new learning. This involves the capability of learning from adversity and developing stronger and more focused business processes, using the disaster as a platform for beneficial transformation.

Resilience has become a concern of businesses operating in every sector and risen to the fore with events such as earthquakes in New Zealand and Japan, flooding in Australia and Thailand, hurricanes and flooding along the US East Coast, and fires in California. Other events have included the global economic crisis and major recent technological changes involving mobility and cloud IT.

Concepts of agility, adaptability, and resilience have been in development for at least the past four decades, with both progress and need accelerating recently in the wake of economic and technological change. These concepts, which began as lean manufacturing, have gained further validation through work on complex adaptive mechanisms and through practical application at the process level in software development, where methodologies such as Scrum and XP have provided significant benefits to companies in meeting the challenges of change.

As a result of growing concern, resilience has come under investigation over the past several years, particularly in the context of integration with existing IT solutions. Because it involves other aspects of risk management, auditing, and compliance and must incorporate the whole enterprise, we have seen the emergence of several frameworks and approaches aligned with existing management frameworks, such as ITIL, COBIT, and CMM. These frameworks, including the IBM Resilience Framework and its Resilience Maturity Model (RMM) and the CERT Resilience Management Model (also RMM), provide guidance for incorporating various elements into a resilience program.

Read more]]> In IT circles, ITIL projects induce feelings of both love and hate. While the IT landscape has many successful ITIL implementations, that landscape is also littered with cost overruns, frustrated IT staff that couldn’t focus on immediate customer demands, and dissatisfied end users whose business “technology” needs were put on hold pending completion of the ITIL projects.

The June 2012 Cutter IT Journal with Guest Editor Bill Keyworth, seeks to identify how ITIL can be used effectively to satisfy the customer goals of IT service management and how IT operations can balance the conflicting demands of IT process and business needs.

Please send us your ideas – proposals of interest are due 6 April 2012.

To respond, please visit

http://www.cutter.com/content-and-analysis/journals-and-reports/cutter-it-journal/callforpapers02.html

Read more]]> It’s transition time for our journal Cutter Benchmark Review. We can’t overstate how much we’ll miss working on a regular basis with our friend and former Editor Gabe Piccoli. We all hope to continue to work with him in other ways, whenever his very busy schedule allows. But tempering our sadness is our excitement at welcoming Cutter Senior Consultant Joseph Feller to CBR‘s editorial helm. Like Gabe, Joe is a truly engaging person and a dynamic thinker. I encourage you to read the introduction to Joe’s inaugural issue, and meet him via video, as he talks about why benchmarking no longer needs to be an idle exercise.

I know you’ll enjoy getting to know Joe. You’ll able to interact with him more, here on the blog, and of course, if you’re a subscriber, you’ll benefit from his insights in the pages of his quarterly CBR issues.

Read more]]> Governance is a fundamental (perhaps the fundamental) process within EA to connect the business aspirations with the current and future enterprise reality. Governance is probably also the most contentious EA process: a necessary evil at best or a dysfunctional rubber stamp or change-prevention mechanism at worst. The current focus on enterprise agility provides a context for refining governance. The conclusion is not to throw out governance or to diminish EA to a laissez-faire view of awareness and simplistic control of the enterprise. Rather, the conclusion is that governance can be made effective, compelling, and a value-add to agility.

Part of the complexity with governance is that it varies widely and is a tradeoff of constraints and flexibilities. The telecom industry in the early 1990s illustrates one view of governance. This was a time of great expansion of ideas/capabilities (e.g., cell phones, network features such as voicemail, data transport in addition to voice) but also a time of great constraint (50% of customers did not even have touch-tone phones — remember those circular dial faces!, and the connections were twisted-pair copper wires, not higher performance wires, cables, or fibers). At that time, the enterprise architecture — the way calls were switched, the way signals were sampled and processed by software, the way endpoint devices (phones) worked, and so on — was a set of relatively immovable constraints. This is not to say that the role of architecture (and the architects) was to limit innovation, but rather help “engineer” tradeoffs based on the current reality to encourage workable innovation and delivery. There was no doubt, though, that governance was there to enforce architectural constraints, and that the flexibilities were limited to understanding what those constraints did and didn’t mean. Outcomes included ways to deliver network services, such as caller ID, call waiting, network voicemail, and so on, to subscribers that had the most basic handsets and far-reaching features like “number portability” (the precursor to today’s ability to migrate phone numbers from landlines to cell phones or between providers). There were disruptive transitions, such as adding area codes or splitting them, that needed to be managed.

Another view of governance is that it manages the roadmap from the “as is” to “to be” architectures. The nature of the concerns can range from explicit transitions (e.g., moving from one document management solution to another) to abstract goals (e.g., adoption of XML, SOA, or REST) to operational concerns (e.g., the JDK version or App Server version used for implementation). In this view of governance, the architectural transition is driving the change, and the enterprise is shifting from a state of compliance with one set of standards to compliance with another set of standards. An audit function is performed to evaluate if each project/delivery is to either be part of the status quo or part of the managed change. This audit often occurs so late in the process that there is not much that can be done if the answer is other than what is expected. The audit and the mandate for compliance may occur at an idea level, not based on an integrated end-to-end capability that involves the details of how the transition was applied.

Enterprise governance ultimately provides answers to the following questions, and the considerations for how to answer them in an agile enterprise are significant:

• What third-party technologies are we using across the enterprise, and is that consistent with what we want to be using (vendor, version, licensing, migrations, policies on open source usage)? Agile methods use the term “situational awareness” as necessary to understand how a project is doing so that next steps can be evaluated. Governance provides an enterprise-level situational awareness.
• What are the third-party technologies being used on a given project, and are those from the approved list of technologies (including transitions)? The system-level projects may or may not be able to avail of the full set of options (i.e., variants) within the enterprise, so there is a fit-for-purpose analysis that is made. In the agile development process, that purpose and fit may emerge rather than be planned and this will need to be addressed with governance.
• When have technology choices changed on a project/system, what are those changes, and are those changes expected/approved? This is similar to the above, but emphasizes that much of the desire/need for technology transition is discovered at the systems project level. There are big ideas that might be able to flow from top down (e.g., adopt messaging over remote procedure call (RPC), or adopt document management over file storage), but within those large domains, the discovery of what works and doesn’t comes from the projects. Governance needs to be aware of these transitions, allowing for a managed, bottom-up innovation.
• What are the internal supplier-consumer (integration) relationships between projects/systems? Governance is not just about technology usage within a system, but also integration between systems. Integration has a technology underpinning, but the more significant concerns are at the level of interface models, data exchange models, and the ability to test system-of-systems integrations for functional, nonfunctional, and error-handling issues. Agile development emphasizes a continuous integration approach that necessities an integration environment that is always available (to each system/project), always up to date (and able to be moved to different dated revisions easily), and supports automated testing. But these responsibilities to the system-of-systems integration are, by their nature, bigger than any one system, and hence effective governance can make this workable and efficient.
• Are the third-party technology and integration choices across projects/systems compatible (note: this does not need to mean identical)? The key is compatibility, not conformity. This is a transition in the governance approach for standardization but appeals to the sense that “one size does not fit all,” and what organizations really need is a way to realize that objective in a way that supports agility without leading to chaos.

Read more]]> It still amazes me how many enterprise data warehousing/business intelligence (DW/BI) projects struggle, often to the point of paralysis, with the “Inmon/Kimball” debate. This impasse revolves around whether a DW/BI program should insist upon routing all information through a complex, third normal form (3NF) data layer or take it straight to a user-intelligible star schema repository from where it can be reported more or less directly. It’s easy to fault the 3NF for more than doubling the complexity, expense, and data latency of a DW/BI project, but also for being of zero direct value to the project sponsors and their stakeholders. On the other hand, projects that deliver data immediately to star schemas can quickly become complex themselves as the scope of the warehouse grows. When the conformed stars scale out, they too end up necessitating enormous reengineering efforts whenever the underlying business requirements change.

Truth is, neither of these architectural paradigms can be agile because they both result in inflexible juggernauts that defy economical impact analysis when new features are needed. Both approaches scale to installations that are so inscrutable and fragile that it becomes more economical simply to solve new requirements with new applications rather than updating the existing assets. Both the Inmon and Kimball approaches leave companies struggling with “legacy” warehouses.

One of the most promising solutions to this juggernaut problem revolves around data architectures that go beyond our traditional 3NF. Warehousing teams have achieved more robust designs when they push their target schemas past fourth and fifth normal forms into a variety of “hypernormal forms” (HNF). These forms include “data vault” forms, associative strategies, and even CJ Date’s new sixth normal form, where all attributes of a source schema should be shredded into something akin to key-value pairs when stored in a target system. My recent year of fieldwork revealed that, indeed, hypernormalized approaches yield warehouse designs that are far more robust in the face of changing requirements.

There are several tool suites and practice communities that have demonstrated the commercial viability of HNF for large projects with tight budgets and even tighter deadlines. What I particularly appreciate is how these new solutions truly enable agile data warehousing by enabling and accelerating incremental warehouse delivery patterns.

In terms of engineering advantages, HNF yields target schemas that are what I call “three-way robust”:

1. Teams can add small increments of functionality without undertaking massive reengineering.
2. Modifications to existing database tables involve only local impacts.
3. As the model grows, the cost of scaling the warehouse increases less than linearly.

My fieldwork’s most remarkable discovery, however, was how the epitome of hypernormalized tools truly enable the fourth tenet of the agile data warehousing manifesto, where we focus on “evolving data models over incrementing application code.” There are tools that can take business models and automatically generate 90% of the integration, presentation, and semantic layers for a team. By leveraging such technology, warehouse developers can focus on the remaining 10% where the business rules are complex and hand coding adds the greatest value. Such tools make the warehouse project and the entire organization agile because IT and business can now collaboratively model a small sliver of the warehouse, generate a user-tangible result, and evaluate it together. With the distraction and time lost to coding now reduced to a minimum, IT is able to rapidly address additional requirements step-by-step by evolving the information users can touch, keeping the business constantly involved in the partnership, and rapidly zeroing in on the exact operational insights the current business challenge demands.

So, the days of endless Inmon/Kimball debates are over. New tools and practices have moved the pivotal question far beyond “star schema versus third normal form.” DW/BI project kickoffs should now be immersing themselves in the question: “Which hypernormalized modeling technique and supporting toolset will work best for our organization?” It is the astounding improvements in programmer productivity and customer satisfaction this new generation of solutions allow that demands we make this change in mindset.

Read more]]> The “consumerization of IT” — in a big picture kind of way (not limited to “Bring Your Own Device”) — is having a huge effect on how IT is structured and delivered. It’s a “tipping point” with far reaching ramifications for companies, for employees and for the industry in general.

The May 2012 Cutter IT Journal with Guest Editor Jim Love, explores what “consumerization” really means and invites submissions from those who are experiencing the changes first hand, from skeptics who see a different point of view and from those who have already started to plan for the changes that they envision.

Please send us your ideas – proposals of interest are due 9 March 2012.

To respond, please visit
http://www.cutter.com/content-and-analysis/journals-and-reports/cutter-it-journal/callforpapers01.html

Read more]]> It often starts as a seemingly plain training request. Having decided to go the agile route, a client would like Cutter to train a certain number of employees in one agile method or another. We collect data on the demographics of the target population: architects, UI designers, product managers, project managers, developers, testers, and so on. We then move on to discuss the way these folks are geographically dispersed and what the team structure for the launched agile teams will be. Once these parameters have been nailed down, it largely becomes a matter of figuring out the logistics for training and coaching. A fairly straightforward process for rolling out the agile process, one might say.

The catch, time and time again, is that the problem to be solved had been defined from the outset in too narrow a manner. Naturally enough, a client wrestling with software challenges tends to gravitate toward improving the software method(s) in use. It is a small step from here to the conclusion that effective training and coaching are the only services that really need to be provided by the consultant(s).

The typical outcome of such a narrowly defined engagement is improved tactical agility. The teams become quite good at responding to frequent change needs. Stories get added, modified, deleted, or reprioritized fairly effectively and (many times) efficiently. The improvement can be assessed, and very possibly quantified, in quite a few ways. Yet, it is quite difficult, if not impossible, to point out the actual business benefits. The improvement in the process is clear enough to anyone who cares to examine it, but the end-to-end functioning of the software still leaves much to be desired. The “so what?!” question is typically raised under such circumstance: could it be the case that the resources put into the agile initiative could be better invested elsewhere?

My view on the subject — before, during, and after the engagement — is that an improved software process is a necessary but not sufficient condition for attaining the desired business results. Obviously, elaborate business plans are next to impossible to implement if the software process crumbles underneath them. However important tactical agility through a robust agile process is, it cannot on its own address strategic adaptability nor supply-demand alignment issues. To succeed as a business from a software perspective, you need all three: agility, adaptability, and alignment.

Lack of strategic adaptability is the end result of the accumulation of all “sins” that had been committed “against” the code prior to starting a new page with agile. The investment you put into improving the software process will probably pay off nicely in terms of quality, productivity, and time-to-market at some point in the future. Unless the level of your technical debt on the existing code is really low, however, your overall software system — old in combination with the new — will be slow to adapt. In addition to investing in agile, you will need to invest in pushing down the level of technical debt that had accrued over the years. You might choose to think of the investment in agile as primarily forward-looking. In contrast, cleaning up the technical debt mess might (erroneously) seem like backward-looking. However, both need to take place. As a matter of fact, doing the two together is quite synergistic, as technical debt is an excellent metric for driving agile software development.

If you step back to examine how the technical debt accumulated in the first place, you are very likely to find out that the vicious cycle depicted in Figure 1 has been taking place for years on end. It is not lack of expertise or the difficulty of development across the pond that got the software in trouble (though these might be contributing factors of secondary importance). Rather, it is the inevitable system dynamics that prevail when supply (i.e., team capacity) is misaligned with demand (i.e., requirements).

Figure 1 — The vicious cycle of technical debt.

Aligning supply with demand is in many ways the acid test for the long-term success of your agile initiative. Our knowledge of the agile process has increased dramatically over the past 10 years. Likewise, progress in technical debt measurement, reduction, and prevention has been quite impressive during the past couple of years. We know enough about agile methods and technical debt techniques to improve the state of affairs in all but the most desperate situations. As Figure 1 illustrates, we also possess a fairly good understanding of how misalignment could lead to the accumulation of technical debt of biblical proportions. What are often lacking are the discipline, will, and determination to force alignment.

My recommendation on supply-demand alignment in the sense defined in this article is to use the data structure of the enterprise depicted in Figure 2 as the tool for forcing alignment at multiple levels of abstraction. The principle is simple: the grand total of development and technical debt reduction work (probably expressed as dollars) at the epic level can’t exceed the amount allocated at the respective theme level. Likewise, the grand total of development and technical debt reduction at the feature level can’t exceed the amount allocated at the respective epic level. Ditto for the story level: the grand total of development and technical debt reduction at the story level can’t exceed the amount allocated at the respective feature level.

Figure 2 — Data structure of the enterprise.

You can opt to traverse the data structure of the enterprise in a top-down manner or in a bottom-up fashion. Whatever direction you choose, you must maintain the integrity of the calculations (i.e., the grand total in one level can’t exceed the respective allocation in the level above). Maintaining this integrity is quite possibly the number-one responsibility of the stakeholders you pick to govern the software process in your company.

– Israel Gat, Director, Agile Product & Project Management Practice