2015 will be the year that agile data warehouse (DW)/business intelligence (BI) takes off. Traditional strategies for DW/BI have been challenged at best, with the running joke being that a DW/BI team will build the first release and nobody will come. On average, Agile strategies provide better time to market, improved stakeholder satisfaction, greater levels of quality, and better return on investment (ROI) than do traditional strategies. The DW/BI community has finally started to accept this reality, and it is now starting to shift gears and adopt agile ways of working. My expectation is that 2015 will see a plethora of books, case studies, and blog postings describing people’s experiences in this area.
There are several key differences with an agile approach over a traditional one. First, agile teams will follow a usage-based strategy, often via user stories or light-weight use cases, instead of a data-driven approach. By focusing on how people will use the data warehouse, agile DW/BI teams are able to deliver valuable functionality that addresses the actual needs of end users, thus avoiding the “build it and nobody comes” problem. Second, agile DW/BI teams deliver incrementally, thereby avoiding the risks associated with a big-bang release strategy. The first release may take a bit of time due to the need to get the fundamental DW infrastructure in place, but after that, new functionality can be released monthly if not weekly. Third, disciplined agile DW/BI teams take a test-driven approach that leads to greater quality and dependability.
Cutter consultants have led the way in this field. Lynn Winterboer, Ken Collier, and Ralph Hughes have all done excellent work over the years describing how to apply agile strategies in the DW/BI space. Pramod Sadalage and I have published seminal work in agile database techniques such as agile data modeling, agile database testing, database refactoring, and continuous database integration.
As you can imagine, transitioning to an agile DW/BI approach requires a cultural shift, a paradigm shift, learning new skills, and adopting new tools. 2015 will prove to be a very exciting year for the data community.
Editor’s Note: This post is part of the annual “Cutter Predicts …” series.]