Metagility: Managing Agile Development for Competitive Advantage

Metagility: Managing Agile Development for Competitive Advantage

by David Bishop


View All Available Formats & Editions
Choose Expedited Shipping at checkout for guaranteed delivery by Friday, July 26


Agile methodologies have become a popular and widely accepted method for managing software development. However, despite this success, managing agile methods has proven to be a real challenge for most companies, particularly those with complex products such as IoT devices and large development environments. Many companies have been forced to adopt a hybrid version of agile and waterfall techniques, and this hybrid approach is fast becoming the norm rather than the exception in the industry. Metagility is the first book to provide a comprehensive approach for managing a new and highly effective breed of agility from the executive level on down. Based on scientific theory and practitioner research, it is the definitive playbook for those seeking the optimal solution for adapting agile to more complex product development and organizational contexts. 

Product Details

ISBN-13: 9781604271553
Publisher: Ross, J. Publishing, Incorporated
Publication date: 01/24/2019
Edition description: None
Pages: 216
Product dimensions: 6.00(w) x 9.00(h) x 0.80(d)

About the Author

Dr. David A. Bishop is a technology consultant, enterprise architect, researcher, and instructor with over 25 years of experience.

Read an Excerpt




Agile methodologies have become a popular and widely respected method for managing software development. Since the inception of the Agile Manifesto, agile development methodologies have superseded waterfall methodology in many, if not most, software development organizations. Despite its apparent success, however, many organizations have struggled with the adoption and implementation of agile and exactly what level of adoption provides the most agility. Agility is commonly held in the literature to be constructed of elements external to a company or project but, in fact, may be composed of both external and internal elements. The exact relationship of the adoption of agile development techniques to a company or organization's true agility remains unclear. Crucially, there are no measurements available for each of these factors in an embedded software development context nor is there a method to determine the impact of one against the other. A major reason for the lack of measurements is due to the somewhat amorphous definition of agile itself. In academic literature, the concept is still relatively young and poorly defined. In practice, organizations have largely opted for a hybrid approach to agile, mixing its concepts and method with existing stage-gate or waterfall methodologies. This has made the management of agile even more complex. These issues beg the following questions:

• How do organizations orchestrate and assess agility?

• How is agile best managed in difficult or complex situations?

• How can some of the commonly known shortcomings of agile be mitigated?

• Most important, given the apparent efficiency and performance gains of agile, how can it be leveraged to maximize organizational performance and product success?

Metagility answers these questions by examining multiple dimensions of the issue (agile definition, history, adoption, implementations, hybrid solutions, project/program management) to determine how agility processes are orchestrated (meta-agility) in a number of companies that were successful in leveraging and maximizing agility to become number one in their respective market(s). Throughout this book, we highlight one of these case studies in particular. A study of an embedded systems development organization was conducted with twelve managers from hardware, firmware, and software development groups within the company. These managers included product, project, people, and technology managers with direct responsibility for managing agility processes. An innovative form of research, referred to as grounded theory, was utilized to perform data analysis and determine a central phenomenon or theory at the crux of the agile question. This research revealed the central phenomenon to be agile vorticity — the point at which maximum agility is achieved. This point is reached through the management (meta-agility) of two main subcategories: process and market agility. The relationships of these categories are illustrated with a thought experiment of a whirlpool vortex. Included is an empirical account of how agility can be managed and tweaked for optimum results. In the chapters that follow, we examine market and process agility to determine the success factors for achieving agile vorticity and becoming number one in your market!

We begin by establishing a historical perspective of agility — its evolution and its current state as compared to when it was first developed.


The Agile Manifesto as it was originally written in 2001 is a set of four now very well-known tenets:

• Individuals and interactions over processes and tools

• Working software over comprehensive documentation

• Customer collaboration over contract negotiation

• Responding to change over following a plan

The idea being that while all elements in the previous statements have value, the ones on the left side are valued even more. In this section, we will explore each of these four tenets and how today's environment is influencing them to change.

Individuals and Interactions over Processes and Tools

In 2001, people worked more independently as waterfall processes tended to limit interactions and focus on well-defined goals. Agilists hoped to break this lack of interaction by emphasizing teamwork, but much of this was predicated on colocated teams. As Table 1.1 illustrates, teams today and the communication channels they use are more complex. Many development organizations are distributed all over the world and use a variety of communication methods and tools. With that in mind, tools themselves have become much more sophisticated and user friendly. Many such tools are easily customizable and have agile concepts built in.

Waterfall was extremely heavy with a plethora of burdensome steps and operating procedures designed for CYA (cover your ass!) rather than focusing on creating a great product. On the other hand, some aspects of agile, such as team collaboration, have proven challenging to implement at scale. Although interactions are certainly very important, the lack of process around documenting the outcomes and decisions of these interactions can result in a great deal of churn. Churn in this context means burning resources and losing time due to repetitive meetings and rework.

Many organizations have found that today's tools can facilitate conversations and can be used to drive, not inhibit, development. Additionally, they have also discovered that implementing agile itself is a process, even if it is much smaller in scope than waterfall was. The right combination of process and tools can be decisive in the success of your business.

Working Software over Comprehensive Documentation

Older waterfall processes often resulted in cumbersome and exhaustive word documents which included requirements documentation, design documents, standard operating procedures, maintenance documents, support documents, and much more. Emphasis was placed on writing many of these documents up front before attempting to build anything. This comprehensive documentation tended to inhibit response to change and resulted in a slowdown of innovation and productivity. Documentation is still needed for today's development, but it takes a different form. It is no longer a series of heavy word documents, rather it comes in the form of features, epics, user stories, and other artifacts that may be documented within collaboration tools or other applications. Table 1.2 illustrates these differences. As mentioned previously, outcomes of key interactions need to be documented, just not in a process heavy or cumbersome document, but within a lean and easily managed set of tools.

The need for documentation is even more critical with today's complex products such as smart devices or Internet of Things (IoT) applications that are created using multiple teams, development tracks, and sub-products.

Customer Collaboration over Contract Negotiation

Contracts are a fact of business life. They aren't going away. Regulatory requirements, idiosyncrasies of specific industries, or the nature of certain products require them. In today's world, contracts and collaboration are not mutually exclusive; in fact, they are anything but. When we discuss contract negotiation, that negotiation within itself is a form of collaboration with the customer. The difference is that this collaboration is not necessarily a one-time thing where all items are discussed and documented up front in a contract, rather it is an ongoing process of communication and alignment.

Collaboration has become more than just talking directly to the customer. Often, the customer may not be able to articulate exactly what they need, and it may require a variety of stakeholders on both the customer side and internal to the development organization to glean succinct requirements and break those down into increments that can be developed into a working product. Table 1.3 illustrates this change.

Responding to Change over Following a Plan

In 2001, plans were synonymous with rigid. A plan was often considered to be a representation of the final work. Making changes midstream once the plan was finalized was discouraged, and if changes were implemented, it often resulted in a painful approval process and reevaluation of delivery timelines, costs, and performance expectations. A great deal of work was performed up front to ensure that the plan, and the requirements it entailed, were as complete as possible so that developers knew exactly what they needed to do down to the finest detail. Different interpretations of the plan, unanticipated engineering challenges, and inability to accommodate changing customer needs often resulted in failure. Today, markets are simply moving too fast to adhere to this kind of rigid approach. At the same time, there must be some kind of plan to ensure that the company is getting the business value out of the development work that not only makes the customer happy but also satisfies their strategic goals. This means that the definition of a plan has changed. As Table 1.4 illustrates, no longer is it a long, exhaustive document that is full of requirements and design specifications, rather it is more of a vision for the organization. Change is inevitable, but every change must be evaluated against the values and needs of the business.

Now that we understand how the Agile Manifesto has changed from a practitioner standpoint, let's take a look at what the business research says about the topic.


The management of agile methodologies can be a challenge to organizations on many levels. The first issue is the concept of agile itself. The current research literature on agility is sparse, particularly in relation to information systems development (ISD). This is largely due to the fact that agile is a relatively new concept in an ISD context and is therefore not entirely solidified. It is, in essence, a set of concepts or ideas, not necessarily a methodology or framework of its own. The second challenge is the management of agility. Most agile-based frameworks available today are operational in nature, focusing on project management indicators such as velocity, release frequency, sprint completion, and so forth (Highsmith, 2010). Still, other methods focus entirely on the outcome of agile adoption in relation to environmental turbulence (Yauch, 2011). This type of study, however, does not allow for the evaluation of the adoption of agile principles, such as people over processes and tools, against quality and customer responsiveness. Giachetti holds that the assessment and management of agile should not only consider performance, but agile characteristics that have been assimilated into the organization (Giachetti et al., 2003). Although there have been methods describing agile adoption, none of these has related the level of adoption to agility outcomes. As a result, these existing methods do not completely address all dimensions of agility. These challenges are compounded by the fact that recent research has revealed that most organizations have not adopted agile in its entirety, but instead have assimilated a variety of agile concepts and methods into existing traditional methods. Such hybrid agile implementations have only added to the complexity of agility management.

Orchestration of agility within any organization is often poorly defined, due to its fluid, quickly changing, and somewhat amorphous concepts, processes, and methodologies. Hybrid implementations, environment specific assessments, and varying degrees of market turbulence are just a few dimensions of agility that should be considered within a management context. In short, the management of agile methods requires their own brand of agility. Our goal from a research perspective was to examine our case studies against the various dimensions of this agility to determine how agile processes are typically managed and orchestrated.


The short answer to that question is that it is becoming increasingly difficult to stay ahead. Technology today is changing at an ever-accelerating rate. In his essay "The Law of Accelerating Returns," futurist Ray Kurzweil said that this rate of acceleration is occurring exponentially and could eventually result in "technological change so rapid and profound it represents a rupture in the fabric of human history." We see evidence of this change in our everyday lives, not just in technology. Companies must work harder and harder to keep up, tweaking ever more agility out of their processes and teams.

Hyper-Accelerated Markets

Many case studies in our research experienced hyper-accelerated markets, which created some very unique conditions. One of these conditions is what is referred to as an agile vortex or whirlpool. Agile vortices are created when extreme levels of market pressure are torqued by high innovation or quickly evolving technology. This provided an excellent laboratory under which it was possible to observe how a company achieved optimum performance, or essentially agile nirvana in today's most challenging markets. This phenomenon is referred to as reaching the point of agile vorticity, which will be explained in a later chapter.

How these hyper-accelerated markets take shape can be illustrated by close examination of one of our cases studies. In this case, the technology developers were in the smart grid market, developing smart meters for power utilities that wanted to upgrade their grids to the latest green technologies. These are the same kind of meters that you may find on the side of your house or business; the difference is they are automated, more accurate, and in most cases communicate wirelessly without the need to send out technicians to read them. Not only do smart meters save money and protect the environment, they provide greater security as well. The subject of our case study faced several challenges which could be summarized in the following ways.

Extreme Innovation

1. Carrier and networking technology was rapidly changing: From low power radio frequency (RF) signals to cellular to power line carrier, there are a host of communication methods that could be used to transport meter data from the field to the utility. Adopting the best technology could make or break a company. Our case study had invested deeply into RF mesh networking technology and was constantly faced with new threats.

2. Meter capabilities dramatically increasing: Originally fairly dumb devices, electric, gas, and water meters were acquiring the ability to manage smart devices throughout the home, communicate pricing to customers, and gather detailed event and power data. In many ways, the growing capabilities of the available technologies were outpacing the company's ability to get them tested and implemented — not to mention the fact that competitors were in the same race. The same thing was happening, and in fact still is happening, with other IoT solutions. Our case study was rushing to create the most intelligent smart meters of all!

Extreme Market Pressure

The customers for the subject of this case study were power utilities. In many cases, governments in many parts of the world provided these utilities with incentives to switch to smart meters. At the same time, these smart meters had a long life span of 10 to 20 years, thus, once a utility selected a vendor, that customer would be out of the market for quite a long time. There were several start-up smart metering vendors and only a fixed number of big utilities. This created a land grab situation where each vendor was rushing to capture as many big utilities as they could. Whoever managed to garner the largest chunk of early market shares would dominate the market for a LONG time. This combination of incentives, long-term contracts, and limited customer base resulted in extremely high market pressure.

Combined, these extremes of market pressure and innovation created a hyper-accelerated market. One in which innovation and market pressure are so great that they result in a virtual hurricane for vendors to navigate.

What Is the Solution?

Transitioning to, adopting, and implementing agile methodologies in a software organization is a costly and time-consuming proposition. Companies and organizations need to know where they stand in terms of adoption/assimilation and its impact to the external agility of the business. They also need a framework to manage their performance so that strategic adjustments can be made. We set out to conduct studies that would provide greater understanding with regard to how agility can be managed and the impact of assimilation and adoption on these management processes. As we will see in a review of the latest research literature, most embedded systems environments, such as the one mentioned previously, with larger, more mature organizations, have taken a hybridized approach to agile adoption by combining agile with existing stage-gate methodologies. The orchestration of the processes required to support agile principles in such an environment and the way those principles are affected can reveal new insights into agility in new and different contexts. Such contexts include:

• Large teams,

• Geographically distributed teams,

• Hybrid agility evolution, and

• Embedded systems (environments which include the synchronization of firmware, software, and hardware development).


Excerpted from "Metagility"
by .
Copyright © 2019 David A. Bishop.
Excerpted by permission of J. Ross Publishing, Inc..
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

Table of Contents

Foreword ix

Preface xi

About the Author xv

WAV™ xvii

Part 1 The Need for Metagility 1

Chapter 1 Agile Methods Today: What Has Changed and Why 3

Introduction 3

The Agile Manifesto Today 4

Individuals and Interactions over Processes and Tools 5

Working Software over Comprehensive Documentation 6

Customer Collaboration over Contract Negotiation 6

Responding to Change over Following a Plan 7

Agile Management Today: A Research Perspective 8

Why Are Agile Methods and Agility Concepts so Critical to Business Success Now? 9

Hyper-Accelerated Markets 9

Approach for Building a New Direction 11

The Evolution of Agility: Engineering and Software Traditions 13

Agility 15

Agile Software Development Methodologies 17

How Have Most Companies Dealt with These Issues? 21

Achieving Agile Vorticity 22

Chapter 2 The Problem: Performance and Limitations of Agile Methods 23

Understanding the Performance of Agile Methods in Challenging Situations 23

Agile's Positive Contributions 24

Agile's Challenges 25

Managing Agility with Embedded and Real-Time Systems 27

Why Are Embedded Systems so Important? 29

Continuous Development and Managing Complex Change 29

Managing Distributed Teams 31

Adoption of Agility: The Need for Agile Transformation 32

Agile Has Become the Industry Standard for Managing ISD Efforts 32

Agile Adoption in Large, Complex Environments: Leveraging a Hybrid Approach 34

Problems with the Current Literature: Limitations of Agile Research 36

What Is Agile, Really? 36

An Apparent Lack of Rigor 37

No Research in Differing Contexts 37

Lack of Research at the Organizational Level or Across Business Units 37

Chapter 3 Hybrid Agility: Determining and Implementing the Right Approach 41

Why Organizations Adopt Some Aspects of Agility and Not Others, and How This Is Changing Agile Methods 41

Agile Is Not Just a Development Process Anymore, Its a New Way of Doing Business 44

The Agile Triangle Replacing the Iron Triangle 44

Obtaining Executive Involvement and Buy-In 45

Old Habits Die Hard: Convincing Others to Break Away from Waterfall 48

Agile Requires Too Many Meetings 48

We Don't Want to Provide High-Level Estimates 49

Agile Does Not Provide for Sound Technical or Architectural Planning 49

Agile Requires Colocation 50

Agile Does Not Allow for Long-Term Planning. 50

Our Product and/or Development Environment Is Too Complicated for Agile 51

Unspoken Concerns 51

Analyzing Your Environment to Determine the Correct Hybrid Agile Approach 51

How Waterfall Is Commonly Blended with Agile Methods 55

Stage-Gate Governed Release Train 58

Hybrid Agile Stage-Gate Definitions 58

Hybrid Agile for Firmware Teams 61

Hybrid Agile for Hardware Teams 62

Part 2 The Agile Business Vortex 63

Chapter 4 Understanding Agile Vortices 65

The Science Behind the Theory 66

Basic Design of the Research 66

How Data Was Collected 67

Qualitative Grounded Theory Analysis 68

Agile Vortices: How and Why They Occur 69

A Vignette of a Case Study 70

Converging Forces 74

The Agile Vortex Thought Experiment 75

Understanding the Systems Release 80

The Creation of Business Momentum 81

Agile Vorticity: The Key Connection Point 81

The Value of Agile Vortex Theory to Complex Contexts 83

Chapter 5 Managing Market Pressure 85

Market Agility 85

Establishing an Agile Market Share 86

How Does Your Customer Base Impact Business Agility? 88

Competition 90

Dealing with Government Regulation 91

Governing Market Pressures by Establishing a Strategic Direction 93

Customer Appetite: Maximizing Collaboration 95

Chapter 6 Product Genesis: Determining What to Build 97

Market Timing: How to Know if This Is the Right Time for Your Product 97

Dynamic Priorities: Managing Ever-Changing Priorities 99

Common Challenges with Dynamic Priorities 101

Requirements Comprehension: Understanding What the Market Needs 103

Decomposition: How to Break Down Requirements into Consumable Chunks 104

Functionality Chunking 106

Estimations 108

Determining Which Requirements to Keep and Which Ones to Drop 110

Customer Needs 111

Customer-Driven Priorities 112

How to Negotiate Scope with Your Customers 114

Field Trials 115

Workshops 116

Pilot Projects 116

Release Acceptance 117

Cultural Differences Affect Negotiation 117

Creation of the Roadmap 119

Instilling a Culture of Agility 120

A Recap of Business Momentum 121

Chapter 7 Organizational (Process) Agility: Building the Capability 123

How Do Hybrid Agile Implementations Work? Are They Successful? 123

How Is Agile Adopted Differently Between Software, Firmware, and Hardware Development? 125

What Is Different About Agile in Software Development and How Can This Difference Be Leveraged? 127

Managing Shared Resources: Resolving Conflicts and Getting the Job Done 127

Case Study Interview: Firmware in Embedded Systems Development 128

Is Waterfall Best in Certain Situations? 151

Prototyping to Accelerate Product Development 153

Making Use of Fast Tracking 154

Managing Customer Expectations for Agile Vorticity 155

Customer Negotiation Techniques for Maintaining Business Momentum 155

What Situations Benefit Most from a Hybrid Agile Mix? 157

Part 3 Practical Issues and Applications 159

Chapter 8 The Art of Metagility: Bringing It All Together 161

Metagility in Complex Environments 161

Embedded Systems: A Case Study in Agile Orchestration 161

Interconnections and Interactions 162

Managing One-Way Dependencies 163

Intertwined Solutions: Managing Interdependencies 164

Defining Formal Linkages: Developing Productive and Impactful Meetings 165

Metagility Metrics: Predictive Analytics Dashboards for Executives (Status Points) 172

Decision Points: The What, When, and Where of Metagility Decision Making 181

Informal Interactions for Maximum Agility: Eliminating Forgetfulness 182

Fine Tuning: How to Optimize 183

Customer Acceptance 184

Scope Adjustment 185

Resource Adjustment 186

Kaizen Culture: The Importance of Constant Reassessment 187

Chapter 9 Soft Skills for Managing Agility 189

Addressing Common Project Management Concerns 189

Documentation 190

Team Management 191

Understanding Tribes and Building Agile Teams 192

Self-Organizing Teams 194

Managing Distributed Teams 195

Interchangeable Teams 197

Techniques for Bringing Your Team up to Speed 198

Teams Are Not Equal 198

Training 199

Pair Programming 199

Executive Management: Managing Your Managers 199

Real-Time Strategic Alignment 200

Communication 210

The Number One Secret Ingredient to Becoming and Remaining Agile 211

Chapter 10 The Future of Agile Methods 215

An Inquiry into Hybrid Agility 215

Fluidity and Continuous Releases: The Cause for Leaner Methodologies 216

Hybrid Agile Implementations: Whirlpools within a River of Innovation 217

Implications for Research 219

Implications for Practice 220

Trends: Looking Ahead 221

Data Science Along with Advanced Tooling Will Drive Decisions 221

Approaches to Communication Methods Must Be Continually Re-evaluated 222

Agile Organizations Require Agile Individuals 222

Agile Companies Require Agile Leaders 223

Agile Is Becoming Integral to Strategic Planning 223

A Caution Regarding Agile Frameworks 224

Agile Evolution and Natural Selection 225

New Markets Must Be Continually Sought Out 225

Agility Will Continue to Evolve 226

References 227

Index 235

Customer Reviews

Most Helpful Customer Reviews

See All Customer Reviews