“Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization’s communication structure”
— Melvin E. Conway
Consider a traditional organization. There will most likely be a hierarchy of decision-making with a small group disseminating strategic decisions to a range of teams or departments for implementation. Each department will have a single function, such as sales, marketing, legal, development, testing, operations etc. and all will compete with each other annually for budget and headcount. Within each department, staff will compete against each other for career advancement opportunities within the department.
Annually, strategic goals are set for the business, outputs are defined for each department, budgets granted and bonuses set against the successful implementation of these outputs.
This model has been around for centuries, so what could possibly be wrong with it?
Well, it operates based upon the assumption that markets change linearly over very long timescales so you can rely upon a few percent of growth per year based upon an annual cadence of strategic input.
This assumption is however invalid in an exponentially changing, technology-driven economy where massive disruption of existing business models can happen in months and true ‘Digital’ businesses leverage machine learning to change business models on a minute-by-minute basis, tracking the behavior of markets in real time.
Under these circumstances, the culture of a traditional organization turns against itself because structure, metrics and incentives are all inappropriate. Staff will resist reacting to unanticipated market changes because their bonuses are tied to annual strategic outputs. Departments will compete rather than collaborate because of their budget structure. Teams will become increasingly risk-averse because change takes them further from the comfortable predictability of the expected small, linear future.
Now consider a high-growth, technology business. This will typically be a much flatter structure where there is a product-centric mission and individuals are rewarded through the increasing value of their shareholding. Cross-functional teams are structured based upon product line or customer need.
Strategic goals are set in the form of outcomes, objectives and key results, with clearly defined metrics that can be collected and evaluated daily or weekly. Goals are updated iteratively based upon the results of experiments run in live environments with customers. Resources are allocated based upon a spread of longer-term ‘bets’ and shorter term results against objectives.
In many cases, decision-making is now offloaded to machine learning models trained to react to changing market behaviors.
So, how does this impact the delivery process?
Well, in a traditional business, the product lifecycle starts with a strategic guess about a customer need. A team then creates a detailed specification for a product to fulfill that need. This specification goes through legal approval and sales and marketing teams start work on promoting the product to customers whilst a development team works on building the product to specification. The output of this development is then tested by another team and passed to operations for deployment to production. It may be a year or more before a customer starts to interact with the product and you discover whether there is any demand for it.
The end result is a very long feedback loop, with the product passing between many silos inside the organization, with each team expected to come up to speed from cold before progressing the delivery. If the original guess proves to be wrong, or the market changes significantly during development, the cost can be fatally high.
The approach in a high growth business is very different. A product manager will make a hypothesis about a potential customer need, based upon conversations with actual or potential customers. They define a minimal experiment to validate this hypothesis with one or more test customers, create a set of metrics by which the outcome will be evaluated and raise funds for running the experiment for a defined time period. A cross-functional team then does all of the work necessary to put this experiment live in a customer-facing environment, collects usage data according to the previously identified metrics and tracks progress daily. This process may evolve into a sequence of iterative experiments that demonstrate product-market fit, or result in a rapid decision to abandon this feature in favor of another approach. Only products demonstrating strong product-market fit are funded to be scaled out.
Under this approach, feedback is very rapid and the organization is able to learn very quickly from the trial and error process whilst minimizing the downside risk. This is particularly important in cutting edge fields where there is no prior experience or best known method to build from.
How is this relevant?
Continuous Delivery is a methodology designed to optimize the delivery process within high growth organizations. It is the answer to the questions “How do we minimize the time it takes to go from a hypothesis about a need to testing and validating that hypothesis in production?” and “How do we make mistakes safely and cheaply so that we can have the fastest possible learning experience without blowing up the business?”
If you are a traditional business in a mature and stable market, you likely have no need for Continuous Delivery and could end up spending a lot of effort implementing processes that yield minimal benefits within your organization.
If you are a tech start-up, you must follow the iterative product commercialization approach if you are to grow fast enough to successfully exit. Continuous Delivery is going to be very beneficial to you and you have many advantages that will help you to implement it. You have a team that is incentivized to build a product that customers love because the value of their equity depends upon it. Your team is not big enough to have developed departmental silos that can’t easily be broken up and restructured to align to the needs of your customers. Your priority is to work out what is the cheapest set of experiments that can be run to demonstrate the existence or lack of product-market fit within the available runway, so this encourages a culture of rapid experimentation with early adopters.
If you are a traditional business, undertaking a ‘digital transformation’, you face the steepest challenge in implementing Continuous Delivery successfully. You may be encountering disruptive competition from digital businesses entering your market and recognize that they are responding to customer needs more rapidly than you because you are only able to deliver change once or twice per year.
It may not be clear to everyone in your business that this means that the clock is now ticking down on a window of time in which you must respond before a competitor takes your most valuable customers. To be able to compete, you must transition to being better than your competitors in running the product commercialization strategy that they are using against you, but this means a major change to your corporate culture.
It is essential to start with communicating the required changes in culture before attempting to implement Continuous Delivery so that you have the correct incentives and policies in place before implementation. If you do not do this, you will typically face a scenario where ‘corporate antibodies’ will react to sabotage the change. Experience shows that DevOps is not a capability that can be acquired and ‘bolted-on’ to an existing delivery team. To succeed in this, it is necessary to break down all of the silos in the organization and create cross-functional product teams that follow a DevOps methodology and are aligned to your customer needs and empowered to deliver. This means not just breaking down departmental boundaries, but also restructuring the way you measure performance and incentivize productivity.
Looking more closely at potential problems, a traditional business often follows a ‘command and control’ model of central decision making with teams following orders in narrowly defined roles. This can create a culture of “only following orders”, “more than my job’s worth”, “our paymasters say we do X, so we do X” and “not my problem”. This can lead to bizarre situations where everyone knows that there are huge gaps in a solution but nobody owns the work to fill the gaps and nobody dares inform the leadership team.
For a high growth product commercialization strategy to work, it is necessary for all teams to be directly empowered to do whatever is necessary (within the bounds of ethics and brand values) to deliver a product that customers want. To do this, it is necessary to create a culture of extreme personal responsibility, where everyone on the team understands that it is expected that they will ensure that all aspects of the delivery are appropriately addressed and communicate any gaps or risks in a timely manner.
To succeed in this, you must first move visibly away from a ‘blame’ culture. Leveraging people’s mistakes as a tool for career advancement is a common strategy in traditional businesses. This behavior will kill your transformation stone dead because it represents the opposite from the desired mindset.
The fastest way to learn how to solve a problem well is to make a bunch of mistakes that give you valuable information about the true nature of the problem, survive the mistakes and then leverage the learning to build a better solution. To do this well, you will need to make a large number of small mistakes whilst avoiding making any fatal ones. In order for that to happen, your team must feel safe to acknowledge errors and communicate the learnings that come with them widely.
This level of psychological safety can be created with a culture of frequent communication that flags risks and blockers as a daily activity, where ‘shooting the messenger’ is clearly rejected as inappropriate and where teams pull together to recover from challenges then hold retrospective investigations to ensure that processes are modified to avoid the same problem causing impact in the future.
It is itself a mistake to try to build a team of ‘perfect’ developers who always produce flawless code. Anywhere that there is a human in the loop, you must plan for a certain number of errors to creep in. Expecting perfection actually creates a culture of risk-aversity and prevents your team from having sufficient learning experiences to make breakthroughs. Instead, you must build psychological safety by a combination of automated verification and peer review to catch problems during the release process.
Similarly, if the Continuous Delivery process is being adhered to correctly, and reviews and tests are passing, team members must be empowered to put changes into production. A classic cause of failure is to have a ‘continuous’ process within engineering, but then require paper sign-off from multiple departments before deliveries can go live.
Much of the literature on Continuous Delivery will be found to focus upon engineering activities, but it is important to realize that much like fish being unaware of water, these texts are usually written from the perspective of engineers who are swimming in an eco-system of product commercialization which has become second nature. For other organizations, however, your job is first to fill the pool, so the engineering parts of the journey can flow smoothly.