Download as PDF

Contents

Warning

To DevOps and Beyond

Get Traction

Compasses

The DORA 4 Key Metrics

Getting Started With Measurement, The Right Way

Beyond Metrics

Maps

The 4 Key Maps Simplified for illustration

The 4 Keys Maps in Context

4x4 In Action

Set the Compass

Map the Path

Case: Ambitious Software Delivery Team

Create your Turn-By-Turn Navigation

Driving Forward

Frequently Asked Questions

Why do so many teams struggle to improve?

Why should we measure these four metrics?

Are these the only metrics I need to improve?

Why do I need maps? Can’t I just start tracking the metrics?

Why is mapping valuable in DevOps?

Is this only for software teams?

Who guides this process?

Who participates in mapping?

How should we build the maps?

When should we do this?

How do we get accurate results?

What does this really look like?

Where can I find more information?

Get Started

How should I get started?


Even with all of our experience and knowledge in adopting new techniques and technologies, 70% of transformation efforts continue to fail. They primarily fail as a result of lacking clarity and process for improvement. Few techniques and strategies exist to address the clarity gap, or provide actionable guidance to make improvements.

The rise of value stream focus has revealed a strategic and tactical approach to transformation that reaches beyond DevOps to support data-driven decisions, and improve performance holistically. Gartner has said that by 2023, 70% of organizations will use value stream management to improve flow in the DevOps pipeline, leading to faster delivery of customer value. The true potential of a value stream focused approach is in aligning and connecting every effort in your organization towards continuous delivery of customer value. In order to deliver on this promise, teams need an on-ramp that will allow them to capitalize on improvement efforts. Plugging in a new tool, no matter how valuable, doesn’t drive change, and can disrupt as much as it assists.

There are massive benefits to be gained through successful adoption of new development, delivery, and operational capabilities. Take it from DORA’s research on thousands of organizations:

Many teams have embraced the idea of tracking DORA’s 4 Key Metrics, and are enthusiastic about using them to measure improvement, but they often lack a clear starting point  - much less a path to success. The 4x4 Method provides that path.

⚠️ Warning ⚠️

The 4 Key Metrics might be a popular focal point in DevOps, but they’re not the only metrics to target, nor the most important. High performance with the 4 Key Metrics is merely highly correlated with high DevOps performance. The most important metrics for you to track depend entirely on the outcome you’re aiming for. In this document, the 4 Key metrics are used as an example of potential proxy metrics you could use the 4 Maps to address. A key feature of the 4 Key Maps is that they allow you to choose valuable metrics once you know what you need to succeed, even if you just have a rough goal to start with.

When a measure becomes a target, it ceases to be a good measure - Goodhart’s Law

To DevOps and Beyond

This document describes a winning approach to making progress with any key initiative, framed within the context of DevOps adoption and improvement. It has evolved through many iterations across several mediums and now in the age of remote-first collaboration, is finally easy to share, and easy to start. The system is collaborative and remote by default, but works equally well in a room full of people and post-its. It can even provide value as a solo investigation. It requires no specific tooling. It can be leveraged to drive both continuous improvement and innovation.

It was created through 20 years of improvement experience. It sits upon principles, practices, and learning from Lean, Agile, Systems Thinking, Visual Learning, Collaborative Facilitation, and many other bodies of knowledge. It was created to help teams make clear and consistent progress towards their performance goals. The 4x4 Method is named not only for the 2 sets of 4 components it comprises, but because it was designed to provide a vehicle to help you navigate challenging terrain.

The 4x4 Method combines the clarity and simplicity of a compass, with the breadth and visibility of a map.

If you’re starting from zero alignment, zero clarity, zero executive buy-in, this is for you. If you’re absolutely knocking it out of the park, but you’d love a way to check your work and insure you don’t slide off course, this is for you. If everything is working, but you need to take it to the next level or scale beyond your comfort zone, this is for you. The process described is by no means exhaustive or a step-by-step tutorial, but I’m happy to make myself available for further guidance. So far, reception to this approach has been overwhelmingly and enthusiastically positive, so more detail will soon follow.

In order to connect the approach with a specific challenge, DORA’s 4 Key Metrics are used to frame the context, but the mapping techniques and order described can be applied to any target goal. By creating and leveraging the 4 Key Maps I describe here, you can choose any set of balanced performance metrics and make progress towards improving them.

Get Traction

The DORA 4 Key Metrics have provided a tremendous amount of clarity and confidence in the DevOps world towards measuring the right things, and how to track progress as a modern software delivery organization. As targets, they act as compasses to guide direction and keep teams from veering away from their true north. However, compasses only truly become powerful for making decisions when they are paired with maps. Without a map, blindly following a compass can lead you to a dead end, or off a cliff. You may be staring directly at a mountain, wondering how to take your first step forward.

If you’re new to the 4 Key Metrics, you may be struggling to imagine how you can start to address them. There’s little guidance on taking that first step with confidence. You may have created a roadmap already. Have you noticed it doesn’t help you make progress? Is a roadmap a map if it's not showing you obstacles, measurements, and how to make decisions on your way? Where you’re going, there are no roads - you need to build them. Pairing these 4 compasses with 4 maps provides step-by-step navigation towards improvement in the form of clear alignment, better prioritization, constructive communication, and valuable collaboration. Mapping pairs the quantitative with the qualitative to build the path you need to follow. The application of these 4 Key Compasses and 4 Key Maps is what I call the 4x4 Method of DevOps.

The 4x4 Method doesn’t dictate any rituals, structure, or dogma. It simply provides a toolset to map a clear path from desire to success for teams who want to make visible, rapid, measurable progress. Once you have your compass points, and your maps of your path through the terrain, the 4x4 Method allows you to drive forward without fear of getting stuck, or sliding off course.

Compasses

4 Key Compasses

Lead Time:

Mean Time To Recover:

Change Fail %:

Deployment Frequency:

Measurement

Time from work start to finish

Time to restore service

Frequency of failed changes

Frequency of delivered change

Focus

Velocity

Quality

Quality

Velocity

Direction

Aim for lower

Aim for lower

Aim for lower

Aim for higher

The DORA 4 Key Metrics

Aspect of Software Delivery Performance

Elite

High

Medium

Low

Deployment frequency

How often does your organization deploy code to production or release it to end users?

On-demand

(multiple deploys per day)

Between 1-2/day and 1/week

Between  1/week and  1/month

Between 1/month and 1/six months

Lead time for changes

How long does it take to go from code committed to code successfully running in production*

Less than one day

Between one day and one week

Between one week and one month

Between one month and six months

Time to restore service

How long does it generally take to restore service when a service incident or a defect that affects users occurs (e.g. unplanned outage or service impairment) 

Less than one hour

Less than one day

Less than one day

Between 1/week and 1/month

Change failure rate

What percentage of changes to production or released to users result in degraded service (e.g. lead to service impairment or service outage) and subsequently require remediation

0-15%

0-15%

0-15%

46-60%

*This is not true lead time, rather lead time in the subcontext of software delivery

Care must be taken with measurement to ensure that behaviours, incentives, culture, and safety are considered along with the target. Before setting your target, you need a clear view of the bigger picture (because the bigger picture will influence, and be influenced by your focus). This is best undertaken by carefully considering your target outcome from a variety of angles, from within and outside your target team. I’ll describe how this is accomplished below.

“What gets measured gets managed - even when it’s pointless to measure and manage it, and even if it harms the purpose of the organisation to do so.”


- Simon Caulkin summarizing V.F. Ridgeway, 1956

Getting Started With Measurement, The Right Way

Many teams are not currently measuring the 4 Key Metrics because they’re not looking at their true value stream. In order to effectively measure lead time and mean time to recover, you have to be looking at the full flow of activities from when work starts to when customers experience the results.

You can certainly count change failure percentage by support tickets, but in order to affect that statistic, you have to look back through the value stream to see where quality is breaking down.

The 4 Key Metrics stretch well beyond your CI/CD pipeline

You can likewise measure deployment frequency by looking at your CI/CD pipeline, but without looking at the contributing activities upstream within your value stream, you may be missing a key contributing factor that could dramatically improve your results.

Many teams see rapid success by focusing efforts on their delivery pipelines, provided a bottleneck is within their build and deploy process,that process is highly independent, and their teams are highly capable of automation. If all of that is true, once your pipeline runs in less than an hour, you’ll see diminishing returns and your bottleneck almost certainly resides either upstream or downstream of your pipeline.

By visualizing and measuring your value stream, you can collect critical metrics data, and see what is contributing to the measurements - all in a picture you can easily share. Before you start mapping your value stream, you need to start with a clear outcome in mind. That’s coming up next.

Beyond Metrics

Tracking the 4 Key Metrics is a great way to measure your team's ability to successfully deliver software. So how can you actually improve these metrics? It’s great to have a target, but how can you actually move towards it without wasting time, effort, money, or morale? This is where maps guide the way.

Imagine the path of your workflow from idea to customer as a stream of value that currently winds throughout the landscape, off into the distance around you, from upstream to downstream. From where you stand, you can see part of the stream as it approaches you from upstream, and you can see a part downstream as it continues into the distance. If you wanted to improve the flow, you could widen the portion around you, or perhaps carve a more direct path from one curve to the next. You wouldn’t truly be able to see what effect your change had on the stream without a view of the entire flow, and the downstream outcome. The stream could be restricted far upstream beyond your view, or downstream. If you had a map of the stream’s path, and measurement of the flow within the stream, you could ensure it followed the most direct path to your destination, and improve the flow precisely where it would have the most positive effect.

This is the power of a mapping approach to improvement. By creating and leveraging maps of the flow of value in your organizations, you can create the optimal flow of value, by changing precisely what will have the greatest impact.

A compass without a map can easily lead you to a dead end.

Maps

4 Key Maps

Outcome Map:

Value Stream Maps:

Dependency Map:

Capability Map:

Visualization

Factors contributing to your desired state

Measured workflow performance and constraints

External influences on the stream

Internal influences on the stream

Focus

Clarity, Direction

Flow

External Effects

Internal Effects

The 4 Key Maps provide a comprehensive view of the landscape that your value stream traverses. Using the maps to guide your focus, decisions, and efforts, you can scientifically approach your improvement efforts using a visual and easily understood medium.

The 4 Key Maps Simplified for illustration

The Outcome Map

Why? Begin with the end in mind. See and share factors between you and your goal. The Outcome Map allows team members to dig into a desired outcome and begin charting the path to achieving it with open eyes. Doing so will help the team define a clear roadmap towards better.

How do you build it? The team comes together to define their desired outcome, and then consider what could impact their ability to deliver it. Defining the Why(s) sets the foundation supporting the effort. Obstacles are the key challenges standing in the teams’ way. Investigation lists your immediate actions to address obstacles. Measures are TBD when you define the map at first, because until you create the following 3 maps, defining effective measures is impossible. You’ll loop back and fill them in. Optionally, you can dig even deeper to add Indicators, Impacts, Roles, and Methods. Indicators list how you will know you’re progressing. Impacts list factors that will affect your progress. Roles list the contributors to the process. Methods, like Measures, wait until you know more. Deciding what to do and how to measure it can’t be done effectively until you truly know where your risks and opportunities are.

How do you use it? The Outcome Map is an excellent way to create energetic communication, clarity, and alignment from the start. It also reminds you to stay on track as you progress, and how to know when we’re drifting from the path. By adding methods and measurements, you can describe where you want to go and how you plan to get there.

Similar to: Impact maps, fishbone diagrams, V2MOM

The highest performing companies build alignment around three questions: Why are we doing this? What should we do? How do we implement the transformation?

Boston Consulting Group, 2020

The Value Stream Maps

Why? Visualize the flow of value through your team and organization. Measure performance, identify bottlenecks, target opportunities.

How do you build it? Your defined outcome allows you to identify the primary contributing stream. The team records the flow of activities towards delivery, and then collects activity and wait timing. Beyond the core components, you can layer on quality, value, role, and other data to create a rich visualization of your current reality. Beyond the current state, you define an ideal state (your ‘what if?’ to inspire innovation) and target state (your ‘what next?’ to define a short term iteration of 3-6 months) to align next steps.

How do you use it? The mapping exercise brings the team together under a common understanding. Everyone is encouraged to contribute their perspective to build the richest illustration. The measurements reveal bottlenecks, quality issues, low value activities, and more. The clarity provided by that measurement allows you to not only focus, but prioritize as well. Your team can hold many opinions on how to prioritize and where to focus, but having data and a clear picture to discuss brings perspectives into alignment. You can use these maps to target improvement efforts to start on immediately, provided we’re not constrained by external dependencies.

Similar to: Traditional Value Stream maps, process maps

The measurements reveal bottlenecks, quality issues, low value activities, and more.

The Dependency Map

Why? Now that you have value stream maps along with a few key bottlenecks and opportunities, you can work together to define what is affecting the team and stream in question from beyond the team boundaries. By mapping dependencies, you can identify opportunities to mitigate them.

How do you build it? The team brainstorms factors (shared services, meetings, SLAs, approvals, challenging work, obligations, etc) that affect their ability to deliver work in the value stream. They then cluster them around common areas or themes to highlight and discuss the effects of each. For extra clarity, the team can also add data related to turnaround time or SLAs.

How do you use it? This map shows every member of the team (and anyone who sees it) what is involved in -and affecting- their ability to deliver value. This brings another dimension of clarity to the bottlenecks identified in the value stream, and confidence that the team has targeted truly valuable focus areas. The map reveals key dependencies that can be mitigated or broken, provided the team possesses the capability.

Similar to: Dependency graphs, system maps

The Dependency map reveals what is externally affecting the team’s ability to deliver value.

The Capability Map

Why? You need to see, consider and share what will affect your ability to execute on improvements and foster more autonomy. Once you know your dependencies, a capability map allows you to see what would allow you to break or mitigate them.

How do you build it? Based on prior maps, the team identifies critical capabilities to deliver the desired outcome, votes to rank them and choose the top 8-10, then votes on each of the contributing factors (Skill, Training, Resources, Comfort) to create a score for each. For an extra dimension of clarity and actionability, a prime owner and a backup owner can be identified for each, ensuring someone (with support) is leading improvement efforts.

How do you use it? The map will identify where the team requires support, training, tooling, etc to reach a point where they can confidently break dependencies and improve delivery. The map allows us to target investment in improved capabilities where it is most needed, directly connected back to your key opportunities and outcome.

Similar to: Skills matrix, competency maps

The Capability map allows us to target investment in improved capabilities where it is most needed.

In the example, you can see how feature delivery is being affected by your two primary bottlenecks which are dependent on validation and test automation, how they are connected to external tooling and infrastructure, and a gap in capabilities. By combining the 4 Key Maps, you can trace a clear path from desired outcome to actionable next steps. In fact, this method will work for any situation, not just identifying how to improve your DORA metrics for technical teams, but any process or outcome you want to address.


The 4 Keys Maps in Context

Below you can see how each map addresses valuable questions, and flows from one to the next. As you progress through them, you add clarity, confidence, and context to support powerful decisions with maximum alignment.

In other words:

4x4 In Action

Below is an abstract overview of the process, from clarifying a desired outcome, to making data-driven decisions to achieve it.

Set the Compass

“We want to deliver software twice as fast (Deployment Frequency + Lead Time) without sacrificing quality (MTTR + CF%)”

Map the Path

1. You clarify the goal and contributing factors (Outcome Map), to explore the challenge/opportunity and align those involved. You can use one of the 4 key metrics as a starting point if you’re struggling to define an outcome. “We want to deliver changes every week” is a fine outcome to aim for, but your odds of success and ability to make key decisions depend on why that’s valuable, as well as what could get in the way.

2. You visualize and measure the current, ideal, and target state workflow (Value Stream Maps) to identify constraints and prioritize opportunities. The current state value stream map can inform immediate actions to mitigate constraints. Most teams immediately save 20% of their total process time by acting on a current state map. The ideal state map follows. It’s an opportunity to reimagine what’s possible, get inspired and excited about the future. The target state map sets our sights at a value stream reality 3-6 months in the future.

3. You identify external influences (Shared Services, Meetings, SLAs etc) on workflow (Dependency Map) to identify contributing factors beyond the team.

4. You identify internal influences (Skills, Resources, Support) on workflow (Capability Map) to identify contributing factors within the team.

5. Armed with clear direction, knowledge of flow and constraints, and influences beyond and within the team, you can make high quality, data-driven decisions as you navigate your next 3-6 months.

6. Every 3-6 months beyond outcome mapping, you can now re-evaluate your current state and set off in a new direction, or set your sights higher.

The 4 Key Maps allow you to see factors affecting your ability to progress towards any compass target

Case: Ambitious Software Delivery Team

Let's look at the example of a team aiming to boost their performance:

Starting from: “We want to deliver software twice as fast without sacrificing quality” we first map that outcome to reveal top contributing factors.

We created an outcome map to clarify a team understanding of the goal, why it's valuable, what obstacles could impact success, key investigations to improve odds of success, and indicators that will reveal progress. The team discovered that out of 8 members, everyone had a different idea of what would contribute to the outcome.

Next we mapped the value stream to reveal the visual flow and collect data on how the work is performing currently, and how it could be improved. We revealed handoffs, delays, tool challenges, meeting impacts and more. The team discovered that 30% of the current process was waste that could be eliminated within the next 3 days. Their target state map was less than 1/3 the time of the current state.

We then mapped out dependencies beyond the team and value stream in question to highlight external factors affecting our desired outcome. Approvals, cross-team meetings, SLAs, and shared services interactions were all clearly revealed as measurable impacts on the team. As one example, the team revealed that there was a 3 business day SLA with shared services that affected the team almost every single sprint.

Finally, we mapped out team capabilities to see what internal factors could affect our progress to the outcome. The team could see how their key capabilities and gaps were affected by available resources and support, as well as ownership and cognitive load. Along with other insights, the team identified ownership gaps and missing backup roles that were needed to support automated testing.

All of this clarity and data fed into prioritization and a clear roadmap defining next steps. The team set off working with awareness, confidence, and alignment. With a few weeks of effort, the team was operating far beyond their target outcome and on to bigger and better things.

Create your Turn-By-Turn Navigation

Let’s look at another case from the perspective of the team response, to illustrate the effects of each mapping exercise on a team working through them.

1. By starting with the clarity of the Outcome Map, the team is heading in one direction together. Having a map makes it easy to communicate within and beyond the team.

"We're all on the same page and aware of what may lie ahead. We revealed that our regulatory requirements will demand a careful mitigation strategy, but our lack of test automation is impacting quality and velocity."

2. By creating the Value Stream Maps, the team can clearly see and share where flow, quality, and value are being affected. The map is easily shared and understood within and beyond the team. A clear set of prioritized countermeasures can be formed based on the data revealed.

"We can now see what's affecting our ability to rapidly deliver high quality work, and what we can address or improve. We've revealed how quality issues are resulting in frequent rework and testing time is 1/3 of the total value stream. Our target state map shows that we can easily deliver in half the time with a few adjustments to workflow and simple automation."

3. By creating the Dependency Map, the team can see how their performance is being impacted by external factors that they could potentially influence or mitigate.

"We can now see how factors beyond our control are affecting our performance, and we can target dependencies to break or mitigate. Our dependence on creating tickets and a long SLA with our infrastructure team keeps us from quickly resolving test environment issues."

4. By creating the Capability Map, the team can see how their performance is being impacted by internal factors that could affect their ability to perform at a higher level, or take on more autonomy. The Value Stream Map has highlighted where constraints exist, and what capabilities are required to address them. You can see how lacking skill, support, or resources is influencing your constraints. You can also evaluate what dependencies we're capable of internalizing within the team.

"We can now see how contributions within the team are affecting our most critical constraints and our ability to address dependencies. If we can create a playbook to fix the simplest 80% of our build failures, we could resolve issues quickly within the team. We've identified a lead and backup to focus on building out test automation, and highlighted necessary support and training."

5. Armed with these compasses and maps, the team has a clear direction, knowledge of the landscape between their position and destination, and supporting metrics to keep them on track. They can easily tackle a prioritized set of initiatives aligned to their true north. They have velocity and quality metrics to inform them when something is pulling them in the wrong direction. They can see the path to success, which they have confidently defined to avoid obstacles and mitigate challenges. They can clearly and constructively collaborate along the way with everyone involved or affected.

"We can now see a clear path to success, together. Everyone from individual contributors to leadership can share the same view and alignment. We went from 8 different ideas of where to focus to 2 clear, prioritized opportunities. We know exactly what to do next, and can act safely in the knowledge that our compasses and maps will keep us on track. We can craft data-driven hypotheses that drive rapid, measurable experiments. We can periodically measure against our baseline to see, share and celebrate progress."

You may have noticed by now that the focus of these efforts is not merely DevOps or software delivery performance, but value stream performance. Well beyond the boundaries of software development and delivery, value streams are the true engines of value in every organization. They encompass every role, department, function, capability, and purpose throughout your company. This means that the skills you develop to improve value stream performance can be leveraged in any scenario where performance is constrained, and targeted improvements can have massive impacts throughout your organization. Value Stream Thinking and these techniques can empower you to drive powerful change.

DevOps is here. Value Streams are next.

This may seem new to you, and a lot of the applications of these techniques are new to the software world, but they are based on over 100 years of production improvement experience, research, learning, and results. Even the practice of Lean Software Development is over 20 years old. In areas and times of uncertainty and new capabilities, we tend to forget the past in order to look for possibilities with fresh eyes, but there is far more we can learn from the past than we often realize. Yes, software development should be a creative exercise, and we should continue to develop new techniques, but there is a great deal we can leverage from manufacturing value in the past (and presently in other industries) that will maximize our creative time, energy, and freedom. This approach brings you proven techniques from the past, and the latest winning techniques of the present, to power a better future.

Using the 4 Key Maps as an on-ramp to a highway of continuous improvement can be seen in the larger context of a Value Stream Enabling Methodology (pictured below). It depicts a clear process starting from outcome definition to scaling and sustaining change in three phases: Focus, Improve, and Thrive. By following this flow you can improve a single team, or an entire organization, with the same clarity, alignment, and confidence.

Start from clarity, build confidence, aim for success

Driving Forward

Once you’ve created your 4 Key Maps and analyzed the revealed data, you’re in an excellent position to create a roadmap. Armed with clarity, alignment, perspective, and data, you can synthesize the insights from your maps into clear opportunities. There are a number of ways to proceed, but prioritization is a key next step to ensure efforts are directed where they are most needed. In most cases, your opportunities will arise out of the maps like so:

Once that list is compiled, several items will be visible across multiple maps, and some perhaps across all, from various perspectives. For instance, testing could be identified as an obstacle in an outcome map in that it takes too long. It can be made visible and measurable in a value stream map with data to support that assertion, and with clear visual context on its place in the stream. It could be seen in the context of an external dependency on a QA team or shared resources that support testing in a dependency map, adding further context and clarity. Finally, testing will likely arise as a key capability within the team, and likely undersupported (hence the constraint). The capability map will reveal the shape of the constraint and how much effort it may take to address.

Once all of this is revealed, creating a matrix to set priority becomes a much simpler exercise. If you have a prioritization strategy already, you’re all set, but a tried and tested model is simply impact x effort (right). Your odds of success with this method are greatly increased by completing the 4 maps, but for even greater depth, you can leverage the RICE model. RICE adds Reach and Confidence to the scope, which can be partially derived by the key maps.

Once you’ve prioritized your list of experiments and initiatives, creating a roadmap you can feel confident in becomes a much easier endeavor. You’ll have all the clarity, alignment, context, data, and supporting evidence to craft a roadmap that impresses even the most skeptical executive, and resources that every individual contributor can leverage as navigation to get them moving in the right direction.

Whether you’re aiming to start improvement, continue successful efforts, or restart in a new direction, the 4x4 Method provides a clear, collaborative, and rapid kick start to get your team on the same page, and confidently moving in the right direction. If you’re interested in getting started with the 4x4 Method and looking for a boost, I’d be happy to guide you. Grab a spot on my calendar here

There you have it. From zero to off to the races. Start your engines!

Frequently Asked Questions

Why do so many teams struggle to improve?

To truly improve, you need to deliver change to people, process, and tools. To make change effective, you need to identify where that change is needed, and what needs to change. Take it from Google, this is complex territory. Below is Google’s map of factors affecting continuous delivery. Where should you start? How should you start? You need your teams aligned to a clear goal, you need a plan of action.

Why should we measure these four metrics?

The short answer is that you shouldn’t. If you’re wondering why the 4 Key metrics are important, you’re on the right track. You need to measure what matters to you, and your desired outcome. They’re just provided here as an example of how you can take any metrics and use them to measure performance while you pursue a specific outcome.

The 4 Key Metrics of DevOps have been successfully linked to high performing software delivery organizations. They’re the result of over 6 years of research and data collection from thousands of organizations along a spectrum of low to high performance. They correspond to the most critical factors affecting delivery performance, and each pair of velocity (Lead Time and Deployment Frequency) and quality (MTTR and Escaped Defects) balance each other to ensure sustained performance. You can absolutely choose any metrics you like and use the 4x4 Method with them. Tracking 2-4 balanced metrics means you can ensure you don’t pull too hard in any single direction and compromise elsewhere.

Are these the only metrics I need to improve?

The 4 Key Metrics are a great balance of velocity and quality measurement, but they are by no means exhaustive. To create the outcomes you want, you may have to measure any number of things that will allow you to track progress and performance. That could include anything from ‘how often developers are paged outside of business hours’ to ‘how many people skip sprint review’. However, any measurement impacts the systems being measured and it is important to take care not to fall prey to carelessly measuring the wrong things. Effects such as perverse incentives, the Hawthorne Effect and Goodhart's law should be taken into account.

Why do I need maps? Can’t I just start tracking the metrics?

Do you know why you want to track a particular metric? Do you know by which priority you’ll track them (because one will always win, and four people may choose four different metrics)? When you want a specific outcome, it makes all the difference in the world to be clear on the why, and the destination you’re heading in. Any metric is a means to an outcome. Mike Burrows created an acronym 2MBM to summarize why mapping helps with metrics:

MBM #1: Meaning before metric. Know why you want a metric before you start to measure. Mapping helps here as a way to clearly define your meaning (outcome) and share it with everyone involved and affected.
MBM #2: Measure before method. Know your measure before you decide how to address it. Mapping helps here to look at everything contributing to your measure, so you can design a method that considers all the variables in play.

Once you know where you’re headed, you’ll want to know what’s in your way! By mapping, you go from not knowing where to start, past knowing what you want, all the way to having a clear path to get there. It’s also important to remember that when it comes to measurement, qualitative measurement is always more valuable.

For every quantitative measure, there is a qualitative measure it is in service of.

Why is mapping valuable in DevOps?

Knowledge work in general, and software delivery specifically is incredibly complex. The systems, human interactions, and pace of change all combine to create a very challenging machine to debug and improve. You can’t get there with only quantitative metrics. You need to be able to see and share, and look beyond the data you have available. DevOps is about collaboration and aligning groups with different incentives, so shared understanding is like gold. If a picture is worth a thousand words, a good map is novel delivered in a second. The best maps can be easily and quickly understood by anyone from the C-Suite to the new hire, and across any department. In an age of constant collaboration, sharing the same understanding is a superpower.

Is this only for software teams?

The 4x4 Method is primarily focused on improving software delivery performance, but you could leverage The 4 Key Maps to improve outcomes in incident response, infrastructure automation, or even a sales process. Anywhere you need to improve the delivery of valuable outcomes, whether it’s to employees, the business or customers.

Who guides this process?

It pays to have skilled, outside facilitation for this process for a few reasons: A perspective from outside the organization is far less likely to bring an agenda, bias, or political influence to the process, which will drive superior results. A skilled facilitator knows where to dig, and when to move on. They will know what questions to ask, and what seems like a strange measurement based on experience with other teams. All of that saves valuable time and maximizes value. Gathering a team can be expensive if you’re not getting value from the session, and it can impact morale and trust to waste time. A skilled facilitator can likely complete a session in half the time, while keeping the team engaged. You may find you have skilled facilitators right now as agile coaches, scrum facilitators, dojo trainers, community of practice leads, or teachers. It’s ok to pilot the process with a facilitator and team willing to try and learn!

Who participates in mapping?

It’s important to involve representation from at least the responsible and accountable parties within a given value stream. That means if design is a part of the stream, there should be someone from design present during mapping. That also means that leadership and those who are able to change the system, workflow, and team must be present and involved. Once you identify your key bottlenecks, you can narrow the involved parties to those who are critical to those areas, but until the number of people exceeds a manageable 12, it’s better to include as many voices and perspectives as possible.

How should we build the maps?

Any shared visual board tool will work well for these maps. There are dozens of free tools that allow for real-time collaboration, and many allow for anonymous voting and other powerful facilitation capabilities. The important part is to build them collaboratively, or at least get fast and varied feedback from everyone involved and affected. These days, that means online, but this is all possible with a whiteboard, paper, post-its or almost anything you can write on together. For each map, you’ll likely need 2 hours for an extremely skilled facilitator with prior experience, or 4 hours for a new attempt.

When should we do this?

You know the saying: The best time to plant a tree is 20 years ago, and the second best time is today. If you don’t have clear performance data on your workflow today, clear pictures of what it looks like, and a clear path to better, wouldn’t you like to have that ASAP? After right away, this practice is valuable every 6 months. Let’s say after 4x4 you identify 3 high priority, relatively simple improvements to implement right away. It will likely take at least 3 months to see measurable progress, and then with 3 more months of progress and operation you’ll have enough data and experience with the current state that you can map it again and look to a new outcome. In six months you’ll have both made enough progress that you’ll have a new desired outcome, as well experience with new bottlenecks to identify and tackle.

How do we get accurate results?

The short answer is, you likely don’t need them (yet). If you are going from nothing to something, all that matters is relative measurement. When collecting data for a value stream map, it’s ok to guess that an activity takes 5 hours instead of its actual 6 hours, because your biggest bottleneck may be days of delay elsewhere. When creating a capability map, every member of the team votes on the capability of the team, and the values average out to what the team believes. Based on the prior mapping activities, the facilitator can act as a control to gauge the accuracy and dig deeper if a measurement seems unrealistic.

Consider a team that rates their test automation capabilities based on a high degree of end-to-end integration testing, but with knowledge of their outcome, value stream and dependency maps I knew that the tests failed often, and there was very little unit testing present in their process. In this case, the inaccuracy may have resulted in the team missing an improvement opportunity, but by leveraging the combined data present across all 4 maps, a clear insight arises.

Once you’ve run the 4x4 even a few times with just your team estimations, you may be able to measure more accurately by looking at your systems for data, but what you find may in fact be less accurate. For instance, you may have lots of data in Jira, but it’s only useful for measurement if people diligently record their activities on time. By getting a few guesses from across your team and averaging them out, you’re asking the people who know, and informing those who don’t. There’s a benefit to the human activity of that data collection. And again, perfect detail isn’t necessary. You’ll find bottlenecks, and you won’t be splitting hairs to prioritize them. Most teams find that 20% of their existing workflow is waste they can get rid of right away - that means seconds and minutes are not the problem.

What does this really look like?

In practice, the mapping process consists of 4-6 2 hour sessions to define, analyze, and design the value stream you target. You can create separate boards for each map, or combine them all. As you go from the raw input towards insight, you can rearrange the maps as you like, and easily export them to be shared throughout the organization. In the board below, you can see a value stream map (and an additional level of detail) in the top middle-to-right, a loosely defined dependency map in progress in the bottom left corner, and a target state map in the bottom centre. Capability mapping is elsewhere and connected here through a link. You can also see various other maps to collect supporting context - the middle row contains information about meetings, collected insights, and a mini retro. Beyond this level, Visible has a 4 phase, 12 week program that takes you all the way from discovery to launching your own Flow Engineering teams.

Example map board in progress

Where can I find more information?

Although these practices are based on decades of experience, study, and existing techniques, this is a new method. You can find a wealth of information about the techniques these maps are based on as well as the measurement context from these sources:

I’ll also be creating much more content on 4x4 which will be published on LinkedIn, Twitter and thinking.visible.is. You can also subscribe here to get the latest info as it’s published.

Get Started

How should I get started?

If you’re not ready to dive in on your own based on the examples here (and they are rather slimmed down for illustration), I’d be happy to guide you. Grab a spot on my calendar here. I’ll gladly give you a tour, and set you up in the right direction!


Steve Pereira    @SteveElsewhere    visible.is                Download as PDF