1 of 56

“Fortunately, maths!” - this talk is about how our approach to planning work, which can seem like a matter of taste or an art form, is actually governed by some mathematical principles. The first one is demonstrated by flipping a coin. Every time it lands on ‘heads’, we add 1 to a cumulative total. Every time it’s a ‘tails’, we subtract one. What do you think will happen?

NOTE: the speaker notes here were originally exactly what I planned to say for each slide. As I became more confident with the material, I moved away from following them so closely. And as I moved the slides around, some speaker notes from different slides got duplicated across. I’ve kept them in because there are definitely some slides for which they’re still useful, but you shouldn’t consider it a transcript. -

2 of 56

What is the point?

Ask the audience

OK, so we’re here to deliver value. I feel like a dirty capitalist saying that, but for most of us, our salaries are paid on the assumption that our employers get more value from us than they pay out. And value doesn’t have to mean money. Some of us are lucky enough to work for non-profits or other contexts where value is expressed in something other than dollar amounts.

For what it’s worth, I’ve also become convinced that, by optimizing our processes to deliver value, we can make our experiences of work fairer, more fun, and avoid rewarding bad behaviours. But that is a topic for an entirely different talk.

So if we’re here to deliver value, we should optimize our processes so that we take actions which maximize value. I’m going to claim that most companies are not doing a great job of this, and look at a few ways that we can use some maths to work out how to do it better.

3 of 56

“Fortunately, maths!”

Or: process for nerds

Tom Stuart

4 of 56

Cost of delay

How many managers do we have in the room? (hands up)

Cool, how many of you know (roughly) how much it costs to run your team (salaries, taxes, benefits, overhead) per month?

Cool. How many of you track the average lead time for delivery of a request by your team?

Cool. How many of you know the lifecycle value that could be realized by any given request?

Not cool. OK, don’t feel bad. I certainly don’t for my teams. My claim is that we’re extremely good at quantifying some of our costs (e.g. cost of running a team, which I’m going to call cost of capacity), but not at others. And if we want to optimize our processes to maximize value, we need to quantify everything.

So what’s cost of delay?

For a given action (think: feature release, but this applies to lots of other things too), what is the total value it can realize, per day (what is the lifecycle profit of this feature, for example)? Now, how many days is this action waiting somewhere (sitting in a backlog, waiting for the next feature release, waiting on the testing department)? Multiply the one by the other and you have the cost of delay.

5 of 56

6 of 56

Cost of delay

“Look at all the value we could have”

In other words, it’s all the money you could be making if you did this action sooner.

OK, so we’re not good at quantifying this, and we need to quantify everything. But it’s especially bad that we’re not quantifying this, because normally the cost of delay dwarves all other costs. We’ll see that later.

You may be wondering why I’m talking in terms of cost rather than value. Why am I being so negative? Well, we want to make decisions based on a number of interacting factors, which means we need our units to be the same everywhere. Since cost of delay and cost of capacity interact, it’s just easier to talk about cost on both sides. Otherwise we have to do some mental gymnastics to try and invert ‘cost of capacity’. Something like “value of not employing people”? No thanks!

7 of 56

Lead time

The most important measure of an engineering team’s success

In other words, it’s all the money you could be making if you did this action sooner.

OK, so we’re not good at quantifying this, and we need to quantify everything. But it’s especially bad that we’re not quantifying this, because normally the cost of delay dwarves all other costs. We’ll see that later.

You may be wondering why I’m talking in terms of cost rather than value. Why am I being so negative? Well, we want to make decisions based on a number of interacting factors, which means we need our units to be the same everywhere. Since cost of delay and cost of capacity interact, it’s just easier to talk about cost on both sides. Otherwise we have to do some mental gymnastics to try and invert ‘cost of capacity’. Something like “value of not employing people”? No thanks!

8 of 56

Randomness

9 of 56

10 of 56

11 of 56

12 of 56

Randomness in our processes

So we know that a random process won’t fix itself. Let’s talk about how we organize our work.

Let’s say we have a process that will be pretty familiar to a lot of people here:

First, a stakeholder makes a feature request.

We do some work with them to clarify the requirements, write it up as some stories, and add them to our backlog.

When an engineer on the team is finished with their current task, they take the next task from the backlog and work on it.

Is this a good process?

Let’s look at both ends:

On the engineering team’s side, we know that we have variability. People get sick, people become less motivated, tasks take longer than expected, some turn out to be impossible. Most engineering teams have a Markovian service process - the time between completion of tasks ranges from ‘none at all’ (actually, there’s no work to be done) to ‘infinite’ (this task is impossible). And the service time for any given task is _memoryless_. It doesn’t depend on the service time of the previous task. So sometimes we’re meeting demand, and sometimes we’re not.

Just like in the coin flip example, the most likely outcome is that we process exactly as much this week as we did last week. But all the other outcomes become relatively more likely over time. If we fail to meet demand in one time period, we have to exceed it in the next two to return to a queue depth of 0, and it’s just as likely that we’ll fail to meet it in one of those.

On the demand side, we also have variability. The world changes, and so what people want changes. We discover bugs in production. The designer’s out of the office for 2 weeks. The arrival process is also Markovian. Sometimes we get two tasks coming in with no time in between them, sometimes it can take weeks. If we have a spike in demand in one time period, we need two subsequent time periods of lower demand, and constant capacity in the engineering team, to get our queue back to 0.

This kind of queue is called an M/M/1/∞ queue, and fortunately, mathematicians have studied these queues in depth.

13 of 56

M/M/1/∞

Arrival:

Markovian

Service:

Markovian

Server

Maximum queue size

So we know that a random process won’t fix itself. Let’s talk about how we organize our work.

Let’s say we have a process that will be pretty familiar to a lot of people here:

First, a stakeholder makes a feature request.

We do some work with them to clarify the requirements, write it up as some stories, and add them to our backlog.

When an engineer on the team is finished with their current task, they take the next task from the backlog and work on it.

Is this a good process?

Let’s look at both ends:

On the engineering team’s side, we know that we have variability. People get sick, people become less motivated, tasks take longer than expected, some turn out to be impossible. Most engineering teams have a Markovian service process - the time between completion of tasks ranges from ‘none at all’ (actually, there’s no work to be done) to ‘infinite’ (this task is impossible). And the service time for any given task is _memoryless_. It doesn’t depend on the service time of the previous task. So sometimes we’re meeting demand, and sometimes we’re not.

Just like in the coin flip example, the most likely outcome is that we process exactly as much this week as we did last week. But all the other outcomes become relatively more likely over time. If we fail to meet demand in one time period, we have to exceed it in the next two to return to a queue depth of 0, and it’s just as likely that we’ll fail to meet it in one of those.

On the demand side, we also have variability. The world changes, and so what people want changes. We discover bugs in production. The designer’s out of the office for 2 weeks. The arrival process is also Markovian. Sometimes we get two tasks coming in with no time in between them, sometimes it can take weeks. If we have a spike in demand in one time period, we need two subsequent time periods of lower demand, and constant capacity in the engineering team, to get our queue back to 0.

This kind of queue is called an M/M/1/∞ queue, and fortunately, mathematicians have studied these queues in depth.

14 of 56

Queues

So we’re going to get into queueing theory today. Tonnes of people have spent tonnes of time studying queues, and they’ve found out some really counterintuitive things. The coin toss example is just one. Here’s another:

You have a bank cashier, and it takes them on average 10 minutes to serve a customer. On average, 5.8 customers arrive per hour. How long do customers normally have to wait?

No more than 10 minutes, right? Since on average the bank cashier can get through 6 customer per hour, they have enough time to deal with everyone. So you only have to wait until the next person is done, right?

Wrong.

The average wait time for these customers is… almost 5 hours!

OK, that’s weird. Queues might be weird?

What happens when we add another cashier and double our capacity? What’s the average wait time now? Just under 2 hours? More? Less?

3 minutes!

OK, so it’s clear that unless we understand how queues really work, rather than how we think they work, we’re going to get the answers drastically wrong. Why does that matter?

15 of 56

Queues increase delay cost

16 of 56

Queues increase the cost of our process

17 of 56

Queues delay feedback

18 of 56

Visualizations: Alicia Hickey - @lonelypeanut

19 of 56

Visualizations: Alicia Hickey - @lonelypeanut

We know that randomness is not our friend, so we decide we need to reduce the variability. This is the approach taken by our Six Sigma friends. Does it work?

Sort of.

The mathematics of queues tells us that variability increases queue size linearly.

Queue size = p^2/1-p * (C^2(arrival) + C^2(process) / 2)

C = Coefficient of variation = Standard Deviation / Mean

A poisson distribution has a C of 1. A uniform distribution has a C of 0.

Some examples at 90% utilization:

C = 1: 8.1

C = 2: 32.4

C = 0.8: 5.184

C = 0.5: 2.025

At 80%:

C = 1: 3.2

C = 2: 12.8

C = 0.8: 2.048

C = 0.5: 0.8

(Draw the graph)

So cutting our variation on both sides by half achieves roughly the same effect as cutting our utilization by just 10%! And it does nothing at the extreme end where capacity utilization is close to 100%, because infinity * anything is still infinity.

20 of 56

Visualizations: Alicia Hickey - @lonelypeanut

We know that randomness is not our friend, so we decide we need to reduce the variability. This is the approach taken by our Six Sigma friends. Does it work?

Sort of.

The mathematics of queues tells us that variability increases queue size linearly.

Queue size = p^2/1-p * (C^2(arrival) + C^2(process) / 2)

C = Coefficient of variation = Standard Deviation / Mean

A poisson distribution has a C of 1. A uniform distribution has a C of 0.

Some examples at 90% utilization:

C = 1: 8.1

C = 2: 32.4

C = 0.8: 5.184

C = 0.5: 2.025

At 80%:

C = 1: 3.2

C = 2: 12.8

C = 0.8: 2.048

C = 0.5: 0.8

(Draw the graph)

So cutting our variation on both sides by half achieves roughly the same effect as cutting our utilization by just 10%! And it does nothing at the extreme end where capacity utilization is close to 100%, because infinity * anything is still infinity.

21 of 56

Visualizations: Alicia Hickey - @lonelypeanut

We know that randomness is not our friend, so we decide we need to reduce the variability. This is the approach taken by our Six Sigma friends. Does it work?

Sort of.

The mathematics of queues tells us that variability increases queue size linearly.

Queue size = p^2/1-p * (C^2(arrival) + C^2(process) / 2)

C = Coefficient of variation = Standard Deviation / Mean

A poisson distribution has a C of 1. A uniform distribution has a C of 0.

Some examples at 90% utilization:

C = 1: 8.1

C = 2: 32.4

C = 0.8: 5.184

C = 0.5: 2.025

At 80%:

C = 1: 3.2

C = 2: 12.8

C = 0.8: 2.048

C = 0.5: 0.8

(Draw the graph)

So cutting our variation on both sides by half achieves roughly the same effect as cutting our utilization by just 10%! And it does nothing at the extreme end where capacity utilization is close to 100%, because infinity * anything is still infinity.

22 of 56

How do we fix this?

23 of 56

Vizualizations: Alicia Hickey - @lonelypeanut

Some clever people worked out the formulas for this, but they’re not too hard to understand.

Cost of capacity is the total cost you pay to run your team, multiplied by the amount you’re not using them

Cost of delay is the cost you pay per time slice for a thing not being delivered, multiplied by the queue time (which is arrival rate divided by capacity - utilization)

So we’re trying to optimize the total cost. How would you describe the shape of this curve? (Prompt the audience)

It’s U-shaped. That’s right! And U-curves are our friends. They have some really important characteristics:

First, we know that if we’re at the extreme ends, we’re not optimal. So running a process at 100% utilization is always wrong.

Second, it’s really flat at the bottom. So the difference between the optimal level of capacity utilization and being off by even 10%, if we’re even close to the optimum value, is actually very small.

This means: we don’t have to get it perfect.

Our capacity utilization can be off. At 90% utilization, the total cost in this example is 2120. At the optimum, it’s around 1640. At 70%, it’s 1820. These are small amounts compared to the differences at the extremes.

Our quantification of cost of delay or cost of capacity can be off. As long as it’s in the right order of magnitude, we’ll be able to get to the bottom of the U. If cost of delay is twice what I plugged in for this graph, the optimal capacity utilization moves to around 70%. If we keep it at 80%, which is what our current graph tells us to do, the total cost difference is less than 5%.

Third, it has really steep sides.

This means: if we are at the extreme levels and we start moving toward the middle, we’ll start to see really dramatic total cost reductions.

But we need to quantify cost of delay and measure total cost to be able to see that we’re making them.

24 of 56

Focus

25 of 56

Finishing, not starting

26 of 56

Minimize handoffs, maximize sharing

27 of 56

Small batches

28 of 56

Let’s get practical

29 of 56

30 of 56

31 of 56

32 of 56

Completed: 5

(EXCESS CAPACITY: 2)

Capacity utilization: 93%

46 of 56

“But we prioritize our backlogs!”

47 of 56

Vizualizations: Alicia Hickey - @lonelypeanut

Task	Cost of delay	Duration
	10	3
	3	3
	1	3

48 of 56

Vizualizations: Alicia Hickey - @lonelypeanut

Task	Cost of delay	Duration
	10	3
	3	3
	1	3

49 of 56

Vizualizations: Alicia Hickey - @lonelypeanut

Task	Cost of delay	Duration
	3	1
	3	3
	3	10

50 of 56

Vizualizations: Alicia Hickey - @lonelypeanut

Task	Cost of delay	Duration
	3	1
	3	3
	3	10

51 of 56

51/27

Vizualizations: @lonelypeanut

Task	Cost of delay	Duration
	10	7
	7	3
	1	10

52 of 56

5227

Vizualizations: @lonelypeanut

Task	Cost of delay	Duration
	10	7
	7	3
	1	10

And here’s how it looks when we take into account both cost of delay and duration. The shaded area is NX smaller. That’s a huge difference in cost.

So before we can make the argument that queues don’t matter for cost of delay, we need to show that we correctly sequence work based both on its expected value and its expected duration. If we’re not good at estimating one or both of these values, we’re going to end up with prioritization rules which are extremely significant for our organizations. And we know that correctly estimating programming tasks is not our forté. And that’s before we get into the other costs of queues.

As a side note, there’s a way to get around needing to prioritize, even when you have queues, but I’m not going to have time to cover it in this talk.

53 of 56

A note about supermarkets

54 of 56

55 of 56

Fortunately, a summary!

We’re here to deliver value.
We’re not good at quantifying cost of delay.
Our processes are subject to randomness.
Running a process at full capacity will cause queues.
WIP limits control queues by taking advantage of randomness.
Prioritization doesn’t fully solve our problems - but can help!

1 of 56

2 of 56

3 of 56

4 of 56

5 of 56

6 of 56

7 of 56

8 of 56

9 of 56

10 of 56

11 of 56

12 of 56

13 of 56

14 of 56

15 of 56

16 of 56

17 of 56

18 of 56

19 of 56

20 of 56

21 of 56

22 of 56

23 of 56

24 of 56

25 of 56

26 of 56

27 of 56

28 of 56

29 of 56

30 of 56

31 of 56

32 of 56

33 of 56

34 of 56

35 of 56

36 of 56

37 of 56

38 of 56

39 of 56

40 of 56

41 of 56

42 of 56

43 of 56

44 of 56

45 of 56

46 of 56

47 of 56

48 of 56

49 of 56

50 of 56

51 of 56

52 of 56

53 of 56

54 of 56

55 of 56

56 of 56