1 of 45

The Complexity Trap: Reevaluating our incentives

2 of 45

Hello there!

Edu Ferro Aldama

Head of Platform

@eferro

@eferro@hachyderm.io

https://eferro.net

3 of 45

Agenda:

Incentives
Concepts

Basal Cost
Essential/Accidental Complexity
Software ages like…
Value

Lean Software Development & Simplicity
Conclusions

4 of 45

Premises about Software

Software development is a team activity
Software is a means to an end
No trade-off between Quality and Speed
High degree of uncertainty
Less than half of ideas have a positive impact
Software has basal cost / is a liability
Options are valuable

4

5 of 45

Incentives

6 of 45

Technology Collector / Resume-Driven Development

7 of 45

Technology Collector / Resume-Driven Development

Why?

Keeping Up with Rapid Technological Advancement: The tech industry evolves rapidly. New tools, languages, and frameworks emerge continuously. Professionals often feel pressure to keep up to remain relevant.
Personal & Professional Growth: Some engineers believe that diversifying their skills is crucial for career advancement. By showcasing a wide array of technologies on their resume, they hope to appeal to a broader range of employers.
Curiosity & Passion: Engineers are, by nature, curious problem-solvers. The allure of a new technology, especially one that promises to simplify tasks or offer better performance, can be irresistible.
Peer & Community Pressure: The tech community, including forums, conferences, and even casual conversations between peers, often buzzes with discussions on the "next big thing." This creates a FOMO (Fear of Missing Out) effect.

8 of 45

“We get paid to solve difficult problems, and if there aren’t any, we make them up.”

Edu Ferro

9 of 45

Yet another…

Blow your audience away with attractive visuals

Framework

Distributed DB

Scheduler

Scalable Web Server

10 of 45

Hero Culture

We see two attitudes.

One of Proactive Prevention: Acting immediately reduces risks and prevents larger problems. It doesn't seek recognition, just to do its job well.

Another is Waiting until the problem is visible and critical to solve it seeks public recognition.

If you have a culture that rewards (publicly) the last-minute solution to a problem instead of its proactive elimination, you will generate a toxic hero culture where individual efforts and heroic actions are rewarded. I believe this is one of the worst cultures to work in.

Origin of the Hero Culture:

- Human Psychology / Hero admiration

- Tight Deadlines and High Pressure -> long Hours

- Lack of monitoring, testing, observability, safety -> unforeseen issues that require “heroic” efforts

- Visibility of last-minute efforts

Consequences of the Hero Culture:

- Increase in avoidable problems.

- Stress and wear on the team.

- Incentivizes reactive behavior over preventive behavior.

- Devalues silent but essential work.

The Antidote: Team Learning and Continuous Improvement.

11 of 45

“Simplicity is a great virtue but it requires hard work to achieve it and education to appreciate it. And to make matters worse: complexity sells better.”

Edsger W. Dijkstra

12 of 45

(Wrong) Incentives

13 of 45

Concepts

14 of 45

Edsger W. Dijkstra

Building Fallacy

Another challenge in software development is the fallacy of construction.

I'm not sure why, but traditionally we have used the building construction metaphor when talking about software development. This metaphor has never made sense and could lead us to think that each feature only incurs costs during development, meaning a team would spend all its time creating features one after the other.

If we're to draw a fitting metaphor, software product development more closely resembles gardening or agriculture. Just as we sow seeds and cultivate plants, requiring regular watering, nurturing, and pruning, in software development we plant ideas that demand continuous care, adaptation, and refinement.

This metaphor didn't make sense before, and now it makes even less sense. �Currently, our systems have the following characteristics:

Continuous Delivery and Adaptation
Rapidly changing environment (cloud, AI, OS, etc.)
Increased number of dependencies (open source libs, SaaS, etc.)

15 of 45

Building Fallacy: Team Capacity

16 of 45

Building Fallacy: Team Capacity

17 of 45

Edsger W. Dijkstra

https://www.eferro.net/2021/02/basal-cost-of-software.html

Basal Cost

18 of 45

Basal Cost

KLO + Cognitive Load + Complexity

19 of 45

Essential vs Accidental Complexity

20 of 45

Essential vs Accidental Complexity

21 of 45

Essential vs Accidental Complexity

22 of 45

Essential vs Accidental Complexity

23 of 45

Essential vs Accidental Complexity

24 of 45

Edsger W. Dijkstra

Essential vs Accidental Complexity

25 of 45

Edsger W. Dijkstra

Software ages like…

26 of 45

Value:

Beyond Revenue Growth

27 of 45

I

Increase Revenue

Increasing sales to new or existing customers. Delighting or Disrupting to increase market share and size

R

Reduce Costs

Costs that we are currently incurring, that can be reduced. More efficient, improved margin or contribution

P

Protect Revenue

Improvements and incremental innovation to sustain current market share and revenue figures

A

Avoid Cost

Improvements to sustain current cost base. Costs we are not currently incurring but may do in the future

Value

28 of 45

Lean Software Development & Simplicity

29 of 45

Edsger W. Dijkstra

Lean Software Development

Reduce Waste

Amplify Learning

Decide as late as possible

Deliver as quickly as possible

Empower the team

Build with Integrity / quality

Optimize the whole

30 of 45

“The biggest cause of failure in software-intensive systems is not technical failure; it’s building the wrong thing.”

Mary Poppendieck

31 of 45

Game Over

I understand that everyone knows the game Tetris. I believe our profession is somewhat similar to this. We continuously make decisions (rotating the pieces, moving them) while accumulating software (which should provide value to our users).

In the game, if we postpone decisions until the last responsible moment, we have more information (like the next piece) to make better decisions. The same principle applies when developing software systems.

There is a capacity limit in the team, and when we exceed it, the team simply cannot innovate (and can only engage in fire fighting).

In this context, we'll explore the winning strategy (and the one that induces the least anxiety). Do we work in large steps, with decisions always being high-risk and hard to reverse, or do we work in very small steps, with feedback, postponing decisions, and making them reversible whenever possible?

32 of 45

Game Over

Waste

33 of 45

Game Over

Waste

Features to adapt or remove

34 of 45

Game Over

Waste

Features to adapt or remove

35 of 45

Lean Software Development / Small Safe Steps

Reduce waste
Decide as late as possible
Deliver as quickly as possible
Build with Integrity / Quality

+

Simplify proactively

36 of 45

Conclusions

37 of 45

Basal Cost

Essential vs Accidental Complexity

Software ages like milk

Value (Increase, Protect, Reduce, Avoid)

38 of 45

From:

Complexity only growing
Any “Shiny” Technology
The “Yet another” syndrome…
Being a Hero

To:

Proactive Simplification
Reduce Waste / Lean
Increase, Protect, Reduce, Avoid
Lean Software Development
Low drama flow

We urgently need to change the current incentives.��We need to be able to change the incentives in our teams and companies from these self-destructive incentives ("More is Better", collecting technologies, being a hero...) to new incentives where we proactively reward simplification, waste reduction, and working in a lean manner.

This is what will allow us to continuously evolve systems and teams with a low level of drama while providing real value to our users.

This shift is essential since the Basal Cost is more critical than ever due to the rapid pace of changes environment (cloud, AI, deployment models, OSs, etc.) and the fact that systems now have an inherently higher essential complexity. That's why it's currently crucial to minimize that accidental complexity as much as possible. Creating as little accidental complexity in the process as possible and proactively eliminating it as we better understand the problem.

Perhaps these incentives don't resonate with the people at McKinsey, but why should we care?

McKinsey has no real experience on how this profession truly works.

We're not code monkeys and we're not paid by the line; we're paid to keep complexity under control.

39 of 45

Right Incentives Drive Success

&

Simplify, Simplify, and Simplify

40 of 45

“Art is the elimination of the unnecessary ”

Pablo Picasso

41 of 45

Via Negativa (Nassim Taleb)

Nassim Nicholas Taleb, in his book "Antifragile: Things That Gain From Disorder," provides a general definition of via negativa as a focus on what something is not, or an indirect definition. In practice, it is a recipe for what to avoid or what not to do, emphasizing subtraction rather than addition, such as in medicine[1].

The concept is also linked to the idea of inversion, a problem-solving technique that involves thinking through problems in reverse. Instead of asking how to succeed, one identifies how something could fail and then avoids those pitfalls. This approach is exemplified by figures like Charlie Munger and Warren Buffett, who prioritize avoiding bad decisions and minimizing downside risk[1][2].

Via negativa is considered powerful because negative knowledge, or knowing what is wrong, is often more certain and useful than positive knowledge, which can be prone to error. By focusing on what not to do, individuals and organizations can avoid mistakes and survive adverse conditions, thereby positioning themselves to benefit from positive outcomes when they do occur[1].

Citations:

[1] https://www.wealest.com/articles/via-negativa

[2] https://www.artofmanliness.com/character/advice/via-negativa-adding-to-your-life-by-subtracting/

42 of 45

What will you eliminate Next Monday?

How will you Simplify?

How will you change your work approach?

43 of 45

Take Action.

Drive Simplicity.

44 of 45

https://cocomaterial.com

45 of 45

Thank You! there!

Edu Ferro Aldama

Head of Platform

@eferro

@eferro@hachyderm.io

https://eferro.net