Measuring Developer Productivity & AI Impact – Reading List
Curated by Laura Tacho
Upcoming courses on developer productivity metrics and measuring AI impact: Sept 4, 2025 at either 9:00 or 17:00 CEST
In this reading list:
DORA State of DevOps Report Library
Other Articles and Videos on Productivity
The AI Measurement Framework
AI-specific metrics to enable organizations to track AI adoption, measure impact, and make smarter investments. When combined with the DX Core 4, which measures overall engineering productivity, get deep insight into how AI is providing value to developers, and what impact AI is having on organizational performance.
DX Core 4
A simple, unified framework that combines DORA, SPACE, and DevEx together and is designed to get you started quickly (I'm a co-author, you don't need to use DX to use this framework)
Also comes with DX Core 4 Benchmarks
SPACE Framework of Developer Productivity
The SPACE of Developer Productivity
The full paper. Myths & research about developer productivity and why it's difficult to measure.
You can use this as a quick reference guide for the SPACE framework: https://lauratacho.com/space-framework
DORA Metrics
DORA is a lot more than just their four key metrics, but that’s what they’re most known for.
DevEx Framework
DevEx: What Actually Drives Productivity - ACM Queue
This is the paper that introduces the DevEx framework (feedback loops, flow state, cognitive load)
GitHub’s Engineering System Success Playbook
Including this here as an example of these frameworks in action. GitHub bases their measurements off of DORA, SPACE, and Core 4. They have lots of thorough guidance on how to actually make change happen with these metrics.
GitHub's Engineering Systems Success Playbook
Industry benchmarks for Core 4 metrics
The AI Measurement Framework
This whitepaper covers the AI Measurement Framework, which I co-authored with Abi Noda, based on 12+ months of research.
Guide: How to measure GenAI adoption and impact
TL;DR Using GenAI isn't a guaranteed way to improve the developer experience and boost productivity. As with any initiative, make sure that you have a good strategy around both rollout and measurement of impact so you can get signals of whether it's doing its intended job.
There are three main ways to get data:
Guide to AI-Assisted Engineering
This is a practical and tactical guide of how to best use AI to accelerate development, based on research of over 180+ companies
For the second year in a row, DORA research finds that the use of AI coding assistants worsens software delivery performance. Also this year, Change Failure Rate is moving slightly differently from the other four key metrics. Plus more discussion about AI, platform engineering, and developer experience.
The 2023 State of DevOps Report explores how teams can improve organizational performance, employee well-being, and team output. Key findings highlight the importance of user-centricity, which increases performance by 40%, and faster code reviews, which boost software delivery by 50%. High-quality documentation and cloud infrastructure flexibility further enhance organizational success, while a generative culture fosters better outcomes.
The report also notes that underrepresented groups face higher burnout, often tied to increased repetitive work. Addressing equitable work distribution and cultural investment is essential for inclusive team success.
Maximizing Developer Effectiveness
A great overview of why it's not just about output.
The article "Maximizing Developer Effectiveness" by Martin Fowler focuses on improving developer productivity by fostering a supportive work environment rather than simply measuring output. Fowler emphasizes the importance of effective feedback loops, proper tooling, and collaboration. He explores the difference between efficiency (output over time) and effectiveness (delivering value), encouraging organizations to align their processes with developer well-being and team goals. Key strategies include reducing interruptions, promoting autonomy, and facilitating continuous learning to enhance long-term effectiveness.
How to Misuse and Abuse DORA Metrics
If you’ve not read this before, I am happy to be the person to share it with you. DORA metrics can be so useful when used appropriately, but you do need to understand what they are designed to measure. Otherwise, they can end up pointing you in some very wrong directions, as Bryan points out.
The article "How to Misuse and Abuse DORA Metrics" by Bryan Finster explores common pitfalls organizations face when implementing DevOps metrics, particularly those associated with continuous delivery (CD). Finster emphasizes that while tracking metrics is vital for improving delivery processes, improper use can lead to misguided conclusions and inefficiencies. He discusses common metric "anti-patterns" and suggests how organizations can use metrics constructively to pinpoint areas for improvement and scale knowledge across teams effectively.
A conversation with Nathen Harvey, DORA lead, and me, about when DORA metrics won’t help you, and other things to avoid.
Yes, you can measure software developer productivity (The now infamous McKinsey article)
AI-generated summary:
The article "Yes, you can measure software developer productivity" from McKinsey explains that while measuring developer productivity is challenging due to the complexity and collaboration involved in software development, it is feasible with the right approach. It highlights using a combination of metrics like DORA and SPACE, and introduces opportunity-focused metrics to provide a holistic view of productivity. The article emphasizes avoiding overly simplistic measures and leveraging insights to optimize both individual and team performance.
[Laura’s note - I don’t recommend measuring things the McKinsey way but adding it here for completeness]
Then please read Measuring developer productivity? A response to McKinsey from The Pragmatic Engineer
Measuring Developer Productivity via Humans
One thing I won’t shut up about is that developers are adults, and if they tell you something is preventing them from working efficiently, you should believe them. Plus, some parts of software development can only be captured by asking the people who participate in the process, because they are not captured in any workflow tool.
AI-generated summary:
In "Measuring Developer Productivity via Humans," Martin Fowler contrasts quantitative data (like lines of code or commit counts) with qualitative insights, emphasizing the latter's importance in measuring productivity. He argues that quantitative metrics are often misleading because they don't capture human elements such as collaboration, creativity, or problem-solving. Qualitative data, like peer feedback and the overall impact on the team's success, offers a richer understanding of productivity. Fowler advocates for a balanced approach that values human-centered evaluation alongside quantitative measures.
Using Metrics to Measure Individual Developer Performance — Laura Tacho
Should you use any of these metrics to measure individual productivity? Probably not.
AI-generated summary:
In her article, Laura Tacho discusses the challenges of using metrics to measure individual developer performance. She argues that metrics like lines of code or Jira tickets closed are flawed for evaluating individuals because they focus on output rather than outcomes and can incentivize wrong behaviors. Instead, Tacho recommends focusing on evidence-based performance management by measuring value delivered, team collaboration, and impact on business outcomes. She emphasizes the need for metrics tailored to company goals and roles, particularly for senior positions that focus on strategy rather than task-level output.