Gamification: a Game Changer for Managing Technical Debt?
A Design Study
What is Technical Debt (TD) ?
A metaphor
Non-optimal technical solutions are debt
Improving the solution is debt repayment
Code smells are a form of code TD: “it works but could be written better”
Once upon a time...
In a French company providing financial services for 7M users, the manager of the team developing the platform wants to improve the quality of their code.
The C-App project has 550k lines of code, developed for over 10 years.
Managers set up SonarQube (a quality management platform) to measure the quality of the code and identify code smells.
Result: 2,000 worker-days to fix over 15,000 code smells.
Problem Characterization
Large backlog of TD
TD repayment is a demotivating, repetitive, and boring task
Negative impact of TD creation is not visible
Challenge of TD management: providing developers with the proper mindset and motivation
J. Yli-Huumo, A. Maglyas, and K. Smolander, “How do software development teams manage technical debt? – An empirical study,” Journal of Systems and Software, pp. 1 – 24, 2016.
Solution: Gamification
Use of game design elements in non-game contexts
Extrinsic motivation (e.g. score) -> Intrinsic motivation (e.g. writing better code)
RQ1: How does gamification impact developer behaviour towards technical debt management?
RQ2: How can managers use gamification to help them monitor and drive developers’ actions on technical debt?
Introducing Themis
A layer on top of SonarQube & other linters
Adding or repaying technical debt generate negative or positive actions, resp.
Actions are rewarded with points (negative or positive)
Developers’ reputation is shown in a leaderboard
Challenges can be issued by managers
Scores are reset for every sprint
Main Developer Dashboard
List of latest actions
Leaderboard
Actions report
Goals�
Suggestions module
List & treemap views
Find files where points can be won �
Action Plans Setup
Time-boxed
Assigned to a set of developers
Reward specific actions
Validation of Themis: Timeline
December 2015
SonarQube was introduced in the C-App project
Discussion with ProMyze (The startup developing Themis)
April 2016: Deployment of Themis in the C-App project
July 2016: Surveying developers and managers
Survey overview
Three managers and eight developers responded
Closed- and open-ended questions
“What information do you find useful in [a dashboard]? How do you use this information?”
“How important is your score to you? Explain why.”
“Did Themis impact your motivation to reduce TD? How?”
Answers Analysis
Coded answers, with a set of provisional codes (TD management activities, extrinsic motivation, intrinsic motivation).
Codes were reviewed by a second coder. Three iterations of codes.
Follow-up questions with one manager and one developer.
Findings: 34 codes, 7 themes.
Themis promotes technical debt reduction
All respondents acknowledge a positive impact
“some seek to be on the top of the leaderboard.” [d3]
Only a preliminary study (three months): TD reduction remains “difficult to quantify for the time being.” [m1]
Themis made developers more attentive to the quality of the code they write
One developer mentioned paying “specific attention every day before committing code” [d2] as well using “quality measurement tools available to [them] before committing [their] code.” [d6]
Extrinsic factor: “There is less TD created because it is visible by others through the leaderboard, so we are more careful.” [d6]
Intrinsic motivation: “I mostly look to improve myself and having fewer negative points helps with that.” [d1]
Developers appreciate and follow the suggestions provided by Themis
One developer mentioned that he looks at the goals defined by managers “to see how make quick progress on the leaderboard.” [d1]
Developers “[use Themis] to look for files containing several anomalies to be fixed.” [d3]
One developer was not doing TD repayment: “making mass TD repayment actions is absolutely not a part of my work. It is uninteresting and there is some development that is way more important that needs to be done.” [d7]
Developers follow an opportunistic approach to TD repayment
“[To improve my score in Themis,] I fix the content of a file that I have to modify for my development. NB: It is not Themis which recommended this file to me (but my development needs), and it is not Themis which pointed me to the existing defects in this file (but the quality measurement tools in my IDE).” [d6]
Points reward are the main incentive for this strategy.
Themis provides monitoring and awareness of TD for individuals and the team
Developers mainly use the monitoring features of the main dashboard to “evaluate [their] work very quickly” [d6] and “see what are [their] areas of improvement” [d1]
Managers use it for TD prioritization and communication “the action reports at the end of the sprint [...] are useful to me in order to create action plans, or to provide reminders to the team or individuals, if needed.” [m3]
Themis allows managers to adapt TDM to the context of their project
“Anomalies created and fixed by developers are strongly connected to the project’s context (age, architectural choices, . . . ).” [m2]
Intentional TD persistence: some TD is not removed because it is too expensive, e.g., “cycles between packages”
Themis promotes ongoing discussion between stakeholders about technical debt
“Different proposals were made (penalizing more TD creation but leaving repayment points low, penalizing more the creation of more severe defects and reward more the correction of severe defects, etc) and a vote was taken to decide which rule to set up. There are multiple arguments here: some consider that creating TD is worse and has to impact more the developer than the correction of an equivalent TD, others will allow the ‘right to make a mistake’ and consider that a developer who fixed their mistake must be in a neutral state” [d6]
Guidelines
Nurture a positive team culture (playfulness)
Tailor gamification to suit different developers (experience, age, gender, personality)
Adapt the game to project context (new/legacy project, intentional debt)
Aim for seamless integration with existing processes and tools
Monitor how extrinsic motivators influence intrinsic motivation over time
Consider when not to gamify
Conclusions & Future work
First step in studying the impact of gamification in TD management
Guidelines that can be extended to build a theory
Longitudinal study to measure the impact over time
Measure the impact in other teams and projects