ABCDEFGHIJKLMNOPQRSTUVWXYZAAABACADAEAFAGAHAIAJAKALAMANAOAPAQARASATAUAVAWAXAYAZBABBBCBDBEBFBGBHBI
1
role
role_6_TEXT
industry
industry_35_TEXT
industry_32_TEXT
experience
Q33
Q33_3_TEXT
fieldQ34Q36Q37
used_toolkit_before
used_toolkit_before_7_TEXT
purpose
purpose_6_TEXT
toolkit_experience
what_is_fairness
what_is_fairness_3_TEXT
Q43Q44
Q44_3_TEXT
Q45
Q45_3_TEXT
Q42_1Q42_2Q42_3Q42_4Q42_5Q42_6Q42_7Q42_8Q46
opportunistic
would_explore
would_explore_4_TEXT
process
solutionism
solutionism_5_TEXT
Q47
time_spent
Q48Q49Q39experts
experts_3_TEXT
can_find_expert
obstacles_1
obstacles_2
obstacles_3
obstacles_4
obstacles_5
obstacles_6
obstacles_7
obstacles_8
org_time
convince
convince_3_TEXT
fastest
anything_else
2
ID
What is your role? - Selected Choice
What is your role? - Other: please specify - Text
What is your industry domain area? - Selected Choice
What is your industry domain area? - Financial services: other - Text
What is your industry domain area? - Other (please specify) - Text
How many years of experience do you have in your role?
Where are you currently located? - Selected Choice
Where are you currently located? - Other (please specify) - Text
Do you have any academic or practical background in the following fields?
How many years of experience do you have in coding (any language)?
How many years of experience do you have in machine learning (ML)?
How many years of experience do you have with issues of ML fairness?
Are you familiar with any of the below open source fairness toolkits? - Selected Choice
Are you familiar with any of the below open source fairness toolkits? - Other - Text
What was your purpose for using the toolkit? - Selected Choice
What was your purpose for using the toolkit? - Other - Text
For the toolkits you have used before, please describe your experience.
What are some of the following options that most likely be your reason(s) to explore an open source fairness toolkit? - Selected Choice
What are some of the following options that most likely be your reason(s) to explore an open source fairness toolkit? - Other (please specify) - Text
Would you use a fairness toolkit off the shelf, extend an existing tool/codebase, or develop your own?
Why do you prefer an out-of-the-box toolkit? - Selected Choice
Why do you prefer an out-of-the-box toolkit? - Other - Text
Why do you prefer to extend an existing code base or to develop your own? - Selected Choice
Why do you prefer to extend an existing code base or to develop your own? - Other - Text
To what extent do you agree: I expect open source fairness toolkits to be able to do the following: - Help me surface the potential sensitive features.
To what extent do you agree: I expect open source fairness toolkits to be able to do the following: - Help me quantify the unfairness in the dataset.
To what extent do you agree: I expect open source fairness toolkits to be able to do the following: - Help me quantify the unfairness in the model.
To what extent do you agree: I expect open source fairness toolkits to be able to do the following: - Help me mitigate the biases that exist in the dataset
To what extent do you agree: I expect open source fairness toolkits to be able to do the following: - Help me mitigate the biases that exist in the model
To what extent do you agree: I expect open source fairness toolkits to be able to do the following: - Help me mitigate the biases that exist throughout the ML pipeline and process
To what extent do you agree: I expect open source fairness toolkits to be able to do the following: - Prove that bias has been mitigated with quantitative results.
To what extent do you agree: I expect open source fairness toolkits to be able to do the following: - Help me generate visualizations to show the existing biases.
Would you expect a fairness toolkit to do anything else not listed above?
If using a fairness toolkit, to what extent do you agree with the statement: it is important for me to understand the low-level implementation details of the methods for determining fairness (i.e. specifics of how they work) in a toolkit.
If there were a fairness-related concern in your model and use case, would you or any of your team members be likely to explore it? - Selected Choice
If there were a fairness-related concern in your model and use case, would you or any of your team members be likely to explore it? - N/A (please explain) - Text
If you were to have a fairness-related concern, is there a defined process in your organisation to raise and address it?
If you had a fairness-related issue in your model and use case, how would you go about dealing with it? - Selected Choice
If you had a fairness-related issue in your model and use case, how would you go about dealing with it? - Hand off to a different team member (please specify to whom) - Text
You selected "Non-technical remedy" in the last question. Could you please explain?
How much time would you typically have available and envisage spending to explore a fairness concern?
Approximately how many hours would you spend?
Approximately how many days would you spend?
Please explain
Would you or your team members seek any of the below domain experts for assistance while using a fairness toolkit? - Selected Choice
Would you or your team members seek any of the below domain experts for assistance while using a fairness toolkit? - Other - Text
Do you agree or disagree with the statement: I am confident that I (or a team member) could find the domain experts for assistance while using a fairness toolkit at my working place.
To what extent do the following factors influence your decision on whether or not to perform fairness testing? - Time
To what extent do the following factors influence your decision on whether or not to perform fairness testing? - Budget
To what extent do the following factors influence your decision on whether or not to perform fairness testing? - Stakeholder demand
To what extent do the following factors influence your decision on whether or not to perform fairness testing? - Organisational process
To what extent do the following factors influence your decision on whether or not to perform fairness testing? - Personal interest
To what extent do the following factors influence your decision on whether or not to perform fairness testing? - Expertise in fair ML algorithms
To what extent do the following factors influence your decision on whether or not to perform fairness testing? - Domain expertise
To what extent do the following factors influence your decision on whether or not to perform fairness testing? - Expertise in social aspects of fair ML
Would your organisation provide sufficient time and resources to explore and address the fairness-related concern?
What do you think are the key drivers for your organisation to invest time and resources into fairness testing and assessment? - Selected Choice
What do you think are the key drivers for your organisation to invest time and resources into fairness testing and assessment? - Other - Text
Do you agree or disagree with the statement: Given the obstacles to fairness testing, I (or my team members) look for the fastest available solution.
Anything else you'd like to share with the researchers? In particular, we would like to know about the fairness-related challenges you have faced in your role and your experience using fairness toolkits.
3
1Technical leadProfessional services (e.g. law, tax, audit, consulting)5-10 yearsUKSocial sciences,Computer science,Machine learning5-10 years5-10 years3-5 yearsAI Fairness 360,Fairlearn,Aequitas Tool,What-if ToolLearn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness,I need to build a fairness tool for my own working pipeline and I’m using the toolkit as a reference for implementationsUse as an out-of-the-box solution for my own fairness-related challenges,As a starting point to build my own tool, as a reference for implementationsExtend an existing code baseNeed for customisation,Need to understand the low-level implementation,Need to integrate with existing workflowAgreeAgreeStrongly agreeDisagreeDisagreeStrongly agreeAgreeStrongly agreeIt may not be possible for a technical toolkit to mitigate all biases, especially if they were introduced in the design process, but a toolkit should help the user identify what types of biases exist and suggest both technical and non-technical mitigation strategies.Somewhat agreeYesYesSoftware-based testing and intervention - open source tools,Software-based testing and intervention - in-house tools,Follow a standard organisational process,Non-technical remedyWe have a risk assessment questionnaire that includes ethical assessmentLess than a day4Unless the project is specifically to look for fairness-related concerns, there is limited time and budget for this.Legal and regulatory experts,Business domain experts,Reputational risk expertsSomewhat agreeExtremely importantExtremely importantExtremely importantExtremely importantVery importantVery importantNeither important nor unimportantNeither important nor unimportantYesRelevant regulations,Concerning initial findings,Compelling visualisations demonstrating initial findings,Reputational risk,Liability and compliance risk,Customer / end user demand or requirementsStrongly agree
4
2Other: please specifyData EngineerProfessional services (e.g. law, tax, audit, consulting)3-5 yearsUKEngineering,Computer science5-10 years3-5 years1-2 yearsscikit-fairness,AI Fairness 360,Aequitas Tool,What-if ToolLearn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness,I need to build a fairness tool for my own working pipeline and I’m using the toolkit as a reference for implementationsMost toolkits involved loading a dataframe into the python package within a Jupyter (IPython) enviroment. Here I could explore and play with the various metrics that the tool would generate to determine the "fairness" of my datasets / models.

It wasn't especially clear what my fairness metric actually meant. Documentation was more focused around the technical implementation rather than the practical consequences of the results.

This also applied to any "de-biasing" techniques the tools might have come with. It was not clear what the techniques were actually doing to the data / model to improve fairness, and what that meant for my model performance.		Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairnessExtend an existing code baseNeed to integrate with existing workflowStrongly agreeStrongly agreeStrongly agreeAgreeAgreeAgreeAgreeNeither agree nor disagreeSomewhat agreeNoNoSoftware-based testing and intervention - open source tools,Software-based testing and intervention - in-house toolsLess than a day8Fairness has not been a priority in my projects.Extremely importantNeither important nor unimportantExtremely importantExtremely importantNeither important nor unimportantNot very importantNot very importantNot very importantNoRelevant regulations,Input from the legal team,Reputational risk,Liability and compliance risk,Customer / end user demand or requirementsStrongly agree
5
3Data scientistRetail banking1-2 yearsOther (please specify)MalaysiaSocial sciences,Machine learning,Law / public policy1-2 years1-2 yearsLess than 1 yearAI Fairness 360Learn more about ML fairness concepts and terminologies.Python classes seemed too heavy handed, wasn't clear how to use toolkit if disparate impact was the opposite of expected. However developer offered helpful support on slack Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness,As a starting point to build my own tool, as a reference for implementationsUse an off-the-shelf toolkitTime constraint,Built-in usabilityAgreeStrongly agreeStrongly agreeNeither agree nor disagreeNeither agree nor disagreeNeither agree nor disagreeAgreeNeither agree nor disagreeSomewhat agreeYesNot sureSoftware-based testing and intervention - open source tools,Software-based testing and intervention - in-house tools,Software-based testing and intervention - build my own toolsMore than a day2Doable if concern raised by superiors Business domain expertsNeither agree nor disagreeVery importantNeither important nor unimportantExtremely importantExtremely importantVery importantVery importantNeither important nor unimportantVery importantNoRelevant regulations,Reputational risk,Liability and compliance risk,Customer / end user demand or requirementsStrongly agreeDifficult to convince higher ups that further testing should be done to asses disparate impact beyond assessing input features. Lack of expertise to validate results of fairness assessment.
6
4Technical leadProfessional services (e.g. law, tax, audit, consulting)5-10 yearsUKComputer science,Machine learning5-10 years5-10 years1-2 yearsscikit-fairnessQuickly onboard the toolkits to use it in my current work.,Learn more about ML fairness concepts and terminologies.Learn more about ML fairness concepts and terminologies.Use an off-the-shelf toolkitTime constraint,Built-in usabilityAgreeStrongly agreeStrongly agreeAgreeAgreeAgreeStrongly agreeStrongly agreeStrongly agreeYesYesSoftware-based testing and intervention - in-house toolsMore than a day3Legal and regulatory experts,Business domain experts,Reputational risk expertsStrongly agreeVery importantVery importantExtremely importantExtremely importantVery importantVery importantVery importantVery importantYesRelevant regulations,Reputational riskSomewhat agree
7
5Data scientistProfessional services (e.g. law, tax, audit, consulting)5-10 yearsOther (please specify)JPEngineering,Computer science,Machine learning10+ years5-10 years3-5 yearsscikit-fairnessLearn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairnessLearn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairnessUse an off-the-shelf toolkitTime constraint,No need for customisationNeither agree nor disagreeDisagreeDisagreeStrongly disagreeStrongly disagreeAgreeAgreeDisagreeSomewhat disagreeYesNoSoftware-based testing and intervention - open source tools,Software-based testing and intervention - in-house toolsLess than a day4Business domain experts Somewhat disagreeNot very importantVery importantExtremely importantNeither important nor unimportantVery importantNeither important nor unimportantVery importantNot very importantNoRelevant regulations,Compelling visualisations demonstrating initial findings,Reputational risk,Liability and compliance risk,Customer / end user demand or requirementsSomewhat agree
8
6Technical leadTechnology10+ yearsOther (please specify)DEEngineering,Computer science,Machine learning,Law / public policy10+ years10+ years5-10 yearsscikit-fairness,AI Fairness 360,Fairlearn,What-if ToolQuickly onboard the toolkits to use it in my current work.,Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness,I need to build a fairness tool for my own working pipeline and I’m using the toolkit as a reference for implementationsUse as an out-of-the-box solution for my own fairness-related challenges,Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness,As a starting point to build my own tool, as a reference for implementationsDevelop my own toolkitNeed for customisation,Need to understand the low-level implementation,Need to integrate with existing workflowAgreeAgreeAgreeAgreeAgreeAgreeAgreeAgreeStrongly agreeYesNot sureSoftware-based testing and intervention - open source tools,Software-based testing and intervention - in-house tools,Software-based testing and intervention - build my own tools,Follow a standard organisational processMore than a day2Legal and regulatory experts,Business domain experts,Reputational risk expertsSomewhat agreeVery importantNeither important nor unimportantVery importantNeither important nor unimportantExtremely importantExtremely importantExtremely importantExtremely importantYesRelevant regulations,Concerning initial findings,Reputational risk,Liability and compliance riskNeither agree nor disagree
9
7Data scientistFinancial services: other1-2 yearsOther (please specify)GermanyEngineering1-2 years1-2 yearsLess than 1 yearLearn more about the typical process of dealing with ML fairness,As a starting point to build my own tool, as a reference for implementationsExtend an existing code baseNeed for customisation,Need to understand the low-level implementation,Need to integrate with existing workflowAgreeAgreeStrongly agreeStrongly agreeAgreeAgreeStrongly agreeStrongly agreeHelp to generate a conscience for legal issues (discrimination)Strongly agreeYesNoSoftware-based testing and intervention - open source tools,Software-based testing and intervention - build my own toolsLess than a day4I would have the time aside the day to day project work to actually explore the biases and why they aroseLegal and regulatory experts,Business domain experts,Reputational risk expertsSomewhat agreeExtremely importantExtremely importantVery importantNot very importantExtremely importantExtremely importantNeither important nor unimportantExtremely importantNoRelevant regulations,Input from the legal team,Reputational risk,Liability and compliance risk,Customer / end user demand or requirementsSomewhat agreeI am a junior data scientist. Being a woman in Machine Learning, I would say, I am personally more invested in issues concerning biases than my male colleagues. Generally speaking, we do not have any processes implemented in my organisation to check AI applications concerning fairness aspects (the same applies for trustworthy AI). There is furthermore no official training / awareness raising for this issue.
10
8Business or product-related role
11
9Data scientistTechnology1-2 yearsOther (please specify)GermanyEngineering3-5 years1-2 yearsLess than 1 yearscikit-fairness,AI Fairness 360,FairlearnLearn more about ML fairness concepts and terminologies.,I need to build a fairness tool for my own working pipeline and I’m using the toolkit as a reference for implementationsLearn more about ML fairness concepts and terminologies.,As a starting point to build my own tool, as a reference for implementationsUse an off-the-shelf toolkitBuilt-in usabilityStrongly agreeStrongly agreeStrongly agreeStrongly agreeStrongly agreeStrongly agreeStrongly agreeStrongly agreeNeither agree nor disagreeNoNoSoftware-based testing and intervention - open source toolsMore than a day5Fairness as the base of application security, robustness and future success must be done right.Legal and regulatory experts,Business domain experts,Reputational risk expertsSomewhat agreeNot very importantVery importantVery importantVery importantNeither important nor unimportantNeither important nor unimportantVery importantVery importantYesCompelling visualisations demonstrating initial findings,Reputational risk,Liability and compliance riskNeither agree nor disagree
12
10Technical leadProfessional services (e.g. law, tax, audit, consulting)5-10 yearsUKComputer science,Machine learning5-10 years5-10 years1-2 yearsscikit-fairness,AI Fairness 360Quickly onboard the toolkits to use it in my current work.Use as an out-of-the-box solution for my own fairness-related challenges,Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairnessExtend an existing code baseNeed for customisation,Need to integrate with existing workflowNeither agree nor disagreeAgreeAgreeNeither agree nor disagreeAgreeNeither agree nor disagreeNeither agree nor disagreeAgreeStrongly agreeYesNoSoftware-based testing and intervention - open source toolsLess than a dayReputational risk expertsSomewhat agreeExtremely importantExtremely importantVery importantNeither important nor unimportantNeither important nor unimportantVery importantVery importantVery importantNoRelevant regulations,Input from the legal team,Reputational risk,Liability and compliance riskSomewhat agree
13
11Data scientistRetail banking5-10 yearsUKComputer science,Machine learning5-10 years5-10 years3-5 yearsAI Fairness 360Learn more about the typical process of dealing with ML fairnessUse as an out-of-the-box solution for my own fairness-related challenges,Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairnessUse an off-the-shelf toolkitTime constraint,Built-in usabilityDisagreeStrongly agreeStrongly agreeStrongly agreeStrongly agreeStrongly agreeAgreeStrongly agreeRoot cause or proxy analysis to understand why the biases existSomewhat disagreeYesNoSoftware-based testing and intervention - open source toolsLess than a day2Business domain expertsSomewhat agreeExtremely importantExtremely importantExtremely importantVery importantVery importantVery importantNot very importantNot very importantNoRelevant regulations,Concerning initial findings,Liability and compliance riskStrongly agree
14
12Data scientistHiring / recruitment1-2 yearsUSSocial sciences,Computer science,Machine learning3-5 years1-2 yearsLess than 1 yearUse as an out-of-the-box solution for my own fairness-related challenges,Learn more about the typical process of dealing with ML fairnessUse an off-the-shelf toolkitTime constraint,Built-in usabilityAgreeAgreeAgreeAgreeAgreeAgreeAgreeAgreeSomewhat agreeYesNoNon-technical remedyLess than a dayLegal and regulatory experts,Business domain experts Somewhat disagreeExtremely importantExtremely importantVery importantNeither important nor unimportantVery importantVery importantVery importantVery importantNoReputational risk,Customer / end user demand or requirementsSomewhat agree
15
13Data scientistPublic sector / government3-5 yearsOther (please specify)NetherlandsSocial sciences,Computer science,Machine learning5-10 years3-5 yearsLess than 1 yearUse as an out-of-the-box solution for my own fairness-related challengesUse an off-the-shelf toolkitTime constraint,No need for customisationNeither agree nor disagreeStrongly agreeAgreeNeither agree nor disagreeAgreeAgreeAgreeNeither agree nor disagree-Somewhat agreeYesNoSoftware-based testing and intervention - open source tools,Follow a standard organisational processMore than a day1000Any number necessary. We are required to take this seriously. Legal and regulatory expertsSomewhat agreeNot important at allNot important at allVery importantVery importantNeither important nor unimportantNeither important nor unimportantNeither important nor unimportantNeither important nor unimportantYesRelevant regulations,Compelling visualisations demonstrating initial findings,Input from the legal team,Reputational risk,Liability and compliance risk,Customer / end user demand or requirementsSomewhat disagreeGovernment care a lot about fairness and have a lot of time and money to spend.
16
14
17
15Data scientistProfessional services (e.g. law, tax, audit, consulting)Less than 1 yearUSSocial sciences,Machine learning3-5 years1-2 yearsLess than 1 yearscikit-fairnessLearn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairnessUse as an out-of-the-box solution for my own fairness-related challenges,Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairnessExtend an existing code baseNeed for customisation,Need to understand the low-level implementation,Need to integrate with existing workflowAgreeAgreeStrongly agreeAgreeStrongly agreeAgreeAgreeAgreeStrongly agreeYesNot sureSoftware-based testing and intervention - open source toolsMore than a day3Or more, depending on the complexity of the issueLegal and regulatory experts,Business domain expertsSomewhat agreeNot very importantNot very importantVery importantVery importantExtremely importantVery importantVery importantVery importantYesRelevant regulations,Concerning initial findings,Compelling visualisations demonstrating initial findings,Input from the legal team,Liability and compliance riskSomewhat disagree
18
16Data scientistProfessional services (e.g. law, tax, audit, consulting)3-5 yearsUSComputer science5-10 years5-10 years3-5 yearsAI Fairness 360,Aequitas Tool,audit-aiLearn more about ML fairness concepts and terminologies.,I need to build a fairness tool for my own working pipeline and I’m using the toolkit as a reference for implementationsI've never used the toolkits I selected; I've just used them to understand the open-source offerings - i.e. identify which features of a fairness toolkit are the most common, identify where there seems to be gaps between methods proposed in research and methods implemented in open-source toolkits. Learn more about ML fairness concepts and terminologies.,As a starting point to build my own tool, as a reference for implementationsDevelop my own toolkitNeed for customisation,Need to understand the low-level implementationDisagreeAgreeAgreeNeither agree nor disagreeNeither agree nor disagreeStrongly disagreeDisagreeStrongly agreeI think an explainability assessment is an important part of assessing AI fairness. Also - in terms of mitigating bias, I think that those steps should be taken with the use case/regulatory landscape/real-world deployment in mind; in most places it will probably not be appropriate to use an out-of-box bias mitigation technique without a deeper subject matter understanding. Strongly agreeYesNot sureSoftware-based testing and intervention - build my own tools,Non-technical remedyMy team has done a lot of work around attempting to mitigate potential harms that could result from a deployed AI system - so even if there is bias in the model/data, even if decisions are incorrect, there would not be actual harms created or the harms would be very small.More than a day5If a fairness concern arises on a project we would have more than a day to deal with it, but on many projects fairness concerns may not be raised at all. Some projects are 100% dedicated to "ethical AI" concepts or related "equitable AI" solutions, so the time varies greatly. Legal and regulatory experts,Business domain expertsStrongly agreeNot very importantNot very importantExtremely importantNeither important nor unimportantNot very importantNot very importantNot very importantNot very importantYesCustomer / end user demand or requirementsSomewhat disagreeThe driving factor behind whether we look into AI fairness is basically if the client asks for it or if it's required from a regulatory standpoint. In these cases, I have found that we have sufficient time/budget to look into the issue. My projects have always used custom code analysis as opposed to out-of-box fairness toolkits. The questions we look into are very use-case/industry dependent which means many are easier to answer non-programmatically or with custom code for that specific client.
19
17AcademicOther (please specify)
20
18Other: please specifyModel validation Retail banking5-10 yearsUKSocial sciences,Machine learning5-10 years3-5 years3-5 yearsscikit-fairness,AI Fairness 360Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairnessPrimarily read the accompanying documentation to help with my understanding and also see how theoretical concepts have been applied in practice
21
19
22
20Data scientistProfessional services (e.g. law, tax, audit, consulting)1-2 yearsUKSocial sciences,Machine learning3-5 years1-2 years1-2 yearsscikit-fairness,AI Fairness 360,Fairlearn
23
21
24
22Business or product-related roleTechnology5-10 yearsOther (please specify)CzComputer science,Machine learning10+ years10+ years10+ yearsWhat-if Tool,audit-aiLearn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairnessLearn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairnessExtend an existing code baseNeed to understand the low-level implementation,Need to integrate with existing workflowAgreeAgreeAgreeAgreeNeither agree nor disagreeNeither agree nor disagreeAgreeNeither agree nor disagreeStrongly agreeYesYesSoftware-based testing and intervention - open source tools,Software-based testing and intervention - in-house tools,Follow a standard organisational processMore than a day
25
23
26
24Data scientistProfessional services (e.g. law, tax, audit, consulting)Less than 1 yearOther (please specify)GermanySocial sciences,Engineering1-2 yearsLess than 1 yearLess than 1 yearscikit-fairness,AI Fairness 360Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairnessLearn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairnessExtend an existing code baseNeed for customisation,Need to understand the low-level implementation,Need to integrate with existing workflowStrongly agreeStrongly agreeAgreeAgreeAgreeAgreeAgreeAgreeStrongly agreeYesNot sureI don't have the background or capacity to answerMore than a day4I would spend enough days to solve the concern which usually takes around 4 days. If not the span would be extended.Business domain experts
27
25
28
26
29
27
30
28Data scientistRegulator3-5 yearsOther (please specify)JP5-10 years3-5 yearsLess than 1 yearAI Fairness 360Quickly onboard the toolkits to use it in my current work.Extend an existing code base
31
29
32
30Data scientistHiring / recruitment3-5 yearsOther (please specify)BrazilComputer science3-5 years1-2 yearsLess than 1 yearAI Fairness 360Quickly onboard the toolkits to use it in my current work.,Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness
33
31Software developerHealthcare1-2 yearsUSComputer science3-5 years1-2 yearsLess than 1 yearAI Fairness 360,Aequitas Tool,What-if ToolLearn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairnessAIF360 provided a nice wrapper to compute various fairness metrics. However it was hard to apply to many of our models which are not binary classification; e.g. for regression models, there is not much guidance on if or how the toolkit should be used.Use as an out-of-the-box solution for my own fairness-related challenges,Learn more about ML fairness concepts and terminologies.,As a starting point to build my own tool, as a reference for implementationsUse an off-the-shelf toolkitTime constraint,Built-in usabilityNeither agree nor disagreeStrongly agreeStrongly agreeAgreeAgreeNeither agree nor disagreeAgreeStrongly agreeStrongly agreeYesNoSoftware-based testing and intervention - open source toolsMore than a day
34
32Data scientistTechnology5-10 yearsOther (please specify)BelgiumEngineering,Computer science,Machine learning5-10 years3-5 years1-2 yearsscikit-fairness,AI Fairness 360,Fairlearn,Aequitas Tool,What-if ToolQuickly onboard the toolkits to use it in my current work.,Learn more about ML fairness concepts and terminologies.,I need to build a fairness tool for my own working pipeline and I’m using the toolkit as a reference for implementationsMainly been using AIF360. It is a good source for example implementation, but the code quality for production is very low.Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness,As a starting point to build my own tool, as a reference for implementationsDevelop my own toolkitNeed for customisation,Need to understand the low-level implementationNeither agree nor disagreeStrongly agreeStrongly agreeStrongly agreeStrongly agreeStrongly agreeStrongly agreeAgreeProvide references to the existing methods, and a good overview of those methods.Strongly disagreeYesNoSoftware-based testing and intervention - open source tools,Software-based testing and intervention - in-house tools,Follow a standard organisational processMore than a day5This is necessary to catch the input of different stakeholders to define what to look for, then look for it.Legal and regulatory experts
35
33Software developerCharity or third-sector organisations10+ yearsUKComputer science10+ years1-2 years1-2 yearsLearn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness,As a starting point to build my own tool, as a reference for implementationsExtend an existing code base
36
34
37
35
38
36Business or product-related roleTechnology3-5 yearsOther (please specify)GermanySocial sciences1-2 years1-2 yearsNone
39
37
40
38Data scientistTechnology1-2 yearsOther (please specify)Germany3-5 yearsLess than 1 yearNoneUse as an out-of-the-box solution for my own fairness-related challenges,Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairnessUse an off-the-shelf toolkitTime constraint,Built-in usability,No need for customisationAgreeAgreeAgreeAgreeAgreeAgreeAgreeAgreeStrongly agreeYesNoI don't have the background or capacity to answerLess than a day2Business domain experts,Reputational risk experts
41
39Role in legal / regulation / complianceFinancial services: other3-5 yearsOther (please specify)Ireland Law / public policyNoneNoneLess than 1 yearUse as an out-of-the-box solution for my own fairness-related challenges,Learn more about ML fairness concepts and terminologies.Use an off-the-shelf toolkitTime constraint,Built-in usabilityStrongly agreeAgreeStrongly agreeStrongly agreeAgreeAgreeAgreeAgreeProduce bench marks against other companies within the same sector Strongly agreeN/A (please explain)Have not been briefed on the model Not sureSoftware-based testing and intervention - in-house tools,I don't have the background or capacity to answerMore than a day7Iterative updates as industry matures Legal and regulatory experts,Business domain experts,Reputational risk experts Somewhat disagreeExtremely importantVery importantVery importantVery importantExtremely importantVery importantVery importantVery importantNoReputational risk,Liability and compliance riskSomewhat disagreeHave not as yet used fairness toolkits
42
40Business or product-related roleInsurance10+ yearsUKEngineering3-5 years3-5 years1-2 yearsscikit-fairness
43
41Business or product-related roleFinancial services: otherLess than 1 yearUKEngineering,Machine learning1-2 yearsLess than 1 yearNone
44
42
45
43AcademicMedia1-2 yearsOther (please specify)Philippines Computer science,Machine learning1-2 yearsLess than 1 yearLess than 1 yearOtherOtherLearn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairnessUse an off-the-shelf toolkit
46
44AcademicHealthcare1-2 yearsUKSocial sciences,EngineeringLess than 1 yearLess than 1 year3-5 yearsAI Fairness 360,Fairlearn,Aequitas Tool,What-if Tool,audit-aiOtherAcademic researchMy impression of the toolkits I've used is that they are serving important - but nevertheless quite narrow - purposes. For example, if you have a specific model that happens to meet the requirements of a given toolkit AND want to test for a specific metric afforded by the same toolkit, then the results are both sound and helpful. In particular, toolkits that help software developers visualise the features of their models may help spark ethical deliberation around design choices. However, often managers and software developers are faced with more complex realities. Hence, the toolkits that currently exist on the market seldom help companies ensure that the technologies they design and deploy are 'ethically sound' in a more general sense. Other (please specify)As one of many tools in the software development process (e.g. external validation, or exploratory testing before piloting the model in real world settings)Use an off-the-shelf toolkitTime constraint,Built-in usabilityStrongly agreeAgreeAgreeNeither agree nor disagreeNeither agree nor disagreeNeither agree nor disagreeAgreeStrongly agreeProvide relevant communication material (e.g. reports that can be circulated internally for the purpose of improving the software development process AND/OR scorecards that can be shared with external stakeholders like customers and investors)Strongly agreeYesNot sureI don't have the background or capacity to answerLess than a day20This question is hard to answer because the time spent would have to be proportional to the impact of the use case in question. For example, an issue related to a marginal feature in a system designed for in-house purposes only would probably receive less attention (and justifiably so) than a major issue related to a system that is determining access to public resources (like university admissions, loan application or medical diagnostics). Legal and regulatory experts,Reputational risk expertsNeither agree nor disagreeExtremely importantVery importantVery importantVery importantNot very importantNeither important nor unimportantNeither important nor unimportantNeither important nor unimportantNoRelevant regulations,Compelling visualisations demonstrating initial findings,Reputational risk,Liability and compliance riskSomewhat agree
47
45AcademicTechnology3-5 yearsUKSocial sciences,Engineering,Computer science,Machine learning10+ years5-10 years3-5 yearsUse as an out-of-the-box solution for my own fairness-related challenges,Learn more about ML fairness concepts and terminologies.,As a starting point to build my own tool, as a reference for implementationsUse an off-the-shelf toolkitTime constraint,Built-in usabilityAgreeAgreeAgreeStrongly agreeDisagreeNeither agree nor disagreeStrongly agreeAgreeSomewhat agreeYesNoSoftware-based testing and intervention - open source tools,Software-based testing and intervention - build my own tools,Non-technical remedyI suppose caveating and communicating any findings in the model with the knowledge that the model is only making predictions on unfair or unbalanced data, and that the findings may differ with a fairer representation. More than a day3My work can go slowly so if something is an issue of fairness it is likely also an issue of validity as my work is academic. If there was a serious issue of fairness that arose it would be scientifically negligent to disregard it, I think. Other-Neither agree nor disagreeVery importantNot important at allNot very importantNot important at allExtremely importantNeither important nor unimportantNeither important nor unimportantNeither important nor unimportantYesNeither agree nor disagree
48
46
49
47
50
48Data scientistProfessional services (e.g. law, tax, audit, consulting)1-2 yearsUKSocial sciences,Computer science3-5 years3-5 years1-2 yearsscikit-fairness,AI Fairness 360,FairlearnLearn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairnessSome of the toolkits are not intuitive to use. It takes some time to familiarise myself with the tool.Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness,As a starting point to build my own tool, as a reference for implementationsDevelop my own toolkitNeed for customisation,Need to understand the low-level implementation,Need to integrate with existing workflow,OtherTo tackle novel problemsAgreeNeither agree nor disagreeNeither agree nor disagreeDisagreeDisagreeDisagreeNeither agree nor disagreeAgreenope.Strongly agreeYesYesSoftware-based testing and intervention - open source tools,Software-based testing and intervention - in-house tools,Software-based testing and intervention - build my own tools,Non-technical remedyI think fairness can be quite subjective and therefore, non-technical remedy must sometimes be used. To create fair ML models, we need both quantitative and qualitative assessments. For example, quantitatively, we can check if one of the protected groups is a minority and then qualitatively, discuss the best way to deal with the imbalance - do we simply re-sample using SMOTE? or should we rectify the data collection process in the first place. Understanding the context of the problem is also important. More than a day5So long we create ML models, fairness must be incorporated in the pipeline.Legal and regulatory experts,Reputational risk expertsStrongly agreeNot very importantNot very importantVery importantVery importantVery importantVery importantVery importantVery importantYesRelevant regulations,Concerning initial findings,Compelling visualisations demonstrating initial findings,Input from the legal team,Reputational risk,Liability and compliance risk,Customer / end user demand or requirementsSomewhat agreeI think most fairness toolkits are tailored for data scientists with overly complicated metrics. This makes it hard to explain the metrics to the business stakeholders which most of the time are looking for something that is intuitive and meaningful.

51
49Other: please specifyStudent
52
50Technical leadProfessional services (e.g. law, tax, audit, consulting)5-10 yearsUKSocial sciences,Computer science,Machine learning5-10 years5-10 yearsAI Fairness 360Learn more about the typical process of dealing with ML fairness,I need to build a fairness tool for my own working pipeline and I’m using the toolkit as a reference for implementationsNot production-ready. Significant effort required for integration and customisationUse as an out-of-the-box solution for my own fairness-related challenges,Learn more about the typical process of dealing with ML fairness,As a starting point to build my own tool, as a reference for implementationsUse an off-the-shelf toolkitTime constraint,Built-in usabilityAgreeStrongly agreeStrongly agreeAgreeAgreeAgreeAgreeStrongly agreeQualitative guidance on types of fairnessSomewhat agreeYesYesSoftware-based testing and intervention - open source tools,Software-based testing and intervention - in-house tools,Software-based testing and intervention - build my own tools,Follow a standard organisational process,Non-technical remedyIn-house risk assessment, which includes an ethics assessmentLess than a day4If there is a significant fairness issue, we would request more time and budget from stakeholders. Otherwise, we have limited time to explore the issues unless it was specified in the project scope.Legal and regulatory experts,Business domain experts,Reputational risk expertsStrongly agreeExtremely importantExtremely importantExtremely importantVery importantVery importantVery importantNot very importantVery importantYesRelevant regulations,Concerning initial findings,Compelling visualisations demonstrating initial findings,Input from the legal team,Liability and compliance riskStrongly agree
53
51
Technical lead
Charity or third-sector organisations
3-5 years
UK
Social sciences
Less than 1 year
Less than 1 year
Less than 1 year
What-if Tool
Learn more about ML fairness concepts and terminologies.
Learn more about ML fairness concepts and terminologies.
Extend an existing code base
Need to understand the low-level implementation,Need to integrate with existing workflow
Strongly agree
Strongly agree
Strongly agree
Neither agree nor disagree
AgreeAgreeAgreeAgree
Strongly agree
Yes
54
52
ML engineer
Healthcare
3-5 years
US
Engineering,Computer science,Machine learning
5-10 years
1-2 years
3-5 years
AI Fairness 360,Fairlearn,What-if Tool
Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness
Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness
Extend an existing code base
Need to understand the low-level implementation,Need to integrate with existing workflow
Neither agree nor disagree
Neither agree nor disagree
Strongly agree
Strongly agree
Strongly agree
Agree
Strongly agree
Agree
Strongly agree
Yes
55
53
56
54
Technical lead
Public sector / government
1-2 years
US
Social sciences,Computer science,Law / public policy
1-2 years
1-2 years
3-5 years
Learn more about ML fairness concepts and terminologies.
Use an off-the-shelf toolkit
Time constraint
Strongly agree
Strongly agree
Strongly agree
Strongly agree
Strongly agree
Strongly agree
Strongly agree
Strongly agree
Neither agree nor disagree
Yes
57
55
Other: please specify
Product Design in AI/ML
Healthcare
10+ years
UK
3-5 years
1-2 years
NoneOther
Not familiar
Learn more about the typical process of dealing with ML fairness
58
56
Academic
Healthcare
3-5 years
UK
Machine learning
3-5 years
Less than 1 year
None
Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness
Use an off-the-shelf toolkit
Built-in usability
Neither agree nor disagree
AgreeAgree
Strongly agree
Strongly agree
Strongly agree
AgreeAgree
Strongly agree
Yes
59
57
Software developer
Charity or third-sector organisations
10+ years
UK
Engineering,Computer science
10+ years
1-2 years
Less than 1 year
Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness
Extend an existing code base
Need to understand the low-level implementation,Need to integrate with existing workflow
Strongly agree
Agree
Strongly agree
DisagreeAgree
Neither agree nor disagree
Neither agree nor disagree
Strongly agree
Strongly agree
Yes
60
58
61
59
Data scientist
Public sector / government
Less than 1 year
UK
Computer science,Machine learning
10+ years
Less than 1 year
None
Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness
Develop my own toolkit
62
60
Technical lead
Technology
3-5 years
UK
Engineering,Computer science,Machine learning
10+ years
5-10 years
1-2 years
scikit-fairness,What-if Tool
Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness
Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness
Use an off-the-shelf toolkit
Time constraint,Built-in usability
Agree
Strongly agree
Strongly agree
DisagreeAgreeAgree
Strongly agree
Disagree
Strongly agree
N/A (please explain)
I don't currently work in implementation
Not sure
More than a day
63
61
Business or product-related role
Charity or third-sector organisations
3-5 years
US
Social sciences,Machine learning
3-5 years
1-2 years
Less than 1 year
AI Fairness 360,Fairlearn,Aequitas Tool,What-if Tool
Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness
Use as an out-of-the-box solution for my own fairness-related challenges,Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness
Use an off-the-shelf toolkit
Time constraint,Built-in usability
AgreeAgreeAgreeAgreeAgreeAgreeAgreeAgree
Strongly agree
YesNot sure
Software-based testing and intervention - open source tools
Less than a day
2
approximately 2hours per day
Legal and regulatory experts,Business domain experts
Somewhat agree
Very important
Very important
Extremely important
Very important
Extremely important
Very important
Very important
Very important
Yes
Relevant regulations,Concerning initial findings,Compelling visualisations demonstrating initial findings,Input from the legal team
Neither agree nor disagree
64
62
Technical lead
FinTech
5-10 years
UK
Social sciences,Computer science,Machine learning
10+ years
5-10 years
3-5 years
scikit-fairness,AI Fairness 360,Fairlearn,Aequitas Tool,What-if Tool,audit-ai
Quickly onboard the toolkits to use it in my current work.,Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness,I need to build a fairness tool for my own working pipeline and I’m using the toolkit as a reference for implementations
Use as an out-of-the-box solution for my own fairness-related challenges,Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness,As a starting point to build my own tool, as a reference for implementations
Extend an existing code base
Need for customisation,Need to understand the low-level implementation,Need to integrate with existing workflow
Agree
Strongly agree
Strongly agree
AgreeAgreeAgreeAgree
Strongly agree
Strongly agree
YesNo
Software-based testing and intervention - build my own tools
Less than a day
4
Legal and regulatory experts,Business domain experts,Reputational risk experts
Somewhat agree
Extremely important
Extremely important
Extremely important
Very important
Very important
Very important
Very important
Very important
No
Relevant regulations,Concerning initial findings,Compelling visualisations demonstrating initial findings,Input from the legal team,Liability and compliance risk,Customer / end user demand or requirements
Strongly agree
65
63
Data scientist
Other (please specify)
Education
3-5 years
UK
Computer science,Machine learning
3-5 years
5-10 years
1-2 years
66
64
67
65
Academic
Other (please specify)
No industry domain in academia
1-2 years
Other (please specify)
The Netherlands
Engineering,Computer science,Machine learning
5-10 years
3-5 years
3-5 years
scikit-fairness,AI Fairness 360,Fairlearn,Aequitas Tool,What-if Tool
Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness
For many of these, the user-facing API was difficult to work with (e.g., AIF360) or the capabilities were fairly limited (e.g., Aequitas).
Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness,As a starting point to build my own tool, as a reference for implementations
Extend an existing code base
Need for customisation,Need to integrate with existing workflow
Disagree
Strongly agree
Strongly agree
Strongly agree
Strongly agree
Neither agree nor disagree
AgreeAgree
Somewhat agree
N/A (please explain)
I'm in academia
Not sure
Software-based testing and intervention - open source tools,Follow a standard organisational process,Non-technical remedy
Rethink problem definition, stakeholder involvement.
More than a day
0
I'm in academia so I don't build specific products.
Legal and regulatory experts,Business domain experts,Reputational risk experts
68
66
69
67
Software developer
Technology
5-10 years
US
Computer science
5-10 years
3-5 years
3-5 years
scikit-fairness,AI Fairness 360,Fairlearn,Aequitas Tool,What-if Tool,audit-ai
70
68
71
69
Business or product-related role
Technology
10+ years
Other (please specify)
Canada
Computer science,Law / public policy
10+ years
3-5 years
1-2 years
Use as an out-of-the-box solution for my own fairness-related challenges
Extend an existing code base
Need for customisation
Strongly agree
Agree
Strongly agree
Neither agree nor disagree
Strongly agree
AgreeAgree
Strongly agree
Strongly agree
YesNo
Software-based testing and intervention - open source tools,Hand off to a different team member (please specify to whom)
Consult our Ethics teams
More than a day
5
Fairness as a component of a "responsible AI solution" would require a significant portion of the total effort. Items such as reliability and safety are much more transferable and as such would require less effort for new implementations.
Legal and regulatory experts,Business domain experts,Reputational risk experts,Other
We have an Ethics team
Strongly agree
Neither important nor unimportant
Very important
Extremely important
Very important
Not very important
Not important at all
Not very important
Not very important
Yes
Reputational risk,Liability and compliance risk
Somewhat disagree
72
70
Business or product-related role
Insurance
10+ years
Other (please specify)
Canada
Social sciences
10+ years
3-5 years
None
Use as an out-of-the-box solution for my own fairness-related challenges,Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness
Use an off-the-shelf toolkit
Built-in usability
Strongly agree
Neither agree nor disagree
Agree
Strongly agree
Agree
Strongly agree
Strongly agree
Disagree
Somewhat agree
YesNo
Software-based testing and intervention - in-house tools,Software-based testing and intervention - build my own tools
More than a day
10
I anticipate needing to loop in subject matter experts and external data providers which lengthens the timeline
Legal and regulatory experts,Business domain experts
Somewhat disagree
Very important
Very important
Not very important
Neither important nor unimportant
Extremely important
Neither important nor unimportant
Extremely important
Extremely important
Yes
Relevant regulations,Input from the legal team,Reputational risk,Liability and compliance risk
Somewhat agree
73
71
Data scientist
Professional services (e.g. law, tax, audit, consulting)
5-10 years
US
Social sciences,Engineering,Computer science,Machine learning,Law / public policy
10+ years
5-10 years
3-5 years
scikit-fairness,Fairlearn
Learn more about ML fairness concepts and terminologies.
Went through some tutorials to learn the basic ideas and concepts, then ported over concepts to custom implementations.
Learn more about ML fairness concepts and terminologies.,Learn more about the typical process of dealing with ML fairness,As a starting point to build my own tool, as a reference for implementations
Extend an existing code base
Need for customisation,Need to understand the low-level implementation,Need to integrate with existing workflow
AgreeAgreeAgreeDisagreeDisagreeDisagree
Strongly disagree
Agree
Strongly agree
YesNot sure
Software-based testing and intervention - open source tools,Software-based testing and intervention - build my own tools,Non-technical remedy
If there are problems in the source data (eg Experian, TU, etc) then ideally it can be addressed upstream of a particular credit model. Otherwise, you are just slapping a band aid over the problem for a specific build of a credit and will have to fix it again in six months when you retrain.
More than a day
3
For lending there's usually a strong compliance process; but that also depends on particular clients.
Legal and regulatory experts
Neither agree nor disagree
Very important
Very important
Extremely important
Neither important nor unimportant
Very important
Neither important nor unimportant
Very important
Neither important nor unimportant
Yes
Relevant regulations,Concerning initial findings,Liability and compliance risk
Neither agree nor disagree
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100