Responsible and Trusted AI

Responsible AI

• Trusted
• Bias Free
• Privacy Preserving
• Equitable
• Traceable
• Reliable
• Governable
• Explainable

Responsible AI: Address Fairness and Bias

• Bias is major concern for data-driven methods such as machine learning
• Fairness is challenged in settings where the available training data is already contaminated by bias
• Unfair machine learning can encode existing human biases and at the same time introduce new ones.
• Example:
• Three commercially released facial-analysis programs from major technology companies demonstrate both skin-type and gender biases¹
• Error rates in determining the gender of light-skinned men were never worse than 0.8 percent
• For darker-skinned women, more than 20 percent in one case and more than 34 percent in the other two.

1. Joy Buolamwini and Timnit Gebru.Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pages 77–91. PMLR, 2018

Effect of Bias

• Effect of biased data
• AI/ML model is dependent on a good dataset and a dataset can lack diversity
• The data itself is often a product of social and historical process that operated to the disadvantage of certain groups
• When trained in such data, off-the-shelf ML may reproduce, reinforce, and exacerbate existing biases
• Biased dataset will perform poorly with minority:
• If most of the samples are white males, the model will fail for women and people of color
• Machine learning techniques are designed to fit the data, and replicate any bias already present in the data.
• In the problem of recidivism prediction used to inform bail and parole decisions, the goal is to predict whether an inmate, if released, will go on to commit another crime within a fixed period.
• But we do not have data on who commits crimes—we have data on who is arrested. Arrest data may be skewed toward minority populations that are monitored by police at a higher rate
• Unfair ML can result in people not getting admission, parole or a very necessary loan assistance
• In military friendly fire can result if AI/ML model is biased

​

​

Bias in the Dataset

• We found presence of human bias in the dataset in research on emotion recognition:
• Keyword searching in google is a popular method of collecting visual (image and video) data
• In our search with keyword “angry face”- 85% of the acceptable images appeared are male
• This pattern holds for other generic keywords like ”sad people”, ”happy human” etc.
• Therefore, a dataset prepared by collecting results from these types of keyword search results in bias
• Same applies to the volunteer choice for creating an acted dataset
• Without careful selection of people from multiple genders and ethnic backgrounds, dataset bias can be easily incorporated into the model

Bias in the Dataset

• One of the widely used facial emotion recognition dataset FER-2013 is suffering from keyword search bias

How to Reduce Dataset Bias:

• Fair representation:
• Better representation of minority groups by using specific keywords
• Using both “happy man face” and “happy woman face” instead of “happy face” keyword
• Diversity: Choosing volunteers from diverse background
• To produce new ML models which assign higher importance on less represented data samples
• Mitigation strategies:
• Incorporating community-driven data instead of police arrest data may helps to attenuate the biasing feedback effects.
• Better Algorithm:
• Data cleaning algorithms
• Developing algorithms capable of detecting, tracing, and handling/mitigating bias

Responsible AI: REALM/SKOD Application

• Modules where responsible AI must be ensured:
1. Video and text dataset collection without bias
2. Neural network training with fairness
3. Unbiased recommendation system
• Potential problem:
• Biased datasets lead to biased models for feature extraction modules
• Decreased accuracy for underrepresented minorities is an important social issue
• Recommender systems suffer from popularity bias and feedback loops (filter bubble)

Trustworthy AI for REALM SKOD

• Dataset Collection:
• Real-world image datasets are not representative of the entire world
• Datasets collected in a particular location (e.g. West Lafayette) may not apply to the population in other location (e.g. California, Texas)
• Privacy laws may differ for different locations of data collection.
• Recommendation Engine:
• With time the model evolves and aligns with the user interests.
• The model learns the user’s interests/preferences and recommends the events within the ‘user bubble’ in future
• The events of interest that are outside of the bubble are not recommended making it harder to deal with anomalies and novelties

​

​

​

​

​

Responsible AI: Neural Network Training

• Dataset Bias. Data drift and domain shift cause significant decrease in accuracy
• Challenge: How to detect data drift and domain shift in a timely fashion?
• Many solutions to combat the issue once it is detected
• Solution 1: Increase the dataset to retrain the model. Collect more representative data to generalize better
• If the location of collecting the data is not representative, the dataset remains biased
• Labeling new data is expensive and lengthy process
• Solution 2: Apply transfer learning techniques

​

​

Data Drift and new Objects, Entities, Classes

• Data drift happens when statistical properties of the dataset change over time
• Novelties can be one of the causes of data drift
• Other causes include seasonality and anomalies

Dataset Augmentation for Better Novelties Prediction

• Differentiate novelties vs. anomalies
• At the first encounter of a new example, how to tell a novelty vs anomaly?
• Novelties come from previously unseen class, may be related to the other detectable classes
• Anomalies are previously unseen instances, may not be related to other classes
• Further encounter of a new instance
• Novelties eventually transition into new normal as time passes, “blend into” the environment
• Anomalies remain abnormal and never become normal instances but may re-appear occasionally
• Hence, two-fold approach for dataset augmentation:
• Distance-based methods to enhance detection of novel instances at the first encounter
• Distribution-based methods to enhance detection of novelties as time passes
• Novel data instances can be generated by varying the distance parameters or the distribution parameters

​

​

Trustworthy and Responsible Recommendation Engine

• Security issues: Biased profiles injected by attackers may force a system to ‘adapt’ in a manner advantageous to attackers
• Degradation of user trust in the accuracy of the system
• Profile evaluation must be implemented to neutralize injection attacks
• Social awareness issues: Feedback loop discriminates minorities and underrepresented groups
• Explainability issues: Does the end user understand the factors that led to a particular recommendation?

​

Trustworthy AI: Best Practices in REALM (SKOD)

• Ethical design comes first:
• Security and privacy policies during data collection, dataset storage phases
• Dataset does not contain personal information and data is anonymized.
• Data transparency: documenting of the dataset collection, labeling guidelines, ensure balance between the racial/gender classes.
• Traceability. The dataset update, data provenance, model retrain, model deployment is documented and logged.
• Continuous monitoring of the deployed model.
• Checking for data shift, data drift and encoded bias.