Preserving Utility in Fair Top-k Ranking with Intersectional Bias
Alessandro Castelnovo, Riccardo Crupi, Fabio Mercorio, Mario Mezzanzanica and Nicola Alimonda
Motivation
The ranking-based proposition has become part of everyday life; it is now commonplace to choose movies, travel, or clothes after seeing sorted tips. �Ranking is typically adopted by companies in High-Risk* domains such as �Credit Worthiness and Hiring
*See The European Commission: Proposal for a Regulation of the European Parliament laying down harmonised rules on AI (AI Act)
This work focus on post-processing approach to ensure a �Final Fair Rank considering multiple protected attributes
The Fair Re-Rank Problem
Color Blind Ranking π
The goal of a ranking is to associate to each item di a score Yi to create a permutation π of all the elements in D. By j = πi we denote the item in D at rank j where j = 1, 2, ..., n.
s
A better position in the rank denotes a better utility of the item for the user.
1
we must verify that the rank π is fair in respect of G. The fair re-rank problem is necessary whenever π is unfair.
The goal is to calculate a new rank ω such that: (i) should fairly represent all the protected groups G and; (ii) preserve as much utility as possible from the original rank π
2
3
4
Fair Ranking
Background – How to measure utility
Reworking of Multinomial FA*IR algorithm taking into account drop of utillity in respect of original rank while remaining into fairness boundaries
Individual Exposure.�Take into account, for each individual the rank position
Discounted Cumulative Gain (DCG) �Take into account Score Y given by the model and also rank position j
Kendal-Ƭ
Utility metric based on similarity between new rank and original rank
Kendal-Ƭ is a statistical measure tipically used to confront similarity between distributions
Background – How to measure fairness
Average Exposure
Disparate Treatment Ratio (DTR)
Cumulative Distribution Function for a Binomial Distribution
Related Work – Multinomial FA*IR
Zehlike, M., S ̈uhr, T., Baeza-Yates, R., Bonchi, F., Castillo, C., Hajian, S.: Fair top-k ranking with multiple protected groups. IPM , 102707 (2022)
Multinomial FA*IR is an algorithm developed by [Zehlike et al.] it is useful to create a new rank, calculating position by position while using an hypothesis test that check if new rank fits binomial distribution. This latter establish if the distribution fits given proportion parameters for each group.
Multinomial FA*IR select iteratively individuals with high Score that belong to low exposure protected groups to fit them to given proportions passed to binomial distribution.
Selected individuals are positioned in a better rank j in ranking ω.
It select which individual ‘d’ to put in ‘j’ simulating the adding of this latter and calculating the value of binomial distribution for each protected group and it select the individual that guarantee the bettervalue of fairness in terms of alphac
However this approach does not guarantee the maximum utility in terms of similarity with original rank and DCG. In fact is possible that a solution thath do not guarantee maximum utility for position j is equally fair but with a better utility.
Contribution
Paper’s contribution lays in the fact that Multinomial FA*IR is extendend in order to (1) allow the user to chose which metric to maximise, if fairness or utility(they were fixed with the original algorithm) (2) users can act in chosing the preferred configuration referred to Fairness and Utility.
experiments are conducted to demonstrate the efficicacy and flexibility of our proposed approach (focus on next slide). In particular, we have demonstrate that we can find better solutions in respect on those founded by Multinomial FA*IR respecting fairness boundaries.
You can act on trade-off, working on L parameter that can vary between 0 and 1. When L is equal to 0 there’s the preservation of fairness obtaining Multinomial FA*IR algorithm, while L is equal to 1 it maximise utility remaining into fairness boundaries.
Experiments Overivew
Experiments are conducted on benchmark dataset Adult.csv.
Following plots shows rank without considering Fairness (Color Blind Ranking) and mitigating with Multinomial FA*IR
Color Blind Ranking
Multinomial Fair
Investigation
In our experiment, we suppose that we want to maximize utility as expressed in terms of Kendall-τ . Under this assumption, our research questions are:
Could there be solutions that maximize Kendall-τ that satisfy the fairness boundaries, that are not covered by Multinomial FA*IR? �In other terms, can the value of L that maximizes Kendall-τ be different from 1 and 0?
Could there be solutions that maximize Kendall-τ that satisfy the fairness
boundaries, that are not covered by Multinomial FA*IR? In other terms, can
the value of L that maximizes Kendall-τ be different from 1 and 0?
Experiments Evidence
Relationship between Kendal-tau and L is not linear
Maximum utility is found between 0 and 1, it confirms the significance of out approach
Results are consistent for different values of alpha
Conclusion
Thank You