Xomnia webinar Fair and Explainable AI
Event Description
The June edition of the Responsible AI webinar series will focus on the topic "Fair and Explainable AI".
Our first guest speaker will be Hilde Weerts, an Artificial Intelligence Engineer at the Eindhoven University of Technology. Her primary research interest is Fair AI, to ensure which ensures that different groups of people are treated fairly by algorithms. She will talk about different ways of defining and measuring fairness when developing a prediction model, and ways to mitigate bias and unfairness when it is encountered in a prediction model.
Our second guest speaker will be Jasper van der Waa, a Research Scientist in Artificial Intelligence at TNO and a Ph.D. candidate at the Delft University of Technology. His primary research topic is Explainable AI, ensuring that ensures that the outcomes of a prediction model are explainable to the stakeholders affected by its outcomes. Jasper will take us through various kinds of explanations, how to obtain them from a model or data set, and which kinds of explanations work for different types of tasks and stakeholders.
Program:
16:00: opening by Marieke Peeters
16:05: talk by Hilde Weerts
16:25: Q&A
16:30: talk by Jasper van der Waa
16:55: Q&A
17:00: closing by Marieke
Hilde Weerts: An Introduction to Algorithmic Fairness in Machine Learning
Hilde Weerts is one of the maintainers of fairlearn!
Why Machine Learning?
- Scalability
- Superhuman Performance
- "More objective"?
Some
- Compas Recidivism algorithms: more false negatives for white Americans, many false positives for black Americans
- Amazon Automated Hiring: biased against women and words related to femininity
- Gender-biased / stereotypical translation
Types of fairness-related harms
- Allocation: extend of withold opportunities or info for particular groups only
- Quality of service: system does not work equally well for all groups
- Stereotyping: reinforcement of stereotypes
- Most prevalent in unstructured data
- Denigration: system is actively derogatory or offensive
- Representation: Development/usage of the system overrepresents or underrepresents certain groups
- Procedural: system makes decisions in a way that violates social norms
- Closely related to interpretable machine learning
This talk focuses on 1 and 2.
Where do problems arise? And what do we optimize for?
1 and 2 are "real world".
-
Problem formulation
-
Does our problem formulation reflect the real-world context and our moral values?
-
Data collection
Is the data a good representation of reality?
-
Training data
-
Are we optimizing the right loss function?
-
Do we have the right Performance metric (model selection?)
-
Prediction
-
Does the model make fair predictions?
-
Decisions
-
Does the system produce fair outcome?
Bias
Let's make this term a bit more explicit.
A systematic or disproportionate tendency towards ... something
- Social bias: prejudice agaainst a person or a group
- Statistical bias: systematic error in estimation of parameter or variables
The danger zone is where they meet.
Historical bias
Social bias encoded in historical data.
If in the past more males were hired, this will make biased prediction in the future as well.
Two flavors of historical bias
- Innate potential: innate potential e.g. for a data scientist position
- Construct: actual suitability for data scientist position
- Measurement: this is where data happens, namely whjen someone gets hired (or not)!
The two types:
- Life Bias: the data can be a good representation of reality, but reality itself is biased.
- Can result in allocation harm
- Moral question: who accounts for this?
- Construct Validity Bias: the data is a biased measurement of the construct we are trying to measure
- Also allocation harm, but "quality-of-service harm in disguise"!
- Can be a data collection problems, but often already originates in the machine learning task formulation.
We cannot distinguish between these biases from data alone (because we only have measurements). Often they occur simultaneously. That's why understanding the context of your use case is so important!
Representation bias
Can result in quality-of-service harm
Can be mitigated by
- Collecting more data for underrepresented groups
- Algorithmic approaches (e.g. reweighing, cost-sensitive learning)
- Diverse data collection team able to spot bias from early on
Measuring fairness
https://www.youtube.com/watch?v=HF0h0GSvwyA
Group Fairness Metrics. Main ones:
- Demographic Parity
- measures allocation harm
- Equal selection rates
- Assumption: regardless of what data says, everyone is or should be equal (in the real world)
- Equalized Odds
- measure quality-of-service harm
- Criterion: equal FPR and FNR
- Assumption: the target variable is a good representation of reality, but the data distribution may differ across groups.
Hilde Weerts is one of the maintainers of fairlearn.
Fairness... mission impossible?
Kleinberg et al (2016) -> Inherent trade-offs in fair ...
It is impossible to both have demographic parity and equalized odds.
Is this a bug?
No not really! The point is that both metrics represent a different understanding of fairness. You should define based on your context which one fits your values and contexts.
Why not removing sensitive features?
If you remove sensitive attributes, you may prevent direct discrimanation but not indirect discrimanation!
Fairness Checlist
Madaio et al (2020) -> Co-designing Checklist to Understand ...
- Problem formulation
- Data collection
- Modelling
- Deployment
At each stage, ask yourself whether ML is really the best solution! Consider abandoning development/deployment.
Part of AIDE lab!
Discussion
Question: Trade-off fairness (e.g. parity) and accuracy?
Paradox in terms of historical data: when we correct past historical bias, it makes sense to perform worse on historically biased data! So the question is more: are we measuring the right thing!
Jasper van der Waa
Jasper found a broader community dealing with transparency and human values in AI, but he distinguished two subcommunities.
Example of a watch:
- Interpretable AI
- Focus on developing more interpretable models
- Explainable AI
- Focused on how to develop AI that can answer specific questions; that help us make the best use of the AI.
- We want transparency and simplicity, so people can answer questions about the AI in a useful manner
Point 1 is more relevant for data scientists, but less the "subjects" that are affected by AI decisions. Point 2 is more relevant for those subjects, they have questions about AI (decisions) that they want a useful answer to.
Consultants are in the middle.
This talk focuses on explainability and takes a systems approach. Trying to find a common problem to find a generic solution: an XAI Design Pattern to solve common industry problems!
Example design patterns
Two example design patterns
- Confidence explanations: can you trust the output?
- Contrastive explanations
Confidence explanations
Goal is that an AI explains and asks: "I have high confidence. Out of 20 similar cases, 18 had the same decision. Care to review them?"
Guideline for XAI pattern:
- Model-agnostic
- Case-based reasoning
- This fits with how people define confidence: similarity to past cases
- Weighed neighbourhoord Waa et al. 2020 Interpretable Confidence Measures.
- Don't provide your confidence scores numerically! Provide as an ordinal scale to standardize what people consider to be high
- Provide a way of progressive disclosure
- Allow interactive exploration
- Let user define their own pace
- If they do want more info, provide e.g. other cases that were similar
- Know your user and adjust your information interface accordingly; e.g. a data scientist may appreciate
Contrastive explanations
Loan rejected. Why did I get rejected rather than accepted?
Use simpler rules to approximate the decision. You may simplify the prediction proces too much, but now people actually understand the explanation. Better than providing very long list.
- Income below 25k
- Your credit score is below 650
Algorithmically speaking:
- Identify partition the user is interested in
- From this partition we extract the rules, ordered by their attribution
- Waa et al., 2018, FoilTree (similar to LIME)
- Uses trees both to partition and explain
- Make sure the rules are understandable
- Don't just use "feature names"
- Order rules by importance, e.g. by measuring feature importance with LIME and then prioritize rules using the most important features
Main point
Explanations solve a problem. So start with identifying this problem! Can explainable AI offer a solution?
Then adopt and develop the needed algorithms and interfaces to get an effective explanation.
Discussion
Black box algorithms are not excluded from explainability!
How do you measure the succes of an explanation? In the end you want to convey information to the user. If you want to measure the succes of an explanation, you almost always have to measure the increase of knowledge on the user side!
How well does the explanation add to how well the user can predict the systems predictions? Or can the user better explain the system?
There's an exception, namely the faithfulness of an explanation.
- Faithfulness of explanation is relevant for model-agnostic approach
- How good is the quality of your explanation w.r.t. the actual behavior of the model?
- e.g. with the decision rules you do a local approximation, but it may be wrong!
- e.g. what if you say "loan declined because income < 50k" but this is not the case
How can we make explanations more actionable? Counterfactual explanations improve actionability.
8 July -> Systems Thinking (Alliander) & EU AI Regulation (Considerati)
CCO -> Annemieke Los
annemieke.los@xomnia.com