The FATE System: FAir, Transparent and Explainable Decision Making

"The FATE (FAir, Transparent, Explainable) system aims to unify a set of AI tools addressing these issues into a ready-to-use decision support system that can be utilized by end users in various roles and domains." (1)

  • "human-AI co-learning for ongoing human-AI performance improvements." (1)
  • Tasks: classifications, predictions, advice, decisions
  • Fair, "understandable-trustworthy", "controllable", "secure".

  • Fairness: through "bias management (i.e., prevention, identification and mitigation)"

  • Understanding and trust: "personalized explanations" (1)
  • Meaningful Human Control by knowledge engineering, integrating "(explicit) human knowledge into the ongoing learning process" (1)
  • incorporating concepts or causal relations
  • Secure learning through federated learning
    • which constraints does this impose on the other aspects?

Innovation is to have a generic domain-agnostic decision support system that nevertheless combines several AI research areas w.r.t. concrete use cases and user preferences.

Fairness

  1. "collect the knowledge for risks, requirements and expectations from experts and stakeholders"
  2. leverage knowledge to assess bias risks in the data
  3. determine priorities, scope, metrics, required model features
  4. assess impact of bias using a baseline model with the determined metrics
  5. bias mitigation
  6. evaluation with metrics, audit process and record in documentation

Hybrid AI

  • Maintain knowledge model as formal representation of domain knowledge
  • Extract meaningful features using domain knowledge
  • Provide constraints on output
  • Improve predictions where data-driven approaches fail
  • Domain knowledge can aid in explanations

Example approach: incorporate some knowledge in a graph neural network.

Explainable AI

  • User modeling
  • Personalize explanation based on user model
  • Confidence estimations for trust calibration

Within FATE, these are the users:

1) Researcher (a domain researcher, e.g. a medical researcher, a criminologist, a sociologist), 2) Consultant (advisor of a subject, e.g. a medical doctor, a judge, a civil servant), 3) Subject (affected person, e.g., a patient/consumer, a convicted person, a naïve users, etc.).

User interaction and co-learning

Archetypical roles:

  • AI developers / data scientist
  • Expert users with domain knowledge
  • Lay users

Allow interactive explanation of the outcomes

Secure learning

  • Federated learning
  • Multi Party Computation
  • Differential Privacy

Why have an integrated system?

Decisions in one module have implications for other modules. In some cases this poses challenges. For example, the federated learning setting poses constraints on explainability. But it also poses possibilities: e.g. world knowledge can be used to improve predictions, but also guide the bias detection component.