Lecture 1: Introduction & Explainable AI
Main takeaway: the addressed problems of GOFAI are nowadays still relevant because of XAI.
Reminder: Marr's three levels of explanation.
Ask for each paper: on what level is the author working?
Main focii of course:
- Abduction problem: making sense of observation
- Connect philosophical issues (non-demonstrative ... ) and computational modeling
- Key contraints: tractability & uncertain information
- Desire for explainable AI
- Computational formalism: Bayesian networks
- Could in principle be another framework that deals with uncertainty
- So focus is mostly computational
Example: Picture of roof: what is relevant for an explanation depends on your goal. If classifying building is the goal, the snow or foreground is irrelevant. Just referring to the statistics of the picture is not enough.
Explainable AI
Gunning, DARPA report on XAI
Gunning, three approaches:
- Learn deep explanations (slide 6): Train net to associate semantic attributes with hidden layer nodes and known ontologies.
An external ontology is necessary because there is no 1-to-1 mapping between hidden layers and human understandable labels.
However, presupposing an external ontology negates other benefits of a machine learning / neural net approach. This negates the benefit of finding statistical patterns that domain experts do not perceive or yet know. Networks can pick up nuances that the domain expert did not see as relevant (or the feature is not visible to them at all). So you limit the capacity to improve on domain experts; find new connections.
If there are unlabeled notes, generate examples from them to find semantic information. But again, if it is a meaningful result it would probably already be in the ontology.
- Generate explanations by a language generating recursive net that augments a CNN (slide 7)
For our Mars Rover example: do we have such an external ontology? We are going there precisely to learn new things, in a very unfamiliar environment.
Short discussion of papers related to XAI.
Pearl: criticaster of purely statistical methods for AI. NIPS 2017.
- Statistical patterns do not allow drawing of causal conclusions
- Intervention (what if I do X)
- Counterfactual (what if I had done Y)
Example: lightswitch and lightbulb.
- Just observating only captures statistical association
- But we know the relation between lightswitch and lightbulb is causal
- What do we need to do to infer a causal relation between the lightswitch and the lightbulb?
- A->B and not(B)->not(A).
- So we can only learn this relation either by switching the switch to on, or breaking the lightbulb.
- Counterfactual
- I treat a patient, patient dies
- What would have happened if I did not do that specific treatment?
- what if? Counterfactual reasoning requires insight in underlying world model
- Statistically you do not get much further than stating the association between treatment and death... what does that say?
Is this required for XAI? Is reasoning about interventions really necessary?
Johan Kwisthout is not sure.
Intentional stance from Dennett
Alarm clock example.
Why do analog clocks snooze every 9 minutes. Physical reason; underlying logical circuit. (My digital clock does 10, but Francesca's 8. Does 8 have a physical rather than psychological reason?)
- Assumes rationality (otherwise all rules for attributing beliefs or desires are nonsensical)
- Agnostic about inherent workings;
- not realistic w.r.t. intentions.
- Instrumentalist: you attribute intentions to explain behavior, irrespective of whether they actually have them
- It's a useful way of modeling
- Dennett is between instrumentalist and realism: once you attribute intentions, they have them; we see them as beliefs.
John McCarthy
Founding father of term AI at Darthmouth Conference.
Bypasses Searle's chinese room.
The question is: is it useful to ascribe mental qualities to machines?
It might be useful to attribute mental capacities to computer systems even if they do not have them.
McCarthy was optimistic about the possibility to formalize or axiomatize mental qualities.
He suspected we would in several decades (so now in the past) still talk about "free will". We would rather discuss the quality of various mathematical foundations. Is that justified? (Well, overly optimistic).
Main issues
- It may be useful to ascribe mental qualities even though we are agnostic about whether agents "really" have those beliefs -> intentional stance.
- What kind of representation do we need for successful AI?
- Pearl: build in causal reasoning
- XAI: even for purely statistical ML we require explanations
- The importance and interest in XAI shows that old philosophical issues, e.g. on what a relevant explanation is, are still very important to consider.
- They are not made superfluous by recent technical successes in AI.