Building Bayesian networks for legal evidence with narratives | Vlek et al. 2014, Artificial Intelligence & Law 22:375-421
Abstract:
In a criminal trial, evidence is used to draw conclusions about what happened concerning a supposed crime. Traditionally, the three main approaches to modeling reasoning with evidence are argumentative, narrative and probabilistic approaches. Integrating these three approaches could arguably enhance the communication between an expert and a judge or jury. In previous work, techniques were proposed to represent narratives in a Bayesian network and to use narratives as a basis for systematizing the construction of a Bayesian network for a legal case. In this paper, these techniques are combined to form a design method for constructing a Bayesian network based on narratives. This design method is evaluated by means of an extensive case study concerning the notorious Dutch case of the Anjum murders. (375)
Introduction
Three main approaches in the study of legal reasoning about evidence:
- argumentative
- narrative
- probabilistic
In a criminal trial, the available evidence is used to draw conclusions about the events that took place (referred to in legal terminology as the facts of the crime). In the formal study of rational legal reasoning about evidence, drawing conclusions from evidence is usually formalized by one of three approaches: argumentative, narrative or probabilistic reasoning (Kaptein et al. 2009). While a judge or jury might be more inclined to use argumentation or narratives, a forensic expert might use probabilities to report his findings. In this paper a method is presented to integrate narrative and probabilistic approaches. (376)
What is the narrative approach?
In the narrative approach to reasoning with evidence, a crime scenario forms a context for the evidence. A scenario or story is a collection of states and events with some coherent structure (Bex 2011). When investigating a crime, scenarios are formed to make sense of the evidence (Poot et al. 2004), and as Pennington and Hastie (1992) showed, jurors use stories or scenarios to organize the evidential data. (376)
Of course, the Bayesian network models a large joint probability table with (conditional) independencies between variables: "In legal applications, a Bayesian network is commonly used to model the (in)dependencies between a hypothesis and the evidence." (377) A known problem is how to fill in the correct probabilities, but this "numbers" problem is not the focus of this paper. The focus of the paper is on the prior step: how to construct a good graphical network in the first place, in this case using narrative elements.
A first big problem is this: "In the legal domain, every case is different, demanding a custom-made Bayesian network." (377). There is therefore a serious need to have reusable "building blocks" for generating a graphical representation of a case. But how to decide which building blocks or variables are relevant to include? Methodology is lacking here.
The "narrative" approach to this problem in this paper is to depart from the "scenario's" that result e.g. from police investigation, which describe which variables are relevant to consider. The analyses thus depart from the narrative scenarios described in a case.
Summary of the design method and the problems it addresses:
The design method aims to alleviate three common difficulties in reasoning with evidence: (1) tunnel vision, (2) the problem of a good story pushing out a true story and (3) finding the relevant variables for a model of the case. An advantage of the narrative approach is the use of multiple stories or scenarios to prevent tunnel vision, as described above. By explicitly taking multiple scenarios into account, the narrative approach in our design method aims to prevent (1). A disadvantage of the narrative approach is the problem of a good story being chosen over a true story (Pennington and Hastie 1993). By integrating a narrative and probabilistic approach, the various scenarios in a case can be compared in their formal representation, the Bayesian network, such that the more likely story can be chosen over what may sound as a good story, addressing (2). Finally, the existing probabilistic approaches to modeling legal cases often do not describe how to find which variables are relevant. By taking narratives as a starting point, heuristics for finding the variables can be formulated, addressing (3). (377)
W.r.t. the problem of specifying probabilities to the network: in some cases conditional probabilities can be inferred from logical relations (p={0,1} I reckon), but in other cases the specification is subjective. The paper gives a typical defense against the criticism of subjectivity by stating that at least the Bayesian approach formalized this subjectivity so that you can factor it into your analysis:
Nonetheless, by making these numbers precise, the subjective interpretation of a legal case is made explicit. Therefore, our design method is intended to formalize a subjective account of a legal case, thus forming a tool for a judge or jury to structure their thoughts rather than a tool for reaching an objective verdict. (378)
Method
Narrative idioms.
These are on top of idioms from previous literature.
Example scenario idiom with a top scenario node:
- A is the burglar; has elements (scenario node)
- A wanted to steal something (element)
- A broke in (element)
- A took items (element)
Relations:
- If the scenario node is true, all of its elements must be true
- So probabilities of elements conditional on scenario node are 1
- If the scenario is not true, elements might still be true
- This requires estimation of conditional probabilities
- Scenario node has its own prior probability
The subscenario idiom has the same structure.
A variation idiom lists variations of a particular elements (e.g. various ways to break in) as multiple binary child nodes, via a "disjunction node" (p. 386). Not as a single node with multiple options to allow for better compatibility with subscenarios, i.e. allow one of the variations to be a subscenario in itself.
Merged scenarios idiom are constraint nodes that ensure that only a single scenario is true. This allows all scenarios to be represented in the same Bayesian network.
Unfolding a scenario: add more detail (e.g. a subscenario plus using the variation idiom) when required in discussing the case.
Design method
- Collect relevant scenarios
- Model each scenario using the scenario idiom, and then unfold further:
- Do not unfold if evidence can be directly connected to an element node
- Is there evidence for a subscenario? Unfold.
- Is there relevant evidence for a detail (variation) of that element? Unfold.
- Merge scenarios
- Include evidence: connect to element nodes that the evidence supports