Approaches to text mining arguments from legal cases | Wyner 2010, Chapter from "Semantic Processing of Legal Texts", eds. E. Francesconi et al.
Introduction
Def. case law: "Case law is that corpus of decisions on cases which judges have made; we refer to this corpus as the case base and the previous decisions as the precedents. " (p. 61)
Case-based reasoning is particularly useful for US and UK common law. Precedents are presented through analogical argument. Case base keeps increasing in size, so navigating this large case base is a complex task requiring interpretation and extensive training.
There are search tools nowadays that help refine the search, but in the end it's still the legal professional to read the case abstracts and determine its relevance.
Advantages of automated text mining on case law:
- mine detailed properties and relationships within and among cases
- easier to keep up with new cases
Notice the difference between information retrieval and information extraction. In IR, you retrieve a set of relevant sources e.g. given keywords. But in information extraction, your output is structured information, e.g. relationships between entities in texts.
Objectives:
- Towards automatically identifying legal arguments, case properties and relationships
- Support legal professionals in activities such as identifying relevant cases from the case base
- Towards a bottom-up understanding of the legal system of common law
- How?
Three approaches:
- Parse arguments into argument interchange format and automatically identify argument sentences from non-argument sentences
- Identify argument characteristics and provide a context-free grammar (that captures those characteristics?)
-
Extraction of various semantically relevant elements, e.g. "case factors" (I don't know what they are yet)
- e.g. Falakmasir2017
-
Argumentation: "about the abstract theory" (62)
- "Argument": "relates to particular instances that may be chained together"
Background
First, we are concerned with information extraction of argument structures from the case base, allowing us to identify decisions and their justifications. Second, we want to extract not only key profile elements (along the lines of the functional elements of [31]), but also the linguistically represented case factors that can be used in case based reasoning (along the lines of [32] and [34]) (63-64)
32: Daniels, J.J., Rissland, E.L.: Finding legally relevant passages in case opinions. In: ICAIL 1997: Proceedings of the 6th International Conference on Artificial intelligence and Law, pp. 39–46. ACM, New York (1997)
34: Brüninghaus, S., Ashley, K.D.: Generating legal arguments and predictions from case texts. In: ICAIL 2005, pp. 65–74. ACM Press, New York (2005)
Argumentation theory
Arguments can be captured as large trees and e.g. represented with XML, but this approach does not scale to large corpora.
Argument theory is more about capturing abstract reasoning schemes or patterns, and then filling in details of concrete arguments. E.g. have a knowledge base with facts and rules, with an additional logic (e.g. Defeasible Logic) to make inferences.
M.b.t. case based reasoning: "An additional realisation of argumentation theory are the argument schemes for case based reasoning of [23], which relates arguments and case based reasoning by allowing schemes to be used to argue for or against a decision given comparative case factors found in the case base." (63)
Text-mining
Traditional services have the following workflow:
- Manually assign indices to cases
- Search these indices
Also searching for semantic content, among others:
- recognising inference
- calculating entailment
- identifying inconsistency
But goal is to extract relations between arguments or case factors. There is some work in that direction. Particularly difficult when those relations do not have explicit indices, but must be inferred.
Argument corpus
This section describes several publicly available datasets for argument mining.
Analysis
Araucaria
Distinguish argument from non-argument sentences
Features:
The features used to distinguish examples are: sequences of words in a sentence (from one (unigrams) to three (trigrams)), words identified by part-of-speech (adverbs, verbs, and modal auxiliaries), collocations (high word pair co-occurrences), text statistics (sentence length, word length, punctuation marks), punctuation, parse features (given a parse of the sentence into phrases, the depth of the tree and number of subclauses), and key words that have been identified as signalling argument (e.g. but, consequently, because of, and others). (66)
The best results were obtained with a combination of collocations, verbs, sentence length, word length, and number of punctuation marks. The latter three broadly can be taken to indicate that sentence complexity (length of sentence and words along with complex structure signaled by punctuation) signal argument. (66)
- So perhaps complexity signals argument-sentences, but this could be an artifact from the selection of argument and non-argument sentences
- Role of specific features is not clear and sometimes ambiguous (certain words can signal both argument and non-argument sentences depending on their interpretation)
Mochales and Moens
"In this section, we formally present a CFG for legal arguments, exploiting the inherently structured nature of argument in case law documents, specifically in the judgements and decisions of the European Court of Human Rights (ECHR). We develop a parsing application to test the grammar accuracy." (67)
Complexity: grammars for a language (e.g. English) are often based on a grammar, whereas a grammar for legal arguments sentences must be investigated in relation to other sentences.
Wyner and Milward
What distinguishes a legal argument from an argument in general?
Case-based reasoning "is a meta-level of argument concerning the cases themselves" (72)
Stages of case-based reasoning:
- Submit problem case and retrieve precedent cases from case base
- Reuse solutions from precedent cases
- solution is (dis)confirmed
- current problem case becomes part of the case base
The main task for a lawyer is selecting relevant cases from the case base. Step 2 and 3 are essentially analogical reasoning.
From among these stages, a key task for a lawyer is to identify on-point-cases from a case base; these are cases which were decided in favour of the lawyer’s side and share the most number of highly valued “factors”. (72)
What are factors? Case factors are not themselves an argument, but a fact pattern that, in a given argument, impacts the decision on the case.
Factors are textually expressed typical fact patterns in a case which bias the decision for or against a side in the case. Importantly, factors are not themselves signalled by argumentative indicators. Yet, identifying the cases which contain the factors is crucial to case based reasoning. (72)
E.g. there are relevant factors for determining culpable intent. Automatically finding these factors is a task and research area in itself, because the same factor can be expressed in many different forms:
This list of factors which are used to determine culpable intent provide a textual “frame” for each factor; however, the form of the factors as they appear in a case base may vary. One of the important tasks of information extraction is to reliably identify the “same” factor in semantic terms while varying the form. This is a general linguistic issue (e.g. passives and actives mean the same thing, but have different forms). (72)
Discussion
- Think about appropriate datasets; European Court decisions are highly complex and irregular; maybe focus on decisions in more typical cases first
- Enthymemes
- "Other issues of interest relate to the identification of enthymemes, which are missing and inferred premises, as well as argument coherence. For information extraction, enthymemes present a very significant issue since they must be inferred semantically and are not in evidence in the text. It is unclear how text mining can yet address this issue.
- Argument structure may be identified, but components may semantically not make sense as an argument
- Using keywords to identify argument is practical but has serious limitations; future research focuses on identifying case factors;
- Lexical, semantic and ontological information should probably be leveraged here
- A main bottleneck for using ontologies is that much knowledge must be drafted manually