Argumentation mining | Raquel Mochales and Marie-Francine Moens, Artificial Intelligence and Law (2011) 19:1-22

This is pretty much the original paper on argumentation mining. It makes sense that the first application is in the legal domain, since legal argument is more structured than free natural language texts in other domains.

I notice I typically write "argument mining" instead of "argumentation mining", but these are not the same for the simple reason that argumentation can be chain of multiple arguments.

Introduction

What does argumentation mean? There's a clear difference between its highly formal understanding and everyday practice:

Argumentation can be a formal demonstration of the truth of a proposition, e.g. when someone expresses a mathematical proof, but in many areas of human reasoning, including law, this formal demonstration is often not possible. In such domains rationality must be based on some more or less informal arguments. Informal arguments are sometimes implicit. That is, the logical structure, the relationship of claims, premises, warrants, relations of implication, and conclusion, is not always spelled out and immediately visible and must sometimes be made explicit by analysis. The analysis of argumentation focuses on understanding the content of argument (2)

A full argument analysis is a very difficult task, requiring many different aspects to come together:

The analysis of argumentation focuses on understanding the content of argument chains, as well as analyzing the linguistic structure, determining the relationship between the preceding and following arguments, recognizing the underlying conceptual beliefs, and understanding the arguments’ coherence in light of general knowledge of the specific topic. A complete argumentative analysis must depend on several aspects of knowledge: linguistic constraints, domain dependent, conceptual relations, and discourse structure. None of the above aspects are sufficient by themselves for a complete analysis, but they all contribute to complete analysis, especially in the situation where one aspect of information is incomplete. (2)

Argument mining is one aspect of this overall bigger picture. This is

Argumentation mining focuses on the detection of all the arguments in a text and their relationships with their preceding and following arguments. Argumentation mining does not analyse the validity of the argumentation or its correctness. The aim is to detect those pieces of text which seem to function as argumentative (from a linguistic and semantic point of view) and the relations between them, i.e. their structure. The result is an argumentative structure of the text from the linguistic analysis of its propositions. (2)

So there are actually two main stages to argument mining (here presented in one breath):

  1. Detection of arguments in text
  2. Finding the particular type of relationship between these arguments

Related research

Two main approaches:

  1. Formal logic: inference with purely formal content, i.e. inference based on application of abstract rules (independent of its content)
  2. Informal logic: focus on understanindg everyday reasoning (which is often not deductive)

Psychologically speaking, argumentation is also a lot about persuasion; the mere repetition, or charisma of the speaker, can make an argument more persuasive and effective. Nothing of this plays a role in a formal approach to argument.

New, third approach:

  1. AI: analyze and perform argumentation/inference by computers.

Argumentation mining

Interdisciplinary discipline which combines 1) NLP, 2) argumentation theory, 3) information retrieval.

At the transition from page 3 to 4, the authors further specify what they mean with "relations":

  • local structure: "rhetorical or argumentative relationships between their propositions" (i.e. the components of arguments)
  • global structure: the relations and interactions between arguments (e.g. in a chain of reasoning).

Which questions need to be answered? See p. 4:

  1. What is the correct abstract structure of an argument? A tree or a graph? Which constraints need to be enforced in these representations?
  2. What are the elementary units of argumentation?
  3. What are the relations that hold between (units of) arguments? And how are they grounded? In worldly events or general principles of rhetoric and linguistics?
  4. Can we automatically determine (the units of) arguments?
  5. Can argumentation structures be determined automatically?

This paper provides suggestions for 1-3, and has some "positive answers" for points 4-5.

Elementary units of argumentation

Elementary units of argumentation are arguments. But what are arguments? Many definitions only seem to agree on the fact that arguments have premises and a conclusion.

Premises and conclusion can be implicit in free text. Apparently, these are called enthymemes. In this case, an argument's representation can be very minimal. The authors argue however that an argument is not recognizable if it does not at least have two propositions.

However, this does not exclude that two propositions occur in a single sentence. This depends on the type of text.

Legal documents are a bit more formal and will present premises and conclusion in separate sentences. This has two purposes: 1) clarify the argument structure and 2) extend the given information easily.

Internal structure of elementary units (i.e. arguments)

The most basic structure is premises + conclusion.

Other theories: Toulmin (1958), Walton (1998), Wigmore (1931).

This work is based on Walton (1996).

  • premises + conclusion are the only recognized units
  • arguments follow an argument scheme which reflects stereotypical reasoning patterns between arguments and conclusions
    • defeasible reasoning in particular

Relations between elementary units

Van Eemeren and Grootendorst (2004):

  • coordination : A -> B -> C
  • subordination : A,B,C,D -> E
  • multiple argumentation : A -> E, B -> E, C -> E

Argumentation structure

Because the authors choose for a simple conclusion-premise relation they can use a tree-structure for the arguments (instead of a graph).

This fits with pragma-dialectic theory: Van Eemeren and Grootendorst. Q: I don't know what that is.

  • leaves of the tree are the arguments
  • parents of an argument are either the premises and one conclusion
  • or another argument

In this paper a relation is identified but not further specified (e.g. "attack" or "support").

Experiments

Data

Corpus 1: Dataset mined from several newspapers, parliamentary records, court reports, magazines, online discussion boards.

  • First argument identified and analysed by hand
  • Create 50-50 split between argumentative and non-argumentative sentences by adding non-argumentative information from same sources added to corpus
  • sentences classified by text type: newspaper, parliamentary records, legal judgment, weekly magazine, discussion fora
  • argumentative sentences grouped as arguments in terms of an argumentation scheme: premises, conclusion, relations, type of reasoning scheme

Corpus 2: European Court of Human Rights ( ECHR ). These texts are suitable because they are quite structured for natural language:

The ECHR, over the years, has developed a standard type of reasoning and structure of argumentation. Therefore, its documents are a perfect test set for our argumentation analysis. It provides a set of documents which lack some natural language problems, such as extensive use of metaphor or high appearance of ambiguity, while maintaning enough complexity to be described as real natural language argumentation. (p. 9-10)

763 premises, 304 conclusions, 1449 non-argumentative sentences.

Argument detection

First we need to know which text contains the arguments of the text. This task can be reduced to a binary classification problem: is a sentence argumentative or non-argumentative.

The second step is to find out how all this argumentative information is split up in multiple arguments, which is a segmentation problem.

Argumentative information detection

  • Requires labels on sentence level
  • Create vector representation of each sentence using generic features
  • Study contribution of features to the classification of sentences as argumentative

Used techniques:

  • Maximum entropy model
    • Often used because information from natural language texts is often incomplete
  • Multinomial naive Bayes classifier with binary feature vector

Features:

  • Unigrams
  • Bigrams
  • Trigrams
  • Adverbs (uses POS tagger)
  • Main verbs (POS; excluding "to be", "to do", "to have")
  • Modal auxiliary (e.g. "may"; POS tagger)
  • Word couples, all possible combinations of two words in the sentence
  • Text statistics
    • Sentence length
    • Average word length
    • Number of punctuation marks
  • Punctuation sequence (all punctuation marks in a row)
    • Example: ":."
    • Recurring punctuation marks are mapped onto the same pattern ",,," -> ",+" for "two comma's or more"
  • Key words
    • Knott and Dale 1993: 286 words or word sequences that are indicative for argumentation
    • See Knott A, Dale R (1993) Using linguistic phenomena to motivate a set of rhetorical relations. Technical report, Discourse Processes
  • Parse features
    • Depth of the parse tree
    • Number of subclauses

Results:

  • 73% accuracy on Araucaria corpus
  • 80% on ECHR corpus
  • Shows that this approach is feasible

Argumentation information boundaries

Ideas:

  • Use structure of the document to determine start and end of an argument
    • Makes explicit assumptions with obvious limitations
  • Group semantically similar sentences into a single argument
    • Deal with ambiguity, coreference, pronoun resolution
    • Assumes relatedness of sentences is a function of their words
    • 1) Ontology based, e.g. distance in WordNet
    • 2) Corpus-based: distributional semantics

Results?

Argumentative proposition classification

If you have detected argumentative sentences, can you also try to classify them as premise or conclusion with the same approach.

Q: Why not use the same classifier and make it a multi-class problem with 1) non-argumentative 2) premise 3) conclusion?

A: Because the proposition classification requires more elaborate features. A second reasoning is that this classifier works on a single argument, whereas the purpose of the first classifier was to find the whole collection of combined arguments.

Used features:

  • Absolute location of sentence in document (measured in terms of 7 segments)
    • You'd expect generally that later sections contain more conclusions
  • Sentence length (binary feature indicating sentence is longer than some threshold words )
  • Tense of Main Verb (nominal: present, past, noverb)
  • History (most probable category of previous and next sentences)
    • I would have called this "context" since you also look at the "future" according to this description
    • The main text however only mentions it takes into account the previous sentence!
    • During testing you only probabilistically know the category of the previous sentence, so beam search is used.
    • What is meant here? And does this suggest that during training they use the true labels here in the feature? That is very weird to me.
  • Info 1st classifier (argumentative or not?)
  • Rhetorical Patterns between current, previous and next sentences (Support, Against, Conclusion, Other or None)
    • Defining this feature is a research problem in itself! How do they get this info?
    • From later text, this againdoes not become clear. Is there a "not" missing in the next sentence? The "but" indicates a contradiction, but there doesn't seem to be any.
    • "the Rhetorical Patterns feature models discursive relations, but it distinguishes the presence of discursive cues highly related to argumentation, expressed in two types (premise or conclusion), and the discursive cues (other) which are not related to the presence of a premise or conclusion" (14)
  • Article reference ( does the sentence contain a reference to an article? Uses POS)
  • Article (does the sentence include the definition of an article? POS + legal terminology)
    • I'm not entirely sure how a definition of an article looks like
  • Argumentative patterns (5 types)
    • e.g. "mutatis mutandis", "having reached this conclusion", "by a majority" (14)
    • TODO which 5 types?
  • Type of subject (agent: applicant, defendant, court, other; POS tagger, roles manually drafted)
  • Argumentative type of main verb (4 types: premise, conclusion, final decision, none)
    • Manual implementation: each type is associated with a list of verbs, which is then detected using a POS tagger

There are apparently five categories?

Some more rationale behind several features. Some are more indicative of premises, other more of conclusions.

The Article and Article reference tend to mark the role of premises, while Type of Main Verb signals that verbs such as conclude or decide have a higher chance of being the main verb of a conclusion than verbs like recall or note. (14)

Detection of argumentation structure

Authors have attempted to parse argumentation, including the relation between arguments. There are several approaches, but they used a context-free grammar hand-designed for the legal domain.

There exist different parsing approaches: rule-based (hand-crafted, transformation-based learning) or statistical (Hidden Markov Model, maximum entropy model, memory-based, decision tree, neural network, linear models), but for the time being we have focused on parsing the texts by means of manually derived rules that are grouped into a context-free grammar (CFG).

See paper for the grammar.

Applications of argumentation mining in law

The following idea combines argumentation mining with information retrieval:

When used for case management, a software using argumentation mining and other natural language and information retrieval techniques could allow the user to structure a collection of case-related documents in terms of the argumentation structure of a case. The structure would capture: the main issues; the main positions and arguments taken by the parties with respect to the issues; the available evidence related to them; and so on. Documents (statutes, case law, journal articles, testimony, letters, and so on) could be indexed following the argumentative structure, i.e. relating them to the relevant argumentation parts of the case, making it easier to find the evidence that supports or attacks that part of the argumentation. (19)

So we could index cases on:

  • main issues
  • main positions and arguments w.r.t. to the issues
  • available evidence per issue

Possible use case: find evidence (in the same case document?) that supports or attacks a particular part of the argumentation.

For case-based reasoning where you have a case base and match a new case/problem against the past cases to find similar cases... :

... argumentation mining offers a new way to compare cases, i.e. by their argumentation structure. Lawyers could compare cases by the amount of arguments that were necessary to prove a common conclusion. Then they would find which argumentation was more efficient and complete, i.e. the minimal set of arguments that can not be defeated. In another more complex application a lawyer could be interested in searching for all the previous cases which used a type of argument to attack another; here we refer to for example the use of an argument of expert opinion to attack a previous argument. The lawyer’s intention could be to find examples where it yields a favourable result and examples where the factfinder (e.g. court) was not convinced. Then the lawyer would analize how those arguments diverge, and therefore he would find the most effective or persuasive presentation of the facts. The current state of argumentation mining research would not allow this. However, we are confident that once the argumentation structure is well determined a classification of its arguments would be possible. (20)

Possible use cases from the quote above:

  • compare cases by amount of arguments given a common conclusion (i.e. common conclusion between the cases) to find a more "efficient" argumentation
  • search on a type of argument used for attack another argument
  • find cases with favorable results and cases where the factfinder was not satisfied for a similar issue -> the point here is to then analyze what distinguishes a successful and unsuccessful argument

Other:

  • Automated decision systems' main bottleneck is knowledge engineering. If argumentation mining would be successful, this would allow larger databases for developing reasoning systems on.
  • Argument visualisation tools
  • In various business areas argumentation mining could help maintain and visualize discussions for very large or long meetings (I guess where it's somehow infeasible if humans do this?)