Toward Artificial Argumentation | Katie Atkinson, Pietro Baroni, Massimiliano Giacomin, Anthony Hunter, Henry Prakken, Chris Reed, Guillermo Simari, Matthias Thimm, Serena Villata
This paper gives a quick overview of the field. For a more detailed overview they recommend some other papers:
- Rahwan and Simari (2009)
- Modgil et al (2013)
- Proceedings of the International Conference on Computational Models of Argument series
- Argument and Computation journal
Motivation: artificial argumentation as a step towards robust AI:
if we are to develop robust intelligent systems, then it is imperative that they can handle incomplete and inconsistent information in a way that somehow emulates the way humans tackle such a complex task. (p. 1)
It is supposed that argumentation, either internal or external, is a main manner in which humans solve complex tasks.
Q: These makes some assumptions. Which ones, and do they hold?
The importance of argument:
As a cognitive capacity, argumentation is important for handling conflicting beliefs, assumptions, viewpoints, opinions, goals, and many other kinds of mental attitudes. When we are faced with a situation where we find that our information is incomplete or inconsistent, we often resort to the use of arguments in favor and against a given position in order to make sense of the situation. When we interact with other people we often exchange arguments in a cooperative or competitive fashion to reach a final agreement or to defend and promote an individual position. (24)
Models of Argument
There is a wide array of possible argumentation structures, but in this section the authors try to identify five abstract layers that most models of argumentation may use:
- Structural: How are argument structured?
- Relational: What are the relationships between arguments?
- Dialogical: How can argumentation be undertaken in dialogues?
- Assessment: How can a constellation of interacting arguments be evaluated and conclusions drawn?
- Rhetorical: How can argumentation be tailored for an audience so that it is persuasive?
Structural layer
Specifies what an argument looks like in terms of the internal structure between components like premises and conclusions. A good example are logic-based approaches where an argument can be constructed over logical symbols by following the rules of that particular logic, given a particular knowledge base.
Prominent examples of rule-based formalisms are ASPIC+, assumption-based argumentation (ABA), and defeasible logic programming (DeLP). For a tutorial introduction to formalisms for structured argu- mentation, see Besnard et al. (2014).
On top of a knowledge base, there are also argumentation schemes (Walton, Reed, and Macagno 2008).
- "stereotypical reasoning patterns" (26)
- Beyond premises and claims these add "a set of critical questions" which "provide a sort of checklist of issues that can be raised to challenge arguments built on the basis of a given scheme" (26)
- Have also been used as "a source of defeasible inference rules in rule-based approaches to argument construction from knowledge bases" (26)
- Used in argument mining as well
How do you share arguments between systems if indeed arguments are so diverse? The argument interchange format is a generic abstract ontology for capturing arguments.
Relational layer
Important relationships:
- subargument and superargument relation
- attack relation (can be contradictory claims, contradiction with a premise or assumption)
- support, "intuitively meaning that an argument provides some backing to another, and admitting several, even rather dissimilar, interpretations, depending on the actual nature of this backing" (26)
- preference: "ranking arguments according to some criterion, and admit- ting again a variety of instantiations ranging from strength to credibility to value-based evaluations." (27)
Word of caution:
Note in particular that identifying argument relations may be an easy mechanical procedure in settings where arguments are formally built from a knowledge base, while in an argument mining scenario it is a task as challenging as the identification of the arguments themselves. (27)
My own project idea actually was to try and assess whether two legal cases with a citation relation have a supporting or attacking relation.
Dialogical layer
Exchange of arguments according to formal dialogue rules.
Forms: " inquiry, negotiation, information-seeking, deliberation, and persuasion" (27)
This layer requires integration with the structural and relational layer because:
- With structural: "It concerns representing and managing the locutions exchanged between the agents involved, as well as specifying the contents of these locutions in terms of entire arguments or components of arguments." (28)
- With relational: A different particular protocol may be chosen given particular argument relationships
Assessment layer
This is the step towards justification.
"he need for this layer arises from the presence of attacks among arguments, pre- venting them so as to be accepted altogether and calling for a formal method to solve the conflict." (28)
-
Abstract argumentation frameworks / argument graph
-
deprived of structural properties
-
treat all arguments as attacks requiring resolving
-
Abstract argumentation semantics
"An abstract argumentation semantics is a formal criterion to determine which sets of arguments, called extensions, are able to survive the conflict together and can be regarded as collectively acceptable." (28)
An example of approach 1 is the framework of Dung (1995), which is so general that the authors argue that approach 2 can be seen as a subfield of it.
Dung’s approach abstracts from the origin and nature of the attack relation. A natural idea is to define this relation in terms of a more basic notion of conflict between arguments (for example, two arguments having contradictory conclusions) and a notion of relative argument strength or preference. In the literature, there are two ways to connect these ideas to Dung’s frameworks. The first approach leaves Dung’s frameworks as they are but connects them with models at the structural layer of argument to define attack in terms of preferences or argument strength while taking the structure of arguments into account. The second approach instead extends Dung’s frameworks with some abstract notion of argument strength or preference, while possibly also adding an abstract support relation between arguments. Moreover, while most approaches consider a qualitative notion of acceptance, quantitative assessments methods are being investigated too. (28)
This is only a first step towards the actual goal of assessment, namely to determine the support and justification of the claim that is being argued for. This is a very complex task that can also be approached in several ways, two reasons being:
Since many arguments may have the same conclusion, assessing the status of a statement involves a synthesis of the statuses of the arguments supporting it. As in real life, the task of deciding what to believe may be carried out adopting different attitudes, ranging from extremely skeptical to extremely credulous, corresponding to different formal methods for statement justification synthesis. (28)
Rhetorical Layer
Arguments are usually made with some goal in mind, e.g. to persuade someone. The rhetorical layer can be absent, but if you have it it most probably interacts with the other layers:
goal-oriented considerations may drive the decisions of which arguments to build, taking into account their relations with other arguments, of whether, how, and when to use the arguments in a dialogue, and of which assessment method (for example, whether a more skeptically or more credulously oriented one) to apply. (29)
Legal argumentation
The legal domain is great for this type of work!
- argumentative in nature
- on top, to a large extent recorded in documents
- legal arguments are highly stylized, which makes all of the above tasks easier
Recommended read: Prakken and Sartor (2015)
Steps of legal evidential reasoning:
- Establish legal facts
- Classify under the condition of legal rules, which requires interpretation of these rules
- Application of legal rules to facts
Step 1: Establishing facts
In legal cases the facts are presented first as evidential support for the final ruling. This type of evidential reasoning uses evidence of various forms and shapes.
Because of the diverse nature of the evidence in most cases and the need for explanation to statistical laypeople, legal evidential reasoning is an excellent test bed for combined qualitative and quantitative models of defeasible reasoning (29)
See defeasible reasoning .
Step 2: Classify facts under rules
These systems "model how lawyers in common-law jurisdictions make use of past decisions when arguing a case" (29):
- HYPO by Kevin Ashley
- CATO by Vincent Aleven
These models assume that a case is a collection of abstract factors or dimension that contribute to a case outcome:
Their underlying argumentation model is for factor- or dimension-based reasoning, where cases are collections of abstract fact patterns that favor or oppose a conclusion, either in an all-or-nothing fashion (factors) or to varying degrees (dimensions). (29)
There are also formal models that formalize various typical legal forms of reasoning such as "appeal to precedent and policy" and "balancing of goals, values, and interests" (29)
Step 3: Apply rules to facts
The caveat is that rules can 1) have exceptions and 2) be conflicting.
A main approach is rule-based argumentation logics with preferences, where preferences can be used to choose one approach over the other in case of conflict.
Outlook
Currently, the existing prototype systems are not used in practice.
"One reason is the conservative attitude to technology in the legal world and its billing-by-the-hour culture, which does not stimulate innovation." (29)
But change is a-coming:
Moreover, the recent advances in natural language processing, machine learning, and data science combined with the massive digital availability of legal data and information have created the prospects for combining AI models of legal argument with argument mining techniques. In fact, two of the first argument mining projects were on legal argument (Palau and Moens 2011). If this technology is combined with AI and law’s computational models of argument, then practical applications of these models could be well within reach. (29)
This is pretty much what I want to do!
How can we combine ML-based approaches to legal argument with argument mining techniques?
TODO: Palau, R. M., and Moens, M. 2011. Argumentation Mining. Artificial Intelligence in Law 19(1): 1–22. doi.org/10.1007/s10506- 010-9104-x
Medical argumentation
"It is common for health-care information to be complex, heterogeneous, incomplete, and inconsistent, and therefore argumentation is appealing for those involved as it allows for important conflicts to be highlighted and analyzed and unimportant conflicts to be suppressed." (29-30)
Think of
- Decision support systems that give possible pros and cons for alternative drug treatments
- Systems for evidence-based reasoning
- The authors give one example, but I'm immediately thinking of Bayesian network-based systems here
- You can factor in e.g. patient preferences, previous response to treatments etc.
- You can factor in the quality of used evidence; the worse the quality, the easier to overthrow
- Generate Argument graph -> abstract argumentation semantic
Further applications include dealing with the conflicts that can occur when using multiple clinical guidelines, supporting multidisciplinary teams of healthcare professionals when dealing with difficult clinical cases, and supporting medical image interpretation. (30)
e-Government
- modeling debates
- facilitating interaction with government bodies
- collaborative decision support, e.g. Zeno en HERMES
- "contained more structure by making use of the IBIS (Issue Based Information Systems) model of argument. Use of this model enabled a particular problem or issue to be decomposed into a number of different positions for which arguments can then be created to attack or defend the positions until the issue is settled." (31)
- etc.
Argument Mining
This tasks became increasingly more useful with the ever expanding amount of online information, which makes it useful to automatically identify, structure, summarize online resources.
The main goal of argument mining is the automated extraction of natural language arguments and their relations from generic textual corpora, with the final goal to provide machine-readable structured data for computational models of argument and reasoning engines. (31)
Two main stages:
- Argument extraction
- Detect arguments in free natural language text
- Nodes in argument graph
- Subtask: detection of argument boundaries
- SVMs, naive Bayes, logistic regression
- Relation extraction
- Internal: between components of arguments, e.g. connection premises and claims
- External: relation between two arguments
- Involves "high-level knowledge"
- "Being an extremely challenging task, existing approaches assume simplifying hypotheses, like the fact that evidence is always associated with a claim." (31)
Identifying the relation between arguments is very difficult, sometimes ambiguous, and requires interpretation. E.g. A1 may have a support relation with A2, but could also be interpreted as a sub-argument. Currently, a deep level of natural language understanding is needed for NLP techniques to really adequately classify argument relations.
Also, NLP and ML-based approaches require high-quality labelled datasets:
- golden standard for argument mining with validation expert annotations
- guidelines to explain how data has been annotated
- unlabelled raw corpus for testing the system
Recommended reading, state-of-the-art argument mining techniques: Lippi and Torroni (2016)
Debating Technologies
These systems have two things in common: first, they implement idiosyncratic and new dialogical structures, or games, with little reuse or incremental development. The second is that little or no contribution to the debate itself is offered by the machine. The system role has been one of support and facilitation only. With a variety of techniques for automatically mining argument structures from both monological and dialogical resources, an exciting new possibility is opened up for not just supporting and enhancing new human-human arguments but also developing new human-machine arguments: this is the space of debate technology, a specific sub-field of argument technology in general. (32)
Interesting, but skipped because currently not my interest.
Argumentation Solvers
- Given an argumentation graph compute the extensions of such a graph i.e. "sets of arguments that can collectively be accepted, given the attack relation represented by the arcs" (33)
- Answer questions like: is an argument contained in all extensions?
- These problems are often harder than NP-hard .