Argument Extraction from News | Sardianos, Manousis Karatis, Petasis, Karkaletsis

  • identify segments containing claims and premises
  • in Greek
  • small set of thematic domains
  • "exploits distributed representations of words, extracted from a large non-annotated corpus" (56) -> word vectors/embeddings
  • justification of application to social web: "The huge increase of social web communities, along with their user tendency to debate, makes the identification of arguments in these texts a necessity." (56)

From conclusion:

In this research paper we propose an approach for argument extraction that exploits distributed representations of words in order to be applicable on multiple thematic domains, without requiring any other linguistic resource beyond a part-of-speech tagger and a small list of cue words. Our goal was to suggest a semi-supervised method, applicable from traditional news and blogs documents to corpora from social web, mainly written in the Greek language. The proposed approach is based on previous research performed on this domain and attempts to extend its existing functionality. As gazetteer lists of entities and cue words play an important role to the argument extraction process, we suggest the expansion of the above gazetteer list which are usually provided by domain experts (in our case policy makers), using semantic similarities. (64)

  • One of their main goals was a semi-supervised approach, but I disagree that they actually have a semi-supervised approach; all their ML-classifiers use a fully annotated dataset
    • The only "unsupervised" aspect are the word embeddings, but these are only used to generate features and have nothing to do with the labels used during trainig
  • I thought the word embeddings would play a more direct role in the classification process, but they are only used to generate a few more domain-specific features
  • Another main goal was to generalize to more domains; I don't really see any experiment that compares generalization

Introduction

Mining on the social web commonly uses sentiment analysis techniques in orde to "capture sentiment polarity of people about a specific topic". (56) This is particularly interesting in areas where public opinion matters a lot, e.g. e-government and policy making.

Further argument extraction can help identify exactly which portions of a text elicit a particular response.

Approach:

  • Conditional Random Fields classifier
  • supervised
  • already assumes to have all the argumentative sentences; focuses on classifying premise vs. claim
  • tries to be more domain-independent: "words, part-of-speech tags, small lists of language-dependent cue words, and distributed representations of words" (57)

Related work

The step that this paper skips, namely the classification of argumentative sentences, is typically a first key step of the argumentation pipeline. This section mentions an interesting paper that uses a different approach altogether:

[Lawrence et al., 2014] combined a machine learning algorithm to extract propositions from philosophical text, with a topic model to determine argu- ment structure, without considering whether a piece of text is part of an argument. Hence, the machine learning algorithm was used in order to define the boundaries and afterwards classify each word as the beginning or end of a proposition. Once the identification of the beginning and the ending of the argument propositions has finished, the text is marked from each starting point till the next ending word. (57)

  • Which ML classifier?
  • How did they combine this with a topic model?

Also interesting combination with topic-based arguments:

[Boltužić and Šnajder, 2014] collected comments from online discussions about two specific topics and created a manually annotated corpus for argument extraction. Afterwards they used a supervised model to match user-created comments to a set of predefined topic-based arguments, which can be either attacked or supported in the comment. In order to achieve this, they used textual entailment (TE) features, semantic text similarity (STS) features, and one “stance alignment” (SA) feature. (57)

Proposed approach

  • Conditional Random Field classifier for segmentation
  • Features:
    • General: words and POS tags
    • Specific cue words signalling arguments
      • distributed representations using word2vec for semantic similarity
    • "widen the “word space” beyond the words observed during the training phase" (59)

Expansion of word feature space

The authors use the word2vec model to expand the "word feature space" with the top k most similar words. The idea is to perform better across multiple thematic domains with a single approach. I'm not entirely sure what their exact approach is. Are these features generated on the fly for each text?

Semi-supervised approach for extracting argument components

Extends approach by Goudas et al, 2014.

  1. identify argument sentences with selection of general state of the art features and domains-specific features
    • general features incl. position of sentence in the document; number of comma's and connectives; number of verbs (active vs passive voice); existence and amount of cue words; entities; number of words and adverbs in sentence context; average word length
    • domain specific: nr. adjectives; number of entities in n-th previous sentence; total entities in n previous sentences;
    • ratio of distributions over unigrams, bigrams, trigrams of words, and over POS tags
  2. identify claim vs premise
    • "CRF classifier on a manually corpus." (60) -> typo, but manually annotated corpus
    • features were: words of sentence; gazetteer lists of known entities for the domain; gazetteer lists of cue words; lexica of verbs and adjectives that appear most frequently in argumentative sentences of the training data
    • NEW: also use features based on word embeddings

Q: What are gazetteer lists?

In our approach, we want to reduce this dependency on gazetteers, by exploiting distributed representation for words, using the proposed method described in subsection 3.1. This will help us widen the spectrum of words that can be handled by our classifier and thus, manage to create a more fine-grained CRF model. (61)

Q: Why is this semi-supervised?

The innovation of this work is to use the "expansion of word feature space"-trick mentioned in the previous section. So I think they call their approach semi-supervised because a part of their feature space comes from an unsupervised technique (namely word2vec)? This is not what I would call semi-supervised, because these features are still used for a fully supervised ML classifier on a manually annotated set...

I also still do not get whether they directly do something on the word2vec vectors, or only use word2vec to dynamically extract some new features for a given domain.

Empirical evaluation

This section is not that well-written. In particular, I find that the autors keep introducing "method" related remarks in later sections. In this "evaluation" section there's even a new "Experimental Setup" section.

Extend gazetteer / lexicon of cue words or domain-specific entities with new "entries" (features?). Again, a bit too vague in my opinion! Same description as before, but now it's used in evaluation? How exactly?

One word2vec model per domain. Given cue words, get top k most similar words. Use these as word features for the classification steps, if they are deemed useful (i.e. they are cue words or indicate relevant entities; this is manually checked).

Q: Does this idea scale? So for each domain, you'd add another set of word features?

According to these results, we can conclude to the fact that “word2vec” can be used for the expansion of the cue word lexicons. In addition, it can be proven a valuable resource as regards to the enrichment of the entities provided by the policy makers. (62)

Evaluation set needed to be created:

The guidelines contained text examples of premises in favor or against the central claim stated by the articles’ author. In these terms, the annotators were initially called to identify the central claims stated from the author of each article. Subsequently, they looked for text segments attacking or supporting every claim respectively. These segments may sometimes start with cue words such as “διότι” (“because”), “για να” (“in order to”), “αλλά” (“but”), or may just follow the usual sentence struc- ture. Each annotator annotated 150 documents with argument components (premises and claims). (63)

Classification task in this case is to segment argumentative sections, i.e. not to classify each sentence first as argumentative or not. For this purpose there are five class labels defined on single tokens (so not on sentences!):

  • B beginning of a segment
  • I inside a segment
  • L last token of segment, i.e. its end
  • O outside a segment
  • U unit segment, edge case where a segment has a single token only