1 of 13

Automatic Detection of Entity-Manipulated Text Using Factual Knowledge

Ganesh Jawahar, Muhammad Abdul-Mageed, Laks V. S. Lakshmanan

2 of 13

Problem

Distinguish a human written news article from a manipulated news article

PubNub, a startup that develops the infrastructure to power key features in real-time applications (...) has raised $23 million in a series D round of funding from Hewlett Packard Enterprise (HPE), Relay Ventures, Sapphire Ventures, Scale Venture Partners, Cisco Investments, Bosch, and Ericsson.

PubNub, a startup that develops the infrastructure to power key features in real-time applications (...) has raised $23 million in a series D round of funding from Hewlett Packard Enterprise (HPE), Samsung, Sapphire Ventures, Scale Venture Partners, Cisco Investments, Bosch, and Ericsson.

We focus only on replacing some entities in a human written news article with manipulated entities.

3 of 13

Motivation

  • “the best policy for the criminal is to tell the truth as nearly as possible.” Raskolnikov, “Crime and Punishment” (Dostoyevsky 1866).
  • Detecting entity-manipulated text helps detect one form of misinformation, which is easy/cheap to create.
  • Understudied subfield of fake news detection
  • Existing detectors don’t perform well as they over rely on stylometric signals

4 of 13

Most Relevant Work

The Limitations of Stylometry for Detecting Machine-Generated Fake News, Schuster et al., Computational Linguistics Journal, 2019

    • Existing detectors (e.g., Grover) fail to identify manipulated text (e.g., adding/deleting negations) well as they over rely on stylometric signals.
    • Humans can only identify the manipulated text well when they are allowed to consult external sources (e.g., internet).

​

5 of 13

Our Approach

Entity-Manipulated Text Creator

Human text

Entity-Manipulated Text Detector

(distinguishes text from human and text from entity-manipulated text creator)

Human text

Consults knowledge base

Manipulated text

Human text

Manipulated text

6 of 13

Entity-Manipulated Text Creator

Entity-Manipulated Text Creator

Human text

Manipulated text

PubNub, a startup that develops the infrastructure to power key features in real-time applications and devices covering chat, tracking, and internet of things (IoT), has raised $23 million in a series D round of funding from Hewlett Packard Enterprise (HPE), Relay Ventures, Sapphire Ventures, Scale Venture Partners, Cisco Investments, Bosch, and Ericsson.

PubNub, a startup that develops the infrastructure to power key features in real-time applications and devices covering chat, tracking, and internet of things (IoT), has raised $23 million in a series D round of funding from Hewlett Packard Enterprise (HPE), Samsung, Sapphire Ventures, Scale Venture Partners, Cisco Investments, Bosch, and Ericsson.

Human text

Manipulated text

Prompt

GPT-2

Samsung

Generated entity

Entity replacement

7 of 13

Entity-Manipulated Text Detector

PubNub, a startup that develops the infrastructure to power key features in real-time applications (...) has raised $23 million in a series D round of funding from Hewlett Packard Enterprise (HPE), Samsung, Sapphire Ventures, Scale Venture Partners, Cisco Investments, Bosch, and Ericsson.

Samsung

type

Organization

Ericsson

memberOf

FIDO Alliance

Entity-relation graph

Graph Convolutional Network

RoBERTa

Manipulated Article Detector

Manipulated text

Manipulated Entity Classifier

Samsung

Detector

Representation

Raw data

Decision

8 of 13

Experimental Settings

  • Dataset – NeuralNews (Zellers et al., 2019)
  • Baseline detector – RoBERTa
  • Other (non-contextual) manipulated text creation strategies:
    • Random least frequent entity
    • Random most frequent entity

​

Sample manipulated entities

9 of 13

Results

Manipulated Article Detection Accuracy

(for different entity replacement strategies)

Random least

1

2

3

RoBERTa

67.09

78.37

84.26

Ours

68.25 (1.7%)

78.99 (0.8%)

83.84 (0.4%)

Random most

1

2

3

RoBERTa

65.56

76.86

83.93

Ours

67.21 (2.5%)

78.26 (1.8%)

84.39 (0.5%)

GPT-2

1

2

3

RoBERTa

67.09

74.12

78.79

Ours

65.84 (1.9%)

74.80 (0.9%)

79.05 (0.3%)

10 of 13

Results

Manipulated Entity Identification

Random least

1

2

3

Precision

81.06

91.76

84.14

Recall

0

3.7

12.12

F-Score

0

7.11

21.19

Random least

1

2

3

Precision

84.71

88.06

86.06

Recall

6.08

4.63

14.03

F-Score

11.35

8.8

24.13

GPT-2

1

2

3

Precision

85.59

85.91

73.8

Recall

9.14

1.64

12.5

F-Score

16.52

3.22

21.38

11 of 13

Final Takeaways

  • Fake news propagator can manipulate exactly one entity to make the detection task harder.
  • Manipulation using GPT-2 can keep the detection task harder even for large number of replacements.
  • Explicit factual knowledge can complement textual knowledge.
  • There’s a lot of room for improvement in detection tasks (>20% accuracy) and entity identification tasks (>75% F-score).

​

Code and data: https://github.com/UBC-NLP/manipulated_entity_detection

12 of 13

Future Directions

  • Bidirectional context for entity generation
  • Principled fact extraction for a given article
  • Tighter coupling of factual and textual information
  • Entity classification loss that’s sensitive to imbalanced nature
  • Other types of manipulations: negation, subject-object exchange
  • Consider other types of entities apart from persons, organization and locations

​

​

13 of 13

Thank you! Questions?