1 of 26

Adversarial

Training for Aspect-based Sentiment

Analysis

Introduction

Previous Approaches

Albat Architecture

Results

Metrics

Case Study

Resource Usage

1

Daniel Williams

2 of 26

Introduction

Task Definitions

2

“The food was served promptly but the meal wasn’t rushed - we had plenty

of time to enjoy the appetizers and our entrees as well as sit and chat while

finishing up our drinks even after we paid.”

3 of 26

Introduction

Task Definitions

3

“The food was served promptly but the meal wasn’t rushed - we had plenty

of time to enjoy the appetizers and our entrees as well as sit and chat while

finishing up our drinks even after we paid.”

  • Aspect Extraction (AE)
  • Aspect Sentiment Classification (ASC)
  • End-to-end Aspect-based Sentiment Analysis (E2E-ABSA)

4 of 26

Introduction

Task Definitions

4

“The food was served promptly but the meal wasn’t rushed - we had plenty

of time to enjoy the appetizers and our entrees as well as sit and chat while

finishing up our drinks even after we paid.”

  • Aspect Extraction (AE)
  • Aspect Sentiment Classification (ASC)
  • End-to-end Aspect-based Sentiment Analysis (E2E-ABSA)

5 of 26

Introduction

Previous Approaches

5

  • Information Retrieval and Supervised Learning

6 of 26

Introduction

Previous Approaches

6

  • Deep Learning: Recursive and Recurrent Neural Networks

7 of 26

Introduction

Previous Approaches

7

  • Deep Learning: Convolutional Neural Networks

8 of 26

Introduction

Previous Approaches

8

  • Deep Learning: Attention Networks and Transformers

9 of 26

Introduction

Previous Approaches

9

  • Deep Learning: BERT

10 of 26

Introduction

Previous Approaches

10

  • BERT: Training

11 of 26

Introduction

Previous Approaches

11

  • BERT: Fine-tuning

12 of 26

Introduction

Previous Approaches

12

  • BERT: Resource Usage

13 of 26

Albat Architecture

Overview

13

14 of 26

Albat Architecture

Adversarial Training

14

15 of 26

Albat Architecture

Further Pre-Training

15

16 of 26

Results

Datasets and Metrics

16

English Datasets

  • Lapt14
  • Rest14
  • Unified
  • MAMS

Mandarin Datasets

  • Camera
  • Car
  • Notebook
  • Phone

AE

  • F1 Score

ASC

  • Accuracy
  • Macro F1 Score

E2E-ABSA

  • Accuracy
  • Macro F1 Score

17 of 26

Results

E2E-ABSA

17

State of the art performance on Lapt14 and Unified:

18 of 26

Results

AE

18

State of the art performance on Car, competitive on all other datasets:

19 of 26

Results

ASC

19

State of the art performance on MAMS, competitive on Notebook and Phone:

20 of 26

Results

Case Study

20

Ground Truth:

The food was served promptly but the meal wasn't rushed - we had plenty of time to enjoy the appetizers and our entrees as well as sit and chat while finishing up our drinks even after we paid.

Albat-1LC Prediction:

The food was served promptly but the meal wasn't rushed - we had plenty of time to enjoy the appetizers and our entrees as well as sit and chat while finishing up our drinks even after we paid.

Albat-2LC Prediction:

The food was served promptly but the meal wasn't rushed - we had plenty of time to enjoy the appetizers and our entrees as well as sit and chat while finishing up our drinks even after we paid.

Albat-3LC Prediction:

The food was served promptly but the meal wasn't rushed - we had plenty of time to enjoy the appetizers and our entrees as well as sit and chat while finishing up our drinks even after we paid.

21 of 26

Results

Case Study

21

Error Analysis

22 of 26

Results

Resource Usage

22

23 of 26

Results

Resource Usage

23

24 of 26

Results

Resource Usage

24

25 of 26

Results

Resource Usage

25

26 of 26

Summary of Contributions

26

  1. Adversarial training for ABSA using Albert;
  2. Adversarial training for E2E-ABSA;
  3. Evaluation with eight datasets across two languages;
  4. State of the art for all E2E-ABSA metrics, two AE metrics, and one ASC metric.