1 of 12

2024710737

Suyeon Lee

LogBERT: Log Anomaly Detection via BERT

IEEE IJCNN, 2021

2 of 12

Contents.

2

  1. Introduction
  2. LogBERT
  3. Dataset
  4. Experiments
  5. Result
  6. Conclusion and Limitation

3 of 12

Introduction

3

  • Motivation�- Manual detection of abnormal events is inefficient due to the scale and complexity of log data

​

  • Limitations of Existing Method�- There is a limit to detecting a new type of abnormal event�- Although used sequence models such as RNN, LSTM, and GRU, these models do not fully utilize the two-way contextual information of log sequences.

​

🡺 Propose a framework that applies the robust bidirectional contextual learning capabilities of Bidirectional Encoder Representations from Transformers (BERT) to log anomaly detection

4 of 12

LogBERT

4

5 of 12

LogBERT

5

  • Log Key Sequence
  • Input Representation
  • DIST: Special tokens representing the entire log sequence
  • MASK: Special tokens representing masked log keys

6 of 12

LogBERT

6

  • Transformer Encoder�- A step to learn the contextual relationship of log sequences by consisting of multiple transformer layers

​

  • Self-supervised Tasks�- Masked Log Key Prediction (MLKP): Capturing two-way contextual information for log sequences�- Volume of Hypersphere Minimization (VHM): Focus on the distribution of normal log sequences�

7 of 12

Dataset

7

  • Dataset
  • Baselines�- Principal Component Analysis (PCA) �- One-Class SVM (OCSVM)�- IsolationForest (iForest) �- LogCluster�- DeepLog�- LogAnomaly

8 of 12

Result

8

  • Experimental Results on HDFS, BGL, and Thunderbird Datasets

9 of 12

Result

9

  • Performance of LogBERT base on One Self-supervised Training Task

10 of 12

Ablation

10

  • MLKP and VHM Individual Performance Evaluation
  • Analyzing Parameters

🡪 VHM operations help the model distinguish �between normal and abnormal data

11 of 12

Conclusion and Limitation

11

  • Conclusion�- LogBERT is a novel log anomaly detection model based on BERT, which is learned through two self-supervised learning tasks.�- As a result of evaluating the performance through experiments on three log datasets, it outperforms the state-of-the-art methods in anomaly detection.

​

  • Limitation�- What log parser method was applied is not shown in the paper.

12 of 12

Thank you.

12

x

x