Toxic Comment Detection for Semnan University

An automated cyber-security pipeline using Hugging Face Transformers and the unitary/toxic-bert model to classify social-media comments as safe or toxic before they appear on the university website.

​

Semnan University | Natural Language Processing (NLP) | Text Classification

WESAM KAREEM

404XXXXXXX

PhD — Second Semester Semnan University

Deep Neural Networks

Student Number

Program

University Course

1

Project Overview

Universities receive hundreds of comments on their official Facebook pages and websites. Some comments are constructive feedback, while others contain offensive language, personal attacks, or cyberbullying. Manual moderation is slow and inconsistent.

This project builds an automated content-moderation system that scans incoming comments in real time and flags toxic content before it is published. The first phase focuses on loading a pre-trained model and validating it against sample comments related to Semnan University.

​

Social Media Comment _→ toxic-bert Model _→ Toxicity Score _→ Safe / Toxic Label

​

​

Goal of Phase 1: Install the required libraries, load the unitary/toxic-bert model from Hugging Face, run five test comments, and verify that safe comments are classified correctly while toxic ones are flagged.

2

Step 1 — Selecting the Model on Hugging Face

The model chosen for this project is unitary /toxic-bert, hosted on the Hugging Face Model Hub. It is a BERT-based classifier fine-tuned specifically for detecting toxic, hateful, and abusive language in English text.

This model is based on the Jigsaw Toxic Comment Classification Challenge dataset ( jigsaw-toxic-comment-classification-challenge ), available on Kaggle: kaggle.com/c/jigsaw-toxic-comment-classification-challenge/data. The dataset contains thousands of Wikipedia talk-page comments labeled as toxic or non-toxic, which was used to train and fine-tune BERT models for automated content moderation.

Why toxic-bert?

Fine-tuned on the Jigsaw toxic comment dataset — a large, labeled corpus of toxic vs. non-toxic comments. Available as a ready-to-use pipeline via the transformers library — no custom training required for Phase 1. Returns a confidence score (0–100%) for each prediction, enabling a clear threshold rule.

Widely used in industry for content moderation and cyberbullying detection.

Figure 1 — The unitary/toxic-bert model page on Hugging Face Hub, confirming the model selection for this project.

3

Step 2 — Environment Setup & Load Model

​

2.1 Install Dependencies

# Install required libraries (run once) pip install transformers torch

2.2 Load the Model & Prepare Test Comments

from transformers import pipeline

​

# Load the toxic-bert classifier from Hugging Face

cyber_detector = pipeline("text-classification", model="unitary/toxic-bert")

​

# Sample comments simulating Facebook posts about Semnan University comments = [

"Semnan University has an amazing campus and the professors are very supportive. Proud to be a student here!",

"The engineering department at Semnan University is doing a great job. Thanks for the guidance.",

"The website of this university is absolute trash, and the IT staff are completely useless and stupid!", "I really hate this place, the exams are unfair and the professors are terrible losers.",

"I faced some challenges with my registration, but the academic staff helped me resolve it quickly."

]

4

Step 2 — Classification Logic & Execution

​

2.3 Classification Logic

Each comment is passed through the model. The returned score represents the model's confidence that the text is toxic. A threshold of 50% is applied:

Score > 50% → classified as Toxic (cyberbullying / offensive) Score ≤ 50% → classified as Safe (constructive / neutral)

print("- Cyber Security: Semnan University Comment Analysis Results - \n") for comment in comments:

result = cyber_detector(comment)[0]

score = result['score'] * 100 # Convert to percentage

​

if score > 50.0:

status = "Toxic (Offensive / Cyberbullying)" confidence = score

else:

status = "Safe (Constructive / Neutral)" confidence = 100 - score

​

print(f"Comment: \"{comment}\"")

print(f"Result: {status} | Confidence: {confidence:.2f}%\n")

Figure 2 — The Python script loaded and executed in the development environment, showing the toxic-bert pipeline in action.

5

Step 3 — Analysis Results

After running the script, the model correctly separated safe comments from toxic ones. The table below summarizes the expected classification for each test comment:

5

Registration issue resolved by staff

Safe

Constructive criticism with positive resolution

Figure 3 — Console output showing three Safe comments (green) and two Toxic comments (red) with confidence percentages.

Validation passed: Comments 1, 2, and 5 received a Safe label because they either praise the university or offer respectful criticism.

Comments 3 and 4 received a Toxic label due to offensive words such as trash, useless, stupid, and losers — confirming the security system successfully filters harmful content on the university page.

6

Technical Summary

Component Details

Platform Hugging Face Model Hub

​

Model unitary/toxic-bert

Training Dataset Jigsaw Toxic Comment Classification Challenge ( jigsaw-toxic-comment-classification-challenge ) Task Text Classification (Toxic vs. Non-Toxic)

​

Library transformers + torch

​

Threshold 50% — score above threshold = Toxic

​

Test Set 5 sample comments (3 safe, 2 toxic)

​

Accuracy (Phase 1) 5 / 5 comments classified correctly