MACHINE LEARNING PROJECT

FTPGuard:

A host-based FTP intrusion detection system using unsupervised machine learning and signature rules

REPOSITORY

github.com/AmraniCh/ftp-ids-ml

Realized by:

• CHAKIR EL AMRANI
• BADR HASSEBI

Supervised by:

• Pr. Abdelhak MAHMOUDI

Plan

01

Context & Objectives

FTP security · IDS approaches · Project goals

02

Data Pipeline

Architecture

· Parsing · Sessions · Case study

03

ML Detection

Features · ML pipeline · Isolation Forest

04

Results & Quality

Demo · CLI · Unit tests

05

Future Work & Conclusion

Limitations & Future Work

· Conclusion

02

FTP at a Glance

FTP CONNECTION

Client

Server (vsftpd)

USER / PASS, RETR, STOR…

Authentication via username + password in plain text (no TLS)

Simple text-based commands (USER, PASS, RETR, STOR, LIST…)

Every action is logged server-side (e.g. vsftpd.log)

Still everywhere: web hosting, internal transfers, IoT, legacy systems

WHY IT'S A TARGET

No encryption by default

Credentials and files travel in plain text over the network.

Large attack surface

Reachable from the Internet on most hosting providers.

A favorite target for bots

Continuously scanned for default or weak credentials.

After-the-fact detection

Without active monitoring, an intrusion can go unnoticed.

Separate data channel for file transfers

03

What Is an IDS?

Signature-Based Detection

Compares every action against a database of known attacks

Very precise on already-catalogued threats

Blind to novel attacks (zero-day)

Requires continuous signature updates

Anomaly-Based Detection

Learns a profile of "normal" behavior

Flags anything that deviates significantly from that profile

Able to detect threats never seen before

Approach chosen for ftp-ids-ml

04

Project Objectives

01

Parse FTP logs

Turn raw vsftpd log lines into structured, usable events.

02

Reconstruct sessions

Group events by source IP into sessions, using a 5-minute inactivity cutoff and connection-end signals (221 Goodbye, abrupt disconnect) as session boundaries.

03

Extract behavioral features

Quantify each session: session duration, downloads, uploads…

04

Detect anomalies (target)

Train an unsupervised model to score each session and flag deviant behavior.

05

05

Pipeline Architecture

FTP Logs

Example: /var/log/vsftpd.log

config.py

VsftpdParser → events

parsers/vsftpd_parser.py

Sessions

Grouping by IP + cutoff rules

core/session_builder.py

Features

Numeric vector per session

core/feature_extractor.py

Detector

ML score + Rules

Parsing

06

Dashboard

Flask + live alerts

HITL Feedback

Mark normal → retrain

Technical Choices

STACK & TOOLING

Python 3.10+

Native type hints (str | None), modern syntax

re (regex)

Core of the parsing logic, no heavy external dependency

pytest

Unit tests for the parser

setuptools

Packaging via pyproject.toml, installable CLI

REPOSITORY LAYOUT

src/ftp_ids/

cli.py

CLI entry point

config.py

paths & thresholds

parsers/

base.py

BaseParser (ABC)

vsftpd_parser.py

vsftpd impl.

core/

session_builder.py

→ sessions

feature_extractor.py

→ features

detector.py

Scaler + IsoForest

rule_engine.py

heuristic checks

storage.py

alerts / clean pool

dashboard/

Flask + Jinja2

tests/

test_vsftpd_parser.py

07

Parsing vsftpd Logs

RAW LOG

Thu Jun 11 15:17:40 2026 [pid 1153056]

[alice] FTP command: Client

"203.0.113.x", "USER alice"

STRUCTURED EVENT

timestamp

2026-06-11 15:17:40

pid

1153056

user

alice

event_type

FTP command

command / argument

USER / alice

src_ip

203.0.113.x

5 RECOGNIZED LINE FORMATS

CONNECT

A new client connection

DEBUG

Internal messages (e.g. abrupt disconnect)

TRANSFER (OK/FAIL)

Upload / download, size, speed

STATUS (OK/FAIL)

Result of an action (LOGIN, MKDIR…)

FTP command / response

Command sent and response code

08

Anatomy of a Regular Expression

(?P<dow>\w{3})\s+(?P<month>\w{3})\s+(?P<day>\d+)\s+(?P<time>\S+)\s+(?P<year>\d{4})\s+\[pid\s+(?P<pid>\d+)\]\s+

(?:\[(?P<user>[^\]]+)\]\s+)?(?P<event_type>FTP command|FTP response):\s+

Client\s+"(?P<raw_ip>[^"]+)"(?:,\s+"(?P<payload>[^"]*)")?

Timestamp

day, month, day-of-month, time, year — rebuilt into a datetime object

Process ID

pid — used to link lines from the same connection

User (optional)

non-capturing group ?: — absent until login has happened

Event type

command | response alternative — distinguishes request from reply

Source IP address

later stripped of the IPv4-mapped ::ffff: prefix

Payload (optional)

the quoted content, e.g. "USER alice"

09

Building Sessions

GROUPING RULES

1. Group by source IP

All events from the same IP are sorted chronologically.

2. Inactivity cutoff

A gap of more than 5 minutes between two events closes the current session.

3. Explicit cutoff

A 221 code (Goodbye) or a detected disconnect closes the session immediately.

4. User attribution

The first non-null user seen in the session is assigned to it (handles the absence of a tag before login).

"SESSION" OBJECT

src_ip

203.0.113.x

user

alice

start_time

15:17:39

end_time

15:17:51

end_type

clean / abrupt / unknown

n_events

14

events

[ list of events ]

→ natively handles the fact that vsftpd spreads a single logical session across multiple PIDs.

10

Step-by-Step Case Study

EVENT STREAM (PID 1153056 / 1153086)

15:17:39 CONNECT Client 203.0.113.x

15:17:40 FTP command USER alice

15:17:40 FTP response 331 Please specify the password.

15:17:41 FTP command PASS ****

15:17:41 OK_LOGIN Client 203.0.113.x

15:17:45 OK_DOWNLOAD report.pdf, 245 Kbyte/sec

15:17:51 FTP response 221 Goodbye.

→ RECONSTRUCTED SESSION

Duration

12 seconds

User

alice

Downloads

1 file (245 Kbyte/sec)

Session end

clean (221 Goodbye)

Time of day

daytime (3 PM)

Judged as normal behavior: no failed attempts, and both the volume and timing are consistent.

11

Feature Extraction

Brute Force

failed_logins

fail_ratio

Protocol Confusion

garbage_cmd_ratio

unique_commands

Data Transfer

total_bytes

bytes_per_file

avg_speed

transfer_ratio

Off-Hours Activity

night_events

night_ratio

Suspicious Disconnect

abrupt_disconnect

session_duration

17 features in total — computed for every session via the FeatureExtractor class before being passed to the detection model.

12

Data Preprocessing: Feature Scaling

WHY IT MATTERS

total_bytes

1 024 000 bytes

failed_logins

47 count

session_duration

38 seconds

fail_ratio

0.94 ratio 0–1

Problem: total_bytes is in the millions, fail_ratio is 0–1. Unscaled, IsolationForest's splits are dominated by the largest-magnitude features.

OBJECT 1 · ON THE FEATURES

StandardScaler()

Z = (X − μ) / σ

total_bytes: 1 024 000 → Z ≈ 1.06 (μ, σ fitted on training sessions)

OBJECT 2 · ON THE ANOMALY SCORE

MinMaxScaler(feature_range=(0,1))

S = (s − min) / (max − min)

Maps raw Isolation Forest score to [0, 1] range

Pipeline: features → StandardScaler → IsolationForest → raw score → MinMaxScaler → score ∈ [0, 1]

13

The Full ML Pipeline

≡

Session

Vector

17 raw features

per session

FeatureExtractor

⚖

Standard

Scaling

Zero mean

unit variance

StandardScaler

🌲

Isolation

Forest

n_estimators=200

contamination=0.05

IsolationForest

◎

Anomaly

Score

Score ∈ [0, 1]

MinMax normalized

score_scaler

⚡

Rule

Engine

Signature rules

Parallel path

RuleEngine

NEW

KEY PARAMETERS

n_estimators=200 — 200 isolation trees in the forest

max_samples='auto' — Subsample size per tree (min(256, n_samples))

contamination=0.05 — Expected 5% anomalies in training data

random_state=42 — Reproducibility seed

★ RuleEngine runs in parallel: 5 signature-based rules (CVE-2011-2523 backdoor, FTP bounce, port scan, brute force, anonymous abuse)

14

⚠

Alert

Flag session

for review

predict()

Isolation Forest — Deep Dive

PATH LENGTH INTUITION

Normal (dense cluster)

→ depth = 8+ cuts

Anomaly

→ depth = 2 cuts

ANOMALY SCORE FORMULA

s(x) = 2^[ −E(h(x)) / c(n) ]

E(h(x)) = avg path length across all trees c(n) = expected path length for n samples

SCORE INTERPRETATION

s ≈ 1.0

Strong anomaly — very short path

s ≈ 0.5

Ambiguous — needs context

s ≈ 0.0

Clearly normal — deep path

Decision: if predict(x) == -1 → flag as anomaly

(set by contamination parameter at fit time)

TRAIN vs INFER

TRAIN

Fit scaler + IsolationForest on normal session vectors. The scaler memorizes X_min and X_max per feature.

INFER

Scale the new session with the saved scaler, then call decision_function() to get the anomaly score in real time.

15

Rule Engine

VSFTPD_234_BACKDOOR

Detects CVE-2011-2523 exploit (USER with :) suffix). FTP_BOUNCE_MGLNDD: Detects MGLNDD scanner probes.

PORT_SCAN / BRUTE_FORCE

PORT_SCAN: Detects connections with no FTP commands. BRUTE_FORCE: Detects 5+ failed login attempts in one session.

ANONYMOUS_ABUSE

Detects anonymous login with write commands (STOR, DELE, MKD).

Why Both ML and Rules?

ML catches unknown attacks (anything anomalous). Rules catch known attacks that ML might score low (simple probes with few features to analyze).

16

Dataset Composition

109

sessions total

98

normal (90%)

11

attack (10%)

NORMAL SESSIONS

59 generated deploy sessions (AI-augmented from real deploy template)

39 real deploy sessions (from production CI/CD pipeline)

File sizes, transfer speeds, and timing match real behavior.

ANONYMIZATION

Personal identifiers replaced before publication. Attacker IPs kept (public hostile actors).

ATTACK SESSIONS (REAL)

TLS / protocol confusion floods

MGLNDD scanner probes

Anonymous brute-force attempts

Banner grabs and port scans

HTTP-on-FTP probes

17

Evaluation Results

MODEL PERFORMANCE

Dataset: 109 sessions (98 normal, 11 attack)

Training: 109 sessions (contamination=0.05) | Threshold: 0.6

CLASSIFICATION REPORT

precision

recall

F1-score

normal

0.99

0.95

0.97

attack

0.67

0.91

0.77

accuracy

0.94

KEY FINDINGS

Attack recall = 0.91: catches 10 out of 11 attacks.

The model rarely misses real threats.

Normal F1 = 0.97: almost all deploys correctly identified.

Very few legitimate sessions are disrupted.

Attack precision = 0.67: some false positives.

Some manual FTP sessions flagged as attacks. Resolved via the HITL feedback loop.

18

Flask Dashboard

19

Flask Dashboard

Flask • Chart.js • Tailwind CSS

Overview

Events today, alerts today, pending review, unique IPs. Alerts-per-hour chart.

Alerts

Paginated table (50/page). Score, source IP, user, rules fired. Click any row to see full session details and all 17 features displayed.

Live Monitoring

Real-time monitoring with toast notifications. Browser notifications for new alerts.

Apply / Retrain

One-click retrain from the dashboard. CLI output shown in collapsible panel. Mark alert as normal button.

20

Demo & CLI Interface

$ ftp-ids train --log vsftpd.log

Model trained and saved to data/models/iso_forest.pkl

SRC_IP USER END EVENTS

203.0.113.10 alice clean 14

198.51.100.23 - abrupt 6

203.0.113.55 bob clean 22

$ ftp-ids parse --log vsftpd.log

parsed : 142

failed : 0

AVAILABLE COMMANDS

21

Tests & Quality

test_connect_line

Verifies correct extraction of a CONNECT event (IP, PID, timestamp).

test_command_with_argument

Verifies the command / argument split (e.g. USER alice).

test_goodbye_sets_session_end

Verifies that a 221 code correctly marks a clean session end.

test_terminated_sets_session_end

Verifies detection of an abrupt disconnect (no SSL shutdown).

test_garbage_line_returns_none

Verifies robustness against unrecognized or empty lines.

22

Limitations & Future Work

CURRENT LIMITATION

FUTURE WORK

Only vsftpd supported

ProFTPd, Pure-FTPd, and IIS need their own parser.

Support other FTP servers

The BaseParser contract is ready. Adding a new server means writing one parser class.

Small dataset (109 sessions)

Enough for proof of concept, not for production.

Publish dataset on Kaggle

Share the labeled sessions publicly and collect more production data.

Attack precision = 0.67

Some manual FTP sessions flagged as false positives.

More data + HITL feedback

More training data and continuous user corrections will improve precision over time.

Limited dashboard analytics

Current dashboard shows alerts and basic stats.

Richer insights

Geo-IP mapping, attack types chart, AI trust score based on HITL corrections.

23

The Project, by the Numbers

17

extracted

features

5

unit

tests

5

signature

rules

7

7 CLI commands

109

109 training sessions

WHAT'S DEMONSTRATED TODAY

✔ End-to-end pipeline: log parsing → sessions → 17 features → ML detection

✔ Isolation Forest (n=200, contamination=0.05) + MinMax scoring [0, 1]

✔ 5 signature rules running in parallel

✔ Flask dashboard with live alerts, manual review, retrain

✔ HITL feedback loop: mark normal → retrain → improved detection

✔ Evaluation: 0.94 accuracy, 0.97 normal F1, 0.91 attack recall

24

Conclusion

A complete detection pipeline: parsing → sessions → features → ML scoring,built and tested on real vsftpd production logs.

A hybrid approach combining unsupervised ML (Isolation Forest) with 5 signature rules to detect both known and unknown attacks.

A human-in-the-loop feedback loop that lets the user correct falsepositives and improve the model over time. Evaluated on 109 sessions: 0.94 accuracy, 0.91 attack recall.

Thank you for your attention