HONOURS RESEARCH PROJECT
Anomaly Detection for Early and Adaptive Fraud Detection in Mobile Money Transactions
Litha Mkonwana
4341225@myuwc.ac.za
Andrew Muller
andrewstevenmuller@gmail.com
Co - Supervisor
Dr Andre Henney
ahenney@uwc.ac.za
Supervisor
Mobile Money: Background
📱
Financial inclusion through mobile technology
Access requires only a SIM card and a phone number, with no bank branch or paperwork necessary.
Mobile Money as a Fraud Target
SIM swap fraud
An attacker hijacks the victim's SIM to intercept one-time passwords and gain access to their wallet.
Social engineering
Phishing messages or fraudulent agents deceive users into disclosing sensitive information.
Account takeover
Compromised credentials grant an attacker full control of the account.
Agent fraud
A mobile money agent exploits their position to misappropriate customer funds.
0.13%
of transactions in this study's dataset were fraudulent
This nonetheless corresponds to 1,840 fraud cases
within 67,953 test transactions.
This pronounced class imbalance is the central challenge this project addresses.
Anomaly Detection: A Conceptual Overview
Rather than training a model to recognise fraud directly, anomaly detection trains a model to characterise normal behaviour and flags deviations from that baseline.
Normal transactions
← The anomaly
Traditional (supervised) approach
Requires substantial labelled fraud data and struggles to adapt when fraud patterns evolve or novel schemes emerge.
Anomaly detection (this project)
Learns normal activity patterns from largely unlabelled data and flags deviations, enabling detection of previously unseen fraud patterns.
Methodology
Dataset: PaySim
A synthetic mobile money simulator modelled on a real African provider, featuring approximately 6.3 million transactions over 30 simulated days. Ground-truth fraud labels were used solely for evaluation.
19 engineered features across four categories
Transactional
amounts, balances
Temporal
time-of-day, velocity
Behavioural
balance probes, micro-tests
Network
sender/receiver connections
Five unsupervised and semi-supervised models compared
Gaussian Mixture Model (GMM)
Isolation Forest
Local Outlier Factor (LOF)
Autoencoder (deep learning)
One-Class SVM
A supervised Logistic Regression model is reported separately as a reference benchmark only, as it relies directly on fraud labels.
Research Hypotheses
Primary Research Question
To what extent can unsupervised and semi-supervised anomaly detection identify fraud earlier and more adaptively than conventional approaches, given evolving fraud patterns (concept drift) and highly imbalanced data?
Three hypotheses were tested against the Term 3 report findings
H1 — Supported
Autoencoder outperformed GMM on F1 (0.724 vs 0.674) and AUPRC (0.710 vs 0.638).
H2 — Supported
The 19-feature GMM improved F1 by 30.7% over a 5-feature raw baseline, exceeding the 20% threshold.
H3 — Not Supported
No drift was detected in PaySim’s synthetic data; the static Isolation Forest marginally outperformed the adaptive model.
Evidence at a glance
Autoencoder F1 0.724 vs GMM 0.674
+30.7% F1 with full 19-feature set
ADWIN drift detector: not triggered
H1 and H2 were supported; H3 was not supported under current ADWIN sensitivity settings, motivating recalibration and validation on real transaction data.
Comparative Model Performance
The Autoencoder achieved the best overall performance, with an F1-score of 0.724, an AUPRC of 0.710, and a precision of 0.969, indicating a low false-positive rate. The Gaussian Mixture Model (GMM) also performed well, achieving an F1-score of 0.674 and serving as a strong baseline. In contrast, the Local Outlier Factor (LOF) struggled in the high-dimensional feature space. A supervised Logistic Regression reference (not shown) reached recall 0.970 and AUPRC 0.895 but only precision 0.146, illustrating the cost of the label-free constraint used throughout this study.
Concept Drift Experiment
Recall declined 20.4% across test windows for the static Isolation Forest, from 0.357 in Window 1 to 0.284 in Window 3, indicating a genuine drift signal. However, the custom ADWIN-inspired drift detector (delta threshold 0.002, minimum window 30) never triggered, so the adaptive model never retrained and could not demonstrate its intended advantage. This is attributed to insufficient sensitivity calibration for PaySim's synthetic environment rather than a failure of the adaptive approach, and Hypothesis 3 was therefore not supported under current settings.
Limitations & Caveats
Balance artefact confound
Fraudulent transactions show the origin balance dropping to exactly zero; engineered features may partly detect this synthetic artefact rather than generalised fraud behaviour.
Synthetic dataset
PaySim's fraud patterns do not evolve organically, limiting the concept drift experiment's ecological validity.
Narrow transaction scope
Fraud in PaySim occurs only in TRANSFER and CASH_OUT transactions, restricting generalisability to PAYMENT and DEBIT activity.
Computational constraints
LOF and One-Class SVM could not complete training on a personal laptop; Autoencoder results also differed between laptop and server environments.
0.970
recall achieved by the supervised reference (Logistic Regression) baseline
Its precision was only 0.146 (F1 0.253)
illustrating the cost of the label-free constraint.
Included only as context; not ranked alongside the unsupervised models.
Key Findings
1
Autoencoder achieves optimal performance
Highest F1-score (0.724) and AUPRC (0.710), with a low false-positive rate.
2
Feature engineering improves detection
Adding behavioural and network features yielded a 30.7% improvement in F1-score over raw transactional data alone.
3
Concept drift requires real-world validation
The drift detector did not trigger on this synthetic dataset; validation on real, evolving fraud data is a clear direction for future work.
The results demonstrate that effective fraud detection is possible without labelled fraud data, provided the model can accurately learn and detect deviations from normal behaviour.
Thank you