DETECTING
DIGITAL
DECEPTION
A comparative study of rule-based and machine learning
approaches for online misinformation detection
Zandile Dladla 4120815
BSc Honours Computer Science
Supervisor: Mr Mogammad Allie
RULES
compared with
MACHINE
LEARNING
The problem and project aim
A lightweight comparison of language-based detection methods
THE PROBLEM
Online misinformation spreads quickly.
Manual verification struggles to keep up.
PROJECT AIM
Build a working prototype and compare weighted
linguistic rules with TF-IDF Logistic Regression.
Lightweight baseline study. BERT, RoBERTa and LLMs provide context for future work.
What the application must do
Requirements informed the interface and functional tests
01
ENTER
Paste an English statement
02
ANALYSE
Submit it for both methods
03
COMPARE
Read labels and rule indicators
04
UPDATE
Replace the text and repeat
Unusable input receives a clear validation message.
The data behind the comparison
LIAR contains fact-checked US political statements from PolitiFact
10,240
TRAINING
Fit vocabulary and model
1,284
VALIDATION
Compare configurations
1,267
TEST
Final evaluation only
Truthful: true and mostly-true
Misleading: half-true, barely-true, false and pants-fire
Half-true retains misleading content under this conservative binary mapping.
Test set: 818 Misleading and 449 Truthful. Fit TF-IDF on training data only.
One statement, two analysis paths
Procedural Python functions connect the interface and both methods
Original
statement
Clean text
TF-IDF features
Weighted phrase rules
Original text
Logistic Regression
Display both outputs
Final label follows ML
The rule score describes matched indicators. It does not override ML.
Implementation
Python in Colab with NLTK, scikit-learn and Gradio
MACHINE LEARNING
Clean and tokenise text
Remove stop words
Build 11,464 TF-IDF features
Apply Logistic Regression
TF-IDF weights words by their importance
within a statement and the training corpus.
WEIGHTED LINGUISTIC RULES
Search the original text with regex
Conspiracy and deceptive claims: 2 each
Sensational, absolute, certainty: 1 each
Score of 2 or more: Misleading
“conspiracy” + “shocking” = 3 / 7
Rule score: 42.86% of maximum weight
The rule score measures matched indicators. It is not a probability of truth.
Model selection and re-evaluation
LR = Logistic Regression. These are validation results, not test results.
Validation model | Accuracy | Balanced accuracy | Macro F1 |
Original unigram LR | 68.15% | 56.63% | 0.5578 |
Balanced unigram + bigram LR | 65.42% | 63.11% | 0.6236 |
Macro F1 gives both classes equal weight.
The balanced model leads this class-sensitive comparison.
The application and reported test results retain the original model.
The re-evaluation qualifies the earlier accuracy-based choice.
The working application
Recorded interface output, redrawn for readability
EXAMPLE STATEMENT
Climate change is a fake conspiracy
created to manipulate the global economy.
Final classification: Misleading
ML confidence: 66.82%
Rule score: 28.57%
Detected indicator: Conspiracy Language
Testing and refinement
Input validation, repeatable predictions and sequential response time
INPUT VALIDATION
Before: 1234567890
Misleading, 66.82% model confidence
After: rejected as unusable input
Retests reject empty, whitespace, punctuation,
number-only and URL-only inputs.
PROCESSING TIME
3.0 ms
Mean across 100 sequential calls
Maximum: 12.7 ms
All completed below 3 seconds
Application time excludes browser and network delays. No concurrent-load or security audit.
Final test results
1,267 held-out statements, with no further model tuning
4
additional correct predictions over the majority baseline
A gain of 0.32 percentage points in test accuracy
The model misses many truthful statements
Recall measures the share of each actual class detected correctly
91 / 449
truthful statements
correctly classified
by ML
ML balanced accuracy: 54.82% Macro F1: 0.5285
Rules detected only 6 of 818 misleading statements.
Representative errors
Historical LIAR labels compared with predictions from the original model
FALSE POSITIVE
“Building a wall on the U.S.-Mexico
border will take literally years.”
LIAR: Truthful
Model: Misleading
The duration claim needs external evidence.
Border vocabulary cannot establish veracity.
FALSE NEGATIVE
Claim: no opposing-party Senate has
confirmed a Supreme Court nominee
since 1888. (Paraphrased)
LIAR: Misleading (false)
Model: Truthful
Preprocessing removes the year.
The historical assertion needs evidence.
Examples from report Section 12.7. Explanations are qualitative, not causal feature attributions.
Limitations and responsible use
Training on US political claims limits everyday use
“I love my sister.”
Model output: Misleading with 71.24% model confidence
This personal statement lies outside the training domain.
The system does not retrieve evidence or verify facts.
Automated labels can mislead users. Human verification remains necessary.
Avoid confidential inputs. Security and scalability remain unevaluated.
Conclusion and future work
A complete comparative prototype with documented limitations
ML outperformed the selected rules
The small gain over baseline and low truthful recall
limit practical reliability.
Next steps: class-sensitive model selection, preserve negation and quantities,
and evaluate evidence retrieval on a new evaluation set.
sites.google.com/myuwc.ac.za/misinformation-detection/home
Thank you