Subtitle

FLORIDA INTERNATIONAL UNIVERSITY

AI-Powered Document Analysis & Fraud Detection

Loan Validation Platform

Students: Carlos Hernandez, Christoph McPhee, Luis Jimenez, Ernesto Blanco, Miguel Restrepo.

Instructor: Masoud Sadjadi

Faculty: Florida International University-College of Engineering and Computing

Subtitle

FLORIDA INTERNATIONAL UNIVERSITY

The Loan Validation System automates document verification using OCR, AI, and fraud-detection logic to help lenders validate applicant information.

The purpose of the Loan Validation System is to build an intelligent, automated platform that helps lenders verify applicant information, detect fraudulent documents, and assess risk using OCR, AI models, and cross-document consistency checks.Our system extracts structured fields from uploaded paystubs, W-2s, bank statements, and application forms, validates them against a ground-truth dataset, and flags discrepancies that could indicate fraud or errors. Lastly, our system provides the probability of loan default so lender can make an assessment of Value at Risk (VaR)

​

Purpose

Subtitle

FLORIDA INTERNATIONAL UNIVERSITY

• Frontend: (User Dashboard, Document Viewer, Fraud and Consistency Summary.
• Backend: (OCR pipeline (Google Vision API + Gemini), Fraud heuristics & cross-validation rules, Ground-truth verification API)
• AI/OCR Layer: (Extract text from PDF/Image, Parse fields, Detect Inconsistencies)
• Loan Default Model (XGBoost)
• Storage: AWS S3 , GCS Bucket

System Architecture Overview

Subtitle

FLORIDA INTERNATIONAL UNIVERSITY

• Frontend: React, TS, CSS
• Backend: Python/Flask
• AI/OCR: Google Vision API, Gemini, pandas, scikit-learn, XGBoost, manTraNet
• Storage: S3 Bucket, GCS Bucket
• Security: IAM, Secret Manager

​

Technology Stack

Subtitle

FLORIDA INTERNATIONAL UNIVERSITY

• Accept uploads → ML Pipeline → JSON response back to frontend.

​

​

​

Backend Processing Logic

FLORIDA INTERNATIONAL UNIVERSITY

• OCR for paystubs, W-2s, statements
• Fraud consistency checks
• Loan Default probability
• Risk Assessment
• Ground-truth dataset

ML Pipeline Features

FLORIDA INTERNATIONAL UNIVERSITY

• OCR for application document (e.g. pay stubs, W-2s, bank statements)
• Extract features from text used by loan officers and risk model
• Cross validate fields extracted across documents

​

OCR & Cross Validation(ML)

FLORIDA INTERNATIONAL UNIVERSITY

• Image Manipulation Analysis: Detects pixel-level tampering using deep-learning anomaly tracing.
• Page-Level Suspicion Scoring: Generates a numeric fraud likelihood score for each document based on visual inconsistency patterns.
• Tamper Localization Heatmaps: Highlights specific regions where manipulation is suspected, enabling targeted human review.
• Low-Data, High-Impact Baseline: Works effectively without labeled fake samples by relying on universal manipulation artifacts rather than predefined templates.

Fraud Detection(ML)

FLORIDA INTERNATIONAL UNIVERSITY

• Leverage information extracted from application documents (OCR)
• Loan Default Model offering probability of default
• Provide Value at Risk (VaR) give loan amount and probability of default

Quantifying Risk(ML)

FLORIDA INTERNATIONAL UNIVERSITY

• Training data obtained in Kaggle
• Features include:
• Marital status
• Applicant Age
• Income
• House/Car ownership
• Years of employment
• Applicant occupation
• Cross validation was used to derive model parameters, reduce bias and maximize variance

Model Results

• 3 models were explored:
• Logistic Regression
• Random Forest
• XGBoost Classifier
• XGBoost and Random Forest performed better, supporting the hypothesis that the relationship between features and predicted label is not linear

Subtitle

FLORIDA INTERNATIONAL UNIVERSITY

• Dashboard shows applicant info, document previews, fraud summary, extracted fields.

Platform Dashboard (Placeholder Image)

Subtitle

FLORIDA INTERNATIONAL UNIVERSITY

• OCR variability: Foreign bank statements format, inconsistent PDF quality.
• Dataset creation: manually building dataset from scratch.
• Frontend-backend syncing: Complex coordination between React dashboard + OCR backend.
• Model Variability: Gemini/Vision API requires careful prompt engineering
• Document diversity: Paystubs, W-2s,1099s, bank statements (All require different parsing logic.)

Challenges

FLORIDA INTERNATIONAL UNIVERSITY

• Custom parsers
• Fraud heuristics
• Ground-truth dataset
• Improved validation rules.
• ManTraNet Model

Solutions

Subtitle

FLORIDA INTERNATIONAL UNIVERSITY

• Use document embeddings for similarity detection.
• Add real-time fraud alert on dashboard.
• Build applicant historical timeline.
• Mobile-friendly loan officer version.
• Add encryption and user-role access control.

Improvement & Future Work

Subtitle

FLORIDA INTERNATIONAL UNIVERSITY

Our Loan Validation platform automates a traditionally manual and error-prone process by combining OCR, AI, and structured fraud-detection logic. The system reduces human workload, improves loan decision accuracy, and provides a scalable foundation for future ML-based risk scoring.

​

Skills Strengthened:

• Frontend development
• Backend model design
• AI/OCR Integration
• Storage
• Collaborative development and Agile work.

​

Conclusion