Saving a Million with AI

How UMBC Built Its Own �Enterprise E-Discovery Solution��Jason Paluck�Division of Information Technology

DOJ Settlement

• UMBC Title IX Settlement
• Emails & documents from January 1, 2015
• Keyword search with wildcards
• “Good faith effort” to ensure all Title IX matters are surfaced
• READ: Not an evidentiary matter!
• Firm deadline

​

​

The Options

The Solution: AWS AI Workflow

• Event-driven AWS architecture
• Native AWS services
• Anthropic AI models via AWS Bedrock API
• Protected data
• Automated, scalable, cost-effective.

Finding Needles in a Haystack

From: Jill Smith <jsmith@umbc.edu>

Subject: Sorry to bother you

Date: July 28, 2024 at 9:20:06 AM EDT

To: Jason Paluck <paluck@umbc.edu>

Hi Jason,

I’m really sorry to bother you, I know how busy you are. I don’t want you to think I’m complaining or trying to get anyone in trouble, but I feel like I just need to vent to someone. I’ve been feeling more and more uncomfortable around Jim Jones, my immediate supervisor. He asked me out a few times, and while I first thought it was just to work on our project off campus, it has become obvious he wants more. I’ve told him I’m not interested, but it doesn’t stop. Yesterday he came up behind me at my desk and started rubbing my shoulders.

Thanks,

Jill

Finding Needles in a Haystack

• AI Prompt
• “You are an expert civil rights advocate who specializes in Title IX of the federal Education Amendments of 1972….”
• Response (JSON payload)
• “This email describes behavior that could constitute harassment, creating a hostile work environment, and involves power dynamics due to the supervisor-subordinate relationship.”
• “I recommend further reviewing this email as potentially related to a Title IX report.”

​

Finding Needles in a Haystack

New Incidents

AWS LEGOLAND

Herding Cats

AWS Step Functions

• Visual workflow orchestration
• Controls 200+ AWS services
• Built-in error handling
• Event-driven execution

AWS EventBridge

• Real-time event routing
• Rule-based filtering
• Trigger Lambda/SNS/SQS
• Starts Step Functions workflows

AWS Discovery Workflow

Data Upload & Workflow Trigger

• Keyword search
• Documents uploaded to S3
• Simple drag & drop UI
• PDF, Word, hand-written scanned images, zip archives
• Google Vault (zipped Mboxes of emails and documents)
• Cloud-based file storage via API (Box)
• EventBridge triggers Lambda
• On-demand workflow automation

Iterative Routing and Pre-processing

• AWS Lambda
• Identify file types
• Routes workflow accordingly
• Invokes Step Functions
• AWS Glue (Apache Spark)
• Unzips as necessary
• Extracts EML files from Mboxes
• Extracts text from emails and documents back to S3
• Process Repeats

Bedrock AI Semantic Analysis

• Bedrock API calls AI model with protected data
• Executes custom prompts against document
• Initial screening by Claude Haiku
• High volume, but inexpensive
• Multi-model consensus by Claude Sonnet
• Low volume, but expensive
• Results written to DynamoDB

Bedrock AI Semantic Matching

• Compares documents to incident database (RAG)
• Identifies matches with known incidents
• Adds decision rationale as metadata
• Links to historical incident records

Output & Review

• Results stored in S3 & DynamoDB
• Unmatched to known incidents
• Multi-model consensus of Title IX
• Amplify enables human review
• React web application
• Built-in authentication
• Triage incidents via web browser

Browser-based Human Review

Impact

• < $2,000 in direct AWS costs
• Model for future AI initiatives at UMBC

Questions

​

Jason Paluck

paluck@umbc.edu