1 of 34

I2 Cloud Community Chatbot

Timothy Manik, Internet2

Date: May 21, 2026

2 of 34

Agenda

Background
Architecture
Demo
What’s next?
Q&A

2

3 of 34

4 of 34

In the beginning…

5 of 34

Background

Early 2024: We want a chatbot. Who can help us build?
Summer 2024: Jan connects us with the CalPoly CIC Team
Fall 2024: They're booked solid. So we wait…
Early 2025: A slot opens!
Mid 2025: Iterations begin
Fall 2025: Prototype is done!

5

6 of 34

7 of 34

8 of 34

9 of 34

10 of 34

11 of 34

12 of 34

13 of 34

14 of 34

15 of 34

Architecture - Web App

15

1. User loads the app

CloudFront serves the static React frontend cached from S3

2. User signs in

Cognito handles authentication (and issues tokens the frontend attaches to API calls)

Cognito User Pool, Lite tier (10K MAU free), self-signup disabled — users only enter via SAML.

Designed to federate to the institution's existing IdP — relevant for Internet2/InCommon.

3. User sends a chat message

The request hits API Gateway, which routes it to Lambda — our serverless backend, so we pay per request and scale automatically.

4. Lambda orchestrates the RAG pipeline

Queries OpenSearch Vector Store to retrieve relevant context (semantic search over embedded documents)

Calls Bedrock to generate the answer using that retrieved context

Reads/writes DynamoDB for chat history and session state

More specifically, embedding → vector search → classifier → (filter) → chat. That's 3–4 Bedrock round-trips serialized per user request. The chat handler timeout is 60 seconds for a reason.

Embedding: per-request, but embedding models are dirt cheap

Classifier: tiny prompt, 100-token cap → pennies, use a Haiku-tier model

Filter: medium prompt, 1000-token cap, only sometimes invoked → cheap

Chat: full context window with retrieved docs → dominates the bill, justifies a Sonnet/Opus-tier model

5. Response flows back through API Gateway to the user.

16 of 34

Architecture - Data Pipeline

16

1. Sources → landing zone

Confluence and Google Drive sync into a single S3 bucket. S3 as the entry point decouples sources from processing — adding SharePoint or a GitHub wiki later means a new sync job, nothing downstream changes.

2. Orchestration — Step Functions

The pink box is one Step Functions state machine: durable execution, per-step IAM, automatic retries, visual debugging. A Map state with bounded concurrency fans out the batch without overwhelming downstream services.

3. Lambda Router — fan-out by file type

Inspects each S3 object and routes by extension: pdf → PDF, mp4 → video, m4a → audio, txt/vtt → text. Also checks a DynamoDB idempotency table (PK = s3_uri) to skip already-ingested files — the pipeline is safe to re-run.

4. Four branches — right compute for the workload

Branch Compute Why

Text Lambda, 128 MB / 1 min Tiny work, cheapest option

PDF Container Lambda, 2 GB / 15 min Needs Textract SDK + native deps, bursty

Audio Fargate, 2 GB / 1 vCPU Exceeds Lambda's 15-min ceiling

Video Fargate, 16 GB / 8 vCPU Heavy CPU + memory, Lambda can't do it

Fargate over EC2: same container model, no cluster to patch, pay per task-second.

5. AWS-native ML for extraction

Textract (PDF): OCR plus table/form extraction — beats open-source on scanned docs.

Transcribe (audio/video): speech-to-text with speaker diarization (up to 5 speakers). Step Functions polls the async job with a Wait state until done.

Zero infra, pay-per-use, accuracy improvements roll in for free.

6. Common terminus — Bedrock → OpenSearch

Every branch converges: chunk the extracted text, call a Bedrock embedding model (Titan), write vectors to OpenSearch Serverless (VECTORSEARCH collection — purpose-built for k-NN). Chunk size and overlap are env-var configurable. OpenSearch Serverless = no shards, no instance sizing, scales automatically.

17 of 34

Architecture - Data Pipeline (Video)

17

Transcribe
Detect Scenes
Transcript-Scene Alignment
OCR Text Detection

Pre-filtering with Tesseract
AWS Textract over textual scenes
Cut Textract costs by ~90%

Large text chunk

Small text chunk

0s

30s

Small text chunk

5s

10s

15s

23s

Content is here, timestamp is 18s early

Timestamp

Chunking

Split long dialogues at speaker change for retrieval
Chunk length 400 - 800 chars

Index chunks into OpenSearch

Timestamp at start of chunk

17

18 of 34

Demo

19 of 34

What’s next?

20 of 34

Links

Website

https://abe.internet2.edu

GitHub Repo

https://github.com/Internet2/i2ccc

20

21 of 34

Thank you!

22 of 34

Q&A

23 of 34

23

24 of 34

24

25 of 34

25

26 of 34

26

27 of 34

27

28 of 34

28

29 of 34

29

30 of 34

30

31 of 34

31

32 of 34

32

33 of 34

33

34 of 34

34