1 of 47

2 of 47

3 of 47

4 of 47

5 of 47

Documentation

Soul of Software Development

6 of 47

Doc-Monitor

An Open Source MCP by Akash Sathish

7 of 47

Self-updating knowledge bases for smarter AI agents

A Presentation by Akash Sathish

8 of 47

9 of 47

10 of 47

11 of 47

12 of 47

13 of 47

Let’s Build an MCP

14 of 47

15 of 47

16 of 47

17 of 47

Shortcomings of Context7

  • Not Fully Open Sourced
  • Fully Community Dependant on getting the latest updates of documentation
  • Documentation Format Limitations
  • No Change Detection

18 of 47

Let’s Build an MCP

19 of 47

20 of 47

How does Doc-Monitor work ?

21 of 47

22 of 47

23 of 47

24 of 47

Architecture Trade Offs: Vector Database Decision Matrix

​

 

 

Performance Benchmarks (Based on VectorDBBench 2024)

Metric

PgVector ✅

FAISS

Elasticsearch

QPS (Queries/sec)

2,500-4,000

8,000-15,000

1,500-3,500

Latency (p99)

15-25ms

5-12ms

25-45ms

Memory Usage

40% lower

Highest

60% higher

Setup Complexity

Low

High

Medium

Operational Cost

$200-400/month

$800-1200/month

$600-1000/month

 

25 of 47

Architecture Trade Offs: Embedding Model Decision Matrix

 

Model

Dimensions

Accuracy (MTEB)

Cost/1M tokens

Latency

Infrastructure

text-embedding-3-small ✅

1536

0.726

$20

50-80ms

Zero

all-mpnet-base-v2

768

0.694

$0

45ms

Self-hosted

text-embedding-3-large

3072

0.746

$130

100-150ms

Zero

all-MiniLM-L6-v2

384

0.637

$0

15ms

Self-hosted

 

26 of 47

Architecture Trade Offs: Embedding Model Decision Matrix

Factor

OpenAI text-embedding-3-small

all-mpnet-base-v2

Winner

Accuracy (MTEB)

0.726

0.694

OpenAI (+4.6%)

Setup Time

5 minutes (API key)

2-3 hours (infrastructure)

OpenAI

Monthly Cost (50K docs)

$15-25

$200-400 (server costs)

OpenAI

Latency

67ms (API + network)

45ms (local inference)

mpnet (+33%)

Maintenance

Zero

High (model updates, scaling)

OpenAI

Reliability

99.97% uptime (SLA)

Self-managed

OpenAI

 

27 of 47

Architecture Trade Offs: Embedding Model Decision Matrix

Factor

OpenAI text-embedding-3-small

all-mpnet-base-v2

Winner

Accuracy (MTEB)

0.726

0.694

OpenAI (+4.6%)

Setup Time

5 minutes (API key)

2-3 hours (infrastructure)

OpenAI

Monthly Cost (50K docs)

$15-25

$200-400 (server costs)

OpenAI

Latency

67ms (API + network)

45ms (local inference)

mpnet (+33%)

Maintenance

Zero

High (model updates, scaling)

OpenAI

Reliability

99.97% uptime (SLA)

Self-managed

OpenAI

 

28 of 47

Problems Faced

 

29 of 47

Problems Faced ft. Smart Crawling

 

30 of 47

Smart Crawling

 

31 of 47

Smart Crawling

 

32 of 47

Smart Crawling

 

33 of 47

Smart Crawling

 

34 of 47

Smart Crawling

 

35 of 47

Smart Crawling

 

36 of 47

Smart Crawling

 

37 of 47

Smart Crawling

 

38 of 47

Problems Faced ft. Documentation Processing

 

39 of 47

Documentation Processing

 

40 of 47

Documentation Processing

 

41 of 47

Documentation Processing

 

42 of 47

Documentation Processing

 

43 of 47

Documentation Processing

 

44 of 47

Documentation Processing

 

45 of 47

Documentation Processing

 

46 of 47

Documentation Processing

 

47 of 47

RoadMap Ahead

  • Automating Developer Setup
  • Inbuilt job scheduler for checking document changes regularly
  • Improving Change Detection Strategy
  • Adding options for embedding models

: