HACKSJSU · EDGE AI
NanoGate
Every AI request takes the cheapest safe route with proof.
An OpenAI-compatible AI control plane that runs 100% on the HP ZGX Nano (NVIDIA GB10).
x-nanogate-receipt
route: local
data_class: Restricted
reason: SENSITIVE_LOCAL_ONLY
remote_bytes: 0
seal: HMAC-SHA256 · verified
Team OK200 · Dev Faldu · Vraj Patel · Rahil Shah · Ranadhir Reddy · Yaksh Dalwadi
NanoGate · HackSJSU Edge AI 2026
1 · TEAM & INTRODUCTIONS
Team OK200: five builders, one ZGX Nano
Dev Faldu
Data & model training
Found datasets, trained and fine-tuned models
Vraj Patel
Integration & evaluation
Wired modules together, ran Nano benchmarks
Rahil Shah
Model runtime
Served both Qwen tiers on vLLM
Ranadhir
Cache & policy
Built verified cache and policy engine
Yaksh Dalwadi
Frontend dashboard
Built React dashboard and receipt views
Built in four days on the event's HP ZGX Nano: ~7,200 lines of Python gateway and a ~4,500-line React dashboard.
2 · PROBLEM STATEMENT
Every AI request makes a trade-off nobody can prove
Privacy
An HR prompt with an employee's SSN goes to a cloud API. Who approved that?
Cost
"How do I reset my VPN?" burns large-model tokens it never needed.
Audit
When the CISO asks what happened to a request, there's no provable answer.
Privacy, quality, cost and auditability are traded off implicitly on every request, by default settings nobody chose.
NanoGate · HackSJSU Edge AI 2026
2 · PROBLEM — WHY TODAY'S TOOLS FAIL
Today's tools forward requests. None decide and prove.
What a team needs | Cloud AI API | Provider proxy | Cosine cache | NanoGate |
Sensitive data stays on-site | No — leaves the building | Only if you pick the model | Not its job | Yes — policy removes remote routes |
Per-department policy | No | Routing, not data-aware | No | Data class × department |
Safe answer reuse | No | No | Similarity only — 9.3% false hits | NLI-verified — 2.3% false hits |
Provable audit trail | Provider's logs | Request logs | No | Hash-chained, sealed receipts |
False-hit rates measured on our authored cache set (n = 80), run 2026-09-23 · NanoGate
3 · ARCHITECTURE
One box. Every model. Every decision.
100% on-device inference
Your apps
OpenAI SDK, employee chat, Assistant
HP ZGX Nano · NVIDIA GB10 Grace Blackwell · 128 GB unified memory
NanoGate gateway · FastAPI · 15-stage pipeline
1 Auth
2 DLP scan
3 Policy
4 Budget
5 Cache
6 Local LLM
7 Risk
8 Route
9 Out DLP
10 Receipt
vLLM on the GB10 (HP Z Runtime)
Qwen 2.5 3B (default) · Qwen 2.5 14B (escalation) · BF16
On-device ML + data
bge-small · NLI verifier · Presidio/spaCy · company knowledge · SQLite
Remote tier
Only with an egress ticket
Confidential+ data never mints an egress ticket · remote tier is simulated by the local 14B in the demo · NanoGate
3 · ARCHITECTURE — TECHNICAL DEPTH
Three on-device models make each call
Calibrated risk router
Logistic regression + isotonic calibration, retrained on 1,791 graded local answers. Predicts p(local answer is wrong).
Verified cache
bge-small nominates; a DeBERTa NLI cross-encoder plus slot checks (owner, scope, polarity, CVE ids) must agree.
Layered DLP
Secret patterns → regex + Luhn/SSN checks → de-obfuscation → Presidio + spaCy NER. Fails closed.
Hard problems we solved on the Nano
Swapped Ollama for vLLM on HP Z Runtime → both tiers in BF16
GPU memory pressure made every request abstain → router retrained without it
No root, no Docker → fully user-space setup
Power-offs mid-run → runs with a dead model marked INVALID
~7,200 lines Python gateway · ~4,500 lines React/TypeScript dashboard · NanoGate
4 · SOLUTION
How NanoGate answers each problem
Privacy
DLP classifies every request; policy strips remote routes for Confidential+ data. Answered on the GB10.
Cost
Verified cache first, then the 3B model. The risk router escalates to 14B only when needed.
Audit
Every decision, answered or denied, is sealed in a hash-chained, HMAC-signed receipt.
Ships with:
2-line OpenAI SDK swap
Employee chat
NanoGate Assistant
Company knowledge
Slack/Teams alerts
Auditor role
Routes, cheapest first: verified cache → Qwen 2.5 3B → Qwen 2.5 14B → remote (only if policy allows) · NanoGate
Source: RESULTS.md, runs of 2026-09-26 on HP Z Runtime (BF16), DLP run 2026-09-23 · synthetic & public data · NanoGate
4 · SOLUTION — MEASURED RESULTS
Measured on the ZGX Nano, not projected
0.999
DLP F1, layered
vs 0.614 regex-only · 340 spans
97.2%
Cache precision
vs 50.0% cosine-only · 17 ms lookup
101/101
Attacks blocked
security suite, 0 failed
60/60
Receipts verified
end-to-end run · 0 remote bytes
Answer accuracy on 362 held-out questions
3B only
74.3% · 5.2 s
NanoGate router
83.7% · 13.4 s
14B always
84.3% · 14.4 s
Router, calibrated
AUROC 0.849. On the half it keeps local, it's right 95% of the time.
0 errors at 8 concurrent requests · offline test PASS
Q&A · TRADE-OFFS WE MADE ON PURPOSE
Every AI request, the cheapest safe route with proof.
Small model first
3B answers by default; 14B only on predicted risk. Some accuracy for speed and energy.
Verify before reuse
~17 ms (P50, on GPU) per lookup buys 4× fewer wrong cached answers.
Fail closed
If DLP is down, requests are denied. Availability traded for privacy.
App-layer egress
Enforced in the gateway, not the OS firewall yet. nftables/eBPF is next.
Next up
OS-level egress · SSO + multi-node · per-tenant router tuning · pilot with one department