TRUSTSQL

Trust SQL.

Verify the answer, not just the syntax.

An AI agent that verifies whether a SQL query's ANSWER can be trusted — before it reaches a dashboard, a deck, or a decision.

Built on CRAFT MCP · Emergence AI

Reasoning by Nebius Token Factory

Hackathon 2026

Emergence AI × Nebius Token Factory Hackathon · July 2026

01

THE PROBLEM

The query executed.

The insight didn't.

A revenue chart goes to leadership. The SQL is green. The dashboard looks great. Nobody questions it — until someone realizes “average monthly revenue” is actually averaging line-item prices, not monthly totals. The join used the wrong key. Filters for completed orders and active customers are missing.

It didn't fail. It succeeded at being wrong.

Query OK · Ran in 0.4s

Avg. Monthly Revenue — Active Customers, 2025

$118,402

This number is wrong — and everyone trusted it.

Shipped to: Leadership deck, Q3 board review

02

THE GAP

Everyone checks that SQL runs.

Nobody checks that it's right.

Syntax checkers

Verify SQL is well-formed

BI tools

Verify queries execute

Text-to-SQL generators

Write SQL fast

TrustSQL

Verify the result matches the business question

AI is generating SQL faster than humans can review it. Verification is the missing layer.

03

THE SOLUTION

Two inputs. One verdict.

TrustSQL takes a business question and a SQL query, and tells you — with evidence — whether the answer is trustworthy.

INPUT

Business question

“What was the average monthly revenue from active customers during 2025?”

SQL query

Pasted by analyst — or generated via CRAFT generate_sql

TRUSTSQL AGENT

9-phase verification

pipeline

Generates validation SQL, executes it on live Snowflake data via CRAFT MCP, then reasons over the evidence with Nebius LLM.

Empirical, not theoretical

OUTPUT

34%

Confidence score

+ 7 dimension cards

Suggested corrected SQL

Policy compliance check

Exportable MD / JSON report

MCP tool audit trail

04

HOW IT WORKS

Hypothesis → SQL → Execute → Evaluate.

Never trust an LLM assertion without data evidence. Every phase below either extracts structure or produces proof.

1

Resolve business terms

CRAFT · resolve_term

2

Extract structured intent

Nebius LLM

3

Parse SQL structure

sqlglot

4

Detect joins & risks

SQLAnalyzer

5

Compare SQL vs. intent

AlignmentChecker

6

Generate hypotheses

falsifiable claims

7

Execute validation SQL

CRAFT · execute_query

8

Reason over evidence

Nebius LLM

9

Score & build report

ReportBuilder

▲ live Snowflake execution — the moment the agent stops guessing and starts proving

05

LIVE DEMO

34% confidence. Caught in seconds.

“What was the average monthly revenue from active customers during 2025?” — verified against live Snowflake data.

CONFIDENCE SCORE

34%

LOW

This query will systematically overstate revenue. Two of three joins use incorrect or unfiltered logic.

7 dimensions scored below →

Business Intent Alignment

Question maps to a revenue-by-month metric — correctly scoped.

Join Validation

Uses o.id instead of order_id — fan-out risk confirmed by test query.

Aggregation

AVG() runs over line items, not monthly totals — wrong grain.

Business Rules

No completed-order filter, no active-customer filter, no 2025 bound.

06

THE MATH

A weighted score, not a vibe.

C = ∑ wi · si

i = 1..7

Each dimension score sᵢ ∈ [0, 100], weighted by business importance wᵢ

High Confidence

C ≥ 80

Moderate

50 ≤ C < 80

Low Confidence

C < 50

Join cardinality red flag

rows after join / rows before join ≫ 1 ⇒ inflated metrics

7 WEIGHTED DIMENSIONS

Business Intent Alignment

22%

Join Validation

18%

Aggregation

18%

Business Rules

12%

Policy Verification

10%

Data Quality

10%

Performance

10%

07

ENTERPRISE GOVERNANCE

Not just for analysts. For governance teams.

Policy compliance baked directly into the review loop — configurable, versionable, enforced automatically.

data_policies.yaml

Org governance rules

•

Read-only SELECT only — no INSERT / UPDATE / DELETE / DROP

•

Revenue metrics must filter status = 'Complete'

•

No SELECT * on PII tables (USERS)

•

Fact-table scans require date filters on created_at

business_rules.yaml

Domain semantics

•

Active customers must have status = 'Active'

•

Revenue metrics exclude non-completed orders

•

Revenue grain awareness — order vs. line item

WHO IT'S FOR

Data Analysts

Sanity check before sharing numbers

Analytics Engineers

Review gate for self-serve & AI-generated SQL

Governance & Compliance

Policy enforcement in the review loop

Leadership

Trust scores instead of blind faith in dashboards

08

TECH STACK

CRAFT handles the what.

Nebius handles the why.

DATA

CRAFT MCP

Emergence AI · on Snowflake

Governed schema access — resolve_term, execute_query, generate_sql

REASONING

Nebius Token Factory

Meta Llama 3.1 70B Instruct

Multi-step LLM reasoning — intent, alignment, evidence, correction

INTERFACE

Streamlit

OAuth 2.0 PKCE sign-in

Interactive demo with live 9-phase progress tracker

PARSING

sqlglot

+ DuckDB fallback

Structural SQL analysis, offline mode when CRAFT is unavailable

MCP endpoint: https://nebius.emergence.ai/mcp Auth: OAuth 2.0 PKCE (Google / Microsoft) — same flow as Cursor IDE

09

BUILT UNDER PRESSURE

What we broke. What we shipped.

Challenges we overcame

OAuth in Streamlit ≠ OAuth in Cursor — full PKCE flow; localhost vs 127.0.0.1 broke session cookies

Real schema quirks — THELOOK uses order_id not id, microsecond epoch timestamps, quoted lowercase Snowflake columns

LLM hallucination risk — solved by requiring executed validation SQL before any failure claim

Fair scoring — mapping heterogeneous checks onto 7 weighted dimensions

Hackathon scope — shipped one thing completely instead of five things partially

What we're proud of

End-to-end 9-phase agent pipeline, working in a production demo

Live Snowflake verification via CRAFT MCP — not mocked

CRAFT OAuth replicated in a standalone Streamlit app

Policy + business rules verification layer

Queries that run perfectly but mean nothing, exposed in seconds

DuckDB offline fallback for demo reliability

Exportable Markdown / JSON reports for stakeholders

10

ROADMAP

This weekend was step one.

From single-query verification to continuous trust across every dashboard in the org.

CI/CD SQL review gates

Block low-confidence queries before they reach production

Slack / Teams alerts

Notify when scheduled reports drop below threshold

Org-wide policy packs

Shareable, versioned governance rules across teams

Semantic lineage

Trace metric definitions through joins & aggregations

Cross-dataset reconciliation

Same KPI computed two ways must agree

Real-time dashboard monitoring

Continuous trust scoring on live BI assets

11

TRUSTSQL

In analytics, the most

expensive bug isn’t the one

that crashes your query.

It’s the one that ships.

Generate fast. Verify before you trust.

Try it: live demo + Streamlit app on request�Built with: CRAFT MCP (Emergence AI) · Nebius Token Factory

SCAN FOR DEMO

Emergence AI × Nebius Token Factory Hackathon · July 2026

12