Private Eye
An on-device assistant for blind people
Point, tap, and it tells you what’s there, out loud.
Private Eye
1
The idea
WHAT IT SHOULD DO
Private Eye
2
Ask about anything in front of you
Tap
“Canon EOS camera with a zoom lens on the desk. Laptop behind it, to the right.”
“Watch for a T-shirt that has code on it”
“Found a t-shirt that has code on it.”
Private Eye
Camera frames and replies from the app’s own log.
3
It remembers your people and your things
Under the hood: YOLOE segmentation, ArcFace for faces, DINOv3 for belongings.
Tap
“Elias to your right. Charles ahead. 3 people in view. Two people walking on sidewalk, shopping bags.”
Tap
“Your water bottle. Sleep mask.”
Private Eye
Camera frames and replies from the app’s own log.
4
The last 10% is the hardest
Private Eye
5
When it can’t see, it makes things up
“What is my blood pressure reading?”
125/80 · 120/80 · 120/75 · 120/80 · 178/92
“How much sodium is in this package?”
1000 · 10mg · 0 mg ·
140 mg · 100 mg
“What kind of medication is in this bottle?”
“Antibiotics” · “antibiotic”
“eye drops”
Private Eye
Answers from 2B models, in red. None of them is in the photo.
6
So we had
to dig in.
Private Eye
7
Into the data, on Databricks
1 · DATASET
VizWiz
20,000 photos taken by blind people, each with a question and 10 crowd answers
→
2 · AI GATEWAY
Relabel in under an hour
Two GPT models behind one endpoint rewrite all 20,000 labels for $87. Key kept as a secret, every call logged
→
3 · UNITY CATALOG
One source of truth
Every label, answer and grade is a row: 12 tables, with 26,000 photos in a volume beside them
4 · FINE-TUNE, RUN 1
20,000 examples, 13 min
Trained on our own GPU. Invented answers fall from 58% to 9%
→
5 · AI GATEWAY + MLFLOW
Grade it, find the faults
gpt-6.1-sol grades each run through the gateway. Run 1 declines 71 photos it could answer and loses left and right
→
6 · FINE-TUNE, RUN 2
40,000 examples, 25 min
Same training, new data: 19,400 tap and where-is-it examples, 6,400 “what to do next” declines. Wrong declines halved. This is Witness-2B
Private Eye
All three runs in MLflow, both model versions in the Unity Catalog registry, dashboard and Genie on top.
8
Witness-2B: our fine-tune of LFM3
“What is my blood pressure reading?”
“Glare obscures the reading. Tilt the phone.”
All 500 answers
Right
Partly
“Can’t tell”
Wrong
Hallucinations, 115 unanswerable photos
Share where the model invents an answer
Witness-2B
66%
13%
8%
Qwen3.5-2B
50%
35%
80%
Qwen3-VL-2B
49%
32%
70%
LFM3-VL-2B (stock)
45%
33%
58%
SmolVLM2-2.2B
45%
31%
57%
Gemma 4 E2B
39%
38%
57%
InternVL3.5-2B
39%
39%
74%
Private Eye
500 VizWiz photos, the same bare question to every model, graded by gpt-6.1-sol.
9
Try it
yourself!
Private Eye
10