1 of 8

Tracing the whole pipeline,

not just the LLM

End-to-end observability for AI systems

The Fifth Elephant 2026 · Track 1 — Data Engineering & Infrastructure · Hands-on workshop

take data → classify → retrieve → validate → route

2 of 8

The model call is the one part you can already see

Everyone instruments the model: tokens, latency, prompt, response. There's a dashboard for it.

But a production AI feature is a pipeline. When something breaks — a wrong answer, a latency spike, a cost surprise — the LLM dashboard says the model was “fine.” And it was.

The break was three stages upstream, in code you never traced.

take data

dark

classify

observed

retrieve

dark

validate

dark

route

dark

3 of 8

An AI feature is a pipeline, not a call

Most stages are plain deterministic code. Only a couple are the model. Both must be in one trace.

take data

code

classify

LLM

retrieve

code

validate

code

route

code

deterministic (assertable)

non-deterministic (model)

4 of 8

What the data actually is

The demo is a customer-support resolver. Here is what flows through each stage.

Stage

Type

What flows through it

take data

code

the incoming support ticket — message text, channel, customer ID; normalize & validate

classify

LLM

the intent of the ticket — billing, refund, how-to, bug report

retrieve

code

matching help-centre articles, policy docs, and past resolved tickets

validate

code

the proposed reply against business rules and groundedness in the retrieved docs

route

code

auto-send the resolved reply, or escalate to a human agent

Same five stages as the abstract — only one is the model. The trace makes every one of them visible.

5 of 8

One trace, two kinds of telemetry

Deterministic spans

code stages — you can test these

  • retrieval recall & hit rate
  • validation pass / fail
  • DB and tool latency
  • error and retry counts

Model generations

non-deterministic — attached to the trace

  • tokens in / out
  • cost per call
  • generation latency

Both hang off the same trace, so a model step sits in context next to the code around it.

6 of 8

What to record at each span

Stage

Record

Assert / alert on

take data

payload size, schema valid

malformed input rate

classify (LLM)

tokens, cost, latency, label

low-confidence share

retrieve

query, k, hit count, latency

zero-hit / recall drop

validate

rule results, groundedness

validation failure rate

route

destination, escalation flag

escalation spike

LLM rows are observed; code rows are tested. Same trace, different contract.

7 of 8

One request, read end to end

Open any request: the waterfall shows every stage as its own span on one timeline.

take data

0.3s

classify

0.9s

retrieve

0.7s

validate

0.4s

route

0.2s

time →

every stage is its own span — the whole lifecycle reads off one trace

8 of 8

The end-to-end trace lifecycle

One request, one trace — built up stage by stage, then read end to end in Langfuse.

1

Request enters

The root trace is created. Everything downstream attaches to it.

→

2

Each stage opens a span

Code stages record assertable attributes; the model stage is a generation with tokens, cost, latency.

→

3

Spans nest under the root

In execution order, so a model step sits in context next to the code around it.

→

4

Trace flushes to Langfuse

Open it and read the full lifecycle of that single request — every stage in one view.

Same trace, read healthy here — the exact view that later localizes any stage to the span that owns it.