ARCESIUM • ENGINEERING DEEP DIVE
Setting up Harness for
Reliable AI Agents
Why reliability in enterprise agents comes from the environment around the model — not the model itself.
Instructions
State
Verification
Session
Scope & Lifecycle
Udit Ujagar ( Arcesium India Pvt Limited)
THE SHIFT
2026 is the year of Harness Engineering
Each year, the lever that most improves AI output moves up a level of abstraction — from the words we send, to the context we assemble, to the entire environment the agent works inside.
2024
Prompt Engineering
Craft better instructions for a single response.
2025
Context Engineering
Assemble the right information into the window.
2026
Harness Engineering
Design the whole working system around the model.
"X Engineering" keeps climbing — but is harness real, or just hype? The evidence says it is real.
Arcesium • Setting up Harness for Reliable AI Agents
2
DEFINITION
What is a harness — and why it matters
Harness Engineering builds the working environment around a foundation model so it produces reliable results. It is not about better prompts — it is system design.
THE KEY INSIGHT
A harness does not make the model smarter. It establishes a closed-loop working system around the model — turning raw capability into dependable output.
Constrain behaviour
Explicit rules and boundaries keep the agent on task.
Maintain context
Carry state across long-running, multi-session work.
Verify the work
Self-reflection plus full-pipeline tests confirm completion.
Stay observable
Make runtime debuggable so agents can't declare early victory.
Arcesium • Setting up Harness for Reliable AI Agents
3
THE EVIDENCE
Does the harness actually deliver value?
Anthropic ran the same one-line prompt — "build a 2D retro game maker" — through a single agent and through a full multi-agent harness. The difference was immediate and dramatic.
Solo agent
20 min • $9
Wasted layout, rigid workflow
Entities appeared but ignored input
Core game was simply broken
Full harness
6 hr • $200
16-feature spec, 10 sprints
Richer editors + built-in AI features
Playable game that actually worked
20× more expensive — but the only run that produced working software. Harness cost buys reliability.
Arcesium • Setting up Harness for Reliable AI Agents
4
INDEPENDENT CONFIRMATION
The same finding, across the industry
This isn't one lab's quirk. Anthropic, OpenAI, and the open-source community independently converged on the same conclusion: a harness turns an unreliable model into a reliable one.
Anthropic
Effective Harnesses (Nov 2025) and Harness Design (Mar 2026) — the planner / generator / evaluator pattern.
OpenAI
Harness Engineering for Codex (Feb 2026): in a well-harnessed repo the same model shifts from unreliable to reliable — a qualitative jump, not a marginal gain.
Community
Geoffrey Huntley's "Ralph Wiggum" technique and open courses converged on the same loop independently.
"The model didn't change. The harness did." — the through-line in every source.
Arcesium • Setting up Harness for Reliable AI Agents
5
ANATOMY
What constitutes an agent harness?
A harness around an agent decomposes into five interlocking sub-areas. Each one closes a specific reliability gap.
01
Instructions
Rules, boundaries & operating procedure the agent follows.
02
State
Persistent memory that survives across sessions.
03
Verification
Self-checks and tests that confirm work is truly done.
04
Session
Clean handoffs between context windows.
05
Scope & Lifecycle
What's in bounds, and how a task starts and ends.
The rest of this deck walks each sub-area, then shows the architectures that wire them together.
Arcesium • Setting up Harness for Reliable AI Agents
6
THE CORE CHALLENGE
Every new session starts with no memory
Imagine a software project staffed by engineers working in shifts, where each new engineer arrives with no memory of what happened on the previous shift.
Failure mode 1
Doing too much at once
The agent tries to one-shot the whole app, runs out of context mid-feature, and leaves the next session a half-built mess to untangle.
Failure mode 2
Declaring victory too early
A later agent sees progress, assumes the job is done, and stops — shipping at 30% complete with full confidence.
7
FOUNDATIONAL PATTERN
The two-fold harness solution
Anthropic's first harness split the work into two roles that hand off through structured artifacts on disk.
Initializer agent
Runs once, on the first session
Expands the prompt into a comprehensive feature list
Writes an init.sh to start the dev server
Creates a progress file + initial git commit
Coding agent
Runs every subsequent session
Reads progress file + git log to get its bearings
Works on ONE feature at a time
Self-verifies, commits, updates progress, leaves clean state
Same model, same tools — the agents differ only in their initial prompt.
Arcesium • Setting up Harness for Reliable AI Agents
8
THE WORKFLOW
From feedback to done: the harness pipeline
A production harness threads one ticket through a fixed sequence of stages. Plans, progress, and evaluator verdicts are persisted as JSON so state survives every context reset.
Triage
Auto-detect: bug or feature?
Clarify
Resolve unknowns up front.
Plan
Decompose into tasks + ACs.
Execute
Implement with TDD.
Evaluate
Skeptical QA gates merge.
Done
Only when ACs pass.
Persisted as JSON: per-ticket plans, evaluator verdicts, and session notes — the agent picks up exactly where the last one stopped, even after a full reset.
Arcesium • Setting up Harness for Reliable AI Agents
9
SUB-AREA 1 · INSTRUCTIONS
The feature list: a contract for "done"
To stop the agent one-shotting or quitting early, the initializer expands the prompt into a structured JSON feature list — every capability marked failing until proven otherwise.
200+ end-to-end features
The claude.ai clone spec listed over 200 concrete, testable features.
JSON, not Markdown
Models are far less likely to silently overwrite or edit a JSON file.
Edits restricted to one field
Agents may only flip the passes flag — "removing tests is unacceptable."
feature_list.json
{
"category": "functional",
"description": "New chat button
creates a fresh conversation",
"steps": [
"Click the New Chat button",
"Verify a new conversation",
"Check welcome state shows"
],
"passes": false
}
Arcesium • Setting up Harness for Reliable AI Agents
10
SUB-AREA 2 · STATE
Bridging the gap between sessions
Compaction alone is not enough — it can pass muddy instructions forward. Durable, on-disk artifacts let a fresh agent rebuild full situational awareness in seconds.
feature_list.json
The source of truth for what's built and what remains.
claude-progress.txt
A running log of what each session accomplished.
git history
Descriptive commits let the agent revert to a known-good state.
init.sh
One command to launch the dev server and smoke-test.
Inspiration came straight from what effective software engineers do every day: leave the codebase clean and documented for whoever picks it up next.
Arcesium • Setting up Harness for Reliable AI Agents
11
SUB-AREA 3 · VERIFICATION
Trust nothing the agent grades itself
Left alone, Claude ran unit tests and curl commands, then marked features done — without ever confirming they worked end-to-end as a user.
The trap
Skews positive when grading its own work
Tests superficially, skipping edge cases
Talks itself out of bugs it has already found
Marks features "passing" without real proof
The fix
Browser automation (Puppeteer / Playwright)
Test every feature the way a real user would.
Default-FAIL contract
Every criterion starts false until evidence proves it.
Fresh-context evaluator
A separate agent grades work it never helped build.
Arcesium • Setting up Harness for Reliable AI Agents
12
SUB-AREA 4 · SESSION
Context resets vs. compaction
As the window fills, models lose coherence — and some develop "context anxiety," wrapping up early as they near a limit they only imagine. The handoff strategy matters.
Compaction
Summarizes earlier conversation in place so the same agent keeps going on a shortened history.
Preserves continuity
No clean slate — anxiety persists
Context reset
Clears the window entirely and starts a fresh agent, with a structured handoff carrying state + next steps.
Clean slate every time
Costs orchestration + tokens
Sonnet 4.5 needed resets to beat context anxiety. Opus 4.5+ largely removed the behaviour — so the harness could drop them. Scaffolding should track the model.
Arcesium • Setting up Harness for Reliable AI Agents
13
SUB-AREA 5 · SCOPE & LIFECYCLE
How every session starts and ends
A fixed startup ritual orients each fresh agent, and a fixed shutdown ritual leaves the environment clean for the next one. Scope stays bounded to the working directory.
1
pwd
Confirm the working directory — the only place edits are allowed.
2
Read state
Progress file + git log to see what was recently done.
3
Pick one feature
Choose the highest-priority item still failing.
4
Smoke test
Run init.sh, verify the app still works before building.
5
Build + verify
Implement the feature; test it end-to-end as a user.
6
Hand off
Commit with a clear message; update the progress file.
Arcesium • Setting up Harness for Reliable AI Agents
14
THE NEXT STEP
From two agents to a GAN-inspired loop
Agents praise their own work — especially on subjective tasks. Borrowing from Generative Adversarial Networks, separate the maker from the judge.
Generator
Builds the work — one feature / sprint at a time.
Evaluator
A skeptical, separately-tuned agent that grades and critiques.
feedback
loop
5–15
iterations per run, each pushing toward a stronger result
Why it works: Tuning a standalone evaluator to be skeptical is far more tractable than making a generator critical of its own work. Once external feedback exists, the generator finally has something concrete to iterate against.
15
ARCHITECTURE
The three-agent harness
Applied to full-stack development, the loop becomes Planner → Generator → Evaluator — each agent closing a gap observed in earlier runs. They communicate through files.
Planner
Expands a 1–4 sentence prompt into an ambitious product spec — staying high-level so early mistakes don't cascade downstream.
Generator
Works in sprints, one feature at a time, on a React / FastAPI / Postgres stack. Self-evaluates before handing off.
Evaluator
Drives the live app via Playwright, files specific bugs, and grades each sprint against hard thresholds.
Before each sprint the generator and evaluator negotiate a "sprint contract" — agreeing what done looks like before any code is written.
Arcesium • Setting up Harness for Reliable AI Agents
16
EVALUATOR IN ACTION
Specific bugs, not vague approval
A well-tuned evaluator clicks through the running app and files findings precise enough to act on without further investigation — Sprint 3 alone had 27 test criteria.
CONTRACT CRITERION
EVALUATOR FINDING
Rectangle fill tool fills a region by click-drag
FAIL Tool only places tiles at drag start/end — fillRectangle never triggers on mouseUp.
User can delete a placed entity spawn point
FAIL Delete handler requires two state flags, but clicking only sets one. Condition is wrong.
Animation frames can be reordered via the API
FAIL PUT /frames/reorder defined after /{frame_id} — FastAPI parses 'reorder' as an integer, 422.
Out of the box Claude is a poor QA agent — it took several rounds of reading logs and tuning the prompt before the evaluator graded the way a careful human would.
Arcesium • Setting up Harness for Reliable AI Agents
17
MAINTAINING A HARNESS
Every component encodes an assumption
As models improve, scaffolding can go stale. When Opus 4.6 landed — planning better and sustaining longer tasks — the harness was deliberately stripped back, one component at a time.
Stress-test assumptions
Each piece of a harness assumes something the model can't do alone — check whether that's still true.
Drop the dead weight
Sprints and context resets, once essential on 4.5, became overhead once 4.6 could handle the work natively.
Keep what's load-bearing
Planner and evaluator stayed — they still added clear value at the edge of the model's ability.
Find the simplest solution possible, and only increase complexity when needed.
The space of useful harnesses doesn't shrink as models improve — it moves. Finding the next combination is the work.
Arcesium • Setting up Harness for Reliable AI Agents
18
DISCIPLINE · CONTEXT
Treat the context window as scarce
The more context an agent consumes, the worse its output. The primary window should act as a scheduler — offloading heavy work to subagents and keeping only the essentials loaded.
Offload to subagents
Fan out read-only work — search, analysis, summarization — at high parallelism; keep write operations serialized.
Load core files every loop
Plan and spec are reloaded each iteration so the agent always works from the same foundation.
Search before implementing
Code search is non-deterministic — instruct the agent to look first, so it doesn't rebuild what already exists.
AGENTS.md is a map,
not an encyclopedia
Keep the top-level instruction file short (~100 lines) — a table of contents pointing into a structured docs/ directory.
Progressive disclosure: a small, stable entry point that teaches the agent where to look — rather than overwhelming it up front.
Arcesium • Setting up Harness for Reliable AI Agents
19
RUNNING AUTONOMOUSLY · SAFETY
Defense in depth for autonomous agents
An agent that edits files and runs commands on its own needs guardrails. Layer three independent controls so no single failure gives the agent free rein.
L1
OS-level sandbox
Isolate the execution environment so the agent can't touch the host system.
L2
Filesystem restrictions
Limit all file operations to the project directory — nothing outside the worktree.
L3
Command allowlist
Permit only the commands the agent needs. Parse with shlex, handle pipes, block the rest.
Add extra validation for sensitive commands — e.g. allow pkill only for dev processes, chmod only for +x.
Arcesium • Setting up Harness for Reliable AI Agents
20
REALITY · RESILIENCE
Expect failures — design for recovery
You will wake up to broken builds. A resilient harness makes recovery cheap, so the answer is always either reset-and-rerun or a quick rescue prompt.
Git is the safety net
Commit after every task; tag known-good states. A broken codebase is one git reset --hard away from recovery.
Regenerate stale plans
Todo lists drift. Periodically delete and rebuild them by comparing the codebase against the spec.
Cap the retry loop
If a task fails evaluation twice, stop rather than looping forever — surface it for a human.
Pay debt continuously
Run recurring cleanup agents that scan for deviations and open small, targeted refactor PRs.
Arcesium • Setting up Harness for Reliable AI Agents
21
GET STARTED
A minimal harness you can add today
You don't need the full multi-agent system to benefit. Drop four structured files into your repo and every agent session starts from the same known state.
YOUR PROJECT ROOT
|--
AGENTS.md
the agent's operating manual
|--
CLAUDE.md
alt. for Claude Code
|--
init.sh
install + verify + start
|--
feature_list.json
features, and which are done
|--
claude-progress.md
what happened each session
|--
src/
your actual code
Sessions start from the same state
No more cold starts or re-explaining the project each time.
Scope stays bounded
The feature list keeps the agent on one task at a time.
You review, not rescue
Verification gates mean you check results instead of cleaning up.
Four files, and your agent sessions are already far more stable than running on prompts alone.
Arcesium • Setting up Harness for Reliable AI Agents
22
BRINGING IT HOME
What this means for enterprise agents
Reliability is an environment problem
Don't wait for a smarter model — build the harness around the one you have.
Make "done" explicit and testable
A structured spec plus default-FAIL criteria stops premature victory.
Separate the maker from the judge
A skeptical, fresh-context evaluator catches what self-grading misses.
Persist state across sessions
Progress files and git let any agent resume cleanly from cold.
Keep runs observable
If you can't see what the agent did, you can't trust that it's done.
Revisit the harness each model
Strip stale scaffolding; add what new capability makes possible.
2026 is the year of Harness Engineering — and the harness, not the model, is where reliability is won.
REFERENCES
Sources & further reading
This deck synthesizes the Arcesium engineering blog with primary research and open-source practice from across the industry.
PRIMARY RESEARCH
Anthropic — Effective Harnesses for Long-Running Agents (Nov 2025)
Anthropic — Harness Design for Long-Running Application Development (Mar 2026)
OpenAI — Harness Engineering: Leveraging Codex in an Agent-First World (Feb 2026)
PRACTICE & COMMUNITY
celesteanders/harness — minimal generator + evaluator harness and best-practices guide
walkinglabs/learn-harness-engineering — 12-lecture, 6-project course
Geoffrey Huntley — the "Ralph Wiggum" technique (Jul 2025)
Arcesium • Setting up Harness for Reliable AI Agents
24