From Inference Loops
to Long-Running Agents
Fundamentals, Workflow, and a Real Example
Shekhar UpadhayaShekhar Upadhaya
CTO · Skyhost · ex-Amazon
§ 01The model
“text in”→LLM→“text out”
An LLM is a function. Text in, text out. Stateless. No memory between calls.
§ 01The model
An agent is a while-true loop
that appends to an array.
agent.py
# What an agent actually is
while True:
user_input = get_input()
response = llm.complete(user_input)
if response.wants_tool:
result = execute_tool(response.tool_call)
response = llm.complete(result)
print(response)Agent · ↻ while true
user input→
LLM
→output↑↓
tool
§ 01The model
The context window
is just an array.
- ◆Every API call sends the entire array.
- ◆Each turn appends.
- ◆The model is stateless.
CONTEXT WINDOW · 200K
system prompt6.3k
tool definitions9.5k
CLAUDE.md2.5k
user message0.4k
assistant1.2k
tool result3.1k
unused context
§ 01The model
The loop, in action.
agent.py
# this iteration takes the tool branch
while True:
user_input = get_input()
response = llm.complete(user_input)
if response.wants_tool:
result = execute_tool(response.tool_call)
response = llm.complete(result)
print(response)Agent · log
// agent ready · waiting for input
→user_input received“find all TODO comments in src/”
→LLM responds: wants_tool=truetool_call: bash(“grep -rn TODO src/”)
→executing tool · bash3 matches in auth.ts, api.ts, db.ts
→LLM responds with answer“Found 3 TODOs across the codebase”
→print(response) · loop iterates ↻
Context window · 200K
system prompt6.3k
tool definitions9.5k
CLAUDE.md2.5k
skills0.9k
user0.3k
assistant · tool_call0.5k
tool_result0.6k
assistant0.4k
~179k remaining · empty
§ 01The model
The agent harness wraps the loop.
Everything that isn’t the LLM — what tools exist, what context loads, when to stop.
Agent harness · Claude Code
Agent · ↻ while true
user input→LLM→output
system prompt
context mgmt
skills & tools
MCPs
sub-agents
plan mode
session persistence
permissions & hooks
§ 01The model
the whole stackLoop.Array.Harness.
An agent is a while-true loop appending to an array. The harness controls what’s in it.
If the context window is just an array,
what goes in the array is everything.
§ 02Context
The instruction ceiling is real.
Frontier thinking models reliably follow ~150–200 instructions. Beyond that, even rules at the top get ignored.
smaller models · exponential decay•frontier thinking · linear decay
source · Dex Horthy / humanlayer.dev · arxiv:2507.11538
§ 02Context
Smart zone, dumb zone.
The window isn’t uniform. The first ~40%is where the model thinks clearly. Past that, attention frays — tool choice gets sloppy, instructions get dropped, the goal drifts.
“The more context you use, the worse results you’ll get.”
Dex Horthy / humanlayer.dev · Geoffrey Huntley
Context window · 200K
smart zonedumb zone
system prompt6.3k
tool definitions9.5k
CLAUDE.md2.5k
skills0.9k
~180k remaining · fresh session
§ 02Context
The allocation problem.
Static fills eat your usable space before the conversation starts.
Context window · 200K
smart zonedumb zone
system prompt6.3k
tool definitions9.5k
CLAUDE.md2.5k
skills0.9k
~180k remaining · ~60k smart available
§ 02Context
The context rot problem.
What’s in there is wrong. Errors propagate. Compaction is lossy.
Context window · 200K
smart zonedumb zone
system prompt6.3k
tool definitions9.5k
CLAUDE.md2.5k
skills0.9k
user · “implement feature X”0.3k
assistant · tool_call0.5k
tool_result · ok0.6k
~179k remaining · clean context
§ 02Context
Good context stays in the smart zone.
- Fresh session per task start clean, don't reuse a tired window
- Only what this task needs drop the MCPs and notes that aren't useful here
- Offload to disk save big stuff as files, keep short summaries in the window
- Send sub-agents for side quests let them explore, return one paragraph
- Leave room below the line finalizing work (tests, commits, lint) still has space
- Split big work across sessions when it won't fit one window, plan it, write the spec to disk, let multiple agents pick it up
Context window · 200K
smart zonedumb zone
system prompt · lean1.2k
tool definitions · 4 tools2k
spec.md · one task3k
skills0.9k
user · one clear goal0.3k
assistant · tool_call0.5k
tool_result1.5k
assistant · tool_call0.5k
tool_result2k
assistant · “done”0.4k
~188k remaining · all above the line
Every session starts from zero. Context doesn’t engineer itself.
Allocation, rot, compaction, recovery. Someone has to handle them.
§ 03The harness
Your harness wraps theirs.
Anthropic ships the agent harness. You ship the layer around it.
User harness · what you build
Agent harness · Claude Code / Cursor / Codex
Agent · ↻ while true
user input→
LLM
→output↑↓
tool
system prompt
context mgmt
skills & tools
MCPs
sub-agents
plan mode
session persistence
permissions & hooks
custom CLAUDE.md
agent_docs/
custom skills
issue tracker
The Ralph loop
3rd-party tools
§ 03The harness
Your harness is the layer you own.
Files, skills, loops, and rules the agent reads every session. That’s what makes long-running runs possible.
§ 03The harness
The setup.
~/project/
├── CLAUDE.md
├── .claude/
│ └── skills/
│ ├── grill-me.md
│ ├── to-prd.md
│ ├── session-planner.md
│ └── improve-codebase-architecture.md
├── .beads/
├── agent_docs/
│ ├── adding-a-feature.md
│ ├── anti-patterns.md
│ ├── architecture.md
│ ├── components.md
│ ├── server.md
│ ├── tech-debt.md
│ ├── workflow.md
│ └── specs/
├── scripts/
│ └── ralph.sh
└── src/
├── app/
│ └── CLAUDE.md
├── modules/
│ └── CLAUDE.md
├── components/
│ └── CLAUDE.md
└── server/
└── CLAUDE.md
0102030405
CLAUDE.md
project map · multi-level
agent_docs/
architecture · conventions · specs
custom skills
slash commands · tiny markdown
issue tracker
Beads · dependency graph · survives sessions
The Ralph loop
the loop · fresh window each pass
§ 03The harness
CLAUDE.md.
~/project/
├── CLAUDE.md
├── .claude/
│ └── skills/
├── .beads/
├── agent_docs/
│ └── ...
├── scripts/
│ └── ralph.sh
└── src/
├── app/
│ └── CLAUDE.md
├── modules/
│ └── CLAUDE.md
├── components/
│ └── CLAUDE.md
└── server/
└── CLAUDE.md
- A map, not a brain dump. Points at docs, doesn't contain them.
- Lists standards, never-rules, skill names. One section each.
- Multi-level. Sub-CLAUDE.mds in each module — app, modules, components, server.
- Progressive disclosure. Root loads at session start. Sub-files load only when the agent enters that directory.
- Same context budget, more steering. Right rules show up at the right moment.
§ 03The harness
agent_docs/
~/project/
├── CLAUDE.md
├── .claude/
│ └── skills/
├── .beads/
├── agent_docs/
│ ├── adding-a-feature.md
│ ├── anti-patterns.md
│ ├── architecture.md
│ ├── components.md
│ ├── server.md
│ ├── tech-debt.md
│ ├── workflow.md
│ └── specs/
├── scripts/
│ └── ralph.sh
└── src/
└── ...
- One concern per file. architecture, conventions, anti-patterns, workflow.
- Linked from CLAUDE.md, not loaded by it. “Before adding a feature, read adding-a-feature.md.”
- On-demand context. Agent reads docs only when relevant — nothing wasted up front.
- Specs split big work into phases. Dated, on disk, diffable. One phase per session.
- Surviveable. Specs outlive context windows. Reset and continue.
§ 03The harness
custom skills
~/project/
├── CLAUDE.md
├── .claude/
│ └── skills/
│ ├── grill-me.md
│ ├── to-prd.md
│ ├── session-planner.md
│ └── improve-codebase-architecture.md
├── .beads/
├── agent_docs/
│ └── ...
├── scripts/
│ └── ralph.sh
└── src/
└── ...
/grill-meinterview before plan
/to-prdturn idea into a PRD
/session-plannerbreak PRD into ralph-ready sessions
/improve-codebase-architectureaudit + propose refactors
Each skill wraps a recurring workflow into one verb. Loaded only when invoked. Stolen from Matt Pocock.
§ 03The harness
issue tracker (Beads)
~/project/
├── CLAUDE.md
├── .claude/
│ └── skills/
├── .beads/
├── agent_docs/
│ └── ...
├── scripts/
│ └── ralph.sh
└── src/
└── ...
- Tasks survive sessions. Not in the context window — on disk, in a graph.
- Dependency graph. Beads knows what's blocked, what's ready, what's done.
- bd ready — the next thing to work on. One command. Top-priority issue with no open blockers.
- Linked to specs. One spec breaks into many issues. Same vocabulary across plan and execution.
- Feeds the loop. Ralph asks Beads what's next, runs it, loops.
§ 03The harness
The Ralph loop.
~/project/
├── CLAUDE.md
├── .claude/
│ └── skills/
├── .beads/
├── agent_docs/
│ └── ...
├── scripts/
│ └── ralph.sh
└── src/
└── ...
while :; do
cat PROMPT.md | claude # Claude Code CLI
done- ▸The window is the budget. A plan becomes epics, epics become issues. Each session takes whatever fits inside the smart zone, could be one issue, could be five.
- ▸PROMPT.md is the instruction sheet. Tells the agent which spec to read, where to find the next task, and the rules of the run. Re-read every iteration. This is where the intelligence lives.
- ▸Reset every loop. Fresh window each pass. No compaction. State that matters lives on disk: Beads, specs, CLAUDE.md.
§ 04The plan
Where my hours actually go.
Plan~2:00 hr
Execute59 min
Review~30 min
Most of my time is here, not in the run. Brainstorm, grill, spec, atomic issues.
§ 04The plan
Two ways to plan.
Write the spec after you understand it, not before.
Plan mode
eager to write
Wants to produce an asset first. Writes the plan before alignment.
/grill-me
eager to understand
Wants shared understanding and alignment first. The asset comes after.
§ 04The plan
Grill before you plan.
·/grill-meThe agent asks now or assumes later. Assumptions become bugs.
Q1.
What happens when a user pulls to refresh while a stream is loading? Cancel? Queue?
Q2.
Bottom-sheet scroll behavior, does it lock the parent scroll or compete with it?
Q3.
Offline state: stale data shown, error shown, or skeleton? Decide once, here.
§ 04The plan
The spec.
·/to-prdSynthesizes the grilling session into a PRD on disk. No new interview.
specs/2026-05-04-mobile-components-phase-4.md
## GOAL
Reimplement 8 web components for React Native.
## CONTEXT
Web components don't translate 1:1 to RN. Preserve the visual language.
## SCOPE
atoms, molecules, weather, explore, profile, places, itineraries, verification.
## ACCEPTANCE
Visual parity at 3 breakpoints. All existing tests pass.
## OUT OF SCOPE
Architecture decisions, navigation refactor.
§ 04The plan
Decompose.
·/session-plannerSpec to issues, sized for the smart zone.
spec.md
→/session-planner
→
feature branch
→
N Beads issues + deps
→
ralph.sh
- ▸Sized for the smart zone. Each issue fits one fresh session.
- ▸Ralph script wired to this queue. Per-epic guard rails, ready to run.
§ 05The run
The run.
The same session that planned the work now runs and watches it.
- ▸Launched from the session, not a terminal. Same session that did grilling, spec, decompose now runs
ralph.sh. - ▸Ralph runs headless. Each iteration spawns a fresh
claude -p, picks the next Beads issue, lints, commits, closes it, loops. Verbose output streams to a log. - ▸The session polls the log. Tails every minute, summarizes progress, flags failures. I’m on my phone.
§ 05The run
Review.
Ralph closes the queue. I open the PR.
- ▸Open the PR. Ralph pushed and opened it. I read the diff.
- ▸Check the preview build. Web → Vercel preview. Mobile → Xcode Cloud build lands in TestFlight. Click through.
- ▸Loop back if needed. Anything off becomes a new issue. Ralph runs again. Otherwise merge.
§ 06Task selection
When the task fits, the result lands at 95–99% of what I wanted.
Task selection is the work.
§ 06Task selection
What fits
01
Large refactors
Class components to hooks. Old testing library to new one. Web app to mobile.
02
Migrations
Database changes, API upgrades, repo-wide pattern swaps.
03
Repetitive product work
Forms, tables, admin pages, login flows. Anything where the requirements are clear.
04
Mechanical cleanup
Dead code, dependency upgrades, documentation, test coverage.
§ 06Task selection
What doesn't fit
01
High-taste UI work
Motion, interaction feel, visual identity. Generators give you good. Taste gives you great.
02
Architecture choices
Service boundaries, data models, scaling tradeoffs. Wrong here is expensive.
03
Open-ended improvement
“Make this better.” “Modernize the product.” If you can’t measure success, the loop has no compass.
§ 07The shift
The harness compounds.
Every repeated mistake→an anti-pattern entry.Every repeated workflow→a slash command.Every recurring correction→a CLAUDE.md edit.Every mechanical rule→a lint.
Each fix is permanent. The next session starts smarter than the last.
Be on the loop. Not in.
With ideas fromGeoffrey Huntley · Dex Horthy · Matt Pocock · Steve Yegge
Ryan Lopopolo · Lance Martin · Mario Zechner · Armin Ronacher
Thank you.
You’ve got the map.
The Deck is free, and it stays free. When you’re ready to build the way it describes, the Harness Starter Kithands you the founder’s actual harness — the custom skills, CLAUDE.md, the ralph-loop script and the agent_docs templates — the moment you grab it.
Follow along — new drops, plus the Screencast free when it ships.
01 / 36
The Deck is the map. The Harness Starter Kit is the toolkit — founders lock the Founding price.