HumanLayer Blog

Context Forking to Save Time, Tokens and Trouble

Kyle — Thu, 14 May 2026 16:00:00 GMT

Context forking is a powerful coding agent primitive that allows you to build up high-quality context and then reuse it multiple times.

Lots of coding agents (OpenCode, Pi, Claude Code, and many others) support context forking, though they often give it different names: rewind, time traveling, or branching - but they're all variants of the same concept.

This post is a tactical guide on how to use context forking to save you time, tokens, money, and/or a great deal of frustration

## Context Windows as Operating System Stacks

I like to conceptualize agent context windows as downwards-growing stacks, inspired by stacks from operating systems:

In an OS stack, frames for new routines are appended to the bottom of the stack as routines are called. You can conceptualize a context window the same way, with each user message-assistant message turn as a new routine with a new stack frame:

Like a stack, coding agent context windows usually prevent random access. You can push things to the end of it by sending a user message, and you can pop (remove) things from the end.

But just like how an OS stack grows downwards as new routines are called & stack frames are pushed onto it, the context window only lets you push or pop from the newest end of the history. You are stuck interacting with the end of it.

Random access isn't allowed for several reasons:

It can cause expensive cache misses against your inference API
It can mangle important accumulated context
It can interfere with your coding agent's internal state

Most coding agents have internal state that tracks which files the agent has read and/or written (among other things). When the agent tries to edit a file, this allows the harness to prompt & force the agent to read the file before attempting to modify it.

Erasing or adding tool calls into the middle of a context window would require surgical manipulation of the coding agent's context AND any internal state, and agents generally don't support that.

## How does context forking work?

Context forking allows you to pop 1 or more messages off the bottom of the context window stack to restore the state to an earlier version.

Just like how an OS stacks pushes & pops entire frames at a time, you can usually only rewind your context window at user-message turn boundaries, not in the middle of a sequence of tool calls:

Usually, you can do this more than once - forking from the same context window in many different ways.

Interface and implementation details vary from agent to agent. Some agents allow you to rewind the state of the code & disk when you rewind the state of the conversation. Others may create a new branch or worktree when you do this.

## When should you fork your context window?

### Rewinding to course-correct an agent

A popular use-case is for forking is to rewind a conversation while the agent is implementing a feature to account for something that was missed:

### Forking to explore different design paths

I often find myself forking sessions during the design phase of a task.

Once I have accumulated high-quality context about the codebase and problem I'm trying to solve, I'll fork the conversation to explore different design & architectural paths.

Then I'll review the results and decide which session to proceed from - or that more research is necessary!

### Forking to preserve a context window from context-inefficient operations

Another great use of forking is to rewind a conversation to preserve high-quality built-up context after the agent does something context-inefficient.

An example of this is when the agent reads a large file or runs a command that generates a TON of output - tens of thousands of tokens worth. Most coding agent harnesses have hooks or other safeguards to prevent this by writing the output to a file and just showing the agent the filepath rather than the entire contents, and instructing the agent to search through it. But sometimes, the agent just reads all 40,000 tokens one chunk at a time and fills up the context window:

Fortunately, we can salvage the conversation with forking!

## Let's recap:

Coding agent context windows can be conceptualized as downwards-growing stacks. You can push to or pop from the bottom, but generally can't interactively mess with the middle or top of the stack
Context forking can be used to rewind a context window to re-steer it when the agent missed something
Context forking can be used to branch and explore multiple different design or implementation paths
Context forking can be used to restore a context window to a state before a large volume of low-quality context was added.

Long-Context Isn't the Answer

Kyle — Sun, 22 Mar 2026 16:00:00 GMT

Anthropic just switched the default model in Claude Code to Opus 4.6 with a 1M context window. We tried it when it launched. But now we're switching back to Opus 4.5

## How about some context?

Unlike previous Claude Opus models which have a ~ 200k context window, Opus 4.6 has as 1M context window. This was very exciting for us!

We spend a lot of time working on hard problems in large enterprise codebases, and if the model can reliably hold more of the codebase in its head at a time, so to speak, then that creates new possibilities for the scale of problem that can be solved in a single context window.

It would mean less compaction (whether frequent intentional compaction or auto-compact), and less "context pressure" to solve the problem before your context window fills up.

As Calvin French-Owen, who helped launch Codex last year, puts it in an episode of YC's Lightcone podcast:

...imagine you're a college student. You're taking an exam. In the first five minutes of that exam, you're like, "Oh, I have all the time in the world. I'll do a great job. I'll think through each of these problems."

Let's say you have like five minutes left and you still have half the exam left. You're like, "Oh man, I just got to do whatever I can." Like, that's the LLM with a context window

## More context, less instruction adherence

While the context window is dramatically larger, we noticed over the course of a couple of weeks that instruction adherence was dramatically degraded, and not just at longer context lengths.

Even well-within what we would consider the smart zone of a 200k-context frontier model, the model was less precise. It would ignore design documents and other inputs when writing a plan file. It would make trivial mistakes, or misunderstand simple instructions - or worse, directly disobey them.

At longer context lengths, the degradation was even steeper - like the user instructions were getting drowned out by the intermediate tool results and mass of accumulated context.

## What determines instruction adherence?

We've written about the concept of the instruction budget before - a measurable property of LLMs which describes how many instructions they can follow reasonably well before instruction adherence drops off.

It's different for each model, but it's a function primarily of the quality of the model's instruction tuning, and more importantly for our purposes, it is strongly correlated with the size of the model.

You can clearly see that the larger models from the study can follow dramatically more instructions before adherence drops off.

## Instruction adherence at long context lengths

Why does this matter?

When a lab offers an extended-context version of a model, you're usually not getting a bigger model with more parameters and therefore a larger "instruction budget" to go with the larger context window - you're likely getting the same model with some clever math (e.g. YaRN) to extend the sequence length the model can attend to, and probably some more post-training to stabilize the model at the longer sequence lengths.

This means that while the context window size increases, the instruction budget remains the same. You can fit more context and more instructions in the context window, but the model isn't actually better at attending to those instructions over the context length.

## Instruction-following as needle in a haystack

You can think of your context window as a haystack where all of your tool calls and documents and files are hay - every line in your CLAUDE.md file, every instruction in your tool descriptions, every tool result, every instruction in your system prompt, and every user message.

The quality & correctness of the agent's next step depends on the LLM's ability to find a needle in there: namely, the instruction(s) in the context window that are most-relevant to the current state of the context window, which give it the information it needs to make the correct decision about its next action.

Now imagine we increase the size of the haystack by 500% - but the size of the needle remains the same. Unless our ability to find the needle also increases by 500%, we will have a dramatically harder time finding it. The extra context isn't really helping us - it's just digging us deeper into the dumb zone.

## What Works Instead

Instead of trying to stuff as much information in a context window as possible and having the LLM reason over it, we found that the context management techniques we had been using had to be used even more aggressively to keep Opus 4.6 1M on track.

In my post on harness engineering, I wrote about this as the limit case for using sub-agents:

The day after I published it, I decided maybe it wasn't such a bad idea, and sat down and wrote this skill:

---
name: subagent-orchestrator
description: orchestrate sub-agents to accomplish complex long-horizon tasks without losing coherency by delegating to sub-agents
---

This skill provides you with **CRITICAL** instructions that will help you to maintain coherency in long-horizon context-heavy tasks.

You have a large number of tools available to you. The most important one is the one that allows you to dispatch sub-agents: either `Agent` or `Task`.

All non-trivial operations should be delegated to sub-agents. You should delegate research and codebase understanding tasks to codebase-analyzer, codebase-locator and pattern-locator sub-agents.

You should delegate running bash commands (particularly ones that are likely to produce lots of output) such as investigating with the `aws` CLI, using the `gh` CLI, digging through logs to `Bash` sub-agents.

You should use separate sub-agents for separate tasks, and you may launch them in parallel - but do not delegate multiple tasks that are likely to have significant overlap to separate sub-agents.

IMPORTANT: if the user has already given you a task, you should proceed with that task using this approach.

If you have not already been explicitly given a taks, you should ask the user what task they would like for you to work on - do not assume or begin working on a ticket automatically.

I found that using this skill in combination with our existing workflow skills helped me keep Opus coherent at longer context lengths. Why? As I wrote about in that post, sub-agents encapsulate context and ensure that only highly-relevant context (the prompt, and the focused sub-agent result) end up in the context window, avoiding context rot:

## How we're incorporating this at HumanLayer

We have had a feature in the tool for a while that warns the user when they're context is getting high. We used to set this at around %40 of sonnets 168k token window (200k - 32k reserved for output). This came out to about 100k tokens.

To help users maximize instruction adherence and intelligence on hard codebase problems, we've updated our context warnings for long-context models to trigger at the 100k token mark instead of 40% of the usable context. For opus 1m this is only 10% of the context window.

## TL;DR

Long-context models degrade at all context lengths, not just long ones.
More context isn't more capability - the instruction budget doesn't scale with the context window.
Context isolation beats context expansion. Sub-agents, progressive disclosure, and context-efficient backpressure keep each context window small, focused, and in the smart zone.

Getting Claude to Actually Read Your CLAUDE.md

Dex — Mon, 16 Mar 2026 16:00:00 GMT

Claude Code wraps your CLAUDE.md in a <system_reminder> that explicitly tells the model the contents "may or may not be relevant." The longer your file gets, the more Claude seems to treat individual sections as optional.

We wrote before about keeping these files concise and avoiding stuff that belongs in a linter. But even after trimming things down, I kept running into cases where Claude would ignore testing instructions while writing tests, or skip our API conventions when building endpoints.

## Conditional `<important if>` blocks

What's been working for me: wrapping sections in <important if="condition"> tags.

<important if="you are writing or modifying tests">
- Use `createTestApp()` helper for integration tests
- Mock database with `dbMock` from `packages/db/test`
- Test fixtures live in `__fixtures__/` directories
</important>

We don't have a rigorous explanation for why this helps. My guess is that the explicit condition gives Claude a clearer signal about when to apply the instructions, rather than leaving it to decide relevance on its own. Whatever the mechanism, we've seen noticeably better adherence on tasks where only some sections of my CLAUDE.md should apply.

A few things I've learned:

Don't wrap everything. Project identity, directory structure, and tech stack are relevant to basically every task. The conditional blocks are for testing setup or deployment procedures that only matter sometimes.

Make conditions specific. <important if="you are writing code"> matches almost everything and defeats the purpose. I try to make each condition narrow enough that it only fires when I actually want those rules applied.

## A skill to automate this

I did a lot of restructuring CLAUDE.md files by hand, so I made a Claude Code skill that does the rewrite. It pulls out the foundational stuff, wraps domain-specific sections in conditional blocks, and cleans up some common problems (stale code snippets, style rules that should be in a linter, vague instructions that don't actually help).

It's not perfect—you'll probably want to review what it produces—but it's a decent starting point.

## Try it

npx skills add humanlayer/skills --skill improve-claude-md

Then in your project:

/improve-claude-md

It reads your existing CLAUDE.md and outputs a rewritten version. Source is at github.com/humanlayer/skills.

Skill Issue: Harness Engineering for Coding Agents

Kyle — Wed, 11 Mar 2026 16:00:00 GMT

We've spent the past year watching coding agents fail in every conceivable way: ignoring instructions, executing dangerous commands un-prompted, and going in circles on the simplest of tasks.

We've seen teams ship immense amounts of slop. We've even shipped a little bit of slop ourselves.

Every time, the instinct was the same:

"We just need better models, GPT-6 will fix it"
"We just need better instruction-following"
"It'll work once [niche library I'm using] is in the training data"

But over the course of dozens of projects and hundreds of agent sessions, we kept arriving at the same conclusion: it's not a model problem. It's a configuration problem.

Yes, models will get smarter, and some existing failure modes will disappear. And then because they are smarter, we will give them new problems which are bigger and harder, and they will continue to fail in unexpected ways. Unexpected failures modes are a fundamental problem for non-deterministic systems.

So instead of praying for gpt-6.4-codex-ultrahigh_extended to save us all, we try to focus on answering the question of "how do we get the most out of today's models?"

There are lots of ways to get better performance out of your coding agent. If you use coding agents for moderately hard tasks, you've probably configured your coding agent a bit. Have you used skills? MCP servers? Sub-agents? Memory? AGENTS.md files?

coding agent = AI model(s) + harness

These are all technically separate concepts, but they are all part of the coding agent's configuration surface. We call this the coding agent's harness, and we think of it as the agent’s runtime, or as its peripherals: what does the model use to interact with its environment?

## Harness Engineering

Harness engineering, coined by Viv, describes the practice of leveraging these configuration points to customize and improve your coding agent's output quality and reliability.

Image from Viv's post

As Mitchell Hashimoto put it, harness engineering

[...] is the idea that anytime you find an agent makes a mistake, you take the time to engineer a solution such that the agent never makes that mistake again.

## ... as a Subset of Context Engineering

We view harness engineering as a subset of context engineering. Coined by my cofounder Dex in 12-factor agents, context engineering is a superset of “prompt engineering” and a variety of other techniques for systematically improving AI agents’ reliability. You can find the original talk on here.

Harness engineering then is the subset of context engineering which primarily involves leveraging harness configuration points to carefully manage the context windows of coding agents.

It answers:

How do we give our coding agent new capabilities?
How do we teach it things about our codebase that aren’t in the training data?
How do we add determinism beyond CRITICAL: always do XYZ in the system message?
How do we adapt the agent’s behavior for our specific codebase?
How do we increase task success rates beyond “magic prompts”?
How do we prevent our context window from inflating too rapidly, or with too much bad context?

Skills, MCP servers, sub-agents, hooks, and back-pressure mechanisms are all tactical solutions we’ve arrived at.

### Views on Harness Engineering

Viv’s posts on harness engineering are worth reading alongside this one — the first frames the four customization levers (system prompt, tools/MCPs, context, sub-agents), and the second works backwards from what models can’t do natively to derive why each harness component exists.

Image from Viv's post

We’d add two levers he doesn’t emphasize:

hooks for automated integration and deterministic control flow
skills for progressive disclosure of knowledge. (Dex likes to refer to them as "Instruction Modules" - more on this in another post.)

After months of solving hard problems in complex brownfield enterprise-scale codebases, we have found that sub-agents are a particularly powerful lever. When working on hard problems that require many, many context windows to solve, sub-agents are the key to maintaining coherency across many sessions. Sub-agents function as a "context firewall" that ensures discrete tasks can run in isolated context windows so none of the intermediate noise accumulates in your parent thread which is responsible for orchestration, and you can maintain coherency for much, much longer.

OpenAI recently wrote a blog post on the topic as well. There's some great content in there, and it seems to indicate that they view harness engineering as configuring everything outside of the agent's runtime. It's more focused on back-pressure and verification mechanisms. (Although this may be a mis-reading; the post is somewhat unclear: the word "harness" only appears once in the text of the post, and in reference to evals rather than harness engineering itself.)

### But What About Post-Training?

Given that frontier coding models are post-trained on their harnesses (e.g. Claude in Claude Code, GPT-5 Codex in Codex), some will argue that the best harness and/or configuration is the one that the model was trained on.

For example, the Codex models are so tightly coupled with the Codex harness's apply_patch tool that OpenCode — built as an open-source alternative to Claude Code — had to add an apply_patch tool specifically for GPT/Codex models to mimic the Codex harness to improve the Codex models' performance in the OpenCode harness - while Claude and other models still use normal edit and write tools.

This can mean that a model will perform better when coupled with the harness it was post-trained on, and some might infer that this means you shouldn't customize the harness at all.

But it cuts both ways: models can be over-fitted to their harness. Viv cites Terminal Bench 2.0 where Opus 4.6 in Claude Code comes in position #33, but when placed in a different harness that wasn't seen during post-training, it comes in at #5 (+/- about 4 positions in either direction).

## Engineering Your Harness

With that in mind, let's walk through the configuration surfaces we've found most impactful.

### CLAUDE.md & AGENTS.md

Before touching any other harness configuration points, it's usually worth customizing your CLAUDE.md / AGENTS.md files. These are markdown files at the top-level of your repository that get deterministically injected into the agent's system prompt by the harness.

We have already shared some opinions about what makes a good CLAUDE.md file and how to use it correctly, so give that a read if you're not familiar with it. Matt Pocock also wrote a great follow-up that more generally applies to AGENTS.md.

#### The ETH Zurich Study

After ETH Zurich published their study testing 138 agentfiles across various repos which indicated that most agentfiles were useless-or-worse, we got a lot of feedback on our CLAUDE.md post:

See, CLAUDE.md files don't even help — they're a waste of time.

Indeed: the study tested many agentfiles across a wide variety of repos, and found:

that LLM-generated ones actually hurt performance while costing 20%+ more
human-written ones only helped about 4%.
Agents spent 14-22% more reasoning tokens processing context file instructions, took more steps to complete tasks, and ran more tools — all without improving resolution rates.
Codebase overviews and directory listings didn't help at all; agents discover repository structure on their own just fine.

A careful reading of the study indicates that what we said in our post was correct:

Agent-generated files were worse. Yes, we said:

avoid auto-generating it. You should carefully craft its contents for best results.
Lots of files too-heavily-steered the model to use specific tools, causing worse outcomes. Yep, we said:

Less (instructions) is more. While you shouldn't omit necessary instructions, you should include as few instructions as reasonably possible in the file.
Files contained irrelevant context. Yep, we said:

Use Progressive Disclosure
The human-written ones barely helped because of too many conditional rules. Yep, we said:

Keep the contents of your CLAUDE.md concise and universally applicable.

Our CLAUDE.md is under 60 lines.

### MCP Servers Are for Tools

MCP servers are primarily for plugging tools into your coding agent to extend its capabilities beyond file I/O and bash commands. The MCP specification includes additional features like resources, prompts, and elicitations, but these are generally not well-supported by MCP clients and coding agent harnesses.

The MCP spec supports servers that run on your (or the agent's) local machine which allow the agent to interact with its local environment, but it also supports HTTP-based MCP servers that can connect your agent with remote tools and services like Linear, Sentry, and more.

When you plug an MCP server into your coding agent, the list of available tools, their descriptions, and the arguments needed to invoke them are injected into your coding agent's system prompt. As a result, the MCP server can use the tool descriptions to customize your agent’s behavior by providing your agent with instructions about when to use them.

WARNING: because MCP servers’ tool descriptions are added to your coding agent’s system prompt, never connect to one you don’t trust. This can be a dangerous vector for prompt injection! STDIO servers and other servers that run client-side with npx or uvx can also execute code on your host in the absence of prompt injection.

#### Too Many Tools Is Bad

We’ve seen this firsthand: plug too many MCP tools into your agent, and the context window fills up with tool descriptions, pushing you into the dumb zone much faster:

The instruction budget matters too — every irrelevant tool description is an instruction the agent has to process without any benefit.

In fact, these failure modes are so common that Anthropic released experimental support for MCP tool search to progressively disclose tools to Claude when the user has too many MCP tools connected. TL;DR: if you’re not actively using a server which provides a large number of tools, turn it off.

We also found that if an MCP server duplicates functionality that’s already available as a CLI well-represented in training data, it works better to just prompt the agent to use the CLI. For things like GitHub, Docker, or most databases, your coding agent can just use the right CLIs and shell commands. The model has seen these tools enough during training that it already knows how to use them, and you gain the added benefit of composability with tools like grep and jq to enable additional context-efficiency.

#### Always Be Context-Engineering

At HumanLayer, we used the Linear MCP server for a while before realizing that we really only used a small subset of the tools it provides - so we wrote a small CLI that wraps the Linear API and provides very context-efficient responses, and we included 6 example usages in our CLAUDE.md file:

## Linear

Use the Linear CLI for:

- **fetching issues**: `linear get-issue ENG-XXXX`
- **listing issues**: `linear list-issues` or `linear my-issues`
- **adding comments**: `linear add-comment -i ENG-XXXX "comment"`
- **adding links**: `linear add-link ENG-XXXX "url" -t "link title"`
- **updating status**: `linear update-status ENG-XXXX "status name"` ("spec needed", "research needed", "research in progress", "research in review", "ready for plan", "plan in progress", "plan in review", "ready for dev", "in dev", "code review")
- **get branch name**: `linear get-issue-v2 ENG-XXXX --fields branch` (use this when creating a worktree)
- **get images from the ticket**: `linear fetch-images ENG-XXXX` (do this any time the ticket has images in the description)

This saved us thousands of tokens from the MCP server's tool definitions that were ending up in our agent’s system prompt, and many more from the verbose MCP server responses.

### Skills Are for Reusable Knowledge (and Tools)

Skills were originally introduced by Anthropic for use with Claude Code, but have since become an open standard supported by other harnesses like Codex and OpenCode. You can read about how they're structured in the Anthropic docs — what matters here is why they're useful.

Before we go further: skill registries have already been caught distributing hundreds of malicious skills. Treat skills like you'd treat npm install random-package — read what you're installing. Registries like ClawHub and skills.sh can execute arbitrary code on your machine.

#### Progressive Disclosure

We learned this one early: we kept stuffing every instruction and tool into the system prompt, and the agent kept getting worse. We were blowing through our instruction budget before the agent even started working. Skills solve this through progressive disclosure — the agent only gets access to specific instructions, knowledge, or tools when it decides (or you decide for it) that it needs them.

#### Skill Activation

When a skill is activated, the SKILL.md file in the skill's directory is loaded into the agent's context window as a user message, and the agent is informed of the directory that the skill file was loaded from. The SKILL.md file may inform the agent of anything else it's bundled with:

example-skill/
|--- SKILL.md
|--- response_template.md
|--- CLIs/
   |--- linear-cli
   |--- tunnel-cli

Since each skill has its own directory, you can get even more creative with progressive disclosure: you might bundle several markdown files in the skill, each of which contains different information about different functionalities or for different purposes, and the main SKILL.md file can tell the agent what the other files in the skill are and if/when it should read them.

#### Distributing Tools with Skills

Unfortunately, it's not possible to bundle MCP servers or custom agent tools directly into a skill - you have to write them into an executable, a CLI, an NPM package, or something else which you can either distribute with your skill or instruct the agent to install in the skill file.

For example, instead of configuring a Playwright MCP server, you could just provide your agent with a skill for web browsing using BrowserBase's agent browser skills or Vercel's agent browser CLI.

### Sub-Agents Are for Context Control

Sub-agents are a popular but often misunderstood harness configuration point. We tried the "frontend engineer" sub-agent and "backend engineer" sub-agent and "data analyst" sub-agent thing. It doesn't work. What does work is using sub-agents for context control.

They provide a way to encapsulate an entire coding agent session's worth of work such that the dispatching agent only sees the prompt it writes for the sub-agent, and the sub-agent's final result. None of the intermediate tool calls, tool results, or other messages end up in the parent coding agent's context window.

Breaking work up into discrete tasks and delegating it to sub-agents is how we keep our primary coding agent thread in the "smart zone." This is how we handle research, implementation, and a number of other context-heavy tasks in our day-to-day workflows.

#### Sub-Agents Avoid Context Rot

Chroma's context rot research provides empirical backing to what we've been saying for a long time: models perform worse at longer context lengths. Avoid the dumb zone.

Chroma researchers tested 18 models on needle-in-a-haystack tasks, which is admittedly quite different from agentic coding. But the finding matches our experience exactly: performance degrades as context length increases — even on simple tasks.

Worse, when there's low semantic similarity between the question and the relevant information in context, the degradation is steeper. We’ve watched this happen in real time: every intermediate tool call, every grep result, every file read in the parent session that doesn’t end up being relevant is a potential distractor, and the Chroma research confirms that distractor effects compound at longer context windows.

#### Aside: Long-Context Models

This is also why we're skeptical of the "just make the context window bigger" approach to coding agents. When a lab offers an extended-context version of a given model, you are usually not getting a bigger model with a larger "instruction budget" - you're getting the same model with some clever math (e.g. YaRN) to extend the length of the sequence the model can attend to.

Consider the needle-in-a-haystack problem. A bigger context window doesn't make the model better at finding the needle — it just makes the haystack bigger. For our purposes, it means you can stuff more instructions (each user message is at least one instruction, and usually several) into the context window - putting you deeper and deeper into the "dumb zone".

If you think you need longer context, you may just need better context window isolation. Sub-agents solve this structurally: each one gets a fresh, small, high-relevance context window with a fresh "instruction budget" for its task, and only the condensed result flows back to the parent - allowing you to stitch together many context windows for a single problem.

The limit case probably looks something like this, although at some point you're crossing Recursive Language Model territory.

(Note: some arrows omitted for brevity.)

#### Sub-Agent Use-Cases

Great examples of things to use sub-agents for include:

Locating specific definitions or implementations in the codebase
Analyzing the codebase to identify patterns for a specific type of work
Tracing the flow of information through the codebase, e.g. tracing a request across service boundaries
Other general code/documentation/web research tasks

These types of tasks often have a straightforward question and simple answer, but require lots of intermediate tool calls that you don't want or need in your parent session. Sub-agents should return highly condensed responses that also follow the principle of progressive disclosure. For example, our sub-agents provide an answer to the question but also cite sources in filepath:line format or with URLs so that the parent agent isn't exposed to all the sources the sub-agent used, but if it needs more details or confirmation, it has the information that it needs to go find the relevant context:

Dex spoke about this more extensively here.

Claude Code and some other coding agents even provide built-in, task-specific sub-agents, e.g. Claude Code's Explore sub-agent for codebase exploration, or their Bash sub-agent which is designed to execute verbose bash commands and extract information to return to the parent agent without polluting its context. Other coding agents support sub-agents but don't define their own, and require that the user manually configure them if desired.

#### Sub-Agents Are (Also) for Cost Control

Sub-agents can also help with cost control. We use an expensive model (Opus) for the parent session where thinking-heavy tasks like planning and orchestration happen, and a cheaper, faster model like Sonnet or Haiku for each sub-agent. Sub-agents receive much smaller and more discrete tasks that can be handled by a less-intelligent model with a smaller "instruction budget" — no need to burn Opus tokens on a codebase grep.

#### So We Heard You Like Sub-Agents

Some harnesses don't support sub-agents at all! Even Codex didn't until recently, and support is still experimental.

Fortunately you can still use this powerful context encapsulation pattern by writing an MCP server that provides a tool for launching a new agent session which receives a prompt from the parent agent, launches a new coding agent session with that prompt as the user message, and which returns the sub-agent's final response message to the parent.

A very rough approximation of a server to do this can be found here. Warning: using this pattern with a coding agent that supports sub-agents will allow the harness's native sub-agents to dispatch sub-agents via MCP. This can result in an unpredictable game of telephone:

Jokes aside, practically you have to be very careful when you write your sub-agents' system prompts to carefully specify the scope of their role:

What is the agent's role - what should it do, but also what should it not do
What information should the agent return, and how should it return it
What tools should the sub-agent have?

It's also worth noting that many harnesses have an MCP tool call timeout - if you implement this pattern, you may need to configure your harness to increase the MCP tool call timeout.

### Hooks Are for Control Flow

Claude Code has the concept of hooks: user-defined commands or scripts that are automatically executed when certain events occur and at various points of the agent's lifecycle. Similarly, Opencode has the concept of plugins which do the same thing. Other coding agents may have similar configuration points. (Sadly, Codex doesn't have an equivalent.)

Hooks are conceptually similar to git hooks, but they're quite flexible. They can be used to add new features, integrate with external services, automate routine actions, modify permissions, and configure default behavior.

Implementation details vary between harnesses, but generally a hook can:

run something automatically but silently when an event occurs
run when a tool is called, and return additional context to the agent in addition to the tool result
surface build/type errors to a coding agent before it finishes, so that it's forced to keep working until it resolves the error

Common use cases include...

Notifications: We wire our agents up to play sounds when they finish or when they need attention (e.g. an approval has been pending for too long)
Approvals: We automatically approve or deny tool calls based on input values and more expressive rules than the coding agent’s default permissions model. For example, we automatically deny any Bash() tool calls that try to run migrations, with an instruction to ask the user to run them instead.
Integrations: We have our agents send a Slack message when they’re finished, create a GitHub PR, or set up a preview environment.
Verification: If your framework and repository can run a typecheck or build in under a handful of seconds, run it every time the agent stops to surface errors to it — this is exactly what the hook below does.

#### An Example Hook

Here's a hook I (via Claude) wrote for our repo which implements the last example I gave. When Claude stops, it runs our biome formatter and TypeScript type checks. If there are errors, they are raised to Claude. If not, the script exits silently.

#!/bin/bash
cd "$CLAUDE_PROJECT_DIR"

# prebuild generates types and builds internal SDK packages so typecheck has
# everything it needs. runs bun install afterward to pick up any new generated files.
PREBUILD_OUTPUT=$(bun run generate-cache-key && turbo run build --filter=@humanlayer/hld-sdk && bun install 2>&1)
if [ $? -ne 0 ]; then
  echo "prebuild failed:" >&2
  echo "$PREBUILD_OUTPUT" >&2
  exit 2
fi

# biome and typecheck run in parallel to keep the feedback loop tight.
# one quirk: biome --write exits with code 1 if it made any changes, even if it
# successfully fixed everything. so we run it twice with ||: if the first pass
# makes changes and exits 1, the second pass will exit 0 since there's nothing
# left to fix. if there are unfixable errors, both passes fail and exit 2.
OUTPUT=$(bun run --parallel \
  "biome check . --write --unsafe || biome check . --write --unsafe" \
  "turbo run typecheck" 2>&1)

if [ $? -ne 0 ]; then
  echo "$OUTPUT" >&2
  exit 2
fi

On success the hook is completely silent — nothing ends up in the agent's context. On failure, only the errors are surfaced, and exit code 2 tells the harness to re-engage the agent so it fixes them before finishing.

## Back-Pressure Increases Your Chances of Success

We've written about back-pressure before. The core insight is that your likelihood of successfully solving a problem with a coding agent is strongly correlated with the agent's ability to verify its own work. We've spent a lot of time building out tests and other back-pressure mechanisms into our repository, and it remains one of the highest-leverage things we have spent time on.

Our codebase has verification mechanisms that allow the agent to check its own work:

typechecks and build steps (probably in a strongly-typed language)
unit tests and/or integration tests
code coverage reporting (we have a Stop hook that prompts the agent to increase coverage if it drops)
UI interaction and testing integrations (playwright, agent-browser, etc)

Critically, these verification mechanisms need to be context-efficient. We learned this one the hard way — early on we had our agent run the full test suite after every change, and 4,000 lines of passing tests would flood the context window. The agent would then lose track of the actual task and start hallucinating about test files it had just read. Now we swallow the output and only surface errors. We do the same with builds — success is silent, and only failures produce verbose output.

We give Claude concise instructions about how to use all of these mechanisms in our CLAUDE.md file. Some are even bundled inside skills for progressive disclosure.

## Closing Notes

It is entirely possible to spend more time optimizing your coding agent setup than actually shipping code with it — we’ve been there.

Our approach: bias towards shipping. We only spend time on harness configuration to the extent that it’s actually enabling us to ship more high-quality code faster. When the agent fails, we take the time to engineer a solution so it doesn’t fail that way again — but we don’t go looking for problems to solve preemptively.

What didn’t work for us:

Trying to design the ideal harness configuration upfront before we’d even hit real failures
Installing dozens of skills and MCP servers "just in case"
Running our entire test suite (5+ minutes) at the end of every agent session (run a subset instead)
Trying to micro-optimize which sub-agents could access which tools. This led to a lot of tool thrash which gave us worse results, not better ones. Most coding agents don't have a robust configuration surface for this anyways.

What did work:

Starting simple and adding configuration only when the agent actually failed
Designing, testing, iterating — and throwing away things that didn’t help. I have thrown away many more hooks than we actually use today.
Distributing battle-tested configurations to the whole team via repository-level config
Optimizing for iteration speed, not "likelihood of 1-shotting it on the first attempt"
Giving the agent a set of capabilities (Linear) and then carefully paring down what we exposed to the model once we knew what we needed.

The next time your coding agent isn’t performing the way you expect, before you blame the model, check the harness. Agentfiles, MCP servers, skills, sub-agents, hooks, and back-pressure — that’s where we’ve found most of the leverage. The model is probably fine. It’s just a skill issue.

A Brief History of Ralph

Dex — Mon, 05 Jan 2026 16:00:00 GMT

The Ralph Wiggum Technique, created by Geoff Huntley, went viral in the final weeks of 2025. Here's the story of ralph since the first time I met Geoff in June of 2025.

I've been messing with ralph since ~June 2025. Here's my story and what I learned along the way.

tl;dr

Jan 1 2026 - If you wanna skip to the end, I did a deep dive on ralph w/ Geoff Huntley on Jan 1 2026. It talks through the history, cursed lang, and compares the original bash-loop ralph implementation with the anthropic stop-hook implementation. You can check it out here:

### June 19th 2025

I attend a meetup with about 15 members of a Twitter GC where we talk about agentic coding. It's the first time I see context7, WisprFlow, specstory, taskmaster, and a whole bunch of other tools and addons, some of which are now quite mainstream. One of our engineers demos an early TUI for Claude approvals and what becomes the foundation of research / plan / implement.

There are about 3 hours of presentations. Geoff shows up 2 hours late and presents last. He completely steals the show, diving deep on ralph, cursed lang (at the time, the compiler stack is written in Rust), livestreaming autonomous coding overnight while asleep in Australia, subagents in amp code, the virtues of drinking 3 margaritas and shouting at cursor, and much, much more.

Geoff talks about the "overbaking" phenomenon. If you leave ralph running too long, you end up with all sorts of bizarre emergent behavior, like post-quantum cryptography support.

It has dimensions of art, deep engineering, the embrace of chaos, and the raw and authentic joy of making a thing.

All ~15 of us have a long and (imo) somewhat unsettling conversation about the future of software dev—about how easy it is to take a SaaS and copy 80-90% of it, and about how many types of work are about to change or disappear entirely.

### July 2025

Geoff Launches ralph in an official blog post.

It includes the basic bash loop structure:

while :; do cat PROMPT.md | npx --yes @sourcegraph/amp ; done

I send it to everyone I know. I hint at it to a few others.

It includes a very lightweight example of one of the prompts (this one came later in the language dev, as far as I can tell, since its about the stdlib)

At the time, it included an ask (paraphrasing here as it appears to be gone now)

you could probably find the cursed lang repo on github if you looked for it, but please don't share it yet

This is exciting.

### August 2025 - Advanced Context Engineering for Coding agents

If you were early to the coding agents poweruser game, you might have seen Getting AI to work in complex codebases on hackernews and the youtube talk it was based on

It references ralph and why its such an important example of why engineering your context window is such a high-leverage activity.

Ralph is also a great example of using declarative specifications over imperative instructions.

### August 2025 - hacking productivity tools

I love hacking on Getting Things Done (GTD) and other productivity systems. I for whatever reason am still stuck on omnifocus which is an ancient but well-honed implementation of the system. But I have always longed for an AI native version.

I wrote a half-pager on what I wanted, I had ralph write the specs from that, and I had another ralph build the implementation from the specs.

I didn't really read the specs, and I didn't read the code.

The output sucked. I went back to read the specs and they were way off base.

That's fine it was a saturday morning toy thing, but I learned stuff.

Lessons:

if the specs are bad, the results will be meh
if you don't actually know your desired end state workflows and how you will test it, you probably won't know what to do when its done
if you are iterating/exploring, you probably don't want ralph in the first place

### August 2025 - we put a coding agent in a while loop and it shipped 6 repos overnight

You may recall this HN discussion - we put a coding agent in a while loop and it shipped 6 repos overnight - here's the original post simon and I wrote up at repomirrorhq/repomirror

### August 2025 - testing ralph for a refactor

I did a few experiments with ralph for side projects, but the most interestinng one was this one.

An engineer was complaining that the frontend code was messy and was hard to work in, sprawling patterns, long components, etc
I spent about 30 minutes back and forth with claude developing a REACT_CODING_STANDARDS.md
I spent another 30 minutes working with the engineer (who has a lot more react experience than I do) to refine and update this doc
I launched a ralph with a prompt "make sure the code base matches the standards" prompt.md
Over the next 6 hours, it developed a REACT_REFACTOR_PLAN.md and blasted through the whole thing
I checked in after 6 hours and it claimed to be finished, so I stopped it and sent it over to team for a look.
Overall the feedback was positive, but the PR quickly got some merge conflicts and so we never ended up merging it - you can view the closed PR here

Lesson - code is cheap - the easier alternative to "merge/rebase" is just to re-run the ralph loop on the fresh code with the same prompt and re-open a PR.

Lesson - for existing codebases, make the change set manageable - we have since set up any ralph-ish desired state loops to once ONCE on a cron overnight, and merge small iterations over time. Waking up to one small refactor every morning is better than both a) waking up to none and b) waking up to 50.

### September 2025 - cursed lang launch

In September 2025, Geoff launches cursed lang officially, the programming language that ralph built. Once in C, once in rust, and then finally in zig. It has a standard library and a stage-2 compiler (cursed lang compiler written in cursed lang).

Check out the Cursed Lang Homepage

Aside - I want to do a cursed lang hackathon next time Geoff is in SF - stay tune or ping me on X if this is something you're be interested in.

### October 2025 - Claude Anonymous SF

I did a 5-minute ralph presentation at claude code anonymous in sf, in a room full of some of the most creative and enterprising claude code and codex users in town. steipete came all the way out from Austria to co-host with the Sentry folks.

It's hard to capture the glory of ralph in a 5 minute lightning talk. But even if all the context engineering concepts get missed, its always a fun time because of the deep art Geoff put into it the naming and branding.

Highlight was a question from the audience "so do you recommend this?" - my answer: "well, I believe you should try everything". Perhaps the point is not the 5-line bash loop. Perhaps the point is dumb things can work surprisingly well, so what could we expect from a smart version of the thing?

### October 2025 - AI That Works w/ Geoff Huntley

I was annoyed that I couldn't really do ralph justice in 5 minutes, so I wrangled Vaibhav and Geoff to do a 75-minute podcast deep diving on ralph and why it works.

We talk a lot about context windows, control loops, and various applications of the technique - refactoring, spec generation, new project setup, etc.

It was a fun time. You can watch it here:

There's some code samples here

### December 2025 - Anthropic Plugin Launches

At some point the anthropic team released an official ralph wiggum plugin.

Many people tweeted at Geoff about it.

I test it out. I am disappointed. It dies in cryptic ways unless you have --dangerously-skip-permissions.

It installs hooks in weird places you can't find, uses a strange markdown file to track state, and adds an opaque stop hook for all claude sessions in a directory until you disable it. If you, in trying to stop it, delete the markdown file before stopping it, you break claude in that repo until you disable the plugin entirely.

Beyond that, it misses the key point of ralph which is not "run forever" but in "carve off small bits of work into independent context windows".

I look forward to see more from Anthropic on this in the future, but for now I'll keep my 5-line bash loops.

### December 2025 - Youtube Slop Everywhere

There were quickly ralph wiggum videos everywhere. Most of them are typical youtube ai hype-slop:

I actually like Matt Pockock's youtube overview because its true to the OG technique, and Matt does a good job of grounding the technique in a workflow like kanban, requirements discovery, etc.

### Jan 1 2026 - Ralph Wiggum Showdown

So much ralph hype, I decided to give the official plugin another shot. I posted this graphic on twitter:

To my pleasant surprise, Geoff texted me and was like "okay I can jump on for the first 30 minutes or so".

So here ya go, a bonafide ralph wiggum yap, the whole story, why it works, with whiteboards and live examples for a side project I'm working on called kustomark.

Here's the repos that got built. I still haven't gotten around to evaluating which one, if any, actually solves my problem:

### What's next

I still need to do a deep dive on what was built on the showdown stream. That's coming in the next few days, hopefully, because the kustomark plugin is a thing I actually want+need for some projects we're exploring.

You should mess with this stuff. It's not exactly the future, but its an imporant slice through a bunch of important concepts that will make you a better Agentic Coder and a better AI Engineer.

### Okay one more thing

A gift and a shameless plug - a lot of you are sitting around on the waitlist wondering "when the heck is codelayer gonna actually launch". Quick update there - we're still working hard in the trenches with customers, and learning a lot that has led us to reexamine the entire product. However I am very proud of the OG codelayer and we have many many very happy users on the free private beta. So -- with the disclaimer that we will likely sunset the current beta in favor of a new thing we're launching soon -- if you read this far and you wanna try codelayer - check out the docs at https://hlyr.dev/docs and have fun.

Excited to share what's next.

In the mean time, happy ralphing. Hurry up before it gets semantically diffused.

-dex

### PPS

We're hiring yada yada come build cool shit

Context-Efficient Backpressure for Coding Agents

Dex — Mon, 08 Dec 2025 16:00:00 GMT

Here's a pattern we use constantly at HumanLayer: swallow all test/build/lint output and replace it with a single ✓ if the stage passes.

If exitCode != 0, dump the stashed output to stdout. Otherwise, the agent never sees it.

## stay in the smart zone

You should try hard to stay in the ~75k token "smart zone" for claude models - every line of PASS src/utils/helper.test.ts is waste.

A jest/maven/pytest run might dump 200+ lines of output, and then your agent has to parse all of that to find the one failure it actually needs to fix.

Or worse, if all tests are passing, you just threw away 2-3% of your context window for an "all good" result you could have conveyed in less than 10 tokens.

You're wasting context - every token you use is diminishing the results and moves you closer to "need to clear or compact to get back to the smart zone".

And you should forget the token cost and time-it-takes-to-run concerns for now (we'll come back to time in a second), since human time wasted on wrangling an agent in the dumb zone is likely more expensive by 10x or more.

## a pretty dumb fix

Not a full fix, but in the right direction:

Rather than letting the model decide what to truncate, we like to take control of output deterministically.

run_silent() {
    local description="$1"
    local command="$2"
    local tmp_file=$(mktemp)

    if eval "$command" > "$tmp_file" 2>&1; then
        printf "  ✓ %s\n" "$description"
        rm -f "$tmp_file"
        return 0
    else
        local exit_code=$?
        printf "  ✗ %s\n" "$description"
        cat "$tmp_file"
        rm -f "$tmp_file"
        return $exit_code
    fi
}

Invoke it with

run_silent "fe integration tests" "bun run test:integration"

Instead of 200 lines of test output, the agent sees:

  ✓ Auth tests
  ✓ Utils tests
  ✗ API tests

  FAIL src/api/users.test.ts
  ● should validate email format
    Expected: true
    Received: false

The model doesn't need to decide whether to truncate. You've already made that decision. Success = ✓. Failure = full output.

Now your only job is to convince the model not to do its own truncation. Maybe you can shout at it in your claude.md.

## make it smaller over time

Once you have the basic wrapper working, iterate:

Enable failFast. pytest -x, jest --bail, go test -failfast. One failure at a time. Don't make the agent context-switch between five different bugs, or re-read the same "tests 2-5 are failing" output when its still trying to fix test #1.

Filter the output. Strip generic stack frames that don't help your agent find the issue. Remove timing info. grep/sed/awk/cut/etc your way to just the assertion that failed.

Framework-specific parsing. even a pretty-hacky run_silent.sh extracts test counts from pytest, jest, go test, and vitest so you still get visibility without the noise.

We use this pattern heavily when working with customers who have Maven and Gradle projects (notoriously verbose), and it works equally well for xcodebuild, cargo, and anything else that spews.

Here's the humanlayer/humanlayer checks and tests for a monorepo - we run these in a pre-push hook, and some subset of these projects for each plan phase.

This would easily be half a context window worth of output without the wrappers.

## im tired of context-anxious models

The latest generation of models has spent so much time in the RL dungeon that they've overcorrected way too far on being conservative with how much context they load.

here's some patterns I've seen in the last few months:

#### output swallowing

piping output to /dev/null and using an || operator to just print a message based on the commands exit code

while its cool that we're being conservative, remember what cat would have printed without the output swallowing:

In this trivial example, the guardrails actually use MORE TOKENS:

Disclaimers - 1) yes if cat had repeated the whole filename, that would have been more tokens. 2) this is an unofficial tokenizer and probably out of date

but you get my point here - the real nasty one comes next -

#### piping to head / tail

I hear so many complaints about "oh it ran a 5 minute test suite with head -n 50 and now it has to run it again"

venv/bin/python -m pytest -n 4 | head -n 50

If your tests take less than 30 seconds, great you probably don't care, but that's not the case for most of us. And again, we're gonna focus on human time wasted while the agent runs and re-runs test suites, or wasted on compacting/restarting a context window because too many tokens got used.

Ironically, in trying to conserve context and serve us better, models end up burning more tokens, more human time, and more mental energy.

I won't opine on why the product teams that focus on model behavior have leaned into this behavior, but I can only assume its at least in-broad-strokes intentional. It helps models perform well in codebases that are not thoughtful about context-efficiency of their backpressure mechanisms

## its not your fault

Okay fine, opining time: things are this way because labs have to take big swings and the majority of their potential user base doesn't know how (or want to learn how) to do context engineering well. Sure, "don't make me think". But also, let me think if I want to.

As we always say - deterministic is better than non-deterministic. If you already know what matters, don't leave it to a model to churn through 1000s of junk tokens to decide.

Writing a good CLAUDE.md

Kyle — Mon, 24 Nov 2025 16:00:00 GMT

Note: this post is also applicable to AGENTS.md, the open-source equivalent of CLAUDE.md for agents and harnesses like OpenCode, Zed, Cursor and Codex.

## Principle: LLMs are (mostly) stateless

LLMs are stateless functions. Their weights are frozen by the time they're used for inference, so they don't learn over time. The only thing that the model knows about your codebase is the tokens you put into it.

Similarly, coding agent harnesses such as Claude Code usually require you to manage agents' memory explicitly. CLAUDE.md (or AGENTS.md) is the only file that by default goes into every single conversation you have with the agent.

This has three important implications:

Coding agents know absolutely nothing about your codebase at the beginning of each session.
The agent must be told anything that's important to know about your codebase each time you start a session.
CLAUDE.md is the preferred way of doing this.

## `CLAUDE.md` onboards Claude to your codebase

Since Claude doesn't know anything about your codebase at the beginning of each session, you should use CLAUDE.md to onboard Claude into your codebase. At a high level, this means it should cover:

WHAT: tell Claude about the tech, your stack, the project structure. Give Claude a map of the codebase. This is especially important in monorepos! Tell Claude what the apps are, what the shared packages are, and what everything is for so that it knows where to look for things
WHY: tell Claude the purpose of the project and what everything is doing in the repository. What are the purpose and function of the different parts of the project?
HOW: tell Claude how it should work on the project. For example, do you use bun instead of node? You want to include all the information it needs to actually do meaningful work on the project. How can Claude verify Claude's changes? How can it run tests, typechecks, and compilation steps?

But the way you do this is important! Don't try to stuff every command Claude could possibly need to run in your CLAUDE.md file - you will get sub-optimal results.

## Claude often ignores `CLAUDE.md`

Regardless of which model you're using, you may notice that Claude frequently ignores your CLAUDE.md file's contents.

You can investigate this yourself by putting a logging proxy between the claude code CLI and the Anthropic API using ANTHROPIC_BASE_URL. Claude code injects the following system reminder with your CLAUDE.md file in the user message to the agent:

<system-reminder>
      IMPORTANT: this context may or may not be relevant to your tasks. 
      You should not respond to this context unless it is highly relevant to your task.
</system-reminder>

As a result, Claude will ignore the contents of your CLAUDE.md if it decides that it is not relevant to its current task. The more information you have in the file that's not universally applicable to the tasks you have it working on, the more likely it is that Claude will ignore your instructions in the file.

Why did Anthropic add this? It's hard to say for sure, but we can speculate a bit. Most CLAUDE.md files we come across include a bunch of instructions in the file that aren't broadly applicable. Many users treat the file as a way to add "hotfixes" to behavior they didn't like by appending lots of instructions that weren't necessarily broadly applicable.

We can only assume that the Claude Code team found that by telling Claude to ignore the bad instructions, the harness actually produced better results.

## Creating a good `CLAUDE.md` file

The following section provides a number of recommendations on how to write a good CLAUDE.md file following context engineering best practices.

Your mileage may vary. Not all of these rules are necessarily optimal for every setup. Like anything else, feel free to break the rules once...

you understand when & why it's okay to break them
you have a good reason to do so

### Less (instructions) is more

It can be tempting to try and stuff every single command that claude could possibly need to run, as well as your code standards and style guidelines into CLAUDE.md. We recommend against this.

Though the topic hasn't been investigated in an incredibly rigorous manner, some research has been done which indicates the following:

Frontier thinking LLMs can follow ~ 150-200 instructions with reasonable consistency. Smaller models can attend to fewer instructions than larger models, and non-thinking models can attend to fewer instructions than thinking models.
Smaller models get MUCH worse, MUCH more quickly. Specifically, smaller models tend to exhibit an expotential decay in instruction-following performance as the number of instructions increase, whereas larger frontier thinking models exhibit a linear decay (see below). For this reason, we recommend against using smaller models for multi-step tasks or complicated implementation plans.
LLMs bias towards instructions that are on the peripheries of the prompt: at the very beginning (the Claude Code system message and CLAUDE.md), and at the very end (the most-recent user messages)
As instruction count increases, instruction-following quality decreases uniformly. This means that as you give the LLM more instructions, it doesn't simply ignore the newer ("further down in the file") instructions - it begins to ignore all of them uniformly

Our analysis of the Claude Code harness indicates that Claude Code's system prompt contains ~50 individual instructions. Depending on the model you're using, that's nearly a third of the instructions your agent can reliably follow already - and that's before rules, plugins, skills, or user messages.

This implies that your CLAUDE.md file should contain as few instructions as possible - ideally only ones which are universally applicable to your task.

### `CLAUDE.md` file length & applicability

All else being equal, an LLM will perform better on a task when its' context window is full of focused, relevant context including examples, related files, tool calls, and tool results compared to when its context window has a lot of irrelevant context.

Since CLAUDE.md goes into every single session, you should ensure that its contents are as universally applicable as possible.

For example, avoid including instructions about (for example) how to structure a new database schema - this won't matter and will distract the model when you're working on something else that's unrelated!

Length-wise, the less is more principle applies as well. While Anthropic does not have an official recommendation on how long your CLAUDE.md file should be, general consensus is that < 300 lines is best, and shorter is even better.

At HumanLayer, our root CLAUDE.md file is less than sixty lines.

### Progressive Disclosure

Writing a concise CLAUDE.md file that covers everything you want Claude to know can be challenging, especially in larger projects.

To address this, we can leverage the principle of Progressive Disclosure to ensure that claude only sees task- or project-specific instructions when it needs them.

Instead of including all your different instructions about building your project, running tests, code conventions, or other important context in your CLAUDE.md file, we recommend keeping task-specific instructions in separate markdown files with self-descriptive names somewhere in your project.

For example:

agent_docs/
  |- building_the_project.md
  |- running_tests.md
  |- code_conventions.md
  |- service_architecture.md
  |- database_schema.md
  |- service_communication_patterns.md

Then, in your CLAUDE.md file, you can include a list of these files with a brief description of each, and instruct Claude to decide which (if any) are relevant and to read them before it starts working. Or, ask Claude to present you with the files it wants to read for aproval first before reading them.

Prefer pointers to copies. Don't include code snippets in these files if possible - they will become out-of-date quickly. Instead, include file:line references to point Claude to the authoritative context.

Conceptually, this is very similar to how Claude Skills are intended to work, although skills are more focused on tool use than instructions.

### Claude is (not) an expensive linter

One of the most common things that we see people put in their CLAUDE.md file is code style guidelines. Never send an LLM to do a linter's job. LLMs are comparably expensive and incredibly slow compared to traditional linters and formatters. We think you should always use deterministic tools whenever you can.

Code style guidelines will inevitably add a bunch of instructions and mostly-irrelevant code snippets into your context window, degrading your LLM's performance and instruction-following and eating up your context window.

LLMs are in-context learners! If your code follows a certain set of style guidelines or patterns, you should find that armed with a few searches of your codebase (or a good research document!) your agent should tend to follow existing code patterns and conventions without being told to.

If you feel very stronly about this, you might even consider setting up a Claude Code Stop hook that runs your formatter & linter and presents errors to Claude for it to fix. Don't make Claude find the formatting issues itself.

Bonus points: use a linter that can automatically fix issues (we like Biome), and carefully tune your rules about what can safely be auto-fixed for maximum (safe) coverage.

You could also create a Slash Command that includes your code guidelines and which points claude at the changes in version control, or at your git status, or similar. This way, you can handle implementation and formatting separately. You will see better results with both as a result.

### Don't use `/init` or auto-generate your `CLAUDE.md`

Both Claude Code and other harnesses with OpenCode come with ways to auto-generate your CLAUDE.md file (or AGENTS.md).

Because CLAUDE.md goes into every single session with Claude code, it is one of the highest leverage points of the harness - for better or for worse, depending on how you use it.

A bad line of code is a bad line of code. A bad line of an implementation plan has the potential to create a lot of bad lines of code. A bad line of a research that misunderstands how the system works has the potential to result in a lot of bad lines in the plan, and therefore a lot more bad lines of code as a result.

But the CLAUDE.md file affects every single phase of your workflow and every single artifact produced by it. As a result, we think you should spend some time thinking very carefully about every single line that goes into it:

## In Conclusion

CLAUDE.md is for onboarding Claude into your codebase. It should define your project's WHY, WHAT, and HOW.
Less (instructions) is more. While you shouldn't omit necessary instructions, you should include as few instructions as reasonably possible in the file.
Keep the contents of your CLAUDE.md concise and universally applicable.
Use Progressive Disclosure - don't tell Claude all the information you could possibly want it to know. Rather, tell it how to find important information so that it can find and use it, but only when it needs to to avoid bloating your context window or instruction count.
Claude is not a linter. Use linters and code formatters, and use other features like Hooks and Slash Commands as necessary.
CLAUDE.md is the highest leverage point of the harness, so avoid auto-generating it. You should carefully craft its contents for best results.

Advanced Context Engineering for Coding Agents

Dex — Thu, 28 Aug 2025 16:00:00 GMT

## Getting AI to Work in Complex Codebases

It seems pretty well-accepted that AI coding tools struggle with real production codebases. The Stanford study on AI's impact on developer productivity found:

A lot of the "extra code" shipped by AI tools ends up just reworking the slop that was shipped last week.
Coding agents are great for new projects or small changes, but in large established codebases, they can often make developers less productive.

The common response is somewhere between the pessimist "this will never work" and the more measured "maybe someday when there are smarter models."

After several months of tinkering, I've found that you can get really far with today's models if you embrace core context engineering principles.

This isn't another "10x your productivity" pitch. I tend to be pretty measured when it comes to interfacing with the ai hype machine. But we've stumbled into workflows that leave me with considerable optimism for what's possible. We've gotten claude code to handle 300k LOC Rust codebases, ship a week's worth of work in a day, and maintain code quality that passes expert review. We use a family of techniques I call "frequent intentional compaction" - deliberately structuring how you feed context to the AI throughout the development process.

I am now fully convinced that AI for coding is not just for toys and prototypes, but rather a deeply technical engineering craft.

Video Version: If you prefer video, this post is based on a talk given at Y Combinator on August 20th

### Grounding Context from AI Engineer

Two talks from AI Engineer 2025 fundamentally shaped my thinking about this problem.

The first is Sean Grove's talk on "Specs are the new code" and the second is the Stanford study on AI's impact on developer productivity.

Sean argued that we’re all vibe coding wrong. The idea of chatting with an AI agent for two hours, specifying what you want, and then throwing away all the prompts while committing only the final code… is like a Java developer compiling a JAR and checking in the compiled binary while throwing away the source.

Sean proposes that in the AI future, the specs will become the real code. That in two years, you'll be opening python files in your IDE with about the same frequency that, today, you might open up a hex editor to read assembly (which, for most of us, is never).

Yegor's talk on developer productivity tackled an orthogonal problem. They analyzed commits from 100k developers and found, among other things,

(1) That AI tools often lead to a lot of rework, diminishing the perceived productivity gains

(2) That AI tools work well for greenfield projects, but are often counter-productive for brownfield codebases and complex tasks

This matched what I heard talking with founders:

“Too much slop.”
“Tech debt factory.”
“Doesn’t work in big repos.”
“Doesn’t work for complex systems.”

The general vibe on AI-coding for hard stuff tends to be

Maybe someday, when models are smarter…

Heck even Amjad was on a lenny's podcast 9 months ago talking about how PMs use Replit agent to prototype new stuff and then they hand it off to engineers to implement for production. (Disclaimer: i haven't caught up with him recently (ok, ever), this stance may have changed)

Whenever I hear "Maybe someday when the models are smart" I generally leap to exclaim that's what context engineering is all about: getting the most out of today's models.

### What's actually possible today

I'll deep dive on this a bit futher down, but to prove this isn't just theory, let me outline a concrete example. A few weeks ago, I decided to test our techniques on BAML, a 300k LOC Rust codebase for a programming language that works with LLMs. I'm at best an amateur Rust dev and had never touched the BAML codebase before.

Within an hour or so, I had a PR fixing a bug which was approved by the maintainer the next morning. A few weeks later, @hellovai and I paired on shipping 35k LOC to BAML, adding cancellation support and WASM compilation - features the team estimated would take a senior engineer 3-5 days each. We got both draft prs ready in about 7 hours.

Again, this is all built around a workflow we call frequent intentional compaction - essentially designing your entire development process around context management, keeping utilization in the 40-60% range, and building in high-leverage human review at exactly the right points. We use a "research, plan, implement" workflow, but the core capabilities/learnings here are FAR more general than any specific workflow or set of prompts.

### Our weird journey to get here

I was working with one of the most productive AI coders I've ever met. Every few days they'd drop 2000-line Go PRs. And this wasn't a nextjs app or a CRUD API. This was complex, race-prone systems code that did JSON RPC over unix sockets and managed streaming stdio from forked unix processes (mostly claude code sdk processes, more on that later 🙂).

The idea of carefully reading 2,000 lines of complex Go code every few days was simply not sustainable. I was starting to feel a bit like Mitchell Hashimoto when he added the AI contributions must be disclosed rules for ghostty.

Our approach was to adopt something like sean's spec-driven development.

It was uncomfortable at first. I had to learn to let go of reading every line of PR code. I still read the tests pretty carefully, but the specs became our source of truth for what was being built and why.

The transformation took about 8 weeks. It was incredibly uncomfortable for everyone involved, not least of all for me. But now we're flying. A few weeks back, I shipped 6 PRs in a day. I can count on one hand the number of times I've edited a non-markdown file by hand in the last three months.

## Advanced Context Engineering for Coding Agents

What we needed was:

AI that Works Well in Brownfield Codebases
AI that Solves Complex Problems
No Slop
Maintain Mental Alignment across the team

(And yeah sure, let's try to spend as many tokens as possible.)

I'll dive into:

what we learned applying context engineering to coding agents
the dimensions along which using these agents is a deeply technical craft
why I don't believe these approaches are generalizable
the number of times I've been repeatedly proven wrong about (3)

### But first: The Naive Way to manage agent context

Most of us start by using a coding agent like a chatbot. You talk (or drunkenly shout) back and forth with it, vibing your way through a problem until you either run out of context, give up, or the agent starts apologizing.

A slightly smarter way is to just start over when you get off track, discarding your session and starting a new one, perhaps with a little more steering in the prompt.

[original prompt], but make sure you use XYZ approach, because ABC approach won't work

### Slightly Smarter: Intentional Compaction

You have probably done something I've come to call "intentional compaction". Whether you're on track or not, as your context starts to fill up, you probably want to pause your work and start over with a fresh context window. To do this, you might use a prompt like

"Write everything we did so far to progress.md, ensure to note the end goal, the approach we're taking, the steps we've done so far, and the current failure we're working on"

You can also use commit messages for intentional compaction.

### What Exactly Are We Compacting?

What eats up context?

Searching for files
Understanding code flow
Applying edits
Test/build logs
Huge JSON blobs from tools

All of these can flood the context window. Compaction is simply distilling them into structured artifacts.

A good output for an intentional compaction might include something like

### Why obsess over context?

As we went deep on in 12-factor agents, LLMs are stateless functions. The only thing that affects the quality of your output (without training/tuning models themselves) is the quality of the inputs.

This is just as true for wielding coding agents as it is for general agent design, you just have a smaller problem space, and rather than building agents, we're talking about using agents.

At any given point, a turn in an agent like claude code is a stateless function call. Context window in, next step out.

That is, the contents of your context window are the ONLY lever you have to affect the quality of your output. So yeah, it's worth obsessing over.

You should optimize your context window for:

Correctness
Completeness
Size
Trajectory

Put another way, the worst things that can happen to your context window, in order, are:

Incorrect Information
Missing Information
Too much Noise

If you like equations, here's a dumb one you can reference:

As Geoff Huntley puts it,

The name of the game is that you only have approximately 170k of context window to work with. So it's essential to use as little of it as possible. The more you use the context window, the worse the outcomes you'll get.

Geoff's solution to this engineering constraint is a technique he calls Ralph Wiggum as a Software Engineer, which basically involves running an agent in a while loop forever with a simple prompt.

while :; do
  cat PROMPT.md | npx --yes @sourcegraph/amp
done

If you wanna learn more about ralph or what's in PROMPT.md, you can check out Geoff's post or dive into the project that @simonfarshid, @lantos1618, @AVGVSTVS96 and I built at last weekend's YC Agents Hackathon, which was able to (mostly) port BrowserUse to TypeScript overnight

Geoff describes ralph as a "hilariously dumb" solution to the context window problem. I'm not entirely sure that it is dumb.

### Back to compaction: Using Sub-Agents

Subagents are another way to manage context, and generic subagents (i.e. not custom ones) have been a feature of claude code and many coding CLIs since the early days.

Subagents are not about playing house and anthropomorphizing roles. Subagents are about context control.

The most common/straightforward use case for subagents is to let you use a fresh context window to do finding/searching/summarizing that enables the parent agent to get straight to work without clouding its context window with Glob / Grep / Read / etc calls.

https://github.com/user-attachments/assets/cb4e7864-9556-4eaa-99ca-a105927f484d

The ideal subagent response probably looks similar to the ideal ad-hoc compaction from above

Getting a subagent to return this is not trivial:

### What works even better: Frequent Intentional Compaction

The techniques I want to talk about and that we've adopted in the last few months fall under what I call "frequent intentional compaction".

Essentially, this means designing your ENTIRE WORKFLOW around context management, and keeping utilization in the 40%-60% range (depends on complexity of the problem ).

The way we do it is to split into three (ish) steps.

I say "ish" because sometimes we skip the research and go straight to planning, and sometimes we'll do multiple passes of compacted research before we're ready to implement.

I'll share example outputs of each step in a concrete example below. For a given feature or bug, we'll tend to do:

Research

Understand the codebase, the files relevant to the issue, and how information flows, and perhaps potential causes of a problem.

here's our research prompt. It currently uses custom subagents, but in other repos I use a more generic version that uses the claude code Task() tool with general-agent. The generic one works almost as well.

Plan

Outline the exact steps we'll take to fix the issue, and the files we'll need to edit and how, being super precise about the testing / verification steps in each phase.

This is the prompt we use for planning.

Implement

Step through the plan, phase by phase. For complex work, I'll often compact the current status back into the original plan file after each implementation phase is verified.

This is the implementation prompt we use.

Aside - if you've been hearing a lot about git worktrees, this is the only step that needs to be done in a worktree. We tend to do everything else on main.

How we manage/share the markdown files

I will skip this part for brevity but feel free to launch a claude session in humanlayer/humanlayer and ask how the "thoughts tool" works.

### Putting this into practice

I do a weekly live-coding session with @vaibhav where we whiteboard and code up a solution to an advanced AI Engineering problem. It's one of the highlights of my week.

Several weeks ago, I decided to share some more about the process, curious if our in-house techniques could one-shot a fix to a 300k LOC Rust codebase for BAML, a programming language for working with LLMs. I picked out an (admittedly small-ish) bug from the @BoundaryML repo and got to work.

You can watch the episode to learn more about the process, but to outline it:

Worth noting: I am at best an amateur Rust dev, and I have never worked in the BAML codebase before.

#### The research

I created a piece of research, I read it. Claude decided the bug was invalid and the codebase was correct.
I threw that research out and kicked off a new one, with more steering.
here is the final research doc i ended up using

#### The plans

While the research was running, I got impatient and kicked off a plan, with no research, to see if claude could go straight to an implementation plan - you can see it here
When the research was done, I kicked off another implementation plan that used the research results - you can see it here

The plans are both fairly short, but they differ significantly. They fix the issue in different ways, and have different testing approaches. Without going too much into detail, they both "would have worked" but the one built with research fixed the problem in the best place and prescribed testing that was in line with the codebase conventions.

#### The implementation

This was all happening the night before the podcast recording. I ran both plans in parallel and submitted both as PRs before signing off for the night.

By the time we were on the show at 10am PT the next day, the PR from the plan with the research was already approved by @aaron, who didn't even know I was doing a bit for a podcast 🙂. We closed the other one.

So out of our original 4 goals, we hit:

✅ Works in brownfield codebases (300k LOC rust project)
Solves complex problems
✅ no slop (pr merged)
Keeps mental alignment

### Solving complex problems

Vaibhav was still skeptical, and I wanted to see if we could solve a more complex problem.

So a few weeks later, the two of us spent 7 hours (3 hours on research/plans, 4 hours on implementation) and shipped 35k LOC to add cancellation and wasm support to BAML. The cancelation PR just got merged last week. The WASM one is still open, but has a working demo of calling the wasm-compiled rust runtime from a JS app in the browser.

While the cancelation PR required a little more love to take things over the line, we got incredible progress in just a day. Vaibhav estimated that each of these PRs would have been 3-5 days of work for a senior engineer on the BAML team to complete.

✅ So we can solve complex problems too.

### This is not Magic

Remember that part in the example where I read the research and threw it out cause it was wrong? Or me and Vaibhav sitting DEEPLY ENGAGED FOR 7 HOURS? You have to engage with your task when you're doing this or it WILL NOT WORK.

There's a certain type of person who is always looking for the one magic prompt that will solve all their problems. It doesn't exist.

Frequent Intentional Compaction via a research/plan/implement flow will make your performance better, but what makes it good enough for hard problems is that you build high-leverage human review into your pipeline.

### Eggs on Faces

A few weeks back, @blakesmith and I sat down for 7 hours and tried to remove hadoop dependencies from parquet java - the deep dive on everything that went wrong and my theories as to why, I'll save for another post, suffice it to say that it did not go well. The tl;dr is that the research steps didn't go deep enough through the dependency tree, and assumed classes could be moved upstream without introducing deeply nested hadoop dependencies.

There are big hard problems you cannot just prompt your way through in 7 hours, and we're still curiously and excitedly hacking on pushing the boundaries with friends and partners. I think the other learning here is that you probably need at least one person who is an expert in the codebase, and for this case, that was neither of us.

### On Human Leverage

If there's one thing you take away from all this, let it be this:

A bad line of code is… a bad line of code. But a bad line of a plan could lead to hundreds of bad lines of code. And a bad line of research, a misunderstanding of how the codebase works or where certain functionality is located, could land you with thousands of bad lines of code.

So you want to focus human effort and attention on the HIGHEST LEVERAGE parts of the pipeline.

When you review the research and the plans, you get more leverage than you do when you review the code. (By the way, one of our primary focuses @ humanlayer is helping teams build and leverage high-quality workflow prompts and crafting great collaboration workflows for ai-generated code and specs).

### What is code review for?

People have a lot of different opinions on what code review is for.

I prefer Blake Smith's framing in Code Review Essentials for Software Teams, where he says the most important part of code review is mental alignment - keeping members of the team on the page as to how the code is changing and why.

Remember those 2k line golang PRs? I cared about them being correct and well designed, but the biggest source of internal unrest and frustration on the team was the lack of mental alignment. I was starting to lose touch with what our product was and how it worked.

I would expect that anyone who's worked with a very productive AI coder has had this experience.

This is actually the most important part of research/plan/implement to us. A guaranteed side effect of everyone shipping way more code is that a much larger proportion of your codebase is going to be unfamiliar to any given engineer at any point in time.

I won't even try to convince you that research/plan/implement is the right approach for most teams - it probably isn't. But you ABSOLUTELY need an engineering process that

keeps team members on the same page
enables team members to quickly learn about unfamiliar parts of the codebase

For most teams, this is pull requests and internal docs. For us, it's now specs, plans, and research.

I can't read 2000 lines of golang daily. But I can read 200 lines of a well-written implementation plan.

I can't go spelunking through 40+ files of daemon code for an hour+ when something is broken (okay, I can, but I don't want to). I can steer a research prompt to give me the speed-run on where I should be looking and why.

### Recap

Basically we got everything we needed.

✅ Works in brownfield codebases
✅ Solves complex problems
✅ No slop
✅ Maintains mental alignment

(oh, and yeah, our team of three is averaging about $12k on opus per month)

So you don't think I'm just another hyped up mustachio'd sales guy, I'll note that this does not work perfectly for every problem (we'll be back for another round sound, parquet-java).

In August the whole team spent 2 weeks spinning circles on a really tricky race condition that spiraled into a rabbit hole of issues with MCP sHTTP keepalives in golang and a whole bunch of other dead ends.

But that's the exception now. In general, this works well for us. Our intern shipped 2 PRs on his first day, and 10 on his 8th day. I was genuinely skeptical that it would work for anyone else, but me and Vaibhav shipped 35k LOC of working BAML code in 7 hours. (And if you haven't met Vaibhav, he's one of the most meticulous engineers I know when it comes to code design and quality.)

### What's coming

I'm reasonably confident that coding agents will be commoditized.

The hard part will be the team and workflow transformation. Everything about collaboration will change in a world where AI writes 99% of our code.

And I believe pretty strongly that if you don't figure this out, you're gonna get lapped by someone who did.

### okay so clearly you have something to sell me

We're pretty bullish on spec-first, agentic workflows, so we're building tools to make it easier. Among many things, I'm obsessed with the problem of scaling these "frequent intentional compaction" workflows collaboratively across large teams.

Today, we're launching CodeLayer, our new "post-IDE IDE" in private beta - think "Superhuman for claude code". If you're a fan of Superhuman and/or vim mode and you're ready to move beyond "vibe coding" and get serious about building with agents, we'd love to have you join the waitlist.

Sign up at https://humanlayer.dev.

## For OSS Maintainers - lets ship something together

If you are a maintainer on a complex OSS project and based in the bay area, my open offer - I will pair with you in-person in SF for 7 hours on a saturday and see if we can ship something big.

I get a lot of learning about the limitations and where these techniques fall short (and, with any luck, a working merged PR that adds a ton of value that I can point to). You get to learn the workflow in the only way I've found that works well - direct 1x1 pairing.

## For Engineering Leaders

If you or someone you know is an engineering leader that wants to 10x their team's productivity with AI, we're forward-deploying with teams of all sizes to help drive the culture/process/tech shift needed to transition to an ai-first coding world.

### Thanks

Thanks to all the friends and founders who've listened through early ramble-y versions of this post - Adam, Josh, Andrew, and many many more
Thanks Sundeep for weathering this wacky storm
Thanks Allison, Geoff, and Gerred for dragging us kicking and screaming into the future

12 Factor Agents

Dex — Wed, 02 Apr 2025 16:00:00 GMT

Based on the original 12 Factor Agents post on GitHub. In the spirit of 12 Factor Apps.

## Introduction

I've tried every agent framework out there, from the plug-and-play crew/langchains to the "minimalist" smolagents of the world to the "production grade" langraph, griptape, etc.

I've talked to a lot of really strong founders, in and out of YC, who are all building really impressive things with AI. Most of them are rolling the stack themselves. I don't see a lot of frameworks in production customer-facing agents.

I've been surprised to find that most of the products out there billing themselves as "AI Agents" are not all that agentic. A lot of them are mostly deterministic code, with LLM steps sprinkled in at just the right points to make the experience truly magical.

Agents, at least the good ones, don't follow the "here's your prompt, here's a bag of tools, loop until you hit the goal" pattern. Rather, they are comprised of mostly just software.

So, I set out to answer:

What are the principles we can use to build LLM-powered software that is actually good enough to put in the hands of production customers?

## Table of Contents

## How We Got Here: A Brief History of Software

### You don't have to listen to me

Whether you're new to agents or an ornery old veteran like me, I'm going to try to convince you to throw out most of what you think about AI Agents, take a step back, and rethink them from first principles. (spoiler alert if you didn't catch the OpenAI responses launch a few weeks back, but pushing MORE agent logic behind an API ain't it)

## Agents are software, and a brief history thereof

let's talk about how we got here

### 60 years ago

We're gonna talk a lot about Directed Graphs (DGs) and their Acyclic friends, DAGs. I'll start by pointing out that...well...software is a directed graph. There's a reason we used to represent programs as flow charts.

### 20 years ago

Around 20 years ago, we started to see DAG orchestrators become popular. We're talking classics like Airflow, Prefect, some predecessors, and some newer ones like (dagster, inggest, windmill). These followed the same graph pattern, with the added benefit of observability, modularity, retries, administration, etc.

### 10-15 years ago

When ML models started to get good enough to be useful, we started to see DAGs with ML models sprinkled in. You might imagine steps like "summarize the text in this column into a new column" or "classify the support issues by severity or sentiment".

But at the end of the day, it's still mostly the same good old deterministic software.

### The promise of agents

I'm not the first person to say this, but my biggest takeaway when I started learning about agents, was that you get to throw the DAG away. Instead of software engineers coding each step and edge case, you can give the agent a goal and a set of transitions:

And let the LLM make decisions in real time to figure out the path

The promise here is that you write less software, you just give the LLM the "edges" of the graph and let it figure out the nodes. You can recover from errors, you can write less code, and you may find that LLMs find novel solutions to problems.

### Agents as loops

Put another way, you've got this loop consisting of 3 steps:

LLM determines the next step in the workflow, outputting structured json ("tool calling")
Deterministic code executes the tool call
The result is appended to the context window
repeat until the next step is determined to be "done"

initial_event = {"message": "..."}
context = [initial_event]
while True:
  next_step = await llm.determine_next_step(context)
  context.append(next_step)

  if (next_step.intent === "done"):
    return next_step.final_answer

  result = await execute_step(next_step)
  context.append(result)

Our initial context is just the starting event (maybe a user message, maybe a cron fired, maybe a webhook, etc), and we ask the llm to choose the next step (tool) or to determine that we're done.

Here's a multi-step example:

And the "materialized" DAG that was generated would look something like:

### The problem with this "loop until you solve it" pattern

The biggest problems with this pattern:

Agents get lost when the context window gets too long - they spin out trying the same broken approach over and over again
literally thats it, but that's enough to kneecap the approach

Even if you haven't hand-rolled an agent, you've probably seen this long-context problem in working with agentic coding tools. They just get lost after a while and you need to start a new chat.

I'll even perhaps posit something I've heard in passing quite a bit, and that YOU probably have developed your own intuition around:

### Even as models support longer and longer context windows, you'll ALWAYS get better results with a small, focused prompt and context

Most builders I've talked to pushed the "tool calling loop" idea to the side when they realized that anything more than 10-20 turns becomes a big mess that the LLM can't recover from. Even if the agent gets it right 90% of the time, that's miles away from "good enough to put in customer hands". Can you imagine a web app that crashed on 10% of page loads?

### What actually works - micro agents

One thing that I have seen in the wild quite a bit is taking the agent pattern and sprinkling it into a broader more deterministic DAG.

You might be asking - "why use agents at all in this case?" - we'll get into that shortly, but basically, having language models managing well-scoped sets of tasks makes it easy to incorporate live human feedback, translating it into workflow steps without spinning out into context error loops.

#### having language models managing well-scoped sets of tasks makes it easy to incorporate live human feedback...without spinning out into context error loops

### A real life micro agent

Here's an example of how deterministic code might run one micro agent responsible for handling the human-in-the-loop steps for deployment.

Human Merges PR to GitHub main branch
Deterministic Code Deploys to staging env
Deterministic Code Runs end-to-end (e2e) tests against staging
Deterministic Code Hands to agent for prod deployment, with initial context: "deploy SHA 4af9ec0 to production"
Agent calls deploy_frontend_to_prod(4af9ec0)
Deterministic code requests human approval on this action
Human Rejects the action with feedback "can you deploy the backend first?"
Agent calls deploy_backend_to_prod(4af9ec0)
Deterministic code requests human approval on this action
Human approves the action
Deterministic code executed the backend deployment
Agent calls deploy_frontend_to_prod(4af9ec0)
Deterministic code requests human approval on this action
Human approves the action
Deterministic code executed the frontend deployment
Agent determines that the task was completed successfully, we're done!
Deterministic code run the end-to-end tests against production
Deterministic code task completed, OR pass to rollback agent to review failures and potentially roll back

This example is based on a real life OSS agent we've shipped to manage our deployments at Humanlayer - here is a real conversation I had with it last week:

We haven't given this agent a huge pile of tools or tasks. The primary value in the LLM is parsing the human's plaintext feedback and proposing an updated course of action. We isolate tasks and contexts as much as possible to keep the LLM focused on a small, 5-10 step workflow.

### So what's an agent really?

prompt - tell an LLM how to behave, and what "tools" it has available. The output of the prompt is a JSON object that describe the next step in the workflow (the "tool call" or "function call").
switch statement - based on the JSON that the LLM returns, decide what to do with it.
accumulated context - store the list of steps that have happened and their results
for loop - until the LLM emits some sort of "Terminal" tool call (or plaintext response), add the result of the switch statement to the context window and ask the LLM to choose the next step.

In the "deploybot" example, we gain a couple benefits from owning the control flow and context accumulation:

In our switch statement and for loop, we can hijack control flow to pause for human input or to wait for completion of long-running tasks
We can trivially serialize the context window for pause+resume
In our prompt, we can optimize the heck out of how we pass instructions and "what happened so far" to the LLM

## Factor 1: Natural Language to Tool Calls

One of the most common patterns in agent building is to convert natural language to structured tool calls. This is a powerful pattern that allows you to build agents that can reason about tasks and execute them.

This pattern, when applied atomically, is the simple translation of a phrase like

can you create a payment link for $750 to Terri for sponsoring the february AI tinkerers meetup?

to a structured object that describes a Stripe API call like

{
  "function": {
    "name": "create_payment_link",
    "parameters": {
      "amount": 750,
      "customer": "cust_128934ddasf9",
      "product": "prod_8675309",
      "price": "prc_09874329fds",
      "quantity": 1,
      "memo": "Hey Jeff - see below for the payment link for the february ai tinkerers meetup"
    }
  }
}

Note: in reality the stripe API is a bit more complex, a real agent that does this (video) would list customers, list products, list prices, etc to build this payload with the proper ids, or include those ids in the prompt/context window (we'll see below how those are kinda the same thing though!)

From there, deterministic code can pick up the payload and do something with it.

# The LLM takes natural language and returns a structured object
nextStep = await llm.determineNextStep(
  """
  create a payment link for $750 to Jeff
  for sponsoring the february AI tinkerers meetup
  """
  )

# Handle the structured output based on its function
if nextStep.function == 'create_payment_link':
    stripe.paymentlinks.create(nextStep.parameters)
    return  # or whatever you want, see below
elif nextStep.function == 'something_else':
    # ... more cases
    pass
else:  # the model didn't call a tool we know about
    # do something else
    pass

NOTE: While a full agent would then receive the API call result and loop with it, eventually returning something like

I've successfully created a payment link for $750 to Terri for sponsoring the february AI tinkerers meetup. Here's the link: https://buy.stripe.com/test_1234567890

Instead, We're actually going to skip that step here, and save it for another factor, which you may or may not want to also incorporate (up to you!)

## Factor 2: Own Your Prompts

Don't outsource your prompt engineering to a framework.

By the way, this is far from novel advice:

Some frameworks provide a "black box" approach like this:

agent = Agent(
  role="...",
  goal="...",
  personality="...",
  tools=[tool1, tool2, tool3]
)

task = Task(
  instructions="...",
  expected_output=OutputModel
)

result = agent.run(task)

This is great for pulling in some TOP NOTCH prompt engineering to get you started, but it is often difficult to tune and/or reverse engineer to get exactly the right tokens into your model.

Instead, own your prompts and treat them as first-class code:

function DetermineNextStep(thread: string) -> DoneForNow | ListGitTags | DeployBackend | DeployFrontend | RequestMoreInformation {
  prompt #"
    {{ _.role("system") }}

    You are a helpful assistant that manages deployments for frontend and backend systems.
    You work diligently to ensure safe and successful deployments by following best practices
    and proper deployment procedures.

    Before deploying any system, you should check:
    - The deployment environment (staging vs production)
    - The correct tag/version to deploy
    - The current system status

    You can use tools like deploy_backend, deploy_frontend, and check_deployment_status
    to manage deployments. For sensitive deployments, use request_approval to get
    human verification.

    Always think about what to do first, like:
    - Check current deployment status
    - Verify the deployment tag exists
    - Request approval if needed
    - Deploy to staging before production
    - Monitor deployment progress

    {{ _.role("user") }}

    {{ thread }}

    What should the next step be?
  "#
}

(the above example uses BAML to generate the prompt, but you can do this with any prompt engineering tool you want, or even just template it manually)

If the signature looks a little funny, we'll get to that in factor 4 - tools are just structured outputs

function DetermineNextStep(thread: string) -> DoneForNow | ListGitTags | DeployBackend | DeployFrontend | RequestMoreInformation {

Key benefits of owning your prompts:

Full Control: Write exactly the instructions your agent needs, no black box abstractions
Testing and Evals: Build tests and evals for your prompts just like you would for any other code
Iteration: Quickly modify prompts based on real-world performance
Transparency: Know exactly what instructions your agent is working with
Role Hacking: take advantage of APIs that support nonstandard usage of user/assistant roles - for example, the now-deprecated non-chat flavor of OpenAI "completions" API. This includes some so-called "model gaslighting" techniques

Remember: Your prompts are the primary interface between your application logic and the LLM.

Having full control over your prompts gives you the flexibility and prompt control you need for production-grade agents.

I don't know what's the best prompt, but I know you want the flexibility to be able to try EVERYTHING.

## Factor 3: Own Your Context Window

You don't necessarily need to use standard message-based formats for conveying context to an LLM.

#### At any given point, your input to an LLM in an agent is "here's what's happened so far, what's the next step"

Everything is context engineering. LLMs are stateless functions that turn inputs into outputs. To get the best outputs, you need to give them the best inputs.

Creating great context means:

The prompt and instructions you give to the model
Any documents or external data you retrieve (e.g. RAG)
Any past state, tool calls, results, or other history
Any past messages or events from related but separate histories/conversations (Memory)
Instructions about what sorts of structured data to output

### on context engineering

This guide is all about getting as much as possible out of today's models. Notably not mentioned are:

Changes to models parameters like temperature, top_p, frequency_penalty, presence_penalty, etc.
Training your own completion or embedding models
Fine-tuning existing models

Again, I don't know what's the best way to hand context to an LLM, but I know you want the flexibility to be able to try EVERYTHING.

#### Standard vs Custom Context Formats

Most LLM clients use a standard message-based format like this:

[
  { 'role': 'system', 'content': 'You are a helpful assistant...' },
  { 'role': 'user', 'content': 'Can you deploy the backend?' },
  {
    'role': 'assistant',
    'content': null,
    'tool_calls': [{ 'id': '1', 'name': 'list_git_tags', 'arguments': '{}' }],
  },
  {
    'role': 'tool',
    'name': 'list_git_tags',
    'content': '{"tags": [{"name": "v1.2.3", "commit": "abc123", "date": "2024-03-15T10:00:00Z"}, {"name": "v1.2.2", "commit": "def456", "date": "2024-03-14T15:30:00Z"}, {"name": "v1.2.1", "commit": "abe033d", "date": "2024-03-13T09:15:00Z"}]}',
    'tool_call_id': '1',
  },
]

While this works great for most use cases, if you want to really get THE MOST out of today's LLMs, you need to get your context into the LLM in the most token- and attention-efficient way you can.

As an alternative to the standard message-based format, you can build your own context format that's optimized for your use case. For example, you can use custom objects and pack/spread them into one or more user, system, assistant, or tool messages as makes sense.

Here's an example of putting the whole context window into a single user message:


[
  {
    "role": "system",
    "content": "You are a helpful assistant..."
  },
  {
    "role": "user",
    "content": |
            Here's everything that happened so far:

        <slack_message>
            From: @alex
            Channel: #deployments
            Text: Can you deploy the backend?
        </slack_message>

        <list_git_tags>
            intent: "list_git_tags"
        </list_git_tags>

        <list_git_tags_result>
            tags:
              - name: "v1.2.3"
                commit: "abc123"
                date: "2024-03-15T10:00:00Z"
              - name: "v1.2.2"
                commit: "def456"
                date: "2024-03-14T15:30:00Z"
              - name: "v1.2.1"
                commit: "ghi789"
                date: "2024-03-13T09:15:00Z"
        </list_git_tags_result>

        what's the next step?
    }
]

The model may infer that you're asking it what's the next step by the tool schemas you supply, but it never hurts to roll it into your prompt template.

### code example

We can build this with something like:


class Thread:
  events: List[Event]

class Event:
  # could just use string, or could be explicit - up to you
  type: Literal["list_git_tags", "deploy_backend", "deploy_frontend", "request_more_information", "done_for_now", "list_git_tags_result", "deploy_backend_result", "deploy_frontend_result", "request_more_information_result", "done_for_now_result", "error"]
  data: ListGitTags | DeployBackend | DeployFrontend | RequestMoreInformation |
        ListGitTagsResult | DeployBackendResult | DeployFrontendResult | RequestMoreInformationResult | string

def event_to_prompt(event: Event) -> str:
    data = event.data if isinstance(event.data, str) \
           else stringifyToYaml(event.data)

    return f"<{event.type}>\n{data}\n</{event.type}>"


def thread_to_prompt(thread: Thread) -> str:
  return '\n\n'.join(event_to_prompt(event) for event in thread.events)

#### Example Context Windows

Here's how context windows might look with this approach:

Initial Slack Request:

<slack_message>
    From: @alex
    Channel: #deployments
    Text: Can you deploy the latest backend to production?
</slack_message>

After Listing Git Tags:

<slack_message>
    From: @alex
    Channel: #deployments
    Text: Can you deploy the latest backend to production?
    Thread: []
</slack_message>

<list_git_tags>
    intent: "list_git_tags"
</list_git_tags>

<list_git_tags_result>
    tags:
      - name: "v1.2.3"
        commit: "abc123"
        date: "2024-03-15T10:00:00Z"
      - name: "v1.2.2"
        commit: "def456"
        date: "2024-03-14T15:30:00Z"
      - name: "v1.2.1"
        commit: "ghi789"
        date: "2024-03-13T09:15:00Z"
</list_git_tags_result>

After Error and Recovery:

<slack_message>
    From: @alex
    Channel: #deployments
    Text: Can you deploy the latest backend to production?
    Thread: []
</slack_message>

<deploy_backend>
    intent: "deploy_backend"
    tag: "v1.2.3"
    environment: "production"
</deploy_backend>

<error>
    error running deploy_backend: Failed to connect to deployment service
</error>

<request_more_information>
    intent: "request_more_information_from_human"
    question: "I had trouble connecting to the deployment service, can you provide more details and/or check on the status of the service?"
</request_more_information>

<human_response>
    data:
      response: "I'm not sure what's going on, can you check on the status of the latest workflow?"
</human_response>

From here your next step might be:

nextStep = await determine_next_step(thread_to_prompt(thread))

{
  "intent": "get_workflow_status",
  "workflow_name": "tag_push_prod.yaml",
}

The XML-style format is just one example - the point is you can build your own format that makes sense for your application. You'll get better quality if you have the flexibility to experiment with different context structures and what you store vs. what you pass to the LLM.

Key benefits of owning your context window:

Information Density: Structure information in ways that maximize the LLM's understanding
Error Handling: Include error information in a format that helps the LLM recover. Consider hiding errors and failed calls from context window once they are resolved.
Safety: Control what information gets passed to the LLM, filtering out sensitive data
Flexibility: Adapt the format as you learn what works best for your use case
Token Efficiency: Optimize context format for token efficiency and LLM understanding

Context includes: prompts, instructions, RAG documents, history, tool calls, memory

Remember: The context window is your primary interface with the LLM. Taking control of how you structure and present information can dramatically improve your agent's performance.

### Don't take it from me

About 2 months after 12-factor agents was published, context engineering started to become a pretty popular term.

There's also a quite good Context Engineering Cheat Sheet from @lenadroid from July 2025.

Recurring theme here: I don't know what's the best approach, but I know you want the flexibility to be able to try EVERYTHING.

## Factor 4: Tools Are Just Structured Outputs

Tools don't need to be complex. At their core, they're just structured output from your LLM that triggers deterministic code.

For example, lets say you have two tools CreateIssue and SearchIssues. To ask an LLM to "use one of several tools" is just to ask it to output JSON we can parse into an object representing those tools.


class Issue:
  title: str
  description: str
  team_id: str
  assignee_id: str

class CreateIssue:
  intent: "create_issue"
  issue: Issue

class SearchIssues:
  intent: "search_issues"
  query: str
  what_youre_looking_for: str

The pattern is simple:

LLM outputs structured JSON
Deterministic code executes the appropriate action (like calling an external API)
Results are captured and fed back into the context

This creates a clean separation between the LLM's decision-making and your application's actions. The LLM decides what to do, but your code controls how it's done. Just because an LLM "called a tool" doesn't mean you have to go execute a specific corresponding function in the same way every time.

If you recall our switch statement from above

if nextStep.intent == 'create_payment_link':
    stripe.paymentlinks.create(nextStep.parameters)
    return # or whatever you want, see below
elif nextStep.intent == 'wait_for_a_while':
    # do something monadic idk
else: #... the model didn't call a tool we know about
    # do something else

Note: there has been a lot said about the benefits of "plain prompting" vs. "tool calling" vs. "JSON mode" and the performance tradeoffs of each. We'll link some resources to that stuff soon, but not gonna get into it here. See Prompting vs JSON Mode vs Function Calling vs Constrained Generation vs SAP, When should I use function calling, structured outputs, or JSON mode? and OpenAI JSON vs Function Calling.

The "next step" might not be as atomic as just "run a pure function and return the result". You unlock a lot of flexibility when you think of "tool calls" as just a model outputting JSON describing what deterministic code should do. Put this together with factor 8 own your control flow.

## Factor 5: Unify Execution State and Business State

Even outside the AI world, many infrastructure systems try to separate "execution state" from "business state". For AI apps, this might involve complex abstractions to track things like current step, next step, waiting status, retry counts, etc. This separation creates complexity that may be worthwhile, but may be overkill for your use case.

As always, it's up to you to decide what's right for your application. But don't think you have to manage them separately.

More clearly:

Execution state: current step, next step, waiting status, retry counts, etc.
Business state: What's happened in the agent workflow so far (e.g. list of OpenAI messages, list of tool calls and results, etc.)

If possible, SIMPLIFY - unify these as much as possible.

In reality, you can engineer your application so that you can infer all execution state from the context window. In many cases, execution state (current step, waiting status, etc.) is just metadata about what has happened so far.

You may have things that can't go in the context window, like session ids, password contexts, etc, but your goal should be to minimize those things. By embracing factor 3 you can control what actually goes into the LLM

This approach has several benefits:

Simplicity: One source of truth for all state
Serialization: The thread is trivially serializable/deserializable
Debugging: The entire history is visible in one place
Flexibility: Easy to add new state by just adding new event types
Recovery: Can resume from any point by just loading the thread
Forking: Can fork the thread at any point by copying some subset of the thread into a new context / state ID
Human Interfaces and Observability: Trivial to convert a thread into a human-readable markdown or a rich Web app UI

## Factor 6: Launch/Pause/Resume with Simple APIs

Agents are just programs, and we have things we expect from how to launch, query, resume, and stop them.

It should be easy for users, apps, pipelines, and other agents to launch an agent with a simple API.

Agents and their orchestrating deterministic code should be able to pause an agent when a long-running operation is needed.

External triggers like webhooks should enable agents to resume from where they left off without deep integration with the agent orchestrator.

Closely related to factor 5 - unify execution state and business state and factor 8 - own your control flow, but can be implemented independently.

Note - often AI orchestrators will allow for pause and resume, but not between the moment of tool selection and tool execution. See also factor 7 - contact humans with tool calls and factor 11 - trigger from anywhere, meet users where they are.

## Factor 7: Contact Humans with Tool Calls

By default, LLM APIs rely on a fundamental HIGH-STAKES token choice: Are we returning plaintext content, or are we returning structured data?

You're putting a lot of weight on that choice of first token, which, in the the weather in tokyo case, is

"the"

but in the fetch_weather case, it's some special token to denote the start of a JSON object.

|JSON>

You might get better results by having the LLM always output json, and then declare it's intent with some natural language tokens like request_human_input or done_for_now (as opposed to a "proper" tool like check_weather_in_city).

Again, you might not get any performance boost from this, but you should experiment, and ensure you're free to try weird stuff to get the best results.


class Options:
  urgency: Literal["low", "medium", "high"]
  format: Literal["free_text", "yes_no", "multiple_choice"]
  choices: List[str]

# Tool definition for human interaction
class RequestHumanInput:
  intent: "request_human_input"
  question: str
  context: str
  options: Options

# Example usage in the agent loop
if nextStep.intent == 'request_human_input':
  thread.events.append({
    type: 'human_input_requested',
    data: nextStep
  })
  thread_id = await save_state(thread)
  await notify_human(nextStep, thread_id)
  return # Break loop and wait for response to come back with thread ID
else:
  # ... other cases

Later, you might receive a webhook from a system that handles slack, email, sms, or other events.


@app.post('/webhook')
def webhook(req: Request):
  thread_id = req.body.threadId
  thread = await load_state(thread_id)
  thread.events.push({
    type: 'response_from_human',
    data: req.body
  })
  # ... simplified for brevity, you likely don't want to block the web worker here
  next_step = await determine_next_step(thread_to_prompt(thread))
  thread.events.append(next_step)
  result = await handle_next_step(thread, next_step)
  # todo - loop or break or whatever you want

  return {"status": "ok"}

The above includes patterns from factor 5 - unify execution state and business state, factor 8 - own your control flow, factor 3 - own your context window, and factor 4 - tools are just structured outputs, and several others.

If we were using the XML-y formatted from factor 3 - own your context window, our context window after a few turns might look like this:


(snipped for brevity)

<slack_message>
    From: @alex
    Channel: #deployments
    Text: Can you deploy backend v1.2.3 to production?
    Thread: []
</slack_message>

<request_human_input>
    intent: "request_human_input"
    question: "Would you like to proceed with deploying v1.2.3 to production?"
    context: "This is a production deployment that will affect live users."
    options: {
        urgency: "high"
        format: "yes_no"
    }
</request_human_input>

<human_response>
    response: "yes please proceed"
    approved: true
    timestamp: "2024-03-15T10:30:00Z"
    user: "alex@company.com"
</human_response>

<deploy_backend>
    intent: "deploy_backend"
    tag: "v1.2.3"
    environment: "production"
</deploy_backend>

<deploy_backend_result>
    status: "success"
    message: "Deployment v1.2.3 to production completed successfully."
    timestamp: "2024-03-15T10:30:00Z"
</deploy_backend_result>

Benefits:

Clear Instructions: Tools for different types of human contact allow for more specificity from the LLM
Inner vs Outer Loop: Enables agents workflows outside of the traditional chatGPT-style interface, where the control flow and context initialization may be Agent->Human rather than Human->Agent (think, agents kicked off by a cron or an event)
Multiple Human Access: Can easily track and coordinate input from different humans through structured events
Multi-Agent: Simple abstraction can be easily extended to support Agent->Agent requests and responses
Durable: Combined with factor 6 - launch/pause/resume with simple APIs, this makes for durable, reliable, and introspectable multiplayer workflows

## Factor 8: Own Your Control Flow

If you own your control flow, you can do lots of fun things.

Build your own control structures that make sense for your specific use case. Specifically, certain types of tool calls may be reason to break out of the loop and wait for a response from a human or another long-running task like a training pipeline. You may also want to incorporate custom implementation of:

summarization or caching of tool call results
LLM-as-judge on structured output
context window compaction or other memory management
logging, tracing, and metrics
client-side rate limiting
durable sleep / pause / "wait for event"

The below example shows three possible control flow patterns:

request_clarification: model asked for more info, break the loop and wait for a response from a human
fetch_git_tags: model asked for a list of git tags, fetch the tags, append to context window, and pass straight back to the model
deploy_backend: model asked to deploy a backend, this is a high-stakes thing, so break the loop and wait for human approval

def handle_next_step(thread: Thread):

  while True:
    next_step = await determine_next_step(thread_to_prompt(thread))

    # inlined for clarity - in reality you could put
    # this in a method, use exceptions for control flow, or whatever you want
    if next_step.intent == 'request_clarification':
      thread.events.append({
        type: 'request_clarification',
          data: nextStep,
        })

      await send_message_to_human(next_step)
      await db.save_thread(thread)
      # async step - break the loop, we'll get a webhook later
      break
    elif next_step.intent == 'fetch_open_issues':
      thread.events.append({
        type: 'fetch_open_issues',
        data: next_step,
      })

      issues = await linear_client.issues()

      thread.events.append({
        type: 'fetch_open_issues_result',
        data: issues,
      })
      # sync step - pass the new context to the LLM to determine the NEXT next step
      continue
    elif next_step.intent == 'create_issue':
      thread.events.append({
        type: 'create_issue',
        data: next_step,
      })

      await request_human_approval(next_step)
      await db.save_thread(thread)
      # async step - break the loop, we'll get a webhook later
      break

This pattern allows you to interrupt and resume your agent's flow as needed, creating more natural conversations and workflows.

Example - the number one feature request I have for every AI framework out there is we need to be able to interrupt a working agent and resume later, ESPECIALLY between the moment of tool selection and the moment of tool invocation.

Without this level of resumability/granularity, there's no way to review/approve the tool call before it runs, which means you're forced to either:

Pause the task in memory while waiting for the long-running thing to complete (think while...sleep) and restart it from the beginning if the process is interrupted
Restrict the agent to only low-stakes, low-risk calls like research and summarization
Give the agent access to do bigger, more useful things, and just yolo hope it doesn't screw up

You may notice this is closely related to factor 5 - unify execution state and business state and factor 6 - launch/pause/resume with simple APIs, but can be implemented independently.

## Factor 9: Compact Errors into Context Window

This one is a little short but is worth mentioning. One of these benefits of agents is "self-healing" - for short tasks, an LLM might call a tool that fails. Good LLMs have a fairly good chance of reading an error message or stack trace and figuring out what to change in a subsequent tool call.

Most frameworks implement this, but you can do JUST THIS without doing any of the other 11 factors. Here's an example:

thread = {"events": [initial_message]}

while True:
  next_step = await determine_next_step(thread_to_prompt(thread))
  thread["events"].append({
    "type": next_step.intent,
    "data": next_step,
  })
  try:
    result = await handle_next_step(thread, next_step) # our switch statement
  except Exception as e:
    # if we get an error, we can add it to the context window and try again
    thread["events"].append({
      "type": 'error',
      "data": format_error(e),
    })
    # loop, or do whatever else here to try to recover

You may want to implement an errorCounter for a specific tool call, to limit to ~3 attempts of a single tool, or whatever other logic makes sense for your use case.

consecutive_errors = 0

while True:

  # ... existing code ...

  try:
    result = await handle_next_step(thread, next_step)
    thread["events"].append({
      "type": next_step.intent + '_result',
      data: result,
    })
    # success! reset the error counter
    consecutive_errors = 0
  except Exception as e:
    consecutive_errors += 1
    if consecutive_errors < 3:
      # do the loop and try again
      thread["events"].append({
        "type": 'error',
        "data": format_error(e),
      })
    else:
      # break the loop, reset parts of the context window, escalate to a human, or whatever else you want to do
      break
  }
}

Hitting some consecutive-error-threshold might be a great place to escalate to a human, whether by model decision or via deterministic takeover of the control flow.

Benefits:

Self-Healing: The LLM can read the error message and figure out what to change in a subsequent tool call
Durable: The agent can continue to run even if one tool call fails

I'm sure you will find that if you do this TOO much, your agent will start to spin out and might repeat the same error over and over again.

That's where factor 8 - own your control flow and factor 3 - own your context building come in - you don't need to just put the raw error back on, you can completely restructure how it's represented, remove previous events from the context window, or whatever deterministic thing you find works to get an agent back on track.

But the number one way to prevent error spin-outs is to embrace factor 10 - small, focused agents.

## Factor 10: Small, Focused Agents

Rather than building monolithic agents that try to do everything, build small, focused agents that do one thing well. Agents are just one building block in a larger, mostly deterministic system.

The key insight here is about LLM limitations: the bigger and more complex a task is, the more steps it will take, which means a longer context window. As context grows, LLMs are more likely to get lost or lose focus. By keeping agents focused on specific domains with 3-10, maybe 20 steps max, we keep context windows manageable and LLM performance high.

#### As context grows, LLMs are more likely to get lost or lose focus

Benefits of small, focused agents:

Manageable Context: Smaller context windows mean better LLM performance
Clear Responsibilities: Each agent has a well-defined scope and purpose
Better Reliability: Less chance of getting lost in complex workflows
Easier Testing: Simpler to test and validate specific functionality
Improved Debugging: Easier to identify and fix issues when they occur

### What if LLMs get smarter?

Do we still need this if LLMs get smart enough to handle 100-step+ workflows?

tl;dr yes. As agents and LLMs improve, they might naturally expand to be able to handle longer context windows. This means handling MORE of a larger DAG. This small, focused approach ensures you can get results TODAY, while preparing you to slowly expand agent scope as LLM context windows become more reliable. (If you've refactored large deterministic code bases before, you may be nodding your head right now).

Being intentional about size/scope of agents, and only growing in ways that allow you to maintain quality, is key here. As the team that built NotebookLM put it:

I feel like consistently, the most magical moments out of AI building come about for me when I'm really, really, really just close to the edge of the model capability

Regardless of where that boundary is, if you can find that boundary and get it right consistently, you'll be building magical experiences. There are many moats to be built here, but as usual, they take some engineering rigor.

## Factor 11: Trigger from Anywhere

If you're waiting for the humanlayer pitch, you made it. If you're doing factor 6 - launch/pause/resume with simple APIs and factor 7 - contact humans with tool calls, you're ready to incorporate this factor.

Enable users to trigger agents from slack, email, sms, or whatever other channel they want. Enable agents to respond via the same channels.

Benefits:

Meet users where they are: This helps you build AI applications that feel like real humans, or at the very least, digital coworkers
Outer Loop Agents: Enable agents to be triggered by non-humans, e.g. events, crons, outages, whatever else. They may work for 5, 20, 90 minutes, but when they get to a critical point, they can contact a human for help, feedback, or approval
High Stakes Tools: If you're able to quickly loop in a variety of humans, you can give agents access to higher stakes operations like sending external emails, updating production data and more. Maintaining clear standards gets you auditability and confidence in agents that perform bigger better things

## Factor 12: Make Your Agent a Stateless Reducer

Okay so we're over 1000 lines of markdown at this point. This one is mostly just for fun.

HumanLayer Blog

Context Forking to Save Time, Tokens and Trouble

## Context Windows as Operating System Stacks

## How does context forking work?

## When should you fork your context window?

### Rewinding to course-correct an agent

### Forking to explore different design paths

### Forking to preserve a context window from context-inefficient operations

## Let's recap:

Long-Context Isn't the Answer

## How about some context?

## More context, less instruction adherence

## What determines instruction adherence?

## Instruction adherence at long context lengths

## Instruction-following as needle in a haystack

## What Works Instead

## How we're incorporating this at HumanLayer

## TL;DR

Getting Claude to Actually Read Your CLAUDE.md

## Conditional <important if> blocks

## A skill to automate this

## Try it

Skill Issue: Harness Engineering for Coding Agents

## Harness Engineering

## ... as a Subset of Context Engineering

### Views on Harness Engineering

### But What About Post-Training?

## Engineering Your Harness

### CLAUDE.md & AGENTS.md

#### The ETH Zurich Study

### MCP Servers Are for Tools

#### Too Many Tools Is Bad

#### Always Be Context-Engineering

### Skills Are for Reusable Knowledge (and Tools)

#### Progressive Disclosure

#### Skill Activation

#### Distributing Tools with Skills

### Sub-Agents Are for Context Control

#### Sub-Agents Avoid Context Rot

#### Aside: Long-Context Models

#### Sub-Agent Use-Cases

#### Sub-Agents Are (Also) for Cost Control

#### So We Heard You Like Sub-Agents

### Hooks Are for Control Flow

#### An Example Hook

## Back-Pressure Increases Your Chances of Success

## Closing Notes

A Brief History of Ralph

### June 19th 2025

### July 2025

### August 2025 - Advanced Context Engineering for Coding agents

### August 2025 - hacking productivity tools

### August 2025 - we put a coding agent in a while loop and it shipped 6 repos overnight

### August 2025 - testing ralph for a refactor

### September 2025 - cursed lang launch

### October 2025 - Claude Anonymous SF

### October 2025 - AI That Works w/ Geoff Huntley

### December 2025 - Anthropic Plugin Launches

### December 2025 - Youtube Slop Everywhere

### Jan 1 2026 - Ralph Wiggum Showdown

### What's next

### Okay one more thing

### PPS

Context-Efficient Backpressure for Coding Agents

## stay in the smart zone

## a pretty dumb fix

## make it smaller over time

## im tired of context-anxious models

#### output swallowing

#### piping to head / tail

## its not your fault

Writing a good CLAUDE.md

## Principle: LLMs are (mostly) stateless

## CLAUDE.md onboards Claude to your codebase

## Claude often ignores CLAUDE.md

## Creating a good CLAUDE.md file

### Less (instructions) is more

### CLAUDE.md file length & applicability

### Progressive Disclosure

### Claude is (not) an expensive linter

## Conditional `<important if>` blocks

## `CLAUDE.md` onboards Claude to your codebase

## Claude often ignores `CLAUDE.md`

## Creating a good `CLAUDE.md` file

### `CLAUDE.md` file length & applicability

### Don't use `/init` or auto-generate your `CLAUDE.md`