How to Build an AI Agent That Gets Better the More You Use It

I've been teaching programming for over twenty years. And the single most consistent thing I see in my students isn't a lack of intelligence. It's a lack of feedback.

They read the material once. They feel like they understand it. They move on. Then the test reveals they didn't understand it the way they thought they did. The students who improve aren't the ones who read more. They're the ones who test themselves, see exactly where they failed, go back to those specific gaps, and test again.

I think about this every time I see how most people build AI agents.

They set up the system prompt. They add a few tools. They run it for a week. And then they slowly drift away from it because the outputs aren't quite what they need and they don't know why. The agent peaked somewhere around day three and has been coasting ever since.

That's not an agent problem. That's a system design problem.

I spent the past week going through Anthropic's internal engineering sessions from Code with Claude London. These are the sessions where the people who actually build Claude - the applied AI team - walked through how they design agents internally. Not the marketing version. The actual architecture.

What I found is that self-improving agents aren't magic. They're five design decisions. And most people are making none of them.

Here's the pattern I see constantly.

Someone builds an agent. It works well. Then they get a new requirement and add more to the system prompt. Then another requirement. And another. Six months in, the system prompt is four hundred lines long, the agent has twelve tools it has to reason across at the start of every session, and the performance is actively worse than it was at the beginning.

The Anthropic team had an inventory management agent that hit this ceiling exactly. Sixty-two percent performance on their test suite. And the fix wasn't a new model. The fix wasn't better prompts. It was architecture.

The team took that four hundred line system prompt down to fifteen lines. Reduced twelve tools to three. And performance jumped to ninety-two percent.

Same model. Better system.

That's the insight the rest of this guide is built on. Self-improvement isn't about the model getting smarter. It's about the system you build around it.

Before you build anything, write down what a great output looks like.

Not "it should be helpful." Something specific enough that you could hand it to someone else and they'd be able to judge whether a given output passes or fails.

This sounds obvious. Almost nobody does it.

The reason it matters is that without a clear quality target, improvement is impossible to measure. You can feel like your agent is getting better without it actually improving in the ways that matter. And you can't fix what you haven't defined as broken.

Anthropic built this into their infrastructure as a feature called Outcomes, which is now in public beta for developers building on Claude Managed Agents. You write a rubric describing what success looks like. A separate evaluator agent scores each output against that rubric. The agent works toward it, sees its score, and adjusts.

You don't need the full infrastructure version to apply this principle today. You need one paragraph.

Write down: what does a 10/10 output look like from this agent? What does a 6/10 look like? What constitutes a fail? Put that in your project instructions so Claude has a quality standard to aim at, not just a task to complete.

Every improvement you make from this point forward has somewhere to point.

The most common cause of agent degradation isn't a bad model. It's a polluted context window.

When an agent has to read four hundred lines of instructions, twelve tool descriptions, and a sprawling system prompt at the start of every session, something gets lost. Policies contradict each other. Rules that apply to one type of task bleed into another. The agent starts making choices that look like reasoning failures but are actually navigation failures - there's simply too much to hold in mind at once.

The fix is called progressive disclosure, and it's one of the most valuable patterns the Anthropic team uses.

The idea is simple: Claude's system prompt should contain only what the agent needs to know regardless of the task. Everything else - policies, workflows, domain-specific rules - should be packaged as a skill that Claude loads when it realizes it needs it.

A skill is a saved set of instructions Claude can pull into its context on demand. Instead of "always having all the forecasting procedures in memory," the agent has a forecasting skill it reaches for when a forecasting task comes in. The rest of the time, that information doesn't exist in the session at all.

Think of it this way: a surgeon doesn't walk into the operating room reciting every procedure they've ever learned. They walk in knowing the specific procedure for today's patient and pull from their broader training only when they hit something unexpected.

Skills are how you build that selectivity into an agent.

The agent's system prompt gets shorter. The agent gets better. Those two things happen at the same time, not in opposition.

This is the pattern I recognized immediately from twenty years of teaching.

The students who improve the fastest aren't the ones who study more. They're the ones who test themselves, see specifically where they failed, study those exact gaps, and test again. The feedback loop is the learning mechanism.

The Anthropic team calls this hill climbing on evals. Here's how it works in practice:

First, define your test cases. What are the five or ten tasks your agent needs to handle? Write them down as specific questions or scenarios with expected outputs.

Second, run the tests. See what your agent produces for each one.

Third, diagnose the failures. Don't just note that it failed. Figure out why. Was the information missing from the context? Was a policy ambiguous? Was the agent routing to the wrong tool? The cause matters more than the failure.

Fourth, fix specifically that. Not the whole system. The one thing that caused this specific failure.

Fifth, run the tests again. See if the score improved. See if anything regressed.

Repeat.

Will from Anthropic's applied AI team ran this loop with the inventory management agent. He started at sixty-two percent. After each architectural improvement - skills replacing system prompt bloat, simpler tools replacing complex custom ones - he'd rerun the evals and watch the number climb. He ended at ninety-two.

The test cases are what made it possible. Without them, there's no signal. You're guessing.

A simple version of this you can run today:

That's the loop. It's not sophisticated. It's reliable.

An agent that starts from zero every session doesn't improve. It repeats.

It makes the same mistakes it made last week. It rediscovers the same workarounds. It never builds the kind of task-specific intuition that makes a human assistant genuinely better over time.

Claude's memory system, available in Settings → Memory, is the foundation of accumulation. Two toggles worth understanding:

"Search and reference past chats" - Claude can look back at previous conversations and carry relevant context forward. Turn it on.

"Generate memory from chat history" - Claude extracts patterns and preferences from your conversations and stores them as memory items. Turn this on too, and then audit it weekly.

That second part is the key. The memory system stores things automatically. Some will be accurate and worth keeping. Some will be approximate or outdated. A five-minute weekly review - delete what's wrong, add what's missing - is what transforms a memory system from a passive accumulator into an active asset.

After a few months of maintaining this, you'll notice the same thing I did: you stop re-explaining yourself. The context is already there. Every session begins already knowing what you're working on and how you like to work.

That's accumulation. It feels slow at first. Then one day it's everywhere.

Everything so far has been about improving your agent by making better design choices. This last pattern is different.

Dreaming is a feature Anthropic launched at Code with Claude on May 6, 2026, as part of Claude Managed Agents. It's a scheduled process that runs between agent sessions - when the agent is idle - and reviews everything the agent has done. Past session transcripts. Memory entries. Recurring patterns. What went wrong repeatedly. What worked every time. What memory entries are outdated and should be replaced.

Then it rewrites the agent's memory. Not the model. The notes. The next session begins already knowing what the previous sessions collectively learned.

Two things make this different from regular memory:

It surfaces patterns no single session can see. If an agent made the same mistake in twelve different sessions across three weeks, no individual session would catch that. Dreaming reads across all of them and flags it.

You decide how much control you want. Dreaming can update memory automatically between sessions. Or it can propose updates and wait for your review before anything lands. The memory notes are plain text. You can read them, edit them, reject them.

Harvey, the legal AI platform, reported a 6x improvement in task completion after enabling it.

Dreaming is currently in research preview on the Claude Platform - developers building on Claude Managed Agents can request access. It's not yet available to regular Claude.ai users. But the design principle it represents is available to everyone: build a regular habit of reviewing your agent's sessions, extracting what went right and wrong, and updating the memory and instructions accordingly.

The manual version is slower than the automated one. It's still dramatically better than not doing it at all.

One more thing the Anthropic team builds that most people don't: agents that act on events instead of waiting to be asked.

Maya from Anthropic's engineering team demonstrated this at Code with Claude London. The feature is called Routines, and it's available in Claude Code today via the /schedule command.

A Routine is a Claude Code session that runs on a trigger. Time-based - every Monday at 10am, review what changed in the codebase and update the documentation. Event-based - every time an issue gets filed, investigate whether it's related to a documentation gap and open a PR if it is.

The agent doesn't wait for a human to notice the problem and ask for help. It notices the problem and responds.

Over time, the triggers you've set up, the context you've given each routine, the quality standards you've defined - all of it gets refined. You see which routines consistently produce useful output and which ones generate noise. The system gets more targeted. More useful. More aligned with what you actually care about.

That's the compounding effect of the proactive layer. The agent that reacts to your environment gradually becomes more accurate at knowing what to react to.

They build on each other.

The quality target from Pattern 1 gives you something to test against in Pattern 3. The clean context from Pattern 2 means the testing loop in Pattern 3 produces signals you can act on instead of noise. The memory from Pattern 4 means each run of the testing loop builds on the last one. The dreaming from Pattern 5 means the memory improves between sessions without requiring your attention. The proactive routines from the last section mean the improvement isn't just happening when you remember to run tests - it's happening continuously, triggered by what actually occurs in your environment.

None of them require a new model. All of them compound.

The system that gets better over time isn't the one with the most capable model. It's the one where every session leaves the agent slightly more capable than the session before it.

Build the feedback loop. Then let it run.

If this changes how you think about your AI setup, share it with one person still waiting for the next model to fix their problems.

That's all I'm asking.

Become a paying subscriber to get access to this post and all other premium content.