Delegating Design Work to Agents

In the last post, I wrote about moving more of my design work from Figma into code. That solved one problem and exposed another: I had become the loop. Prototyping was faster, but mostly because I was moving faster. The team was still waiting for a designer to make the next screen.

So this July, I started rethinking how our design team should work.

The problem was simple. Screen-by-screen design does not scale. We have two designers working across six product surfaces. We cannot personally make every screen, review every state, and correct every small inconsistency. But the system that produces those screens can scale.

I was not trying to remove designers from the process. I wanted the parts we already understood to become more self-serve. PMs, engineers, and agents should be able to use an existing pattern without waiting for a designer to make every screen. Designers could then spend more time on the places where the pattern was still unknown.

We classified the roadmap to see whether this was realistic. Of the items that touched UI, about 85% fit patterns we already had.

The operating model I sketched had four layers: design tokens, rules that run in CI, pattern documentation written for agents, and an agent harness that connects everything. Three of those already existed in some form. The harness was still a box on a slide.

This post is about what happened when I tried to make that box real.

The work was already there

When I looked closely at my workflow, I realized I was repeating the same sequence. I gathered context from meetings and Slack, found the closest product pattern, built or repaired something in code, reviewed it against our guidelines, and wrote down what changed. Most of the work already existed. It just lived in separate prompts, documents, scripts, and habits.

That is what made me think it could become a harness. The recurring steps could be connected so the output of one became the input to the next. I still wanted a designer to make the decisions. The harness would keep the context, patterns, rules, and checks from getting lost between steps.

Claude Code already provides the basic loop: plan the work, use subagents, change files, verify the result, and prepare a pull request. It also gives us extension points such as skills, agent definitions, hooks, and headless runs. I organized the work I was already doing around those extension points:

• Rules describe what can be checked, stored as YAML and read by a small engine.
• Patterns index settled screen types and point to working examples in the product.
• Skills are runbooks for recurring work such as auditing, generating, and exploring.
• Agents handle narrow jobs in separate contexts.
• Evals contain fixtures and repeatable tasks that tell us whether the system is actually helping.

Once connected, one step could hand something useful to the next. A brief could feed exploration, a chosen pattern could constrain what the agent built, and the rules and evals could check the result. I would no longer have to remember and manually rebuild the whole sequence every time.

Three lanes, three levels of trust

Design requests are not one kind of work, so I did not want one pipeline for all of them. I separated the work into three lanes:

• Explore means we do not know what the answer should be yet. The agent creates directions and prototypes. A designer chooses what is worth pursuing. This lane has the most human involvement.
• Generate means the pattern is already known. The agent matches the request to an existing pattern, builds the screen, reviews the result in a separate context, and opens a draft pull request.
• Repair means the product has drifted from rules we already agreed on. Scheduled audits find the drift. Only safe, exact fixes can become small pull requests, and a human still merges them.

If something is not covered by a rule or pattern, the agent reports it and stops. The generate lane does the same when it cannot find a matching pattern. I wanted that behavior because an agent that improvises when it is uncertain can look very productive while quietly creating more design debt.

What one project looked like

One of the first real uses was a new expense-entry flow. This project mostly sat in the Explore lane, but it was the first time I could see the separate pieces working as one flow. The review was one week away, and the requirements were spread across meeting transcripts and two Slack channels.

A context-gathering skill turned that material into one brief in the repository. Each claim included a speaker and timestamp. The brief separated what had been decided, what had only been discussed, and what had been ruled out. Anything the source material could not support was labeled as an assumption.

Open UI questions became small ASCII sketch files. Each direction pointed to the real patterns and components it would use. The file ended with a simple table: what ships if nobody makes a decision? That framing helped the team decide what actually needed discussion.

The parts that were still genuinely open became playground prototypes. After the review, the decisions went back into the repository instead of staying in a meeting or someone's memory.

What made this useful was the source trail. Every step produced a sourced document before the next step consumed it. By review day, “why is it like this?” could be answered by a file with a timestamp, not by whoever happened to remember the conversation.

Connecting the steps was one problem. Deciding which automated checks were safe enough to trust was another.

The clean fixture is the product

Each deterministic rule lives in one YAML file. It has an ID, a severity, a detection method, and a fix mode. It also ships with two kinds of fixtures: a violation that the rule must catch, and a clean case that it must leave alone.

I originally thought the violation fixture was the important one. The clean fixture turned out to matter more.

It is the false-positive test. It is what lets us say a rule is trustworthy with a measurement instead of a feeling. My first version of an opacity rule found 144 violations. Almost all of them were legitimate exceptions. After narrowing the rule against those cases, it found 23, and all 23 were real. The rejected shapes now live in the clean fixture so the same mistake cannot quietly return.

That experience led to three operating rules:

• When a rule misfires, add the case to the clean fixture and improve the rule. Do not add a special case to the engine. A rejected fix should become permanent evidence.
• Auto-fix only means byte-exact. p-[16px] can become p-4 because they represent the same value in our system. Changing p-[13px] to the nearest token would move the layout, so the agent should report it instead.
• Trust has an order. A new rule starts in dry-run. It audits for a while before it can open a repair pull request. If people keep rejecting its fixes, it goes back to dry-run.

Not every design problem belongs in this system. A screen can pass every rule and still be wrong: the main decision is below the fold, one table uses three date formats, or an empty state only says “No data.” Those are judgments, not deterministic violations.

I keep those findings in a separate heuristic layer. Heuristics have their own IDs, and they can never auto-fix code. A rule is a check backed by clean fixtures. A heuristic is an argument that still needs a person.

Prose is not a gate

The most useful week was when the harness got its second real user: another designer on my team. In one 1:1, she reported three different misfires.

An ambiguous request went directly to generation without asking a question. A generated form used the browser's native date picker instead of our calendar component. The automatic routing did not recognize the way she described her request.

At first these looked like three separate bugs. They had the same cause. Every failed gate existed only as prose.

“Confirm the requirements first” was a sentence. “Use only components in the manifest” was a sentence. The agent could read both and still continue past them.

Each fix turned the instruction into something checkable. The confirmation step now has to write its answers before generation can begin. A reviewer agent treats a missing answer as a failure. The native-control issue became a rule with violation and clean fixtures. Each original prompt became an eval task, including one that makes sure the router stays quiet when the request is already clear.

I still write instructions in plain language. But I no longer assume that a load-bearing sentence is a reliable gate. If ignoring it would create a bad result, it needs a check behind it.

If you are starting from zero

I would build this in a different order than I first imagined:

1. Write the operating model first, even if it is only two pages. What kinds of design work exist on your team? How much autonomy can each kind tolerate?
2. Start with audit, not generation. Turn the rules your team already repeats in review comments into data, with both violation and clean fixtures. Run them in dry-run and share the report.
3. Add repair only for exact fixes. Keep pull requests small, require human merge, and include a kill switch from the beginning.
4. Add generation after the pattern index is good enough to refuse unknown work. Review the generated result in a separate context.
5. Evaluate against a baseline. Run the same task without the harness so you can see whether the system improves the outcome.

And throughout the process, when an important sentence in the instructions gets ignored, do not just rewrite the sentence. Build the check.

Where it stands

I do not want to make this sound more finished than it is.

Deciding what should graduate into a rule is still slow human judgment. Visual evaluation beyond deterministic accessibility checks is difficult to keep stable. The reasoning behind a decision still depends on people choosing to write it down.

There is also a maintenance loop that cannot be delegated. The system produces evidence—misfires, repeated findings, improvised patterns, and escalations—but a person still has to decide whether each one becomes a rule, a heuristic, a pattern, or something we reject and record.

But I notice what changed.

The other designer now reports agent misfires the way an engineer reports bugs. Each report makes the system permanently better instead of disappearing in a Slack thread. The design system is no longer only a document we ask people to remember. Parts of it now have a false-positive rate, fixtures, and a history of decisions.

In my notes from that work, I wrote a line that felt like a slogan: the designer's craft shifts from making the thing to making the thing that makes the thing.

After building this for a few weeks, I would add one thing. The second thing still needs judgment, maintenance, and restraint. The work did not become easier when it moved up a level. It moved to a place where it can compound.