Appearance
Building agents with agents
Give a coding agent the goal. The built-in skills guide it through scaffolding, channels, verification, evals, and measured improvement.
What can a coding agent build for me?
A coding agent can take an Agent SDK project from an idea to a verified first version. It can:
- Create the project and choose the smallest useful surface
- Add tools, skills, channels, MCP connections, and approval gates
- Run a real turn and inspect its trace
- Add evals for the behavior you want to keep
- Improve the agent against fixed inputs
This works because the project surface lives in files. The CLI also returns structured JSON and records each run as an NDJSON trace. Your coding agent can inspect what the framework discovered, run the agent, and verify the result without reading terminal prose.
How do I create an agent with the built-in skill?
Have the coding agent read skills/create-agent/SKILL.md (in the monorepo: packages/agent-serve/skills/create-agent/SKILL.md) and follow it.
The skill asks about your agent's purpose, runtime, model, channels, MCP connections, and capabilities. It then shows you a plan, writes the project, and verifies the result.
For example:
Use the Agent SDK create-agent skill to build a PR triage agent reachable through GitHub. It should summarize failed checks, require approval before posting a review, and include one smoke eval.
The skill starts with a small surface: short instructions, one or two tools, the channels you chose, and a smoke eval. You can add more after the first end-to-end turn works.
Which built-in skill should I use?
The package ships task-specific guides under skills/:
| What you want to do | Skill |
|---|---|
| Understand the project layout and runtimes | framework-map |
| Create and verify a new agent | create-agent |
| Write fixtures and regression checks | evals |
Live A/B metrics on traffic (defineAB) | ab |
| Improve an agent against fixed inputs | hillclimb |
| Add GitHub webhooks and replay events | github |
| Connect an agent to Slack | setup-slack |
| Diagnose a local run | debug |
Point your coding agent at the matching SKILL.md. The guide contains the workflow, commands, and common mistakes for that task.
NOTE
The skill bodies use the current agent-serve CLI names. This guide uses the upcoming agent-sdk names. See Run the CLI for the full rename table.
How does a coding agent verify its work?
The coding agent should discover the project, test each server tool, run a real turn, and finish with evals:
bash
agent-sdk validate --dir .
agent-sdk info --dir . --json
agent-sdk call inspect_pr --dir . \
--input '{"prUrl":"https://github.com/acme/checkout/pull/42"}'
agent-sdk run --dir . \
--message "Is https://github.com/acme/checkout/pull/42 ready to approve?"
agent-sdk trajectory --events .agent-serve/traces/<sessionId>.ndjson
agent-sdk eval --dir . --list
agent-sdk eval --dir . --jsonserve boots without an API key, which is enough to check channel mounts and the playground shell. Model turns need a credential. When the environment lacks one, finish every key-free check, then hand the run and eval commands to the operator.
Test server tools with call before tuning the prompt. It runs a tool in-process with schema validation and no model turn. If the tool returns the wrong data, a prompt change won't fix it.
validate and run don't type-check the project because tsx strips types. Run the project's TypeScript check before shipping. Tool results must also be JSON-shaped. Use object literals or type aliases for execute return types instead of interface types.
How do I improve the agent after its first run?
Scaffolding proves the agent runs. It says nothing about quality. Once a smoke turn passes, give the hillclimb skill:
- Fixtures: one to three fixed inputs, such as a PR URL, a canonical question, or a saved webhook payload
- Success criteria: correct tool choice, fewer tool calls, lower wall time, or better output
- The freeze line: the API contract, output shape, and existing evals that must stay unchanged
Have the coding agent read skills/hillclimb/SKILL.md. It measures the current run, proposes one change, remeasures the same fixtures, and adds an eval for each kept improvement.