A nightly load into sales_fact fails. Ask an orchestrating agent to break down the diagnosis and it will usually stop at the first two subtasks that come to mind: check the logs, check the schema. That decomposition is narrow, and a fix built on it can miss the real cause. The problem is not the model, it is that the coordinator never questioned its own first list before delegating.
1. See what narrow decomposition misses
A first-pass breakdown of "why did sales_fact fail" often lands on log-analyzer and schema-inspector and stops there. Both are reasonable, and both are incomplete. Neither one checks whether the upstream extract delivered bad or late data, whether another job holds a lock on the same table, or which downstream dashboards are now stale because the load never completed. A fix based on two subtasks treats the visible half of the failure as the whole failure.
2. Add a self-critique step to the coordinator prompt
The fix is a coordinator prompt that forces a second pass before any delegation happens: generate an initial subtask list, ask what is missing, add subtasks to cover the gap, and only then hand work to subagents.
You are a coordinator diagnosing an ETL failure. When
decomposing the task:
1. Generate an initial list of subtasks.
2. Ask yourself: what systems, data sources, or failure
modes are missing from that list?
3. Add subtasks to cover those gaps.
4. Only then begin delegating to subagents.
For an ETL failure specifically, consider:
- the failing job AND its upstream dependencies
- a structural cause (schema) AND a data cause (quality)
- this run AND whether it is a recurring pattern
- the technical root cause AND the downstream impact
Run the sales_fact example through that loop and the initial two-item list grows to four: log-analyzer and schema-inspector, plus a source-data-validator that checks the upstream extract for null spikes or row-count drops, and a downstream-impact-checker that lists which dashboards or scheduled jobs depend on the table. The self-critique step costs one extra generation and catches the two subtasks that a narrow first pass always skips.
3. Define each subtask as a subagent
With Claude's Agent SDK, each subtask becomes an AgentDefinition, with a description Claude matches against the task and a prompt scoped to that one job.
from claude_agent_sdk import AgentDefinition
log_analyzer = AgentDefinition(
description="Diagnoses ETL failures from pipeline logs. "
"Use when a nightly load job fails and the "
"cause is unclear.",
prompt="Find the failing step and report the exact "
"error message and line number. Do not propose "
"a fix.",
tools=["Read", "Grep", "Bash"],
model="haiku",
)
schema_inspector = AgentDefinition(
description="Checks table structure for drift against "
"the last known good schema.",
prompt="Compare current columns, types, and "
"constraints to the last successful load.",
tools=["Read", "Bash"],
)
source_data_validator = AgentDefinition(
description="Checks the upstream extract for the failed "
"load. Use for row-count drops or null spikes.",
prompt="Compare today's source row count and null "
"rate to the 7-day average.",
tools=["Read", "Bash"],
model="haiku",
)
downstream_impact_checker = AgentDefinition(
description="Lists dashboards and jobs that depend on "
"the table that failed to load.",
prompt="Find consumers of sales_fact and flag which "
"are now stale.",
tools=["Read", "Grep"],
)
AgentDefinition also takes a model field, so a subtask that is closer to pattern matching, like reading a log file or comparing two row counts, can run on a cheaper model while one that needs judgment, like assessing schema drift or downstream impact, stays on the default. That is the same idea covered in matching the model to the task, not the app, applied at the subagent level instead of a single orchestrator.
4. Wire the coordinator to call the agents
The coordinator prompt from step 2 becomes the system_prompt, and the four AgentDefinition objects from step 3 go into agents, keyed by the name Claude will use to call each one. Claude decides on its own which of the four to invoke, based on how the task matches each description.
from claude_agent_sdk import query, ClaudeAgentOptions
AGENTS = {
"log-analyzer": log_analyzer,
"schema-inspector": schema_inspector,
"source-data-validator": source_data_validator,
"downstream-impact-checker": downstream_impact_checker,
}
async for message in query(
prompt="The sales_fact load failed last night, find out why.",
options=ClaudeAgentOptions(
system_prompt=COORDINATOR_PROMPT,
# "Agent" auto-approves each subagent call
allowed_tools=["Read", "Grep", "Bash", "Agent"],
agents=AGENTS,
),
):
for block in getattr(message, "content", None) or []:
if getattr(block, "name", None) == "Agent":
print("routed to:", block.input.get("subagent_type"))
Claude does not call a Python function directly, it emits a tool_use block named Agent, and block.input["subagent_type"] holds the name of the agent it picked, one of the four keys in AGENTS. Logging that value is how you confirm the coordinator actually routed to source-data-validator or downstream-impact-checker instead of quietly falling back to the two obvious subtasks. To force a specific agent instead of letting Claude choose, name it in the prompt: "use the downstream-impact-checker agent to list what depends on sales_fact."
5. The same routing in OpenAI's Agents SDK
OpenAI's Agents SDK expresses the same coordinator through handoffs. The triage agent's instructions carry the same four-step self-critique, and each of the four specialists is passed in its handoffs list, exposed to the model as a callable tool named transfer_to_<agent_name>. When Runner.run finishes, result.new_items contains a HandoffOutputItem for any handoff that fired, and its target_agent.name is the concrete signal, the OpenAI equivalent of reading subagent_type off Claude's Agent tool call.
6. One failure, traced end to end
Take one run instead of a list of what each agent can do. At 02:14, the nightly load into sales_fact fails, and the engineer on call prompts the coordinator with "the sales_fact load failed last night, find out why." The coordinator's first move is always the same: call log-analyzer, since nothing else can be scoped correctly before the actual error is known.
routed to: log-analyzer
MERGE INTO sales_fact failed at 02:14:07 with:
duplicate key value violates unique constraint
"sales_fact_pkey" (order_id, sale_date)
A duplicate key on a merge has two plausible explanations: something changed about the table that now lets duplicates in, or the source data itself contains rows that collide on that key. Both are live possibilities under the self-critique step from earlier, so the coordinator calls schema-inspector and source-data-validator next. It does not call downstream-impact-checker yet, since nothing so far says anything about who reads this table.
routed to: schema-inspector
no DDL changes on sales_fact in the last 14 days
routed to: source-data-validator
source row count: 41,209 (7-day avg: 68,450, -40%)
null rate on customer_id: 6.2% (7-day avg: 0.1%)
schema-inspector rules out a structural cause. source-data-validator explains the collision: a partial extract with a null spike on customer_id delivered rows that collapsed onto the same order_id and sale_date, tripping the unique constraint on merge. The coordinator stops here and never calls downstream-impact-checker, because the merge failed and rolled back, so nothing downstream ever read the bad batch. That agent earns its place in a run where a bad load succeeds silently instead of failing loudly, a different failure mode from this one.
Three of the four candidate agents ran, in two rounds instead of one batch, and the proposed fix follows directly from what they found: quarantine the extract that arrived overnight rather than retrying the load as is, and add a row-count and null-rate check ahead of the merge so a batch shaped like this one gets rejected before it reaches sales_fact, not after.
Before You Trust the Decomposition
Test the coordinator prompt against a case where you already know the full list of causes, and check whether its self-critique step actually surfaces the ones a naive first pass would miss. If it keeps landing on the same two or three subtasks regardless of the failure, the gap-checking questions in the prompt are too generic and need to name the specific systems in your pipeline.
Not every failure needs any of this. A connection timeout, a transient throttling error, a job that fails because another job it depends on is still running, these have deterministic fixes: retry with backoff, wait and requeue, alert and stop. Route the alert through a cheap classifier first, or a plain if-statement on the error code, and reserve the coordinator and its agents for failures that pattern-matching cannot already explain. Calling four agents to conclude "retry it" is not decomposition, it is waste.
