On this article, you’ll find out how software design — not mannequin functionality — is the foundation explanation for most AI agent failures, and what concrete design patterns you’ll be able to apply to repair it.
Matters we’ll cowl embody:
Device design practices that enhance agent reliability, together with single-responsibility instruments, tight schemas, and structured error returns.
Widespread failure modes akin to unfiltered API publicity, silent partial success, and overlapping software names that break real-world workloads.
Schema and error dealing with patterns that cut back hallucination and unreliable conduct on the software boundary.
Let’s get into it.
AI Agent Device Design: What Works and What Doesn’t
Introduction
Most AI agent failures appear like mannequin errors: selecting the incorrect software, passing dangerous arguments, or mishandling errors. However in follow, the mannequin is normally working with the interface it was given. The underlying situation is usually the software design itself.
A mannequin can solely purpose from the knowledge uncovered by the software interface: the software title, its description, the parameter schema, and the parameter descriptions. These particulars form how the mannequin interprets intent, plans actions, and executes duties. When the software design is unclear, incomplete, or loosely structured, failures turn into predictable relatively than unintended.
Issues like imprecise naming, ambiguous directions, inconsistent schemas, weak parameter definitions, and poor error dealing with all improve the chance of failures. Stronger fashions can cut back some errors, however they can not reliably compensate for a flawed interface. This text covers:
Device design practices that enhance reliability
Failure modes that look wonderful in demos however break underneath actual workloads
Schema and error design that reduces hallucination on the software boundary
Every sample is paired with its failure counterpart, as a result of understanding why a design fails is as essential as realizing what to interchange it with.
What Works in AI Agent Device Design
1. One Device, One Accountability
In most agent techniques, a software ought to characterize a single, clear operation. When one software handles a number of behaviors by an motion parameter, the mannequin should first work out which mode to invoke earlier than it may possibly resolve the precise process.
The distinction turns into clearer when evaluating a multi-action software towards devoted single-purpose instruments:
# Keep away from: action-based multi-behavior software
@software
def manage_customer(
motion: str,
customer_id: str | None = None,
knowledge: dict | None = None
):
“””
motion: create | get | replace | delete | droop
“””
…
# Desire: single-responsibility instruments
@software
def create_customer(knowledge: CustomerInput) -> Buyer:
“””Create a brand new buyer file.”””
…
@software
def get_customer(customer_id: str) -> Buyer:
“””Retrieve a buyer by ID.”””
…
@software
def suspend_customer(customer_id: str, purpose: str) -> SuspensionResult:
“””Droop a buyer account.”””
…
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# Keep away from: action-based multi-behavior software
@software
def manage_customer(
motion: str,
customer_id: str | None = None,
knowledge: dict | None = None
):
“”“
motion: create | get | replace | delete | droop
““”
...
# Desire: single-responsibility instruments
@software
def create_customer(knowledge: CustomerInput) -> Buyer:
“”“Create a brand new buyer file.”“”
...
@software
def get_customer(customer_id: str) -> Buyer:
“”“Retrieve a buyer by ID.”“”
...
@software
def suspend_customer(customer_id: str, purpose: str) -> SuspensionResult:
“”“Droop a buyer account.”“”
...
One Device, One Accountability
Single-responsibility instruments give the mannequin an unambiguous operate and offer you cleaner error dealing with and simpler observability.
⚠️ Be aware: It is a helpful default relatively than a common rule. Some domains — akin to shell, filesystem, browser, or calendar instruments — could profit from a constrained multi-action interface as a result of the motion area itself is a part of the underlying abstraction.
2. Schemas That Make Invalid States Inconceivable
In tool-calling brokers, the mannequin constructs software name arguments by reasoning out of your schema.
A free schema means the mannequin guesses at constraints.
A decent schema encodes these constraints so no guessing is required.
Right here’s an instance:
from pydantic import BaseModel, Discipline
from enum import Enum
class Precedence(str, Enum):
LOW = “low”
MEDIUM = “medium”
HIGH = “excessive”
class CreateTaskInput(BaseModel):
title: str = Discipline(
description=”Brief, actionable process title. Use crucial type: ‘Overview PR’, not ‘PR Overview’.”,
min_length=5,
max_length=100
)
precedence: Precedence = Discipline(
description=”Activity precedence. Use HIGH just for blockers affecting different work.”,
default=Precedence.MEDIUM
)
due_date: str = Discipline(
description=”Due date in ISO 8601 format: YYYY-MM-DD. Have to be a future date.”,
sample=r”^d{4}-d{2}-d{2}$”
)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
from pydantic import BaseModel, Discipline
from enum import Enum
class Precedence(str, Enum):
LOW = “low”
MEDIUM = “medium”
HIGH = “excessive”
class CreateTaskInput(BaseModel):
title: str = Discipline(
description=“Brief, actionable process title. Use crucial type: ‘Overview PR’, not ‘PR Overview’.”,
min_length=5,
max_length=100
)
precedence: Precedence = Discipline(
description=“Activity precedence. Use HIGH just for blockers affecting different work.”,
default=Precedence.MEDIUM
)
due_date: str = Discipline(
description=“Due date in ISO 8601 format: YYYY-MM-DD. Have to be a future date.”,
sample=r“^d{4}-d{2}-d{2}$”
)
Enums are notably helpful for fields with a small set of legitimate values as a result of they remove a category of plausible-but-invalid outputs. Validation failures floor on the software boundary relatively than as cryptic downstream errors.
3. Descriptions That Outline Scope, Not Simply Goal
Device descriptions are model-facing documentation. They should do two issues: clarify when to make use of the software, and clarify when to not. Most descriptions solely do the primary.
# Weak: explains what it does, not when to not use it
“””Seek for paperwork within the information base.”””
# Robust: defines function, scope, and limits
“””
Search the interior information base for paperwork, insurance policies, and reference materials.
Use this when the consumer asks about firm procedures, product specs, or documented workflows.
Do NOT use this for real-time knowledge (costs, availability, present standing) — use get_live_data() as an alternative.
Returns as much as 5 outcomes ranked by relevance. If no outcomes are returned, the knowledge is just not within the information base.
“””
# Weak: explains what it does, not when to not use it
“”“Seek for paperwork within the information base.”“”
# Robust: defines function, scope, and limits
“”“
Search the interior information base for paperwork, insurance policies, and reference materials.
Use this when the consumer asks about firm procedures, product specs, or documented workflows.
Do NOT use this for real-time knowledge (costs, availability, present standing) — use get_live_data() as an alternative.
Returns as much as 5 outcomes ranked by relevance. If no outcomes are returned, the knowledge is just not within the information base.
““”
With out the disambiguation, the mannequin infers scope from the software title alone, which is usually a dependable supply of choice errors at scale. A superb software definition consists of clear boundaries from different instruments, not simply utilization directions.
4. Structured, Actionable Error Returns
When a software fails, the mannequin reads the error and decides what to do subsequent. An unhandled exception or stack hint produces noise-driven follow-up conduct. A structured error provides the mannequin one thing to department on.
Structured errors shouldn’t solely report what failed but additionally assist the agent determine what to do subsequent. A superb error format makes retry conduct express and provides the mannequin a transparent restoration path:
class ToolError(BaseModel):
error_code: str # machine-readable, for the mannequin to department on
message: str # human-readable description
recoverable: bool # can the agent retry?
suggested_action: str # what the agent ought to do subsequent
# Report not discovered: retryable
return ToolError(
error_code=”RECORD_NOT_FOUND”,
message=”No consumer file discovered with ID ‘usr_123’.”,
recoverable=True,
suggested_action=”Use list_users() to get legitimate consumer IDs earlier than calling get_user().”
)
# Quota exceeded: not retryable
return ToolError(
error_code=”QUOTA_EXCEEDED”,
message=”API quota for this software has been reached for at this time.”,
recoverable=False,
suggested_action=”Notify the consumer and cease. Don’t retry this software at this time.”
)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
class ToolError(BaseModel):
error_code: str # machine-readable, for the mannequin to department on
message: str # human-readable description
recoverable: bool # can the agent retry?
suggested_action: str # what the agent ought to do subsequent
# Report not discovered: retryable
return ToolError(
error_code=“RECORD_NOT_FOUND”,
message=“No consumer file discovered with ID ‘usr_123’.”,
recoverable=True,
suggested_action=“Use list_users() to get legitimate consumer IDs earlier than calling get_user().”
)
# Quota exceeded: not retryable
return ToolError(
error_code=“QUOTA_EXCEEDED”,
message=“API quota for this software has been reached for at this time.”,
recoverable=False,
suggested_action=“Notify the consumer and cease. Don’t retry this software at this time.”
)
The recoverable flag and suggested_action subject are what change agent conduct. With out them, fashions retry non-retryable errors or abandon recoverable ones.
5. Idempotent State-Altering Operations
Each software that mutates state — creates a file, sends a message, transfers funds — should be protected to name twice. In follow, brokers retry, networks fail, and the LLM loop could situation a second name as a result of affirmation of the primary by no means arrived.
A easy solution to forestall duplicate unwanted effects is to require an idempotency key for each write operation:
@software
def send_email(
to: str,
topic: str,
physique: str,
idempotency_key: str = Discipline(
description=”Distinctive key for this ship operation. Use a hash of recipient + topic + timestamp. “
“Similar key on retry returns the unique end result with out re-sending.”
)
) -> dict:
“””Ship an e mail. Idempotent: the identical idempotency_key is not going to set off a second ship.”””
current = idempotency_store.get(idempotency_key)
if current:
return current
end result = email_service.ship(to=to, topic=topic, physique=physique)
idempotency_store.set(idempotency_key, end result, ttl=86400)
return end result
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
@software
def send_email(
to: str,
topic: str,
physique: str,
idempotency_key: str = Discipline(
description=“Distinctive key for this ship operation. Use a hash of recipient + topic + timestamp. “
“Similar key on retry returns the unique end result with out re-sending.”
)
) -> dict:
“”“Ship an e mail. Idempotent: the identical idempotency_key is not going to set off a second ship.”“”
current = idempotency_store.get(idempotency_key)
if current:
return current
end result = email_service.ship(to=to, topic=topic, physique=physique)
idempotency_store.set(idempotency_key, end result, ttl=86400)
return end result
With out idempotency ensures, transient failures can simply flip into duplicate actions.
What Doesn’t Work in AI Agent Device Design
1. Skinny Wrappers Round Unfiltered APIs
Pointing an agent at a REST API and surfacing it as a software is the most typical shortcut and the most typical supply of manufacturing failures. APIs constructed for builders typically expose way more element than brokers really want. Responses come full of lots of of fields, even when solely a handful are related. They depend on pagination, use opaque inside IDs with little contextual which means, and return error codes that require deep area information to interpret.
A purpose-built wrapper handles pagination internally, initiatives solely the fields the agent wants, and maps API errors to the structured ToolError format mentioned above. The agent by no means constructs API paths or manages pages; it receives typed objects it may possibly purpose about.
That stated, over-wrapping may also be dangerous. If each endpoint turns into a separate, narrowly outlined software with no shared construction, the software floor can turn into fragmented and tougher for the mannequin to navigate. The purpose is just not maximal abstraction, however a constant, agent-friendly abstraction layer.
2. Loading All Instruments Into Each Context
Accuracy degrades because the software catalog grows. LongFuncEval, a 2025 examine on tool-calling efficiency throughout lengthy contexts, discovered efficiency drops considerably because the software catalog measurement elevated — even in fashions with 128K context home windows. Loading each software into each system immediate compounds this by consuming token price range earlier than any process content material is processed.
Dynamic software loading addresses each issues. Decide which instruments are related to the present step and embody solely these:
STEP_TOOL_MAP = {
“analysis”: [“search_documents”, “search_web”, “get_url_content”],
“write”: [“create_document”, “update_document”, “format_text”],
“ship”: [“send_email”, “post_to_slack”, “create_calendar_event”],
}
def get_tools_for_step(step_type: str, available_tools: record) -> record:
relevant_names = STEP_TOOL_MAP.get(step_type, [])
return [t for t in available_tools if t.name in relevant_names]
STEP_TOOL_MAP = {
“analysis”: [“search_documents”, “search_web”, “get_url_content”],
“write”: [“create_document”, “update_document”, “format_text”],
“ship”: [“send_email”, “post_to_slack”, “create_calendar_event”],
}
def get_tools_for_step(step_type: str, available_tools: record) -> record:
relevant_names = STEP_TOOL_MAP.get(step_type, [])
return [t for t in available_tools if t.name in relevant_names]
Dynamic Device Loading
Exposing solely a small, related subset of instruments at every step — relatively than the complete toolset — usually improves choice accuracy and reduces per-call token value.
3. Silent Partial Success
Partial success turns into an issue when a software completes solely a part of the requested work however returns a response that appears totally profitable. The agent continues execution with an incomplete or deceptive view of the system state.
This normally occurs when instruments suppress inside failures and return solely the profitable portion of the end result:
# This model silently misleads the agent
@software
def bulk_create_tasks(duties: record) -> dict:
created = []
for process in duties:
attempt:
end result = task_api.create(process)
created.append(end result.id)
besides Exception:
cross # silent failure: that is the bug
return {“created”: created}
# This model makes partial success express
@software
def bulk_create_tasks(duties: record) -> BulkCreateResult:
created, failed = [], []
for process in duties:
attempt:
created.append(task_api.create(process).id)
besides TaskCreationError as e:
failed.append({“enter”: process.title, “purpose”: str(e)})
return BulkCreateResult(
created_ids=created,
failed_items=failed,
success=len(failed) == 0,
partial_success=len(created) > 0 and len(failed) > 0
)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# This model silently misleads the agent
@software
def bulk_create_tasks(duties: record) -> dict:
created = []
for process in duties:
attempt:
end result = task_api.create(process)
created.append(end result.id)
besides Exception:
cross # silent failure: that is the bug
return {“created”: created}
# This model makes partial success express
@software
def bulk_create_tasks(duties: record) -> BulkCreateResult:
created, failed = [], []
for process in duties:
attempt:
created.append(task_api.create(process).id)
besides TaskCreationError as e:
failed.append({“enter”: process.title, “purpose”: str(e)})
return BulkCreateResult(
created_ids=created,
failed_items=failed,
success=len(failed) == 0,
partial_success=len(created) > 0 and len(failed) > 0
)
The partial_success flag provides the mannequin one thing to department on: retry the failed objects, floor the partial end result to the consumer, or halt the workflow.
4. Overlapping Device Names and Descriptions
When two instruments do related issues, the mannequin causes about which to make use of on each name. That reasoning prices tokens and introduces errors. Some frequent examples embody:
search_documents and find_documents with an identical function
get_user and fetch_user_profile with unclear variations
create_task, add_task, and new_task as three instruments for one operation
In such instances, renaming alone isn’t the repair. Each software wants a function that may be described irrespective of different instruments within the set. If an outline requires “in contrast to X, this one…” to make sense, that’s a design downside. Device sprawl — too many instruments with overlapping scope — is a supply of unreliable agent conduct in enterprise deployments.
5. Damaging Actions With out a Affirmation Gate
Any software that takes an irreversible motion — deleting data, messaging actual customers, executing monetary transactions — wants a structural two-step affirmation, not an in-prompt “are you positive?” A staged method introduces an express affirmation boundary that reduces the danger of unintended or unauthorized execution.
The most secure sample is to separate staging from execution and require a short-lived affirmation token between the 2 steps:
@software
def stage_deletion(record_ids: record[str], purpose: str) -> StagedDeletion:
“””Stage data for deletion. Does NOT delete something.
Returns a affirmation token that expires in 60 seconds.
Name confirm_deletion() with this token to proceed.”””
token = generate_deletion_token(record_ids)
staged_deletions[token] = {“ids”: record_ids, “expires”: now() + 60}
return StagedDeletion(token=token, records_to_delete=len(record_ids), expires_in_seconds=60)
@software
def confirm_deletion(token: str) -> DeletionResult:
“””Execute a staged deletion. IRREVERSIBLE. Affirm solely after express consumer approval.”””
staged = staged_deletions.get(token)
if not staged or staged[“expires”] < now():
elevate ValueError(“Token invalid or expired. Stage the deletion once more.”)
# proceed
@software
def stage_deletion(record_ids: record[str], purpose: str) -> StagedDeletion:
“”“Stage data for deletion. Does NOT delete something.
Returns a affirmation token that expires in 60 seconds.
Name confirm_deletion() with this token to proceed.”“”
token = generate_deletion_token(record_ids)
staged_deletions[token] = {“ids”: record_ids, “expires”: now() + 60}
return StagedDeletion(token=token, records_to_delete=len(record_ids), expires_in_seconds=60)
@software
def confirm_deletion(token: str) -> DeletionResult:
“”“Execute a staged deletion. IRREVERSIBLE. Affirm solely after express consumer approval.”“”
staged = staged_deletions.get(token)
if not staged or staged[“expires”] < now():
elevate ValueError(“Token invalid or expired. Stage the deletion once more.”)
# proceed
Damaging Actions With out a Affirmation Gate
Two distinct software calls imply the mannequin can not full a damaging operation in a single reasoning step, which is the purpose.
⚠️ Be aware: Two-step security flows, nevertheless, are sometimes not ample on their very own in lots of techniques. Even when staging and affirmation are used, further safeguards — akin to short-lived, single-use tokens, strict session binding, and replay safety — are vital to stop token reuse, leakage, or cross-session execution that may bypass the meant security boundary.
AI Agent Device Design Selections at a Look
Each row represents a key resolution in AI agent software design:
Design Space
Works
Doesn’t Work
Device Scope
Single accountability per software
Motion-parameter instruments like manage_database(motion=”create”)
Schema
Tight: enums, validators, typed fields
Unfastened: free strings, untyped dicts
Descriptions
Embody scope boundaries and when to not use
Joyful path solely
Write Operations
Idempotent with idempotency keys
Hearth-and-forget, no retry security
Error Returns
Structured: error_code, recoverable, suggested_action
Unhandled exceptions or untyped strings
Device Rely
Dynamic loading per step
All instruments in each context
API Wrapping
Goal-built wrapper with agent-facing schema
Unfiltered API publicity
Partial Success
Specific partial_success subject in return
Silent exception swallowing
Damaging Actions
Two-step staging + affirmation
Single-call delete/ship/execute
Device Overlap
Semantically distinct, audited earlier than deploy
Related names and descriptions competing
Writing efficient instruments for AI brokers — utilizing AI brokers from Anthropic is a helpful reference on software design.
