How to build an AI agent that actually works

You want to build an agent. You've read the papers. You know the loop: think, act, observe, repeat. But when you sit down to wire it up, the questions pile up fast. How do you actually call tools? What happens when the model hallucinates a function signature? How do you keep the thing from looping forever? How do you know if it worked?

This is what that looks like in practice.

The core loop

An agent is a state machine. The simplest version:

1. You give it a goal and context.
2. The model decides what to do next: call a tool, return an answer, or ask for clarification.
3. You execute that action and feed the result back.
4. Repeat until the model says it's done or you hit a limit.

The trap is treating this as a simple function call. It's not. You need to manage:

• Conversation history. The model needs context about what it's already tried. This grows unbounded. You'll need to truncate or summarize.
• Tool definitions. The model needs to know what tools exist and how to use them. Format matters. JSON schema is standard; get it wrong and the model will hallucinate parameters.
• Execution state. You need to track whether a tool call succeeded, failed, or timed out. The model needs to know which so it can recover.
• Termination. The agent needs a reason to stop. Max steps. Token budget. A clear success condition. All three if possible.

Tool calling in practice

Here's where most implementations break. The model outputs something like:

{
  "tool": "search",
  "parameters": {"query": "weather in NYC"}
}

You need to:

1. Parse it. JSON parsing fails sometimes. Have a fallback.
2. Validate it. Is search a real tool? Are the parameters the right type? Do they pass your constraints?
3. Execute it. Wrap the call in a timeout. Catch exceptions. Return structured errors, not stack traces.
4. Feed it back to the model in a format it recognizes.

The model will get tool definitions wrong. It will call tools that don't exist. It will pass the wrong types. Build for this:

def call_tool(tool_name: str, params: dict) -> str:
    if tool_name not in TOOL_REGISTRY:
        return f"Error: tool '{tool_name}' not found"
    
    tool = TOOL_REGISTRY[tool_name]
    try:
        # Validate params against schema
        validated = tool.schema.validate(params)
        result = tool.fn(**validated)
        return str(result)
    except TimeoutError:
        return "Error: tool timed out after 30s"
    except Exception as e:
        return f"Error: {type(e).__name__}: {str(e)}"

Return errors as strings the model can read. Not tracebacks. Not None. The model needs to understand what went wrong to recover.

State and recovery

An agent that stops mid-task is useless. You need to save state between calls. At minimum:

• The conversation history (all messages and tool results).
• The current step count.
• Which tools have been called and when (to detect loops).

When you resume, feed the model the full history. It will pick up where it left off. This is expensive in tokens, but it's reliable.

Detect loops before they happen. If the agent calls the same tool with the same parameters twice in a row, stop it. If it exceeds your max steps, stop it. If the token count hits your budget, stop it. Give the model a chance to wrap up cleanly—ask it to summarize what it found—before you cut it off.

The decision flow

flowchart TD
    A["User goal + context"] --> B["Call LLM with history"]
    B --> C{"Model output type?"}
    C -->|Tool call| D["Validate + execute tool"]
    D --> E{"Success?"}
    E -->|Yes| F["Add result to history"]
    E -->|No| G["Add error to history"]
    F --> H{"Done or max steps?"}
    G --> H
    C -->|Final answer| I["Return to user"]
    C -->|Invalid| J["Add parse error to history"]
    J --> H
    H -->|Continue| B
    H -->|Stop| I

What you'll actually encounter

Hallucinated tool calls. The model will invent tools or parameters that don't exist. This is normal. Your validation layer catches it. Feed the error back as a tool result and the model will try again. Usually it corrects itself on the second attempt.

Token explosion. The conversation history grows with every step. After 10-15 tool calls, you're sending kilobytes of context. Consider summarizing old turns or using sliding windows. The tradeoff: you lose some context but keep costs and latency reasonable.

Infinite loops. The agent gets stuck trying the same thing. Detect this with step limits and call frequency tracking. When you stop it, be explicit: "You've tried this 3 times. Try a different approach or return what you know."

Tool timeouts. External services are slow or fail. Always set timeouts on tool calls. Return a clear error message. Don't crash the agent.

Model inconsistency. Different models produce different tool call formats. Some models are better at following JSON schema. Some hallucinate less. If you support multiple models, normalize their outputs before validation.

Observability

You can't fix what you can't see. Log:

• The input (user goal, context).
• Each turn: what the model decided, what tool was called, what the result was.
• The final output.
• How many steps it took. How many tokens.

When an agent fails, you need the full trace. Use structured logging (JSON). Include timestamps. Make it queryable.

Track success rates by goal type. If agents fail consistently on a certain class of task, that's a signal to change the tool set or the prompt.

The honest tradeoffs

Building a reliable agent is not a one-liner. You trade simplicity for robustness at every layer:

• Validation vs. flexibility. Strict schemas catch errors early but make it hard to add new tools. Loose schemas are flexible but fail at runtime.
• History vs. cost. Full conversation history is reliable but expensive. Truncation saves tokens but loses context.
• Retries vs. latency. Letting the model retry failed tools improves success rate but makes the system slower.

There is no universal right answer. It depends on your use case. A customer support agent can afford to take 30 seconds and make 20 tool calls. A real-time trading agent cannot.

Start here

1. Define your tools precisely. Write the schema. Make sure the model can understand it.
2. Build the loop. LLM call, parse output, execute tool, feed back result. Handle errors at each step.
3. Add state management. Save and restore conversation history.
4. Set limits. Max steps, token budget, timeout per tool.
5. Log everything. You'll debug this.
6. Test with real goals. Evals on a fixed set of tasks. Track success rate and cost.

The agent will fail in ways you didn't expect. That's fine. The system is designed to survive that. Your job is to observe, understand why it failed, and adjust the tools, the prompt, or the limits.

Start simple. Add complexity only when you see it fail.