Tool calling

Tool calling, also called function calling, is the mechanism by which a language model asks the surrounding software to do something. Instead of answering in prose, the model emits a structured request that names a tool and its arguments. The application runs the tool, returns the result, and the model continues. The model itself never executes anything.

How does tool calling work?

The application tells the model which tools exist. Each tool comes with a name, a description and a schema of its arguments, for example lookup_customer with a required email field.

When the model decides that a tool is needed, it outputs a call rather than an answer: the tool name and the argument values, as structured data. The application validates the call, runs the real function against your system, and appends the result to the conversation.

The model reads the result and either calls another tool or writes its answer. That loop, repeated until the task is done, is what makes an AI agent. The tool definitions and the transport between application and tools are what the Model Context Protocol standardises.

What separates a chatbot from an agent?

Access. A chatbot without tools can only talk. A model with tools can read your CRM, create a ticket, post an entry, send an email.

That is the whole difference and also the whole risk. Everything an agent can do wrong, it does through a tool call. The decisions that matter therefore sit in the tool layer: which tools exist, with which permissions they run, and which checks happen before a call executes.

What has to be designed?

Granularity. "Create a ticket in queue X" is a better tool than "run arbitrary SQL". The narrower the tool, the less a wrong judgement by the model can damage.

Permissions. A tool runs under a service account. Whatever that account may do, the model may now do. A tool layer running with admin rights hands admin rights to the model.

Confirmation. Some calls execute immediately. Others should wait for a person. Human-in-the-loop is implemented as exactly that: a tool call that pauses until someone approves, typically at an irreversible step such as sending, paying or deleting.

Logging. Every call with its arguments and its result, tied to the case that triggered it. This is the audit trail, and it exists only if it was built.

Cost. Each tool round trip is another model call. An agent that makes many calls per case pays for each one, in money and in latency, which is one reason to put deterministic work into code rather than asking the model to do it step by step.

Example: an invoice-check agent

The agent receives supplier invoices and checks them against orders. Its tools are get_order, which fetches an order by number, extract_lines, which reads the invoice PDF into a table, compare_lines, which is plain code that diffs two tables, and flag_for_review, which takes a reason.

The model's job is judgement: work out which order the invoice refers to, call the comparison, read the differences, decide whether a discrepancy is within tolerance or must be flagged. The arithmetic is in code. The model never adds up a column.

Where tool calling fails

Models sometimes call a tool with wrong arguments, or call a tool that does not exist. Schema validation catches the first. A strict tool list, with no free-form fallback, catches the second.

Error messages matter more than they seem. A tool that answers a bad call with "expected an order number in the format A-12345" leads to a corrected second attempt. A tool that merely fails leads to an invented answer. When an agent is built as an agent graph, each tool node can be tested on its own with exactly these bad inputs.

The tool layer is where most of the engineering in an agent project actually happens. It is what we mean when we say an agent is built into a client's stack rather than placed next to it; see software development.