Function calling (tool use)

Overview

Function calling, also called tool use or tool calling, is the ability of a language model to ask the surrounding program to run a function on its behalf. The model does not run anything itself. It writes a structured request, for example "call get_weather with city set to Istanbul", and the program runs the function, then gives the result back to the model.

A good comparison is a manager and an assistant. The manager cannot reach the filing cabinet from the desk, but can write a precise note: "Bring me the March invoices for client 4417." The assistant fetches the folder and hands it over, and the manager continues working with the real documents. The manager decides what is needed; the assistant does the fetching. In function calling the model is the manager writing precise notes, and your code is the assistant.

Function calling is the basic building block of every AI agent. Retrieval, database queries, code execution, sending email and operating other software all reach the model through this one mechanism.

What it is

The application gives the model a list of tool definitions along with the conversation. Each definition has:

• a name, such as search_orders;
• a description that says what the tool does and when to use it;
• an input schema, usually JSON Schema, that lists the arguments, their types and which are required.

When the model decides a tool would help, it returns a tool call instead of, or in addition to, ordinary text. The tool call names the tool and gives arguments as JSON that should match the schema. The application runs the function, sends back a tool result tied to that call, and asks the model to continue.

Where it came from

Early approaches asked the model to write a special text format, such as an action line inside its reasoning, and then parsed that text with regular expressions. The ReAct pattern from 2022 popularised this style. It worked, but parsing was fragile and models often broke the format.

In June 2023 OpenAI added function calling to its chat API, with models fine-tuned to emit structured calls against JSON Schema definitions. Other providers followed with their own tool use APIs during 2023 and 2024, and models were trained specifically to choose tools, fill arguments and use results. Later additions included parallel calls, strict schema adherence, tool choice controls and built-in server-side tools such as web search and code execution. Protocols such as MCP then standardised how tools are discovered and reached, while function calling remained the way the model expresses its choice.

How it works

The call loop

A tool-using exchange is a loop:

1. The application sends the conversation and the tool definitions to the model.
2. The model returns either a final answer or one or more tool calls.
3. For each tool call, the application validates the arguments, runs the function and records the result.
4. The application sends the results back to the model, linked to the call ids.
5. The model continues, perhaps calling more tools, until it gives a final answer or a limit is reached.

This loop is the heart of an agent. Everything else, memory, planning, multiple agents, sits on top of it.

Tool choice

Most APIs let the application control whether tools are used. The model can decide freely, can be required to call some tool, can be forced to call a specific tool, or can be told not to call tools at all. Forcing a specific tool is useful for extraction tasks where you always want structured output.

Parallel calls

A model can return several tool calls in one turn when they are independent, for example fetching the weather in three cities. The application can run them concurrently and return all results together, which cuts latency. When calls depend on each other, the model makes them one after another across turns.

Validation and strictness

Models usually produce valid arguments, but not always. A field can be missing, a date can be in the wrong format, or an enum can get an unexpected value. Many APIs offer a strict mode that constrains generation so arguments always match the schema. Even then, the application must check business rules: that an account belongs to the user, that an amount is within limits, that a date range makes sense.

Server-side tools

Some providers run certain tools themselves, such as web search, code execution or file search. The model calls them the same way, but the provider executes them and returns results inside the response. Your application only handles the tools it defines.

Key concepts and terms

A worked example

A finance assistant helps staff answer questions about invoices. The application defines two tools.

[
  {
    "name": "search_invoices",
    "description": "Search invoices by customer and date range. Use for any question about amounts, due dates or payment status. Returns up to 20 invoices.",
    "input_schema": {
      "type": "object",
      "properties": {
        "customer": {"type": "string"},
        "from_date": {"type": "string", "format": "date"},
        "to_date": {"type": "string", "format": "date"},
        "status": {"type": "string", "enum": ["paid", "unpaid", "overdue", "any"]}
      },
      "required": ["customer"]
    }
  },
  {
    "name": "send_reminder",
    "description": "Send a payment reminder email for one overdue invoice. Only use after the user confirms.",
    "input_schema": {
      "type": "object",
      "properties": {"invoice_id": {"type": "string"}},
      "required": ["invoice_id"]
    }
  }
]

A user asks: "Which invoices for Kara Logistics are overdue this quarter?"

1. The model returns a call to search_invoices with the customer, the quarter's start and end dates and status overdue.
2. The application checks that the user may see this customer, runs the query and returns two invoices.
3. The model answers with the two invoice numbers, amounts and days overdue, and asks whether to send reminders.
4. The user says yes for the older one. The model calls send_reminder. Because this tool has a side effect, the application shows a confirmation before running it.
5. The application returns a success result, and the model confirms the reminder was sent.

The model never touched the database or the email system. It only wrote two precise requests, and the application kept full control.

Strengths and limitations

Strengths

• Grounding. Answers can use live, exact data instead of the model's memory.
• Action. Models can trigger real work: tickets, emails, updates, code runs.
• Control. The application decides which tools exist, validates every call and can require approval.
• Structure. JSON arguments are easy to log, test and audit.
• Composability. Many tools can be combined in one loop to solve multi-step problems.

Limitations

• Wrong tool or wrong arguments. Models can pick a poor tool or fill arguments incorrectly, especially with vague descriptions.
• Context cost. Every tool definition takes tokens on every turn. Large tool sets slow responses and raise cost.
• Loops. Without limits, a model can repeat calls without progress.
• Injection through results. Tool outputs can contain instructions that try to steer the model.
• Latency. Each round trip adds time. Multi-step tasks can take many seconds.

How it compares with alternatives

Role in tabular foundation models

Function calling is how an agent puts a tabular model to work. A tool such as predict can accept a table reference, a target column and the rows to score, and return predictions with uncertainty. An agent can first call a SQL tool to select the right context rows, then call the prediction tool, then explain the result. Because tabular outputs are numbers and labels, returning them as structured tool results rather than prose keeps the agent's later reasoning accurate.

Practical guidance and best practices

1. Name tools by intent. find_customer_orders is clearer than api_v2_get.
2. Write descriptions as instructions. Say when to use the tool, when not to, what it returns and any side effects.
3. Keep schemas tight. Use enums, formats and required fields. Describe every argument.
4. Keep the tool set small. Offer only tools relevant to the current task; filter or search tools when there are many.
5. Validate on the server. Check permissions and business rules for every call, even in strict mode.
6. Gate side effects. Require confirmation for writes, payments and messages.
7. Return compact results. Summarise or paginate large outputs and include identifiers for follow-up calls.
8. Return helpful errors. Tell the model what was wrong and how to fix the call.
9. Set step and time limits. Stop runaway loops and return a clear message to the user.
10. Evaluate tool selection. Keep a test set of questions with expected tool calls and run it after every change.

Common mistakes

• Overlapping tools. Two tools that do almost the same thing confuse the model.
• Missing argument descriptions. The model guesses formats and gets them wrong.
• Trusting arguments. Running a call without permission checks because "the model chose it".
• Dumping raw payloads. Large JSON responses waste context and hide the key fields.
• No call ids in logs. Debugging becomes guesswork.

Frequently asked questions

Does the model actually run the function?

No. The model only writes a structured request. Your application runs the function, enforces permissions and sends back the result.

What is the difference between function calling and tool use?

They are the same idea under different names. Some providers say function calling, others say tool use or tool calling.

How many tools can I give a model?

There is no fixed limit, but accuracy and cost get worse as the list grows. Many production agents work best with fewer than twenty active tools, using filtering or tool search when more are available.

What is strict mode?

A setting that constrains generation so tool arguments always match the JSON Schema. It removes format errors but not logic errors, so server-side validation is still needed.

Can a model call several tools at once?

Yes. Many models can return parallel tool calls when the calls are independent. The application can run them concurrently and return all results.

How does function calling relate to MCP?

Function calling is how a model chooses and fills a tool call. MCP is how an application discovers and reaches tools on external servers. Hosts use MCP to load tools and function calling to let the model use them.

How do I stop an agent from looping?

Set a maximum number of steps and a time budget, detect repeated identical calls, and return a clear message when a limit is reached.

Glossary

• Agent: a model-driven loop that calls tools and reads results.
• Argument: a named input value for a tool call.
• Enum: a field restricted to a fixed list of values.
• JSON Schema: a vocabulary for describing JSON structure.
• Latency: the time between request and response.
• Prompt injection: hostile instructions hidden in content the model reads.
• Side effect: a change in the outside world caused by a tool.
• Token: a unit of text that models read and write.

Summary

Function calling lets a model ask your program to run a tool, using a name, a description and a JSON Schema for arguments. The program validates and runs the call, returns the result, and the loop continues until the model answers. It is the foundation of agents, retrieval and automation. Good tool names, precise descriptions, tight schemas, server-side checks, approval for side effects and step limits turn it from a demo feature into a dependable system.

Related

It belongs to Protocols in my stack, under Agentic AI. Others in the same group: