R_REDDYX.XYZ
Agents2026-06-15

Function calling in 2026: how to teach LLM to use tools

Function calling in 2026: how to teach LLM to use tools
TL;DR: Function calling is when an LLM doesn't write a text response but returns a structured call to your function as described in a JSON schema. In 2026 this is the foundation of any agent: the model decides which tool to invoke, you execute the code and return the result back into the dialog. Below — the working cycle, Anthropic and OpenAI formats, typical pitfalls, and how to raise call reliability to 95%+.

WHAT IS FUNCTION CALLING AND WHY IT'S NEEDED

A bare LLM does one thing — generates text. It doesn't know the weather right now, doesn't see your DB, doesn't reliably calculate large numbers. Function calling (aka tool use) plugs this hole: you describe a set of tools, the model itself decides when and with what arguments to call them, while execution stays with your code.

The key point that confuses beginners: the model does not execute the function. It merely returns JSON like "I want to call get_weather with argument city=Berlin". The actual HTTP request, DB trip, script launch — all that you do. The model is a dispatcher, not an executor. This separation is the security foundation of the whole scheme.

Why did this take off as a separate feature rather than "let's ask the model to return JSON in text"? Because modern models are fine-tuned on tool use: they emit calls in a separate response channel, respect the argument schema, and understand what to do with the result. Regex-parsing JSON from free text in 2026 is an anti-pattern.

WHAT THE CALL CYCLE LOOKS LIKE

A full tool-use round trip is always at least two API calls. The model doesn't answer the user immediately if it needs a tool:

  1. Request. You send the user message plus a list of tool descriptions.
  2. Model decision. The model returns not text but a call request: function name + arguments, already validated against your JSON schema.
  3. Execution. Your code runs the function — hits an API, DB, filesystem.
  4. Return result. You append the result to the dialog history and call the model again.
  5. Final answer. The model formulates a human-readable answer based on the tool data. Or decides to call another tool — then the cycle repeats.

It's exactly this looped nature ("call → result → call → …") that turns a plain chat into an agent. An agent is function calling in a loop with a stopping condition.

JSON SCHEMA — THE HEART OF TOOL DESCRIPTION

The model understands a tool only as well as you described it. The description is a JSON schema: name, human-readable description, types and requiredness of parameters. The quality of this text directly determines whether the model will hit the right function with the right arguments.

Practical minimum that actually affects hit rate:

  • description of the function itself — 3-4 sentences on when to call it, not just what it does. Write as if for a junior.
  • description of each parameter — format, units, examples. "City name in English" is better than "city".
  • enum where values are fixed. This cuts argument hallucinations almost to zero.
  • required — list explicitly. Otherwise the model will start inventing optional fields.
import anthropic

client = anthropic.Anthropic()

tools = [
    {
        "name": "get_weather",
        "description": "Возвращает текущую погоду в городе. "
                       "Вызывай, когда пользователь спрашивает про погоду, "
                       "температуру или осадки в конкретном месте.",
        "input_schema": {
            "type": "object",
            "properties": {
                "city": {
                    "type": "string",
                    "description": "Название города на английском, напр. 'Berlin'"
                },
                "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"],
                    "description": "Единица измерения температуры"
                }
            },
            "required": ["city"]
        }
    }
]

resp = client.messages.create(
    model="claude-opus-4-5",
    max_tokens=1024,
    tools=tools,
    messages=[{"role": "user", "content": "Какая погода в Берлине?"}]
)

# resp.stop_reason == "tool_use" — модель просит вызвать функцию
for block in resp.content:
    if block.type == "tool_use":
        print(block.name, block.input)   # get_weather {'city': 'Berlin'}

Next you execute get_weather('Berlin'), and return the result in a new message with role user and a tool_result block, referencing the tool_use_id. On the second request the model will output text like "In Berlin it's currently +12°C, cloudy".

FORMATS: ANTHROPIC VS OPENAI VS OPEN-SOURCE

Concept is one, wrappers differ. If you are building a multi-provider agent, knowing the differences is mandatory — otherwise you catch schema errors when switching backend.

AspectAnthropic (Claude)OpenAIOpen-source (via vLLM/Ollama)
Schema keyinput_schemaparametersUsually OpenAI-compatible
Tool wrapperflat object{"type":"function","function":{…}}depends on model template
Completion flagstop_reason: tool_usefinish_reason: tool_callsvaries
Parallel callsyes, out of the boxyespartially, depends on model
Strict schema compliancehighStructured Outputs (strict)lower, needs a validator

Trend 2026: most inference servers (vLLM, llama.cpp, Ollama) provide an OpenAI-compatible endpoint with tool calling, so the de facto standard description format is OpenAI. However, the hit quality on open-source 7-8B models is noticeably lower: without external validation of arguments, do not go to production. Fresh releases of orchestration frameworks and local engines are conveniently tracked in the REDDYX catalog.

PARALLEL CALLS AND CHAINS

Two behavior models that are important to distinguish:

  • Parallel calls. For the request «compare weather in Berlin and Tokyo», the model returns two independent tool_use blocks in one turn. You execute them concurrently (asyncio, thread pool) and return both results at once. It saves both time and tokens.
  • Sequential chains. «Find the cheapest flight and book it» — first search_flights, then based on the result book_flight. The model cannot call the second until it sees the result of the first. This is already a full-fledged agent cycle.

According to community observations, about half of real-world agent tasks contain at least one chain of two or more steps. Therefore, design the orchestrator as a loop `while stop_reason == tool_use`, not as a single exchange.

RELIABILITY: HOW TO INCREASE HIT RATE

Top models of 2026 hit the correct tool with correct arguments in about 90-97% of cases on simple sets — but as the number of tools grows, accuracy drops. What really helps:

  1. Keep the tool set narrow. 5-8 functions — comfortable. 30+ — the model starts to get confused. If you have many tools, introduce a router: first choose a category, then a narrow subset.
  2. Validate arguments with your code. Never trust that city is really a city. The model may hallucinate IDs that are not in the database.
  3. Return meaningful errors. If a function fails, return in tool_result the error text with is_error: true. The model is great at retrying with corrected arguments.
  4. Don't hide state in description. Tool descriptions are cached and must be stable. Pass dynamics in the system prompt or arguments.
  5. Test on edge cases. Empty input, ambiguous request, request where the tool is not needed at all. A good schema forces the model to respond with text rather than calling the function uselessly.

MCP AND THE FUTURE OF TOOLS

The major shift of the last one and a half years — Model Context Protocol (MCP). Instead of manually describing each tool in code for each model, you spin up an MCP server that provides a standardized list of tools. The client (your agent, IDE, desktop application) connects and receives a ready set.

What does this change in practice:

  • Tools become reusable between projects and providers — one MCP server for Postgres serves any client.
  • An ecosystem of ready servers appears: files, DB, search, Git, browser. Many open servers and connectors can be conveniently monitored via the REDDYX catalog.
  • Function calling from the «API feature» becomes a transport layer between the model and the outside world.

Under the hood, MCP still relies on the same JSON schema and the same call-result cycle. Once you grasp basic tool use, you’ll master MCP in an evening.

Frequently Asked Questions

How does function calling differ from structured output?

Structured output forces the model to return an answer strictly according to the given JSON schema — this is about the format of the final answer. Function calling is about the model deciding to call an external function and get its result. They are often combined: strict mode guarantees that the call arguments exactly match the schema.

Does the model execute my function?

No. The LLM only returns the function name and arguments in a structured form. Actual execution — HTTP request, database query, code execution — is done by your backend. This is a key security boundary: the model never gets direct access to your systems.

How many tools can be connected to one model?

Technically dozens, but selection accuracy drops as the number grows. Practical comfort — 5-8 functions. With a larger number, build a two-level router: the model first selects a category, then works with a narrow subset of tools.

What to do if the model calls the wrong tool?

Improve the description — add explicit conditions ‘when to call’ and negative examples. Use enum for fixed values and validate arguments with your code. On error, return its text in tool_result — the model can self-correct on the next turn.

Function calling — is the skill that separates a chatbot from a true agent: learn to describe tools cleanly, run the call-result cycle and validate arguments — and then everything scales from a single function to a whole MCP park. Want to see fresh tools, framework orchestrators, and MCP servers first — Telegram-канал REDDYX AI — new repositories every 30-60 minutes.

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram

← All articles

Function calling in 2026: how to teach LLM to use tools | REDDYX AI