What Is Function Calling in LLMs? Tool Use Explained

Function calling is a mode where a large language model, instead of only producing free-form text, outputs a structured request to invoke a specific function with specific arguments — and your application actually runs it. The model doesn’t execute code itself; it decides when a function is needed and what arguments to pass, and the calling application is responsible for running the function and feeding the result back.

The problem it solves

An LLM generates text by predicting tokens. Left alone, it can only answer from what it learned during training and whatever’s in its context window — it has no way to check today’s exchange rate, query a database, or send an email. Function calling bridges that gap: the model is given a list of available functions (name, description, and a schema for their parameters), and when a user’s request needs one, the model responds with a structured call instead of a guess.

For example, given a get_weather(location: string) function, a user asking “what’s the weather in Tokyo” causes the model to emit something like {"name": "get_weather", "arguments": {"location": "Tokyo"}} rather than hallucinating an answer.

How the round trip works

Function calling is a multi-step exchange between your application and the model, not a single request:

1. Describe available functions. Your app sends the model a list of functions it can call, each with a name, description, and a parameter schema (commonly JSON Schema).
2. Model decides. Given the user’s message and the function list, the model either answers directly or responds with a function call — a name and a JSON object of arguments matching the schema.
3. Application executes. Your code parses the model’s structured output, validates the arguments, and actually runs the corresponding function — hitting a database, calling an API, whatever the function does.
4. Result goes back to the model. The function’s return value is appended to the conversation, and the model is called again to produce a final natural-language answer that incorporates the result.

The model never runs anything itself. All the actual execution — and all the actual security boundary — lives in your application code.

Function calling vs an AI agent

These terms get used almost interchangeably, but function calling is a building block, not the whole system. An AI agent is a loop that repeatedly calls a model, lets it choose from a set of tools, executes those tools, and feeds results back — often across many iterations, planning and adjusting as it goes. Function calling is the mechanism that makes each individual step of that loop possible: it’s how the model expresses “run this tool with these arguments” in a format your code can parse reliably. A single function call is one turn; an agent is the orchestration loop built on top of many of them.

The Model Context Protocol standardizes how those tool definitions and results are described and passed around, so the same tool works across different agent frameworks without custom glue code for each one.

Why structured output matters here

Before native function calling support, developers tried to get the same behavior by asking the model to output JSON in its regular text response and then parsing that text — fragile, since the model could add explanatory prose around the JSON, use inconsistent formatting, or produce invalid JSON outright. Native function calling instead constrains generation so the arguments conform to the declared schema, which is far more reliable for anything a program needs to parse deterministically.

This same schema-constrained approach is what lets a model return validated structured data even outside tool-use scenarios — extracting a specific set of fields from a document, for instance, is a lightweight variant of the same underlying mechanism.

Practical considerations

• Describe functions precisely. Vague names and descriptions lead to the model calling the wrong function or guessing at arguments. Treat the description the way you’d write a docstring for a human developer.
• Validate everything. The model’s arguments are a suggestion your code must still validate before execution — never trust generated input as if it came from a typed compiler.
• Watch context and cost. Every function definition you provide consumes context window space and gets re-sent on each turn; keep the tool list scoped to what’s relevant. If you’re tracking API spend, our LLM token cost calculator can help estimate what a chatty tool-heavy conversation costs across turns.
• Handle the “no function fits” case. A well-designed system prompt should make it clear the model can decline to call anything and just answer directly when no function applies.

The takeaway

Function calling gives an LLM a structured way to say “call this function with these arguments,” turning it from a text generator into a component that can trigger real actions in your application — with the actual execution, and the actual trust boundary, staying in your code. It’s the low-level primitive that makes agent loops and tool-using assistants possible, and getting the function descriptions and argument validation right matters more to reliability than which model you’re using.