deepset-ai--haystack/docs-website/docs/pipeline-components/generators/guides-to-generators/function-calling.mdx

---
title: "Function Calling"
id: function-calling
slug: "/function-calling"
description: "Learn about function calling and how to use it in Haystack."
---

# Function Calling

Learn about function calling and how to use it in Haystack.

Function calling is a powerful feature that significantly enhances the capabilities of Large Language Models (LLMs). It enables better functionality, immediate data access, and interaction, and sets up for integration with external APIs and services. Function calling turns LLMs into adaptable tools for various use case scenarios.

## Use Cases

Function calling is useful for a variety of purposes, but two main points are particularly notable:

1. **Enhanced LLM Functionality**: Function calling enhances the capabilities of LLMs beyond just text generation. It allows to convert human-generated prompts into precise function invocation descriptors. These descriptors can then be used by connected LLM frameworks to perform computations, manipulate data, and interact with external APIs. This expansion of functionality makes LLMs adaptable tools for a wide array of tasks and industries.
2. **Real-Time Data Access and Interaction**: Function calling lets LLMs create function calls that access and interact with real-time data. This is necessary for apps that need current data, like news, weather, or financial market updates. By giving access to the latest information, this feature greatly improves the usefulness and trustworthiness of LLMs in changing and time-critical situations.

:::note[Important Note]

The model doesn't actually call the function. Function calling returns the name of a function and the arguments to invoke it. The actual invocation is performed by your code (or by a Haystack component such as `ToolInvoker` or `Agent`).
:::

## Example

Let's walk through function calling in Haystack in three steps: define a tool, let the LLM pick it, and then actually invoke it.

:::tip[Real-world usage]

We split function calling into separate steps below for clarity. In real applications, the [`Agent`](../../agents-1/agent.mdx) component handles the full loop for you. See [Using the Agent component](#using-the-agent-component) at the end of this section.
:::

### 1. Define a Tool

The simplest way to expose a Python function to an LLM is the `@tool` decorator. Type hints (and `Annotated` metadata) are used to automatically build the JSON schema the LLM needs.

```python
from typing import Annotated, Literal
from haystack.tools import tool


@tool
def get_weather(
    city: Annotated[str, "The city to get the weather for"],
    unit: Annotated[Literal["celsius", "fahrenheit"], "Temperature unit"] = "celsius",
):
    """Get the current weather for a city."""
    # In a real application, this would call a weather API.
    return {"city": city, "temperature": 18, "unit": unit, "condition": "Partly Cloudy"}
```

This produces a [`Tool`](../../../tools/tool.mdx) instance you can pass directly to a ChatGenerator. If you prefer to build it explicitly, you can also instantiate `Tool(...)` yourself.

### 2. Let the LLM Pick the Tool

Pass the tool to a ChatGenerator and run it on a user message. The model decides whether to call the tool and, if so, with which arguments. The result lives on `replies[0].tool_calls`.

```python
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage

generator = OpenAIChatGenerator(tools=[get_weather])
response = generator.run(
    messages=[ChatMessage.from_user("What's the weather like in Berlin?")],
)

print(response["replies"][0].tool_calls)
# >> [ToolCall(tool_name='get_weather', arguments={'city': 'Berlin', 'unit': 'celsius'}, id='call_...')]
```

At this point the model has only *requested* a call. Nothing has been executed yet.

### 3. Actually Invoke the Tool

To execute the requested tool calls, use the `ToolInvoker` component. It takes a list of [`ChatMessage`](../../../concepts/data-classes/chatmessage.mdx) objects containing tool calls, runs the corresponding tools, and returns new `ChatMessage` objects (with role `tool`) carrying the results as `ToolCallResult`s. To get a final natural-language answer, feed those results back to the ChatGenerator.

```python
from haystack.components.tools import ToolInvoker

invoker = ToolInvoker(tools=[get_weather])

# Run the tools requested by the assistant.
tool_messages = invoker.run(messages=response["replies"])["tool_messages"]

# Send the full conversation back to the model for the final reply.
messages = [
    ChatMessage.from_user("What's the weather like in Berlin?"),
    *response["replies"],  # assistant message with the tool call
    *tool_messages,  # tool result messages
]
final = generator.run(messages=messages)
print(final["replies"][0].text)
# The weather in Berlin is partly cloudy with a temperature of 18°C.
```

#### Using the Agent component

In real applications, you typically don't manage the loop yourself: the [`Agent`](../../agents-1/agent.mdx) component does it for you. Internally, it wraps a ChatGenerator and a `ToolInvoker` and iterates until the model produces a final answer.

```python
from haystack.components.agents import Agent
from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.dataclasses import ChatMessage

agent = Agent(
    chat_generator=OpenAIChatGenerator(),
    tools=[get_weather],
)

response = agent.run(
    messages=[ChatMessage.from_user("What's the weather like in Berlin?")],
)
print(response["messages"][-1].text)
# The weather in Berlin is partly cloudy with a temperature of 18°C.
```

## Additional References

- [`Tool`](../../../tools/tool.mdx), [`Toolset`](../../../tools/toolset.mdx), and [`ComponentTool`](../../../tools/componenttool.mdx) cover the building blocks for defining tools, grouping them, and exposing existing Haystack components as tools.
- To connect an LLM to external services or other Haystack applications, the recommended approach is the [Model Context Protocol (MCP)](https://modelcontextprotocol.io/). Haystack provides [`MCPTool`](../../../tools/mcptool.mdx) and [`MCPToolset`](../../../tools/mcptoolset.mdx) for connecting to MCP servers over Streamable HTTP or stdio. Install them with `pip install mcp-haystack`.

:notebook: **Tutorial:** [Building a Chat Application with Function Calling](https://haystack.deepset.ai/tutorials/40_building_chat_application_with_function_calling)