c56bef871b
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled
173 lines
8.7 KiB
Plaintext
173 lines
8.7 KiB
Plaintext
---
|
|
title: "MirageShellTool"
|
|
id: mirageshelltool
|
|
slug: "/mirageshelltool"
|
|
description: "A Tool that gives Agents a bash shell over a Mirage unified virtual filesystem, mounting backends like S3, Google Drive, and Postgres as one file tree."
|
|
---
|
|
|
|
# MirageShellTool
|
|
|
|
A Tool that gives Agents a bash shell over a [Mirage](https://github.com/strukto-ai/mirage) unified virtual filesystem, mounting backends like S3, Google Drive, and Postgres as one file tree.
|
|
|
|
<div className="key-value-table">
|
|
|
|
| | |
|
|
| --- | --- |
|
|
| **Mandatory init variables** | `workspace`: A `MirageWorkspace` describing the mount tree the Agent can access. |
|
|
| **API reference** | [Mirage](/reference/integrations-mirage) |
|
|
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/mirage |
|
|
| **Package name** | `mirage-haystack` |
|
|
|
|
</div>
|
|
|
|
## Overview
|
|
|
|
`MirageShellTool` hands an [Agent](../../pipeline-components/agents-1/agent.mdx) a single shell over a [Mirage](https://github.com/strukto-ai/mirage) *unified virtual filesystem*: one directory tree that mounts heterogeneous backends — object storage, databases, SaaS apps, and local disk — side by side. Instead of pre-loading file contents into the prompt, the Agent explores the mounted data itself by running ordinary bash commands (`ls`, `cat`, `grep`, `wc`, …). Command output is normalized to text and truncated before it reaches the model.
|
|
|
|
The tool is backed by two serializable helpers you compose the workspace with:
|
|
|
|
- **`MirageWorkspace`**: A description of the mount tree that lazily builds a live Mirage workspace. It is the shared backend behind the tool, and you can also use it directly — without an Agent — through its `run()` / `run_async()` methods.
|
|
- **`MirageMount`**: A declarative description of a single backend: *where* it is mounted (`path`), *which* backend it is (`resource`, a Mirage registry name such as `"s3"`, `"gdrive"`, `"postgres"`, `"disk"`, or `"ram"`), and *how* it is configured (`config`). Credentials can be passed as Haystack `Secret`s and are resolved only when the live workspace is built.
|
|
|
|
Because every backend is mounted the same way, one tool gives the Agent uniform access to S3, Google Drive, Slack, Gmail, Redis, Postgres, local disk, and more — swap a `MirageMount` and the Agent's commands stay the same. Mirage never shells out to the host, so the Agent's blast radius is confined to the mounts you attach (see [Security model](#security-model)).
|
|
|
|
### Parameters
|
|
|
|
- `workspace` is _mandatory_ and must be a `MirageWorkspace` describing the mounts the Agent can access.
|
|
- `name` is _optional_ and defaults to `"mirage_shell"`. Sets the tool name exposed to the LLM.
|
|
- `description` is _optional_. A custom tool description; when not set, one is generated from the mount tree.
|
|
- `invocation_timeout` is _optional_ and defaults to `60.0`. Maximum seconds to wait for a command to finish.
|
|
- `max_output_chars` is _optional_ and defaults to `20000`. Command output is truncated to this many characters before being returned to the model.
|
|
- `allowed_commands` is _optional_. If set, only these command names may run (for example `["ls", "cat", "grep"]`). See [Security model](#security-model).
|
|
- `denied_paths` is _optional_. If set, any command referencing one of these path substrings is rejected.
|
|
|
|
## Usage
|
|
|
|
Install the Mirage integration to use `MirageShellTool`:
|
|
|
|
```shell
|
|
pip install mirage-haystack
|
|
```
|
|
|
|
### With an Agent
|
|
|
|
You can use `MirageShellTool` with the [Agent](../../pipeline-components/agents-1/agent.mdx) component. The Agent starts the workspace on `warm_up()`, then drives the tool with bash to answer questions by exploring the mounted files itself.
|
|
|
|
The example below builds a small "log triage" Agent. A directory of log files is mounted read-only, and the Agent inspects it with bash to answer a question. It uses a local `disk` mount so it is fully self-contained; swap the `MirageMount` for `s3`, `gdrive`, `postgres`, … to point the same Agent at a different backend.
|
|
|
|
```python
|
|
import os
|
|
import tempfile
|
|
|
|
from haystack.components.agents import Agent
|
|
from haystack.components.generators.chat import OpenAIChatGenerator
|
|
from haystack.dataclasses import ChatMessage
|
|
|
|
from haystack_integrations.tools.mirage import (
|
|
MirageMount,
|
|
MirageShellTool,
|
|
MirageWorkspace,
|
|
)
|
|
|
|
# Create some sample data on disk (in a real setup this already exists).
|
|
data_dir = tempfile.mkdtemp(prefix="mirage-logs-")
|
|
with open(os.path.join(data_dir, "api.log"), "w") as fh:
|
|
fh.write(
|
|
"INFO request /health 200\nERROR db connection timeout\nERROR db connection timeout\n",
|
|
)
|
|
with open(os.path.join(data_dir, "worker.log"), "w") as fh:
|
|
fh.write("INFO job 41 done\nERROR job 42 failed: OutOfMemory\n")
|
|
|
|
# Describe the workspace. The read-only mount is the authoritative write boundary:
|
|
# Mirage refuses any write to it regardless of the command the model chooses.
|
|
workspace = MirageWorkspace(
|
|
mounts=[
|
|
MirageMount(
|
|
path="/logs",
|
|
resource="disk",
|
|
config={"root": data_dir},
|
|
read_only=True,
|
|
),
|
|
],
|
|
)
|
|
|
|
tool = MirageShellTool(workspace, allowed_commands=["ls", "cat", "grep", "head", "wc"])
|
|
|
|
agent = Agent(
|
|
chat_generator=OpenAIChatGenerator(model="gpt-4o-mini"),
|
|
tools=[tool],
|
|
system_prompt=(
|
|
"You are a log-triage assistant. A virtual filesystem is available through the `mirage_shell` "
|
|
"tool. Use bash commands (ls, cat, grep, wc, ...) to inspect the mounted files under /logs before "
|
|
"answering. Base your answer only on what the files actually show."
|
|
),
|
|
)
|
|
|
|
response = agent.run(
|
|
messages=[
|
|
ChatMessage.from_user(
|
|
"Across all files in /logs, what is the single most common ERROR message, "
|
|
"and how many times does it occur?",
|
|
),
|
|
],
|
|
)
|
|
print(response["last_message"].text)
|
|
|
|
tool.close()
|
|
```
|
|
|
|
### Running commands without an Agent
|
|
|
|
`MirageWorkspace` can be used on its own, which is handy for testing a mount tree or building non-agentic pipelines. When using it standalone, call `warm_up()` before the first invocation (or let the first `run()` build it lazily) and `close()` when you are done to release resources.
|
|
|
|
```python
|
|
from haystack_integrations.tools.mirage import MirageMount, MirageWorkspace
|
|
|
|
workspace = MirageWorkspace(
|
|
mounts=[
|
|
MirageMount(path="/data", resource="ram"), # in-memory scratch space
|
|
MirageMount(
|
|
path="/s3",
|
|
resource="s3",
|
|
config={"bucket": "my-bucket"},
|
|
read_only=True,
|
|
),
|
|
],
|
|
)
|
|
print(workspace.run("ls /s3"))
|
|
print(workspace.run("grep -r alert /s3/logs | wc -l"))
|
|
workspace.close()
|
|
```
|
|
|
|
### Mounting credentialed backends
|
|
|
|
Backends that need credentials take them through `config`. Pass secrets as Haystack `Secret`s so they are resolved only when the live workspace is built and are never serialized in plaintext:
|
|
|
|
```python
|
|
from haystack.utils import Secret
|
|
from haystack_integrations.tools.mirage import MirageMount
|
|
|
|
MirageMount(path="/data", resource="ram") # in-memory scratch
|
|
MirageMount(path="/local", resource="disk", config={"root": "/srv/data"}) # local disk
|
|
MirageMount(path="/s3", resource="s3", config={"bucket": "my-bucket"}, read_only=True)
|
|
MirageMount(
|
|
path="/drive",
|
|
resource="gdrive",
|
|
config={
|
|
"client_id": "...",
|
|
"refresh_token": Secret.from_env_var("GDRIVE_REFRESH_TOKEN"),
|
|
},
|
|
read_only=True,
|
|
)
|
|
```
|
|
|
|
Discover the backend names available in your Mirage install with `MirageMount.available_resources()`; the config keys each backend expects come from that backend's Mirage config class.
|
|
|
|
## Security model
|
|
|
|
Mirage never shells out to the host: every command runs inside Mirage's own virtual-filesystem interpreter, so an Agent's blast radius is confined to the mounts you attach. Two controls shape what an Agent can do:
|
|
|
|
- **Per-mount read-only mode** (`MirageMount(..., read_only=True)`) is the authoritative write boundary. Mirage refuses any write to a read-only mount regardless of the command used — this is how you prevent modification or deletion. Mount anything the Agent should not change as read-only.
|
|
- **The command allowlist** (`allowed_commands`) restricts *which* commands may run. It is enforced against every command Mirage would execute, including commands nested inside `$(...)`, backticks, `<(...)`, and subshells, so `ls "$(rm x)"` is rejected unless `rm` is also allowed. Treat it as a best-effort filter to steer the Agent, not a sandbox: allowing a command that itself runs other commands (`eval`, `bash`, `sh`, `source`, `xargs`, `timeout`) effectively allows anything, so do not list those for untrusted or hosted use.
|
|
- **`denied_paths`** rejects any command whose text references one of the given path substrings.
|