strukto-ai--mirage/docs/python/resource/new.mdx

---
title: Adding a New Resource
icon: plus
description: Step-by-step guide for implementing a new MIRAGE resource.
---

This guide covers what you need to implement when adding a new resource
to MIRAGE. Use the Discord or Slack resources as reference.

## File Structure

```text
mirage/
  resource/<name>/
    __init__.py          # lazy-loading exports
    config.py            # Pydantic config (credentials)
    <name>.py            # BaseResource subclass
  accessor/<name>.py     # Accessor wrapping config
  core/<name>/
    __init__.py
    _client.py           # HTTP client (get/post with rate limiting)
    readdir.py           # directory listing
    read.py              # file reading
    stat.py              # file metadata
    scope.py             # scope detection
    glob.py              # glob pattern resolution
    ...                  # data fetching modules (history, search, post, etc.)
  ops/<name>/
    __init__.py           # exports OPS list
    read.py               # @op wrapper
    readdir.py            # @op wrapper
    stat.py               # @op wrapper
  commands/builtin/<name>/
    __init__.py           # exports COMMANDS list
    _plan.py              # cost estimation helpers
    cat.py, ls.py, ...    # standard commands
    <name>_send.py, ...   # resource-specific commands
```

## Implementation Steps

### 1. Config, Accessor, ResourceName

Create a Pydantic config model to hold credentials and an accessor
class that wraps it.

```python
# mirage/resource/<name>/config.py
from pydantic import BaseModel


class MyConfig(BaseModel):
    token: str
```

```python
# mirage/accessor/<name>.py
from mirage.accessor.base import Accessor
from mirage.resource.<name>.config import MyConfig


class MyAccessor(Accessor):

    def __init__(self, config: MyConfig) -> None:
        self.config = config
```

Add `MY_RESOURCE = "<name>"` to the `ResourceName` enum in `mirage/types.py`.

### 2. HTTP Client

Wrap the resource's API with rate-limit handling. All resources follow
the same pattern: async get/post functions with retry on 429.

```python
# mirage/core/<name>/_client.py
async def my_get(config, endpoint, params=None) -> dict: ...
async def my_post(config, endpoint, body=None) -> dict: ...
```

### 3. Core VFS

Implement the three VFS operations that map API data to a filesystem:

- **`readdir.py`** -- Returns `list[str]` of child paths for a directory.
- **`read.py`** -- Returns `bytes` content of a file.
- **`stat.py`** -- Returns `FileStat` with name, type, and extras (e.g., IDs).

All three accept `(accessor, path, index, prefix)` and use `IndexCacheStore`
to cache name-to-ID mappings.

### 4. Scope Detection and GlobScope Optimization

`GlobScope` carries the raw path and pattern **before** expansion. This
lets commands decide how to resolve paths efficiently -- skipping
expensive glob expansion when the resource has a native API for the
operation.

**`scope.py`** parses the unexpanded path to determine the level:

```python
@dataclass
class MyScope:
    level: str  # "root", "category", "item", "file"
    item_id: str | None = None
```

**How commands use scope for optimization:**

```python
# Example: grep at different scopes

@command("grep", resource="my_resource", spec=SPECS["grep"])
async def grep(accessor, paths, *texts, **_extra):
    pattern = texts[0]
    scope = detect_scope(paths[0], index)

    if scope.level in ("category", "item"):
        # CHEAP: use native search API (1 API call)
        results = await search_api(accessor.config, scope.item_id, pattern)
        return format_results(results), IOResult()

    # EXPENSIVE fallback: expand glob, download each file, grep locally
    paths = await resolve_glob(accessor, paths, index=index)
    for p in paths:
        data = await read(accessor, p.original, index, prefix=p.prefix)
        # ... grep the bytes ...
```

**When to use this pattern:**

| Scenario                                   | Approach                                                  |
| ------------------------------------------ | --------------------------------------------------------- |
| Resource has a search API (Discord, Slack) | Use scope to route to native API at directory level       |
| Resource has no search API                 | Always fall through to file-level reads; scope is a noop  |
| `head`/`tail` with direct message fetch    | Use scope to detect file level, fetch N messages directly |

**When the resource has no search API**, keep `scope.py` as a noop for
structural consistency. The file still parses path parts but does not
trigger any API calls:

```python
# Noop scope -- no search API, no resource-relative paths
def detect_scope(path: str | GlobScope) -> MyScope:
    key = path.strip("/")
    parts = key.split("/")
    # Just parse path structure, no index lookups
    ...
```

### 5. Glob Resolution

```python
# mirage/core/<name>/glob.py
async def resolve_glob(accessor, paths, index=None) -> list[GlobScope]:
    # Expand patterns via readdir + fnmatch
```

### 6. Ops Layer

Thin wrappers that bridge core functions to the command framework:

```python
@op("read", resource="<name>")
async def read(accessor, scope, **kwargs) -> bytes:
    return await core_read(accessor, scope.original,
                           kwargs.get("index"), prefix=scope.prefix)
```

### 7. Commands

Copy from an existing resource (Discord/Slack), then:

1. Replace accessor and core imports.
1. Set `resource="<name>"` in decorators.
1. Add or remove scope-based optimizations depending on API capabilities.
1. Add resource-specific commands (send message, etc.).

**Provision (dry-run estimates).** Factory-built commands
(`make_generic_commands`) get family-default estimators for free, so
`ws.execute(cmd, provision=True)` works without extra wiring. For
bespoke commands, pass `provision=` to the decorator, reusing the
shared helpers from `mirage.commands.builtin.generic_bind`:

```python
from mirage.commands.builtin.generic_bind.provision import (
    make_file_read_provision, metadata_provision)
from mirage.core.<name>.stat import stat as my_stat

@command("cat", resource="<name>", spec=SPECS["cat"],
         provision=make_file_read_provision(my_stat))
async def cat(...): ...
```

Omit `provision=` and the planner honestly reports
`precision=unknown`. If your resource renders virtual files (a
`chat.jsonl` built from messages), have `stat` report `size=None`
rather than 0 so estimates degrade to floors instead of lying.

### 8. Resource Class

```python
class MyResource(BaseResource):
    name: str = ResourceName.MY_RESOURCE
    caches_reads: bool = True

    def __init__(self, config: MyConfig) -> None:
        super().__init__()
        self.config = config
        self.accessor = MyAccessor(self.config)
        from mirage.commands.builtin.<name> import COMMANDS
        from mirage.ops.<name> import OPS

        for fn in COMMANDS:
            self.register(fn)
        for fn in OPS:
            self.register_op(fn)
```