Files
wehub-resource-sync bcbd1bdb22
Integ / changes (push) Has been skipped
Pre-commit / pre-commit (push) Failing after 1s
CLI exit codes / changes (push) Has been skipped
Test (Install) / changes (push) Has been skipped
Test (Python) / changes (push) Has been skipped
Test (TypeScript) / changes (push) Has been skipped
CLI exit codes / cli-gate (push) Has been cancelled
Test (Install) / test-install-gate (push) Has been cancelled
Integ / integ-gate (push) Has been cancelled
Test (Python) / test-python-gate (push) Has been cancelled
Test (TypeScript) / test-typescript-gate (push) Has been cancelled
Test (Install) / python-minimal (3.12) (push) Has been cancelled
Test (Install) / python-minimal (3.11) (push) Has been cancelled
Test (Install) / python-extra (agno, mirage.agents.agno) (push) Has been cancelled
Test (Install) / python-extra (chroma, mirage.resource.chroma) (push) Has been cancelled
Test (Install) / python-extra (pdf, mirage.core.filetype.pdf) (push) Has been cancelled
Integ / integ (push) Has been cancelled
Integ / integ-database (push) Has been cancelled
Integ / integ-database-ts (push) Has been cancelled
Integ / integ-data (push) Has been cancelled
Integ / integ-ssh (push) Has been cancelled
Integ / integ-ssh-ts (push) Has been cancelled
Test (Python) / audit (push) Has been cancelled
Test (TypeScript) / test (push) Has been cancelled
Test (TypeScript) / python-fs-shim (push) Has been cancelled
CLI exit codes / Python CLI (push) Has been cancelled
CLI exit codes / TypeScript CLI (push) Has been cancelled
CLI exit codes / Cross-language snapshot interop (push) Has been cancelled
Test (Python) / test (push) Has been cancelled
Test (Python) / import-isolation (deepagents, openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Python) / import-isolation (deepagents, pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Integ / integ-ts (push) Has been cancelled
Integ / integ-fuse (push) Has been cancelled
Test (Install) / python-extra (databricks, mirage.resource.databricks_volume) (push) Has been cancelled
Test (Install) / python-extra (deepagents, mirage.agents.langchain) (push) Has been cancelled
Test (Install) / python-extra (email, mirage.resource.email) (push) Has been cancelled
Test (Install) / python-extra (fuse, mirage.fuse.mount) (push) Has been cancelled
Test (Install) / python-extra (hdf5, mirage.core.filetype.hdf5) (push) Has been cancelled
Test (Install) / python-extra (hf, mirage.resource.hf_buckets) (push) Has been cancelled
Test (Install) / python-extra (lancedb, mirage.resource.lancedb) (push) Has been cancelled
Test (Install) / python-extra (langfuse, mirage.resource.langfuse) (push) Has been cancelled
Test (Install) / python-extra (mongodb, mirage.resource.mongodb) (push) Has been cancelled
Test (Install) / python-extra (nextcloud, mirage.resource.nextcloud) (push) Has been cancelled
Test (Install) / python-extra (openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Install) / python-extra (openhands, mirage.agents.openhands, 3.12) (push) Has been cancelled
Test (Install) / python-extra (parquet, mirage.core.filetype.parquet) (push) Has been cancelled
Test (Install) / python-extra (postgres, mirage.resource.postgres) (push) Has been cancelled
Test (Install) / python-extra (pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Test (Install) / python-extra (qdrant, mirage.resource.qdrant) (push) Has been cancelled
Test (Install) / python-extra (redis, mirage.resource.redis) (push) Has been cancelled
Test (Install) / python-extra (s3, mirage.resource.s3) (push) Has been cancelled
Test (Install) / python-extra (ssh, mirage.resource.ssh) (push) Has been cancelled
Test (Install) / ts-minimal (push) Has been cancelled
chore: import upstream snapshot with attribution
2026-07-13 12:30:44 +08:00

349 lines
11 KiB
Plaintext
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: Discord
description: Mount Discord guilds, channels, messages, attachments, and members as a Mirage virtual filesystem for Python agents.
icon: discord
---
The Discord resource exposes guild, channel, and member data as a virtual
filesystem mounted at some prefix such as `/discord/`.
For token setup, see [Discord Setup](/python/setup/discord).
## Config
```python
import os
from mirage import MountMode, Workspace
from mirage.resource.discord import DiscordConfig, DiscordResource
config = DiscordConfig(token=os.environ["DISCORD_BOT_TOKEN"])
resource = DiscordResource(config=config)
ws = Workspace({"/discord": resource}, mode=MountMode.READ)
```
## Filesystem Layout
```text
/discord/
<guild-name>__<guild-id>/
channels/
<channel-name>__<channel-id>/
<yyyy-mm-dd>/
chat.jsonl
files/
<stem>__<attachment-id>.<ext>
...
...
members/
<username>__<user-id>.json
...
```
Example:
```text
/discord/
My Server__111222333444555666/
channels/
general__777888999000111222/
2026-04-04/
chat.jsonl
files/
screenshot__1488111222333444555.png
2026-04-05/
chat.jsonl
random__777888999000111223/
2026-04-11/
chat.jsonl
members/
alice__444555666777888999.json
bob__444555666777888900.json
```
Display names keep their original spelling from Discord (spaces,
apostrophes, emoji are all preserved). Only `/` is replaced with ``
(U+2215) so it cannot collide with a directory boundary. The Discord
snowflake ID is appended after `__` (double underscore) on guild,
channel, member, and attachment names so resource specific commands can
extract it without an extra lookup, and so two same named entities never
collide. Quote names containing spaces in shell commands. `stat` also
exposes the ID in the `extra` dict (see [Finding IDs](#finding-ids)).
### Guilds
The root lists one directory per guild the bot has access to.
### Channels
`/discord/<guild>/channels/` lists text channels (types 0, 5, 15).
Each channel directory contains day-partitioned **directories** for the
30 days leading up to the channel's last message. Each day directory
holds:
- `chat.jsonl`, the day's messages (one JSON object per line).
- `files/`, attachments posted on that day. Each blob is named
`<stem>__<attachment-id>.<ext>`, where the stem keeps the original
filename's spelling (only `/` is replaced). The ID suffix keeps the
filename collision-free. `cat`'ing a blob downloads it from the
Discord CDN.
The date range is derived from `last_message_id` on the channel object,
so inactive channels show dates around their last activity, not today.
Soft errors (403 missing permissions, 404 unknown channel, 429 rate
limit) on a single day are swallowed so listings, `find`, and `grep`
keep working across the rest of the tree.
### Members
`/discord/<guild>/members/` lists one `.json` file per member.
Reading a member file returns the full member payload from the
Discord API.
## Smart Commands
### grep / rg at different scopes
When `grep` or `rg` target a channel or guild directory (not a specific file),
they use the Discord search API instead of downloading every `.jsonl` file:
```bash
# FILE level - downloads the .jsonl, greps locally
grep hello "/discord/My Server__111222333444555666/channels/general__777888999000111222/2026-04-04/chat.jsonl"
# CHANNEL level - uses Discord search API (GET /guilds/{id}/messages/search)
grep hello "/discord/My Server__111222333444555666/channels/general__777888999000111222/"
# GUILD level - searches across all channels
grep hello "/discord/My Server__111222333444555666/"
```
Scope detection is handled by `mirage/core/discord/scope.py`.
### head / tail
`head` and `tail` on file-level paths use the Discord messages API directly
(`GET /channels/{id}/messages`) instead of downloading the full day's history.
## Cache
The Discord resource uses `IndexCacheStore` (same as RAM/S3/disk/GitHub).
Index entries store guild IDs, channel IDs, and `last_message_id` for
date range computation. There is no separate content cache - file content
caching is handled by the workspace `IOResult` mechanism.
## Example
```python
import asyncio
import os
from dotenv import load_dotenv
from mirage import MountMode, Workspace
from mirage.resource.discord import DiscordConfig, DiscordResource
load_dotenv(".env.development")
config = DiscordConfig(token=os.environ["DISCORD_BOT_TOKEN"])
resource = DiscordResource(config=config)
async def main():
ws = Workspace({"/discord": resource}, mode=MountMode.READ)
# List guilds
r = await ws.execute("ls /discord/")
print(await r.stdout_str())
guild = r.stdout_str().strip().split("\n")[0].strip()
# List channels
r = await ws.execute(f'ls "/discord/{guild}/channels/"')
print(await r.stdout_str())
ch = r.stdout_str().strip().splitlines()[0].strip()
base = f"/discord/{guild}/channels/{ch}"
# Read messages from a specific date
r = await ws.execute(f'cat "{base}/2026-04-04/chat.jsonl" | head -n 3')
print(await r.stdout_str())
# Extract usernames with jq
r = await ws.execute(
f'jq -r ".[] | .author.username" "{base}/2026-04-04/chat.jsonl"')
print(await r.stdout_str())
# Count messages per user
r = await ws.execute(
f'cat "{base}/2026-04-04/chat.jsonl"'
' | jq -r ".[] | .author.username" | sort | uniq -c')
print(await r.stdout_str())
# List attachments for a day and download one
r = await ws.execute(f'ls "{base}/2026-04-04/files/"')
print(await r.stdout_str())
r = await ws.execute(
f'cat "{base}/2026-04-04/files/screenshot__1488111222333444555.png"')
blob = await r.materialize_stdout()
print(f"downloaded {len(blob)} bytes")
# Search across channel (uses Discord search API)
r = await ws.execute(f'grep hello "{base}/"')
print(await r.stdout_str())
# Search across guild
r = await ws.execute(f'grep hello "/discord/{guild}/"')
print(await r.stdout_str())
# Navigate with cd/pwd
await ws.execute(f'cd "{base}"')
r = await ws.execute("pwd")
print(await r.stdout_str())
# Relative paths after cd
r = await ws.execute("ls | tail -n 5")
print(await r.stdout_str())
if __name__ == "__main__":
asyncio.run(main())
```
See `examples/python/discord/discord.py` for the full working example.
## Finding IDs
Resource-specific commands require Discord snowflake IDs
(`channel_id`, `guild_id`, `message_id`). These can be extracted
from the filesystem:
```bash
# Guild ID - use stat
stat "/discord/My Server__111222333444555666"
# → extra={"guild_id": "1256522563555819574"}
# Channel ID - use stat
stat "/discord/My Server__111222333444555666/channels/general__777888999000111222"
# → extra={"channel_id": "1256522563555819574"}
# Message ID - inside JSONL messages
jq -r '.[] | "\(.id) [\(.author.username)] \(.content)"' \
"/discord/My Server__111222333444555666/channels/general__777888999000111222/2026-04-04/chat.jsonl"
# → 1489887688978075769 [alice] hello world
# Find a message then reply
jq -r '.[] | select(.content | test("hello")) | .id' \
"/discord/My Server__111222333444555666/channels/general__777888999000111222/2026-04-04/chat.jsonl"
# → 1489887688978075769
discord-send-message --channel_id 1256522563555819574 \
--text "Reply" --message_id 1489887688978075769
```
## Working with Large Channels
Tips for efficient access on busy channels:
```bash
# Check message count per day
wc -l "/discord/My Server__111222333444555666/channels/general__777888999000111222/2026-04-04/chat.jsonl"
# Read only recent messages
tail -n 10 "/discord/My Server__111222333444555666/channels/general__777888999000111222/2026-04-04/chat.jsonl"
# Search uses Discord API at channel/guild level (no file download)
grep "keyword" "/discord/My Server__111222333444555666/channels/general__777888999000111222/"
# Extract specific fields
jq -r '.[] | "\(.author.username): \(.content)"' \
"/discord/My Server__111222333444555666/channels/general__777888999000111222/2026-04-04/chat.jsonl" | head -n 20
# Count messages per user
cat "/discord/My Server__111222333444555666/channels/general__777888999000111222/2026-04-04/chat.jsonl" \
| jq -r '.[] | .author.username' | sort | uniq -c
```
Note: `grep`/`rg` at channel or guild level uses the Discord search
API instead of downloading every `.jsonl` file, making it efficient
even for large channels.
## Shell Commands
Standard commands available on the mounted Discord tree:
| Command | Notes |
| --------------- | -------------------------------------------------------------- |
| `ls` | List guilds, channels, members, dates, attachments |
| `cat` | Read `chat.jsonl`, member `.json`, or download an attachment |
| `head` / `tail` | Smart: uses messages API for file scope |
| `grep` / `rg` | Smart: uses search API for channel/guild scope (with fallback) |
| `jq` | Query JSON; use `.[]` prefix for JSONL files |
| `wc` | Line/word/byte counts |
| `stat` | File metadata (name, size, type, ID via `extra`) |
| `find` | Recursive search with `-name`, `-maxdepth` |
| `tree` | Directory tree view |
Resource-specific commands:
### `discord-send-message`
Post a message to a channel, optionally as a reply.
```bash
discord-send-message --channel_id 1256522563555819574 --text "Hello from MIRAGE"
discord-send-message --channel_id 1256522563555819574 --text "Reply" --message_id 1489887688978075769
```
| Option | Required | Description |
| -------------- | -------- | ---------------------------- |
| `--channel_id` | yes | Discord channel snowflake ID |
| `--text` | yes | Message text to send |
| `--message_id` | no | Message ID to reply to |
The channel ID can be found in directory names under
`/discord/<guild>/channels/` or via `stat`. Returns the
posted message JSON.
### `discord-add-reaction`
Add an emoji reaction to a message.
```bash
discord-add-reaction --channel_id 1256522563555819574 --message_id 1489887688978075769 --reaction 👍
```
| Option | Required | Description |
| -------------- | -------- | ---------------------------- |
| `--channel_id` | yes | Discord channel snowflake ID |
| `--message_id` | yes | Message snowflake ID |
| `--reaction` | yes | Emoji (unicode or name) |
### `discord-list-members`
Search guild members by name.
```bash
discord-list-members --guild_id 1256522563555819574 --query "alice"
```
| Option | Required | Description |
| ------------ | -------- | ----------------------- |
| `--guild_id` | yes | Discord guild snowflake |
| `--query` | yes | Username search query |
Returns matching members as JSON array.
### `discord-get-server-info`
Get full guild metadata from the Discord API.
```bash
discord-get-server-info --guild_id 1256522563555819574
```
| Option | Required | Description |
| ------------ | -------- | ----------------------- |
| `--guild_id` | yes | Discord guild snowflake |
Returns the guild object JSON (name, icon, member count, etc.).