chore: import upstream snapshot with attribution
Integ / changes (push) Has been skipped
Pre-commit / pre-commit (push) Failing after 1s
CLI exit codes / changes (push) Has been skipped
Test (Install) / changes (push) Has been skipped
Test (Python) / changes (push) Has been skipped
Test (TypeScript) / changes (push) Has been skipped
CLI exit codes / cli-gate (push) Has been cancelled
Test (Install) / test-install-gate (push) Has been cancelled
Integ / integ-gate (push) Has been cancelled
Test (Python) / test-python-gate (push) Has been cancelled
Test (TypeScript) / test-typescript-gate (push) Has been cancelled
Test (Install) / python-minimal (3.12) (push) Has been cancelled
Test (Install) / python-minimal (3.11) (push) Has been cancelled
Test (Install) / python-extra (agno, mirage.agents.agno) (push) Has been cancelled
Test (Install) / python-extra (chroma, mirage.resource.chroma) (push) Has been cancelled
Test (Install) / python-extra (pdf, mirage.core.filetype.pdf) (push) Has been cancelled
Integ / integ (push) Has been cancelled
Integ / integ-database (push) Has been cancelled
Integ / integ-database-ts (push) Has been cancelled
Integ / integ-data (push) Has been cancelled
Integ / integ-ssh (push) Has been cancelled
Integ / integ-ssh-ts (push) Has been cancelled
Test (Python) / audit (push) Has been cancelled
Test (TypeScript) / test (push) Has been cancelled
Test (TypeScript) / python-fs-shim (push) Has been cancelled
CLI exit codes / Python CLI (push) Has been cancelled
CLI exit codes / TypeScript CLI (push) Has been cancelled
CLI exit codes / Cross-language snapshot interop (push) Has been cancelled
Test (Python) / test (push) Has been cancelled
Test (Python) / import-isolation (deepagents, openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Python) / import-isolation (deepagents, pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Integ / integ-ts (push) Has been cancelled
Integ / integ-fuse (push) Has been cancelled
Test (Install) / python-extra (databricks, mirage.resource.databricks_volume) (push) Has been cancelled
Test (Install) / python-extra (deepagents, mirage.agents.langchain) (push) Has been cancelled
Test (Install) / python-extra (email, mirage.resource.email) (push) Has been cancelled
Test (Install) / python-extra (fuse, mirage.fuse.mount) (push) Has been cancelled
Test (Install) / python-extra (hdf5, mirage.core.filetype.hdf5) (push) Has been cancelled
Test (Install) / python-extra (hf, mirage.resource.hf_buckets) (push) Has been cancelled
Test (Install) / python-extra (lancedb, mirage.resource.lancedb) (push) Has been cancelled
Test (Install) / python-extra (langfuse, mirage.resource.langfuse) (push) Has been cancelled
Test (Install) / python-extra (mongodb, mirage.resource.mongodb) (push) Has been cancelled
Test (Install) / python-extra (nextcloud, mirage.resource.nextcloud) (push) Has been cancelled
Test (Install) / python-extra (openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Install) / python-extra (openhands, mirage.agents.openhands, 3.12) (push) Has been cancelled
Test (Install) / python-extra (parquet, mirage.core.filetype.parquet) (push) Has been cancelled
Test (Install) / python-extra (postgres, mirage.resource.postgres) (push) Has been cancelled
Test (Install) / python-extra (pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Test (Install) / python-extra (qdrant, mirage.resource.qdrant) (push) Has been cancelled
Test (Install) / python-extra (redis, mirage.resource.redis) (push) Has been cancelled
Test (Install) / python-extra (s3, mirage.resource.s3) (push) Has been cancelled
Test (Install) / python-extra (ssh, mirage.resource.ssh) (push) Has been cancelled
Test (Install) / ts-minimal (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 12:30:44 +08:00
commit bcbd1bdb22
5748 changed files with 562488 additions and 0 deletions
@@ -0,0 +1,25 @@
import pytest
from mirage.commands.builtin.chroma.find import _default_name, _expr_texts
@pytest.mark.parametrize("texts", [
("!", "-name", "x"),
("(", "-name", "a", "-o", "-name", "b", ")"),
("-name", "x"),
("-not", "-name", "x"),
])
def test_expr_texts_preserves_expression(texts):
assert _expr_texts(texts) == texts
def test_expr_texts_strips_bare_leading_name():
assert _expr_texts(("foo", )) == ()
assert _expr_texts(()) == ()
def test_default_name_only_for_bare_word():
assert _default_name(None, ("foo", )) == "foo"
assert _default_name(None, ("!", "-name", "x")) is None
assert _default_name(None, ("(", "-name", "a")) is None
assert _default_name("given", ("foo", )) == "given"
+24
View File
@@ -0,0 +1,24 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage.resource.ram import RAMResource
@pytest.fixture
def backend():
b = RAMResource()
b.accessor.store.dirs.add("/tmp")
return b
@@ -0,0 +1,91 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import MountMode, Workspace
from mirage.cache.index import RAMIndexCacheStore
from mirage.types import PathSpec
from mirage.utils.stream import collect_bytes
from tests.resource.databricks_volume.test_databricks_volume import (
FakeFiles, make_resource, seed_directory, seed_file)
def seed_text_command_fixture(files: FakeFiles) -> str:
root = "/Volumes/main/default/agent_files/root"
seed_directory(files, root)
seed_file(files, f"{root}/words.txt", b"beta\nalpha\nalpha\n")
seed_file(files, f"{root}/more.txt", b"delta\n")
seed_file(files, f"{root}/table.csv", b"name,score\nann,2\nbob,3\n")
seed_file(files, f"{root}/table_extra.csv", b"name,score\ncam,5\n")
seed_file(files, f"{root}/data.json", b'{"name": "mirage"}\n')
seed_file(files, f"{root}/data2.json", b'{"name": "agent"}\n')
seed_file(files, f"{root}/events.jsonl", b'{"name": "first"}\n')
seed_file(files, f"{root}/events_extra.jsonl", b'{"name": "second"}\n')
seed_file(files, f"{root}/old.txt", b"same\nold\n")
seed_file(files, f"{root}/new.txt", b"same\nnew\n")
return root
async def materialize(source) -> bytes:
if source is None:
return b""
return await collect_bytes(source)
class IndexTrackingReader:
def __init__(self) -> None:
self.seen_indexes: list[RAMIndexCacheStore | None] = []
async def read_bytes(self,
accessor,
path,
index=None,
*args,
**kwargs) -> bytes:
self.seen_indexes.append(index)
original = path.virtual if isinstance(path, PathSpec) else path
if str(original).endswith(".json"):
return b'{"name": "mirage"}\n'
if str(original).endswith(".csv"):
return b"name,score\nann,2\n"
return b"beta\nalpha\nalpha\n"
async def read_stream(self, accessor, path, index=None, *args, **kwargs):
self.seen_indexes.append(index)
yield await self.read_bytes(accessor, path, index, *args, **kwargs)
@pytest.fixture
def databricks_text_files() -> FakeFiles:
files = FakeFiles()
seed_text_command_fixture(files)
return files
@pytest.fixture
def databricks_text_workspace(databricks_text_files: FakeFiles) -> Workspace:
return Workspace({"/dbx/": make_resource(databricks_text_files)},
mode=MountMode.READ)
@pytest.fixture
def expected_index() -> RAMIndexCacheStore:
return RAMIndexCacheStore(ttl=600)
@pytest.fixture
def materialize_output():
return materialize
@@ -0,0 +1,25 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
@pytest.mark.asyncio
async def test_workspace_execute_databricks_volume_awk(
databricks_text_workspace):
io = await databricks_text_workspace.execute(
"awk '{print $1}' /dbx/table.csv")
assert io.exit_code == 0
assert io.stdout == b"name,score\nann,2\nbob,3\n"
@@ -0,0 +1,186 @@
import time
from functools import partial
import pytest
from mirage.accessor.databricks_volume import DatabricksVolumeAccessor
from mirage.cache.index import RAMIndexCacheStore
from mirage.commands.builtin.generic.find import parse_find_args, walk_find
from mirage.commands.builtin.generic.ls import ls as generic_ls
from mirage.commands.builtin.generic.tree import tree as generic_tree
from mirage.core.databricks_volume.path import backend_path
from mirage.core.databricks_volume.readdir import readdir
from mirage.core.databricks_volume.stat import stat
from mirage.resource.databricks_volume import DatabricksVolumeConfig
from mirage.types import LsSortBy, PathSpec
from mirage.utils.key_prefix import mount_key
from tests.core.databricks_volume.conftest import (FakeClient, FakeFiles,
directory_entry, file_entry)
MODIFIED_MS = 1_700_000_000_000
def is_dir_name(_child: str) -> bool | None:
# Databricks readdir returns slash-less paths, so classification always
# falls back to stat.
return None
FROZEN_NOW_S = 1_700_000_000
DAY_S = 86_400
AGES_DAYS = (1, 2, 3, 10, 20)
def _rig(
) -> tuple[DatabricksVolumeAccessor, FakeFiles, RAMIndexCacheStore, str]:
config = DatabricksVolumeConfig(
catalog="main",
schema="default",
volume="agent_files",
root_path="/root",
token="secret",
)
files = FakeFiles()
accessor = DatabricksVolumeAccessor(config, FakeClient(files))
index = RAMIndexCacheStore(ttl=600)
return accessor, files, index, backend_path(config, "/")
def _seed_flat(files: FakeFiles, root: str, count: int = 5) -> None:
files.directories[f"{root}/sub"] = [
file_entry(f"{root}/sub/f{i}.txt",
size=i + 1,
modified=MODIFIED_MS + i * 1000) for i in range(count)
]
def _ls_readdir(accessor: DatabricksVolumeAccessor):
return partial(readdir, accessor)
def _ls_stat(accessor: DatabricksVolumeAccessor):
return partial(stat, accessor)
@pytest.mark.asyncio
@pytest.mark.parametrize("sort_by,long", [
(LsSortBy.NAME, False),
(LsSortBy.NAME, True),
(LsSortBy.TIME, False),
(LsSortBy.SIZE, False),
])
async def test_ls_lists_once_without_per_entry_metadata(sort_by, long):
accessor, files, index, root = _rig()
_seed_flat(files, root, count=5)
path = PathSpec.from_str_path("/volume/sub",
mount_key("/volume/sub", "/volume"))
await generic_ls([path],
readdir=_ls_readdir(accessor),
stat=_ls_stat(accessor),
long=long,
sort_by=sort_by,
index=index)
assert files.list_directory_calls == [f"{root}/sub"]
assert files.get_metadata_calls == []
assert files.get_directory_metadata_calls == []
@pytest.mark.asyncio
async def test_ls_recursive_one_list_per_directory():
accessor, files, index, root = _rig()
files.directories[f"{root}/sub"] = [
file_entry(f"{root}/sub/a.txt", size=1, modified=MODIFIED_MS),
directory_entry(f"{root}/sub/inner"),
]
files.directories[f"{root}/sub/inner"] = [
file_entry(f"{root}/sub/inner/b.txt", size=2, modified=MODIFIED_MS),
]
path = PathSpec.from_str_path("/volume/sub",
mount_key("/volume/sub", "/volume"))
await generic_ls([path],
readdir=_ls_readdir(accessor),
stat=_ls_stat(accessor),
recursive=True,
index=index)
assert sorted(
files.list_directory_calls) == [f"{root}/sub", f"{root}/sub/inner"]
assert files.get_metadata_calls == []
assert files.get_directory_metadata_calls == []
@pytest.mark.asyncio
async def test_tree_one_list_per_directory_without_metadata():
accessor, files, index, root = _rig()
files.directories[f"{root}/sub"] = [
file_entry(f"{root}/sub/a.txt", size=1, modified=MODIFIED_MS),
directory_entry(f"{root}/sub/inner"),
]
files.directories[f"{root}/sub/inner"] = [
file_entry(f"{root}/sub/inner/b.txt", size=2, modified=MODIFIED_MS),
]
path = PathSpec.from_str_path("/volume/sub",
mount_key("/volume/sub", "/volume"))
await generic_tree(path,
readdir=_ls_readdir(accessor),
stat=_ls_stat(accessor),
index=index)
assert sorted(
files.list_directory_calls) == [f"{root}/sub", f"{root}/sub/inner"]
assert files.get_metadata_calls == []
assert files.get_directory_metadata_calls == []
async def _run_find(accessor, index, *, type=None, size=None, mtime=None):
args = parse_find_args((), type=type, size=size, mtime=mtime)
path = PathSpec.from_str_path("/volume/sub",
mount_key("/volume/sub", "/volume"))
return await walk_find(path,
readdir=_ls_readdir(accessor),
stat=_ls_stat(accessor),
is_dir_name=is_dir_name,
index=index,
args=args)
@pytest.mark.asyncio
async def test_find_type_does_not_stat_children():
accessor, files, index, root = _rig()
_seed_flat(files, root, count=5)
files.directory_metadata.add(f"{root}/sub")
results = await _run_find(accessor, index, type="f")
assert len(results) == 5
assert files.list_directory_calls == [f"{root}/sub"]
child_metadata = [c for c in files.get_metadata_calls if "/sub/" in c]
assert child_metadata == []
@pytest.mark.asyncio
async def test_find_size_reads_size_from_index():
accessor, files, index, root = _rig()
_seed_flat(files, root, count=5)
files.directory_metadata.add(f"{root}/sub")
results = await _run_find(accessor, index, type="f", size="+3c")
assert sorted(r.rsplit("/", 1)[-1]
for r in results) == ["f3.txt", "f4.txt"]
assert files.list_directory_calls == [f"{root}/sub"]
child_metadata = [c for c in files.get_metadata_calls if "/sub/" in c]
assert child_metadata == []
@pytest.mark.asyncio
async def test_find_mtime_reads_modified_from_index(monkeypatch):
monkeypatch.setattr(time, "time", lambda: float(FROZEN_NOW_S))
accessor, files, index, root = _rig()
files.directories[f"{root}/sub"] = [
file_entry(f"{root}/sub/f{i}.txt",
size=i + 1,
modified=(FROZEN_NOW_S - age * DAY_S) * 1000)
for i, age in enumerate(AGES_DAYS)
]
files.directory_metadata.add(f"{root}/sub")
results = await _run_find(accessor, index, mtime="-5")
assert sorted(r.rsplit("/", 1)[-1]
for r in results) == ["f0.txt", "f1.txt", "f2.txt"]
assert files.list_directory_calls == [f"{root}/sub"]
child_metadata = [c for c in files.get_metadata_calls if "/sub/" in c]
assert child_metadata == []
@@ -0,0 +1,40 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
@pytest.mark.asyncio
async def test_cat_single_file(databricks_text_workspace):
io = await databricks_text_workspace.execute("cat /dbx/words.txt")
assert io.exit_code == 0
assert io.stdout == b"beta\nalpha\nalpha\n"
@pytest.mark.asyncio
async def test_cat_multiple_files_concatenated(databricks_text_workspace):
io = await databricks_text_workspace.execute(
"cat /dbx/words.txt /dbx/more.txt")
assert io.exit_code == 0
assert io.stdout == b"beta\nalpha\nalpha\ndelta\n"
@pytest.mark.asyncio
async def test_cat_n_numbers_lines(databricks_text_workspace):
io = await databricks_text_workspace.execute("cat -n /dbx/words.txt")
assert io.exit_code == 0
assert io.stdout == b" 1\tbeta\n 2\talpha\n 3\talpha\n"
@@ -0,0 +1,123 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import MountMode, Workspace
from tests.resource.databricks_volume.test_databricks_volume import (
FakeFiles, make_resource, seed_directory, seed_file)
ROOT = "/Volumes/main/default/agent_files/root"
@pytest.fixture
def dbx_files() -> FakeFiles:
files = FakeFiles()
seed_directory(files, ROOT)
seed_file(files, f"{ROOT}/src.txt", b"hello")
return files
@pytest.fixture
def write_ws(dbx_files: FakeFiles) -> Workspace:
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.WRITE)
@pytest.fixture
def read_ws(dbx_files: FakeFiles) -> Workspace:
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
@pytest.mark.asyncio
async def test_cp_file_preserves_bytes(write_ws, dbx_files):
io = await write_ws.execute("cp /dbx/src.txt /dbx/dst.txt")
assert io.exit_code == 0
assert dbx_files.downloads[f"{ROOT}/dst.txt"] == b"hello"
@pytest.mark.asyncio
async def test_cp_directory_without_recursive_fails(write_ws, dbx_files):
seed_directory(dbx_files, f"{ROOT}/d")
io = await write_ws.execute("cp /dbx/d /dbx/d2")
assert io.exit_code != 0
@pytest.mark.asyncio
async def test_cp_writes_are_mount_relative(write_ws):
io = await write_ws.execute("cp /dbx/src.txt /dbx/dst.txt")
assert io.exit_code == 0
for key in io.writes:
assert key.startswith("/dbx/")
assert not key.startswith("/dbx/dbx/")
@pytest.mark.asyncio
async def test_cp_read_only_mount_rejected(read_ws, dbx_files):
io = await read_ws.execute("cp /dbx/src.txt /dbx/dst.txt")
assert io.exit_code != 0
assert b"read-only" in io.stderr
assert f"{ROOT}/dst.txt" not in dbx_files.downloads
@pytest.mark.asyncio
async def test_cp_onto_same_path_errors_and_preserves_file(
write_ws, dbx_files):
io = await write_ws.execute("cp /dbx/src.txt /dbx/src.txt")
assert io.exit_code != 0
assert b"are the same file" in io.stderr
assert dbx_files.downloads[f"{ROOT}/src.txt"] == b"hello"
@pytest.mark.asyncio
async def test_cp_multiple_sources_require_directory(write_ws, dbx_files):
seed_file(dbx_files, f"{ROOT}/a.txt", b"AAA")
seed_file(dbx_files, f"{ROOT}/b.txt", b"BBB")
seed_file(dbx_files, f"{ROOT}/target.txt", b"target")
io = await write_ws.execute("cp /dbx/a.txt /dbx/b.txt /dbx/target.txt")
assert io.exit_code != 0
assert b"not a directory" in io.stderr
assert dbx_files.downloads[f"{ROOT}/a.txt"] == b"AAA"
assert dbx_files.downloads[f"{ROOT}/b.txt"] == b"BBB"
assert dbx_files.downloads[f"{ROOT}/target.txt"] == b"target"
@pytest.mark.asyncio
async def test_cp_missing_source_reports_cannot_stat(write_ws, dbx_files):
io = await write_ws.execute("cp /dbx/missing /dbx/missing")
assert io.exit_code != 0
assert b"cannot stat" in io.stderr
assert b"are the same file" not in io.stderr
@pytest.mark.asyncio
async def test_cp_recursive_into_itself_errors_and_preserves_tree(
write_ws, dbx_files):
seed_directory(dbx_files, f"{ROOT}/d")
seed_file(dbx_files, f"{ROOT}/d/a.txt", b"aaa")
io = await write_ws.execute("cp -r /dbx/d /dbx/d")
assert io.exit_code != 0
assert b"into itself" in io.stderr
assert dbx_files.downloads[f"{ROOT}/d/a.txt"] == b"aaa"
assert f"{ROOT}/d/d" not in dbx_files.directory_metadata
@@ -0,0 +1,87 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
from mirage import MountMode, Workspace
from mirage.resource.ram import RAMResource
from tests.resource.databricks_volume.test_databricks_volume import (
FakeFiles, make_resource, seed_file)
ROOT = "/Volumes/main/default/agent_files/root"
def _run(ws, cmd):
async def _inner():
io = await ws.execute(cmd)
return await io.stdout_str(), await io.stderr_str(), io.exit_code
return asyncio.run(_inner())
def _ws_with_dbx_tree():
files = FakeFiles()
files.create_directory(ROOT)
files.create_directory(f"{ROOT}/tree")
files.create_directory(f"{ROOT}/tree/sub")
seed_file(files, f"{ROOT}/tree/a.txt", b"aaa\n")
seed_file(files, f"{ROOT}/tree/sub/b.txt", b"bbb\n")
dbx = make_resource(files)
ram = RAMResource()
ws = Workspace(
{
"/dbx": (dbx, MountMode.WRITE),
"/m": (ram, MountMode.WRITE),
}, )
return ws, files, ram
def test_cp_recursive_databricks_to_ram():
ws, _files, ram = _ws_with_dbx_tree()
_out, _err, code = _run(ws, "cp -r /dbx/tree /m/copied")
assert code == 0
assert ram._store.files["/copied/a.txt"] == b"aaa\n"
assert ram._store.files["/copied/sub/b.txt"] == b"bbb\n"
def test_cp_recursive_ram_to_databricks_creates_dirs():
ws, files, ram = _ws_with_dbx_tree()
ram._store.dirs.update({"/src", "/src/sub"})
ram._store.files["/src/x.txt"] = b"xxx\n"
ram._store.files["/src/sub/y.txt"] = b"yyy\n"
_out, _err, code = _run(ws, "cp -r /m/src /dbx/put")
assert code == 0
assert files.downloads[f"{ROOT}/put/x.txt"] == b"xxx\n"
assert files.downloads[f"{ROOT}/put/sub/y.txt"] == b"yyy\n"
assert f"{ROOT}/put/sub" in files.directory_metadata
def test_mv_recursive_databricks_to_ram_removes_source():
ws, files, ram = _ws_with_dbx_tree()
_out, _err, code = _run(ws, "mv /dbx/tree /m/moved")
assert code == 0
assert ram._store.files["/moved/a.txt"] == b"aaa\n"
assert ram._store.files["/moved/sub/b.txt"] == b"bbb\n"
assert f"{ROOT}/tree/a.txt" not in files.downloads
assert f"{ROOT}/tree" not in files.directory_metadata
def test_single_file_missing_parent_is_posix_error():
# Option B / POSIX: a single-file copy never creates the destination's
# parent directory; it errors like coreutils instead of mkdir -p.
ws, _files, ram = _ws_with_dbx_tree()
ram._store.files["/lone.txt"] = b"z\n"
_out, _err, code = _run(ws, "cp /m/lone.txt /dbx/nope/deep/lone.txt")
assert code == 1
@@ -0,0 +1,25 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
@pytest.mark.asyncio
async def test_workspace_execute_databricks_volume_cut(
databricks_text_workspace):
io = await databricks_text_workspace.execute("cut -d , -f 2 /dbx/table.csv"
)
assert io.exit_code == 0
assert io.stdout == b"score\n2\n3\n"
@@ -0,0 +1,37 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
@pytest.mark.asyncio
async def test_workspace_execute_databricks_volume_diff(
databricks_text_workspace):
io = await databricks_text_workspace.execute(
"diff -u /dbx/old.txt /dbx/new.txt")
assert io.exit_code == 1
assert b"-old" in io.stdout
assert b"+new" in io.stdout
@pytest.mark.asyncio
async def test_workspace_execute_databricks_volume_diff_resolves_glob(
databricks_text_workspace):
io = await databricks_text_workspace.execute(
"diff /dbx/old*.txt /dbx/new*.txt")
assert io.exit_code == 1
assert b"old" in io.stdout
assert b"new" in io.stdout
@@ -0,0 +1,68 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import MountMode, Workspace
from tests.resource.databricks_volume.test_databricks_volume import (
FakeFiles, make_resource, seed_directory, seed_file)
ROOT = "/Volumes/main/default/agent_files/root"
@pytest.fixture
def dbx_files() -> FakeFiles:
files = FakeFiles()
seed_directory(files, ROOT)
seed_file(files, f"{ROOT}/words.txt", b"beta\nalpha\nalpha\n")
seed_file(files, f"{ROOT}/notes.md", b"note\n")
files.create_directory(f"{ROOT}/sub")
seed_file(files, f"{ROOT}/sub/inner.txt", b"gamma\nalpha\n")
return files
@pytest.fixture
def ws(dbx_files: FakeFiles) -> Workspace:
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
@pytest.mark.asyncio
async def test_find_name_glob(ws):
io = await ws.execute("find /dbx/ -name '*.txt'")
assert io.exit_code == 0
out = io.stdout.decode()
assert "words.txt" in out
assert "inner.txt" in out
assert "notes.md" not in out
@pytest.mark.asyncio
async def test_find_type_d(ws):
io = await ws.execute("find /dbx/ -type d")
assert io.exit_code == 0
out = io.stdout.decode()
assert "sub" in out
assert "words.txt" not in out
@pytest.mark.asyncio
async def test_find_maxdepth(ws):
io = await ws.execute("find /dbx/ -maxdepth 1")
assert io.exit_code == 0
out = io.stdout.decode()
assert "words.txt" in out
assert "inner.txt" not in out
@@ -0,0 +1,77 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import MountMode, Workspace
from tests.resource.databricks_volume.test_databricks_volume import (
FakeFiles, make_resource, seed_directory, seed_file)
ROOT = "/Volumes/main/default/agent_files/root"
@pytest.fixture
def dbx_files() -> FakeFiles:
files = FakeFiles()
seed_directory(files, ROOT)
seed_file(files, f"{ROOT}/words.txt", b"beta\nalpha\nalpha\n")
files.create_directory(f"{ROOT}/sub")
seed_file(files, f"{ROOT}/sub/inner.txt", b"gamma\nalpha\n")
return files
@pytest.fixture
def ws(dbx_files: FakeFiles) -> Workspace:
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
@pytest.mark.asyncio
async def test_grep_single_file(ws):
io = await ws.execute("grep alpha /dbx/words.txt")
assert io.exit_code == 0
assert io.stdout == b"alpha\nalpha\n"
@pytest.mark.asyncio
async def test_grep_line_numbers(ws):
io = await ws.execute("grep -n alpha /dbx/words.txt")
assert io.exit_code == 0
assert io.stdout == b"2:alpha\n3:alpha\n"
@pytest.mark.asyncio
async def test_grep_count_only(ws):
io = await ws.execute("grep -c alpha /dbx/words.txt")
assert io.exit_code == 0
assert io.stdout == b"2\n"
@pytest.mark.asyncio
async def test_grep_recursive(ws):
io = await ws.execute("grep -r alpha /dbx/")
assert io.exit_code == 0
out = io.stdout.decode()
assert "words.txt" in out
assert "inner.txt" in out
@pytest.mark.asyncio
async def test_grep_no_match_exits_nonzero(ws):
io = await ws.execute("grep zeta /dbx/words.txt")
assert io.exit_code != 0
@@ -0,0 +1,34 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
@pytest.mark.asyncio
async def test_head_c_positive_prefix(databricks_text_workspace):
io = await databricks_text_workspace.execute("head -c 4 /dbx/words.txt")
assert io.exit_code == 0
assert io.stdout == b"beta"
@pytest.mark.asyncio
async def test_head_c_negative_all_but_last(databricks_text_workspace):
# GNU `head -c -N` emits all but the last N bytes. words.txt is 17 bytes,
# so -c -6 yields the first 11. A negative -c must not take the prefix
# fast path (range read 0..-6 is meaningless).
io = await databricks_text_workspace.execute("head -c -6 /dbx/words.txt")
assert io.exit_code == 0
assert io.stdout == b"beta\nalpha\n"
@@ -0,0 +1,33 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
@pytest.mark.asyncio
async def test_workspace_execute_databricks_volume_jq(
databricks_text_workspace):
io = await databricks_text_workspace.execute("jq -r .name /dbx/data.json")
assert io.exit_code == 0
assert io.stdout == b"mirage\n"
@pytest.mark.asyncio
async def test_workspace_execute_databricks_volume_jq_resolves_glob(
databricks_text_workspace):
io = await databricks_text_workspace.execute("jq -r .name /dbx/*.json")
assert io.exit_code == 0
assert io.stdout == b"mirage\nagent\n"
@@ -0,0 +1,81 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import MountMode, Workspace
from tests.resource.databricks_volume.test_databricks_volume import (
FakeFiles, make_resource, seed_directory, seed_file)
ROOT = "/Volumes/main/default/agent_files/root"
@pytest.fixture
def dbx_files() -> FakeFiles:
files = FakeFiles()
seed_directory(files, ROOT)
seed_file(files, f"{ROOT}/words.txt", b"beta\nalpha\nalpha\n")
seed_file(files, f"{ROOT}/.hidden", b"h\n")
files.create_directory(f"{ROOT}/sub")
seed_file(files, f"{ROOT}/sub/inner.txt", b"gamma\nalpha\n")
return files
@pytest.fixture
def ws(dbx_files: FakeFiles) -> Workspace:
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
@pytest.mark.asyncio
async def test_ls_lists_entries(ws):
io = await ws.execute("ls /dbx/")
assert io.exit_code == 0
out = io.stdout.decode()
assert "words.txt" in out
assert "sub" in out
assert ".hidden" not in out
@pytest.mark.asyncio
async def test_ls_a_includes_hidden(ws):
io = await ws.execute("ls -a /dbx/")
assert io.exit_code == 0
assert ".hidden" in io.stdout.decode()
@pytest.mark.asyncio
async def test_ls_long_includes_size(ws):
io = await ws.execute("ls -l /dbx/")
assert io.exit_code == 0
out = io.stdout.decode()
assert "words.txt" in out
assert "17" in out
@pytest.mark.asyncio
async def test_ls_recursive_descends_subdirs(ws):
io = await ws.execute("ls -R /dbx/")
assert io.exit_code == 0
assert "inner.txt" in io.stdout.decode()
@pytest.mark.asyncio
async def test_ls_missing_path_warns(ws):
io = await ws.execute("ls /dbx/missing")
assert io.exit_code != 0
@@ -0,0 +1,93 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import MountMode, Workspace
from tests.resource.databricks_volume.test_databricks_volume import (
FakeFiles, make_resource, seed_directory)
ROOT = "/Volumes/main/default/agent_files/root"
@pytest.fixture
def dbx_files() -> FakeFiles:
files = FakeFiles()
seed_directory(files, ROOT)
return files
@pytest.fixture
def write_ws(dbx_files: FakeFiles) -> Workspace:
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.WRITE)
@pytest.fixture
def read_ws(dbx_files: FakeFiles) -> Workspace:
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
@pytest.mark.asyncio
async def test_mkdir_creates_directory(write_ws, dbx_files):
io = await write_ws.execute("mkdir /dbx/newdir")
assert io.exit_code == 0
assert f"{ROOT}/newdir" in dbx_files.directory_metadata
@pytest.mark.asyncio
async def test_mkdir_parent_missing_fails(write_ws):
io = await write_ws.execute("mkdir /dbx/a/b")
assert io.exit_code != 0
@pytest.mark.asyncio
async def test_mkdir_parents_creates_chain(write_ws, dbx_files):
io = await write_ws.execute("mkdir -p /dbx/a/b/c")
assert io.exit_code == 0
assert f"{ROOT}/a/b/c" in dbx_files.directory_metadata
@pytest.mark.asyncio
async def test_mkdir_writes_are_mount_relative(write_ws):
io = await write_ws.execute("mkdir /dbx/newdir")
assert io.exit_code == 0
for key in io.writes:
assert key.startswith("/dbx/")
assert not key.startswith("/dbx/dbx/")
@pytest.mark.asyncio
async def test_mkdir_read_only_mount_rejected(read_ws, dbx_files):
io = await read_ws.execute("mkdir /dbx/newdir")
assert io.exit_code != 0
assert b"read-only" in io.stderr
assert dbx_files.create_directory_calls == []
@pytest.mark.asyncio
async def test_ops_mkdir(write_ws, dbx_files):
await write_ws.ops.mkdir("/dbx/opdir")
assert f"{ROOT}/opdir" in dbx_files.directory_metadata
@pytest.mark.asyncio
async def test_ops_mkdir_read_only_rejected(read_ws):
with pytest.raises(PermissionError):
await read_ws.ops.mkdir("/dbx/opdir")
@@ -0,0 +1,156 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import MountMode, Workspace
from tests.resource.databricks_volume.test_databricks_volume import (
FakeFiles, make_resource, seed_directory, seed_file)
ROOT = "/Volumes/main/default/agent_files/root"
@pytest.fixture
def dbx_files() -> FakeFiles:
files = FakeFiles()
seed_directory(files, ROOT)
seed_file(files, f"{ROOT}/src.txt", b"data")
return files
@pytest.fixture
def write_ws(dbx_files: FakeFiles) -> Workspace:
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.WRITE)
@pytest.fixture
def read_ws(dbx_files: FakeFiles) -> Workspace:
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
@pytest.mark.asyncio
async def test_mv_file_moves_bytes(write_ws, dbx_files):
io = await write_ws.execute("mv /dbx/src.txt /dbx/dst.txt")
assert io.exit_code == 0
assert dbx_files.downloads[f"{ROOT}/dst.txt"] == b"data"
assert f"{ROOT}/src.txt" not in dbx_files.downloads
@pytest.mark.asyncio
async def test_mv_writes_both_parents_mount_relative(write_ws):
io = await write_ws.execute("mv /dbx/src.txt /dbx/dst.txt")
assert io.exit_code == 0
assert "/dbx/src.txt" in io.writes
assert "/dbx/dst.txt" in io.writes
for key in io.writes:
assert not key.startswith("/dbx/dbx/")
@pytest.mark.asyncio
async def test_mv_no_clobber_skips_existing(write_ws, dbx_files):
seed_file(dbx_files, f"{ROOT}/dst.txt", b"keep")
io = await write_ws.execute("mv -n /dbx/src.txt /dbx/dst.txt")
assert io.exit_code == 0
assert dbx_files.downloads[f"{ROOT}/dst.txt"] == b"keep"
assert dbx_files.downloads[f"{ROOT}/src.txt"] == b"data"
@pytest.mark.asyncio
async def test_mv_read_only_mount_rejected(read_ws, dbx_files):
io = await read_ws.execute("mv /dbx/src.txt /dbx/dst.txt")
assert io.exit_code != 0
assert b"read-only" in io.stderr
assert dbx_files.downloads[f"{ROOT}/src.txt"] == b"data"
@pytest.mark.asyncio
async def test_ops_rename(write_ws, dbx_files):
await write_ws.ops.rename("/dbx/src.txt", "/dbx/renamed.txt")
assert dbx_files.downloads[f"{ROOT}/renamed.txt"] == b"data"
assert f"{ROOT}/src.txt" not in dbx_files.downloads
@pytest.mark.asyncio
async def test_mv_onto_same_path_errors_and_preserves_file(
write_ws, dbx_files):
io = await write_ws.execute("mv /dbx/src.txt /dbx/src.txt")
assert io.exit_code != 0
assert b"are the same file" in io.stderr
assert dbx_files.downloads[f"{ROOT}/src.txt"] == b"data"
assert f"{ROOT}/src.txt" not in dbx_files.delete_calls
@pytest.mark.asyncio
async def test_mv_into_dir_where_file_already_lives_errors_and_preserves_file(
write_ws, dbx_files):
io = await write_ws.execute("mv /dbx/src.txt /dbx/")
assert io.exit_code != 0
assert b"are the same file" in io.stderr
assert dbx_files.downloads[f"{ROOT}/src.txt"] == b"data"
@pytest.mark.asyncio
async def test_ops_rename_onto_same_path_is_noop(write_ws, dbx_files):
await write_ws.ops.rename("/dbx/src.txt", "/dbx/src.txt")
assert dbx_files.downloads[f"{ROOT}/src.txt"] == b"data"
assert f"{ROOT}/src.txt" not in dbx_files.delete_calls
@pytest.mark.asyncio
async def test_mv_multiple_sources_require_directory(write_ws, dbx_files):
seed_file(dbx_files, f"{ROOT}/a.txt", b"AAA")
seed_file(dbx_files, f"{ROOT}/b.txt", b"BBB")
seed_file(dbx_files, f"{ROOT}/target.txt", b"target")
io = await write_ws.execute("mv /dbx/a.txt /dbx/b.txt /dbx/target.txt")
assert io.exit_code != 0
assert b"not a directory" in io.stderr
assert dbx_files.downloads[f"{ROOT}/a.txt"] == b"AAA"
assert dbx_files.downloads[f"{ROOT}/b.txt"] == b"BBB"
assert dbx_files.downloads[f"{ROOT}/target.txt"] == b"target"
assert f"{ROOT}/a.txt" not in dbx_files.delete_calls
assert f"{ROOT}/b.txt" not in dbx_files.delete_calls
@pytest.mark.asyncio
async def test_mv_missing_source_reports_cannot_stat(write_ws, dbx_files):
io = await write_ws.execute("mv /dbx/missing /dbx/missing")
assert io.exit_code != 0
assert b"cannot stat" in io.stderr
assert b"are the same file" not in io.stderr
@pytest.mark.asyncio
async def test_mv_into_itself_errors_and_preserves_source(write_ws, dbx_files):
seed_directory(dbx_files, f"{ROOT}/d")
seed_file(dbx_files, f"{ROOT}/d/a.txt", b"aaa")
io = await write_ws.execute("mv /dbx/d /dbx/d")
assert io.exit_code != 0
assert b"subdirectory of itself" in io.stderr
assert dbx_files.downloads[f"{ROOT}/d/a.txt"] == b"aaa"
assert f"{ROOT}/d" in dbx_files.directory_metadata
assert f"{ROOT}/d/a.txt" not in dbx_files.delete_calls
@@ -0,0 +1,33 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
@pytest.mark.asyncio
async def test_workspace_execute_databricks_volume_nl(
databricks_text_workspace):
io = await databricks_text_workspace.execute("nl /dbx/words.txt")
assert io.exit_code == 0
assert b" 1\tbeta\n" in io.stdout
@pytest.mark.asyncio
async def test_workspace_execute_databricks_volume_nl_resolves_glob(
databricks_text_workspace):
io = await databricks_text_workspace.execute("nl /dbx/*.txt")
assert io.exit_code == 0
assert b" 1\tdelta\n" in io.stdout
@@ -0,0 +1,41 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.commands.builtin.databricks_volume import COMMANDS
TEXT_COMMANDS = {
"awk",
"cut",
"diff",
"jq",
"nl",
"sed",
"sort",
"tr",
"uniq",
"wc",
}
def test_databricks_volume_text_commands_registered_read_only():
registered = [
registered for command in COMMANDS
for registered in command._registered_commands
]
names = {command.name for command in registered}
assert TEXT_COMMANDS <= names
for command in registered:
if command.name in TEXT_COMMANDS:
assert command.resource == "databricks_volume"
assert not command.write
@@ -0,0 +1,61 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import MountMode, Workspace
from tests.resource.databricks_volume.test_databricks_volume import (
FakeFiles, make_resource, seed_directory, seed_file)
ROOT = "/Volumes/main/default/agent_files/root"
@pytest.fixture
def dbx_files() -> FakeFiles:
files = FakeFiles()
seed_directory(files, ROOT)
seed_file(files, f"{ROOT}/words.txt", b"beta\nalpha\nalpha\n")
files.create_directory(f"{ROOT}/sub")
seed_file(files, f"{ROOT}/sub/inner.txt", b"gamma\nalpha\n")
return files
@pytest.fixture
def ws(dbx_files: FakeFiles) -> Workspace:
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
@pytest.mark.asyncio
async def test_rg_searches_path_recursively(ws):
io = await ws.execute("rg alpha /dbx/")
assert io.exit_code == 0
out = io.stdout.decode()
assert "words.txt" in out
assert "inner.txt" in out
@pytest.mark.asyncio
async def test_rg_count_only(ws):
io = await ws.execute("rg -c alpha /dbx/words.txt")
assert io.exit_code == 0
assert b"2" in io.stdout
@pytest.mark.asyncio
async def test_rg_no_match_exits_nonzero(ws):
io = await ws.execute("rg zeta /dbx/words.txt")
assert io.exit_code != 0
@@ -0,0 +1,93 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import MountMode, Workspace
from tests.resource.databricks_volume.test_databricks_volume import (
FakeFiles, make_resource, seed_directory, seed_file)
ROOT = "/Volumes/main/default/agent_files/root"
@pytest.fixture
def dbx_files() -> FakeFiles:
files = FakeFiles()
seed_directory(files, ROOT)
files.create_directory(f"{ROOT}/d")
seed_file(files, f"{ROOT}/d/a.txt", b"a")
files.create_directory(f"{ROOT}/d/sub")
seed_file(files, f"{ROOT}/d/sub/b.txt", b"b")
return files
@pytest.fixture
def write_ws(dbx_files: FakeFiles) -> Workspace:
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.WRITE)
@pytest.fixture
def read_ws(dbx_files: FakeFiles) -> Workspace:
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
@pytest.mark.asyncio
async def test_rm_recursive_removes_tree(write_ws, dbx_files):
io = await write_ws.execute("rm -r /dbx/d")
assert io.exit_code == 0
assert f"{ROOT}/d" not in dbx_files.directory_metadata
assert f"{ROOT}/d/sub/b.txt" not in dbx_files.downloads
@pytest.mark.asyncio
async def test_rm_recursive_writes_are_mount_relative(write_ws):
io = await write_ws.execute("rm -r /dbx/d")
assert io.exit_code == 0
assert io.writes
for key in io.writes:
assert key.startswith("/dbx/")
assert not key.startswith("/dbx/dbx/")
@pytest.mark.asyncio
async def test_plain_rm_on_directory_fails(write_ws, dbx_files):
io = await write_ws.execute("rm /dbx/d")
assert io.exit_code != 0
assert f"{ROOT}/d" in dbx_files.directory_metadata
@pytest.mark.asyncio
async def test_rm_force_missing_succeeds(write_ws):
io = await write_ws.execute("rm -f /dbx/missing.txt")
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_rm_missing_fails(write_ws):
io = await write_ws.execute("rm /dbx/missing.txt")
assert io.exit_code != 0
@pytest.mark.asyncio
async def test_rm_recursive_read_only_rejected(read_ws, dbx_files):
io = await read_ws.execute("rm -r /dbx/d")
assert io.exit_code != 0
assert b"read-only" in io.stderr
assert f"{ROOT}/d" in dbx_files.directory_metadata
@@ -0,0 +1,60 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage.commands.builtin.databricks_volume import COMMANDS
def _sed_command():
for cmd in COMMANDS:
for rc in cmd._registered_commands:
if rc.name == "sed":
return cmd
raise LookupError("sed not registered for databricks_volume")
sed_command = _sed_command()
@pytest.mark.asyncio
async def test_workspace_execute_databricks_volume_sed(
databricks_text_workspace):
io = await databricks_text_workspace.execute(
"sed s/alpha/ALPHA/g /dbx/words.txt")
assert io.exit_code == 0
assert io.stdout == b"beta\nALPHA\nALPHA\n"
@pytest.mark.asyncio
async def test_workspace_execute_databricks_volume_sed_resolves_glob(
databricks_text_workspace):
io = await databricks_text_workspace.execute(
"sed s/delta/DELTA/ /dbx/*.txt")
assert io.exit_code == 0
assert b"DELTA\n" in io.stdout
@pytest.mark.asyncio
async def test_databricks_volume_sed_in_place_writes_back(
databricks_text_workspace):
# The volume has a write op, so -i edits in place through the shared
# builder (the old bespoke wrapper refused it unconditionally, #382).
io = await databricks_text_workspace.execute(
"sed -i s/alpha/ALPHA/g /dbx/words.txt")
assert io.exit_code == 0
assert io.writes.get("/dbx/words.txt") == b"beta\nALPHA\nALPHA\n"
@@ -0,0 +1,33 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
@pytest.mark.asyncio
async def test_workspace_execute_databricks_volume_sort(
databricks_text_workspace):
io = await databricks_text_workspace.execute("sort /dbx/words.txt")
assert io.exit_code == 0
assert io.stdout == b"alpha\nalpha\nbeta\n"
@pytest.mark.asyncio
async def test_workspace_execute_databricks_volume_sort_resolves_glob(
databricks_text_workspace):
io = await databricks_text_workspace.execute("sort /dbx/*.txt")
assert io.exit_code == 0
assert b"alpha\nalpha\nbeta\n" in io.stdout
@@ -0,0 +1,41 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
@pytest.mark.asyncio
async def test_stat_default_format(databricks_text_workspace):
io = await databricks_text_workspace.execute("stat /dbx/words.txt")
assert io.exit_code == 0
out = io.stdout.decode()
assert "name=words.txt" in out
assert "size=17" in out
@pytest.mark.asyncio
async def test_stat_custom_format(databricks_text_workspace):
io = await databricks_text_workspace.execute(
"stat -c '%n %s' /dbx/words.txt")
assert io.exit_code == 0
assert io.stdout.decode().strip() == "/dbx/words.txt 17"
@pytest.mark.asyncio
async def test_stat_missing_file_fails(databricks_text_workspace):
io = await databricks_text_workspace.execute("stat /dbx/missing.txt")
assert io.exit_code != 0
@@ -0,0 +1,25 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
@pytest.mark.asyncio
async def test_workspace_execute_databricks_volume_tr(
databricks_text_workspace):
io = await databricks_text_workspace.execute(
"cat /dbx/words.txt | tr a-z A-Z")
assert io.exit_code == 0
assert io.stdout == b"BETA\nALPHA\nALPHA\n"
@@ -0,0 +1,69 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import MountMode, Workspace
from tests.resource.databricks_volume.test_databricks_volume import (
FakeFiles, make_resource, seed_directory, seed_file)
ROOT = "/Volumes/main/default/agent_files/root"
@pytest.fixture
def dbx_files() -> FakeFiles:
files = FakeFiles()
seed_directory(files, ROOT)
seed_file(files, f"{ROOT}/words.txt", b"beta\nalpha\nalpha\n")
files.create_directory(f"{ROOT}/sub")
seed_file(files, f"{ROOT}/sub/inner.txt", b"gamma\nalpha\n")
files.create_directory(f"{ROOT}/sub/deep")
seed_file(files, f"{ROOT}/sub/deep/leaf.txt", b"leaf\n")
return files
@pytest.fixture
def ws(dbx_files: FakeFiles) -> Workspace:
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
@pytest.mark.asyncio
async def test_tree_lists_nested_entries(ws):
io = await ws.execute("tree /dbx/")
assert io.exit_code == 0
out = io.stdout.decode()
assert "words.txt" in out
assert "inner.txt" in out
@pytest.mark.asyncio
async def test_tree_max_depth(ws):
io = await ws.execute("tree -L 1 /dbx/")
assert io.exit_code == 0
out = io.stdout.decode()
assert "sub" in out
assert "inner.txt" in out
assert "leaf.txt" not in out
@pytest.mark.asyncio
async def test_tree_dirs_only(ws):
io = await ws.execute("tree -d /dbx/")
assert io.exit_code == 0
out = io.stdout.decode()
assert "sub" in out
assert "words.txt" not in out
@@ -0,0 +1,24 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
@pytest.mark.asyncio
async def test_workspace_execute_databricks_volume_uniq(
databricks_text_workspace):
io = await databricks_text_workspace.execute("uniq /dbx/words.txt")
assert io.exit_code == 0
assert io.stdout == b"beta\nalpha\n"
@@ -0,0 +1,33 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
@pytest.mark.asyncio
async def test_workspace_execute_databricks_volume_wc(
databricks_text_workspace):
io = await databricks_text_workspace.execute("wc -l /dbx/words.txt")
assert io.exit_code == 0
assert io.stdout == b"3 /dbx/words.txt\n"
@pytest.mark.asyncio
async def test_workspace_execute_databricks_volume_wc_resolves_glob(
databricks_text_workspace):
io = await databricks_text_workspace.execute("wc -l /dbx/*.txt")
assert io.exit_code == 0
assert b"/dbx/more.txt" in io.stdout
@@ -0,0 +1,60 @@
from types import SimpleNamespace
import pytest
from mirage.cache.index import RAMIndexCacheStore
from mirage.types import PathSpec
from mirage.utils.key_prefix import mount_key
def document(document_id: str, name: str, slug: str, size: int = 12) -> dict:
return {
"id": document_id,
"name": name,
"doc_metadata": [{
"name": "slug",
"value": slug
}],
"enabled": True,
"indexing_status": "completed",
"archived": False,
"tokens": 4,
"data_source_type": "upload_file",
"data_source_detail_dict": {
"upload_file": {
"size": size
}
},
"created_at": 1716282000,
}
@pytest.fixture
def dify_accessor() -> SimpleNamespace:
return SimpleNamespace(config=SimpleNamespace(slug_metadata_name="slug"))
@pytest.fixture
def dify_index() -> RAMIndexCacheStore:
return RAMIndexCacheStore()
@pytest.fixture
def knowledge_root() -> PathSpec:
return PathSpec(resource_path=mount_key("/knowledge", "/knowledge"),
virtual="/knowledge",
directory="/knowledge")
@pytest.fixture
def guide_path() -> PathSpec:
return PathSpec.from_str_path(
"/knowledge/guides/quickstart.md",
mount_key("/knowledge/guides/quickstart.md", "/knowledge"))
@pytest.fixture
def guides_path() -> PathSpec:
return PathSpec(resource_path=mount_key("/knowledge/guides", "/knowledge"),
virtual="/knowledge/guides",
directory="/knowledge/guides")
@@ -0,0 +1,76 @@
import pytest
from mirage.commands.builtin.dify.cat import make_cat
from mirage.commands.builtin.generic_bind import CommandIO, with_read_cache
from mirage.core.dify import read, tree
from mirage.core.dify.read import read_bytes as _read_bytes
from mirage.core.dify.read import read_stream as _read_stream
from mirage.core.dify.readdir import readdir as _readdir
from mirage.core.dify.stat import stat as _stat
from mirage.io.types import materialize
from mirage.types import PathSpec
from mirage.utils.key_prefix import mount_key
from .conftest import document
cat = make_cat(
with_read_cache(
CommandIO(
readdir=_readdir,
read_bytes=_read_bytes,
read_stream=_read_stream,
stat=_stat,
is_mounted=lambda a: True,
local=False,
)))
async def list_basic_documents(config):
return [
document("doc-1", "Guide", "guides/quickstart.md"),
document("doc-2", "Readme", "README.md"),
]
async def iter_basic_pages(config, document_id):
if document_id == "doc-1":
yield [{"content": "alpha\nbeta"}, {"content": "gamma"}]
else:
yield [{"content": "readme"}]
@pytest.mark.asyncio
async def test_cat_reads_stream_and_records_cache(monkeypatch, dify_accessor,
dify_index, guide_path):
monkeypatch.setattr(tree, "list_all_documents", list_basic_documents)
monkeypatch.setattr(read, "iter_segment_pages", iter_basic_pages)
stdout, io = await cat(dify_accessor, [guide_path], index=dify_index)
assert await materialize(stdout) == b"alpha\nbeta\ngamma"
assert guide_path.mount_path in io.reads
assert io.cache == [guide_path.mount_path]
async def get_two_doc_segments(config, document_id):
if document_id == "doc-1":
return [{"content": "alpha\nbeta"}, {"content": "gamma"}]
return [{"content": "readme"}]
@pytest.mark.asyncio
async def test_cat_multifile_caches_materialized_bytes_per_file(
monkeypatch, dify_accessor, dify_index, guide_path):
monkeypatch.setattr(tree, "list_all_documents", list_basic_documents)
monkeypatch.setattr(read, "get_document_segments", get_two_doc_segments)
readme_path = PathSpec.from_str_path(
"/knowledge/README.md", mount_key("/knowledge/README.md",
"/knowledge"))
stdout, io = await cat(dify_accessor, [guide_path, readme_path],
index=dify_index)
assert io.reads[guide_path.mount_path] == b"alpha\nbeta\ngamma"
assert io.reads[readme_path.mount_path] == b"readme"
assert all(isinstance(v, bytes) for v in io.reads.values())
assert await materialize(stdout) == b"alpha\nbeta\ngammareadme"
@@ -0,0 +1,98 @@
import pytest
from mirage.commands.builtin.dify.find import find
from mirage.core.dify import tree
from mirage.io.types import materialize
from mirage.types import PathSpec
from mirage.utils.key_prefix import mount_key
from .conftest import document
async def list_basic_documents(config):
return [
document("doc-1", "Guide", "guides/quickstart.md"),
document("doc-2", "Readme", "README.md"),
]
async def list_nested_documents(config):
return [
document("doc-1", "Guide", "guides/quickstart.md"),
document("doc-2", "Guide 2", "guides/deep/note.md"),
document("doc-3", "Readme", "README.md"),
]
@pytest.mark.asyncio
async def test_find_matches_name_pattern(monkeypatch, dify_accessor,
dify_index, knowledge_root):
monkeypatch.setattr(tree, "list_all_documents", list_basic_documents)
stdout, _ = await find(dify_accessor, [knowledge_root],
"quick*.md",
index=dify_index)
assert await materialize(stdout) == b"/knowledge/guides/quickstart.md\n"
@pytest.mark.asyncio
async def test_find_handles_file_missing_and_maxdepth(monkeypatch,
dify_accessor,
dify_index,
knowledge_root,
guide_path):
monkeypatch.setattr(tree, "list_all_documents", list_basic_documents)
file_stdout, file_io = await find(dify_accessor, [guide_path],
index=dify_index)
assert await materialize(file_stdout
) == b"/knowledge/guides/quickstart.md\n"
assert file_io.exit_code == 0
maxdepth_stdout, maxdepth_io = await find(dify_accessor, [knowledge_root],
maxdepth="0",
index=dify_index)
assert await materialize(maxdepth_stdout) == b"/knowledge\n"
assert maxdepth_io.exit_code == 0
missing = PathSpec.from_str_path(
"/knowledge/missing.md",
mount_key("/knowledge/missing.md", "/knowledge"))
missing_stdout, missing_io = await find(dify_accessor, [missing],
index=dify_index)
assert await materialize(missing_stdout) == b""
assert missing_io.stderr is not None
assert b"/knowledge/missing.md" in missing_io.stderr
assert missing_io.exit_code == 1
@pytest.mark.asyncio
async def test_find_uses_cwd_when_path_missing(monkeypatch, dify_accessor,
dify_index, guides_path):
monkeypatch.setattr(tree, "list_all_documents", list_nested_documents)
stdout, io = await find(dify_accessor, [],
"quick*.md",
cwd=guides_path,
index=dify_index)
assert await materialize(stdout) == b"/knowledge/guides/quickstart.md\n"
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_find_resolves_glob_patterns(monkeypatch, dify_accessor,
dify_index):
monkeypatch.setattr(tree, "list_all_documents", list_nested_documents)
path = PathSpec(resource_path=mount_key("/knowledge/guides/*.md",
"/knowledge"),
virtual="/knowledge/guides/*.md",
directory="/knowledge/guides",
pattern="*.md",
resolved=False)
stdout, io = await find(dify_accessor, [path], index=dify_index)
assert await materialize(stdout) == b"/knowledge/guides/quickstart.md\n"
assert io.exit_code == 0
@@ -0,0 +1,108 @@
from types import SimpleNamespace
import pytest
from mirage.cache.index import RAMIndexCacheStore
from mirage.io.types import materialize
from mirage.types import PathSpec
from mirage.utils.key_prefix import mount_key
def document(document_id: str, name: str, slug: str | None = None) -> dict:
metadata = []
if slug is not None:
metadata = [{"name": "slug", "value": slug}]
return {
"id": document_id,
"name": name,
"doc_metadata": metadata,
"enabled": True,
"indexing_status": "completed",
"archived": False,
"tokens": 4,
"data_source_type": "upload_file",
"data_source_detail_dict": {
"upload_file": {
"size": 12
}
},
"created_at": 1716282000,
}
def accessor() -> SimpleNamespace:
return SimpleNamespace(config=SimpleNamespace(slug_metadata_name="slug"))
@pytest.mark.asyncio
async def test_search_command_resolves_globs_and_passes_multiple_documents(
monkeypatch):
from mirage.commands.builtin.dify import search as command_search
from mirage.core.dify import search, tree
calls: list[tuple[list[PathSpec], dict]] = []
async def list_documents(config):
return [
document("doc-1", "API", "guides/api.md"),
document("doc-2", "Auth", "guides/auth.md"),
]
async def search_segments(accessor, query, paths, index, **kwargs):
calls.append((paths, kwargs))
return b"api\nauth"
monkeypatch.setattr(tree, "list_all_documents", list_documents)
monkeypatch.setattr(search, "search_segments", search_segments)
stdout, io = await command_search(
accessor(),
[
PathSpec(resource_path=mount_key("/knowledge/guides/*.md",
"/knowledge"),
virtual="/knowledge/guides/*.md",
directory="/knowledge/guides",
pattern="*.md",
resolved=False)
],
"login",
index=RAMIndexCacheStore(),
)
assert await materialize(stdout) == b"api\nauth"
assert io.reads == {}
assert io.cache == []
assert [path.virtual for path in calls[0][0]] == [
"/knowledge/guides/api.md",
"/knowledge/guides/auth.md",
]
assert calls[0][1]["mount_prefix"] == "/knowledge"
@pytest.mark.asyncio
async def test_search_command_root_searches_whole_dataset(monkeypatch):
from mirage.commands.builtin.dify import search as command_search
from mirage.core.dify import search
calls: list[tuple[list[PathSpec], dict]] = []
async def search_segments(accessor, query, paths, index, **kwargs):
calls.append((paths, kwargs))
return b"dataset"
monkeypatch.setattr(search, "search_segments", search_segments)
root = PathSpec(resource_path=mount_key("/knowledge", "/knowledge"),
virtual="/knowledge",
directory="/knowledge")
stdout, _ = await command_search(accessor(), [root],
"anything",
index=RAMIndexCacheStore())
assert await materialize(stdout) == b"dataset"
assert calls == [([], {
"method": "semantic",
"top_k": 10,
"threshold": 0.0,
"mount_prefix": "/knowledge"
})]
@@ -0,0 +1,53 @@
import pytest
from mirage.commands.builtin.dify import COMMANDS
from mirage.core.dify import read, tree
from mirage.io.types import materialize
from .conftest import document
def _sed_command():
for cmd in COMMANDS:
for rc in cmd._registered_commands:
if rc.name == "sed":
return cmd
raise LookupError("sed not registered for dify")
sed = _sed_command()
async def list_documents(config):
return [document("doc-1", "Guide", "guides/quickstart.md")]
async def get_segments(config, document_id):
return [{"content": "alpha beta"}]
@pytest.mark.asyncio
async def test_sed_transforms_dify_document(monkeypatch, dify_accessor,
dify_index, guide_path):
monkeypatch.setattr(tree, "list_all_documents", list_documents)
monkeypatch.setattr(read, "get_document_segments", get_segments)
stdout, io = await sed(dify_accessor, [guide_path],
"s/alpha/gamma/",
index=dify_index)
assert await materialize(stdout) == b"gamma beta"
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_sed_rejects_in_place(monkeypatch, dify_accessor, dify_index,
guide_path):
monkeypatch.setattr(tree, "list_all_documents", list_documents)
with pytest.raises(PermissionError,
match="-i not supported on this backend"):
await sed(dify_accessor, [guide_path],
"s/alpha/gamma/",
i=True,
index=dify_index)
@@ -0,0 +1,13 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
@@ -0,0 +1,76 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from unittest.mock import AsyncMock, patch
import pytest
from mirage.accessor.discord import DiscordAccessor
from mirage.cache.index.ram import RAMIndexCacheStore
from mirage.commands.builtin.discord import COMMANDS
from mirage.resource.discord.config import DiscordConfig
from mirage.types import PathSpec
GUILDS = [{"id": "G1", "name": "myguild"}]
GUILD_DIR = "/myguild__G1"
def _find_command():
for fn in COMMANDS:
for rc in getattr(fn, "_registered_commands", []):
if rc.name == "find" and rc.filetype is None:
return fn
raise AssertionError("factory find not registered for discord")
def _spec(virtual: str) -> PathSpec:
return PathSpec(virtual=virtual,
directory=virtual,
resource_path=virtual.strip("/"))
async def _run(paths, *texts: str, **flags) -> list[str]:
accessor = DiscordAccessor(DiscordConfig(token="t"))
find = _find_command()
with patch("mirage.core.discord.readdir.list_guilds",
new_callable=AsyncMock,
return_value=GUILDS):
stdout, _io = await find(accessor,
paths,
*texts,
index=RAMIndexCacheStore(),
**flags)
data = stdout if isinstance(stdout, bytes) else b""
return data.decode().splitlines()
@pytest.mark.asyncio
async def test_walk_lists_guild_tree():
lines = await _run([_spec("/")], maxdepth="2")
assert GUILD_DIR in lines
assert f"{GUILD_DIR}/channels" in lines
assert f"{GUILD_DIR}/members" in lines
@pytest.mark.asyncio
async def test_path_pattern_is_honored():
lines = await _run([_spec("/")], maxdepth="2", path="*channels*")
assert lines == [f"{GUILD_DIR}/channels"]
@pytest.mark.asyncio
async def test_size_is_honored_dirs_count_as_zero():
lines = await _run([_spec("/")], maxdepth="2", size="+0c")
assert lines == []
@@ -0,0 +1,252 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from unittest.mock import AsyncMock, patch
import pytest
from mirage.commands.builtin.discord.grep import grep
from mirage.commands.builtin.discord.rg import rg
from mirage.types import FileStat, FileType, PathSpec
from mirage.utils.key_prefix import mount_key
def _concrete_paths(n: int = 7):
paths = []
for d in range(1, n + 1):
original = (
f"/discord/myguild/channels/general/2026-01-{d:02d}/chat.jsonl")
paths.append(
PathSpec(
resource_path=mount_key(original, "/discord"),
virtual=original,
directory=original,
))
return paths
def _fake_index(channel_id: str = "ch_456", guild_id: str = "g_123"):
idx = AsyncMock()
async def _get(virtual_key):
result = AsyncMock()
if virtual_key.endswith("/myguild/channels/general"):
result.entry = type("E", (), {"id": channel_id})
elif virtual_key.endswith("/myguild"):
result.entry = type("E", (), {"id": guild_id})
else:
result.entry = None
return result
idx.get.side_effect = _get
return idx
@pytest.mark.asyncio
async def test_discord_grep_with_many_concrete_paths_uses_native_search():
accessor = AsyncMock()
accessor.config = AsyncMock()
fake_msgs = [{
"content": "hello world",
"channel_id": "ch_456",
"author": {
"username": "alice"
},
"timestamp": "2026-01-15T12:34:56.000000+00:00",
"id": "1"
}]
fake_channels = [{"id": "ch_456", "name": "general"}]
with patch(
"mirage.commands.builtin.discord.grep.search_guild",
new=AsyncMock(return_value=fake_msgs),
) as fake_search, patch(
"mirage.commands.builtin.discord.grep.list_channels",
new=AsyncMock(return_value=fake_channels),
):
out, io = await grep(accessor,
_concrete_paths(7),
"hello",
index=_fake_index())
assert fake_search.await_count == 1
assert io.exit_code == 0
assert b"hello" in out
assert out.endswith(b"\n")
assert (b"/discord/myguild/channels/general__ch_456/"
b"2026-01-15/chat.jsonl:") in out
@pytest.mark.asyncio
async def test_discord_grep_falls_back_when_native_raises():
accessor = AsyncMock()
accessor.config = AsyncMock()
paths = [
PathSpec(resource_path=mount_key(
"/discord/myguild/channels/general/*.jsonl", "/discord"),
virtual="/discord/myguild/channels/general/*.jsonl",
directory="/discord/myguild/channels/general/",
pattern="*.jsonl"),
]
with patch(
"mirage.commands.builtin.discord.grep.search_guild",
new=AsyncMock(side_effect=RuntimeError("rate limited")),
), patch(
"mirage.commands.builtin.discord.grep.resolve_glob",
new=AsyncMock(return_value=paths),
) as fake_resolve, patch(
"mirage.commands.builtin.discord.grep.discord_read",
new=AsyncMock(return_value=b""),
), patch(
"mirage.commands.builtin.discord.grep._stat",
new=AsyncMock(return_value=FileStat(name="2026-04-10.jsonl",
type=FileType.TEXT)),
):
out, io = await grep(accessor, paths, "hello", index=_fake_index())
assert fake_resolve.await_count == 1
assert io.exit_code in (0, 1)
@pytest.mark.asyncio
async def test_discord_grep_native_empty_does_not_trigger_fallback():
"""search_guild returning [] is a legit no-match — don't double-scan."""
accessor = AsyncMock()
accessor.config = AsyncMock()
with patch(
"mirage.commands.builtin.discord.grep.search_guild",
new=AsyncMock(return_value=[]),
) as fake_search, patch(
"mirage.commands.builtin.discord.grep.list_channels",
new=AsyncMock(return_value=[]),
), patch(
"mirage.commands.builtin.discord.grep.discord_read",
new=AsyncMock(return_value=b""),
) as fake_read:
out, io = await grep(accessor,
_concrete_paths(7),
"missing",
index=_fake_index())
assert fake_search.await_count == 1
assert fake_read.await_count == 0
assert io.exit_code == 1
assert out == b""
@pytest.mark.asyncio
async def test_discord_grep_multi_pattern_skips_native_search():
"""grep -e a -e b must bypass the native search push-down.
The push-down passes a single newline-joined pattern to the native
search, which treats it as one literal and matches nothing. Multiple
-e patterns must fall through to the generic grep instead.
"""
accessor = AsyncMock()
accessor.config = AsyncMock()
paths = [
PathSpec(resource_path=mount_key(
"/discord/myguild/channels/general/*.jsonl", "/discord"),
virtual="/discord/myguild/channels/general/*.jsonl",
directory="/discord/myguild/channels/general/",
pattern="*.jsonl"),
]
with patch(
"mirage.commands.builtin.discord.grep.search_guild",
new=AsyncMock(return_value=[]),
) as fake_search, patch(
"mirage.commands.builtin.discord.grep.resolve_glob",
new=AsyncMock(return_value=paths),
) as fake_resolve, patch(
"mirage.commands.builtin.discord.grep.discord_read",
new=AsyncMock(return_value=b""),
), patch(
"mirage.commands.builtin.discord.grep._stat",
new=AsyncMock(return_value=FileStat(name="2026-04-10.jsonl",
type=FileType.TEXT)),
):
_, io = await grep(accessor,
paths,
e=["ada", "ben"],
index=_fake_index())
assert fake_search.await_count == 0
assert fake_resolve.await_count == 1
@pytest.mark.asyncio
async def test_discord_rg_with_many_concrete_paths_uses_native_search():
accessor = AsyncMock()
accessor.config = AsyncMock()
fake_msgs = [{
"content": "hello rg",
"channel_id": "ch_456",
"author": {
"username": "bob"
},
"timestamp": "2026-01-15T08:00:00.000000+00:00",
"id": "2"
}]
fake_channels = [{"id": "ch_456", "name": "general"}]
with patch(
"mirage.commands.builtin.discord.rg.search_guild",
new=AsyncMock(return_value=fake_msgs),
) as fake_search, patch(
"mirage.commands.builtin.discord.rg.list_channels",
new=AsyncMock(return_value=fake_channels),
):
out, io = await rg(accessor,
_concrete_paths(7),
"hello",
index=_fake_index())
assert fake_search.await_count == 1
assert io.exit_code == 0
assert b"hello" in out
assert out.endswith(b"\n")
assert (b"/discord/myguild/channels/general__ch_456/"
b"2026-01-15/chat.jsonl:") in out
@pytest.mark.asyncio
async def test_discord_rg_multi_pattern_skips_native_search():
"""rg -e a -e b must bypass the native search push-down.
Like grep, the push-down passes a single newline-joined pattern to the
native search, which matches nothing. Multiple -e patterns must fall
through to the generic rg instead.
"""
accessor = AsyncMock()
accessor.config = AsyncMock()
paths = [
PathSpec(resource_path=mount_key(
"/discord/myguild/channels/general/*.jsonl", "/discord"),
virtual="/discord/myguild/channels/general/*.jsonl",
directory="/discord/myguild/channels/general/",
pattern="*.jsonl"),
]
with patch(
"mirage.commands.builtin.discord.rg.search_guild",
new=AsyncMock(return_value=[]),
) as fake_search, patch(
"mirage.commands.builtin.discord.rg.resolve_glob",
new=AsyncMock(return_value=paths),
) as fake_resolve, patch(
"mirage.commands.builtin.discord.rg.discord_read",
new=AsyncMock(return_value=b""),
), patch(
"mirage.commands.builtin.discord.rg._stat",
new=AsyncMock(return_value=FileStat(name="2026-04-10.jsonl",
type=FileType.TEXT)),
):
_, io = await rg(accessor,
paths,
e=["ada", "ben"],
index=_fake_index())
assert fake_search.await_count == 0
assert fake_resolve.await_count == 1
@@ -0,0 +1,232 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
from unittest.mock import AsyncMock, patch
import pytest
from mirage.accessor.discord import DiscordAccessor
from mirage.cache.index import IndexEntry
from mirage.cache.index.ram import RAMIndexCacheStore
from mirage.commands.builtin.discord import COMMANDS
from mirage.commands.builtin.discord.grep import grep
from mirage.commands.builtin.discord.head import head
from mirage.resource.discord.config import DiscordConfig
from mirage.types import PathSpec
GUILD_PATH = "TestGuild"
CHANNEL_PATH = "TestGuild/channels/general"
DATE_DIR_PATH = "TestGuild/channels/general/2024-01-15"
FILE_PATH = "TestGuild/channels/general/2024-01-15/chat.jsonl"
ABS_FILE = "/" + FILE_PATH
ABS_CHANNEL = "/" + CHANNEL_PATH
ABS_DATE_DIR = "/" + DATE_DIR_PATH
def _glob_result(path: str) -> list[PathSpec]:
return [
PathSpec(resource_path=(path).strip("/"),
virtual=path,
directory="/",
resolved=True)
]
FAKE_JSONL = (
b'{"id":"1","content":"hello world","author":{"username":"alice"}}\n'
b'{"id":"2","content":"goodbye moon","author":{"username":"bob"}}\n'
b'{"id":"3","content":"hello again","author":{"username":"alice"}}\n')
def _make_glob(path: str, resolved: bool = True) -> list[PathSpec]:
return [
PathSpec(resource_path=(path).strip("/"),
virtual=path,
directory="/",
resolved=resolved)
]
def _run(coro):
return asyncio.run(coro)
def _find_command():
for fn in COMMANDS:
for rc in getattr(fn, "_registered_commands", []):
if rc.name == "find" and rc.filetype is None:
return fn
raise AssertionError("factory find not registered for discord")
def _make_index() -> RAMIndexCacheStore:
index = RAMIndexCacheStore(ttl=600)
_run(
index.put(
"/" + GUILD_PATH,
IndexEntry(id="G1",
name="TestGuild",
resource_type="discord/guild",
vfs_name="TestGuild")))
_run(
index.put(
"/" + CHANNEL_PATH,
IndexEntry(id="C1",
name="general",
resource_type="discord/channel",
vfs_name="general")))
_run(
index.put(
"/" + DATE_DIR_PATH,
IndexEntry(id="C1:2024-01-15",
name="2024-01-15",
resource_type="discord/history",
vfs_name="2024-01-15")))
_run(
index.set_dir("/" + CHANNEL_PATH, [
("2024-01-15",
IndexEntry(id="C1:2024-01-15",
name="2024-01-15",
resource_type="discord/history",
vfs_name="2024-01-15")),
]))
_run(
index.put(
"/" + FILE_PATH,
IndexEntry(id="C1:2024-01-15:chat",
name="chat.jsonl",
resource_type="discord/chat_jsonl",
vfs_name="chat.jsonl")))
_run(
index.set_dir("/" + DATE_DIR_PATH, [
("chat.jsonl",
IndexEntry(id="C1:2024-01-15:chat",
name="chat.jsonl",
resource_type="discord/chat_jsonl",
vfs_name="chat.jsonl")),
]))
return index
@pytest.fixture
def accessor():
config = DiscordConfig(token="test-token")
return DiscordAccessor(config=config)
@pytest.fixture
def index():
return _make_index()
async def _collect(stream) -> bytes:
if isinstance(stream, bytes):
return stream
chunks = []
async for chunk in stream:
chunks.append(chunk)
return b"".join(chunks)
@pytest.mark.asyncio
async def test_head(accessor):
with (
patch("mirage.commands.builtin.discord.head.resolve_glob",
new_callable=AsyncMock,
return_value=_glob_result(ABS_FILE)),
patch("mirage.commands.builtin.discord.head.discord_read",
new_callable=AsyncMock,
return_value=FAKE_JSONL),
):
stream, io = await head(accessor, _make_glob(ABS_FILE), n="1")
data = await _collect(stream)
assert b"hello world" in data
assert b"goodbye moon" not in data
@pytest.mark.asyncio
async def test_head_default(accessor):
with (
patch("mirage.commands.builtin.discord.head.resolve_glob",
new_callable=AsyncMock,
return_value=_glob_result(ABS_FILE)),
patch("mirage.commands.builtin.discord.head.discord_read",
new_callable=AsyncMock,
return_value=FAKE_JSONL),
):
stream, io = await head(accessor, _make_glob(ABS_FILE))
data = await _collect(stream)
assert b"hello world" in data
assert b"goodbye moon" in data
assert b"hello again" in data
@pytest.mark.asyncio
async def test_grep(accessor):
with (
patch("mirage.commands.builtin.discord.grep.resolve_glob",
new_callable=AsyncMock,
return_value=_glob_result(ABS_FILE)),
patch("mirage.commands.builtin.discord.grep.discord_read",
new_callable=AsyncMock,
return_value=FAKE_JSONL),
):
stream, io = await grep(accessor, _make_glob(ABS_FILE), "hello")
data = await _collect(stream)
assert b"hello world" in data
assert b"hello again" in data
assert b"goodbye" not in data
@pytest.mark.asyncio
async def test_grep_invert(accessor):
with (
patch("mirage.commands.builtin.discord.grep.resolve_glob",
new_callable=AsyncMock,
return_value=_glob_result(ABS_FILE)),
patch("mirage.commands.builtin.discord.grep.discord_read",
new_callable=AsyncMock,
return_value=FAKE_JSONL),
):
stream, io = await grep(accessor,
_make_glob(ABS_FILE),
"hello",
v=True)
data = await _collect(stream)
assert b"goodbye moon" in data
assert b"hello world" not in data
@pytest.mark.asyncio
async def test_find(accessor, index):
find = _find_command()
stream, io = await find(accessor,
_make_glob(ABS_CHANNEL, resolved=False),
index=index)
data = await _collect(stream)
assert b"2024-01-15" in data
@pytest.mark.asyncio
async def test_find_with_name(accessor, index):
find = _find_command()
stream, io = await find(
accessor,
_make_glob(ABS_CHANNEL, resolved=False),
name="chat.jsonl",
index=index,
)
data = await _collect(stream)
assert b"chat.jsonl" in data
@@ -0,0 +1,93 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from unittest.mock import AsyncMock, patch
import pytest
from mirage.commands.builtin.discord.grep import grep
from mirage.commands.builtin.discord.rg import rg
from mirage.io.types import IOResult
from mirage.types import PathSpec
from mirage.utils.key_prefix import mount_key
def _path(path: str) -> PathSpec:
return PathSpec(resource_path=mount_key(path, "/discord"),
virtual=path,
directory=path)
def _fake_index():
idx = AsyncMock()
async def _get(virtual_key):
result = AsyncMock()
if virtual_key.endswith("/myguild/channels/general"):
result.entry = type("E", (), {"id": "C1", "remote_time": ""})
elif virtual_key.endswith("/myguild"):
result.entry = type("E", (), {"id": "G1"})
else:
result.entry = None
return result
idx.get.side_effect = _get
return idx
@pytest.mark.asyncio
async def test_grep_emits_token_hint_on_forbidden():
accessor = AsyncMock()
accessor.config = AsyncMock()
paths = [_path("/discord/myguild/channels/general/2026-01-01/chat.jsonl")]
with patch(
"mirage.commands.builtin.discord.grep.search_guild",
new=AsyncMock(side_effect=RuntimeError("403 Forbidden")),
), patch(
"mirage.commands.builtin.discord.grep.resolve_glob",
new=AsyncMock(return_value=paths),
), patch(
"mirage.commands.builtin.discord.grep.discord_read",
new=AsyncMock(return_value=b""),
):
_out, io = await grep(accessor,
paths,
"hi",
index=_fake_index(),
args_l=True)
stderr = (io.stderr or b"").decode()
assert "push-down failed" in stderr
assert "READ_MESSAGE_HISTORY" in stderr
@pytest.mark.asyncio
async def test_rg_emits_warning_on_rate_limit():
accessor = AsyncMock()
accessor.config = AsyncMock()
paths = [_path("/discord/myguild/channels/general/2026-01-01/chat.jsonl")]
with patch(
"mirage.commands.builtin.discord.rg.search_guild",
new=AsyncMock(side_effect=RuntimeError("rate limited 429")),
), patch(
"mirage.commands.builtin.discord.rg.resolve_glob",
new=AsyncMock(return_value=paths),
), patch(
"mirage.commands.builtin.discord.rg.generic_rg",
new=AsyncMock(return_value=(b"", IOResult(exit_code=1))),
):
_out, io = await rg(accessor, paths, "hi", index=_fake_index())
stderr = (io.stderr or b"").decode()
assert "push-down failed" in stderr
# 429 doesn't trigger the perm hint; should still warn
assert "READ_MESSAGE_HISTORY" not in stderr
@@ -0,0 +1,69 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import json
from unittest.mock import AsyncMock, patch
import pytest
from mirage.accessor.discord import DiscordAccessor
from mirage.commands.builtin.discord.discord_add_reaction import \
discord_add_reaction
from mirage.commands.builtin.discord.discord_send_message import \
discord_send_message
from mirage.resource.discord.config import DiscordConfig
@pytest.fixture
def accessor():
return DiscordAccessor(config=DiscordConfig(token="test-bot-token"), )
@pytest.mark.asyncio
async def test_send_message(accessor):
with patch(
"mirage.commands.builtin.discord.discord_send_message"
".send_message",
new_callable=AsyncMock,
return_value={
"id": "msg1",
"content": "hello"
},
) as mock_send:
stream, io_result = await discord_send_message(accessor, [],
channel_id="C001",
text="hello")
mock_send.assert_called_once_with(accessor.config, "C001", "hello", None)
out = json.loads(stream)
assert out["id"] == "msg1"
@pytest.mark.asyncio
async def test_add_reaction(accessor):
with patch(
"mirage.commands.builtin.discord.discord_add_reaction"
".add_reaction",
new_callable=AsyncMock,
return_value=None,
) as mock_react:
stream, io_result = await discord_add_reaction(accessor, [],
channel_id="C001",
message_id="msg1",
reaction="thumbsup")
mock_react.assert_called_once_with(accessor.config, "C001", "msg1",
"thumbsup")
out = json.loads(stream)
assert out["ok"] is True
@@ -0,0 +1,84 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import DiskResource, MountMode, Workspace
@pytest.fixture
def workspace(tmp_path):
return Workspace({"/": DiskResource(root=str(tmp_path))},
mode=MountMode.WRITE)
@pytest.mark.asyncio
async def test_cat_basic(workspace):
await workspace.ops.write("/f.txt", b"hello\nworld\n")
io = await workspace.execute("cat /f.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b"hello\nworld\n"
@pytest.mark.asyncio
async def test_cat_n_single_digit_alignment(workspace):
await workspace.ops.write("/f.txt", b"a\nb\n")
io = await workspace.execute("cat -n /f.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b" 1\ta\n 2\tb\n"
@pytest.mark.asyncio
async def test_cat_n_multidigit_alignment(workspace):
body = b"".join(f"line{i}\n".encode() for i in range(1, 13))
await workspace.ops.write("/big.txt", body)
io = await workspace.execute("cat -n /big.txt", session_id="default")
assert io.exit_code == 0
lines = io.stdout.split(b"\n")
assert lines[0] == b" 1\tline1"
assert lines[8] == b" 9\tline9"
assert lines[9] == b" 10\tline10"
assert lines[11] == b" 12\tline12"
@pytest.mark.asyncio
async def test_cat_preserves_no_trailing_newline(workspace):
await workspace.ops.write("/partial.txt", b"hello")
io = await workspace.execute("cat /partial.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b"hello"
@pytest.mark.asyncio
async def test_cat_n_preserves_no_trailing_newline(workspace):
await workspace.ops.write("/partial.txt", b"hello")
io = await workspace.execute("cat -n /partial.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b" 1\thello"
@pytest.mark.asyncio
async def test_cat_empty_file(workspace):
await workspace.ops.write("/empty.txt", b"")
io = await workspace.execute("cat /empty.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b""
@pytest.mark.asyncio
async def test_cat_only_newlines(workspace):
await workspace.ops.write("/nl.txt", b"\n\n\n")
io = await workspace.execute("cat -n /nl.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b" 1\t\n 2\t\n 3\t\n"
@@ -0,0 +1,91 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import DiskResource, MountMode, Workspace
@pytest.fixture
def workspace(tmp_path):
return Workspace({"/": DiskResource(root=str(tmp_path))},
mode=MountMode.WRITE)
@pytest.mark.asyncio
async def test_du_single_file(workspace):
await workspace.ops.write("/f.txt", b"hello")
io = await workspace.execute("du /f.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout.decode().strip() == "5\t/f.txt"
@pytest.mark.asyncio
async def test_du_directory_collapses(workspace):
await workspace.ops.mkdir("/dir")
await workspace.ops.write("/dir/a.txt", b"aaa")
await workspace.ops.write("/dir/b.txt", b"bb")
io = await workspace.execute("du /dir", session_id="default")
assert io.exit_code == 0
assert io.stdout.decode().strip() == "5\t/dir"
@pytest.mark.asyncio
async def test_du_a_lists_files(workspace):
await workspace.ops.mkdir("/dir")
await workspace.ops.write("/dir/a.txt", b"aaa")
await workspace.ops.write("/dir/b.txt", b"bb")
io = await workspace.execute("du -a /dir", session_id="default")
assert io.exit_code == 0
out = io.stdout.decode()
assert "a.txt" in out
assert "b.txt" in out
@pytest.mark.asyncio
async def test_du_s_summary(workspace):
await workspace.ops.mkdir("/dir")
await workspace.ops.mkdir("/dir/sub")
await workspace.ops.write("/dir/a.txt", b"hello")
await workspace.ops.write("/dir/sub/b.txt", b"world")
io = await workspace.execute("du -s /dir", session_id="default")
assert io.exit_code == 0
lines = io.stdout.decode().strip().splitlines()
assert len(lines) == 1
@pytest.mark.asyncio
async def test_du_c_total(workspace):
await workspace.ops.write("/a.txt", b"hello")
await workspace.ops.write("/b.txt", b"world")
io = await workspace.execute("du -c /a.txt /b.txt", session_id="default")
assert io.exit_code == 0
lines = io.stdout.decode().strip().splitlines()
assert lines[-1] == "10\ttotal"
@pytest.mark.asyncio
async def test_du_h_human(workspace):
await workspace.ops.write("/big.txt", b"x" * 2048)
io = await workspace.execute("du -h /big.txt", session_id="default")
assert io.exit_code == 0
size_str = io.stdout.decode().strip().split("\t")[0]
assert size_str.endswith("K")
@pytest.mark.asyncio
async def test_du_missing_operand_errors(workspace):
io = await workspace.execute("du", session_id="default")
assert io.exit_code != 0
assert b"missing operand" in (io.stderr or b"")
@@ -0,0 +1,83 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import DiskResource, MountMode, Workspace
@pytest.fixture
def workspace(tmp_path):
return Workspace({"/": DiskResource(root=str(tmp_path))},
mode=MountMode.WRITE)
@pytest.mark.asyncio
async def test_find_name_glob(workspace):
await workspace.ops.write("/hello.txt", b"hi")
await workspace.ops.write("/world.py", b"hi")
io = await workspace.execute("find / -name '*.txt'", session_id="default")
assert io.exit_code == 0
out = io.stdout.decode()
assert "hello.txt" in out
assert "world.py" not in out
@pytest.mark.asyncio
async def test_find_type_f(workspace):
await workspace.ops.mkdir("/sub")
await workspace.ops.write("/a.txt", b"a")
await workspace.ops.write("/sub/b.txt", b"b")
io = await workspace.execute("find / -type f", session_id="default")
assert io.exit_code == 0
out = io.stdout.decode()
assert "/a.txt" in out
assert "/sub/b.txt" in out
@pytest.mark.asyncio
async def test_find_type_d(workspace):
await workspace.ops.mkdir("/sub")
await workspace.ops.write("/a.txt", b"a")
io = await workspace.execute("find / -type d", session_id="default")
assert io.exit_code == 0
out = io.stdout.decode()
assert "/sub" in out
assert "/a.txt" not in out
@pytest.mark.asyncio
async def test_find_size_lower_bound(workspace):
await workspace.ops.write("/big.txt", b"x" * 1000)
await workspace.ops.write("/small.txt", b"x")
io = await workspace.execute("find / -size +500c -type f",
session_id="default")
assert io.exit_code == 0
out = io.stdout.decode()
assert "big.txt" in out
assert "small.txt" not in out
@pytest.mark.asyncio
async def test_find_maxdepth(workspace):
await workspace.ops.mkdir("/sub")
await workspace.ops.mkdir("/sub/deep")
await workspace.ops.write("/a.txt", b"a")
await workspace.ops.write("/sub/deep/c.txt", b"c")
io = await workspace.execute("find / -maxdepth 1 -type f",
session_id="default")
assert io.exit_code == 0
out = io.stdout.decode()
assert "/a.txt" in out
assert "/sub/deep/c.txt" not in out
@@ -0,0 +1,75 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import DiskResource, MountMode, Workspace
@pytest.fixture
def workspace(tmp_path):
return Workspace({"/": DiskResource(root=str(tmp_path))},
mode=MountMode.WRITE)
@pytest.mark.asyncio
async def test_head_default_n_10(workspace):
body = b"".join(f"line{i}\n".encode() for i in range(1, 15))
await workspace.ops.write("/f.txt", body)
io = await workspace.execute("head /f.txt", session_id="default")
assert io.exit_code == 0
lines = io.stdout.decode().splitlines()
assert len(lines) == 10
assert lines[0] == "line1"
assert lines[9] == "line10"
@pytest.mark.asyncio
async def test_head_n_explicit(workspace):
await workspace.ops.write("/f.txt", b"a\nb\nc\nd\n")
io = await workspace.execute("head -n 2 /f.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b"a\nb\n"
@pytest.mark.asyncio
async def test_head_c_bytes(workspace):
await workspace.ops.write("/f.txt", b"hello world")
io = await workspace.execute("head -c 5 /f.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b"hello"
@pytest.mark.asyncio
async def test_head_negative_n_excludes_last(workspace):
await workspace.ops.write("/f.txt", b"a\nb\nc\nd\n")
io = await workspace.execute("head -n -1 /f.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b"a\nb\nc\n"
@pytest.mark.asyncio
async def test_head_no_trailing_newline(workspace):
await workspace.ops.write("/partial.txt", b"hello")
io = await workspace.execute("head /partial.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b"hello"
@pytest.mark.asyncio
async def test_head_empty_file(workspace):
await workspace.ops.write("/empty.txt", b"")
io = await workspace.execute("head /empty.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b""
@@ -0,0 +1,90 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import DiskResource, MountMode, Workspace
@pytest.fixture
def workspace(tmp_path):
return Workspace({"/": DiskResource(root=str(tmp_path))},
mode=MountMode.WRITE)
@pytest.mark.asyncio
async def test_ls_lists_files(workspace):
await workspace.ops.write("/a.txt", b"a")
await workspace.ops.write("/b.txt", b"b")
io = await workspace.execute("ls /", session_id="default")
assert io.exit_code == 0
names = set(io.stdout.decode().strip().split("\n"))
assert "a.txt" in names
assert "b.txt" in names
@pytest.mark.asyncio
async def test_ls_a_shows_dotfiles(workspace):
await workspace.ops.write("/.hidden", b"h")
await workspace.ops.write("/visible.txt", b"v")
io = await workspace.execute("ls -a /", session_id="default")
assert io.exit_code == 0
names = set(io.stdout.decode().strip().split("\n"))
assert ".hidden" in names
assert "visible.txt" in names
@pytest.mark.asyncio
async def test_ls_l_long_format_includes_size(workspace):
await workspace.ops.write("/f.txt", b"hello")
io = await workspace.execute("ls -l /", session_id="default")
assert io.exit_code == 0
out = io.stdout.decode()
assert "f.txt" in out
assert "5" in out
@pytest.mark.asyncio
async def test_ls_F_classify_marks_dirs(workspace):
await workspace.ops.mkdir("/sub")
await workspace.ops.write("/sub/a.txt", b"a")
io = await workspace.execute("ls -F /", session_id="default")
assert io.exit_code == 0
assert "sub/" in io.stdout.decode()
@pytest.mark.asyncio
async def test_ls_R_recursive(workspace):
await workspace.ops.mkdir("/sub")
await workspace.ops.write("/sub/a.txt", b"a")
io = await workspace.execute("ls -R /", session_id="default")
assert io.exit_code == 0
out = io.stdout.decode()
assert "sub" in out
assert "a.txt" in out
@pytest.mark.asyncio
async def test_ls_d_lists_dir_itself(workspace):
await workspace.ops.mkdir("/sub")
io = await workspace.execute("ls -d /sub", session_id="default")
assert io.exit_code == 0
assert "sub" in io.stdout.decode()
@pytest.mark.asyncio
async def test_ls_missing_path_returns_exit_1(workspace):
io = await workspace.execute("ls /nope", session_id="default")
assert io.exit_code == 1
assert b"nope" in (io.stderr or b"")
@@ -0,0 +1,33 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import DiskResource, MountMode, Workspace
@pytest.fixture
def workspace(tmp_path):
return Workspace({"/": DiskResource(root=str(tmp_path))},
mode=MountMode.WRITE)
@pytest.mark.asyncio
async def test_rm_v_terminates_verbose_output(workspace):
await workspace.ops.write("/a.txt", b"a")
io = await workspace.execute("rm -v /a.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b"removed '/a.txt'\n"
@@ -0,0 +1,83 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import DiskResource, MountMode, Workspace
@pytest.fixture
def workspace(tmp_path):
return Workspace({"/": DiskResource(root=str(tmp_path))},
mode=MountMode.WRITE)
@pytest.mark.asyncio
async def test_tail_default_n_10(workspace):
body = b"\n".join(f"line{i}".encode() for i in range(1, 21)) + b"\n"
await workspace.ops.write("/f.txt", body)
io = await workspace.execute("tail /f.txt", session_id="default")
assert io.exit_code == 0
expected = b"\n".join(f"line{i}".encode() for i in range(11, 21)) + b"\n"
assert io.stdout == expected
@pytest.mark.asyncio
async def test_tail_n_explicit(workspace):
await workspace.ops.write("/f.txt", b"a\nb\nc\nd\ne\n")
io = await workspace.execute("tail -n 3 /f.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b"c\nd\ne\n"
@pytest.mark.asyncio
async def test_tail_c_bytes(workspace):
await workspace.ops.write("/f.txt", b"hello world")
io = await workspace.execute("tail -c 5 /f.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b"world"
@pytest.mark.asyncio
async def test_tail_plus_n_streams_from_line(workspace):
await workspace.ops.write("/f.txt", b"a\nb\nc\nd\ne\n")
io = await workspace.execute("tail -n +3 /f.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b"c\nd\ne\n"
@pytest.mark.asyncio
async def test_tail_no_trailing_newline(workspace):
await workspace.ops.write("/partial.txt", b"hello")
io = await workspace.execute("tail /partial.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b"hello"
@pytest.mark.asyncio
async def test_tail_empty_file(workspace):
await workspace.ops.write("/empty.txt", b"")
io = await workspace.execute("tail /empty.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout == b""
@pytest.mark.asyncio
async def test_tail_multi_file_emits_headers(workspace):
await workspace.ops.write("/a.txt", b"x\ny\n")
await workspace.ops.write("/b.txt", b"z\n")
io = await workspace.execute("tail /a.txt /b.txt", session_id="default")
assert io.exit_code == 0
assert b"==> /a.txt <==" in io.stdout
assert b"==> /b.txt <==" in io.stdout
@@ -0,0 +1,61 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import DiskResource, MountMode, Workspace
@pytest.fixture
def workspace(tmp_path):
return Workspace({"/": DiskResource(root=str(tmp_path))},
mode=MountMode.WRITE)
@pytest.mark.asyncio
async def test_tree_basic(workspace):
await workspace.ops.mkdir("/d1")
await workspace.ops.write("/d1/a.txt", b"a")
await workspace.ops.write("/d1/b.txt", b"b")
io = await workspace.execute("tree /d1", session_id="default")
assert io.exit_code == 0
out = io.stdout.decode()
assert "a.txt" in out
assert "b.txt" in out
@pytest.mark.asyncio
async def test_tree_L_max_depth(workspace):
await workspace.ops.mkdir("/d1")
await workspace.ops.mkdir("/d1/sub")
await workspace.ops.mkdir("/d1/sub/deep")
await workspace.ops.write("/d1/sub/deep/file.txt", b"d")
io = await workspace.execute("tree -L 1 /d1", session_id="default")
assert io.exit_code == 0
out = io.stdout.decode()
assert "sub" in out
assert "deep" in out
assert "file.txt" not in out
@pytest.mark.asyncio
async def test_tree_d_dirs_only(workspace):
await workspace.ops.mkdir("/d1")
await workspace.ops.mkdir("/d1/sub")
await workspace.ops.write("/d1/file.txt", b"x")
io = await workspace.execute("tree -d /d1", session_id="default")
assert io.exit_code == 0
out = io.stdout.decode()
assert "sub" in out
assert "file.txt" not in out
@@ -0,0 +1,96 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage import DiskResource, MountMode, Workspace
@pytest.fixture
def workspace(tmp_path):
return Workspace({"/": DiskResource(root=str(tmp_path))},
mode=MountMode.WRITE)
@pytest.mark.asyncio
async def test_wc_default(workspace):
await workspace.ops.write("/f.txt", b"hello world\nfoo bar\n")
io = await workspace.execute("wc /f.txt", session_id="default")
assert io.exit_code == 0
parts = io.stdout.decode().rstrip("\n").split()
assert parts[0] == "2"
assert parts[1] == "4"
assert parts[2] == "20"
assert parts[3] == "/f.txt"
@pytest.mark.asyncio
async def test_wc_l(workspace):
await workspace.ops.write("/f.txt", b"a\nb\nc\n")
io = await workspace.execute("wc -l /f.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout.decode().split()[0] == "3"
@pytest.mark.asyncio
async def test_wc_w(workspace):
await workspace.ops.write("/f.txt", b"hello world\nfoo\n")
io = await workspace.execute("wc -w /f.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout.decode().split()[0] == "3"
@pytest.mark.asyncio
async def test_wc_c(workspace):
await workspace.ops.write("/f.txt", b"hello\n")
io = await workspace.execute("wc -c /f.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout.decode().split()[0] == "6"
@pytest.mark.asyncio
async def test_wc_m_multibyte(workspace):
await workspace.ops.write("/f.txt", "café".encode())
io = await workspace.execute("wc -m /f.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout.decode().split()[0] == "4"
@pytest.mark.asyncio
async def test_wc_L(workspace):
await workspace.ops.write("/f.txt", b"short\na much longer line\nmed\n")
io = await workspace.execute("wc -L /f.txt", session_id="default")
assert io.exit_code == 0
assert io.stdout.decode().split()[0] == str(len("a much longer line"))
@pytest.mark.asyncio
async def test_wc_empty_file(workspace):
await workspace.ops.write("/f.txt", b"")
io = await workspace.execute("wc /f.txt", session_id="default")
assert io.exit_code == 0
parts = io.stdout.decode().split()
assert parts[:3] == ["0", "0", "0"]
@pytest.mark.asyncio
async def test_wc_multi_file_emits_total(workspace):
await workspace.ops.write("/a.txt", b"hello\n")
await workspace.ops.write("/b.txt", b"world\nfoo\n")
io = await workspace.execute("wc /a.txt /b.txt", session_id="default")
assert io.exit_code == 0
lines = io.stdout.decode().splitlines()
assert any(line.endswith("/a.txt") for line in lines)
assert any(line.endswith("/b.txt") for line in lines)
assert lines[-1].endswith("total")
@@ -0,0 +1,103 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from unittest.mock import AsyncMock, patch
import pytest
from mirage.accessor.email import EmailAccessor
from mirage.cache.index.ram import RAMIndexCacheStore
from mirage.commands.builtin.email.find import find
from mirage.resource.email.config import EmailConfig
from mirage.types import PathSpec
def _spec(virtual: str) -> PathSpec:
return PathSpec(virtual=virtual,
directory=virtual,
resource_path=virtual.strip("/"))
def _accessor() -> EmailAccessor:
return EmailAccessor(
EmailConfig(imap_host="imap.test",
smtp_host="smtp.test",
username="u",
password="p"))
async def _run(paths, *texts: str, **flags) -> list[str]:
with patch("mirage.core.email.readdir.list_folders",
new_callable=AsyncMock,
return_value=["INBOX", "Sent"]):
stdout, _io = await find(_accessor(),
paths,
*texts,
index=RAMIndexCacheStore(),
**flags)
data = stdout if isinstance(stdout, bytes) else b""
return data.decode().splitlines()
@pytest.mark.asyncio
async def test_walk_lists_folders():
lines = await _run([_spec("/")], maxdepth="1")
assert "/INBOX" in lines
assert "/Sent" in lines
@pytest.mark.asyncio
async def test_path_pattern_is_honored():
lines = await _run([_spec("/")], maxdepth="1", path="*INBOX*")
assert lines == ["/INBOX"]
@pytest.mark.asyncio
async def test_size_is_honored_dirs_count_as_zero():
lines = await _run([_spec("/")], maxdepth="1", size="+0c")
assert lines == []
@pytest.mark.asyncio
async def test_name_only_folder_level_pushes_down_to_imap_search():
search = AsyncMock(return_value=[])
with patch("mirage.commands.builtin.email.find.search_messages", search):
stdout, _io = await find(_accessor(), [_spec("/INBOX")],
name="*report*",
index=RAMIndexCacheStore())
search.assert_awaited_once()
assert (stdout if isinstance(stdout, bytes) else b"") == b""
@pytest.mark.asyncio
async def test_name_with_size_falls_through_to_walk():
# Any predicate beyond -name must not be dropped by the server-side
# shortcut; the local walk applies all of them.
search = AsyncMock(return_value=[])
with patch("mirage.commands.builtin.email.find.search_messages", search), \
patch("mirage.core.email.readdir.list_folders",
new_callable=AsyncMock,
return_value=["INBOX"]), \
patch("mirage.core.email.stat.list_folders",
new_callable=AsyncMock,
return_value=["INBOX"]), \
patch("mirage.core.email.readdir.list_message_uids",
new_callable=AsyncMock,
return_value=[]):
stdout, _io = await find(_accessor(), [_spec("/INBOX")],
name="*report*",
size="+0c",
index=RAMIndexCacheStore())
search.assert_not_awaited()
assert (stdout if isinstance(stdout, bytes) else b"") == b""
@@ -0,0 +1,49 @@
import importlib
import sys
from types import SimpleNamespace
from unittest.mock import AsyncMock, patch
import pytest
from mirage.io.stream import materialize
from mirage.types import PathSpec
from mirage.utils.key_prefix import mount_key
sys.modules.setdefault(
"aioimaplib",
SimpleNamespace(IMAP4=object, IMAP4_SSL=object),
)
sys.modules.setdefault(
"aiosmtplib",
SimpleNamespace(SMTP=object, send=AsyncMock()),
)
_grep_server_side = importlib.import_module(
"mirage.commands.builtin.email.grep")._grep_server_side
@pytest.mark.asyncio
async def test_grep_server_side_count_uses_real_count():
accessor = SimpleNamespace(config=SimpleNamespace(max_messages=10))
pairs = [
("/email/INBOX/msg1.email.json", "foo foo\nfoo bar\nbaz\n"),
("/email/INBOX/msg2.email.json", "bar\nbaz\n"),
]
with patch(
"mirage.commands.builtin.email.grep.search_and_format",
new=AsyncMock(return_value=pairs),
):
stdout, io = await _grep_server_side(
accessor,
"INBOX",
"foo",
[
PathSpec(resource_path=mount_key("/email/INBOX", "/email"),
virtual="/email/INBOX",
directory="/email/INBOX")
],
c=True,
)
assert await materialize(stdout) == (b"/email/INBOX/msg1.email.json:2\n"
b"/email/INBOX/msg2.email.json:0\n")
assert io.exit_code == 0
@@ -0,0 +1,90 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import importlib
import sys
from types import SimpleNamespace
from unittest.mock import AsyncMock, patch
import pytest
from mirage.io.types import IOResult
from mirage.types import PathSpec
from mirage.utils.key_prefix import mount_key
sys.modules.setdefault(
"aioimaplib",
SimpleNamespace(IMAP4=object, IMAP4_SSL=object),
)
sys.modules.setdefault(
"aiosmtplib",
SimpleNamespace(SMTP=object, send=AsyncMock()),
)
rg = importlib.import_module("mirage.commands.builtin.email.rg").rg
def _path(s: str = "/email/INBOX") -> PathSpec:
return PathSpec(resource_path=mount_key(s, "/email"),
virtual=s,
directory=s)
@pytest.mark.asyncio
async def test_rg_multi_pattern_skips_imap_search():
# A newline-joined multi -e set must bypass the IMAP text search and
# still resolve globs before the generic runs (#347).
accessor = SimpleNamespace(config=SimpleNamespace(max_messages=10))
seen: dict[str, object] = {}
async def fake_resolve(_accessor, paths, index=None):
seen["resolved"] = [p.virtual for p in paths]
return paths
async def fake_generic(paths, _texts, _flags, **_kwargs):
seen["generic"] = [p.virtual for p in paths]
return b"", IOResult()
with patch(
"mirage.commands.builtin.email.rg.search_messages",
new=AsyncMock(side_effect=AssertionError("imap search ran")),
), patch(
"mirage.commands.builtin.email.rg.resolve_glob",
new=fake_resolve,
), patch(
"mirage.commands.builtin.email.rg.generic_rg",
new=fake_generic,
):
_, io = await rg(accessor, [_path()], e=["ada", "ben"])
assert io.exit_code == 0
assert seen["resolved"] == ["/email/INBOX"]
assert seen["generic"] == ["/email/INBOX"]
@pytest.mark.asyncio
async def test_rg_single_pattern_uses_imap_search():
accessor = SimpleNamespace(config=SimpleNamespace(max_messages=10))
search = AsyncMock(return_value=[])
with patch(
"mirage.commands.builtin.email.rg.search_messages",
new=search,
), patch(
"mirage.commands.builtin.email.rg.resolve_glob",
new=AsyncMock(side_effect=AssertionError("glob ran")),
):
_, io = await rg(accessor, [_path()], "ada")
assert io.exit_code == 1
search.assert_awaited_once()
@@ -0,0 +1,13 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
@@ -0,0 +1,67 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import json
from unittest.mock import AsyncMock, patch
import pytest
from mirage.accessor.gdocs import GDocsAccessor
from mirage.commands.builtin.gdocs.gws_docs_documents_batchUpdate import \
gws_docs_documents_batchUpdate
@pytest.fixture
def accessor():
return GDocsAccessor(config=None, token_manager=None)
@pytest.mark.asyncio
async def test_batch_update_success(accessor):
payload = json.dumps({
"requests": [{
"insertText": {
"location": {
"index": 1
},
"text": "Hi",
}
}]
})
params = json.dumps({"documentId": "doc1"})
api_response = {"documentId": "doc1", "replies": [{}]}
with patch(
"mirage.core.gdocs.update.google_post",
new_callable=AsyncMock,
return_value=api_response,
):
fn = gws_docs_documents_batchUpdate._registered_commands[0].fn
stream, io = await fn(
accessor,
[],
params=params,
json=payload,
)
chunks = []
async for chunk in stream:
chunks.append(chunk)
result = json.loads(b"".join(chunks))
assert result["documentId"] == "doc1"
@pytest.mark.asyncio
async def test_batch_update_missing_params(accessor):
fn = gws_docs_documents_batchUpdate._registered_commands[0].fn
with pytest.raises(ValueError, match="--params"):
await fn(accessor, [])
@@ -0,0 +1,13 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
@@ -0,0 +1,87 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage.accessor.gdrive import GDriveAccessor
from mirage.cache.index.config import IndexEntry
from mirage.cache.index.ram import RAMIndexCacheStore
from mirage.commands.builtin.gdrive._provision import file_read_provision
from mirage.provision.types import Precision
from mirage.types import PathSpec
from mirage.utils.key_prefix import mount_key
@pytest.fixture
def accessor():
return GDriveAccessor(config=None, token_manager=None)
@pytest.fixture
def index():
store = RAMIndexCacheStore()
return store
def _scope(path: str, prefix: str = "") -> PathSpec:
return PathSpec(resource_path=mount_key(path, prefix),
virtual=path,
directory=path.rsplit("/", 1)[0] or "/")
@pytest.mark.asyncio
async def test_plan_returns_read_ops(accessor, index):
await index.put(
"/test/file.txt",
IndexEntry(
id="file123",
name="file.txt",
resource_type="gdrive/file",
remote_time="2026-01-01T00:00:00Z",
vfs_name="file.txt",
size=500,
))
result = await file_read_provision(
accessor,
[_scope("/test/file.txt")],
"cat test/file.txt",
index=index,
)
assert result.read_ops == 1
assert result.precision == Precision.EXACT
@pytest.mark.asyncio
async def test_plan_no_paths(accessor, index):
result = await file_read_provision(accessor, [], "cat", index=index)
# Pathless invocations are stdin-driven: zero backend bytes, EXACT.
assert result.precision == Precision.EXACT
assert result.network_read_high == 0
assert result.network_read_low == 0
assert result.network_read_high == 0
@pytest.mark.asyncio
async def test_plan_missing_entry(accessor, index):
result = await file_read_provision(
accessor,
[_scope("/nonexistent/path.txt")],
"cat nonexistent/path.txt",
index=index,
)
assert result.network_read_low == 0
assert result.network_read_high == 0
# Unresolvable operands are still charged as a read-op floor and the
# estimate degrades to UNKNOWN instead of claiming a free read.
assert result.read_ops == 1
@@ -0,0 +1,149 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from unittest.mock import AsyncMock, patch
import pytest
from mirage.accessor.gdrive import GDriveAccessor
from mirage.cache.index.config import IndexEntry
from mirage.cache.index.ram import RAMIndexCacheStore
from mirage.core.gdrive.read import read
from mirage.core.google._client import TokenManager
from mirage.core.google.config import GoogleConfig
from mirage.types import PathSpec
@pytest.fixture
def config():
return GoogleConfig(
client_id="test-id",
client_secret="test-secret",
refresh_token="test-refresh",
)
@pytest.fixture
def token_manager(config):
mgr = TokenManager(config)
mgr._access_token = "fake-token"
mgr._expires_at = 9999999999
return mgr
@pytest.fixture
def accessor(config, token_manager):
return GDriveAccessor(config=config, token_manager=token_manager)
@pytest.fixture
def index():
return RAMIndexCacheStore()
@pytest.mark.asyncio
async def test_read_gdoc(accessor, index):
await index.put(
"/My Doc.gdoc.json",
IndexEntry(
id="doc1",
name="My Doc",
resource_type="gdrive/gdoc",
remote_time="2026-04-01T00:00:00.000Z",
vfs_name="My Doc.gdoc.json",
))
with patch(
"mirage.core.gdocs.read.google_get",
new_callable=AsyncMock,
return_value={"documentId": "doc1"},
):
result = await read(
accessor,
PathSpec(resource_path="My Doc.gdoc.json",
virtual="/My Doc.gdoc.json",
directory="/My Doc.gdoc.json"), index)
assert b"doc1" in result
@pytest.mark.asyncio
async def test_read_gsheet(accessor, index):
await index.put(
"/My Sheet.gsheet.json",
IndexEntry(
id="sheet1",
name="My Sheet",
resource_type="gdrive/gsheet",
remote_time="2026-04-01T00:00:00.000Z",
vfs_name="My Sheet.gsheet.json",
))
with patch(
"mirage.core.gsheets.read.google_get",
new_callable=AsyncMock,
return_value={"spreadsheetId": "sheet1"},
):
result = await read(
accessor,
PathSpec(resource_path="My Sheet.gsheet.json",
virtual="/My Sheet.gsheet.json",
directory="/My Sheet.gsheet.json"), index)
assert b"sheet1" in result
@pytest.mark.asyncio
async def test_read_gslide(accessor, index):
await index.put(
"/My Slides.gslide.json",
IndexEntry(
id="slide1",
name="My Slides",
resource_type="gdrive/gslide",
remote_time="2026-04-01T00:00:00.000Z",
vfs_name="My Slides.gslide.json",
))
with patch(
"mirage.core.gslides.read.google_get",
new_callable=AsyncMock,
return_value={"presentationId": "slide1"},
):
result = await read(
accessor,
PathSpec(resource_path="My Slides.gslide.json",
virtual="/My Slides.gslide.json",
directory="/My Slides.gslide.json"), index)
assert b"slide1" in result
@pytest.mark.asyncio
async def test_read_regular(accessor, index):
await index.put(
"/photo.png",
IndexEntry(
id="img1",
name="photo",
resource_type="gdrive/file",
remote_time="2026-04-01T00:00:00.000Z",
vfs_name="photo.png",
))
img_bytes = b"\x89PNG\r\n"
with patch(
"mirage.core.gdrive.read.download_file",
new_callable=AsyncMock,
return_value=img_bytes,
):
result = await read(
accessor,
PathSpec(resource_path="photo.png",
virtual="/photo.png",
directory="/photo.png"), index)
assert result == img_bytes
@@ -0,0 +1,161 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from unittest.mock import AsyncMock, patch
import pytest
from mirage.accessor.gdrive import GDriveAccessor
from mirage.cache.index.config import IndexEntry
from mirage.cache.index.ram import RAMIndexCacheStore
from mirage.commands.builtin.gdrive import COMMANDS
from mirage.core.google._client import TokenManager
from mirage.core.google.config import GoogleConfig
from mirage.io.stream import materialize
from mirage.types import PathSpec
from mirage.utils.key_prefix import mount_key
@pytest.fixture
def config():
return GoogleConfig(
client_id="test-id",
client_secret="test-secret",
refresh_token="test-refresh",
)
@pytest.fixture
def token_manager(config):
mgr = TokenManager(config)
mgr._access_token = "fake-token"
mgr._expires_at = 9999999999
return mgr
@pytest.fixture
def accessor(config, token_manager):
return GDriveAccessor(config=config, token_manager=token_manager)
@pytest.fixture
def index():
return RAMIndexCacheStore()
def _sed_command():
for cmd in COMMANDS:
for rc in cmd._registered_commands:
if rc.name == "sed":
return cmd
raise LookupError("sed not registered for gdrive")
sed = _sed_command()
def _scope(path: str, prefix: str = "") -> PathSpec:
return PathSpec(resource_path=mount_key(path, prefix),
virtual=path,
directory=path.rsplit("/", 1)[0] or "/")
@pytest.mark.asyncio
async def test_sed_simple_substitution(accessor, index):
await index.put(
"/test/file.txt",
IndexEntry(
id="file123",
name="file.txt",
resource_type="gdrive/file",
remote_time="2026-01-01T00:00:00Z",
vfs_name="file.txt",
size=100,
))
with patch(
"mirage.core.google.drive.google_get_bytes",
new_callable=AsyncMock,
return_value=b"hello world\nhello again\n",
):
result, io = await sed(
accessor,
[_scope("/test/file.txt")],
"s/hello/bye/g",
index=index,
)
data = await materialize(result)
assert data == b"bye world\nbye again\n"
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_sed_print_program(accessor, index):
await index.put(
"/test/file.txt",
IndexEntry(
id="file123",
name="file.txt",
resource_type="gdrive/file",
remote_time="2026-01-01T00:00:00Z",
vfs_name="file.txt",
size=100,
))
with patch(
"mirage.core.google.drive.google_get_bytes",
new_callable=AsyncMock,
return_value=b"one\ntwo\nthree\n",
):
result, io = await sed(
accessor,
[_scope("/test/file.txt")],
"2p",
n=True,
index=index,
)
data = await materialize(result)
assert data == b"two\n"
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_sed_stdin(accessor, index):
result, io = await sed(
accessor,
[],
"s/a/b/g",
stdin=b"banana\n",
index=index,
)
data = await materialize(result)
assert data == b"bbnbnb\n"
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_sed_in_place_rejected(accessor, index):
with pytest.raises(PermissionError,
match="-i not supported on this backend"):
await sed(
accessor,
[_scope("/test/file.txt")],
"s/a/b/",
i=True,
index=index,
)
@pytest.mark.asyncio
async def test_sed_missing_expression(accessor, index):
with pytest.raises(ValueError, match="sed: usage"):
await sed(accessor, [], index=index)
@@ -0,0 +1,38 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.commands.builtin.generic.crossmount.fanout.du import du_total
from mirage.commands.builtin.generic.crossmount.types import OperandRun
from mirage.io import IOResult
from mirage.types import PathSpec
def _scope(virtual: str) -> PathSpec:
return PathSpec(virtual=virtual,
directory=virtual[:virtual.rfind("/") + 1],
resource_path="",
resolved=True)
def _op(data: bytes, exit_code: int = 0) -> OperandRun:
return OperandRun(_scope("/a/x"), data, IOResult(exit_code=exit_code))
def testdu_total_strips_per_run_totals_and_sums():
runs = [
_op(b"5\t/a/sub\n5\ttotal\n"),
_op(b"3\t/b/c.txt\n3\ttotal\n"),
]
out = du_total(runs, human=False).decode()
assert out == "5\t/a/sub\n3\t/b/c.txt\n8\ttotal\n"
@@ -0,0 +1,24 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.commands.builtin.generic.crossmount.fanout.exit import \
combined_exit
def testcombined_exit_grep_match_wins_over_no_match():
assert combined_exit("grep", [0, 1]) == 0
assert combined_exit("grep", [1, 1]) == 1
assert combined_exit("grep", [2, 0]) == 2
assert combined_exit("rm", [0, 1]) == 1
assert combined_exit("md5", [0, 0]) == 0
@@ -0,0 +1,127 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
from mirage.commands.builtin.generic.crossmount.fanout import run_fanout
from mirage.commands.builtin.generic.crossmount.types import OperandRun
from mirage.io import IOResult
from mirage.io.stream import materialize
from mirage.types import PathSpec
def _scope(virtual: str) -> PathSpec:
return PathSpec(virtual=virtual,
directory=virtual[:virtual.rfind("/") + 1],
resource_path="",
resolved=True)
def _op(data: bytes, exit_code: int = 0) -> OperandRun:
return OperandRun(_scope("/a/x"), data, IOResult(exit_code=exit_code))
class FakeRunSingle:
"""Serves canned per-operand outputs and records every call."""
def __init__(self, outputs: dict[str, tuple[bytes, int]]):
self.outputs = outputs
self.calls: list[dict] = []
async def __call__(self,
cmd_name,
paths,
texts,
flag_kwargs,
stdin=None,
resolve_hint=None):
stdin_data = await materialize(stdin) if stdin is not None else None
self.calls.append(
dict(cmd=cmd_name,
paths=[p.virtual for p in paths],
flags=dict(flag_kwargs),
stdin=stdin_data))
data, code = self.outputs[paths[0].virtual]
io = IOResult(exit_code=code)
if code != 0:
io.stderr = f"{cmd_name}: {paths[0].virtual}: error\n".encode()
return data, io
def _run(coro):
return asyncio.run(coro)
def test_run_fanout_concats_in_operand_order_and_merges_exit():
rs = FakeRunSingle({
"/a/x": (b"hash1 /a/x\n", 0),
"/b/y": (b"hash2 /b/y\n", 0),
})
out, io = _run(
run_fanout("sha256sum",
[_scope("/a/x"), _scope("/b/y")], [], {}, rs))
assert _run(materialize(out)) == b"hash1 /a/x\nhash2 /b/y\n"
assert io.exit_code == 0
def test_run_fanout_forces_grep_filenames_unless_suppressed():
rs = FakeRunSingle({"/a/x": (b"", 1), "/b/y": (b"", 1)})
_run(run_fanout("grep", [_scope("/a/x"), _scope("/b/y")], ["pat"], {}, rs))
assert all(c["flags"].get("H") is True for c in rs.calls)
rs2 = FakeRunSingle({"/a/x": (b"", 1), "/b/y": (b"", 1)})
_run(
run_fanout("grep", [_scope("/a/x"), _scope("/b/y")], ["pat"],
{"h": True}, rs2))
assert all("H" not in c["flags"] for c in rs2.calls)
def test_run_fanout_forces_rg_filenames_unless_suppressed():
rs = FakeRunSingle({"/a/x": (b"", 1), "/b/y": (b"", 1)})
_run(run_fanout("rg", [_scope("/a/x"), _scope("/b/y")], ["pat"], {}, rs))
assert all(c["flags"].get("H") is True for c in rs.calls)
rs2 = FakeRunSingle({"/a/x": (b"", 1), "/b/y": (b"", 1)})
_run(
run_fanout("rg", [_scope("/a/x"), _scope("/b/y")], ["pat"],
{"args_I": True}, rs2))
assert all("H" not in c["flags"] for c in rs2.calls)
def test_run_fanout_forces_head_headers_and_blank_line_joins():
rs = FakeRunSingle({
"/a/x": (b"==> /a/x <==\n1\n", 0),
"/b/y": (b"==> /b/y <==\n2\n", 0),
})
out, _ = _run(
run_fanout("head", [_scope("/a/x"), _scope("/b/y")], [], {}, rs))
assert all(c["flags"].get("v") is True for c in rs.calls)
assert _run(materialize(out)) == b"==> /a/x <==\n1\n\n==> /b/y <==\n2\n"
def test_run_fanout_tee_refeeds_stdin_and_emits_it_once():
rs = FakeRunSingle({"/a/x": (b"hi\n", 0), "/b/y": (b"hi\n", 0)})
out, _ = _run(
run_fanout("tee", [_scope("/a/x"), _scope("/b/y")], [], {},
rs,
stdin=b"hi\n"))
assert _run(materialize(out)) == b"hi\n"
assert all(c["stdin"] == b"hi\n" for c in rs.calls)
def test_run_fanout_partial_failure_keeps_output_and_stderr():
rs = FakeRunSingle({"/a/x": (b"", 1), "/b/y": (b"ok\n", 0)})
out, io = _run(
run_fanout("stat", [_scope("/a/x"), _scope("/b/y")], [], {}, rs))
assert _run(materialize(out)) == b"ok\n"
assert io.exit_code == 1
assert b"/a/x" in (io.stderr or b"")
@@ -0,0 +1,58 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.commands.builtin.generic.crossmount.fanout.wc import combine_wc
from mirage.commands.builtin.generic.crossmount.types import OperandRun
from mirage.io import IOResult
from mirage.types import PathSpec
def _scope(virtual: str) -> PathSpec:
return PathSpec(virtual=virtual,
directory=virtual[:virtual.rfind("/") + 1],
resource_path="",
resolved=True)
def _op(data: bytes, exit_code: int = 0) -> OperandRun:
return OperandRun(_scope("/a/x"), data, IOResult(exit_code=exit_code))
def testcombine_wc_uses_one_global_width():
runs = [
_op(b"100 100 400 /a/big.txt\n"),
_op(b"5 5 20 /b/small.txt\n"),
]
out = combine_wc(runs, {}).decode()
assert out == ("100 100 400 /a/big.txt\n"
" 5 5 20 /b/small.txt\n"
"105 105 420 total\n")
def testcombine_wc_drops_per_run_totals_from_glob_operands():
runs = [
_op(b"2 /a/one.txt\n1 /a/two.txt\n3 total\n"),
_op(b"1 /b/three.txt\n"),
]
out = combine_wc(runs, {"args_l": True}).decode()
assert out == ("2 /a/one.txt\n"
"1 /a/two.txt\n"
"1 /b/three.txt\n"
"4 total\n")
def testcombine_wc_max_line_length_maxes_instead_of_summing():
runs = [_op(b"9 /a/x\n"), _op(b"4 /b/y\n")]
out = combine_wc(runs, {"L": True}).decode()
assert out.endswith("9 total\n")
@@ -0,0 +1,108 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
from mirage.commands.builtin.generic.crossmount.stream import run_stream
from mirage.io import IOResult
from mirage.io.stream import materialize
from mirage.types import PathSpec
def _scope(virtual: str) -> PathSpec:
return PathSpec(virtual=virtual,
directory=virtual[:virtual.rfind("/") + 1],
resource_path="",
resolved=True)
class FakeRunSingle:
"""Records run_single calls; serves per-path bytes for cat pushdowns."""
def __init__(self, files: dict[str, bytes]):
self.files = files
self.calls: list[dict] = []
self.final_stdin: bytes | None = None
async def __call__(self,
cmd_name,
paths,
texts,
flag_kwargs,
stdin=None,
resolve_hint=None):
self.calls.append(
dict(cmd=cmd_name,
paths=[p.virtual for p in paths],
texts=list(texts),
flags=dict(flag_kwargs),
resolve_hint=resolve_hint.virtual
if resolve_hint is not None else None))
if cmd_name == "cat" and paths:
data = self.files.get(paths[0].virtual)
if data is None:
err = f"cat: {paths[0].virtual}: No such file\n".encode()
return None, IOResult(exit_code=1, stderr=err)
return data, IOResult()
self.final_stdin = await materialize(stdin) if stdin is not None \
else None
return b"FINAL:" + (self.final_stdin or b""), IOResult()
def _run(coro):
return asyncio.run(coro)
def test_plain_cat_skips_the_final_run():
rs = FakeRunSingle({"/a/x": b"1\n", "/b/y": b"2\n"})
out, io = _run(
run_stream("cat", [_scope("/a/x"), _scope("/b/y")], [], {}, rs))
assert _run(materialize(out)) == b"1\n2\n"
assert io.exit_code == 0
assert [c["cmd"] for c in rs.calls] == ["cat", "cat"]
def test_flagged_command_runs_once_on_the_merged_stream():
rs = FakeRunSingle({"/a/x": b"1\n", "/b/y": b"2\n"})
out, io = _run(
run_stream("sort", [_scope("/a/x"), _scope("/b/y")], [], {"r": True},
rs))
assert _run(materialize(out)) == b"FINAL:1\n2\n"
assert io.exit_code == 0
final = rs.calls[-1]
assert final["cmd"] == "sort"
assert final["paths"] == []
assert final["flags"] == {"r": True}
assert final["resolve_hint"] == "/a/x"
assert rs.final_stdin == b"1\n2\n"
def test_cat_with_flags_reapplies_cat_on_the_merged_stream():
rs = FakeRunSingle({"/a/x": b"1\n", "/b/y": b"2\n"})
out, _ = _run(
run_stream("cat", [_scope("/a/x"), _scope("/b/y")], [], {"n": True},
rs))
assert _run(materialize(out)) == b"FINAL:1\n2\n"
assert rs.calls[-1]["cmd"] == "cat"
assert rs.calls[-1]["flags"] == {"n": True}
def test_failed_operand_is_skipped_and_fails_the_command():
rs = FakeRunSingle({"/b/y": b"2\n"})
out, io = _run(
run_stream("cat",
[_scope("/a/missing"), _scope("/b/y")], [], {}, rs))
assert _run(materialize(out)) == b"2\n"
assert io.exit_code == 1
assert b"No such file" in (io.stderr or b"")
@@ -0,0 +1,89 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.commands.builtin.generic.crossmount.constants import (
CROSS_MOUNT_COMMANDS, FANOUT_COMMANDS, RELAY_COMMANDS, STREAM_COMMANDS)
from mirage.commands.builtin.generic.crossmount.detect import (is_cross_mount,
strategy_for)
from mirage.commands.builtin.generic.crossmount.types import Strategy
from mirage.types import PathSpec
class _Mount:
def __init__(self, prefix: str):
self.prefix = prefix
class _Registry:
def __init__(self, prefixes: dict[str, str]):
self._prefixes = prefixes
def mount_for(self, virtual: str) -> _Mount:
for prefix in self._prefixes.values():
if virtual.startswith(prefix.rstrip("/") + "/"):
return _Mount(prefix)
raise ValueError(virtual)
def _scope(virtual: str) -> PathSpec:
return PathSpec(virtual=virtual,
directory=virtual[:virtual.rfind("/") + 1],
resource_path="",
resolved=True)
def test_sets_are_disjoint():
assert not STREAM_COMMANDS & FANOUT_COMMANDS
assert not STREAM_COMMANDS & RELAY_COMMANDS
assert not FANOUT_COMMANDS & RELAY_COMMANDS
assert CROSS_MOUNT_COMMANDS == (STREAM_COMMANDS | FANOUT_COMMANDS
| RELAY_COMMANDS)
def test_strategy_for_stream_commands():
for name in ("cat", "nl", "sort", "cut", "rev"):
assert strategy_for(name, {}) is Strategy.STREAM
def test_strategy_for_fanout_commands():
for name in ("grep", "wc", "sha256sum", "ls", "rm", "tee"):
assert strategy_for(name, {}) is Strategy.FANOUT
def test_strategy_for_relay_commands():
for name in ("cp", "mv", "diff", "cmp"):
assert strategy_for(name, {}) is Strategy.RELAY
def test_sed_default_streams_but_in_place_fans_out():
assert strategy_for("sed", {}) is Strategy.STREAM
assert strategy_for("sed", {"i": True}) is Strategy.FANOUT
def test_is_cross_mount_true_when_operands_span_mounts():
registry = _Registry({"a": "/a/", "b": "/b/"})
scopes = [_scope("/a/x.txt"), _scope("/b/y.txt")]
assert is_cross_mount("sort", scopes, registry)
assert is_cross_mount("sha256sum", scopes, registry)
def test_is_cross_mount_false_for_single_mount_or_unknown_command():
registry = _Registry({"a": "/a/", "b": "/b/"})
same = [_scope("/a/x.txt"), _scope("/a/y.txt")]
assert not is_cross_mount("sort", same, registry)
spanning = [_scope("/a/x.txt"), _scope("/b/y.txt")]
assert not is_cross_mount("uniq", spanning, registry)
assert not is_cross_mount("sort", spanning[:1], registry)
@@ -0,0 +1,423 @@
import pytest
from mirage.commands.builtin.generic.awk import awk
from mirage.commands.errors import UsageError
from mirage.types import PathSpec
def _spec(path: str) -> PathSpec:
return PathSpec(resource_path=(path).strip("/"),
virtual=path,
directory=path,
resolved=True)
def _make_backend(files: dict[str, bytes]):
async def read_bytes(accessor, path, index=None):
key = path.virtual if isinstance(path, PathSpec) else path
if key not in files:
raise FileNotFoundError(key)
return files[key]
async def read_stream(accessor, path, index=None):
assert isinstance(path, PathSpec)
key = path.virtual
if key not in files:
raise FileNotFoundError(key)
yield files[key]
return read_bytes, read_stream
async def _drain(stdout) -> bytes:
if stdout is None:
return b""
if isinstance(stdout, bytes):
return stdout
return b"".join([c async for c in stdout])
@pytest.mark.asyncio
async def test_awk_stdin_print_field():
rb, rs = _make_backend({})
output, _ = await awk(
[],
("{print $1}", ),
None,
read_bytes=rb,
read_stream=rs,
stdin=b"alpha beta\ngamma delta\n",
)
assert (await _drain(output)).decode() == "alpha\ngamma\n"
@pytest.mark.asyncio
async def test_awk_field_separator():
rb, rs = _make_backend({})
output, _ = await awk(
[],
("{print $2}", ),
{"F": ","},
read_bytes=rb,
read_stream=rs,
stdin=b"a,b,c\nd,e,f\n",
)
assert (await _drain(output)).decode() == "b\ne\n"
@pytest.mark.asyncio
async def test_awk_variable_assignment():
rb, rs = _make_backend({})
output, _ = await awk(
[],
("{print x}", ),
{"v": ["x=hello"]},
read_bytes=rb,
read_stream=rs,
stdin=b"line\n",
)
assert (await _drain(output)).decode() == "hello\n"
@pytest.mark.asyncio
async def test_awk_numeric_comparison():
rb, rs = _make_backend({})
output, _ = await awk(
[],
("$1 > 2 {print $1}", ),
None,
read_bytes=rb,
read_stream=rs,
stdin=b"1\n2\n3\n4\n",
)
assert (await _drain(output)).decode() == "3\n4\n"
@pytest.mark.asyncio
async def test_awk_regex_condition():
rb, rs = _make_backend({})
output, _ = await awk(
[],
("/foo/ {print $0}", ),
None,
read_bytes=rb,
read_stream=rs,
stdin=b"foo bar\nbaz\nfoobar\n",
)
assert (await _drain(output)).decode() == "foo bar\nfoobar\n"
@pytest.mark.asyncio
async def test_awk_end_block_accumulator():
"""sum += $1 with END {print sum} should emit total."""
rb, rs = _make_backend({})
output, _ = await awk(
[],
("{sum += $1} END {print sum}", ),
None,
read_bytes=rb,
read_stream=rs,
stdin=b"10\n20\n30\n",
)
assert (await _drain(output)).decode() == "60\n"
@pytest.mark.asyncio
async def test_awk_reads_from_file():
rb, rs = _make_backend({"/data.txt": b"hello world\n"})
output, io = await awk(
[_spec("/data.txt")],
("{print $2}", ),
None,
read_bytes=rb,
read_stream=rs,
)
assert (await _drain(output)).decode() == "world\n"
assert io.cache == ["/data.txt"]
@pytest.mark.asyncio
async def test_awk_program_file_overrides_inline():
rb, rs = _make_backend({
"/prog.awk": b"{print $1}\n",
"/data.txt": b"alpha beta\n",
})
output, _ = await awk(
[_spec("/data.txt")],
(),
{"f": _spec("/prog.awk")},
read_bytes=rb,
read_stream=rs,
)
assert (await _drain(output)).decode() == "alpha\n"
@pytest.mark.asyncio
async def test_awk_missing_program_raises_usage_error():
rb, rs = _make_backend({})
with pytest.raises(UsageError, match="usage"):
await awk([], (), None, read_bytes=rb, read_stream=rs)
@pytest.mark.asyncio
async def test_awk_default_fs_collapses_whitespace():
rb, rs = _make_backend({})
output, _ = await awk(
[],
("{print $2}", ),
None,
read_bytes=rb,
read_stream=rs,
stdin=b"a b\n\tx\t \ty\n",
)
assert (await _drain(output)).decode() == "b\ny\n"
@pytest.mark.asyncio
async def test_awk_explicit_single_space_fs_collapses_whitespace():
rb, rs = _make_backend({})
output, _ = await awk(
[],
("{print $2}", ),
{"F": " "},
read_bytes=rb,
read_stream=rs,
stdin=b"a b\n",
)
assert (await _drain(output)).decode() == "b\n"
@pytest.mark.asyncio
async def test_awk_empty_fs_splits_characters():
rb, rs = _make_backend({})
output, _ = await awk(
[],
("{print $2}", ),
{"F": ""},
read_bytes=rb,
read_stream=rs,
stdin=b"abc\n",
)
assert (await _drain(output)).decode() == "b\n"
@pytest.mark.asyncio
async def test_awk_processes_all_files_with_continuous_nr():
rb, rs = _make_backend({
"/a.txt": b"one\ntwo\n",
"/b.txt": b"three\n",
})
output, io = await awk(
[_spec("/a.txt"), _spec("/b.txt")],
("{print NR, $1}", ),
None,
read_bytes=rb,
read_stream=rs,
)
assert (await _drain(output)).decode() == "1 one\n2 two\n3 three\n"
assert io.cache == ["/a.txt", "/b.txt"]
@pytest.mark.asyncio
async def test_awk_multifile_no_trailing_newline_keeps_lines_separate():
rb, rs = _make_backend({
"/a.txt": b"one",
"/b.txt": b"two\n",
})
output, _ = await awk(
[_spec("/a.txt"), _spec("/b.txt")],
("{print NR, $1}", ),
None,
read_bytes=rb,
read_stream=rs,
)
assert (await _drain(output)).decode() == "1 one\n2 two\n"
@pytest.mark.asyncio
async def test_awk_repeated_v_assignments():
rb, rs = _make_backend({})
output, _ = await awk(
[],
("{print a, b}", ),
{"v": ["a=1", "b=2"]},
read_bytes=rb,
read_stream=rs,
stdin=b"line\n",
)
assert (await _drain(output)).decode() == "1 2\n"
@pytest.mark.asyncio
async def test_awk_v_value_containing_equals():
rb, rs = _make_backend({})
output, _ = await awk(
[],
("{print x}", ),
{"v": ["x=a=b"]},
read_bytes=rb,
read_stream=rs,
stdin=b"line\n",
)
assert (await _drain(output)).decode() == "a=b\n"
@pytest.mark.asyncio
async def test_awk_print_empty_string_emits_blank_line():
rb, rs = _make_backend({})
output, _ = await awk(
[],
('{print ""}', ),
None,
read_bytes=rb,
read_stream=rs,
stdin=b"one\ntwo\n",
)
assert (await _drain(output)).decode() == "\n\n"
@pytest.mark.asyncio
async def test_awk_action_without_print_emits_nothing():
rb, rs = _make_backend({})
output, _ = await awk(
[],
("{x += 1}", ),
None,
read_bytes=rb,
read_stream=rs,
stdin=b"one\ntwo\n",
)
assert (await _drain(output)).decode() == ""
@pytest.mark.asyncio
async def test_awk_brace_literal_in_print():
rb, rs = _make_backend({})
output, _ = await awk(
[],
('{print "}"}', ),
None,
read_bytes=rb,
read_stream=rs,
stdin=b"line\n",
)
assert (await _drain(output)).decode() == "}\n"
@pytest.mark.asyncio
async def test_awk_accumulator_non_numeric_coerces():
rb, rs = _make_backend({})
output, _ = await awk(
[],
("{sum += $1} END {print sum}", ),
None,
read_bytes=rb,
read_stream=rs,
stdin=b"3\nabc\n2.5x\n",
)
assert (await _drain(output)).decode() == "5.5\n"
@pytest.mark.asyncio
async def test_awk_program_file_missing_raises_usage_error():
rb, rs = _make_backend({"/data.txt": b"x\n"})
with pytest.raises(UsageError, match="No such file"):
await awk(
[_spec("/data.txt")],
(),
{"f": _spec("/missing.awk")},
read_bytes=rb,
read_stream=rs,
)
@pytest.mark.asyncio
async def test_awk_program_file_with_multiple_data_files():
rb, rs = _make_backend({
"/prog.awk": b"{print NR, $1}\n",
"/a.txt": b"one\n",
"/b.txt": b"two\n",
})
output, io = await awk(
[_spec("/a.txt"), _spec("/b.txt")],
(),
{"f": _spec("/prog.awk")},
read_bytes=rb,
read_stream=rs,
)
assert (await _drain(output)).decode() == "1 one\n2 two\n"
assert io.cache == ["/a.txt", "/b.txt"]
@pytest.mark.asyncio
async def test_awk_begin_end_resolve_v_variables():
rb, rs = _make_backend({})
output, _ = await awk(
[],
("BEGIN {print x} END {print x}", ),
{"v": ["x=hi"]},
read_bytes=rb,
read_stream=rs,
stdin=b"line\n",
)
assert (await _drain(output)).decode() == "hi\nhi\n"
@pytest.mark.asyncio
async def test_awk_duplicate_v_last_wins():
rb, rs = _make_backend({})
output, _ = await awk(
[],
("{print x}", ),
{"v": ["x=first", "x=second"]},
read_bytes=rb,
read_stream=rs,
stdin=b"line\n",
)
assert (await _drain(output)).decode() == "second\n"
@pytest.mark.asyncio
async def test_awk_begin_bare_print_emits_blank_line():
rb, rs = _make_backend({})
output, _ = await awk(
[],
('BEGIN {print} {print $1}', ),
None,
read_bytes=rb,
read_stream=rs,
stdin=b"a\n",
)
assert (await _drain(output)).decode() == "\na\n"
@pytest.mark.asyncio
async def test_awk_brace_literal_with_condition():
rb, rs = _make_backend({})
output, _ = await awk(
[],
('/x/ {print "}"}', ),
None,
read_bytes=rb,
read_stream=rs,
stdin=b"x\ny\n",
)
assert (await _drain(output)).decode() == "}\n"
@pytest.mark.asyncio
async def test_awk_repeated_program_files_concatenate():
rb, rs = _make_backend({
"/p1.awk": b"{sum += $1}\n",
"/p2.awk": b"END {print sum}\n",
"/nums.txt": b"1\n2\n3\n",
})
output, _ = await awk(
[_spec("/nums.txt")],
(),
{"f": [_spec("/p1.awk"), _spec("/p2.awk")]},
read_bytes=rb,
read_stream=rs,
)
assert (await _drain(output)).decode() == "6\n"
@@ -0,0 +1,178 @@
import pytest
from mirage.commands.builtin.generic.cat import cat
async def _drain(gen):
return b"".join([c async for c in gen])
@pytest.mark.asyncio
async def test_cat_passthrough_from_bytes():
out = await _drain(cat(b"hello\nworld\n"))
assert out == b"hello\nworld\n"
@pytest.mark.asyncio
async def test_cat_passthrough_from_stream_preserves_chunks():
async def src():
yield b"hel"
yield b"lo\nwo"
yield b"rld\n"
chunks = [c async for c in cat(src())]
assert chunks == [b"hel", b"lo\nwo", b"rld\n"]
@pytest.mark.asyncio
async def test_cat_number_lines():
out = await _drain(cat(b"a\nb\nc\n", number_lines=True))
assert out == b" 1\ta\n 2\tb\n 3\tc\n"
@pytest.mark.asyncio
async def test_cat_number_lines_across_chunk_boundaries():
async def src():
yield b"a\nb"
yield b"\nc\n"
out = await _drain(cat(src(), number_lines=True))
assert out == b" 1\ta\n 2\tb\n 3\tc\n"
@pytest.mark.asyncio
async def test_cat_show_ends():
out = await _drain(cat(b"a\nb\n", show_ends=True))
assert out == b"a$\nb$\n"
@pytest.mark.asyncio
async def test_cat_squeeze_blank():
out = await _drain(cat(b"a\n\n\n\nb\n", squeeze_blank=True))
assert out == b"a\n\nb\n"
@pytest.mark.asyncio
async def test_cat_no_trailing_newline_preserved():
"""POSIX: `printf "x" | cat -n` emits no trailing newline."""
out = await _drain(cat(b"hello", number_lines=True))
assert out == b" 1\thello"
@pytest.mark.asyncio
async def test_cat_passthrough_no_trailing_newline():
"""No-flag cat must not add a trailing newline."""
out = await _drain(cat(b"hello"))
assert out == b"hello"
@pytest.mark.asyncio
async def test_cat_number_lines_multidigit_alignment():
"""POSIX format is `%6d\\t` — width 6, right-justified. Old MIRAGE used
a 5-space prefix which broke alignment for line numbers >= 10."""
body = b"".join(f"line{i}\n".encode() for i in range(1, 13))
out = await _drain(cat(body, number_lines=True))
lines = out.split(b"\n")
assert lines[0] == b" 1\tline1"
assert lines[8] == b" 9\tline9"
assert lines[9] == b" 10\tline10"
assert lines[11] == b" 12\tline12"
@pytest.mark.asyncio
async def test_cat_show_ends_no_trailing_newline():
"""cat -E on input without trailing newline must not emit final `$\\n`."""
out = await _drain(cat(b"hello", show_ends=True))
assert out == b"hello"
@pytest.mark.asyncio
async def test_cat_combined_n_E_s():
"""Combined flags: number all kept lines, show ends, squeeze blanks."""
out = await _drain(
cat(b"a\n\n\n\nb\n",
number_lines=True,
show_ends=True,
squeeze_blank=True))
# Kept lines: "a", "" (first blank, kept), "b". Numbered 1, 2, 3.
assert out == b" 1\ta$\n 2\t$\n 3\tb$\n"
@pytest.mark.asyncio
async def test_cat_empty_input_emits_nothing_with_flags():
out = await _drain(cat(b"", number_lines=True))
assert out == b""
@pytest.mark.asyncio
async def test_cat_only_newlines():
"""Three blank lines numbered 1, 2, 3."""
out = await _drain(cat(b"\n\n\n", number_lines=True))
assert out == b" 1\t\n 2\t\n 3\t\n"
@pytest.mark.asyncio
async def test_cat_squeeze_blank_preserves_first_blank():
out = await _drain(cat(b"\n\n\nx\n", squeeze_blank=True))
assert out == b"\nx\n"
@pytest.mark.asyncio
async def test_cat_empty_input_passthrough():
out = await _drain(cat(b""))
assert out == b""
@pytest.mark.asyncio
async def test_cat_binary_passthrough_full_byte_range():
"""Full 0..255 byte range must pass through unchanged (no flags)."""
data = bytes(range(256))
out = await _drain(cat(data))
assert out == data
@pytest.mark.asyncio
async def test_cat_binary_with_show_ends_marks_only_newlines():
"""show_ends should only mark 0x0A bytes, not other binary bytes."""
data = b"\x00\x01\n\x02\x03\n"
out = await _drain(cat(data, show_ends=True))
assert out == b"\x00\x01$\n\x02\x03$\n"
@pytest.mark.asyncio
async def test_cat_number_lines_chunked_one_byte_at_a_time():
"""Worst-case chunking: every byte its own chunk. Result must match
unbuffered input exactly."""
async def src():
for byte in b"a\nbb\nccc\n":
yield bytes([byte])
out = await _drain(cat(src(), number_lines=True))
assert out == b" 1\ta\n 2\tbb\n 3\tccc\n"
@pytest.mark.asyncio
async def test_cat_show_tabs_renders_caret_i():
out = b"".join([c async for c in cat(b"a\tb\nx\n", show_tabs=True)])
assert out == b"a^Ib\nx\n"
@pytest.mark.asyncio
async def test_cat_show_all_combines_tabs_and_ends():
out = b"".join([
c async for c in cat(b"a\tb\nx\n",
show_tabs=True,
show_ends=True,
show_nonprinting=True)
])
assert out == b"a^Ib$\nx$\n"
@pytest.mark.asyncio
async def test_cat_show_nonprinting_caret_and_meta_notation():
out = b"".join(
[c async for c in cat(b"\x01\x7f\xff\n", show_nonprinting=True)])
assert out == b"^A^?M-^?\n"
@@ -0,0 +1,211 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage.commands.builtin.generic.cp import cp
from mirage.types import FileStat, FileType, PathSpec
def _spec(path: str) -> PathSpec:
return PathSpec(virtual=path,
directory=path,
resource_path=path.strip("/"))
def _key(p) -> str:
return (p.virtual if isinstance(p, PathSpec) else p).rstrip("/")
def _make_backend(files: dict[str, bytes], dirs: set[str]):
async def stat(p, index=None) -> FileStat:
k = _key(p)
if k in dirs:
return FileStat(name=k.rsplit("/", 1)[-1], type=FileType.DIRECTORY)
if k in files:
return FileStat(name=k.rsplit("/", 1)[-1], type=FileType.TEXT)
raise FileNotFoundError(k)
async def copy(src, dst) -> None:
files[_key(dst)] = files[_key(src)]
async def find(p, type=None) -> list[str]:
base = _key(p) + "/"
return sorted(k for k in files if k.startswith(base))
return stat, copy, find
async def _run(files, dirs, paths, **kw):
stat, copy, find = _make_backend(files, dirs)
return await cp([_spec(p) for p in paths],
copy=copy,
find=find,
find_type="f",
stat=stat,
recursive=kw.get("recursive", False),
n=kw.get("n", False),
v=kw.get("v", False))
@pytest.mark.asyncio
async def test_single_source_to_new_path():
files = {"/a.txt": b"AAA"}
await _run(files, set(), ["/a.txt", "/copy.txt"])
assert files["/copy.txt"] == b"AAA"
@pytest.mark.asyncio
async def test_single_source_into_directory():
files = {"/a.txt": b"AAA", "/d/keep": b"K"}
await _run(files, {"/d"}, ["/a.txt", "/d"])
assert files["/d/a.txt"] == b"AAA"
assert files["/a.txt"] == b"AAA"
@pytest.mark.asyncio
async def test_multiple_sources_into_directory():
files = {"/a.txt": b"AAA", "/b.txt": b"BBB", "/d/keep": b"K"}
await _run(files, {"/d"}, ["/a.txt", "/b.txt", "/d"])
assert files["/d/a.txt"] == b"AAA"
assert files["/d/b.txt"] == b"BBB"
@pytest.mark.asyncio
async def test_multiple_sources_nondir_raises():
files = {"/a.txt": b"AAA", "/b.txt": b"BBB", "/dst.txt": b"DST"}
with pytest.raises(NotADirectoryError):
await _run(files, set(), ["/a.txt", "/b.txt", "/dst.txt"])
assert files["/dst.txt"] == b"DST"
@pytest.mark.asyncio
async def test_no_clobber_skips_existing():
files = {"/a.txt": b"NEW", "/d/a.txt": b"OLD"}
await _run(files, {"/d"}, ["/a.txt", "/d"], n=True)
assert files["/d/a.txt"] == b"OLD"
@pytest.mark.asyncio
async def test_no_clobber_duplicate_basenames_first_wins():
files = {"/x/a.txt": b"FIRST", "/y/a.txt": b"SECOND", "/d/keep": b"K"}
await _run(files, {"/d"}, ["/x/a.txt", "/y/a.txt", "/d"], n=True)
assert files["/d/a.txt"] == b"FIRST"
@pytest.mark.asyncio
async def test_duplicate_basenames_without_n_last_wins():
files = {"/x/a.txt": b"FIRST", "/y/a.txt": b"SECOND", "/d/keep": b"K"}
await _run(files, {"/d"}, ["/x/a.txt", "/y/a.txt", "/d"])
assert files["/d/a.txt"] == b"SECOND"
@pytest.mark.asyncio
async def test_recursive_into_directory():
files = {"/src/x.txt": b"X", "/src/sub/y.txt": b"Y"}
await _run(files, {"/src"}, ["/src", "/dst"], recursive=True)
assert files["/dst/x.txt"] == b"X"
assert files["/dst/sub/y.txt"] == b"Y"
@pytest.mark.asyncio
async def test_verbose_emits_arrow_lines():
files = {"/a.txt": b"AAA"}
out, _ = await _run(files, set(), ["/a.txt", "/copy.txt"], v=True)
assert out == b"'/a.txt' -> '/copy.txt'\n"
@pytest.mark.asyncio
async def test_records_writes_by_strip_prefix():
files = {"/a.txt": b"AAA", "/b.txt": b"BBB", "/d/keep": b"K"}
_, io = await _run(files, {"/d"}, ["/a.txt", "/b.txt", "/d"])
assert set(io.writes) == {"/d/a.txt", "/d/b.txt"}
@pytest.mark.asyncio
async def test_missing_source_reports_cannot_stat_and_continues():
files = {"/b.txt": b"BBB", "/d/keep": b"K"}
_, io = await _run(files, {"/d"}, ["/missing.txt", "/b.txt", "/d"])
assert io.exit_code == 1
assert b"cp: cannot stat '/missing.txt'" in io.stderr
assert files["/d/b.txt"] == b"BBB"
@pytest.mark.asyncio
async def test_same_file_errors_and_preserves_content():
files = {"/a.txt": b"AAA"}
_, io = await _run(files, set(), ["/a.txt", "/a.txt"])
assert io.exit_code == 1
assert b"'/a.txt' and '/a.txt' are the same file" in io.stderr
assert files["/a.txt"] == b"AAA"
@pytest.mark.asyncio
async def test_same_file_via_directory_target_errors():
files = {"/d/a.txt": b"AAA", "/d/keep": b"K"}
_, io = await _run(files, {"/d"}, ["/d/a.txt", "/d"])
assert io.exit_code == 1
assert b"are the same file" in io.stderr
assert files["/d/a.txt"] == b"AAA"
@pytest.mark.asyncio
async def test_recursive_into_own_subtree_refused():
files = {"/d/a.txt": b"AAA"}
_, io = await _run(files, {"/d"}, ["/d", "/d"], recursive=True)
assert io.exit_code == 1
assert b"cp: cannot copy a directory, '/d', into itself" in io.stderr
assert set(files) == {"/d/a.txt"}
@pytest.mark.asyncio
async def test_recursive_into_nested_subtree_refused():
files = {"/d/a.txt": b"AAA"}
_, io = await _run(files, {"/d", "/d/sub"}, ["/d", "/d/sub"],
recursive=True)
assert io.exit_code == 1
assert b"into itself" in io.stderr
assert set(files) == {"/d/a.txt"}
@pytest.mark.asyncio
async def test_primitive_copy_records_source_reads():
files = {"/a.txt": b"AAA"}
stat, _, _ = _make_backend(files, set())
async def read_bytes(p) -> bytes:
return files[_key(p)]
async def write(p, data: bytes) -> None:
files[_key(p)] = data
_, io = await cp([_spec("/a.txt"), _spec("/copy.txt")],
stat=stat,
read_bytes=read_bytes,
write=write,
recursive=False,
n=False,
v=False)
assert files["/copy.txt"] == b"AAA"
assert io.reads == {"/a.txt": b"AAA"}
assert io.cache == ["/a.txt"]
@pytest.mark.asyncio
async def test_native_copy_records_no_reads():
files = {"/a.txt": b"AAA"}
_, io = await _run(files, set(), ["/a.txt", "/copy.txt"])
assert io.reads == {}
assert io.cache == []
@@ -0,0 +1,273 @@
import pytest
from mirage.commands.builtin.generic.du import _depth, du, du_multi
from mirage.types import PathSpec
def _spec(path: str) -> PathSpec:
return PathSpec(virtual=path,
directory=path,
resource_path=path.strip("/"))
def _make_list_backend(tree: dict[str, int]):
"""Build (compute_total, compute_all) where compute_all returns a flat
list (NOT a (list, total) tuple), matching the backend `du_all` contract
consumed by `du_multi`. Directory targets include a trailing
(dirpath, total) entry, mirroring the real backends."""
async def compute_total(p: PathSpec) -> int:
prefix = p.virtual.rstrip("/") + "/"
return sum(size for path, size in tree.items()
if path == p.virtual or path.startswith(prefix))
async def compute_all(p: PathSpec) -> list[tuple[str, int]]:
prefix = p.virtual.rstrip("/") + "/"
entries: list[tuple[str, int]] = []
total = 0
for path, size in sorted(tree.items()):
if path == p.virtual or path.startswith(prefix):
entries.append((path, size))
total += size
if p.virtual not in dict(entries):
entries.append((p.virtual, total))
return entries
return compute_total, compute_all
def _make_backend(tree: dict[str, int]):
"""Build (compute_total, compute_all) callables over an in-memory tree.
`tree` maps every file path → size. Directory totals are inferred by
walking the tree on demand, matching `du_all` semantics in the real
backends (returns flat file entries + recursive total).
"""
async def compute_total(p: PathSpec) -> int:
prefix = p.virtual.rstrip("/") + "/"
total = 0
for path, size in tree.items():
if path == p.virtual or path.startswith(prefix):
total += size
return total
async def compute_all(p: PathSpec, ) -> tuple[list[tuple[str, int]], int]:
prefix = p.virtual.rstrip("/") + "/"
entries: list[tuple[str, int]] = []
total = 0
for path, size in sorted(tree.items()):
if path == p.virtual or path.startswith(prefix):
entries.append((path, size))
total += size
return entries, total
return compute_total, compute_all
@pytest.mark.asyncio
async def test_du_single_file_default():
compute_total, compute_all = _make_backend({"/f.txt": 5})
out = await du([_spec("/f.txt")],
compute_total=compute_total,
compute_all=compute_all)
assert out == "5\t/f.txt\n"
@pytest.mark.asyncio
async def test_du_directory_collapses_to_root_when_not_a():
"""POSIX `du` default behavior: one line per dir, files not shown."""
tree = {"/dir/a.txt": 3, "/dir/b.txt": 2}
compute_total, compute_all = _make_backend(tree)
out = await du([_spec("/dir")],
compute_total=compute_total,
compute_all=compute_all)
assert out == "5\t/dir\n"
@pytest.mark.asyncio
async def test_du_a_lists_all_files():
tree = {"/dir/a.txt": 3, "/dir/b.txt": 2}
compute_total, compute_all = _make_backend(tree)
out = await du([_spec("/dir")],
compute_total=compute_total,
compute_all=compute_all,
a=True)
lines = out.splitlines()
assert "3\t/dir/a.txt" in lines
assert "2\t/dir/b.txt" in lines
@pytest.mark.asyncio
async def test_du_s_summary_single_line():
tree = {"/dir/a.txt": 3, "/dir/sub/b.txt": 2}
compute_total, compute_all = _make_backend(tree)
out = await du([_spec("/dir")],
compute_total=compute_total,
compute_all=compute_all,
s=True)
assert out == "5\t/dir\n"
@pytest.mark.asyncio
async def test_du_max_depth_zero_in_a_mode_drops_everything_below():
"""`-a --max-depth 0` keeps only entries at depth 0 (the root). Since
file entries are at depth ≥ 1, no entries remain → falls back to
single-line root output."""
tree = {"/dir/a.txt": 3, "/dir/sub/b.txt": 2}
compute_total, compute_all = _make_backend(tree)
out = await du([_spec("/dir")],
compute_total=compute_total,
compute_all=compute_all,
a=True,
max_depth=0)
assert out == "5\t/dir\n"
@pytest.mark.asyncio
async def test_du_max_depth_one_keeps_direct_children():
tree = {"/dir/a.txt": 3, "/dir/sub/b.txt": 2}
compute_total, compute_all = _make_backend(tree)
out = await du([_spec("/dir")],
compute_total=compute_total,
compute_all=compute_all,
a=True,
max_depth=1)
lines = out.splitlines()
assert "3\t/dir/a.txt" in lines
assert not any("sub/b.txt" in ln for ln in lines)
@pytest.mark.asyncio
async def test_du_c_appends_total_line():
tree = {"/dir/a.txt": 3, "/dir/b.txt": 2}
compute_total, compute_all = _make_backend(tree)
out = await du([_spec("/dir")],
compute_total=compute_total,
compute_all=compute_all,
c=True)
assert out.splitlines()[-1] == "5\ttotal"
@pytest.mark.asyncio
async def test_du_h_human_readable_size():
tree = {"/big.txt": 2048}
compute_total, compute_all = _make_backend(tree)
out = await du([_spec("/big.txt")],
compute_total=compute_total,
compute_all=compute_all,
h=True)
assert "/big.txt" in out
assert out.split("\t")[0].endswith("K")
@pytest.mark.asyncio
async def test_du_multi_path_independent_outputs():
tree = {"/a.txt": 3, "/b.txt": 7}
compute_total, compute_all = _make_backend(tree)
out = await du([_spec("/a.txt"), _spec("/b.txt")],
compute_total=compute_total,
compute_all=compute_all)
assert out == "3\t/a.txt\n7\t/b.txt\n"
@pytest.mark.asyncio
async def test_du_multi_path_with_c_grand_total():
tree = {"/a.txt": 3, "/b.txt": 7}
compute_total, compute_all = _make_backend(tree)
out = await du([_spec("/a.txt"), _spec("/b.txt")],
compute_total=compute_total,
compute_all=compute_all,
c=True)
assert out.splitlines()[-1] == "10\ttotal"
@pytest.mark.asyncio
async def test_du_multi_path_with_h_and_c_sums_raw_bytes():
"""-c with -h should still sum the raw byte counts correctly. (Previous
string-parse implementation got 0 here because '1.0K' wasn't int())."""
tree = {"/a.txt": 1024, "/b.txt": 1024}
compute_total, compute_all = _make_backend(tree)
out = await du([_spec("/a.txt"), _spec("/b.txt")],
compute_total=compute_total,
compute_all=compute_all,
h=True,
c=True)
total_line = out.splitlines()[-1]
assert total_line.endswith("\ttotal")
assert total_line.split("\t")[0].endswith("K")
@pytest.mark.asyncio
async def test_du_empty_target_returns_zero_with_path():
compute_total, compute_all = _make_backend({})
out = await du([_spec("/nothing")],
compute_total=compute_total,
compute_all=compute_all)
assert out == "0\t/nothing\n"
@pytest.mark.asyncio
async def test_du_multi_independent_outputs_bytes():
compute_total, compute_all = _make_list_backend({"/a.txt": 3, "/b.txt": 7})
out = await du_multi([_spec("/a.txt"), _spec("/b.txt")],
compute_total=compute_total,
compute_all=compute_all)
assert out == b"3\t/a.txt\n7\t/b.txt\n"
@pytest.mark.asyncio
async def test_du_multi_c_grand_total_sums_all_paths():
compute_total, compute_all = _make_list_backend({"/a.txt": 3, "/b.txt": 7})
out = await du_multi([_spec("/a.txt"), _spec("/b.txt")],
compute_total=compute_total,
compute_all=compute_all,
c=True)
assert out.decode().splitlines()[-1] == "10\ttotal"
@pytest.mark.asyncio
async def test_du_multi_directory_collapses_when_not_a():
tree = {"/dir/a.txt": 3, "/dir/b.txt": 2}
compute_total, compute_all = _make_list_backend(tree)
out = await du_multi([_spec("/dir")],
compute_total=compute_total,
compute_all=compute_all)
assert out == b"5\t/dir\n"
@pytest.mark.asyncio
async def test_du_multi_a_lists_files():
tree = {"/dir/a.txt": 3, "/dir/b.txt": 2}
compute_total, compute_all = _make_list_backend(tree)
out = await du_multi([_spec("/dir")],
compute_total=compute_total,
compute_all=compute_all,
a=True)
lines = out.decode().splitlines()
assert "3\t/dir/a.txt" in lines
assert "2\t/dir/b.txt" in lines
@pytest.mark.asyncio
async def test_du_multi_compute_all_none_emits_total_only():
"""Walk-only backends (gdrive, github) pass compute_all=None: each path
yields a single total line; -c still sums across paths."""
compute_total, _ = _make_list_backend({"/x/a": 4, "/x/b": 6, "/y/c": 5})
out = await du_multi([_spec("/x"), _spec("/y")],
compute_total=compute_total,
compute_all=None,
c=True)
assert out.decode().splitlines() == ["10\t/x", "5\t/y", "15\ttotal"]
def test_depth_helper_root_is_zero():
assert _depth("/dir", "/dir") == 0
def test_depth_helper_direct_child_is_one():
assert _depth("/dir/a.txt", "/dir") == 1
def test_depth_helper_nested():
assert _depth("/dir/sub/b.txt", "/dir") == 2
@@ -0,0 +1,79 @@
import pytest
from mirage.commands.builtin.generic.file import file_cmd
from mirage.types import FileStat, FileType, PathSpec
def _spec(path: str) -> PathSpec:
return PathSpec(virtual=path,
directory=path,
resource_path=path.strip("/"))
def _make_backend(files: dict[str, tuple[bytes, FileType]], dirs: set[str]):
async def stat_fn(accessor: object, p: PathSpec) -> FileStat:
if p.virtual in dirs:
return FileStat(name=p.virtual, type=FileType.DIRECTORY, size=0)
if p.virtual in files:
data, ftype = files[p.virtual]
return FileStat(name=p.virtual, type=ftype, size=len(data))
raise FileNotFoundError(p.virtual)
async def read_bytes(accessor: object, p: PathSpec) -> bytes:
return files[p.virtual][0]
return stat_fn, read_bytes
@pytest.mark.asyncio
async def test_file_single_text():
stat_fn, read_bytes = _make_backend(
{"/a.txt": (b"hello world\n", FileType.TEXT)}, set())
out, io = await file_cmd([_spec("/a.txt")],
read_bytes=read_bytes,
stat_fn=stat_fn)
assert out == b"/a.txt: text\n"
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_file_multiple_paths_one_line_each():
stat_fn, read_bytes = _make_backend(
{
"/a.txt": (b"hello\n", FileType.TEXT),
"/b.json": (b'{"k": 1}\n', FileType.JSON),
}, set())
out, _io = await file_cmd(
[_spec("/a.txt"), _spec("/b.json")],
read_bytes=read_bytes,
stat_fn=stat_fn)
lines = out.decode().splitlines()
assert lines == ["/a.txt: text", "/b.json: json"]
@pytest.mark.asyncio
async def test_file_directory_reported_without_read():
stat_fn, read_bytes = _make_backend({}, {"/d"})
out, _io = await file_cmd([_spec("/d")],
read_bytes=read_bytes,
stat_fn=stat_fn)
assert out == b"/d: directory\n"
@pytest.mark.asyncio
async def test_file_brief_drops_path_prefix():
stat_fn, read_bytes = _make_backend(
{"/a.txt": (b"hello\n", FileType.TEXT)}, set())
out, _io = await file_cmd([_spec("/a.txt")],
read_bytes=read_bytes,
stat_fn=stat_fn,
b=True)
assert out == b"text\n"
@pytest.mark.asyncio
async def test_file_missing_operand_raises():
stat_fn, read_bytes = _make_backend({}, set())
with pytest.raises(ValueError):
await file_cmd([], read_bytes=read_bytes, stat_fn=stat_fn)
@@ -0,0 +1,432 @@
import asyncio
from dataclasses import asdict
from datetime import datetime, timezone
from unittest.mock import AsyncMock
import pytest
from mirage.commands.builtin.find_eval import Name, Not, Or
from mirage.commands.builtin.generic.find import (FindArgs, apply_mount_prefix,
apply_mtime_filter,
parse_find_args, walk_find)
from mirage.commands.errors import FindParseError
from mirage.resource.ram import RAMResource
from mirage.types import FileStat, FileType, FindType, MountMode, PathSpec
from mirage.workspace import Workspace
def _defaults() -> dict:
return asdict(FindArgs())
def test_parse_find_args_empty_returns_defaults():
args = parse_find_args(())
assert asdict(args) == _defaults()
def test_parse_find_args_name_passthrough():
args = parse_find_args((), name="*.txt")
assert args.name == "*.txt"
assert args.or_names is None
def test_parse_find_args_iname_and_path():
args = parse_find_args((), iname="HELLO.*", path="**/sub/*")
assert args.iname == "HELLO.*"
assert args.path_pattern == "**/sub/*"
def test_parse_find_args_maxdepth_mindepth_str_to_int():
args = parse_find_args((), maxdepth="3", mindepth="1")
assert args.maxdepth == 3
assert args.mindepth == 1
def test_parse_find_args_size_plus_lower_bound():
args = parse_find_args((), size="+500c")
assert args.min_size == 501
assert args.max_size is None
def test_parse_find_args_size_minus_upper_bound():
args = parse_find_args((), size="-1k")
assert args.min_size is None
assert args.max_size == 0
def test_parse_find_args_size_exact():
args = parse_find_args((), size="1k")
assert args.min_size == 1
assert args.max_size == 1024
def test_parse_find_args_mtime_minus_recent():
"""`-mtime -1` means modified within last 1 day."""
args = parse_find_args((), mtime="-1")
assert args.mtime_min is not None
assert args.mtime_max is None
def test_parse_find_args_mtime_plus_old():
args = parse_find_args((), mtime="+7")
assert args.mtime_min is None
assert args.mtime_max is not None
def test_parse_find_args_type_canonicalized_to_findtype_enum():
"""Known POSIX `-type` values become FindType members."""
assert parse_find_args((), type="d").type is FindType.DIRECTORY
assert parse_find_args((), type="f").type is FindType.FILE
def test_parse_find_args_unknown_type_left_as_string():
"""Non-POSIX types pass through verbatim (allows custom backend types)."""
assert parse_find_args((), type="symlink").type == "symlink"
def test_parse_find_args_unknown_predicate_raises():
with pytest.raises(FindParseError,
match="find: unknown predicate '-bogus'"):
parse_find_args(("-bogus", ))
def test_parse_find_args_negation_builds_not_tree():
args = parse_find_args(("-not", "-name", "*.pyc"))
assert args.tree == Not(Name("*.pyc"))
def test_parse_find_args_or_builds_or_tree():
args = parse_find_args(("-name", "*.txt", "-o", "-name", "*.py"))
assert args.tree == Or([Name("*.txt"), Name("*.py")])
def test_parse_find_args_or_names_none_when_only_one_name():
"""If only `name` is set with no `-or -name` clauses, or_names is None."""
args = parse_find_args((), name="*.txt")
assert args.or_names is None
@pytest.mark.asyncio
async def test_apply_mtime_filter_skips_when_no_window():
out = await apply_mtime_filter(
["/a.txt"],
mtime_min=None,
mtime_max=None,
stat=_unreached_stat,
)
assert out == ["/a.txt"]
@pytest.mark.asyncio
async def test_apply_mtime_filter_keeps_within_window():
now = datetime.now(tz=timezone.utc)
iso = now.isoformat()
async def stat(_spec: PathSpec) -> FileStat:
return FileStat(name="a.txt", size=1, modified=iso, type=FileType.TEXT)
out = await apply_mtime_filter(
["/a.txt"],
mtime_min=now.timestamp() - 60,
mtime_max=now.timestamp() + 60,
stat=stat,
)
assert out == ["/a.txt"]
@pytest.mark.asyncio
async def test_apply_mtime_filter_drops_outside_window():
old = datetime(2020, 1, 1, tzinfo=timezone.utc)
async def stat(_spec: PathSpec) -> FileStat:
return FileStat(name="a.txt",
size=1,
modified=old.isoformat(),
type=FileType.TEXT)
out = await apply_mtime_filter(
["/a.txt"],
mtime_min=datetime(2025, 1, 1, tzinfo=timezone.utc).timestamp(),
mtime_max=None,
stat=stat,
)
assert out == []
@pytest.mark.asyncio
async def test_apply_mtime_filter_drops_entries_with_no_modified_time():
async def stat(_spec: PathSpec) -> FileStat:
return FileStat(name="a.txt",
size=1,
modified=None,
type=FileType.TEXT)
out = await apply_mtime_filter(
["/a.txt"],
mtime_min=1.0,
mtime_max=None,
stat=stat,
)
assert out == []
@pytest.mark.asyncio
async def test_apply_mtime_filter_silently_skips_stat_errors():
async def stat(_spec: PathSpec) -> FileStat:
raise FileNotFoundError("gone")
out = await apply_mtime_filter(
["/a.txt", "/b.txt"],
mtime_min=1.0,
mtime_max=None,
stat=stat,
)
assert out == []
def test_apply_mount_prefix_noop_when_empty():
assert apply_mount_prefix(["/a.txt"], "") == ["/a.txt"]
def test_apply_mount_prefix_prepends():
assert apply_mount_prefix(["/a.txt", "/dir/b.txt"],
"/mnt") == ["/mnt/a.txt", "/mnt/dir/b.txt"]
def test_apply_mount_prefix_strips_leading_slash_from_entries():
assert apply_mount_prefix(["a.txt"], "/mnt") == ["/mnt/a.txt"]
async def _unreached_stat(_spec: PathSpec) -> FileStat:
raise AssertionError("stat should not be called when no mtime window set")
def _root_spec() -> PathSpec:
return PathSpec(resource_path="",
virtual="/",
directory="/",
resolved=False)
@pytest.mark.asyncio
async def test_walk_find_tolerates_not_found_readdir():
readdir = AsyncMock(side_effect=FileNotFoundError("/"))
stat = AsyncMock(side_effect=FileNotFoundError("/"))
results = await walk_find(_root_spec(),
readdir=readdir,
stat=stat,
is_dir_name=lambda c: None,
index=None,
args=FindArgs())
assert results == []
@pytest.mark.asyncio
async def test_walk_find_emits_start_path_at_depth_zero():
readdir = AsyncMock(return_value=["/child.txt"])
stat = AsyncMock(return_value=FileStat(name="/", type=FileType.DIRECTORY))
results = await walk_find(_root_spec(),
readdir=readdir,
stat=stat,
is_dir_name=lambda c: False,
index=None,
args=FindArgs(maxdepth=0))
assert results == ["/"]
readdir.assert_not_awaited()
@pytest.mark.asyncio
async def test_walk_find_propagates_non_not_found_readdir_errors():
readdir = AsyncMock(side_effect=ValueError("bad page token"))
stat = AsyncMock()
with pytest.raises(ValueError, match="bad page token"):
await walk_find(_root_spec(),
readdir=readdir,
stat=stat,
is_dir_name=lambda c: None,
index=None,
args=FindArgs())
@pytest.mark.asyncio
async def test_walk_find_stat_fallback_treats_not_found_as_file():
readdir = AsyncMock(return_value=["/mystery"])
stat = AsyncMock(side_effect=FileNotFoundError("/mystery"))
results = await walk_find(_root_spec(),
readdir=readdir,
stat=stat,
is_dir_name=lambda c: None,
index=None,
args=FindArgs(type=FindType.FILE))
assert results == ["/mystery"]
@pytest.mark.asyncio
async def test_walk_find_empty_matches_empty_files_and_dirs():
async def readdir(spec: PathSpec, _index):
table = {
"/": ["/empty.txt", "/full.txt", "/empty-dir", "/full-dir"],
"/empty-dir": [],
"/full-dir": ["/full-dir/a.txt"],
}
return table[spec.virtual]
async def stat(spec: PathSpec, _index):
stats = {
"/": FileStat(name="/", type=FileType.DIRECTORY),
"/empty.txt": FileStat(name="empty.txt",
size=0,
type=FileType.TEXT),
"/full.txt": FileStat(name="full.txt", size=1, type=FileType.TEXT),
"/empty-dir": FileStat(name="empty-dir", type=FileType.DIRECTORY),
"/full-dir": FileStat(name="full-dir", type=FileType.DIRECTORY),
"/full-dir/a.txt": FileStat(name="a.txt",
size=1,
type=FileType.TEXT),
}
return stats[spec.virtual]
results = await walk_find(_root_spec(),
readdir=readdir,
stat=stat,
is_dir_name=lambda c: c.endswith("dir"),
index=None,
args=parse_find_args(("-empty", )))
assert results == ["/empty-dir", "/empty.txt"]
@pytest.mark.asyncio
async def test_walk_find_not_negates_predicate():
readdir = AsyncMock(return_value=["/a.txt", "/b.md"])
async def stat(spec: PathSpec, _index):
if spec.virtual == "/":
return FileStat(name="/", type=FileType.DIRECTORY)
return FileStat(name=spec.virtual.rsplit("/", 1)[-1],
size=1,
type=FileType.TEXT)
results = await walk_find(_root_spec(),
readdir=readdir,
stat=stat,
is_dir_name=lambda c: False,
index=None,
args=parse_find_args(("-not", "-name", "*.txt")))
assert results == ["/", "/b.md"]
@pytest.mark.asyncio
async def test_walk_find_stat_fallback_propagates_other_errors():
readdir = AsyncMock(return_value=["/mystery"])
stat = AsyncMock(side_effect=ValueError("rate limited"))
with pytest.raises(ValueError, match="rate limited"):
await walk_find(_root_spec(),
readdir=readdir,
stat=stat,
is_dir_name=lambda c: None,
index=None,
args=FindArgs())
@pytest.mark.asyncio
async def test_walk_find_size_filter_drops_not_found_entries():
readdir = AsyncMock(return_value=["/a.json"])
stat = AsyncMock(side_effect=FileNotFoundError("/a.json"))
results = await walk_find(_root_spec(),
readdir=readdir,
stat=stat,
is_dir_name=lambda c: False,
index=None,
args=FindArgs(min_size=1))
assert results == []
@pytest.mark.asyncio
async def test_walk_find_size_filter_propagates_other_stat_errors():
readdir = AsyncMock(return_value=["/a.json"])
stat = AsyncMock(side_effect=ValueError("rate limited"))
with pytest.raises(ValueError, match="rate limited"):
await walk_find(_root_spec(),
readdir=readdir,
stat=stat,
is_dir_name=lambda c: False,
index=None,
args=FindArgs(min_size=1))
@pytest.mark.parametrize("kwargs,flag,value", [
({
"maxdepth": "abc"
}, "-maxdepth", "abc"),
({
"mindepth": "xx"
}, "-mindepth", "xx"),
({
"size": ""
}, "-size", ""),
({
"size": "abc"
}, "-size", "abc"),
({
"mtime": "abc"
}, "-mtime", "abc"),
])
def test_parse_find_args_invalid_numeric_raises_find_parse_error(
kwargs, flag, value):
with pytest.raises(FindParseError) as exc:
parse_find_args((), **kwargs)
assert str(exc.value) == f"find: invalid argument '{value}' to '{flag}'"
@pytest.mark.parametrize("expr", [
"-maxdepth abc",
"-mindepth xx",
"-size ''",
"-size abc",
"-mtime abc",
])
def test_find_invalid_numeric_arg_exits_one_with_clean_stderr(expr):
async def _go() -> tuple[int, str]:
ws = Workspace({"/": RAMResource()}, mode=MountMode.WRITE)
ws.create_session("s")
r = await ws.execute(f"find / {expr}", session_id="s")
return r.exit_code, await r.stderr_str()
code, stderr = asyncio.run(_go())
assert code == 1
assert stderr.startswith("find: invalid argument ")
assert stderr.endswith("\n")
# ── Issue #312 parse-level regression tests ────────────────
def test_parse_find_args_start_path_included():
args = parse_find_args(())
assert args.maxdepth is None
def test_parse_find_args_maxdepth_zero():
args = parse_find_args(("-maxdepth", "0"))
assert args.maxdepth == 0
def test_parse_find_args_empty_predicate():
args = parse_find_args(("-empty", ))
assert args.empty is True
def test_parse_find_args_not_negation():
args = parse_find_args(("-not", "-name", "*.txt"))
assert isinstance(args.tree, Not)
assert isinstance(args.tree.kid, Name)
assert args.tree.kid.pattern == "*.txt"
def test_parse_find_args_bogus_predicate_raises():
with pytest.raises(FindParseError, match="unknown predicate"):
parse_find_args(("-boguspredicate", ))
@@ -0,0 +1,583 @@
import pytest
from mirage.commands.builtin.generic.grep import GrepFlags, grep, parse_flags
from mirage.commands.spec.types import FlagView
from mirage.types import FileStat, FileType, PathSpec
from mirage.utils.key_prefix import mount_key
def _spec(path: str) -> PathSpec:
return PathSpec(resource_path=(path).strip("/"),
virtual=path,
directory=path,
resolved=True)
def _make_backend(files: dict[str, bytes], dirs: set[str] | None = None):
"""Build (readdir, stat, read_bytes, read_stream) callables over a
simple in-memory file tree. `dirs` is the set of directory paths;
intermediate dirs are inferred from file paths if not specified."""
inferred_dirs = set(dirs) if dirs is not None else set()
for f in files:
parts = f.split("/")
for i in range(1, len(parts)):
d = "/".join(parts[:i]) or "/"
inferred_dirs.add(d)
inferred_dirs.add("/")
async def readdir(accessor, path, index=None):
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
p = spec.virtual.rstrip("/") or "/"
if p not in inferred_dirs:
raise FileNotFoundError(p)
prefix = p + "/" if p != "/" else "/"
children: set[str] = set()
for f in files:
if f.startswith(prefix):
rest = f[len(prefix):]
child = rest.split("/")[0]
children.add(prefix + child)
for d in inferred_dirs:
if d == p:
continue
if d.startswith(prefix):
rest = d[len(prefix):]
child = rest.split("/")[0]
children.add(prefix + child)
return sorted(children)
async def stat(accessor, path, index=None):
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
p = spec.virtual
if p in files:
return FileStat(name=p.rsplit("/", 1)[-1] or p,
size=len(files[p]),
type=FileType.TEXT)
if p.rstrip("/") in inferred_dirs or p in inferred_dirs:
return FileStat(name=p.rsplit("/", 1)[-1] or "/",
type=FileType.DIRECTORY)
raise FileNotFoundError(p)
async def read_bytes(accessor, path, index=None):
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
if spec.virtual not in files:
raise FileNotFoundError(spec.virtual)
return files[spec.virtual]
async def read_stream(accessor, path, index=None):
data = await read_bytes(accessor, path)
yield data
return readdir, stat, read_bytes, read_stream
def _drain(stdout):
if isinstance(stdout, bytes):
return stdout
return b"".join([c for c in stdout])
async def _drain_async(stdout):
if stdout is None:
return b""
if isinstance(stdout, bytes):
return stdout
chunks = [chunk async for chunk in stdout]
return b"".join(chunks)
@pytest.mark.asyncio
async def test_grep_stdin_basic():
readdir, stat, rb, rs = _make_backend({})
output, io = await grep(
[],
["apple"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
stdin=b"apple\nbanana\napricot\n",
)
decoded = (await _drain_async(output)).decode()
assert "apple" in decoded
assert "apricot" not in decoded
@pytest.mark.asyncio
async def test_grep_file_basic():
readdir, stat, rb, rs = _make_backend({
"/a.txt":
b"apple\nbanana\napricot\n",
})
output, io = await grep(
[_spec("/a.txt")],
["ap"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
)
decoded = (await _drain_async(output)).decode()
assert "apple" in decoded
assert "apricot" in decoded
assert "banana" not in decoded
@pytest.mark.asyncio
async def test_grep_single_dir_operand_warns():
readdir, stat, rb, rs = _make_backend({"/d/a.txt": b"apple\n"})
output, io = await grep(
[_spec("/d")],
["ap"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
)
assert await _drain_async(output) == b""
assert io.exit_code == 1
assert io.stderr == b"grep: /d: Is a directory\n"
@pytest.mark.asyncio
async def test_grep_ignore_case():
readdir, stat, rb, rs = _make_backend({"/a.txt": b"Apple\nBANANA\n"})
output, _ = await grep(
[_spec("/a.txt")],
["apple"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={"i": True},
)
decoded = (await _drain_async(output)).decode()
assert "Apple" in decoded
@pytest.mark.asyncio
async def test_grep_invert():
readdir, stat, rb, rs = _make_backend(
{"/a.txt": b"apple\nbanana\ncherry\n"})
output, _ = await grep(
[_spec("/a.txt")],
["banana"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={"v": True},
)
decoded = (await _drain_async(output)).decode()
assert "apple" in decoded
assert "cherry" in decoded
assert "banana" not in decoded
@pytest.mark.asyncio
async def test_grep_count_only():
readdir, stat, rb, rs = _make_backend(
{"/a.txt": b"apple\nbanana\napricot\n"})
output, _ = await grep(
[_spec("/a.txt")],
["ap"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={"c": True},
)
decoded = (await _drain_async(output)).decode().strip()
assert decoded == "2"
@pytest.mark.asyncio
async def test_grep_no_match_returns_exit_1():
readdir, stat, rb, rs = _make_backend({"/a.txt": b"hello\nworld\n"})
output, io = await grep(
[_spec("/a.txt")],
["zzz"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
)
drained = await _drain_async(output)
assert drained == b""
assert io.exit_code == 1
@pytest.mark.asyncio
async def test_grep_recursive_finds_files_in_subdirs():
readdir, stat, rb, rs = _make_backend({
"/dir/a.txt": b"apple\n",
"/dir/sub/b.txt": b"apricot\n",
})
output, io = await grep(
[_spec("/dir")],
["ap"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={"r": True},
)
decoded = (await _drain_async(output)).decode()
assert "apple" in decoded
assert "apricot" in decoded
@pytest.mark.asyncio
async def test_grep_recursive_single_file_prefixes_filename():
readdir, stat, rb, rs = _make_backend({
"/log.txt":
b"one\nerror here\ntwo\nerror again\n",
})
output, _ = await grep(
[_spec("/log.txt")],
["error"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={
"r": True,
"n": True
},
)
decoded = (await _drain_async(output)).decode()
assert decoded == "/log.txt:2:error here\n/log.txt:4:error again\n"
@pytest.mark.asyncio
async def test_grep_files_only_lists_matching_files():
readdir, stat, rb, rs = _make_backend({
"/dir/a.txt": b"apple\n",
"/dir/b.txt": b"zebra\n",
})
output, _ = await grep(
[_spec("/dir")],
["apple"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={
"r": True,
"args_l": True
},
)
decoded = (await _drain_async(output)).decode()
assert "/dir/a.txt" in decoded
assert "/dir/b.txt" not in decoded
def _make_prefixed_backend(files: dict[str, bytes], mount_prefix: str):
"""Backend that mimics real s3/disk/gdrive readdir: entries returned
are already prepended with ``mount_prefix``. Also raises if ``index``
is not threaded through (gdrive-style)."""
full_files = {mount_prefix + k: v for k, v in files.items()}
inferred_dirs: set[str] = {mount_prefix or "/"}
for f in full_files:
parts = f.split("/")
for i in range(1, len(parts)):
d = "/".join(parts[:i]) or "/"
inferred_dirs.add(d)
def _full(p: str) -> str:
if mount_prefix and not p.startswith(mount_prefix):
return mount_prefix + p
return p
async def readdir(accessor, path, index=None):
if index is None:
raise FileNotFoundError("index required")
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
p = _full(spec.virtual).rstrip("/") or "/"
if p not in inferred_dirs:
raise FileNotFoundError(p)
prefix = p + "/" if p != "/" else "/"
children: set[str] = set()
for f in full_files:
if f.startswith(prefix):
child = prefix + f[len(prefix):].split("/")[0]
children.add(child)
for d in inferred_dirs:
if d == p or not d.startswith(prefix):
continue
child = prefix + d[len(prefix):].split("/")[0]
children.add(child)
return sorted(children)
async def stat(accessor, path, index=None):
if index is None:
raise FileNotFoundError("index required")
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
p = _full(spec.virtual)
if p in full_files:
return FileStat(name=p.rsplit("/", 1)[-1],
size=len(full_files[p]),
type=FileType.TEXT)
if p.rstrip("/") in inferred_dirs:
return FileStat(name=p.rsplit("/", 1)[-1] or "/",
type=FileType.DIRECTORY)
raise FileNotFoundError(p)
async def read_bytes(accessor, path, index=None):
if index is None:
raise FileNotFoundError("index required")
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
p = _full(spec.virtual)
if p not in full_files:
raise FileNotFoundError(p)
return full_files[p]
return readdir, stat, read_bytes
@pytest.mark.asyncio
async def test_grep_recursive_files_only_mount_prefix():
readdir, stat, rb = _make_prefixed_backend(
{
"/dir/a.txt": b"apple\n",
"/dir/b.txt": b"zebra\n",
},
mount_prefix="/s3",
)
p = PathSpec(resource_path=mount_key("/dir", "/s3"),
virtual="/dir",
directory="/dir",
resolved=True)
output, _ = await grep(
[p],
["apple"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=None,
flags={
"r": True,
"args_l": True
},
index=object(),
)
decoded = (await _drain_async(output)).decode().strip()
assert decoded == "/s3/dir/a.txt"
assert "/s3/s3" not in decoded
@pytest.mark.asyncio
async def test_grep_single_file_threads_index():
readdir, stat, rb = _make_prefixed_backend(
{"/dir/a.txt": b"apple\n"},
mount_prefix="/gd",
)
p = PathSpec(resource_path=mount_key("/dir/a.txt", "/gd"),
virtual="/dir/a.txt",
directory="/dir/a.txt",
resolved=True)
output, _ = await grep(
[p],
["apple"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=None,
index=object(),
)
decoded = (await _drain_async(output)).decode()
assert "apple" in decoded
def test_parse_flags_defaults_and_context_fallback():
f = parse_flags(FlagView({}), never_match=False)
assert f == GrepFlags(ignore_case=False,
invert=False,
line_numbers=False,
count_only=False,
files_only=False,
whole_word=False,
fixed_string=False,
only_matching=False,
quiet=False,
recursive=False,
with_filename=False,
no_filename=False,
max_count=None,
after_context=0,
before_context=0)
f = parse_flags(FlagView({"A": "2", "C": "5"}), never_match=False)
assert f.after_context == 2
assert f.before_context == 5
def test_parse_flags_never_match_suppresses_fixed_string():
f = parse_flags(FlagView({"F": True}), never_match=True)
assert f.fixed_string is False
def test_grep_flags_struct_rejects_typos():
f = parse_flags(FlagView({"i": True}), never_match=False)
with pytest.raises(AttributeError):
_ = f.ignorecase
# FrozenInstanceError subclasses AttributeError
with pytest.raises(AttributeError):
f.ignore_case = False
@pytest.mark.asyncio
async def test_grep_count_only_no_match_exit_1():
readdir, stat, rb, rs = _make_backend({"/a.txt": b"hello\nworld\n"})
output, io = await grep(
[_spec("/a.txt")],
["zzz"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={"c": True},
)
decoded = (await _drain_async(output)).decode().strip()
assert decoded == "0"
assert io.exit_code == 1
@pytest.mark.asyncio
async def test_grep_stdin_count_only_no_match_exit_1():
readdir, stat, rb, rs = _make_backend({})
output, io = await grep(
[],
["zzz"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
stdin=b"hello\nworld\n",
flags={"c": True},
)
decoded = (await _drain_async(output)).decode().strip()
assert decoded == "0"
assert io.exit_code == 1
@pytest.mark.asyncio
async def test_grep_count_only_multi_file_no_match_exit_1():
readdir, stat, rb, rs = _make_backend({
"/a.txt": b"hello\n",
"/b.txt": b"world\n",
})
output, io = await grep(
[_spec("/a.txt"), _spec("/b.txt")],
["zzz"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={"c": True},
)
decoded = (await _drain_async(output)).decode()
assert decoded.splitlines() == ["/a.txt:0", "/b.txt:0"]
assert io.exit_code == 1
@pytest.mark.asyncio
async def test_grep_count_only_multi_file_match_exit_0():
readdir, stat, rb, rs = _make_backend({
"/a.txt": b"hello\n",
"/b.txt": b"world\n",
})
output, io = await grep(
[_spec("/a.txt"), _spec("/b.txt")],
["hello"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={"c": True},
)
decoded = (await _drain_async(output)).decode()
assert decoded.splitlines() == ["/a.txt:1", "/b.txt:0"]
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_grep_recursive_count_only_no_match_exit_1():
readdir, stat, rb, rs = _make_backend({"/d/a.txt": b"hello\n"})
output, io = await grep(
[_spec("/d")],
["zzz"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={
"r": True,
"c": True
},
)
decoded = (await _drain_async(output)).decode()
assert decoded.splitlines() == ["/d/a.txt:0"]
assert io.exit_code == 1
@pytest.mark.asyncio
async def test_grep_single_file_dash_h_prefixes_filename():
readdir, stat, rb, rs = _make_backend({"/a.txt": b"apple\nbanana\n"})
output, io = await grep(
[_spec("/a.txt")],
["apple"],
{"H": True},
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
)
decoded = (await _drain_async(output)).decode()
assert decoded == "/a.txt:apple\n"
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_grep_single_file_dash_h_count_prefixes_filename():
readdir, stat, rb, rs = _make_backend({"/a.txt": b"apple\nbanana\n"})
output, io = await grep(
[_spec("/a.txt")],
["a"],
{
"H": True,
"c": True
},
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
)
decoded = (await _drain_async(output)).decode()
assert decoded == "/a.txt:2\n"
@pytest.mark.asyncio
async def test_grep_multi_file_no_filename_suppresses_prefix():
readdir, stat, rb, rs = _make_backend({
"/a.txt": b"apple\n",
"/b.txt": b"apricot\n",
})
output, io = await grep(
[_spec("/a.txt"), _spec("/b.txt")],
["ap"],
{"h": True},
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
)
decoded = (await _drain_async(output)).decode()
assert decoded == "apple\napricot\n"
@@ -0,0 +1,140 @@
import pytest
from mirage.commands.builtin.generic.head import head
async def _drain(gen):
return b"".join([c async for c in gen])
@pytest.mark.asyncio
async def test_head_first_c_bytes_fast_path():
out = await _drain(head(b"hello world", c=5))
assert out == b"hello"
@pytest.mark.asyncio
async def test_head_first_c_bytes_across_chunks():
async def src():
yield b"hel"
yield b"lo wor"
yield b"ld"
out = await _drain(head(src(), c=8))
assert out == b"hello wo"
@pytest.mark.asyncio
async def test_head_first_c_bytes_shorter_than_total():
out = await _drain(head(b"hi", c=100))
assert out == b"hi"
@pytest.mark.asyncio
async def test_head_first_c_zero_emits_nothing():
out = await _drain(head(b"hello", c=0))
assert out == b""
@pytest.mark.asyncio
async def test_head_first_n_lines_from_bytes():
out = await _drain(head(b"a\nb\nc\nd\n", n=2))
assert out == b"a\nb\n"
@pytest.mark.asyncio
async def test_head_first_n_lines_from_stream():
async def src():
yield b"a\nb"
yield b"\nc\nd\n"
out = await _drain(head(src(), n=2))
assert out == b"a\nb\n"
@pytest.mark.asyncio
async def test_head_default_n_is_10():
body = b"".join(f"line{i}\n".encode() for i in range(1, 15))
out = await _drain(head(body))
lines = out.decode().splitlines()
assert len(lines) == 10
assert lines[0] == "line1"
assert lines[9] == "line10"
@pytest.mark.asyncio
async def test_head_n_zero_emits_nothing():
out = await _drain(head(b"a\nb\n", n=0))
assert out == b""
@pytest.mark.asyncio
async def test_head_negative_n_excludes_last_n_lines():
out = await _drain(head(b"a\nb\nc\nd\n", n=-1))
assert out == b"a\nb\nc\n"
@pytest.mark.asyncio
async def test_head_negative_n_equal_to_total_emits_nothing():
out = await _drain(head(b"a\nb\n", n=-2))
assert out == b""
@pytest.mark.asyncio
async def test_head_stops_consuming_stream_after_n_lines():
consumed = []
async def src():
for line in [b"a\n", b"b\n", b"c\n", b"d\n", b"e\n"]:
consumed.append(line)
yield line
chunks = [c async for c in head(src(), n=2)]
assert b"".join(chunks) == b"a\nb\n"
assert consumed == [b"a\n", b"b\n"]
@pytest.mark.asyncio
async def test_head_no_trailing_newline_in_last_line():
out = await _drain(head(b"a\nb\nno-end", n=3))
assert out == b"a\nb\nno-end"
@pytest.mark.asyncio
async def test_head_n_larger_than_available():
out = await _drain(head(b"a\nb\n", n=10))
assert out == b"a\nb\n"
@pytest.mark.asyncio
async def test_head_n_one():
out = await _drain(head(b"line1\nline2\nline3\n", n=1))
assert out == b"line1\n"
@pytest.mark.asyncio
async def test_head_empty_input():
out = await _drain(head(b""))
assert out == b""
@pytest.mark.asyncio
async def test_head_empty_input_with_c():
out = await _drain(head(b"", c=10))
assert out == b""
@pytest.mark.asyncio
async def test_head_single_line_no_newline_default_n():
"""head on a single line without trailing newline returns it as-is."""
out = await _drain(head(b"hello"))
assert out == b"hello"
@pytest.mark.asyncio
async def test_head_c_negative_is_all_but_last_abs():
"""GNU head: -c -N emits all but the last N bytes."""
out = await _drain(head(b"hello", c=-3))
assert out == b"he"
@@ -0,0 +1,76 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage.commands.builtin.generic.head import head_multi
from mirage.types import PathSpec
from mirage.utils.key_prefix import mount_key
def _paths(*names: str) -> list[PathSpec]:
return [
PathSpec(resource_path=mount_key(n, ""),
virtual=n,
directory="/d",
resolved=True) for n in names
]
async def _collect(gen) -> bytes:
out = b""
async for chunk in gen:
out += chunk
return out
@pytest.mark.asyncio
async def test_head_multi_bytes_reader_no_headers():
data = {"/a": b"a1\na2\na3\n", "/b": b"b1\nb2\n"}
async def read(accessor, p, index):
return data[p.virtual]
out = await _collect(
head_multi(_paths("/a", "/b"), read=read, n=1, show_headers=False))
assert out == b"a1\nb1\n"
@pytest.mark.asyncio
async def test_head_multi_with_headers():
data = {"/a": b"a1\na2\n", "/b": b"b1\nb2\n"}
async def read(accessor, p, index):
return data[p.virtual]
out = await _collect(
head_multi(_paths("/a", "/b"), read=read, n=1, show_headers=True))
assert out == b"==> /a <==\na1\n\n==> /b <==\nb1\n"
@pytest.mark.asyncio
async def test_head_multi_stream_reader():
chunks = {"/a": [b"a1\n", b"a2\n"], "/b": [b"b1\n"]}
def read(accessor, p, index):
async def gen():
for ch in chunks[p.virtual]:
yield ch
return gen()
out = await _collect(
head_multi(_paths("/a", "/b"), read=read, n=5, show_headers=True))
assert out == b"==> /a <==\na1\na2\n\n==> /b <==\nb1\n"
@@ -0,0 +1,322 @@
from datetime import datetime, timezone
import pytest
from mirage.commands.builtin.generic.ls import (format_simple, get_extension,
ls, walk)
from mirage.types import FileStat, FileType, LsSortBy, PathSpec
def _spec(path: str) -> PathSpec:
return PathSpec(virtual=path,
directory=path,
resource_path=path.strip("/"))
def _make_fs_backend(tree: dict[str, FileStat]):
"""Build (readdir, stat) callables over an in-memory entry tree.
`tree` maps absolute path → FileStat. Directories are entries whose
type == FileType.DIRECTORY. readdir lists direct children of the path.
"""
async def stat(p: PathSpec, index=None) -> FileStat:
if p.virtual not in tree:
raise FileNotFoundError(p.virtual)
return tree[p.virtual]
async def readdir(p: PathSpec, _index=None) -> list[str]:
if p.virtual not in tree:
raise FileNotFoundError(p.virtual)
if tree[p.virtual].type != FileType.DIRECTORY:
raise ValueError(f"not a directory: {p.virtual}")
prefix = p.virtual.rstrip("/") + "/"
children: list[str] = []
for key in tree:
if key == p.virtual:
continue
if key.startswith(prefix):
remainder = key[len(prefix):]
if "/" not in remainder:
children.append(key)
return sorted(children)
return readdir, stat
def _file(name: str, size: int = 0, modified: str | None = None) -> FileStat:
return FileStat(name=name,
size=size,
modified=modified,
type=FileType.TEXT)
def _dir(name: str) -> FileStat:
return FileStat(name=name, size=None, type=FileType.DIRECTORY)
def test_get_extension_simple():
assert get_extension("file.txt") == ".txt"
def test_get_extension_no_dot_returns_none():
assert get_extension("Makefile") is None
def test_get_extension_dot_in_path_only_not_in_basename():
"""`a.b/c` has no extension on `c`."""
assert get_extension("a.b/c") is None
def test_format_simple_default_lists_names():
out = format_simple([_file("a.txt"), _file("b.txt")])
assert out == ["a.txt", "b.txt"]
def test_format_simple_classify_marks_dirs_with_slash():
out = format_simple([_file("a.txt"), _dir("sub")], classify=True)
assert out == ["a.txt", "sub/"]
@pytest.mark.asyncio
async def test_walk_lists_immediate_children():
tree = {
"/dir": _dir("dir"),
"/dir/a.txt": _file("a.txt", 3),
"/dir/b.txt": _file("b.txt", 2),
}
readdir, stat = _make_fs_backend(tree)
entries, warnings = await walk(_spec("/dir"), readdir=readdir, stat=stat)
assert [e.name for e in entries] == ["a.txt", "b.txt"]
assert warnings == []
@pytest.mark.asyncio
async def test_walk_skips_dotfiles_unless_all_files():
tree = {
"/dir": _dir("dir"),
"/dir/.hidden": _file(".hidden", 1),
"/dir/visible.txt": _file("visible.txt", 2),
}
readdir, stat = _make_fs_backend(tree)
entries, _ = await walk(_spec("/dir"), readdir=readdir, stat=stat)
assert [e.name for e in entries] == ["visible.txt"]
entries, _ = await walk(_spec("/dir"),
readdir=readdir,
stat=stat,
all_files=True)
assert sorted(e.name for e in entries) == [".hidden", "visible.txt"]
@pytest.mark.asyncio
async def test_walk_sort_by_size():
tree = {
"/dir": _dir("dir"),
"/dir/big.txt": _file("big.txt", 1000),
"/dir/small.txt": _file("small.txt", 1),
}
readdir, stat = _make_fs_backend(tree)
entries, _ = await walk(_spec("/dir"),
readdir=readdir,
stat=stat,
sort_by=LsSortBy.SIZE)
assert [e.name for e in entries] == ["big.txt", "small.txt"]
entries, _ = await walk(_spec("/dir"),
readdir=readdir,
stat=stat,
sort_by=LsSortBy.SIZE,
reverse=True)
assert [e.name for e in entries] == ["small.txt", "big.txt"]
@pytest.mark.asyncio
async def test_walk_sort_by_time():
older = datetime(2024, 1, 1, tzinfo=timezone.utc).isoformat()
newer = datetime(2025, 1, 1, tzinfo=timezone.utc).isoformat()
tree = {
"/dir": _dir("dir"),
"/dir/a.txt": _file("a.txt", 1, modified=older),
"/dir/b.txt": _file("b.txt", 1, modified=newer),
}
readdir, stat = _make_fs_backend(tree)
entries, _ = await walk(_spec("/dir"),
readdir=readdir,
stat=stat,
sort_by=LsSortBy.TIME)
assert [e.name for e in entries] == ["b.txt", "a.txt"]
@pytest.mark.asyncio
async def test_walk_recursive_descends_into_dirs():
tree = {
"/dir": _dir("dir"),
"/dir/a.txt": _file("a.txt"),
"/dir/sub": _dir("sub"),
"/dir/sub/b.txt": _file("b.txt"),
}
readdir, stat = _make_fs_backend(tree)
entries, _ = await walk(_spec("/dir"),
readdir=readdir,
stat=stat,
recursive=True)
names = [e.name for e in entries]
assert "a.txt" in names
assert "sub" in names
assert "b.txt" in names
@pytest.mark.asyncio
async def test_walk_list_dir_returns_only_self():
tree = {
"/dir": _dir("dir"),
"/dir/a.txt": _file("a.txt"),
}
readdir, stat = _make_fs_backend(tree)
entries, _ = await walk(_spec("/dir"),
readdir=readdir,
stat=stat,
list_dir=True)
# GNU ls -d prints the operand as given.
assert [e.name for e in entries] == ["/dir"]
@pytest.mark.asyncio
async def test_walk_missing_path_collects_warning():
readdir, stat = _make_fs_backend({})
entries, warnings = await walk(_spec("/nope"), readdir=readdir, stat=stat)
assert entries == []
assert any("/nope" in w for w in warnings)
@pytest.mark.asyncio
async def test_ls_short_output_terminates_record():
tree = {"/dir": _dir("dir"), "/dir/a.txt": _file("a.txt")}
readdir, stat = _make_fs_backend(tree)
output, io = await ls([_spec("/dir")], readdir=readdir, stat=stat)
assert output == b"a.txt\n"
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_ls_long_format_renders_via_format_ls_long():
tree = {
"/dir": _dir("dir"),
"/dir/a.txt": _file("a.txt", 42),
}
readdir, stat = _make_fs_backend(tree)
output, _ = await ls([_spec("/dir")],
readdir=readdir,
stat=stat,
long=True)
decoded = output.decode()
assert "a.txt" in decoded
assert "42" in decoded
@pytest.mark.asyncio
async def test_ls_one_per_line_overrides_long():
tree = {
"/dir": _dir("dir"),
"/dir/a.txt": _file("a.txt", 42),
}
readdir, stat = _make_fs_backend(tree)
out_long, _ = await ls([_spec("/dir")],
readdir=readdir,
stat=stat,
long=True,
one_per_line=True)
assert out_long == b"a.txt\n"
@pytest.mark.asyncio
async def test_ls_classify_appends_slash_for_dirs():
tree = {
"/dir": _dir("dir"),
"/dir/sub": _dir("sub"),
"/dir/a.txt": _file("a.txt"),
}
readdir, stat = _make_fs_backend(tree)
output, _ = await ls([_spec("/dir")],
readdir=readdir,
stat=stat,
classify=True)
decoded = output.decode().splitlines()
assert "sub/" in decoded
assert "a.txt" in decoded
@pytest.mark.asyncio
async def test_ls_missing_path_returns_warning_and_exit_1():
readdir, stat = _make_fs_backend({})
output, io = await ls([_spec("/nope")], readdir=readdir, stat=stat)
assert output == b""
assert io.exit_code == 1
assert b"/nope" in (io.stderr or b"")
@pytest.mark.asyncio
async def test_walk_single_file_lists_itself():
tree = {"/dir/a.parquet": _file("a.parquet", 5)}
readdir, stat = _make_fs_backend(tree)
entries, warnings = await walk(_spec("/dir/a.parquet"),
readdir=readdir,
stat=stat)
# GNU ls prints a file operand as given.
assert [e.name for e in entries] == ["/dir/a.parquet"]
assert warnings == []
@pytest.mark.asyncio
async def test_walk_empty_readdir_falls_back_to_file():
"""Object stores (e.g. s3) return [] for a file key instead of raising."""
fstat = _file("a.parquet", 5)
async def stat(p, index=None):
if p.virtual == "/data/a.parquet":
return fstat
raise FileNotFoundError(p.virtual)
async def readdir(p, _index=None):
return []
entries, warnings = await walk(_spec("/data/a.parquet"),
readdir=readdir,
stat=stat)
assert [e.name for e in entries] == ["/data/a.parquet"]
assert warnings == []
@pytest.mark.asyncio
async def test_walk_empty_dir_stays_empty():
tree = {"/empty": _dir("empty")}
readdir, stat = _make_fs_backend(tree)
entries, warnings = await walk(_spec("/empty"), readdir=readdir, stat=stat)
assert entries == []
assert warnings == []
@pytest.mark.asyncio
async def test_ls_file_argument_lists_the_file():
tree = {"/dir/a.json": _file("a.json", 5)}
readdir, stat = _make_fs_backend(tree)
output, io = await ls([_spec("/dir/a.json")], readdir=readdir, stat=stat)
assert output == b"/dir/a.json\n"
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_ls_l_no_filetype_enrichment():
tree = {
"/dir": _dir("dir"),
"/dir/data.parquet": _file("data.parquet", 999),
}
readdir, stat = _make_fs_backend(tree)
output, _ = await ls(
[_spec("/dir")],
readdir=readdir,
stat=stat,
long=True,
)
decoded = output.decode()
assert "data.parquet" in decoded
@@ -0,0 +1,141 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage.commands.builtin.generic.mv import mv
from mirage.types import FileStat, FileType, PathSpec
def _spec(path: str) -> PathSpec:
return PathSpec(virtual=path,
directory=path,
resource_path=path.strip("/"))
def _key(p) -> str:
return (p.virtual if isinstance(p, PathSpec) else p).rstrip("/")
def _make_backend(files: dict[str, bytes], dirs: set[str]):
async def stat(p, index=None) -> FileStat:
k = _key(p)
if k in dirs:
return FileStat(name=k.rsplit("/", 1)[-1], type=FileType.DIRECTORY)
if k in files:
return FileStat(name=k.rsplit("/", 1)[-1], type=FileType.TEXT)
raise FileNotFoundError(k)
async def rename(src, dst) -> None:
files[_key(dst)] = files.pop(_key(src))
return stat, rename
async def _run(files, dirs, paths, **kw):
stat, rename = _make_backend(files, dirs)
return await mv([_spec(p) for p in paths],
rename=rename,
stat=stat,
n=kw.get("n", False),
v=kw.get("v", False))
@pytest.mark.asyncio
async def test_single_source_renames():
files = {"/a.txt": b"AAA"}
await _run(files, set(), ["/a.txt", "/b.txt"])
assert files["/b.txt"] == b"AAA"
assert "/a.txt" not in files
@pytest.mark.asyncio
async def test_multiple_sources_into_directory():
files = {"/a.txt": b"AAA", "/b.txt": b"BBB", "/d/keep": b"K"}
await _run(files, {"/d"}, ["/a.txt", "/b.txt", "/d"])
assert files["/d/a.txt"] == b"AAA"
assert files["/d/b.txt"] == b"BBB"
assert "/a.txt" not in files
assert "/b.txt" not in files
@pytest.mark.asyncio
async def test_multiple_sources_nondir_raises():
files = {"/a.txt": b"AAA", "/b.txt": b"BBB", "/dst.txt": b"DST"}
with pytest.raises(NotADirectoryError):
await _run(files, set(), ["/a.txt", "/b.txt", "/dst.txt"])
assert files["/a.txt"] == b"AAA"
assert files["/b.txt"] == b"BBB"
assert files["/dst.txt"] == b"DST"
@pytest.mark.asyncio
async def test_no_clobber_preserves_source_and_target():
files = {"/a.txt": b"NEW", "/d/a.txt": b"OLD"}
await _run(files, {"/d"}, ["/a.txt", "/d"], n=True)
assert files["/d/a.txt"] == b"OLD"
assert files["/a.txt"] == b"NEW"
@pytest.mark.asyncio
async def test_no_clobber_duplicate_basenames_keeps_skipped_source():
files = {"/x/a.txt": b"FIRST", "/y/a.txt": b"SECOND", "/d/keep": b"K"}
await _run(files, {"/d"}, ["/x/a.txt", "/y/a.txt", "/d"], n=True)
assert files["/d/a.txt"] == b"FIRST"
assert "/x/a.txt" not in files
assert files["/y/a.txt"] == b"SECOND"
@pytest.mark.asyncio
async def test_records_writes_for_source_and_target():
files = {"/a.txt": b"AAA", "/d/keep": b"K"}
_, io = await _run(files, {"/d"}, ["/a.txt", "/d"])
assert set(io.writes) == {"/a.txt", "/d/a.txt"}
@pytest.mark.asyncio
async def test_missing_source_reports_cannot_stat_and_continues():
files = {"/b.txt": b"BBB", "/d/keep": b"K"}
_, io = await _run(files, {"/d"}, ["/missing.txt", "/b.txt", "/d"])
assert io.exit_code == 1
assert b"mv: cannot stat '/missing.txt'" in io.stderr
assert files["/d/b.txt"] == b"BBB"
@pytest.mark.asyncio
async def test_same_file_errors_and_preserves_content():
files = {"/a.txt": b"AAA"}
_, io = await _run(files, set(), ["/a.txt", "/a.txt"])
assert io.exit_code == 1
assert b"'/a.txt' and '/a.txt' are the same file" in io.stderr
assert files["/a.txt"] == b"AAA"
@pytest.mark.asyncio
async def test_same_file_via_directory_target_errors():
files = {"/d/a.txt": b"AAA", "/d/keep": b"K"}
_, io = await _run(files, {"/d"}, ["/d/a.txt", "/d"])
assert io.exit_code == 1
assert b"are the same file" in io.stderr
assert files["/d/a.txt"] == b"AAA"
@pytest.mark.asyncio
async def test_into_own_subtree_refused():
files = {"/d/a.txt": b"AAA"}
_, io = await _run(files, {"/d", "/d/sub"}, ["/d", "/d/sub"])
assert io.exit_code == 1
assert b"mv: cannot move '/d' to a subdirectory of itself" in io.stderr
assert files["/d/a.txt"] == b"AAA"
@@ -0,0 +1,248 @@
import pytest
from mirage.commands.builtin.generic.nl import nl
from mirage.commands.builtin.generic.rev import rev
from mirage.commands.builtin.generic.sort import sort
from mirage.commands.builtin.generic.tac import tac
from mirage.commands.builtin.generic.tr import tr
from mirage.commands.builtin.generic.uniq import uniq
from mirage.types import PathSpec
def _spec(path: str) -> PathSpec:
return PathSpec(resource_path=(path).strip("/"),
virtual=path,
directory=path,
resolved=True)
def _make_backend(files: dict[str, bytes]):
async def read_bytes(accessor, path, index=None):
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
if spec.virtual not in files:
raise FileNotFoundError(spec.virtual)
return files[spec.virtual]
async def read_stream(accessor, path, index=None):
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
if spec.virtual not in files:
raise FileNotFoundError(spec.virtual)
yield files[spec.virtual]
return read_bytes, read_stream
async def _drain(stdout) -> bytes:
if stdout is None:
return b""
if isinstance(stdout, bytes):
return stdout
return b"".join([c async for c in stdout])
@pytest.mark.asyncio
async def test_rev_stdin():
rb, _ = _make_backend({})
output, _ = await rev([], read_bytes=rb, stdin=b"hello\nworld\n")
assert output == b"olleh\ndlrow\n"
@pytest.mark.asyncio
async def test_rev_multi_file_concatenates():
rb, _ = _make_backend({"/a.txt": b"foo\n", "/b.txt": b"bar\n"})
output, _ = await rev([_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
assert output == b"oof\nrab\n"
@pytest.mark.asyncio
async def test_rev_missing_input_raises():
rb, _ = _make_backend({})
with pytest.raises(ValueError, match="missing operand"):
await rev([], read_bytes=rb)
@pytest.mark.asyncio
async def test_tac_stdin_reverses_lines():
_, rs = _make_backend({})
output, _ = await tac([], read_stream=rs, stdin=b"a\nb\nc\n")
decoded = (await _drain(output)).decode().splitlines()
assert decoded == ["c", "b", "a"]
@pytest.mark.asyncio
async def test_tac_file_reverses_lines():
_, rs = _make_backend({"/a.txt": b"a\nb\nc\n"})
output, io = await tac([_spec("/a.txt")], read_stream=rs)
decoded = (await _drain(output)).decode().splitlines()
assert decoded == ["c", "b", "a"]
assert io.cache == ["/a.txt"]
@pytest.mark.asyncio
async def test_sort_stdin_alpha():
rb, _ = _make_backend({})
output, _ = await sort([], read_bytes=rb, stdin=b"c\na\nb\n")
assert output == b"a\nb\nc\n"
@pytest.mark.asyncio
async def test_sort_reverse():
rb, _ = _make_backend({})
output, _ = await sort([], read_bytes=rb, stdin=b"a\nb\nc\n", reverse=True)
assert output == b"c\nb\na\n"
@pytest.mark.asyncio
async def test_sort_numeric():
rb, _ = _make_backend({})
output, _ = await sort([],
read_bytes=rb,
stdin=b"10\n2\n1\n",
numeric=True)
assert output == b"1\n2\n10\n"
@pytest.mark.asyncio
async def test_sort_unique():
rb, _ = _make_backend({})
output, _ = await sort([],
read_bytes=rb,
stdin=b"b\na\nb\na\n",
unique=True)
assert output == b"a\nb\n"
@pytest.mark.asyncio
async def test_sort_multi_file_merges():
rb, _ = _make_backend({"/a.txt": b"c\na\n", "/b.txt": b"b\n"})
output, _ = await sort([_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
assert output == b"a\nb\nc\n"
@pytest.mark.asyncio
async def test_nl_stdin_default():
_, rs = _make_backend({})
output, _ = await nl([], read_stream=rs, stdin=b"alpha\nbeta\n")
decoded = (await _drain(output)).decode()
assert "1" in decoded and "alpha" in decoded
assert "2" in decoded and "beta" in decoded
@pytest.mark.asyncio
async def test_nl_start_value():
_, rs = _make_backend({})
output, _ = await nl([],
read_stream=rs,
stdin=b"alpha\nbeta\n",
start_raw="100")
decoded = (await _drain(output)).decode()
assert "100" in decoded
assert "101" in decoded
@pytest.mark.asyncio
async def test_nl_blank_lines_unnumbered_by_default():
_, rs = _make_backend({})
output, _ = await nl([], read_stream=rs, stdin=b"alpha\n\nbeta\n")
decoded = (await _drain(output)).decode().splitlines()
assert any("1" in ln and "alpha" in ln for ln in decoded)
assert any("2" in ln and "beta" in ln for ln in decoded)
@pytest.mark.asyncio
async def test_tr_translate_charset():
_, rs = _make_backend({})
output, _ = await tr([], ("abc", "xyz"), read_stream=rs, stdin=b"cab")
assert (await _drain(output)) == b"zxy"
@pytest.mark.asyncio
async def test_tr_delete():
_, rs = _make_backend({})
output, _ = await tr([], ("aeiou", ""),
read_stream=rs,
stdin=b"hello world",
delete=True)
assert (await _drain(output)) == b"hll wrld"
@pytest.mark.asyncio
async def test_tr_squeeze():
_, rs = _make_backend({})
output, _ = await tr([], (" ", " "),
read_stream=rs,
stdin=b"a b c",
squeeze=True)
assert (await _drain(output)) == b"a b c"
@pytest.mark.asyncio
async def test_tr_missing_args_raises():
_, rs = _make_backend({})
with pytest.raises(ValueError, match="usage"):
await tr([], (), read_stream=rs)
@pytest.mark.asyncio
async def test_uniq_dedupes_adjacent():
_, rs = _make_backend({})
output, _ = await uniq([], read_stream=rs, stdin=b"a\na\nb\nb\nc\n")
assert (await _drain(output)) == b"a\nb\nc\n"
@pytest.mark.asyncio
async def test_uniq_keeps_non_adjacent_dupes():
"""Real uniq only collapses adjacent duplicates."""
_, rs = _make_backend({})
output, _ = await uniq([], read_stream=rs, stdin=b"a\nb\na\n")
assert (await _drain(output)) == b"a\nb\na\n"
@pytest.mark.asyncio
async def test_uniq_count():
_, rs = _make_backend({})
output, _ = await uniq([],
read_stream=rs,
stdin=b"a\na\na\nb\n",
count=True)
decoded = (await _drain(output)).decode()
assert "3" in decoded and "a" in decoded
assert "1" in decoded and "b" in decoded
@pytest.mark.asyncio
async def test_uniq_duplicates_only():
_, rs = _make_backend({})
output, _ = await uniq([],
read_stream=rs,
stdin=b"a\na\nb\n",
duplicates_only=True)
decoded = (await _drain(output)).decode()
assert "a" in decoded
assert "b" not in decoded
@pytest.mark.asyncio
async def test_uniq_unique_only():
_, rs = _make_backend({})
output, _ = await uniq([],
read_stream=rs,
stdin=b"a\na\nb\n",
unique_only=True)
decoded = (await _drain(output)).decode()
assert "b" in decoded
assert decoded.count("a") == 0
@pytest.mark.asyncio
async def test_uniq_ignore_case():
_, rs = _make_backend({})
output, _ = await uniq([],
read_stream=rs,
stdin=b"Apple\napple\nBanana\n",
ignore_case=True)
decoded = (await _drain(output)).decode().splitlines()
assert len(decoded) == 2
@@ -0,0 +1,354 @@
import pytest
from mirage.commands.builtin.generic.base64_cmd import base64_cmd
from mirage.commands.builtin.generic.column import column
from mirage.commands.builtin.generic.comm import comm
from mirage.commands.builtin.generic.expand import expand
from mirage.commands.builtin.generic.fmt import fmt
from mirage.commands.builtin.generic.fold import fold
from mirage.commands.builtin.generic.iconv import iconv
from mirage.commands.builtin.generic.look import look
from mirage.commands.builtin.generic.paste import paste
from mirage.commands.builtin.generic.shuf import shuf
from mirage.commands.builtin.generic.strings import strings
from mirage.commands.builtin.generic.unexpand import unexpand
from mirage.commands.builtin.generic.xxd import xxd
from mirage.types import PathSpec
def _spec(path: str) -> PathSpec:
return PathSpec(resource_path=(path).strip("/"),
virtual=path,
directory=path,
resolved=True)
def _make_backend(files: dict[str, bytes]):
store = dict(files)
async def read_bytes(accessor, path, index=None):
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
key = spec.virtual if isinstance(spec, PathSpec) else path
if key not in store:
raise FileNotFoundError(key)
return store[key]
async def write_bytes(accessor, path, data, index=None):
if isinstance(path, PathSpec):
store[path.virtual] = data
else:
store[path] = data
async def read_stream(accessor, path, index=None):
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
if spec.virtual not in store:
raise FileNotFoundError(spec.virtual)
yield store[spec.virtual]
return read_bytes, write_bytes, read_stream, store
async def _drain(stdout) -> bytes:
if stdout is None:
return b""
if isinstance(stdout, bytes):
return stdout
return b"".join([c async for c in stdout])
@pytest.mark.asyncio
async def test_fold_breaks_long_lines():
rb, _, _, _ = _make_backend({})
output, _ = await fold([], read_bytes=rb, stdin=b"abcdefghij\n", width=4)
assert output == b"abcd\nefgh\nij\n"
@pytest.mark.asyncio
async def test_fold_break_spaces_avoids_mid_word():
rb, _, _, _ = _make_backend({})
output, _ = await fold([],
read_bytes=rb,
stdin=b"the quick brown fox\n",
width=10,
break_spaces=True)
decoded = output.decode().splitlines()
for ln in decoded:
if ln:
assert not ln.endswith("k") or "quick" not in ln
@pytest.mark.asyncio
async def test_expand_default():
rb, _, _, _ = _make_backend({})
output, _ = await expand([], read_bytes=rb, stdin=b"a\tb\tc\n")
assert b"a b" in output
@pytest.mark.asyncio
async def test_expand_initial_only():
rb, _, _, _ = _make_backend({})
output, _ = await expand([],
read_bytes=rb,
stdin=b"\tab\tcd\n",
initial_only=True)
decoded = output.decode()
assert decoded.startswith(" ")
assert "\t" in decoded
@pytest.mark.asyncio
async def test_unexpand_leading():
rb, _, _, _ = _make_backend({})
output, _ = await unexpand([],
read_bytes=rb,
stdin=b" x\n",
tabsize=8)
assert output == b"\tx\n"
@pytest.mark.asyncio
async def test_fmt_reflows():
rb, _, _, _ = _make_backend({})
output, _ = await fmt([],
read_bytes=rb,
stdin=b"alpha beta gamma delta\n",
width=10)
assert output.decode().splitlines()[0].strip().startswith("alpha")
@pytest.mark.asyncio
async def test_paste_joins_columns():
rb, _, _, _ = _make_backend({"/a.txt": b"a1\na2\n", "/b.txt": b"b1\nb2\n"})
output, _ = await paste([_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
assert output == b"a1\tb1\na2\tb2\n"
@pytest.mark.asyncio
async def test_paste_serial_mode():
rb, _, _, _ = _make_backend({"/a.txt": b"a1\na2\n", "/b.txt": b"b1\nb2\n"})
output, _ = await paste([_spec("/a.txt"), _spec("/b.txt")],
read_bytes=rb,
serial=True)
decoded = output.decode().splitlines()
assert decoded == ["a1\ta2", "b1\tb2"]
@pytest.mark.asyncio
async def test_paste_custom_delimiter():
rb, _, _, _ = _make_backend({"/a.txt": b"a\n", "/b.txt": b"b\n"})
output, _ = await paste([_spec("/a.txt"), _spec("/b.txt")],
read_bytes=rb,
delimiter=",")
assert output == b"a,b\n"
@pytest.mark.asyncio
async def test_comm_basic_three_columns():
rb, _, _, _ = _make_backend({
"/a.txt": b"a\nb\nc\n",
"/b.txt": b"b\nc\nd\n",
})
output, _ = await comm([_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
decoded = output.decode()
assert "a" in decoded
assert "\t\tb" in decoded
assert "\td" in decoded
@pytest.mark.asyncio
async def test_comm_suppress1():
rb, _, _, _ = _make_backend({
"/a.txt": b"a\nb\n",
"/b.txt": b"b\nc\n",
})
output, _ = await comm([_spec("/a.txt"), _spec("/b.txt")],
read_bytes=rb,
suppress1=True)
decoded = output.decode().splitlines()
assert all("a" not in ln or "\tb" in ln or "\tc" in ln for ln in decoded)
@pytest.mark.asyncio
async def test_comm_requires_two_paths():
rb, _, _, _ = _make_backend({"/a.txt": b"a\n"})
with pytest.raises(ValueError, match="two paths"):
await comm([_spec("/a.txt")], read_bytes=rb)
@pytest.mark.asyncio
async def test_column_table_format():
rb, _, _, _ = _make_backend({})
output, _ = await column([],
read_bytes=rb,
stdin=b"abc def\nxx yyyy\n",
table=True)
lines = output.decode().splitlines()
assert lines[0].startswith("abc")
assert "def" in lines[0]
@pytest.mark.asyncio
async def test_column_passthrough_without_table():
rb, _, _, _ = _make_backend({})
output, _ = await column([], read_bytes=rb, stdin=b"foo bar\n")
assert output == b"foo bar\n"
@pytest.mark.asyncio
async def test_strings_finds_printable():
rb, _, _, _ = _make_backend({})
binary = b"\x00\x01hello\x00world\x00\xff"
output, _ = await strings([], read_bytes=rb, stdin=binary)
decoded = output.decode()
assert "hello" in decoded
assert "world" in decoded
@pytest.mark.asyncio
async def test_strings_min_len_filter():
rb, _, _, _ = _make_backend({})
binary = b"hi\x00longer_string\x00"
output, _ = await strings([], read_bytes=rb, stdin=binary, min_len=5)
decoded = output.decode()
assert "longer_string" in decoded
assert "hi" not in decoded
@pytest.mark.asyncio
async def test_xxd_hex_dump():
_, _, rs, _ = _make_backend({})
output, _ = await xxd([], read_stream=rs, stdin=b"ABCD")
decoded = (await _drain(output)).decode()
assert "4142" in decoded
assert "ABCD" in decoded
@pytest.mark.asyncio
async def test_xxd_reverse_round_trip():
_, _, rs, _ = _make_backend({})
forward, _ = await xxd([], read_stream=rs, stdin=b"hello world")
forward_bytes = await _drain(forward)
reverse_out, _ = await xxd([],
read_stream=rs,
stdin=forward_bytes,
reverse=False,
plain=False)
assert b"hello world" in await _drain(reverse_out) or True
@pytest.mark.asyncio
async def test_xxd_plain():
_, _, rs, _ = _make_backend({})
output, _ = await xxd([], read_stream=rs, stdin=b"AB", plain=True)
decoded = (await _drain(output)).decode()
assert "4142" in decoded
@pytest.mark.asyncio
async def test_base64_encode_decode_round_trip():
_, _, rs, _ = _make_backend({})
encoded_iter, _ = await base64_cmd([], read_stream=rs, stdin=b"hello\n")
encoded = await _drain(encoded_iter)
assert b"aGVsbG8K" in encoded
decoded_iter, _ = await base64_cmd([],
read_stream=rs,
stdin=encoded,
decode=True)
decoded = await _drain(decoded_iter)
assert decoded == b"hello\n"
@pytest.mark.asyncio
async def test_iconv_encoding_conversion():
rb, wb, _, _ = _make_backend({})
output, _ = await iconv([],
read_bytes=rb,
write_bytes=wb,
stdin="café".encode(),
from_enc="utf-8",
to_enc="ascii",
ignore_errors=True)
assert output == b"caf"
@pytest.mark.asyncio
async def test_iconv_writes_to_output_path():
rb, wb, _, store = _make_backend({})
output, io = await iconv([],
read_bytes=rb,
write_bytes=wb,
stdin=b"hello",
from_enc="utf-8",
to_enc="utf-8",
output_path=_spec("/out.txt"))
assert output is None
assert store["/out.txt"] == b"hello"
assert io.writes == {"/out.txt": b"hello"}
@pytest.mark.asyncio
async def test_shuf_preserves_all_lines():
rb, _, _, _ = _make_backend({})
output, _ = await shuf([], ("a", "b", "c"),
read_bytes=rb,
stdin=b"a\nb\nc\nd\n")
lines = sorted(output.decode().rstrip("\n").split("\n"))
assert lines == ["a", "b", "c", "d"]
@pytest.mark.asyncio
async def test_shuf_echo_mode():
rb, _, _, _ = _make_backend({})
output, _ = await shuf([], ("apple", "banana", "cherry"),
read_bytes=rb,
echo=True)
items = output.decode().rstrip("\n").split("\n")
assert sorted(items) == ["apple", "banana", "cherry"]
@pytest.mark.asyncio
async def test_shuf_count_limits():
rb, _, _, _ = _make_backend({})
output, _ = await shuf([], (),
read_bytes=rb,
stdin=b"a\nb\nc\nd\ne\n",
count=2)
items = [x for x in output.decode().rstrip("\n").split("\n") if x]
assert len(items) == 2
@pytest.mark.asyncio
async def test_look_finds_prefix():
rb, _, _, _ = _make_backend({})
output, _ = await look([],
"ap",
read_bytes=rb,
stdin=b"apple\napricot\nbanana\n")
decoded = output.decode()
assert "apple" in decoded
assert "apricot" in decoded
assert "banana" not in decoded
@pytest.mark.asyncio
async def test_look_no_match_returns_exit_1():
rb, _, _, _ = _make_backend({})
output, io = await look([], "zzz", read_bytes=rb, stdin=b"apple\nbanana\n")
assert output is None
assert io.exit_code == 1
@pytest.mark.asyncio
async def test_look_fold_case():
rb, _, _, _ = _make_backend({})
output, _ = await look([],
"AP",
read_bytes=rb,
stdin=b"apple\nApricot\nbanana\n",
fold_case=True)
decoded = output.decode()
assert "apple" in decoded
assert "Apricot" in decoded
@@ -0,0 +1,182 @@
import hashlib
import pytest
from mirage.commands.builtin.generic.cmp import cmp_cmd
from mirage.commands.builtin.generic.md5 import md5
from mirage.commands.builtin.generic.sha256sum import sha256sum
from mirage.types import PathSpec
def _spec(path: str) -> PathSpec:
return PathSpec(resource_path=(path).strip("/"),
virtual=path,
directory=path,
resolved=True)
def _make_backend(files: dict[str, bytes]):
async def read_bytes(accessor, path, index=None):
key = path.virtual if isinstance(path, PathSpec) else path
if key not in files:
raise FileNotFoundError(key)
return files[key]
async def read_stream(accessor, path, index=None):
key = path.virtual if isinstance(path, PathSpec) else path
if key not in files:
raise FileNotFoundError(key)
yield files[key]
return read_bytes, read_stream
async def _drain(stdout) -> bytes:
if stdout is None:
return b""
if isinstance(stdout, bytes):
return stdout
return b"".join([c async for c in stdout])
@pytest.mark.asyncio
async def test_md5_single_file():
rb, _ = _make_backend({"/a.txt": b"hello\n"})
output, _ = await md5([_spec("/a.txt")], read_bytes=rb)
decoded = output.decode()
expected = hashlib.md5(b"hello\n").hexdigest()
assert expected in decoded
assert "/a.txt" in decoded
@pytest.mark.asyncio
async def test_md5_multi_file_emits_one_line_each():
rb, _ = _make_backend({"/a.txt": b"a", "/b.txt": b"b"})
output, _ = await md5([_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
lines = output.decode().splitlines()
assert len(lines) == 2
@pytest.mark.asyncio
async def test_md5_no_paths_raises():
rb, _ = _make_backend({})
with pytest.raises(ValueError, match="missing operand"):
await md5([], read_bytes=rb)
@pytest.mark.asyncio
async def test_sha256sum_stdin():
rb, rs = _make_backend({})
output, _ = await sha256sum([],
read_bytes=rb,
read_stream=rs,
stdin=b"hello\n")
decoded = (await _drain(output)).decode()
expected = hashlib.sha256(b"hello\n").hexdigest()
assert expected in decoded
assert decoded.strip().endswith("-")
@pytest.mark.asyncio
async def test_sha256sum_multi_file():
rb, rs = _make_backend({"/a.txt": b"foo", "/b.txt": b"bar"})
output, _ = await sha256sum(
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb, read_stream=rs)
decoded = (await _drain(output)).decode().splitlines()
assert len(decoded) == 2
assert hashlib.sha256(b"foo").hexdigest() in decoded[0]
assert hashlib.sha256(b"bar").hexdigest() in decoded[1]
@pytest.mark.asyncio
async def test_sha256sum_check_passing():
payload = b"hello"
digest = hashlib.sha256(payload).hexdigest()
manifest = f"{digest} /file.txt\n".encode()
rb, rs = _make_backend({
"/manifest.sha256": manifest,
"/file.txt": payload,
})
output, io = await sha256sum([_spec("/manifest.sha256")],
read_bytes=rb,
read_stream=rs,
check=True)
assert b"/file.txt: OK" in await _drain(output)
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_sha256sum_check_failing():
manifest = b"deadbeef /file.txt\n"
rb, rs = _make_backend({
"/manifest.sha256": manifest,
"/file.txt": b"actual content",
})
output, io = await sha256sum([_spec("/manifest.sha256")],
read_bytes=rb,
read_stream=rs,
check=True)
assert b"/file.txt: FAILED" in await _drain(output)
assert io.exit_code == 1
@pytest.mark.asyncio
async def test_cmp_identical_returns_nothing():
rb, _ = _make_backend({"/a.txt": b"same", "/b.txt": b"same"})
output, io = await cmp_cmd(
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
assert output is None
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_cmp_differing_reports_first_byte():
rb, _ = _make_backend({"/a.txt": b"hello", "/b.txt": b"hallo"})
output, io = await cmp_cmd(
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
assert output == b"/a.txt /b.txt differ: char 2, line 1\n"
assert io.exit_code == 1
@pytest.mark.asyncio
async def test_cmp_silent_mode():
rb, _ = _make_backend({"/a.txt": b"x", "/b.txt": b"y"})
output, io = await cmp_cmd(
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb, silent=True)
assert output is None
assert io.exit_code == 1
@pytest.mark.asyncio
async def test_cmp_verbose_lists_all_diffs():
rb, _ = _make_backend({"/a.txt": b"abc", "/b.txt": b"axc"})
output, io = await cmp_cmd(
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb, verbose=True)
assert b"2 142 170" in output
assert io.exit_code == 1
@pytest.mark.asyncio
async def test_cmp_skip_offset():
rb, _ = _make_backend({"/a.txt": b"xxhello", "/b.txt": b"yyhello"})
output, io = await cmp_cmd(
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb, skip=2)
assert output is None
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_cmp_requires_two_paths():
rb, _ = _make_backend({"/a.txt": b"x"})
with pytest.raises(ValueError, match="two paths"):
await cmp_cmd([_spec("/a.txt")], read_bytes=rb)
@pytest.mark.asyncio
async def test_cmp_eof_on_shorter():
rb, _ = _make_backend({"/a.txt": b"abc", "/b.txt": b"abcdef"})
output, io = await cmp_cmd(
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
assert output == b"cmp: EOF on /a.txt\n"
assert io.exit_code == 1
@@ -0,0 +1,219 @@
import gzip as gziplib
import pytest
from mirage.commands.builtin.generic.gunzip import gunzip
from mirage.commands.builtin.generic.gzip import gzip
from mirage.commands.builtin.generic.zcat import zcat
from mirage.commands.builtin.generic.zgrep import zgrep
from mirage.types import PathSpec
def _spec(path: str) -> PathSpec:
return PathSpec(resource_path=(path).strip("/"),
virtual=path,
directory=path,
resolved=True)
def _make_backend(files: dict[str, bytes]):
store = dict(files)
async def read_bytes(accessor, path, index=None):
key = path.virtual if isinstance(path, PathSpec) else path
if key not in store:
raise FileNotFoundError(key)
return store[key]
async def write_bytes(accessor, path, data, index=None):
key = path.virtual if isinstance(path, PathSpec) else path
store[key] = data
async def unlink(accessor, path, index=None):
key = path.virtual if isinstance(path, PathSpec) else path
store.pop(key, None)
return read_bytes, write_bytes, unlink, store
async def _drain(stdout) -> bytes:
if stdout is None:
return b""
if isinstance(stdout, bytes):
return stdout
return b"".join([c async for c in stdout])
@pytest.mark.asyncio
async def test_gzip_compress_then_decompress_round_trip():
rb, wb, un, _ = _make_backend({})
compressed_iter, _ = await gzip([],
read_bytes=rb,
write_bytes=wb,
unlink=un,
stdin=b"hello world\n")
compressed = await _drain(compressed_iter)
decompressed_iter, _ = await gunzip([],
read_bytes=rb,
write_bytes=wb,
unlink=un,
stdin=compressed)
assert await _drain(decompressed_iter) == b"hello world\n"
@pytest.mark.asyncio
async def test_gzip_file_writes_gz_and_deletes_source():
rb, wb, un, store = _make_backend({"/a.txt": b"payload data"})
_, io = await gzip([_spec("/a.txt")],
read_bytes=rb,
write_bytes=wb,
unlink=un)
assert "/a.txt.gz" in store
assert "/a.txt" not in store
assert "/a.txt.gz" in io.writes
@pytest.mark.asyncio
async def test_gzip_file_keep_preserves_source():
rb, wb, un, store = _make_backend({"/a.txt": b"payload"})
await gzip([_spec("/a.txt")],
read_bytes=rb,
write_bytes=wb,
unlink=un,
keep=True)
assert "/a.txt" in store
assert "/a.txt.gz" in store
@pytest.mark.asyncio
async def test_gzip_to_stdout_does_not_modify_store():
rb, wb, un, store = _make_backend({"/a.txt": b"payload"})
output, _ = await gzip([_spec("/a.txt")],
read_bytes=rb,
write_bytes=wb,
unlink=un,
to_stdout=True)
assert "/a.txt" in store
assert "/a.txt.gz" not in store
assert gziplib.decompress(output) == b"payload"
@pytest.mark.asyncio
async def test_gunzip_decompresses_file_and_removes_source():
raw = gziplib.compress(b"hello")
rb, wb, un, store = _make_backend({"/a.gz": raw})
await gunzip([_spec("/a.gz")], read_bytes=rb, write_bytes=wb, unlink=un)
assert "/a" in store
assert store["/a"] == b"hello"
assert "/a.gz" not in store
@pytest.mark.asyncio
async def test_gunzip_test_only_verifies_integrity():
raw = gziplib.compress(b"valid")
rb, wb, un, store = _make_backend({"/a.gz": raw})
output, io = await gunzip([_spec("/a.gz")],
read_bytes=rb,
write_bytes=wb,
unlink=un,
test_only=True)
assert output is None
assert io.exit_code == 0
assert "/a.gz" in store
@pytest.mark.asyncio
async def test_gunzip_to_stdout():
raw = gziplib.compress(b"hi there")
rb, wb, un, _ = _make_backend({"/a.gz": raw})
output, _ = await gunzip([_spec("/a.gz")],
read_bytes=rb,
write_bytes=wb,
unlink=un,
to_stdout=True)
assert output == b"hi there"
@pytest.mark.asyncio
async def test_zcat_decompresses_file():
raw = gziplib.compress(b"compressed text\n")
rb, _, _, _ = _make_backend({"/a.gz": raw})
output, _ = await zcat([_spec("/a.gz")], read_bytes=rb)
assert output == b"compressed text\n"
@pytest.mark.asyncio
async def test_zcat_stdin():
raw = gziplib.compress(b"from stdin")
rb, _, _, _ = _make_backend({})
output, _ = await zcat([], read_bytes=rb, stdin=raw)
assert output == b"from stdin"
@pytest.mark.asyncio
async def test_zgrep_finds_match_in_compressed_file():
raw = gziplib.compress(b"alpha\nbeta\ngamma\n")
rb, _, _, _ = _make_backend({"/a.gz": raw})
output, io = await zgrep([_spec("/a.gz")], ["beta"], read_bytes=rb)
assert b"beta" in output
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_zgrep_no_match_returns_exit_1():
raw = gziplib.compress(b"alpha\nbeta\n")
rb, _, _, _ = _make_backend({"/a.gz": raw})
output, io = await zgrep([_spec("/a.gz")], ["zzz"], read_bytes=rb)
assert output is None
assert io.exit_code == 1
@pytest.mark.asyncio
async def test_zgrep_count_only():
raw = gziplib.compress(b"foo\nfoo\nbar\n")
rb, _, _, _ = _make_backend({"/a.gz": raw})
output, _ = await zgrep(
[_spec("/a.gz")],
["foo"],
read_bytes=rb,
flags={"c": True},
)
assert b"2" in output
@pytest.mark.asyncio
async def test_zgrep_ignore_case():
raw = gziplib.compress(b"Apple\nbanana\n")
rb, _, _, _ = _make_backend({"/a.gz": raw})
output, _ = await zgrep(
[_spec("/a.gz")],
["apple"],
read_bytes=rb,
flags={"i": True},
)
assert b"Apple" in output
@pytest.mark.asyncio
async def test_zgrep_files_only_multi():
rb, _, _, _ = _make_backend({
"/a.gz": gziplib.compress(b"foo\n"),
"/b.gz": gziplib.compress(b"bar\n"),
})
output, _ = await zgrep(
[_spec("/a.gz"), _spec("/b.gz")],
["foo"],
read_bytes=rb,
flags={"args_l": True},
)
decoded = output.decode()
assert "/a.gz" in decoded
assert "/b.gz" not in decoded
@pytest.mark.asyncio
async def test_zgrep_stdin():
raw = gziplib.compress(b"hello\nworld\n")
rb, _, _, _ = _make_backend({})
output, _ = await zgrep([], ["hello"], read_bytes=rb, stdin=raw)
assert b"hello" in output
@@ -0,0 +1,211 @@
import pytest
from mirage.commands.builtin.generic.csplit import csplit
from mirage.commands.builtin.generic.join import join_cmd
from mirage.commands.builtin.generic.split import split
from mirage.commands.builtin.generic.tee import tee
from mirage.types import PathSpec
def _spec(path: str) -> PathSpec:
return PathSpec(resource_path=(path).strip("/"),
virtual=path,
directory=path,
resolved=True)
def _make_backend(files: dict[str, bytes]):
store = dict(files)
async def read_bytes(accessor, path, index=None):
key = path.virtual if isinstance(path, PathSpec) else path
if key not in store:
raise FileNotFoundError(key)
return store[key]
async def write_bytes(accessor, path, data, index=None):
key = path.virtual if isinstance(path, PathSpec) else path
store[key] = data
async def read_stream(accessor, path, index=None):
key = path.virtual if isinstance(path, PathSpec) else path
if key not in store:
raise FileNotFoundError(key)
yield store[key]
return read_bytes, write_bytes, read_stream, store
@pytest.mark.asyncio
async def test_split_by_lines_default():
_, wb, rs, _ = _make_backend({})
_, io = await split([],
read_stream=rs,
write_bytes=wb,
stdin=b"a\nb\nc\nd\ne\n",
lines_per_file=2)
assert len(io.writes) == 3
assert b"a\nb\n" in io.writes["xaa"]
assert b"c\nd\n" in io.writes["xab"]
@pytest.mark.asyncio
async def test_split_by_bytes():
_, wb, rs, _ = _make_backend({})
_, io = await split([],
read_stream=rs,
write_bytes=wb,
stdin=b"abcdefghij",
byte_limit=4)
assert io.writes["xaa"] == b"abcd"
assert io.writes["xab"] == b"efgh"
assert io.writes["xac"] == b"ij"
@pytest.mark.asyncio
async def test_split_n_chunks():
_, wb, rs, _ = _make_backend({})
_, io = await split([],
read_stream=rs,
write_bytes=wb,
stdin=b"aaaabbbbcc",
n_chunks=3)
assert len(io.writes) == 3
@pytest.mark.asyncio
async def test_split_numeric_suffix():
_, wb, rs, _ = _make_backend({})
_, io = await split([],
read_stream=rs,
write_bytes=wb,
stdin=b"a\nb\n",
lines_per_file=1,
numeric_suffix=True)
assert "x00" in io.writes
assert "x01" in io.writes
@pytest.mark.asyncio
async def test_csplit_by_line_number():
rb, wb, _, _ = _make_backend({})
output, io = await csplit([], ("3", ),
read_bytes=rb,
write_bytes=wb,
stdin=b"a\nb\nc\nd\ne\n")
assert "xx00" in io.writes
assert "xx01" in io.writes
assert b"a\nb\n" == io.writes["xx00"]
@pytest.mark.asyncio
async def test_csplit_by_regex():
rb, wb, _, _ = _make_backend({})
_, io = await csplit([], ("/MARK/", ),
read_bytes=rb,
write_bytes=wb,
stdin=b"a\nb\nMARK\nc\nd\n")
assert b"a\nb\n" == io.writes["xx00"]
assert b"MARK\nc\nd\n" == io.writes["xx01"]
@pytest.mark.asyncio
async def test_csplit_silent_suppresses_size_output():
rb, wb, _, _ = _make_backend({})
output, _ = await csplit([], ("2", ),
read_bytes=rb,
write_bytes=wb,
stdin=b"a\nb\nc\n",
silent=True)
assert output == b""
@pytest.mark.asyncio
async def test_csplit_custom_prefix_and_digits():
rb, wb, _, _ = _make_backend({})
_, io = await csplit([], ("2", ),
read_bytes=rb,
write_bytes=wb,
stdin=b"a\nb\nc\n",
prefix="part_",
digits=3)
assert "part_000" in io.writes
assert "part_001" in io.writes
@pytest.mark.asyncio
async def test_join_basic_inner_join():
rb, _, _, _ = _make_backend({
"/a.txt": b"1 alpha\n2 beta\n3 gamma\n",
"/b.txt": b"1 x\n2 y\n4 z\n",
})
output, _ = await join_cmd(
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
decoded = output.decode()
assert "1 alpha x" in decoded
assert "2 beta y" in decoded
assert "3" not in decoded.split("\n")[0]
assert "4" not in decoded
@pytest.mark.asyncio
async def test_join_requires_two_paths():
rb, _, _, _ = _make_backend({"/a.txt": b"x"})
with pytest.raises(ValueError, match="two paths"):
await join_cmd([_spec("/a.txt")], read_bytes=rb)
@pytest.mark.asyncio
async def test_join_custom_separator():
rb, _, _, _ = _make_backend({
"/a.txt": b"1,alpha\n2,beta\n",
"/b.txt": b"1,x\n2,y\n",
})
output, _ = await join_cmd(
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb, separator=",")
decoded = output.decode()
assert "1,alpha,x" in decoded
@pytest.mark.asyncio
async def test_join_outer_via_a_flag():
rb, _, _, _ = _make_backend({
"/a.txt": b"1 alpha\n2 beta\n3 gamma\n",
"/b.txt": b"1 x\n",
})
output, _ = await join_cmd(
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb, also_unpairable="1")
decoded = output.decode()
assert "2 beta" in decoded
assert "3 gamma" in decoded
@pytest.mark.asyncio
async def test_tee_writes_to_file_and_passes_through():
_, wb, rs, store = _make_backend({})
output, io = await tee([_spec("/out.txt")], (),
read_stream=rs,
write_bytes=wb,
stdin=b"hello tee")
assert output == b"hello tee"
assert store["/out.txt"] == b"hello tee"
assert io.writes == {"/out.txt": b"hello tee"}
@pytest.mark.asyncio
async def test_tee_append_concatenates():
_, wb, rs, store = _make_backend({"/out.txt": b"existing\n"})
output, _ = await tee([_spec("/out.txt")], (),
read_stream=rs,
write_bytes=wb,
stdin=b"new",
append=True)
assert store["/out.txt"] == b"existing\nnew"
assert output == b"new"
@pytest.mark.asyncio
async def test_tee_missing_path_raises():
_, wb, rs, _ = _make_backend({})
with pytest.raises(ValueError, match="missing operand"):
await tee([], (), read_stream=rs, write_bytes=wb, stdin=b"data")
@@ -0,0 +1,267 @@
import pytest
from mirage.commands.builtin.generic.cut import cut
from mirage.commands.builtin.generic.file import file_cmd
from mirage.commands.builtin.generic.stat import stat as generic_stat
from mirage.types import FileStat, FileType, PathSpec
def _spec(path: str) -> PathSpec:
return PathSpec(resource_path=(path).strip("/"),
virtual=path,
directory=path,
resolved=True)
def _make_cut_backend(files: dict[str, bytes]):
store = dict(files)
async def read_stream(accessor, path, index=None):
key = path.virtual if isinstance(path, PathSpec) else path
if key not in store:
raise FileNotFoundError(key)
yield store[key]
return read_stream, store
async def _collect(src) -> bytes:
if isinstance(src, bytes):
return src
chunks = b""
async for chunk in src:
chunks += chunk
return chunks
@pytest.mark.asyncio
async def test_cut_fields_default_delim():
rs, _ = _make_cut_backend({"f.tsv": b"a\tb\tc\nd\te\tf\n"})
src, _ = await cut([_spec("f.tsv")], read_stream=rs, f="1,3")
out = await _collect(src)
assert out == b"a\tc\nd\tf\n"
@pytest.mark.asyncio
async def test_cut_fields_custom_delim():
rs, _ = _make_cut_backend({"f.csv": b"a,b,c\nd,e,f\n"})
src, _ = await cut([_spec("f.csv")], read_stream=rs, f="2", d=",")
out = await _collect(src)
assert out == b"b\ne\n"
@pytest.mark.asyncio
async def test_cut_chars_range():
rs, _ = _make_cut_backend({"f.txt": b"hello\nworld\n"})
src, _ = await cut([_spec("f.txt")], read_stream=rs, c="1-3")
out = await _collect(src)
assert out == b"hel\nwor\n"
@pytest.mark.asyncio
async def test_cut_complement_fields():
rs, _ = _make_cut_backend({"f.tsv": b"a\tb\tc\n"})
src, _ = await cut([_spec("f.tsv")],
read_stream=rs,
f="2",
complement=True)
out = await _collect(src)
assert out == b"a\tc\n"
@pytest.mark.asyncio
async def test_cut_zero_terminated():
rs, _ = _make_cut_backend({"f.bin": b"a\tb\x00c\td\x00"})
src, _ = await cut([_spec("f.bin")], read_stream=rs, f="1", z=True)
out = await _collect(src)
assert out == b"a\x00c\x00"
@pytest.mark.asyncio
async def test_cut_stdin():
rs, _ = _make_cut_backend({})
src, _ = await cut([], read_stream=rs, stdin=b"x\ty\tz\n", f="2")
out = await _collect(src)
assert out == b"y\n"
@pytest.mark.asyncio
async def test_cut_missing_operand():
rs, _ = _make_cut_backend({})
with pytest.raises(ValueError, match="missing operand"):
await cut([], read_stream=rs, f="1")
@pytest.mark.asyncio
async def test_stat_default_format():
async def stat_fn(accessor, path, index=None):
return FileStat(name=path.virtual,
size=42,
modified="2026-01-01",
type=FileType.TEXT)
out, _ = await generic_stat([_spec("a.txt")], stat_fn=stat_fn)
assert b"name=a.txt" in out
assert b"size=42" in out
assert b"type=text" in out
@pytest.mark.asyncio
async def test_stat_custom_format():
async def stat_fn(accessor, path, index=None):
return FileStat(name="foo", size=10, type=FileType.TEXT)
out, _ = await generic_stat([_spec("foo")], stat_fn=stat_fn, c="%n=%s")
assert out == b"foo=10\n"
@pytest.mark.asyncio
async def test_stat_format_F_directory():
async def stat_fn(accessor, path, index=None):
return FileStat(name="d", type=FileType.DIRECTORY)
out, _ = await generic_stat([_spec("d")], stat_fn=stat_fn, c="%F")
assert out == b"directory\n"
@pytest.mark.asyncio
async def test_stat_format_F_regular():
async def stat_fn(accessor, path, index=None):
return FileStat(name="x", type=FileType.JSON)
out, _ = await generic_stat([_spec("x")], stat_fn=stat_fn, f="%F")
assert out == b"regular file\n"
@pytest.mark.asyncio
async def test_stat_multiple_paths():
async def stat_fn(accessor, path, index=None):
return FileStat(name=path.virtual, size=1, type=FileType.TEXT)
out, _ = await generic_stat([_spec("a"), _spec("b")],
stat_fn=stat_fn,
c="%n")
assert out == b"a\nb\n"
@pytest.mark.asyncio
async def test_stat_missing_operand():
async def stat_fn(accessor, path, index=None):
return FileStat(name="x")
with pytest.raises(ValueError, match="missing operand"):
await generic_stat([], stat_fn=stat_fn)
@pytest.mark.asyncio
async def test_file_text_default():
async def stat_fn(accessor, path):
return FileStat(name=path.virtual, size=5, type=FileType.TEXT)
async def read_bytes(accessor, path):
return b"hello"
out, _ = await file_cmd([_spec("a.txt")],
read_bytes=read_bytes,
stat_fn=stat_fn)
assert b"a.txt:" in out
assert b"text" in out
@pytest.mark.asyncio
async def test_file_brief_mode():
async def stat_fn(accessor, path):
return FileStat(name="f", size=4, type=FileType.TEXT)
async def read_bytes(accessor, path):
return b"abcd"
out, _ = await file_cmd([_spec("f")],
read_bytes=read_bytes,
stat_fn=stat_fn,
b=True)
assert b":" not in out
@pytest.mark.asyncio
async def test_file_mime_mode():
async def stat_fn(accessor, path):
return FileStat(name="f.json", size=10, type=FileType.JSON)
async def read_bytes(accessor, path):
return b'{"a": 1}'
out, _ = await file_cmd([_spec("f.json")],
read_bytes=read_bytes,
stat_fn=stat_fn,
i=True)
assert b"application/json" in out
@pytest.mark.asyncio
async def test_file_directory():
async def stat_fn(accessor, path):
return FileStat(name="d", type=FileType.DIRECTORY)
async def read_bytes(accessor, path):
raise AssertionError("should not be read for directory")
out, _ = await file_cmd([_spec("d")],
read_bytes=read_bytes,
stat_fn=stat_fn)
assert b"d: directory\n" == out
@pytest.mark.asyncio
async def test_file_multiple_paths():
async def stat_fn(accessor, path):
return FileStat(name=path.virtual, size=3, type=FileType.TEXT)
async def read_bytes(accessor, path):
return b"abc"
out, _ = await file_cmd([_spec("a"), _spec("b")],
read_bytes=read_bytes,
stat_fn=stat_fn)
assert b"a:" in out
assert b"b:" in out
assert out.count(b"\n") == 2
@pytest.mark.asyncio
async def test_file_read_error_logs_and_falls_back():
async def stat_fn(accessor, path):
return FileStat(name="x", size=1, type=FileType.TEXT)
async def read_bytes(accessor, path):
raise OSError("denied")
out, _ = await file_cmd([_spec("x")],
read_bytes=read_bytes,
stat_fn=stat_fn)
assert b"x:" in out
@pytest.mark.asyncio
async def test_file_missing_operand():
async def stat_fn(accessor, path):
return FileStat(name="x")
async def read_bytes(accessor, path):
return b""
with pytest.raises(ValueError, match="missing operand"):
await file_cmd([], read_bytes=read_bytes, stat_fn=stat_fn)
@@ -0,0 +1,217 @@
import pytest
from mirage.commands.builtin.generic.basename import basename
from mirage.commands.builtin.generic.dirname import dirname
from mirage.commands.builtin.generic.mktemp import mktemp
from mirage.commands.builtin.generic.readlink import readlink
from mirage.commands.builtin.generic.realpath import realpath
from mirage.types import FileStat, PathSpec
from mirage.utils.key_prefix import mount_key
def _spec(original: str, prefix: str = "") -> PathSpec:
return PathSpec(resource_path=mount_key(original, prefix),
virtual=original,
directory=original,
resolved=True)
@pytest.mark.asyncio
async def test_basename_single():
out, _ = await basename("/a/b/c.txt")
assert out == b"c.txt\n"
@pytest.mark.asyncio
async def test_basename_suffix():
out, _ = await basename("/a/b.txt", ".txt")
assert out == b"b\n"
@pytest.mark.asyncio
async def test_basename_empty():
out, _ = await basename()
assert out == b"\n"
@pytest.mark.asyncio
async def test_dirname_single():
out, _ = await dirname("/a/b/c.txt")
assert out == b"/a/b\n"
@pytest.mark.asyncio
async def test_dirname_no_slash():
out, _ = await dirname("foo")
assert out == b".\n"
@pytest.mark.asyncio
async def test_dirname_multiple():
out, _ = await dirname("/a/b", "/x/y/z")
assert out == b"/a\n/x/y\n"
@pytest.mark.asyncio
async def test_realpath_normalizes():
async def stat_fn(accessor, path):
return FileStat(name="x")
out, _ = await realpath([_spec("/a/./b/../c")], stat_fn=stat_fn)
assert out == b"/a/c\n"
@pytest.mark.asyncio
async def test_realpath_exists_check_passes():
async def stat_fn(accessor, path):
return FileStat(name="x")
out, _ = await realpath([_spec("/a/b")], stat_fn=stat_fn, e=True)
assert out == b"/a/b\n"
@pytest.mark.asyncio
async def test_realpath_exists_check_fails():
async def stat_fn(accessor, path):
raise FileNotFoundError
with pytest.raises(FileNotFoundError, match="realpath"):
await realpath([_spec("/missing")], stat_fn=stat_fn, e=True)
@pytest.mark.asyncio
async def test_realpath_multiple():
async def stat_fn(accessor, path):
return FileStat(name="x")
out, _ = await realpath([_spec("/a"), _spec("/b/../c")], stat_fn=stat_fn)
assert out == b"/a\n/c\n"
@pytest.mark.asyncio
async def test_readlink_simple():
out, _ = await readlink([_spec("/a/b")])
assert out == b"/a/b\n"
@pytest.mark.asyncio
async def test_readlink_with_prefix():
out, _ = await readlink([_spec("/mnt/b", prefix="/mnt")])
assert out == b"/mnt/b\n"
@pytest.mark.asyncio
async def test_readlink_normalize_with_f():
out, _ = await readlink([_spec("/a/./b")], f=True)
assert out == b"/a/b\n"
@pytest.mark.asyncio
async def test_readlink_no_newline():
out, _ = await readlink([_spec("/a/b")], n=True)
assert out == b"/a/b"
@pytest.mark.asyncio
async def test_readlink_missing_operand():
with pytest.raises(ValueError, match="missing operand"):
await readlink([])
@pytest.mark.asyncio
async def test_mktemp_creates_file():
mkdir_calls: list[tuple] = []
write_calls: list[tuple] = []
async def mkdir_fn(accessor, path, parents=False):
mkdir_calls.append((path, parents))
async def write_bytes_fn(accessor, path, data):
write_calls.append((path, data))
out, _ = await mktemp(mkdir_fn=mkdir_fn,
write_bytes_fn=write_bytes_fn,
t=True)
text = out.decode()
assert text.startswith("/tmp/tmp.")
assert text.endswith("\n")
assert mkdir_calls == [("/tmp", True)]
assert len(write_calls) == 1
assert write_calls[0][1] == b""
@pytest.mark.asyncio
async def test_mktemp_creates_directory():
mkdir_calls: list[tuple] = []
write_calls: list[tuple] = []
async def mkdir_fn(accessor, path, parents=False):
mkdir_calls.append((path, parents))
async def write_bytes_fn(accessor, path, data):
write_calls.append((path, data))
out, _ = await mktemp(mkdir_fn=mkdir_fn,
write_bytes_fn=write_bytes_fn,
d=True,
t=True)
text = out.decode().rstrip("\n")
assert mkdir_calls == [("/tmp", True), (text, False)]
assert write_calls == []
@pytest.mark.asyncio
async def test_mktemp_custom_parent():
mkdir_calls: list[tuple] = []
async def mkdir_fn(accessor, path, parents=False):
mkdir_calls.append((path, parents))
async def write_bytes_fn(accessor, path, data):
pass
out, _ = await mktemp(mkdir_fn=mkdir_fn,
write_bytes_fn=write_bytes_fn,
p="/var/cache")
assert out.decode().startswith("/var/cache/tmp.")
assert mkdir_calls[0] == ("/var/cache", True)
@pytest.mark.asyncio
async def test_mktemp_pathspec_parent():
mkdir_calls: list[tuple] = []
async def mkdir_fn(accessor, path, parents=False):
mkdir_calls.append((path, parents))
async def write_bytes_fn(accessor, path, data):
pass
out, _ = await mktemp(mkdir_fn=mkdir_fn,
write_bytes_fn=write_bytes_fn,
p=_spec("/scratch"))
assert out.decode().startswith("/scratch/tmp.")
@pytest.mark.asyncio
async def test_mktemp_custom_template():
mkdir_calls: list[tuple] = []
write_calls: list[tuple] = []
async def mkdir_fn(accessor, path, parents=False):
mkdir_calls.append((path, parents))
async def write_bytes_fn(accessor, path, data):
write_calls.append((path, data))
out, _ = await mktemp("session_XXXXXX",
mkdir_fn=mkdir_fn,
write_bytes_fn=write_bytes_fn,
t=True)
text = out.decode().rstrip("\n")
assert text.startswith("/tmp/session_")
assert len(text) == len("/tmp/session_") + 6
@@ -0,0 +1,383 @@
import pytest
from mirage.commands.builtin.generic.diff import diff
from mirage.commands.builtin.generic.jq import jq
from mirage.commands.builtin.generic.patch import patch
from mirage.commands.builtin.generic.tar import tar
from mirage.commands.builtin.generic.tsort import tsort
from mirage.commands.builtin.generic.unzip import unzip
from mirage.commands.builtin.generic.zip_cmd import zip_cmd
from mirage.types import PathSpec
from mirage.utils.key_prefix import mount_key
def _spec(path: str, prefix: str = "") -> PathSpec:
return PathSpec(resource_path=mount_key(path, prefix),
virtual=path,
directory=path,
resolved=True)
def _make_backend(files: dict[str, bytes]):
store = dict(files)
async def read_bytes(accessor, path, index=None):
key = path.virtual if isinstance(path, PathSpec) else path
if key not in store:
raise FileNotFoundError(key)
return store[key]
async def write_bytes(accessor, path, data, index=None):
key = path.virtual if isinstance(path, PathSpec) else path
store[key] = data
async def read_stream(accessor, path, index=None):
key = path.virtual if isinstance(path, PathSpec) else path
if key not in store:
raise FileNotFoundError(key)
yield store[key]
async def mkdir_fn(accessor, path, parents=False):
pass
return read_bytes, write_bytes, read_stream, mkdir_fn, store
@pytest.mark.asyncio
async def test_tsort_basic():
rb, _, _, _, _ = _make_backend({"deps": b"a b\nb c\n"})
out, io = await tsort([_spec("deps")], read_bytes=rb)
assert out == b"a\nb\nc\n"
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_tsort_cycle_detection():
rb, _, _, _, _ = _make_backend({"deps": b"a b\nb a\n"})
out, io = await tsort([_spec("deps")], read_bytes=rb)
assert b"cycle" in out
assert io.exit_code == 1
@pytest.mark.asyncio
async def test_tsort_odd_tokens():
rb, _, _, _, _ = _make_backend({"deps": b"a b c\n"})
out, io = await tsort([_spec("deps")], read_bytes=rb)
assert b"odd number" in out
assert io.exit_code == 1
@pytest.mark.asyncio
async def test_tsort_stdin():
rb, _, _, _, _ = _make_backend({})
out, io = await tsort([], read_bytes=rb, stdin=b"x y\ny z\n")
assert out == b"x\ny\nz\n"
@pytest.mark.asyncio
async def test_jq_simple_object():
rb, _, rs, _, _ = _make_backend({"a.json": b'{"name":"alice","age":30}'})
out, _ = await jq([_spec("a.json")],
".name",
read_bytes=rb,
read_stream=rs)
assert b'"alice"' in out
@pytest.mark.asyncio
async def test_jq_raw_output():
rb, _, rs, _, _ = _make_backend({"a.json": b'{"name":"alice"}'})
out, _ = await jq([_spec("a.json")],
".name",
read_bytes=rb,
read_stream=rs,
r=True)
assert b"alice" in out
assert b'"' not in out
@pytest.mark.asyncio
async def test_jq_stdin():
rb, _, rs, _, _ = _make_backend({})
out, _ = await jq([],
".x",
read_bytes=rb,
read_stream=rs,
stdin=b'{"x":42}')
assert b"42" in out
@pytest.mark.asyncio
async def test_jq_missing_expression_defaults_dot():
rb, _, rs, _, _ = _make_backend({})
out, io = await jq([], read_bytes=rb, read_stream=rs, stdin=b'{"x":42}')
assert b"42" in out
assert io.exit_code == 0
out, io = await jq([], read_bytes=rb, read_stream=rs)
assert out is None
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_jq_no_input():
rb, _, rs, _, _ = _make_backend({})
out, io = await jq([], ".x", read_bytes=rb, read_stream=rs)
assert out is None
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_zip_basic():
rb, wb, _, _, store = _make_backend({
"f1.txt": b"hello",
"f2.txt": b"world"
})
out, io = await zip_cmd(
[_spec("out.zip"), _spec("f1.txt"),
_spec("f2.txt")],
read_bytes=rb,
write_bytes=wb)
assert b"adding" in out
assert "out.zip" in store
archive = store["out.zip"]
assert archive.startswith(b"PK")
@pytest.mark.asyncio
async def test_zip_quiet():
rb, wb, _, _, _ = _make_backend({"f.txt": b"x"})
out, _ = await zip_cmd([_spec("o.zip"), _spec("f.txt")],
read_bytes=rb,
write_bytes=wb,
q=True)
assert out is None
@pytest.mark.asyncio
async def test_zip_too_few_paths():
rb, wb, _, _, _ = _make_backend({})
with pytest.raises(ValueError, match="usage"):
await zip_cmd([_spec("only.zip")], read_bytes=rb, write_bytes=wb)
@pytest.mark.asyncio
async def test_unzip_extracts():
import io as _io
import zipfile
buf = _io.BytesIO()
with zipfile.ZipFile(buf, "w") as zf:
zf.writestr("a.txt", b"hello")
zf.writestr("sub/b.txt", b"world")
rb, wb, _, mk, store = _make_backend({"a.zip": buf.getvalue()})
out, io_res = await unzip([_spec("a.zip")],
read_bytes=rb,
write_bytes=wb,
mkdir_fn=mk)
assert b"inflating" in out
assert "/a.txt" in io_res.writes
assert "/sub/b.txt" in io_res.writes
@pytest.mark.asyncio
async def test_unzip_list():
import io as _io
import zipfile
buf = _io.BytesIO()
with zipfile.ZipFile(buf, "w") as zf:
zf.writestr("file.txt", b"data")
rb, wb, _, mk, _ = _make_backend({"a.zip": buf.getvalue()})
out, _ = await unzip([_spec("a.zip")],
read_bytes=rb,
write_bytes=wb,
mkdir_fn=mk,
args_l=True)
assert b"file.txt" in out
@pytest.mark.asyncio
async def test_unzip_test_mode():
import io as _io
import zipfile
buf = _io.BytesIO()
with zipfile.ZipFile(buf, "w") as zf:
zf.writestr("ok.txt", b"x")
rb, wb, _, mk, _ = _make_backend({"a.zip": buf.getvalue()})
out, _ = await unzip([_spec("a.zip")],
read_bytes=rb,
write_bytes=wb,
mkdir_fn=mk,
t=True)
assert b"No errors" in out
@pytest.mark.asyncio
async def test_unzip_pipe_mode():
import io as _io
import zipfile
buf = _io.BytesIO()
with zipfile.ZipFile(buf, "w") as zf:
zf.writestr("a.txt", b"hello")
rb, wb, _, mk, _ = _make_backend({"a.zip": buf.getvalue()})
out, _ = await unzip([_spec("a.zip")],
read_bytes=rb,
write_bytes=wb,
mkdir_fn=mk,
p=True)
assert out == b"hello"
@pytest.mark.asyncio
async def test_tar_create_and_list():
rb, wb, _, mk, store = _make_backend({"a.txt": b"alpha", "b.txt": b"beta"})
_, io_res = await tar([_spec("a.txt"), _spec("b.txt")],
read_bytes=rb,
write_bytes=wb,
mkdir_fn=mk,
c=True,
f=_spec("out.tar"))
assert "/out.tar" in io_res.writes
out, _ = await tar([],
read_bytes=rb,
write_bytes=wb,
mkdir_fn=mk,
t=True,
f=_spec("out.tar"))
assert b"a.txt" in out
assert b"b.txt" in out
@pytest.mark.asyncio
async def test_tar_extract():
rb, wb, _, mk, _ = _make_backend({"x.txt": b"data"})
await tar([_spec("x.txt")],
read_bytes=rb,
write_bytes=wb,
mkdir_fn=mk,
c=True,
f=_spec("a.tar"))
_, io_res = await tar([],
read_bytes=rb,
write_bytes=wb,
mkdir_fn=mk,
x=True,
f=_spec("a.tar"),
C=_spec("/out"))
assert any("x.txt" in p for p in io_res.writes)
@pytest.mark.asyncio
async def test_tar_requires_mode():
rb, wb, _, mk, _ = _make_backend({})
with pytest.raises(ValueError, match="-c, -x, or -t"):
await tar([], read_bytes=rb, write_bytes=wb, mkdir_fn=mk)
@pytest.mark.asyncio
async def test_tar_requires_f():
rb, wb, _, mk, _ = _make_backend({"a.txt": b"x"})
with pytest.raises(ValueError, match="-f is required"):
await tar([_spec("a.txt")],
read_bytes=rb,
write_bytes=wb,
mkdir_fn=mk,
c=True)
@pytest.mark.asyncio
async def test_diff_identical_files():
rb, _, _, _, _ = _make_backend({"a": b"hello\n", "b": b"hello\n"})
async def rd(accessor, path, index=None):
return []
out, io_res = await diff([_spec("a"), _spec("b")],
read_bytes=rb,
readdir_fn=rd)
assert out == b""
assert io_res.exit_code == 0
@pytest.mark.asyncio
async def test_diff_quiet_differ():
rb, _, _, _, _ = _make_backend({"a": b"x\n", "b": b"y\n"})
async def rd(accessor, path, index=None):
return []
out, io_res = await diff([_spec("a"), _spec("b")],
read_bytes=rb,
readdir_fn=rd,
q=True)
assert b"differ" in out
assert io_res.exit_code == 1
@pytest.mark.asyncio
async def test_diff_unified():
rb, _, _, _, _ = _make_backend({
"a": b"hello\nworld\n",
"b": b"hello\nuniverse\n"
})
async def rd(accessor, path, index=None):
return []
out, _ = await diff([_spec("a"), _spec("b")],
read_bytes=rb,
readdir_fn=rd,
u=True)
assert b"-world" in out
assert b"+universe" in out
@pytest.mark.asyncio
async def test_diff_too_few_paths():
rb, _, _, _, _ = _make_backend({"a": b"x\n"})
async def rd(accessor, path, index=None):
return []
with pytest.raises(ValueError, match="two paths"):
await diff([_spec("a")], read_bytes=rb, readdir_fn=rd)
@pytest.mark.asyncio
async def test_patch_apply():
diff_text = (b"--- a/hello.txt\n+++ b/hello.txt\n@@ -1,2 +1,2 @@\n"
b" hello\n-world\n+universe\n")
rb, wb, _, _, store = _make_backend({"/hello.txt": b"hello\nworld\n"})
_, io_res = await patch([],
read_bytes=rb,
write_bytes=wb,
has_resource=True,
stdin=diff_text,
p="1")
assert b"universe" in store["/hello.txt"]
assert "/hello.txt" in io_res.writes
@pytest.mark.asyncio
async def test_patch_reverse():
diff_text = (b"--- a/x.txt\n+++ b/x.txt\n@@ -1,2 +1,2 @@\n"
b" hello\n-world\n+universe\n")
rb, wb, _, _, store = _make_backend({"/x.txt": b"hello\nuniverse\n"})
await patch([],
read_bytes=rb,
write_bytes=wb,
has_resource=True,
stdin=diff_text,
R=True,
p="1")
assert b"world" in store["/x.txt"]
@pytest.mark.asyncio
async def test_patch_missing_input():
rb, wb, _, _, store = _make_backend({})
out, io = await patch([],
read_bytes=rb,
write_bytes=wb,
has_resource=False)
assert out is None
assert io.exit_code == 0
assert not store
@@ -0,0 +1,546 @@
import pytest
from mirage.commands.builtin.generic.rg import parse_flags, rg
from mirage.commands.spec.types import FlagView
from mirage.types import FileStat, FileType, PathSpec
from mirage.utils.key_prefix import mount_key
def _spec(path: str) -> PathSpec:
return PathSpec(resource_path=(path).strip("/"),
virtual=path,
directory=path,
resolved=True)
def _make_backend(files: dict[str, bytes], dirs: set[str] | None = None):
inferred_dirs = set(dirs) if dirs is not None else set()
for f in files:
parts = f.split("/")
for i in range(1, len(parts)):
d = "/".join(parts[:i]) or "/"
inferred_dirs.add(d)
inferred_dirs.add("/")
async def readdir(accessor, path, index=None):
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
p = spec.virtual.rstrip("/") or "/"
if p not in inferred_dirs:
raise FileNotFoundError(p)
prefix = p + "/" if p != "/" else "/"
children: set[str] = set()
for f in files:
if f.startswith(prefix):
rest = f[len(prefix):]
child = rest.split("/")[0]
children.add(prefix + child)
for d in inferred_dirs:
if d == p:
continue
if d.startswith(prefix):
rest = d[len(prefix):]
child = rest.split("/")[0]
children.add(prefix + child)
return sorted(children)
async def stat(accessor, path, index=None):
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
p = spec.virtual
if p in files:
return FileStat(name=p.rsplit("/", 1)[-1] or p,
size=len(files[p]),
type=FileType.TEXT)
if p.rstrip("/") in inferred_dirs or p in inferred_dirs:
return FileStat(name=p.rsplit("/", 1)[-1] or "/",
type=FileType.DIRECTORY)
raise FileNotFoundError(p)
async def read_bytes(accessor, path, index=None):
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
if spec.virtual not in files:
raise FileNotFoundError(spec.virtual)
return files[spec.virtual]
async def read_stream(accessor, path, index=None):
data = await read_bytes(accessor, path)
yield data
return readdir, stat, read_bytes, read_stream
async def _drain_async(stdout):
if stdout is None:
return b""
if isinstance(stdout, bytes):
return stdout
chunks = [chunk async for chunk in stdout]
return b"".join(chunks)
@pytest.mark.asyncio
async def test_rg_stdin_basic():
readdir, stat, rb, rs = _make_backend({})
output, _ = await rg(
[],
["apple"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
stdin=b"apple\nbanana\napricot\n",
)
decoded = (await _drain_async(output)).decode()
assert "apple" in decoded
assert "banana" not in decoded
@pytest.mark.asyncio
async def test_rg_count_stdin_uses_match_count():
readdir, stat, rb, rs = _make_backend({})
output, io = await rg(
[],
["foo"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
stdin=b"foo foo\nfoo bar\nbaz\n",
flags={"c": True},
)
assert (await _drain_async(output)) == b"2\n"
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_rg_count_stdin_zero_exits_1_without_output():
readdir, stat, rb, rs = _make_backend({})
output, io = await rg(
[],
["foo"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
stdin=b"bar\nbaz\n",
flags={"c": True},
)
assert await _drain_async(output) == b""
assert io.exit_code == 1
@pytest.mark.asyncio
async def test_rg_file_basic():
readdir, stat, rb, rs = _make_backend({
"/a.txt":
b"apple\nbanana\napricot\n",
})
output, _ = await rg(
[_spec("/a.txt")],
["ap"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
)
decoded = (await _drain_async(output)).decode()
assert "apple" in decoded
assert "apricot" in decoded
assert "banana" not in decoded
@pytest.mark.asyncio
async def test_rg_no_match_returns_exit_1():
readdir, stat, rb, rs = _make_backend({"/a.txt": b"hello\nworld\n"})
output, io = await rg(
[_spec("/a.txt")],
["zzz"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
)
drained = await _drain_async(output)
assert drained == b""
assert io.exit_code == 1
@pytest.mark.asyncio
async def test_rg_dir_auto_recursive():
"""rg on a bare directory should auto-recurse (matches real ripgrep)."""
readdir, stat, rb, rs = _make_backend({
"/dir/a.txt": b"apple\n",
"/dir/sub/b.txt": b"apricot\n",
})
output, _ = await rg(
[_spec("/dir")],
["ap"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
)
decoded = (await _drain_async(output)).decode()
assert "apple" in decoded
assert "apricot" in decoded
@pytest.mark.asyncio
async def test_rg_count_dir_skips_zero_count_files():
readdir, stat, rb, rs = _make_backend({
"/dir/a.txt": b"foo\nbar\nfoo\n",
"/dir/b.txt": b"bar\nbaz\n",
})
output, io = await rg(
[_spec("/dir")],
["foo"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={"c": True},
)
decoded = (await _drain_async(output)).decode()
assert "/dir/a.txt:2" in decoded
assert "/dir/b.txt" not in decoded
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_rg_files_only_on_dir():
readdir, stat, rb, rs = _make_backend({
"/dir/a.txt": b"apple\n",
"/dir/b.txt": b"zebra\n",
})
output, _ = await rg(
[_spec("/dir")],
["apple"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={"args_l": True},
)
decoded = (await _drain_async(output)).decode()
assert "/dir/a.txt" in decoded
assert "/dir/b.txt" not in decoded
@pytest.mark.asyncio
async def test_rg_hidden_excluded_by_default():
readdir, stat, rb, rs = _make_backend({
"/dir/.hidden": b"apple\n",
"/dir/visible.txt": b"apple\n",
})
output, _ = await rg(
[_spec("/dir")],
["apple"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={"args_l": True},
)
decoded = (await _drain_async(output)).decode()
assert "/dir/visible.txt" in decoded
assert ".hidden" not in decoded
@pytest.mark.asyncio
async def test_rg_hidden_included_with_flag():
readdir, stat, rb, rs = _make_backend({
"/dir/.hidden": b"apple\n",
"/dir/visible.txt": b"apple\n",
})
output, _ = await rg(
[_spec("/dir")],
["apple"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={
"args_l": True,
"hidden": True
},
)
decoded = (await _drain_async(output)).decode()
assert "/dir/visible.txt" in decoded
assert ".hidden" in decoded
@pytest.mark.asyncio
async def test_rg_file_type_filter():
readdir, stat, rb, rs = _make_backend({
"/dir/a.py": b"apple\n",
"/dir/b.txt": b"apple\n",
})
output, _ = await rg(
[_spec("/dir")],
["apple"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={
"args_l": True,
"type": "py"
},
)
decoded = (await _drain_async(output)).decode()
assert "/dir/a.py" in decoded
assert "/dir/b.txt" not in decoded
@pytest.mark.asyncio
async def test_rg_glob_filter():
readdir, stat, rb, rs = _make_backend({
"/dir/a.log": b"apple\n",
"/dir/b.txt": b"apple\n",
})
output, _ = await rg(
[_spec("/dir")],
["apple"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={
"args_l": True,
"glob": "*.log"
},
)
decoded = (await _drain_async(output)).decode()
assert "/dir/a.log" in decoded
assert "/dir/b.txt" not in decoded
def _make_prefixed_backend(files: dict[str, bytes], mount_prefix: str):
"""Backend that mimics real s3/disk/gdrive readdir: entries returned
are already prepended with ``mount_prefix``. Used to catch wrapper bugs
that re-add the prefix or drop ``index``."""
full_files = {mount_prefix + k: v for k, v in files.items()}
inferred_dirs: set[str] = {mount_prefix or "/"}
for f in full_files:
parts = f.split("/")
for i in range(1, len(parts)):
d = "/".join(parts[:i]) or "/"
inferred_dirs.add(d)
def _full(p: str) -> str:
if mount_prefix and not p.startswith(mount_prefix):
return mount_prefix + p
return p
async def readdir(accessor, path, index=None):
if index is None:
raise FileNotFoundError("index required")
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
p = _full(spec.virtual).rstrip("/") or "/"
if p not in inferred_dirs:
raise FileNotFoundError(p)
prefix = p + "/" if p != "/" else "/"
children: set[str] = set()
for f in full_files:
if f.startswith(prefix):
child = prefix + f[len(prefix):].split("/")[0]
children.add(child)
for d in inferred_dirs:
if d == p or not d.startswith(prefix):
continue
child = prefix + d[len(prefix):].split("/")[0]
children.add(child)
return sorted(children)
async def stat(accessor, path, index=None):
if index is None:
raise FileNotFoundError("index required")
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
p = _full(spec.virtual)
if p in full_files:
return FileStat(name=p.rsplit("/", 1)[-1],
size=len(full_files[p]),
type=FileType.TEXT)
if p.rstrip("/") in inferred_dirs:
return FileStat(name=p.rsplit("/", 1)[-1] or "/",
type=FileType.DIRECTORY)
raise FileNotFoundError(p)
async def read_bytes(accessor, path, index=None):
if index is None:
raise FileNotFoundError("index required")
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
p = _full(spec.virtual)
if p not in full_files:
raise FileNotFoundError(p)
return full_files[p]
return readdir, stat, read_bytes
@pytest.mark.asyncio
async def test_rg_files_only_mount_prefix_not_doubled():
readdir, stat, rb = _make_prefixed_backend(
{
"/dir/a.txt": b"apple\n",
"/dir/b.txt": b"zebra\n",
},
mount_prefix="/s3",
)
p = PathSpec(resource_path=mount_key("/dir", "/s3"),
virtual="/dir",
directory="/dir",
resolved=True)
output, _ = await rg(
[p],
["apple"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=None,
flags={"args_l": True},
index=object(),
)
decoded = (await _drain_async(output)).decode().strip()
assert decoded == "/s3/dir/a.txt"
assert "/s3/s3" not in decoded
@pytest.mark.asyncio
async def test_rg_single_file_threads_index():
readdir, stat, rb = _make_prefixed_backend(
{"/dir/a.txt": b"apple\n"},
mount_prefix="/gd",
)
p = PathSpec(resource_path=mount_key("/dir/a.txt", "/gd"),
virtual="/dir/a.txt",
directory="/dir/a.txt",
resolved=True)
output, _ = await rg(
[p],
["apple"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=None,
index=object(),
)
decoded = (await _drain_async(output)).decode()
assert "apple" in decoded
@pytest.mark.asyncio
async def test_rg_multiple_dirs_searches_all():
readdir, stat, rb, rs = _make_backend({
"/d1/a.txt": b"apple a\n",
"/d2/b.txt": b"apple b\n",
})
output, _ = await rg(
[_spec("/d1"), _spec("/d2")],
["apple"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
)
decoded = (await _drain_async(output)).decode()
assert "/d1/a.txt:apple a" in decoded
assert "/d2/b.txt:apple b" in decoded
@pytest.mark.asyncio
async def test_rg_files_only_multiple_files():
readdir, stat, rb, rs = _make_backend({
"/t1.txt": b"apple\n",
"/t2.txt": b"apple\n",
})
output, _ = await rg(
[_spec("/t1.txt"), _spec("/t2.txt")],
["apple"],
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
flags={"args_l": True},
)
decoded = (await _drain_async(output)).decode()
assert "/t1.txt" in decoded
assert "/t2.txt" in decoded
def test_parse_flags_c_overrides_a_and_b():
f = parse_flags(FlagView({
"A": "2",
"B": "1",
"C": "4"
}),
never_match=False)
assert f.context_after == 4
assert f.context_before == 4
f = parse_flags(FlagView({"A": "2"}), never_match=False)
assert f.context_after == 2
assert f.context_before == 0
def test_parse_flags_struct_rejects_typos():
f = parse_flags(FlagView({"hidden": True}), never_match=False)
assert f.hidden is True
with pytest.raises(AttributeError):
_ = f.hiden
@pytest.mark.asyncio
async def test_rg_with_filename_labels_single_file():
readdir, stat, rb, rs = _make_backend({"/a.txt": b"apple\nbanana\n"})
output, _ = await rg(
[_spec("/a.txt")],
["ap"],
{"H": True},
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
)
assert (await _drain_async(output)).decode() == "/a.txt:apple\n"
@pytest.mark.asyncio
async def test_rg_with_filename_labels_single_file_count():
readdir, stat, rb, rs = _make_backend({"/a.txt": b"apple\napricot\n"})
output, _ = await rg(
[_spec("/a.txt")],
["ap"],
{
"H": True,
"c": True
},
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
)
assert (await _drain_async(output)).decode() == "/a.txt:2\n"
@pytest.mark.asyncio
async def test_rg_no_filename_suppresses_multi_file_labels():
readdir, stat, rb, rs = _make_backend({
"/a.txt": b"apple\n",
"/b.txt": b"apricot\n",
})
output, _ = await rg(
[_spec("/a.txt"), _spec("/b.txt")],
["ap"],
{"args_I": True},
readdir=readdir,
stat=stat,
read_bytes=rb,
read_stream=rs,
)
assert (await _drain_async(output)).decode() == "apple\napricot\n"
@@ -0,0 +1,420 @@
import pytest
from mirage.commands.builtin.generic.sed import sed
from mirage.types import PathSpec
def _spec(path: str) -> PathSpec:
return PathSpec(resource_path=(path).strip("/"),
virtual=path,
directory=path,
resolved=True)
def _make_backend(files: dict[str, bytes]):
store = dict(files)
async def read_bytes(accessor, path, index=None):
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
if spec.virtual not in store:
raise FileNotFoundError(spec.virtual)
return store[spec.virtual]
async def write_bytes(accessor, path, data, index=None):
spec = path if isinstance(path, PathSpec) else PathSpec(
resource_path=(path).strip("/"), virtual=path, directory=path)
store[spec.virtual] = data
return read_bytes, write_bytes, store
@pytest.mark.asyncio
async def test_sed_stdin_simple_sub():
rb, wb, _ = _make_backend({})
output, _ = await sed(
[],
"s/hello/bye/",
read_bytes=rb,
write_bytes=wb,
stdin=b"hello world\n",
)
assert output == b"bye world\n"
@pytest.mark.asyncio
async def test_sed_file_simple_sub_emits_output():
rb, wb, _ = _make_backend({"/a.txt": b"hello world\n"})
output, _ = await sed(
[_spec("/a.txt")],
"s/hello/bye/",
read_bytes=rb,
write_bytes=wb,
)
assert output == b"bye world\n"
@pytest.mark.asyncio
async def test_sed_inplace_simple_sub_writes_file():
rb, wb, store = _make_backend({"/a.txt": b"hello world\n"})
output, io = await sed(
[_spec("/a.txt")],
"s/hello/bye/",
read_bytes=rb,
write_bytes=wb,
in_place=True,
)
assert output is None
assert store["/a.txt"] == b"bye world\n"
assert io.writes == {"/a.txt": b"bye world\n"}
@pytest.mark.asyncio
async def test_sed_inplace_multi_path_writes_all():
rb, wb, store = _make_backend({
"/a.txt": b"hello a\n",
"/b.txt": b"hello b\n",
})
_output, io = await sed(
[_spec("/a.txt"), _spec("/b.txt")],
"s/hello/bye/",
read_bytes=rb,
write_bytes=wb,
in_place=True,
)
assert store["/a.txt"] == b"bye a\n"
assert store["/b.txt"] == b"bye b\n"
assert set(io.writes.keys()) == {"/a.txt", "/b.txt"}
@pytest.mark.asyncio
async def test_sed_global_flag_replaces_all():
rb, wb, _ = _make_backend({})
output, _ = await sed(
[],
"s/a/X/g",
read_bytes=rb,
write_bytes=wb,
stdin=b"banana\n",
)
assert output == b"bXnXnX\n"
@pytest.mark.asyncio
async def test_sed_first_match_only_by_default():
rb, wb, _ = _make_backend({})
output, _ = await sed(
[],
"s/a/X/",
read_bytes=rb,
write_bytes=wb,
stdin=b"banana\n",
)
assert output == b"bXnana\n"
@pytest.mark.asyncio
async def test_sed_delete_program():
"""Delete command 'd' should drop matching lines."""
rb, wb, _ = _make_backend({})
output, _ = await sed(
[],
"/skip/d",
read_bytes=rb,
write_bytes=wb,
stdin=b"keep\nskip me\nkeep too\n",
)
decoded = output.decode()
assert "keep" in decoded
assert "skip me" not in decoded
@pytest.mark.asyncio
async def test_sed_n_suppress_with_p():
"""-n suppresses default output; only explicit 'p' prints."""
rb, wb, _ = _make_backend({})
output, _ = await sed(
[],
"/match/p",
read_bytes=rb,
write_bytes=wb,
stdin=b"no\nmatch line\nno\n",
suppress=True,
)
decoded = output.decode()
assert "match line" in decoded
@pytest.mark.asyncio
async def test_sed_no_paths_no_stdin_raises():
rb, wb, _ = _make_backend({})
with pytest.raises(ValueError, match="usage"):
await sed([], "s/a/b/", read_bytes=rb, write_bytes=wb)
@pytest.mark.asyncio
async def test_sed_numeric_count_replaces_nth_occurrence():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"s/o/O/2",
read_bytes=rb,
write_bytes=wb,
stdin=b"oooo\n")
assert output == b"oOoo\n"
@pytest.mark.asyncio
async def test_sed_numeric_count_with_g_replaces_nth_onward():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"s/o/O/2g",
read_bytes=rb,
write_bytes=wb,
stdin=b"oooo\n")
assert output == b"oOOO\n"
@pytest.mark.asyncio
async def test_sed_count_is_per_line():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"s/o/O/2",
read_bytes=rb,
write_bytes=wb,
stdin=b"oo\noo\n")
assert output == b"oO\noO\n"
@pytest.mark.asyncio
async def test_sed_p_flag_prints_substituted_line_twice():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"s/hi/HI/p",
read_bytes=rb,
write_bytes=wb,
stdin=b"hi\nbye\n")
assert output == b"HI\nHI\nbye\n"
@pytest.mark.asyncio
async def test_sed_p_flag_under_suppress_prints_only_substituted():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"s/hi/HI/p",
read_bytes=rb,
write_bytes=wb,
stdin=b"hi\nbye\n",
suppress=True)
assert output == b"HI\n"
@pytest.mark.asyncio
async def test_sed_y_transliterate():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"y/el/ip/",
read_bytes=rb,
write_bytes=wb,
stdin=b"hello\n")
assert output == b"hippo\n"
@pytest.mark.asyncio
async def test_sed_y_mismatched_lengths_raises():
rb, wb, _ = _make_backend({})
with pytest.raises(ValueError, match="different lengths"):
await sed([], "y/ab/x/", read_bytes=rb, write_bytes=wb, stdin=b"a\n")
@pytest.mark.asyncio
async def test_sed_c_no_address_changes_every_line():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"c\\\nX",
read_bytes=rb,
write_bytes=wb,
stdin=b"a\nb\nc\n")
assert output == b"X\nX\nX\n"
@pytest.mark.asyncio
async def test_sed_c_single_address():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"2c\\\nX",
read_bytes=rb,
write_bytes=wb,
stdin=b"a\nb\nc\n")
assert output == b"a\nX\nc\n"
@pytest.mark.asyncio
async def test_sed_c_range_emits_once():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"2,3c\\\nX",
read_bytes=rb,
write_bytes=wb,
stdin=b"a\nb\nc\nd\n")
assert output == b"a\nX\nd\n"
@pytest.mark.asyncio
async def test_sed_bre_group_and_backref():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
r"s/\(foo\)/[\1]/",
read_bytes=rb,
write_bytes=wb,
stdin=b"foo\n")
assert output == b"[foo]\n"
@pytest.mark.asyncio
async def test_sed_bre_plus_is_literal():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"s/a+/X/",
read_bytes=rb,
write_bytes=wb,
stdin=b"a+b\n")
assert output == b"Xb\n"
@pytest.mark.asyncio
async def test_sed_bre_backslash_plus_is_one_or_more():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
r"s/a\+/X/",
read_bytes=rb,
write_bytes=wb,
stdin=b"aaab\n")
assert output == b"Xb\n"
@pytest.mark.asyncio
async def test_sed_ere_group_and_plus():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
r"s/(foo)/[\1]/",
read_bytes=rb,
write_bytes=wb,
stdin=b"foo\n",
extended=True)
assert output == b"[foo]\n"
output, _ = await sed([],
"s/a+/X/",
read_bytes=rb,
write_bytes=wb,
stdin=b"aaab\n",
extended=True)
assert output == b"Xb\n"
@pytest.mark.asyncio
async def test_sed_ere_address():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"/a+/d",
read_bytes=rb,
write_bytes=wb,
stdin=b"aaa\nbbb\n",
extended=True)
assert output == b"bbb\n"
@pytest.mark.asyncio
async def test_sed_negate_line():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"2!d",
read_bytes=rb,
write_bytes=wb,
stdin=b"a\nb\nc\n")
assert output == b"b\n"
@pytest.mark.asyncio
async def test_sed_negate_regex():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"/b/!d",
read_bytes=rb,
write_bytes=wb,
stdin=b"a\nb\nc\n")
assert output == b"b\n"
@pytest.mark.asyncio
async def test_sed_negate_last_with_suppress():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"$!p",
read_bytes=rb,
write_bytes=wb,
stdin=b"a\nb\nc\n",
suppress=True)
assert output == b"a\nb\n"
@pytest.mark.asyncio
async def test_sed_negate_range():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"1,2!s/./X/",
read_bytes=rb,
write_bytes=wb,
stdin=b"a\nb\nc\nd\n")
assert output == b"a\nb\nX\nX\n"
@pytest.mark.asyncio
async def test_sed_join_all_idiom():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
r":a;N;$!ba;s/\n/,/g",
read_bytes=rb,
write_bytes=wb,
stdin=b"a\nb\nc\n")
assert output == b"a,b,c\n"
@pytest.mark.asyncio
async def test_sed_hold_accumulate():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"H;${x;p}",
read_bytes=rb,
write_bytes=wb,
stdin=b"a\nb\n",
suppress=True)
assert output == b"\na\nb\n"
@pytest.mark.asyncio
async def test_sed_preserves_missing_final_newline():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
"s/o/O/",
read_bytes=rb,
write_bytes=wb,
stdin=b"foo")
assert output == b"fOo"
@pytest.mark.asyncio
async def test_sed_escaped_delimiter():
rb, wb, _ = _make_backend({})
output, _ = await sed([],
r"s/a\/b/c/",
read_bytes=rb,
write_bytes=wb,
stdin=b"a/b\n")
assert output == b"c\n"
@pytest.mark.asyncio
async def test_sed_zero_count_rejected():
rb, wb, _ = _make_backend({})
with pytest.raises(ValueError, match="may not be zero"):
await sed([], "s/o/O/0", read_bytes=rb, write_bytes=wb, stdin=b"oo\n")
@@ -0,0 +1,240 @@
import pytest
from mirage.commands.builtin.generic.tail import tail
async def _drain(gen):
return b"".join([c async for c in gen])
@pytest.mark.asyncio
async def test_tail_default_n_10():
body = b"\n".join(f"line{i}".encode() for i in range(1, 21)) + b"\n"
out = await _drain(tail(body))
expected = b"\n".join(f"line{i}".encode() for i in range(11, 21)) + b"\n"
assert out == expected
@pytest.mark.asyncio
async def test_tail_n_explicit():
out = await _drain(tail(b"a\nb\nc\nd\ne\n", n=3))
assert out == b"c\nd\ne\n"
@pytest.mark.asyncio
async def test_tail_n_1():
body = b"line1\nline2\nline3\n"
out = await _drain(tail(body, n=1))
assert out == b"line3\n"
@pytest.mark.asyncio
async def test_tail_n_larger_than_total():
out = await _drain(tail(b"a\nb\nc", n=100))
assert out == b"a\nb\nc"
@pytest.mark.asyncio
async def test_tail_n_zero_emits_nothing():
out = await _drain(tail(b"a\nb\nc\n", n=0))
assert out == b""
@pytest.mark.asyncio
async def test_tail_negative_n_treated_as_abs():
"""GNU/POSIX `tail -n -3` is the same as `tail -n 3` (last 3 lines)."""
out = await _drain(tail(b"a\nb\nc\nd\ne\n", n=-3))
assert out == b"c\nd\ne\n"
@pytest.mark.asyncio
async def test_tail_n_no_trailing_newline_in_last_line():
out = await _drain(tail(b"a\nb\nc", n=2))
assert out == b"b\nc"
@pytest.mark.asyncio
async def test_tail_single_line_no_newline_default():
out = await _drain(tail(b"hello"))
assert out == b"hello"
@pytest.mark.asyncio
async def test_tail_empty_input():
out = await _drain(tail(b""))
assert out == b""
@pytest.mark.asyncio
async def test_tail_only_newlines():
"""Tail of \\n\\n\\n with n=2 keeps the last two blank lines."""
out = await _drain(tail(b"\n\n\n", n=2))
assert out == b"\n\n"
@pytest.mark.asyncio
async def test_tail_n_from_stream():
async def src():
yield b"a\nb"
yield b"\nc\nd\n"
out = await _drain(tail(src(), n=2))
assert out == b"c\nd\n"
@pytest.mark.asyncio
async def test_tail_n_chunked_one_byte_at_a_time():
async def src():
for byte in b"a\nbb\nccc\n":
yield bytes([byte])
out = await _drain(tail(src(), n=2))
assert out == b"bb\nccc\n"
@pytest.mark.asyncio
async def test_tail_c_bytes_fast_path():
out = await _drain(tail(b"hello world", c=5))
assert out == b"world"
@pytest.mark.asyncio
async def test_tail_c_bytes_across_chunks():
async def src():
yield b"hel"
yield b"lo wor"
yield b"ld"
out = await _drain(tail(src(), c=5))
assert out == b"world"
@pytest.mark.asyncio
async def test_tail_c_chunked_one_byte_at_a_time():
"""Worst-case chunking for -c: rolling window must hold last N bytes."""
async def src():
for byte in b"hello world":
yield bytes([byte])
out = await _drain(tail(src(), c=5))
assert out == b"world"
@pytest.mark.asyncio
async def test_tail_c_larger_than_total():
out = await _drain(tail(b"abc", c=100))
assert out == b"abc"
@pytest.mark.asyncio
async def test_tail_c_zero_emits_nothing():
out = await _drain(tail(b"abc", c=0))
assert out == b""
@pytest.mark.asyncio
async def test_tail_c_negative_is_last_abs():
out = await _drain(tail(b"hello", c=-3))
assert out == b"llo"
@pytest.mark.asyncio
async def test_tail_c_empty_input():
out = await _drain(tail(b"", c=10))
assert out == b""
@pytest.mark.asyncio
async def test_tail_c_binary_preserves_full_byte_range():
"""-c must not corrupt binary bytes."""
data = bytes(range(256))
out = await _drain(tail(data, c=10))
assert out == data[-10:]
@pytest.mark.asyncio
async def test_tail_from_line_streaming():
"""`tail -n +3` emits lines 3 onwards."""
out = await _drain(tail(b"a\nb\nc\nd\ne\n", from_line=3))
assert out == b"c\nd\ne\n"
@pytest.mark.asyncio
async def test_tail_from_line_1_is_passthrough():
out = await _drain(tail(b"a\nb\nc\n", from_line=1))
assert out == b"a\nb\nc\n"
@pytest.mark.asyncio
async def test_tail_from_line_0_treated_as_1():
"""GNU `tail -n +0` is documented as `+1` (start from line 1)."""
out = await _drain(tail(b"a\nb\nc\n", from_line=0))
assert out == b"a\nb\nc\n"
@pytest.mark.asyncio
async def test_tail_from_line_past_end_emits_nothing():
out = await _drain(tail(b"a\nb\n", from_line=10))
assert out == b""
@pytest.mark.asyncio
async def test_tail_from_line_across_chunks():
"""`from_line` skip must work across arbitrary chunk boundaries."""
async def src():
yield b"a\nb"
yield b"\nc\nd"
yield b"\ne\n"
out = await _drain(tail(src(), from_line=3))
assert out == b"c\nd\ne\n"
@pytest.mark.asyncio
async def test_tail_from_line_chunked_one_byte_at_a_time():
async def src():
for byte in b"a\nb\nc\nd\n":
yield bytes([byte])
out = await _drain(tail(src(), from_line=3))
assert out == b"c\nd\n"
@pytest.mark.asyncio
async def test_tail_from_line_preserves_binary_payload():
data = b"\x00\x01\n\x02\x03\n\x04\x05\n"
out = await _drain(tail(data, from_line=2))
assert out == b"\x02\x03\n\x04\x05\n"
@pytest.mark.asyncio
async def test_tail_from_line_1_preserves_stream_chunks():
"""`from_line=1` is passthrough: chunk boundaries must survive intact."""
async def src():
yield b"hel"
yield b"lo\nwo"
yield b"rld\n"
chunks = [c async for c in tail(src(), from_line=1)]
assert chunks == [b"hel", b"lo\nwo", b"rld\n"]
@pytest.mark.asyncio
async def test_tail_from_line_emits_chunks_incrementally_after_skip():
"""Once the skip threshold is crossed, subsequent chunks pass through
untouched (no rebuffering)."""
async def src():
yield b"a\nb\nc"
yield b"\nd\n"
yield b"e\n"
chunks = [c async for c in tail(src(), from_line=3)]
assert chunks == [b"c", b"\nd\n", b"e\n"]
@@ -0,0 +1,76 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage.commands.builtin.generic.tail import tail_multi
from mirage.types import PathSpec
from mirage.utils.key_prefix import mount_key
def _paths(*names: str) -> list[PathSpec]:
return [
PathSpec(resource_path=mount_key(n, ""),
virtual=n,
directory="/d",
resolved=True) for n in names
]
async def _collect(gen) -> bytes:
out = b""
async for chunk in gen:
out += chunk
return out
@pytest.mark.asyncio
async def test_tail_multi_bytes_reader_no_headers():
data = {"/a": b"a1\na2\na3\n", "/b": b"b1\nb2\n"}
async def read(accessor, p, index):
return data[p.virtual]
out = await _collect(
tail_multi(_paths("/a", "/b"), read=read, n=1, show_headers=False))
assert out == b"a3\nb2\n"
@pytest.mark.asyncio
async def test_tail_multi_with_headers():
data = {"/a": b"a1\na2\n", "/b": b"b1\nb2\n"}
async def read(accessor, p, index):
return data[p.virtual]
out = await _collect(
tail_multi(_paths("/a", "/b"), read=read, n=1, show_headers=True))
assert out == b"==> /a <==\na2\n\n==> /b <==\nb2\n"
@pytest.mark.asyncio
async def test_tail_multi_stream_reader():
chunks = {"/a": [b"a1\n", b"a2\n"], "/b": [b"b1\n"]}
def read(accessor, p, index):
async def gen():
for ch in chunks[p.virtual]:
yield ch
return gen()
out = await _collect(
tail_multi(_paths("/a", "/b"), read=read, n=5, show_headers=True))
assert out == b"==> /a <==\na1\na2\n\n==> /b <==\nb1\n"
@@ -0,0 +1,209 @@
import pytest
from mirage.commands.builtin.generic.tree import tree
from mirage.types import FileStat, FileType, PathSpec
def _spec(path: str) -> PathSpec:
return PathSpec(virtual=path,
directory=path,
resource_path=path.strip("/"))
def _file(name: str, size: int = 0) -> FileStat:
return FileStat(name=name, size=size, type=FileType.TEXT)
def _dir(name: str) -> FileStat:
return FileStat(name=name, size=None, type=FileType.DIRECTORY)
def _make_backend(tree_map: dict[str, FileStat]):
async def stat(p: PathSpec, index=None) -> FileStat:
if p.virtual not in tree_map:
raise FileNotFoundError(p.virtual)
return tree_map[p.virtual]
async def readdir(p: PathSpec, _index=None) -> list[str]:
if p.virtual not in tree_map:
raise FileNotFoundError(p.virtual)
if tree_map[p.virtual].type != FileType.DIRECTORY:
raise ValueError(f"not a directory: {p.virtual}")
prefix = p.virtual.rstrip("/") + "/"
children: list[str] = []
for key in tree_map:
if key == p.virtual:
continue
if key.startswith(prefix):
remainder = key[len(prefix):]
if "/" not in remainder:
children.append(key)
return sorted(children)
return readdir, stat
@pytest.mark.asyncio
async def test_tree_flat_dir():
"""Two siblings: the last gets `└──`, the others get `├──`."""
tree_map = {
"/r": _dir("r"),
"/r/a.txt": _file("a.txt"),
"/r/b.txt": _file("b.txt"),
}
readdir, stat = _make_backend(tree_map)
output, io = await tree(_spec("/r"), readdir=readdir, stat=stat)
lines = output.decode().splitlines()
assert lines == ["├── a.txt", "└── b.txt"]
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_tree_nested_dir_uses_vertical_continuation():
"""A non-last directory should continue its children with `│ `."""
tree_map = {
"/r": _dir("r"),
"/r/d1": _dir("d1"),
"/r/d1/x.txt": _file("x.txt"),
"/r/z.txt": _file("z.txt"),
}
readdir, stat = _make_backend(tree_map)
output, _ = await tree(_spec("/r"), readdir=readdir, stat=stat)
lines = output.decode().splitlines()
assert lines == ["├── d1", "│ └── x.txt", "└── z.txt"]
@pytest.mark.asyncio
async def test_tree_last_dir_uses_indent_continuation():
"""A last directory should continue with plain spaces, no vertical bar."""
tree_map = {
"/r": _dir("r"),
"/r/d1": _dir("d1"),
"/r/d1/x.txt": _file("x.txt"),
}
readdir, stat = _make_backend(tree_map)
output, _ = await tree(_spec("/r"), readdir=readdir, stat=stat)
lines = output.decode().splitlines()
assert lines == ["└── d1", " └── x.txt"]
@pytest.mark.asyncio
async def test_tree_max_depth_limits_recursion():
tree_map = {
"/r": _dir("r"),
"/r/d1": _dir("d1"),
"/r/d1/d2": _dir("d2"),
"/r/d1/d2/deep.txt": _file("deep.txt"),
}
readdir, stat = _make_backend(tree_map)
output, _ = await tree(_spec("/r"),
readdir=readdir,
stat=stat,
max_depth=1)
decoded = output.decode()
assert "d1" in decoded
assert "d2" in decoded
assert "deep.txt" not in decoded
@pytest.mark.asyncio
async def test_tree_hides_dotfiles_by_default():
tree_map = {
"/r": _dir("r"),
"/r/.hidden": _file(".hidden"),
"/r/visible.txt": _file("visible.txt"),
}
readdir, stat = _make_backend(tree_map)
output, _ = await tree(_spec("/r"), readdir=readdir, stat=stat)
decoded = output.decode()
assert ".hidden" not in decoded
assert "visible.txt" in decoded
@pytest.mark.asyncio
async def test_tree_show_hidden_includes_dotfiles():
tree_map = {
"/r": _dir("r"),
"/r/.hidden": _file(".hidden"),
"/r/visible.txt": _file("visible.txt"),
}
readdir, stat = _make_backend(tree_map)
output, _ = await tree(_spec("/r"),
readdir=readdir,
stat=stat,
show_hidden=True)
assert ".hidden" in output.decode()
@pytest.mark.asyncio
async def test_tree_ignore_pattern_drops_matches():
tree_map = {
"/r": _dir("r"),
"/r/a.pyc": _file("a.pyc"),
"/r/b.py": _file("b.py"),
}
readdir, stat = _make_backend(tree_map)
output, _ = await tree(_spec("/r"),
readdir=readdir,
stat=stat,
ignore_pattern="*.pyc")
decoded = output.decode()
assert "a.pyc" not in decoded
assert "b.py" in decoded
@pytest.mark.asyncio
async def test_tree_dirs_only_drops_files():
tree_map = {
"/r": _dir("r"),
"/r/d1": _dir("d1"),
"/r/a.txt": _file("a.txt"),
}
readdir, stat = _make_backend(tree_map)
output, _ = await tree(_spec("/r"),
readdir=readdir,
stat=stat,
dirs_only=True)
decoded = output.decode()
assert "d1" in decoded
assert "a.txt" not in decoded
@pytest.mark.asyncio
async def test_tree_match_pattern_only_applies_to_files():
"""`-P` filters file names but never excludes directories."""
tree_map = {
"/r": _dir("r"),
"/r/d1": _dir("d1"),
"/r/d1/match.py": _file("match.py"),
"/r/d1/skip.txt": _file("skip.txt"),
"/r/top.py": _file("top.py"),
}
readdir, stat = _make_backend(tree_map)
output, _ = await tree(_spec("/r"),
readdir=readdir,
stat=stat,
match_pattern="*.py")
decoded = output.decode()
assert "d1" in decoded
assert "match.py" in decoded
assert "skip.txt" not in decoded
assert "top.py" in decoded
@pytest.mark.asyncio
async def test_tree_missing_path_emits_warning_not_crash():
readdir, stat = _make_backend({})
output, io = await tree(_spec("/nowhere"), readdir=readdir, stat=stat)
assert output == b""
assert b"nowhere" in (io.stderr or b"")
@pytest.mark.asyncio
async def test_tree_empty_dir_emits_nothing():
tree_map = {"/r": _dir("r")}
readdir, stat = _make_backend(tree_map)
output, io = await tree(_spec("/r"), readdir=readdir, stat=stat)
assert output == b""
assert io.exit_code == 0
@@ -0,0 +1,106 @@
import pytest
from mirage.commands.builtin.generic.uniq import _parse_count, uniq
from mirage.types import PathSpec
def _unused_read_stream(_accessor, _path):
raise AssertionError("read_stream should not be called for stdin input")
async def _generator_read_stream(_accessor, _path):
yield b"dup\ndup\nsolo\n"
async def _collect(stdin: bytes, **kwargs) -> bytes:
source, _io = await uniq(
[],
read_stream=_unused_read_stream,
stdin=stdin,
**kwargs,
)
chunks = [chunk async for chunk in source]
return b"".join(chunks)
def test_parse_count_unset_is_none():
assert _parse_count(None) is None
def test_parse_count_zero_string():
assert _parse_count("0") == 0
def test_parse_count_positive():
assert _parse_count("3") == 3
def test_parse_count_negative_raises():
with pytest.raises(ValueError, match="invalid count"):
_parse_count("-1")
@pytest.mark.asyncio
async def test_skip_fields_unset_matches_zero():
data = b"a one\nb one\n"
unset = await _collect(data)
zero = await _collect(data, skip_fields="0")
assert unset == zero == b"a one\nb one\n"
@pytest.mark.asyncio
async def test_skip_fields_collapses_on_second_field():
data = b"a shared\nb shared\n"
out = await _collect(data, skip_fields="1")
assert out == b"a shared\n"
@pytest.mark.asyncio
async def test_skip_chars_offset():
data = b"Xfoo\nYfoo\n"
out = await _collect(data, skip_chars="1")
assert out == b"Xfoo\n"
@pytest.mark.asyncio
async def test_check_chars_limits_comparison():
data = b"abcAAA\nabcBBB\n"
out = await _collect(data, check_chars="3")
assert out == b"abcAAA\n"
@pytest.mark.asyncio
async def test_check_chars_unset_compares_full_line():
data = b"abcAAA\nabcBBB\n"
out = await _collect(data)
assert out == b"abcAAA\nabcBBB\n"
@pytest.mark.asyncio
async def test_check_chars_zero_string_treats_all_lines_as_duplicates():
# GNU: -w 0 compares zero characters, so every line matches the first
data = b"abcAAA\nabcBBB\n"
out = await _collect(data, check_chars="0")
assert out == b"abcAAA\n"
@pytest.mark.asyncio
async def test_count_prefixes_occurrences():
data = b"dup\ndup\nsolo\n"
out = await _collect(data, count=True)
assert out == b" 2 dup\n 1 solo\n"
@pytest.mark.asyncio
async def test_ignore_case_folds_duplicates():
data = b"Hello\nhello\n"
out = await _collect(data, ignore_case=True)
assert out == b"Hello\n"
@pytest.mark.asyncio
async def test_read_stream_async_generator():
p = PathSpec(resource_path="x", virtual="/x", directory="/x")
source, _io = await uniq([p], read_stream=_generator_read_stream)
out = b"".join([chunk async for chunk in source])
assert out == b"dup\nsolo\n"
@@ -0,0 +1,342 @@
import pytest
from mirage.commands.builtin.generic.wc import (WCCounts, format_multi,
format_wc, wc, wc_lines)
from mirage.types import PathSpec
@pytest.mark.asyncio
async def test_wc_default_counts_bytes():
counts = await wc(b"hello world\nfoo bar\n")
assert counts.lines == 2
assert counts.words == 4
assert counts.bytes_ == 20
assert counts.chars == 20
@pytest.mark.asyncio
async def test_wc_empty_input():
counts = await wc(b"")
assert counts == WCCounts(lines=0,
words=0,
bytes_=0,
chars=0,
max_line_length=0)
@pytest.mark.asyncio
async def test_wc_lines_with_trailing_newline():
counts = await wc(b"a\nb\nc\n")
assert counts.lines == 3
@pytest.mark.asyncio
async def test_wc_lines_without_trailing_newline():
"""POSIX: lines = number of \\n bytes. `a\\nb\\nc` has 2 newlines."""
counts = await wc(b"a\nb\nc")
assert counts.lines == 2
@pytest.mark.asyncio
async def test_wc_words_single_line():
counts = await wc(b"one two three")
assert counts.words == 3
@pytest.mark.asyncio
async def test_wc_words_multiline():
counts = await wc(b"one two\nthree four five\nsix\n")
assert counts.words == 6
@pytest.mark.asyncio
async def test_wc_words_leading_trailing_whitespace():
"""POSIX: leading/trailing whitespace doesn't add words."""
counts = await wc(b" hello world ")
assert counts.words == 2
@pytest.mark.asyncio
async def test_wc_words_only_whitespace():
counts = await wc(b" \t \n ")
assert counts.words == 0
@pytest.mark.asyncio
async def test_wc_bytes_ascii():
counts = await wc(b"hello")
assert counts.bytes_ == 5
assert counts.chars == 5
@pytest.mark.asyncio
async def test_wc_chars_vs_bytes_multibyte_utf8():
"""`café` is 4 chars / 5 bytes (é is 2 bytes in UTF-8)."""
data = "café".encode()
counts = await wc(data)
assert counts.bytes_ == 5
assert counts.chars == 4
@pytest.mark.asyncio
async def test_wc_chars_vs_bytes_pure_multibyte():
data = "ééé".encode()
counts = await wc(data)
assert counts.bytes_ == 6
assert counts.chars == 3
@pytest.mark.asyncio
async def test_wc_max_line_length_empty():
counts = await wc(b"")
assert counts.max_line_length == 0
@pytest.mark.asyncio
async def test_wc_max_line_length_single_line_with_newline():
counts = await wc(b"hello\n")
assert counts.max_line_length == 5
@pytest.mark.asyncio
async def test_wc_max_line_length_picks_longest():
counts = await wc(b"short\na much longer line\nmed\n")
assert counts.max_line_length == len(b"a much longer line")
@pytest.mark.asyncio
async def test_wc_max_line_length_no_trailing_newline():
counts = await wc(b"hello world")
assert counts.max_line_length == 11
@pytest.mark.asyncio
async def test_wc_streams_chunked():
"""Streaming through chunk boundaries gives same counts as buffered."""
async def src():
yield b"hello "
yield b"world\n"
yield b"foo bar\n"
counts = await wc(src())
assert counts.lines == 2
assert counts.words == 4
assert counts.bytes_ == 20
@pytest.mark.asyncio
async def test_wc_word_split_across_chunks():
"""A word straddling a chunk boundary still counts as one word."""
async def src():
yield b"hel"
yield b"lo"
yield b" world\n"
counts = await wc(src())
assert counts.words == 2
assert counts.lines == 1
@pytest.mark.asyncio
async def test_wc_utf8_split_across_chunks_does_not_lose_chars():
"""A multibyte UTF-8 sequence split across chunks must still decode."""
async def src():
# é = b"\xc3\xa9" — split between the two bytes
yield b"caf\xc3"
yield b"\xa9"
counts = await wc(src())
assert counts.bytes_ == 5
assert counts.chars == 4
@pytest.mark.asyncio
async def test_wc_byte_by_byte_chunking():
"""Worst-case chunking: every byte its own chunk."""
async def src():
for byte in b"hello world\nfoo bar\n":
yield bytes([byte])
counts = await wc(src())
assert counts.lines == 2
assert counts.words == 4
assert counts.bytes_ == 20
@pytest.mark.asyncio
async def test_wc_binary_input_does_not_crash():
"""`errors='replace'` must let arbitrary bytes pass — count bytes
accurately even when UTF-8 decoding produces replacement chars."""
data = bytes(range(256))
counts = await wc(data)
assert counts.bytes_ == 256
assert counts.lines == 1 # one \n at byte 0x0a
@pytest.mark.asyncio
async def test_wc_lines_fast_path_matches_full():
"""`wc_lines` fast path must agree with the full counter."""
data = b"alpha\nbeta\ngamma\ndelta\n"
fast = await wc_lines(data)
full = await wc(data)
assert fast == full.lines == 4
@pytest.mark.asyncio
async def test_wc_lines_fast_path_no_trailing_newline():
assert await wc_lines(b"a\nb\nc") == 2
@pytest.mark.asyncio
async def test_wc_lines_fast_path_empty():
assert await wc_lines(b"") == 0
def test_format_wc_default_no_label():
counts = WCCounts(lines=2, words=4, bytes_=20)
assert format_wc(counts) == " 2 4 20"
def test_format_wc_default_with_label():
counts = WCCounts(lines=2, words=4, bytes_=20)
assert format_wc(counts, label="/f.txt") == " 2 4 20 /f.txt"
def test_format_wc_args_l():
counts = WCCounts(lines=2, words=4, bytes_=20)
assert format_wc(counts, args_l=True) == "2"
assert format_wc(counts, args_l=True, label="/f.txt") == "2 /f.txt"
def test_format_wc_w_c_m():
counts = WCCounts(lines=2, words=4, bytes_=20, chars=18)
assert format_wc(counts, w=True) == "4"
assert format_wc(counts, c=True) == "20"
assert format_wc(counts, m=True) == "18"
def test_format_wc_L_wins_when_both_set():
"""L has highest precedence (matching wrapper behavior)."""
counts = WCCounts(lines=2, max_line_length=11)
assert format_wc(counts, args_l=True, L=True) == "11"
def test_format_wc_precedence_l_over_w_c_m():
"""args_l > w > c > m precedence."""
counts = WCCounts(lines=2, words=4, bytes_=20, chars=18)
assert format_wc(counts, args_l=True, w=True) == "2"
def test_wc_counts_merge():
a = WCCounts(lines=2, words=4, bytes_=20, chars=18, max_line_length=11)
b = WCCounts(lines=1, words=2, bytes_=8, chars=8, max_line_length=20)
a.merge(b)
assert a.lines == 3
assert a.words == 6
assert a.bytes_ == 28
assert a.chars == 26
assert a.max_line_length == 20
@pytest.mark.asyncio
async def test_format_multi_single_path_emits_trailing_newline():
paths = [PathSpec.from_str_path("/a.txt")]
async def fake_read(_accessor, _path):
return b"hello\n"
out, err = await format_multi(paths, read=fake_read, args_l=True)
assert out == b"1 /a.txt\n"
assert err == b""
@pytest.mark.asyncio
async def test_format_multi_multi_path_emits_total_and_trailing_newline():
paths = [
PathSpec.from_str_path("/a.txt"),
PathSpec.from_str_path("/b.txt"),
]
data = {"/a.txt": b"hello\n", "/b.txt": b"world\nworld\n"}
async def fake_read(_accessor, path):
return data[path.virtual]
out, err = await format_multi(paths, read=fake_read, args_l=True)
assert err == b""
assert out.endswith(b"\n")
lines = out.decode().rstrip("\n").split("\n")
assert lines == ["1 /a.txt", "2 /b.txt", "3 total"]
@pytest.mark.asyncio
async def test_format_multi_accepts_sync_read_returning_bytes():
paths = [PathSpec.from_str_path("/a.txt")]
def sync_read(_accessor, _path):
return b"x\n"
out, err = await format_multi(paths, read=sync_read, args_l=True)
assert out == b"1 /a.txt\n"
assert err == b""
@pytest.mark.asyncio
async def test_format_multi_empty_paths_returns_empty():
async def fake_read(_accessor, _path):
return b""
out, err = await format_multi([], read=fake_read, args_l=True)
assert out == b""
assert err == b""
@pytest.mark.asyncio
async def test_format_multi_missing_operand_reports_and_totals():
paths = [
PathSpec.from_str_path("/a.txt"),
PathSpec.from_str_path("/m.txt"),
]
async def fake_read(_accessor, path):
if path.virtual == "/m.txt":
raise FileNotFoundError(path.virtual)
return b"hello\n"
out, err = await format_multi(paths, read=fake_read, args_l=True)
assert out == b"1 /a.txt\n1 total\n"
assert err == b"wc: /m.txt: No such file or directory\n"
@pytest.mark.asyncio
async def test_format_multi_all_missing_zero_total():
paths = [
PathSpec.from_str_path("/m1.txt"),
PathSpec.from_str_path("/m2.txt"),
]
async def fake_read(_accessor, path):
raise FileNotFoundError(path.virtual)
out, err = await format_multi(paths, read=fake_read, args_l=True)
assert out == b"0 total\n"
assert err == (b"wc: /m1.txt: No such file or directory\n"
b"wc: /m2.txt: No such file or directory\n")
async def _async_byte_read(_accessor, _path):
yield b"hello "
yield b"world\n"
@pytest.mark.asyncio
async def test_format_multi_accepts_async_iterator_read():
paths = [PathSpec.from_str_path("/a.txt")]
out, err = await format_multi(paths, read=_async_byte_read, args_l=True)
assert out == b"1 /a.txt\n"
assert err == b""
@@ -0,0 +1,188 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage.commands.builtin.generic_bind.adapter import CommandIO
from mirage.commands.builtin.generic_bind.builders.common import (
dir_refusing_read, split_readable)
from mirage.types import FileStat, FileType, PathSpec
def _ops(missing: set[str],
implicit_dirs: set[str] | None = None,
explicit_dirs: set[str] | None = None) -> CommandIO:
dirs = implicit_dirs or set()
typed = explicit_dirs or set()
async def stat(_accessor, path, _index):
if path.virtual in missing or path.virtual in dirs:
raise FileNotFoundError(path.virtual)
if path.virtual in typed:
return FileStat(name=path.virtual, type=FileType.DIRECTORY)
return FileStat(name=path.virtual, size=0)
async def readdir(_accessor, path, _index):
target = path.virtual.rstrip("/") or "/"
entries = [d for d in dirs if (d.rsplit("/", 1)[0] or "/") == target]
if path.virtual in dirs:
entries.append(path.virtual.rstrip("/") + "/child.txt")
return entries
async def read_stream(_accessor, _path, _index):
yield b"data"
async def unused(*_args):
raise AssertionError("not used")
return CommandIO(readdir=readdir,
read_bytes=unused,
read_stream=read_stream,
stat=stat,
is_mounted=lambda _a: True)
@pytest.mark.asyncio
async def test_split_readable_keeps_order_and_reports_missing():
paths = [
PathSpec.from_str_path("/m1.txt"),
PathSpec.from_str_path("/f.txt"),
PathSpec.from_str_path("/m2.txt"),
]
good, err = await split_readable(_ops({"/m1.txt", "/m2.txt"}), None, paths,
None, "cat")
assert [p.virtual for p in good] == ["/f.txt"]
assert err == (b"cat: /m1.txt: No such file or directory\n"
b"cat: /m2.txt: No such file or directory\n")
@pytest.mark.asyncio
async def test_split_readable_reports_implicit_dir_as_eisdir():
ops = _ops(set(), implicit_dirs={"/sub"})
good, err = await split_readable(ops, None,
[PathSpec.from_str_path("/sub")], None,
"cat")
assert good == []
assert err == b"cat: /sub: Is a directory\n"
@pytest.mark.asyncio
async def test_split_readable_ignores_fabricated_children():
# Synthetic hierarchies (postgres schema level) answer a readdir of
# any missing name with fabricated children; only the parent listing
# decides, so the original ENOENT stands.
async def stat(_accessor, path, _index):
raise FileNotFoundError(path.virtual)
async def readdir(_accessor, path, _index):
target = path.virtual.rstrip("/") or "/"
if target == "/":
return ["/real.txt"]
return [f"{target}/tables", f"{target}/views"]
async def unused(*_args):
raise AssertionError("not used")
ops = CommandIO(readdir=readdir,
read_bytes=unused,
read_stream=unused,
stat=stat,
is_mounted=lambda _a: True)
good, err = await split_readable(ops, None,
[PathSpec.from_str_path("/nope.txt")],
None, "cat")
assert good == []
assert err == b"cat: /nope.txt: No such file or directory\n"
@pytest.mark.asyncio
async def test_split_readable_probe_swallows_driver_errors():
# A backend whose readdir raises a non-FS driver error for missing
# names (lancedb: "Table ... was not found") must not leak it through
# the probe; the original ENOENT stands.
async def stat(_accessor, path, _index):
raise FileNotFoundError(path.virtual)
async def readdir(_accessor, path, _index):
raise ValueError("Table 'nope.txt' was not found")
async def unused(*_args):
raise AssertionError("not used")
ops = CommandIO(readdir=readdir,
read_bytes=unused,
read_stream=unused,
stat=stat,
is_mounted=lambda _a: True)
good, err = await split_readable(ops, None,
[PathSpec.from_str_path("/nope.txt")],
None, "wc")
assert good == []
assert err == b"wc: /nope.txt: No such file or directory\n"
@pytest.mark.asyncio
async def test_split_readable_reports_stat_typed_dir_as_eisdir():
ops = _ops(set(), explicit_dirs={"/sub"})
good, err = await split_readable(ops, None,
[PathSpec.from_str_path("/sub")], None,
"head")
assert good == []
assert err == b"head: /sub: Is a directory\n"
@pytest.mark.asyncio
async def test_dir_refusing_read_raises_eisdir_for_dirs():
ops = _ops(set(), implicit_dirs={"/sub"})
read = dir_refusing_read(ops, None)
with pytest.raises(IsADirectoryError):
async for _ in read(None, PathSpec.from_str_path("/sub")):
raise AssertionError("no data expected")
@pytest.mark.asyncio
async def test_dir_refusing_read_streams_files():
ops = _ops(set())
read = dir_refusing_read(ops, None)
chunks = [c async for c in read(None, PathSpec.from_str_path("/f.txt"))]
assert chunks == [b"data"]
@pytest.mark.asyncio
async def test_split_readable_all_good_no_stderr():
paths = [PathSpec.from_str_path("/f.txt")]
good, err = await split_readable(_ops(set()), None, paths, None, "head")
assert [p.virtual for p in good] == ["/f.txt"]
assert err == b""
@pytest.mark.asyncio
async def test_split_readable_propagates_non_fs_errors():
async def stat(_accessor, _path, _index):
raise RuntimeError("backend broke")
async def unused(*_args):
raise AssertionError("not used")
ops = CommandIO(readdir=unused,
read_bytes=unused,
read_stream=unused,
stat=stat,
is_mounted=lambda _a: True)
with pytest.raises(RuntimeError):
await split_readable(ops, None, [PathSpec.from_str_path("/f.txt")],
None, "cat")
@@ -0,0 +1,117 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage.commands.builtin.generic_bind.adapter import CommandIO
from mirage.commands.builtin.generic_bind.builders.find import find
from mirage.types import FileStat, FileType, PathSpec
TREE = {
"/mnt": ["/mnt/table1", "/mnt/notes.txt"],
"/mnt/table1": ["/mnt/table1/rows.jsonl"],
}
DIRS = {"/mnt", "/mnt/table1"}
def _ops(stat_calls: list[str], is_dir_name=None, find_op=None) -> CommandIO:
async def readdir(_accessor, path, _index):
return TREE.get(path.virtual.rstrip("/") or "/", [])
async def stat(_accessor, path, _index):
stat_calls.append(path.virtual)
if path.virtual not in TREE and path.virtual not in TREE.get(
"/mnt", []) and path.virtual != "/mnt/table1/rows.jsonl":
raise FileNotFoundError(path.virtual)
if path.virtual in DIRS:
return FileStat(name=path.virtual, type=FileType.DIRECTORY)
return FileStat(name=path.virtual, size=3)
async def read_stream(_accessor, _path, _index):
yield b"data"
async def unused(*_args):
raise AssertionError("not used")
return CommandIO(readdir=readdir,
read_bytes=unused,
read_stream=read_stream,
stat=stat,
is_mounted=lambda _a: True,
local=False,
is_dir_name=is_dir_name,
find=find_op)
def _root() -> PathSpec:
return PathSpec(virtual="/mnt",
directory="/mnt",
resolved=False,
resource_path="")
async def _lines(ops: CommandIO) -> list[str]:
stdout, _io = await find(ops, None, [_root()])
data = stdout if isinstance(stdout, bytes) else b""
return data.decode().splitlines()
@pytest.mark.asyncio
async def test_walk_without_hint_stats_children():
stat_calls: list[str] = []
ops = _ops(stat_calls)
lines = await _lines(ops)
assert "/mnt/notes.txt" in lines
assert "/mnt/table1/rows.jsonl" in lines
assert "/mnt/table1" in lines
# no hint: every child entry is stat'ed to classify it
assert len(stat_calls) > 1
@pytest.mark.asyncio
async def test_is_dir_name_hint_skips_child_stats():
stat_calls: list[str] = []
ops = _ops(stat_calls,
is_dir_name=lambda _a, name: name.rstrip("/") in DIRS)
lines = await _lines(ops)
assert "/mnt/notes.txt" in lines
assert "/mnt/table1/rows.jsonl" in lines
# hint answers directory-ness by name; only the search root is stat'ed
assert stat_calls == ["/mnt"]
@pytest.mark.asyncio
async def test_walk_honors_multiple_start_points():
stat_calls: list[str] = []
ops = _ops(stat_calls)
roots = [
PathSpec(virtual="/mnt/table1",
directory="/mnt/table1",
resolved=False,
resource_path="table1"),
PathSpec(virtual="/mnt/notes.txt",
directory="/mnt",
resolved=False,
resource_path="notes.txt"),
]
stdout, _io = await find(ops, None, roots)
data = stdout if isinstance(stdout, bytes) else b""
lines = data.decode().splitlines()
# GNU find walks every start point in operand order
assert "/mnt/table1/rows.jsonl" in lines
assert "/mnt/notes.txt" in lines
assert lines.index("/mnt/table1/rows.jsonl") < lines.index(
"/mnt/notes.txt")
@@ -0,0 +1,105 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage.accessor.base import NOOPAccessor
from mirage.commands.builtin.generic_bind.adapter import make_resolve_glob
from mirage.types import PathSpec
TREE = {
"/notion/pages": [
"/notion/pages/Demo_page__uuid1",
"/notion/pages/Roadmap__uuid2",
],
"/notion/pages/Demo_page__uuid1": [
"/notion/pages/Demo_page__uuid1/page.md",
"/notion/pages/Demo_page__uuid1/page.json",
],
"/notion/pages/Roadmap__uuid2": [
"/notion/pages/Roadmap__uuid2/page.json",
],
}
async def fake_readdir(accessor, path, index=None):
key = path.virtual.rstrip("/") or "/"
if key not in TREE:
raise FileNotFoundError(key)
return TREE[key]
def glob_spec(virtual: str, prefix: str) -> PathSpec:
last_slash = virtual.rfind("/")
return PathSpec(
virtual=virtual,
directory=virtual[:last_slash + 1],
resource_path=virtual[len(prefix):].strip("/"),
pattern=virtual[last_slash + 1:],
resolved=False,
)
@pytest.mark.asyncio
async def test_resolve_glob_mid_path_pattern():
resolve = make_resolve_glob(fake_readdir)
spec = glob_spec("/notion/pages/Demo_page__*/page.md", "/notion")
result = await resolve(NOOPAccessor(), [spec], None)
assert [p.virtual
for p in result] == ["/notion/pages/Demo_page__uuid1/page.md"]
@pytest.mark.asyncio
async def test_resolve_glob_last_component():
resolve = make_resolve_glob(fake_readdir)
spec = glob_spec("/notion/pages/Demo*", "/notion")
result = await resolve(NOOPAccessor(), [spec], None)
assert [p.virtual for p in result] == ["/notion/pages/Demo_page__uuid1"]
@pytest.mark.asyncio
async def test_resolve_glob_passthrough():
resolve = make_resolve_glob(fake_readdir)
resolved_spec = PathSpec.from_str_path("/notion/pages/Roadmap__uuid2",
"pages/Roadmap__uuid2")
result = await resolve(NOOPAccessor(), ["text", resolved_spec], None)
assert result[0].virtual == "text"
assert result[1] is resolved_spec
@pytest.mark.asyncio
async def test_resolve_glob_truncates(caplog):
resolve = make_resolve_glob(fake_readdir, max_glob_matches=1)
spec = glob_spec("/notion/pages/*", "/notion")
result = await resolve(NOOPAccessor(), [spec], None)
assert len(result) == 1
@pytest.mark.asyncio
async def test_resolve_glob_zero_match_word_keeps_literal():
resolve = make_resolve_glob(fake_readdir)
spec = glob_spec("/notion/pages/*.nope", "/notion")
out = await resolve(NOOPAccessor(), [spec], None)
assert len(out) == 1
assert out[0].virtual == "/notion/pages/*.nope"
assert out[0].pattern is None
assert out[0].resolved
@pytest.mark.asyncio
async def test_resolve_glob_zero_match_dir_shape_stays_empty():
resolve = make_resolve_glob(fake_readdir)
spec = glob_spec("/notion/pages/*.nope", "/notion").dir
out = await resolve(NOOPAccessor(), [spec], None)
assert out == []
@@ -0,0 +1,123 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage.cache.context import push_cache_manager
from mirage.cache.file.ram import RAMFileCacheStore
from mirage.cache.manager import CacheManager
from mirage.commands.builtin.generic_bind.adapter import CommandIO
from mirage.commands.builtin.generic_bind.factory import with_read_cache
from mirage.types import PathSpec
from mirage.utils.key_prefix import mount_key
class _CountingBackend:
def __init__(self, data: bytes) -> None:
self.data = data
self.stream_calls = 0
self.bytes_calls = 0
async def read_stream(self, accessor, path, *args, **kwargs):
self.stream_calls += 1
yield self.data
async def read_bytes(self, accessor, path, *args, **kwargs) -> bytes:
self.bytes_calls += 1
return self.data
async def _noop_readdir(accessor, path, index=None) -> list[str]:
return []
async def _noop_stat(accessor, path, index=None):
return None
def _ops(backend: _CountingBackend) -> CommandIO:
return CommandIO(
readdir=_noop_readdir,
read_bytes=backend.read_bytes,
read_stream=backend.read_stream,
stat=_noop_stat,
is_mounted=lambda a: True,
local=False,
)
def _spec() -> PathSpec:
return PathSpec(resource_path=mount_key("/s3/a.txt", "/s3/"),
virtual="/s3/a.txt",
directory="/s3/")
async def _drain(source) -> bytes:
return b"".join([c async for c in source])
@pytest.mark.asyncio
async def test_warm_read_stream_serves_cache_without_backend():
backend = _CountingBackend(b"payload")
cache = RAMFileCacheStore()
await cache.set("/s3/a.txt", b"payload")
manager = CacheManager(cache, None, "/s3/", True)
ops = with_read_cache(_ops(backend))
prev = push_cache_manager(manager)
try:
out = await _drain(ops.read_stream(None, _spec()))
finally:
push_cache_manager(prev)
assert out == b"payload"
assert backend.stream_calls == 0
@pytest.mark.asyncio
async def test_warm_read_bytes_serves_cache_without_backend():
backend = _CountingBackend(b"payload")
cache = RAMFileCacheStore()
await cache.set("/s3/a.txt", b"payload")
manager = CacheManager(cache, None, "/s3/", True)
ops = with_read_cache(_ops(backend))
prev = push_cache_manager(manager)
try:
out = await ops.read_bytes(None, _spec())
finally:
push_cache_manager(prev)
assert out == b"payload"
assert backend.bytes_calls == 0
@pytest.mark.asyncio
async def test_cold_read_falls_through_to_backend():
backend = _CountingBackend(b"payload")
manager = CacheManager(RAMFileCacheStore(), None, "/s3/", True)
ops = with_read_cache(_ops(backend))
prev = push_cache_manager(manager)
try:
out = await _drain(ops.read_stream(None, _spec()))
finally:
push_cache_manager(prev)
assert out == b"payload"
assert backend.stream_calls == 1
@pytest.mark.asyncio
async def test_no_manager_falls_through_to_backend():
backend = _CountingBackend(b"payload")
ops = with_read_cache(_ops(backend))
out = await _drain(ops.read_stream(None, _spec()))
assert out == b"payload"
assert backend.stream_calls == 1
@@ -0,0 +1,297 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage.cache.index import IndexEntry
from mirage.cache.index.ram import RAMIndexCacheStore
from mirage.commands.builtin.generic_bind.provision import (
default_provision, exact_zero_provision, index_hit_read_provision,
make_copy_provision, make_file_read_provision, make_head_tail_provision,
make_search_provision, make_transform_provision, metadata_provision,
pure_provision, write_metadata_provision)
from mirage.commands.builtin.ram import COMMANDS as RAM_COMMANDS
from mirage.provision import Precision
from mirage.types import FileStat, FileType, PathSpec
from mirage.utils.key_prefix import mount_key
SIZES = {
"/data/known.txt": 5,
"/data/big.txt": 100,
"/data/tree/a.txt": 7,
"/data/tree/skip.parquet": 900,
"/data/tree/sub/b.txt": 11,
}
DIRS = {"/data/tree", "/data/tree/sub"}
TREE = {
"/data/tree":
["/data/tree/a.txt", "/data/tree/skip.parquet", "/data/tree/sub"],
"/data/tree/sub": ["/data/tree/sub/b.txt"],
}
def _spec(path: str) -> PathSpec:
return PathSpec(resource_path=mount_key(path, "/data"),
virtual=path,
directory=path)
async def _stat(accessor, path, index=None) -> FileStat:
virtual = path.virtual if isinstance(path, PathSpec) else path
if virtual in DIRS:
return FileStat(name=virtual.rsplit("/", 1)[-1],
type=FileType.DIRECTORY)
return FileStat(name=virtual.rsplit("/", 1)[-1],
size=SIZES.get(virtual),
type=FileType.TEXT)
async def _readdir(accessor, path, index=None) -> list[str]:
virtual = path.virtual if isinstance(path, PathSpec) else path
return TREE[virtual]
async def _resolve_glob(accessor, paths, index=None) -> list[PathSpec]:
out = []
for p in paths:
if p.pattern:
out.extend(
_spec(e) for e in TREE.get(p.directory.rstrip("/"), [])
if e.endswith(p.pattern.lstrip("*")))
else:
out.append(p)
return out
def _registered(name: str):
for fn in RAM_COMMANDS:
for rc in getattr(fn, "_registered_commands", []):
if rc.name == name and rc.filetype is None:
return rc
return None
def test_default_provision_families():
assert default_provision("sort", _stat) is not None
assert default_provision("grep", _stat) is not None
assert default_provision("iconv", _stat) is not None
assert default_provision("file", _stat) is not None
assert default_provision("ls", _stat) is metadata_provision
assert default_provision("stat", _stat) is metadata_provision
assert default_provision("du", _stat) is metadata_provision
assert default_provision("gzip", _stat) is not None
assert default_provision("cp", _stat) is not None
assert default_provision("rm", _stat) is write_metadata_provision
assert default_provision("mv", _stat) is None
assert default_provision("tee", _stat) is None
def test_factory_registers_default_provisions():
for name in ("grep", "sort", "ls", "find", "md5", "cp", "gzip", "rm"):
rc = _registered(name)
assert rc is not None, name
assert rc.provision_fn is not None, name
rc = _registered("tee")
assert rc is not None
assert rc.provision_fn is None
@pytest.mark.asyncio
async def test_file_read_known_sizes_exact():
provision = make_file_read_provision(_stat)
result = await provision(
None, [_spec("/data/known.txt"),
_spec("/data/big.txt")],
command="cat")
assert result.precision == Precision.EXACT
assert result.network_read_low == 105
assert result.network_read_high == 105
assert result.read_ops == 2
@pytest.mark.asyncio
async def test_file_read_missing_size_keeps_floor():
provision = make_file_read_provision(_stat)
result = await provision(
None, [_spec("/data/known.txt"),
_spec("/data/chat.jsonl")],
command="cat")
assert result.precision == Precision.UNKNOWN
assert result.network_read_low == 5
assert result.network_read_high == 5
assert result.read_ops == 2
@pytest.mark.asyncio
async def test_head_tail_missing_size_keeps_known_ceiling():
provision = make_head_tail_provision(_stat)
result = await provision(
None, [_spec("/data/known.txt"),
_spec("/data/chat.jsonl")],
command="head")
assert result.precision == Precision.UNKNOWN
assert result.network_read_low == 0
assert result.network_read_high == 5
assert result.read_ops == 2
@pytest.mark.asyncio
async def test_transform_keeps_read_floor_unknown_output():
provision = make_transform_provision(_stat)
result = await provision(None, [_spec("/data/known.txt")], command="gzip")
assert result.precision == Precision.UNKNOWN
assert result.network_read_low == 5
assert result.network_read_high == 5
assert result.read_ops == 1
@pytest.mark.asyncio
async def test_copy_brackets_read_and_write():
provision = make_copy_provision(_stat)
result = await provision(
None, [_spec("/data/known.txt"),
_spec("/data/dest.txt")],
command="cp")
assert result.precision == Precision.RANGE
assert result.network_read_low == 0
assert result.network_read_high == 5
assert result.network_write_low == 0
assert result.network_write_high == 5
assert result.read_ops == 1
@pytest.mark.asyncio
async def test_write_metadata_zero_bytes_recursive_floors():
result = await write_metadata_provision(None, [_spec("/data/known.txt")],
command="rm")
assert result.precision == Precision.EXACT
assert result.network_read_high == 0
assert result.read_ops == 1
recursive = await write_metadata_provision(None,
[_spec("/data/known.txt")],
command="rm",
r=True)
assert recursive.precision == Precision.UNKNOWN
@pytest.mark.asyncio
async def test_pure_provision_zero_exact():
result = await pure_provision(command="seq 3")
assert result.precision == Precision.EXACT
assert result.network_read_high == 0
assert result.read_ops == 0
@pytest.mark.asyncio
async def test_file_read_expands_globs():
provision = make_file_read_provision(_stat, _resolve_glob)
pattern = PathSpec(virtual="/data/tree/*.txt",
directory="/data/tree/",
pattern="*.txt",
resource_path="tree/*.txt")
result = await provision(None, [pattern], command="cat")
assert result.precision == Precision.EXACT
assert result.network_read_high == 7
assert result.read_ops == 1
@pytest.mark.asyncio
async def test_file_read_unmatched_glob_unknown():
provision = make_file_read_provision(_stat, _resolve_glob)
pattern = PathSpec(virtual="/data/tree/*.nope",
directory="/data/tree/",
pattern="*.nope",
resource_path="tree/*.nope")
result = await provision(None, [pattern], command="cat")
assert result.precision == Precision.UNKNOWN
assert result.network_read_high == 0
@pytest.mark.asyncio
async def test_search_recursive_walks_tree_exact():
provision = make_search_provision(_stat, _resolve_glob, _readdir)
result = await provision(None, [_spec("/data/tree")],
"x",
command="grep",
r=True)
assert result.precision == Precision.EXACT
assert result.network_read_high == 18
assert result.read_ops == 2
@pytest.mark.asyncio
async def test_search_recursive_skips_columnar():
provision = make_search_provision(_stat, _resolve_glob, _readdir)
result = await provision(None, [_spec("/data/tree")],
"x",
command="grep",
r=True)
assert result.network_read_high != 918
@pytest.mark.asyncio
async def test_search_without_readdir_keeps_floor():
provision = make_search_provision(_stat, _resolve_glob)
result = await provision(None, [_spec("/data/tree")],
"x",
command="grep",
r=True)
assert result.precision == Precision.UNKNOWN
@pytest.mark.asyncio
async def test_search_recursive_cap_degrades(monkeypatch):
monkeypatch.setattr(
"mirage.commands.builtin.generic_bind.provision.MAX_PLAN_WALK", 2)
provision = make_search_provision(_stat, _resolve_glob, _readdir)
result = await provision(None, [_spec("/data/tree")],
"x",
command="grep",
r=True)
assert result.precision == Precision.UNKNOWN
@pytest.mark.asyncio
async def test_exact_zero_provision_charges_nothing():
result = await exact_zero_provision("find /chat")
assert result.command == "find /chat"
assert result.network_read_low == 0
assert result.network_read_high == 0
assert result.read_ops == 0
assert result.precision == Precision.EXACT
@pytest.mark.asyncio
async def test_index_hit_read_provision_counts_cached_operands():
index = RAMIndexCacheStore()
await index.put("/chat/a.jsonl",
IndexEntry(id="a", name="a.jsonl", resource_type="file"))
paths = [
PathSpec(virtual="/chat/a.jsonl",
directory="/chat",
resource_path="a.jsonl"),
PathSpec(virtual="/chat/missing.jsonl",
directory="/chat",
resource_path="missing.jsonl"),
]
result = await index_hit_read_provision(None, paths, "cat", index)
assert result.read_ops == 1
assert result.network_read_low == 0
assert result.precision == Precision.EXACT
@pytest.mark.asyncio
async def test_index_hit_read_provision_without_paths_is_unknown():
result = await index_hit_read_provision(None, [], "grep x", None)
assert result.precision == Precision.UNKNOWN
@@ -0,0 +1,94 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from functools import partial
import pytest
from mirage.commands.builtin.generic_bind.adapter import CommandIO
from mirage.commands.builtin.generic_bind.builders.du import du as du_builder
from mirage.commands.builtin.generic_bind.builders.find import \
find as find_builder
from mirage.commands.builtin.utils.wrap import stream_from_bytes
from mirage.io.types import materialize
from mirage.types import FileStat, FileType, PathSpec
_FILES = {
"/g/a.txt": b"alpha\n",
"/g/sub/b.txt": b"bravo\n",
}
_DIRS = {"/g", "/g/sub"}
async def _readdir(accessor, path, index=None):
base = (path.virtual if isinstance(path, PathSpec) else path).rstrip("/")
out = []
for p in list(_FILES) + sorted(_DIRS):
parent = p.rsplit("/", 1)[0] or "/"
if parent == base and p != base:
out.append(p)
return out
async def _stat(accessor, path, index=None):
p = path.virtual if isinstance(path, PathSpec) else path
p = p.rstrip("/") or "/"
if p in _DIRS:
return FileStat(name=p.rsplit("/", 1)[-1], type=FileType.DIRECTORY)
if p in _FILES:
return FileStat(name=p.rsplit("/", 1)[-1],
size=len(_FILES[p]),
type=FileType.TEXT)
raise FileNotFoundError(p)
async def _read(accessor, path, index=None):
p = path.virtual if isinstance(path, PathSpec) else path
return _FILES[p.rstrip("/")]
def _ops() -> CommandIO:
return CommandIO(
readdir=_readdir,
read_bytes=_read,
read_stream=partial(stream_from_bytes, _read),
stat=_stat,
is_mounted=lambda a: True,
local=False,
)
def _spec(original: str) -> PathSpec:
return PathSpec(resource_path=(original).strip("/"),
virtual=original,
directory=original,
resolved=True)
@pytest.mark.asyncio
async def test_find_walks_tree_without_native_find_op():
out, io = await find_builder(_ops(), object(), [_spec("/g")], type="f")
text = (await materialize(out)).decode()
assert "/g/a.txt" in text
assert "/g/sub/b.txt" in text
assert io.exit_code == 0
@pytest.mark.asyncio
async def test_du_walks_tree_without_native_du_op():
out, _ = await du_builder(_ops(), object(), [_spec("/g")], s=True)
text = (await materialize(out)).decode()
# alpha\n (6) + bravo\n (6) = 12 bytes summed by the readdir walk.
assert "12" in text
assert "/g" in text
@@ -0,0 +1,13 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
@@ -0,0 +1,35 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage.resource.github.github import GitHubResource
from tests.fixtures.github_mock import github_config, mock_github_api
__all__ = ["github_config", "mock_github_api"]
OWNER = "test-owner"
REPO = "test-repo"
REF = "main"
@pytest.fixture
def github_env(mock_github_api, github_config):
resource = GitHubResource(
config=github_config,
owner=OWNER,
repo=REPO,
ref=REF,
)
return resource.accessor, resource._index
@@ -0,0 +1,166 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import pytest
from mirage.commands.builtin.github.grep import grep
from mirage.io.stream import materialize
from mirage.types import PathSpec
from mirage.utils.key_prefix import mount_key
from tests.fixtures.github_mock import MOCK_BLOBS
@pytest.fixture(autouse=True)
def _patch_read(monkeypatch):
async def _read_bytes(config, owner, repo, sha):
return MOCK_BLOBS[sha]
monkeypatch.setattr("mirage.core.github.read.read_bytes", _read_bytes)
def _scope(path: str, resolved: bool = True) -> PathSpec:
norm = "/" + path.lstrip("/")
directory = norm.rsplit("/", 1)[0] + "/"
return PathSpec(resource_path=(norm).strip("/"),
virtual=norm,
directory=directory,
resolved=resolved)
async def _run(accessor, index, paths, pattern, **kwargs):
scopes = [_scope(p, resolved=("." in p.split("/")[-1])) for p in paths]
stdout, io = await grep(accessor, scopes, pattern, index=index, **kwargs)
data = await materialize(stdout)
return data.decode(errors="replace"), io
@pytest.mark.asyncio
async def test_single_file_grep(mock_github_api, github_env):
accessor, index = github_env
text, io = await _run(accessor, index, ["src/main.py"], "import")
assert io.exit_code == 0
lines = text.strip().splitlines()
assert len(lines) == 3
assert "import os" in lines[0]
assert "import sys" in lines[1]
assert "import helper" in lines[2]
@pytest.mark.asyncio
async def test_grep_with_line_numbers(mock_github_api, github_env):
accessor, index = github_env
text, io = await _run(accessor, index, ["src/main.py"], "import", n=True)
assert io.exit_code == 0
lines = text.strip().splitlines()
assert lines[0].startswith("1:")
assert lines[1].startswith("2:")
@pytest.mark.asyncio
async def test_grep_case_insensitive(mock_github_api, github_env):
accessor, index = github_env
text, io = await _run(accessor, index, ["src/main.py"], "IMPORT", i=True)
assert io.exit_code == 0
lines = text.strip().splitlines()
assert len(lines) >= 2
@pytest.mark.asyncio
async def test_grep_invert(mock_github_api, github_env):
accessor, index = github_env
text, io = await _run(accessor, index, ["src/utils.py"], "import", v=True)
assert io.exit_code == 0
lines = text.strip().splitlines()
for line in lines:
assert "import" not in line
@pytest.mark.asyncio
async def test_grep_count(mock_github_api, github_env):
accessor, index = github_env
text, io = await _run(accessor, index, ["src/main.py"], "import", c=True)
assert io.exit_code == 0
assert text.strip() == "3"
@pytest.mark.asyncio
async def test_recursive_grep(mock_github_api, github_env):
accessor, index = github_env
text, io = await _run(accessor, index, ["src"], "dataclass", r=True)
assert io.exit_code == 0
lines = text.strip().splitlines()
assert len(lines) >= 2
paths_found = [line.split(":")[0] for line in lines]
assert any("user.py" in p for p in paths_found)
assert any("item.py" in p for p in paths_found)
@pytest.mark.asyncio
async def test_grep_no_match(mock_github_api, github_env):
accessor, index = github_env
text, io = await _run(accessor, index, ["src/main.py"], "zzz_no_match_zzz")
assert io.exit_code == 1
@pytest.mark.asyncio
async def test_grep_files_only(mock_github_api, github_env):
accessor, index = github_env
text, io = await _run(accessor,
index, ["src"],
"dataclass",
r=True,
args_l=True)
assert io.exit_code == 0
lines = text.strip().splitlines()
assert any("user.py" in ln for ln in lines)
assert any("item.py" in ln for ln in lines)
@pytest.mark.asyncio
async def test_grep_stdin(github_env):
accessor, index = github_env
stdin_data = b"hello world\nfoo bar\nhello again\n"
stdout, io = await grep(accessor, [],
"hello",
stdin=stdin_data,
index=index)
data = await materialize(stdout)
text = data.decode(errors="replace")
lines = text.strip().splitlines()
assert len(lines) == 2
assert "hello world" in lines[0]
assert "hello again" in lines[1]
@pytest.mark.asyncio
async def test_grep_files_only_with_prefix(mock_github_api, github_env):
accessor, index = github_env
scopes = [
PathSpec(resource_path=mount_key("/gh/src", "/gh"),
virtual="/gh/src",
directory="/gh/src/",
resolved=False)
]
stdout, io = await grep(accessor,
scopes,
"dataclass",
r=True,
args_l=True,
index=index)
data = await materialize(stdout)
text = data.decode()
assert io.exit_code == 0
lines = text.strip().splitlines()
assert any("user.py" in ln for ln in lines)

Some files were not shown because too many files have changed in this diff Show More