chore: import upstream snapshot with attribution
Integ / changes (push) Has been skipped
Pre-commit / pre-commit (push) Failing after 1s
CLI exit codes / changes (push) Has been skipped
Test (Install) / changes (push) Has been skipped
Test (Python) / changes (push) Has been skipped
Test (TypeScript) / changes (push) Has been skipped
CLI exit codes / cli-gate (push) Has been cancelled
Test (Install) / test-install-gate (push) Has been cancelled
Integ / integ-gate (push) Has been cancelled
Test (Python) / test-python-gate (push) Has been cancelled
Test (TypeScript) / test-typescript-gate (push) Has been cancelled
Test (Install) / python-minimal (3.12) (push) Has been cancelled
Test (Install) / python-minimal (3.11) (push) Has been cancelled
Test (Install) / python-extra (agno, mirage.agents.agno) (push) Has been cancelled
Test (Install) / python-extra (chroma, mirage.resource.chroma) (push) Has been cancelled
Test (Install) / python-extra (pdf, mirage.core.filetype.pdf) (push) Has been cancelled
Integ / integ (push) Has been cancelled
Integ / integ-database (push) Has been cancelled
Integ / integ-database-ts (push) Has been cancelled
Integ / integ-data (push) Has been cancelled
Integ / integ-ssh (push) Has been cancelled
Integ / integ-ssh-ts (push) Has been cancelled
Test (Python) / audit (push) Has been cancelled
Test (TypeScript) / test (push) Has been cancelled
Test (TypeScript) / python-fs-shim (push) Has been cancelled
CLI exit codes / Python CLI (push) Has been cancelled
CLI exit codes / TypeScript CLI (push) Has been cancelled
CLI exit codes / Cross-language snapshot interop (push) Has been cancelled
Test (Python) / test (push) Has been cancelled
Test (Python) / import-isolation (deepagents, openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Python) / import-isolation (deepagents, pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Integ / integ-ts (push) Has been cancelled
Integ / integ-fuse (push) Has been cancelled
Test (Install) / python-extra (databricks, mirage.resource.databricks_volume) (push) Has been cancelled
Test (Install) / python-extra (deepagents, mirage.agents.langchain) (push) Has been cancelled
Test (Install) / python-extra (email, mirage.resource.email) (push) Has been cancelled
Test (Install) / python-extra (fuse, mirage.fuse.mount) (push) Has been cancelled
Test (Install) / python-extra (hdf5, mirage.core.filetype.hdf5) (push) Has been cancelled
Test (Install) / python-extra (hf, mirage.resource.hf_buckets) (push) Has been cancelled
Test (Install) / python-extra (lancedb, mirage.resource.lancedb) (push) Has been cancelled
Test (Install) / python-extra (langfuse, mirage.resource.langfuse) (push) Has been cancelled
Test (Install) / python-extra (mongodb, mirage.resource.mongodb) (push) Has been cancelled
Test (Install) / python-extra (nextcloud, mirage.resource.nextcloud) (push) Has been cancelled
Test (Install) / python-extra (openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Install) / python-extra (openhands, mirage.agents.openhands, 3.12) (push) Has been cancelled
Test (Install) / python-extra (parquet, mirage.core.filetype.parquet) (push) Has been cancelled
Test (Install) / python-extra (postgres, mirage.resource.postgres) (push) Has been cancelled
Test (Install) / python-extra (pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Test (Install) / python-extra (qdrant, mirage.resource.qdrant) (push) Has been cancelled
Test (Install) / python-extra (redis, mirage.resource.redis) (push) Has been cancelled
Test (Install) / python-extra (s3, mirage.resource.s3) (push) Has been cancelled
Test (Install) / python-extra (ssh, mirage.resource.ssh) (push) Has been cancelled
Test (Install) / ts-minimal (push) Has been cancelled
Integ / changes (push) Has been skipped
Pre-commit / pre-commit (push) Failing after 1s
CLI exit codes / changes (push) Has been skipped
Test (Install) / changes (push) Has been skipped
Test (Python) / changes (push) Has been skipped
Test (TypeScript) / changes (push) Has been skipped
CLI exit codes / cli-gate (push) Has been cancelled
Test (Install) / test-install-gate (push) Has been cancelled
Integ / integ-gate (push) Has been cancelled
Test (Python) / test-python-gate (push) Has been cancelled
Test (TypeScript) / test-typescript-gate (push) Has been cancelled
Test (Install) / python-minimal (3.12) (push) Has been cancelled
Test (Install) / python-minimal (3.11) (push) Has been cancelled
Test (Install) / python-extra (agno, mirage.agents.agno) (push) Has been cancelled
Test (Install) / python-extra (chroma, mirage.resource.chroma) (push) Has been cancelled
Test (Install) / python-extra (pdf, mirage.core.filetype.pdf) (push) Has been cancelled
Integ / integ (push) Has been cancelled
Integ / integ-database (push) Has been cancelled
Integ / integ-database-ts (push) Has been cancelled
Integ / integ-data (push) Has been cancelled
Integ / integ-ssh (push) Has been cancelled
Integ / integ-ssh-ts (push) Has been cancelled
Test (Python) / audit (push) Has been cancelled
Test (TypeScript) / test (push) Has been cancelled
Test (TypeScript) / python-fs-shim (push) Has been cancelled
CLI exit codes / Python CLI (push) Has been cancelled
CLI exit codes / TypeScript CLI (push) Has been cancelled
CLI exit codes / Cross-language snapshot interop (push) Has been cancelled
Test (Python) / test (push) Has been cancelled
Test (Python) / import-isolation (deepagents, openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Python) / import-isolation (deepagents, pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Integ / integ-ts (push) Has been cancelled
Integ / integ-fuse (push) Has been cancelled
Test (Install) / python-extra (databricks, mirage.resource.databricks_volume) (push) Has been cancelled
Test (Install) / python-extra (deepagents, mirage.agents.langchain) (push) Has been cancelled
Test (Install) / python-extra (email, mirage.resource.email) (push) Has been cancelled
Test (Install) / python-extra (fuse, mirage.fuse.mount) (push) Has been cancelled
Test (Install) / python-extra (hdf5, mirage.core.filetype.hdf5) (push) Has been cancelled
Test (Install) / python-extra (hf, mirage.resource.hf_buckets) (push) Has been cancelled
Test (Install) / python-extra (lancedb, mirage.resource.lancedb) (push) Has been cancelled
Test (Install) / python-extra (langfuse, mirage.resource.langfuse) (push) Has been cancelled
Test (Install) / python-extra (mongodb, mirage.resource.mongodb) (push) Has been cancelled
Test (Install) / python-extra (nextcloud, mirage.resource.nextcloud) (push) Has been cancelled
Test (Install) / python-extra (openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Install) / python-extra (openhands, mirage.agents.openhands, 3.12) (push) Has been cancelled
Test (Install) / python-extra (parquet, mirage.core.filetype.parquet) (push) Has been cancelled
Test (Install) / python-extra (postgres, mirage.resource.postgres) (push) Has been cancelled
Test (Install) / python-extra (pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Test (Install) / python-extra (qdrant, mirage.resource.qdrant) (push) Has been cancelled
Test (Install) / python-extra (redis, mirage.resource.redis) (push) Has been cancelled
Test (Install) / python-extra (s3, mirage.resource.s3) (push) Has been cancelled
Test (Install) / python-extra (ssh, mirage.resource.ssh) (push) Has been cancelled
Test (Install) / ts-minimal (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,25 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.chroma.find import _default_name, _expr_texts
|
||||
|
||||
|
||||
@pytest.mark.parametrize("texts", [
|
||||
("!", "-name", "x"),
|
||||
("(", "-name", "a", "-o", "-name", "b", ")"),
|
||||
("-name", "x"),
|
||||
("-not", "-name", "x"),
|
||||
])
|
||||
def test_expr_texts_preserves_expression(texts):
|
||||
assert _expr_texts(texts) == texts
|
||||
|
||||
|
||||
def test_expr_texts_strips_bare_leading_name():
|
||||
assert _expr_texts(("foo", )) == ()
|
||||
assert _expr_texts(()) == ()
|
||||
|
||||
|
||||
def test_default_name_only_for_bare_word():
|
||||
assert _default_name(None, ("foo", )) == "foo"
|
||||
assert _default_name(None, ("!", "-name", "x")) is None
|
||||
assert _default_name(None, ("(", "-name", "a")) is None
|
||||
assert _default_name("given", ("foo", )) == "given"
|
||||
@@ -0,0 +1,24 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.resource.ram import RAMResource
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def backend():
|
||||
b = RAMResource()
|
||||
b.accessor.store.dirs.add("/tmp")
|
||||
return b
|
||||
@@ -0,0 +1,91 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import MountMode, Workspace
|
||||
from mirage.cache.index import RAMIndexCacheStore
|
||||
from mirage.types import PathSpec
|
||||
from mirage.utils.stream import collect_bytes
|
||||
from tests.resource.databricks_volume.test_databricks_volume import (
|
||||
FakeFiles, make_resource, seed_directory, seed_file)
|
||||
|
||||
|
||||
def seed_text_command_fixture(files: FakeFiles) -> str:
|
||||
root = "/Volumes/main/default/agent_files/root"
|
||||
seed_directory(files, root)
|
||||
seed_file(files, f"{root}/words.txt", b"beta\nalpha\nalpha\n")
|
||||
seed_file(files, f"{root}/more.txt", b"delta\n")
|
||||
seed_file(files, f"{root}/table.csv", b"name,score\nann,2\nbob,3\n")
|
||||
seed_file(files, f"{root}/table_extra.csv", b"name,score\ncam,5\n")
|
||||
seed_file(files, f"{root}/data.json", b'{"name": "mirage"}\n')
|
||||
seed_file(files, f"{root}/data2.json", b'{"name": "agent"}\n')
|
||||
seed_file(files, f"{root}/events.jsonl", b'{"name": "first"}\n')
|
||||
seed_file(files, f"{root}/events_extra.jsonl", b'{"name": "second"}\n')
|
||||
seed_file(files, f"{root}/old.txt", b"same\nold\n")
|
||||
seed_file(files, f"{root}/new.txt", b"same\nnew\n")
|
||||
return root
|
||||
|
||||
|
||||
async def materialize(source) -> bytes:
|
||||
if source is None:
|
||||
return b""
|
||||
return await collect_bytes(source)
|
||||
|
||||
|
||||
class IndexTrackingReader:
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.seen_indexes: list[RAMIndexCacheStore | None] = []
|
||||
|
||||
async def read_bytes(self,
|
||||
accessor,
|
||||
path,
|
||||
index=None,
|
||||
*args,
|
||||
**kwargs) -> bytes:
|
||||
self.seen_indexes.append(index)
|
||||
original = path.virtual if isinstance(path, PathSpec) else path
|
||||
if str(original).endswith(".json"):
|
||||
return b'{"name": "mirage"}\n'
|
||||
if str(original).endswith(".csv"):
|
||||
return b"name,score\nann,2\n"
|
||||
return b"beta\nalpha\nalpha\n"
|
||||
|
||||
async def read_stream(self, accessor, path, index=None, *args, **kwargs):
|
||||
self.seen_indexes.append(index)
|
||||
yield await self.read_bytes(accessor, path, index, *args, **kwargs)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def databricks_text_files() -> FakeFiles:
|
||||
files = FakeFiles()
|
||||
seed_text_command_fixture(files)
|
||||
return files
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def databricks_text_workspace(databricks_text_files: FakeFiles) -> Workspace:
|
||||
return Workspace({"/dbx/": make_resource(databricks_text_files)},
|
||||
mode=MountMode.READ)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def expected_index() -> RAMIndexCacheStore:
|
||||
return RAMIndexCacheStore(ttl=600)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def materialize_output():
|
||||
return materialize
|
||||
@@ -0,0 +1,25 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_workspace_execute_databricks_volume_awk(
|
||||
databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute(
|
||||
"awk '{print $1}' /dbx/table.csv")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"name,score\nann,2\nbob,3\n"
|
||||
@@ -0,0 +1,186 @@
|
||||
import time
|
||||
from functools import partial
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.accessor.databricks_volume import DatabricksVolumeAccessor
|
||||
from mirage.cache.index import RAMIndexCacheStore
|
||||
from mirage.commands.builtin.generic.find import parse_find_args, walk_find
|
||||
from mirage.commands.builtin.generic.ls import ls as generic_ls
|
||||
from mirage.commands.builtin.generic.tree import tree as generic_tree
|
||||
from mirage.core.databricks_volume.path import backend_path
|
||||
from mirage.core.databricks_volume.readdir import readdir
|
||||
from mirage.core.databricks_volume.stat import stat
|
||||
from mirage.resource.databricks_volume import DatabricksVolumeConfig
|
||||
from mirage.types import LsSortBy, PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
from tests.core.databricks_volume.conftest import (FakeClient, FakeFiles,
|
||||
directory_entry, file_entry)
|
||||
|
||||
MODIFIED_MS = 1_700_000_000_000
|
||||
|
||||
|
||||
def is_dir_name(_child: str) -> bool | None:
|
||||
# Databricks readdir returns slash-less paths, so classification always
|
||||
# falls back to stat.
|
||||
return None
|
||||
|
||||
|
||||
FROZEN_NOW_S = 1_700_000_000
|
||||
DAY_S = 86_400
|
||||
AGES_DAYS = (1, 2, 3, 10, 20)
|
||||
|
||||
|
||||
def _rig(
|
||||
) -> tuple[DatabricksVolumeAccessor, FakeFiles, RAMIndexCacheStore, str]:
|
||||
config = DatabricksVolumeConfig(
|
||||
catalog="main",
|
||||
schema="default",
|
||||
volume="agent_files",
|
||||
root_path="/root",
|
||||
token="secret",
|
||||
)
|
||||
files = FakeFiles()
|
||||
accessor = DatabricksVolumeAccessor(config, FakeClient(files))
|
||||
index = RAMIndexCacheStore(ttl=600)
|
||||
return accessor, files, index, backend_path(config, "/")
|
||||
|
||||
|
||||
def _seed_flat(files: FakeFiles, root: str, count: int = 5) -> None:
|
||||
files.directories[f"{root}/sub"] = [
|
||||
file_entry(f"{root}/sub/f{i}.txt",
|
||||
size=i + 1,
|
||||
modified=MODIFIED_MS + i * 1000) for i in range(count)
|
||||
]
|
||||
|
||||
|
||||
def _ls_readdir(accessor: DatabricksVolumeAccessor):
|
||||
return partial(readdir, accessor)
|
||||
|
||||
|
||||
def _ls_stat(accessor: DatabricksVolumeAccessor):
|
||||
return partial(stat, accessor)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("sort_by,long", [
|
||||
(LsSortBy.NAME, False),
|
||||
(LsSortBy.NAME, True),
|
||||
(LsSortBy.TIME, False),
|
||||
(LsSortBy.SIZE, False),
|
||||
])
|
||||
async def test_ls_lists_once_without_per_entry_metadata(sort_by, long):
|
||||
accessor, files, index, root = _rig()
|
||||
_seed_flat(files, root, count=5)
|
||||
path = PathSpec.from_str_path("/volume/sub",
|
||||
mount_key("/volume/sub", "/volume"))
|
||||
await generic_ls([path],
|
||||
readdir=_ls_readdir(accessor),
|
||||
stat=_ls_stat(accessor),
|
||||
long=long,
|
||||
sort_by=sort_by,
|
||||
index=index)
|
||||
assert files.list_directory_calls == [f"{root}/sub"]
|
||||
assert files.get_metadata_calls == []
|
||||
assert files.get_directory_metadata_calls == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_recursive_one_list_per_directory():
|
||||
accessor, files, index, root = _rig()
|
||||
files.directories[f"{root}/sub"] = [
|
||||
file_entry(f"{root}/sub/a.txt", size=1, modified=MODIFIED_MS),
|
||||
directory_entry(f"{root}/sub/inner"),
|
||||
]
|
||||
files.directories[f"{root}/sub/inner"] = [
|
||||
file_entry(f"{root}/sub/inner/b.txt", size=2, modified=MODIFIED_MS),
|
||||
]
|
||||
path = PathSpec.from_str_path("/volume/sub",
|
||||
mount_key("/volume/sub", "/volume"))
|
||||
await generic_ls([path],
|
||||
readdir=_ls_readdir(accessor),
|
||||
stat=_ls_stat(accessor),
|
||||
recursive=True,
|
||||
index=index)
|
||||
assert sorted(
|
||||
files.list_directory_calls) == [f"{root}/sub", f"{root}/sub/inner"]
|
||||
assert files.get_metadata_calls == []
|
||||
assert files.get_directory_metadata_calls == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_one_list_per_directory_without_metadata():
|
||||
accessor, files, index, root = _rig()
|
||||
files.directories[f"{root}/sub"] = [
|
||||
file_entry(f"{root}/sub/a.txt", size=1, modified=MODIFIED_MS),
|
||||
directory_entry(f"{root}/sub/inner"),
|
||||
]
|
||||
files.directories[f"{root}/sub/inner"] = [
|
||||
file_entry(f"{root}/sub/inner/b.txt", size=2, modified=MODIFIED_MS),
|
||||
]
|
||||
path = PathSpec.from_str_path("/volume/sub",
|
||||
mount_key("/volume/sub", "/volume"))
|
||||
await generic_tree(path,
|
||||
readdir=_ls_readdir(accessor),
|
||||
stat=_ls_stat(accessor),
|
||||
index=index)
|
||||
assert sorted(
|
||||
files.list_directory_calls) == [f"{root}/sub", f"{root}/sub/inner"]
|
||||
assert files.get_metadata_calls == []
|
||||
assert files.get_directory_metadata_calls == []
|
||||
|
||||
|
||||
async def _run_find(accessor, index, *, type=None, size=None, mtime=None):
|
||||
args = parse_find_args((), type=type, size=size, mtime=mtime)
|
||||
path = PathSpec.from_str_path("/volume/sub",
|
||||
mount_key("/volume/sub", "/volume"))
|
||||
return await walk_find(path,
|
||||
readdir=_ls_readdir(accessor),
|
||||
stat=_ls_stat(accessor),
|
||||
is_dir_name=is_dir_name,
|
||||
index=index,
|
||||
args=args)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_type_does_not_stat_children():
|
||||
accessor, files, index, root = _rig()
|
||||
_seed_flat(files, root, count=5)
|
||||
files.directory_metadata.add(f"{root}/sub")
|
||||
results = await _run_find(accessor, index, type="f")
|
||||
assert len(results) == 5
|
||||
assert files.list_directory_calls == [f"{root}/sub"]
|
||||
child_metadata = [c for c in files.get_metadata_calls if "/sub/" in c]
|
||||
assert child_metadata == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_size_reads_size_from_index():
|
||||
accessor, files, index, root = _rig()
|
||||
_seed_flat(files, root, count=5)
|
||||
files.directory_metadata.add(f"{root}/sub")
|
||||
results = await _run_find(accessor, index, type="f", size="+3c")
|
||||
assert sorted(r.rsplit("/", 1)[-1]
|
||||
for r in results) == ["f3.txt", "f4.txt"]
|
||||
assert files.list_directory_calls == [f"{root}/sub"]
|
||||
child_metadata = [c for c in files.get_metadata_calls if "/sub/" in c]
|
||||
assert child_metadata == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_mtime_reads_modified_from_index(monkeypatch):
|
||||
monkeypatch.setattr(time, "time", lambda: float(FROZEN_NOW_S))
|
||||
accessor, files, index, root = _rig()
|
||||
files.directories[f"{root}/sub"] = [
|
||||
file_entry(f"{root}/sub/f{i}.txt",
|
||||
size=i + 1,
|
||||
modified=(FROZEN_NOW_S - age * DAY_S) * 1000)
|
||||
for i, age in enumerate(AGES_DAYS)
|
||||
]
|
||||
files.directory_metadata.add(f"{root}/sub")
|
||||
results = await _run_find(accessor, index, mtime="-5")
|
||||
assert sorted(r.rsplit("/", 1)[-1]
|
||||
for r in results) == ["f0.txt", "f1.txt", "f2.txt"]
|
||||
assert files.list_directory_calls == [f"{root}/sub"]
|
||||
child_metadata = [c for c in files.get_metadata_calls if "/sub/" in c]
|
||||
assert child_metadata == []
|
||||
@@ -0,0 +1,40 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_single_file(databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute("cat /dbx/words.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"beta\nalpha\nalpha\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_multiple_files_concatenated(databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute(
|
||||
"cat /dbx/words.txt /dbx/more.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"beta\nalpha\nalpha\ndelta\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_n_numbers_lines(databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute("cat -n /dbx/words.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b" 1\tbeta\n 2\talpha\n 3\talpha\n"
|
||||
@@ -0,0 +1,123 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import MountMode, Workspace
|
||||
from tests.resource.databricks_volume.test_databricks_volume import (
|
||||
FakeFiles, make_resource, seed_directory, seed_file)
|
||||
|
||||
ROOT = "/Volumes/main/default/agent_files/root"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def dbx_files() -> FakeFiles:
|
||||
files = FakeFiles()
|
||||
seed_directory(files, ROOT)
|
||||
seed_file(files, f"{ROOT}/src.txt", b"hello")
|
||||
return files
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def write_ws(dbx_files: FakeFiles) -> Workspace:
|
||||
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.WRITE)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def read_ws(dbx_files: FakeFiles) -> Workspace:
|
||||
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cp_file_preserves_bytes(write_ws, dbx_files):
|
||||
io = await write_ws.execute("cp /dbx/src.txt /dbx/dst.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert dbx_files.downloads[f"{ROOT}/dst.txt"] == b"hello"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cp_directory_without_recursive_fails(write_ws, dbx_files):
|
||||
seed_directory(dbx_files, f"{ROOT}/d")
|
||||
|
||||
io = await write_ws.execute("cp /dbx/d /dbx/d2")
|
||||
|
||||
assert io.exit_code != 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cp_writes_are_mount_relative(write_ws):
|
||||
io = await write_ws.execute("cp /dbx/src.txt /dbx/dst.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
for key in io.writes:
|
||||
assert key.startswith("/dbx/")
|
||||
assert not key.startswith("/dbx/dbx/")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cp_read_only_mount_rejected(read_ws, dbx_files):
|
||||
io = await read_ws.execute("cp /dbx/src.txt /dbx/dst.txt")
|
||||
|
||||
assert io.exit_code != 0
|
||||
assert b"read-only" in io.stderr
|
||||
assert f"{ROOT}/dst.txt" not in dbx_files.downloads
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cp_onto_same_path_errors_and_preserves_file(
|
||||
write_ws, dbx_files):
|
||||
io = await write_ws.execute("cp /dbx/src.txt /dbx/src.txt")
|
||||
|
||||
assert io.exit_code != 0
|
||||
assert b"are the same file" in io.stderr
|
||||
assert dbx_files.downloads[f"{ROOT}/src.txt"] == b"hello"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cp_multiple_sources_require_directory(write_ws, dbx_files):
|
||||
seed_file(dbx_files, f"{ROOT}/a.txt", b"AAA")
|
||||
seed_file(dbx_files, f"{ROOT}/b.txt", b"BBB")
|
||||
seed_file(dbx_files, f"{ROOT}/target.txt", b"target")
|
||||
|
||||
io = await write_ws.execute("cp /dbx/a.txt /dbx/b.txt /dbx/target.txt")
|
||||
|
||||
assert io.exit_code != 0
|
||||
assert b"not a directory" in io.stderr
|
||||
assert dbx_files.downloads[f"{ROOT}/a.txt"] == b"AAA"
|
||||
assert dbx_files.downloads[f"{ROOT}/b.txt"] == b"BBB"
|
||||
assert dbx_files.downloads[f"{ROOT}/target.txt"] == b"target"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cp_missing_source_reports_cannot_stat(write_ws, dbx_files):
|
||||
io = await write_ws.execute("cp /dbx/missing /dbx/missing")
|
||||
|
||||
assert io.exit_code != 0
|
||||
assert b"cannot stat" in io.stderr
|
||||
assert b"are the same file" not in io.stderr
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cp_recursive_into_itself_errors_and_preserves_tree(
|
||||
write_ws, dbx_files):
|
||||
seed_directory(dbx_files, f"{ROOT}/d")
|
||||
seed_file(dbx_files, f"{ROOT}/d/a.txt", b"aaa")
|
||||
|
||||
io = await write_ws.execute("cp -r /dbx/d /dbx/d")
|
||||
|
||||
assert io.exit_code != 0
|
||||
assert b"into itself" in io.stderr
|
||||
assert dbx_files.downloads[f"{ROOT}/d/a.txt"] == b"aaa"
|
||||
assert f"{ROOT}/d/d" not in dbx_files.directory_metadata
|
||||
@@ -0,0 +1,87 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import asyncio
|
||||
|
||||
from mirage import MountMode, Workspace
|
||||
from mirage.resource.ram import RAMResource
|
||||
from tests.resource.databricks_volume.test_databricks_volume import (
|
||||
FakeFiles, make_resource, seed_file)
|
||||
|
||||
ROOT = "/Volumes/main/default/agent_files/root"
|
||||
|
||||
|
||||
def _run(ws, cmd):
|
||||
|
||||
async def _inner():
|
||||
io = await ws.execute(cmd)
|
||||
return await io.stdout_str(), await io.stderr_str(), io.exit_code
|
||||
|
||||
return asyncio.run(_inner())
|
||||
|
||||
|
||||
def _ws_with_dbx_tree():
|
||||
files = FakeFiles()
|
||||
files.create_directory(ROOT)
|
||||
files.create_directory(f"{ROOT}/tree")
|
||||
files.create_directory(f"{ROOT}/tree/sub")
|
||||
seed_file(files, f"{ROOT}/tree/a.txt", b"aaa\n")
|
||||
seed_file(files, f"{ROOT}/tree/sub/b.txt", b"bbb\n")
|
||||
dbx = make_resource(files)
|
||||
ram = RAMResource()
|
||||
ws = Workspace(
|
||||
{
|
||||
"/dbx": (dbx, MountMode.WRITE),
|
||||
"/m": (ram, MountMode.WRITE),
|
||||
}, )
|
||||
return ws, files, ram
|
||||
|
||||
|
||||
def test_cp_recursive_databricks_to_ram():
|
||||
ws, _files, ram = _ws_with_dbx_tree()
|
||||
_out, _err, code = _run(ws, "cp -r /dbx/tree /m/copied")
|
||||
assert code == 0
|
||||
assert ram._store.files["/copied/a.txt"] == b"aaa\n"
|
||||
assert ram._store.files["/copied/sub/b.txt"] == b"bbb\n"
|
||||
|
||||
|
||||
def test_cp_recursive_ram_to_databricks_creates_dirs():
|
||||
ws, files, ram = _ws_with_dbx_tree()
|
||||
ram._store.dirs.update({"/src", "/src/sub"})
|
||||
ram._store.files["/src/x.txt"] = b"xxx\n"
|
||||
ram._store.files["/src/sub/y.txt"] = b"yyy\n"
|
||||
_out, _err, code = _run(ws, "cp -r /m/src /dbx/put")
|
||||
assert code == 0
|
||||
assert files.downloads[f"{ROOT}/put/x.txt"] == b"xxx\n"
|
||||
assert files.downloads[f"{ROOT}/put/sub/y.txt"] == b"yyy\n"
|
||||
assert f"{ROOT}/put/sub" in files.directory_metadata
|
||||
|
||||
|
||||
def test_mv_recursive_databricks_to_ram_removes_source():
|
||||
ws, files, ram = _ws_with_dbx_tree()
|
||||
_out, _err, code = _run(ws, "mv /dbx/tree /m/moved")
|
||||
assert code == 0
|
||||
assert ram._store.files["/moved/a.txt"] == b"aaa\n"
|
||||
assert ram._store.files["/moved/sub/b.txt"] == b"bbb\n"
|
||||
assert f"{ROOT}/tree/a.txt" not in files.downloads
|
||||
assert f"{ROOT}/tree" not in files.directory_metadata
|
||||
|
||||
|
||||
def test_single_file_missing_parent_is_posix_error():
|
||||
# Option B / POSIX: a single-file copy never creates the destination's
|
||||
# parent directory; it errors like coreutils instead of mkdir -p.
|
||||
ws, _files, ram = _ws_with_dbx_tree()
|
||||
ram._store.files["/lone.txt"] = b"z\n"
|
||||
_out, _err, code = _run(ws, "cp /m/lone.txt /dbx/nope/deep/lone.txt")
|
||||
assert code == 1
|
||||
@@ -0,0 +1,25 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_workspace_execute_databricks_volume_cut(
|
||||
databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute("cut -d , -f 2 /dbx/table.csv"
|
||||
)
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"score\n2\n3\n"
|
||||
@@ -0,0 +1,37 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_workspace_execute_databricks_volume_diff(
|
||||
databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute(
|
||||
"diff -u /dbx/old.txt /dbx/new.txt")
|
||||
|
||||
assert io.exit_code == 1
|
||||
assert b"-old" in io.stdout
|
||||
assert b"+new" in io.stdout
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_workspace_execute_databricks_volume_diff_resolves_glob(
|
||||
databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute(
|
||||
"diff /dbx/old*.txt /dbx/new*.txt")
|
||||
|
||||
assert io.exit_code == 1
|
||||
assert b"old" in io.stdout
|
||||
assert b"new" in io.stdout
|
||||
@@ -0,0 +1,68 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import MountMode, Workspace
|
||||
from tests.resource.databricks_volume.test_databricks_volume import (
|
||||
FakeFiles, make_resource, seed_directory, seed_file)
|
||||
|
||||
ROOT = "/Volumes/main/default/agent_files/root"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def dbx_files() -> FakeFiles:
|
||||
files = FakeFiles()
|
||||
seed_directory(files, ROOT)
|
||||
seed_file(files, f"{ROOT}/words.txt", b"beta\nalpha\nalpha\n")
|
||||
seed_file(files, f"{ROOT}/notes.md", b"note\n")
|
||||
files.create_directory(f"{ROOT}/sub")
|
||||
seed_file(files, f"{ROOT}/sub/inner.txt", b"gamma\nalpha\n")
|
||||
return files
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def ws(dbx_files: FakeFiles) -> Workspace:
|
||||
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_name_glob(ws):
|
||||
io = await ws.execute("find /dbx/ -name '*.txt'")
|
||||
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "words.txt" in out
|
||||
assert "inner.txt" in out
|
||||
assert "notes.md" not in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_type_d(ws):
|
||||
io = await ws.execute("find /dbx/ -type d")
|
||||
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "sub" in out
|
||||
assert "words.txt" not in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_maxdepth(ws):
|
||||
io = await ws.execute("find /dbx/ -maxdepth 1")
|
||||
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "words.txt" in out
|
||||
assert "inner.txt" not in out
|
||||
@@ -0,0 +1,77 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import MountMode, Workspace
|
||||
from tests.resource.databricks_volume.test_databricks_volume import (
|
||||
FakeFiles, make_resource, seed_directory, seed_file)
|
||||
|
||||
ROOT = "/Volumes/main/default/agent_files/root"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def dbx_files() -> FakeFiles:
|
||||
files = FakeFiles()
|
||||
seed_directory(files, ROOT)
|
||||
seed_file(files, f"{ROOT}/words.txt", b"beta\nalpha\nalpha\n")
|
||||
files.create_directory(f"{ROOT}/sub")
|
||||
seed_file(files, f"{ROOT}/sub/inner.txt", b"gamma\nalpha\n")
|
||||
return files
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def ws(dbx_files: FakeFiles) -> Workspace:
|
||||
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_single_file(ws):
|
||||
io = await ws.execute("grep alpha /dbx/words.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"alpha\nalpha\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_line_numbers(ws):
|
||||
io = await ws.execute("grep -n alpha /dbx/words.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"2:alpha\n3:alpha\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_count_only(ws):
|
||||
io = await ws.execute("grep -c alpha /dbx/words.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"2\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_recursive(ws):
|
||||
io = await ws.execute("grep -r alpha /dbx/")
|
||||
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "words.txt" in out
|
||||
assert "inner.txt" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_no_match_exits_nonzero(ws):
|
||||
io = await ws.execute("grep zeta /dbx/words.txt")
|
||||
|
||||
assert io.exit_code != 0
|
||||
@@ -0,0 +1,34 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_c_positive_prefix(databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute("head -c 4 /dbx/words.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"beta"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_c_negative_all_but_last(databricks_text_workspace):
|
||||
# GNU `head -c -N` emits all but the last N bytes. words.txt is 17 bytes,
|
||||
# so -c -6 yields the first 11. A negative -c must not take the prefix
|
||||
# fast path (range read 0..-6 is meaningless).
|
||||
io = await databricks_text_workspace.execute("head -c -6 /dbx/words.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"beta\nalpha\n"
|
||||
@@ -0,0 +1,33 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_workspace_execute_databricks_volume_jq(
|
||||
databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute("jq -r .name /dbx/data.json")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"mirage\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_workspace_execute_databricks_volume_jq_resolves_glob(
|
||||
databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute("jq -r .name /dbx/*.json")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"mirage\nagent\n"
|
||||
@@ -0,0 +1,81 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import MountMode, Workspace
|
||||
from tests.resource.databricks_volume.test_databricks_volume import (
|
||||
FakeFiles, make_resource, seed_directory, seed_file)
|
||||
|
||||
ROOT = "/Volumes/main/default/agent_files/root"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def dbx_files() -> FakeFiles:
|
||||
files = FakeFiles()
|
||||
seed_directory(files, ROOT)
|
||||
seed_file(files, f"{ROOT}/words.txt", b"beta\nalpha\nalpha\n")
|
||||
seed_file(files, f"{ROOT}/.hidden", b"h\n")
|
||||
files.create_directory(f"{ROOT}/sub")
|
||||
seed_file(files, f"{ROOT}/sub/inner.txt", b"gamma\nalpha\n")
|
||||
return files
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def ws(dbx_files: FakeFiles) -> Workspace:
|
||||
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_lists_entries(ws):
|
||||
io = await ws.execute("ls /dbx/")
|
||||
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "words.txt" in out
|
||||
assert "sub" in out
|
||||
assert ".hidden" not in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_a_includes_hidden(ws):
|
||||
io = await ws.execute("ls -a /dbx/")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert ".hidden" in io.stdout.decode()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_long_includes_size(ws):
|
||||
io = await ws.execute("ls -l /dbx/")
|
||||
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "words.txt" in out
|
||||
assert "17" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_recursive_descends_subdirs(ws):
|
||||
io = await ws.execute("ls -R /dbx/")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert "inner.txt" in io.stdout.decode()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_missing_path_warns(ws):
|
||||
io = await ws.execute("ls /dbx/missing")
|
||||
|
||||
assert io.exit_code != 0
|
||||
@@ -0,0 +1,93 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import MountMode, Workspace
|
||||
from tests.resource.databricks_volume.test_databricks_volume import (
|
||||
FakeFiles, make_resource, seed_directory)
|
||||
|
||||
ROOT = "/Volumes/main/default/agent_files/root"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def dbx_files() -> FakeFiles:
|
||||
files = FakeFiles()
|
||||
seed_directory(files, ROOT)
|
||||
return files
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def write_ws(dbx_files: FakeFiles) -> Workspace:
|
||||
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.WRITE)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def read_ws(dbx_files: FakeFiles) -> Workspace:
|
||||
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mkdir_creates_directory(write_ws, dbx_files):
|
||||
io = await write_ws.execute("mkdir /dbx/newdir")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert f"{ROOT}/newdir" in dbx_files.directory_metadata
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mkdir_parent_missing_fails(write_ws):
|
||||
io = await write_ws.execute("mkdir /dbx/a/b")
|
||||
|
||||
assert io.exit_code != 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mkdir_parents_creates_chain(write_ws, dbx_files):
|
||||
io = await write_ws.execute("mkdir -p /dbx/a/b/c")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert f"{ROOT}/a/b/c" in dbx_files.directory_metadata
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mkdir_writes_are_mount_relative(write_ws):
|
||||
io = await write_ws.execute("mkdir /dbx/newdir")
|
||||
|
||||
assert io.exit_code == 0
|
||||
for key in io.writes:
|
||||
assert key.startswith("/dbx/")
|
||||
assert not key.startswith("/dbx/dbx/")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mkdir_read_only_mount_rejected(read_ws, dbx_files):
|
||||
io = await read_ws.execute("mkdir /dbx/newdir")
|
||||
|
||||
assert io.exit_code != 0
|
||||
assert b"read-only" in io.stderr
|
||||
assert dbx_files.create_directory_calls == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ops_mkdir(write_ws, dbx_files):
|
||||
await write_ws.ops.mkdir("/dbx/opdir")
|
||||
|
||||
assert f"{ROOT}/opdir" in dbx_files.directory_metadata
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ops_mkdir_read_only_rejected(read_ws):
|
||||
with pytest.raises(PermissionError):
|
||||
await read_ws.ops.mkdir("/dbx/opdir")
|
||||
@@ -0,0 +1,156 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import MountMode, Workspace
|
||||
from tests.resource.databricks_volume.test_databricks_volume import (
|
||||
FakeFiles, make_resource, seed_directory, seed_file)
|
||||
|
||||
ROOT = "/Volumes/main/default/agent_files/root"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def dbx_files() -> FakeFiles:
|
||||
files = FakeFiles()
|
||||
seed_directory(files, ROOT)
|
||||
seed_file(files, f"{ROOT}/src.txt", b"data")
|
||||
return files
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def write_ws(dbx_files: FakeFiles) -> Workspace:
|
||||
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.WRITE)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def read_ws(dbx_files: FakeFiles) -> Workspace:
|
||||
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mv_file_moves_bytes(write_ws, dbx_files):
|
||||
io = await write_ws.execute("mv /dbx/src.txt /dbx/dst.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert dbx_files.downloads[f"{ROOT}/dst.txt"] == b"data"
|
||||
assert f"{ROOT}/src.txt" not in dbx_files.downloads
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mv_writes_both_parents_mount_relative(write_ws):
|
||||
io = await write_ws.execute("mv /dbx/src.txt /dbx/dst.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert "/dbx/src.txt" in io.writes
|
||||
assert "/dbx/dst.txt" in io.writes
|
||||
for key in io.writes:
|
||||
assert not key.startswith("/dbx/dbx/")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mv_no_clobber_skips_existing(write_ws, dbx_files):
|
||||
seed_file(dbx_files, f"{ROOT}/dst.txt", b"keep")
|
||||
|
||||
io = await write_ws.execute("mv -n /dbx/src.txt /dbx/dst.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert dbx_files.downloads[f"{ROOT}/dst.txt"] == b"keep"
|
||||
assert dbx_files.downloads[f"{ROOT}/src.txt"] == b"data"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mv_read_only_mount_rejected(read_ws, dbx_files):
|
||||
io = await read_ws.execute("mv /dbx/src.txt /dbx/dst.txt")
|
||||
|
||||
assert io.exit_code != 0
|
||||
assert b"read-only" in io.stderr
|
||||
assert dbx_files.downloads[f"{ROOT}/src.txt"] == b"data"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ops_rename(write_ws, dbx_files):
|
||||
await write_ws.ops.rename("/dbx/src.txt", "/dbx/renamed.txt")
|
||||
|
||||
assert dbx_files.downloads[f"{ROOT}/renamed.txt"] == b"data"
|
||||
assert f"{ROOT}/src.txt" not in dbx_files.downloads
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mv_onto_same_path_errors_and_preserves_file(
|
||||
write_ws, dbx_files):
|
||||
io = await write_ws.execute("mv /dbx/src.txt /dbx/src.txt")
|
||||
|
||||
assert io.exit_code != 0
|
||||
assert b"are the same file" in io.stderr
|
||||
assert dbx_files.downloads[f"{ROOT}/src.txt"] == b"data"
|
||||
assert f"{ROOT}/src.txt" not in dbx_files.delete_calls
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mv_into_dir_where_file_already_lives_errors_and_preserves_file(
|
||||
write_ws, dbx_files):
|
||||
io = await write_ws.execute("mv /dbx/src.txt /dbx/")
|
||||
|
||||
assert io.exit_code != 0
|
||||
assert b"are the same file" in io.stderr
|
||||
assert dbx_files.downloads[f"{ROOT}/src.txt"] == b"data"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ops_rename_onto_same_path_is_noop(write_ws, dbx_files):
|
||||
await write_ws.ops.rename("/dbx/src.txt", "/dbx/src.txt")
|
||||
|
||||
assert dbx_files.downloads[f"{ROOT}/src.txt"] == b"data"
|
||||
assert f"{ROOT}/src.txt" not in dbx_files.delete_calls
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mv_multiple_sources_require_directory(write_ws, dbx_files):
|
||||
seed_file(dbx_files, f"{ROOT}/a.txt", b"AAA")
|
||||
seed_file(dbx_files, f"{ROOT}/b.txt", b"BBB")
|
||||
seed_file(dbx_files, f"{ROOT}/target.txt", b"target")
|
||||
|
||||
io = await write_ws.execute("mv /dbx/a.txt /dbx/b.txt /dbx/target.txt")
|
||||
|
||||
assert io.exit_code != 0
|
||||
assert b"not a directory" in io.stderr
|
||||
assert dbx_files.downloads[f"{ROOT}/a.txt"] == b"AAA"
|
||||
assert dbx_files.downloads[f"{ROOT}/b.txt"] == b"BBB"
|
||||
assert dbx_files.downloads[f"{ROOT}/target.txt"] == b"target"
|
||||
assert f"{ROOT}/a.txt" not in dbx_files.delete_calls
|
||||
assert f"{ROOT}/b.txt" not in dbx_files.delete_calls
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mv_missing_source_reports_cannot_stat(write_ws, dbx_files):
|
||||
io = await write_ws.execute("mv /dbx/missing /dbx/missing")
|
||||
|
||||
assert io.exit_code != 0
|
||||
assert b"cannot stat" in io.stderr
|
||||
assert b"are the same file" not in io.stderr
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mv_into_itself_errors_and_preserves_source(write_ws, dbx_files):
|
||||
seed_directory(dbx_files, f"{ROOT}/d")
|
||||
seed_file(dbx_files, f"{ROOT}/d/a.txt", b"aaa")
|
||||
|
||||
io = await write_ws.execute("mv /dbx/d /dbx/d")
|
||||
|
||||
assert io.exit_code != 0
|
||||
assert b"subdirectory of itself" in io.stderr
|
||||
assert dbx_files.downloads[f"{ROOT}/d/a.txt"] == b"aaa"
|
||||
assert f"{ROOT}/d" in dbx_files.directory_metadata
|
||||
assert f"{ROOT}/d/a.txt" not in dbx_files.delete_calls
|
||||
@@ -0,0 +1,33 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_workspace_execute_databricks_volume_nl(
|
||||
databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute("nl /dbx/words.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert b" 1\tbeta\n" in io.stdout
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_workspace_execute_databricks_volume_nl_resolves_glob(
|
||||
databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute("nl /dbx/*.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert b" 1\tdelta\n" in io.stdout
|
||||
@@ -0,0 +1,41 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
from mirage.commands.builtin.databricks_volume import COMMANDS
|
||||
|
||||
TEXT_COMMANDS = {
|
||||
"awk",
|
||||
"cut",
|
||||
"diff",
|
||||
"jq",
|
||||
"nl",
|
||||
"sed",
|
||||
"sort",
|
||||
"tr",
|
||||
"uniq",
|
||||
"wc",
|
||||
}
|
||||
|
||||
|
||||
def test_databricks_volume_text_commands_registered_read_only():
|
||||
registered = [
|
||||
registered for command in COMMANDS
|
||||
for registered in command._registered_commands
|
||||
]
|
||||
names = {command.name for command in registered}
|
||||
assert TEXT_COMMANDS <= names
|
||||
for command in registered:
|
||||
if command.name in TEXT_COMMANDS:
|
||||
assert command.resource == "databricks_volume"
|
||||
assert not command.write
|
||||
@@ -0,0 +1,61 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import MountMode, Workspace
|
||||
from tests.resource.databricks_volume.test_databricks_volume import (
|
||||
FakeFiles, make_resource, seed_directory, seed_file)
|
||||
|
||||
ROOT = "/Volumes/main/default/agent_files/root"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def dbx_files() -> FakeFiles:
|
||||
files = FakeFiles()
|
||||
seed_directory(files, ROOT)
|
||||
seed_file(files, f"{ROOT}/words.txt", b"beta\nalpha\nalpha\n")
|
||||
files.create_directory(f"{ROOT}/sub")
|
||||
seed_file(files, f"{ROOT}/sub/inner.txt", b"gamma\nalpha\n")
|
||||
return files
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def ws(dbx_files: FakeFiles) -> Workspace:
|
||||
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_searches_path_recursively(ws):
|
||||
io = await ws.execute("rg alpha /dbx/")
|
||||
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "words.txt" in out
|
||||
assert "inner.txt" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_count_only(ws):
|
||||
io = await ws.execute("rg -c alpha /dbx/words.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert b"2" in io.stdout
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_no_match_exits_nonzero(ws):
|
||||
io = await ws.execute("rg zeta /dbx/words.txt")
|
||||
|
||||
assert io.exit_code != 0
|
||||
@@ -0,0 +1,93 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import MountMode, Workspace
|
||||
from tests.resource.databricks_volume.test_databricks_volume import (
|
||||
FakeFiles, make_resource, seed_directory, seed_file)
|
||||
|
||||
ROOT = "/Volumes/main/default/agent_files/root"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def dbx_files() -> FakeFiles:
|
||||
files = FakeFiles()
|
||||
seed_directory(files, ROOT)
|
||||
files.create_directory(f"{ROOT}/d")
|
||||
seed_file(files, f"{ROOT}/d/a.txt", b"a")
|
||||
files.create_directory(f"{ROOT}/d/sub")
|
||||
seed_file(files, f"{ROOT}/d/sub/b.txt", b"b")
|
||||
return files
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def write_ws(dbx_files: FakeFiles) -> Workspace:
|
||||
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.WRITE)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def read_ws(dbx_files: FakeFiles) -> Workspace:
|
||||
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rm_recursive_removes_tree(write_ws, dbx_files):
|
||||
io = await write_ws.execute("rm -r /dbx/d")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert f"{ROOT}/d" not in dbx_files.directory_metadata
|
||||
assert f"{ROOT}/d/sub/b.txt" not in dbx_files.downloads
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rm_recursive_writes_are_mount_relative(write_ws):
|
||||
io = await write_ws.execute("rm -r /dbx/d")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.writes
|
||||
for key in io.writes:
|
||||
assert key.startswith("/dbx/")
|
||||
assert not key.startswith("/dbx/dbx/")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_plain_rm_on_directory_fails(write_ws, dbx_files):
|
||||
io = await write_ws.execute("rm /dbx/d")
|
||||
|
||||
assert io.exit_code != 0
|
||||
assert f"{ROOT}/d" in dbx_files.directory_metadata
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rm_force_missing_succeeds(write_ws):
|
||||
io = await write_ws.execute("rm -f /dbx/missing.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rm_missing_fails(write_ws):
|
||||
io = await write_ws.execute("rm /dbx/missing.txt")
|
||||
|
||||
assert io.exit_code != 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rm_recursive_read_only_rejected(read_ws, dbx_files):
|
||||
io = await read_ws.execute("rm -r /dbx/d")
|
||||
|
||||
assert io.exit_code != 0
|
||||
assert b"read-only" in io.stderr
|
||||
assert f"{ROOT}/d" in dbx_files.directory_metadata
|
||||
@@ -0,0 +1,60 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.databricks_volume import COMMANDS
|
||||
|
||||
|
||||
def _sed_command():
|
||||
for cmd in COMMANDS:
|
||||
for rc in cmd._registered_commands:
|
||||
if rc.name == "sed":
|
||||
return cmd
|
||||
raise LookupError("sed not registered for databricks_volume")
|
||||
|
||||
|
||||
sed_command = _sed_command()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_workspace_execute_databricks_volume_sed(
|
||||
databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute(
|
||||
"sed s/alpha/ALPHA/g /dbx/words.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"beta\nALPHA\nALPHA\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_workspace_execute_databricks_volume_sed_resolves_glob(
|
||||
databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute(
|
||||
"sed s/delta/DELTA/ /dbx/*.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert b"DELTA\n" in io.stdout
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_databricks_volume_sed_in_place_writes_back(
|
||||
databricks_text_workspace):
|
||||
# The volume has a write op, so -i edits in place through the shared
|
||||
# builder (the old bespoke wrapper refused it unconditionally, #382).
|
||||
io = await databricks_text_workspace.execute(
|
||||
"sed -i s/alpha/ALPHA/g /dbx/words.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.writes.get("/dbx/words.txt") == b"beta\nALPHA\nALPHA\n"
|
||||
@@ -0,0 +1,33 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_workspace_execute_databricks_volume_sort(
|
||||
databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute("sort /dbx/words.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"alpha\nalpha\nbeta\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_workspace_execute_databricks_volume_sort_resolves_glob(
|
||||
databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute("sort /dbx/*.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert b"alpha\nalpha\nbeta\n" in io.stdout
|
||||
@@ -0,0 +1,41 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_stat_default_format(databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute("stat /dbx/words.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "name=words.txt" in out
|
||||
assert "size=17" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_stat_custom_format(databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute(
|
||||
"stat -c '%n %s' /dbx/words.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout.decode().strip() == "/dbx/words.txt 17"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_stat_missing_file_fails(databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute("stat /dbx/missing.txt")
|
||||
|
||||
assert io.exit_code != 0
|
||||
@@ -0,0 +1,25 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_workspace_execute_databricks_volume_tr(
|
||||
databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute(
|
||||
"cat /dbx/words.txt | tr a-z A-Z")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"BETA\nALPHA\nALPHA\n"
|
||||
@@ -0,0 +1,69 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import MountMode, Workspace
|
||||
from tests.resource.databricks_volume.test_databricks_volume import (
|
||||
FakeFiles, make_resource, seed_directory, seed_file)
|
||||
|
||||
ROOT = "/Volumes/main/default/agent_files/root"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def dbx_files() -> FakeFiles:
|
||||
files = FakeFiles()
|
||||
seed_directory(files, ROOT)
|
||||
seed_file(files, f"{ROOT}/words.txt", b"beta\nalpha\nalpha\n")
|
||||
files.create_directory(f"{ROOT}/sub")
|
||||
seed_file(files, f"{ROOT}/sub/inner.txt", b"gamma\nalpha\n")
|
||||
files.create_directory(f"{ROOT}/sub/deep")
|
||||
seed_file(files, f"{ROOT}/sub/deep/leaf.txt", b"leaf\n")
|
||||
return files
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def ws(dbx_files: FakeFiles) -> Workspace:
|
||||
return Workspace({"/dbx/": make_resource(dbx_files)}, mode=MountMode.READ)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_lists_nested_entries(ws):
|
||||
io = await ws.execute("tree /dbx/")
|
||||
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "words.txt" in out
|
||||
assert "inner.txt" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_max_depth(ws):
|
||||
io = await ws.execute("tree -L 1 /dbx/")
|
||||
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "sub" in out
|
||||
assert "inner.txt" in out
|
||||
assert "leaf.txt" not in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_dirs_only(ws):
|
||||
io = await ws.execute("tree -d /dbx/")
|
||||
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "sub" in out
|
||||
assert "words.txt" not in out
|
||||
@@ -0,0 +1,24 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_workspace_execute_databricks_volume_uniq(
|
||||
databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute("uniq /dbx/words.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"beta\nalpha\n"
|
||||
@@ -0,0 +1,33 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_workspace_execute_databricks_volume_wc(
|
||||
databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute("wc -l /dbx/words.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"3 /dbx/words.txt\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_workspace_execute_databricks_volume_wc_resolves_glob(
|
||||
databricks_text_workspace):
|
||||
io = await databricks_text_workspace.execute("wc -l /dbx/*.txt")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert b"/dbx/more.txt" in io.stdout
|
||||
@@ -0,0 +1,60 @@
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.cache.index import RAMIndexCacheStore
|
||||
from mirage.types import PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
|
||||
def document(document_id: str, name: str, slug: str, size: int = 12) -> dict:
|
||||
return {
|
||||
"id": document_id,
|
||||
"name": name,
|
||||
"doc_metadata": [{
|
||||
"name": "slug",
|
||||
"value": slug
|
||||
}],
|
||||
"enabled": True,
|
||||
"indexing_status": "completed",
|
||||
"archived": False,
|
||||
"tokens": 4,
|
||||
"data_source_type": "upload_file",
|
||||
"data_source_detail_dict": {
|
||||
"upload_file": {
|
||||
"size": size
|
||||
}
|
||||
},
|
||||
"created_at": 1716282000,
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def dify_accessor() -> SimpleNamespace:
|
||||
return SimpleNamespace(config=SimpleNamespace(slug_metadata_name="slug"))
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def dify_index() -> RAMIndexCacheStore:
|
||||
return RAMIndexCacheStore()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def knowledge_root() -> PathSpec:
|
||||
return PathSpec(resource_path=mount_key("/knowledge", "/knowledge"),
|
||||
virtual="/knowledge",
|
||||
directory="/knowledge")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def guide_path() -> PathSpec:
|
||||
return PathSpec.from_str_path(
|
||||
"/knowledge/guides/quickstart.md",
|
||||
mount_key("/knowledge/guides/quickstart.md", "/knowledge"))
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def guides_path() -> PathSpec:
|
||||
return PathSpec(resource_path=mount_key("/knowledge/guides", "/knowledge"),
|
||||
virtual="/knowledge/guides",
|
||||
directory="/knowledge/guides")
|
||||
@@ -0,0 +1,76 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.dify.cat import make_cat
|
||||
from mirage.commands.builtin.generic_bind import CommandIO, with_read_cache
|
||||
from mirage.core.dify import read, tree
|
||||
from mirage.core.dify.read import read_bytes as _read_bytes
|
||||
from mirage.core.dify.read import read_stream as _read_stream
|
||||
from mirage.core.dify.readdir import readdir as _readdir
|
||||
from mirage.core.dify.stat import stat as _stat
|
||||
from mirage.io.types import materialize
|
||||
from mirage.types import PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
from .conftest import document
|
||||
|
||||
cat = make_cat(
|
||||
with_read_cache(
|
||||
CommandIO(
|
||||
readdir=_readdir,
|
||||
read_bytes=_read_bytes,
|
||||
read_stream=_read_stream,
|
||||
stat=_stat,
|
||||
is_mounted=lambda a: True,
|
||||
local=False,
|
||||
)))
|
||||
|
||||
|
||||
async def list_basic_documents(config):
|
||||
return [
|
||||
document("doc-1", "Guide", "guides/quickstart.md"),
|
||||
document("doc-2", "Readme", "README.md"),
|
||||
]
|
||||
|
||||
|
||||
async def iter_basic_pages(config, document_id):
|
||||
if document_id == "doc-1":
|
||||
yield [{"content": "alpha\nbeta"}, {"content": "gamma"}]
|
||||
else:
|
||||
yield [{"content": "readme"}]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_reads_stream_and_records_cache(monkeypatch, dify_accessor,
|
||||
dify_index, guide_path):
|
||||
monkeypatch.setattr(tree, "list_all_documents", list_basic_documents)
|
||||
monkeypatch.setattr(read, "iter_segment_pages", iter_basic_pages)
|
||||
|
||||
stdout, io = await cat(dify_accessor, [guide_path], index=dify_index)
|
||||
|
||||
assert await materialize(stdout) == b"alpha\nbeta\ngamma"
|
||||
assert guide_path.mount_path in io.reads
|
||||
assert io.cache == [guide_path.mount_path]
|
||||
|
||||
|
||||
async def get_two_doc_segments(config, document_id):
|
||||
if document_id == "doc-1":
|
||||
return [{"content": "alpha\nbeta"}, {"content": "gamma"}]
|
||||
return [{"content": "readme"}]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_multifile_caches_materialized_bytes_per_file(
|
||||
monkeypatch, dify_accessor, dify_index, guide_path):
|
||||
monkeypatch.setattr(tree, "list_all_documents", list_basic_documents)
|
||||
monkeypatch.setattr(read, "get_document_segments", get_two_doc_segments)
|
||||
|
||||
readme_path = PathSpec.from_str_path(
|
||||
"/knowledge/README.md", mount_key("/knowledge/README.md",
|
||||
"/knowledge"))
|
||||
stdout, io = await cat(dify_accessor, [guide_path, readme_path],
|
||||
index=dify_index)
|
||||
|
||||
assert io.reads[guide_path.mount_path] == b"alpha\nbeta\ngamma"
|
||||
assert io.reads[readme_path.mount_path] == b"readme"
|
||||
assert all(isinstance(v, bytes) for v in io.reads.values())
|
||||
assert await materialize(stdout) == b"alpha\nbeta\ngammareadme"
|
||||
@@ -0,0 +1,98 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.dify.find import find
|
||||
from mirage.core.dify import tree
|
||||
from mirage.io.types import materialize
|
||||
from mirage.types import PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
from .conftest import document
|
||||
|
||||
|
||||
async def list_basic_documents(config):
|
||||
return [
|
||||
document("doc-1", "Guide", "guides/quickstart.md"),
|
||||
document("doc-2", "Readme", "README.md"),
|
||||
]
|
||||
|
||||
|
||||
async def list_nested_documents(config):
|
||||
return [
|
||||
document("doc-1", "Guide", "guides/quickstart.md"),
|
||||
document("doc-2", "Guide 2", "guides/deep/note.md"),
|
||||
document("doc-3", "Readme", "README.md"),
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_matches_name_pattern(monkeypatch, dify_accessor,
|
||||
dify_index, knowledge_root):
|
||||
monkeypatch.setattr(tree, "list_all_documents", list_basic_documents)
|
||||
|
||||
stdout, _ = await find(dify_accessor, [knowledge_root],
|
||||
"quick*.md",
|
||||
index=dify_index)
|
||||
|
||||
assert await materialize(stdout) == b"/knowledge/guides/quickstart.md\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_handles_file_missing_and_maxdepth(monkeypatch,
|
||||
dify_accessor,
|
||||
dify_index,
|
||||
knowledge_root,
|
||||
guide_path):
|
||||
monkeypatch.setattr(tree, "list_all_documents", list_basic_documents)
|
||||
|
||||
file_stdout, file_io = await find(dify_accessor, [guide_path],
|
||||
index=dify_index)
|
||||
assert await materialize(file_stdout
|
||||
) == b"/knowledge/guides/quickstart.md\n"
|
||||
assert file_io.exit_code == 0
|
||||
|
||||
maxdepth_stdout, maxdepth_io = await find(dify_accessor, [knowledge_root],
|
||||
maxdepth="0",
|
||||
index=dify_index)
|
||||
assert await materialize(maxdepth_stdout) == b"/knowledge\n"
|
||||
assert maxdepth_io.exit_code == 0
|
||||
|
||||
missing = PathSpec.from_str_path(
|
||||
"/knowledge/missing.md",
|
||||
mount_key("/knowledge/missing.md", "/knowledge"))
|
||||
missing_stdout, missing_io = await find(dify_accessor, [missing],
|
||||
index=dify_index)
|
||||
assert await materialize(missing_stdout) == b""
|
||||
assert missing_io.stderr is not None
|
||||
assert b"/knowledge/missing.md" in missing_io.stderr
|
||||
assert missing_io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_uses_cwd_when_path_missing(monkeypatch, dify_accessor,
|
||||
dify_index, guides_path):
|
||||
monkeypatch.setattr(tree, "list_all_documents", list_nested_documents)
|
||||
|
||||
stdout, io = await find(dify_accessor, [],
|
||||
"quick*.md",
|
||||
cwd=guides_path,
|
||||
index=dify_index)
|
||||
|
||||
assert await materialize(stdout) == b"/knowledge/guides/quickstart.md\n"
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_resolves_glob_patterns(monkeypatch, dify_accessor,
|
||||
dify_index):
|
||||
monkeypatch.setattr(tree, "list_all_documents", list_nested_documents)
|
||||
path = PathSpec(resource_path=mount_key("/knowledge/guides/*.md",
|
||||
"/knowledge"),
|
||||
virtual="/knowledge/guides/*.md",
|
||||
directory="/knowledge/guides",
|
||||
pattern="*.md",
|
||||
resolved=False)
|
||||
|
||||
stdout, io = await find(dify_accessor, [path], index=dify_index)
|
||||
|
||||
assert await materialize(stdout) == b"/knowledge/guides/quickstart.md\n"
|
||||
assert io.exit_code == 0
|
||||
@@ -0,0 +1,108 @@
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.cache.index import RAMIndexCacheStore
|
||||
from mirage.io.types import materialize
|
||||
from mirage.types import PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
|
||||
def document(document_id: str, name: str, slug: str | None = None) -> dict:
|
||||
metadata = []
|
||||
if slug is not None:
|
||||
metadata = [{"name": "slug", "value": slug}]
|
||||
return {
|
||||
"id": document_id,
|
||||
"name": name,
|
||||
"doc_metadata": metadata,
|
||||
"enabled": True,
|
||||
"indexing_status": "completed",
|
||||
"archived": False,
|
||||
"tokens": 4,
|
||||
"data_source_type": "upload_file",
|
||||
"data_source_detail_dict": {
|
||||
"upload_file": {
|
||||
"size": 12
|
||||
}
|
||||
},
|
||||
"created_at": 1716282000,
|
||||
}
|
||||
|
||||
|
||||
def accessor() -> SimpleNamespace:
|
||||
return SimpleNamespace(config=SimpleNamespace(slug_metadata_name="slug"))
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_command_resolves_globs_and_passes_multiple_documents(
|
||||
monkeypatch):
|
||||
from mirage.commands.builtin.dify import search as command_search
|
||||
from mirage.core.dify import search, tree
|
||||
|
||||
calls: list[tuple[list[PathSpec], dict]] = []
|
||||
|
||||
async def list_documents(config):
|
||||
return [
|
||||
document("doc-1", "API", "guides/api.md"),
|
||||
document("doc-2", "Auth", "guides/auth.md"),
|
||||
]
|
||||
|
||||
async def search_segments(accessor, query, paths, index, **kwargs):
|
||||
calls.append((paths, kwargs))
|
||||
return b"api\nauth"
|
||||
|
||||
monkeypatch.setattr(tree, "list_all_documents", list_documents)
|
||||
monkeypatch.setattr(search, "search_segments", search_segments)
|
||||
|
||||
stdout, io = await command_search(
|
||||
accessor(),
|
||||
[
|
||||
PathSpec(resource_path=mount_key("/knowledge/guides/*.md",
|
||||
"/knowledge"),
|
||||
virtual="/knowledge/guides/*.md",
|
||||
directory="/knowledge/guides",
|
||||
pattern="*.md",
|
||||
resolved=False)
|
||||
],
|
||||
"login",
|
||||
index=RAMIndexCacheStore(),
|
||||
)
|
||||
|
||||
assert await materialize(stdout) == b"api\nauth"
|
||||
assert io.reads == {}
|
||||
assert io.cache == []
|
||||
assert [path.virtual for path in calls[0][0]] == [
|
||||
"/knowledge/guides/api.md",
|
||||
"/knowledge/guides/auth.md",
|
||||
]
|
||||
assert calls[0][1]["mount_prefix"] == "/knowledge"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_command_root_searches_whole_dataset(monkeypatch):
|
||||
from mirage.commands.builtin.dify import search as command_search
|
||||
from mirage.core.dify import search
|
||||
|
||||
calls: list[tuple[list[PathSpec], dict]] = []
|
||||
|
||||
async def search_segments(accessor, query, paths, index, **kwargs):
|
||||
calls.append((paths, kwargs))
|
||||
return b"dataset"
|
||||
|
||||
monkeypatch.setattr(search, "search_segments", search_segments)
|
||||
root = PathSpec(resource_path=mount_key("/knowledge", "/knowledge"),
|
||||
virtual="/knowledge",
|
||||
directory="/knowledge")
|
||||
|
||||
stdout, _ = await command_search(accessor(), [root],
|
||||
"anything",
|
||||
index=RAMIndexCacheStore())
|
||||
|
||||
assert await materialize(stdout) == b"dataset"
|
||||
assert calls == [([], {
|
||||
"method": "semantic",
|
||||
"top_k": 10,
|
||||
"threshold": 0.0,
|
||||
"mount_prefix": "/knowledge"
|
||||
})]
|
||||
@@ -0,0 +1,53 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.dify import COMMANDS
|
||||
from mirage.core.dify import read, tree
|
||||
from mirage.io.types import materialize
|
||||
|
||||
from .conftest import document
|
||||
|
||||
|
||||
def _sed_command():
|
||||
for cmd in COMMANDS:
|
||||
for rc in cmd._registered_commands:
|
||||
if rc.name == "sed":
|
||||
return cmd
|
||||
raise LookupError("sed not registered for dify")
|
||||
|
||||
|
||||
sed = _sed_command()
|
||||
|
||||
|
||||
async def list_documents(config):
|
||||
return [document("doc-1", "Guide", "guides/quickstart.md")]
|
||||
|
||||
|
||||
async def get_segments(config, document_id):
|
||||
return [{"content": "alpha beta"}]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_transforms_dify_document(monkeypatch, dify_accessor,
|
||||
dify_index, guide_path):
|
||||
monkeypatch.setattr(tree, "list_all_documents", list_documents)
|
||||
monkeypatch.setattr(read, "get_document_segments", get_segments)
|
||||
|
||||
stdout, io = await sed(dify_accessor, [guide_path],
|
||||
"s/alpha/gamma/",
|
||||
index=dify_index)
|
||||
|
||||
assert await materialize(stdout) == b"gamma beta"
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_rejects_in_place(monkeypatch, dify_accessor, dify_index,
|
||||
guide_path):
|
||||
monkeypatch.setattr(tree, "list_all_documents", list_documents)
|
||||
|
||||
with pytest.raises(PermissionError,
|
||||
match="-i not supported on this backend"):
|
||||
await sed(dify_accessor, [guide_path],
|
||||
"s/alpha/gamma/",
|
||||
i=True,
|
||||
index=dify_index)
|
||||
@@ -0,0 +1,13 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
@@ -0,0 +1,76 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.accessor.discord import DiscordAccessor
|
||||
from mirage.cache.index.ram import RAMIndexCacheStore
|
||||
from mirage.commands.builtin.discord import COMMANDS
|
||||
from mirage.resource.discord.config import DiscordConfig
|
||||
from mirage.types import PathSpec
|
||||
|
||||
GUILDS = [{"id": "G1", "name": "myguild"}]
|
||||
|
||||
GUILD_DIR = "/myguild__G1"
|
||||
|
||||
|
||||
def _find_command():
|
||||
for fn in COMMANDS:
|
||||
for rc in getattr(fn, "_registered_commands", []):
|
||||
if rc.name == "find" and rc.filetype is None:
|
||||
return fn
|
||||
raise AssertionError("factory find not registered for discord")
|
||||
|
||||
|
||||
def _spec(virtual: str) -> PathSpec:
|
||||
return PathSpec(virtual=virtual,
|
||||
directory=virtual,
|
||||
resource_path=virtual.strip("/"))
|
||||
|
||||
|
||||
async def _run(paths, *texts: str, **flags) -> list[str]:
|
||||
accessor = DiscordAccessor(DiscordConfig(token="t"))
|
||||
find = _find_command()
|
||||
with patch("mirage.core.discord.readdir.list_guilds",
|
||||
new_callable=AsyncMock,
|
||||
return_value=GUILDS):
|
||||
stdout, _io = await find(accessor,
|
||||
paths,
|
||||
*texts,
|
||||
index=RAMIndexCacheStore(),
|
||||
**flags)
|
||||
data = stdout if isinstance(stdout, bytes) else b""
|
||||
return data.decode().splitlines()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_lists_guild_tree():
|
||||
lines = await _run([_spec("/")], maxdepth="2")
|
||||
assert GUILD_DIR in lines
|
||||
assert f"{GUILD_DIR}/channels" in lines
|
||||
assert f"{GUILD_DIR}/members" in lines
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_path_pattern_is_honored():
|
||||
lines = await _run([_spec("/")], maxdepth="2", path="*channels*")
|
||||
assert lines == [f"{GUILD_DIR}/channels"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_size_is_honored_dirs_count_as_zero():
|
||||
lines = await _run([_spec("/")], maxdepth="2", size="+0c")
|
||||
assert lines == []
|
||||
@@ -0,0 +1,252 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.discord.grep import grep
|
||||
from mirage.commands.builtin.discord.rg import rg
|
||||
from mirage.types import FileStat, FileType, PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
|
||||
def _concrete_paths(n: int = 7):
|
||||
paths = []
|
||||
for d in range(1, n + 1):
|
||||
original = (
|
||||
f"/discord/myguild/channels/general/2026-01-{d:02d}/chat.jsonl")
|
||||
paths.append(
|
||||
PathSpec(
|
||||
resource_path=mount_key(original, "/discord"),
|
||||
virtual=original,
|
||||
directory=original,
|
||||
))
|
||||
return paths
|
||||
|
||||
|
||||
def _fake_index(channel_id: str = "ch_456", guild_id: str = "g_123"):
|
||||
idx = AsyncMock()
|
||||
|
||||
async def _get(virtual_key):
|
||||
result = AsyncMock()
|
||||
if virtual_key.endswith("/myguild/channels/general"):
|
||||
result.entry = type("E", (), {"id": channel_id})
|
||||
elif virtual_key.endswith("/myguild"):
|
||||
result.entry = type("E", (), {"id": guild_id})
|
||||
else:
|
||||
result.entry = None
|
||||
return result
|
||||
|
||||
idx.get.side_effect = _get
|
||||
return idx
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_discord_grep_with_many_concrete_paths_uses_native_search():
|
||||
accessor = AsyncMock()
|
||||
accessor.config = AsyncMock()
|
||||
fake_msgs = [{
|
||||
"content": "hello world",
|
||||
"channel_id": "ch_456",
|
||||
"author": {
|
||||
"username": "alice"
|
||||
},
|
||||
"timestamp": "2026-01-15T12:34:56.000000+00:00",
|
||||
"id": "1"
|
||||
}]
|
||||
fake_channels = [{"id": "ch_456", "name": "general"}]
|
||||
with patch(
|
||||
"mirage.commands.builtin.discord.grep.search_guild",
|
||||
new=AsyncMock(return_value=fake_msgs),
|
||||
) as fake_search, patch(
|
||||
"mirage.commands.builtin.discord.grep.list_channels",
|
||||
new=AsyncMock(return_value=fake_channels),
|
||||
):
|
||||
out, io = await grep(accessor,
|
||||
_concrete_paths(7),
|
||||
"hello",
|
||||
index=_fake_index())
|
||||
assert fake_search.await_count == 1
|
||||
assert io.exit_code == 0
|
||||
assert b"hello" in out
|
||||
assert out.endswith(b"\n")
|
||||
assert (b"/discord/myguild/channels/general__ch_456/"
|
||||
b"2026-01-15/chat.jsonl:") in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_discord_grep_falls_back_when_native_raises():
|
||||
accessor = AsyncMock()
|
||||
accessor.config = AsyncMock()
|
||||
paths = [
|
||||
PathSpec(resource_path=mount_key(
|
||||
"/discord/myguild/channels/general/*.jsonl", "/discord"),
|
||||
virtual="/discord/myguild/channels/general/*.jsonl",
|
||||
directory="/discord/myguild/channels/general/",
|
||||
pattern="*.jsonl"),
|
||||
]
|
||||
with patch(
|
||||
"mirage.commands.builtin.discord.grep.search_guild",
|
||||
new=AsyncMock(side_effect=RuntimeError("rate limited")),
|
||||
), patch(
|
||||
"mirage.commands.builtin.discord.grep.resolve_glob",
|
||||
new=AsyncMock(return_value=paths),
|
||||
) as fake_resolve, patch(
|
||||
"mirage.commands.builtin.discord.grep.discord_read",
|
||||
new=AsyncMock(return_value=b""),
|
||||
), patch(
|
||||
"mirage.commands.builtin.discord.grep._stat",
|
||||
new=AsyncMock(return_value=FileStat(name="2026-04-10.jsonl",
|
||||
type=FileType.TEXT)),
|
||||
):
|
||||
out, io = await grep(accessor, paths, "hello", index=_fake_index())
|
||||
assert fake_resolve.await_count == 1
|
||||
assert io.exit_code in (0, 1)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_discord_grep_native_empty_does_not_trigger_fallback():
|
||||
"""search_guild returning [] is a legit no-match — don't double-scan."""
|
||||
accessor = AsyncMock()
|
||||
accessor.config = AsyncMock()
|
||||
with patch(
|
||||
"mirage.commands.builtin.discord.grep.search_guild",
|
||||
new=AsyncMock(return_value=[]),
|
||||
) as fake_search, patch(
|
||||
"mirage.commands.builtin.discord.grep.list_channels",
|
||||
new=AsyncMock(return_value=[]),
|
||||
), patch(
|
||||
"mirage.commands.builtin.discord.grep.discord_read",
|
||||
new=AsyncMock(return_value=b""),
|
||||
) as fake_read:
|
||||
out, io = await grep(accessor,
|
||||
_concrete_paths(7),
|
||||
"missing",
|
||||
index=_fake_index())
|
||||
assert fake_search.await_count == 1
|
||||
assert fake_read.await_count == 0
|
||||
assert io.exit_code == 1
|
||||
assert out == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_discord_grep_multi_pattern_skips_native_search():
|
||||
"""grep -e a -e b must bypass the native search push-down.
|
||||
|
||||
The push-down passes a single newline-joined pattern to the native
|
||||
search, which treats it as one literal and matches nothing. Multiple
|
||||
-e patterns must fall through to the generic grep instead.
|
||||
"""
|
||||
accessor = AsyncMock()
|
||||
accessor.config = AsyncMock()
|
||||
paths = [
|
||||
PathSpec(resource_path=mount_key(
|
||||
"/discord/myguild/channels/general/*.jsonl", "/discord"),
|
||||
virtual="/discord/myguild/channels/general/*.jsonl",
|
||||
directory="/discord/myguild/channels/general/",
|
||||
pattern="*.jsonl"),
|
||||
]
|
||||
with patch(
|
||||
"mirage.commands.builtin.discord.grep.search_guild",
|
||||
new=AsyncMock(return_value=[]),
|
||||
) as fake_search, patch(
|
||||
"mirage.commands.builtin.discord.grep.resolve_glob",
|
||||
new=AsyncMock(return_value=paths),
|
||||
) as fake_resolve, patch(
|
||||
"mirage.commands.builtin.discord.grep.discord_read",
|
||||
new=AsyncMock(return_value=b""),
|
||||
), patch(
|
||||
"mirage.commands.builtin.discord.grep._stat",
|
||||
new=AsyncMock(return_value=FileStat(name="2026-04-10.jsonl",
|
||||
type=FileType.TEXT)),
|
||||
):
|
||||
_, io = await grep(accessor,
|
||||
paths,
|
||||
e=["ada", "ben"],
|
||||
index=_fake_index())
|
||||
assert fake_search.await_count == 0
|
||||
assert fake_resolve.await_count == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_discord_rg_with_many_concrete_paths_uses_native_search():
|
||||
accessor = AsyncMock()
|
||||
accessor.config = AsyncMock()
|
||||
fake_msgs = [{
|
||||
"content": "hello rg",
|
||||
"channel_id": "ch_456",
|
||||
"author": {
|
||||
"username": "bob"
|
||||
},
|
||||
"timestamp": "2026-01-15T08:00:00.000000+00:00",
|
||||
"id": "2"
|
||||
}]
|
||||
fake_channels = [{"id": "ch_456", "name": "general"}]
|
||||
with patch(
|
||||
"mirage.commands.builtin.discord.rg.search_guild",
|
||||
new=AsyncMock(return_value=fake_msgs),
|
||||
) as fake_search, patch(
|
||||
"mirage.commands.builtin.discord.rg.list_channels",
|
||||
new=AsyncMock(return_value=fake_channels),
|
||||
):
|
||||
out, io = await rg(accessor,
|
||||
_concrete_paths(7),
|
||||
"hello",
|
||||
index=_fake_index())
|
||||
assert fake_search.await_count == 1
|
||||
assert io.exit_code == 0
|
||||
assert b"hello" in out
|
||||
assert out.endswith(b"\n")
|
||||
assert (b"/discord/myguild/channels/general__ch_456/"
|
||||
b"2026-01-15/chat.jsonl:") in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_discord_rg_multi_pattern_skips_native_search():
|
||||
"""rg -e a -e b must bypass the native search push-down.
|
||||
|
||||
Like grep, the push-down passes a single newline-joined pattern to the
|
||||
native search, which matches nothing. Multiple -e patterns must fall
|
||||
through to the generic rg instead.
|
||||
"""
|
||||
accessor = AsyncMock()
|
||||
accessor.config = AsyncMock()
|
||||
paths = [
|
||||
PathSpec(resource_path=mount_key(
|
||||
"/discord/myguild/channels/general/*.jsonl", "/discord"),
|
||||
virtual="/discord/myguild/channels/general/*.jsonl",
|
||||
directory="/discord/myguild/channels/general/",
|
||||
pattern="*.jsonl"),
|
||||
]
|
||||
with patch(
|
||||
"mirage.commands.builtin.discord.rg.search_guild",
|
||||
new=AsyncMock(return_value=[]),
|
||||
) as fake_search, patch(
|
||||
"mirage.commands.builtin.discord.rg.resolve_glob",
|
||||
new=AsyncMock(return_value=paths),
|
||||
) as fake_resolve, patch(
|
||||
"mirage.commands.builtin.discord.rg.discord_read",
|
||||
new=AsyncMock(return_value=b""),
|
||||
), patch(
|
||||
"mirage.commands.builtin.discord.rg._stat",
|
||||
new=AsyncMock(return_value=FileStat(name="2026-04-10.jsonl",
|
||||
type=FileType.TEXT)),
|
||||
):
|
||||
_, io = await rg(accessor,
|
||||
paths,
|
||||
e=["ada", "ben"],
|
||||
index=_fake_index())
|
||||
assert fake_search.await_count == 0
|
||||
assert fake_resolve.await_count == 1
|
||||
@@ -0,0 +1,232 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import asyncio
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.accessor.discord import DiscordAccessor
|
||||
from mirage.cache.index import IndexEntry
|
||||
from mirage.cache.index.ram import RAMIndexCacheStore
|
||||
from mirage.commands.builtin.discord import COMMANDS
|
||||
from mirage.commands.builtin.discord.grep import grep
|
||||
from mirage.commands.builtin.discord.head import head
|
||||
from mirage.resource.discord.config import DiscordConfig
|
||||
from mirage.types import PathSpec
|
||||
|
||||
GUILD_PATH = "TestGuild"
|
||||
CHANNEL_PATH = "TestGuild/channels/general"
|
||||
DATE_DIR_PATH = "TestGuild/channels/general/2024-01-15"
|
||||
FILE_PATH = "TestGuild/channels/general/2024-01-15/chat.jsonl"
|
||||
ABS_FILE = "/" + FILE_PATH
|
||||
ABS_CHANNEL = "/" + CHANNEL_PATH
|
||||
ABS_DATE_DIR = "/" + DATE_DIR_PATH
|
||||
|
||||
|
||||
def _glob_result(path: str) -> list[PathSpec]:
|
||||
return [
|
||||
PathSpec(resource_path=(path).strip("/"),
|
||||
virtual=path,
|
||||
directory="/",
|
||||
resolved=True)
|
||||
]
|
||||
|
||||
|
||||
FAKE_JSONL = (
|
||||
b'{"id":"1","content":"hello world","author":{"username":"alice"}}\n'
|
||||
b'{"id":"2","content":"goodbye moon","author":{"username":"bob"}}\n'
|
||||
b'{"id":"3","content":"hello again","author":{"username":"alice"}}\n')
|
||||
|
||||
|
||||
def _make_glob(path: str, resolved: bool = True) -> list[PathSpec]:
|
||||
return [
|
||||
PathSpec(resource_path=(path).strip("/"),
|
||||
virtual=path,
|
||||
directory="/",
|
||||
resolved=resolved)
|
||||
]
|
||||
|
||||
|
||||
def _run(coro):
|
||||
return asyncio.run(coro)
|
||||
|
||||
|
||||
def _find_command():
|
||||
for fn in COMMANDS:
|
||||
for rc in getattr(fn, "_registered_commands", []):
|
||||
if rc.name == "find" and rc.filetype is None:
|
||||
return fn
|
||||
raise AssertionError("factory find not registered for discord")
|
||||
|
||||
|
||||
def _make_index() -> RAMIndexCacheStore:
|
||||
index = RAMIndexCacheStore(ttl=600)
|
||||
_run(
|
||||
index.put(
|
||||
"/" + GUILD_PATH,
|
||||
IndexEntry(id="G1",
|
||||
name="TestGuild",
|
||||
resource_type="discord/guild",
|
||||
vfs_name="TestGuild")))
|
||||
_run(
|
||||
index.put(
|
||||
"/" + CHANNEL_PATH,
|
||||
IndexEntry(id="C1",
|
||||
name="general",
|
||||
resource_type="discord/channel",
|
||||
vfs_name="general")))
|
||||
_run(
|
||||
index.put(
|
||||
"/" + DATE_DIR_PATH,
|
||||
IndexEntry(id="C1:2024-01-15",
|
||||
name="2024-01-15",
|
||||
resource_type="discord/history",
|
||||
vfs_name="2024-01-15")))
|
||||
_run(
|
||||
index.set_dir("/" + CHANNEL_PATH, [
|
||||
("2024-01-15",
|
||||
IndexEntry(id="C1:2024-01-15",
|
||||
name="2024-01-15",
|
||||
resource_type="discord/history",
|
||||
vfs_name="2024-01-15")),
|
||||
]))
|
||||
_run(
|
||||
index.put(
|
||||
"/" + FILE_PATH,
|
||||
IndexEntry(id="C1:2024-01-15:chat",
|
||||
name="chat.jsonl",
|
||||
resource_type="discord/chat_jsonl",
|
||||
vfs_name="chat.jsonl")))
|
||||
_run(
|
||||
index.set_dir("/" + DATE_DIR_PATH, [
|
||||
("chat.jsonl",
|
||||
IndexEntry(id="C1:2024-01-15:chat",
|
||||
name="chat.jsonl",
|
||||
resource_type="discord/chat_jsonl",
|
||||
vfs_name="chat.jsonl")),
|
||||
]))
|
||||
return index
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def accessor():
|
||||
config = DiscordConfig(token="test-token")
|
||||
return DiscordAccessor(config=config)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def index():
|
||||
return _make_index()
|
||||
|
||||
|
||||
async def _collect(stream) -> bytes:
|
||||
if isinstance(stream, bytes):
|
||||
return stream
|
||||
chunks = []
|
||||
async for chunk in stream:
|
||||
chunks.append(chunk)
|
||||
return b"".join(chunks)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head(accessor):
|
||||
with (
|
||||
patch("mirage.commands.builtin.discord.head.resolve_glob",
|
||||
new_callable=AsyncMock,
|
||||
return_value=_glob_result(ABS_FILE)),
|
||||
patch("mirage.commands.builtin.discord.head.discord_read",
|
||||
new_callable=AsyncMock,
|
||||
return_value=FAKE_JSONL),
|
||||
):
|
||||
stream, io = await head(accessor, _make_glob(ABS_FILE), n="1")
|
||||
data = await _collect(stream)
|
||||
assert b"hello world" in data
|
||||
assert b"goodbye moon" not in data
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_default(accessor):
|
||||
with (
|
||||
patch("mirage.commands.builtin.discord.head.resolve_glob",
|
||||
new_callable=AsyncMock,
|
||||
return_value=_glob_result(ABS_FILE)),
|
||||
patch("mirage.commands.builtin.discord.head.discord_read",
|
||||
new_callable=AsyncMock,
|
||||
return_value=FAKE_JSONL),
|
||||
):
|
||||
stream, io = await head(accessor, _make_glob(ABS_FILE))
|
||||
data = await _collect(stream)
|
||||
assert b"hello world" in data
|
||||
assert b"goodbye moon" in data
|
||||
assert b"hello again" in data
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep(accessor):
|
||||
with (
|
||||
patch("mirage.commands.builtin.discord.grep.resolve_glob",
|
||||
new_callable=AsyncMock,
|
||||
return_value=_glob_result(ABS_FILE)),
|
||||
patch("mirage.commands.builtin.discord.grep.discord_read",
|
||||
new_callable=AsyncMock,
|
||||
return_value=FAKE_JSONL),
|
||||
):
|
||||
stream, io = await grep(accessor, _make_glob(ABS_FILE), "hello")
|
||||
data = await _collect(stream)
|
||||
assert b"hello world" in data
|
||||
assert b"hello again" in data
|
||||
assert b"goodbye" not in data
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_invert(accessor):
|
||||
with (
|
||||
patch("mirage.commands.builtin.discord.grep.resolve_glob",
|
||||
new_callable=AsyncMock,
|
||||
return_value=_glob_result(ABS_FILE)),
|
||||
patch("mirage.commands.builtin.discord.grep.discord_read",
|
||||
new_callable=AsyncMock,
|
||||
return_value=FAKE_JSONL),
|
||||
):
|
||||
stream, io = await grep(accessor,
|
||||
_make_glob(ABS_FILE),
|
||||
"hello",
|
||||
v=True)
|
||||
data = await _collect(stream)
|
||||
assert b"goodbye moon" in data
|
||||
assert b"hello world" not in data
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find(accessor, index):
|
||||
find = _find_command()
|
||||
stream, io = await find(accessor,
|
||||
_make_glob(ABS_CHANNEL, resolved=False),
|
||||
index=index)
|
||||
data = await _collect(stream)
|
||||
assert b"2024-01-15" in data
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_with_name(accessor, index):
|
||||
find = _find_command()
|
||||
stream, io = await find(
|
||||
accessor,
|
||||
_make_glob(ABS_CHANNEL, resolved=False),
|
||||
name="chat.jsonl",
|
||||
index=index,
|
||||
)
|
||||
data = await _collect(stream)
|
||||
assert b"chat.jsonl" in data
|
||||
@@ -0,0 +1,93 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.discord.grep import grep
|
||||
from mirage.commands.builtin.discord.rg import rg
|
||||
from mirage.io.types import IOResult
|
||||
from mirage.types import PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
|
||||
def _path(path: str) -> PathSpec:
|
||||
return PathSpec(resource_path=mount_key(path, "/discord"),
|
||||
virtual=path,
|
||||
directory=path)
|
||||
|
||||
|
||||
def _fake_index():
|
||||
idx = AsyncMock()
|
||||
|
||||
async def _get(virtual_key):
|
||||
result = AsyncMock()
|
||||
if virtual_key.endswith("/myguild/channels/general"):
|
||||
result.entry = type("E", (), {"id": "C1", "remote_time": ""})
|
||||
elif virtual_key.endswith("/myguild"):
|
||||
result.entry = type("E", (), {"id": "G1"})
|
||||
else:
|
||||
result.entry = None
|
||||
return result
|
||||
|
||||
idx.get.side_effect = _get
|
||||
return idx
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_emits_token_hint_on_forbidden():
|
||||
accessor = AsyncMock()
|
||||
accessor.config = AsyncMock()
|
||||
paths = [_path("/discord/myguild/channels/general/2026-01-01/chat.jsonl")]
|
||||
with patch(
|
||||
"mirage.commands.builtin.discord.grep.search_guild",
|
||||
new=AsyncMock(side_effect=RuntimeError("403 Forbidden")),
|
||||
), patch(
|
||||
"mirage.commands.builtin.discord.grep.resolve_glob",
|
||||
new=AsyncMock(return_value=paths),
|
||||
), patch(
|
||||
"mirage.commands.builtin.discord.grep.discord_read",
|
||||
new=AsyncMock(return_value=b""),
|
||||
):
|
||||
_out, io = await grep(accessor,
|
||||
paths,
|
||||
"hi",
|
||||
index=_fake_index(),
|
||||
args_l=True)
|
||||
stderr = (io.stderr or b"").decode()
|
||||
assert "push-down failed" in stderr
|
||||
assert "READ_MESSAGE_HISTORY" in stderr
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_emits_warning_on_rate_limit():
|
||||
accessor = AsyncMock()
|
||||
accessor.config = AsyncMock()
|
||||
paths = [_path("/discord/myguild/channels/general/2026-01-01/chat.jsonl")]
|
||||
with patch(
|
||||
"mirage.commands.builtin.discord.rg.search_guild",
|
||||
new=AsyncMock(side_effect=RuntimeError("rate limited 429")),
|
||||
), patch(
|
||||
"mirage.commands.builtin.discord.rg.resolve_glob",
|
||||
new=AsyncMock(return_value=paths),
|
||||
), patch(
|
||||
"mirage.commands.builtin.discord.rg.generic_rg",
|
||||
new=AsyncMock(return_value=(b"", IOResult(exit_code=1))),
|
||||
):
|
||||
_out, io = await rg(accessor, paths, "hi", index=_fake_index())
|
||||
stderr = (io.stderr or b"").decode()
|
||||
assert "push-down failed" in stderr
|
||||
# 429 doesn't trigger the perm hint; should still warn
|
||||
assert "READ_MESSAGE_HISTORY" not in stderr
|
||||
@@ -0,0 +1,69 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import json
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.accessor.discord import DiscordAccessor
|
||||
from mirage.commands.builtin.discord.discord_add_reaction import \
|
||||
discord_add_reaction
|
||||
from mirage.commands.builtin.discord.discord_send_message import \
|
||||
discord_send_message
|
||||
from mirage.resource.discord.config import DiscordConfig
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def accessor():
|
||||
return DiscordAccessor(config=DiscordConfig(token="test-bot-token"), )
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_send_message(accessor):
|
||||
with patch(
|
||||
"mirage.commands.builtin.discord.discord_send_message"
|
||||
".send_message",
|
||||
new_callable=AsyncMock,
|
||||
return_value={
|
||||
"id": "msg1",
|
||||
"content": "hello"
|
||||
},
|
||||
) as mock_send:
|
||||
stream, io_result = await discord_send_message(accessor, [],
|
||||
channel_id="C001",
|
||||
text="hello")
|
||||
|
||||
mock_send.assert_called_once_with(accessor.config, "C001", "hello", None)
|
||||
out = json.loads(stream)
|
||||
assert out["id"] == "msg1"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_add_reaction(accessor):
|
||||
with patch(
|
||||
"mirage.commands.builtin.discord.discord_add_reaction"
|
||||
".add_reaction",
|
||||
new_callable=AsyncMock,
|
||||
return_value=None,
|
||||
) as mock_react:
|
||||
stream, io_result = await discord_add_reaction(accessor, [],
|
||||
channel_id="C001",
|
||||
message_id="msg1",
|
||||
reaction="thumbsup")
|
||||
|
||||
mock_react.assert_called_once_with(accessor.config, "C001", "msg1",
|
||||
"thumbsup")
|
||||
out = json.loads(stream)
|
||||
assert out["ok"] is True
|
||||
@@ -0,0 +1,84 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import DiskResource, MountMode, Workspace
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def workspace(tmp_path):
|
||||
return Workspace({"/": DiskResource(root=str(tmp_path))},
|
||||
mode=MountMode.WRITE)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_basic(workspace):
|
||||
await workspace.ops.write("/f.txt", b"hello\nworld\n")
|
||||
io = await workspace.execute("cat /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"hello\nworld\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_n_single_digit_alignment(workspace):
|
||||
await workspace.ops.write("/f.txt", b"a\nb\n")
|
||||
io = await workspace.execute("cat -n /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b" 1\ta\n 2\tb\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_n_multidigit_alignment(workspace):
|
||||
body = b"".join(f"line{i}\n".encode() for i in range(1, 13))
|
||||
await workspace.ops.write("/big.txt", body)
|
||||
io = await workspace.execute("cat -n /big.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
lines = io.stdout.split(b"\n")
|
||||
assert lines[0] == b" 1\tline1"
|
||||
assert lines[8] == b" 9\tline9"
|
||||
assert lines[9] == b" 10\tline10"
|
||||
assert lines[11] == b" 12\tline12"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_preserves_no_trailing_newline(workspace):
|
||||
await workspace.ops.write("/partial.txt", b"hello")
|
||||
io = await workspace.execute("cat /partial.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"hello"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_n_preserves_no_trailing_newline(workspace):
|
||||
await workspace.ops.write("/partial.txt", b"hello")
|
||||
io = await workspace.execute("cat -n /partial.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b" 1\thello"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_empty_file(workspace):
|
||||
await workspace.ops.write("/empty.txt", b"")
|
||||
io = await workspace.execute("cat /empty.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_only_newlines(workspace):
|
||||
await workspace.ops.write("/nl.txt", b"\n\n\n")
|
||||
io = await workspace.execute("cat -n /nl.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b" 1\t\n 2\t\n 3\t\n"
|
||||
@@ -0,0 +1,91 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import DiskResource, MountMode, Workspace
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def workspace(tmp_path):
|
||||
return Workspace({"/": DiskResource(root=str(tmp_path))},
|
||||
mode=MountMode.WRITE)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_single_file(workspace):
|
||||
await workspace.ops.write("/f.txt", b"hello")
|
||||
io = await workspace.execute("du /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout.decode().strip() == "5\t/f.txt"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_directory_collapses(workspace):
|
||||
await workspace.ops.mkdir("/dir")
|
||||
await workspace.ops.write("/dir/a.txt", b"aaa")
|
||||
await workspace.ops.write("/dir/b.txt", b"bb")
|
||||
io = await workspace.execute("du /dir", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout.decode().strip() == "5\t/dir"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_a_lists_files(workspace):
|
||||
await workspace.ops.mkdir("/dir")
|
||||
await workspace.ops.write("/dir/a.txt", b"aaa")
|
||||
await workspace.ops.write("/dir/b.txt", b"bb")
|
||||
io = await workspace.execute("du -a /dir", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "a.txt" in out
|
||||
assert "b.txt" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_s_summary(workspace):
|
||||
await workspace.ops.mkdir("/dir")
|
||||
await workspace.ops.mkdir("/dir/sub")
|
||||
await workspace.ops.write("/dir/a.txt", b"hello")
|
||||
await workspace.ops.write("/dir/sub/b.txt", b"world")
|
||||
io = await workspace.execute("du -s /dir", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
lines = io.stdout.decode().strip().splitlines()
|
||||
assert len(lines) == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_c_total(workspace):
|
||||
await workspace.ops.write("/a.txt", b"hello")
|
||||
await workspace.ops.write("/b.txt", b"world")
|
||||
io = await workspace.execute("du -c /a.txt /b.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
lines = io.stdout.decode().strip().splitlines()
|
||||
assert lines[-1] == "10\ttotal"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_h_human(workspace):
|
||||
await workspace.ops.write("/big.txt", b"x" * 2048)
|
||||
io = await workspace.execute("du -h /big.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
size_str = io.stdout.decode().strip().split("\t")[0]
|
||||
assert size_str.endswith("K")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_missing_operand_errors(workspace):
|
||||
io = await workspace.execute("du", session_id="default")
|
||||
assert io.exit_code != 0
|
||||
assert b"missing operand" in (io.stderr or b"")
|
||||
@@ -0,0 +1,83 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import DiskResource, MountMode, Workspace
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def workspace(tmp_path):
|
||||
return Workspace({"/": DiskResource(root=str(tmp_path))},
|
||||
mode=MountMode.WRITE)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_name_glob(workspace):
|
||||
await workspace.ops.write("/hello.txt", b"hi")
|
||||
await workspace.ops.write("/world.py", b"hi")
|
||||
io = await workspace.execute("find / -name '*.txt'", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "hello.txt" in out
|
||||
assert "world.py" not in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_type_f(workspace):
|
||||
await workspace.ops.mkdir("/sub")
|
||||
await workspace.ops.write("/a.txt", b"a")
|
||||
await workspace.ops.write("/sub/b.txt", b"b")
|
||||
io = await workspace.execute("find / -type f", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "/a.txt" in out
|
||||
assert "/sub/b.txt" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_type_d(workspace):
|
||||
await workspace.ops.mkdir("/sub")
|
||||
await workspace.ops.write("/a.txt", b"a")
|
||||
io = await workspace.execute("find / -type d", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "/sub" in out
|
||||
assert "/a.txt" not in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_size_lower_bound(workspace):
|
||||
await workspace.ops.write("/big.txt", b"x" * 1000)
|
||||
await workspace.ops.write("/small.txt", b"x")
|
||||
io = await workspace.execute("find / -size +500c -type f",
|
||||
session_id="default")
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "big.txt" in out
|
||||
assert "small.txt" not in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_maxdepth(workspace):
|
||||
await workspace.ops.mkdir("/sub")
|
||||
await workspace.ops.mkdir("/sub/deep")
|
||||
await workspace.ops.write("/a.txt", b"a")
|
||||
await workspace.ops.write("/sub/deep/c.txt", b"c")
|
||||
io = await workspace.execute("find / -maxdepth 1 -type f",
|
||||
session_id="default")
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "/a.txt" in out
|
||||
assert "/sub/deep/c.txt" not in out
|
||||
@@ -0,0 +1,75 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import DiskResource, MountMode, Workspace
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def workspace(tmp_path):
|
||||
return Workspace({"/": DiskResource(root=str(tmp_path))},
|
||||
mode=MountMode.WRITE)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_default_n_10(workspace):
|
||||
body = b"".join(f"line{i}\n".encode() for i in range(1, 15))
|
||||
await workspace.ops.write("/f.txt", body)
|
||||
io = await workspace.execute("head /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
lines = io.stdout.decode().splitlines()
|
||||
assert len(lines) == 10
|
||||
assert lines[0] == "line1"
|
||||
assert lines[9] == "line10"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_n_explicit(workspace):
|
||||
await workspace.ops.write("/f.txt", b"a\nb\nc\nd\n")
|
||||
io = await workspace.execute("head -n 2 /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"a\nb\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_c_bytes(workspace):
|
||||
await workspace.ops.write("/f.txt", b"hello world")
|
||||
io = await workspace.execute("head -c 5 /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"hello"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_negative_n_excludes_last(workspace):
|
||||
await workspace.ops.write("/f.txt", b"a\nb\nc\nd\n")
|
||||
io = await workspace.execute("head -n -1 /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"a\nb\nc\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_no_trailing_newline(workspace):
|
||||
await workspace.ops.write("/partial.txt", b"hello")
|
||||
io = await workspace.execute("head /partial.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"hello"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_empty_file(workspace):
|
||||
await workspace.ops.write("/empty.txt", b"")
|
||||
io = await workspace.execute("head /empty.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b""
|
||||
@@ -0,0 +1,90 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import DiskResource, MountMode, Workspace
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def workspace(tmp_path):
|
||||
return Workspace({"/": DiskResource(root=str(tmp_path))},
|
||||
mode=MountMode.WRITE)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_lists_files(workspace):
|
||||
await workspace.ops.write("/a.txt", b"a")
|
||||
await workspace.ops.write("/b.txt", b"b")
|
||||
io = await workspace.execute("ls /", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
names = set(io.stdout.decode().strip().split("\n"))
|
||||
assert "a.txt" in names
|
||||
assert "b.txt" in names
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_a_shows_dotfiles(workspace):
|
||||
await workspace.ops.write("/.hidden", b"h")
|
||||
await workspace.ops.write("/visible.txt", b"v")
|
||||
io = await workspace.execute("ls -a /", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
names = set(io.stdout.decode().strip().split("\n"))
|
||||
assert ".hidden" in names
|
||||
assert "visible.txt" in names
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_l_long_format_includes_size(workspace):
|
||||
await workspace.ops.write("/f.txt", b"hello")
|
||||
io = await workspace.execute("ls -l /", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "f.txt" in out
|
||||
assert "5" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_F_classify_marks_dirs(workspace):
|
||||
await workspace.ops.mkdir("/sub")
|
||||
await workspace.ops.write("/sub/a.txt", b"a")
|
||||
io = await workspace.execute("ls -F /", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert "sub/" in io.stdout.decode()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_R_recursive(workspace):
|
||||
await workspace.ops.mkdir("/sub")
|
||||
await workspace.ops.write("/sub/a.txt", b"a")
|
||||
io = await workspace.execute("ls -R /", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "sub" in out
|
||||
assert "a.txt" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_d_lists_dir_itself(workspace):
|
||||
await workspace.ops.mkdir("/sub")
|
||||
io = await workspace.execute("ls -d /sub", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert "sub" in io.stdout.decode()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_missing_path_returns_exit_1(workspace):
|
||||
io = await workspace.execute("ls /nope", session_id="default")
|
||||
assert io.exit_code == 1
|
||||
assert b"nope" in (io.stderr or b"")
|
||||
@@ -0,0 +1,33 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import DiskResource, MountMode, Workspace
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def workspace(tmp_path):
|
||||
return Workspace({"/": DiskResource(root=str(tmp_path))},
|
||||
mode=MountMode.WRITE)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rm_v_terminates_verbose_output(workspace):
|
||||
await workspace.ops.write("/a.txt", b"a")
|
||||
|
||||
io = await workspace.execute("rm -v /a.txt", session_id="default")
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"removed '/a.txt'\n"
|
||||
@@ -0,0 +1,83 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import DiskResource, MountMode, Workspace
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def workspace(tmp_path):
|
||||
return Workspace({"/": DiskResource(root=str(tmp_path))},
|
||||
mode=MountMode.WRITE)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_default_n_10(workspace):
|
||||
body = b"\n".join(f"line{i}".encode() for i in range(1, 21)) + b"\n"
|
||||
await workspace.ops.write("/f.txt", body)
|
||||
io = await workspace.execute("tail /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
expected = b"\n".join(f"line{i}".encode() for i in range(11, 21)) + b"\n"
|
||||
assert io.stdout == expected
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_n_explicit(workspace):
|
||||
await workspace.ops.write("/f.txt", b"a\nb\nc\nd\ne\n")
|
||||
io = await workspace.execute("tail -n 3 /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"c\nd\ne\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_c_bytes(workspace):
|
||||
await workspace.ops.write("/f.txt", b"hello world")
|
||||
io = await workspace.execute("tail -c 5 /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"world"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_plus_n_streams_from_line(workspace):
|
||||
await workspace.ops.write("/f.txt", b"a\nb\nc\nd\ne\n")
|
||||
io = await workspace.execute("tail -n +3 /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"c\nd\ne\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_no_trailing_newline(workspace):
|
||||
await workspace.ops.write("/partial.txt", b"hello")
|
||||
io = await workspace.execute("tail /partial.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b"hello"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_empty_file(workspace):
|
||||
await workspace.ops.write("/empty.txt", b"")
|
||||
io = await workspace.execute("tail /empty.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_multi_file_emits_headers(workspace):
|
||||
await workspace.ops.write("/a.txt", b"x\ny\n")
|
||||
await workspace.ops.write("/b.txt", b"z\n")
|
||||
io = await workspace.execute("tail /a.txt /b.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert b"==> /a.txt <==" in io.stdout
|
||||
assert b"==> /b.txt <==" in io.stdout
|
||||
@@ -0,0 +1,61 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import DiskResource, MountMode, Workspace
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def workspace(tmp_path):
|
||||
return Workspace({"/": DiskResource(root=str(tmp_path))},
|
||||
mode=MountMode.WRITE)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_basic(workspace):
|
||||
await workspace.ops.mkdir("/d1")
|
||||
await workspace.ops.write("/d1/a.txt", b"a")
|
||||
await workspace.ops.write("/d1/b.txt", b"b")
|
||||
io = await workspace.execute("tree /d1", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "a.txt" in out
|
||||
assert "b.txt" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_L_max_depth(workspace):
|
||||
await workspace.ops.mkdir("/d1")
|
||||
await workspace.ops.mkdir("/d1/sub")
|
||||
await workspace.ops.mkdir("/d1/sub/deep")
|
||||
await workspace.ops.write("/d1/sub/deep/file.txt", b"d")
|
||||
io = await workspace.execute("tree -L 1 /d1", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "sub" in out
|
||||
assert "deep" in out
|
||||
assert "file.txt" not in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_d_dirs_only(workspace):
|
||||
await workspace.ops.mkdir("/d1")
|
||||
await workspace.ops.mkdir("/d1/sub")
|
||||
await workspace.ops.write("/d1/file.txt", b"x")
|
||||
io = await workspace.execute("tree -d /d1", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
out = io.stdout.decode()
|
||||
assert "sub" in out
|
||||
assert "file.txt" not in out
|
||||
@@ -0,0 +1,96 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage import DiskResource, MountMode, Workspace
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def workspace(tmp_path):
|
||||
return Workspace({"/": DiskResource(root=str(tmp_path))},
|
||||
mode=MountMode.WRITE)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_default(workspace):
|
||||
await workspace.ops.write("/f.txt", b"hello world\nfoo bar\n")
|
||||
io = await workspace.execute("wc /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
parts = io.stdout.decode().rstrip("\n").split()
|
||||
assert parts[0] == "2"
|
||||
assert parts[1] == "4"
|
||||
assert parts[2] == "20"
|
||||
assert parts[3] == "/f.txt"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_l(workspace):
|
||||
await workspace.ops.write("/f.txt", b"a\nb\nc\n")
|
||||
io = await workspace.execute("wc -l /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout.decode().split()[0] == "3"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_w(workspace):
|
||||
await workspace.ops.write("/f.txt", b"hello world\nfoo\n")
|
||||
io = await workspace.execute("wc -w /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout.decode().split()[0] == "3"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_c(workspace):
|
||||
await workspace.ops.write("/f.txt", b"hello\n")
|
||||
io = await workspace.execute("wc -c /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout.decode().split()[0] == "6"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_m_multibyte(workspace):
|
||||
await workspace.ops.write("/f.txt", "café".encode())
|
||||
io = await workspace.execute("wc -m /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout.decode().split()[0] == "4"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_L(workspace):
|
||||
await workspace.ops.write("/f.txt", b"short\na much longer line\nmed\n")
|
||||
io = await workspace.execute("wc -L /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
assert io.stdout.decode().split()[0] == str(len("a much longer line"))
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_empty_file(workspace):
|
||||
await workspace.ops.write("/f.txt", b"")
|
||||
io = await workspace.execute("wc /f.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
parts = io.stdout.decode().split()
|
||||
assert parts[:3] == ["0", "0", "0"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_multi_file_emits_total(workspace):
|
||||
await workspace.ops.write("/a.txt", b"hello\n")
|
||||
await workspace.ops.write("/b.txt", b"world\nfoo\n")
|
||||
io = await workspace.execute("wc /a.txt /b.txt", session_id="default")
|
||||
assert io.exit_code == 0
|
||||
lines = io.stdout.decode().splitlines()
|
||||
assert any(line.endswith("/a.txt") for line in lines)
|
||||
assert any(line.endswith("/b.txt") for line in lines)
|
||||
assert lines[-1].endswith("total")
|
||||
@@ -0,0 +1,103 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.accessor.email import EmailAccessor
|
||||
from mirage.cache.index.ram import RAMIndexCacheStore
|
||||
from mirage.commands.builtin.email.find import find
|
||||
from mirage.resource.email.config import EmailConfig
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
def _spec(virtual: str) -> PathSpec:
|
||||
return PathSpec(virtual=virtual,
|
||||
directory=virtual,
|
||||
resource_path=virtual.strip("/"))
|
||||
|
||||
|
||||
def _accessor() -> EmailAccessor:
|
||||
return EmailAccessor(
|
||||
EmailConfig(imap_host="imap.test",
|
||||
smtp_host="smtp.test",
|
||||
username="u",
|
||||
password="p"))
|
||||
|
||||
|
||||
async def _run(paths, *texts: str, **flags) -> list[str]:
|
||||
with patch("mirage.core.email.readdir.list_folders",
|
||||
new_callable=AsyncMock,
|
||||
return_value=["INBOX", "Sent"]):
|
||||
stdout, _io = await find(_accessor(),
|
||||
paths,
|
||||
*texts,
|
||||
index=RAMIndexCacheStore(),
|
||||
**flags)
|
||||
data = stdout if isinstance(stdout, bytes) else b""
|
||||
return data.decode().splitlines()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_lists_folders():
|
||||
lines = await _run([_spec("/")], maxdepth="1")
|
||||
assert "/INBOX" in lines
|
||||
assert "/Sent" in lines
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_path_pattern_is_honored():
|
||||
lines = await _run([_spec("/")], maxdepth="1", path="*INBOX*")
|
||||
assert lines == ["/INBOX"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_size_is_honored_dirs_count_as_zero():
|
||||
lines = await _run([_spec("/")], maxdepth="1", size="+0c")
|
||||
assert lines == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_name_only_folder_level_pushes_down_to_imap_search():
|
||||
search = AsyncMock(return_value=[])
|
||||
with patch("mirage.commands.builtin.email.find.search_messages", search):
|
||||
stdout, _io = await find(_accessor(), [_spec("/INBOX")],
|
||||
name="*report*",
|
||||
index=RAMIndexCacheStore())
|
||||
search.assert_awaited_once()
|
||||
assert (stdout if isinstance(stdout, bytes) else b"") == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_name_with_size_falls_through_to_walk():
|
||||
# Any predicate beyond -name must not be dropped by the server-side
|
||||
# shortcut; the local walk applies all of them.
|
||||
search = AsyncMock(return_value=[])
|
||||
with patch("mirage.commands.builtin.email.find.search_messages", search), \
|
||||
patch("mirage.core.email.readdir.list_folders",
|
||||
new_callable=AsyncMock,
|
||||
return_value=["INBOX"]), \
|
||||
patch("mirage.core.email.stat.list_folders",
|
||||
new_callable=AsyncMock,
|
||||
return_value=["INBOX"]), \
|
||||
patch("mirage.core.email.readdir.list_message_uids",
|
||||
new_callable=AsyncMock,
|
||||
return_value=[]):
|
||||
stdout, _io = await find(_accessor(), [_spec("/INBOX")],
|
||||
name="*report*",
|
||||
size="+0c",
|
||||
index=RAMIndexCacheStore())
|
||||
search.assert_not_awaited()
|
||||
assert (stdout if isinstance(stdout, bytes) else b"") == b""
|
||||
@@ -0,0 +1,49 @@
|
||||
import importlib
|
||||
import sys
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.io.stream import materialize
|
||||
from mirage.types import PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
sys.modules.setdefault(
|
||||
"aioimaplib",
|
||||
SimpleNamespace(IMAP4=object, IMAP4_SSL=object),
|
||||
)
|
||||
sys.modules.setdefault(
|
||||
"aiosmtplib",
|
||||
SimpleNamespace(SMTP=object, send=AsyncMock()),
|
||||
)
|
||||
|
||||
_grep_server_side = importlib.import_module(
|
||||
"mirage.commands.builtin.email.grep")._grep_server_side
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_server_side_count_uses_real_count():
|
||||
accessor = SimpleNamespace(config=SimpleNamespace(max_messages=10))
|
||||
pairs = [
|
||||
("/email/INBOX/msg1.email.json", "foo foo\nfoo bar\nbaz\n"),
|
||||
("/email/INBOX/msg2.email.json", "bar\nbaz\n"),
|
||||
]
|
||||
with patch(
|
||||
"mirage.commands.builtin.email.grep.search_and_format",
|
||||
new=AsyncMock(return_value=pairs),
|
||||
):
|
||||
stdout, io = await _grep_server_side(
|
||||
accessor,
|
||||
"INBOX",
|
||||
"foo",
|
||||
[
|
||||
PathSpec(resource_path=mount_key("/email/INBOX", "/email"),
|
||||
virtual="/email/INBOX",
|
||||
directory="/email/INBOX")
|
||||
],
|
||||
c=True,
|
||||
)
|
||||
assert await materialize(stdout) == (b"/email/INBOX/msg1.email.json:2\n"
|
||||
b"/email/INBOX/msg2.email.json:0\n")
|
||||
assert io.exit_code == 0
|
||||
@@ -0,0 +1,90 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import importlib
|
||||
import sys
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.io.types import IOResult
|
||||
from mirage.types import PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
sys.modules.setdefault(
|
||||
"aioimaplib",
|
||||
SimpleNamespace(IMAP4=object, IMAP4_SSL=object),
|
||||
)
|
||||
sys.modules.setdefault(
|
||||
"aiosmtplib",
|
||||
SimpleNamespace(SMTP=object, send=AsyncMock()),
|
||||
)
|
||||
|
||||
rg = importlib.import_module("mirage.commands.builtin.email.rg").rg
|
||||
|
||||
|
||||
def _path(s: str = "/email/INBOX") -> PathSpec:
|
||||
return PathSpec(resource_path=mount_key(s, "/email"),
|
||||
virtual=s,
|
||||
directory=s)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_multi_pattern_skips_imap_search():
|
||||
# A newline-joined multi -e set must bypass the IMAP text search and
|
||||
# still resolve globs before the generic runs (#347).
|
||||
accessor = SimpleNamespace(config=SimpleNamespace(max_messages=10))
|
||||
seen: dict[str, object] = {}
|
||||
|
||||
async def fake_resolve(_accessor, paths, index=None):
|
||||
seen["resolved"] = [p.virtual for p in paths]
|
||||
return paths
|
||||
|
||||
async def fake_generic(paths, _texts, _flags, **_kwargs):
|
||||
seen["generic"] = [p.virtual for p in paths]
|
||||
return b"", IOResult()
|
||||
|
||||
with patch(
|
||||
"mirage.commands.builtin.email.rg.search_messages",
|
||||
new=AsyncMock(side_effect=AssertionError("imap search ran")),
|
||||
), patch(
|
||||
"mirage.commands.builtin.email.rg.resolve_glob",
|
||||
new=fake_resolve,
|
||||
), patch(
|
||||
"mirage.commands.builtin.email.rg.generic_rg",
|
||||
new=fake_generic,
|
||||
):
|
||||
_, io = await rg(accessor, [_path()], e=["ada", "ben"])
|
||||
|
||||
assert io.exit_code == 0
|
||||
assert seen["resolved"] == ["/email/INBOX"]
|
||||
assert seen["generic"] == ["/email/INBOX"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_single_pattern_uses_imap_search():
|
||||
accessor = SimpleNamespace(config=SimpleNamespace(max_messages=10))
|
||||
search = AsyncMock(return_value=[])
|
||||
with patch(
|
||||
"mirage.commands.builtin.email.rg.search_messages",
|
||||
new=search,
|
||||
), patch(
|
||||
"mirage.commands.builtin.email.rg.resolve_glob",
|
||||
new=AsyncMock(side_effect=AssertionError("glob ran")),
|
||||
):
|
||||
_, io = await rg(accessor, [_path()], "ada")
|
||||
|
||||
assert io.exit_code == 1
|
||||
search.assert_awaited_once()
|
||||
@@ -0,0 +1,13 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
@@ -0,0 +1,67 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import json
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.accessor.gdocs import GDocsAccessor
|
||||
from mirage.commands.builtin.gdocs.gws_docs_documents_batchUpdate import \
|
||||
gws_docs_documents_batchUpdate
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def accessor():
|
||||
return GDocsAccessor(config=None, token_manager=None)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_batch_update_success(accessor):
|
||||
payload = json.dumps({
|
||||
"requests": [{
|
||||
"insertText": {
|
||||
"location": {
|
||||
"index": 1
|
||||
},
|
||||
"text": "Hi",
|
||||
}
|
||||
}]
|
||||
})
|
||||
params = json.dumps({"documentId": "doc1"})
|
||||
api_response = {"documentId": "doc1", "replies": [{}]}
|
||||
with patch(
|
||||
"mirage.core.gdocs.update.google_post",
|
||||
new_callable=AsyncMock,
|
||||
return_value=api_response,
|
||||
):
|
||||
fn = gws_docs_documents_batchUpdate._registered_commands[0].fn
|
||||
stream, io = await fn(
|
||||
accessor,
|
||||
[],
|
||||
params=params,
|
||||
json=payload,
|
||||
)
|
||||
chunks = []
|
||||
async for chunk in stream:
|
||||
chunks.append(chunk)
|
||||
result = json.loads(b"".join(chunks))
|
||||
assert result["documentId"] == "doc1"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_batch_update_missing_params(accessor):
|
||||
fn = gws_docs_documents_batchUpdate._registered_commands[0].fn
|
||||
with pytest.raises(ValueError, match="--params"):
|
||||
await fn(accessor, [])
|
||||
@@ -0,0 +1,13 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
@@ -0,0 +1,87 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.accessor.gdrive import GDriveAccessor
|
||||
from mirage.cache.index.config import IndexEntry
|
||||
from mirage.cache.index.ram import RAMIndexCacheStore
|
||||
from mirage.commands.builtin.gdrive._provision import file_read_provision
|
||||
from mirage.provision.types import Precision
|
||||
from mirage.types import PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def accessor():
|
||||
return GDriveAccessor(config=None, token_manager=None)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def index():
|
||||
store = RAMIndexCacheStore()
|
||||
return store
|
||||
|
||||
|
||||
def _scope(path: str, prefix: str = "") -> PathSpec:
|
||||
return PathSpec(resource_path=mount_key(path, prefix),
|
||||
virtual=path,
|
||||
directory=path.rsplit("/", 1)[0] or "/")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_plan_returns_read_ops(accessor, index):
|
||||
await index.put(
|
||||
"/test/file.txt",
|
||||
IndexEntry(
|
||||
id="file123",
|
||||
name="file.txt",
|
||||
resource_type="gdrive/file",
|
||||
remote_time="2026-01-01T00:00:00Z",
|
||||
vfs_name="file.txt",
|
||||
size=500,
|
||||
))
|
||||
result = await file_read_provision(
|
||||
accessor,
|
||||
[_scope("/test/file.txt")],
|
||||
"cat test/file.txt",
|
||||
index=index,
|
||||
)
|
||||
assert result.read_ops == 1
|
||||
assert result.precision == Precision.EXACT
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_plan_no_paths(accessor, index):
|
||||
result = await file_read_provision(accessor, [], "cat", index=index)
|
||||
# Pathless invocations are stdin-driven: zero backend bytes, EXACT.
|
||||
assert result.precision == Precision.EXACT
|
||||
assert result.network_read_high == 0
|
||||
assert result.network_read_low == 0
|
||||
assert result.network_read_high == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_plan_missing_entry(accessor, index):
|
||||
result = await file_read_provision(
|
||||
accessor,
|
||||
[_scope("/nonexistent/path.txt")],
|
||||
"cat nonexistent/path.txt",
|
||||
index=index,
|
||||
)
|
||||
assert result.network_read_low == 0
|
||||
assert result.network_read_high == 0
|
||||
# Unresolvable operands are still charged as a read-op floor and the
|
||||
# estimate degrades to UNKNOWN instead of claiming a free read.
|
||||
assert result.read_ops == 1
|
||||
@@ -0,0 +1,149 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.accessor.gdrive import GDriveAccessor
|
||||
from mirage.cache.index.config import IndexEntry
|
||||
from mirage.cache.index.ram import RAMIndexCacheStore
|
||||
from mirage.core.gdrive.read import read
|
||||
from mirage.core.google._client import TokenManager
|
||||
from mirage.core.google.config import GoogleConfig
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def config():
|
||||
return GoogleConfig(
|
||||
client_id="test-id",
|
||||
client_secret="test-secret",
|
||||
refresh_token="test-refresh",
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def token_manager(config):
|
||||
mgr = TokenManager(config)
|
||||
mgr._access_token = "fake-token"
|
||||
mgr._expires_at = 9999999999
|
||||
return mgr
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def accessor(config, token_manager):
|
||||
return GDriveAccessor(config=config, token_manager=token_manager)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def index():
|
||||
return RAMIndexCacheStore()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_gdoc(accessor, index):
|
||||
await index.put(
|
||||
"/My Doc.gdoc.json",
|
||||
IndexEntry(
|
||||
id="doc1",
|
||||
name="My Doc",
|
||||
resource_type="gdrive/gdoc",
|
||||
remote_time="2026-04-01T00:00:00.000Z",
|
||||
vfs_name="My Doc.gdoc.json",
|
||||
))
|
||||
with patch(
|
||||
"mirage.core.gdocs.read.google_get",
|
||||
new_callable=AsyncMock,
|
||||
return_value={"documentId": "doc1"},
|
||||
):
|
||||
result = await read(
|
||||
accessor,
|
||||
PathSpec(resource_path="My Doc.gdoc.json",
|
||||
virtual="/My Doc.gdoc.json",
|
||||
directory="/My Doc.gdoc.json"), index)
|
||||
assert b"doc1" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_gsheet(accessor, index):
|
||||
await index.put(
|
||||
"/My Sheet.gsheet.json",
|
||||
IndexEntry(
|
||||
id="sheet1",
|
||||
name="My Sheet",
|
||||
resource_type="gdrive/gsheet",
|
||||
remote_time="2026-04-01T00:00:00.000Z",
|
||||
vfs_name="My Sheet.gsheet.json",
|
||||
))
|
||||
with patch(
|
||||
"mirage.core.gsheets.read.google_get",
|
||||
new_callable=AsyncMock,
|
||||
return_value={"spreadsheetId": "sheet1"},
|
||||
):
|
||||
result = await read(
|
||||
accessor,
|
||||
PathSpec(resource_path="My Sheet.gsheet.json",
|
||||
virtual="/My Sheet.gsheet.json",
|
||||
directory="/My Sheet.gsheet.json"), index)
|
||||
assert b"sheet1" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_gslide(accessor, index):
|
||||
await index.put(
|
||||
"/My Slides.gslide.json",
|
||||
IndexEntry(
|
||||
id="slide1",
|
||||
name="My Slides",
|
||||
resource_type="gdrive/gslide",
|
||||
remote_time="2026-04-01T00:00:00.000Z",
|
||||
vfs_name="My Slides.gslide.json",
|
||||
))
|
||||
with patch(
|
||||
"mirage.core.gslides.read.google_get",
|
||||
new_callable=AsyncMock,
|
||||
return_value={"presentationId": "slide1"},
|
||||
):
|
||||
result = await read(
|
||||
accessor,
|
||||
PathSpec(resource_path="My Slides.gslide.json",
|
||||
virtual="/My Slides.gslide.json",
|
||||
directory="/My Slides.gslide.json"), index)
|
||||
assert b"slide1" in result
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_regular(accessor, index):
|
||||
await index.put(
|
||||
"/photo.png",
|
||||
IndexEntry(
|
||||
id="img1",
|
||||
name="photo",
|
||||
resource_type="gdrive/file",
|
||||
remote_time="2026-04-01T00:00:00.000Z",
|
||||
vfs_name="photo.png",
|
||||
))
|
||||
img_bytes = b"\x89PNG\r\n"
|
||||
with patch(
|
||||
"mirage.core.gdrive.read.download_file",
|
||||
new_callable=AsyncMock,
|
||||
return_value=img_bytes,
|
||||
):
|
||||
result = await read(
|
||||
accessor,
|
||||
PathSpec(resource_path="photo.png",
|
||||
virtual="/photo.png",
|
||||
directory="/photo.png"), index)
|
||||
assert result == img_bytes
|
||||
@@ -0,0 +1,161 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
from unittest.mock import AsyncMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.accessor.gdrive import GDriveAccessor
|
||||
from mirage.cache.index.config import IndexEntry
|
||||
from mirage.cache.index.ram import RAMIndexCacheStore
|
||||
from mirage.commands.builtin.gdrive import COMMANDS
|
||||
from mirage.core.google._client import TokenManager
|
||||
from mirage.core.google.config import GoogleConfig
|
||||
from mirage.io.stream import materialize
|
||||
from mirage.types import PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def config():
|
||||
return GoogleConfig(
|
||||
client_id="test-id",
|
||||
client_secret="test-secret",
|
||||
refresh_token="test-refresh",
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def token_manager(config):
|
||||
mgr = TokenManager(config)
|
||||
mgr._access_token = "fake-token"
|
||||
mgr._expires_at = 9999999999
|
||||
return mgr
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def accessor(config, token_manager):
|
||||
return GDriveAccessor(config=config, token_manager=token_manager)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def index():
|
||||
return RAMIndexCacheStore()
|
||||
|
||||
|
||||
def _sed_command():
|
||||
for cmd in COMMANDS:
|
||||
for rc in cmd._registered_commands:
|
||||
if rc.name == "sed":
|
||||
return cmd
|
||||
raise LookupError("sed not registered for gdrive")
|
||||
|
||||
|
||||
sed = _sed_command()
|
||||
|
||||
|
||||
def _scope(path: str, prefix: str = "") -> PathSpec:
|
||||
return PathSpec(resource_path=mount_key(path, prefix),
|
||||
virtual=path,
|
||||
directory=path.rsplit("/", 1)[0] or "/")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_simple_substitution(accessor, index):
|
||||
await index.put(
|
||||
"/test/file.txt",
|
||||
IndexEntry(
|
||||
id="file123",
|
||||
name="file.txt",
|
||||
resource_type="gdrive/file",
|
||||
remote_time="2026-01-01T00:00:00Z",
|
||||
vfs_name="file.txt",
|
||||
size=100,
|
||||
))
|
||||
with patch(
|
||||
"mirage.core.google.drive.google_get_bytes",
|
||||
new_callable=AsyncMock,
|
||||
return_value=b"hello world\nhello again\n",
|
||||
):
|
||||
result, io = await sed(
|
||||
accessor,
|
||||
[_scope("/test/file.txt")],
|
||||
"s/hello/bye/g",
|
||||
index=index,
|
||||
)
|
||||
data = await materialize(result)
|
||||
assert data == b"bye world\nbye again\n"
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_print_program(accessor, index):
|
||||
await index.put(
|
||||
"/test/file.txt",
|
||||
IndexEntry(
|
||||
id="file123",
|
||||
name="file.txt",
|
||||
resource_type="gdrive/file",
|
||||
remote_time="2026-01-01T00:00:00Z",
|
||||
vfs_name="file.txt",
|
||||
size=100,
|
||||
))
|
||||
with patch(
|
||||
"mirage.core.google.drive.google_get_bytes",
|
||||
new_callable=AsyncMock,
|
||||
return_value=b"one\ntwo\nthree\n",
|
||||
):
|
||||
result, io = await sed(
|
||||
accessor,
|
||||
[_scope("/test/file.txt")],
|
||||
"2p",
|
||||
n=True,
|
||||
index=index,
|
||||
)
|
||||
data = await materialize(result)
|
||||
assert data == b"two\n"
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_stdin(accessor, index):
|
||||
result, io = await sed(
|
||||
accessor,
|
||||
[],
|
||||
"s/a/b/g",
|
||||
stdin=b"banana\n",
|
||||
index=index,
|
||||
)
|
||||
data = await materialize(result)
|
||||
assert data == b"bbnbnb\n"
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_in_place_rejected(accessor, index):
|
||||
with pytest.raises(PermissionError,
|
||||
match="-i not supported on this backend"):
|
||||
await sed(
|
||||
accessor,
|
||||
[_scope("/test/file.txt")],
|
||||
"s/a/b/",
|
||||
i=True,
|
||||
index=index,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_missing_expression(accessor, index):
|
||||
with pytest.raises(ValueError, match="sed: usage"):
|
||||
await sed(accessor, [], index=index)
|
||||
@@ -0,0 +1,38 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
from mirage.commands.builtin.generic.crossmount.fanout.du import du_total
|
||||
from mirage.commands.builtin.generic.crossmount.types import OperandRun
|
||||
from mirage.io import IOResult
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
def _scope(virtual: str) -> PathSpec:
|
||||
return PathSpec(virtual=virtual,
|
||||
directory=virtual[:virtual.rfind("/") + 1],
|
||||
resource_path="",
|
||||
resolved=True)
|
||||
|
||||
|
||||
def _op(data: bytes, exit_code: int = 0) -> OperandRun:
|
||||
return OperandRun(_scope("/a/x"), data, IOResult(exit_code=exit_code))
|
||||
|
||||
|
||||
def testdu_total_strips_per_run_totals_and_sums():
|
||||
runs = [
|
||||
_op(b"5\t/a/sub\n5\ttotal\n"),
|
||||
_op(b"3\t/b/c.txt\n3\ttotal\n"),
|
||||
]
|
||||
out = du_total(runs, human=False).decode()
|
||||
assert out == "5\t/a/sub\n3\t/b/c.txt\n8\ttotal\n"
|
||||
@@ -0,0 +1,24 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
from mirage.commands.builtin.generic.crossmount.fanout.exit import \
|
||||
combined_exit
|
||||
|
||||
|
||||
def testcombined_exit_grep_match_wins_over_no_match():
|
||||
assert combined_exit("grep", [0, 1]) == 0
|
||||
assert combined_exit("grep", [1, 1]) == 1
|
||||
assert combined_exit("grep", [2, 0]) == 2
|
||||
assert combined_exit("rm", [0, 1]) == 1
|
||||
assert combined_exit("md5", [0, 0]) == 0
|
||||
@@ -0,0 +1,127 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import asyncio
|
||||
|
||||
from mirage.commands.builtin.generic.crossmount.fanout import run_fanout
|
||||
from mirage.commands.builtin.generic.crossmount.types import OperandRun
|
||||
from mirage.io import IOResult
|
||||
from mirage.io.stream import materialize
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
def _scope(virtual: str) -> PathSpec:
|
||||
return PathSpec(virtual=virtual,
|
||||
directory=virtual[:virtual.rfind("/") + 1],
|
||||
resource_path="",
|
||||
resolved=True)
|
||||
|
||||
|
||||
def _op(data: bytes, exit_code: int = 0) -> OperandRun:
|
||||
return OperandRun(_scope("/a/x"), data, IOResult(exit_code=exit_code))
|
||||
|
||||
|
||||
class FakeRunSingle:
|
||||
"""Serves canned per-operand outputs and records every call."""
|
||||
|
||||
def __init__(self, outputs: dict[str, tuple[bytes, int]]):
|
||||
self.outputs = outputs
|
||||
self.calls: list[dict] = []
|
||||
|
||||
async def __call__(self,
|
||||
cmd_name,
|
||||
paths,
|
||||
texts,
|
||||
flag_kwargs,
|
||||
stdin=None,
|
||||
resolve_hint=None):
|
||||
stdin_data = await materialize(stdin) if stdin is not None else None
|
||||
self.calls.append(
|
||||
dict(cmd=cmd_name,
|
||||
paths=[p.virtual for p in paths],
|
||||
flags=dict(flag_kwargs),
|
||||
stdin=stdin_data))
|
||||
data, code = self.outputs[paths[0].virtual]
|
||||
io = IOResult(exit_code=code)
|
||||
if code != 0:
|
||||
io.stderr = f"{cmd_name}: {paths[0].virtual}: error\n".encode()
|
||||
return data, io
|
||||
|
||||
|
||||
def _run(coro):
|
||||
return asyncio.run(coro)
|
||||
|
||||
|
||||
def test_run_fanout_concats_in_operand_order_and_merges_exit():
|
||||
rs = FakeRunSingle({
|
||||
"/a/x": (b"hash1 /a/x\n", 0),
|
||||
"/b/y": (b"hash2 /b/y\n", 0),
|
||||
})
|
||||
out, io = _run(
|
||||
run_fanout("sha256sum",
|
||||
[_scope("/a/x"), _scope("/b/y")], [], {}, rs))
|
||||
assert _run(materialize(out)) == b"hash1 /a/x\nhash2 /b/y\n"
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
def test_run_fanout_forces_grep_filenames_unless_suppressed():
|
||||
rs = FakeRunSingle({"/a/x": (b"", 1), "/b/y": (b"", 1)})
|
||||
_run(run_fanout("grep", [_scope("/a/x"), _scope("/b/y")], ["pat"], {}, rs))
|
||||
assert all(c["flags"].get("H") is True for c in rs.calls)
|
||||
rs2 = FakeRunSingle({"/a/x": (b"", 1), "/b/y": (b"", 1)})
|
||||
_run(
|
||||
run_fanout("grep", [_scope("/a/x"), _scope("/b/y")], ["pat"],
|
||||
{"h": True}, rs2))
|
||||
assert all("H" not in c["flags"] for c in rs2.calls)
|
||||
|
||||
|
||||
def test_run_fanout_forces_rg_filenames_unless_suppressed():
|
||||
rs = FakeRunSingle({"/a/x": (b"", 1), "/b/y": (b"", 1)})
|
||||
_run(run_fanout("rg", [_scope("/a/x"), _scope("/b/y")], ["pat"], {}, rs))
|
||||
assert all(c["flags"].get("H") is True for c in rs.calls)
|
||||
rs2 = FakeRunSingle({"/a/x": (b"", 1), "/b/y": (b"", 1)})
|
||||
_run(
|
||||
run_fanout("rg", [_scope("/a/x"), _scope("/b/y")], ["pat"],
|
||||
{"args_I": True}, rs2))
|
||||
assert all("H" not in c["flags"] for c in rs2.calls)
|
||||
|
||||
|
||||
def test_run_fanout_forces_head_headers_and_blank_line_joins():
|
||||
rs = FakeRunSingle({
|
||||
"/a/x": (b"==> /a/x <==\n1\n", 0),
|
||||
"/b/y": (b"==> /b/y <==\n2\n", 0),
|
||||
})
|
||||
out, _ = _run(
|
||||
run_fanout("head", [_scope("/a/x"), _scope("/b/y")], [], {}, rs))
|
||||
assert all(c["flags"].get("v") is True for c in rs.calls)
|
||||
assert _run(materialize(out)) == b"==> /a/x <==\n1\n\n==> /b/y <==\n2\n"
|
||||
|
||||
|
||||
def test_run_fanout_tee_refeeds_stdin_and_emits_it_once():
|
||||
rs = FakeRunSingle({"/a/x": (b"hi\n", 0), "/b/y": (b"hi\n", 0)})
|
||||
out, _ = _run(
|
||||
run_fanout("tee", [_scope("/a/x"), _scope("/b/y")], [], {},
|
||||
rs,
|
||||
stdin=b"hi\n"))
|
||||
assert _run(materialize(out)) == b"hi\n"
|
||||
assert all(c["stdin"] == b"hi\n" for c in rs.calls)
|
||||
|
||||
|
||||
def test_run_fanout_partial_failure_keeps_output_and_stderr():
|
||||
rs = FakeRunSingle({"/a/x": (b"", 1), "/b/y": (b"ok\n", 0)})
|
||||
out, io = _run(
|
||||
run_fanout("stat", [_scope("/a/x"), _scope("/b/y")], [], {}, rs))
|
||||
assert _run(materialize(out)) == b"ok\n"
|
||||
assert io.exit_code == 1
|
||||
assert b"/a/x" in (io.stderr or b"")
|
||||
@@ -0,0 +1,58 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
from mirage.commands.builtin.generic.crossmount.fanout.wc import combine_wc
|
||||
from mirage.commands.builtin.generic.crossmount.types import OperandRun
|
||||
from mirage.io import IOResult
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
def _scope(virtual: str) -> PathSpec:
|
||||
return PathSpec(virtual=virtual,
|
||||
directory=virtual[:virtual.rfind("/") + 1],
|
||||
resource_path="",
|
||||
resolved=True)
|
||||
|
||||
|
||||
def _op(data: bytes, exit_code: int = 0) -> OperandRun:
|
||||
return OperandRun(_scope("/a/x"), data, IOResult(exit_code=exit_code))
|
||||
|
||||
|
||||
def testcombine_wc_uses_one_global_width():
|
||||
runs = [
|
||||
_op(b"100 100 400 /a/big.txt\n"),
|
||||
_op(b"5 5 20 /b/small.txt\n"),
|
||||
]
|
||||
out = combine_wc(runs, {}).decode()
|
||||
assert out == ("100 100 400 /a/big.txt\n"
|
||||
" 5 5 20 /b/small.txt\n"
|
||||
"105 105 420 total\n")
|
||||
|
||||
|
||||
def testcombine_wc_drops_per_run_totals_from_glob_operands():
|
||||
runs = [
|
||||
_op(b"2 /a/one.txt\n1 /a/two.txt\n3 total\n"),
|
||||
_op(b"1 /b/three.txt\n"),
|
||||
]
|
||||
out = combine_wc(runs, {"args_l": True}).decode()
|
||||
assert out == ("2 /a/one.txt\n"
|
||||
"1 /a/two.txt\n"
|
||||
"1 /b/three.txt\n"
|
||||
"4 total\n")
|
||||
|
||||
|
||||
def testcombine_wc_max_line_length_maxes_instead_of_summing():
|
||||
runs = [_op(b"9 /a/x\n"), _op(b"4 /b/y\n")]
|
||||
out = combine_wc(runs, {"L": True}).decode()
|
||||
assert out.endswith("9 total\n")
|
||||
@@ -0,0 +1,108 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import asyncio
|
||||
|
||||
from mirage.commands.builtin.generic.crossmount.stream import run_stream
|
||||
from mirage.io import IOResult
|
||||
from mirage.io.stream import materialize
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
def _scope(virtual: str) -> PathSpec:
|
||||
return PathSpec(virtual=virtual,
|
||||
directory=virtual[:virtual.rfind("/") + 1],
|
||||
resource_path="",
|
||||
resolved=True)
|
||||
|
||||
|
||||
class FakeRunSingle:
|
||||
"""Records run_single calls; serves per-path bytes for cat pushdowns."""
|
||||
|
||||
def __init__(self, files: dict[str, bytes]):
|
||||
self.files = files
|
||||
self.calls: list[dict] = []
|
||||
self.final_stdin: bytes | None = None
|
||||
|
||||
async def __call__(self,
|
||||
cmd_name,
|
||||
paths,
|
||||
texts,
|
||||
flag_kwargs,
|
||||
stdin=None,
|
||||
resolve_hint=None):
|
||||
self.calls.append(
|
||||
dict(cmd=cmd_name,
|
||||
paths=[p.virtual for p in paths],
|
||||
texts=list(texts),
|
||||
flags=dict(flag_kwargs),
|
||||
resolve_hint=resolve_hint.virtual
|
||||
if resolve_hint is not None else None))
|
||||
if cmd_name == "cat" and paths:
|
||||
data = self.files.get(paths[0].virtual)
|
||||
if data is None:
|
||||
err = f"cat: {paths[0].virtual}: No such file\n".encode()
|
||||
return None, IOResult(exit_code=1, stderr=err)
|
||||
return data, IOResult()
|
||||
self.final_stdin = await materialize(stdin) if stdin is not None \
|
||||
else None
|
||||
return b"FINAL:" + (self.final_stdin or b""), IOResult()
|
||||
|
||||
|
||||
def _run(coro):
|
||||
return asyncio.run(coro)
|
||||
|
||||
|
||||
def test_plain_cat_skips_the_final_run():
|
||||
rs = FakeRunSingle({"/a/x": b"1\n", "/b/y": b"2\n"})
|
||||
out, io = _run(
|
||||
run_stream("cat", [_scope("/a/x"), _scope("/b/y")], [], {}, rs))
|
||||
assert _run(materialize(out)) == b"1\n2\n"
|
||||
assert io.exit_code == 0
|
||||
assert [c["cmd"] for c in rs.calls] == ["cat", "cat"]
|
||||
|
||||
|
||||
def test_flagged_command_runs_once_on_the_merged_stream():
|
||||
rs = FakeRunSingle({"/a/x": b"1\n", "/b/y": b"2\n"})
|
||||
out, io = _run(
|
||||
run_stream("sort", [_scope("/a/x"), _scope("/b/y")], [], {"r": True},
|
||||
rs))
|
||||
assert _run(materialize(out)) == b"FINAL:1\n2\n"
|
||||
assert io.exit_code == 0
|
||||
final = rs.calls[-1]
|
||||
assert final["cmd"] == "sort"
|
||||
assert final["paths"] == []
|
||||
assert final["flags"] == {"r": True}
|
||||
assert final["resolve_hint"] == "/a/x"
|
||||
assert rs.final_stdin == b"1\n2\n"
|
||||
|
||||
|
||||
def test_cat_with_flags_reapplies_cat_on_the_merged_stream():
|
||||
rs = FakeRunSingle({"/a/x": b"1\n", "/b/y": b"2\n"})
|
||||
out, _ = _run(
|
||||
run_stream("cat", [_scope("/a/x"), _scope("/b/y")], [], {"n": True},
|
||||
rs))
|
||||
assert _run(materialize(out)) == b"FINAL:1\n2\n"
|
||||
assert rs.calls[-1]["cmd"] == "cat"
|
||||
assert rs.calls[-1]["flags"] == {"n": True}
|
||||
|
||||
|
||||
def test_failed_operand_is_skipped_and_fails_the_command():
|
||||
rs = FakeRunSingle({"/b/y": b"2\n"})
|
||||
out, io = _run(
|
||||
run_stream("cat",
|
||||
[_scope("/a/missing"), _scope("/b/y")], [], {}, rs))
|
||||
assert _run(materialize(out)) == b"2\n"
|
||||
assert io.exit_code == 1
|
||||
assert b"No such file" in (io.stderr or b"")
|
||||
@@ -0,0 +1,89 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
from mirage.commands.builtin.generic.crossmount.constants import (
|
||||
CROSS_MOUNT_COMMANDS, FANOUT_COMMANDS, RELAY_COMMANDS, STREAM_COMMANDS)
|
||||
from mirage.commands.builtin.generic.crossmount.detect import (is_cross_mount,
|
||||
strategy_for)
|
||||
from mirage.commands.builtin.generic.crossmount.types import Strategy
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
class _Mount:
|
||||
|
||||
def __init__(self, prefix: str):
|
||||
self.prefix = prefix
|
||||
|
||||
|
||||
class _Registry:
|
||||
|
||||
def __init__(self, prefixes: dict[str, str]):
|
||||
self._prefixes = prefixes
|
||||
|
||||
def mount_for(self, virtual: str) -> _Mount:
|
||||
for prefix in self._prefixes.values():
|
||||
if virtual.startswith(prefix.rstrip("/") + "/"):
|
||||
return _Mount(prefix)
|
||||
raise ValueError(virtual)
|
||||
|
||||
|
||||
def _scope(virtual: str) -> PathSpec:
|
||||
return PathSpec(virtual=virtual,
|
||||
directory=virtual[:virtual.rfind("/") + 1],
|
||||
resource_path="",
|
||||
resolved=True)
|
||||
|
||||
|
||||
def test_sets_are_disjoint():
|
||||
assert not STREAM_COMMANDS & FANOUT_COMMANDS
|
||||
assert not STREAM_COMMANDS & RELAY_COMMANDS
|
||||
assert not FANOUT_COMMANDS & RELAY_COMMANDS
|
||||
assert CROSS_MOUNT_COMMANDS == (STREAM_COMMANDS | FANOUT_COMMANDS
|
||||
| RELAY_COMMANDS)
|
||||
|
||||
|
||||
def test_strategy_for_stream_commands():
|
||||
for name in ("cat", "nl", "sort", "cut", "rev"):
|
||||
assert strategy_for(name, {}) is Strategy.STREAM
|
||||
|
||||
|
||||
def test_strategy_for_fanout_commands():
|
||||
for name in ("grep", "wc", "sha256sum", "ls", "rm", "tee"):
|
||||
assert strategy_for(name, {}) is Strategy.FANOUT
|
||||
|
||||
|
||||
def test_strategy_for_relay_commands():
|
||||
for name in ("cp", "mv", "diff", "cmp"):
|
||||
assert strategy_for(name, {}) is Strategy.RELAY
|
||||
|
||||
|
||||
def test_sed_default_streams_but_in_place_fans_out():
|
||||
assert strategy_for("sed", {}) is Strategy.STREAM
|
||||
assert strategy_for("sed", {"i": True}) is Strategy.FANOUT
|
||||
|
||||
|
||||
def test_is_cross_mount_true_when_operands_span_mounts():
|
||||
registry = _Registry({"a": "/a/", "b": "/b/"})
|
||||
scopes = [_scope("/a/x.txt"), _scope("/b/y.txt")]
|
||||
assert is_cross_mount("sort", scopes, registry)
|
||||
assert is_cross_mount("sha256sum", scopes, registry)
|
||||
|
||||
|
||||
def test_is_cross_mount_false_for_single_mount_or_unknown_command():
|
||||
registry = _Registry({"a": "/a/", "b": "/b/"})
|
||||
same = [_scope("/a/x.txt"), _scope("/a/y.txt")]
|
||||
assert not is_cross_mount("sort", same, registry)
|
||||
spanning = [_scope("/a/x.txt"), _scope("/b/y.txt")]
|
||||
assert not is_cross_mount("uniq", spanning, registry)
|
||||
assert not is_cross_mount("sort", spanning[:1], registry)
|
||||
@@ -0,0 +1,423 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.awk import awk
|
||||
from mirage.commands.errors import UsageError
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(resource_path=(path).strip("/"),
|
||||
virtual=path,
|
||||
directory=path,
|
||||
resolved=True)
|
||||
|
||||
|
||||
def _make_backend(files: dict[str, bytes]):
|
||||
|
||||
async def read_bytes(accessor, path, index=None):
|
||||
key = path.virtual if isinstance(path, PathSpec) else path
|
||||
if key not in files:
|
||||
raise FileNotFoundError(key)
|
||||
return files[key]
|
||||
|
||||
async def read_stream(accessor, path, index=None):
|
||||
assert isinstance(path, PathSpec)
|
||||
key = path.virtual
|
||||
if key not in files:
|
||||
raise FileNotFoundError(key)
|
||||
yield files[key]
|
||||
|
||||
return read_bytes, read_stream
|
||||
|
||||
|
||||
async def _drain(stdout) -> bytes:
|
||||
if stdout is None:
|
||||
return b""
|
||||
if isinstance(stdout, bytes):
|
||||
return stdout
|
||||
return b"".join([c async for c in stdout])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_stdin_print_field():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
("{print $1}", ),
|
||||
None,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"alpha beta\ngamma delta\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "alpha\ngamma\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_field_separator():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
("{print $2}", ),
|
||||
{"F": ","},
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"a,b,c\nd,e,f\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "b\ne\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_variable_assignment():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
("{print x}", ),
|
||||
{"v": ["x=hello"]},
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"line\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "hello\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_numeric_comparison():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
("$1 > 2 {print $1}", ),
|
||||
None,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"1\n2\n3\n4\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "3\n4\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_regex_condition():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
("/foo/ {print $0}", ),
|
||||
None,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"foo bar\nbaz\nfoobar\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "foo bar\nfoobar\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_end_block_accumulator():
|
||||
"""sum += $1 with END {print sum} should emit total."""
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
("{sum += $1} END {print sum}", ),
|
||||
None,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"10\n20\n30\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "60\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_reads_from_file():
|
||||
rb, rs = _make_backend({"/data.txt": b"hello world\n"})
|
||||
output, io = await awk(
|
||||
[_spec("/data.txt")],
|
||||
("{print $2}", ),
|
||||
None,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
assert (await _drain(output)).decode() == "world\n"
|
||||
assert io.cache == ["/data.txt"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_program_file_overrides_inline():
|
||||
rb, rs = _make_backend({
|
||||
"/prog.awk": b"{print $1}\n",
|
||||
"/data.txt": b"alpha beta\n",
|
||||
})
|
||||
output, _ = await awk(
|
||||
[_spec("/data.txt")],
|
||||
(),
|
||||
{"f": _spec("/prog.awk")},
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
assert (await _drain(output)).decode() == "alpha\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_missing_program_raises_usage_error():
|
||||
rb, rs = _make_backend({})
|
||||
with pytest.raises(UsageError, match="usage"):
|
||||
await awk([], (), None, read_bytes=rb, read_stream=rs)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_default_fs_collapses_whitespace():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
("{print $2}", ),
|
||||
None,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"a b\n\tx\t \ty\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "b\ny\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_explicit_single_space_fs_collapses_whitespace():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
("{print $2}", ),
|
||||
{"F": " "},
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"a b\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "b\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_empty_fs_splits_characters():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
("{print $2}", ),
|
||||
{"F": ""},
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"abc\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "b\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_processes_all_files_with_continuous_nr():
|
||||
rb, rs = _make_backend({
|
||||
"/a.txt": b"one\ntwo\n",
|
||||
"/b.txt": b"three\n",
|
||||
})
|
||||
output, io = await awk(
|
||||
[_spec("/a.txt"), _spec("/b.txt")],
|
||||
("{print NR, $1}", ),
|
||||
None,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
assert (await _drain(output)).decode() == "1 one\n2 two\n3 three\n"
|
||||
assert io.cache == ["/a.txt", "/b.txt"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_multifile_no_trailing_newline_keeps_lines_separate():
|
||||
rb, rs = _make_backend({
|
||||
"/a.txt": b"one",
|
||||
"/b.txt": b"two\n",
|
||||
})
|
||||
output, _ = await awk(
|
||||
[_spec("/a.txt"), _spec("/b.txt")],
|
||||
("{print NR, $1}", ),
|
||||
None,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
assert (await _drain(output)).decode() == "1 one\n2 two\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_repeated_v_assignments():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
("{print a, b}", ),
|
||||
{"v": ["a=1", "b=2"]},
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"line\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "1 2\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_v_value_containing_equals():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
("{print x}", ),
|
||||
{"v": ["x=a=b"]},
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"line\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "a=b\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_print_empty_string_emits_blank_line():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
('{print ""}', ),
|
||||
None,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"one\ntwo\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "\n\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_action_without_print_emits_nothing():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
("{x += 1}", ),
|
||||
None,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"one\ntwo\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == ""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_brace_literal_in_print():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
('{print "}"}', ),
|
||||
None,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"line\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "}\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_accumulator_non_numeric_coerces():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
("{sum += $1} END {print sum}", ),
|
||||
None,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"3\nabc\n2.5x\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "5.5\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_program_file_missing_raises_usage_error():
|
||||
rb, rs = _make_backend({"/data.txt": b"x\n"})
|
||||
with pytest.raises(UsageError, match="No such file"):
|
||||
await awk(
|
||||
[_spec("/data.txt")],
|
||||
(),
|
||||
{"f": _spec("/missing.awk")},
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_program_file_with_multiple_data_files():
|
||||
rb, rs = _make_backend({
|
||||
"/prog.awk": b"{print NR, $1}\n",
|
||||
"/a.txt": b"one\n",
|
||||
"/b.txt": b"two\n",
|
||||
})
|
||||
output, io = await awk(
|
||||
[_spec("/a.txt"), _spec("/b.txt")],
|
||||
(),
|
||||
{"f": _spec("/prog.awk")},
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
assert (await _drain(output)).decode() == "1 one\n2 two\n"
|
||||
assert io.cache == ["/a.txt", "/b.txt"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_begin_end_resolve_v_variables():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
("BEGIN {print x} END {print x}", ),
|
||||
{"v": ["x=hi"]},
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"line\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "hi\nhi\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_duplicate_v_last_wins():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
("{print x}", ),
|
||||
{"v": ["x=first", "x=second"]},
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"line\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "second\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_begin_bare_print_emits_blank_line():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
('BEGIN {print} {print $1}', ),
|
||||
None,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"a\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "\na\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_brace_literal_with_condition():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await awk(
|
||||
[],
|
||||
('/x/ {print "}"}', ),
|
||||
None,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"x\ny\n",
|
||||
)
|
||||
assert (await _drain(output)).decode() == "}\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_awk_repeated_program_files_concatenate():
|
||||
rb, rs = _make_backend({
|
||||
"/p1.awk": b"{sum += $1}\n",
|
||||
"/p2.awk": b"END {print sum}\n",
|
||||
"/nums.txt": b"1\n2\n3\n",
|
||||
})
|
||||
output, _ = await awk(
|
||||
[_spec("/nums.txt")],
|
||||
(),
|
||||
{"f": [_spec("/p1.awk"), _spec("/p2.awk")]},
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
assert (await _drain(output)).decode() == "6\n"
|
||||
@@ -0,0 +1,178 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.cat import cat
|
||||
|
||||
|
||||
async def _drain(gen):
|
||||
return b"".join([c async for c in gen])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_passthrough_from_bytes():
|
||||
out = await _drain(cat(b"hello\nworld\n"))
|
||||
assert out == b"hello\nworld\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_passthrough_from_stream_preserves_chunks():
|
||||
|
||||
async def src():
|
||||
yield b"hel"
|
||||
yield b"lo\nwo"
|
||||
yield b"rld\n"
|
||||
|
||||
chunks = [c async for c in cat(src())]
|
||||
assert chunks == [b"hel", b"lo\nwo", b"rld\n"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_number_lines():
|
||||
out = await _drain(cat(b"a\nb\nc\n", number_lines=True))
|
||||
assert out == b" 1\ta\n 2\tb\n 3\tc\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_number_lines_across_chunk_boundaries():
|
||||
|
||||
async def src():
|
||||
yield b"a\nb"
|
||||
yield b"\nc\n"
|
||||
|
||||
out = await _drain(cat(src(), number_lines=True))
|
||||
assert out == b" 1\ta\n 2\tb\n 3\tc\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_show_ends():
|
||||
out = await _drain(cat(b"a\nb\n", show_ends=True))
|
||||
assert out == b"a$\nb$\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_squeeze_blank():
|
||||
out = await _drain(cat(b"a\n\n\n\nb\n", squeeze_blank=True))
|
||||
assert out == b"a\n\nb\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_no_trailing_newline_preserved():
|
||||
"""POSIX: `printf "x" | cat -n` emits no trailing newline."""
|
||||
out = await _drain(cat(b"hello", number_lines=True))
|
||||
assert out == b" 1\thello"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_passthrough_no_trailing_newline():
|
||||
"""No-flag cat must not add a trailing newline."""
|
||||
out = await _drain(cat(b"hello"))
|
||||
assert out == b"hello"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_number_lines_multidigit_alignment():
|
||||
"""POSIX format is `%6d\\t` — width 6, right-justified. Old MIRAGE used
|
||||
a 5-space prefix which broke alignment for line numbers >= 10."""
|
||||
body = b"".join(f"line{i}\n".encode() for i in range(1, 13))
|
||||
out = await _drain(cat(body, number_lines=True))
|
||||
lines = out.split(b"\n")
|
||||
assert lines[0] == b" 1\tline1"
|
||||
assert lines[8] == b" 9\tline9"
|
||||
assert lines[9] == b" 10\tline10"
|
||||
assert lines[11] == b" 12\tline12"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_show_ends_no_trailing_newline():
|
||||
"""cat -E on input without trailing newline must not emit final `$\\n`."""
|
||||
out = await _drain(cat(b"hello", show_ends=True))
|
||||
assert out == b"hello"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_combined_n_E_s():
|
||||
"""Combined flags: number all kept lines, show ends, squeeze blanks."""
|
||||
out = await _drain(
|
||||
cat(b"a\n\n\n\nb\n",
|
||||
number_lines=True,
|
||||
show_ends=True,
|
||||
squeeze_blank=True))
|
||||
# Kept lines: "a", "" (first blank, kept), "b". Numbered 1, 2, 3.
|
||||
assert out == b" 1\ta$\n 2\t$\n 3\tb$\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_empty_input_emits_nothing_with_flags():
|
||||
out = await _drain(cat(b"", number_lines=True))
|
||||
assert out == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_only_newlines():
|
||||
"""Three blank lines numbered 1, 2, 3."""
|
||||
out = await _drain(cat(b"\n\n\n", number_lines=True))
|
||||
assert out == b" 1\t\n 2\t\n 3\t\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_squeeze_blank_preserves_first_blank():
|
||||
out = await _drain(cat(b"\n\n\nx\n", squeeze_blank=True))
|
||||
assert out == b"\nx\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_empty_input_passthrough():
|
||||
out = await _drain(cat(b""))
|
||||
assert out == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_binary_passthrough_full_byte_range():
|
||||
"""Full 0..255 byte range must pass through unchanged (no flags)."""
|
||||
data = bytes(range(256))
|
||||
out = await _drain(cat(data))
|
||||
assert out == data
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_binary_with_show_ends_marks_only_newlines():
|
||||
"""show_ends should only mark 0x0A bytes, not other binary bytes."""
|
||||
data = b"\x00\x01\n\x02\x03\n"
|
||||
out = await _drain(cat(data, show_ends=True))
|
||||
assert out == b"\x00\x01$\n\x02\x03$\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_number_lines_chunked_one_byte_at_a_time():
|
||||
"""Worst-case chunking: every byte its own chunk. Result must match
|
||||
unbuffered input exactly."""
|
||||
|
||||
async def src():
|
||||
for byte in b"a\nbb\nccc\n":
|
||||
yield bytes([byte])
|
||||
|
||||
out = await _drain(cat(src(), number_lines=True))
|
||||
assert out == b" 1\ta\n 2\tbb\n 3\tccc\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_show_tabs_renders_caret_i():
|
||||
out = b"".join([c async for c in cat(b"a\tb\nx\n", show_tabs=True)])
|
||||
assert out == b"a^Ib\nx\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_show_all_combines_tabs_and_ends():
|
||||
out = b"".join([
|
||||
c async for c in cat(b"a\tb\nx\n",
|
||||
show_tabs=True,
|
||||
show_ends=True,
|
||||
show_nonprinting=True)
|
||||
])
|
||||
assert out == b"a^Ib$\nx$\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cat_show_nonprinting_caret_and_meta_notation():
|
||||
out = b"".join(
|
||||
[c async for c in cat(b"\x01\x7f\xff\n", show_nonprinting=True)])
|
||||
assert out == b"^A^?M-^?\n"
|
||||
@@ -0,0 +1,211 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.cp import cp
|
||||
from mirage.types import FileStat, FileType, PathSpec
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(virtual=path,
|
||||
directory=path,
|
||||
resource_path=path.strip("/"))
|
||||
|
||||
|
||||
def _key(p) -> str:
|
||||
return (p.virtual if isinstance(p, PathSpec) else p).rstrip("/")
|
||||
|
||||
|
||||
def _make_backend(files: dict[str, bytes], dirs: set[str]):
|
||||
|
||||
async def stat(p, index=None) -> FileStat:
|
||||
k = _key(p)
|
||||
if k in dirs:
|
||||
return FileStat(name=k.rsplit("/", 1)[-1], type=FileType.DIRECTORY)
|
||||
if k in files:
|
||||
return FileStat(name=k.rsplit("/", 1)[-1], type=FileType.TEXT)
|
||||
raise FileNotFoundError(k)
|
||||
|
||||
async def copy(src, dst) -> None:
|
||||
files[_key(dst)] = files[_key(src)]
|
||||
|
||||
async def find(p, type=None) -> list[str]:
|
||||
base = _key(p) + "/"
|
||||
return sorted(k for k in files if k.startswith(base))
|
||||
|
||||
return stat, copy, find
|
||||
|
||||
|
||||
async def _run(files, dirs, paths, **kw):
|
||||
stat, copy, find = _make_backend(files, dirs)
|
||||
return await cp([_spec(p) for p in paths],
|
||||
copy=copy,
|
||||
find=find,
|
||||
find_type="f",
|
||||
stat=stat,
|
||||
recursive=kw.get("recursive", False),
|
||||
n=kw.get("n", False),
|
||||
v=kw.get("v", False))
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_single_source_to_new_path():
|
||||
files = {"/a.txt": b"AAA"}
|
||||
await _run(files, set(), ["/a.txt", "/copy.txt"])
|
||||
assert files["/copy.txt"] == b"AAA"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_single_source_into_directory():
|
||||
files = {"/a.txt": b"AAA", "/d/keep": b"K"}
|
||||
await _run(files, {"/d"}, ["/a.txt", "/d"])
|
||||
assert files["/d/a.txt"] == b"AAA"
|
||||
assert files["/a.txt"] == b"AAA"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_multiple_sources_into_directory():
|
||||
files = {"/a.txt": b"AAA", "/b.txt": b"BBB", "/d/keep": b"K"}
|
||||
await _run(files, {"/d"}, ["/a.txt", "/b.txt", "/d"])
|
||||
assert files["/d/a.txt"] == b"AAA"
|
||||
assert files["/d/b.txt"] == b"BBB"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_multiple_sources_nondir_raises():
|
||||
files = {"/a.txt": b"AAA", "/b.txt": b"BBB", "/dst.txt": b"DST"}
|
||||
with pytest.raises(NotADirectoryError):
|
||||
await _run(files, set(), ["/a.txt", "/b.txt", "/dst.txt"])
|
||||
assert files["/dst.txt"] == b"DST"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_no_clobber_skips_existing():
|
||||
files = {"/a.txt": b"NEW", "/d/a.txt": b"OLD"}
|
||||
await _run(files, {"/d"}, ["/a.txt", "/d"], n=True)
|
||||
assert files["/d/a.txt"] == b"OLD"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_no_clobber_duplicate_basenames_first_wins():
|
||||
files = {"/x/a.txt": b"FIRST", "/y/a.txt": b"SECOND", "/d/keep": b"K"}
|
||||
await _run(files, {"/d"}, ["/x/a.txt", "/y/a.txt", "/d"], n=True)
|
||||
assert files["/d/a.txt"] == b"FIRST"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_duplicate_basenames_without_n_last_wins():
|
||||
files = {"/x/a.txt": b"FIRST", "/y/a.txt": b"SECOND", "/d/keep": b"K"}
|
||||
await _run(files, {"/d"}, ["/x/a.txt", "/y/a.txt", "/d"])
|
||||
assert files["/d/a.txt"] == b"SECOND"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_recursive_into_directory():
|
||||
files = {"/src/x.txt": b"X", "/src/sub/y.txt": b"Y"}
|
||||
await _run(files, {"/src"}, ["/src", "/dst"], recursive=True)
|
||||
assert files["/dst/x.txt"] == b"X"
|
||||
assert files["/dst/sub/y.txt"] == b"Y"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_verbose_emits_arrow_lines():
|
||||
files = {"/a.txt": b"AAA"}
|
||||
out, _ = await _run(files, set(), ["/a.txt", "/copy.txt"], v=True)
|
||||
assert out == b"'/a.txt' -> '/copy.txt'\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_records_writes_by_strip_prefix():
|
||||
files = {"/a.txt": b"AAA", "/b.txt": b"BBB", "/d/keep": b"K"}
|
||||
_, io = await _run(files, {"/d"}, ["/a.txt", "/b.txt", "/d"])
|
||||
assert set(io.writes) == {"/d/a.txt", "/d/b.txt"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_missing_source_reports_cannot_stat_and_continues():
|
||||
files = {"/b.txt": b"BBB", "/d/keep": b"K"}
|
||||
_, io = await _run(files, {"/d"}, ["/missing.txt", "/b.txt", "/d"])
|
||||
assert io.exit_code == 1
|
||||
assert b"cp: cannot stat '/missing.txt'" in io.stderr
|
||||
assert files["/d/b.txt"] == b"BBB"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_same_file_errors_and_preserves_content():
|
||||
files = {"/a.txt": b"AAA"}
|
||||
_, io = await _run(files, set(), ["/a.txt", "/a.txt"])
|
||||
assert io.exit_code == 1
|
||||
assert b"'/a.txt' and '/a.txt' are the same file" in io.stderr
|
||||
assert files["/a.txt"] == b"AAA"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_same_file_via_directory_target_errors():
|
||||
files = {"/d/a.txt": b"AAA", "/d/keep": b"K"}
|
||||
_, io = await _run(files, {"/d"}, ["/d/a.txt", "/d"])
|
||||
assert io.exit_code == 1
|
||||
assert b"are the same file" in io.stderr
|
||||
assert files["/d/a.txt"] == b"AAA"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_recursive_into_own_subtree_refused():
|
||||
files = {"/d/a.txt": b"AAA"}
|
||||
_, io = await _run(files, {"/d"}, ["/d", "/d"], recursive=True)
|
||||
assert io.exit_code == 1
|
||||
assert b"cp: cannot copy a directory, '/d', into itself" in io.stderr
|
||||
assert set(files) == {"/d/a.txt"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_recursive_into_nested_subtree_refused():
|
||||
files = {"/d/a.txt": b"AAA"}
|
||||
_, io = await _run(files, {"/d", "/d/sub"}, ["/d", "/d/sub"],
|
||||
recursive=True)
|
||||
assert io.exit_code == 1
|
||||
assert b"into itself" in io.stderr
|
||||
assert set(files) == {"/d/a.txt"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_primitive_copy_records_source_reads():
|
||||
files = {"/a.txt": b"AAA"}
|
||||
stat, _, _ = _make_backend(files, set())
|
||||
|
||||
async def read_bytes(p) -> bytes:
|
||||
return files[_key(p)]
|
||||
|
||||
async def write(p, data: bytes) -> None:
|
||||
files[_key(p)] = data
|
||||
|
||||
_, io = await cp([_spec("/a.txt"), _spec("/copy.txt")],
|
||||
stat=stat,
|
||||
read_bytes=read_bytes,
|
||||
write=write,
|
||||
recursive=False,
|
||||
n=False,
|
||||
v=False)
|
||||
assert files["/copy.txt"] == b"AAA"
|
||||
assert io.reads == {"/a.txt": b"AAA"}
|
||||
assert io.cache == ["/a.txt"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_native_copy_records_no_reads():
|
||||
files = {"/a.txt": b"AAA"}
|
||||
_, io = await _run(files, set(), ["/a.txt", "/copy.txt"])
|
||||
assert io.reads == {}
|
||||
assert io.cache == []
|
||||
@@ -0,0 +1,273 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.du import _depth, du, du_multi
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(virtual=path,
|
||||
directory=path,
|
||||
resource_path=path.strip("/"))
|
||||
|
||||
|
||||
def _make_list_backend(tree: dict[str, int]):
|
||||
"""Build (compute_total, compute_all) where compute_all returns a flat
|
||||
list (NOT a (list, total) tuple), matching the backend `du_all` contract
|
||||
consumed by `du_multi`. Directory targets include a trailing
|
||||
(dirpath, total) entry, mirroring the real backends."""
|
||||
|
||||
async def compute_total(p: PathSpec) -> int:
|
||||
prefix = p.virtual.rstrip("/") + "/"
|
||||
return sum(size for path, size in tree.items()
|
||||
if path == p.virtual or path.startswith(prefix))
|
||||
|
||||
async def compute_all(p: PathSpec) -> list[tuple[str, int]]:
|
||||
prefix = p.virtual.rstrip("/") + "/"
|
||||
entries: list[tuple[str, int]] = []
|
||||
total = 0
|
||||
for path, size in sorted(tree.items()):
|
||||
if path == p.virtual or path.startswith(prefix):
|
||||
entries.append((path, size))
|
||||
total += size
|
||||
if p.virtual not in dict(entries):
|
||||
entries.append((p.virtual, total))
|
||||
return entries
|
||||
|
||||
return compute_total, compute_all
|
||||
|
||||
|
||||
def _make_backend(tree: dict[str, int]):
|
||||
"""Build (compute_total, compute_all) callables over an in-memory tree.
|
||||
|
||||
`tree` maps every file path → size. Directory totals are inferred by
|
||||
walking the tree on demand, matching `du_all` semantics in the real
|
||||
backends (returns flat file entries + recursive total).
|
||||
"""
|
||||
|
||||
async def compute_total(p: PathSpec) -> int:
|
||||
prefix = p.virtual.rstrip("/") + "/"
|
||||
total = 0
|
||||
for path, size in tree.items():
|
||||
if path == p.virtual or path.startswith(prefix):
|
||||
total += size
|
||||
return total
|
||||
|
||||
async def compute_all(p: PathSpec, ) -> tuple[list[tuple[str, int]], int]:
|
||||
prefix = p.virtual.rstrip("/") + "/"
|
||||
entries: list[tuple[str, int]] = []
|
||||
total = 0
|
||||
for path, size in sorted(tree.items()):
|
||||
if path == p.virtual or path.startswith(prefix):
|
||||
entries.append((path, size))
|
||||
total += size
|
||||
return entries, total
|
||||
|
||||
return compute_total, compute_all
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_single_file_default():
|
||||
compute_total, compute_all = _make_backend({"/f.txt": 5})
|
||||
out = await du([_spec("/f.txt")],
|
||||
compute_total=compute_total,
|
||||
compute_all=compute_all)
|
||||
assert out == "5\t/f.txt\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_directory_collapses_to_root_when_not_a():
|
||||
"""POSIX `du` default behavior: one line per dir, files not shown."""
|
||||
tree = {"/dir/a.txt": 3, "/dir/b.txt": 2}
|
||||
compute_total, compute_all = _make_backend(tree)
|
||||
out = await du([_spec("/dir")],
|
||||
compute_total=compute_total,
|
||||
compute_all=compute_all)
|
||||
assert out == "5\t/dir\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_a_lists_all_files():
|
||||
tree = {"/dir/a.txt": 3, "/dir/b.txt": 2}
|
||||
compute_total, compute_all = _make_backend(tree)
|
||||
out = await du([_spec("/dir")],
|
||||
compute_total=compute_total,
|
||||
compute_all=compute_all,
|
||||
a=True)
|
||||
lines = out.splitlines()
|
||||
assert "3\t/dir/a.txt" in lines
|
||||
assert "2\t/dir/b.txt" in lines
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_s_summary_single_line():
|
||||
tree = {"/dir/a.txt": 3, "/dir/sub/b.txt": 2}
|
||||
compute_total, compute_all = _make_backend(tree)
|
||||
out = await du([_spec("/dir")],
|
||||
compute_total=compute_total,
|
||||
compute_all=compute_all,
|
||||
s=True)
|
||||
assert out == "5\t/dir\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_max_depth_zero_in_a_mode_drops_everything_below():
|
||||
"""`-a --max-depth 0` keeps only entries at depth 0 (the root). Since
|
||||
file entries are at depth ≥ 1, no entries remain → falls back to
|
||||
single-line root output."""
|
||||
tree = {"/dir/a.txt": 3, "/dir/sub/b.txt": 2}
|
||||
compute_total, compute_all = _make_backend(tree)
|
||||
out = await du([_spec("/dir")],
|
||||
compute_total=compute_total,
|
||||
compute_all=compute_all,
|
||||
a=True,
|
||||
max_depth=0)
|
||||
assert out == "5\t/dir\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_max_depth_one_keeps_direct_children():
|
||||
tree = {"/dir/a.txt": 3, "/dir/sub/b.txt": 2}
|
||||
compute_total, compute_all = _make_backend(tree)
|
||||
out = await du([_spec("/dir")],
|
||||
compute_total=compute_total,
|
||||
compute_all=compute_all,
|
||||
a=True,
|
||||
max_depth=1)
|
||||
lines = out.splitlines()
|
||||
assert "3\t/dir/a.txt" in lines
|
||||
assert not any("sub/b.txt" in ln for ln in lines)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_c_appends_total_line():
|
||||
tree = {"/dir/a.txt": 3, "/dir/b.txt": 2}
|
||||
compute_total, compute_all = _make_backend(tree)
|
||||
out = await du([_spec("/dir")],
|
||||
compute_total=compute_total,
|
||||
compute_all=compute_all,
|
||||
c=True)
|
||||
assert out.splitlines()[-1] == "5\ttotal"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_h_human_readable_size():
|
||||
tree = {"/big.txt": 2048}
|
||||
compute_total, compute_all = _make_backend(tree)
|
||||
out = await du([_spec("/big.txt")],
|
||||
compute_total=compute_total,
|
||||
compute_all=compute_all,
|
||||
h=True)
|
||||
assert "/big.txt" in out
|
||||
assert out.split("\t")[0].endswith("K")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_multi_path_independent_outputs():
|
||||
tree = {"/a.txt": 3, "/b.txt": 7}
|
||||
compute_total, compute_all = _make_backend(tree)
|
||||
out = await du([_spec("/a.txt"), _spec("/b.txt")],
|
||||
compute_total=compute_total,
|
||||
compute_all=compute_all)
|
||||
assert out == "3\t/a.txt\n7\t/b.txt\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_multi_path_with_c_grand_total():
|
||||
tree = {"/a.txt": 3, "/b.txt": 7}
|
||||
compute_total, compute_all = _make_backend(tree)
|
||||
out = await du([_spec("/a.txt"), _spec("/b.txt")],
|
||||
compute_total=compute_total,
|
||||
compute_all=compute_all,
|
||||
c=True)
|
||||
assert out.splitlines()[-1] == "10\ttotal"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_multi_path_with_h_and_c_sums_raw_bytes():
|
||||
"""-c with -h should still sum the raw byte counts correctly. (Previous
|
||||
string-parse implementation got 0 here because '1.0K' wasn't int())."""
|
||||
tree = {"/a.txt": 1024, "/b.txt": 1024}
|
||||
compute_total, compute_all = _make_backend(tree)
|
||||
out = await du([_spec("/a.txt"), _spec("/b.txt")],
|
||||
compute_total=compute_total,
|
||||
compute_all=compute_all,
|
||||
h=True,
|
||||
c=True)
|
||||
total_line = out.splitlines()[-1]
|
||||
assert total_line.endswith("\ttotal")
|
||||
assert total_line.split("\t")[0].endswith("K")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_empty_target_returns_zero_with_path():
|
||||
compute_total, compute_all = _make_backend({})
|
||||
out = await du([_spec("/nothing")],
|
||||
compute_total=compute_total,
|
||||
compute_all=compute_all)
|
||||
assert out == "0\t/nothing\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_multi_independent_outputs_bytes():
|
||||
compute_total, compute_all = _make_list_backend({"/a.txt": 3, "/b.txt": 7})
|
||||
out = await du_multi([_spec("/a.txt"), _spec("/b.txt")],
|
||||
compute_total=compute_total,
|
||||
compute_all=compute_all)
|
||||
assert out == b"3\t/a.txt\n7\t/b.txt\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_multi_c_grand_total_sums_all_paths():
|
||||
compute_total, compute_all = _make_list_backend({"/a.txt": 3, "/b.txt": 7})
|
||||
out = await du_multi([_spec("/a.txt"), _spec("/b.txt")],
|
||||
compute_total=compute_total,
|
||||
compute_all=compute_all,
|
||||
c=True)
|
||||
assert out.decode().splitlines()[-1] == "10\ttotal"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_multi_directory_collapses_when_not_a():
|
||||
tree = {"/dir/a.txt": 3, "/dir/b.txt": 2}
|
||||
compute_total, compute_all = _make_list_backend(tree)
|
||||
out = await du_multi([_spec("/dir")],
|
||||
compute_total=compute_total,
|
||||
compute_all=compute_all)
|
||||
assert out == b"5\t/dir\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_multi_a_lists_files():
|
||||
tree = {"/dir/a.txt": 3, "/dir/b.txt": 2}
|
||||
compute_total, compute_all = _make_list_backend(tree)
|
||||
out = await du_multi([_spec("/dir")],
|
||||
compute_total=compute_total,
|
||||
compute_all=compute_all,
|
||||
a=True)
|
||||
lines = out.decode().splitlines()
|
||||
assert "3\t/dir/a.txt" in lines
|
||||
assert "2\t/dir/b.txt" in lines
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_multi_compute_all_none_emits_total_only():
|
||||
"""Walk-only backends (gdrive, github) pass compute_all=None: each path
|
||||
yields a single total line; -c still sums across paths."""
|
||||
compute_total, _ = _make_list_backend({"/x/a": 4, "/x/b": 6, "/y/c": 5})
|
||||
out = await du_multi([_spec("/x"), _spec("/y")],
|
||||
compute_total=compute_total,
|
||||
compute_all=None,
|
||||
c=True)
|
||||
assert out.decode().splitlines() == ["10\t/x", "5\t/y", "15\ttotal"]
|
||||
|
||||
|
||||
def test_depth_helper_root_is_zero():
|
||||
assert _depth("/dir", "/dir") == 0
|
||||
|
||||
|
||||
def test_depth_helper_direct_child_is_one():
|
||||
assert _depth("/dir/a.txt", "/dir") == 1
|
||||
|
||||
|
||||
def test_depth_helper_nested():
|
||||
assert _depth("/dir/sub/b.txt", "/dir") == 2
|
||||
@@ -0,0 +1,79 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.file import file_cmd
|
||||
from mirage.types import FileStat, FileType, PathSpec
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(virtual=path,
|
||||
directory=path,
|
||||
resource_path=path.strip("/"))
|
||||
|
||||
|
||||
def _make_backend(files: dict[str, tuple[bytes, FileType]], dirs: set[str]):
|
||||
|
||||
async def stat_fn(accessor: object, p: PathSpec) -> FileStat:
|
||||
if p.virtual in dirs:
|
||||
return FileStat(name=p.virtual, type=FileType.DIRECTORY, size=0)
|
||||
if p.virtual in files:
|
||||
data, ftype = files[p.virtual]
|
||||
return FileStat(name=p.virtual, type=ftype, size=len(data))
|
||||
raise FileNotFoundError(p.virtual)
|
||||
|
||||
async def read_bytes(accessor: object, p: PathSpec) -> bytes:
|
||||
return files[p.virtual][0]
|
||||
|
||||
return stat_fn, read_bytes
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_single_text():
|
||||
stat_fn, read_bytes = _make_backend(
|
||||
{"/a.txt": (b"hello world\n", FileType.TEXT)}, set())
|
||||
out, io = await file_cmd([_spec("/a.txt")],
|
||||
read_bytes=read_bytes,
|
||||
stat_fn=stat_fn)
|
||||
assert out == b"/a.txt: text\n"
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_multiple_paths_one_line_each():
|
||||
stat_fn, read_bytes = _make_backend(
|
||||
{
|
||||
"/a.txt": (b"hello\n", FileType.TEXT),
|
||||
"/b.json": (b'{"k": 1}\n', FileType.JSON),
|
||||
}, set())
|
||||
out, _io = await file_cmd(
|
||||
[_spec("/a.txt"), _spec("/b.json")],
|
||||
read_bytes=read_bytes,
|
||||
stat_fn=stat_fn)
|
||||
lines = out.decode().splitlines()
|
||||
assert lines == ["/a.txt: text", "/b.json: json"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_directory_reported_without_read():
|
||||
stat_fn, read_bytes = _make_backend({}, {"/d"})
|
||||
out, _io = await file_cmd([_spec("/d")],
|
||||
read_bytes=read_bytes,
|
||||
stat_fn=stat_fn)
|
||||
assert out == b"/d: directory\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_brief_drops_path_prefix():
|
||||
stat_fn, read_bytes = _make_backend(
|
||||
{"/a.txt": (b"hello\n", FileType.TEXT)}, set())
|
||||
out, _io = await file_cmd([_spec("/a.txt")],
|
||||
read_bytes=read_bytes,
|
||||
stat_fn=stat_fn,
|
||||
b=True)
|
||||
assert out == b"text\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_missing_operand_raises():
|
||||
stat_fn, read_bytes = _make_backend({}, set())
|
||||
with pytest.raises(ValueError):
|
||||
await file_cmd([], read_bytes=read_bytes, stat_fn=stat_fn)
|
||||
@@ -0,0 +1,432 @@
|
||||
import asyncio
|
||||
from dataclasses import asdict
|
||||
from datetime import datetime, timezone
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.find_eval import Name, Not, Or
|
||||
from mirage.commands.builtin.generic.find import (FindArgs, apply_mount_prefix,
|
||||
apply_mtime_filter,
|
||||
parse_find_args, walk_find)
|
||||
from mirage.commands.errors import FindParseError
|
||||
from mirage.resource.ram import RAMResource
|
||||
from mirage.types import FileStat, FileType, FindType, MountMode, PathSpec
|
||||
from mirage.workspace import Workspace
|
||||
|
||||
|
||||
def _defaults() -> dict:
|
||||
return asdict(FindArgs())
|
||||
|
||||
|
||||
def test_parse_find_args_empty_returns_defaults():
|
||||
args = parse_find_args(())
|
||||
assert asdict(args) == _defaults()
|
||||
|
||||
|
||||
def test_parse_find_args_name_passthrough():
|
||||
args = parse_find_args((), name="*.txt")
|
||||
assert args.name == "*.txt"
|
||||
assert args.or_names is None
|
||||
|
||||
|
||||
def test_parse_find_args_iname_and_path():
|
||||
args = parse_find_args((), iname="HELLO.*", path="**/sub/*")
|
||||
assert args.iname == "HELLO.*"
|
||||
assert args.path_pattern == "**/sub/*"
|
||||
|
||||
|
||||
def test_parse_find_args_maxdepth_mindepth_str_to_int():
|
||||
args = parse_find_args((), maxdepth="3", mindepth="1")
|
||||
assert args.maxdepth == 3
|
||||
assert args.mindepth == 1
|
||||
|
||||
|
||||
def test_parse_find_args_size_plus_lower_bound():
|
||||
args = parse_find_args((), size="+500c")
|
||||
assert args.min_size == 501
|
||||
assert args.max_size is None
|
||||
|
||||
|
||||
def test_parse_find_args_size_minus_upper_bound():
|
||||
args = parse_find_args((), size="-1k")
|
||||
assert args.min_size is None
|
||||
assert args.max_size == 0
|
||||
|
||||
|
||||
def test_parse_find_args_size_exact():
|
||||
args = parse_find_args((), size="1k")
|
||||
assert args.min_size == 1
|
||||
assert args.max_size == 1024
|
||||
|
||||
|
||||
def test_parse_find_args_mtime_minus_recent():
|
||||
"""`-mtime -1` means modified within last 1 day."""
|
||||
args = parse_find_args((), mtime="-1")
|
||||
assert args.mtime_min is not None
|
||||
assert args.mtime_max is None
|
||||
|
||||
|
||||
def test_parse_find_args_mtime_plus_old():
|
||||
args = parse_find_args((), mtime="+7")
|
||||
assert args.mtime_min is None
|
||||
assert args.mtime_max is not None
|
||||
|
||||
|
||||
def test_parse_find_args_type_canonicalized_to_findtype_enum():
|
||||
"""Known POSIX `-type` values become FindType members."""
|
||||
assert parse_find_args((), type="d").type is FindType.DIRECTORY
|
||||
assert parse_find_args((), type="f").type is FindType.FILE
|
||||
|
||||
|
||||
def test_parse_find_args_unknown_type_left_as_string():
|
||||
"""Non-POSIX types pass through verbatim (allows custom backend types)."""
|
||||
assert parse_find_args((), type="symlink").type == "symlink"
|
||||
|
||||
|
||||
def test_parse_find_args_unknown_predicate_raises():
|
||||
with pytest.raises(FindParseError,
|
||||
match="find: unknown predicate '-bogus'"):
|
||||
parse_find_args(("-bogus", ))
|
||||
|
||||
|
||||
def test_parse_find_args_negation_builds_not_tree():
|
||||
args = parse_find_args(("-not", "-name", "*.pyc"))
|
||||
assert args.tree == Not(Name("*.pyc"))
|
||||
|
||||
|
||||
def test_parse_find_args_or_builds_or_tree():
|
||||
args = parse_find_args(("-name", "*.txt", "-o", "-name", "*.py"))
|
||||
assert args.tree == Or([Name("*.txt"), Name("*.py")])
|
||||
|
||||
|
||||
def test_parse_find_args_or_names_none_when_only_one_name():
|
||||
"""If only `name` is set with no `-or -name` clauses, or_names is None."""
|
||||
args = parse_find_args((), name="*.txt")
|
||||
assert args.or_names is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_apply_mtime_filter_skips_when_no_window():
|
||||
out = await apply_mtime_filter(
|
||||
["/a.txt"],
|
||||
mtime_min=None,
|
||||
mtime_max=None,
|
||||
stat=_unreached_stat,
|
||||
)
|
||||
assert out == ["/a.txt"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_apply_mtime_filter_keeps_within_window():
|
||||
now = datetime.now(tz=timezone.utc)
|
||||
iso = now.isoformat()
|
||||
|
||||
async def stat(_spec: PathSpec) -> FileStat:
|
||||
return FileStat(name="a.txt", size=1, modified=iso, type=FileType.TEXT)
|
||||
|
||||
out = await apply_mtime_filter(
|
||||
["/a.txt"],
|
||||
mtime_min=now.timestamp() - 60,
|
||||
mtime_max=now.timestamp() + 60,
|
||||
stat=stat,
|
||||
)
|
||||
assert out == ["/a.txt"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_apply_mtime_filter_drops_outside_window():
|
||||
old = datetime(2020, 1, 1, tzinfo=timezone.utc)
|
||||
|
||||
async def stat(_spec: PathSpec) -> FileStat:
|
||||
return FileStat(name="a.txt",
|
||||
size=1,
|
||||
modified=old.isoformat(),
|
||||
type=FileType.TEXT)
|
||||
|
||||
out = await apply_mtime_filter(
|
||||
["/a.txt"],
|
||||
mtime_min=datetime(2025, 1, 1, tzinfo=timezone.utc).timestamp(),
|
||||
mtime_max=None,
|
||||
stat=stat,
|
||||
)
|
||||
assert out == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_apply_mtime_filter_drops_entries_with_no_modified_time():
|
||||
|
||||
async def stat(_spec: PathSpec) -> FileStat:
|
||||
return FileStat(name="a.txt",
|
||||
size=1,
|
||||
modified=None,
|
||||
type=FileType.TEXT)
|
||||
|
||||
out = await apply_mtime_filter(
|
||||
["/a.txt"],
|
||||
mtime_min=1.0,
|
||||
mtime_max=None,
|
||||
stat=stat,
|
||||
)
|
||||
assert out == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_apply_mtime_filter_silently_skips_stat_errors():
|
||||
|
||||
async def stat(_spec: PathSpec) -> FileStat:
|
||||
raise FileNotFoundError("gone")
|
||||
|
||||
out = await apply_mtime_filter(
|
||||
["/a.txt", "/b.txt"],
|
||||
mtime_min=1.0,
|
||||
mtime_max=None,
|
||||
stat=stat,
|
||||
)
|
||||
assert out == []
|
||||
|
||||
|
||||
def test_apply_mount_prefix_noop_when_empty():
|
||||
assert apply_mount_prefix(["/a.txt"], "") == ["/a.txt"]
|
||||
|
||||
|
||||
def test_apply_mount_prefix_prepends():
|
||||
assert apply_mount_prefix(["/a.txt", "/dir/b.txt"],
|
||||
"/mnt") == ["/mnt/a.txt", "/mnt/dir/b.txt"]
|
||||
|
||||
|
||||
def test_apply_mount_prefix_strips_leading_slash_from_entries():
|
||||
assert apply_mount_prefix(["a.txt"], "/mnt") == ["/mnt/a.txt"]
|
||||
|
||||
|
||||
async def _unreached_stat(_spec: PathSpec) -> FileStat:
|
||||
raise AssertionError("stat should not be called when no mtime window set")
|
||||
|
||||
|
||||
def _root_spec() -> PathSpec:
|
||||
return PathSpec(resource_path="",
|
||||
virtual="/",
|
||||
directory="/",
|
||||
resolved=False)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_find_tolerates_not_found_readdir():
|
||||
readdir = AsyncMock(side_effect=FileNotFoundError("/"))
|
||||
stat = AsyncMock(side_effect=FileNotFoundError("/"))
|
||||
results = await walk_find(_root_spec(),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
is_dir_name=lambda c: None,
|
||||
index=None,
|
||||
args=FindArgs())
|
||||
assert results == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_find_emits_start_path_at_depth_zero():
|
||||
readdir = AsyncMock(return_value=["/child.txt"])
|
||||
stat = AsyncMock(return_value=FileStat(name="/", type=FileType.DIRECTORY))
|
||||
results = await walk_find(_root_spec(),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
is_dir_name=lambda c: False,
|
||||
index=None,
|
||||
args=FindArgs(maxdepth=0))
|
||||
assert results == ["/"]
|
||||
readdir.assert_not_awaited()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_find_propagates_non_not_found_readdir_errors():
|
||||
readdir = AsyncMock(side_effect=ValueError("bad page token"))
|
||||
stat = AsyncMock()
|
||||
with pytest.raises(ValueError, match="bad page token"):
|
||||
await walk_find(_root_spec(),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
is_dir_name=lambda c: None,
|
||||
index=None,
|
||||
args=FindArgs())
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_find_stat_fallback_treats_not_found_as_file():
|
||||
readdir = AsyncMock(return_value=["/mystery"])
|
||||
stat = AsyncMock(side_effect=FileNotFoundError("/mystery"))
|
||||
results = await walk_find(_root_spec(),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
is_dir_name=lambda c: None,
|
||||
index=None,
|
||||
args=FindArgs(type=FindType.FILE))
|
||||
assert results == ["/mystery"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_find_empty_matches_empty_files_and_dirs():
|
||||
|
||||
async def readdir(spec: PathSpec, _index):
|
||||
table = {
|
||||
"/": ["/empty.txt", "/full.txt", "/empty-dir", "/full-dir"],
|
||||
"/empty-dir": [],
|
||||
"/full-dir": ["/full-dir/a.txt"],
|
||||
}
|
||||
return table[spec.virtual]
|
||||
|
||||
async def stat(spec: PathSpec, _index):
|
||||
stats = {
|
||||
"/": FileStat(name="/", type=FileType.DIRECTORY),
|
||||
"/empty.txt": FileStat(name="empty.txt",
|
||||
size=0,
|
||||
type=FileType.TEXT),
|
||||
"/full.txt": FileStat(name="full.txt", size=1, type=FileType.TEXT),
|
||||
"/empty-dir": FileStat(name="empty-dir", type=FileType.DIRECTORY),
|
||||
"/full-dir": FileStat(name="full-dir", type=FileType.DIRECTORY),
|
||||
"/full-dir/a.txt": FileStat(name="a.txt",
|
||||
size=1,
|
||||
type=FileType.TEXT),
|
||||
}
|
||||
return stats[spec.virtual]
|
||||
|
||||
results = await walk_find(_root_spec(),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
is_dir_name=lambda c: c.endswith("dir"),
|
||||
index=None,
|
||||
args=parse_find_args(("-empty", )))
|
||||
assert results == ["/empty-dir", "/empty.txt"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_find_not_negates_predicate():
|
||||
readdir = AsyncMock(return_value=["/a.txt", "/b.md"])
|
||||
|
||||
async def stat(spec: PathSpec, _index):
|
||||
if spec.virtual == "/":
|
||||
return FileStat(name="/", type=FileType.DIRECTORY)
|
||||
return FileStat(name=spec.virtual.rsplit("/", 1)[-1],
|
||||
size=1,
|
||||
type=FileType.TEXT)
|
||||
|
||||
results = await walk_find(_root_spec(),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
is_dir_name=lambda c: False,
|
||||
index=None,
|
||||
args=parse_find_args(("-not", "-name", "*.txt")))
|
||||
assert results == ["/", "/b.md"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_find_stat_fallback_propagates_other_errors():
|
||||
readdir = AsyncMock(return_value=["/mystery"])
|
||||
stat = AsyncMock(side_effect=ValueError("rate limited"))
|
||||
with pytest.raises(ValueError, match="rate limited"):
|
||||
await walk_find(_root_spec(),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
is_dir_name=lambda c: None,
|
||||
index=None,
|
||||
args=FindArgs())
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_find_size_filter_drops_not_found_entries():
|
||||
readdir = AsyncMock(return_value=["/a.json"])
|
||||
stat = AsyncMock(side_effect=FileNotFoundError("/a.json"))
|
||||
results = await walk_find(_root_spec(),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
is_dir_name=lambda c: False,
|
||||
index=None,
|
||||
args=FindArgs(min_size=1))
|
||||
assert results == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_find_size_filter_propagates_other_stat_errors():
|
||||
readdir = AsyncMock(return_value=["/a.json"])
|
||||
stat = AsyncMock(side_effect=ValueError("rate limited"))
|
||||
with pytest.raises(ValueError, match="rate limited"):
|
||||
await walk_find(_root_spec(),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
is_dir_name=lambda c: False,
|
||||
index=None,
|
||||
args=FindArgs(min_size=1))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("kwargs,flag,value", [
|
||||
({
|
||||
"maxdepth": "abc"
|
||||
}, "-maxdepth", "abc"),
|
||||
({
|
||||
"mindepth": "xx"
|
||||
}, "-mindepth", "xx"),
|
||||
({
|
||||
"size": ""
|
||||
}, "-size", ""),
|
||||
({
|
||||
"size": "abc"
|
||||
}, "-size", "abc"),
|
||||
({
|
||||
"mtime": "abc"
|
||||
}, "-mtime", "abc"),
|
||||
])
|
||||
def test_parse_find_args_invalid_numeric_raises_find_parse_error(
|
||||
kwargs, flag, value):
|
||||
with pytest.raises(FindParseError) as exc:
|
||||
parse_find_args((), **kwargs)
|
||||
assert str(exc.value) == f"find: invalid argument '{value}' to '{flag}'"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("expr", [
|
||||
"-maxdepth abc",
|
||||
"-mindepth xx",
|
||||
"-size ''",
|
||||
"-size abc",
|
||||
"-mtime abc",
|
||||
])
|
||||
def test_find_invalid_numeric_arg_exits_one_with_clean_stderr(expr):
|
||||
|
||||
async def _go() -> tuple[int, str]:
|
||||
ws = Workspace({"/": RAMResource()}, mode=MountMode.WRITE)
|
||||
ws.create_session("s")
|
||||
r = await ws.execute(f"find / {expr}", session_id="s")
|
||||
return r.exit_code, await r.stderr_str()
|
||||
|
||||
code, stderr = asyncio.run(_go())
|
||||
assert code == 1
|
||||
assert stderr.startswith("find: invalid argument ")
|
||||
assert stderr.endswith("\n")
|
||||
|
||||
|
||||
# ── Issue #312 parse-level regression tests ────────────────
|
||||
|
||||
|
||||
def test_parse_find_args_start_path_included():
|
||||
args = parse_find_args(())
|
||||
assert args.maxdepth is None
|
||||
|
||||
|
||||
def test_parse_find_args_maxdepth_zero():
|
||||
args = parse_find_args(("-maxdepth", "0"))
|
||||
assert args.maxdepth == 0
|
||||
|
||||
|
||||
def test_parse_find_args_empty_predicate():
|
||||
args = parse_find_args(("-empty", ))
|
||||
assert args.empty is True
|
||||
|
||||
|
||||
def test_parse_find_args_not_negation():
|
||||
args = parse_find_args(("-not", "-name", "*.txt"))
|
||||
assert isinstance(args.tree, Not)
|
||||
assert isinstance(args.tree.kid, Name)
|
||||
assert args.tree.kid.pattern == "*.txt"
|
||||
|
||||
|
||||
def test_parse_find_args_bogus_predicate_raises():
|
||||
with pytest.raises(FindParseError, match="unknown predicate"):
|
||||
parse_find_args(("-boguspredicate", ))
|
||||
@@ -0,0 +1,583 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.grep import GrepFlags, grep, parse_flags
|
||||
from mirage.commands.spec.types import FlagView
|
||||
from mirage.types import FileStat, FileType, PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(resource_path=(path).strip("/"),
|
||||
virtual=path,
|
||||
directory=path,
|
||||
resolved=True)
|
||||
|
||||
|
||||
def _make_backend(files: dict[str, bytes], dirs: set[str] | None = None):
|
||||
"""Build (readdir, stat, read_bytes, read_stream) callables over a
|
||||
simple in-memory file tree. `dirs` is the set of directory paths;
|
||||
intermediate dirs are inferred from file paths if not specified."""
|
||||
|
||||
inferred_dirs = set(dirs) if dirs is not None else set()
|
||||
for f in files:
|
||||
parts = f.split("/")
|
||||
for i in range(1, len(parts)):
|
||||
d = "/".join(parts[:i]) or "/"
|
||||
inferred_dirs.add(d)
|
||||
inferred_dirs.add("/")
|
||||
|
||||
async def readdir(accessor, path, index=None):
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
p = spec.virtual.rstrip("/") or "/"
|
||||
if p not in inferred_dirs:
|
||||
raise FileNotFoundError(p)
|
||||
prefix = p + "/" if p != "/" else "/"
|
||||
children: set[str] = set()
|
||||
for f in files:
|
||||
if f.startswith(prefix):
|
||||
rest = f[len(prefix):]
|
||||
child = rest.split("/")[0]
|
||||
children.add(prefix + child)
|
||||
for d in inferred_dirs:
|
||||
if d == p:
|
||||
continue
|
||||
if d.startswith(prefix):
|
||||
rest = d[len(prefix):]
|
||||
child = rest.split("/")[0]
|
||||
children.add(prefix + child)
|
||||
return sorted(children)
|
||||
|
||||
async def stat(accessor, path, index=None):
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
p = spec.virtual
|
||||
if p in files:
|
||||
return FileStat(name=p.rsplit("/", 1)[-1] or p,
|
||||
size=len(files[p]),
|
||||
type=FileType.TEXT)
|
||||
if p.rstrip("/") in inferred_dirs or p in inferred_dirs:
|
||||
return FileStat(name=p.rsplit("/", 1)[-1] or "/",
|
||||
type=FileType.DIRECTORY)
|
||||
raise FileNotFoundError(p)
|
||||
|
||||
async def read_bytes(accessor, path, index=None):
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
if spec.virtual not in files:
|
||||
raise FileNotFoundError(spec.virtual)
|
||||
return files[spec.virtual]
|
||||
|
||||
async def read_stream(accessor, path, index=None):
|
||||
data = await read_bytes(accessor, path)
|
||||
yield data
|
||||
|
||||
return readdir, stat, read_bytes, read_stream
|
||||
|
||||
|
||||
def _drain(stdout):
|
||||
if isinstance(stdout, bytes):
|
||||
return stdout
|
||||
return b"".join([c for c in stdout])
|
||||
|
||||
|
||||
async def _drain_async(stdout):
|
||||
if stdout is None:
|
||||
return b""
|
||||
if isinstance(stdout, bytes):
|
||||
return stdout
|
||||
chunks = [chunk async for chunk in stdout]
|
||||
return b"".join(chunks)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_stdin_basic():
|
||||
readdir, stat, rb, rs = _make_backend({})
|
||||
output, io = await grep(
|
||||
[],
|
||||
["apple"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"apple\nbanana\napricot\n",
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "apple" in decoded
|
||||
assert "apricot" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_file_basic():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/a.txt":
|
||||
b"apple\nbanana\napricot\n",
|
||||
})
|
||||
output, io = await grep(
|
||||
[_spec("/a.txt")],
|
||||
["ap"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "apple" in decoded
|
||||
assert "apricot" in decoded
|
||||
assert "banana" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_single_dir_operand_warns():
|
||||
readdir, stat, rb, rs = _make_backend({"/d/a.txt": b"apple\n"})
|
||||
output, io = await grep(
|
||||
[_spec("/d")],
|
||||
["ap"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
assert await _drain_async(output) == b""
|
||||
assert io.exit_code == 1
|
||||
assert io.stderr == b"grep: /d: Is a directory\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_ignore_case():
|
||||
readdir, stat, rb, rs = _make_backend({"/a.txt": b"Apple\nBANANA\n"})
|
||||
output, _ = await grep(
|
||||
[_spec("/a.txt")],
|
||||
["apple"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={"i": True},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "Apple" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_invert():
|
||||
readdir, stat, rb, rs = _make_backend(
|
||||
{"/a.txt": b"apple\nbanana\ncherry\n"})
|
||||
output, _ = await grep(
|
||||
[_spec("/a.txt")],
|
||||
["banana"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={"v": True},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "apple" in decoded
|
||||
assert "cherry" in decoded
|
||||
assert "banana" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_count_only():
|
||||
readdir, stat, rb, rs = _make_backend(
|
||||
{"/a.txt": b"apple\nbanana\napricot\n"})
|
||||
output, _ = await grep(
|
||||
[_spec("/a.txt")],
|
||||
["ap"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={"c": True},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode().strip()
|
||||
assert decoded == "2"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_no_match_returns_exit_1():
|
||||
readdir, stat, rb, rs = _make_backend({"/a.txt": b"hello\nworld\n"})
|
||||
output, io = await grep(
|
||||
[_spec("/a.txt")],
|
||||
["zzz"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
drained = await _drain_async(output)
|
||||
assert drained == b""
|
||||
assert io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_recursive_finds_files_in_subdirs():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/dir/a.txt": b"apple\n",
|
||||
"/dir/sub/b.txt": b"apricot\n",
|
||||
})
|
||||
output, io = await grep(
|
||||
[_spec("/dir")],
|
||||
["ap"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={"r": True},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "apple" in decoded
|
||||
assert "apricot" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_recursive_single_file_prefixes_filename():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/log.txt":
|
||||
b"one\nerror here\ntwo\nerror again\n",
|
||||
})
|
||||
output, _ = await grep(
|
||||
[_spec("/log.txt")],
|
||||
["error"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={
|
||||
"r": True,
|
||||
"n": True
|
||||
},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert decoded == "/log.txt:2:error here\n/log.txt:4:error again\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_files_only_lists_matching_files():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/dir/a.txt": b"apple\n",
|
||||
"/dir/b.txt": b"zebra\n",
|
||||
})
|
||||
output, _ = await grep(
|
||||
[_spec("/dir")],
|
||||
["apple"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={
|
||||
"r": True,
|
||||
"args_l": True
|
||||
},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "/dir/a.txt" in decoded
|
||||
assert "/dir/b.txt" not in decoded
|
||||
|
||||
|
||||
def _make_prefixed_backend(files: dict[str, bytes], mount_prefix: str):
|
||||
"""Backend that mimics real s3/disk/gdrive readdir: entries returned
|
||||
are already prepended with ``mount_prefix``. Also raises if ``index``
|
||||
is not threaded through (gdrive-style)."""
|
||||
|
||||
full_files = {mount_prefix + k: v for k, v in files.items()}
|
||||
inferred_dirs: set[str] = {mount_prefix or "/"}
|
||||
for f in full_files:
|
||||
parts = f.split("/")
|
||||
for i in range(1, len(parts)):
|
||||
d = "/".join(parts[:i]) or "/"
|
||||
inferred_dirs.add(d)
|
||||
|
||||
def _full(p: str) -> str:
|
||||
if mount_prefix and not p.startswith(mount_prefix):
|
||||
return mount_prefix + p
|
||||
return p
|
||||
|
||||
async def readdir(accessor, path, index=None):
|
||||
if index is None:
|
||||
raise FileNotFoundError("index required")
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
p = _full(spec.virtual).rstrip("/") or "/"
|
||||
if p not in inferred_dirs:
|
||||
raise FileNotFoundError(p)
|
||||
prefix = p + "/" if p != "/" else "/"
|
||||
children: set[str] = set()
|
||||
for f in full_files:
|
||||
if f.startswith(prefix):
|
||||
child = prefix + f[len(prefix):].split("/")[0]
|
||||
children.add(child)
|
||||
for d in inferred_dirs:
|
||||
if d == p or not d.startswith(prefix):
|
||||
continue
|
||||
child = prefix + d[len(prefix):].split("/")[0]
|
||||
children.add(child)
|
||||
return sorted(children)
|
||||
|
||||
async def stat(accessor, path, index=None):
|
||||
if index is None:
|
||||
raise FileNotFoundError("index required")
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
p = _full(spec.virtual)
|
||||
if p in full_files:
|
||||
return FileStat(name=p.rsplit("/", 1)[-1],
|
||||
size=len(full_files[p]),
|
||||
type=FileType.TEXT)
|
||||
if p.rstrip("/") in inferred_dirs:
|
||||
return FileStat(name=p.rsplit("/", 1)[-1] or "/",
|
||||
type=FileType.DIRECTORY)
|
||||
raise FileNotFoundError(p)
|
||||
|
||||
async def read_bytes(accessor, path, index=None):
|
||||
if index is None:
|
||||
raise FileNotFoundError("index required")
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
p = _full(spec.virtual)
|
||||
if p not in full_files:
|
||||
raise FileNotFoundError(p)
|
||||
return full_files[p]
|
||||
|
||||
return readdir, stat, read_bytes
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_recursive_files_only_mount_prefix():
|
||||
readdir, stat, rb = _make_prefixed_backend(
|
||||
{
|
||||
"/dir/a.txt": b"apple\n",
|
||||
"/dir/b.txt": b"zebra\n",
|
||||
},
|
||||
mount_prefix="/s3",
|
||||
)
|
||||
p = PathSpec(resource_path=mount_key("/dir", "/s3"),
|
||||
virtual="/dir",
|
||||
directory="/dir",
|
||||
resolved=True)
|
||||
output, _ = await grep(
|
||||
[p],
|
||||
["apple"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=None,
|
||||
flags={
|
||||
"r": True,
|
||||
"args_l": True
|
||||
},
|
||||
index=object(),
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode().strip()
|
||||
assert decoded == "/s3/dir/a.txt"
|
||||
assert "/s3/s3" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_single_file_threads_index():
|
||||
readdir, stat, rb = _make_prefixed_backend(
|
||||
{"/dir/a.txt": b"apple\n"},
|
||||
mount_prefix="/gd",
|
||||
)
|
||||
p = PathSpec(resource_path=mount_key("/dir/a.txt", "/gd"),
|
||||
virtual="/dir/a.txt",
|
||||
directory="/dir/a.txt",
|
||||
resolved=True)
|
||||
output, _ = await grep(
|
||||
[p],
|
||||
["apple"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=None,
|
||||
index=object(),
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "apple" in decoded
|
||||
|
||||
|
||||
def test_parse_flags_defaults_and_context_fallback():
|
||||
f = parse_flags(FlagView({}), never_match=False)
|
||||
assert f == GrepFlags(ignore_case=False,
|
||||
invert=False,
|
||||
line_numbers=False,
|
||||
count_only=False,
|
||||
files_only=False,
|
||||
whole_word=False,
|
||||
fixed_string=False,
|
||||
only_matching=False,
|
||||
quiet=False,
|
||||
recursive=False,
|
||||
with_filename=False,
|
||||
no_filename=False,
|
||||
max_count=None,
|
||||
after_context=0,
|
||||
before_context=0)
|
||||
f = parse_flags(FlagView({"A": "2", "C": "5"}), never_match=False)
|
||||
assert f.after_context == 2
|
||||
assert f.before_context == 5
|
||||
|
||||
|
||||
def test_parse_flags_never_match_suppresses_fixed_string():
|
||||
f = parse_flags(FlagView({"F": True}), never_match=True)
|
||||
assert f.fixed_string is False
|
||||
|
||||
|
||||
def test_grep_flags_struct_rejects_typos():
|
||||
f = parse_flags(FlagView({"i": True}), never_match=False)
|
||||
with pytest.raises(AttributeError):
|
||||
_ = f.ignorecase
|
||||
# FrozenInstanceError subclasses AttributeError
|
||||
with pytest.raises(AttributeError):
|
||||
f.ignore_case = False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_count_only_no_match_exit_1():
|
||||
readdir, stat, rb, rs = _make_backend({"/a.txt": b"hello\nworld\n"})
|
||||
output, io = await grep(
|
||||
[_spec("/a.txt")],
|
||||
["zzz"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={"c": True},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode().strip()
|
||||
assert decoded == "0"
|
||||
assert io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_stdin_count_only_no_match_exit_1():
|
||||
readdir, stat, rb, rs = _make_backend({})
|
||||
output, io = await grep(
|
||||
[],
|
||||
["zzz"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"hello\nworld\n",
|
||||
flags={"c": True},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode().strip()
|
||||
assert decoded == "0"
|
||||
assert io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_count_only_multi_file_no_match_exit_1():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/a.txt": b"hello\n",
|
||||
"/b.txt": b"world\n",
|
||||
})
|
||||
output, io = await grep(
|
||||
[_spec("/a.txt"), _spec("/b.txt")],
|
||||
["zzz"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={"c": True},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert decoded.splitlines() == ["/a.txt:0", "/b.txt:0"]
|
||||
assert io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_count_only_multi_file_match_exit_0():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/a.txt": b"hello\n",
|
||||
"/b.txt": b"world\n",
|
||||
})
|
||||
output, io = await grep(
|
||||
[_spec("/a.txt"), _spec("/b.txt")],
|
||||
["hello"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={"c": True},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert decoded.splitlines() == ["/a.txt:1", "/b.txt:0"]
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_recursive_count_only_no_match_exit_1():
|
||||
readdir, stat, rb, rs = _make_backend({"/d/a.txt": b"hello\n"})
|
||||
output, io = await grep(
|
||||
[_spec("/d")],
|
||||
["zzz"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={
|
||||
"r": True,
|
||||
"c": True
|
||||
},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert decoded.splitlines() == ["/d/a.txt:0"]
|
||||
assert io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_single_file_dash_h_prefixes_filename():
|
||||
readdir, stat, rb, rs = _make_backend({"/a.txt": b"apple\nbanana\n"})
|
||||
output, io = await grep(
|
||||
[_spec("/a.txt")],
|
||||
["apple"],
|
||||
{"H": True},
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert decoded == "/a.txt:apple\n"
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_single_file_dash_h_count_prefixes_filename():
|
||||
readdir, stat, rb, rs = _make_backend({"/a.txt": b"apple\nbanana\n"})
|
||||
output, io = await grep(
|
||||
[_spec("/a.txt")],
|
||||
["a"],
|
||||
{
|
||||
"H": True,
|
||||
"c": True
|
||||
},
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert decoded == "/a.txt:2\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_multi_file_no_filename_suppresses_prefix():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/a.txt": b"apple\n",
|
||||
"/b.txt": b"apricot\n",
|
||||
})
|
||||
output, io = await grep(
|
||||
[_spec("/a.txt"), _spec("/b.txt")],
|
||||
["ap"],
|
||||
{"h": True},
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert decoded == "apple\napricot\n"
|
||||
@@ -0,0 +1,140 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.head import head
|
||||
|
||||
|
||||
async def _drain(gen):
|
||||
return b"".join([c async for c in gen])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_first_c_bytes_fast_path():
|
||||
out = await _drain(head(b"hello world", c=5))
|
||||
assert out == b"hello"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_first_c_bytes_across_chunks():
|
||||
|
||||
async def src():
|
||||
yield b"hel"
|
||||
yield b"lo wor"
|
||||
yield b"ld"
|
||||
|
||||
out = await _drain(head(src(), c=8))
|
||||
assert out == b"hello wo"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_first_c_bytes_shorter_than_total():
|
||||
out = await _drain(head(b"hi", c=100))
|
||||
assert out == b"hi"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_first_c_zero_emits_nothing():
|
||||
out = await _drain(head(b"hello", c=0))
|
||||
assert out == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_first_n_lines_from_bytes():
|
||||
out = await _drain(head(b"a\nb\nc\nd\n", n=2))
|
||||
assert out == b"a\nb\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_first_n_lines_from_stream():
|
||||
|
||||
async def src():
|
||||
yield b"a\nb"
|
||||
yield b"\nc\nd\n"
|
||||
|
||||
out = await _drain(head(src(), n=2))
|
||||
assert out == b"a\nb\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_default_n_is_10():
|
||||
body = b"".join(f"line{i}\n".encode() for i in range(1, 15))
|
||||
out = await _drain(head(body))
|
||||
lines = out.decode().splitlines()
|
||||
assert len(lines) == 10
|
||||
assert lines[0] == "line1"
|
||||
assert lines[9] == "line10"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_n_zero_emits_nothing():
|
||||
out = await _drain(head(b"a\nb\n", n=0))
|
||||
assert out == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_negative_n_excludes_last_n_lines():
|
||||
out = await _drain(head(b"a\nb\nc\nd\n", n=-1))
|
||||
assert out == b"a\nb\nc\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_negative_n_equal_to_total_emits_nothing():
|
||||
out = await _drain(head(b"a\nb\n", n=-2))
|
||||
assert out == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_stops_consuming_stream_after_n_lines():
|
||||
consumed = []
|
||||
|
||||
async def src():
|
||||
for line in [b"a\n", b"b\n", b"c\n", b"d\n", b"e\n"]:
|
||||
consumed.append(line)
|
||||
yield line
|
||||
|
||||
chunks = [c async for c in head(src(), n=2)]
|
||||
assert b"".join(chunks) == b"a\nb\n"
|
||||
assert consumed == [b"a\n", b"b\n"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_no_trailing_newline_in_last_line():
|
||||
out = await _drain(head(b"a\nb\nno-end", n=3))
|
||||
assert out == b"a\nb\nno-end"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_n_larger_than_available():
|
||||
out = await _drain(head(b"a\nb\n", n=10))
|
||||
assert out == b"a\nb\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_n_one():
|
||||
out = await _drain(head(b"line1\nline2\nline3\n", n=1))
|
||||
assert out == b"line1\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_empty_input():
|
||||
out = await _drain(head(b""))
|
||||
assert out == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_empty_input_with_c():
|
||||
out = await _drain(head(b"", c=10))
|
||||
assert out == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_single_line_no_newline_default_n():
|
||||
"""head on a single line without trailing newline returns it as-is."""
|
||||
out = await _drain(head(b"hello"))
|
||||
assert out == b"hello"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_c_negative_is_all_but_last_abs():
|
||||
"""GNU head: -c -N emits all but the last N bytes."""
|
||||
out = await _drain(head(b"hello", c=-3))
|
||||
assert out == b"he"
|
||||
@@ -0,0 +1,76 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.head import head_multi
|
||||
from mirage.types import PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
|
||||
def _paths(*names: str) -> list[PathSpec]:
|
||||
return [
|
||||
PathSpec(resource_path=mount_key(n, ""),
|
||||
virtual=n,
|
||||
directory="/d",
|
||||
resolved=True) for n in names
|
||||
]
|
||||
|
||||
|
||||
async def _collect(gen) -> bytes:
|
||||
out = b""
|
||||
async for chunk in gen:
|
||||
out += chunk
|
||||
return out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_multi_bytes_reader_no_headers():
|
||||
data = {"/a": b"a1\na2\na3\n", "/b": b"b1\nb2\n"}
|
||||
|
||||
async def read(accessor, p, index):
|
||||
return data[p.virtual]
|
||||
|
||||
out = await _collect(
|
||||
head_multi(_paths("/a", "/b"), read=read, n=1, show_headers=False))
|
||||
assert out == b"a1\nb1\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_multi_with_headers():
|
||||
data = {"/a": b"a1\na2\n", "/b": b"b1\nb2\n"}
|
||||
|
||||
async def read(accessor, p, index):
|
||||
return data[p.virtual]
|
||||
|
||||
out = await _collect(
|
||||
head_multi(_paths("/a", "/b"), read=read, n=1, show_headers=True))
|
||||
assert out == b"==> /a <==\na1\n\n==> /b <==\nb1\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_multi_stream_reader():
|
||||
chunks = {"/a": [b"a1\n", b"a2\n"], "/b": [b"b1\n"]}
|
||||
|
||||
def read(accessor, p, index):
|
||||
|
||||
async def gen():
|
||||
for ch in chunks[p.virtual]:
|
||||
yield ch
|
||||
|
||||
return gen()
|
||||
|
||||
out = await _collect(
|
||||
head_multi(_paths("/a", "/b"), read=read, n=5, show_headers=True))
|
||||
assert out == b"==> /a <==\na1\na2\n\n==> /b <==\nb1\n"
|
||||
@@ -0,0 +1,322 @@
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.ls import (format_simple, get_extension,
|
||||
ls, walk)
|
||||
from mirage.types import FileStat, FileType, LsSortBy, PathSpec
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(virtual=path,
|
||||
directory=path,
|
||||
resource_path=path.strip("/"))
|
||||
|
||||
|
||||
def _make_fs_backend(tree: dict[str, FileStat]):
|
||||
"""Build (readdir, stat) callables over an in-memory entry tree.
|
||||
|
||||
`tree` maps absolute path → FileStat. Directories are entries whose
|
||||
type == FileType.DIRECTORY. readdir lists direct children of the path.
|
||||
"""
|
||||
|
||||
async def stat(p: PathSpec, index=None) -> FileStat:
|
||||
if p.virtual not in tree:
|
||||
raise FileNotFoundError(p.virtual)
|
||||
return tree[p.virtual]
|
||||
|
||||
async def readdir(p: PathSpec, _index=None) -> list[str]:
|
||||
if p.virtual not in tree:
|
||||
raise FileNotFoundError(p.virtual)
|
||||
if tree[p.virtual].type != FileType.DIRECTORY:
|
||||
raise ValueError(f"not a directory: {p.virtual}")
|
||||
prefix = p.virtual.rstrip("/") + "/"
|
||||
children: list[str] = []
|
||||
for key in tree:
|
||||
if key == p.virtual:
|
||||
continue
|
||||
if key.startswith(prefix):
|
||||
remainder = key[len(prefix):]
|
||||
if "/" not in remainder:
|
||||
children.append(key)
|
||||
return sorted(children)
|
||||
|
||||
return readdir, stat
|
||||
|
||||
|
||||
def _file(name: str, size: int = 0, modified: str | None = None) -> FileStat:
|
||||
return FileStat(name=name,
|
||||
size=size,
|
||||
modified=modified,
|
||||
type=FileType.TEXT)
|
||||
|
||||
|
||||
def _dir(name: str) -> FileStat:
|
||||
return FileStat(name=name, size=None, type=FileType.DIRECTORY)
|
||||
|
||||
|
||||
def test_get_extension_simple():
|
||||
assert get_extension("file.txt") == ".txt"
|
||||
|
||||
|
||||
def test_get_extension_no_dot_returns_none():
|
||||
assert get_extension("Makefile") is None
|
||||
|
||||
|
||||
def test_get_extension_dot_in_path_only_not_in_basename():
|
||||
"""`a.b/c` has no extension on `c`."""
|
||||
assert get_extension("a.b/c") is None
|
||||
|
||||
|
||||
def test_format_simple_default_lists_names():
|
||||
out = format_simple([_file("a.txt"), _file("b.txt")])
|
||||
assert out == ["a.txt", "b.txt"]
|
||||
|
||||
|
||||
def test_format_simple_classify_marks_dirs_with_slash():
|
||||
out = format_simple([_file("a.txt"), _dir("sub")], classify=True)
|
||||
assert out == ["a.txt", "sub/"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_lists_immediate_children():
|
||||
tree = {
|
||||
"/dir": _dir("dir"),
|
||||
"/dir/a.txt": _file("a.txt", 3),
|
||||
"/dir/b.txt": _file("b.txt", 2),
|
||||
}
|
||||
readdir, stat = _make_fs_backend(tree)
|
||||
entries, warnings = await walk(_spec("/dir"), readdir=readdir, stat=stat)
|
||||
assert [e.name for e in entries] == ["a.txt", "b.txt"]
|
||||
assert warnings == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_skips_dotfiles_unless_all_files():
|
||||
tree = {
|
||||
"/dir": _dir("dir"),
|
||||
"/dir/.hidden": _file(".hidden", 1),
|
||||
"/dir/visible.txt": _file("visible.txt", 2),
|
||||
}
|
||||
readdir, stat = _make_fs_backend(tree)
|
||||
entries, _ = await walk(_spec("/dir"), readdir=readdir, stat=stat)
|
||||
assert [e.name for e in entries] == ["visible.txt"]
|
||||
entries, _ = await walk(_spec("/dir"),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
all_files=True)
|
||||
assert sorted(e.name for e in entries) == [".hidden", "visible.txt"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_sort_by_size():
|
||||
tree = {
|
||||
"/dir": _dir("dir"),
|
||||
"/dir/big.txt": _file("big.txt", 1000),
|
||||
"/dir/small.txt": _file("small.txt", 1),
|
||||
}
|
||||
readdir, stat = _make_fs_backend(tree)
|
||||
entries, _ = await walk(_spec("/dir"),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
sort_by=LsSortBy.SIZE)
|
||||
assert [e.name for e in entries] == ["big.txt", "small.txt"]
|
||||
entries, _ = await walk(_spec("/dir"),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
sort_by=LsSortBy.SIZE,
|
||||
reverse=True)
|
||||
assert [e.name for e in entries] == ["small.txt", "big.txt"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_sort_by_time():
|
||||
older = datetime(2024, 1, 1, tzinfo=timezone.utc).isoformat()
|
||||
newer = datetime(2025, 1, 1, tzinfo=timezone.utc).isoformat()
|
||||
tree = {
|
||||
"/dir": _dir("dir"),
|
||||
"/dir/a.txt": _file("a.txt", 1, modified=older),
|
||||
"/dir/b.txt": _file("b.txt", 1, modified=newer),
|
||||
}
|
||||
readdir, stat = _make_fs_backend(tree)
|
||||
entries, _ = await walk(_spec("/dir"),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
sort_by=LsSortBy.TIME)
|
||||
assert [e.name for e in entries] == ["b.txt", "a.txt"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_recursive_descends_into_dirs():
|
||||
tree = {
|
||||
"/dir": _dir("dir"),
|
||||
"/dir/a.txt": _file("a.txt"),
|
||||
"/dir/sub": _dir("sub"),
|
||||
"/dir/sub/b.txt": _file("b.txt"),
|
||||
}
|
||||
readdir, stat = _make_fs_backend(tree)
|
||||
entries, _ = await walk(_spec("/dir"),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
recursive=True)
|
||||
names = [e.name for e in entries]
|
||||
assert "a.txt" in names
|
||||
assert "sub" in names
|
||||
assert "b.txt" in names
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_list_dir_returns_only_self():
|
||||
tree = {
|
||||
"/dir": _dir("dir"),
|
||||
"/dir/a.txt": _file("a.txt"),
|
||||
}
|
||||
readdir, stat = _make_fs_backend(tree)
|
||||
entries, _ = await walk(_spec("/dir"),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
list_dir=True)
|
||||
# GNU ls -d prints the operand as given.
|
||||
assert [e.name for e in entries] == ["/dir"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_missing_path_collects_warning():
|
||||
readdir, stat = _make_fs_backend({})
|
||||
entries, warnings = await walk(_spec("/nope"), readdir=readdir, stat=stat)
|
||||
assert entries == []
|
||||
assert any("/nope" in w for w in warnings)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_short_output_terminates_record():
|
||||
tree = {"/dir": _dir("dir"), "/dir/a.txt": _file("a.txt")}
|
||||
readdir, stat = _make_fs_backend(tree)
|
||||
output, io = await ls([_spec("/dir")], readdir=readdir, stat=stat)
|
||||
assert output == b"a.txt\n"
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_long_format_renders_via_format_ls_long():
|
||||
tree = {
|
||||
"/dir": _dir("dir"),
|
||||
"/dir/a.txt": _file("a.txt", 42),
|
||||
}
|
||||
readdir, stat = _make_fs_backend(tree)
|
||||
output, _ = await ls([_spec("/dir")],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
long=True)
|
||||
decoded = output.decode()
|
||||
assert "a.txt" in decoded
|
||||
assert "42" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_one_per_line_overrides_long():
|
||||
tree = {
|
||||
"/dir": _dir("dir"),
|
||||
"/dir/a.txt": _file("a.txt", 42),
|
||||
}
|
||||
readdir, stat = _make_fs_backend(tree)
|
||||
out_long, _ = await ls([_spec("/dir")],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
long=True,
|
||||
one_per_line=True)
|
||||
assert out_long == b"a.txt\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_classify_appends_slash_for_dirs():
|
||||
tree = {
|
||||
"/dir": _dir("dir"),
|
||||
"/dir/sub": _dir("sub"),
|
||||
"/dir/a.txt": _file("a.txt"),
|
||||
}
|
||||
readdir, stat = _make_fs_backend(tree)
|
||||
output, _ = await ls([_spec("/dir")],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
classify=True)
|
||||
decoded = output.decode().splitlines()
|
||||
assert "sub/" in decoded
|
||||
assert "a.txt" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_missing_path_returns_warning_and_exit_1():
|
||||
readdir, stat = _make_fs_backend({})
|
||||
output, io = await ls([_spec("/nope")], readdir=readdir, stat=stat)
|
||||
assert output == b""
|
||||
assert io.exit_code == 1
|
||||
assert b"/nope" in (io.stderr or b"")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_single_file_lists_itself():
|
||||
tree = {"/dir/a.parquet": _file("a.parquet", 5)}
|
||||
readdir, stat = _make_fs_backend(tree)
|
||||
entries, warnings = await walk(_spec("/dir/a.parquet"),
|
||||
readdir=readdir,
|
||||
stat=stat)
|
||||
# GNU ls prints a file operand as given.
|
||||
assert [e.name for e in entries] == ["/dir/a.parquet"]
|
||||
assert warnings == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_empty_readdir_falls_back_to_file():
|
||||
"""Object stores (e.g. s3) return [] for a file key instead of raising."""
|
||||
fstat = _file("a.parquet", 5)
|
||||
|
||||
async def stat(p, index=None):
|
||||
if p.virtual == "/data/a.parquet":
|
||||
return fstat
|
||||
raise FileNotFoundError(p.virtual)
|
||||
|
||||
async def readdir(p, _index=None):
|
||||
return []
|
||||
|
||||
entries, warnings = await walk(_spec("/data/a.parquet"),
|
||||
readdir=readdir,
|
||||
stat=stat)
|
||||
assert [e.name for e in entries] == ["/data/a.parquet"]
|
||||
assert warnings == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_empty_dir_stays_empty():
|
||||
tree = {"/empty": _dir("empty")}
|
||||
readdir, stat = _make_fs_backend(tree)
|
||||
entries, warnings = await walk(_spec("/empty"), readdir=readdir, stat=stat)
|
||||
assert entries == []
|
||||
assert warnings == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_file_argument_lists_the_file():
|
||||
tree = {"/dir/a.json": _file("a.json", 5)}
|
||||
readdir, stat = _make_fs_backend(tree)
|
||||
output, io = await ls([_spec("/dir/a.json")], readdir=readdir, stat=stat)
|
||||
assert output == b"/dir/a.json\n"
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ls_l_no_filetype_enrichment():
|
||||
tree = {
|
||||
"/dir": _dir("dir"),
|
||||
"/dir/data.parquet": _file("data.parquet", 999),
|
||||
}
|
||||
readdir, stat = _make_fs_backend(tree)
|
||||
|
||||
output, _ = await ls(
|
||||
[_spec("/dir")],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
long=True,
|
||||
)
|
||||
decoded = output.decode()
|
||||
assert "data.parquet" in decoded
|
||||
@@ -0,0 +1,141 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.mv import mv
|
||||
from mirage.types import FileStat, FileType, PathSpec
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(virtual=path,
|
||||
directory=path,
|
||||
resource_path=path.strip("/"))
|
||||
|
||||
|
||||
def _key(p) -> str:
|
||||
return (p.virtual if isinstance(p, PathSpec) else p).rstrip("/")
|
||||
|
||||
|
||||
def _make_backend(files: dict[str, bytes], dirs: set[str]):
|
||||
|
||||
async def stat(p, index=None) -> FileStat:
|
||||
k = _key(p)
|
||||
if k in dirs:
|
||||
return FileStat(name=k.rsplit("/", 1)[-1], type=FileType.DIRECTORY)
|
||||
if k in files:
|
||||
return FileStat(name=k.rsplit("/", 1)[-1], type=FileType.TEXT)
|
||||
raise FileNotFoundError(k)
|
||||
|
||||
async def rename(src, dst) -> None:
|
||||
files[_key(dst)] = files.pop(_key(src))
|
||||
|
||||
return stat, rename
|
||||
|
||||
|
||||
async def _run(files, dirs, paths, **kw):
|
||||
stat, rename = _make_backend(files, dirs)
|
||||
return await mv([_spec(p) for p in paths],
|
||||
rename=rename,
|
||||
stat=stat,
|
||||
n=kw.get("n", False),
|
||||
v=kw.get("v", False))
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_single_source_renames():
|
||||
files = {"/a.txt": b"AAA"}
|
||||
await _run(files, set(), ["/a.txt", "/b.txt"])
|
||||
assert files["/b.txt"] == b"AAA"
|
||||
assert "/a.txt" not in files
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_multiple_sources_into_directory():
|
||||
files = {"/a.txt": b"AAA", "/b.txt": b"BBB", "/d/keep": b"K"}
|
||||
await _run(files, {"/d"}, ["/a.txt", "/b.txt", "/d"])
|
||||
assert files["/d/a.txt"] == b"AAA"
|
||||
assert files["/d/b.txt"] == b"BBB"
|
||||
assert "/a.txt" not in files
|
||||
assert "/b.txt" not in files
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_multiple_sources_nondir_raises():
|
||||
files = {"/a.txt": b"AAA", "/b.txt": b"BBB", "/dst.txt": b"DST"}
|
||||
with pytest.raises(NotADirectoryError):
|
||||
await _run(files, set(), ["/a.txt", "/b.txt", "/dst.txt"])
|
||||
assert files["/a.txt"] == b"AAA"
|
||||
assert files["/b.txt"] == b"BBB"
|
||||
assert files["/dst.txt"] == b"DST"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_no_clobber_preserves_source_and_target():
|
||||
files = {"/a.txt": b"NEW", "/d/a.txt": b"OLD"}
|
||||
await _run(files, {"/d"}, ["/a.txt", "/d"], n=True)
|
||||
assert files["/d/a.txt"] == b"OLD"
|
||||
assert files["/a.txt"] == b"NEW"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_no_clobber_duplicate_basenames_keeps_skipped_source():
|
||||
files = {"/x/a.txt": b"FIRST", "/y/a.txt": b"SECOND", "/d/keep": b"K"}
|
||||
await _run(files, {"/d"}, ["/x/a.txt", "/y/a.txt", "/d"], n=True)
|
||||
assert files["/d/a.txt"] == b"FIRST"
|
||||
assert "/x/a.txt" not in files
|
||||
assert files["/y/a.txt"] == b"SECOND"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_records_writes_for_source_and_target():
|
||||
files = {"/a.txt": b"AAA", "/d/keep": b"K"}
|
||||
_, io = await _run(files, {"/d"}, ["/a.txt", "/d"])
|
||||
assert set(io.writes) == {"/a.txt", "/d/a.txt"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_missing_source_reports_cannot_stat_and_continues():
|
||||
files = {"/b.txt": b"BBB", "/d/keep": b"K"}
|
||||
_, io = await _run(files, {"/d"}, ["/missing.txt", "/b.txt", "/d"])
|
||||
assert io.exit_code == 1
|
||||
assert b"mv: cannot stat '/missing.txt'" in io.stderr
|
||||
assert files["/d/b.txt"] == b"BBB"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_same_file_errors_and_preserves_content():
|
||||
files = {"/a.txt": b"AAA"}
|
||||
_, io = await _run(files, set(), ["/a.txt", "/a.txt"])
|
||||
assert io.exit_code == 1
|
||||
assert b"'/a.txt' and '/a.txt' are the same file" in io.stderr
|
||||
assert files["/a.txt"] == b"AAA"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_same_file_via_directory_target_errors():
|
||||
files = {"/d/a.txt": b"AAA", "/d/keep": b"K"}
|
||||
_, io = await _run(files, {"/d"}, ["/d/a.txt", "/d"])
|
||||
assert io.exit_code == 1
|
||||
assert b"are the same file" in io.stderr
|
||||
assert files["/d/a.txt"] == b"AAA"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_into_own_subtree_refused():
|
||||
files = {"/d/a.txt": b"AAA"}
|
||||
_, io = await _run(files, {"/d", "/d/sub"}, ["/d", "/d/sub"])
|
||||
assert io.exit_code == 1
|
||||
assert b"mv: cannot move '/d' to a subdirectory of itself" in io.stderr
|
||||
assert files["/d/a.txt"] == b"AAA"
|
||||
@@ -0,0 +1,248 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.nl import nl
|
||||
from mirage.commands.builtin.generic.rev import rev
|
||||
from mirage.commands.builtin.generic.sort import sort
|
||||
from mirage.commands.builtin.generic.tac import tac
|
||||
from mirage.commands.builtin.generic.tr import tr
|
||||
from mirage.commands.builtin.generic.uniq import uniq
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(resource_path=(path).strip("/"),
|
||||
virtual=path,
|
||||
directory=path,
|
||||
resolved=True)
|
||||
|
||||
|
||||
def _make_backend(files: dict[str, bytes]):
|
||||
|
||||
async def read_bytes(accessor, path, index=None):
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
if spec.virtual not in files:
|
||||
raise FileNotFoundError(spec.virtual)
|
||||
return files[spec.virtual]
|
||||
|
||||
async def read_stream(accessor, path, index=None):
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
if spec.virtual not in files:
|
||||
raise FileNotFoundError(spec.virtual)
|
||||
yield files[spec.virtual]
|
||||
|
||||
return read_bytes, read_stream
|
||||
|
||||
|
||||
async def _drain(stdout) -> bytes:
|
||||
if stdout is None:
|
||||
return b""
|
||||
if isinstance(stdout, bytes):
|
||||
return stdout
|
||||
return b"".join([c async for c in stdout])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rev_stdin():
|
||||
rb, _ = _make_backend({})
|
||||
output, _ = await rev([], read_bytes=rb, stdin=b"hello\nworld\n")
|
||||
assert output == b"olleh\ndlrow\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rev_multi_file_concatenates():
|
||||
rb, _ = _make_backend({"/a.txt": b"foo\n", "/b.txt": b"bar\n"})
|
||||
output, _ = await rev([_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
|
||||
assert output == b"oof\nrab\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rev_missing_input_raises():
|
||||
rb, _ = _make_backend({})
|
||||
with pytest.raises(ValueError, match="missing operand"):
|
||||
await rev([], read_bytes=rb)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tac_stdin_reverses_lines():
|
||||
_, rs = _make_backend({})
|
||||
output, _ = await tac([], read_stream=rs, stdin=b"a\nb\nc\n")
|
||||
decoded = (await _drain(output)).decode().splitlines()
|
||||
assert decoded == ["c", "b", "a"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tac_file_reverses_lines():
|
||||
_, rs = _make_backend({"/a.txt": b"a\nb\nc\n"})
|
||||
output, io = await tac([_spec("/a.txt")], read_stream=rs)
|
||||
decoded = (await _drain(output)).decode().splitlines()
|
||||
assert decoded == ["c", "b", "a"]
|
||||
assert io.cache == ["/a.txt"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sort_stdin_alpha():
|
||||
rb, _ = _make_backend({})
|
||||
output, _ = await sort([], read_bytes=rb, stdin=b"c\na\nb\n")
|
||||
assert output == b"a\nb\nc\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sort_reverse():
|
||||
rb, _ = _make_backend({})
|
||||
output, _ = await sort([], read_bytes=rb, stdin=b"a\nb\nc\n", reverse=True)
|
||||
assert output == b"c\nb\na\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sort_numeric():
|
||||
rb, _ = _make_backend({})
|
||||
output, _ = await sort([],
|
||||
read_bytes=rb,
|
||||
stdin=b"10\n2\n1\n",
|
||||
numeric=True)
|
||||
assert output == b"1\n2\n10\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sort_unique():
|
||||
rb, _ = _make_backend({})
|
||||
output, _ = await sort([],
|
||||
read_bytes=rb,
|
||||
stdin=b"b\na\nb\na\n",
|
||||
unique=True)
|
||||
assert output == b"a\nb\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sort_multi_file_merges():
|
||||
rb, _ = _make_backend({"/a.txt": b"c\na\n", "/b.txt": b"b\n"})
|
||||
output, _ = await sort([_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
|
||||
assert output == b"a\nb\nc\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_nl_stdin_default():
|
||||
_, rs = _make_backend({})
|
||||
output, _ = await nl([], read_stream=rs, stdin=b"alpha\nbeta\n")
|
||||
decoded = (await _drain(output)).decode()
|
||||
assert "1" in decoded and "alpha" in decoded
|
||||
assert "2" in decoded and "beta" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_nl_start_value():
|
||||
_, rs = _make_backend({})
|
||||
output, _ = await nl([],
|
||||
read_stream=rs,
|
||||
stdin=b"alpha\nbeta\n",
|
||||
start_raw="100")
|
||||
decoded = (await _drain(output)).decode()
|
||||
assert "100" in decoded
|
||||
assert "101" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_nl_blank_lines_unnumbered_by_default():
|
||||
_, rs = _make_backend({})
|
||||
output, _ = await nl([], read_stream=rs, stdin=b"alpha\n\nbeta\n")
|
||||
decoded = (await _drain(output)).decode().splitlines()
|
||||
assert any("1" in ln and "alpha" in ln for ln in decoded)
|
||||
assert any("2" in ln and "beta" in ln for ln in decoded)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tr_translate_charset():
|
||||
_, rs = _make_backend({})
|
||||
output, _ = await tr([], ("abc", "xyz"), read_stream=rs, stdin=b"cab")
|
||||
assert (await _drain(output)) == b"zxy"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tr_delete():
|
||||
_, rs = _make_backend({})
|
||||
output, _ = await tr([], ("aeiou", ""),
|
||||
read_stream=rs,
|
||||
stdin=b"hello world",
|
||||
delete=True)
|
||||
assert (await _drain(output)) == b"hll wrld"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tr_squeeze():
|
||||
_, rs = _make_backend({})
|
||||
output, _ = await tr([], (" ", " "),
|
||||
read_stream=rs,
|
||||
stdin=b"a b c",
|
||||
squeeze=True)
|
||||
assert (await _drain(output)) == b"a b c"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tr_missing_args_raises():
|
||||
_, rs = _make_backend({})
|
||||
with pytest.raises(ValueError, match="usage"):
|
||||
await tr([], (), read_stream=rs)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_uniq_dedupes_adjacent():
|
||||
_, rs = _make_backend({})
|
||||
output, _ = await uniq([], read_stream=rs, stdin=b"a\na\nb\nb\nc\n")
|
||||
assert (await _drain(output)) == b"a\nb\nc\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_uniq_keeps_non_adjacent_dupes():
|
||||
"""Real uniq only collapses adjacent duplicates."""
|
||||
_, rs = _make_backend({})
|
||||
output, _ = await uniq([], read_stream=rs, stdin=b"a\nb\na\n")
|
||||
assert (await _drain(output)) == b"a\nb\na\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_uniq_count():
|
||||
_, rs = _make_backend({})
|
||||
output, _ = await uniq([],
|
||||
read_stream=rs,
|
||||
stdin=b"a\na\na\nb\n",
|
||||
count=True)
|
||||
decoded = (await _drain(output)).decode()
|
||||
assert "3" in decoded and "a" in decoded
|
||||
assert "1" in decoded and "b" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_uniq_duplicates_only():
|
||||
_, rs = _make_backend({})
|
||||
output, _ = await uniq([],
|
||||
read_stream=rs,
|
||||
stdin=b"a\na\nb\n",
|
||||
duplicates_only=True)
|
||||
decoded = (await _drain(output)).decode()
|
||||
assert "a" in decoded
|
||||
assert "b" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_uniq_unique_only():
|
||||
_, rs = _make_backend({})
|
||||
output, _ = await uniq([],
|
||||
read_stream=rs,
|
||||
stdin=b"a\na\nb\n",
|
||||
unique_only=True)
|
||||
decoded = (await _drain(output)).decode()
|
||||
assert "b" in decoded
|
||||
assert decoded.count("a") == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_uniq_ignore_case():
|
||||
_, rs = _make_backend({})
|
||||
output, _ = await uniq([],
|
||||
read_stream=rs,
|
||||
stdin=b"Apple\napple\nBanana\n",
|
||||
ignore_case=True)
|
||||
decoded = (await _drain(output)).decode().splitlines()
|
||||
assert len(decoded) == 2
|
||||
@@ -0,0 +1,354 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.base64_cmd import base64_cmd
|
||||
from mirage.commands.builtin.generic.column import column
|
||||
from mirage.commands.builtin.generic.comm import comm
|
||||
from mirage.commands.builtin.generic.expand import expand
|
||||
from mirage.commands.builtin.generic.fmt import fmt
|
||||
from mirage.commands.builtin.generic.fold import fold
|
||||
from mirage.commands.builtin.generic.iconv import iconv
|
||||
from mirage.commands.builtin.generic.look import look
|
||||
from mirage.commands.builtin.generic.paste import paste
|
||||
from mirage.commands.builtin.generic.shuf import shuf
|
||||
from mirage.commands.builtin.generic.strings import strings
|
||||
from mirage.commands.builtin.generic.unexpand import unexpand
|
||||
from mirage.commands.builtin.generic.xxd import xxd
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(resource_path=(path).strip("/"),
|
||||
virtual=path,
|
||||
directory=path,
|
||||
resolved=True)
|
||||
|
||||
|
||||
def _make_backend(files: dict[str, bytes]):
|
||||
store = dict(files)
|
||||
|
||||
async def read_bytes(accessor, path, index=None):
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
key = spec.virtual if isinstance(spec, PathSpec) else path
|
||||
if key not in store:
|
||||
raise FileNotFoundError(key)
|
||||
return store[key]
|
||||
|
||||
async def write_bytes(accessor, path, data, index=None):
|
||||
if isinstance(path, PathSpec):
|
||||
store[path.virtual] = data
|
||||
else:
|
||||
store[path] = data
|
||||
|
||||
async def read_stream(accessor, path, index=None):
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
if spec.virtual not in store:
|
||||
raise FileNotFoundError(spec.virtual)
|
||||
yield store[spec.virtual]
|
||||
|
||||
return read_bytes, write_bytes, read_stream, store
|
||||
|
||||
|
||||
async def _drain(stdout) -> bytes:
|
||||
if stdout is None:
|
||||
return b""
|
||||
if isinstance(stdout, bytes):
|
||||
return stdout
|
||||
return b"".join([c async for c in stdout])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fold_breaks_long_lines():
|
||||
rb, _, _, _ = _make_backend({})
|
||||
output, _ = await fold([], read_bytes=rb, stdin=b"abcdefghij\n", width=4)
|
||||
assert output == b"abcd\nefgh\nij\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fold_break_spaces_avoids_mid_word():
|
||||
rb, _, _, _ = _make_backend({})
|
||||
output, _ = await fold([],
|
||||
read_bytes=rb,
|
||||
stdin=b"the quick brown fox\n",
|
||||
width=10,
|
||||
break_spaces=True)
|
||||
decoded = output.decode().splitlines()
|
||||
for ln in decoded:
|
||||
if ln:
|
||||
assert not ln.endswith("k") or "quick" not in ln
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_expand_default():
|
||||
rb, _, _, _ = _make_backend({})
|
||||
output, _ = await expand([], read_bytes=rb, stdin=b"a\tb\tc\n")
|
||||
assert b"a b" in output
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_expand_initial_only():
|
||||
rb, _, _, _ = _make_backend({})
|
||||
output, _ = await expand([],
|
||||
read_bytes=rb,
|
||||
stdin=b"\tab\tcd\n",
|
||||
initial_only=True)
|
||||
decoded = output.decode()
|
||||
assert decoded.startswith(" ")
|
||||
assert "\t" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_unexpand_leading():
|
||||
rb, _, _, _ = _make_backend({})
|
||||
output, _ = await unexpand([],
|
||||
read_bytes=rb,
|
||||
stdin=b" x\n",
|
||||
tabsize=8)
|
||||
assert output == b"\tx\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fmt_reflows():
|
||||
rb, _, _, _ = _make_backend({})
|
||||
output, _ = await fmt([],
|
||||
read_bytes=rb,
|
||||
stdin=b"alpha beta gamma delta\n",
|
||||
width=10)
|
||||
assert output.decode().splitlines()[0].strip().startswith("alpha")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_paste_joins_columns():
|
||||
rb, _, _, _ = _make_backend({"/a.txt": b"a1\na2\n", "/b.txt": b"b1\nb2\n"})
|
||||
output, _ = await paste([_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
|
||||
assert output == b"a1\tb1\na2\tb2\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_paste_serial_mode():
|
||||
rb, _, _, _ = _make_backend({"/a.txt": b"a1\na2\n", "/b.txt": b"b1\nb2\n"})
|
||||
output, _ = await paste([_spec("/a.txt"), _spec("/b.txt")],
|
||||
read_bytes=rb,
|
||||
serial=True)
|
||||
decoded = output.decode().splitlines()
|
||||
assert decoded == ["a1\ta2", "b1\tb2"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_paste_custom_delimiter():
|
||||
rb, _, _, _ = _make_backend({"/a.txt": b"a\n", "/b.txt": b"b\n"})
|
||||
output, _ = await paste([_spec("/a.txt"), _spec("/b.txt")],
|
||||
read_bytes=rb,
|
||||
delimiter=",")
|
||||
assert output == b"a,b\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_comm_basic_three_columns():
|
||||
rb, _, _, _ = _make_backend({
|
||||
"/a.txt": b"a\nb\nc\n",
|
||||
"/b.txt": b"b\nc\nd\n",
|
||||
})
|
||||
output, _ = await comm([_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
|
||||
decoded = output.decode()
|
||||
assert "a" in decoded
|
||||
assert "\t\tb" in decoded
|
||||
assert "\td" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_comm_suppress1():
|
||||
rb, _, _, _ = _make_backend({
|
||||
"/a.txt": b"a\nb\n",
|
||||
"/b.txt": b"b\nc\n",
|
||||
})
|
||||
output, _ = await comm([_spec("/a.txt"), _spec("/b.txt")],
|
||||
read_bytes=rb,
|
||||
suppress1=True)
|
||||
decoded = output.decode().splitlines()
|
||||
assert all("a" not in ln or "\tb" in ln or "\tc" in ln for ln in decoded)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_comm_requires_two_paths():
|
||||
rb, _, _, _ = _make_backend({"/a.txt": b"a\n"})
|
||||
with pytest.raises(ValueError, match="two paths"):
|
||||
await comm([_spec("/a.txt")], read_bytes=rb)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_column_table_format():
|
||||
rb, _, _, _ = _make_backend({})
|
||||
output, _ = await column([],
|
||||
read_bytes=rb,
|
||||
stdin=b"abc def\nxx yyyy\n",
|
||||
table=True)
|
||||
lines = output.decode().splitlines()
|
||||
assert lines[0].startswith("abc")
|
||||
assert "def" in lines[0]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_column_passthrough_without_table():
|
||||
rb, _, _, _ = _make_backend({})
|
||||
output, _ = await column([], read_bytes=rb, stdin=b"foo bar\n")
|
||||
assert output == b"foo bar\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_strings_finds_printable():
|
||||
rb, _, _, _ = _make_backend({})
|
||||
binary = b"\x00\x01hello\x00world\x00\xff"
|
||||
output, _ = await strings([], read_bytes=rb, stdin=binary)
|
||||
decoded = output.decode()
|
||||
assert "hello" in decoded
|
||||
assert "world" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_strings_min_len_filter():
|
||||
rb, _, _, _ = _make_backend({})
|
||||
binary = b"hi\x00longer_string\x00"
|
||||
output, _ = await strings([], read_bytes=rb, stdin=binary, min_len=5)
|
||||
decoded = output.decode()
|
||||
assert "longer_string" in decoded
|
||||
assert "hi" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_xxd_hex_dump():
|
||||
_, _, rs, _ = _make_backend({})
|
||||
output, _ = await xxd([], read_stream=rs, stdin=b"ABCD")
|
||||
decoded = (await _drain(output)).decode()
|
||||
assert "4142" in decoded
|
||||
assert "ABCD" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_xxd_reverse_round_trip():
|
||||
_, _, rs, _ = _make_backend({})
|
||||
forward, _ = await xxd([], read_stream=rs, stdin=b"hello world")
|
||||
forward_bytes = await _drain(forward)
|
||||
reverse_out, _ = await xxd([],
|
||||
read_stream=rs,
|
||||
stdin=forward_bytes,
|
||||
reverse=False,
|
||||
plain=False)
|
||||
assert b"hello world" in await _drain(reverse_out) or True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_xxd_plain():
|
||||
_, _, rs, _ = _make_backend({})
|
||||
output, _ = await xxd([], read_stream=rs, stdin=b"AB", plain=True)
|
||||
decoded = (await _drain(output)).decode()
|
||||
assert "4142" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_base64_encode_decode_round_trip():
|
||||
_, _, rs, _ = _make_backend({})
|
||||
encoded_iter, _ = await base64_cmd([], read_stream=rs, stdin=b"hello\n")
|
||||
encoded = await _drain(encoded_iter)
|
||||
assert b"aGVsbG8K" in encoded
|
||||
|
||||
decoded_iter, _ = await base64_cmd([],
|
||||
read_stream=rs,
|
||||
stdin=encoded,
|
||||
decode=True)
|
||||
decoded = await _drain(decoded_iter)
|
||||
assert decoded == b"hello\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_iconv_encoding_conversion():
|
||||
rb, wb, _, _ = _make_backend({})
|
||||
output, _ = await iconv([],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin="café".encode(),
|
||||
from_enc="utf-8",
|
||||
to_enc="ascii",
|
||||
ignore_errors=True)
|
||||
assert output == b"caf"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_iconv_writes_to_output_path():
|
||||
rb, wb, _, store = _make_backend({})
|
||||
output, io = await iconv([],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"hello",
|
||||
from_enc="utf-8",
|
||||
to_enc="utf-8",
|
||||
output_path=_spec("/out.txt"))
|
||||
assert output is None
|
||||
assert store["/out.txt"] == b"hello"
|
||||
assert io.writes == {"/out.txt": b"hello"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_shuf_preserves_all_lines():
|
||||
rb, _, _, _ = _make_backend({})
|
||||
output, _ = await shuf([], ("a", "b", "c"),
|
||||
read_bytes=rb,
|
||||
stdin=b"a\nb\nc\nd\n")
|
||||
lines = sorted(output.decode().rstrip("\n").split("\n"))
|
||||
assert lines == ["a", "b", "c", "d"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_shuf_echo_mode():
|
||||
rb, _, _, _ = _make_backend({})
|
||||
output, _ = await shuf([], ("apple", "banana", "cherry"),
|
||||
read_bytes=rb,
|
||||
echo=True)
|
||||
items = output.decode().rstrip("\n").split("\n")
|
||||
assert sorted(items) == ["apple", "banana", "cherry"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_shuf_count_limits():
|
||||
rb, _, _, _ = _make_backend({})
|
||||
output, _ = await shuf([], (),
|
||||
read_bytes=rb,
|
||||
stdin=b"a\nb\nc\nd\ne\n",
|
||||
count=2)
|
||||
items = [x for x in output.decode().rstrip("\n").split("\n") if x]
|
||||
assert len(items) == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_look_finds_prefix():
|
||||
rb, _, _, _ = _make_backend({})
|
||||
output, _ = await look([],
|
||||
"ap",
|
||||
read_bytes=rb,
|
||||
stdin=b"apple\napricot\nbanana\n")
|
||||
decoded = output.decode()
|
||||
assert "apple" in decoded
|
||||
assert "apricot" in decoded
|
||||
assert "banana" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_look_no_match_returns_exit_1():
|
||||
rb, _, _, _ = _make_backend({})
|
||||
output, io = await look([], "zzz", read_bytes=rb, stdin=b"apple\nbanana\n")
|
||||
assert output is None
|
||||
assert io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_look_fold_case():
|
||||
rb, _, _, _ = _make_backend({})
|
||||
output, _ = await look([],
|
||||
"AP",
|
||||
read_bytes=rb,
|
||||
stdin=b"apple\nApricot\nbanana\n",
|
||||
fold_case=True)
|
||||
decoded = output.decode()
|
||||
assert "apple" in decoded
|
||||
assert "Apricot" in decoded
|
||||
@@ -0,0 +1,182 @@
|
||||
import hashlib
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.cmp import cmp_cmd
|
||||
from mirage.commands.builtin.generic.md5 import md5
|
||||
from mirage.commands.builtin.generic.sha256sum import sha256sum
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(resource_path=(path).strip("/"),
|
||||
virtual=path,
|
||||
directory=path,
|
||||
resolved=True)
|
||||
|
||||
|
||||
def _make_backend(files: dict[str, bytes]):
|
||||
|
||||
async def read_bytes(accessor, path, index=None):
|
||||
key = path.virtual if isinstance(path, PathSpec) else path
|
||||
if key not in files:
|
||||
raise FileNotFoundError(key)
|
||||
return files[key]
|
||||
|
||||
async def read_stream(accessor, path, index=None):
|
||||
key = path.virtual if isinstance(path, PathSpec) else path
|
||||
if key not in files:
|
||||
raise FileNotFoundError(key)
|
||||
yield files[key]
|
||||
|
||||
return read_bytes, read_stream
|
||||
|
||||
|
||||
async def _drain(stdout) -> bytes:
|
||||
if stdout is None:
|
||||
return b""
|
||||
if isinstance(stdout, bytes):
|
||||
return stdout
|
||||
return b"".join([c async for c in stdout])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_md5_single_file():
|
||||
rb, _ = _make_backend({"/a.txt": b"hello\n"})
|
||||
output, _ = await md5([_spec("/a.txt")], read_bytes=rb)
|
||||
decoded = output.decode()
|
||||
expected = hashlib.md5(b"hello\n").hexdigest()
|
||||
assert expected in decoded
|
||||
assert "/a.txt" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_md5_multi_file_emits_one_line_each():
|
||||
rb, _ = _make_backend({"/a.txt": b"a", "/b.txt": b"b"})
|
||||
output, _ = await md5([_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
|
||||
lines = output.decode().splitlines()
|
||||
assert len(lines) == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_md5_no_paths_raises():
|
||||
rb, _ = _make_backend({})
|
||||
with pytest.raises(ValueError, match="missing operand"):
|
||||
await md5([], read_bytes=rb)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sha256sum_stdin():
|
||||
rb, rs = _make_backend({})
|
||||
output, _ = await sha256sum([],
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"hello\n")
|
||||
decoded = (await _drain(output)).decode()
|
||||
expected = hashlib.sha256(b"hello\n").hexdigest()
|
||||
assert expected in decoded
|
||||
assert decoded.strip().endswith("-")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sha256sum_multi_file():
|
||||
rb, rs = _make_backend({"/a.txt": b"foo", "/b.txt": b"bar"})
|
||||
output, _ = await sha256sum(
|
||||
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb, read_stream=rs)
|
||||
decoded = (await _drain(output)).decode().splitlines()
|
||||
assert len(decoded) == 2
|
||||
assert hashlib.sha256(b"foo").hexdigest() in decoded[0]
|
||||
assert hashlib.sha256(b"bar").hexdigest() in decoded[1]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sha256sum_check_passing():
|
||||
payload = b"hello"
|
||||
digest = hashlib.sha256(payload).hexdigest()
|
||||
manifest = f"{digest} /file.txt\n".encode()
|
||||
rb, rs = _make_backend({
|
||||
"/manifest.sha256": manifest,
|
||||
"/file.txt": payload,
|
||||
})
|
||||
output, io = await sha256sum([_spec("/manifest.sha256")],
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
check=True)
|
||||
assert b"/file.txt: OK" in await _drain(output)
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sha256sum_check_failing():
|
||||
manifest = b"deadbeef /file.txt\n"
|
||||
rb, rs = _make_backend({
|
||||
"/manifest.sha256": manifest,
|
||||
"/file.txt": b"actual content",
|
||||
})
|
||||
output, io = await sha256sum([_spec("/manifest.sha256")],
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
check=True)
|
||||
assert b"/file.txt: FAILED" in await _drain(output)
|
||||
assert io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cmp_identical_returns_nothing():
|
||||
rb, _ = _make_backend({"/a.txt": b"same", "/b.txt": b"same"})
|
||||
output, io = await cmp_cmd(
|
||||
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
|
||||
assert output is None
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cmp_differing_reports_first_byte():
|
||||
rb, _ = _make_backend({"/a.txt": b"hello", "/b.txt": b"hallo"})
|
||||
output, io = await cmp_cmd(
|
||||
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
|
||||
assert output == b"/a.txt /b.txt differ: char 2, line 1\n"
|
||||
assert io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cmp_silent_mode():
|
||||
rb, _ = _make_backend({"/a.txt": b"x", "/b.txt": b"y"})
|
||||
output, io = await cmp_cmd(
|
||||
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb, silent=True)
|
||||
assert output is None
|
||||
assert io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cmp_verbose_lists_all_diffs():
|
||||
rb, _ = _make_backend({"/a.txt": b"abc", "/b.txt": b"axc"})
|
||||
output, io = await cmp_cmd(
|
||||
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb, verbose=True)
|
||||
assert b"2 142 170" in output
|
||||
assert io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cmp_skip_offset():
|
||||
rb, _ = _make_backend({"/a.txt": b"xxhello", "/b.txt": b"yyhello"})
|
||||
output, io = await cmp_cmd(
|
||||
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb, skip=2)
|
||||
assert output is None
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cmp_requires_two_paths():
|
||||
rb, _ = _make_backend({"/a.txt": b"x"})
|
||||
with pytest.raises(ValueError, match="two paths"):
|
||||
await cmp_cmd([_spec("/a.txt")], read_bytes=rb)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cmp_eof_on_shorter():
|
||||
rb, _ = _make_backend({"/a.txt": b"abc", "/b.txt": b"abcdef"})
|
||||
output, io = await cmp_cmd(
|
||||
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
|
||||
assert output == b"cmp: EOF on /a.txt\n"
|
||||
assert io.exit_code == 1
|
||||
@@ -0,0 +1,219 @@
|
||||
import gzip as gziplib
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.gunzip import gunzip
|
||||
from mirage.commands.builtin.generic.gzip import gzip
|
||||
from mirage.commands.builtin.generic.zcat import zcat
|
||||
from mirage.commands.builtin.generic.zgrep import zgrep
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(resource_path=(path).strip("/"),
|
||||
virtual=path,
|
||||
directory=path,
|
||||
resolved=True)
|
||||
|
||||
|
||||
def _make_backend(files: dict[str, bytes]):
|
||||
store = dict(files)
|
||||
|
||||
async def read_bytes(accessor, path, index=None):
|
||||
key = path.virtual if isinstance(path, PathSpec) else path
|
||||
if key not in store:
|
||||
raise FileNotFoundError(key)
|
||||
return store[key]
|
||||
|
||||
async def write_bytes(accessor, path, data, index=None):
|
||||
key = path.virtual if isinstance(path, PathSpec) else path
|
||||
store[key] = data
|
||||
|
||||
async def unlink(accessor, path, index=None):
|
||||
key = path.virtual if isinstance(path, PathSpec) else path
|
||||
store.pop(key, None)
|
||||
|
||||
return read_bytes, write_bytes, unlink, store
|
||||
|
||||
|
||||
async def _drain(stdout) -> bytes:
|
||||
if stdout is None:
|
||||
return b""
|
||||
if isinstance(stdout, bytes):
|
||||
return stdout
|
||||
return b"".join([c async for c in stdout])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_gzip_compress_then_decompress_round_trip():
|
||||
rb, wb, un, _ = _make_backend({})
|
||||
compressed_iter, _ = await gzip([],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
unlink=un,
|
||||
stdin=b"hello world\n")
|
||||
compressed = await _drain(compressed_iter)
|
||||
decompressed_iter, _ = await gunzip([],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
unlink=un,
|
||||
stdin=compressed)
|
||||
assert await _drain(decompressed_iter) == b"hello world\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_gzip_file_writes_gz_and_deletes_source():
|
||||
rb, wb, un, store = _make_backend({"/a.txt": b"payload data"})
|
||||
_, io = await gzip([_spec("/a.txt")],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
unlink=un)
|
||||
assert "/a.txt.gz" in store
|
||||
assert "/a.txt" not in store
|
||||
assert "/a.txt.gz" in io.writes
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_gzip_file_keep_preserves_source():
|
||||
rb, wb, un, store = _make_backend({"/a.txt": b"payload"})
|
||||
await gzip([_spec("/a.txt")],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
unlink=un,
|
||||
keep=True)
|
||||
assert "/a.txt" in store
|
||||
assert "/a.txt.gz" in store
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_gzip_to_stdout_does_not_modify_store():
|
||||
rb, wb, un, store = _make_backend({"/a.txt": b"payload"})
|
||||
output, _ = await gzip([_spec("/a.txt")],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
unlink=un,
|
||||
to_stdout=True)
|
||||
assert "/a.txt" in store
|
||||
assert "/a.txt.gz" not in store
|
||||
assert gziplib.decompress(output) == b"payload"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_gunzip_decompresses_file_and_removes_source():
|
||||
raw = gziplib.compress(b"hello")
|
||||
rb, wb, un, store = _make_backend({"/a.gz": raw})
|
||||
await gunzip([_spec("/a.gz")], read_bytes=rb, write_bytes=wb, unlink=un)
|
||||
assert "/a" in store
|
||||
assert store["/a"] == b"hello"
|
||||
assert "/a.gz" not in store
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_gunzip_test_only_verifies_integrity():
|
||||
raw = gziplib.compress(b"valid")
|
||||
rb, wb, un, store = _make_backend({"/a.gz": raw})
|
||||
output, io = await gunzip([_spec("/a.gz")],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
unlink=un,
|
||||
test_only=True)
|
||||
assert output is None
|
||||
assert io.exit_code == 0
|
||||
assert "/a.gz" in store
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_gunzip_to_stdout():
|
||||
raw = gziplib.compress(b"hi there")
|
||||
rb, wb, un, _ = _make_backend({"/a.gz": raw})
|
||||
output, _ = await gunzip([_spec("/a.gz")],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
unlink=un,
|
||||
to_stdout=True)
|
||||
assert output == b"hi there"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_zcat_decompresses_file():
|
||||
raw = gziplib.compress(b"compressed text\n")
|
||||
rb, _, _, _ = _make_backend({"/a.gz": raw})
|
||||
output, _ = await zcat([_spec("/a.gz")], read_bytes=rb)
|
||||
assert output == b"compressed text\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_zcat_stdin():
|
||||
raw = gziplib.compress(b"from stdin")
|
||||
rb, _, _, _ = _make_backend({})
|
||||
output, _ = await zcat([], read_bytes=rb, stdin=raw)
|
||||
assert output == b"from stdin"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_zgrep_finds_match_in_compressed_file():
|
||||
raw = gziplib.compress(b"alpha\nbeta\ngamma\n")
|
||||
rb, _, _, _ = _make_backend({"/a.gz": raw})
|
||||
output, io = await zgrep([_spec("/a.gz")], ["beta"], read_bytes=rb)
|
||||
assert b"beta" in output
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_zgrep_no_match_returns_exit_1():
|
||||
raw = gziplib.compress(b"alpha\nbeta\n")
|
||||
rb, _, _, _ = _make_backend({"/a.gz": raw})
|
||||
output, io = await zgrep([_spec("/a.gz")], ["zzz"], read_bytes=rb)
|
||||
assert output is None
|
||||
assert io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_zgrep_count_only():
|
||||
raw = gziplib.compress(b"foo\nfoo\nbar\n")
|
||||
rb, _, _, _ = _make_backend({"/a.gz": raw})
|
||||
output, _ = await zgrep(
|
||||
[_spec("/a.gz")],
|
||||
["foo"],
|
||||
read_bytes=rb,
|
||||
flags={"c": True},
|
||||
)
|
||||
assert b"2" in output
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_zgrep_ignore_case():
|
||||
raw = gziplib.compress(b"Apple\nbanana\n")
|
||||
rb, _, _, _ = _make_backend({"/a.gz": raw})
|
||||
output, _ = await zgrep(
|
||||
[_spec("/a.gz")],
|
||||
["apple"],
|
||||
read_bytes=rb,
|
||||
flags={"i": True},
|
||||
)
|
||||
assert b"Apple" in output
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_zgrep_files_only_multi():
|
||||
rb, _, _, _ = _make_backend({
|
||||
"/a.gz": gziplib.compress(b"foo\n"),
|
||||
"/b.gz": gziplib.compress(b"bar\n"),
|
||||
})
|
||||
output, _ = await zgrep(
|
||||
[_spec("/a.gz"), _spec("/b.gz")],
|
||||
["foo"],
|
||||
read_bytes=rb,
|
||||
flags={"args_l": True},
|
||||
)
|
||||
decoded = output.decode()
|
||||
assert "/a.gz" in decoded
|
||||
assert "/b.gz" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_zgrep_stdin():
|
||||
raw = gziplib.compress(b"hello\nworld\n")
|
||||
rb, _, _, _ = _make_backend({})
|
||||
output, _ = await zgrep([], ["hello"], read_bytes=rb, stdin=raw)
|
||||
assert b"hello" in output
|
||||
@@ -0,0 +1,211 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.csplit import csplit
|
||||
from mirage.commands.builtin.generic.join import join_cmd
|
||||
from mirage.commands.builtin.generic.split import split
|
||||
from mirage.commands.builtin.generic.tee import tee
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(resource_path=(path).strip("/"),
|
||||
virtual=path,
|
||||
directory=path,
|
||||
resolved=True)
|
||||
|
||||
|
||||
def _make_backend(files: dict[str, bytes]):
|
||||
store = dict(files)
|
||||
|
||||
async def read_bytes(accessor, path, index=None):
|
||||
key = path.virtual if isinstance(path, PathSpec) else path
|
||||
if key not in store:
|
||||
raise FileNotFoundError(key)
|
||||
return store[key]
|
||||
|
||||
async def write_bytes(accessor, path, data, index=None):
|
||||
key = path.virtual if isinstance(path, PathSpec) else path
|
||||
store[key] = data
|
||||
|
||||
async def read_stream(accessor, path, index=None):
|
||||
key = path.virtual if isinstance(path, PathSpec) else path
|
||||
if key not in store:
|
||||
raise FileNotFoundError(key)
|
||||
yield store[key]
|
||||
|
||||
return read_bytes, write_bytes, read_stream, store
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_split_by_lines_default():
|
||||
_, wb, rs, _ = _make_backend({})
|
||||
_, io = await split([],
|
||||
read_stream=rs,
|
||||
write_bytes=wb,
|
||||
stdin=b"a\nb\nc\nd\ne\n",
|
||||
lines_per_file=2)
|
||||
assert len(io.writes) == 3
|
||||
assert b"a\nb\n" in io.writes["xaa"]
|
||||
assert b"c\nd\n" in io.writes["xab"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_split_by_bytes():
|
||||
_, wb, rs, _ = _make_backend({})
|
||||
_, io = await split([],
|
||||
read_stream=rs,
|
||||
write_bytes=wb,
|
||||
stdin=b"abcdefghij",
|
||||
byte_limit=4)
|
||||
assert io.writes["xaa"] == b"abcd"
|
||||
assert io.writes["xab"] == b"efgh"
|
||||
assert io.writes["xac"] == b"ij"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_split_n_chunks():
|
||||
_, wb, rs, _ = _make_backend({})
|
||||
_, io = await split([],
|
||||
read_stream=rs,
|
||||
write_bytes=wb,
|
||||
stdin=b"aaaabbbbcc",
|
||||
n_chunks=3)
|
||||
assert len(io.writes) == 3
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_split_numeric_suffix():
|
||||
_, wb, rs, _ = _make_backend({})
|
||||
_, io = await split([],
|
||||
read_stream=rs,
|
||||
write_bytes=wb,
|
||||
stdin=b"a\nb\n",
|
||||
lines_per_file=1,
|
||||
numeric_suffix=True)
|
||||
assert "x00" in io.writes
|
||||
assert "x01" in io.writes
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_csplit_by_line_number():
|
||||
rb, wb, _, _ = _make_backend({})
|
||||
output, io = await csplit([], ("3", ),
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"a\nb\nc\nd\ne\n")
|
||||
assert "xx00" in io.writes
|
||||
assert "xx01" in io.writes
|
||||
assert b"a\nb\n" == io.writes["xx00"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_csplit_by_regex():
|
||||
rb, wb, _, _ = _make_backend({})
|
||||
_, io = await csplit([], ("/MARK/", ),
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"a\nb\nMARK\nc\nd\n")
|
||||
assert b"a\nb\n" == io.writes["xx00"]
|
||||
assert b"MARK\nc\nd\n" == io.writes["xx01"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_csplit_silent_suppresses_size_output():
|
||||
rb, wb, _, _ = _make_backend({})
|
||||
output, _ = await csplit([], ("2", ),
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"a\nb\nc\n",
|
||||
silent=True)
|
||||
assert output == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_csplit_custom_prefix_and_digits():
|
||||
rb, wb, _, _ = _make_backend({})
|
||||
_, io = await csplit([], ("2", ),
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"a\nb\nc\n",
|
||||
prefix="part_",
|
||||
digits=3)
|
||||
assert "part_000" in io.writes
|
||||
assert "part_001" in io.writes
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_join_basic_inner_join():
|
||||
rb, _, _, _ = _make_backend({
|
||||
"/a.txt": b"1 alpha\n2 beta\n3 gamma\n",
|
||||
"/b.txt": b"1 x\n2 y\n4 z\n",
|
||||
})
|
||||
output, _ = await join_cmd(
|
||||
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb)
|
||||
decoded = output.decode()
|
||||
assert "1 alpha x" in decoded
|
||||
assert "2 beta y" in decoded
|
||||
assert "3" not in decoded.split("\n")[0]
|
||||
assert "4" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_join_requires_two_paths():
|
||||
rb, _, _, _ = _make_backend({"/a.txt": b"x"})
|
||||
with pytest.raises(ValueError, match="two paths"):
|
||||
await join_cmd([_spec("/a.txt")], read_bytes=rb)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_join_custom_separator():
|
||||
rb, _, _, _ = _make_backend({
|
||||
"/a.txt": b"1,alpha\n2,beta\n",
|
||||
"/b.txt": b"1,x\n2,y\n",
|
||||
})
|
||||
output, _ = await join_cmd(
|
||||
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb, separator=",")
|
||||
decoded = output.decode()
|
||||
assert "1,alpha,x" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_join_outer_via_a_flag():
|
||||
rb, _, _, _ = _make_backend({
|
||||
"/a.txt": b"1 alpha\n2 beta\n3 gamma\n",
|
||||
"/b.txt": b"1 x\n",
|
||||
})
|
||||
output, _ = await join_cmd(
|
||||
[_spec("/a.txt"), _spec("/b.txt")], read_bytes=rb, also_unpairable="1")
|
||||
decoded = output.decode()
|
||||
assert "2 beta" in decoded
|
||||
assert "3 gamma" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tee_writes_to_file_and_passes_through():
|
||||
_, wb, rs, store = _make_backend({})
|
||||
output, io = await tee([_spec("/out.txt")], (),
|
||||
read_stream=rs,
|
||||
write_bytes=wb,
|
||||
stdin=b"hello tee")
|
||||
assert output == b"hello tee"
|
||||
assert store["/out.txt"] == b"hello tee"
|
||||
assert io.writes == {"/out.txt": b"hello tee"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tee_append_concatenates():
|
||||
_, wb, rs, store = _make_backend({"/out.txt": b"existing\n"})
|
||||
output, _ = await tee([_spec("/out.txt")], (),
|
||||
read_stream=rs,
|
||||
write_bytes=wb,
|
||||
stdin=b"new",
|
||||
append=True)
|
||||
assert store["/out.txt"] == b"existing\nnew"
|
||||
assert output == b"new"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tee_missing_path_raises():
|
||||
_, wb, rs, _ = _make_backend({})
|
||||
with pytest.raises(ValueError, match="missing operand"):
|
||||
await tee([], (), read_stream=rs, write_bytes=wb, stdin=b"data")
|
||||
@@ -0,0 +1,267 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.cut import cut
|
||||
from mirage.commands.builtin.generic.file import file_cmd
|
||||
from mirage.commands.builtin.generic.stat import stat as generic_stat
|
||||
from mirage.types import FileStat, FileType, PathSpec
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(resource_path=(path).strip("/"),
|
||||
virtual=path,
|
||||
directory=path,
|
||||
resolved=True)
|
||||
|
||||
|
||||
def _make_cut_backend(files: dict[str, bytes]):
|
||||
store = dict(files)
|
||||
|
||||
async def read_stream(accessor, path, index=None):
|
||||
key = path.virtual if isinstance(path, PathSpec) else path
|
||||
if key not in store:
|
||||
raise FileNotFoundError(key)
|
||||
yield store[key]
|
||||
|
||||
return read_stream, store
|
||||
|
||||
|
||||
async def _collect(src) -> bytes:
|
||||
if isinstance(src, bytes):
|
||||
return src
|
||||
chunks = b""
|
||||
async for chunk in src:
|
||||
chunks += chunk
|
||||
return chunks
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cut_fields_default_delim():
|
||||
rs, _ = _make_cut_backend({"f.tsv": b"a\tb\tc\nd\te\tf\n"})
|
||||
src, _ = await cut([_spec("f.tsv")], read_stream=rs, f="1,3")
|
||||
out = await _collect(src)
|
||||
assert out == b"a\tc\nd\tf\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cut_fields_custom_delim():
|
||||
rs, _ = _make_cut_backend({"f.csv": b"a,b,c\nd,e,f\n"})
|
||||
src, _ = await cut([_spec("f.csv")], read_stream=rs, f="2", d=",")
|
||||
out = await _collect(src)
|
||||
assert out == b"b\ne\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cut_chars_range():
|
||||
rs, _ = _make_cut_backend({"f.txt": b"hello\nworld\n"})
|
||||
src, _ = await cut([_spec("f.txt")], read_stream=rs, c="1-3")
|
||||
out = await _collect(src)
|
||||
assert out == b"hel\nwor\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cut_complement_fields():
|
||||
rs, _ = _make_cut_backend({"f.tsv": b"a\tb\tc\n"})
|
||||
src, _ = await cut([_spec("f.tsv")],
|
||||
read_stream=rs,
|
||||
f="2",
|
||||
complement=True)
|
||||
out = await _collect(src)
|
||||
assert out == b"a\tc\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cut_zero_terminated():
|
||||
rs, _ = _make_cut_backend({"f.bin": b"a\tb\x00c\td\x00"})
|
||||
src, _ = await cut([_spec("f.bin")], read_stream=rs, f="1", z=True)
|
||||
out = await _collect(src)
|
||||
assert out == b"a\x00c\x00"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cut_stdin():
|
||||
rs, _ = _make_cut_backend({})
|
||||
src, _ = await cut([], read_stream=rs, stdin=b"x\ty\tz\n", f="2")
|
||||
out = await _collect(src)
|
||||
assert out == b"y\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cut_missing_operand():
|
||||
rs, _ = _make_cut_backend({})
|
||||
with pytest.raises(ValueError, match="missing operand"):
|
||||
await cut([], read_stream=rs, f="1")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_stat_default_format():
|
||||
|
||||
async def stat_fn(accessor, path, index=None):
|
||||
return FileStat(name=path.virtual,
|
||||
size=42,
|
||||
modified="2026-01-01",
|
||||
type=FileType.TEXT)
|
||||
|
||||
out, _ = await generic_stat([_spec("a.txt")], stat_fn=stat_fn)
|
||||
assert b"name=a.txt" in out
|
||||
assert b"size=42" in out
|
||||
assert b"type=text" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_stat_custom_format():
|
||||
|
||||
async def stat_fn(accessor, path, index=None):
|
||||
return FileStat(name="foo", size=10, type=FileType.TEXT)
|
||||
|
||||
out, _ = await generic_stat([_spec("foo")], stat_fn=stat_fn, c="%n=%s")
|
||||
assert out == b"foo=10\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_stat_format_F_directory():
|
||||
|
||||
async def stat_fn(accessor, path, index=None):
|
||||
return FileStat(name="d", type=FileType.DIRECTORY)
|
||||
|
||||
out, _ = await generic_stat([_spec("d")], stat_fn=stat_fn, c="%F")
|
||||
assert out == b"directory\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_stat_format_F_regular():
|
||||
|
||||
async def stat_fn(accessor, path, index=None):
|
||||
return FileStat(name="x", type=FileType.JSON)
|
||||
|
||||
out, _ = await generic_stat([_spec("x")], stat_fn=stat_fn, f="%F")
|
||||
assert out == b"regular file\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_stat_multiple_paths():
|
||||
|
||||
async def stat_fn(accessor, path, index=None):
|
||||
return FileStat(name=path.virtual, size=1, type=FileType.TEXT)
|
||||
|
||||
out, _ = await generic_stat([_spec("a"), _spec("b")],
|
||||
stat_fn=stat_fn,
|
||||
c="%n")
|
||||
assert out == b"a\nb\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_stat_missing_operand():
|
||||
|
||||
async def stat_fn(accessor, path, index=None):
|
||||
return FileStat(name="x")
|
||||
|
||||
with pytest.raises(ValueError, match="missing operand"):
|
||||
await generic_stat([], stat_fn=stat_fn)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_text_default():
|
||||
|
||||
async def stat_fn(accessor, path):
|
||||
return FileStat(name=path.virtual, size=5, type=FileType.TEXT)
|
||||
|
||||
async def read_bytes(accessor, path):
|
||||
return b"hello"
|
||||
|
||||
out, _ = await file_cmd([_spec("a.txt")],
|
||||
read_bytes=read_bytes,
|
||||
stat_fn=stat_fn)
|
||||
assert b"a.txt:" in out
|
||||
assert b"text" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_brief_mode():
|
||||
|
||||
async def stat_fn(accessor, path):
|
||||
return FileStat(name="f", size=4, type=FileType.TEXT)
|
||||
|
||||
async def read_bytes(accessor, path):
|
||||
return b"abcd"
|
||||
|
||||
out, _ = await file_cmd([_spec("f")],
|
||||
read_bytes=read_bytes,
|
||||
stat_fn=stat_fn,
|
||||
b=True)
|
||||
assert b":" not in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_mime_mode():
|
||||
|
||||
async def stat_fn(accessor, path):
|
||||
return FileStat(name="f.json", size=10, type=FileType.JSON)
|
||||
|
||||
async def read_bytes(accessor, path):
|
||||
return b'{"a": 1}'
|
||||
|
||||
out, _ = await file_cmd([_spec("f.json")],
|
||||
read_bytes=read_bytes,
|
||||
stat_fn=stat_fn,
|
||||
i=True)
|
||||
assert b"application/json" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_directory():
|
||||
|
||||
async def stat_fn(accessor, path):
|
||||
return FileStat(name="d", type=FileType.DIRECTORY)
|
||||
|
||||
async def read_bytes(accessor, path):
|
||||
raise AssertionError("should not be read for directory")
|
||||
|
||||
out, _ = await file_cmd([_spec("d")],
|
||||
read_bytes=read_bytes,
|
||||
stat_fn=stat_fn)
|
||||
assert b"d: directory\n" == out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_multiple_paths():
|
||||
|
||||
async def stat_fn(accessor, path):
|
||||
return FileStat(name=path.virtual, size=3, type=FileType.TEXT)
|
||||
|
||||
async def read_bytes(accessor, path):
|
||||
return b"abc"
|
||||
|
||||
out, _ = await file_cmd([_spec("a"), _spec("b")],
|
||||
read_bytes=read_bytes,
|
||||
stat_fn=stat_fn)
|
||||
assert b"a:" in out
|
||||
assert b"b:" in out
|
||||
assert out.count(b"\n") == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_read_error_logs_and_falls_back():
|
||||
|
||||
async def stat_fn(accessor, path):
|
||||
return FileStat(name="x", size=1, type=FileType.TEXT)
|
||||
|
||||
async def read_bytes(accessor, path):
|
||||
raise OSError("denied")
|
||||
|
||||
out, _ = await file_cmd([_spec("x")],
|
||||
read_bytes=read_bytes,
|
||||
stat_fn=stat_fn)
|
||||
assert b"x:" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_missing_operand():
|
||||
|
||||
async def stat_fn(accessor, path):
|
||||
return FileStat(name="x")
|
||||
|
||||
async def read_bytes(accessor, path):
|
||||
return b""
|
||||
|
||||
with pytest.raises(ValueError, match="missing operand"):
|
||||
await file_cmd([], read_bytes=read_bytes, stat_fn=stat_fn)
|
||||
@@ -0,0 +1,217 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.basename import basename
|
||||
from mirage.commands.builtin.generic.dirname import dirname
|
||||
from mirage.commands.builtin.generic.mktemp import mktemp
|
||||
from mirage.commands.builtin.generic.readlink import readlink
|
||||
from mirage.commands.builtin.generic.realpath import realpath
|
||||
from mirage.types import FileStat, PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
|
||||
def _spec(original: str, prefix: str = "") -> PathSpec:
|
||||
return PathSpec(resource_path=mount_key(original, prefix),
|
||||
virtual=original,
|
||||
directory=original,
|
||||
resolved=True)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_basename_single():
|
||||
out, _ = await basename("/a/b/c.txt")
|
||||
assert out == b"c.txt\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_basename_suffix():
|
||||
out, _ = await basename("/a/b.txt", ".txt")
|
||||
assert out == b"b\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_basename_empty():
|
||||
out, _ = await basename()
|
||||
assert out == b"\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_dirname_single():
|
||||
out, _ = await dirname("/a/b/c.txt")
|
||||
assert out == b"/a/b\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_dirname_no_slash():
|
||||
out, _ = await dirname("foo")
|
||||
assert out == b".\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_dirname_multiple():
|
||||
out, _ = await dirname("/a/b", "/x/y/z")
|
||||
assert out == b"/a\n/x/y\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_realpath_normalizes():
|
||||
|
||||
async def stat_fn(accessor, path):
|
||||
return FileStat(name="x")
|
||||
|
||||
out, _ = await realpath([_spec("/a/./b/../c")], stat_fn=stat_fn)
|
||||
assert out == b"/a/c\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_realpath_exists_check_passes():
|
||||
|
||||
async def stat_fn(accessor, path):
|
||||
return FileStat(name="x")
|
||||
|
||||
out, _ = await realpath([_spec("/a/b")], stat_fn=stat_fn, e=True)
|
||||
assert out == b"/a/b\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_realpath_exists_check_fails():
|
||||
|
||||
async def stat_fn(accessor, path):
|
||||
raise FileNotFoundError
|
||||
|
||||
with pytest.raises(FileNotFoundError, match="realpath"):
|
||||
await realpath([_spec("/missing")], stat_fn=stat_fn, e=True)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_realpath_multiple():
|
||||
|
||||
async def stat_fn(accessor, path):
|
||||
return FileStat(name="x")
|
||||
|
||||
out, _ = await realpath([_spec("/a"), _spec("/b/../c")], stat_fn=stat_fn)
|
||||
assert out == b"/a\n/c\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_readlink_simple():
|
||||
out, _ = await readlink([_spec("/a/b")])
|
||||
assert out == b"/a/b\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_readlink_with_prefix():
|
||||
out, _ = await readlink([_spec("/mnt/b", prefix="/mnt")])
|
||||
assert out == b"/mnt/b\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_readlink_normalize_with_f():
|
||||
out, _ = await readlink([_spec("/a/./b")], f=True)
|
||||
assert out == b"/a/b\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_readlink_no_newline():
|
||||
out, _ = await readlink([_spec("/a/b")], n=True)
|
||||
assert out == b"/a/b"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_readlink_missing_operand():
|
||||
with pytest.raises(ValueError, match="missing operand"):
|
||||
await readlink([])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mktemp_creates_file():
|
||||
mkdir_calls: list[tuple] = []
|
||||
write_calls: list[tuple] = []
|
||||
|
||||
async def mkdir_fn(accessor, path, parents=False):
|
||||
mkdir_calls.append((path, parents))
|
||||
|
||||
async def write_bytes_fn(accessor, path, data):
|
||||
write_calls.append((path, data))
|
||||
|
||||
out, _ = await mktemp(mkdir_fn=mkdir_fn,
|
||||
write_bytes_fn=write_bytes_fn,
|
||||
t=True)
|
||||
text = out.decode()
|
||||
assert text.startswith("/tmp/tmp.")
|
||||
assert text.endswith("\n")
|
||||
assert mkdir_calls == [("/tmp", True)]
|
||||
assert len(write_calls) == 1
|
||||
assert write_calls[0][1] == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mktemp_creates_directory():
|
||||
mkdir_calls: list[tuple] = []
|
||||
write_calls: list[tuple] = []
|
||||
|
||||
async def mkdir_fn(accessor, path, parents=False):
|
||||
mkdir_calls.append((path, parents))
|
||||
|
||||
async def write_bytes_fn(accessor, path, data):
|
||||
write_calls.append((path, data))
|
||||
|
||||
out, _ = await mktemp(mkdir_fn=mkdir_fn,
|
||||
write_bytes_fn=write_bytes_fn,
|
||||
d=True,
|
||||
t=True)
|
||||
text = out.decode().rstrip("\n")
|
||||
assert mkdir_calls == [("/tmp", True), (text, False)]
|
||||
assert write_calls == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mktemp_custom_parent():
|
||||
mkdir_calls: list[tuple] = []
|
||||
|
||||
async def mkdir_fn(accessor, path, parents=False):
|
||||
mkdir_calls.append((path, parents))
|
||||
|
||||
async def write_bytes_fn(accessor, path, data):
|
||||
pass
|
||||
|
||||
out, _ = await mktemp(mkdir_fn=mkdir_fn,
|
||||
write_bytes_fn=write_bytes_fn,
|
||||
p="/var/cache")
|
||||
assert out.decode().startswith("/var/cache/tmp.")
|
||||
assert mkdir_calls[0] == ("/var/cache", True)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mktemp_pathspec_parent():
|
||||
mkdir_calls: list[tuple] = []
|
||||
|
||||
async def mkdir_fn(accessor, path, parents=False):
|
||||
mkdir_calls.append((path, parents))
|
||||
|
||||
async def write_bytes_fn(accessor, path, data):
|
||||
pass
|
||||
|
||||
out, _ = await mktemp(mkdir_fn=mkdir_fn,
|
||||
write_bytes_fn=write_bytes_fn,
|
||||
p=_spec("/scratch"))
|
||||
assert out.decode().startswith("/scratch/tmp.")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_mktemp_custom_template():
|
||||
mkdir_calls: list[tuple] = []
|
||||
write_calls: list[tuple] = []
|
||||
|
||||
async def mkdir_fn(accessor, path, parents=False):
|
||||
mkdir_calls.append((path, parents))
|
||||
|
||||
async def write_bytes_fn(accessor, path, data):
|
||||
write_calls.append((path, data))
|
||||
|
||||
out, _ = await mktemp("session_XXXXXX",
|
||||
mkdir_fn=mkdir_fn,
|
||||
write_bytes_fn=write_bytes_fn,
|
||||
t=True)
|
||||
text = out.decode().rstrip("\n")
|
||||
assert text.startswith("/tmp/session_")
|
||||
assert len(text) == len("/tmp/session_") + 6
|
||||
@@ -0,0 +1,383 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.diff import diff
|
||||
from mirage.commands.builtin.generic.jq import jq
|
||||
from mirage.commands.builtin.generic.patch import patch
|
||||
from mirage.commands.builtin.generic.tar import tar
|
||||
from mirage.commands.builtin.generic.tsort import tsort
|
||||
from mirage.commands.builtin.generic.unzip import unzip
|
||||
from mirage.commands.builtin.generic.zip_cmd import zip_cmd
|
||||
from mirage.types import PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
|
||||
def _spec(path: str, prefix: str = "") -> PathSpec:
|
||||
return PathSpec(resource_path=mount_key(path, prefix),
|
||||
virtual=path,
|
||||
directory=path,
|
||||
resolved=True)
|
||||
|
||||
|
||||
def _make_backend(files: dict[str, bytes]):
|
||||
store = dict(files)
|
||||
|
||||
async def read_bytes(accessor, path, index=None):
|
||||
key = path.virtual if isinstance(path, PathSpec) else path
|
||||
if key not in store:
|
||||
raise FileNotFoundError(key)
|
||||
return store[key]
|
||||
|
||||
async def write_bytes(accessor, path, data, index=None):
|
||||
key = path.virtual if isinstance(path, PathSpec) else path
|
||||
store[key] = data
|
||||
|
||||
async def read_stream(accessor, path, index=None):
|
||||
key = path.virtual if isinstance(path, PathSpec) else path
|
||||
if key not in store:
|
||||
raise FileNotFoundError(key)
|
||||
yield store[key]
|
||||
|
||||
async def mkdir_fn(accessor, path, parents=False):
|
||||
pass
|
||||
|
||||
return read_bytes, write_bytes, read_stream, mkdir_fn, store
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tsort_basic():
|
||||
rb, _, _, _, _ = _make_backend({"deps": b"a b\nb c\n"})
|
||||
out, io = await tsort([_spec("deps")], read_bytes=rb)
|
||||
assert out == b"a\nb\nc\n"
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tsort_cycle_detection():
|
||||
rb, _, _, _, _ = _make_backend({"deps": b"a b\nb a\n"})
|
||||
out, io = await tsort([_spec("deps")], read_bytes=rb)
|
||||
assert b"cycle" in out
|
||||
assert io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tsort_odd_tokens():
|
||||
rb, _, _, _, _ = _make_backend({"deps": b"a b c\n"})
|
||||
out, io = await tsort([_spec("deps")], read_bytes=rb)
|
||||
assert b"odd number" in out
|
||||
assert io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tsort_stdin():
|
||||
rb, _, _, _, _ = _make_backend({})
|
||||
out, io = await tsort([], read_bytes=rb, stdin=b"x y\ny z\n")
|
||||
assert out == b"x\ny\nz\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_jq_simple_object():
|
||||
rb, _, rs, _, _ = _make_backend({"a.json": b'{"name":"alice","age":30}'})
|
||||
out, _ = await jq([_spec("a.json")],
|
||||
".name",
|
||||
read_bytes=rb,
|
||||
read_stream=rs)
|
||||
assert b'"alice"' in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_jq_raw_output():
|
||||
rb, _, rs, _, _ = _make_backend({"a.json": b'{"name":"alice"}'})
|
||||
out, _ = await jq([_spec("a.json")],
|
||||
".name",
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
r=True)
|
||||
assert b"alice" in out
|
||||
assert b'"' not in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_jq_stdin():
|
||||
rb, _, rs, _, _ = _make_backend({})
|
||||
out, _ = await jq([],
|
||||
".x",
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b'{"x":42}')
|
||||
assert b"42" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_jq_missing_expression_defaults_dot():
|
||||
rb, _, rs, _, _ = _make_backend({})
|
||||
out, io = await jq([], read_bytes=rb, read_stream=rs, stdin=b'{"x":42}')
|
||||
assert b"42" in out
|
||||
assert io.exit_code == 0
|
||||
out, io = await jq([], read_bytes=rb, read_stream=rs)
|
||||
assert out is None
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_jq_no_input():
|
||||
rb, _, rs, _, _ = _make_backend({})
|
||||
out, io = await jq([], ".x", read_bytes=rb, read_stream=rs)
|
||||
assert out is None
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_zip_basic():
|
||||
rb, wb, _, _, store = _make_backend({
|
||||
"f1.txt": b"hello",
|
||||
"f2.txt": b"world"
|
||||
})
|
||||
out, io = await zip_cmd(
|
||||
[_spec("out.zip"), _spec("f1.txt"),
|
||||
_spec("f2.txt")],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb)
|
||||
assert b"adding" in out
|
||||
assert "out.zip" in store
|
||||
archive = store["out.zip"]
|
||||
assert archive.startswith(b"PK")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_zip_quiet():
|
||||
rb, wb, _, _, _ = _make_backend({"f.txt": b"x"})
|
||||
out, _ = await zip_cmd([_spec("o.zip"), _spec("f.txt")],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
q=True)
|
||||
assert out is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_zip_too_few_paths():
|
||||
rb, wb, _, _, _ = _make_backend({})
|
||||
with pytest.raises(ValueError, match="usage"):
|
||||
await zip_cmd([_spec("only.zip")], read_bytes=rb, write_bytes=wb)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_unzip_extracts():
|
||||
import io as _io
|
||||
import zipfile
|
||||
buf = _io.BytesIO()
|
||||
with zipfile.ZipFile(buf, "w") as zf:
|
||||
zf.writestr("a.txt", b"hello")
|
||||
zf.writestr("sub/b.txt", b"world")
|
||||
rb, wb, _, mk, store = _make_backend({"a.zip": buf.getvalue()})
|
||||
out, io_res = await unzip([_spec("a.zip")],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
mkdir_fn=mk)
|
||||
assert b"inflating" in out
|
||||
assert "/a.txt" in io_res.writes
|
||||
assert "/sub/b.txt" in io_res.writes
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_unzip_list():
|
||||
import io as _io
|
||||
import zipfile
|
||||
buf = _io.BytesIO()
|
||||
with zipfile.ZipFile(buf, "w") as zf:
|
||||
zf.writestr("file.txt", b"data")
|
||||
rb, wb, _, mk, _ = _make_backend({"a.zip": buf.getvalue()})
|
||||
out, _ = await unzip([_spec("a.zip")],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
mkdir_fn=mk,
|
||||
args_l=True)
|
||||
assert b"file.txt" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_unzip_test_mode():
|
||||
import io as _io
|
||||
import zipfile
|
||||
buf = _io.BytesIO()
|
||||
with zipfile.ZipFile(buf, "w") as zf:
|
||||
zf.writestr("ok.txt", b"x")
|
||||
rb, wb, _, mk, _ = _make_backend({"a.zip": buf.getvalue()})
|
||||
out, _ = await unzip([_spec("a.zip")],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
mkdir_fn=mk,
|
||||
t=True)
|
||||
assert b"No errors" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_unzip_pipe_mode():
|
||||
import io as _io
|
||||
import zipfile
|
||||
buf = _io.BytesIO()
|
||||
with zipfile.ZipFile(buf, "w") as zf:
|
||||
zf.writestr("a.txt", b"hello")
|
||||
rb, wb, _, mk, _ = _make_backend({"a.zip": buf.getvalue()})
|
||||
out, _ = await unzip([_spec("a.zip")],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
mkdir_fn=mk,
|
||||
p=True)
|
||||
assert out == b"hello"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tar_create_and_list():
|
||||
rb, wb, _, mk, store = _make_backend({"a.txt": b"alpha", "b.txt": b"beta"})
|
||||
_, io_res = await tar([_spec("a.txt"), _spec("b.txt")],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
mkdir_fn=mk,
|
||||
c=True,
|
||||
f=_spec("out.tar"))
|
||||
assert "/out.tar" in io_res.writes
|
||||
out, _ = await tar([],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
mkdir_fn=mk,
|
||||
t=True,
|
||||
f=_spec("out.tar"))
|
||||
assert b"a.txt" in out
|
||||
assert b"b.txt" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tar_extract():
|
||||
rb, wb, _, mk, _ = _make_backend({"x.txt": b"data"})
|
||||
await tar([_spec("x.txt")],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
mkdir_fn=mk,
|
||||
c=True,
|
||||
f=_spec("a.tar"))
|
||||
_, io_res = await tar([],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
mkdir_fn=mk,
|
||||
x=True,
|
||||
f=_spec("a.tar"),
|
||||
C=_spec("/out"))
|
||||
assert any("x.txt" in p for p in io_res.writes)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tar_requires_mode():
|
||||
rb, wb, _, mk, _ = _make_backend({})
|
||||
with pytest.raises(ValueError, match="-c, -x, or -t"):
|
||||
await tar([], read_bytes=rb, write_bytes=wb, mkdir_fn=mk)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tar_requires_f():
|
||||
rb, wb, _, mk, _ = _make_backend({"a.txt": b"x"})
|
||||
with pytest.raises(ValueError, match="-f is required"):
|
||||
await tar([_spec("a.txt")],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
mkdir_fn=mk,
|
||||
c=True)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_diff_identical_files():
|
||||
rb, _, _, _, _ = _make_backend({"a": b"hello\n", "b": b"hello\n"})
|
||||
|
||||
async def rd(accessor, path, index=None):
|
||||
return []
|
||||
|
||||
out, io_res = await diff([_spec("a"), _spec("b")],
|
||||
read_bytes=rb,
|
||||
readdir_fn=rd)
|
||||
assert out == b""
|
||||
assert io_res.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_diff_quiet_differ():
|
||||
rb, _, _, _, _ = _make_backend({"a": b"x\n", "b": b"y\n"})
|
||||
|
||||
async def rd(accessor, path, index=None):
|
||||
return []
|
||||
|
||||
out, io_res = await diff([_spec("a"), _spec("b")],
|
||||
read_bytes=rb,
|
||||
readdir_fn=rd,
|
||||
q=True)
|
||||
assert b"differ" in out
|
||||
assert io_res.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_diff_unified():
|
||||
rb, _, _, _, _ = _make_backend({
|
||||
"a": b"hello\nworld\n",
|
||||
"b": b"hello\nuniverse\n"
|
||||
})
|
||||
|
||||
async def rd(accessor, path, index=None):
|
||||
return []
|
||||
|
||||
out, _ = await diff([_spec("a"), _spec("b")],
|
||||
read_bytes=rb,
|
||||
readdir_fn=rd,
|
||||
u=True)
|
||||
assert b"-world" in out
|
||||
assert b"+universe" in out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_diff_too_few_paths():
|
||||
rb, _, _, _, _ = _make_backend({"a": b"x\n"})
|
||||
|
||||
async def rd(accessor, path, index=None):
|
||||
return []
|
||||
|
||||
with pytest.raises(ValueError, match="two paths"):
|
||||
await diff([_spec("a")], read_bytes=rb, readdir_fn=rd)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_patch_apply():
|
||||
diff_text = (b"--- a/hello.txt\n+++ b/hello.txt\n@@ -1,2 +1,2 @@\n"
|
||||
b" hello\n-world\n+universe\n")
|
||||
rb, wb, _, _, store = _make_backend({"/hello.txt": b"hello\nworld\n"})
|
||||
_, io_res = await patch([],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
has_resource=True,
|
||||
stdin=diff_text,
|
||||
p="1")
|
||||
assert b"universe" in store["/hello.txt"]
|
||||
assert "/hello.txt" in io_res.writes
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_patch_reverse():
|
||||
diff_text = (b"--- a/x.txt\n+++ b/x.txt\n@@ -1,2 +1,2 @@\n"
|
||||
b" hello\n-world\n+universe\n")
|
||||
rb, wb, _, _, store = _make_backend({"/x.txt": b"hello\nuniverse\n"})
|
||||
await patch([],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
has_resource=True,
|
||||
stdin=diff_text,
|
||||
R=True,
|
||||
p="1")
|
||||
assert b"world" in store["/x.txt"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_patch_missing_input():
|
||||
rb, wb, _, _, store = _make_backend({})
|
||||
out, io = await patch([],
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
has_resource=False)
|
||||
assert out is None
|
||||
assert io.exit_code == 0
|
||||
assert not store
|
||||
@@ -0,0 +1,546 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.rg import parse_flags, rg
|
||||
from mirage.commands.spec.types import FlagView
|
||||
from mirage.types import FileStat, FileType, PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(resource_path=(path).strip("/"),
|
||||
virtual=path,
|
||||
directory=path,
|
||||
resolved=True)
|
||||
|
||||
|
||||
def _make_backend(files: dict[str, bytes], dirs: set[str] | None = None):
|
||||
inferred_dirs = set(dirs) if dirs is not None else set()
|
||||
for f in files:
|
||||
parts = f.split("/")
|
||||
for i in range(1, len(parts)):
|
||||
d = "/".join(parts[:i]) or "/"
|
||||
inferred_dirs.add(d)
|
||||
inferred_dirs.add("/")
|
||||
|
||||
async def readdir(accessor, path, index=None):
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
p = spec.virtual.rstrip("/") or "/"
|
||||
if p not in inferred_dirs:
|
||||
raise FileNotFoundError(p)
|
||||
prefix = p + "/" if p != "/" else "/"
|
||||
children: set[str] = set()
|
||||
for f in files:
|
||||
if f.startswith(prefix):
|
||||
rest = f[len(prefix):]
|
||||
child = rest.split("/")[0]
|
||||
children.add(prefix + child)
|
||||
for d in inferred_dirs:
|
||||
if d == p:
|
||||
continue
|
||||
if d.startswith(prefix):
|
||||
rest = d[len(prefix):]
|
||||
child = rest.split("/")[0]
|
||||
children.add(prefix + child)
|
||||
return sorted(children)
|
||||
|
||||
async def stat(accessor, path, index=None):
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
p = spec.virtual
|
||||
if p in files:
|
||||
return FileStat(name=p.rsplit("/", 1)[-1] or p,
|
||||
size=len(files[p]),
|
||||
type=FileType.TEXT)
|
||||
if p.rstrip("/") in inferred_dirs or p in inferred_dirs:
|
||||
return FileStat(name=p.rsplit("/", 1)[-1] or "/",
|
||||
type=FileType.DIRECTORY)
|
||||
raise FileNotFoundError(p)
|
||||
|
||||
async def read_bytes(accessor, path, index=None):
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
if spec.virtual not in files:
|
||||
raise FileNotFoundError(spec.virtual)
|
||||
return files[spec.virtual]
|
||||
|
||||
async def read_stream(accessor, path, index=None):
|
||||
data = await read_bytes(accessor, path)
|
||||
yield data
|
||||
|
||||
return readdir, stat, read_bytes, read_stream
|
||||
|
||||
|
||||
async def _drain_async(stdout):
|
||||
if stdout is None:
|
||||
return b""
|
||||
if isinstance(stdout, bytes):
|
||||
return stdout
|
||||
chunks = [chunk async for chunk in stdout]
|
||||
return b"".join(chunks)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_stdin_basic():
|
||||
readdir, stat, rb, rs = _make_backend({})
|
||||
output, _ = await rg(
|
||||
[],
|
||||
["apple"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"apple\nbanana\napricot\n",
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "apple" in decoded
|
||||
assert "banana" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_count_stdin_uses_match_count():
|
||||
readdir, stat, rb, rs = _make_backend({})
|
||||
output, io = await rg(
|
||||
[],
|
||||
["foo"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"foo foo\nfoo bar\nbaz\n",
|
||||
flags={"c": True},
|
||||
)
|
||||
assert (await _drain_async(output)) == b"2\n"
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_count_stdin_zero_exits_1_without_output():
|
||||
readdir, stat, rb, rs = _make_backend({})
|
||||
output, io = await rg(
|
||||
[],
|
||||
["foo"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
stdin=b"bar\nbaz\n",
|
||||
flags={"c": True},
|
||||
)
|
||||
assert await _drain_async(output) == b""
|
||||
assert io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_file_basic():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/a.txt":
|
||||
b"apple\nbanana\napricot\n",
|
||||
})
|
||||
output, _ = await rg(
|
||||
[_spec("/a.txt")],
|
||||
["ap"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "apple" in decoded
|
||||
assert "apricot" in decoded
|
||||
assert "banana" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_no_match_returns_exit_1():
|
||||
readdir, stat, rb, rs = _make_backend({"/a.txt": b"hello\nworld\n"})
|
||||
output, io = await rg(
|
||||
[_spec("/a.txt")],
|
||||
["zzz"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
drained = await _drain_async(output)
|
||||
assert drained == b""
|
||||
assert io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_dir_auto_recursive():
|
||||
"""rg on a bare directory should auto-recurse (matches real ripgrep)."""
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/dir/a.txt": b"apple\n",
|
||||
"/dir/sub/b.txt": b"apricot\n",
|
||||
})
|
||||
output, _ = await rg(
|
||||
[_spec("/dir")],
|
||||
["ap"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "apple" in decoded
|
||||
assert "apricot" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_count_dir_skips_zero_count_files():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/dir/a.txt": b"foo\nbar\nfoo\n",
|
||||
"/dir/b.txt": b"bar\nbaz\n",
|
||||
})
|
||||
output, io = await rg(
|
||||
[_spec("/dir")],
|
||||
["foo"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={"c": True},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "/dir/a.txt:2" in decoded
|
||||
assert "/dir/b.txt" not in decoded
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_files_only_on_dir():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/dir/a.txt": b"apple\n",
|
||||
"/dir/b.txt": b"zebra\n",
|
||||
})
|
||||
output, _ = await rg(
|
||||
[_spec("/dir")],
|
||||
["apple"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={"args_l": True},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "/dir/a.txt" in decoded
|
||||
assert "/dir/b.txt" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_hidden_excluded_by_default():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/dir/.hidden": b"apple\n",
|
||||
"/dir/visible.txt": b"apple\n",
|
||||
})
|
||||
output, _ = await rg(
|
||||
[_spec("/dir")],
|
||||
["apple"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={"args_l": True},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "/dir/visible.txt" in decoded
|
||||
assert ".hidden" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_hidden_included_with_flag():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/dir/.hidden": b"apple\n",
|
||||
"/dir/visible.txt": b"apple\n",
|
||||
})
|
||||
output, _ = await rg(
|
||||
[_spec("/dir")],
|
||||
["apple"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={
|
||||
"args_l": True,
|
||||
"hidden": True
|
||||
},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "/dir/visible.txt" in decoded
|
||||
assert ".hidden" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_file_type_filter():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/dir/a.py": b"apple\n",
|
||||
"/dir/b.txt": b"apple\n",
|
||||
})
|
||||
output, _ = await rg(
|
||||
[_spec("/dir")],
|
||||
["apple"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={
|
||||
"args_l": True,
|
||||
"type": "py"
|
||||
},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "/dir/a.py" in decoded
|
||||
assert "/dir/b.txt" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_glob_filter():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/dir/a.log": b"apple\n",
|
||||
"/dir/b.txt": b"apple\n",
|
||||
})
|
||||
output, _ = await rg(
|
||||
[_spec("/dir")],
|
||||
["apple"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={
|
||||
"args_l": True,
|
||||
"glob": "*.log"
|
||||
},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "/dir/a.log" in decoded
|
||||
assert "/dir/b.txt" not in decoded
|
||||
|
||||
|
||||
def _make_prefixed_backend(files: dict[str, bytes], mount_prefix: str):
|
||||
"""Backend that mimics real s3/disk/gdrive readdir: entries returned
|
||||
are already prepended with ``mount_prefix``. Used to catch wrapper bugs
|
||||
that re-add the prefix or drop ``index``."""
|
||||
|
||||
full_files = {mount_prefix + k: v for k, v in files.items()}
|
||||
inferred_dirs: set[str] = {mount_prefix or "/"}
|
||||
for f in full_files:
|
||||
parts = f.split("/")
|
||||
for i in range(1, len(parts)):
|
||||
d = "/".join(parts[:i]) or "/"
|
||||
inferred_dirs.add(d)
|
||||
|
||||
def _full(p: str) -> str:
|
||||
if mount_prefix and not p.startswith(mount_prefix):
|
||||
return mount_prefix + p
|
||||
return p
|
||||
|
||||
async def readdir(accessor, path, index=None):
|
||||
if index is None:
|
||||
raise FileNotFoundError("index required")
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
p = _full(spec.virtual).rstrip("/") or "/"
|
||||
if p not in inferred_dirs:
|
||||
raise FileNotFoundError(p)
|
||||
prefix = p + "/" if p != "/" else "/"
|
||||
children: set[str] = set()
|
||||
for f in full_files:
|
||||
if f.startswith(prefix):
|
||||
child = prefix + f[len(prefix):].split("/")[0]
|
||||
children.add(child)
|
||||
for d in inferred_dirs:
|
||||
if d == p or not d.startswith(prefix):
|
||||
continue
|
||||
child = prefix + d[len(prefix):].split("/")[0]
|
||||
children.add(child)
|
||||
return sorted(children)
|
||||
|
||||
async def stat(accessor, path, index=None):
|
||||
if index is None:
|
||||
raise FileNotFoundError("index required")
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
p = _full(spec.virtual)
|
||||
if p in full_files:
|
||||
return FileStat(name=p.rsplit("/", 1)[-1],
|
||||
size=len(full_files[p]),
|
||||
type=FileType.TEXT)
|
||||
if p.rstrip("/") in inferred_dirs:
|
||||
return FileStat(name=p.rsplit("/", 1)[-1] or "/",
|
||||
type=FileType.DIRECTORY)
|
||||
raise FileNotFoundError(p)
|
||||
|
||||
async def read_bytes(accessor, path, index=None):
|
||||
if index is None:
|
||||
raise FileNotFoundError("index required")
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
p = _full(spec.virtual)
|
||||
if p not in full_files:
|
||||
raise FileNotFoundError(p)
|
||||
return full_files[p]
|
||||
|
||||
return readdir, stat, read_bytes
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_files_only_mount_prefix_not_doubled():
|
||||
readdir, stat, rb = _make_prefixed_backend(
|
||||
{
|
||||
"/dir/a.txt": b"apple\n",
|
||||
"/dir/b.txt": b"zebra\n",
|
||||
},
|
||||
mount_prefix="/s3",
|
||||
)
|
||||
p = PathSpec(resource_path=mount_key("/dir", "/s3"),
|
||||
virtual="/dir",
|
||||
directory="/dir",
|
||||
resolved=True)
|
||||
output, _ = await rg(
|
||||
[p],
|
||||
["apple"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=None,
|
||||
flags={"args_l": True},
|
||||
index=object(),
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode().strip()
|
||||
assert decoded == "/s3/dir/a.txt"
|
||||
assert "/s3/s3" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_single_file_threads_index():
|
||||
readdir, stat, rb = _make_prefixed_backend(
|
||||
{"/dir/a.txt": b"apple\n"},
|
||||
mount_prefix="/gd",
|
||||
)
|
||||
p = PathSpec(resource_path=mount_key("/dir/a.txt", "/gd"),
|
||||
virtual="/dir/a.txt",
|
||||
directory="/dir/a.txt",
|
||||
resolved=True)
|
||||
output, _ = await rg(
|
||||
[p],
|
||||
["apple"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=None,
|
||||
index=object(),
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "apple" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_multiple_dirs_searches_all():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/d1/a.txt": b"apple a\n",
|
||||
"/d2/b.txt": b"apple b\n",
|
||||
})
|
||||
output, _ = await rg(
|
||||
[_spec("/d1"), _spec("/d2")],
|
||||
["apple"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "/d1/a.txt:apple a" in decoded
|
||||
assert "/d2/b.txt:apple b" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_files_only_multiple_files():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/t1.txt": b"apple\n",
|
||||
"/t2.txt": b"apple\n",
|
||||
})
|
||||
output, _ = await rg(
|
||||
[_spec("/t1.txt"), _spec("/t2.txt")],
|
||||
["apple"],
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
flags={"args_l": True},
|
||||
)
|
||||
decoded = (await _drain_async(output)).decode()
|
||||
assert "/t1.txt" in decoded
|
||||
assert "/t2.txt" in decoded
|
||||
|
||||
|
||||
def test_parse_flags_c_overrides_a_and_b():
|
||||
f = parse_flags(FlagView({
|
||||
"A": "2",
|
||||
"B": "1",
|
||||
"C": "4"
|
||||
}),
|
||||
never_match=False)
|
||||
assert f.context_after == 4
|
||||
assert f.context_before == 4
|
||||
f = parse_flags(FlagView({"A": "2"}), never_match=False)
|
||||
assert f.context_after == 2
|
||||
assert f.context_before == 0
|
||||
|
||||
|
||||
def test_parse_flags_struct_rejects_typos():
|
||||
f = parse_flags(FlagView({"hidden": True}), never_match=False)
|
||||
assert f.hidden is True
|
||||
with pytest.raises(AttributeError):
|
||||
_ = f.hiden
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_with_filename_labels_single_file():
|
||||
readdir, stat, rb, rs = _make_backend({"/a.txt": b"apple\nbanana\n"})
|
||||
output, _ = await rg(
|
||||
[_spec("/a.txt")],
|
||||
["ap"],
|
||||
{"H": True},
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
assert (await _drain_async(output)).decode() == "/a.txt:apple\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_with_filename_labels_single_file_count():
|
||||
readdir, stat, rb, rs = _make_backend({"/a.txt": b"apple\napricot\n"})
|
||||
output, _ = await rg(
|
||||
[_spec("/a.txt")],
|
||||
["ap"],
|
||||
{
|
||||
"H": True,
|
||||
"c": True
|
||||
},
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
assert (await _drain_async(output)).decode() == "/a.txt:2\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_rg_no_filename_suppresses_multi_file_labels():
|
||||
readdir, stat, rb, rs = _make_backend({
|
||||
"/a.txt": b"apple\n",
|
||||
"/b.txt": b"apricot\n",
|
||||
})
|
||||
output, _ = await rg(
|
||||
[_spec("/a.txt"), _spec("/b.txt")],
|
||||
["ap"],
|
||||
{"args_I": True},
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
read_bytes=rb,
|
||||
read_stream=rs,
|
||||
)
|
||||
assert (await _drain_async(output)).decode() == "apple\napricot\n"
|
||||
@@ -0,0 +1,420 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.sed import sed
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(resource_path=(path).strip("/"),
|
||||
virtual=path,
|
||||
directory=path,
|
||||
resolved=True)
|
||||
|
||||
|
||||
def _make_backend(files: dict[str, bytes]):
|
||||
store = dict(files)
|
||||
|
||||
async def read_bytes(accessor, path, index=None):
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
if spec.virtual not in store:
|
||||
raise FileNotFoundError(spec.virtual)
|
||||
return store[spec.virtual]
|
||||
|
||||
async def write_bytes(accessor, path, data, index=None):
|
||||
spec = path if isinstance(path, PathSpec) else PathSpec(
|
||||
resource_path=(path).strip("/"), virtual=path, directory=path)
|
||||
store[spec.virtual] = data
|
||||
|
||||
return read_bytes, write_bytes, store
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_stdin_simple_sub():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed(
|
||||
[],
|
||||
"s/hello/bye/",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"hello world\n",
|
||||
)
|
||||
assert output == b"bye world\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_file_simple_sub_emits_output():
|
||||
rb, wb, _ = _make_backend({"/a.txt": b"hello world\n"})
|
||||
output, _ = await sed(
|
||||
[_spec("/a.txt")],
|
||||
"s/hello/bye/",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
)
|
||||
assert output == b"bye world\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_inplace_simple_sub_writes_file():
|
||||
rb, wb, store = _make_backend({"/a.txt": b"hello world\n"})
|
||||
output, io = await sed(
|
||||
[_spec("/a.txt")],
|
||||
"s/hello/bye/",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
in_place=True,
|
||||
)
|
||||
assert output is None
|
||||
assert store["/a.txt"] == b"bye world\n"
|
||||
assert io.writes == {"/a.txt": b"bye world\n"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_inplace_multi_path_writes_all():
|
||||
rb, wb, store = _make_backend({
|
||||
"/a.txt": b"hello a\n",
|
||||
"/b.txt": b"hello b\n",
|
||||
})
|
||||
_output, io = await sed(
|
||||
[_spec("/a.txt"), _spec("/b.txt")],
|
||||
"s/hello/bye/",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
in_place=True,
|
||||
)
|
||||
assert store["/a.txt"] == b"bye a\n"
|
||||
assert store["/b.txt"] == b"bye b\n"
|
||||
assert set(io.writes.keys()) == {"/a.txt", "/b.txt"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_global_flag_replaces_all():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed(
|
||||
[],
|
||||
"s/a/X/g",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"banana\n",
|
||||
)
|
||||
assert output == b"bXnXnX\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_first_match_only_by_default():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed(
|
||||
[],
|
||||
"s/a/X/",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"banana\n",
|
||||
)
|
||||
assert output == b"bXnana\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_delete_program():
|
||||
"""Delete command 'd' should drop matching lines."""
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed(
|
||||
[],
|
||||
"/skip/d",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"keep\nskip me\nkeep too\n",
|
||||
)
|
||||
decoded = output.decode()
|
||||
assert "keep" in decoded
|
||||
assert "skip me" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_n_suppress_with_p():
|
||||
"""-n suppresses default output; only explicit 'p' prints."""
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed(
|
||||
[],
|
||||
"/match/p",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"no\nmatch line\nno\n",
|
||||
suppress=True,
|
||||
)
|
||||
decoded = output.decode()
|
||||
assert "match line" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_no_paths_no_stdin_raises():
|
||||
rb, wb, _ = _make_backend({})
|
||||
with pytest.raises(ValueError, match="usage"):
|
||||
await sed([], "s/a/b/", read_bytes=rb, write_bytes=wb)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_numeric_count_replaces_nth_occurrence():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"s/o/O/2",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"oooo\n")
|
||||
assert output == b"oOoo\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_numeric_count_with_g_replaces_nth_onward():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"s/o/O/2g",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"oooo\n")
|
||||
assert output == b"oOOO\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_count_is_per_line():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"s/o/O/2",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"oo\noo\n")
|
||||
assert output == b"oO\noO\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_p_flag_prints_substituted_line_twice():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"s/hi/HI/p",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"hi\nbye\n")
|
||||
assert output == b"HI\nHI\nbye\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_p_flag_under_suppress_prints_only_substituted():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"s/hi/HI/p",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"hi\nbye\n",
|
||||
suppress=True)
|
||||
assert output == b"HI\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_y_transliterate():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"y/el/ip/",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"hello\n")
|
||||
assert output == b"hippo\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_y_mismatched_lengths_raises():
|
||||
rb, wb, _ = _make_backend({})
|
||||
with pytest.raises(ValueError, match="different lengths"):
|
||||
await sed([], "y/ab/x/", read_bytes=rb, write_bytes=wb, stdin=b"a\n")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_c_no_address_changes_every_line():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"c\\\nX",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"a\nb\nc\n")
|
||||
assert output == b"X\nX\nX\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_c_single_address():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"2c\\\nX",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"a\nb\nc\n")
|
||||
assert output == b"a\nX\nc\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_c_range_emits_once():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"2,3c\\\nX",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"a\nb\nc\nd\n")
|
||||
assert output == b"a\nX\nd\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_bre_group_and_backref():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
r"s/\(foo\)/[\1]/",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"foo\n")
|
||||
assert output == b"[foo]\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_bre_plus_is_literal():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"s/a+/X/",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"a+b\n")
|
||||
assert output == b"Xb\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_bre_backslash_plus_is_one_or_more():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
r"s/a\+/X/",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"aaab\n")
|
||||
assert output == b"Xb\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_ere_group_and_plus():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
r"s/(foo)/[\1]/",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"foo\n",
|
||||
extended=True)
|
||||
assert output == b"[foo]\n"
|
||||
output, _ = await sed([],
|
||||
"s/a+/X/",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"aaab\n",
|
||||
extended=True)
|
||||
assert output == b"Xb\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_ere_address():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"/a+/d",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"aaa\nbbb\n",
|
||||
extended=True)
|
||||
assert output == b"bbb\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_negate_line():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"2!d",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"a\nb\nc\n")
|
||||
assert output == b"b\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_negate_regex():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"/b/!d",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"a\nb\nc\n")
|
||||
assert output == b"b\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_negate_last_with_suppress():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"$!p",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"a\nb\nc\n",
|
||||
suppress=True)
|
||||
assert output == b"a\nb\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_negate_range():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"1,2!s/./X/",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"a\nb\nc\nd\n")
|
||||
assert output == b"a\nb\nX\nX\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_join_all_idiom():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
r":a;N;$!ba;s/\n/,/g",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"a\nb\nc\n")
|
||||
assert output == b"a,b,c\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_hold_accumulate():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"H;${x;p}",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"a\nb\n",
|
||||
suppress=True)
|
||||
assert output == b"\na\nb\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_preserves_missing_final_newline():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
"s/o/O/",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"foo")
|
||||
assert output == b"fOo"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_escaped_delimiter():
|
||||
rb, wb, _ = _make_backend({})
|
||||
output, _ = await sed([],
|
||||
r"s/a\/b/c/",
|
||||
read_bytes=rb,
|
||||
write_bytes=wb,
|
||||
stdin=b"a/b\n")
|
||||
assert output == b"c\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_sed_zero_count_rejected():
|
||||
rb, wb, _ = _make_backend({})
|
||||
with pytest.raises(ValueError, match="may not be zero"):
|
||||
await sed([], "s/o/O/0", read_bytes=rb, write_bytes=wb, stdin=b"oo\n")
|
||||
@@ -0,0 +1,240 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.tail import tail
|
||||
|
||||
|
||||
async def _drain(gen):
|
||||
return b"".join([c async for c in gen])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_default_n_10():
|
||||
body = b"\n".join(f"line{i}".encode() for i in range(1, 21)) + b"\n"
|
||||
out = await _drain(tail(body))
|
||||
expected = b"\n".join(f"line{i}".encode() for i in range(11, 21)) + b"\n"
|
||||
assert out == expected
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_n_explicit():
|
||||
out = await _drain(tail(b"a\nb\nc\nd\ne\n", n=3))
|
||||
assert out == b"c\nd\ne\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_n_1():
|
||||
body = b"line1\nline2\nline3\n"
|
||||
out = await _drain(tail(body, n=1))
|
||||
assert out == b"line3\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_n_larger_than_total():
|
||||
out = await _drain(tail(b"a\nb\nc", n=100))
|
||||
assert out == b"a\nb\nc"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_n_zero_emits_nothing():
|
||||
out = await _drain(tail(b"a\nb\nc\n", n=0))
|
||||
assert out == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_negative_n_treated_as_abs():
|
||||
"""GNU/POSIX `tail -n -3` is the same as `tail -n 3` (last 3 lines)."""
|
||||
out = await _drain(tail(b"a\nb\nc\nd\ne\n", n=-3))
|
||||
assert out == b"c\nd\ne\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_n_no_trailing_newline_in_last_line():
|
||||
out = await _drain(tail(b"a\nb\nc", n=2))
|
||||
assert out == b"b\nc"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_single_line_no_newline_default():
|
||||
out = await _drain(tail(b"hello"))
|
||||
assert out == b"hello"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_empty_input():
|
||||
out = await _drain(tail(b""))
|
||||
assert out == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_only_newlines():
|
||||
"""Tail of \\n\\n\\n with n=2 keeps the last two blank lines."""
|
||||
out = await _drain(tail(b"\n\n\n", n=2))
|
||||
assert out == b"\n\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_n_from_stream():
|
||||
|
||||
async def src():
|
||||
yield b"a\nb"
|
||||
yield b"\nc\nd\n"
|
||||
|
||||
out = await _drain(tail(src(), n=2))
|
||||
assert out == b"c\nd\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_n_chunked_one_byte_at_a_time():
|
||||
|
||||
async def src():
|
||||
for byte in b"a\nbb\nccc\n":
|
||||
yield bytes([byte])
|
||||
|
||||
out = await _drain(tail(src(), n=2))
|
||||
assert out == b"bb\nccc\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_c_bytes_fast_path():
|
||||
out = await _drain(tail(b"hello world", c=5))
|
||||
assert out == b"world"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_c_bytes_across_chunks():
|
||||
|
||||
async def src():
|
||||
yield b"hel"
|
||||
yield b"lo wor"
|
||||
yield b"ld"
|
||||
|
||||
out = await _drain(tail(src(), c=5))
|
||||
assert out == b"world"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_c_chunked_one_byte_at_a_time():
|
||||
"""Worst-case chunking for -c: rolling window must hold last N bytes."""
|
||||
|
||||
async def src():
|
||||
for byte in b"hello world":
|
||||
yield bytes([byte])
|
||||
|
||||
out = await _drain(tail(src(), c=5))
|
||||
assert out == b"world"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_c_larger_than_total():
|
||||
out = await _drain(tail(b"abc", c=100))
|
||||
assert out == b"abc"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_c_zero_emits_nothing():
|
||||
out = await _drain(tail(b"abc", c=0))
|
||||
assert out == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_c_negative_is_last_abs():
|
||||
out = await _drain(tail(b"hello", c=-3))
|
||||
assert out == b"llo"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_c_empty_input():
|
||||
out = await _drain(tail(b"", c=10))
|
||||
assert out == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_c_binary_preserves_full_byte_range():
|
||||
"""-c must not corrupt binary bytes."""
|
||||
data = bytes(range(256))
|
||||
out = await _drain(tail(data, c=10))
|
||||
assert out == data[-10:]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_from_line_streaming():
|
||||
"""`tail -n +3` emits lines 3 onwards."""
|
||||
out = await _drain(tail(b"a\nb\nc\nd\ne\n", from_line=3))
|
||||
assert out == b"c\nd\ne\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_from_line_1_is_passthrough():
|
||||
out = await _drain(tail(b"a\nb\nc\n", from_line=1))
|
||||
assert out == b"a\nb\nc\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_from_line_0_treated_as_1():
|
||||
"""GNU `tail -n +0` is documented as `+1` (start from line 1)."""
|
||||
out = await _drain(tail(b"a\nb\nc\n", from_line=0))
|
||||
assert out == b"a\nb\nc\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_from_line_past_end_emits_nothing():
|
||||
out = await _drain(tail(b"a\nb\n", from_line=10))
|
||||
assert out == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_from_line_across_chunks():
|
||||
"""`from_line` skip must work across arbitrary chunk boundaries."""
|
||||
|
||||
async def src():
|
||||
yield b"a\nb"
|
||||
yield b"\nc\nd"
|
||||
yield b"\ne\n"
|
||||
|
||||
out = await _drain(tail(src(), from_line=3))
|
||||
assert out == b"c\nd\ne\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_from_line_chunked_one_byte_at_a_time():
|
||||
|
||||
async def src():
|
||||
for byte in b"a\nb\nc\nd\n":
|
||||
yield bytes([byte])
|
||||
|
||||
out = await _drain(tail(src(), from_line=3))
|
||||
assert out == b"c\nd\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_from_line_preserves_binary_payload():
|
||||
data = b"\x00\x01\n\x02\x03\n\x04\x05\n"
|
||||
out = await _drain(tail(data, from_line=2))
|
||||
assert out == b"\x02\x03\n\x04\x05\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_from_line_1_preserves_stream_chunks():
|
||||
"""`from_line=1` is passthrough: chunk boundaries must survive intact."""
|
||||
|
||||
async def src():
|
||||
yield b"hel"
|
||||
yield b"lo\nwo"
|
||||
yield b"rld\n"
|
||||
|
||||
chunks = [c async for c in tail(src(), from_line=1)]
|
||||
assert chunks == [b"hel", b"lo\nwo", b"rld\n"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_from_line_emits_chunks_incrementally_after_skip():
|
||||
"""Once the skip threshold is crossed, subsequent chunks pass through
|
||||
untouched (no rebuffering)."""
|
||||
|
||||
async def src():
|
||||
yield b"a\nb\nc"
|
||||
yield b"\nd\n"
|
||||
yield b"e\n"
|
||||
|
||||
chunks = [c async for c in tail(src(), from_line=3)]
|
||||
assert chunks == [b"c", b"\nd\n", b"e\n"]
|
||||
@@ -0,0 +1,76 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.tail import tail_multi
|
||||
from mirage.types import PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
|
||||
def _paths(*names: str) -> list[PathSpec]:
|
||||
return [
|
||||
PathSpec(resource_path=mount_key(n, ""),
|
||||
virtual=n,
|
||||
directory="/d",
|
||||
resolved=True) for n in names
|
||||
]
|
||||
|
||||
|
||||
async def _collect(gen) -> bytes:
|
||||
out = b""
|
||||
async for chunk in gen:
|
||||
out += chunk
|
||||
return out
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_multi_bytes_reader_no_headers():
|
||||
data = {"/a": b"a1\na2\na3\n", "/b": b"b1\nb2\n"}
|
||||
|
||||
async def read(accessor, p, index):
|
||||
return data[p.virtual]
|
||||
|
||||
out = await _collect(
|
||||
tail_multi(_paths("/a", "/b"), read=read, n=1, show_headers=False))
|
||||
assert out == b"a3\nb2\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_multi_with_headers():
|
||||
data = {"/a": b"a1\na2\n", "/b": b"b1\nb2\n"}
|
||||
|
||||
async def read(accessor, p, index):
|
||||
return data[p.virtual]
|
||||
|
||||
out = await _collect(
|
||||
tail_multi(_paths("/a", "/b"), read=read, n=1, show_headers=True))
|
||||
assert out == b"==> /a <==\na2\n\n==> /b <==\nb2\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tail_multi_stream_reader():
|
||||
chunks = {"/a": [b"a1\n", b"a2\n"], "/b": [b"b1\n"]}
|
||||
|
||||
def read(accessor, p, index):
|
||||
|
||||
async def gen():
|
||||
for ch in chunks[p.virtual]:
|
||||
yield ch
|
||||
|
||||
return gen()
|
||||
|
||||
out = await _collect(
|
||||
tail_multi(_paths("/a", "/b"), read=read, n=5, show_headers=True))
|
||||
assert out == b"==> /a <==\na1\na2\n\n==> /b <==\nb1\n"
|
||||
@@ -0,0 +1,209 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.tree import tree
|
||||
from mirage.types import FileStat, FileType, PathSpec
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(virtual=path,
|
||||
directory=path,
|
||||
resource_path=path.strip("/"))
|
||||
|
||||
|
||||
def _file(name: str, size: int = 0) -> FileStat:
|
||||
return FileStat(name=name, size=size, type=FileType.TEXT)
|
||||
|
||||
|
||||
def _dir(name: str) -> FileStat:
|
||||
return FileStat(name=name, size=None, type=FileType.DIRECTORY)
|
||||
|
||||
|
||||
def _make_backend(tree_map: dict[str, FileStat]):
|
||||
|
||||
async def stat(p: PathSpec, index=None) -> FileStat:
|
||||
if p.virtual not in tree_map:
|
||||
raise FileNotFoundError(p.virtual)
|
||||
return tree_map[p.virtual]
|
||||
|
||||
async def readdir(p: PathSpec, _index=None) -> list[str]:
|
||||
if p.virtual not in tree_map:
|
||||
raise FileNotFoundError(p.virtual)
|
||||
if tree_map[p.virtual].type != FileType.DIRECTORY:
|
||||
raise ValueError(f"not a directory: {p.virtual}")
|
||||
prefix = p.virtual.rstrip("/") + "/"
|
||||
children: list[str] = []
|
||||
for key in tree_map:
|
||||
if key == p.virtual:
|
||||
continue
|
||||
if key.startswith(prefix):
|
||||
remainder = key[len(prefix):]
|
||||
if "/" not in remainder:
|
||||
children.append(key)
|
||||
return sorted(children)
|
||||
|
||||
return readdir, stat
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_flat_dir():
|
||||
"""Two siblings: the last gets `└──`, the others get `├──`."""
|
||||
tree_map = {
|
||||
"/r": _dir("r"),
|
||||
"/r/a.txt": _file("a.txt"),
|
||||
"/r/b.txt": _file("b.txt"),
|
||||
}
|
||||
readdir, stat = _make_backend(tree_map)
|
||||
output, io = await tree(_spec("/r"), readdir=readdir, stat=stat)
|
||||
lines = output.decode().splitlines()
|
||||
assert lines == ["├── a.txt", "└── b.txt"]
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_nested_dir_uses_vertical_continuation():
|
||||
"""A non-last directory should continue its children with `│ `."""
|
||||
tree_map = {
|
||||
"/r": _dir("r"),
|
||||
"/r/d1": _dir("d1"),
|
||||
"/r/d1/x.txt": _file("x.txt"),
|
||||
"/r/z.txt": _file("z.txt"),
|
||||
}
|
||||
readdir, stat = _make_backend(tree_map)
|
||||
output, _ = await tree(_spec("/r"), readdir=readdir, stat=stat)
|
||||
lines = output.decode().splitlines()
|
||||
assert lines == ["├── d1", "│ └── x.txt", "└── z.txt"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_last_dir_uses_indent_continuation():
|
||||
"""A last directory should continue with plain spaces, no vertical bar."""
|
||||
tree_map = {
|
||||
"/r": _dir("r"),
|
||||
"/r/d1": _dir("d1"),
|
||||
"/r/d1/x.txt": _file("x.txt"),
|
||||
}
|
||||
readdir, stat = _make_backend(tree_map)
|
||||
output, _ = await tree(_spec("/r"), readdir=readdir, stat=stat)
|
||||
lines = output.decode().splitlines()
|
||||
assert lines == ["└── d1", " └── x.txt"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_max_depth_limits_recursion():
|
||||
tree_map = {
|
||||
"/r": _dir("r"),
|
||||
"/r/d1": _dir("d1"),
|
||||
"/r/d1/d2": _dir("d2"),
|
||||
"/r/d1/d2/deep.txt": _file("deep.txt"),
|
||||
}
|
||||
readdir, stat = _make_backend(tree_map)
|
||||
output, _ = await tree(_spec("/r"),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
max_depth=1)
|
||||
decoded = output.decode()
|
||||
assert "d1" in decoded
|
||||
assert "d2" in decoded
|
||||
assert "deep.txt" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_hides_dotfiles_by_default():
|
||||
tree_map = {
|
||||
"/r": _dir("r"),
|
||||
"/r/.hidden": _file(".hidden"),
|
||||
"/r/visible.txt": _file("visible.txt"),
|
||||
}
|
||||
readdir, stat = _make_backend(tree_map)
|
||||
output, _ = await tree(_spec("/r"), readdir=readdir, stat=stat)
|
||||
decoded = output.decode()
|
||||
assert ".hidden" not in decoded
|
||||
assert "visible.txt" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_show_hidden_includes_dotfiles():
|
||||
tree_map = {
|
||||
"/r": _dir("r"),
|
||||
"/r/.hidden": _file(".hidden"),
|
||||
"/r/visible.txt": _file("visible.txt"),
|
||||
}
|
||||
readdir, stat = _make_backend(tree_map)
|
||||
output, _ = await tree(_spec("/r"),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
show_hidden=True)
|
||||
assert ".hidden" in output.decode()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_ignore_pattern_drops_matches():
|
||||
tree_map = {
|
||||
"/r": _dir("r"),
|
||||
"/r/a.pyc": _file("a.pyc"),
|
||||
"/r/b.py": _file("b.py"),
|
||||
}
|
||||
readdir, stat = _make_backend(tree_map)
|
||||
output, _ = await tree(_spec("/r"),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
ignore_pattern="*.pyc")
|
||||
decoded = output.decode()
|
||||
assert "a.pyc" not in decoded
|
||||
assert "b.py" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_dirs_only_drops_files():
|
||||
tree_map = {
|
||||
"/r": _dir("r"),
|
||||
"/r/d1": _dir("d1"),
|
||||
"/r/a.txt": _file("a.txt"),
|
||||
}
|
||||
readdir, stat = _make_backend(tree_map)
|
||||
output, _ = await tree(_spec("/r"),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
dirs_only=True)
|
||||
decoded = output.decode()
|
||||
assert "d1" in decoded
|
||||
assert "a.txt" not in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_match_pattern_only_applies_to_files():
|
||||
"""`-P` filters file names but never excludes directories."""
|
||||
tree_map = {
|
||||
"/r": _dir("r"),
|
||||
"/r/d1": _dir("d1"),
|
||||
"/r/d1/match.py": _file("match.py"),
|
||||
"/r/d1/skip.txt": _file("skip.txt"),
|
||||
"/r/top.py": _file("top.py"),
|
||||
}
|
||||
readdir, stat = _make_backend(tree_map)
|
||||
output, _ = await tree(_spec("/r"),
|
||||
readdir=readdir,
|
||||
stat=stat,
|
||||
match_pattern="*.py")
|
||||
decoded = output.decode()
|
||||
assert "d1" in decoded
|
||||
assert "match.py" in decoded
|
||||
assert "skip.txt" not in decoded
|
||||
assert "top.py" in decoded
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_missing_path_emits_warning_not_crash():
|
||||
readdir, stat = _make_backend({})
|
||||
output, io = await tree(_spec("/nowhere"), readdir=readdir, stat=stat)
|
||||
assert output == b""
|
||||
assert b"nowhere" in (io.stderr or b"")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tree_empty_dir_emits_nothing():
|
||||
tree_map = {"/r": _dir("r")}
|
||||
readdir, stat = _make_backend(tree_map)
|
||||
output, io = await tree(_spec("/r"), readdir=readdir, stat=stat)
|
||||
assert output == b""
|
||||
assert io.exit_code == 0
|
||||
@@ -0,0 +1,106 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.uniq import _parse_count, uniq
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
def _unused_read_stream(_accessor, _path):
|
||||
raise AssertionError("read_stream should not be called for stdin input")
|
||||
|
||||
|
||||
async def _generator_read_stream(_accessor, _path):
|
||||
yield b"dup\ndup\nsolo\n"
|
||||
|
||||
|
||||
async def _collect(stdin: bytes, **kwargs) -> bytes:
|
||||
source, _io = await uniq(
|
||||
[],
|
||||
read_stream=_unused_read_stream,
|
||||
stdin=stdin,
|
||||
**kwargs,
|
||||
)
|
||||
chunks = [chunk async for chunk in source]
|
||||
return b"".join(chunks)
|
||||
|
||||
|
||||
def test_parse_count_unset_is_none():
|
||||
assert _parse_count(None) is None
|
||||
|
||||
|
||||
def test_parse_count_zero_string():
|
||||
assert _parse_count("0") == 0
|
||||
|
||||
|
||||
def test_parse_count_positive():
|
||||
assert _parse_count("3") == 3
|
||||
|
||||
|
||||
def test_parse_count_negative_raises():
|
||||
with pytest.raises(ValueError, match="invalid count"):
|
||||
_parse_count("-1")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_skip_fields_unset_matches_zero():
|
||||
data = b"a one\nb one\n"
|
||||
unset = await _collect(data)
|
||||
zero = await _collect(data, skip_fields="0")
|
||||
assert unset == zero == b"a one\nb one\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_skip_fields_collapses_on_second_field():
|
||||
data = b"a shared\nb shared\n"
|
||||
out = await _collect(data, skip_fields="1")
|
||||
assert out == b"a shared\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_skip_chars_offset():
|
||||
data = b"Xfoo\nYfoo\n"
|
||||
out = await _collect(data, skip_chars="1")
|
||||
assert out == b"Xfoo\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_check_chars_limits_comparison():
|
||||
data = b"abcAAA\nabcBBB\n"
|
||||
out = await _collect(data, check_chars="3")
|
||||
assert out == b"abcAAA\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_check_chars_unset_compares_full_line():
|
||||
data = b"abcAAA\nabcBBB\n"
|
||||
out = await _collect(data)
|
||||
assert out == b"abcAAA\nabcBBB\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_check_chars_zero_string_treats_all_lines_as_duplicates():
|
||||
# GNU: -w 0 compares zero characters, so every line matches the first
|
||||
data = b"abcAAA\nabcBBB\n"
|
||||
out = await _collect(data, check_chars="0")
|
||||
assert out == b"abcAAA\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_count_prefixes_occurrences():
|
||||
data = b"dup\ndup\nsolo\n"
|
||||
out = await _collect(data, count=True)
|
||||
assert out == b" 2 dup\n 1 solo\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ignore_case_folds_duplicates():
|
||||
data = b"Hello\nhello\n"
|
||||
out = await _collect(data, ignore_case=True)
|
||||
assert out == b"Hello\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_read_stream_async_generator():
|
||||
p = PathSpec(resource_path="x", virtual="/x", directory="/x")
|
||||
source, _io = await uniq([p], read_stream=_generator_read_stream)
|
||||
out = b"".join([chunk async for chunk in source])
|
||||
assert out == b"dup\nsolo\n"
|
||||
@@ -0,0 +1,342 @@
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic.wc import (WCCounts, format_multi,
|
||||
format_wc, wc, wc_lines)
|
||||
from mirage.types import PathSpec
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_default_counts_bytes():
|
||||
counts = await wc(b"hello world\nfoo bar\n")
|
||||
assert counts.lines == 2
|
||||
assert counts.words == 4
|
||||
assert counts.bytes_ == 20
|
||||
assert counts.chars == 20
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_empty_input():
|
||||
counts = await wc(b"")
|
||||
assert counts == WCCounts(lines=0,
|
||||
words=0,
|
||||
bytes_=0,
|
||||
chars=0,
|
||||
max_line_length=0)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_lines_with_trailing_newline():
|
||||
counts = await wc(b"a\nb\nc\n")
|
||||
assert counts.lines == 3
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_lines_without_trailing_newline():
|
||||
"""POSIX: lines = number of \\n bytes. `a\\nb\\nc` has 2 newlines."""
|
||||
counts = await wc(b"a\nb\nc")
|
||||
assert counts.lines == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_words_single_line():
|
||||
counts = await wc(b"one two three")
|
||||
assert counts.words == 3
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_words_multiline():
|
||||
counts = await wc(b"one two\nthree four five\nsix\n")
|
||||
assert counts.words == 6
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_words_leading_trailing_whitespace():
|
||||
"""POSIX: leading/trailing whitespace doesn't add words."""
|
||||
counts = await wc(b" hello world ")
|
||||
assert counts.words == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_words_only_whitespace():
|
||||
counts = await wc(b" \t \n ")
|
||||
assert counts.words == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_bytes_ascii():
|
||||
counts = await wc(b"hello")
|
||||
assert counts.bytes_ == 5
|
||||
assert counts.chars == 5
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_chars_vs_bytes_multibyte_utf8():
|
||||
"""`café` is 4 chars / 5 bytes (é is 2 bytes in UTF-8)."""
|
||||
data = "café".encode()
|
||||
counts = await wc(data)
|
||||
assert counts.bytes_ == 5
|
||||
assert counts.chars == 4
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_chars_vs_bytes_pure_multibyte():
|
||||
data = "ééé".encode()
|
||||
counts = await wc(data)
|
||||
assert counts.bytes_ == 6
|
||||
assert counts.chars == 3
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_max_line_length_empty():
|
||||
counts = await wc(b"")
|
||||
assert counts.max_line_length == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_max_line_length_single_line_with_newline():
|
||||
counts = await wc(b"hello\n")
|
||||
assert counts.max_line_length == 5
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_max_line_length_picks_longest():
|
||||
counts = await wc(b"short\na much longer line\nmed\n")
|
||||
assert counts.max_line_length == len(b"a much longer line")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_max_line_length_no_trailing_newline():
|
||||
counts = await wc(b"hello world")
|
||||
assert counts.max_line_length == 11
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_streams_chunked():
|
||||
"""Streaming through chunk boundaries gives same counts as buffered."""
|
||||
|
||||
async def src():
|
||||
yield b"hello "
|
||||
yield b"world\n"
|
||||
yield b"foo bar\n"
|
||||
|
||||
counts = await wc(src())
|
||||
assert counts.lines == 2
|
||||
assert counts.words == 4
|
||||
assert counts.bytes_ == 20
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_word_split_across_chunks():
|
||||
"""A word straddling a chunk boundary still counts as one word."""
|
||||
|
||||
async def src():
|
||||
yield b"hel"
|
||||
yield b"lo"
|
||||
yield b" world\n"
|
||||
|
||||
counts = await wc(src())
|
||||
assert counts.words == 2
|
||||
assert counts.lines == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_utf8_split_across_chunks_does_not_lose_chars():
|
||||
"""A multibyte UTF-8 sequence split across chunks must still decode."""
|
||||
|
||||
async def src():
|
||||
# é = b"\xc3\xa9" — split between the two bytes
|
||||
yield b"caf\xc3"
|
||||
yield b"\xa9"
|
||||
|
||||
counts = await wc(src())
|
||||
assert counts.bytes_ == 5
|
||||
assert counts.chars == 4
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_byte_by_byte_chunking():
|
||||
"""Worst-case chunking: every byte its own chunk."""
|
||||
|
||||
async def src():
|
||||
for byte in b"hello world\nfoo bar\n":
|
||||
yield bytes([byte])
|
||||
|
||||
counts = await wc(src())
|
||||
assert counts.lines == 2
|
||||
assert counts.words == 4
|
||||
assert counts.bytes_ == 20
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_binary_input_does_not_crash():
|
||||
"""`errors='replace'` must let arbitrary bytes pass — count bytes
|
||||
accurately even when UTF-8 decoding produces replacement chars."""
|
||||
data = bytes(range(256))
|
||||
counts = await wc(data)
|
||||
assert counts.bytes_ == 256
|
||||
assert counts.lines == 1 # one \n at byte 0x0a
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_lines_fast_path_matches_full():
|
||||
"""`wc_lines` fast path must agree with the full counter."""
|
||||
data = b"alpha\nbeta\ngamma\ndelta\n"
|
||||
fast = await wc_lines(data)
|
||||
full = await wc(data)
|
||||
assert fast == full.lines == 4
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_lines_fast_path_no_trailing_newline():
|
||||
assert await wc_lines(b"a\nb\nc") == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_wc_lines_fast_path_empty():
|
||||
assert await wc_lines(b"") == 0
|
||||
|
||||
|
||||
def test_format_wc_default_no_label():
|
||||
counts = WCCounts(lines=2, words=4, bytes_=20)
|
||||
assert format_wc(counts) == " 2 4 20"
|
||||
|
||||
|
||||
def test_format_wc_default_with_label():
|
||||
counts = WCCounts(lines=2, words=4, bytes_=20)
|
||||
assert format_wc(counts, label="/f.txt") == " 2 4 20 /f.txt"
|
||||
|
||||
|
||||
def test_format_wc_args_l():
|
||||
counts = WCCounts(lines=2, words=4, bytes_=20)
|
||||
assert format_wc(counts, args_l=True) == "2"
|
||||
assert format_wc(counts, args_l=True, label="/f.txt") == "2 /f.txt"
|
||||
|
||||
|
||||
def test_format_wc_w_c_m():
|
||||
counts = WCCounts(lines=2, words=4, bytes_=20, chars=18)
|
||||
assert format_wc(counts, w=True) == "4"
|
||||
assert format_wc(counts, c=True) == "20"
|
||||
assert format_wc(counts, m=True) == "18"
|
||||
|
||||
|
||||
def test_format_wc_L_wins_when_both_set():
|
||||
"""L has highest precedence (matching wrapper behavior)."""
|
||||
counts = WCCounts(lines=2, max_line_length=11)
|
||||
assert format_wc(counts, args_l=True, L=True) == "11"
|
||||
|
||||
|
||||
def test_format_wc_precedence_l_over_w_c_m():
|
||||
"""args_l > w > c > m precedence."""
|
||||
counts = WCCounts(lines=2, words=4, bytes_=20, chars=18)
|
||||
assert format_wc(counts, args_l=True, w=True) == "2"
|
||||
|
||||
|
||||
def test_wc_counts_merge():
|
||||
a = WCCounts(lines=2, words=4, bytes_=20, chars=18, max_line_length=11)
|
||||
b = WCCounts(lines=1, words=2, bytes_=8, chars=8, max_line_length=20)
|
||||
a.merge(b)
|
||||
assert a.lines == 3
|
||||
assert a.words == 6
|
||||
assert a.bytes_ == 28
|
||||
assert a.chars == 26
|
||||
assert a.max_line_length == 20
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_format_multi_single_path_emits_trailing_newline():
|
||||
paths = [PathSpec.from_str_path("/a.txt")]
|
||||
|
||||
async def fake_read(_accessor, _path):
|
||||
return b"hello\n"
|
||||
|
||||
out, err = await format_multi(paths, read=fake_read, args_l=True)
|
||||
assert out == b"1 /a.txt\n"
|
||||
assert err == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_format_multi_multi_path_emits_total_and_trailing_newline():
|
||||
paths = [
|
||||
PathSpec.from_str_path("/a.txt"),
|
||||
PathSpec.from_str_path("/b.txt"),
|
||||
]
|
||||
data = {"/a.txt": b"hello\n", "/b.txt": b"world\nworld\n"}
|
||||
|
||||
async def fake_read(_accessor, path):
|
||||
return data[path.virtual]
|
||||
|
||||
out, err = await format_multi(paths, read=fake_read, args_l=True)
|
||||
assert err == b""
|
||||
assert out.endswith(b"\n")
|
||||
lines = out.decode().rstrip("\n").split("\n")
|
||||
assert lines == ["1 /a.txt", "2 /b.txt", "3 total"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_format_multi_accepts_sync_read_returning_bytes():
|
||||
paths = [PathSpec.from_str_path("/a.txt")]
|
||||
|
||||
def sync_read(_accessor, _path):
|
||||
return b"x\n"
|
||||
|
||||
out, err = await format_multi(paths, read=sync_read, args_l=True)
|
||||
assert out == b"1 /a.txt\n"
|
||||
assert err == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_format_multi_empty_paths_returns_empty():
|
||||
|
||||
async def fake_read(_accessor, _path):
|
||||
return b""
|
||||
|
||||
out, err = await format_multi([], read=fake_read, args_l=True)
|
||||
assert out == b""
|
||||
assert err == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_format_multi_missing_operand_reports_and_totals():
|
||||
paths = [
|
||||
PathSpec.from_str_path("/a.txt"),
|
||||
PathSpec.from_str_path("/m.txt"),
|
||||
]
|
||||
|
||||
async def fake_read(_accessor, path):
|
||||
if path.virtual == "/m.txt":
|
||||
raise FileNotFoundError(path.virtual)
|
||||
return b"hello\n"
|
||||
|
||||
out, err = await format_multi(paths, read=fake_read, args_l=True)
|
||||
assert out == b"1 /a.txt\n1 total\n"
|
||||
assert err == b"wc: /m.txt: No such file or directory\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_format_multi_all_missing_zero_total():
|
||||
paths = [
|
||||
PathSpec.from_str_path("/m1.txt"),
|
||||
PathSpec.from_str_path("/m2.txt"),
|
||||
]
|
||||
|
||||
async def fake_read(_accessor, path):
|
||||
raise FileNotFoundError(path.virtual)
|
||||
|
||||
out, err = await format_multi(paths, read=fake_read, args_l=True)
|
||||
assert out == b"0 total\n"
|
||||
assert err == (b"wc: /m1.txt: No such file or directory\n"
|
||||
b"wc: /m2.txt: No such file or directory\n")
|
||||
|
||||
|
||||
async def _async_byte_read(_accessor, _path):
|
||||
yield b"hello "
|
||||
yield b"world\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_format_multi_accepts_async_iterator_read():
|
||||
paths = [PathSpec.from_str_path("/a.txt")]
|
||||
|
||||
out, err = await format_multi(paths, read=_async_byte_read, args_l=True)
|
||||
assert out == b"1 /a.txt\n"
|
||||
assert err == b""
|
||||
@@ -0,0 +1,188 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic_bind.adapter import CommandIO
|
||||
from mirage.commands.builtin.generic_bind.builders.common import (
|
||||
dir_refusing_read, split_readable)
|
||||
from mirage.types import FileStat, FileType, PathSpec
|
||||
|
||||
|
||||
def _ops(missing: set[str],
|
||||
implicit_dirs: set[str] | None = None,
|
||||
explicit_dirs: set[str] | None = None) -> CommandIO:
|
||||
dirs = implicit_dirs or set()
|
||||
typed = explicit_dirs or set()
|
||||
|
||||
async def stat(_accessor, path, _index):
|
||||
if path.virtual in missing or path.virtual in dirs:
|
||||
raise FileNotFoundError(path.virtual)
|
||||
if path.virtual in typed:
|
||||
return FileStat(name=path.virtual, type=FileType.DIRECTORY)
|
||||
return FileStat(name=path.virtual, size=0)
|
||||
|
||||
async def readdir(_accessor, path, _index):
|
||||
target = path.virtual.rstrip("/") or "/"
|
||||
entries = [d for d in dirs if (d.rsplit("/", 1)[0] or "/") == target]
|
||||
if path.virtual in dirs:
|
||||
entries.append(path.virtual.rstrip("/") + "/child.txt")
|
||||
return entries
|
||||
|
||||
async def read_stream(_accessor, _path, _index):
|
||||
yield b"data"
|
||||
|
||||
async def unused(*_args):
|
||||
raise AssertionError("not used")
|
||||
|
||||
return CommandIO(readdir=readdir,
|
||||
read_bytes=unused,
|
||||
read_stream=read_stream,
|
||||
stat=stat,
|
||||
is_mounted=lambda _a: True)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_split_readable_keeps_order_and_reports_missing():
|
||||
paths = [
|
||||
PathSpec.from_str_path("/m1.txt"),
|
||||
PathSpec.from_str_path("/f.txt"),
|
||||
PathSpec.from_str_path("/m2.txt"),
|
||||
]
|
||||
good, err = await split_readable(_ops({"/m1.txt", "/m2.txt"}), None, paths,
|
||||
None, "cat")
|
||||
assert [p.virtual for p in good] == ["/f.txt"]
|
||||
assert err == (b"cat: /m1.txt: No such file or directory\n"
|
||||
b"cat: /m2.txt: No such file or directory\n")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_split_readable_reports_implicit_dir_as_eisdir():
|
||||
ops = _ops(set(), implicit_dirs={"/sub"})
|
||||
good, err = await split_readable(ops, None,
|
||||
[PathSpec.from_str_path("/sub")], None,
|
||||
"cat")
|
||||
assert good == []
|
||||
assert err == b"cat: /sub: Is a directory\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_split_readable_ignores_fabricated_children():
|
||||
# Synthetic hierarchies (postgres schema level) answer a readdir of
|
||||
# any missing name with fabricated children; only the parent listing
|
||||
# decides, so the original ENOENT stands.
|
||||
|
||||
async def stat(_accessor, path, _index):
|
||||
raise FileNotFoundError(path.virtual)
|
||||
|
||||
async def readdir(_accessor, path, _index):
|
||||
target = path.virtual.rstrip("/") or "/"
|
||||
if target == "/":
|
||||
return ["/real.txt"]
|
||||
return [f"{target}/tables", f"{target}/views"]
|
||||
|
||||
async def unused(*_args):
|
||||
raise AssertionError("not used")
|
||||
|
||||
ops = CommandIO(readdir=readdir,
|
||||
read_bytes=unused,
|
||||
read_stream=unused,
|
||||
stat=stat,
|
||||
is_mounted=lambda _a: True)
|
||||
good, err = await split_readable(ops, None,
|
||||
[PathSpec.from_str_path("/nope.txt")],
|
||||
None, "cat")
|
||||
assert good == []
|
||||
assert err == b"cat: /nope.txt: No such file or directory\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_split_readable_probe_swallows_driver_errors():
|
||||
# A backend whose readdir raises a non-FS driver error for missing
|
||||
# names (lancedb: "Table ... was not found") must not leak it through
|
||||
# the probe; the original ENOENT stands.
|
||||
|
||||
async def stat(_accessor, path, _index):
|
||||
raise FileNotFoundError(path.virtual)
|
||||
|
||||
async def readdir(_accessor, path, _index):
|
||||
raise ValueError("Table 'nope.txt' was not found")
|
||||
|
||||
async def unused(*_args):
|
||||
raise AssertionError("not used")
|
||||
|
||||
ops = CommandIO(readdir=readdir,
|
||||
read_bytes=unused,
|
||||
read_stream=unused,
|
||||
stat=stat,
|
||||
is_mounted=lambda _a: True)
|
||||
good, err = await split_readable(ops, None,
|
||||
[PathSpec.from_str_path("/nope.txt")],
|
||||
None, "wc")
|
||||
assert good == []
|
||||
assert err == b"wc: /nope.txt: No such file or directory\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_split_readable_reports_stat_typed_dir_as_eisdir():
|
||||
ops = _ops(set(), explicit_dirs={"/sub"})
|
||||
good, err = await split_readable(ops, None,
|
||||
[PathSpec.from_str_path("/sub")], None,
|
||||
"head")
|
||||
assert good == []
|
||||
assert err == b"head: /sub: Is a directory\n"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_dir_refusing_read_raises_eisdir_for_dirs():
|
||||
ops = _ops(set(), implicit_dirs={"/sub"})
|
||||
read = dir_refusing_read(ops, None)
|
||||
with pytest.raises(IsADirectoryError):
|
||||
async for _ in read(None, PathSpec.from_str_path("/sub")):
|
||||
raise AssertionError("no data expected")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_dir_refusing_read_streams_files():
|
||||
ops = _ops(set())
|
||||
read = dir_refusing_read(ops, None)
|
||||
chunks = [c async for c in read(None, PathSpec.from_str_path("/f.txt"))]
|
||||
assert chunks == [b"data"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_split_readable_all_good_no_stderr():
|
||||
paths = [PathSpec.from_str_path("/f.txt")]
|
||||
good, err = await split_readable(_ops(set()), None, paths, None, "head")
|
||||
assert [p.virtual for p in good] == ["/f.txt"]
|
||||
assert err == b""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_split_readable_propagates_non_fs_errors():
|
||||
|
||||
async def stat(_accessor, _path, _index):
|
||||
raise RuntimeError("backend broke")
|
||||
|
||||
async def unused(*_args):
|
||||
raise AssertionError("not used")
|
||||
|
||||
ops = CommandIO(readdir=unused,
|
||||
read_bytes=unused,
|
||||
read_stream=unused,
|
||||
stat=stat,
|
||||
is_mounted=lambda _a: True)
|
||||
with pytest.raises(RuntimeError):
|
||||
await split_readable(ops, None, [PathSpec.from_str_path("/f.txt")],
|
||||
None, "cat")
|
||||
@@ -0,0 +1,117 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic_bind.adapter import CommandIO
|
||||
from mirage.commands.builtin.generic_bind.builders.find import find
|
||||
from mirage.types import FileStat, FileType, PathSpec
|
||||
|
||||
TREE = {
|
||||
"/mnt": ["/mnt/table1", "/mnt/notes.txt"],
|
||||
"/mnt/table1": ["/mnt/table1/rows.jsonl"],
|
||||
}
|
||||
|
||||
DIRS = {"/mnt", "/mnt/table1"}
|
||||
|
||||
|
||||
def _ops(stat_calls: list[str], is_dir_name=None, find_op=None) -> CommandIO:
|
||||
|
||||
async def readdir(_accessor, path, _index):
|
||||
return TREE.get(path.virtual.rstrip("/") or "/", [])
|
||||
|
||||
async def stat(_accessor, path, _index):
|
||||
stat_calls.append(path.virtual)
|
||||
if path.virtual not in TREE and path.virtual not in TREE.get(
|
||||
"/mnt", []) and path.virtual != "/mnt/table1/rows.jsonl":
|
||||
raise FileNotFoundError(path.virtual)
|
||||
if path.virtual in DIRS:
|
||||
return FileStat(name=path.virtual, type=FileType.DIRECTORY)
|
||||
return FileStat(name=path.virtual, size=3)
|
||||
|
||||
async def read_stream(_accessor, _path, _index):
|
||||
yield b"data"
|
||||
|
||||
async def unused(*_args):
|
||||
raise AssertionError("not used")
|
||||
|
||||
return CommandIO(readdir=readdir,
|
||||
read_bytes=unused,
|
||||
read_stream=read_stream,
|
||||
stat=stat,
|
||||
is_mounted=lambda _a: True,
|
||||
local=False,
|
||||
is_dir_name=is_dir_name,
|
||||
find=find_op)
|
||||
|
||||
|
||||
def _root() -> PathSpec:
|
||||
return PathSpec(virtual="/mnt",
|
||||
directory="/mnt",
|
||||
resolved=False,
|
||||
resource_path="")
|
||||
|
||||
|
||||
async def _lines(ops: CommandIO) -> list[str]:
|
||||
stdout, _io = await find(ops, None, [_root()])
|
||||
data = stdout if isinstance(stdout, bytes) else b""
|
||||
return data.decode().splitlines()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_without_hint_stats_children():
|
||||
stat_calls: list[str] = []
|
||||
ops = _ops(stat_calls)
|
||||
lines = await _lines(ops)
|
||||
assert "/mnt/notes.txt" in lines
|
||||
assert "/mnt/table1/rows.jsonl" in lines
|
||||
assert "/mnt/table1" in lines
|
||||
# no hint: every child entry is stat'ed to classify it
|
||||
assert len(stat_calls) > 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_is_dir_name_hint_skips_child_stats():
|
||||
stat_calls: list[str] = []
|
||||
ops = _ops(stat_calls,
|
||||
is_dir_name=lambda _a, name: name.rstrip("/") in DIRS)
|
||||
lines = await _lines(ops)
|
||||
assert "/mnt/notes.txt" in lines
|
||||
assert "/mnt/table1/rows.jsonl" in lines
|
||||
# hint answers directory-ness by name; only the search root is stat'ed
|
||||
assert stat_calls == ["/mnt"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_walk_honors_multiple_start_points():
|
||||
stat_calls: list[str] = []
|
||||
ops = _ops(stat_calls)
|
||||
roots = [
|
||||
PathSpec(virtual="/mnt/table1",
|
||||
directory="/mnt/table1",
|
||||
resolved=False,
|
||||
resource_path="table1"),
|
||||
PathSpec(virtual="/mnt/notes.txt",
|
||||
directory="/mnt",
|
||||
resolved=False,
|
||||
resource_path="notes.txt"),
|
||||
]
|
||||
stdout, _io = await find(ops, None, roots)
|
||||
data = stdout if isinstance(stdout, bytes) else b""
|
||||
lines = data.decode().splitlines()
|
||||
# GNU find walks every start point in operand order
|
||||
assert "/mnt/table1/rows.jsonl" in lines
|
||||
assert "/mnt/notes.txt" in lines
|
||||
assert lines.index("/mnt/table1/rows.jsonl") < lines.index(
|
||||
"/mnt/notes.txt")
|
||||
@@ -0,0 +1,105 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.accessor.base import NOOPAccessor
|
||||
from mirage.commands.builtin.generic_bind.adapter import make_resolve_glob
|
||||
from mirage.types import PathSpec
|
||||
|
||||
TREE = {
|
||||
"/notion/pages": [
|
||||
"/notion/pages/Demo_page__uuid1",
|
||||
"/notion/pages/Roadmap__uuid2",
|
||||
],
|
||||
"/notion/pages/Demo_page__uuid1": [
|
||||
"/notion/pages/Demo_page__uuid1/page.md",
|
||||
"/notion/pages/Demo_page__uuid1/page.json",
|
||||
],
|
||||
"/notion/pages/Roadmap__uuid2": [
|
||||
"/notion/pages/Roadmap__uuid2/page.json",
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
async def fake_readdir(accessor, path, index=None):
|
||||
key = path.virtual.rstrip("/") or "/"
|
||||
if key not in TREE:
|
||||
raise FileNotFoundError(key)
|
||||
return TREE[key]
|
||||
|
||||
|
||||
def glob_spec(virtual: str, prefix: str) -> PathSpec:
|
||||
last_slash = virtual.rfind("/")
|
||||
return PathSpec(
|
||||
virtual=virtual,
|
||||
directory=virtual[:last_slash + 1],
|
||||
resource_path=virtual[len(prefix):].strip("/"),
|
||||
pattern=virtual[last_slash + 1:],
|
||||
resolved=False,
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_glob_mid_path_pattern():
|
||||
resolve = make_resolve_glob(fake_readdir)
|
||||
spec = glob_spec("/notion/pages/Demo_page__*/page.md", "/notion")
|
||||
result = await resolve(NOOPAccessor(), [spec], None)
|
||||
assert [p.virtual
|
||||
for p in result] == ["/notion/pages/Demo_page__uuid1/page.md"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_glob_last_component():
|
||||
resolve = make_resolve_glob(fake_readdir)
|
||||
spec = glob_spec("/notion/pages/Demo*", "/notion")
|
||||
result = await resolve(NOOPAccessor(), [spec], None)
|
||||
assert [p.virtual for p in result] == ["/notion/pages/Demo_page__uuid1"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_glob_passthrough():
|
||||
resolve = make_resolve_glob(fake_readdir)
|
||||
resolved_spec = PathSpec.from_str_path("/notion/pages/Roadmap__uuid2",
|
||||
"pages/Roadmap__uuid2")
|
||||
result = await resolve(NOOPAccessor(), ["text", resolved_spec], None)
|
||||
assert result[0].virtual == "text"
|
||||
assert result[1] is resolved_spec
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_glob_truncates(caplog):
|
||||
resolve = make_resolve_glob(fake_readdir, max_glob_matches=1)
|
||||
spec = glob_spec("/notion/pages/*", "/notion")
|
||||
result = await resolve(NOOPAccessor(), [spec], None)
|
||||
assert len(result) == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_glob_zero_match_word_keeps_literal():
|
||||
resolve = make_resolve_glob(fake_readdir)
|
||||
spec = glob_spec("/notion/pages/*.nope", "/notion")
|
||||
out = await resolve(NOOPAccessor(), [spec], None)
|
||||
assert len(out) == 1
|
||||
assert out[0].virtual == "/notion/pages/*.nope"
|
||||
assert out[0].pattern is None
|
||||
assert out[0].resolved
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_resolve_glob_zero_match_dir_shape_stays_empty():
|
||||
resolve = make_resolve_glob(fake_readdir)
|
||||
spec = glob_spec("/notion/pages/*.nope", "/notion").dir
|
||||
out = await resolve(NOOPAccessor(), [spec], None)
|
||||
assert out == []
|
||||
@@ -0,0 +1,123 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.cache.context import push_cache_manager
|
||||
from mirage.cache.file.ram import RAMFileCacheStore
|
||||
from mirage.cache.manager import CacheManager
|
||||
from mirage.commands.builtin.generic_bind.adapter import CommandIO
|
||||
from mirage.commands.builtin.generic_bind.factory import with_read_cache
|
||||
from mirage.types import PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
|
||||
class _CountingBackend:
|
||||
|
||||
def __init__(self, data: bytes) -> None:
|
||||
self.data = data
|
||||
self.stream_calls = 0
|
||||
self.bytes_calls = 0
|
||||
|
||||
async def read_stream(self, accessor, path, *args, **kwargs):
|
||||
self.stream_calls += 1
|
||||
yield self.data
|
||||
|
||||
async def read_bytes(self, accessor, path, *args, **kwargs) -> bytes:
|
||||
self.bytes_calls += 1
|
||||
return self.data
|
||||
|
||||
|
||||
async def _noop_readdir(accessor, path, index=None) -> list[str]:
|
||||
return []
|
||||
|
||||
|
||||
async def _noop_stat(accessor, path, index=None):
|
||||
return None
|
||||
|
||||
|
||||
def _ops(backend: _CountingBackend) -> CommandIO:
|
||||
return CommandIO(
|
||||
readdir=_noop_readdir,
|
||||
read_bytes=backend.read_bytes,
|
||||
read_stream=backend.read_stream,
|
||||
stat=_noop_stat,
|
||||
is_mounted=lambda a: True,
|
||||
local=False,
|
||||
)
|
||||
|
||||
|
||||
def _spec() -> PathSpec:
|
||||
return PathSpec(resource_path=mount_key("/s3/a.txt", "/s3/"),
|
||||
virtual="/s3/a.txt",
|
||||
directory="/s3/")
|
||||
|
||||
|
||||
async def _drain(source) -> bytes:
|
||||
return b"".join([c async for c in source])
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_warm_read_stream_serves_cache_without_backend():
|
||||
backend = _CountingBackend(b"payload")
|
||||
cache = RAMFileCacheStore()
|
||||
await cache.set("/s3/a.txt", b"payload")
|
||||
manager = CacheManager(cache, None, "/s3/", True)
|
||||
ops = with_read_cache(_ops(backend))
|
||||
prev = push_cache_manager(manager)
|
||||
try:
|
||||
out = await _drain(ops.read_stream(None, _spec()))
|
||||
finally:
|
||||
push_cache_manager(prev)
|
||||
assert out == b"payload"
|
||||
assert backend.stream_calls == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_warm_read_bytes_serves_cache_without_backend():
|
||||
backend = _CountingBackend(b"payload")
|
||||
cache = RAMFileCacheStore()
|
||||
await cache.set("/s3/a.txt", b"payload")
|
||||
manager = CacheManager(cache, None, "/s3/", True)
|
||||
ops = with_read_cache(_ops(backend))
|
||||
prev = push_cache_manager(manager)
|
||||
try:
|
||||
out = await ops.read_bytes(None, _spec())
|
||||
finally:
|
||||
push_cache_manager(prev)
|
||||
assert out == b"payload"
|
||||
assert backend.bytes_calls == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cold_read_falls_through_to_backend():
|
||||
backend = _CountingBackend(b"payload")
|
||||
manager = CacheManager(RAMFileCacheStore(), None, "/s3/", True)
|
||||
ops = with_read_cache(_ops(backend))
|
||||
prev = push_cache_manager(manager)
|
||||
try:
|
||||
out = await _drain(ops.read_stream(None, _spec()))
|
||||
finally:
|
||||
push_cache_manager(prev)
|
||||
assert out == b"payload"
|
||||
assert backend.stream_calls == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_no_manager_falls_through_to_backend():
|
||||
backend = _CountingBackend(b"payload")
|
||||
ops = with_read_cache(_ops(backend))
|
||||
out = await _drain(ops.read_stream(None, _spec()))
|
||||
assert out == b"payload"
|
||||
assert backend.stream_calls == 1
|
||||
@@ -0,0 +1,297 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.cache.index import IndexEntry
|
||||
from mirage.cache.index.ram import RAMIndexCacheStore
|
||||
from mirage.commands.builtin.generic_bind.provision import (
|
||||
default_provision, exact_zero_provision, index_hit_read_provision,
|
||||
make_copy_provision, make_file_read_provision, make_head_tail_provision,
|
||||
make_search_provision, make_transform_provision, metadata_provision,
|
||||
pure_provision, write_metadata_provision)
|
||||
from mirage.commands.builtin.ram import COMMANDS as RAM_COMMANDS
|
||||
from mirage.provision import Precision
|
||||
from mirage.types import FileStat, FileType, PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
|
||||
SIZES = {
|
||||
"/data/known.txt": 5,
|
||||
"/data/big.txt": 100,
|
||||
"/data/tree/a.txt": 7,
|
||||
"/data/tree/skip.parquet": 900,
|
||||
"/data/tree/sub/b.txt": 11,
|
||||
}
|
||||
DIRS = {"/data/tree", "/data/tree/sub"}
|
||||
TREE = {
|
||||
"/data/tree":
|
||||
["/data/tree/a.txt", "/data/tree/skip.parquet", "/data/tree/sub"],
|
||||
"/data/tree/sub": ["/data/tree/sub/b.txt"],
|
||||
}
|
||||
|
||||
|
||||
def _spec(path: str) -> PathSpec:
|
||||
return PathSpec(resource_path=mount_key(path, "/data"),
|
||||
virtual=path,
|
||||
directory=path)
|
||||
|
||||
|
||||
async def _stat(accessor, path, index=None) -> FileStat:
|
||||
virtual = path.virtual if isinstance(path, PathSpec) else path
|
||||
if virtual in DIRS:
|
||||
return FileStat(name=virtual.rsplit("/", 1)[-1],
|
||||
type=FileType.DIRECTORY)
|
||||
return FileStat(name=virtual.rsplit("/", 1)[-1],
|
||||
size=SIZES.get(virtual),
|
||||
type=FileType.TEXT)
|
||||
|
||||
|
||||
async def _readdir(accessor, path, index=None) -> list[str]:
|
||||
virtual = path.virtual if isinstance(path, PathSpec) else path
|
||||
return TREE[virtual]
|
||||
|
||||
|
||||
async def _resolve_glob(accessor, paths, index=None) -> list[PathSpec]:
|
||||
out = []
|
||||
for p in paths:
|
||||
if p.pattern:
|
||||
out.extend(
|
||||
_spec(e) for e in TREE.get(p.directory.rstrip("/"), [])
|
||||
if e.endswith(p.pattern.lstrip("*")))
|
||||
else:
|
||||
out.append(p)
|
||||
return out
|
||||
|
||||
|
||||
def _registered(name: str):
|
||||
for fn in RAM_COMMANDS:
|
||||
for rc in getattr(fn, "_registered_commands", []):
|
||||
if rc.name == name and rc.filetype is None:
|
||||
return rc
|
||||
return None
|
||||
|
||||
|
||||
def test_default_provision_families():
|
||||
assert default_provision("sort", _stat) is not None
|
||||
assert default_provision("grep", _stat) is not None
|
||||
assert default_provision("iconv", _stat) is not None
|
||||
assert default_provision("file", _stat) is not None
|
||||
assert default_provision("ls", _stat) is metadata_provision
|
||||
assert default_provision("stat", _stat) is metadata_provision
|
||||
assert default_provision("du", _stat) is metadata_provision
|
||||
assert default_provision("gzip", _stat) is not None
|
||||
assert default_provision("cp", _stat) is not None
|
||||
assert default_provision("rm", _stat) is write_metadata_provision
|
||||
assert default_provision("mv", _stat) is None
|
||||
assert default_provision("tee", _stat) is None
|
||||
|
||||
|
||||
def test_factory_registers_default_provisions():
|
||||
for name in ("grep", "sort", "ls", "find", "md5", "cp", "gzip", "rm"):
|
||||
rc = _registered(name)
|
||||
assert rc is not None, name
|
||||
assert rc.provision_fn is not None, name
|
||||
rc = _registered("tee")
|
||||
assert rc is not None
|
||||
assert rc.provision_fn is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_read_known_sizes_exact():
|
||||
provision = make_file_read_provision(_stat)
|
||||
result = await provision(
|
||||
None, [_spec("/data/known.txt"),
|
||||
_spec("/data/big.txt")],
|
||||
command="cat")
|
||||
assert result.precision == Precision.EXACT
|
||||
assert result.network_read_low == 105
|
||||
assert result.network_read_high == 105
|
||||
assert result.read_ops == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_read_missing_size_keeps_floor():
|
||||
provision = make_file_read_provision(_stat)
|
||||
result = await provision(
|
||||
None, [_spec("/data/known.txt"),
|
||||
_spec("/data/chat.jsonl")],
|
||||
command="cat")
|
||||
assert result.precision == Precision.UNKNOWN
|
||||
assert result.network_read_low == 5
|
||||
assert result.network_read_high == 5
|
||||
assert result.read_ops == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_head_tail_missing_size_keeps_known_ceiling():
|
||||
provision = make_head_tail_provision(_stat)
|
||||
result = await provision(
|
||||
None, [_spec("/data/known.txt"),
|
||||
_spec("/data/chat.jsonl")],
|
||||
command="head")
|
||||
assert result.precision == Precision.UNKNOWN
|
||||
assert result.network_read_low == 0
|
||||
assert result.network_read_high == 5
|
||||
assert result.read_ops == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_transform_keeps_read_floor_unknown_output():
|
||||
provision = make_transform_provision(_stat)
|
||||
result = await provision(None, [_spec("/data/known.txt")], command="gzip")
|
||||
assert result.precision == Precision.UNKNOWN
|
||||
assert result.network_read_low == 5
|
||||
assert result.network_read_high == 5
|
||||
assert result.read_ops == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_copy_brackets_read_and_write():
|
||||
provision = make_copy_provision(_stat)
|
||||
result = await provision(
|
||||
None, [_spec("/data/known.txt"),
|
||||
_spec("/data/dest.txt")],
|
||||
command="cp")
|
||||
assert result.precision == Precision.RANGE
|
||||
assert result.network_read_low == 0
|
||||
assert result.network_read_high == 5
|
||||
assert result.network_write_low == 0
|
||||
assert result.network_write_high == 5
|
||||
assert result.read_ops == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_write_metadata_zero_bytes_recursive_floors():
|
||||
result = await write_metadata_provision(None, [_spec("/data/known.txt")],
|
||||
command="rm")
|
||||
assert result.precision == Precision.EXACT
|
||||
assert result.network_read_high == 0
|
||||
assert result.read_ops == 1
|
||||
recursive = await write_metadata_provision(None,
|
||||
[_spec("/data/known.txt")],
|
||||
command="rm",
|
||||
r=True)
|
||||
assert recursive.precision == Precision.UNKNOWN
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_pure_provision_zero_exact():
|
||||
result = await pure_provision(command="seq 3")
|
||||
assert result.precision == Precision.EXACT
|
||||
assert result.network_read_high == 0
|
||||
assert result.read_ops == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_read_expands_globs():
|
||||
provision = make_file_read_provision(_stat, _resolve_glob)
|
||||
pattern = PathSpec(virtual="/data/tree/*.txt",
|
||||
directory="/data/tree/",
|
||||
pattern="*.txt",
|
||||
resource_path="tree/*.txt")
|
||||
result = await provision(None, [pattern], command="cat")
|
||||
assert result.precision == Precision.EXACT
|
||||
assert result.network_read_high == 7
|
||||
assert result.read_ops == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_read_unmatched_glob_unknown():
|
||||
provision = make_file_read_provision(_stat, _resolve_glob)
|
||||
pattern = PathSpec(virtual="/data/tree/*.nope",
|
||||
directory="/data/tree/",
|
||||
pattern="*.nope",
|
||||
resource_path="tree/*.nope")
|
||||
result = await provision(None, [pattern], command="cat")
|
||||
assert result.precision == Precision.UNKNOWN
|
||||
assert result.network_read_high == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_recursive_walks_tree_exact():
|
||||
provision = make_search_provision(_stat, _resolve_glob, _readdir)
|
||||
result = await provision(None, [_spec("/data/tree")],
|
||||
"x",
|
||||
command="grep",
|
||||
r=True)
|
||||
assert result.precision == Precision.EXACT
|
||||
assert result.network_read_high == 18
|
||||
assert result.read_ops == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_recursive_skips_columnar():
|
||||
provision = make_search_provision(_stat, _resolve_glob, _readdir)
|
||||
result = await provision(None, [_spec("/data/tree")],
|
||||
"x",
|
||||
command="grep",
|
||||
r=True)
|
||||
assert result.network_read_high != 918
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_without_readdir_keeps_floor():
|
||||
provision = make_search_provision(_stat, _resolve_glob)
|
||||
result = await provision(None, [_spec("/data/tree")],
|
||||
"x",
|
||||
command="grep",
|
||||
r=True)
|
||||
assert result.precision == Precision.UNKNOWN
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_search_recursive_cap_degrades(monkeypatch):
|
||||
monkeypatch.setattr(
|
||||
"mirage.commands.builtin.generic_bind.provision.MAX_PLAN_WALK", 2)
|
||||
provision = make_search_provision(_stat, _resolve_glob, _readdir)
|
||||
result = await provision(None, [_spec("/data/tree")],
|
||||
"x",
|
||||
command="grep",
|
||||
r=True)
|
||||
assert result.precision == Precision.UNKNOWN
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_exact_zero_provision_charges_nothing():
|
||||
result = await exact_zero_provision("find /chat")
|
||||
assert result.command == "find /chat"
|
||||
assert result.network_read_low == 0
|
||||
assert result.network_read_high == 0
|
||||
assert result.read_ops == 0
|
||||
assert result.precision == Precision.EXACT
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_index_hit_read_provision_counts_cached_operands():
|
||||
index = RAMIndexCacheStore()
|
||||
await index.put("/chat/a.jsonl",
|
||||
IndexEntry(id="a", name="a.jsonl", resource_type="file"))
|
||||
paths = [
|
||||
PathSpec(virtual="/chat/a.jsonl",
|
||||
directory="/chat",
|
||||
resource_path="a.jsonl"),
|
||||
PathSpec(virtual="/chat/missing.jsonl",
|
||||
directory="/chat",
|
||||
resource_path="missing.jsonl"),
|
||||
]
|
||||
result = await index_hit_read_provision(None, paths, "cat", index)
|
||||
assert result.read_ops == 1
|
||||
assert result.network_read_low == 0
|
||||
assert result.precision == Precision.EXACT
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_index_hit_read_provision_without_paths_is_unknown():
|
||||
result = await index_hit_read_provision(None, [], "grep x", None)
|
||||
assert result.precision == Precision.UNKNOWN
|
||||
@@ -0,0 +1,94 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
from functools import partial
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.generic_bind.adapter import CommandIO
|
||||
from mirage.commands.builtin.generic_bind.builders.du import du as du_builder
|
||||
from mirage.commands.builtin.generic_bind.builders.find import \
|
||||
find as find_builder
|
||||
from mirage.commands.builtin.utils.wrap import stream_from_bytes
|
||||
from mirage.io.types import materialize
|
||||
from mirage.types import FileStat, FileType, PathSpec
|
||||
|
||||
_FILES = {
|
||||
"/g/a.txt": b"alpha\n",
|
||||
"/g/sub/b.txt": b"bravo\n",
|
||||
}
|
||||
_DIRS = {"/g", "/g/sub"}
|
||||
|
||||
|
||||
async def _readdir(accessor, path, index=None):
|
||||
base = (path.virtual if isinstance(path, PathSpec) else path).rstrip("/")
|
||||
out = []
|
||||
for p in list(_FILES) + sorted(_DIRS):
|
||||
parent = p.rsplit("/", 1)[0] or "/"
|
||||
if parent == base and p != base:
|
||||
out.append(p)
|
||||
return out
|
||||
|
||||
|
||||
async def _stat(accessor, path, index=None):
|
||||
p = path.virtual if isinstance(path, PathSpec) else path
|
||||
p = p.rstrip("/") or "/"
|
||||
if p in _DIRS:
|
||||
return FileStat(name=p.rsplit("/", 1)[-1], type=FileType.DIRECTORY)
|
||||
if p in _FILES:
|
||||
return FileStat(name=p.rsplit("/", 1)[-1],
|
||||
size=len(_FILES[p]),
|
||||
type=FileType.TEXT)
|
||||
raise FileNotFoundError(p)
|
||||
|
||||
|
||||
async def _read(accessor, path, index=None):
|
||||
p = path.virtual if isinstance(path, PathSpec) else path
|
||||
return _FILES[p.rstrip("/")]
|
||||
|
||||
|
||||
def _ops() -> CommandIO:
|
||||
return CommandIO(
|
||||
readdir=_readdir,
|
||||
read_bytes=_read,
|
||||
read_stream=partial(stream_from_bytes, _read),
|
||||
stat=_stat,
|
||||
is_mounted=lambda a: True,
|
||||
local=False,
|
||||
)
|
||||
|
||||
|
||||
def _spec(original: str) -> PathSpec:
|
||||
return PathSpec(resource_path=(original).strip("/"),
|
||||
virtual=original,
|
||||
directory=original,
|
||||
resolved=True)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_find_walks_tree_without_native_find_op():
|
||||
out, io = await find_builder(_ops(), object(), [_spec("/g")], type="f")
|
||||
text = (await materialize(out)).decode()
|
||||
assert "/g/a.txt" in text
|
||||
assert "/g/sub/b.txt" in text
|
||||
assert io.exit_code == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_du_walks_tree_without_native_du_op():
|
||||
out, _ = await du_builder(_ops(), object(), [_spec("/g")], s=True)
|
||||
text = (await materialize(out)).decode()
|
||||
# alpha\n (6) + bravo\n (6) = 12 bytes summed by the readdir walk.
|
||||
assert "12" in text
|
||||
assert "/g" in text
|
||||
@@ -0,0 +1,13 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
@@ -0,0 +1,35 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.resource.github.github import GitHubResource
|
||||
from tests.fixtures.github_mock import github_config, mock_github_api
|
||||
|
||||
__all__ = ["github_config", "mock_github_api"]
|
||||
|
||||
OWNER = "test-owner"
|
||||
REPO = "test-repo"
|
||||
REF = "main"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def github_env(mock_github_api, github_config):
|
||||
resource = GitHubResource(
|
||||
config=github_config,
|
||||
owner=OWNER,
|
||||
repo=REPO,
|
||||
ref=REF,
|
||||
)
|
||||
return resource.accessor, resource._index
|
||||
@@ -0,0 +1,166 @@
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
|
||||
|
||||
import pytest
|
||||
|
||||
from mirage.commands.builtin.github.grep import grep
|
||||
from mirage.io.stream import materialize
|
||||
from mirage.types import PathSpec
|
||||
from mirage.utils.key_prefix import mount_key
|
||||
from tests.fixtures.github_mock import MOCK_BLOBS
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _patch_read(monkeypatch):
|
||||
|
||||
async def _read_bytes(config, owner, repo, sha):
|
||||
return MOCK_BLOBS[sha]
|
||||
|
||||
monkeypatch.setattr("mirage.core.github.read.read_bytes", _read_bytes)
|
||||
|
||||
|
||||
def _scope(path: str, resolved: bool = True) -> PathSpec:
|
||||
norm = "/" + path.lstrip("/")
|
||||
directory = norm.rsplit("/", 1)[0] + "/"
|
||||
return PathSpec(resource_path=(norm).strip("/"),
|
||||
virtual=norm,
|
||||
directory=directory,
|
||||
resolved=resolved)
|
||||
|
||||
|
||||
async def _run(accessor, index, paths, pattern, **kwargs):
|
||||
scopes = [_scope(p, resolved=("." in p.split("/")[-1])) for p in paths]
|
||||
stdout, io = await grep(accessor, scopes, pattern, index=index, **kwargs)
|
||||
data = await materialize(stdout)
|
||||
return data.decode(errors="replace"), io
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_single_file_grep(mock_github_api, github_env):
|
||||
accessor, index = github_env
|
||||
text, io = await _run(accessor, index, ["src/main.py"], "import")
|
||||
assert io.exit_code == 0
|
||||
lines = text.strip().splitlines()
|
||||
assert len(lines) == 3
|
||||
assert "import os" in lines[0]
|
||||
assert "import sys" in lines[1]
|
||||
assert "import helper" in lines[2]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_with_line_numbers(mock_github_api, github_env):
|
||||
accessor, index = github_env
|
||||
text, io = await _run(accessor, index, ["src/main.py"], "import", n=True)
|
||||
assert io.exit_code == 0
|
||||
lines = text.strip().splitlines()
|
||||
assert lines[0].startswith("1:")
|
||||
assert lines[1].startswith("2:")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_case_insensitive(mock_github_api, github_env):
|
||||
accessor, index = github_env
|
||||
text, io = await _run(accessor, index, ["src/main.py"], "IMPORT", i=True)
|
||||
assert io.exit_code == 0
|
||||
lines = text.strip().splitlines()
|
||||
assert len(lines) >= 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_invert(mock_github_api, github_env):
|
||||
accessor, index = github_env
|
||||
text, io = await _run(accessor, index, ["src/utils.py"], "import", v=True)
|
||||
assert io.exit_code == 0
|
||||
lines = text.strip().splitlines()
|
||||
for line in lines:
|
||||
assert "import" not in line
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_count(mock_github_api, github_env):
|
||||
accessor, index = github_env
|
||||
text, io = await _run(accessor, index, ["src/main.py"], "import", c=True)
|
||||
assert io.exit_code == 0
|
||||
assert text.strip() == "3"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_recursive_grep(mock_github_api, github_env):
|
||||
accessor, index = github_env
|
||||
text, io = await _run(accessor, index, ["src"], "dataclass", r=True)
|
||||
assert io.exit_code == 0
|
||||
lines = text.strip().splitlines()
|
||||
assert len(lines) >= 2
|
||||
paths_found = [line.split(":")[0] for line in lines]
|
||||
assert any("user.py" in p for p in paths_found)
|
||||
assert any("item.py" in p for p in paths_found)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_no_match(mock_github_api, github_env):
|
||||
accessor, index = github_env
|
||||
text, io = await _run(accessor, index, ["src/main.py"], "zzz_no_match_zzz")
|
||||
assert io.exit_code == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_files_only(mock_github_api, github_env):
|
||||
accessor, index = github_env
|
||||
text, io = await _run(accessor,
|
||||
index, ["src"],
|
||||
"dataclass",
|
||||
r=True,
|
||||
args_l=True)
|
||||
assert io.exit_code == 0
|
||||
lines = text.strip().splitlines()
|
||||
assert any("user.py" in ln for ln in lines)
|
||||
assert any("item.py" in ln for ln in lines)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_stdin(github_env):
|
||||
accessor, index = github_env
|
||||
stdin_data = b"hello world\nfoo bar\nhello again\n"
|
||||
stdout, io = await grep(accessor, [],
|
||||
"hello",
|
||||
stdin=stdin_data,
|
||||
index=index)
|
||||
data = await materialize(stdout)
|
||||
text = data.decode(errors="replace")
|
||||
lines = text.strip().splitlines()
|
||||
assert len(lines) == 2
|
||||
assert "hello world" in lines[0]
|
||||
assert "hello again" in lines[1]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_grep_files_only_with_prefix(mock_github_api, github_env):
|
||||
accessor, index = github_env
|
||||
scopes = [
|
||||
PathSpec(resource_path=mount_key("/gh/src", "/gh"),
|
||||
virtual="/gh/src",
|
||||
directory="/gh/src/",
|
||||
resolved=False)
|
||||
]
|
||||
stdout, io = await grep(accessor,
|
||||
scopes,
|
||||
"dataclass",
|
||||
r=True,
|
||||
args_l=True,
|
||||
index=index)
|
||||
data = await materialize(stdout)
|
||||
text = data.decode()
|
||||
assert io.exit_code == 0
|
||||
lines = text.strip().splitlines()
|
||||
assert any("user.py" in ln for ln in lines)
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user