chore: import upstream snapshot with attribution
Integ / changes (push) Has been skipped
Pre-commit / pre-commit (push) Failing after 1s
CLI exit codes / changes (push) Has been skipped
Test (Install) / changes (push) Has been skipped
Test (Python) / changes (push) Has been skipped
Test (TypeScript) / changes (push) Has been skipped
CLI exit codes / cli-gate (push) Has been cancelled
Test (Install) / test-install-gate (push) Has been cancelled
Integ / integ-gate (push) Has been cancelled
Test (Python) / test-python-gate (push) Has been cancelled
Test (TypeScript) / test-typescript-gate (push) Has been cancelled
Test (Install) / python-minimal (3.12) (push) Has been cancelled
Test (Install) / python-minimal (3.11) (push) Has been cancelled
Test (Install) / python-extra (agno, mirage.agents.agno) (push) Has been cancelled
Test (Install) / python-extra (chroma, mirage.resource.chroma) (push) Has been cancelled
Test (Install) / python-extra (pdf, mirage.core.filetype.pdf) (push) Has been cancelled
Integ / integ (push) Has been cancelled
Integ / integ-database (push) Has been cancelled
Integ / integ-database-ts (push) Has been cancelled
Integ / integ-data (push) Has been cancelled
Integ / integ-ssh (push) Has been cancelled
Integ / integ-ssh-ts (push) Has been cancelled
Test (Python) / audit (push) Has been cancelled
Test (TypeScript) / test (push) Has been cancelled
Test (TypeScript) / python-fs-shim (push) Has been cancelled
CLI exit codes / Python CLI (push) Has been cancelled
CLI exit codes / TypeScript CLI (push) Has been cancelled
CLI exit codes / Cross-language snapshot interop (push) Has been cancelled
Test (Python) / test (push) Has been cancelled
Test (Python) / import-isolation (deepagents, openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Python) / import-isolation (deepagents, pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Integ / integ-ts (push) Has been cancelled
Integ / integ-fuse (push) Has been cancelled
Test (Install) / python-extra (databricks, mirage.resource.databricks_volume) (push) Has been cancelled
Test (Install) / python-extra (deepagents, mirage.agents.langchain) (push) Has been cancelled
Test (Install) / python-extra (email, mirage.resource.email) (push) Has been cancelled
Test (Install) / python-extra (fuse, mirage.fuse.mount) (push) Has been cancelled
Test (Install) / python-extra (hdf5, mirage.core.filetype.hdf5) (push) Has been cancelled
Test (Install) / python-extra (hf, mirage.resource.hf_buckets) (push) Has been cancelled
Test (Install) / python-extra (lancedb, mirage.resource.lancedb) (push) Has been cancelled
Test (Install) / python-extra (langfuse, mirage.resource.langfuse) (push) Has been cancelled
Test (Install) / python-extra (mongodb, mirage.resource.mongodb) (push) Has been cancelled
Test (Install) / python-extra (nextcloud, mirage.resource.nextcloud) (push) Has been cancelled
Test (Install) / python-extra (openai, mirage.agents.openai_agents) (push) Has been cancelled
Test (Install) / python-extra (openhands, mirage.agents.openhands, 3.12) (push) Has been cancelled
Test (Install) / python-extra (parquet, mirage.core.filetype.parquet) (push) Has been cancelled
Test (Install) / python-extra (postgres, mirage.resource.postgres) (push) Has been cancelled
Test (Install) / python-extra (pydantic-ai, mirage.agents.pydantic_ai) (push) Has been cancelled
Test (Install) / python-extra (qdrant, mirage.resource.qdrant) (push) Has been cancelled
Test (Install) / python-extra (redis, mirage.resource.redis) (push) Has been cancelled
Test (Install) / python-extra (s3, mirage.resource.s3) (push) Has been cancelled
Test (Install) / python-extra (ssh, mirage.resource.ssh) (push) Has been cancelled
Test (Install) / ts-minimal (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 12:30:44 +08:00
commit bcbd1bdb22
5748 changed files with 562488 additions and 0 deletions
@@ -0,0 +1,13 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
@@ -0,0 +1,20 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.provision import Precision, ProvisionResult
async def handle_builtin_provision() -> ProvisionResult:
"""Plan for shell builtins (cd, export, etc.): zero cost."""
return ProvisionResult(precision=Precision.EXACT)
@@ -0,0 +1,191 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import dataclasses
from mirage.cache.file.mixin import FileCacheMixin
from mirage.commands.builtin.generic.crossmount import is_cross_mount
from mirage.commands.resolve import get_extension
from mirage.commands.spec import parse_command, parse_to_kwargs
from mirage.provision import Precision, ProvisionResult, combine_sum
from mirage.types import PathSpec
from mirage.utils.key_prefix import mount_key
from mirage.workspace.mount import MountRegistry
from mirage.workspace.mount.namespace import Namespace
from mirage.workspace.session import Session
async def _check_cache_hits(
cache: FileCacheMixin | None,
parts: list[str | PathSpec],
) -> int:
"""Count how many path args are already cached."""
if cache is None:
return 0
hits = 0
for p in parts[1:]:
if isinstance(p, PathSpec) and await cache.exists(p.virtual):
hits += 1
return hits
def _mount_groups(registry: MountRegistry,
parts: list[str | PathSpec]) -> list[list[PathSpec]]:
"""Group path args by their own mount, in first-appearance order.
Args that resolve to no mount (glob patterns, expression operands)
ride with the first group: they scoped to the primary mount before
and must not fabricate a cross-mount split.
"""
groups: list[list[PathSpec]] = []
seen: dict[str, int] = {}
unresolved: list[PathSpec] = []
for p in parts[1:]:
if not isinstance(p, PathSpec):
continue
if p.pattern:
# Globs are not expanded during planning; a pattern operand
# (find -name, ls *.txt) must not fabricate a mount group.
unresolved.append(p)
continue
try:
prefix = registry.mount_for(p.virtual).prefix
except ValueError:
unresolved.append(p)
continue
idx = seen.get(prefix)
if idx is None:
seen[prefix] = len(groups)
groups.append([p])
else:
groups[idx].append(p)
if unresolved:
if groups:
groups[0].extend(unresolved)
else:
groups.append(unresolved)
return groups
async def handle_command_provision(
registry: MountRegistry,
parts: list[str | PathSpec],
session: Session,
namespace: Namespace | None = None,
) -> ProvisionResult:
"""Estimate cost of a simple command.
Paths are namespace-followed first (a symlinked read costs its
target, and the cache-hit check sees the entry the executor would
actually serve), then grouped by mount: a command spanning mounts
is estimated per mount against each mount's own backend and the
results summed, instead of statting foreign paths against the
first path's backend.
"""
if not parts:
return ProvisionResult(precision=Precision.EXACT)
if namespace is not None:
for i, p in enumerate(parts):
if isinstance(p, PathSpec):
followed = namespace.follow(p.virtual)
if followed != p.virtual:
parts[i] = PathSpec.from_str_path(followed)
cmd_name = str(parts[0])
cmd_str = " ".join(p.virtual if isinstance(p, PathSpec) else p
for p in parts)
groups = _mount_groups(registry, parts)
if len(groups) > 1:
path_parts = [p for p in parts[1:] if isinstance(p, PathSpec)]
if not is_cross_mount(cmd_name, path_parts, registry):
# The executor rejects this command across mounts, so an
# aggregated byte estimate would cost a run that errors.
return ProvisionResult(command=cmd_str,
precision=Precision.UNKNOWN)
texts = [p for p in parts[1:] if not isinstance(p, PathSpec)]
children = []
for group in groups:
sub: list[str | PathSpec] = [cmd_name, *texts, *group]
children.append(await
handle_command_provision(registry, sub, session))
combined = combine_sum(";", children)
combined.command = cmd_str
return combined
first_scope = None
for p in parts[1:]:
if isinstance(p, PathSpec):
first_scope = p
break
mount_path = first_scope.virtual if first_scope else session.cwd
try:
mount = registry.mount_for(mount_path)
except ValueError:
# Pathless commands (seq, date, ...) still need a mount to
# resolve their registration; any mount carries the general
# commands, so fall back to the first one.
mounts = registry.mounts()
if not mounts:
return ProvisionResult(command=cmd_str,
precision=Precision.UNKNOWN)
mount = mounts[0]
extension = get_extension(first_scope.virtual) if first_scope else None
cmd = mount.resolve_command(cmd_name, extension)
if cmd is None or cmd.provision_fn is None:
return ProvisionResult(command=cmd_str, precision=Precision.UNKNOWN)
mount_prefix = mount.prefix.rstrip("/")
resource_scopes = []
for i, p in enumerate(parts[1:], start=1):
if isinstance(p, PathSpec):
scoped = dataclasses.replace(p,
resource_path=mount_key(
p.virtual, mount_prefix))
parts[i] = scoped
resource_scopes.append(scoped)
# Parse flags so plan functions receive them as kwargs (e.g. r=True)
argv = [p.virtual if isinstance(p, PathSpec) else p for p in parts[1:]]
spec = mount.spec_for(cmd_name)
if spec is not None:
parsed = parse_command(spec, argv, cwd=session.cwd)
flag_kwargs = parse_to_kwargs(parsed)
text_args = parsed.texts()
else:
flag_kwargs = {}
text_args = [p for p in parts[1:] if not isinstance(p, PathSpec)]
result = await cmd.provision_fn(mount.resource.accessor,
resource_scopes,
*text_args,
command=cmd_str,
prefix=mount.prefix.rstrip("/"),
index=mount.resource.index,
**flag_kwargs)
if not result.command:
result.command = cmd_str
hits = await _check_cache_hits(registry.file_cache, parts)
if hits > 0:
result.cache_hits = hits
result.cache_read_low = result.network_read_low
result.cache_read_high = result.network_read_high
result.network_read_low = 0
result.network_read_high = 0
return result
@@ -0,0 +1,110 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from typing import Any
from mirage.provision import Precision, ProvisionResult
from mirage.workspace.provision.rollup import rollup_list
from mirage.workspace.session import Session
async def _plan_body(provision_node_fn, body: list,
session) -> ProvisionResult:
"""Plan a multi-statement body."""
children = []
for cmd in body:
children.append(await provision_node_fn(cmd, session))
if not children:
return ProvisionResult(precision=Precision.EXACT)
if len(children) == 1:
return children[0]
return rollup_list(";", children)
async def handle_function_provision(
provision_node_fn,
name: str,
body: list,
planning: set[str],
session: Session,
) -> ProvisionResult:
"""Plan a shell function call: the body's cost.
Recursive functions would loop the planner, so a function already
being planned reports UNKNOWN instead of recursing again.
Args:
provision_node_fn: recursive planner.
name (str): function name.
body (list): function body statement nodes.
planning (set[str]): names currently being planned (guard).
session (Session): shell session state.
"""
if name in planning:
return ProvisionResult(command=name, precision=Precision.UNKNOWN)
planning.add(name)
try:
result = await _plan_body(provision_node_fn, body, session)
finally:
planning.discard(name)
if not result.command:
result.command = name
return result
async def handle_if_provision(
provision_node_fn,
branches: list[tuple[Any, Any]],
else_body: Any | None,
session: Session,
) -> ProvisionResult:
"""Plan an if: branches bracket as alternatives.
Taking branch i evaluates conditions 1..i plus body i, so each
alternative sums its condition ladder with its body. The else (or,
without one, the fall-through) still pays every condition.
"""
cond_costs: list[ProvisionResult] = []
children = []
for condition, body in branches:
cond_costs.append(await provision_node_fn(condition, session))
body_result = await _plan_body(provision_node_fn, body, session)
children.append(rollup_list(";", cond_costs + [body_result]))
else_result = (await _plan_body(provision_node_fn, else_body, session)
if else_body is not None else ProvisionResult(
precision=Precision.EXACT))
children.append(rollup_list(";", cond_costs + [else_result]))
return rollup_list("||", children)
async def handle_for_provision(
provision_node_fn,
body: list,
n: int,
session: Session,
) -> ProvisionResult:
"""Plan a for loop: body cost x iteration count."""
result = await _plan_body(provision_node_fn, body, session)
return result.scaled(n, command="for")
async def handle_while_provision(
provision_node_fn,
body: list,
session: Session,
) -> ProvisionResult:
"""Plan while: unknown iterations."""
result = await _plan_body(provision_node_fn, body, session)
result.precision = Precision.UNKNOWN
return result
@@ -0,0 +1,45 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from typing import Any
from mirage.provision import ProvisionResult
from mirage.workspace.provision.rollup import rollup_list, rollup_pipe
from mirage.workspace.session import Session
async def handle_pipe_provision(
provision_node_fn,
commands: list[Any],
session: Session,
) -> ProvisionResult:
"""Plan a pipe: all commands run."""
children = []
for cmd in commands:
children.append(await provision_node_fn(cmd, session))
return rollup_pipe(children)
async def handle_connection_provision(
provision_node_fn,
left: Any,
op: str,
right: Any,
session: Session,
) -> ProvisionResult:
"""Plan &&, ||"""
children = []
children.append(await provision_node_fn(left, session))
children.append(await provision_node_fn(right, session))
return rollup_list(str(op), children)
@@ -0,0 +1,72 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from typing import Any
from mirage.provision import Precision, ProvisionResult
from mirage.shell.types import RedirectKind
from mirage.types import PathSpec
from mirage.workspace.mount import MountRegistry
from mirage.workspace.mount.namespace import Namespace
from mirage.workspace.provision.command import handle_command_provision
from mirage.workspace.provision.rollup import rollup_list
from mirage.workspace.session import Session
async def handle_redirect_provision(
provision_node_fn,
registry: MountRegistry,
command: Any,
targets: list[tuple[RedirectKind, PathSpec]],
session: Session,
namespace: Namespace | None = None,
) -> ProvisionResult:
"""Plan a redirect: the inner command plus the redirect I/O.
A `< file` source is read fully, so it is planned as a cat of the
source (exact when the size resolves). A `>`/`>>` target writes
the inner command's stdout, whose size is only knowable when the
inner read total is: the write is bracketed 0..inner read high as
a RANGE, or UNKNOWN when the inner plan has no usable ceiling.
stderr redirects, fd duplications, /dev targets, and heredocs are
filtered out by the caller and cost nothing.
Args:
provision_node_fn: recursive planner.
registry (MountRegistry): mount registry for the source read.
command (Any): the redirected command node.
targets (list[tuple[RedirectKind, PathSpec]]): resolved
stdin/stdout redirect targets on mounts.
session (Session): shell session state.
"""
inner = await provision_node_fn(command, session)
if not targets:
return inner
children = [inner]
for kind, target in targets:
if kind == RedirectKind.STDIN:
children.append(await
handle_command_provision(registry, ["cat", target],
session, namespace))
continue
if inner.network_read_high > 0:
children.append(
ProvisionResult(
network_write_low=0,
network_write_high=inner.network_read_high,
precision=Precision.RANGE,
))
else:
children.append(ProvisionResult(precision=Precision.UNKNOWN))
return rollup_list(";", children)
@@ -0,0 +1,57 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.provision import (Precision, ProvisionResult, combine_alternative,
combine_sum)
def rollup_pipe(children: list[ProvisionResult]) -> ProvisionResult:
"""Aggregate plan results for a pipe (all stages run).
A stage downstream of an UNKNOWN stage cannot be trusted either (its
input volume is unknowable), so its precision is degraded before the
field-wise sum.
Args:
children (list[ProvisionResult]): Per-stage results.
Returns:
ProvisionResult: Field-wise sums under op "|".
"""
unknown_seen = False
for child in children:
if unknown_seen:
child.precision = Precision.UNKNOWN
elif child.precision == Precision.UNKNOWN:
unknown_seen = True
return combine_sum("|", children)
def rollup_list(
op: str,
children: list[ProvisionResult],
) -> ProvisionResult:
"""Aggregate plan results for ;, &&, ||.
Args:
op (str): List operator.
children (list[ProvisionResult]): Per-command results.
Returns:
ProvisionResult: Sums for ;/&& (every command runs); a min/max
envelope for || (only one branch runs).
"""
if op == "||":
return combine_alternative(op, children)
return combine_sum(op, children)