wehub-resource-sync
426e9eeabd
Voice Workbench / headless workbench (mocked backends) (push) Has been cancelled
Voice Workbench / real acoustic lane (nightly, provisioned only) (push) Has been cancelled
ci / test (push) Has been cancelled
ci / lint-and-format (push) Has been cancelled
ci / build (push) Has been cancelled
ci / dev-startup (push) Has been cancelled
gitleaks / gitleaks (push) Has been cancelled
Markdown Links / Relative Markdown Links (push) Has been cancelled
Quality (Extended) / Homepage Build (PR smoke) (push) Has been cancelled
Quality (Extended) / Comment-only diff guard (push) Has been cancelled
Quality (Extended) / Format + Type Safety Ratchet (push) Has been cancelled
Quality (Extended) / Develop Gate (secret scan + UI determinism) (push) Has been cancelled
Quality (Extended) / Develop Gate (lint) (push) Has been cancelled
Chat shell gestures / Chat shell gesture + parity e2e (push) Has been cancelled
Cloud Gateway Discord / Test (push) Has been cancelled
Benchmark Bridge Tests / benchmark (bunx @biomejs/biome check packages/lifeops-bench/src, benchmark-lint) (push) Has been cancelled
Benchmark Bridge Tests / benchmark (bunx vitest run --config packages/lifeops-bench/vitest.config.ts --root packages/lifeops-bench --passWithNoTests, benchmark-tests) (push) Has been cancelled
Build Agent Image / build-and-push (push) Has been cancelled
Dev Smoke / bun run dev onboarding chat (push) Has been cancelled
Dev Smoke / Vite HMR dependency-level smoke (push) Has been cancelled
Electrobun Submodule Guard / electrobun gitlink is fetchable (push) Has been cancelled
Publish @elizaos/example-code / check_npm (push) Has been cancelled
Publish @elizaos/example-code / publish_npm (push) Has been cancelled
Publish @elizaos/plugin-elizacloud / verify_version (push) Has been cancelled
Publish @elizaos/plugin-elizacloud / publish_npm (push) Has been cancelled
Sandbox Live Smoke / Sandbox live smoke (push) Has been cancelled
Snap Build & Test / Build Snap (amd64) (push) Has been cancelled
Snap Build & Test / Build Snap (arm64) (push) Has been cancelled
Test Packaging / elizaos CLI global-install smoke (node + bun) (push) Has been cancelled
Cloud Gateway Webhook / Test (push) Has been cancelled
Cloud Tests / lint-and-types (push) Has been cancelled
Cloud Tests / unit-tests (push) Has been cancelled
Cloud Tests / integration-tests (push) Has been cancelled
Cloud Tests / e2e-tests (push) Has been cancelled
CodeQL Advanced / Analyze (javascript-typescript) (push) Has been cancelled
Deploy Apps Worker (Product 2) / Determine environment (push) Has been cancelled
Deploy Apps Worker (Product 2) / Deploy apps worker to apps-control host (${{ needs.determine-env.outputs.environment }}) (push) Has been cancelled
Deploy Eliza Provisioning Worker / Determine environment (push) Has been cancelled
Deploy Eliza Provisioning Worker / Deploy worker to Hetzner host (${{ needs.determine-env.outputs.environment }} @ ${{ needs.determine-env.outputs.deployment_sha }}) (push) Has been cancelled
Dev Smoke / Classify changed paths (push) Has been cancelled
supply-chain / sbom (push) Has been cancelled
supply-chain / vulnerability-scan (push) Has been cancelled
Build, Push & Deploy to Phala Cloud / build-and-push (push) Has been cancelled
Test Packaging / Validate Packaging Configs (push) Has been cancelled
Test Packaging / Build & Test PyPI Package (push) Has been cancelled
Test Packaging / PyPI on Python ${{ matrix.python }} (push) Has been cancelled
Test Packaging / Pack & Test JS Tarballs (push) Has been cancelled
UI Fixture E2E / ui-fixture-e2e (push) Has been cancelled
UI Fixture E2E / fixture-e2e (push) Has been cancelled
UI Story Gate / story-gate (push) Has been cancelled
vault-ci / test (macos-latest) (push) Has been cancelled
vault-ci / test (ubuntu-latest) (push) Has been cancelled
vault-ci / test (windows-latest) (push) Has been cancelled
vault-ci / app-core wiring tests (push) Has been cancelled
verify-patches / verify patches/CHECKSUMS.sha256 (push) Has been cancelled
Voice Benchmark Smoke / voice-emotion fixture smoke (push) Has been cancelled
Voice Benchmark Smoke / voiceagentbench fixture smoke (push) Has been cancelled
Voice Benchmark Smoke / voicebench-quality unit smoke (push) Has been cancelled
Voice Benchmark Smoke / voicebench TypeScript unit (no audio) (push) Has been cancelled
Voice Benchmark Smoke / voice bench smoke summary (push) Has been cancelled
Windows CI / windows ([bun run --cwd packages/app-core test bun run --cwd packages/elizaos test bun run --cwd packages/cloud/shared test], app-and-cli) (push) Has been cancelled
Windows CI / windows ([bun run --cwd packages/scenario-runner test bun run --cwd packages/vault test bun run --cwd packages/security test bun run --cwd plugins/plugin-coding-tools test], framework-packages) (push) Has been cancelled
Windows CI / windows ([bun run --cwd plugins/plugin-elizacloud test bun run --cwd plugins/plugin-discord test bun run --cwd plugins/plugin-anthropic test bun run --cwd plugins/plugin-openai test bun run --cwd plugins/plugin-app-control test bun run --cwd plugins/pl… (push) Has been cancelled
Windows CI / windows ([node packages/scripts/run-turbo.mjs run build --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/agent --concurrency=4 node packages/scripts/run-bash-linux-only.mjs scripts/verify-riscv64-buildpaths.sh node packages/scripts/run… (push) Has been cancelled
Windows CI / windows ([node packages/scripts/run-turbo.mjs run typecheck --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/cloud-shared --concurrency=4 bun run --cwd packages/core test bun run --cwd packages/shared test], core-runtime, 75) (push) Has been cancelled
@elizaos/plugin-computeruse
Desktop automation plugin for elizaOS agents — screenshots, mouse / keyboard control, browser CDP automation, window management, clipboard, and the OCR provider registry that other plugins contribute to.
Ported from
coasty-ai/open-computer-use
(Apache 2.0).
Boundary with @elizaos/plugin-vision
This plugin owns the OS surfaces:
- screen / display capture (
src/platform/capture.ts,src/platform/displays.ts,ComputerUseService.captureScreen()), - input + windows + clipboard + accessibility,
- the OCR provider registries —
OcrProvider(line-level) andCoordOcrProvider(hierarchical with absolute coords), defined insrc/mobile/ocr-provider.ts.
@elizaos/plugin-vision owns the camera pipeline, scene description
via runtime.useModel(IMAGE_DESCRIPTION), the screen tiler, the
detector pipeline (faces / people / objects), and the OCR
implementations themselves. plugin-vision consumes capture from this
plugin via runtime.getService("computeruse") and contributes the
hierarchical OCR adapter into this plugin's registerCoordOcrProvider
seam at boot.
Both seams are runtime feature-detected — neither package depends on the other.
Enabling
- Config:
features.computeruse: true - Env:
COMPUTER_USE_ENABLED=1
Platform requirements
| OS | Capture | Input |
|---|---|---|
| macOS | screencapture (built-in) |
cliclick (brew install cliclick), AppleScript |
| Linux | import (ImageMagick) / scrot |
xdotool (sudo apt install xdotool) |
| Windows | PowerShell + System.Drawing |
PowerShell |
| Browser | — | puppeteer-core + Chrome / Edge / Brave |
Surface
- Actions —
COMPUTER_USE(canonical screenshot / click / key / scroll / etc.),WINDOW(list / focus / arrange / move /...), andCOMPUTER_USE_AGENT(high-level goal-driven autonomous desktop loop: Brain → Cascade → dispatch up tomaxStepsiterations). Subactions ofCOMPUTER_USEandWINDOWare promoted to virtual top-level actions (e.g.COMPUTER_USE_CLICK,WINDOW_FOCUS) so the planner picks a specific verb directly from the catalogue. - Services —
ComputerUseService(serviceType = "computeruse") andVisionContextProvider. - Providers —
computerStateProvider,sceneProvider. - Routes — approval inbox + SSE stream + approval-mode toggle under
/api/computer-use/....
File operations + shell
File operations live on the FILE action; shell / terminal access lives on the SHELL action. They are not exposed by this plugin.
Further reading
docs/MULTI_MONITOR.md— multi-display capture and coordinate translation.docs/SCENE_BUILDER.md— how windows, a11y, screen, and OCR are composed into a singleScene.docs/IOS_CONSTRAINTS.md/docs/ANDROID_CONSTRAINTS.md— honest scope on mobile.docs/MOBILE_ASSISTANT_ROUTING.md— mobile request routing.docs/AOSP_SYSTEM_APP.md— AOSP system-app deployment notes.