elizaos--eliza/plugins/plugin-native-mlkit-text/AGENTS.md

@elizaos/capacitor-mlkit-text

Capacitor plugin for on-device OCR text recognition through Android ML Kit.

Purpose / Role

This is a Capacitor plugin, not an elizaOS runtime plugin. It exposes a Tesseract-compatible JavaScript surface (recognize({ image, psm? })) so the renderer-pulled OCR bridge in @elizaos/plugin-vision can run native Android OCR without bundling Tesseract binaries.

The package intentionally avoids checked-in model/native binaries. Android uses Google ML Kit's text-recognition dependency; web rejects with a clear unsupported error.

Layout

src/
  definitions.ts   JS contract and word shape
  index.ts         registerPlugin("Tesseract")
  web.ts           unsupported web fallback
android/
  build.gradle
  src/main/AndroidManifest.xml
  src/main/java/ai/eliza/plugins/mlkittext/
    MlKitTextPlugin.kt   Capacitor surface (base64 decode + JS bridge)
    MlKitTextReader.kt   Engine wrapper: ML Kit recognizer + word mapping
  src/androidTest/java/ai/eliza/plugins/mlkittext/
    MlKitTextReaderInstrumentedTest.kt   On-device OCR of a rendered bitmap

Commands

Scripts are defined in package.json; run them from the repo root with bun run --cwd:

bun run --cwd plugins/plugin-native-mlkit-text clean           # remove build output
bun run --cwd plugins/plugin-native-mlkit-text build           # build package artifacts
bun run --cwd plugins/plugin-native-mlkit-text typecheck       # TypeScript typecheck
bun run --cwd plugins/plugin-native-mlkit-text lint            # mutating Biome check
bun run --cwd plugins/plugin-native-mlkit-text lint:check      # read-only Biome check
bun run --cwd plugins/plugin-native-mlkit-text format          # write formatting
bun run --cwd plugins/plugin-native-mlkit-text format:check    # read-only formatting check
bun run --cwd plugins/plugin-native-mlkit-text test            # run package tests
bun run --cwd plugins/plugin-native-mlkit-text prepublishOnly  # publish-time build hook
bun run --cwd plugins/plugin-native-mlkit-text watch           # watch TypeScript sources
bun run --cwd plugins/plugin-native-mlkit-text build:unlocked  # bun run clean && tsc && bunx rollup -c rollup.config.mjs

Gotchas

• Keep the plugin name as Tesseract for compatibility with the existing UI bridge lookup and the stale #9649 branch's renderer contract.
• Do not add committed OCR model binaries to this package. If Android OCR needs a downloadable model in the future, stage it through the normal artifact path and document the proof.