From 0730c856fa365432e1732e5d68b02023fcf3a16a Mon Sep 17 00:00:00 2001 From: wehub-resource-sync Date: Mon, 13 Jul 2026 10:41:37 +0000 Subject: [PATCH] docs: make Chinese README the default --- README.md | 351 +++++++++++++++++++++++++----------------------------- 1 file changed, 161 insertions(+), 190 deletions(-) diff --git a/README.md b/README.md index cf6aabf..596db21 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,9 @@ + +> [!NOTE] +> 本文档由 WeHub 基于上游 README 翻译整理，属于社区翻译，非官方中文文档。 +> [English](./README.en.md) · [原始项目](https://github.com/jundot/omlx) · [上游 README](https://github.com/jundot/omlx/blob/HEAD/README.md) +> 原作者、版权与许可证归属以原始项目及本仓库 LICENSE 文件为准。 +

@@ -7,7 +13,7 @@

oMLX

-

LLM inference, optimized for your Mac
Continuous batching and tiered KV caching, managed directly from your menu bar.

+

LLM 推理，为你的 Mac 优化
连续批处理和分层 KV 缓存，直接从菜单栏管理。

Buy Me A Coffee @@ -15,7 +21,7 @@

License - Python 3.11-3.13 + Python 3.10+ Apple Silicon

@@ -24,18 +30,18 @@

- Install · - Quickstart · - Features · - Models · - CLI Configuration · - Benchmarks · + 安装 · + 快速开始 · + 功能 · + 模型 · + CLI 配置 · + 基准测试 · oMLX.ai

- English · - 中文 · + English · + 中文 · 한국어 · 日本語

@@ -43,18 +49,18 @@ ---

- oMLX Admin Dashboard + oMLX 管理后台

-> *Every LLM server I tried made me choose between convenience and control. I wanted to pin everyday models in memory, auto-swap heavier ones on demand, set context limits - and manage it all from a menu bar.* +> *我试过的每个 LLM 服务器都让我在便利性和可控性之间做选择。我想把常用模型固定在内存中，按需自动切换较重的模型，设置上下文限制，并从菜单栏管理这一切。* > -> *oMLX persists KV cache across a hot in-memory tier and cold SSD tier - even when context changes mid-conversation, all past context stays cached and reusable across requests, making local LLMs practical for real coding work with tools like Claude Code. That's why I built it.* +> *oMLX 将 KV 缓存持久化在热内存层和冷 SSD 层之间。即使对话中途上下文发生变化，所有历史上下文仍然保留在缓存中，可跨请求复用，让本地 LLM 在配合 Claude Code 等工具做实际编码时真正变得可用。这就是我做 oMLX 的原因。* -## Install +## 安装 -### macOS App +### macOS 应用 -Download the `.dmg` from [Releases](https://github.com/jundot/omlx/releases), drag to Applications, done. The app includes in-app auto-update, so future upgrades are just one click. The macOS app also installs a lightweight `~/.omlx/bin/omlx` CLI shim so terminal commands and Apple Shortcuts can control the app-managed server. +从 [Releases](https://github.com/jundot/omlx/releases) 下载 `.dmg`，拖到 Applications 即可。应用支持自动更新，后续升级只需一键完成。macOS 应用也会安装轻量的 `~/.omlx/bin/omlx` CLI shim，因此可以从终端命令或 Apple Shortcuts 控制由应用管理的服务器。 ### Homebrew @@ -62,217 +68,191 @@ Download the `.dmg` from [Releases](https://github.com/jundot/omlx/releases), dr brew tap jundot/omlx https://github.com/jundot/omlx brew install omlx -# Upgrade to the latest version +# 升级到最新版本 brew update && brew upgrade omlx -# Run as a background service (auto-restarts on crash) -omlx start +# 作为后台服务运行（崩溃时自动重启） +brew services start omlx -# Optional: MCP (Model Context Protocol) support +# 可选：MCP（Model Context Protocol）支持 /opt/homebrew/opt/omlx/libexec/bin/pip install mcp ``` -Optional GLM-5.2 / MiniMax M3 native custom kernels currently require a HEAD build: +可选的 GLM-5.2 / MiniMax M3 原生自定义内核目前需要 HEAD 构建： ```bash brew install omlx --HEAD --with-custom-kernel ``` -### From Source +### 从源码安装 ```bash git clone https://github.com/jundot/omlx.git cd omlx -pip install -e . # Core only -pip install -e ".[mcp]" # With MCP (Model Context Protocol) support +pip install -e . # 仅核心 +pip install -e ".[mcp]" # 含 MCP（Model Context Protocol）支持 -# GLM-5.2 / MiniMax M3 / Qwen3.5 native custom kernels (strongly recommended -# if you serve those families -- see note below) +# 可选：GLM-5.2 / MiniMax M3 原生自定义内核 OMLX_WITH_CUSTOM_KERNEL=1 pip install -e . ``` -Requires macOS 15.0+ (Sequoia), Python 3.11–3.13, and Apple Silicon (M1/M2/M3/M4). +需要 macOS 15.0+ (Sequoia), Python 3.10+ 和 Apple Silicon（M1/M2/M3/M4）。 -> **Note on native custom kernels:** a plain `pip install -e .` does NOT build -> them, and the affected model families then silently fall back to much slower -> generic paths -- for GLM-5.2 the fused DSA prefill is roughly 30x faster with -> the kernels (measured 845 vs ~29 tok/s on an M3 Ultra), and the fallback also -> uses more memory (#2137). Building them requires the Metal toolchain, which -> Command Line Tools alone do not provide (`xcrun: error: unable to find utility -> "metal"`): install full Xcode, or use the official DMG which ships the kernels -> precompiled. Homebrew can build them with `brew install omlx --HEAD -> --with-custom-kernel`, but that build also needs full Xcode. To verify your -> install: -> -> ```bash -> python -c "from omlx.custom_kernels import native_kernel_status; print(native_kernel_status())" -> ``` +## 快速开始 -## Quickstart +### macOS 应用 -### macOS App - -Launch oMLX from your Applications folder. The Welcome screen guides you through three steps - model directory, server start, and first model download. That's it. To connect OpenClaw, OpenCode, Codex, Hermes Agent, or Copilot, see [Integrations](#integrations). +从 Applications 文件夹启动 oMLX。欢迎界面会引导你完成三个步骤 — 模型目录设置、服务器启动、首个模型下载。就是这样。要连接 OpenClaw、OpenCode、Codex、Hermes Agent 或 Copilot，请参阅[集成](#集成)。

- oMLX Welcome Screen - oMLX Menubar + oMLX 欢迎界面 + oMLX 菜单栏

### CLI ```bash -# Managed background server (macOS app or Homebrew install) -omlx start -omlx stop -omlx restart - -# Foreground server attached to this terminal omlx serve --model-dir ~/models ``` -The server discovers LLMs, VLMs, embedding models, and rerankers from subdirectories automatically. Any OpenAI-compatible client can connect to `http://localhost:8000/v1`. A built-in chat UI is also available at `http://localhost:8000/admin/chat`. +服务器会自动从子目录中发现 LLM、VLM、嵌入模型和重排序模型。任何 OpenAI 兼容客户端都可以连接到 `http://localhost:8000/v1`。内置聊天 UI 也可在 `http://localhost:8000/admin/chat` 使用。 -### Homebrew Service +### Homebrew 服务 -If you installed via Homebrew, you can run oMLX as a managed background service: +如果通过 Homebrew 安装，可以将 oMLX 作为托管后台服务运行： ```bash -omlx start # Start via brew services -omlx stop # Stop -omlx restart # Restart - -brew services start omlx # Start (auto-restarts on crash) -brew services stop omlx # Stop -brew services restart omlx # Restart -brew services info omlx # Check status +brew services start omlx # 启动（崩溃时自动重启） +brew services stop omlx # 停止 +brew services restart omlx # 重启 +brew services info omlx # 查看状态 ``` -The service runs `omlx serve` with zero-config defaults (`~/.omlx/models`, port 8000). `omlx start`, `omlx stop`, and `omlx restart` are the portable lifecycle commands; Homebrew installs delegate them to `brew services`. To customize, either set environment variables (`OMLX_MODEL_DIR`, `OMLX_PORT`, etc.) or run `omlx serve --model-dir /your/path` once to persist settings to `~/.omlx/settings.json`. +服务使用默认配置运行 `omlx serve`（`~/.omlx/models`，端口 8000）。要自定义，可以设置环境变量（`OMLX_MODEL_DIR`、`OMLX_PORT` 等），或运行一次 `omlx serve --model-dir /your/path` 将设置保存到 `~/.omlx/settings.json`。 -Logs are written to two locations: -- **Service log**: `$(brew --prefix)/var/log/omlx.log` (stdout/stderr) -- **Server log**: `~/.omlx/logs/server.log` (structured application log) +日志写入两个位置： +- **服务日志**: `$(brew --prefix)/var/log/omlx.log`（stdout/stderr） +- **服务器日志**: `~/.omlx/logs/server.log`（结构化应用日志） -## Features +## 功能 -Supports text LLMs, vision-language models (VLM), OCR models, embeddings, and rerankers on Apple Silicon. +在 Apple Silicon 上支持文本 LLM、视觉语言模型（VLM）、OCR 模型、嵌入模型和重排序模型。 -### Admin Dashboard +### 管理后台 -Web UI at `/admin` for real-time monitoring, model management, chat, benchmark, and per-model settings. Supports English, Korean, Japanese, Chinese, French, Russian, Spanish, and Brazilian Portuguese. All CDN dependencies are vendored for fully offline operation. +在 `/admin` 提供实时监控、模型管理、聊天、基准测试和模型级设置的 Web UI。支持英语、韩语、日语、中文和俄语。所有 CDN 依赖已内置，完全支持离线运行。

- oMLX Admin Dashboard + oMLX 管理后台

-### Vision-Language Models +### 视觉语言模型 -Run VLMs with the same continuous batching and tiered KV cache stack as text LLMs. Supports multi-image chat, base64/URL/file image inputs, and tool calling with vision context. OCR models (DeepSeek-OCR, DOTS-OCR, GLM-OCR) are auto-detected with optimized prompts. +使用与文本 LLM 相同的连续批处理和分层 KV 缓存堆栈运行 VLM。支持多图聊天、base64/URL/文件图像输入，以及带视觉上下文的工具调用。OCR 模型（DeepSeek-OCR、DOTS-OCR、GLM-OCR）会被自动识别，并使用优化的提示词。 -### Tiered KV Cache (Hot + Cold) +### 分层 KV 缓存（热缓存 + 冷缓存） -Block-based KV cache management inspired by vLLM, with prefix sharing and Copy-on-Write. The cache operates across two tiers: +借鉴 vLLM 的基于块的 KV 缓存管理，支持前缀共享和写时复制（Copy-on-Write）。缓存分为两个层级： -- **Hot tier (RAM)**: Frequently accessed blocks stay in memory for fast access. -- **Cold tier (SSD)**: When the hot cache fills up, blocks are offloaded to SSD in safetensors format. On the next request with a matching prefix, they're restored from disk instead of recomputed from scratch - even after a server restart. +- **热缓存（RAM）**: 频繁访问的块保留在内存中，实现快速读取。 +- **冷缓存（SSD）**: 热缓存满时，块会以 safetensors 格式转存到 SSD。下次请求命中相同前缀时，直接从磁盘恢复，无需重新计算 — 即使服务器重启也不会丢失。

- oMLX Hot & Cold Cache + oMLX 热缓存与冷缓存

-### Continuous Batching +### 连续批处理 -Handles concurrent requests through mlx-lm's BatchGenerator. Max concurrent requests is configurable via CLI or admin panel. +通过 mlx-lm 的 BatchGenerator 处理并发请求。最大并发请求数可通过 CLI 或管理面板配置。 -### Claude Code Optimization +### Claude Code 优化 -Context scaling support for running smaller context models with Claude Code. Scales reported token counts so that auto-compact triggers at the right timing, and SSE keep-alive prevents read timeouts during long prefill. +支持在 Claude Code 中使用较小上下文模型的上下文缩放。通过缩放上报的 Token 数量，让自动压缩在合适的时机触发，同时提供 SSE keep-alive 防止长时间预填充导致的读取超时。 -### Multi-Model Serving +### 多模型服务 -Load LLMs, VLMs, embedding models, and rerankers within the same server. Models are managed through a combination of automatic and manual controls: +在同一服务器中加载 LLM、VLM、嵌入模型和重排序模型。通过自动和手动控制的组合管理模型： -- **LRU eviction**: Least-recently-used models are evicted automatically when memory runs low. -- **Manual load/unload**: Interactive status badges in the admin panel let you load or unload models on demand. -- **Model pinning**: Pin frequently used models to keep them always loaded. -- **Per-model TTL**: Set an idle timeout per model to auto-unload after a period of inactivity. -- **Process memory enforcement**: Total memory limit (default: system RAM - 8GB) prevents system-wide OOM. +- **LRU 驱逐**: 内存不足时，最近最少使用的模型会被自动卸载。 +- **手动加载/卸载**: 在管理后台通过状态标识按需加载或卸载模型。 +- **模型固定**: 固定常用模型使其始终保持加载状态。 +- **模型级 TTL**: 为每个模型设置空闲超时，在一段时间不活动后自动卸载。 +- **进程内存限制**: 总内存限制（默认：系统 RAM - 8GB）防止系统级 OOM。 -### Per-Model Settings +### 模型级设置 -Configure sampling parameters, chat template kwargs, TTL, model alias, model type override, and more per model directly from the admin panel. Changes apply immediately without server restart. +在管理后台直接配置每个模型的采样参数、聊天模板参数、TTL、模型别名、模型类型覆盖等。修改即时生效，无需重启服务器。 -- **Model alias**: set a custom API-visible name. `/v1/models` returns the alias, and requests accept both the alias and directory name. -- **Model type override**: manually set a model as LLM or VLM regardless of auto-detection. -- **Profiles**: save named bundles of per-model settings and switch between them from the admin panel. A profile can optionally be exposed as its own model: `/v1/models` then also lists `:` (e.g. `qwen3-8b:thinking`), which serves on the same engine as the base model with the profile's settings overlaid per request — no extra memory, no reload. When the base model has an alias, the exposed ID is advertised as `:`; the directory-name form keeps working, just like for the base model. +- **模型别名**: 设置自定义 API 显示名称。`/v1/models` 返回别名，请求时别名和目录名均可使用。 +- **模型类型覆盖**: 无论自动检测结果如何，手动设置为 LLM 或 VLM。 +- **配置文件**: 保存每个模型的命名设置组合，并在管理后台中切换。配置文件可以选择性地作为独立模型公开：`/v1/models` 随后也会列出 `<模型>:<配置文件>`（例如 `qwen3-8b:thinking`），它在与基础模型相同的引擎上运行，并按请求叠加该配置文件的设置 — 无需额外内存，无需重新加载。当基础模型有别名时，公开的 ID 以 `<别名>:<配置文件>` 形式呈现；目录名形式仍然有效，与基础模型一样。

- oMLX Chat Template Kwargs + oMLX 聊天模板参数

-### Built-in Chat +### 内置聊天 -Chat directly with any loaded model from the admin panel. Supports conversation history, model switching, dark mode, reasoning model output, and image upload for VLM/OCR models. +从管理后台直接与已加载的模型聊天。支持对话历史、模型切换、深色模式、推理模型输出，以及 VLM/OCR 模型的图片上传。

- oMLX Chat + oMLX 聊天

-### Model Downloader +### 模型下载器 -Search and download MLX models from HuggingFace directly in the admin dashboard. Browse model cards, check file sizes, and download with one click. +在管理后台中直接搜索和下载 HuggingFace 上的 MLX 模型。浏览模型卡片、查看文件大小，一键下载。

- oMLX Model Downloader + oMLX 模型下载器

-### Integrations +### 集成 -Set up OpenClaw, OpenCode, Codex, Hermes Agent, Copilot, and Pi directly from the admin dashboard with a single click. No manual config editing required. +在管理后台中一键设置 OpenClaw、OpenCode、Codex、Hermes Agent、Copilot 和 Pi。无需手动编辑配置文件。

- oMLX Integrations + oMLX 集成

-### Performance Benchmark +### 性能基准测试 -One-click benchmarking from the admin panel. Measures prefill (PP) and text generation (TG) tokens per second, with partial prefix cache hit testing for realistic performance numbers. +从管理后台一键运行基准测试。测量预填充（PP）和 Token 生成（TG）的每秒 Token 数，包含部分前缀缓存命中测试以获得真实的性能数据。

- oMLX Benchmark Tool + oMLX 基准测试工具

-### macOS Menubar App +### macOS 菜单栏应用 -Native Swift / SwiftUI menubar app (not Electron). Start, stop, and monitor the server without opening a terminal. Includes persistent serving stats (survives restarts), auto-restart on crash, and Sparkle-driven auto-update. +原生 Swift / SwiftUI 菜单栏应用（非 Electron）。无需打开终端即可启动、停止和监控服务器。包含持久化服务统计（重启后保留）、崩溃自动重启和基于 Sparkle 的自动更新。

- oMLX Menubar Stats + oMLX 菜单栏统计

-### API Compatibility +### API 兼容性 -Drop-in replacement for OpenAI and Anthropic APIs. Supports streaming usage stats (`stream_options.include_usage`), Anthropic adaptive thinking, and vision inputs (base64, URL). +OpenAI 和 Anthropic API 的直接替代品。支持流式使用统计（`stream_options.include_usage`）、Anthropic adaptive thinking 和视觉输入（base64、URL）。 -| Endpoint | Description | -|----------|-------------| -| `POST /v1/chat/completions` | Chat completions (streaming) | -| `POST /v1/completions` | Text completions (streaming) | +| 端点 | 说明 | +|----------|------| +| `POST /v1/chat/completions` | 聊天补全（流式） | +| `POST /v1/completions` | 文本补全（流式） | | `POST /v1/messages` | Anthropic Messages API | -| `POST /v1/embeddings` | Text embeddings | -| `POST /v1/rerank` | Document reranking | -| `GET /v1/models` | List available models | +| `POST /v1/embeddings` | 文本嵌入 | +| `POST /v1/rerank` | 文档重排序 | +| `GET /v1/models` | 列出可用模型 | -### Tool Calling & Structured Output +### 工具调用与结构化输出 -Supports all function calling formats available in mlx-lm, JSON schema validation, and MCP tool integration. Tool calling requires the model's chat template to support the `tools` parameter. The following model families are auto-detected via mlx-lm's built-in tool parsers: +支持 mlx-lm 中所有可用的函数调用格式、JSON Schema 验证和 MCP 工具集成。工具调用需要模型的聊天模板支持 `tools` 参数。以下模型系列通过 mlx-lm 的内置工具解析器自动检测： -| Model Family | Format | +| 模型系列 | 格式 | |---|---| -| Llama, Qwen, DeepSeek, etc. | JSON `` | -| Qwen3.5 Series | XML `` | +| Llama、Qwen、DeepSeek 等 | JSON `` | +| Qwen3.5 系列 | XML `` | | Gemma | `` | | GLM (4.7, 5) | `/` XML | | MiniMax | Namespaced `` | @@ -280,11 +260,11 @@ Supports all function calling formats available in mlx-lm, JSON schema validatio | Kimi K2 | `<\|tool_calls_section_begin\|>` | | Longcat | `` | -Models not listed above may still work if their chat template accepts `tools` and their output uses a recognized `` XML format. For tool-enabled streaming, assistant text is emitted incrementally while known tool-call control markup is suppressed from visible content; structured tool calls are emitted after parsing the completed turn. +上表未列出的模型，只要聊天模板支持 `tools` 参数且输出采用可识别的 `` XML 格式，也有可能正常工作。针对支持工具调用的流式请求，系统会增量发射助手文本，同时隐藏已知的工具调用控制标记；结构化工具调用将在完成整个回合解析后发射。 -## Models +## 模型 -Point `--model-dir` at a directory containing MLX-format model subdirectories. Two-level organization folders (e.g., `mlx-community/model-name/`) are also supported. +将 `--model-dir` 指向包含 MLX 格式模型子目录的目录。支持两级目录结构（如 `mlx-community/model-name/`）。 ``` ~/models/ @@ -295,83 +275,75 @@ Point `--model-dir` at a directory containing MLX-format model subdirectories. T └── bge-m3/ ``` -Models are auto-detected by type. You can also download models directly from the admin dashboard. +模型会按类型自动识别。也可以直接在管理后台下载模型。 -| Type | Models | -|------|--------| -| LLM | Any model supported by [mlx-lm](https://github.com/ml-explore/mlx-lm) | -| VLM | Qwen3.5 Series, GLM-4V, Pixtral, and other [mlx-vlm](https://github.com/Blaizzy/mlx-vlm) models | -| OCR | DeepSeek-OCR, DOTS-OCR, GLM-OCR | -| Embedding | BERT, BGE-M3, ModernBERT | -| Reranker | ModernBERT, XLM-RoBERTa | +| 类型 | 模型 | +|------|------| +| LLM | [mlx-lm](https://github.com/ml-explore/mlx-lm) 支持的所有模型 | +| VLM | Qwen3.5 系列、GLM-4V、Pixtral 及其他 [mlx-vlm](https://github.com/Blaizzy/mlx-vlm) 模型 | +| OCR | DeepSeek-OCR、DOTS-OCR、GLM-OCR | +| 嵌入 | BERT、BGE-M3、ModernBERT | +| 重排序 | ModernBERT、XLM-RoBERTa | -## CLI Configuration +## CLI 配置 ```bash -# Managed background server (macOS app or Homebrew install) -omlx start -omlx stop -omlx restart +# 已加载模型的内存限制 +omlx serve --model-dir ~/models --max-model-memory 32GB -# Start with default settings (memory guard tier = balanced, manage via admin UI) -omlx serve --model-dir ~/models +# 进程级内存限制（默认：auto = RAM - 8GB） +omlx serve --model-dir ~/models --max-process-memory 80% -# Choose a memory guard tier at startup -omlx serve --model-dir ~/models --memory-guard safe - -# Set a custom memory guard ceiling in GB -omlx serve --model-dir ~/models --memory-guard-gb 48 - -# Enable SSD cache for KV blocks +# 启用 KV 块的 SSD 缓存 omlx serve --model-dir ~/models --paged-ssd-cache-dir ~/.omlx/cache -# Set in-memory hot cache size +# 设置内存热缓存大小 omlx serve --model-dir ~/models --hot-cache-max-size 20% -# Adjust max concurrent requests (default: 8) +# 调整最大并发请求数（默认: 8） omlx serve --model-dir ~/models --max-concurrent-requests 16 -# With MCP tools +# 使用 MCP 工具 omlx serve --model-dir ~/models --mcp-config mcp.json -# HuggingFace mirror endpoint (for restricted regions) +# HuggingFace 镜像端点（适用于受限地区） omlx serve --model-dir ~/models --hf-endpoint https://hf-mirror.com -# API key authentication +# API 密钥认证 omlx serve --model-dir ~/models --api-key your-secret-key -# Localhost-only: skip verification via admin panel global settings +# 仅限 Localhost：在管理后台全局设置中跳过验证 ``` -All settings can also be configured from the web admin panel at `/admin`. Settings are persisted to `~/.omlx/settings.json`, and CLI flags take precedence. +以上所有设置也可以在 `/admin` 的 Web 管理后台中配置。设置保存在 `~/.omlx/settings.json`，CLI 参数优先级更高。
-Architecture +架构 ``` FastAPI Server (OpenAI / Anthropic API) │ - ├── EnginePool (multi-model, LRU eviction, TTL, manual load/unload) - │ ├── BatchedEngine (LLMs, continuous batching) - │ ├── VLMEngine (vision-language models) + ├── EnginePool (多模型、LRU 驱逐、TTL、手动加载/卸载) + │ ├── BatchedEngine (LLM，连续批处理) + │ ├── VLMEngine (视觉语言模型) │ ├── EmbeddingEngine │ └── RerankerEngine │ - ├── ProcessMemoryEnforcer (total memory limit, TTL checks) + ├── ProcessMemoryEnforcer (总内存限制、TTL 检查) │ - ├── Scheduler (FCFS, configurable concurrency) + ├── Scheduler (FCFS，可配置并发数) │ └── mlx-lm BatchGenerator │ └── Cache Stack - ├── PagedCacheManager (GPU, block-based, CoW, prefix sharing) - ├── Hot Cache (in-memory tier, write-back) - └── PagedSSDCacheManager (SSD cold tier, safetensors format) + ├── PagedCacheManager (GPU，基于块，CoW，前缀共享) + ├── Hot Cache (内存缓存，write-back) + └── PagedSSDCacheManager (SSD 冷缓存，safetensors 格式) ```
-## Development +## 开发 -### CLI Server +### CLI 服务器 ```bash git clone https://github.com/jundot/omlx.git @@ -380,44 +352,43 @@ pip install -e ".[dev]" pytest -m "not slow" ``` -### macOS App +### macOS 应用 -The native SwiftUI app lives at `apps/omlx-mac/`. Requires Xcode 26.5+ and Python 3.11+. venvstacks is declared as a dev dependency so `pip install -e ".[dev]"` (or `uv sync --dev`) brings the pinned version in. The build script also falls back to `uvx venvstacks` or `pipx run venvstacks` if you prefer a host-global tool runner. +原生 SwiftUI 应用位于 `apps/omlx-mac/`，需要 Xcode 26.5+ 和 Python 3.11+。venvstacks 已声明为 dev 依赖，因此 `pip install -e ".[dev]"`（或 `uv sync --dev`）会引入固定版本。若偏好主机全局工具运行器，也可使用 `uvx venvstacks` 或 `pipx run venvstacks`。 ```bash -# Stage a runnable oMLX.app (xcodebuild + venvstacks Python layers + ad-hoc sign) +# 暂存可运行的 oMLX.app（xcodebuild + venvstacks Python 层 + ad-hoc 签名） apps/omlx-mac/Scripts/build.sh release -# Result lands at apps/omlx-mac/build/Stage/oMLX.app +# 结果在 apps/omlx-mac/build/Stage/oMLX.app open apps/omlx-mac/build/Stage/oMLX.app -# Force a fresh venvstacks rebuild (otherwise it's cached by fingerprint) +# 强制重建 venvstacks（默认按指纹缓存） apps/omlx-mac/Scripts/build.sh release --rebuild-donor -# Stage with optional GLM-5.2 / MiniMax M3 native custom kernels +# 暂存包含可选 GLM-5.2 / MiniMax M3 原生自定义内核的应用 apps/omlx-mac/Scripts/build.sh release --with-custom-kernel ``` -First cold build takes 10–20 minutes (venvstacks Python layer assembly). Subsequent builds reuse the cached `packaging/_export/` and finish in about 4 minutes. See [packaging/README.md](packaging/README.md) for the layer configuration and [apps/omlx-mac/](apps/omlx-mac/) for the Swift sources. +首次 cold 构建需要 10–20 分钟（venvstacks Python 层组装）。后续构建复用 `packaging/_export/` 缓存，约 4 分钟完成。层配置请参阅 [packaging/README.md](packaging/README.md)，Swift 源码请参阅 [apps/omlx-mac/](apps/omlx-mac/)。 -## Contributing +## 贡献 -Contributions are welcome! See [Contributing Guide](docs/CONTRIBUTING.md) for details. +欢迎贡献！详情请参阅[贡献指南](docs/CONTRIBUTING.md)。 -- Bug fixes and improvements -- Performance optimizations -- Documentation improvements +- Bug 修复和改进 +- 性能优化 +- 文档改进 -## License +## 许可证 [Apache 2.0](LICENSE) -## Acknowledgments +## 致谢 -- [MLX](https://github.com/ml-explore/mlx) and [mlx-lm](https://github.com/ml-explore/mlx-lm) by Apple -- [mlx-vlm](https://github.com/Blaizzy/mlx-vlm) - Vision-language model inference on Apple Silicon -- [vllm-mlx](https://github.com/waybarrios/vllm-mlx) - oMLX started from vllm-mlx v0.1.0 and evolved significantly with multi-model serving, tiered KV caching, VLM with full paged cache support, an admin panel, and a macOS menu bar app -- [venvstacks](https://venvstacks.lmstudio.ai) - Portable Python environment layering for the macOS app bundle -- [mlx-embeddings](https://github.com/Blaizzy/mlx-embeddings) - Embedding model support for Apple Silicon -- [dflash-mlx](https://github.com/bstnxbt/dflash-mlx) - Block diffusion speculative decoding on Apple Silicon -- [MTPLX](https://github.com/youssofal/mtplx) - Lightning MTP's verify-shape Metal kernels are powered by MTPLX by Youssof Altoukhi, which also inspired the depth-k pipeline +- [MLX](https://github.com/ml-explore/mlx) 和 [mlx-lm](https://github.com/ml-explore/mlx-lm) by Apple +- [mlx-vlm](https://github.com/Blaizzy/mlx-vlm) - Apple Silicon 上的视觉语言模型推理 +- [vllm-mlx](https://github.com/waybarrios/vllm-mlx) - oMLX 从 vllm-mlx v0.1.0 起步，经过大幅演进，增加了多模型服务、分层 KV 缓存、完整分页缓存支持的 VLM、管理后台和 macOS 菜单栏应用 +- [venvstacks](https://venvstacks.lmstudio.ai) - macOS 应用包的便携 Python 环境分层 +- [mlx-embeddings](https://github.com/Blaizzy/mlx-embeddings) - Apple Silicon 嵌入模型支持 +- [dflash-mlx](https://github.com/bstnxbt/dflash-mlx) - Apple Silicon 上的块扩散推测解码 (Block diffusion speculative decoding)