chore: import upstream snapshot with attribution
Tests / Import Check (Python 3.13) (push) Has been cancelled
Tests / Import Check (Python 3.14) (push) Has been cancelled
Tests / Python Tests (Python 3.11) (push) Has been cancelled
Tests / Python Tests (Python 3.12) (push) Has been cancelled
Tests / Python Tests (Python 3.14) (push) Has been cancelled
Tests / Test Summary (push) Has been cancelled
Tests / Lint and Format (push) Has been cancelled
Tests / Web Node Tests (push) Has been cancelled
Tests / Import Check (Python 3.11) (push) Has been cancelled
Tests / Import Check (Python 3.12) (push) Has been cancelled
Tests / Python Tests (Python 3.13) (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 13:00:43 +08:00
commit e4dcfc49aa
1668 changed files with 324490 additions and 0 deletions
+252
View File
@@ -0,0 +1,252 @@
# DeepTutor CLI
Agent-first 的命令行界面。两条核心路径:
- **`run`** — 单次执行任意 capability(为 agent 调用设计)
- **`chat`** — 交互式 REPL(为人类设计)
## 安装
```bash
# 仅 CLI(本地源码安装,含 RAG / 文档解析 / 各家 LLM provider SDK
git clone https://github.com/HKUDS/DeepTutor.git
cd DeepTutor
python3 -m venv .venv-cli
source .venv-cli/bin/activate
python -m pip install --upgrade pip
python -m pip install -e ./packaging/deeptutor-cli
deeptutor init --cli
# CLI + Web/API 服务
pip install deeptutor
deeptutor init
# 源码开发
pip install -e .
deeptutor init
# 可选附加组件
pip install -e ".[partners]" # Partners 渠道 SDK + MCP 客户端
pip install -e ".[math-animator]" # 数学动画(另需系统 LaTeX/ffmpeg
pip install -e ".[all]" # 全部依赖(含开发工具)
```
`deeptutor init --cli` 和普通 `deeptutor init` 使用同一套 `data/user/settings/` 配置目录;区别是 `--cli` 不询问 Web 后端/前端端口,仍会创建 `system.json``auth.json``integrations.json``model_catalog.json``main.yaml``agents.yaml`,并继续询问 LLM 配置。Embedding 配置默认跳过;如果要使用 `deeptutor kb ...` 或 RAG,请在向导里选择配置 embedding,或稍后编辑 `data/user/settings/model_catalog.json`
Windows PowerShell 可使用:
```powershell
py -3.11 -m venv .venv-cli
.\.venv-cli\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -e ./packaging/deeptutor-cli
deeptutor init --cli
```
---
## `run` — 执行 Capability
统一入口,单次执行任意 capability。Agent 只需掌握这一个命令。
```bash
deeptutor run <capability> <message> [options]
```
### 内置 Capability
| Capability | 说明 |
|------------|------|
| `chat` | 对话(默认,可挂载工具) |
| `deep_solve` | 多阶段深度解题 |
| `deep_question` | 智能出题 |
| `deep_research` | 多 agent 深度研究 |
| `visualize` | 生成图表、图解、Mermaid、HTML 或 Manim 可视化 |
| `math_animator` | 数学动画生成 |
| `mastery_path` | 掌握式学习路径与测评循环 |
### 选项
| 选项 | 缩写 | 说明 |
|------|------|------|
| `--tool` | `-t` | 启用工具(可多次指定):`rag`, `web_search`, `code_execution`, `reason`, `brainstorm`, `paper_search`, `geogebra_analysis`, `imagegen`, `videogen` |
| `--kb` | | 挂载知识库 |
| `--language` | `-l` | 回复语言(默认 `en` |
| `--session` | | 继续已有会话 |
| `--config` | | capability 配置 `key=value`(可多次指定) |
| `--config-json` | | capability 配置(JSON 字符串) |
| `--notebook-ref` | | 笔记本引用 |
| `--history-ref` | | 引用历史会话 |
| `--format` | `-f` | 输出格式:`rich`(默认)\| `json` |
### 示例
```bash
# 对话
deeptutor run chat "什么是傅里叶变换?" -l zh
# 深度解题
deeptutor run deep_solve "证明 n^3-n 能被 6 整除" -t rag --kb math-textbook
# 简要回答
deeptutor run deep_solve "求 sin(x) 的导数" --config detailed_answer=false
# 智能出题
deeptutor run deep_question "线性代数" --config num_questions=5 --config difficulty=hard
# 仿真出题
deeptutor run deep_question "模拟考试" --config mode=mimic --config paper_path=exam.json
# 深度研究
deeptutor run deep_research "Transformer 最新进展" \
--config-json '{"mode":"report","depth":"deep","sources":["web","papers"]}'
# 可视化
deeptutor run visualize "画出注意力机制的数据流图" --config render_mode=mermaid
# 数学动画
deeptutor run math_animator "展示正弦函数变换" --config quality=high
# 掌握式学习
deeptutor run mastery_path "带我系统掌握特征值和特征向量"
# JSON 输出(适合 agent 解析)
deeptutor run deep_solve "求解 x^2=4" -f json
```
---
## `chat` — 交互式 REPL
进入多轮对话界面,在 REPL 内通过 `/` 命令切换 capability、工具、知识库等。
```bash
deeptutor chat [options]
```
| 选项 | 说明 |
|------|------|
| `--session` | 恢复已有会话 |
| `--tool`, `-t` | 预启用工具 |
| `--capability`, `-c` | 初始 capability(默认 `chat` |
| `--kb` | 预挂载知识库 |
| `--language`, `-l` | 回复语言 |
### REPL 内置命令
| 命令 | 说明 |
|------|------|
| `/quit` | 退出 |
| `/session` | 显示当前 session ID |
| `/new` | 新建会话 |
| `/tool on\|off <name>` | 启用/关闭工具 |
| `/cap <name>` | 切换 capability |
| `/kb <name>\|none` | 切换知识库 |
| `/history add <id>\|clear` | 管理历史引用 |
| `/notebook add <ref>\|clear` | 管理笔记本引用 |
| `/regenerate`(别名 `/retry` | 重跑上一条用户消息 |
| `/show last\|<n>` | 展开被截断的工具结果或折叠的思考过程 |
| `/refs` | 查看当前设置 |
| `/config show\|set\|clear` | 管理 capability 配置 |
回答生成期间按 `Ctrl-C` 会取消当前 turn 并回到输入提示符;模型通过
`ask_user` 提问时,会在终端内渲染选项卡片并等待输入(非交互式 stdin
下自动提交空回复,turn 不会挂起)。
---
## `serve` — 启动 API 服务
```bash
deeptutor serve [--host 0.0.0.0] [--port 8001] [--reload]
```
`deeptutor serve` 需要完整 Web/API 依赖;如果你是通过本地 `./packaging/deeptutor-cli` 安装的 CLI-only 包,请先卸载本地 CLI 包并切换到 `pip install -U deeptutor`
---
## 资源管理命令
### `kb` — 知识库
```bash
deeptutor kb list # 列出所有知识库
deeptutor kb info <name> # 查看详情
deeptutor kb create <name> --doc file.pdf # 创建并导入文档
deeptutor kb create <name> --docs-dir ./docs/ # 从目录批量导入
deeptutor kb add <name> --doc extra.pdf # 追加文档
deeptutor kb set-default <name> # 设为默认
deeptutor kb search <name> "查询内容" # 搜索
deeptutor kb delete <name> --force # 删除
```
### `session` — 会话
```bash
deeptutor session list [--limit 20]
deeptutor session show <id>
deeptutor session open <id> # 进入 REPL 继续对话
deeptutor session rename <id> --title "新标题"
deeptutor session delete <id>
```
### `notebook` — 笔记本
```bash
deeptutor notebook list
deeptutor notebook create "笔记" --description "描述"
deeptutor notebook show <id>
deeptutor notebook add-md <id> ./notes.md
deeptutor notebook replace-md <id> <record_id> ./updated.md
deeptutor notebook remove-record <id> <record_id>
```
### `memory` — 长期记忆
```bash
deeptutor memory show
deeptutor memory clear --force
```
### `plugin` — 插件信息
```bash
deeptutor plugin list # 查看所有工具和 capability
deeptutor plugin info <name> # 查看详情
```
### `config` — 配置
```bash
deeptutor config show
```
### `provider` — 提供方认证 / 校验
```bash
deeptutor provider login openai-codex # 执行 OpenAI Codex OAuth 登录
deeptutor provider login github-copilot # 校验现有 GitHub Copilot 认证是否可用
```
---
## 典型工作流
```bash
# 1. 创建知识库
deeptutor kb create calculus --doc 微积分教材.pdf
# 2. 用知识库解题
deeptutor run deep_solve "求 ∫sin(x)cos(x)dx" -t rag --kb calculus -l zh
# 3. 基于知识库出题
deeptutor run deep_question "微积分" --kb calculus \
--config num_questions=5 --config difficulty=medium -l zh
# 4. 深度研究某课题
deeptutor run deep_research "注意力机制演进" \
--config-json '{"mode":"report","depth":"deep","sources":["papers","web"]}' -l zh
# 5. 查看会话记录
deeptutor session list
```
+7
View File
@@ -0,0 +1,7 @@
"""
DeepTutor CLI
=============
Command-line interface for DeepTutor.
Supports: ``python -m deeptutor`` or the ``deeptutor`` entry point.
"""
+5
View File
@@ -0,0 +1,5 @@
"""Allow running as ``python -m deeptutor_cli`` or ``deeptutor``."""
from deeptutor_cli.main import main
main()
+166
View File
@@ -0,0 +1,166 @@
"""Tool-result truncation and on-demand expansion buffer for the CLI.
Tool outputs in agent loops (RAG hits, web search dumps, file reads, code
execution stdout) can run to hundreds of lines. Streaming the full body into
the terminal drowns the reasoning around it and forces the user to scroll
past noise on every turn. We instead show a short head and stash the full
body so the REPL can re-print it on request — mirroring the
``… +N lines (ctrl+o to expand)`` UX users already know from other CLI
agents.
This module is pure logic so it can be unit-tested without a live REPL:
* :func:`truncate_for_display` decides what to show now.
* :class:`ToolResultBuffer` remembers the full bodies so the
``/show`` REPL command (and any future Ctrl+O keybinding) can expand
them later.
"""
from __future__ import annotations
from dataclasses import dataclass, field
# Defaults tuned for "agent reasoning is the signal, tool dumps are the
# noise". Ten lines is enough to recognise the result; long single lines
# (a giant JSON blob, a binary blob) get a hard character cap so the head
# never blows past one screen.
DEFAULT_HEAD_LINES = 10
DEFAULT_LINE_HARD_CAP = 240
# Cap how many recent tool results we hold onto so a long REPL session
# doesn't keep every tool dump in memory forever. The /show command only
# needs the recent ones; older results stay in the session DB if anyone
# wants the full archive.
DEFAULT_RING_CAPACITY = 32
def truncate_for_display(
body: str,
*,
head_lines: int = DEFAULT_HEAD_LINES,
line_hard_cap: int = DEFAULT_LINE_HARD_CAP,
) -> tuple[str, int]:
"""Return ``(visible_text, hidden_line_count)`` for an inline preview.
``hidden_line_count == 0`` means the full body fits within the budget
and the caller should not append an expansion hint. The returned
``visible_text`` already has overlong single lines clipped with an
ellipsis so the head can't blow past one screen on JSON-style output.
"""
if head_lines <= 0:
return "", _line_count(body)
lines = body.split("\n")
if len(lines) <= head_lines:
head_slice = lines
hidden = 0
else:
head_slice = lines[:head_lines]
hidden = len(lines) - head_lines
clipped = [_clip_long_line(line, line_hard_cap) for line in head_slice]
return "\n".join(clipped), hidden
def _line_count(body: str) -> int:
if not body:
return 0
return body.count("\n") + 1
def _clip_long_line(line: str, line_hard_cap: int) -> str:
if line_hard_cap <= 0 or len(line) <= line_hard_cap:
return line
# Keep most of the line, append a marker so the reader knows the line
# itself was clipped (independent of the line-count hidden marker).
return line[: line_hard_cap - 1] + ""
@dataclass(slots=True)
class ToolResultEntry:
"""One captured tool result that can later be expanded by ``/show``."""
index: int # 1-based, monotonically increasing per REPL session
label: str # tool name as reported by the stream
body: str # untruncated text
@dataclass(slots=True)
class ToolResultBuffer:
"""Bounded ring of recent tool results.
The buffer is intentionally tiny — its only purpose is to back the
``/show`` REPL command, not to be a transcript store.
"""
capacity: int = DEFAULT_RING_CAPACITY
head_lines: int = DEFAULT_HEAD_LINES
line_hard_cap: int = DEFAULT_LINE_HARD_CAP
_entries: list[ToolResultEntry] = field(default_factory=list)
_next_index: int = 1
def remember(self, label: str, body: str) -> ToolResultEntry:
entry = ToolResultEntry(index=self._next_index, label=label or "tool", body=body)
self._next_index += 1
self._entries.append(entry)
if len(self._entries) > self.capacity:
# Drop oldest; preserve indices so ``/show 7`` keeps meaning
# "the seventh tool result of this session" even after older
# entries fall out of the ring.
del self._entries[: len(self._entries) - self.capacity]
return entry
def truncate(self, body: str) -> tuple[str, int]:
"""Front-door to :func:`truncate_for_display` using this buffer's policy."""
return truncate_for_display(
body,
head_lines=self.head_lines,
line_hard_cap=self.line_hard_cap,
)
def last(self) -> ToolResultEntry | None:
return self._entries[-1] if self._entries else None
def get(self, selector: str | int | None) -> ToolResultEntry | None:
"""Resolve ``/show`` arguments: ``None``/``"last"`` → most recent;
a positive integer → entry with that 1-based index;
any other string → most recent entry whose label matches.
"""
if selector is None or selector == "" or selector == "last":
return self.last()
if isinstance(selector, int):
return self._by_index(selector)
text = str(selector).strip()
if text.isdigit():
return self._by_index(int(text))
for entry in reversed(self._entries):
if entry.label == text:
return entry
return None
def entries(self) -> list[ToolResultEntry]:
"""Snapshot of current contents (newest last). For inspection/tests."""
return list(self._entries)
def clear(self) -> None:
self._entries.clear()
self._next_index = 1
def _by_index(self, idx: int) -> ToolResultEntry | None:
for entry in self._entries:
if entry.index == idx:
return entry
return None
__all__ = [
"DEFAULT_HEAD_LINES",
"DEFAULT_LINE_HARD_CAP",
"DEFAULT_RING_CAPACITY",
"ToolResultBuffer",
"ToolResultEntry",
"truncate_for_display",
]
+61
View File
@@ -0,0 +1,61 @@
"""``deeptutor book ...`` CLI commands for the new BookEngine.
Currently exposes maintenance commands. (Authoring/reading still goes through
the API + web frontend.)
"""
from __future__ import annotations
import json
import typer
from .common import console
def register(app: typer.Typer) -> None:
@app.command("list")
def list_books() -> None:
"""List all books in the local workspace."""
from deeptutor.book import get_book_engine
engine = get_book_engine()
books = engine.list_books()
if not books:
console.print("[yellow]No books yet.[/yellow]")
return
for book in books:
stale = len(book.stale_page_ids or [])
stale_label = f" [red]({stale} stale)[/red]" if stale else ""
console.print(
f"[bold]{book.title or '(untitled)'}[/bold] "
f"[dim]{book.id}[/dim] "
f"[cyan]{book.status.value}[/cyan]"
f"{stale_label}"
)
@app.command("health")
def health(
book_id: str = typer.Argument(..., help="Book id."),
) -> None:
"""Inspect KB drift + log.md health for a book."""
from deeptutor.book import get_book_engine
engine = get_book_engine()
drift = engine.kb_drift_report(book_id)
log = engine.log_health(book_id)
console.print_json(json.dumps({"kb_drift": drift, "log_health": log}))
@app.command("refresh-fingerprints")
def refresh_fingerprints(
book_id: str = typer.Argument(..., help="Book id."),
) -> None:
"""Re-snapshot KB fingerprints; clears the stale-page list."""
from deeptutor.book import get_book_engine
engine = get_book_engine()
result = engine.refresh_kb_fingerprints(book_id)
if result is None:
console.print(f"[red]Book {book_id} not found.[/red]")
raise typer.Exit(code=1)
console.print_json(json.dumps(result))
+361
View File
@@ -0,0 +1,361 @@
"""Interactive chat REPL."""
from __future__ import annotations
from contextlib import suppress
from dataclasses import dataclass, field
import json
import shlex
from typing import Any
from rich.panel import Panel
import typer
from deeptutor.app import DeepTutorApp, TurnRequest
from .common import (
console,
maybe_run,
parse_config_items,
parse_json_object,
regenerate_and_render,
render_tool_result_entry,
run_turn_and_render,
tool_results,
)
@dataclass
class ChatState:
session_id: str | None = None
capability: str = "chat"
tools: list[str] = field(default_factory=list)
knowledge_bases: list[str] = field(default_factory=list)
language: str = "en"
notebook_references: list[dict[str, Any]] = field(default_factory=list)
history_references: list[str] = field(default_factory=list)
config: dict[str, Any] = field(default_factory=dict)
def register(app: typer.Typer) -> None:
@app.callback(invoke_without_command=True)
def chat(
ctx: typer.Context,
session: str | None = typer.Option(None, "--session", help="Resume an existing session."),
tool: list[str] = typer.Option([], "--tool", "-t", help="Pre-enable tool(s)."),
capability: str = typer.Option("chat", "--capability", "-c", help="Initial capability."),
kb: list[str] = typer.Option([], "--kb", help="Pre-attach knowledge base(s)."),
notebook_ref: list[str] = typer.Option([], "--notebook-ref", help="Notebook references."),
history_ref: list[str] = typer.Option([], "--history-ref", help="Referenced session ids."),
language: str = typer.Option("en", "--language", "-l", help="Response language."),
config: list[str] = typer.Option([], "--config", help="Initial config key=value."),
config_json: str | None = typer.Option(
None, "--config-json", help="Initial config as JSON."
),
) -> None:
"""Enter interactive chat REPL. Use `deeptutor run` for single-turn execution."""
if ctx.invoked_subcommand is not None:
return
try:
initial_config = parse_json_object(config_json)
initial_config.update(parse_config_items(config))
except ValueError as exc:
raise typer.BadParameter(str(exc)) from exc
state = ChatState(
session_id=session,
capability=capability,
tools=list(tool),
knowledge_bases=list(kb),
language=language,
notebook_references=_parse_notebook_refs(notebook_ref),
history_references=[item.strip() for item in history_ref if item.strip()],
config=initial_config,
)
maybe_run(_chat_repl(state))
async def _chat_repl(state: ChatState) -> None:
client = DeepTutorApp()
cron_service = None
try:
from deeptutor.services.cron import get_cron_service
cron_service = get_cron_service()
await cron_service.start()
except Exception:
cron_service = None
if state.session_id:
existing = await client.get_session(state.session_id)
if existing is None:
console.print(f"[red]Session not found:[/] {state.session_id}")
raise typer.Exit(code=1)
preferences = existing.get("preferences", {}) or {}
state.capability = str(preferences.get("capability") or state.capability or "chat")
state.tools = list(preferences.get("tools") or state.tools)
state.knowledge_bases = list(preferences.get("knowledge_bases") or state.knowledge_bases)
state.language = str(preferences.get("language") or state.language)
state.notebook_references = list(
preferences.get("notebook_references") or state.notebook_references
)
state.history_references = list(
preferences.get("history_references") or state.history_references
)
console.print(
Panel(
"[bold]DeepTutor CLI[/]\n"
"Type a message to chat. Ctrl-C interrupts a running turn. Commands:\n"
" /quit /session /status /new /clear\n"
" /regenerate (alias /retry) — re-run the last user message\n"
" /tool on|off <name>\n"
" /cap <name>\n"
" /kb <name>|none\n"
" /history add <id> | /history clear\n"
" /notebook add <ref> | /notebook clear\n"
" /show last|<n> — expand a tool result or captured thinking\n"
" /refs /config show|set|clear",
title="deeptutor chat",
)
)
_print_state(state)
try:
while True:
try:
user_input = _read_repl_input()
except (EOFError, KeyboardInterrupt):
console.print()
break
if not user_input:
continue
if user_input.startswith("/"):
command = user_input.split(maxsplit=1)[0].lower()
if command in {"/regenerate", "/retry"}:
if not state.session_id:
console.print("[yellow]No active session yet — send a message first.[/]")
continue
result = await regenerate_and_render(
app=client,
session_id=state.session_id,
capability=state.capability,
fmt="rich",
)
if result is not None:
session, _turn = result
state.session_id = str(session["id"])
continue
should_continue = _apply_command(user_input, state)
if should_continue:
continue
break
request = TurnRequest(
content=user_input,
capability=state.capability,
session_id=state.session_id,
tools=list(state.tools),
knowledge_bases=list(state.knowledge_bases),
language=state.language,
config=dict(state.config),
notebook_references=list(state.notebook_references),
history_references=list(state.history_references),
)
session, _turn = await run_turn_and_render(app=client, request=request, fmt="rich")
state.session_id = str(session["id"])
finally:
if cron_service is not None:
with suppress(Exception):
await cron_service.stop()
def _apply_command(raw: str, state: ChatState) -> bool:
try:
parts = shlex.split(raw)
except ValueError as exc:
console.print(f"[yellow]Could not parse command:[/] {exc}")
return True
if not parts:
return True
command = parts[0].lower()
if command == "/quit":
return False
if command == "/session":
console.print(f"session={state.session_id or '(new)'}")
return True
if command == "/status":
_print_state(state)
return True
if command in {"/new", "/clear"}:
state.session_id = None
console.print("[dim]Started a new chat context.[/]")
return True
if command == "/refs":
_print_refs(state)
return True
if command == "/tool" and len(parts) >= 3:
action, tool_name = parts[1], parts[2]
if action == "on" and tool_name not in state.tools:
state.tools.append(tool_name)
elif action == "off" and tool_name in state.tools:
state.tools.remove(tool_name)
_print_state(state)
return True
if command == "/cap" and len(parts) >= 2:
state.capability = parts[1]
_print_state(state)
return True
if command == "/kb" and len(parts) >= 2:
value = parts[1]
state.knowledge_bases = [] if value == "none" else [value]
_print_state(state)
return True
if command == "/history" and len(parts) >= 2:
if parts[1] == "clear":
state.history_references = []
elif parts[1] == "add" and len(parts) >= 3:
state.history_references.append(parts[2])
_print_state(state)
return True
if command == "/notebook" and len(parts) >= 2:
if parts[1] == "clear":
state.notebook_references = []
elif parts[1] == "add" and len(parts) >= 3:
state.notebook_references.extend(_parse_notebook_refs([parts[2]]))
_print_state(state)
return True
if command == "/show":
selector = parts[1] if len(parts) >= 2 else "last"
entry = tool_results.get(selector)
if entry is None:
if selector == "last":
console.print("[dim]No tool result captured yet in this session.[/]")
else:
console.print(
f"[dim]No tool result matches [bold]{selector}[/]. "
f"Available: {[e.index for e in tool_results.entries()] or 'none'}.[/]"
)
else:
render_tool_result_entry(entry)
return True
if command == "/config" and len(parts) >= 2:
subcommand = parts[1]
if subcommand == "show":
console.print_json(_format_config(state.config))
elif subcommand == "clear":
state.config = {}
elif subcommand == "set":
parsed = _parse_config_assignment(parts)
if parsed is None:
console.print("[yellow]Usage:[/] /config set key=value or /config set key value")
return True
key, value = parsed
state.config[key] = _parse_config_value(value)
_print_state(state)
return True
console.print("[dim]Unknown command.[/]")
return True
def _read_repl_input() -> str:
"""Read one REPL message, supporting backslash-continued multi-line input."""
lines: list[str] = []
prompt = "[bold green]You>[/] "
while True:
line = console.input(prompt)
if line.endswith("\\"):
lines.append(line[:-1])
prompt = "[dim]...[/] "
continue
lines.append(line)
return "\n".join(lines).strip()
def _print_state(state: ChatState) -> None:
console.print(
"[dim]"
f"session={state.session_id or '(new)'} "
f"capability={state.capability} "
f"tools={_format_list(state.tools)} "
f"kb={_format_list(state.knowledge_bases)} "
f"history={_format_list(state.history_references)} "
f"notebook_refs={_format_notebook_refs(state.notebook_references)} "
f"language={state.language} "
f"config={_format_config(state.config)}"
"[/]",
highlight=False,
)
def _print_refs(state: ChatState) -> None:
console.print("[bold]Current state:[/]")
console.print(f" session {state.session_id or '(new)'}")
console.print(f" capability {state.capability}")
console.print(f" tools {_format_list(state.tools)}")
console.print(f" kb {_format_list(state.knowledge_bases)}")
console.print(f" history {_format_list(state.history_references)}")
console.print(f" notebooks {_format_notebook_refs(state.notebook_references)}")
console.print(f" language {state.language}")
console.print(f" config {_format_config(state.config)}")
def _format_list(items: list[str]) -> str:
return "[" + ", ".join(items) + "]" if items else "[]"
def _format_notebook_refs(refs: list[dict[str, Any]]) -> str:
if not refs:
return "[]"
rendered = []
for ref in refs:
notebook_id = str(ref.get("notebook_id") or "")
record_ids = [str(item) for item in ref.get("record_ids") or []]
rendered.append(f"{notebook_id}:{','.join(record_ids)}" if record_ids else notebook_id)
return "[" + ", ".join(rendered) + "]"
def _format_config(config: dict[str, Any]) -> str:
return json.dumps(config, ensure_ascii=False, sort_keys=True)
def _parse_config_assignment(parts: list[str]) -> tuple[str, str] | None:
if len(parts) >= 3 and "=" in parts[2]:
key, _, value = parts[2].partition("=")
key = key.strip()
return (key, value) if key else None
if len(parts) >= 4:
key = parts[2].strip()
value = " ".join(parts[3:]).strip()
return (key, value) if key and value else None
return None
def _parse_notebook_refs(values: list[str]) -> list[dict[str, Any]]:
refs = []
for value in values:
notebook_id, _, record_ids_part = value.partition(":")
notebook_id = notebook_id.strip()
if not notebook_id:
raise typer.BadParameter(f"Invalid notebook reference `{value}`.")
record_ids = [item.strip() for item in record_ids_part.split(",") if item.strip()]
refs.append({"notebook_id": notebook_id, "record_ids": record_ids})
return refs
def _parse_config_value(raw_value: str) -> Any:
try:
return json.loads(raw_value)
except (json.JSONDecodeError, TypeError):
lowered = raw_value.lower()
if lowered == "true":
return True
if lowered == "false":
return False
if lowered in {"null", "none"}:
return None
return raw_value
+859
View File
@@ -0,0 +1,859 @@
"""Shared CLI helpers."""
from __future__ import annotations
import asyncio
import contextlib
import json
from pathlib import Path
import signal
import sys
from typing import Any, Callable
from rich.console import Console
from rich.markdown import Markdown
from rich.markup import escape
from rich.status import Status
from rich.table import Table
from rich.text import Text
from deeptutor.app import DeepTutorApp, TurnRequest
from ._tool_result import ToolResultBuffer, ToolResultEntry
console = Console()
# Process-wide buffer that backs the ``/show`` REPL command. The buffer
# lives at module scope so a single ``deeptutor chat`` session shares one
# ring across turns; ``deeptutor run`` doesn't read it (single-shot mode),
# but populating it is harmless.
tool_results = ToolResultBuffer()
# Content call_kinds whose ``call_status: running`` marker drives the
# spinner instead of printing a progress line (one LLM round each).
_LLM_ROUND_CALL_KINDS = frozenset({"agent_loop_round", "llm_final_response"})
class TurnInterrupted(Exception):
"""User aborted the running turn (Ctrl-C / Ctrl-D at a prompt)."""
def parse_config_items(items: list[str]) -> dict[str, Any]:
config: dict[str, Any] = {}
for item in items:
key, sep, raw_value = item.partition("=")
if not sep or not key.strip():
raise ValueError(f"Invalid --config item `{item}`. Expected KEY=VALUE.")
config[key.strip()] = _parse_scalar_value(raw_value.strip())
return config
def parse_json_object(raw: str | None) -> dict[str, Any]:
normalized = (raw or "").strip()
if not normalized:
return {}
try:
value = json.loads(normalized)
except (json.JSONDecodeError, TypeError) as exc:
raise ValueError(f"Invalid JSON config: {exc}") from exc
if not isinstance(value, dict):
raise ValueError("JSON config must be an object.")
return value
def parse_notebook_references(items: list[str]) -> list[dict[str, Any]]:
refs: list[dict[str, Any]] = []
for item in items:
notebook_id, _, record_part = item.partition(":")
resolved_notebook_id = notebook_id.strip()
if not resolved_notebook_id:
raise ValueError(f"Invalid notebook reference `{item}`.")
record_ids = [
record_id.strip() for record_id in record_part.split(",") if record_id.strip()
]
refs.append({"notebook_id": resolved_notebook_id, "record_ids": record_ids})
return refs
async def run_turn_and_render(
*,
app: DeepTutorApp,
request: TurnRequest,
fmt: str,
) -> tuple[dict[str, Any], dict[str, Any]]:
session, turn = await app.start_turn(request)
if fmt == "json":
await stream_turn_as_json(app=app, turn_id=turn["id"])
return session, turn
summary = await render_turn_stream(app=app, turn_id=turn["id"])
console.print(
f"[dim]session={session['id']} turn={turn['id']} "
f"capability={request.capability}{summary}[/]",
highlight=False,
)
return session, turn
async def regenerate_and_render(
*,
app: DeepTutorApp,
session_id: str,
capability: str = "chat",
fmt: str = "rich",
) -> tuple[dict[str, Any], dict[str, Any]] | None:
try:
session, turn = await app.regenerate_last_turn(session_id)
except RuntimeError as exc:
reason = str(exc)
if reason == "regenerate_busy":
console.print(
"[yellow]Cannot regenerate while another turn is running. "
"Wait for it to finish or cancel it first.[/]"
)
elif reason == "nothing_to_regenerate":
console.print("[yellow]Nothing to regenerate yet — send a message first.[/]")
else:
console.print(f"[red]Regenerate failed:[/] {reason}")
return None
if fmt == "json":
await stream_turn_as_json(app=app, turn_id=turn["id"])
return session, turn
summary = await render_turn_stream(app=app, turn_id=turn["id"])
console.print(
f"[dim]session={session['id']} turn={turn['id']} "
f"capability={capability}{summary} (regenerated)[/]",
highlight=False,
)
return session, turn
async def stream_turn_as_json(*, app: DeepTutorApp, turn_id: str) -> None:
"""NDJSON passthrough for ``--format json``.
``ask_user`` pauses are auto-resolved with an empty reply so headless
runs cannot hang on a question no one will answer — the model sees
"(empty reply)" as the tool result and must proceed on its own.
"""
async for item in app.stream_turn(turn_id):
# One event per line: rich wraps long lines at terminal width and
# interprets ``[...]`` as markup, both of which corrupt NDJSON.
console.print(
json.dumps(item, ensure_ascii=False),
soft_wrap=True,
markup=False,
highlight=False,
)
if _ask_user_payload(item) is not None:
await app.submit_user_reply(turn_id, text="")
async def render_turn_stream(*, app: DeepTutorApp, turn_id: str) -> str:
"""Render a turn's event stream; returns a ``key=value`` summary suffix
(rounds / tools / tokens / cost) for the caller's session line.
Ctrl-C while the turn is streaming cancels the turn server-side and
returns control to the caller (the REPL keeps running) instead of
unwinding the whole CLI. While an ``ask_user`` prompt is on screen the
default KeyboardInterrupt behaviour is restored so Ctrl-C aborts the
prompt (and with it the turn).
"""
task = asyncio.current_task()
interrupted = False
def _on_sigint() -> None:
nonlocal interrupted
if interrupted:
return
interrupted = True
if task is not None:
task.cancel()
sigint = _SigintInterceptor(_on_sigint)
renderer = TurnStreamRenderer(app=app, turn_id=turn_id, sigint=sigint)
sigint.resume()
try:
async for item in app.stream_turn(turn_id):
await renderer.handle(item)
except TurnInterrupted:
await _cancel_interrupted_turn(app=app, turn_id=turn_id, renderer=renderer)
except asyncio.CancelledError:
if not interrupted:
renderer.close()
raise
# Our own SIGINT handler cancelled us; absorb the cancellation so
# the REPL survives, then cancel the turn server-side. Drain the
# whole cancel count — a hammered Ctrl-C must not leave a pending
# cancellation that detonates at the next await.
if task is not None:
while task.cancelling() > 0:
task.uncancel()
await _cancel_interrupted_turn(app=app, turn_id=turn_id, renderer=renderer)
finally:
sigint.suspend()
renderer.close()
return renderer.summary_suffix()
async def _cancel_interrupted_turn(
*,
app: DeepTutorApp,
turn_id: str,
renderer: "TurnStreamRenderer",
) -> None:
renderer.abort()
with contextlib.suppress(Exception):
await app.cancel_turn(turn_id)
console.print("\n[dim]Interrupted — turn cancelled.[/]")
class _SigintInterceptor:
"""Route Ctrl-C to a callback while a turn is streaming (POSIX only).
``suspend``/``resume`` bracket blocking prompts so ``console.input``
gets the default KeyboardInterrupt behaviour back (an installed
asyncio handler never fires while the loop is blocked in a read).
On platforms without ``add_signal_handler`` (Windows) this is a no-op
and Ctrl-C falls through to ``maybe_run``'s graceful exit.
"""
def __init__(self, on_sigint: Callable[[], None]) -> None:
self._on_sigint = on_sigint
self._installed = False
def resume(self) -> None:
if self._installed:
return
try:
asyncio.get_running_loop().add_signal_handler(signal.SIGINT, self._on_sigint)
self._installed = True
except (NotImplementedError, RuntimeError, ValueError, AttributeError):
self._installed = False
def suspend(self) -> None:
if not self._installed:
return
with contextlib.suppress(Exception):
asyncio.get_running_loop().remove_signal_handler(signal.SIGINT)
self._installed = False
def _stdin_interactive() -> bool:
try:
return sys.stdin is not None and sys.stdin.isatty()
except Exception:
return False
def _ask_user_payload(item: dict[str, Any]) -> dict[str, Any] | None:
"""The ``ask_user`` question payload carried by a tool_result event."""
if str(item.get("type", "")) != "tool_result":
return None
metadata = item.get("metadata") or {}
tool_meta = metadata.get("tool_metadata")
if not isinstance(tool_meta, dict):
return None
ask = tool_meta.get("ask_user")
if isinstance(ask, dict) and ask.get("questions"):
return ask
return None
class TurnStreamRenderer:
"""State machine that renders one turn's event stream.
The chat agent loop streams EVERY round's text as ``content`` chunks
and only labels the round once it completes — a ``call_status`` marker
whose ``call_role`` says whether that text was ``narration`` (preamble
to tool calls; rendered dim, in place, before its tools) or the
``finish`` (the user-facing answer; rendered as Markdown). Chunks are
therefore buffered per ``call_id`` and settled when the round's marker
arrives — the marker is emitted before the round's tool calls
dispatch, so terminal order matches the model's order.
Content without that trace metadata (other capabilities) keeps the
legacy behaviour: buffer and render at stage boundaries / done.
"""
def __init__(
self,
*,
app: DeepTutorApp,
turn_id: str,
sigint: _SigintInterceptor | None = None,
) -> None:
self.app = app
self.turn_id = turn_id
self._sigint = sigint
self._legacy_buf = ""
self._round_bufs: dict[str, str] = {}
self._round_order: list[str] = []
self._thinking_bufs: dict[str, str] = {}
self._thinking_indicated: set[str] = set()
self._status: Status | None = None
self._sources: list[dict[str, Any]] = []
self._result_meta: dict[str, Any] = {}
async def handle(self, item: dict[str, Any]) -> None:
handler = getattr(self, f"_on_{str(item.get('type', ''))}", None)
if handler is not None:
await handler(item)
# ---- lifecycle -------------------------------------------------------
def abort(self) -> None:
"""Settle whatever already streamed (the backend persists the same
partial text on cancel), then stop the spinner."""
with contextlib.suppress(Exception):
self._flush_pending()
self._status_stop()
def close(self) -> None:
self._status_stop()
def summary_suffix(self) -> str:
meta = self._result_meta
parts: list[str] = []
rounds = meta.get("rounds")
if isinstance(rounds, int) and rounds > 0:
parts.append(f"rounds={rounds}")
tool_steps = meta.get("tool_steps")
if isinstance(tool_steps, int) and tool_steps > 0:
parts.append(f"tools={tool_steps}")
cost = (meta.get("metadata") or {}).get("cost_summary") or {}
tokens = cost.get("total_tokens")
if isinstance(tokens, (int, float)) and tokens > 0:
parts.append(f"tokens={_format_tokens(int(tokens))}")
cost_usd = cost.get("total_cost_usd")
if isinstance(cost_usd, (int, float)) and cost_usd > 0:
parts.append(f"cost=${cost_usd:.4f}")
return (" " + " ".join(parts)) if parts else ""
# ---- event handlers --------------------------------------------------
async def _on_stage_start(self, item: dict[str, Any]) -> None:
self._flush_pending()
stage = str(item.get("stage", "") or "")
if str(item.get("source", "")) == "chat" and stage == "responding":
# The chat loop is one wrapper stage; a banner adds nothing.
return
self._status_stop()
console.print(f"\n[bold cyan]▶ {stage or 'working'}[/]", highlight=False)
async def _on_stage_end(self, item: dict[str, Any]) -> None:
self._flush_pending()
async def _on_thinking(self, item: dict[str, Any]) -> None:
metadata = item.get("metadata") or {}
call_id = str(metadata.get("call_id") or "")
text = str(item.get("content", "") or "")
if not call_id:
# Legacy capabilities emit coarse-grained thinking lines.
if text.strip():
console.print(f" [dim]{escape(text)}[/]", highlight=False)
return
# Chat-loop reasoning streams chunk-by-chunk: collapse it to one
# indicator and stash the full text for ``/show`` at settle time.
self._thinking_bufs[call_id] = self._thinking_bufs.get(call_id, "") + text
if call_id not in self._thinking_indicated:
self._thinking_indicated.add(call_id)
if self._status is not None:
self._status.update("[dim]thinking…[/]")
console.print(" [dim]✻ thinking…[/]", highlight=False)
else:
console.print(" [dim]✻ thinking…[/]", highlight=False)
async def _on_progress(self, item: dict[str, Any]) -> None:
metadata = item.get("metadata") or {}
content = str(item.get("content", "") or "")
if metadata.get("trace_kind") == "call_status":
await self._on_call_status(content, metadata)
return
if not content.strip():
return
console.print(f" [dim]{escape(content)}[/]", highlight=False)
async def _on_call_status(self, content: str, metadata: dict[str, Any]) -> None:
call_state = str(metadata.get("call_state") or "")
if call_state == "complete":
call_role = str(metadata.get("call_role") or "")
if call_role in {"narration", "finish"}:
self._settle_round(str(metadata.get("call_id") or ""), role=call_role)
return
if content.strip():
console.print(f" [dim]{escape(content)}[/]", highlight=False)
return
if metadata.get("call_kind") in _LLM_ROUND_CALL_KINDS:
# One LLM round starting — show liveness without a printed line.
self._status_start(content.strip() or "working")
return
if content.strip():
console.print(f" [dim]{escape(content)}[/]", highlight=False)
async def _on_content(self, item: dict[str, Any]) -> None:
metadata = item.get("metadata") or {}
text = str(item.get("content", "") or "")
if not text:
return
call_id = str(metadata.get("call_id") or "")
trace_kind = str(metadata.get("trace_kind") or "")
if call_id and trace_kind == "llm_chunk":
if call_id not in self._round_bufs:
self._round_bufs[call_id] = ""
self._round_order.append(call_id)
self._round_bufs[call_id] += text
if self._status is not None:
self._status.update("[dim]writing…[/]")
return
if trace_kind == "llm_output":
# Whole-text emission (terminator tool / section / fallback).
# Flush buffered chunks first so blocks keep the model's order.
self._flush_pending()
self._status_stop()
console.print(Markdown(text))
return
self._legacy_buf += text
async def _on_tool_call(self, item: dict[str, Any]) -> None:
_render_tool_call(item)
async def _on_tool_result(self, item: dict[str, Any]) -> None:
ask = _ask_user_payload(item)
if ask is not None:
await self._handle_ask_user(ask)
return
_render_tool_result(item)
async def _on_error(self, item: dict[str, Any]) -> None:
self._status_stop()
console.print(f"[bold red]Error:[/] {escape(str(item.get('content', '') or ''))}")
async def _on_sources(self, item: dict[str, Any]) -> None:
entries = (item.get("metadata") or {}).get("sources")
if isinstance(entries, list):
self._sources.extend(entry for entry in entries if isinstance(entry, dict))
async def _on_result(self, item: dict[str, Any]) -> None:
metadata = item.get("metadata")
if isinstance(metadata, dict):
self._result_meta = metadata
async def _on_done(self, item: dict[str, Any]) -> None:
self._flush_pending()
self._status_stop()
self._print_sources()
async def _on_wait_for_input(self, item: dict[str, Any]) -> None:
"""Legacy in-band input request (``StreamBus.wait_for_input``)."""
from deeptutor.core.stream_bus import get_bus
self._status_stop()
bus = get_bus(self.turn_id)
if bus is None:
return
if not _stdin_interactive():
bus.submit_input("")
return
prompt = str(item.get("content", "") or "").strip()
if prompt:
console.print(f"\n [bold cyan]?[/] {escape(prompt)}", highlight=False)
raw = self._read_line(" [bold green]answer>[/] ")
if raw is None:
raise TurnInterrupted
bus.submit_input(raw)
# ---- ask_user --------------------------------------------------------
async def _handle_ask_user(self, ask: dict[str, Any]) -> None:
self._status_stop()
if not _stdin_interactive():
console.print(
" [yellow]●[/] [dim]ask_user: stdin is not interactive — sending an "
"empty reply so the turn can continue.[/]",
highlight=False,
)
await self.app.submit_user_reply(self.turn_id, text="")
return
answers = self._prompt_ask_user(ask)
if answers is None:
raise TurnInterrupted
await self.app.submit_user_reply(self.turn_id, answers=answers)
def _prompt_ask_user(self, ask: dict[str, Any]) -> list[dict[str, str]] | None:
"""Render the question card and collect one answer per question.
Returns ``None`` when the user aborts (Ctrl-C / Ctrl-D) — the
caller cancels the turn, mirroring the web composer's stop button.
"""
questions = [q for q in (ask.get("questions") or []) if isinstance(q, dict)]
if not questions:
return []
console.print()
console.print("[bold cyan]?[/] [bold]The model needs your input[/]", highlight=False)
intro = str(ask.get("intro") or "").strip()
if intro:
console.print(f" {escape(intro)}", highlight=False)
answers: list[dict[str, str]] = []
for index, question in enumerate(questions):
reply = self._prompt_one_question(question, index=index, total=len(questions))
if reply is None:
return None
answers.append(
{"questionId": str(question.get("id") or f"q{index + 1}"), "text": reply}
)
return answers
def _prompt_one_question(
self,
question: dict[str, Any],
*,
index: int,
total: int,
) -> str | None:
prompt = str(question.get("prompt") or "").strip()
header = str(question.get("header") or "").strip()
numbering = f"({index + 1}/{total}) " if total > 1 else ""
chip = f"[cyan]{escape(header)}[/] " if header else ""
console.print(f"\n {numbering}{chip}{escape(prompt)}", highlight=False)
labels: list[str] = []
for option in question.get("options") or []:
if not isinstance(option, dict):
continue
label = str(option.get("label") or "").strip()
if not label:
continue
labels.append(label)
description = str(option.get("description") or "").strip()
suffix = f" [dim]— {escape(description)}[/]" if description else ""
console.print(f" {len(labels)}. {escape(label)}{suffix}", highlight=False)
multi = bool(question.get("multi_select"))
hint = _answer_hint(
has_options=bool(labels),
multi=multi,
placeholder=str(question.get("placeholder") or "").strip(),
)
raw = self._read_line(f" [bold green]{hint}>[/] ")
if raw is None:
return None
return _resolve_answer(raw, labels, multi)
def _read_line(self, prompt: str) -> str | None:
"""Blocking prompt with default Ctrl-C semantics; ``None`` = abort."""
if self._sigint is not None:
self._sigint.suspend()
try:
return console.input(prompt)
except (KeyboardInterrupt, EOFError):
return None
finally:
if self._sigint is not None:
self._sigint.resume()
# ---- rendering internals ---------------------------------------------
def _settle_round(self, call_id: str, *, role: str) -> None:
"""Render a completed chat-loop round's buffered text.
``narration`` stays dim and lands before the round's tool lines
(the marker precedes tool dispatch); ``finish`` is the answer.
"""
text = self._round_bufs.pop(call_id, "")
if call_id in self._round_order:
self._round_order.remove(call_id)
thinking = self._thinking_bufs.pop(call_id, "").strip()
if thinking:
tool_results.remember("thinking", thinking)
body = text.strip()
if not body:
return
self._status_stop()
if role == "narration":
console.print(Text(body, style="dim"))
else:
console.print(Markdown(text))
def _flush_pending(self) -> None:
"""Render any unsettled buffered text as answer Markdown.
Chat rounds normally settle via their ``call_role`` marker; this is
the stage-boundary / done / llm_output fallback so no text is ever
dropped (and so other capabilities keep their old flush points).
"""
for call_id in list(self._round_order):
self._settle_round(call_id, role="finish")
if self._legacy_buf:
self._status_stop()
console.print(Markdown(self._legacy_buf))
self._legacy_buf = ""
def _print_sources(self) -> None:
if not self._sources:
return
seen: set[str] = set()
rows: list[str] = []
for source in self._sources:
title = str(
source.get("title")
or source.get("filename")
or source.get("file_name")
or source.get("source")
or source.get("url")
or ""
).strip()
location = str(source.get("url") or source.get("file_path") or "").strip()
key = location or title
if not key or key in seen:
continue
seen.add(key)
row = title or location
if location and location != row:
row = f"{row}{location}"
rows.append(row)
if not rows:
return
shown = rows[:8]
console.print(f"[dim]sources ({len(rows)}):[/]", highlight=False)
for index, row in enumerate(shown):
console.print(f" [dim][{index + 1}] {escape(row)}[/]", highlight=False)
if len(rows) > len(shown):
console.print(f" [dim]… +{len(rows) - len(shown)} more[/]", highlight=False)
def _status_start(self, label: str) -> None:
if not console.is_terminal:
return
text = f"[dim]{escape(label)}[/]"
if self._status is None:
self._status = console.status(text, spinner="dots")
self._status.start()
else:
self._status.update(text)
def _status_stop(self) -> None:
if self._status is not None:
with contextlib.suppress(Exception):
self._status.stop()
self._status = None
def _answer_hint(*, has_options: bool, multi: bool, placeholder: str) -> str:
if has_options and multi:
return "answer (numbers/text, comma-separated; Enter to skip)"
if has_options:
return "answer (number or text; Enter to skip)"
if placeholder:
return f"answer ({escape(placeholder)}; Enter to skip)"
return "answer (Enter to skip)"
def _resolve_answer(raw: str, labels: list[str], multi: bool) -> str:
"""Map ``1``-style selections onto option labels; pass text through.
Multi-select accepts comma-separated tokens, each a number or free
text; results join with ", " (the reply travels as one string).
"""
cleaned = raw.strip()
if not cleaned:
return ""
tokens = [t.strip() for t in cleaned.split(",") if t.strip()] if multi else [cleaned]
resolved: list[str] = []
for token in tokens:
if token.isdigit() and labels and 1 <= int(token) <= len(labels):
resolved.append(labels[int(token) - 1])
else:
resolved.append(token)
return ", ".join(resolved)
def _format_tokens(value: int) -> str:
if value >= 1000:
return f"{value / 1000:.1f}k"
return str(value)
def _render_tool_call(item: dict[str, Any]) -> None:
"""Print a one-line tool-call header. Long arg payloads are summarised
so the call stays scannable; the full body lands in tool_result if the
tool echoes it back, or in the stream metadata for debug tooling."""
tool_name = str(item.get("content", "") or "tool")
metadata = item.get("metadata", {}) or {}
args = metadata.get("args", {})
# Budget the args summary so the whole header — " ● <name>(<args>)" —
# fits the current terminal width on one line. We pick a soft floor so
# very narrow terminals still get something useful.
overhead = len(f"{tool_name}()")
budget = max(20, (console.width or 100) - overhead)
summary = _summarize_call_args(args, max_len=budget)
if summary:
console.print(f" [yellow]●[/] {tool_name}([dim]{escape(summary)}[/])", highlight=False)
else:
console.print(f" [yellow]●[/] {tool_name}", highlight=False)
def _render_tool_result(item: dict[str, Any]) -> None:
"""Print a truncated preview of a tool result, stashing the full text
in the shared :data:`tool_results` buffer so ``/show`` can expand it."""
body = str(item.get("content", "") or "")
metadata = item.get("metadata", {}) or {}
label = str(metadata.get("tool") or "tool")
entry = tool_results.remember(label, body)
head, hidden = tool_results.truncate(body)
# Empty result still gets a marker so the user can see the call closed.
if not head.strip() and not hidden:
console.print(
f" [green]└[/] [dim]#{entry.index} {label} → (empty result)[/]", highlight=False
)
return
if head:
for line in head.split("\n"):
console.print(f" [green]│[/] {escape(line)}", highlight=False)
if hidden:
console.print(
f" [green]└[/] [dim]#{entry.index} {label} — +{hidden} more line"
f"{'s' if hidden != 1 else ''}; "
f"run [bold]/show {entry.index}[/] (or [bold]/show last[/]) to expand[/]",
highlight=False,
)
else:
console.print(f" [green]└[/] [dim]#{entry.index} {label}[/]", highlight=False)
def _summarize_call_args(args: Any, max_len: int = 120) -> str:
"""Render call args as a short ``key=value, …`` string.
The full rendering is assembled first, then a single trailing-ellipsis
clip is applied so we never leave a dangling ``", "`` at the end when
the last key's value runs over the budget.
"""
if isinstance(args, dict) and args:
rendered = ", ".join(f"{key}={_one_line(value)}" for key, value in args.items())
elif args:
rendered = _one_line(args)
else:
return ""
if len(rendered) > max_len:
return rendered[: max_len - 1].rstrip(", ") + ""
return rendered
def _one_line(value: Any) -> str:
"""Compact one-line repr for a single arg value. No truncation here —
the caller's overall budget handles that uniformly so we don't double-
clip a dict and end up with a half-finished key=value pair."""
if isinstance(value, str):
text = value
else:
try:
text = json.dumps(value, ensure_ascii=False)
except (TypeError, ValueError):
text = repr(value)
return text.replace("\n", " ")
def render_tool_result_entry(entry: ToolResultEntry) -> None:
"""Fully print a stored tool result. Backs the ``/show`` REPL command."""
from rich.panel import Panel
console.print(
Panel(
escape(entry.body) if entry.body else "[dim](empty result)[/]",
title=f"#{entry.index} {entry.label}",
border_style="green",
),
highlight=False,
)
def build_turn_request(
*,
content: str,
capability: str,
session_id: str | None,
tools: list[str],
knowledge_bases: list[str],
language: str,
config_items: list[str],
config_json: str | None,
notebook_refs: list[str],
history_refs: list[str],
) -> TurnRequest:
config = parse_json_object(config_json)
config.update(parse_config_items(config_items))
return TurnRequest(
content=content,
capability=capability,
session_id=session_id,
tools=tools,
knowledge_bases=knowledge_bases,
language=language,
config=config,
notebook_references=parse_notebook_references(notebook_refs),
history_references=[item.strip() for item in history_refs if item.strip()],
)
def maybe_run(coro): # noqa: ANN001
try:
return asyncio.run(coro)
except KeyboardInterrupt:
console.print("\n[dim]Interrupted.[/]")
return None
def print_session_table(sessions: list[dict[str, Any]]) -> None:
table = Table(title="Sessions")
table.add_column("ID")
table.add_column("Title")
table.add_column("Capability")
table.add_column("Status")
table.add_column("Messages", justify="right")
for session in sessions:
table.add_row(
str(session.get("id", "")),
str(session.get("title", "")),
str(session.get("capability", "") or "chat"),
str(session.get("status", "")),
str(session.get("message_count", 0)),
)
console.print(table)
def print_notebook_table(notebooks: list[dict[str, Any]]) -> None:
table = Table(title="Notebooks")
table.add_column("ID")
table.add_column("Name")
table.add_column("Records", justify="right")
table.add_column("Description")
for notebook in notebooks:
table.add_row(
str(notebook.get("id", "")),
str(notebook.get("name", "")),
str(notebook.get("record_count", 0)),
str(notebook.get("description", "")),
)
console.print(table)
def print_path_result(path: str | Path) -> None:
console.print(f"[dim]{Path(path).resolve()}[/]")
def _parse_scalar_value(raw_value: str) -> Any:
lowered = raw_value.lower()
if lowered in {"true", "false"}:
return lowered == "true"
if lowered in {"null", "none"}:
return None
try:
return json.loads(raw_value)
except (json.JSONDecodeError, TypeError):
return raw_value
+92
View File
@@ -0,0 +1,92 @@
"""
CLI Config Command
==================
View and update DeepTutor configuration.
"""
from __future__ import annotations
from rich.console import Console
import typer
console = Console()
def register(app: typer.Typer) -> None:
@app.command("show")
def config_show() -> None:
"""Show current configuration."""
import json
from deeptutor.services.config import (
load_config_with_main,
load_system_settings,
resolve_embedding_runtime_config,
resolve_llm_runtime_config,
resolve_search_runtime_config,
)
system_settings = load_system_settings()
llm_runtime = resolve_llm_runtime_config()
search_runtime = resolve_search_runtime_config()
llm_info = {
"binding_hint": llm_runtime.binding_hint,
"provider": llm_runtime.provider_name,
"provider_mode": llm_runtime.provider_mode,
"model": llm_runtime.model,
"base_url": llm_runtime.effective_url,
"api_version": llm_runtime.api_version,
"extra_headers": llm_runtime.extra_headers,
"api_key": "***" if llm_runtime.api_key else "(not set)",
}
try:
embedding_runtime = resolve_embedding_runtime_config()
embedding_info = {
"status": "configured",
"binding_hint": embedding_runtime.binding_hint,
"provider": embedding_runtime.provider_name,
"provider_mode": embedding_runtime.provider_mode,
"model": embedding_runtime.model,
"base_url": embedding_runtime.effective_url,
"api_version": embedding_runtime.api_version,
"extra_headers": embedding_runtime.extra_headers,
"api_key": "***" if embedding_runtime.api_key else "(not set)",
"dimension": embedding_runtime.dimension,
}
except ValueError as exc:
embedding_info = {
"status": "not_configured",
"message": str(exc),
}
try:
main_cfg = load_config_with_main("main.yaml")
except Exception:
main_cfg = {}
console.print_json(
json.dumps(
{
"ports": {
"backend": system_settings["backend_port"],
"frontend": system_settings["frontend_port"],
},
"llm": llm_info,
"embedding": embedding_info,
"search": {
"provider": search_runtime.provider or "(optional)",
"requested_provider": search_runtime.requested_provider or "(optional)",
"status": search_runtime.status,
"fallback_reason": search_runtime.fallback_reason,
"base_url": search_runtime.base_url,
"proxy": search_runtime.proxy,
"api_key": "***" if search_runtime.api_key else "(not set)",
},
"language": main_cfg.get("system", {}).get("language", "en"),
"tools": list(main_cfg.get("tools", {}).keys()),
},
indent=2,
ensure_ascii=False,
)
)
+517
View File
@@ -0,0 +1,517 @@
"""Interactive runtime settings initializer.
``deeptutor init`` walks the user through a four-step wizard (ports → LLM →
embedding → review) that writes the same files as the Web Settings page.
Heavy lifting (provider menu, live ``/models`` fetch, connectivity probe,
review panel) lives in :mod:`deeptutor_cli.init_wizard`. This module is
intentionally thin so the order of steps is easy to read top-to-bottom.
"""
from __future__ import annotations
from pathlib import Path
from rich.console import Console
import typer
from deeptutor.runtime.home import DEEPTUTOR_HOME_ENV, get_runtime_home
from . import init_wizard as wiz
def _reset_runtime_singletons() -> None:
"""Drop cached service instances so the new DEEPTUTOR_HOME takes effect.
``deeptutor init`` may pass ``--home`` to target a different workspace; the
singletons cache paths from the *previous* PathService and will silently
write to the wrong place if not cleared.
"""
try:
from deeptutor.services.path_service import PathService
PathService.reset_instance()
except Exception:
pass
try:
from deeptutor.services.config.runtime_settings import RuntimeSettingsService
RuntimeSettingsService._instances.clear()
except Exception:
pass
try:
from deeptutor.services.config.model_catalog import ModelCatalogService
ModelCatalogService._instances.clear()
except Exception:
pass
def _ensure_model_service(catalog: dict, service_name: str, profile_id: str, model_id: str):
"""Locate or create the default profile + model rows we'll mutate in place."""
services = catalog.setdefault("services", {})
service = services.setdefault(
service_name,
{"active_profile_id": profile_id, "active_model_id": model_id, "profiles": []},
)
profiles = service.setdefault("profiles", [])
profile = next(
(item for item in profiles if item.get("id") == service.get("active_profile_id")), None
)
if profile is None:
profile = {
"id": profile_id,
"name": "Default LLM Endpoint"
if service_name == "llm"
else "Default Embedding Endpoint",
"binding": "openai",
"base_url": "",
"api_key": "",
"api_version": "",
"extra_headers": {},
"models": [],
}
profiles.append(profile)
service["active_profile_id"] = profile_id
models = profile.setdefault("models", [])
model = next(
(item for item in models if item.get("id") == service.get("active_model_id")), None
)
if model is None:
model = {"id": model_id, "name": "Default Model", "model": ""}
models.append(model)
service["active_model_id"] = model_id
return profile, model
def _llm_step(
console: Console,
strings: dict,
current_profile: dict,
current_model: dict,
) -> wiz.LLMChoice:
spec = wiz.select_llm_provider(
console,
strings,
current_binding=str(current_profile.get("binding") or "openai"),
)
if spec is not None:
binding = spec.name
default_base = spec.default_api_base or str(current_profile.get("base_url") or "")
display_provider = spec.label
env_key = spec.env_key
else:
binding = (
typer.prompt(
strings["init.binding"],
default=str(current_profile.get("binding") or "openai"),
).strip()
or "openai"
)
default_base = str(current_profile.get("base_url") or "")
display_provider = "Custom"
env_key = ""
edit_base = typer.confirm(strings["init.edit_base_url"], default=not bool(default_base))
if edit_base:
base_url = typer.prompt(strings["init.new_base_url"], default=default_base or "")
else:
base_url = default_base
wiz.info(console, f"Base URL · {base_url or '(empty)'}")
api_key = wiz.capture_api_key(
console,
strings,
env_key=env_key,
current=str(current_profile.get("api_key") or ""),
)
models = wiz.fetch_models(
console,
strings,
base_url=base_url,
api_key=api_key,
binding=binding,
)
if not models:
models = list(wiz.LLM_FALLBACK_MODELS.get(binding, ()))
model = wiz.select_model(
console,
strings,
models=models,
current=str(current_model.get("model") or ""),
)
choice = wiz.LLMChoice(
binding=binding,
base_url=base_url,
api_key=api_key,
model=model,
display_provider=display_provider,
)
if typer.confirm(strings["init.probe_offer"], default=True):
_probe_llm_with_retry(console, strings, choice)
return choice
def _probe_llm_with_retry(console: Console, strings: dict, choice: wiz.LLMChoice) -> None:
"""Run the probe; on failure, offer a single retry with a fresh API key."""
while True:
wiz.info(console, strings["init.probe_running"].format(what=choice.display_provider))
ok_result, elapsed_ms, error = wiz.probe_llm(
base_url=choice.base_url,
api_key=choice.api_key,
binding=choice.binding,
model=choice.model,
)
choice.probed = True
choice.probe_ok = ok_result
choice.probe_ms = elapsed_ms
if ok_result:
wiz.ok(
console,
strings["init.probe_ok"].format(what=choice.display_provider, ms=elapsed_ms),
)
return
wiz.fail(
console, strings["init.probe_fail"].format(what=choice.display_provider, error=error)
)
if not typer.confirm(strings["init.probe_retry"], default=False):
return
choice.api_key = typer.prompt(
strings["init.api_key_prompt"], default="", hide_input=True, show_default=False
)
def _embedding_step(
console: Console,
strings: dict,
catalog: dict,
llm_api_key: str,
) -> wiz.EmbeddingChoice | None:
"""Returns ``None`` when the user picks ``[s] Skip``."""
from deeptutor.services.config.embedding_endpoint import (
EMBEDDING_PROVIDER_LABELS,
normalize_embedding_endpoint_for_display,
)
from deeptutor.services.config.provider_runtime import EMBEDDING_PROVIDERS
current_profile = (catalog.get("services", {}).get("embedding", {}).get("profiles") or [{}])[
0
] or {}
current_binding = str(current_profile.get("binding") or "openai")
provider_pick = wiz.select_embedding_provider(console, strings, current=current_binding)
if provider_pick == wiz.SKIP_SENTINEL:
wiz.info(console, strings["init.skipped"])
return None
if provider_pick is None:
provider = typer.prompt(strings["init.binding"], default=current_binding or "openai")
else:
provider = provider_pick
spec = EMBEDDING_PROVIDERS.get(provider)
display_provider = (
spec.label if spec else EMBEDDING_PROVIDER_LABELS.get(provider, provider.title())
)
default_endpoint = spec.default_api_base if spec else str(current_profile.get("base_url") or "")
edit_endpoint = typer.confirm(strings["init.edit_base_url"], default=not bool(default_endpoint))
endpoint = (
typer.prompt(strings["init.embedding_endpoint"], default=default_endpoint)
if edit_endpoint
else default_endpoint
)
endpoint = normalize_embedding_endpoint_for_display(provider, endpoint)
if not edit_endpoint:
wiz.info(console, f"Endpoint · {endpoint or '(empty)'}")
# Reuse the LLM key by default — most users share creds across services.
masked = wiz._mask_secret(llm_api_key)
if llm_api_key and typer.confirm(
strings["init.api_key_reuse_llm"].format(masked=masked), default=True
):
api_key = llm_api_key
else:
api_key = typer.prompt(
strings["init.embedding_api_key"], default="", hide_input=True, show_default=False
)
# Try live ``/models`` first; fall back to the curated list (spec default
# first, then EMBEDDING_FALLBACK_MODELS) when the fetch returns nothing.
models = wiz.fetch_embedding_models(
console, strings, endpoint=endpoint, api_key=api_key, provider=provider
)
if not models:
models = list(wiz.EMBEDDING_FALLBACK_MODELS.get(provider, ()))
if spec and spec.default_model and spec.default_model not in models:
models = [spec.default_model] + models
model = wiz.select_model(
console,
strings,
models=models,
current=str((current_profile.get("models") or [{}])[0].get("model") or ""),
custom_prompt_label=strings["init.embedding_model"],
)
dimension = typer.prompt(strings["init.embedding_dimension"], default="")
choice = wiz.EmbeddingChoice(
binding=provider,
base_url=endpoint,
api_key=api_key,
model=model,
dimension=str(dimension or "").strip(),
display_provider=display_provider,
)
if typer.confirm(strings["init.probe_offer"], default=True):
wiz.info(console, strings["init.probe_running"].format(what=display_provider))
ok_result, elapsed_ms, error = wiz.probe_embedding(
base_url=choice.base_url, api_key=choice.api_key, model=choice.model
)
choice.probed = True
choice.probe_ok = ok_result
choice.probe_ms = elapsed_ms
if ok_result:
wiz.ok(console, strings["init.probe_ok"].format(what=display_provider, ms=elapsed_ms))
else:
wiz.fail(console, strings["init.probe_fail"].format(what=display_provider, error=error))
return choice
def _search_step(
console: Console,
strings: dict,
catalog: dict,
) -> wiz.SearchChoice | None:
"""Returns ``None`` when the user picks ``[s] Skip``.
A ``provider == "none"`` result is NOT skip — it's "explicitly disable
web search". We still write that into the catalog so agents stop trying.
"""
current_profile = (catalog.get("services", {}).get("search", {}).get("profiles") or [{}])[
0
] or {}
current_provider = str(current_profile.get("provider") or "tavily")
spec = wiz.select_search_provider(console, strings, current=current_provider)
if spec is None:
wiz.info(console, strings["init.skipped"])
return None
if spec.name == "none":
wiz.info(console, strings["init.search_disabled_note"])
return wiz.SearchChoice(provider="none", label=spec.label)
api_key = ""
if spec.requires_api_key:
env_key, env_name = wiz.search_api_key_from_env(spec.env_keys)
if env_key:
masked = wiz._mask_secret(env_key)
offer = strings["init.api_key_env_detected"].format(env_var=env_name, masked=masked)
if typer.confirm(offer, default=True):
api_key = env_key
if not api_key:
current_key = str(current_profile.get("api_key") or "")
if current_key:
masked = wiz._mask_secret(current_key)
if typer.confirm(
strings["init.api_key_reuse_llm"].format(masked=masked), default=True
):
api_key = current_key
if not api_key:
api_key = typer.prompt(
strings["init.search_api_key_prompt"],
default="",
hide_input=True,
show_default=False,
)
else:
wiz.info(console, strings["init.search_no_key_note"].format(label=spec.label))
base_url = ""
if spec.requires_base_url:
default_url = str(current_profile.get("base_url") or "") or spec.default_base_url
base_url = typer.prompt(strings["init.search_base_url_prompt"], default=default_url)
return wiz.SearchChoice(
provider=spec.name,
label=spec.label,
api_key=api_key,
base_url=base_url,
)
def _ensure_search_service(catalog: dict, profile_id: str) -> dict:
"""Locate or create the default search profile we'll mutate in place."""
services = catalog.setdefault("services", {})
service = services.setdefault(
"search",
{"active_profile_id": profile_id, "profiles": []},
)
profiles = service.setdefault("profiles", [])
profile = next(
(item for item in profiles if item.get("id") == service.get("active_profile_id")), None
)
if profile is None:
profile = {
"id": profile_id,
"name": "Default Search",
"provider": "brave",
"base_url": "",
"api_key": "",
"api_version": "",
"extra_headers": {},
"proxy": "",
"models": [],
}
profiles.append(profile)
service["active_profile_id"] = profile_id
return profile
def run_init(*, cli_only: bool = False, home: str | Path | None = None) -> None:
runtime_home = get_runtime_home(home)
runtime_home.mkdir(parents=True, exist_ok=True)
import os
os.environ[DEEPTUTOR_HOME_ENV] = str(runtime_home)
_reset_runtime_singletons()
from deeptutor.runtime.banner import labels_for, print_banner, resolve_language
from deeptutor.services.config import get_model_catalog_service, get_runtime_settings_service
from deeptutor.services.setup import init_user_directories
init_user_directories(runtime_home)
language = resolve_language()
strings = labels_for(language)
console = Console()
# CLI-only: LLM, Embedding, Search, Review = 4 steps.
# Full: Ports, LLM, Embedding, Search, Review = 5 steps.
total_steps = 4 if cli_only else 5
try:
print_banner(console, language=language, mode_key="init.mode")
console.print(f"{strings['init.workspace']}: [bold]{runtime_home}[/bold]")
console.print(f"[dim]{strings['init.note_settings_dir']}[/dim]")
runtime = get_runtime_settings_service()
system = runtime.load_system(include_process_overrides=False)
# --- Step 1 (CLI mode skips ports) ---
step_num = 0
if not cli_only:
step_num += 1
wiz.step_header(
console,
strings["init.step_ports"].format(n=step_num, total=total_steps),
)
system["backend_port"] = int(
typer.prompt(
strings["init.backend_port"],
default=str(system.get("backend_port") or 8001),
)
)
system["frontend_port"] = int(
typer.prompt(
strings["init.frontend_port"],
default=str(system.get("frontend_port") or 3782),
)
)
# --- Step 2: LLM ---
catalog_service = get_model_catalog_service()
catalog = catalog_service.load()
llm_profile, llm_model = _ensure_model_service(
catalog, "llm", "llm-profile-default", "llm-model-default"
)
step_num += 1
wiz.step_header(console, strings["init.step_llm"].format(n=step_num, total=total_steps))
llm_choice = _llm_step(console, strings, llm_profile, llm_model)
# Apply LLM choice back into the catalog draft.
llm_profile["binding"] = llm_choice.binding
llm_profile["base_url"] = llm_choice.base_url
llm_profile["api_key"] = llm_choice.api_key
llm_model["model"] = llm_choice.model
llm_model["name"] = llm_choice.model or "Default Model"
# --- Step 3: Embedding (skip via [s] inside the picker) ---
embedding_choice: wiz.EmbeddingChoice | None = None
step_num += 1
wiz.step_header(
console, strings["init.step_embedding"].format(n=step_num, total=total_steps)
)
embedding_choice = _embedding_step(console, strings, catalog, llm_choice.api_key)
if embedding_choice is not None:
emb_profile, emb_model = _ensure_model_service(
catalog,
"embedding",
"embedding-profile-default",
"embedding-model-default",
)
emb_profile["binding"] = embedding_choice.binding
emb_profile["base_url"] = embedding_choice.base_url
emb_profile["api_key"] = embedding_choice.api_key
emb_model["model"] = embedding_choice.model
emb_model["name"] = embedding_choice.model or "Default Embedding Model"
if embedding_choice.dimension:
emb_model["dimension"] = embedding_choice.dimension
# --- Step 4: Search (skip via [s] inside the picker) ---
search_choice: wiz.SearchChoice | None = None
step_num += 1
wiz.step_header(console, strings["init.step_search"].format(n=step_num, total=total_steps))
search_choice = _search_step(console, strings, catalog)
if search_choice is not None:
search_profile = _ensure_search_service(catalog, "search-profile-default")
search_profile["provider"] = search_choice.provider
search_profile["api_key"] = search_choice.api_key
search_profile["base_url"] = search_choice.base_url
# --- Step 5: Review & save ---
step_num += 1
wiz.step_header(console, strings["init.step_review"].format(n=step_num, total=total_steps))
wiz.render_review_panel(
console,
strings,
llm=llm_choice,
embedding=embedding_choice,
search=search_choice,
backend_port=None if cli_only else system.get("backend_port"),
frontend_port=None if cli_only else system.get("frontend_port"),
)
if not typer.confirm(strings["init.confirm_save"], default=True):
wiz.warn(console, strings["init.cancelled"])
raise typer.Exit(code=1)
if not cli_only:
runtime.save_system(system)
catalog_service.save(catalog)
console.print()
wiz.ok(console, strings["init.saved"])
console.print(f"[dim]{strings['init.next_step']}[/dim]")
except (KeyboardInterrupt, typer.Abort):
console.print()
wiz.warn(console, strings["init.cancelled"])
raise typer.Exit(code=130)
def register(app: typer.Typer) -> None:
@app.command("init")
def init_command(
cli: bool = typer.Option(False, "--cli", help="Initialize for CLI-only use."),
home: Path | None = typer.Option(None, "--home", help="Runtime workspace root."),
) -> None:
"""Create or update data/user/settings for this workspace."""
run_init(cli_only=cli, home=home)
+926
View File
@@ -0,0 +1,926 @@
"""Interactive setup helpers used by ``deeptutor init``.
Drives the multi-step wizard: provider menu, API-key capture (with env-var
auto-detect), live model-list fetch from ``GET {base_url}/models`` with a
curated fallback list, and an optional connectivity probe before save.
Everything that touches I/O (HTTP, env, stdin) goes through small helpers so
the orchestrator in ``init_cmd.py`` stays a thin sequence of steps.
"""
from __future__ import annotations
from dataclasses import dataclass
import os
import time
from typing import Any
import httpx
from rich.console import Console
from rich.markup import escape as rich_escape
from rich.panel import Panel
from rich.table import Table
from rich.text import Text
import typer
from deeptutor.services.llm.config import get_token_limit_kwargs
from deeptutor.services.provider_registry import PROVIDERS, ProviderSpec, find_by_name
# --- Featured selection ------------------------------------------------------
# Hand-picked, in display order, for the LLM step. Everything else is reachable
# via the "Show all" option. Names match ProviderSpec.name in provider_registry.
FEATURED_LLM_PROVIDERS: tuple[str, ...] = (
"openai",
"anthropic",
"deepseek",
"dashscope",
"zhipu",
"moonshot",
"gemini",
"siliconflow",
"openrouter",
"ollama",
)
# Fallback model lists used only when ``GET {base_url}/models`` fails or the
# provider is "custom". Live fetch is preferred — keep these short, just enough
# to unblock common cases.
LLM_FALLBACK_MODELS: dict[str, tuple[str, ...]] = {
"openai": ("gpt-4o-mini", "gpt-4o", "o4-mini", "gpt-4.1", "gpt-4.1-mini"),
"anthropic": (
"claude-sonnet-4-6",
"claude-opus-4-7",
"claude-haiku-4-5-20251001",
),
"deepseek": ("deepseek-chat", "deepseek-reasoner"),
"dashscope": ("qwen-plus", "qwen-turbo", "qwen-max", "qwen3-coder-plus"),
"zhipu": ("glm-4.6", "glm-4.5", "glm-4-flash"),
"moonshot": ("kimi-k2.6", "kimi-k2.5", "kimi-latest"),
"gemini": ("gemini-2.5-pro", "gemini-2.5-flash", "gemini-2.5-flash-lite"),
"siliconflow": (
"Qwen/Qwen3-Coder-480B-A35B-Instruct",
"deepseek-ai/DeepSeek-V3",
),
"openrouter": (
"openai/gpt-4o-mini",
"anthropic/claude-sonnet-4-6",
"deepseek/deepseek-chat",
),
"ollama": ("llama3.2", "qwen2.5", "mistral"),
}
# Featured embedding providers — display order. Source of truth for label /
# default URL / default model is ``EMBEDDING_PROVIDERS`` in
# ``deeptutor.services.config.provider_runtime``. Adding a new featured entry
# just means appending its key here.
FEATURED_EMBEDDING_PROVIDERS: tuple[str, ...] = (
"openai",
"gemini",
"aliyun", # DashScope / Qwen multimodal embeddings
"siliconflow",
"jina",
"cohere",
"openrouter",
"azure_openai",
"vllm", # also covers LM Studio, llama.cpp via the same OpenAI-compatible adapter
"ollama",
)
# Fallback model lists used only when live ``/models`` fetch fails. For
# providers where ``EmbeddingProviderSpec.default_model`` is set, that's
# preferred and these are extras.
EMBEDDING_FALLBACK_MODELS: dict[str, tuple[str, ...]] = {
"openai": ("text-embedding-3-large", "text-embedding-3-small"),
"gemini": ("gemini-embedding-001", "text-embedding-004"),
"aliyun": ("qwen3-vl-embedding", "text-embedding-v3", "text-embedding-v2"),
"siliconflow": (
"Qwen/Qwen3-Embedding-8B",
"BAAI/bge-m3",
"BAAI/bge-large-en-v1.5",
),
"jina": ("jina-embeddings-v3", "jina-embeddings-v2-base-en"),
"cohere": ("embed-v4.0", "embed-multilingual-v3.0", "embed-english-v3.0"),
"openrouter": ("openai/text-embedding-3-large",),
"vllm": ("BAAI/bge-m3",),
"ollama": ("nomic-embed-text", "mxbai-embed-large", "snowflake-arctic-embed"),
}
# --- Search providers ----------------------------------------------------------
# Source of truth: ``SUPPORTED_SEARCH_PROVIDERS`` in
# ``deeptutor.services.config.provider_runtime``. Each entry below describes
# how the wizard captures the credentials/config for that provider.
@dataclass(frozen=True)
class SearchProviderSpec:
"""How the init wizard handles one search provider."""
name: str # canonical key written into catalog.services.search.profiles[].provider
label: str
requires_api_key: bool
env_keys: tuple[str, ...] = () # checked in order — first non-empty wins
requires_base_url: bool = False
default_base_url: str = ""
hint: str = ""
SEARCH_PROVIDERS: tuple[SearchProviderSpec, ...] = (
SearchProviderSpec(
name="brave",
label="Brave Search",
requires_api_key=True,
env_keys=("BRAVE_API_KEY", "SEARCH_API_KEY"),
hint="independent index · paid tier",
),
SearchProviderSpec(
name="tavily",
label="Tavily",
requires_api_key=True,
env_keys=("TAVILY_API_KEY", "SEARCH_API_KEY"),
hint="LLM-friendly · free tier",
),
SearchProviderSpec(
name="jina",
label="Jina Reader Search",
requires_api_key=True,
env_keys=("JINA_API_KEY", "SEARCH_API_KEY"),
hint="returns full page content",
),
SearchProviderSpec(
name="serper",
label="Serper",
requires_api_key=True,
env_keys=("SERPER_API_KEY", "SEARCH_API_KEY"),
hint="Google results · paid",
),
SearchProviderSpec(
name="perplexity",
label="Perplexity",
requires_api_key=True,
env_keys=("PERPLEXITY_API_KEY", "SEARCH_API_KEY"),
hint="answer-style search",
),
SearchProviderSpec(
name="duckduckgo",
label="DuckDuckGo",
requires_api_key=False,
hint="no API key needed",
),
SearchProviderSpec(
name="searxng",
label="SearXNG",
requires_api_key=False,
requires_base_url=True,
default_base_url="http://localhost:8888",
hint="self-hosted · provide your instance URL",
),
SearchProviderSpec(
name="none",
label="Disable web search",
requires_api_key=False,
hint="agents will skip all search tools",
),
)
# --- Data ----------------------------------------------------------------------
@dataclass
class LLMChoice:
"""User-confirmed LLM step result, ready to write into the catalog."""
binding: str
base_url: str
api_key: str
model: str
display_provider: str # human-friendly label for the review panel
probed: bool = False
probe_ok: bool = False
probe_ms: int = 0
@dataclass
class EmbeddingChoice:
binding: str
base_url: str # full /embeddings URL (already normalised)
api_key: str
model: str
dimension: str
display_provider: str
probed: bool = False
probe_ok: bool = False
probe_ms: int = 0
@dataclass
class SearchChoice:
"""User-confirmed Search step result. ``provider == 'none'`` means
disable web search entirely."""
provider: str
label: str
api_key: str = ""
base_url: str = ""
# --- Rendering helpers ---------------------------------------------------------
def step_header(console: Console, label: str) -> None:
console.print()
bar = "" * 8
console.print(
f"[bright_cyan]{bar}[/bright_cyan] [bold]{label}[/bold] [bright_cyan]{bar}[/bright_cyan]"
)
console.print()
def info(console: Console, message: str) -> None:
console.print(f"[dim]{message}[/dim]")
def ok(console: Console, message: str) -> None:
console.print(f"[green]✓[/green] {message}")
def warn(console: Console, message: str) -> None:
console.print(f"[yellow]![/yellow] {message}")
def fail(console: Console, message: str) -> None:
console.print(f"[red]✗[/red] {message}")
def _mask_secret(value: str) -> str:
"""Show first 4 + last 4 chars of an API key. Empty / short → fully masked."""
if not value:
return "(empty)"
if len(value) <= 8:
return "*" * len(value)
return f"{value[:4]}...{value[-4:]}"
# --- Numbered-list picker ------------------------------------------------------
def select_from_options(
console: Console,
*,
title: str,
options: list[tuple[str, str, str]], # [(key, label, hint), ...]
default_key: str | None = None,
extra_keys: dict[str, str] | None = None,
prompt_label: str = "Choice",
invalid_label: str = "Invalid choice. Try again.",
) -> str:
"""Render a numbered/keyed menu, return the selected key.
``options`` is the visible numbered list. ``extra_keys`` adds letter
shortcuts (e.g. ``{"s": "Show all providers", "c": "Custom"}``) — these
show up after the numbered rows and are accepted as input.
"""
# Titles come from i18n and may contain `[c]`-style brackets that Rich
# would otherwise interpret as markup tags.
console.print(f"[bold]{rich_escape(title)}[/bold]")
console.print()
table = Table.grid(padding=(0, 1))
table.add_column(style="bright_cyan", justify="right")
table.add_column(style="bold")
table.add_column(style="dim")
# Rich Table cells parse markup, so e.g. `[s]` would be eaten as a
# nonexistent tag. Wrap markers in Text so they render verbatim.
def _marker(text: str) -> Text:
return Text(text, style="bright_cyan", justify="right")
for idx, (_key, label, hint) in enumerate(options, start=1):
table.add_row(_marker(f"[{idx}]"), label, hint or "")
if extra_keys:
for short, label in extra_keys.items():
table.add_row(_marker(f"[{short}]"), label, "")
console.print(table)
console.print()
valid_numbers = {str(i): options[i - 1][0] for i in range(1, len(options) + 1)}
valid_letters = {k.lower(): k.lower() for k in (extra_keys or {})}
default_input: str | None = None
if default_key is not None:
for idx, (key, _label, _hint) in enumerate(options, start=1):
if key == default_key:
default_input = str(idx)
break
if default_input is None and default_key.lower() in valid_letters:
default_input = default_key.lower()
while True:
raw = typer.prompt(prompt_label, default=default_input or "")
choice = str(raw).strip().lower()
if choice in valid_numbers:
return valid_numbers[choice]
if choice in valid_letters:
return valid_letters[choice]
fail(console, invalid_label)
# --- Provider selection --------------------------------------------------------
def _ordered_providers(featured: tuple[str, ...]) -> list[ProviderSpec]:
"""Return featured provider specs in the given order, dropping unknowns."""
out: list[ProviderSpec] = []
seen: set[str] = set()
for name in featured:
spec = find_by_name(name)
if spec and spec.name not in seen:
out.append(spec)
seen.add(spec.name)
return out
def _all_providers_except(featured: set[str]) -> list[ProviderSpec]:
"""All providers from the registry that aren't already in the featured list."""
return [
spec
for spec in PROVIDERS
if spec.name not in featured and not spec.is_oauth # OAuth flows use `deeptutor login`
]
def select_llm_provider(
console: Console,
strings: dict[str, str],
*,
current_binding: str | None = None,
) -> ProviderSpec | None:
"""Walk the user through provider selection. ``None`` means custom/manual."""
featured = _ordered_providers(FEATURED_LLM_PROVIDERS)
featured_names = {spec.name for spec in featured}
options: list[tuple[str, str, str]] = []
for spec in featured:
hint = spec.default_api_base or ("local" if spec.is_local else "")
options.append((spec.name, spec.label, hint))
# ``[s]`` is reserved for the "Skip" shortcut in optional steps
# (embedding / search). LLM is mandatory, so we use ``[a]`` for "show all".
extra = {
"a": strings["init.show_all"],
"c": strings["init.custom_provider"],
}
default_key = current_binding if current_binding in featured_names else "openai"
pick = select_from_options(
console,
title=strings["init.pick_provider"],
options=options,
default_key=default_key,
extra_keys=extra,
prompt_label=strings["init.choice"],
invalid_label=strings["init.choice_invalid"],
)
if pick == "c":
return None
if pick == "a":
return _select_provider_full_list(console, strings, exclude=featured_names)
return find_by_name(pick)
def _select_provider_full_list(
console: Console,
strings: dict[str, str],
*,
exclude: set[str],
) -> ProviderSpec | None:
rest = _all_providers_except(exclude)
options: list[tuple[str, str, str]] = [
(spec.name, spec.label, spec.default_api_base or ("local" if spec.is_local else ""))
for spec in rest
]
extra = {"b": strings["init.back"], "c": strings["init.custom_provider"]}
pick = select_from_options(
console,
title=strings["init.pick_provider"],
options=options,
extra_keys=extra,
prompt_label=strings["init.choice"],
invalid_label=strings["init.choice_invalid"],
)
if pick == "b":
return select_llm_provider(console, strings, current_binding=None)
if pick == "c":
return None
return find_by_name(pick)
SKIP_SENTINEL = "__skip__"
def select_embedding_provider(
console: Console,
strings: dict[str, str],
*,
current: str | None = None,
) -> str | None:
"""Pick an embedding provider key. Returns one of:
- canonical provider name (e.g. ``"openai"``, ``"aliyun"``)
- ``None`` → user wants to type their own (custom)
- :data:`SKIP_SENTINEL` → user wants to skip this step entirely
The featured list is driven by :data:`FEATURED_EMBEDDING_PROVIDERS`; labels
and default endpoints come from ``EMBEDDING_PROVIDERS`` in
``provider_runtime`` so we don't duplicate the source of truth.
"""
from deeptutor.services.config.provider_runtime import EMBEDDING_PROVIDERS
options: list[tuple[str, str, str]] = []
for name in FEATURED_EMBEDDING_PROVIDERS:
spec = EMBEDDING_PROVIDERS.get(name)
if not spec:
continue
hint = spec.default_api_base or ("local" if spec.is_local else "")
options.append((name, spec.label, hint))
extra = {
"s": strings["init.skip_step"],
"c": strings["init.custom_provider"],
}
default_key = current if current in {n for n, _, _ in options} else "openai"
pick = select_from_options(
console,
title=strings["init.pick_embedding_provider"],
options=options,
default_key=default_key,
extra_keys=extra,
prompt_label=strings["init.choice"],
invalid_label=strings["init.choice_invalid"],
)
if pick == "s":
return SKIP_SENTINEL
if pick == "c":
return None
return pick
def select_search_provider(
console: Console,
strings: dict[str, str],
*,
current: str | None = None,
) -> SearchProviderSpec | None:
"""Pick a search provider. Returns the :class:`SearchProviderSpec` for the
chosen entry, or ``None`` when the user picks ``[s] Skip``."""
options = [(spec.name, spec.label, spec.hint) for spec in SEARCH_PROVIDERS]
extra = {"s": strings["init.skip_step"]}
default_key = current if current in {s.name for s in SEARCH_PROVIDERS} else "tavily"
pick = select_from_options(
console,
title=strings["init.pick_search_provider"],
options=options,
default_key=default_key,
extra_keys=extra,
prompt_label=strings["init.choice"],
invalid_label=strings["init.choice_invalid"],
)
if pick == "s":
return None
return next((spec for spec in SEARCH_PROVIDERS if spec.name == pick), None)
def search_api_key_from_env(env_keys: tuple[str, ...]) -> tuple[str, str]:
"""Return ``(key, env_name)`` of the first non-empty env var, else ``("", "")``."""
for env_name in env_keys:
value = os.environ.get(env_name, "")
if value:
return value, env_name
return "", ""
# --- API key capture -----------------------------------------------------------
def capture_api_key(
console: Console,
strings: dict[str, str],
*,
env_key: str,
current: str = "",
) -> str:
"""Prompt for an API key, with env-var auto-detect + saved-value fallback.
Preference order:
1. Existing saved key — confirm with masked display.
2. ``env_key`` environment variable — confirm with masked display.
3. Plain hidden prompt.
"""
if current:
masked = _mask_secret(current)
if typer.confirm(
strings["init.api_key_reuse_llm"].format(masked=masked),
default=True,
):
return current
if env_key:
from_env = os.environ.get(env_key, "")
if from_env:
masked = _mask_secret(from_env)
offer = strings["init.api_key_env_detected"].format(env_var=env_key, masked=masked)
if typer.confirm(offer, default=True):
return from_env
return typer.prompt(
strings["init.api_key_prompt"], default="", hide_input=True, show_default=False
)
# --- Live /models fetch --------------------------------------------------------
def fetch_models(
console: Console,
strings: dict[str, str],
*,
base_url: str,
api_key: str,
binding: str,
) -> list[str]:
"""Query the provider for an available-model list.
Returns ``[]`` on any failure — callers should fall back to the curated
list in ``LLM_FALLBACK_MODELS`` / ``EMBEDDING_FALLBACK_MODELS``.
"""
if not base_url:
return []
url = base_url.rstrip("/") + "/models"
headers: dict[str, str] = {}
if binding == "anthropic":
# Anthropic uses different auth headers.
if api_key:
headers["x-api-key"] = api_key
headers["anthropic-version"] = "2023-06-01"
else:
if api_key:
headers["Authorization"] = f"Bearer {api_key}"
info(console, strings["init.fetch_models"].format(url=url))
try:
with httpx.Client(timeout=5.0) as client:
response = client.get(url, headers=headers)
response.raise_for_status()
payload = response.json()
except Exception as exc:
warn(console, strings["init.fetch_models_fail"].format(error=str(exc)[:160]))
return []
raw_items: list[Any]
if isinstance(payload, dict) and isinstance(payload.get("data"), list):
raw_items = payload["data"]
elif isinstance(payload, dict) and isinstance(payload.get("models"), list):
# Ollama: GET /api/tags returns {"models": [{"name": "...", ...}]}
raw_items = payload["models"]
elif isinstance(payload, list):
raw_items = payload
else:
warn(console, strings["init.fetch_models_fail"].format(error="unexpected response shape"))
return []
names: list[str] = []
for item in raw_items:
if isinstance(item, str):
names.append(item)
elif isinstance(item, dict):
# OpenAI: {"id": "..."}. Ollama: {"name": "..."}. Anthropic: {"id": "..."}.
name = item.get("id") or item.get("name") or item.get("model")
if isinstance(name, str) and name:
names.append(name)
# Dedupe preserving order
seen: set[str] = set()
deduped: list[str] = []
for n in names:
if n in seen:
continue
seen.add(n)
deduped.append(n)
if deduped:
ok(console, strings["init.fetch_models_ok"].format(count=len(deduped)))
return deduped
def _derive_embedding_models_url(endpoint: str, provider: str) -> str:
"""Convert a (full) embedding endpoint URL into its sibling ``/models`` URL.
Embedding endpoints are stored as the *exact* URL adapters POST to
(e.g. ``https://api.openai.com/v1/embeddings``), not a base. To list
available models we have to strip the embedding-specific path segment.
Ollama is special-cased: it exposes installed models at ``/api/tags``,
not ``/models``.
"""
url = endpoint.rstrip("/")
if provider == "ollama" or url.endswith("/api/embed"):
base = url
for suffix in ("/api/embed", "/api/embeddings"):
if base.endswith(suffix):
base = base[: -len(suffix)]
break
return f"{base.rstrip('/')}/api/tags"
for suffix in ("/embeddings", "/embed"):
if url.endswith(suffix):
return f"{url[: -len(suffix)]}/models"
return f"{url}/models"
# Strict "embed" substring match. Broader heuristics (``e5-``, ``nomic``,
# ``voyage``...) drag too many LLMs in. Embedding models that don't follow
# the naming convention (``bge-m3``, ``qwen3-embedding-8b``) are picked up
# from the curated EMBEDDING_FALLBACK_MODELS list instead.
def _looks_like_embedding_model(name: str) -> bool:
return "embed" in name.lower()
def fetch_embedding_models(
console: Console,
strings: dict[str, str],
*,
endpoint: str,
api_key: str,
provider: str,
) -> list[str]:
"""Live-list embedding models from the provider's ``/models`` endpoint.
Returns ``[]`` on any failure so callers can fall back to the curated
list. When the provider's ``/models`` includes non-embedding models
(typical for OpenAI-compatible endpoints), the result is filtered down
to entries whose name looks like an embedding model. If filtering
leaves nothing, the unfiltered list is returned as a safety net.
"""
if not endpoint:
return []
models_url = _derive_embedding_models_url(endpoint, provider)
headers: dict[str, str] = {}
if api_key:
headers["Authorization"] = f"Bearer {api_key}"
info(console, strings["init.fetch_models"].format(url=models_url))
try:
with httpx.Client(timeout=5.0) as client:
response = client.get(models_url, headers=headers)
response.raise_for_status()
payload = response.json()
except Exception as exc:
warn(console, strings["init.fetch_models_fail"].format(error=str(exc)[:160]))
return []
raw_items: list[Any] = []
if isinstance(payload, dict):
for key in ("data", "models"):
value = payload.get(key)
if isinstance(value, list):
raw_items = value
break
elif isinstance(payload, list):
raw_items = payload
names: list[str] = []
for item in raw_items:
if isinstance(item, str):
names.append(item)
elif isinstance(item, dict):
name = item.get("id") or item.get("name") or item.get("model")
if isinstance(name, str) and name:
names.append(name)
if not names:
warn(console, strings["init.fetch_models_fail"].format(error="empty model list"))
return []
# Mixed lists (OpenAI returns gpt-4o, dall-e, etc. alongside embeddings).
# Strict ``embed`` filter; if it matches nothing, return empty so the
# caller falls through to the curated EMBEDDING_FALLBACK_MODELS list.
filtered = [n for n in names if _looks_like_embedding_model(n)]
if not filtered:
return []
seen: set[str] = set()
deduped: list[str] = []
for n in filtered:
if n in seen:
continue
seen.add(n)
deduped.append(n)
ok(console, strings["init.fetch_models_ok"].format(count=len(deduped)))
return deduped
def select_model(
console: Console,
strings: dict[str, str],
*,
models: list[str],
current: str = "",
custom_prompt_label: str | None = None,
) -> str:
"""Numbered-list model picker with ``[c] Custom`` escape."""
if not models:
return typer.prompt(
custom_prompt_label or strings["init.custom_model"],
default=current or "",
)
options = [(m, m, "") for m in models]
extra = {"c": strings["init.custom_model"]}
default_key = current if current in models else models[0]
pick = select_from_options(
console,
title=strings["init.pick_model"].format(marker="[c]"),
options=options,
default_key=default_key,
extra_keys=extra,
prompt_label=strings["init.choice"],
invalid_label=strings["init.choice_invalid"],
)
if pick == "c":
return typer.prompt(
custom_prompt_label or strings["init.custom_model"],
default=current or "",
)
return pick
# --- Connectivity probe --------------------------------------------------------
def probe_llm(*, base_url: str, api_key: str, binding: str, model: str) -> tuple[bool, int, str]:
"""Send a single-token completion to verify credentials.
Returns ``(ok, elapsed_ms, error_or_empty)``. Network failures, auth
failures, 4xx, 5xx all surface as ``ok=False`` with a short error string.
"""
if not base_url or not model:
return False, 0, "missing base_url or model"
started = time.monotonic()
try:
if binding == "anthropic":
url = base_url.rstrip("/") + "/messages"
headers = {
"x-api-key": api_key or "",
"anthropic-version": "2023-06-01",
"Content-Type": "application/json",
}
body = {
"model": model,
"max_tokens": 1,
"messages": [{"role": "user", "content": "ping"}],
}
else:
url = base_url.rstrip("/") + "/chat/completions"
headers = {
"Authorization": f"Bearer {api_key or 'sk-no-key-required'}",
"Content-Type": "application/json",
}
body = {
"model": model,
**get_token_limit_kwargs(model, 1),
"messages": [{"role": "user", "content": "ping"}],
}
with httpx.Client(timeout=15.0) as client:
response = client.post(url, headers=headers, json=body)
elapsed = int((time.monotonic() - started) * 1000)
if response.status_code >= 400:
snippet = response.text[:200]
return False, elapsed, f"HTTP {response.status_code} · {snippet}"
return True, elapsed, ""
except Exception as exc:
elapsed = int((time.monotonic() - started) * 1000)
return False, elapsed, str(exc)[:200]
def probe_embedding(*, base_url: str, api_key: str, model: str) -> tuple[bool, int, str]:
"""POST a tiny embedding request. Returns ``(ok, elapsed_ms, error)``."""
if not base_url or not model:
return False, 0, "missing base_url or model"
started = time.monotonic()
try:
headers = {
"Authorization": f"Bearer {api_key or 'sk-no-key-required'}",
"Content-Type": "application/json",
}
body = {"model": model, "input": "ping"}
with httpx.Client(timeout=15.0) as client:
response = client.post(base_url, headers=headers, json=body)
elapsed = int((time.monotonic() - started) * 1000)
if response.status_code >= 400:
return False, elapsed, f"HTTP {response.status_code} · {response.text[:200]}"
return True, elapsed, ""
except Exception as exc:
elapsed = int((time.monotonic() - started) * 1000)
return False, elapsed, str(exc)[:200]
# --- Review panel --------------------------------------------------------------
def render_review_panel(
console: Console,
strings: dict[str, str],
*,
llm: LLMChoice | None,
embedding: EmbeddingChoice | None,
search: SearchChoice | None,
backend_port: int | None,
frontend_port: int | None,
) -> None:
body = Text()
def _row(label: str, value: str, probe: tuple[bool, bool] | None = None) -> None:
body.append(f"{label:>12} ", style="bold")
body.append(value)
if probe is not None:
probed, ok_flag = probe
if probed:
if ok_flag:
body.append(" ✓ probed", style="green")
else:
body.append(" ! probe failed", style="yellow")
body.append("\n")
if llm:
_row(
strings["init.review_llm"],
f"{llm.display_provider} · {llm.model} · {llm.base_url}",
probe=(llm.probed, llm.probe_ok),
)
if embedding:
_row(
strings["init.review_embedding"],
f"{embedding.display_provider} · {embedding.model} · {embedding.base_url}",
probe=(embedding.probed, embedding.probe_ok),
)
if search:
if search.provider == "none":
value = strings["init.review_search_disabled"]
elif search.base_url:
value = f"{search.label} · {search.base_url}"
else:
value = search.label
_row(strings["init.review_search"], value)
if backend_port is not None and frontend_port is not None:
_row(
strings["init.review_ports"],
strings["init.review_ports_value"].format(backend=backend_port, frontend=frontend_port),
)
console.print(
Panel(
body,
title=f"[bold]{rich_escape(strings['init.review_title'])}[/]",
border_style="bright_cyan",
padding=(1, 2),
)
)
__all__ = [
"EMBEDDING_FALLBACK_MODELS",
"EmbeddingChoice",
"FEATURED_EMBEDDING_PROVIDERS",
"FEATURED_LLM_PROVIDERS",
"LLMChoice",
"LLM_FALLBACK_MODELS",
"SEARCH_PROVIDERS",
"SKIP_SENTINEL",
"SearchChoice",
"SearchProviderSpec",
"capture_api_key",
"fail",
"fetch_embedding_models",
"fetch_models",
"info",
"ok",
"probe_embedding",
"probe_llm",
"render_review_panel",
"search_api_key_from_env",
"select_embedding_provider",
"select_from_options",
"select_llm_provider",
"select_model",
"select_search_provider",
"step_header",
"warn",
]
+286
View File
@@ -0,0 +1,286 @@
"""
CLI Knowledge Base Command
===========================
Manage llamaindex knowledge bases from the command line.
"""
from __future__ import annotations
import asyncio
import json
from pathlib import Path
from typing import Optional
from rich.console import Console
from rich.table import Table
import typer
from deeptutor.knowledge.manager import KnowledgeBaseManager
from deeptutor.knowledge.naming import validate_knowledge_base_name
from deeptutor.services.path_service import get_path_service
from deeptutor.services.rag.factory import DEFAULT_PROVIDER
from deeptutor.services.rag.file_routing import FileTypeRouter
console = Console()
def _get_kb_manager() -> KnowledgeBaseManager:
"""Return a KnowledgeBaseManager rooted at the canonical project-level KB directory."""
base_dir = get_path_service().project_root / "data" / "knowledge_bases"
return KnowledgeBaseManager(base_dir=str(base_dir))
def _collect_documents(docs: list[str], docs_dir: Optional[str]) -> list[str]:
"""Collect and de-duplicate document files from explicit paths and a directory."""
candidates: list[Path] = []
for doc in docs:
path = Path(doc).expanduser().resolve()
if path.exists() and path.is_file():
candidates.append(path)
if docs_dir:
base = Path(docs_dir).expanduser().resolve()
if not base.exists() or not base.is_dir():
raise typer.BadParameter(f"docs directory does not exist: {base}")
candidates.extend(FileTypeRouter.collect_supported_files(base, recursive=True))
unique: list[str] = []
seen: set[str] = set()
for path in candidates:
key = str(path)
if key in seen:
continue
seen.add(key)
unique.append(key)
return unique
def register(app: typer.Typer) -> None:
@app.command("list")
def kb_list(
fmt: str = typer.Option("rich", "--format", "-f", help="Output format: rich | json."),
) -> None:
"""List all knowledge bases."""
mgr = _get_kb_manager()
kb_names = mgr.list_knowledge_bases()
if not kb_names:
if fmt == "json":
console.print_json("[]")
else:
console.print("[dim]No knowledge bases found.[/]")
return
if fmt == "json":
items = []
for name in kb_names:
info = mgr.get_info(name)
stats = info.get("statistics", {})
metadata = info.get("metadata", {})
items.append(
{
"name": name,
"status": info.get("status", "unknown"),
"documents": stats.get("raw_documents", 0),
"rag_provider": metadata.get(
"rag_provider", stats.get("rag_provider", DEFAULT_PROVIDER)
),
"is_default": bool(info.get("is_default")),
}
)
console.print_json(json.dumps(items, ensure_ascii=False, default=str))
return
table = Table(title="Knowledge Bases")
table.add_column("Name", style="bold")
table.add_column("Status")
table.add_column("Documents", justify="right")
table.add_column("RAG Provider")
table.add_column("Default")
for name in kb_names:
info = mgr.get_info(name)
stats = info.get("statistics", {})
metadata = info.get("metadata", {})
table.add_row(
name,
str(info.get("status", "unknown")),
str(stats.get("raw_documents", 0)),
str(metadata.get("rag_provider", stats.get("rag_provider", DEFAULT_PROVIDER))),
"yes" if info.get("is_default") else "",
)
console.print(table)
@app.command("info")
def kb_info(name: str = typer.Argument(..., help="Knowledge base name.")) -> None:
"""Show details of a knowledge base."""
mgr = _get_kb_manager()
try:
info = mgr.get_info(name)
except Exception as exc:
console.print(f"[red]Knowledge base '{name}' not found: {exc}[/]")
raise typer.Exit(code=1) from exc
console.print_json(json.dumps(info, indent=2, ensure_ascii=False, default=str))
@app.command("set-default")
def kb_set_default(name: str = typer.Argument(..., help="Knowledge base name.")) -> None:
"""Set the default knowledge base."""
mgr = _get_kb_manager()
try:
mgr.set_default(name)
except Exception as exc:
console.print(f"[red]Failed to set default KB '{name}': {exc}[/]")
raise typer.Exit(code=1) from exc
console.print(f"[green]Set '{name}' as default knowledge base.[/]")
@app.command("create")
def kb_create(
name: str = typer.Argument(..., help="New KB name."),
docs: list[str] = typer.Option([], "--doc", "-d", help="Document paths."),
docs_dir: Optional[str] = typer.Option(None, "--docs-dir", help="Directory of documents."),
) -> None:
"""Initialize a new knowledge base from documents."""
mgr = _get_kb_manager()
try:
name = validate_knowledge_base_name(name)
except ValueError as exc:
console.print(f"[red]{exc}[/]")
raise typer.Exit(code=1) from exc
if name in mgr.list_knowledge_bases():
console.print(f"[red]Knowledge base '{name}' already exists.[/]")
raise typer.Exit(code=1)
try:
doc_paths = _collect_documents(docs, docs_dir)
except typer.BadParameter as exc:
console.print(f"[red]{exc}[/]")
raise typer.Exit(code=1) from exc
if not doc_paths:
console.print("[red]Provide at least one supported document (--doc or --docs-dir).[/]")
raise typer.Exit(code=1)
console.print(
f"Creating KB [bold]{name}[/] with {len(doc_paths)} document(s) via [bold]LlamaIndex[/]..."
)
from deeptutor.knowledge.initializer import initialize_knowledge_base
try:
asyncio.run(
initialize_knowledge_base(
kb_name=name,
source_files=doc_paths,
base_dir=str(mgr.base_dir),
)
)
except Exception as exc:
console.print(f"[red]KB creation failed: {exc}[/]")
raise typer.Exit(code=1) from exc
console.print("[green]Knowledge base created successfully.[/]")
@app.command("add")
def kb_add(
name: str = typer.Argument(..., help="KB name."),
docs: list[str] = typer.Option([], "--doc", "-d", help="Document paths to add."),
docs_dir: Optional[str] = typer.Option(None, "--docs-dir", help="Directory of documents."),
) -> None:
"""Add documents to an existing knowledge base."""
mgr = _get_kb_manager()
if name not in mgr.list_knowledge_bases():
console.print(f"[red]Knowledge base '{name}' not found.[/]")
raise typer.Exit(code=1)
try:
doc_paths = _collect_documents(docs, docs_dir)
except typer.BadParameter as exc:
console.print(f"[red]{exc}[/]")
raise typer.Exit(code=1) from exc
if not doc_paths:
console.print("[red]Provide at least one supported document.[/]")
raise typer.Exit(code=1)
console.print(f"Adding {len(doc_paths)} document(s) to [bold]{name}[/]...")
from deeptutor.knowledge.add_documents import add_documents
try:
processed_count = asyncio.run(
add_documents(
kb_name=name,
source_files=doc_paths,
base_dir=str(mgr.base_dir),
allow_duplicates=False,
)
)
except Exception as exc:
console.print(f"[red]Document upload failed: {exc}[/]")
raise typer.Exit(code=1) from exc
if processed_count:
console.print(f"[green]Done. Indexed {processed_count} document(s).[/]")
else:
console.print("[yellow]No new unique documents were indexed.[/]")
@app.command("delete")
def kb_delete(
name: str = typer.Argument(..., help="KB name."),
force: bool = typer.Option(False, "--force", "-f", help="Skip confirmation."),
) -> None:
"""Delete a knowledge base."""
if not force:
confirm = typer.confirm(f"Delete knowledge base '{name}'?")
if not confirm:
raise typer.Abort()
mgr = _get_kb_manager()
try:
deleted = mgr.delete_knowledge_base(name, confirm=True)
except Exception as exc:
console.print(f"[red]Failed to delete '{name}': {exc}[/]")
raise typer.Exit(code=1) from exc
if deleted:
console.print(f"[green]Deleted '{name}'.[/]")
else:
console.print(f"[yellow]Knowledge base '{name}' was not deleted.[/]")
@app.command("search")
def kb_search(
name: str = typer.Argument(..., help="KB name."),
query: str = typer.Argument(..., help="Search query."),
mode: str = typer.Option("hybrid", help="Search mode."),
fmt: str = typer.Option("rich", "--format", "-f", help="Output format: rich | json."),
) -> None:
"""Search a knowledge base."""
from deeptutor.tools.rag_tool import rag_search
mgr = _get_kb_manager()
if name not in mgr.list_knowledge_bases():
console.print(f"[red]Knowledge base '{name}' not found.[/]")
raise typer.Exit(code=1)
try:
result = asyncio.run(
rag_search(
query=query,
kb_name=name,
mode=mode,
kb_base_dir=str(mgr.base_dir),
)
)
except Exception as exc:
console.print(f"[red]Search failed: {exc}[/]")
raise typer.Exit(code=1) from exc
if fmt == "json":
console.print_json(json.dumps(result, indent=2, ensure_ascii=False, default=str))
return
answer = result.get("answer") or result.get("content", "")
provider = result.get("provider", DEFAULT_PROVIDER)
console.print(f"[bold]Provider:[/] {provider}")
console.print(f"[bold]Answer:[/]\n{answer}")
+172
View File
@@ -0,0 +1,172 @@
"""CLI entry point for the standalone ``deeptutor-cli`` package."""
from __future__ import annotations
from pathlib import Path
import typer
from deeptutor.logging import configure_logging
from deeptutor.runtime.mode import RunMode, set_mode
from .book import register as register_book
from .chat import register as register_chat
from .common import build_turn_request, console, maybe_run
from .config_cmd import register as register_config
from .init_cmd import register as register_init
from .kb import register as register_kb
from .memory import register as register_memory
from .notebook import register as register_notebook
from .partner import register as register_partner
from .plugin import register as register_plugin
from .provider_cmd import register as register_provider
from .session_cmd import register as register_session
from .skill import register as register_skill
set_mode(RunMode.CLI)
configure_logging()
app = typer.Typer(
name="deeptutor",
help="DeepTutor CLI agent-first interface for capabilities, tools, and knowledge.",
no_args_is_help=True,
add_completion=False,
)
partner_app = typer.Typer(help="Manage partners (IM-connected companions).")
chat_app = typer.Typer(help="Interactive chat REPL.")
kb_app = typer.Typer(help="Manage knowledge bases.")
skill_app = typer.Typer(help="Manage skills and install from hubs (ClawHub, …).")
memory_app = typer.Typer(help="View and manage lightweight memory.")
plugin_app = typer.Typer(help="List plugins.")
config_app = typer.Typer(help="Inspect configuration.")
session_app = typer.Typer(help="Manage shared sessions.")
notebook_app = typer.Typer(help="Manage notebooks and imported markdown records.")
provider_app = typer.Typer(help="Manage provider OAuth login.")
book_app = typer.Typer(help="Manage interactive Books (BookEngine).")
app.add_typer(partner_app, name="partner")
app.add_typer(chat_app, name="chat")
app.add_typer(kb_app, name="kb")
app.add_typer(skill_app, name="skill")
app.add_typer(skill_app, name="skills") # alias: `deeptutor skills …`
app.add_typer(memory_app, name="memory")
app.add_typer(plugin_app, name="plugin")
app.add_typer(config_app, name="config")
app.add_typer(session_app, name="session")
app.add_typer(notebook_app, name="notebook")
app.add_typer(provider_app, name="provider")
app.add_typer(book_app, name="book")
register_partner(partner_app)
register_chat(chat_app)
register_kb(kb_app)
register_skill(skill_app)
register_memory(memory_app)
register_plugin(plugin_app)
register_config(config_app)
register_session(session_app)
register_notebook(notebook_app)
register_provider(provider_app)
register_book(book_app)
register_init(app)
@app.command("run")
def run_capability(
capability: str = typer.Argument(
...,
help=(
"Capability name (e.g. chat, deep_solve, deep_question, "
"deep_research, visualize, math_animator, mastery_path)."
),
),
message: str = typer.Argument(..., help="Message to send."),
session: str | None = typer.Option(None, "--session", help="Existing session id."),
tool: list[str] = typer.Option([], "--tool", "-t", help="Enabled tool(s)."),
kb: list[str] = typer.Option([], "--kb", help="Knowledge base name."),
notebook_ref: list[str] = typer.Option([], "--notebook-ref", help="Notebook references."),
history_ref: list[str] = typer.Option([], "--history-ref", help="Referenced session ids."),
language: str = typer.Option("en", "--language", "-l", help="Response language."),
config: list[str] = typer.Option([], "--config", help="Capability config key=value."),
config_json: str | None = typer.Option(
None, "--config-json", help="Capability config as JSON."
),
fmt: str = typer.Option("rich", "--format", "-f", help="Output format: rich | json."),
) -> None:
"""Run any capability in a single turn (agent-first entry point)."""
from deeptutor.app import DeepTutorApp
from .common import run_turn_and_render
request = build_turn_request(
content=message,
capability=capability,
session_id=session,
tools=tool,
knowledge_bases=kb,
language=language,
config_items=config,
config_json=config_json,
notebook_refs=notebook_ref,
history_refs=history_ref,
)
maybe_run(run_turn_and_render(app=DeepTutorApp(), request=request, fmt=fmt))
@app.command()
def start(
home: Path | None = typer.Option(None, "--home", help="Runtime workspace root."),
) -> None:
"""Launch backend + frontend together. Press Ctrl+C to stop."""
from deeptutor.runtime.launcher import start as start_web
start_web(home=home)
@app.command()
def serve(
host: str = typer.Option("0.0.0.0", help="Bind address."),
port: int | None = typer.Option(None, help="Port number."),
reload: bool = typer.Option(False, help="Enable auto-reload for development."),
) -> None:
"""Start the DeepTutor API server."""
import asyncio
import sys
set_mode(RunMode.SERVER)
if port is None:
from deeptutor.services.setup import get_backend_port
port = get_backend_port()
# Windows: uvicorn defaults to SelectorEventLoop which does not support
# asyncio.create_subprocess_exec. Switch to ProactorEventLoop so that
# child-process APIs (used by Math Animator renderer, etc.) work correctly.
if sys.platform == "win32":
asyncio.set_event_loop_policy(asyncio.WindowsProactorEventLoopPolicy())
try:
import uvicorn
except ImportError:
console.print(
"[bold red]Error:[/] API server dependencies not installed.\n"
"Run: pip install -U deeptutor"
)
raise typer.Exit(code=1)
uvicorn.run(
"deeptutor.api.main:app",
host=host,
port=port,
reload=reload,
reload_excludes=["web/*", "data/*"] if reload else None,
)
def main() -> None:
app()
if __name__ == "__main__":
main()
+99
View File
@@ -0,0 +1,99 @@
"""CLI memory commands for the three-layer memory subsystem (v2)."""
from __future__ import annotations
from rich.console import Console
from rich.markdown import Markdown
from rich.panel import Panel
import typer
from deeptutor.services.memory import (
L3_SLOTS,
SURFACES,
get_memory_store,
paths,
)
console = Console()
def register(app: typer.Typer) -> None:
@app.command("show")
def memory_show(
target: str = typer.Argument(
"L3",
help="What to show: 'L3' (all four global docs), 'L2' (all seven surface docs), or a single doc name (e.g. 'profile', 'chat').",
),
) -> None:
"""Display memory document content."""
store = get_memory_store()
if target == "L3":
text = store.read_l3_concat()
console.print(Panel(Markdown(text), title="[bold]L3 (concatenated)[/]"))
return
if target == "L2":
for surface in SURFACES:
content = store.read_raw("L2", surface)
if content.strip():
console.print(Panel(Markdown(content), title=f"[bold]L2/{surface}.md[/]"))
else:
console.print(f"[dim]L2/{surface}.md: (empty)[/]")
return
# Single doc name — resolve to L2 or L3.
if target in L3_SLOTS:
content = store.read_raw("L3", target)
label = f"L3/{target}.md"
elif target in SURFACES:
content = store.read_raw("L2", target)
label = f"L2/{target}.md"
else:
console.print(
f"[red]Unknown doc: {target}. Use 'L2', 'L3', a surface ({', '.join(SURFACES)}), or a slot ({', '.join(L3_SLOTS)}).[/]"
)
raise typer.Exit(1)
if content.strip():
console.print(Panel(Markdown(content), title=f"[bold]{label}[/]"))
else:
console.print(f"[dim]{label}: (empty)[/]")
@app.command("clear")
def memory_clear(
target: str = typer.Argument(
"all",
help="What to clear: 'all' (entire memory), 'trace' (all L1 trace), or a surface name to clear that surface's L1.",
),
force: bool = typer.Option(False, "--force", "-f", help="Skip confirmation."),
) -> None:
"""Clear memory (use 'all' for a full reset, or a surface name for L1 only)."""
if target != "all" and target != "trace" and target not in SURFACES:
console.print(f"[red]Unknown target: {target}[/]")
raise typer.Exit(1)
if not force:
label = "all memory" if target == "all" else f"L1 trace for {target}"
if not typer.confirm(f"Clear {label}?"):
raise typer.Abort()
if target == "all":
root = paths.memory_root()
for entry in root.iterdir() if root.exists() else []:
if entry.is_file():
entry.unlink()
elif entry.is_dir() and entry.name in {"trace", "L2", "L3"}:
for child in entry.rglob("*"):
if child.is_file():
child.unlink()
console.print("[green]Cleared all memory.[/]")
elif target == "trace":
for surface in SURFACES:
for f in paths.trace_dir(surface).glob("*.jsonl"):
f.unlink()
console.print("[green]Cleared all L1 trace.[/]")
else:
for f in paths.trace_dir(target).glob("*.jsonl"): # type: ignore[arg-type]
f.unlink()
console.print(f"[green]Cleared L1 trace for {target}.[/]")
+121
View File
@@ -0,0 +1,121 @@
"""CLI commands for notebook record management."""
from __future__ import annotations
import json
from pathlib import Path
import typer
from deeptutor.app import DeepTutorApp
from .common import console, print_notebook_table
def register(app: typer.Typer) -> None:
@app.command("list")
def list_notebooks() -> None:
"""List notebooks."""
client = DeepTutorApp()
print_notebook_table(client.list_notebooks())
@app.command("create")
def create_notebook(
name: str = typer.Argument(..., help="Notebook name."),
description: str = typer.Option("", "--description", help="Notebook description."),
) -> None:
"""Create a notebook."""
client = DeepTutorApp()
notebook = client.create_notebook(name=name, description=description)
console.print(json.dumps(notebook, ensure_ascii=False, indent=2, default=str))
@app.command("show")
def show_notebook(
notebook_id: str = typer.Argument(..., help="Notebook id."),
fmt: str = typer.Option("rich", "--format", help="Output format: rich | json."),
) -> None:
"""Show a notebook and its records."""
client = DeepTutorApp()
notebook = client.get_notebook(notebook_id)
if notebook is None:
console.print(f"[red]Notebook not found:[/] {notebook_id}")
raise typer.Exit(code=1)
if fmt == "json":
console.print(json.dumps(notebook, ensure_ascii=False, indent=2, default=str))
return
console.print(f"[bold]{notebook.get('name', '')}[/] ({notebook.get('id', '')})")
console.print(str(notebook.get("description", "") or ""))
for record in notebook.get("records", []):
console.print(
f"\n[cyan]{record.get('id', '')}[/] "
f"{record.get('type', '')} "
f"{record.get('title', '')}"
)
@app.command("remove-record")
def remove_record(
notebook_id: str = typer.Argument(..., help="Notebook id."),
record_id: str = typer.Argument(..., help="Record id."),
) -> None:
"""Delete a notebook record."""
client = DeepTutorApp()
success = client.remove_record(notebook_id, record_id)
if not success:
console.print(f"[red]Record not found:[/] {record_id}")
raise typer.Exit(code=1)
console.print(f"Removed record {record_id} from notebook {notebook_id}")
@app.command("add-md")
def add_md(
notebook_id: str = typer.Argument(..., help="Notebook id."),
file_path: str = typer.Argument(..., help="Path to the markdown file."),
title: str = typer.Option("", "--title", help="Record title (defaults to filename)."),
record_type: str = typer.Option(
"chat",
"--type",
help="Record type: chat, question, research, solve.",
),
) -> None:
"""Add a markdown file as a record to a notebook."""
path = Path(file_path)
if not path.exists():
console.print(f"[red]File not found:[/] {file_path}")
raise typer.Exit(code=1)
content = path.read_text(encoding="utf-8")
record_title = title or path.stem
client = DeepTutorApp()
result = client.add_record(
notebook_ids=[notebook_id],
record_type=record_type,
title=record_title,
user_query="",
output=content,
)
record = result.get("record", {})
console.print(
f"[green]Added record[/] {record.get('id', '')} "
f"to notebook {notebook_id}: {record_title}"
)
@app.command("replace-md")
def replace_md(
notebook_id: str = typer.Argument(..., help="Notebook id."),
record_id: str = typer.Argument(..., help="Record id."),
file_path: str = typer.Argument(..., help="Path to the markdown file."),
) -> None:
"""Replace a notebook record's output with content from a markdown file."""
path = Path(file_path)
if not path.exists():
console.print(f"[red]File not found:[/] {file_path}")
raise typer.Exit(code=1)
content = path.read_text(encoding="utf-8")
client = DeepTutorApp()
updated = client.update_record(notebook_id, record_id, output=content)
if updated is None:
console.print(f"[red]Record not found:[/] {record_id}")
raise typer.Exit(code=1)
console.print(f"[green]Updated record[/] {record_id} in notebook {notebook_id}")
+103
View File
@@ -0,0 +1,103 @@
"""
CLI commands for managing partner instances.
"""
from __future__ import annotations
import asyncio
from rich.console import Console
from rich.table import Table
import typer
console = Console()
def register(app: typer.Typer) -> None:
@app.command("list")
def partner_list() -> None:
"""List all partners."""
from deeptutor.services.partners import get_partner_manager
partners = get_partner_manager().list_partners()
if not partners:
console.print("[dim]No partners configured.[/]")
return
table = Table(title="Partners")
table.add_column("ID", style="cyan")
table.add_column("Name")
table.add_column("Status")
table.add_column("Model", style="dim")
table.add_column("Channels", style="dim")
for p in partners:
status = "[green]running[/]" if p.get("running") else "[dim]stopped[/]"
selection = p.get("llm_selection") or {}
model = selection.get("model_id") or p.get("model") or "(default)"
table.add_row(
p["partner_id"],
p.get("name", ""),
status,
model,
", ".join(p.get("channels", [])) or "-",
)
console.print(table)
@app.command("start")
def partner_start(
name: str = typer.Argument(..., help="Partner ID to start."),
) -> None:
"""Start a partner."""
from deeptutor.services.partners import get_partner_manager
mgr = get_partner_manager()
try:
instance = asyncio.run(mgr.start_partner(name))
console.print(f"[green]Started partner '{instance.config.name}' ({name})[/]")
except RuntimeError as e:
console.print(f"[red]Failed to start: {e}[/]")
raise typer.Exit(1)
@app.command("stop")
def partner_stop(
name: str = typer.Argument(..., help="Partner ID to stop."),
) -> None:
"""Stop a running partner."""
from deeptutor.services.partners import get_partner_manager
mgr = get_partner_manager()
stopped = asyncio.run(mgr.stop_partner(name))
if stopped:
console.print(f"[green]Stopped partner '{name}'[/]")
else:
console.print(f"[yellow]Partner '{name}' not found or not running.[/]")
@app.command("create")
def partner_create(
name: str = typer.Argument(..., help="Partner ID."),
display_name: str = typer.Option("", "--name", "-n", help="Display name."),
soul: str = typer.Option("", "--soul", "-s", help="Soul markdown (the persona)."),
model: str = typer.Option("", "--model", "-m", help="Model override."),
) -> None:
"""Create a new partner configuration and start it."""
from deeptutor.services.partners import get_partner_manager
from deeptutor.services.partners.manager import PartnerConfig
from deeptutor.services.partners.workspace import write_soul
config = PartnerConfig(
name=display_name or name,
model=model or None,
)
mgr = get_partner_manager()
try:
mgr.save_config(name, config, auto_start=True)
if soul:
write_soul(name, soul)
instance = asyncio.run(mgr.start_partner(name, config))
console.print(
f"[green]Created and started partner '{instance.config.name}' ({name})[/]"
)
except RuntimeError as e:
console.print(f"[red]Failed: {e}[/]")
raise typer.Exit(1)
+81
View File
@@ -0,0 +1,81 @@
"""
CLI Plugin Command
==================
List and inspect registered plugins (tools, capabilities, playground).
"""
from __future__ import annotations
from dataclasses import asdict
from rich.console import Console
from rich.table import Table
import typer
console = Console()
def register(app: typer.Typer) -> None:
@app.command("list")
def plugin_list() -> None:
"""List all registered tools and capabilities."""
from deeptutor.runtime.registry.capability_registry import get_capability_registry
from deeptutor.runtime.registry.tool_registry import get_tool_registry
tr = get_tool_registry()
cr = get_capability_registry()
table = Table(title="Registered Plugins")
table.add_column("Name", style="bold")
table.add_column("Type")
table.add_column("Description")
for defn in tr.get_definitions():
table.add_row(defn.name, "tool", defn.description[:80])
for m in cr.get_manifests():
table.add_row(m["name"], "capability", m["description"][:80])
console.print(table)
@app.command("info")
def plugin_info(name: str = typer.Argument(..., help="Tool or capability name.")) -> None:
"""Show details of a tool or capability."""
import json
from deeptutor.runtime.registry.capability_registry import get_capability_registry
from deeptutor.runtime.registry.tool_registry import get_tool_registry
tr = get_tool_registry()
cr = get_capability_registry()
tool = tr.get(name)
if tool:
defn = tool.get_definition()
console.print_json(json.dumps(defn.to_openai_schema(), indent=2))
return
cap = cr.get(name)
if cap:
from deeptutor.app import DeepTutorApp
availability = DeepTutorApp().get_capability_availability(name)
console.print_json(
json.dumps(
{
"name": cap.manifest.name,
"description": cap.manifest.description,
"cli_aliases": cap.manifest.cli_aliases,
"stages": cap.manifest.stages,
"tools_used": cap.manifest.tools_used,
"config_defaults": cap.manifest.config_defaults,
"availability": asdict(availability),
},
indent=2,
)
)
return
console.print(f"[red]'{name}' not found.[/]")
raise typer.Exit(code=1)
+81
View File
@@ -0,0 +1,81 @@
"""CLI commands for provider auth and access validation."""
from __future__ import annotations
import typer
from .common import maybe_run
def register(app: typer.Typer) -> None:
@app.command("login")
def provider_login(
provider: str = typer.Argument(
...,
help="Provider: openai-codex (OAuth login) | github-copilot (validate existing Copilot auth)",
),
) -> None:
"""Authenticate or validate provider access."""
key = provider.strip().lower().replace("-", "_")
if key == "openai_codex":
_login_openai_codex()
return
if key == "github_copilot":
maybe_run(_login_github_copilot())
return
raise typer.BadParameter(
f"Unknown provider `{provider}`. Supported: openai-codex, github-copilot"
)
def _login_openai_codex() -> None:
try:
from oauth_cli_kit import get_token, login_oauth_interactive
except ImportError:
typer.echo(
"oauth_cli_kit is not installed. Install CLI deps from a local checkout: "
"python -m pip install -e ./packaging/deeptutor-cli"
)
raise typer.Exit(code=1)
token = None
try:
token = get_token()
except Exception:
token = None
if not (token and getattr(token, "access", None)):
token = login_oauth_interactive(
print_fn=typer.echo,
prompt_fn=typer.prompt,
)
if not (token and getattr(token, "access", None)):
typer.echo("OpenAI Codex OAuth authentication failed.")
raise typer.Exit(code=1)
typer.echo("OpenAI Codex OAuth authentication succeeded.")
async def _login_github_copilot() -> None:
"""Validate an existing GitHub Copilot auth session via a lightweight request."""
try:
from openai import AsyncOpenAI
except ImportError:
typer.echo(
"openai is not installed. Install CLI deps from a local checkout: "
"python -m pip install -e ./packaging/deeptutor-cli"
)
raise typer.Exit(code=1)
try:
client = AsyncOpenAI(
api_key="copilot",
base_url="https://api.githubcopilot.com",
max_retries=0,
)
await client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "ping"}],
max_tokens=1,
)
except Exception as exc:
typer.echo(f"GitHub Copilot auth validation failed: {exc}")
raise typer.Exit(code=1) from exc
typer.echo("GitHub Copilot auth validation succeeded.")
+101
View File
@@ -0,0 +1,101 @@
"""CLI commands for shared session management."""
from __future__ import annotations
import json
import typer
from deeptutor.app import DeepTutorApp
from .chat import ChatState, _chat_repl
from .common import console, maybe_run, print_session_table
def register(app: typer.Typer) -> None:
@app.command("list")
def list_sessions(
limit: int = typer.Option(20, "--limit", help="Maximum sessions to show."),
) -> None:
"""List existing sessions."""
maybe_run(_list_sessions(limit))
@app.command("show")
def show_session(
session_id: str = typer.Argument(..., help="Session id."),
fmt: str = typer.Option("rich", "--format", help="Output format: rich | json."),
) -> None:
"""Show a session and its persisted messages."""
maybe_run(_show_session(session_id, fmt))
@app.command("open")
def open_session(
session_id: str = typer.Argument(..., help="Session id."),
) -> None:
"""Enter the interactive chat REPL with an existing session."""
maybe_run(_chat_repl(ChatState(session_id=session_id)))
@app.command("delete")
def delete_session(
session_id: str = typer.Argument(..., help="Session id."),
) -> None:
"""Delete a session and all of its turns/messages."""
maybe_run(_delete_session(session_id))
@app.command("rename")
def rename_session(
session_id: str = typer.Argument(..., help="Session id."),
title: str = typer.Option(..., "--title", help="New session title."),
) -> None:
"""Rename a session."""
maybe_run(_rename_session(session_id, title))
async def _list_sessions(limit: int) -> None:
client = DeepTutorApp()
sessions = await client.list_sessions(limit=limit)
print_session_table(sessions)
async def _show_session(session_id: str, fmt: str) -> None:
client = DeepTutorApp()
session = await client.get_session(session_id)
if session is None:
console.print(f"[red]Session not found:[/] {session_id}")
raise typer.Exit(code=1)
if fmt == "json":
console.print(json.dumps(session, ensure_ascii=False, indent=2, default=str))
return
console.print(f"[bold]{session.get('title', '')}[/] ({session.get('id', '')})")
console.print(
f"[dim]capability={session.get('capability', '') or 'chat'} "
f"status={session.get('status', '')} "
f"messages={len(session.get('messages', []))}[/]",
highlight=False,
)
for message in session.get("messages", []):
role = str(message.get("role", "")).upper()
content = str(message.get("content", "") or "").strip()
console.print(f"\n[cyan]{role}[/]")
if content:
console.print(content)
async def _delete_session(session_id: str) -> None:
client = DeepTutorApp()
success = await client.delete_session(session_id)
if not success:
console.print(f"[red]Session not found:[/] {session_id}")
raise typer.Exit(code=1)
console.print(f"Deleted session {session_id}")
async def _rename_session(session_id: str, title: str) -> None:
client = DeepTutorApp()
success = await client.rename_session(session_id, title)
if not success:
console.print(f"[red]Session not found:[/] {session_id}")
raise typer.Exit(code=1)
console.print(f"Renamed {session_id} -> {title}")
+563
View File
@@ -0,0 +1,563 @@
"""
CLI Skill Commands
==================
Manage local skills and install packages from external hubs (EduHub, ClawHub, …).
Hub references use ``<hub>:<slug>[@version]``; the hub prefix defaults to
``eduhub`` (DeepTutor's native open skill registry). Installs run the full
import gate: hub security verdict →
safe extraction → frontmatter adaptation (``always`` stripped, flat
``bins``/``env`` folded into ``requires``) → provenance in ``.hub-lock.json``.
In a multi-user deployment this CLI operates the owner (admin) workspace, so
an installed skill lands in the admin catalog and stays invisible to other
users until a grant assigns it.
"""
from __future__ import annotations
from rich.table import Table
import typer
from .common import console
def _resolve_token(explicit: str | None, hub: str) -> str | None:
"""Publish token precedence: --token → env → ``skill login`` store."""
import os
from deeptutor.services.skill import credentials
return (
explicit
or os.environ.get("DEEPTUTOR_HUB_TOKEN")
or os.environ.get("EDUHUB_TOKEN")
or credentials.get_stored_token(hub)
)
def _summary_table(
title: str,
*,
hub: str,
slug: str,
version: str,
overrides: dict[str, str],
) -> Table:
"""Confirmation table for the resolved classification (publish/update)."""
from deeptutor.services.skill.taxonomy import domain_label, track_label
def shown(key: str, empty: str, *, as_domain: bool = False) -> str:
vals = [v for v in overrides.get(key, "").split(",") if v]
if not vals:
return empty
return "".join(domain_label(v) if as_domain else v for v in vals)
table = Table(title=title, show_header=False, title_style="bold")
table.add_column("字段", style="dim")
table.add_column("")
table.add_row("hub", hub)
table.add_row("slug", slug)
table.add_row("version", version or "[red]缺失[/]")
track = overrides.get("track", "")
table.add_row("track", f"{track_label(track)} [dim]{track}[/]" if track else "[red]缺失[/]")
table.add_row("language", overrides.get("language", ""))
table.add_row("domains", shown("domains", "[dim]通用[/]", as_domain=True))
table.add_row("stages", shown("stages", "[dim]全学段[/]"))
table.add_row("forms", shown("forms", "[dim]—[/]"))
table.add_row("audiences", shown("audiences", "[dim]学习者[/]"))
table.add_row("tags", shown("tags", "[dim]—[/]"))
return table
def register(app: typer.Typer) -> None:
@app.command("search")
def skill_search(
query: str = typer.Argument(..., help="Natural-language search query."),
hub: str | None = typer.Option(
None, "--hub", help="Hub to search (default from settings)."
),
limit: int = typer.Option(10, "--limit", min=1, max=50, help="Max results."),
) -> None:
"""Search a skill hub."""
from deeptutor.services.skill.hub import HubError, default_hub, get_hub_provider
target = (hub or default_hub()).strip().lower()
try:
refs = get_hub_provider(target).search(query, limit=limit)
except HubError as exc:
console.print(f"[bold red]Search failed:[/] {exc}")
raise typer.Exit(code=1)
if not refs:
console.print("[dim]No skills matched.[/]")
return
table = Table(title=f"{target}: {query}")
table.add_column("Ref", style="bold")
table.add_column("Version")
table.add_column("Summary")
for ref in refs:
table.add_row(
f"{ref.hub}:{ref.slug}",
ref.version or "-",
ref.summary[:100] or ref.display_name,
)
console.print(table)
console.print("[dim]Install with: deeptutor skill install <ref>[/]")
@app.command("install")
def skill_install(
ref: str = typer.Argument(..., help="Skill ref: <hub>:<slug>[@version]."),
name: str | None = typer.Option(
None, "--name", help="Install under a different local skill name."
),
force: bool = typer.Option(
False, "--force", help="Overwrite an existing skill with the same name."
),
allow_unverified: bool = typer.Option(
False,
"--allow-unverified",
help="Install even when the hub flags the package as suspicious.",
),
) -> None:
"""Install a skill from a hub into the local skill library."""
from deeptutor.services.skill.hub import HubError, install_from_hub
from deeptutor.services.skill.service import (
InvalidSkillNameError,
SkillExistsError,
SkillImportError,
get_skill_service,
)
service = get_skill_service()
try:
outcome = install_from_hub(
ref,
service=service,
rename_to=name,
force=force,
allow_unverified=allow_unverified,
)
except SkillExistsError as exc:
console.print(
f"[bold red]Skill `{exc}` already exists.[/] Re-run with --force to replace it."
)
raise typer.Exit(code=1)
except (HubError, SkillImportError, InvalidSkillNameError) as exc:
console.print(f"[bold red]Install failed:[/] {exc}")
raise typer.Exit(code=1)
info = outcome.result.info
verdict = outcome.verdict
verdict_style = {"ok": "green", "suspicious": "red"}.get(verdict.status, "yellow")
console.print(
f"[bold green]Installed[/] [bold]{info.name}[/]"
+ (f" [dim]({outcome.ref.hub}@{outcome.ref.version})[/]" if outcome.ref.version else "")
)
console.print(
f" verdict: [{verdict_style}]{verdict.status}[/]"
+ (f" [dim]{verdict.detail}[/]" if verdict.detail else "")
)
if verdict.status != "ok":
console.print(
f" [yellow]Review before use:[/] [dim]{service.root / info.name / 'SKILL.md'}[/]"
)
for entry in service.summary_entries():
if entry.name == info.name and not entry.available:
console.print(f" [yellow]unavailable until:[/] {', '.join(entry.missing)}")
for rel, reason in outcome.result.skipped:
console.print(f" [dim]skipped {rel}{reason}[/]")
@app.command("login")
def skill_login(
provider: str | None = typer.Argument(
None, help="登录方式:github | google(不填则在终端询问)。"
),
hub: str | None = typer.Option(None, "--hub", help="Target hub (default from settings)."),
no_browser: bool = typer.Option(
False, "--no-browser", help="不自动打开浏览器,只打印授权链接。"
),
) -> None:
"""浏览器授权登录到 skill hub,令牌保存在本地供 publish / update 使用。"""
from deeptutor.services.skill import credentials
from deeptutor.services.skill.hub import HubError, default_hub, get_hub_provider
from deeptutor.services.skill.taxonomy import Option
from .skill_login import hub_origin_from_base, run_login
from .skill_prompts import select_one
target_hub = (hub or default_hub()).strip().lower()
try:
provider_obj = get_hub_provider(target_hub)
except HubError as exc:
console.print(f"[bold red]{exc}[/]")
raise typer.Exit(code=1)
base_url = getattr(provider_obj, "base_url", None)
if not base_url:
console.print(f"[bold red]Hub `{target_hub}` 不支持网页登录(非 clawhub 型)。[/]")
raise typer.Exit(code=1)
origin = hub_origin_from_base(str(base_url))
prov = (provider or "").strip().lower()
if prov not in ("github", "google"):
prov = select_one(
[Option("github", "GitHub", "GitHub"), Option("google", "Google", "Google")],
"选择登录方式",
)
console.print(f"[dim]在浏览器完成 {prov} 授权(hub: {target_hub})…[/]")
result = run_login(
origin,
prov,
on_url=lambda u: console.print(
f" 若浏览器未自动打开,请手动访问:\n [underline]{u}[/]"
),
open_browser=not no_browser,
)
if result.error or not result.token:
console.print(f"[bold red]登录失败:[/] {result.error or '未收到令牌'}")
raise typer.Exit(code=1)
try:
credentials.store_token(target_hub, result.token, login=result.login)
except RuntimeError as exc:
console.print(f"[bold red]令牌保存失败:[/] {exc}")
raise typer.Exit(code=1)
who = f" as @{result.login}" if result.login else ""
console.print(f"[bold green]已登录[/] {target_hub}{who},令牌已保存到本地。")
@app.command("logout")
def skill_logout(
hub: str | None = typer.Option(None, "--hub", help="Target hub (default from settings)."),
) -> None:
"""清除本地保存的某个 hub 登录令牌。"""
from deeptutor.services.skill import credentials
from deeptutor.services.skill.hub import default_hub
target_hub = (hub or default_hub()).strip().lower()
if credentials.clear_token(target_hub):
console.print(f"[green]已登出[/] {target_hub}")
else:
console.print(f"[dim]{target_hub} 本来就没有保存的令牌。[/]")
@app.command("publish")
def skill_publish(
directory: str = typer.Argument(..., help="Skill directory containing SKILL.md."),
version: str | None = typer.Option(
None, "--version", help="semver to publish; overrides SKILL.md `version:`."
),
slug: str | None = typer.Option(
None, "--slug", help="Override slug (default: SKILL.md slug/name)."
),
track: str | None = typer.Option(
None, "--track", help="Track (skips the track prompt when set)."
),
hub: str | None = typer.Option(None, "--hub", help="Target hub (default from settings)."),
token: str | None = typer.Option(
None, "--token", help="Publish token; else env / `deeptutor skills login`."
),
yes: bool = typer.Option(
False,
"--yes",
"-y",
help="Non-interactive: take classification from SKILL.md/flags as-is (CI).",
),
) -> None:
"""Publish a skill: interactively tag it (track + facets), then upload.
Pre-flights the package format, walks the required track (one-of) and
optional facets — pre-filled from SKILL.md frontmatter — shows a summary
for confirmation, then publishes. ``--yes`` skips the prompts for CI.
"""
import sys
from deeptutor.services.skill.hub import (
HubError,
default_hub,
preflight_skill_dir,
publish_to_hub,
read_skill_metadata,
resolve_publish_identity,
)
from deeptutor.services.skill.service import SkillImportError
from .skill_prompts import collect_classification
target_hub = (hub or default_hub()).strip().lower()
# ── step 3: local format pre-flight ────────────────────────────────
pre = preflight_skill_dir(directory)
for warning in pre.warnings:
console.print(f"[yellow]⚠[/] {warning}")
if not pre.ok:
console.print("[bold red]格式预检未通过,请先修复:[/]")
for err in pre.errors:
console.print(f" [red]✗[/] {err}")
console.print(" [dim]格式规范见 https://eduhub.deeptutor.info/skill-format.md[/]")
raise typer.Exit(code=1)
console.print(
f"[green]✓[/] 格式预检通过 "
f"[dim]({pre.file_count} 个文件, {pre.total_bytes // 1024} KB)[/]"
)
fm = read_skill_metadata(directory)
interactive = (not yes) and sys.stdin.isatty()
overrides: dict[str, str] = {}
eff_version = (version or str(fm.get("version") or "")).strip()
if interactive:
# step 4: required track + optional facets, pre-filled from frontmatter
overrides = collect_classification({**fm, "track": track or fm.get("track")})
if not eff_version:
eff_version = typer.prompt("\n版本号 version (semver)", default="1.0.0").strip()
# step 5: confirmation summary
preview_slug, _ = resolve_publish_identity(directory, slug=slug, version=eff_version)
console.print(
_summary_table(
"确认发布信息",
hub=target_hub,
slug=preview_slug,
version=eff_version,
overrides=overrides,
)
)
if not typer.confirm("确认提交?", default=True):
console.print("[dim]已取消。[/]")
raise typer.Exit(code=0)
else:
# Non-interactive (CI): track must come from --track or frontmatter.
eff_track = track or str(fm.get("track") or "")
if not eff_track:
console.print(
"[bold red]缺少 track。[/] 加 --track,或在 SKILL.md 写 track:"
"或去掉 --yes 走交互式打标。"
)
raise typer.Exit(code=1)
tok = _resolve_token(token, target_hub)
if not tok:
console.print(
"[bold red]未登录。[/] 运行 [bold]deeptutor skills login[/] 完成浏览器授权,"
"或用 --token / $DEEPTUTOR_HUB_TOKEN 传入令牌。"
)
raise typer.Exit(code=1)
try:
outcome = publish_to_hub(
directory,
token=tok,
version=eff_version or None,
slug=slug,
track=track,
hub=target_hub,
overrides=overrides or None,
)
except (HubError, SkillImportError) as exc:
console.print(f"[bold red]Publish failed:[/] {exc}")
raise typer.Exit(code=1)
console.print(
f"[bold green]Published[/] [bold]{outcome.slug}@{outcome.version}[/] → {outcome.hub}"
)
console.print(
f" [dim]install with:[/] deeptutor skill install {outcome.hub}:{outcome.slug}"
)
@app.command("update")
def skill_update(
directory: str | None = typer.Argument(
None, help="新版本的 skill 目录(升级新版本时用;回退不需要)。"
),
hub: str | None = typer.Option(None, "--hub", help="Target hub (default from settings)."),
token: str | None = typer.Option(
None, "--token", help="Publish token; else env / `deeptutor skills login`."
),
) -> None:
"""维护已发布技能:列出我的技能 → 选一个 → 回退旧版本,或发布新版本。
升级新版本时,打标环节默认沿用该技能当前的 track 与各维度标签;回退则
把 ``latest`` 指针指回某个更旧的已发布版本,不新建版本。
"""
import sys
from deeptutor.services.skill.hub import (
HubError,
default_hub,
get_hub_provider,
preflight_skill_dir,
publish_to_hub,
)
from deeptutor.services.skill.service import SkillImportError
from deeptutor.services.skill.taxonomy import Option
from .skill_prompts import collect_classification, select_one
target_hub = (hub or default_hub()).strip().lower()
if not sys.stdin.isatty():
console.print("[bold red]update 是交互式命令,请在终端中运行。[/]")
raise typer.Exit(code=1)
tok = _resolve_token(token, target_hub)
if not tok:
console.print(
"[bold red]未登录。[/] 运行 [bold]deeptutor skills login[/] 完成浏览器授权,"
"或用 --token 传入令牌。"
)
raise typer.Exit(code=1)
provider = get_hub_provider(target_hub)
lister = getattr(provider, "list_my_skills", None)
if not callable(lister):
console.print(f"[bold red]Hub `{target_hub}` 不支持列出已发布技能。[/]")
raise typer.Exit(code=1)
try:
mine = [s for s in lister(tok) if s.get("slug")]
except HubError as exc:
console.print(f"[bold red]获取失败:[/] {exc}")
raise typer.Exit(code=1)
if not mine:
console.print(f"[dim]你还没有在 {target_hub} 发布过技能。[/]")
raise typer.Exit(code=0)
# ── step 3: pick a skill, then pick the action ────────────────────
skill_opts = [
Option(
str(s["slug"]),
f"{s.get('displayName') or s['slug']} v{s.get('version') or '-'}",
str(s["slug"]),
)
for s in mine
]
chosen_slug = select_one(skill_opts, "选择要更新的技能")
chosen = next(s for s in mine if s.get("slug") == chosen_slug)
versions = [str(v) for v in (chosen.get("versions") or [])]
current = str(chosen.get("version") or "")
action = select_one(
[
Option("rollback", "回退版本(把 latest 指回旧版本)", "Roll back"),
Option("upgrade", "发布新版本", "Publish a new version"),
],
f"{chosen_slug} 做什么?",
)
# ── rollback: move latest to an older version ─────────────────────
if action == "rollback":
if len([v for v in versions if v != current]) == 0:
console.print(f"[dim]{chosen_slug} 只有一个版本({current}),无法回退。[/]")
raise typer.Exit(code=0)
version_opts = [Option(v, "← 当前 latest" if v == current else "", v) for v in versions]
target_version = select_one(version_opts, "回退 latest 到哪个版本?")
if target_version == current:
console.print("[dim]已是当前 latest,无需回退。[/]")
raise typer.Exit(code=0)
if not typer.confirm(
f"{chosen_slug} 的 latest 从 {current or '-'} 回退到 {target_version}",
default=True,
):
console.print("[dim]已取消。[/]")
raise typer.Exit(code=0)
tag_setter = getattr(provider, "set_dist_tag", None)
if not callable(tag_setter):
console.print(f"[bold red]Hub `{target_hub}` 不支持回退 latest。[/]")
raise typer.Exit(code=1)
try:
tag_setter(chosen_slug, version=target_version, token=tok)
except HubError as exc:
console.print(f"[bold red]回退失败:[/] {exc}")
raise typer.Exit(code=1)
console.print(
f"[bold green]已回退[/] {chosen_slug} 的 latest → [bold]{target_version}[/]"
)
return
# ── upgrade: publish a new version (tags default to current) ──────
if not directory:
directory = typer.prompt("新版本的 skill 目录路径").strip()
pre = preflight_skill_dir(directory)
for warning in pre.warnings:
console.print(f"[yellow]⚠[/] {warning}")
if not pre.ok:
console.print("[bold red]格式预检未通过,请先修复:[/]")
for err in pre.errors:
console.print(f" [red]✗[/] {err}")
raise typer.Exit(code=1)
console.print(
f"[green]✓[/] 格式预检通过 "
f"[dim]({pre.file_count} 个文件, {pre.total_bytes // 1024} KB)[/]"
)
overrides = collect_classification(chosen)
new_version = typer.prompt(
f"\n新版本号 version (semver,当前 latest = {current or '-'})"
).strip()
console.print(
_summary_table(
"确认升级信息",
hub=target_hub,
slug=chosen_slug,
version=new_version,
overrides=overrides,
)
)
if not typer.confirm("确认发布新版本?", default=True):
console.print("[dim]已取消。[/]")
raise typer.Exit(code=0)
try:
outcome = publish_to_hub(
directory,
token=tok,
version=new_version or None,
slug=chosen_slug,
hub=target_hub,
overrides=overrides,
)
except (HubError, SkillImportError) as exc:
console.print(f"[bold red]Update failed:[/] {exc}")
raise typer.Exit(code=1)
console.print(
f"[bold green]Updated[/] [bold]{outcome.slug}@{outcome.version}[/] → {outcome.hub}"
)
@app.command("list")
def skill_list() -> None:
"""List local skills, including hub provenance."""
from deeptutor.services.skill.service import get_skill_service
service = get_skill_service()
table = Table(title="Skills")
table.add_column("Name", style="bold")
table.add_column("Source")
table.add_column("Origin")
table.add_column("Description")
for info in service.list_skills():
origin = service.hub_origin(info.name)
origin_label = "-"
if origin:
version = str(origin.get("version") or "").strip()
origin_label = str(origin.get("hub") or "hub") + (f"@{version}" if version else "")
table.add_row(info.name, info.source, origin_label, info.description[:80])
console.print(table)
@app.command("remove")
def skill_remove(
name: str = typer.Argument(..., help="Skill name to remove."),
) -> None:
"""Remove a user-layer skill (builtin skills are read-only)."""
from deeptutor.services.skill.service import (
InvalidSkillNameError,
SkillNotFoundError,
SkillReadOnlyError,
get_skill_service,
)
try:
get_skill_service().delete(name)
except (SkillNotFoundError, InvalidSkillNameError):
console.print(f"[bold red]Skill not found:[/] {name}")
raise typer.Exit(code=1)
except SkillReadOnlyError as exc:
console.print(f"[bold red]{exc}[/]")
raise typer.Exit(code=1)
console.print(f"[green]Removed[/] {name}")
+134
View File
@@ -0,0 +1,134 @@
"""
Browser OAuth login for skill hubs (loopback token capture)
============================================================
``deeptutor skills login`` opens the hub's GitHub/Google authorization page in
a browser and captures the minted API token via a one-shot loopback server on
``127.0.0.1``. The hub only ever redirects the token to ``127.0.0.1:<port>``
(host fixed server-side), and we verify an opaque ``state`` nonce on the way
back — so a stray request can't slip a token into the store.
The URL building and origin derivation are pure functions so they can be
unit-tested without standing up a browser flow.
"""
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
import http.server
import secrets
import threading
import urllib.parse
import webbrowser
_SUCCESS_HTML = (
"<!doctype html><meta charset=utf-8><title>登录成功</title>"
"<body style='font:16px system-ui;text-align:center;padding:3rem'>"
"<h2>✓ 已登录 EduHub</h2><p>令牌已交给命令行,可以关闭此页面返回终端。</p></body>"
)
_ERROR_HTML = (
"<!doctype html><meta charset=utf-8><title>登录失败</title>"
"<body style='font:16px system-ui;text-align:center;padding:3rem'>"
"<h2>登录失败</h2><p>请回到终端查看错误信息并重试。</p></body>"
)
@dataclass(slots=True)
class LoginResult:
token: str | None
login: str | None
error: str | None
def hub_origin_from_base(base_url: str) -> str:
"""Derive the web origin from a hub's ``/api/v1`` base URL.
``https://eduhub.deeptutor.info/api/v1`` -> ``https://eduhub.deeptutor.info``.
"""
b = base_url.rstrip("/")
for suffix in ("/api/v1", "/api"):
if b.endswith(suffix):
return b[: -len(suffix)]
return b
def oauth_start_url(origin: str, provider: str, *, port: int, state: str) -> str:
"""The hub OAuth start URL carrying the CLI loopback port + state nonce."""
query = urllib.parse.urlencode({"cli_port": port, "cli_state": state})
return f"{origin.rstrip('/')}/api/auth/oauth/{provider}?{query}"
def run_login(
origin: str,
provider: str,
*,
on_url: Callable[[str], None] | None = None,
open_browser: bool = True,
timeout: float = 300.0,
) -> LoginResult:
"""Run the loopback OAuth dance and return the captured token.
Stands up a loopback server on an ephemeral port, opens (or prints) the
authorize URL, then blocks until the hub redirects back with a token whose
``state`` matches, or ``timeout`` elapses.
"""
state = secrets.token_urlsafe(24)
captured: dict[str, str | None] = {}
done = threading.Event()
class Handler(http.server.BaseHTTPRequestHandler):
def do_GET(self) -> None: # noqa: N802
parsed = urllib.parse.urlparse(self.path)
if parsed.path != "/callback":
self.send_response(404)
self.end_headers()
return
params = urllib.parse.parse_qs(parsed.query)
def first(key: str) -> str | None:
values = params.get(key)
return values[0] if values else None
if first("state") != state:
self.send_response(400)
self.end_headers()
self.wfile.write(b"state mismatch")
return
captured["token"] = first("token")
captured["login"] = first("login")
captured["error"] = first("error")
ok = bool(captured.get("token")) and not captured.get("error")
self.send_response(200)
self.send_header("Content-Type", "text/html; charset=utf-8")
self.end_headers()
self.wfile.write((_SUCCESS_HTML if ok else _ERROR_HTML).encode("utf-8"))
done.set()
def log_message(self, *args: object) -> None: # silence default logging
return
server = http.server.HTTPServer(("127.0.0.1", 0), Handler)
port = server.server_address[1]
url = oauth_start_url(origin, provider, port=port, state=state)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
try:
if on_url:
on_url(url)
if open_browser:
try:
webbrowser.open(url)
except Exception:
pass # headless / no browser — user opens the printed URL
if not done.wait(timeout):
return LoginResult(None, None, "登录超时(未在浏览器完成授权)")
finally:
server.shutdown()
server.server_close()
return LoginResult(captured.get("token"), captured.get("login"), captured.get("error"))
__all__ = ["LoginResult", "hub_origin_from_base", "oauth_start_url", "run_login"]
+208
View File
@@ -0,0 +1,208 @@
"""
Interactive taxonomy pickers for ``skill publish`` / ``skill update``.
Terminal prompts that mirror EduHub's web upload form: a required single-select
``track`` plus the optional multi-select facets (language is single-select with
a default). Selections can be pre-filled — from SKILL.md frontmatter on publish,
or from the current skill's labels on update — so the common path is just
pressing Enter through the prompts.
Kept presentation-only and dependency-light (rich console + ``typer.prompt``)
so the publish/update commands stay readable and these are easy to unit-test.
"""
from __future__ import annotations
from collections.abc import Sequence
from typing import Any
import typer
from deeptutor.services.skill.taxonomy import (
AUDIENCE_OPTIONS,
DOMAIN_TREE,
FORM_OPTIONS,
LANGUAGE_OPTIONS,
STAGE_OPTIONS,
TRACK_OPTIONS,
Option,
is_valid_track,
)
from .common import console
def _render_options(options: Sequence[Option], selected: set[str], locale: str) -> None:
for i, opt in enumerate(options, 1):
mark = "[green]●[/]" if opt.value in selected else "[dim]○[/]"
console.print(f" {mark} [bold]{i}[/]. {opt.label(locale)} [dim]{opt.value}[/]")
def _parse_indices(raw: str, count: int) -> list[int] | None:
"""Parse ``"1, 3 4"`` into 0-based indices; None if any token is invalid."""
out: list[int] = []
for tok in raw.replace(",", " ").split():
if not tok.isdigit():
return None
n = int(tok)
if not (1 <= n <= count):
return None
if n - 1 not in out:
out.append(n - 1)
return out
def select_one(
options: Sequence[Option],
title: str,
*,
hint: str = "",
default: str | None = None,
locale: str = "zh",
) -> str:
"""Prompt for exactly one value. Enter accepts ``default`` when given."""
console.print(f"\n[bold]{title}[/]" + (f" [dim]{hint}[/]" if hint else ""))
_render_options(options, {default} if default else set(), locale)
default_idx = next((i + 1 for i, o in enumerate(options) if o.value == default), None)
suffix = f"(回车=默认 {default_idx}" if default_idx else ""
while True:
raw = typer.prompt(f" 输入编号{suffix}", default="", show_default=False).strip()
if not raw and default_idx:
return options[default_idx - 1].value
idx = _parse_indices(raw, len(options))
if idx and len(idx) == 1:
return options[idx[0]].value
console.print(" [red]请输入一个有效编号。[/]")
def select_many(
options: Sequence[Option],
title: str,
*,
hint: str = "可多选,逗号分隔;回车=保留当前;输入 - 清空",
preselected: Sequence[str] = (),
locale: str = "zh",
) -> list[str]:
"""Prompt for zero or more values. Enter keeps ``preselected``; ``-`` clears."""
current = [v for v in preselected if any(o.value == v for o in options)]
console.print(f"\n[bold]{title}[/] [dim]{hint}[/]")
_render_options(options, set(current), locale)
while True:
raw = typer.prompt(" 输入编号", default="", show_default=False).strip()
if not raw:
return list(current)
if raw == "-":
return []
idx = _parse_indices(raw, len(options))
if idx is not None:
return [options[i].value for i in idx]
console.print(" [red]请输入有效编号(逗号分隔),或回车/-。[/]")
def select_domains(
*,
preselected: Sequence[str] = (),
locale: str = "zh",
) -> list[str]:
"""Two-level domain picker: pick top-level groups, then optional children.
A group with chosen children contributes those children; a group with none
contributes the group itself — matching the web form's ``resolveDomains``.
"""
pre_roots = {v.split(".")[0] for v in preselected}
root_options = [Option(n.value, n.zh, n.en) for n in DOMAIN_TREE]
roots = select_many(
root_options,
"领域 (domains) — 可多选;选中后可再挑细分",
preselected=[r for r in pre_roots if any(o.value == r for o in root_options)],
locale=locale,
)
result: list[str] = []
for root in roots:
node = next((n for n in DOMAIN_TREE if n.value == root), None)
if node is None or not node.children:
result.append(root)
continue
pre_children = [v for v in preselected if v.startswith(f"{root}.")]
children = select_many(
list(node.children),
f"{node.label(locale)}」的细分 — 可多选;回车=用「{node.label(locale)}」整体",
hint="可多选,逗号分隔;回车=保留当前/整体;输入 - 清空",
preselected=pre_children,
locale=locale,
)
result.extend(children if children else [root])
return result
def _as_list(defaults: dict[str, Any], key: str) -> list[str]:
value = defaults.get(key)
if isinstance(value, list):
return [str(x).strip() for x in value if str(x).strip()]
if isinstance(value, str) and value.strip():
return [s.strip() for s in value.split(",") if s.strip()]
return []
def collect_classification(defaults: dict[str, Any], *, locale: str = "zh") -> dict[str, str]:
"""Walk the full tagging flow (track + facets), pre-filled from ``defaults``.
Shared by ``skill publish`` (defaults = SKILL.md frontmatter) and
``skill update`` (defaults = the current skill's labels). Returns the
comma-joined ``overrides`` dict ``publish_to_hub`` expects.
"""
track_default = str(defaults.get("track") or "")
track = select_one(
TRACK_OPTIONS,
"主类目 track — 必选",
hint="这个技能服务什么场景?",
default=track_default if is_valid_track(track_default) else None,
locale=locale,
)
lang_default = str(defaults.get("language") or "zh")
language = select_one(
LANGUAGE_OPTIONS,
"语言 language — 必选",
hint="技能与用户交流所用的语言",
default=lang_default if any(o.value == lang_default for o in LANGUAGE_OPTIONS) else "zh",
locale=locale,
)
domains = select_domains(preselected=_as_list(defaults, "domains"), locale=locale)
stages = select_many(
STAGE_OPTIONS,
"学段 stages — 可选;空=全学段通用",
preselected=_as_list(defaults, "stages"),
locale=locale,
)
forms = select_many(
FORM_OPTIONS,
"形式 forms — 可选;技能如何与用户交互",
preselected=_as_list(defaults, "forms"),
locale=locale,
)
audiences = select_many(
AUDIENCE_OPTIONS,
"受众 audiences — 可选;空=面向学习者",
preselected=_as_list(defaults, "audiences"),
locale=locale,
)
tags_default = ",".join(_as_list(defaults, "tags"))
tags = typer.prompt(
"\n标签 tags — 比领域更细,逗号分隔,可空",
default=tags_default,
show_default=bool(tags_default),
).strip()
return {
"track": track,
"language": language,
"domains": ",".join(domains),
"stages": ",".join(stages),
"forms": ",".join(forms),
"audiences": ",".join(audiences),
"tags": tags,
}
__all__ = ["collect_classification", "select_domains", "select_many", "select_one"]