chore: import upstream snapshot with attribution
Validate YAML Workflows / Validate YAML Configuration Files (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 12:37:51 +08:00
commit d0e4308def
614 changed files with 74458 additions and 0 deletions
+83
View File
@@ -0,0 +1,83 @@
# Attachment & Artifact API Guide
> Audience: operators integrating directly with backend REST/WS endpoints. The Web UI already handles most scenarios.
Attachments are files that can be uploaded, downloaded, or registered during a session. Artifacts are events emitted when attachments change so clients can listen in real time. This guide summarizes the REST endpoints, WebSocket mirrors, and storage policies.
## 1. Upload & List
### 1.1 Upload file
`POST /api/uploads/{session_id}`
- Headers: `Content-Type: multipart/form-data`
- Form field: `file`
- Response:
```json
{
"attachment_id": "att_bxabcd",
"name": "spec.md",
"mime": "text/markdown",
"size": 12345
}
```
- Files land in `WareHouse/<session>/code_workspace/attachments/` and are recorded in `attachments_manifest.json`.
### 1.2 List attachments
`GET /api/uploads/{session_id}`
- Returns metadata for all attachments in the session (ID, file name, mime, size, source).
### 1.3 Reference in execution requests
- `POST /api/workflow/execute` or WebSocket `human_input` payloads can include `attachments: ["att_xxx"]`. You still must supply `task_prompt`, even when you only want file uploads.
## 2. Artifact Events & Downloads
### 2.1 Real-time artifact events
`GET /api/sessions/{session_id}/artifact-events`
- Query params: `after`, `wait_seconds`, `include_mime`, `include_ext`, `max_size`, `limit`.
- Response includes `events[]`, `next_cursor`, `has_more`, `timed_out`.
- Event sample:
```json
{
"artifact_id": "art_123",
"attachment_id": "att_456",
"node_id": "python_runner",
"path": "code_workspace/result.json",
"size": 2048,
"mime": "application/json",
"hash": "sha256:...",
"timestamp": 1732699900
}
```
- WebSocket emits the same data via `artifact_created`, so dashboard clients can subscribe live.
### 2.2 Download a single artifact
`GET /api/sessions/{session_id}/artifacts/{artifact_id}`
- Query: `mode=meta|stream`, `download=true|false`.
- `meta` → metadata only; `stream` → file content. Add `download=true` to include `Content-Disposition`.
- Small files may be returned as `data_uri` when the server enables it.
### 2.3 Download an entire session
`GET /api/sessions/{session_id}/download`
- Packages `WareHouse/<session>/` into a zip for batch download.
## 3. File Lifecycle
1. Upload stage: files go under `code_workspace/attachments/`, and the manifest records `source`, `workspace_path`, `storage`, etc.
2. Python nodes/tools can call `AttachmentStore.register_file()` to turn workspace files into attachments; `WorkspaceArtifactHook` syncs events.
3. By default we retain all attachments for post-run downloads. Set `MAC_AUTO_CLEAN_ATTACHMENTS=1` to delete the `attachments/` directory after the session completes.
4. WareHouse zip downloads do **not** delete originals; schedule your own archival/cleanup jobs.
## 4. Size & Security
- **Size limits**: No hard cap in backend; enforce via reverse proxy (`client_max_body_size`, `max_request_body_size`) or customize `AttachmentService.save_upload_file`.
- **File types**: MIME detection controls `MessageBlockType` (image/audio/video/file); filter via `include_mime` as needed.
- **Virus/sensitive data**: Clients should pre-scan uploads; you can also trigger scanning services after save.
- **Permissions**: Attachment APIs require the session ID. In production guard with proxy-layer auth or internal JWT checks to prevent unauthorized downloads.
## 5. FAQ
| Issue | Mitigation |
| --- | --- |
| Upload 413 Payload Too Large | Raise proxy limits or FastAPI `client_max_size`; confirm disk quota. |
| Download link 404 | Check `session_id` spelling (allowed chars: letters/digits/`_-`) and confirm the session hasnt been purged. |
| Missing artifact events | Ensure WebSocket is connected or use `artifact-events` REST polling with the `after` cursor. |
| Attachment not visible in Python node | Verify `code_workspace/attachments/` hasnt been cleaned and `_context[python_workspace_root]` is correct. |
## 6. Client Patterns
- **Web UI**: Use artifact long-polling or WebSocket to refresh lists in real time; offer a “download all” button once nodes finish.
- **CLI/automation**: After runs complete, call `/download` for the zip; if you need just a subset, combine `artifact-events` with `include_ext` filters.
- **Test rigs**: Script the upload/download flow to validate proxy limits and CORS before shipping.
+111
View File
@@ -0,0 +1,111 @@
# Config Schema API Contract
This reference explains how `/api/config/schema` and `/api/config/schema/validate` expose DevAll's dynamic config metadata so IDEs, frontend form builders, and CLI tools can scope schemas with breadcrumbs.
## 1. Endpoints
| Method & Path | Purpose |
| --- | --- |
| `POST /api/config/schema` | Returns the schema for a config node described by breadcrumbs. |
| `POST /api/config/schema/validate` | Validates a YAML/JSON document and optionally echoes the scoped schema. |
### 1.1 Request body (shared fields)
```json
{
"breadcrumbs": [
{"node": "DesignConfig", "field": "graph"},
{"node": "GraphConfig", "field": "nodes"},
{"node": "NodeConfig", "value": "model"}
]
}
```
- `node` (required): class name (`DesignConfig`, `GraphConfig`, `NodeConfig`, etc.) that must match the class reached so far.
- `field` (optional): child field to traverse. When omitted, the breadcrumb only asserts you remain on `node`.
- `value` (optional): use when the child class depends on a discriminator (e.g., node `type`). Supply the value as it would appear in YAML.
- `index` (optional int): reserved for list traversal; current configs rely on `value`/`field` for navigation.
### 1.2 `/schema` response
```json
{
"schemaVersion": "0.1.0",
"node": "NodeConfig",
"fields": [
{
"name": "id",
"typeHint": "str",
"required": true,
"description": "Unique node identifier"
},
{
"name": "type",
"typeHint": "str",
"required": true,
"enum": ["model", "python", "agent"],
"enumOptions": [
{"value": "model", "label": "LLM Node", "description": "Runs provider-backed models"}
]
}
],
"constraints": [...],
"breadcrumbs": [...],
"cacheKey": "f90d..."
}
```
- `fields`: serialized `ConfigFieldSpec` entries; nested targets include `childRoutes`.
- `constraints`: emitted from `collect_schema()` (mutual exclusivity, required combos, etc.).
- `cacheKey`: SHA-1 hash of `{node, breadcrumbs}` so clients can memoize responses.
### 1.3 `/schema/validate` payloads
Add `document` alongside breadcrumbs:
```json
{
"breadcrumbs": [{"node": "DesignConfig"}],
"document": """
name: demo
version: 0.4.0
workflow:
nodes: []
edges: []
"""
}
```
Responses:
- Valid document: `{ "valid": true, "schema": { ... } }`
- Config error:
```json
{
"valid": false,
"error": "field 'nodes' must not be empty",
"path": ["workflow", "nodes"],
"schema": { ... }
}
```
- Malformed YAML: HTTP 400 with `{ "message": "invalid_yaml", "error": "..." }`.
## 2. Breadcrumb Tips
- Begin with `{ "node": "DesignConfig" }`.
- Each hops `node` must match the current config class or the API returns HTTP 422.
- Use `field` to step into nested configs (graph → nodes → config, etc.).
- Use `value` for discriminator-based children (node `type`, tooling `type`, etc.).
- Non-navigable targets raise `field '<name>' on <node> is not navigable`.
## 3. CLI Helper
```bash
python run.py --inspect-schema --schema-breadcrumbs '[{"node":"DesignConfig","field":"graph"}]'
```
The CLI prints the same JSON as `/schema`, which is useful while editing `FIELD_SPECS` or debugging registries before exporting templates.
## 4. Frontend Pattern
1. Fetch base schema with `[{node: 'DesignConfig', field: 'graph'}]` to render the workflow form.
2. When users open nested modals (node config, tooling config, etc.), append breadcrumbs describing the path and refetch.
3. Cache responses using `cacheKey` + breadcrumbs.
4. Before saving, call `/schema/validate` to surface `error` + `path` inline.
## 5. Error Reference
| HTTP Code | Situation | Detail payload |
| --- | --- | --- |
| 400 | YAML parse failure | `{ "message": "invalid_yaml", "error": "..." }` |
| 422 | Breadcrumb resolution failure | `{ "message": "breadcrumb node 'X'..." }` |
| 200 + `valid=false` | Backend `ConfigError` | `{ "error": "...", "path": ["workflow", ...] }` |
| 200 + `valid=true` | Document OK | Schema echoed back for the requested breadcrumbs. |
Pair this contract with `FIELD_SPECS` to build schema-aware experiences without hardcoding config structures.
+276
View File
@@ -0,0 +1,276 @@
# Dynamic Execution Mode Guide
Dynamic execution mode enables parallel processing behavior defined at the edge level, supporting Map (fan-out) and Tree (fan-out + reduce) modes. When messages pass through edges configured with `dynamic`, the target node dynamically expands into multiple parallel instances based on split results.
## 1. Overview
| Mode | Description | Output | Use Cases |
|------|-------------|--------|-----------|
| **Map** | Fan-out execution, splits messages into multiple units for parallel processing | `List[Message]` (flattened results) | Batch processing, parallel queries |
| **Tree** | Fan-out + reduce, parallel processing followed by recursive group merging | Single `Message` | Long text summarization, hierarchical aggregation |
## 2. Configuration Structure
Dynamic configuration is defined on **edges**, not nodes:
```yaml
edges:
- from: Source Node
to: Target Node
trigger: true
carry_data: true
dynamic: # Edge-level dynamic execution config
type: map # map or tree
split: # Message splitting strategy
type: message # message | regex | json_path
# pattern: "..." # Required for regex mode
# json_path: "..." # Required for json_path mode
config: # Mode-specific config
max_parallel: 5 # Maximum concurrency
```
### 2.1 Core Concepts
- **Dynamic edge**: An edge with `dynamic` configured; messages passing through trigger dynamic expansion of the target node
- **Static edge**: An edge without `dynamic` configured; messages are **replicated** to all dynamic expansion instances
- **Target node expansion**: The target node is "virtually" expanded into multiple parallel instances based on split results
### 2.2 Multi-Edge Consistency Rule
> [!IMPORTANT]
> When a node has multiple incoming edges with `dynamic` configured, all dynamic edge configurations **must be identical** (type, split, config). Otherwise, execution will fail with an error.
## 3. Split Strategies
Split defines how messages passing through the edge are partitioned into parallel execution units.
### 3.1 message Mode (Default)
Each message passing through the edge becomes an independent execution unit. This is the most common mode.
```yaml
split:
type: message
```
**Execution behavior**:
- Source node outputs 4 messages through the dynamic edge
- Splits into 4 parallel units, target node executes 4 times
### 3.2 regex Mode
Uses regular expressions to extract matches from text content.
```yaml
split:
type: regex
pattern: "(?s).{1,2000}(?:\\s|$)" # Split every ~2000 characters
```
**Typical uses**:
- Split by paragraph: `pattern: "\\n\\n"`
- Split by line: `pattern: ".+"`
- Fixed-length chunks: `pattern: "(?s).{1,N}"`
### 3.3 json_path Mode
Extracts array elements from JSON-formatted output using JSONPath expressions.
```yaml
split:
type: json_path
json_path: "$.items[*]" # JSONPath expression
```
## 4. Map Mode Details
Map mode splits messages and executes the target node in parallel, flattening outputs into `List[Message]`.
### 4.1 Configuration
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `max_parallel` | int | 10 | Maximum concurrent executions |
### 4.2 Execution Flow
```mermaid
flowchart LR
Source["Source Node Output"] --> Edge["Dynamic Edge (map)"]
Edge --> Split["Split"]
Split --> U1["Unit 1"]
Split --> U2["Unit 2"]
Split --> U3["Unit N"]
U1 --> P1["Target Node #1"]
U2 --> P2["Target Node #2"]
U3 --> P3["Target Node #N"]
P1 --> Merge["Merge Results"]
P2 --> Merge
P3 --> Merge
Merge --> Output["List[Message]"]
```
## 5. Tree Mode Details
Tree mode adds reduction layers on top of Map, recursively merging parallel results by group until a single output remains.
### 5.1 Configuration
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `group_size` | int | 3 | Number of elements per reduction group, minimum 2 |
| `max_parallel` | int | 10 | Maximum concurrent executions per layer |
### 5.2 Execution Flow
```mermaid
flowchart TB
subgraph Layer1["Layer 1: Parallel Execution"]
I1["Unit 1"] --> R1["Result 1"]
I2["Unit 2"] --> R2["Result 2"]
I3["Unit 3"] --> R3["Result 3"]
I4["Unit 4"] --> R4["Result 4"]
I5["Unit 5"] --> R5["Result 5"]
I6["Unit 6"] --> R6["Result 6"]
end
subgraph Layer2["Layer 2: Group Reduction (group_size=3)"]
R1 & R2 & R3 --> G1["Reduction Group 1"]
R4 & R5 & R6 --> G2["Reduction Group 2"]
end
subgraph Layer3["Layer 3: Final Reduction"]
G1 & G2 --> Final["Final Result"]
end
```
## 6. Static Edge Message Replication
When a target node has both dynamic and static incoming edges:
- **Dynamic edge messages**: Split according to strategy, each unit executes the target node once
- **Static edge messages**: **Replicated** to every dynamic expansion instance
```yaml
nodes:
- id: Task Generator
type: passthrough
config: ...
- id: Extra Requirement
type: literal
config:
content: "Please use concise language"
- id: Processor
type: agent
config:
name: gpt-4o
role: Process task
edges:
- from: Task Generator
to: Processor
dynamic: # Dynamic edge: 4 tasks → 4 parallel units
type: map
split:
type: message
config:
max_parallel: 10
- from: Extra Requirement
to: Processor # Static edge: replicated to all 4 instances
trigger: true
carry_data: true
```
**Execution result**: Processor executes 4 times, each receiving 1 task + "Please use concise language"
## 7. Complete Examples
### 7.1 Travel Planning (Map + Tree Combination)
```yaml
graph:
nodes:
- id: Eat Planner
type: literal
config:
content: Plan what to eat in Shanghai
role: user
- id: Play Planner
type: literal
config:
content: Plan what to do in Shanghai
role: user
- id: Stay Planner
type: literal
config:
content: Plan where to stay in Shanghai
role: user
- id: Collector
type: passthrough
config:
only_last_message: false
- id: Travel Executor
type: agent
config:
name: gpt-4o
role: You are a travel planner. Please plan according to user requests.
- id: Final Aggregator
type: agent
config:
name: gpt-4o
role: Please integrate the inputs into a complete travel plan.
edges:
- from: Eat Planner
to: Collector
- from: Play Planner
to: Collector
- from: Stay Planner
to: Collector
- from: Collector
to: Travel Executor
dynamic: # Map fan-out: 3 planning requests → 3 parallel executions
type: map
split:
type: message
config:
max_parallel: 10
- from: Travel Executor
to: Final Aggregator
dynamic: # Tree reduce: 3 results → 1 final plan
type: tree
split:
type: message
config:
group_size: 2
max_parallel: 10
```
### 7.2 Long Document Summarization (Tree Mode)
```yaml
edges:
- from: Document Source
to: Summarizer
dynamic:
type: tree
split:
type: regex
pattern: "(?s).{1,2000}(?:\\s|$)" # 2000-character chunks
config:
group_size: 3
max_parallel: 10
```
## 8. Performance Tips
- **Control concurrency**: Set reasonable `max_parallel` to avoid API rate limiting
- **Optimize split granularity**: Too fine increases overhead, too coarse limits parallelism
- **Tree group size**: `group_size=2-4` is usually optimal
- **Monitor costs**: Dynamic mode significantly increases API calls
## 9. Related Documentation
- [Edge Configuration Guide](../edges.md)
- [Workflow Authoring Guide](../workflow_authoring.md)
- [Agent Node Configuration](../nodes/agent.md)
+230
View File
@@ -0,0 +1,230 @@
# Graph Execution Logic
> Version: 2025-12-16
This document explains how the DevAll backend parses and executes workflow graphs, with particular focus on handling complex graphs containing cyclic structures.
## 1. Execution Engine Overview
The DevAll workflow execution engine supports two types of graph structures:
| Graph Type | Characteristics | Execution Strategy |
|------------|-----------------|-------------------|
| **DAG (Directed Acyclic Graph)** | No cyclic dependencies between nodes | Topological sort + parallel layer execution |
| **Cyclic Directed Graph** | Contains one or more loop structures | Recursive super-node scheduling |
The execution engine automatically detects the graph structure and selects the appropriate execution strategy.
## 2. DAG Execution Flow
For workflow graphs without cycles, the engine uses standard DAG scheduling:
1. **Build predecessor/successor relationships**: Parse edge definitions to establish `predecessors` and `successors` lists for each node
2. **Calculate in-degrees**: Count the number of predecessors for each node
3. **Topological sort**: Place nodes with in-degree 0 in the first layer; after execution, decrement successor in-degrees; new zero in-degree nodes enter the next layer
4. **Parallel layer execution**: Nodes within the same layer have no dependencies and can execute concurrently
```mermaid
flowchart LR
subgraph Layer1["Execution Layer 1"]
A["Node A"]
B["Node B"]
end
subgraph Layer2["Execution Layer 2"]
C["Node C"]
end
subgraph Layer3["Execution Layer 3"]
D["Node D"]
end
A --> C
B --> C
C --> D
```
## 3. Cyclic Graph Execution Flow
### 3.1 Tarjan's Strongly Connected Components Detection
When cyclic structures exist in the graph, the execution engine first uses **Tarjan's algorithm** to detect all Strongly Connected Components (SCCs). Tarjan's algorithm identifies all cycles in the graph in O(|V|+|E|) time complexity through depth-first search.
An SCC containing more than one node constitutes a cycle structure.
### 3.2 Super Node Construction
After detecting cycles, the execution engine abstracts each cycle into a "Super Node":
- All nodes within the cycle are encapsulated in the super node
- Dependencies between super nodes derive from cross-cycle edges between original nodes
- The resulting super node graph is guaranteed to be a DAG, enabling topological sorting
```mermaid
flowchart TB
subgraph Original["Original Graph"]
direction TB
A1["A"] --> B1["B"]
B1 --> C1["C"]
C1 --> B1
C1 --> D1["D"]
end
subgraph Abstracted["Super Node Graph"]
direction TB
A2["Node A"] --> S1["Super Node<br/>(B, C cycle)"]
S1 --> D2["Node D"]
end
Original -.->|"Abstract"| Abstracted
```
### 3.3 Recursive Cycle Execution Strategy
For cycle super nodes, the system employs a recursive execution strategy:
#### Step 1: Unique Initial Node Identification
Analyze the cycle boundary to identify the uniquely triggered entry node as the "initial node". This node must satisfy:
- Triggered by a predecessor node outside the cycle via a condition-satisfying edge
- Exactly one node meets this criterion
#### Step 2: Build Scoped Subgraph
Using all nodes in the current cycle as the scope, **logically remove all incoming edges to the initial node**. This operation breaks the outer cycle boundary, ensuring subsequent cycle detection only targets nested structures within.
#### Step 3: Nested Cycle Detection
Apply Tarjan's algorithm again to the subgraph to detect nested cycles within the scope. Since the initial node's incoming edges are removed, detected SCCs represent only true inner nested cycles.
#### Step 4: Inner Super Node Construction and Topological Sort
If nested cycles are detected:
- Abstract each inner cycle as a super node
- Build a super node dependency graph within the scope
- Perform topological sort on this super node graph
If no nested cycles are detected, perform direct DAG topological sort.
#### Step 5: Layered Execution
Execute according to the topological order:
- **Regular nodes**: Execute after checking trigger state; initial node executes unconditionally in the first iteration
- **Inner cycle super nodes**: **Recursively invoke Steps 1-6**, forming a nested execution structure
#### Step 6: Exit Condition Check
After completing each round of in-cycle execution, the system checks these exit conditions:
- **Exit edge triggered**: If any in-cycle node triggers an edge to an out-of-cycle node, exit the loop
- **Maximum iterations reached**: If the configured maximum (default 100) is reached, force termination
- **Time limit reached**: If a `loop_timer` node within the cycle reaches its configured time limit, exit the loop
- **Initial node not re-triggered**: If the initial node isn't re-triggered by in-cycle predecessors, the loop naturally terminates
If none of the conditions are met, return to Step 2 for the next iteration.
### 3.4 Cycle Execution Flowchart
```mermaid
flowchart TB
A["Cycle super node scheduled"] --> B["Identify uniquely triggered initial node"]
B --> C{"Valid initial node?"}
C -->|"None"| D["Skip this cycle"]
C -->|"Multiple"| E["Report configuration error"]
C -->|"Unique"| F["Build scoped subgraph<br/>Remove initial node's incoming edges"]
F --> G["Tarjan algorithm: detect nested cycles"]
G --> H{"Inner nested cycles exist?"}
H -->|"No"| I["DAG topological sort"]
H -->|"Yes"| J["Build inner super nodes<br/>Topological sort"]
I --> K["Layered execution"]
J --> K
K --> L["Execute regular nodes"]
K --> M["Recursively execute inner cycles"]
L --> N{"Check exit conditions"}
M --> N
N -->|"Exit edge triggered"| O["Exit cycle"]
N -->|"Max iterations reached"| O
N -->|"Initial node not re-triggered"| O
N -->|"Continue iteration"| F
```
## 4. Edge Conditions and Trigger Mechanism
### 4.1 Edge Trigger
Each edge has a `trigger` attribute that determines whether it participates in execution order calculation:
| trigger value | Behavior |
|---------------|----------|
| `true` (default) | Edge participates in topological sort; target node waits for source completion |
| `false` | Edge doesn't participate in topological sort; used only for data transfer |
### 4.2 Edge Condition
Edge conditions determine whether data flows along the edge:
- `true` (default): Always transfer
- `keyword`: Check if upstream output contains/excludes specific keywords
- `function`: Invoke custom function for evaluation
- Other custom condition types
The target node is triggered for execution only when the condition is satisfied.
## 5. Typical Cycle Scenario Examples
### 5.1 Human Review Loop
```yaml
nodes:
- id: Writer
type: agent
config:
name: gpt-4o
role: You are a professional technical writer
- id: Reviewer
type: human
config:
description: Please review the article, enter ACCEPT if satisfied
edges:
- from: Writer
to: Reviewer
- from: Reviewer
to: Writer
condition:
type: keyword
config:
none: [ACCEPT] # Continue loop when ACCEPT is not present
```
Execution flow:
1. Writer generates article
2. Reviewer performs human review
3. If input doesn't contain "ACCEPT", return to Writer for revision
4. If input contains "ACCEPT", exit the loop
### 5.2 Nested Loops
The system supports arbitrarily deep nested loops. For example, an outer "review-revise" loop can contain an inner "generate-validate" loop:
```
Outer Loop (Writer -> Reviewer -> Writer)
└── Inner Loop (Generator -> Validator -> Generator)
```
The recursive execution strategy automatically handles such nested structures.
## 6. Key Code Modules
| Module | Function |
|--------|----------|
| `workflow/cycle_manager.py` | Tarjan algorithm implementation, cycle info management |
| `workflow/topology_builder.py` | Super node graph construction, topological sorting |
| `workflow/executor/cycle_executor.py` | Recursive cycle executor |
| `workflow/graph.py` | Main graph execution entry point |
## 7. Changelog
- **2025-12-16**: Added graph execution logic documentation, detailing DAG and cyclic graph execution strategies.
+61
View File
@@ -0,0 +1,61 @@
# FIELD_SPECS Authoring Guide
This guide explains how to write `FIELD_SPECS` for new configs so the Web UI forms and `python -m tools.export_design_template` stay in sync. It applies to any `BaseConfig` subclass (nodes, Memory, Thinking, Tooling, etc.).
## 1. Why FIELD_SPECS matter
- UI forms rely on `FIELD_SPECS` to render inputs, defaults, and helper text.
- The design template exporter reads `FIELD_SPECS` to populate `yaml_template/design*.yaml` and the mirrored files under `frontend/public/`.
- Fields missing from `FIELD_SPECS` will not appear in the UI or generated templates.
## 2. Basic structure
`FIELD_SPECS` is a dict mapping field name → `ConfigFieldSpec`, usually declared inside the config class:
```python
FIELD_SPECS = {
"interpreter": ConfigFieldSpec(
name="interpreter",
display_name="Interpreter",
type_hint="str",
required=False,
default="python3",
description="Path to the Python executable",
),
...
}
```
Key attributes:
- `name`: same as the YAML field.
- `display_name`: optional human label shown in the UI; falls back to `name` when omitted.
- `type_hint`: string describing the shape (`str`, `list[str]`, `dict[str, Any]`, etc.).
- `required`: whether the UI treats the field as mandatory; usually `False` if a default exists.
- `default`: scalar or JSON-serializable default value.
- `description`: helper text/tooltips in forms and docs.
- `enum`: array of allowed values (strings).
- `enumOptions`: richer metadata per enum entry (label, description). Use it alongside `enum` for user-friendly dropdowns.
- `child`: reference to another `BaseConfig` subclass for nested structures.
## 3. Authoring flow
1. **Validate in `from_dict`** ensure YAML parsing enforces types and emits clear `ConfigError`s (see `entity/configs/python_runner.py`).
2. **Define `FIELD_SPECS`** cover all public fields with type, description, default, etc.
3. **Handle dynamic fields** when options depend on registries or filesystem scans, override `field_specs()` and use `replace()` to inject real-time `enum`/`description` (e.g., `FunctionToolEntryConfig.field_specs()` lists functions on disk).
4. **Export templates** after edits run:
```bash
python -m tools.export_design_template --output yaml_template/design.yaml --mirror frontend/public/design.yaml
```
The command uses the latest `FIELD_SPECS` to regenerate YAML templates and the frontend mirror—no manual edits needed.
## 4. Common patterns
- **Scalar fields**: see `entity/configs/python_runner.py` for `timeout_seconds` (integer default + validation).
- **Nested lists**: `entity/configs/memory.py` uses `child=FileSourceConfig` for `file_sources`, enabling repeatable subforms.
- **Dynamic enums**: `Node.field_specs()` (around `entity/configs/node.py:304`) pulls node types from the registry and supplies `enumOptions`; `FunctionToolEntryConfig.field_specs()` builds enumerations from the function catalog.
- **Registry-driven descriptions**: when calling `register_node_type` / `register_memory_store` / `register_thinking_mode` / `register_tooling_type`, always provide `summary`/`description`. Those strings populate `enumOptions` and keep dropdowns self-explanatory.
- **Optional blocks**: combine `required=False` with sensible `default` values and make sure `from_dict` honors the same defaults.
## 5. Best practices
- Keep descriptions user-friendly and clarify units (e.g., “Timeout (seconds)”).
- Align defaults with parsing logic so UI expectations match backend behavior.
- For nested configs, provide concise examples or cross-links so UI users understand the structure.
- After changing `FIELD_SPECS`, re-run the export command and commit the updated templates/mirror files.
For more examples inspect `entity/configs/model.py`, `entity/configs/tooling.py`, or other existing node/Memory/Thinking configs.
+45
View File
@@ -0,0 +1,45 @@
# DevAll Backend User Guide (English)
This landing page helps operators, workflow authors, and extension developers find the right documentation for DevAll backend components. Use the table below as your navigation map; drill into the linked chapters for full procedures and examples.
## 1. Documentation Map
| Topic | Highlights |
| --- | --- |
| [Web UI Quick Start](web_ui_guide.md) | Frontend interface operations, workflow execution, human review, troubleshooting. |
| [Workflow Authoring](workflow_authoring.md) | YAML structure, node types, provider/edge conditions, template export, CLI execution. |
| [Graph Execution Logic](execution_logic.md) | DAG/cyclic graph execution, Tarjan cycle detection, super node construction, recursive cycle execution. |
| [Dynamic Parallel Execution](dynamic_execution.md) | Map/Tree modes, split strategies, parallel processing and hierarchical reduction. |
| [Memory Module](modules/memory.md) | Memory list architecture, built-in `simple`/`file`/`blackboard` behaviors, embedding config, troubleshooting. |
| [Thinking Module](modules/thinking.md) | Reasoning enhancement, self-reflection mode, extending custom thinking modes. |
| [Tooling Module](modules/tooling/README.md) | Function/MCP modes, context injection, built-in function catalog, MCP launch patterns. |
| [Node Types Reference](nodes/) | Agent, Python, Human, Subgraph, Passthrough, Literal, Loop Counter node configurations. |
| [Attachment & Artifact APIs](attachments.md) | Upload/list/download endpoints, manifest schema, cleanup strategies, security constraints. |
| [`FIELD_SPECS` Standard](field_specs.md) | Field metadata contract that powers UI forms and template export—required reading before customizing modules. |
| [Config Schema API Contract](config_schema_contract.md) | `/api/config/schema(*)` request examples and breadcrumbs protocol (mostly for frontend/IDE integrations). |
## 2. Product Overview (Backend Focus)
- **Workflow orchestration engine**: Parses YAML DAGs, coordinates `model`, `python`, `tooling`, and `human` nodes inside a shared context, and writes node outputs into `WareHouse/<session>/`.
- **Provider abstraction**: The `runtime/node/agent/providers/` layer encapsulates OpenAI, Gemini, and other APIs so each node can swap models, base URLs, credentials, plus optional `thinking` and `memories` settings.
- **Real-time observability**: FastAPI + WebSocket streams node states, stdout/stderr, and artifact events to the Web UI, while structured logs land in `logs/` for centralized collection.
- **Run asset management**: Every execution creates an isolated session; attachments, Python workspace, context snapshots, and summary outputs are downloadable for later review.
## 3. Architecture & Execution Flow
1. **Entry**: Web UI and CLI call the FastAPI server exposed via `server_main.py` (e.g., `/api/workflow/execute`).
2. **Validation/queueing**: `WorkflowRunService` validates YAML, creates a session, prepares `code_workspace/attachments/`, then hands the DAG to the scheduler in `workflow/`.
3. **Execution**: Node executors resolve dependencies, propagate context, call tools, and retrieve memories; `MemoryManager`, `ToolingConfig`, and `ThinkingManager` trigger inside agent nodes as needed.
4. **Observability**: WebSocket pushes states, logs, and artifact events; JSON logs stay in `logs/`, and `WareHouse/` stores run assets.
5. **Cleanup & download**: After completion you can bundle the session for download or fetch files individually via the attachment APIs; retention policies are deployment-specific.
## 4. Role-based Navigation
- **Solutions/Prompt Engineers**: Begin with [Workflow Authoring](workflow_authoring.md); read the Memory and Tooling module docs when you need context memories or custom tools.
- **Extension Developers**: Combine [`FIELD_SPECS`](field_specs.md) with the [Tooling module guide](modules/tooling/README.md) to register new components; reference [config_schema_contract.md](config_schema_contract.md) if you need to debug schema-driven UI.
## 5. Glossary
- **Session**: Unique ID (timestamp + name) for a single run, used across the Web UI, backend, and `WareHouse/`.
- **code_workspace**: Shared directory for Python nodes at `WareHouse/<session>/code_workspace/`, synchronized with relevant attachments.
- **Attachment**: Files uploaded by users or generated during runs; list/download via REST or WebSocket APIs.
- **Memory Store / Attachment**: Stores define persistence backends; memory attachments describe how agent nodes read/write those stores across phases.
- **Tooling**: Execution environment bound to agent nodes (Function or MCP implementations).
If you spot gaps or outdated instructions, open an issue/PR or edit the docs directly (remember to keep Chinese and English versions in sync).
+136
View File
@@ -0,0 +1,136 @@
# Memory Module Guide
This document explains DevAll's memory system: memory list config, built-in store implementations, how agent nodes attach memories, and troubleshooting tips. Core code lives in `entity/configs/memory.py` and `node/agent/memory/*.py`.
## 1. Architecture
1. **Memory Store** Declared under `memory[]` in YAML with `name`, `type`, and `config`. Types are registered via `register_memory_store()` and point to concrete implementations.
2. **Memory Attachment** Referenced inside agent nodes via `AgentConfig.memories`. Each `MemoryAttachmentConfig` defines read/write strategy and retrieval stages.
3. **MemoryManager** Builds store instances at runtime based on attachments and orchestrates `load()`, `retrieve()`, `update()`, `save()`.
4. **Embedding** `SimpleMemoryConfig` and `FileMemoryConfig` embed `EmbeddingConfig`, and `EmbeddingFactory` instantiates OpenAI or local vector models.
## 2. Memory Sample Config
```yaml
memory:
- name: convo_cache
type: simple
config:
memory_path: WareHouse/shared/simple.json
embedding:
provider: openai
model: text-embedding-3-small
api_key: ${API_KEY}
- name: project_docs
type: file
config:
index_path: WareHouse/index/project_docs.json
file_sources:
- path: docs/
file_types: [".md", ".mdx"]
recursive: true
embedding:
provider: openai
model: text-embedding-3-small
```
### Mem0 Memory Config
```yaml
memory:
- name: agent_memory
type: mem0
config:
api_key: ${MEM0_API_KEY}
agent_id: my-agent
```
## 3. Built-in Store Comparison
| Type | Path | Highlights | Best for |
| --- | --- | --- | --- |
| `simple` | `node/agent/memory/simple_memory.py` | Optional disk persistence (JSON) after runs; FAISS + semantic rerank; read/write capable. | Small conversation history, prototypes. |
| `file` | `node/agent/memory/file_memory.py` | Chunks files/dirs into a vector index, read-only, auto rebuilds when files change. | Knowledge bases, doc QA. |
| `blackboard` | `node/agent/memory/blackboard_memory.py` | Lightweight append-only log trimmed by time/count; no vector search. | Broadcast boards, pipeline debugging. |
| `mem0` | `node/agent/memory/mem0_memory.py` | Cloud-managed by Mem0; semantic search + graph relationships; no local embeddings or persistence needed. Requires `mem0ai` package. | Production memory, cross-session persistence, multi-agent memory sharing. |
All stores register through `register_memory_store()` so summaries show up in UI via `MemoryStoreConfig.field_specs()`.
## 4. MemoryAttachmentConfig Fields
| Field | Description |
| --- | --- |
| `name` | Target Memory Store name (must be unique inside `stores[]`). |
| `retrieve_stage` | Optional list limiting retrieval to certain `AgentExecFlowStage` values (`pre`, `plan`, `gen`, `critique`, etc.). Empty means all stages. |
| `top_k` | Number of items per retrieval (default 3). |
| `similarity_threshold` | Minimum similarity cutoff (`-1` disables filtering). |
| `read` / `write` | Whether this node can read from / write back to the store. |
Agent node example:
```yaml
nodes:
- id: answer
type: agent
config:
provider: openai
model: gpt-4o-mini
prompt_template: answer_user
memories:
- name: convo_cache
retrieve_stage: ["gen"]
top_k: 5
read: true
write: true
- name: project_docs
read: true
write: false
```
Execution order:
1. When the node enters `gen`, `MemoryManager` iterates attachments.
2. Attachments matching the stage and `read=true` call `retrieve()` on their store.
3. Retrieved items are formatted under a "===== Related Memories =====" block in the agent context.
4. After completion, attachments with `write=true` call `update()` and optionally `save()`.
## 5. Store Details
All memory stores persist a unified `MemoryItem` structure containing:
- `content_summary` trimmed text used for embedding search.
- `input_snapshot` / `output_snapshot` serialized message blocks (with base64 attachments) preserving multimodal context.
- `metadata` store-specific telemetry (role, previews, attachment IDs, etc.).
This schema lets multimodal outputs flow into Memory/Thinking modules without extra plumbing.
### 5.1 SimpleMemory
- **Path** `SimpleMemoryConfig.memory_path` (or `auto`). Defaults to in-memory.
- **Retrieval** Build a query from the prompt, trim it, embed, query FAISS `IndexFlatIP`, then apply semantic rerank (Jaccard/LCS).
- **Write** `update()` builds a `MemoryContentSnapshot` (text + blocks) for both input/output, deduplicates via hashed summary, embeds the summary, and stores the snapshots/attachments metadata.
- **Tips** Tune `max_content_length`, `top_k`, and `similarity_threshold` to avoid irrelevant context.
### 5.2 FileMemory
- **Config** Requires at least one `file_sources` entry (paths, suffix filters, recursion, encoding). `index_path` is mandatory for incremental updates.
- **Indexing** Scan files → chunk (default 500 chars, 50 overlap) → embed → persist JSON with `file_metadata`.
- **Retrieval** Uses FAISS cosine similarity. Read-only; `update()` unsupported.
- **Maintenance** `load()` checks file hashes and rebuilds if needed. Store `index_path` on persistent storage.
### 5.3 BlackboardMemory
- **Config** `memory_path` (or `auto`) plus `max_items`. Creates the file in the session directory if missing.
- **Retrieval** Returns the latest `top_k` entries ordered by time.
- **Write** `update()` appends the latest snapshot (input/output blocks, attachments, previews). No embeddings are generated, so retrieval is purely recency-based.
### 5.4 Mem0Memory
- **Config** Requires `api_key` (from [app.mem0.ai](https://app.mem0.ai)). Optional `user_id`, `agent_id`, `org_id`, `project_id` for scoping.
- **Entity scoping**: `user_id` and `agent_id` are independent dimensions — both can be included simultaneously in `add()` and `search()` calls. When both are configured, retrieval uses an OR filter (`{"OR": [{"user_id": ...}, {"agent_id": ...}]}`) to search across both scopes. Writes include both IDs when available.
- **Retrieval** Uses Mem0's server-side semantic search. Supports `top_k` and `similarity_threshold` via `MemoryAttachmentConfig`.
- **Write** `update()` sends only user input to Mem0 via the SDK (as `role: "user"` messages). Assistant output is excluded to prevent noise memories from the LLM's responses being extracted as facts.
- **Persistence** Fully cloud-managed. `load()` and `save()` are no-ops. Memories persist across runs and sessions automatically.
- **Dependencies** Requires `mem0ai` package (`pip install mem0ai`).
## 6. EmbeddingConfig Notes
- Fields: `provider`, `model`, `api_key`, `base_url`, `params`.
- `provider=openai` uses the official client; override `base_url` for compatibility layers.
- `params` can include `use_chunking`, `chunk_strategy`, `max_length`, etc.
- `provider=local` expects `params.model_path` and depends on `sentence-transformers`.
## 7. Troubleshooting & Best Practices
- **Duplicate names** The memory list enforces unique `memory[]` names. Duplicates raise `ConfigError`.
- **Missing embeddings** `SimpleMemory` without embeddings downgrades to append-only; `FileMemory` errors out. Provide an embedding config whenever semantic search is required.
- **Permissions** Ensure directories for `memory_path`/`index_path` are writable. Mount volumes when running inside containers.
- **Performance** Pre-build large `FileMemory` indexes offline, use `retrieve_stage` to limit retrieval frequency, and tune `top_k`/`similarity_threshold` to balance recall vs. token cost.
## 8. Extending Memory
1. Implement a Config + Store (subclass `MemoryBase`).
2. Register via `register_memory_store("my_store", config_cls=..., factory=..., summary="...")` in `node/agent/memory/registry.py`.
3. Add `FIELD_SPECS`, then run `python -m tools.export_design_template ...` so the frontend picks up the enum.
4. Update this guide or ship a README detailing configuration knobs and boundaries.
+108
View File
@@ -0,0 +1,108 @@
# Thinking Module Guide
The Thinking module provides reasoning enhancement capabilities for Agent nodes, enabling the model to perform additional inference before or after generating results. This document covers the Thinking module architecture, built-in modes, and configuration methods.
## 1. Architecture
1. **ThinkingConfig**: Declared in YAML under `nodes[].config.thinking`, containing `type` and `config` fields.
2. **ThinkingManagerBase**: Abstract base class defining thinking logic for two timing hooks: `_before_gen_think` and `_after_gen_think`.
3. **Registry**: New thinking modes are registered via `register_thinking_mode()`, and Schema API automatically displays available options.
## 2. Configuration Example
```yaml
nodes:
- id: Thoughtful Agent
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
thinking:
type: reflection
config:
reflection_prompt: |
Please carefully review your response, considering:
1. Is the logic sound?
2. Are there any factual errors?
3. Is the expression clear?
Then provide an improved response.
```
## 3. Built-in Thinking Modes
| Type | Description | Trigger Timing | Config Fields |
|------|-------------|----------------|---------------|
| `reflection` | Model reflects on and refines its output after generation | After generation (`after_gen`) | `reflection_prompt` |
### 3.1 Reflection Mode
Self-Reflection mode allows the model to reflect on and improve its initial output. The execution flow:
1. Agent node calls the model to generate initial response
2. ThinkingManager concatenates conversation history (system role, user input, model output) as reflection context
3. Calls the model again with `reflection_prompt` to generate reflection result
4. Reflection result replaces the original output as the final node output
#### Configuration
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `reflection_prompt` | string | Yes | Prompt guiding model reflection, specifying reflection dimensions and expected improvements |
#### Use Cases
- **Writing refinement**: Self-review and correct grammar, logic issues
- **Code review**: Automatic security and quality checks after code generation
- **Complex reasoning**: Verify and correct multi-step reasoning results
## 4. Execution Timing
ThinkingManager supports two execution timings:
| Timing | Property | Description |
|--------|----------|-------------|
| Before generation (`before_gen`) | `before_gen_think_enabled` | Execute thinking before model call for input preprocessing |
| After generation (`after_gen`) | `after_gen_think_enabled` | Execute thinking after model output for post-processing or refinement |
The built-in `reflection` mode only enables after-generation thinking. Extension developers can implement before-generation thinking as needed.
## 5. Interaction with Memory
The Thinking module can access Memory context:
- `ThinkingPayload.text`: Text content at current stage
- `ThinkingPayload.blocks`: Multimodal content blocks (images, attachments, etc.)
- `ThinkingPayload.metadata`: Additional metadata
Memory retrieval results are passed to thinking functions via the `memory` parameter, allowing reflection to reference historical memories.
## 6. Custom Thinking Mode Extension
1. **Create config class**: Inherit from `BaseConfig`, define required configuration fields
2. **Implement ThinkingManager**: Inherit from `ThinkingManagerBase`, implement `_before_gen_think` or `_after_gen_think`
3. **Register mode**:
```python
from runtime.node.agent.thinking.registry import register_thinking_mode
register_thinking_mode(
"my_thinking",
config_cls=MyThinkingConfig,
manager_cls=MyThinkingManager,
summary="Custom thinking mode description",
)
```
4. **Export template**: Run `python -m tools.export_design_template` to update frontend options
## 7. Best Practices
- **Control reflection rounds**: Current reflection is single-round; specify iteration requirements in `reflection_prompt` for multi-round
- **Concise prompts**: Lengthy `reflection_prompt` increases token consumption; focus on key improvement points
- **Combine with Memory**: Store important reflection results in Memory for downstream nodes
- **Monitor costs**: Reflection makes additional model calls; track token usage
## 8. Related Documentation
- [Agent Node Configuration](../nodes/agent.md)
- [Memory Module](memory.md)
- [Workflow Authoring Guide](../workflow_authoring.md)
+47
View File
@@ -0,0 +1,47 @@
# Tooling Module Overview
DevAll currently exposes two tool binding modes for agent nodes:
1. **Function Tooling** call in-repo Python functions from `functions/function_calling/`, with JSON Schema auto-generated from type hints.
2. **MCP Tooling** connect to external services that implement the Model Context Protocol, including FastMCP, Claude Desktop, or any MCP-compatible tool stack.
All tooling configs hang off `AgentConfig.tooling`:
```yaml
nodes:
- id: solve
type: agent
config:
provider: openai
model: gpt-4o-mini
prompt_template: solver
tooling:
type: function
config:
tools:
- name: describe_available_files
- name: load_file
auto_load: true
timeout: 20
```
## 1. Lifecycle
1. **Parse** `ToolingConfig` selects `FunctionToolConfig`, `McpRemoteConfig`, or `McpLocalConfig` based on `type`. Field definitions live in `entity/configs/tooling.py`.
2. **Runtime** When the LLM chooses a tool, the executor injects `_context` (attachment store, workspace paths, etc.) for Function tools or forwards the request through MCP.
3. **Completion** Tool outputs are appended to the agent message stream and, when relevant, registered as attachments (e.g., `load_file`).
## 2. Documentation Map
- [function.md](function.md) Function Tooling config, context injection, best practices.
- [function_catalog.md](function_catalog.md) Built-in function list with usage notes.
- [mcp.md](mcp.md) MCP Tooling config, auto-launch, FastMCP example, security guidance.
## 3. Quick Comparison
| Dimension | Function | MCP |
| --- | --- | --- |
| Deployment | In-process Python functions shipped with the backend. | Remote: call an HTTP MCP endpoint. Local: launch a process and talk over stdio. |
| Schemas | Derived from annotations + `ParamMeta`. | Provided by the MCP server's JSON Schema. |
| Context | `_context` provides attachments + workspace helpers automatically. | Depends on the MCP server implementation. |
| Typical use | File I/O, local scripts, internal APIs. | Third-party tool suites, browsers, database agents. |
## 4. Security Notes
- Function Tooling runs inside the backend process, so keep functions least-privileged and avoid executing arbitrary shell commands without validation.
- MCP Tooling now has explicit **remote (HTTP)** and **local (stdio)** modes. Remote only needs an existing server URL; Local launches your binary, so constrain the command/env vars and rely on `wait_for_log` + timeouts to detect readiness.
- Tools that mutate attachments or `code_workspace/` should respect the lifecycle described in the [Attachment guide](../../attachments.md) (Chinese for now) to avoid leaking artifacts.
+77
View File
@@ -0,0 +1,77 @@
# Function Tooling Configuration Guide
`FunctionToolConfig` lets agent nodes call Python functions defined in the repo. Implementation lives in `entity/configs/tooling.py`, `utils/function_catalog.py`, and `functions/function_calling/`.
## 1. Config Fields
| Field | Description |
| --- | --- |
| `tools` | List of `FunctionToolEntryConfig`. Each entry requires `name`. |
| `timeout` | Tool execution timeout (seconds). |
`FunctionToolEntryConfig` specifics:
- `name`: top-level function name in `functions/function_calling/`.
### Function picker (`module_name:function_name`) & `module_name:All`
- The dropdown displays each function as `module_name:function_name`, where `module_name` is the relative Python file under `functions/function_calling/` (without `.py`, nested folders joined by `/`). This preserves semantic grouping for large catalogs.
- Every module automatically prepends a `module_name:All` entry, and all `All` entries are sorted lexicographically ahead of concrete functions. Choosing it expands to all functions in that module during config parsing, preserving alphabetical order.
- `module_name:All` is strictly for bulk imports; overriding `description`/`parameters`/`auto_fill` alongside it raises a validation error. Customize individual functions after expansion if needed.
- Both modules and functions are sorted alphabetically, and YAML still stores the plain function names; `module_name:All` is merely an input shortcut.
## 2. Function Directory Requirements
- Path: `functions/function_calling/` (override with `MAC_FUNCTIONS_DIR`).
- Functions must live at module top level.
- Provide Python type hints; for enums/descriptions use `typing.Annotated[..., ParamMeta(...)]`.
- Parameters beginning with `_` or splats (`*args`/`**kwargs`) are hidden from the agent call.
- The docstrings first paragraph becomes the description (truncated to ~600 chars).
- `utils/function_catalog.py` builds JSON Schemas at startup for the frontend/CLI.
## 3. Context Injection
The executor passes `_context` into each function:
| Key | Value |
| --- | --- |
| `attachment_store` | `utils.attachments.AttachmentStore` for querying/registering attachments. |
| `python_workspace_root` | Session `code_workspace/` shared by Python nodes. |
| `graph_directory` | Session root directory for relative path helpers. |
| others | Environment-specific extras (session/node IDs, etc.). |
Functions can declare `_context: dict | None = None` and parse it (see `functions/function_calling/file.py`s `FileToolContext`).
## 4. Example: Read Text File
```python
from typing import Annotated
from utils.function_catalog import ParamMeta
def read_text_file(
path: Annotated[str, ParamMeta(description="workspace-relative path")],
*,
encoding: str = "utf-8",
_context: dict | None = None,
) -> str:
ctx = FileToolContext(_context)
target = ctx.resolve_under_workspace(path)
return target.read_text(encoding=encoding)
```
YAML usage:
```yaml
nodes:
- id: summarize
type: agent
config:
tooling:
type: function
config:
tools:
- name: describe_available_files
- name: read_text_file
```
## 5. Extension Flow
1. Add your function under `functions/function_calling/`.
2. Supply type hints + `ParamMeta`; set `auto_fill: false` with custom `parameters` if you need manual JSON Schema.
3. If the function needs extra packages, declare them in `pyproject.toml`/`requirements.txt`, or use the bundled `install_python_packages` sparingly.
4. Run `python -m tools.export_design_template ...` so the frontend picks up new enums.
## 6. Debugging
- If the frontend/CLI reports function `foo` not found, double-check the name and ensure it resides under `MAC_FUNCTIONS_DIR`.
- When `function_catalog` fails to load, `FunctionToolEntryConfig.field_specs()` includes the error—fix syntax or dependencies first.
- Tool timeouts bubble up to the agent; raise `timeout` or handle exceptions inside the function for friendlier responses.
+168
View File
@@ -0,0 +1,168 @@
# Built-in Function Tool Catalog
This document lists all preset tools in the `functions/function_calling/` directory for Agent nodes to use via Function Tooling.
## Quick Import
Reference tools in YAML as follows:
```yaml
tooling:
- type: function
config:
tools:
- name: file:All # Import entire module
- name: save_file # Import single function
- name: deep_research:All
```
---
## File Operations (file.py)
Tools for file and directory management within `code_workspace/`.
| Function | Description |
|----------|-------------|
| `describe_available_files` | List available files in attachment store and code_workspace |
| `list_directory` | List contents of a directory |
| `create_folder` | Create a folder (supports nested directories) |
| `delete_path` | Delete a file or directory |
| `load_file` | Load a file and register as attachment, supports multimodal (text/image/audio) |
| `save_file` | Save text content to a file |
| `read_text_file_snippet` | Read text snippet (offset + limit), suitable for large files |
| `read_file_segment` | Read file by line range, supports line number metadata |
| `apply_text_edits` | Apply multiple text edits while preserving newlines and encoding |
| `rename_path` | Rename a file or directory |
| `copy_path` | Copy a file or directory tree |
| `move_path` | Move a file or directory |
| `search_in_files` | Search for text or regex patterns in workspace files |
**Example YAML**: [ChatDev_v1.yaml](../../../../../yaml_instance/ChatDev_v1.yaml), [file_tool_use_case.yaml](../../../../../yaml_instance/file_tool_use_case.yaml)
---
## Python Environment Management (uv_related.py)
Manage Python environments and dependencies using uv.
| Function | Description |
|----------|-------------|
| `install_python_packages` | Install Python packages using `uv add` |
| `init_python_env` | Initialize Python environment (uv lock + venv) |
| `uv_run` | Execute uv run in workspace to run modules or scripts |
**Example YAML**: [ChatDev_v1.yaml](../../../../../yaml_instance/ChatDev_v1.yaml)
---
## Deep Research (deep_research.py)
Search result management and report generation tools for automated research workflows.
### Search Result Management
| Function | Description |
|----------|-------------|
| `search_save_result` | Save or update a search result (URL, title, abstract, details) |
| `search_load_all` | Load all saved search results |
| `search_load_by_url` | Load a specific search result by URL |
| `search_high_light_key` | Save highlighted keywords for a search result |
### Report Management
| Function | Description |
|----------|-------------|
| `report_read` | Read full report content |
| `report_read_chapter` | Read a specific chapter (supports multi-level paths like `Intro/Background`) |
| `report_outline` | Get report outline (header hierarchy) |
| `report_create_chapter` | Create a new chapter |
| `report_rewrite_chapter` | Rewrite chapter content |
| `report_continue_chapter` | Append content to an existing chapter |
| `report_reorder_chapters` | Reorder chapters |
| `report_del_chapter` | Delete a chapter |
| `report_export_pdf` | Export report to PDF |
**Example YAML**: [deep_research_v1.yaml](../../../../../yaml_instance/deep_research_v1.yaml)
---
## Web Tools (web.py)
Web search and webpage content retrieval.
| Function | Description |
|----------|-------------|
| `web_search` | Perform web search using Serper.dev, supports pagination and multiple languages |
| `read_webpage_content` | Read webpage content using Jina Reader, supports rate limiting |
**Environment Variables**:
- `SERPER_DEV_API_KEY`: Serper.dev API key
- `JINA_API_KEY`: Jina API key (optional, auto rate-limited to 20 RPM without key)
**Example YAML**: [deep_research_v1.yaml](../../../../../yaml_instance/deep_research_v1.yaml)
---
## Video Tools (video.py)
Manim animation rendering and video processing.
| Function | Description |
|----------|-------------|
| `render_manim` | Render Manim script, auto-detects scene class and outputs video |
| `concat_videos` | Concatenate multiple video files using FFmpeg |
**Example YAML**: [teach_video.yaml](../../../../../yaml_instance/teach_video.yaml), [teach_video.yaml](../../../../../yaml_instance/teach_video.yaml)
---
## Code Execution (code_executor.py)
| Function | Description |
|----------|-------------|
| `execute_code` | Execute Python code string, returns stdout and stderr |
> ⚠️ **Security Note**: This tool has elevated privileges and should only be used in trusted workflows.
---
## User Interaction (user.py)
| Function | Description |
|----------|-------------|
| `call_user` | Send instructions to the user and get a response, for scenarios requiring human input |
---
## Weather Query (weather.py)
Demo tools to illustrate Function Calling workflow.
| Function | Description |
|----------|-------------|
| `get_city_num` | Return city code (hardcoded example) |
| `get_weather` | Return weather info by city code (hardcoded example) |
---
## Adding Custom Tools
1. Create a Python file in `functions/function_calling/` directory
2. Define parameters using type annotations:
```python
from typing import Annotated
from utils.function_catalog import ParamMeta
def my_tool(
param1: Annotated[str, ParamMeta(description="Parameter description")],
*,
_context: dict | None = None, # Optional, auto-injected by system
) -> str:
"""Function description (shown to LLM)"""
return "result"
```
3. Restart the backend server
4. Reference in Agent node via `name: my_tool` or `name: my_module:All`
+92
View File
@@ -0,0 +1,92 @@
# MCP Tooling Guide
MCP tooling is split into two explicit modes: **Remote (HTTP)** and **Local (stdio)**. They map to `tooling.type: mcp_remote` and `tooling.type: mcp_local`. The legacy `type: mcp` schema is no longer supported.
## 1. Mode overview
| Mode | Tooling type | When to use | Key fields |
| --- | --- | --- | --- |
| Remote | `mcp_remote` | A hosted HTTP(S) MCP server (FastMCP, Claude Desktop Connector, custom gateways) | `server`, `headers`, `timeout` |
| Local | `mcp_local` | A local executable that speaks MCP over stdio (Blender MCP, CLI tools, etc.) | `command`, `args`, `cwd`, `env`, timeouts |
## 2. `McpRemoteConfig`
| Field | Description |
| --- | --- |
| `server` | Required. MCP HTTP(S) endpoint, e.g. `https://api.example.com/mcp`. |
| `headers` | Optional. Extra HTTP headers such as `Authorization`. |
| `timeout` | Optional per-request timeout (seconds). |
**YAML example**
```yaml
nodes:
- id: remote_mcp
type: agent
config:
tooling:
type: mcp_remote
config:
server: https://mcp.mycompany.com/mcp
headers:
Authorization: Bearer ${MY_MCP_TOKEN}
timeout: 15
```
DevAll connects to the URL for each list/call request and passes `headers`. If the server is unreachable, an error is raised immediately—there is no local fallback.
## 3. `McpLocalConfig` fields
`mcp_local` declares the process arguments directly under `config`:
- `command` / `args`: executable and arguments (e.g., `uvx blender-mcp`).
- `cwd`: optional working directory.
- `env` / `inherit_env`: environment overrides.
- `startup_timeout`: max seconds to wait for `wait_for_log`.
- `wait_for_log`: regex matched against stdout to mark readiness.
**YAML example**
```yaml
nodes:
- id: local_mcp
type: agent
config:
tooling:
type: mcp_local
config:
command: uvx
args:
- blender-mcp
cwd: ${REPO_ROOT}
wait_for_log: "MCP ready"
startup_timeout: 8
```
DevAll keeps the process alive and relays MCP frames over stdio.
## 4. FastMCP sample server
`mcp_example/mcp_server.py`:
```python
from fastmcp import FastMCP
import random
mcp = FastMCP("Company Simple MCP Server", debug=True)
@mcp.tool
def rand_num(a: int, b: int) -> int:
return random.randint(a, b)
if __name__ == "__main__":
mcp.run()
```
Launch:
```bash
uv run fastmcp run mcp_example/mcp_server.py --transport streamable-http --port 8010
```
- Remote mode: set `server` to `http://127.0.0.1:8010/mcp`.
- Local mode: run the script with `transport=stdio` and point `command` to that invocation.
## 5. Security & operations
- **Network exposure**: Remote mode should sit behind HTTPS + ACL/API keys. Local mode still has access to the host filesystem, so keep the script sandboxed.
- **Resource cleanup**: Local mode processes are terminated by DevAll; make sure they gracefully handle SIGTERM/SIGKILL.
- **Logs**: Emit a clear readiness line that matches `wait_for_log` to debug startup issues.
- **Auth**: Remote mode handles tokens via `headers`; Local mode can receive secrets via `env` (never commit them).
- **Multi-session**: If the MCP server is single-tenant, cap concurrency (e.g., `max_concurrency=1`) and share the same YAML config.
## 6. Debugging checklist
1. Remote: ping the HTTP endpoint via curl or `fastmcp client`. Local: run the binary manually and confirm the readiness log.
2. Start DevAll (optionally with `--reload`) and observe backend logs for tool discovery.
3. When calls fail, inspect the Web UI tool traces or the structured logs under `logs/`.
+186
View File
@@ -0,0 +1,186 @@
# Agent Node
The Agent node is the most fundamental node type in the DevAll platform, used to invoke Large Language Models (LLMs) for text generation, conversation, reasoning, and other tasks. It supports multiple model providers (OpenAI, Gemini, etc.) and can be configured with advanced features like tool calling, chain-of-thought, and memory.
## Configuration
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `provider` | string | Yes | `openai` | Model provider name, e.g., `openai`, `gemini` |
| `name` | string | Yes | - | Model name, e.g., `gpt-4o`, `gemini-2.0-flash-001` |
| `role` | text | No | - | System prompt |
| `base_url` | string | No | Provider default | API endpoint URL, supports `${VAR}` placeholders |
| `api_key` | string | No | - | API key, recommend using environment variable `${API_KEY}` |
| `params` | dict | No | `{}` | Model call parameters (temperature, top_p, etc.) |
| `tooling` | object | No | - | Tool calling configuration, see [Tooling Module](../modules/tooling/README.md) |
| `thinking` | object | No | - | Chain-of-thought configuration, e.g., chain-of-thought, reflection |
| `memories` | list | No | `[]` | Memory binding configuration, see [Memory Module](../modules/memory.md) |
| `skills` | object | No | - | Agent Skills discovery and built-in skill activation/file-read tools |
| `retry` | object | No | - | Automatic retry strategy configuration |
### Retry Strategy Configuration (retry)
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `enabled` | bool | `true` | Whether to enable automatic retry |
| `max_attempts` | int | `5` | Maximum number of attempts (including first attempt) |
| `min_wait_seconds` | float | `1.0` | Minimum backoff wait time |
| `max_wait_seconds` | float | `6.0` | Maximum backoff wait time |
| `retry_on_status_codes` | list[int] | `[408,409,425,429,500,502,503,504]` | HTTP status codes that trigger retry |
### Agent Skills Configuration (skills)
| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `enabled` | bool | `false` | Enable Agent Skills discovery for this node |
| `allow` | list[object] | `[]` | Optional allowlist of skills from the project-level `.agents/skills/` directory; each entry uses `name` |
### Agent Skills Notes
- Skills are discovered from the fixed project-level `.agents/skills/` directory.
- The runtime exposes two built-in skill tools: `activate_skill` and `read_skill_file`.
- `read_skill_file` only works after the relevant skill has been activated.
- Skill `SKILL.md` frontmatter may include optional `allowed-tools` using the Agent Skills spec format, for example `allowed-tools: execute_code`.
- If a selected skill requires tools that are not bound on the node, that skill is skipped at runtime.
- If no compatible skills remain, the agent is explicitly instructed not to claim skill usage.
## When to Use
- **Text generation**: Writing, translation, summarization, Q&A, etc.
- **Intelligent conversation**: Multi-turn dialogue, customer service bots
- **Tool calling**: Enable the model to call external APIs or execute functions
- **Complex reasoning**: Use with thinking configuration for deep thought
- **Knowledge retrieval**: Use with memories to implement RAG patterns
## Examples
### Basic Configuration
```yaml
nodes:
- id: Writer
type: agent
config:
provider: openai
base_url: ${BASE_URL}
api_key: ${API_KEY}
name: gpt-4o
role: |
You are a professional technical documentation writer. Please answer questions in clear and concise language.
params:
temperature: 0.7
max_tokens: 2000
```
### Configuring Tool Calling
```yaml
nodes:
- id: Assistant
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
tooling:
type: function # Tool type: function, mcp_remote, mcp_local
config:
tools: # List of function tools from functions/function_calling/ directory
- name: describe_available_files
- name: load_file
timeout: 20 # Optional: execution timeout (seconds)
```
### Configuring MCP Tools (Remote HTTP)
```yaml
nodes:
- id: MCP Agent
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
tooling:
type: mcp_remote
config:
server: http://localhost:8080/mcp # MCP server endpoint
headers: # Optional: custom request headers
Authorization: Bearer ${MCP_TOKEN}
timeout: 30 # Optional: request timeout (seconds)
```
### Configuring MCP Tools (Local stdio)
```yaml
nodes:
- id: Local MCP Agent
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
tooling:
type: mcp_local
config:
command: uvx # Launch command
args: ["mcp-server-sqlite", "--db-path", "data.db"]
cwd: ${WORKSPACE} # Optional, usually not needed
env: # Optional, usually not needed
DEBUG: "true"
startup_timeout: 10 # Optional: startup timeout (seconds)
```
### Gemini Multimodal Configuration
```yaml
nodes:
- id: Vision Agent
type: agent
config:
provider: gemini
base_url: https://generativelanguage.googleapis.com
api_key: ${GEMINI_API_KEY}
name: gemini-2.5-flash-image
role: You need to generate corresponding image content based on user input.
```
### Configuring Retry Strategy
```yaml
nodes:
- id: Robust Agent
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
retry: # Retry is enabled by default, you can customize it
enabled: true
max_attempts: 3
min_wait_seconds: 2.0
max_wait_seconds: 10.0
```
### Configuring Agent Skills
```yaml
nodes:
- id: Skilled Agent
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
skills:
enabled: true
allow:
- name: python-scratchpad
- name: rest-api-caller
```
## Related Documentation
- [Tooling Module Configuration](../modules/tooling/README.md)
- [Memory Module Configuration](../modules/memory.md)
- [Workflow Authoring Guide](../workflow_authoring.md)
+114
View File
@@ -0,0 +1,114 @@
# Human Node
The Human node is used to introduce human interaction during workflow execution, allowing users to view the current state and provide input through the Web UI. This node blocks workflow execution until the user submits a response.
## Configuration
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `description` | text | No | - | Task description displayed to the user, explaining the operation that requires human completion |
## Core Concepts
### Blocking Wait Mechanism
When the workflow reaches a Human node:
1. The workflow pauses, waiting for human input
2. The Web UI displays the current context and task description
3. The user enters a response in the interface
4. The workflow continues execution, passing the user input to downstream nodes
### Web UI Interaction
- Human nodes are presented as a conversation in the Launch interface
- Users can view previous execution history
- Supports attachment uploads (e.g., files, images)
## When to Use
- **Review and confirmation**: Have humans review LLM output before continuing
- **Modification suggestions**: Collect user suggestions for modifying generated content
- **Critical decisions**: Points where human judgment is needed to continue
- **Data supplementation**: When additional information from the user is needed
- **Quality control**: Introduce human quality checks at critical nodes
## Examples
### Basic Configuration
```yaml
nodes:
- id: Human Reviewer
type: human
config:
description: Please review the above content. If satisfied, enter ACCEPT; otherwise, enter your modification suggestions.
```
### Human-Machine Collaboration Loop
```yaml
nodes:
- id: Article Writer
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
role: You are a professional writer who writes articles based on user requirements.
- id: Human Reviewer
type: human
config:
description: |
Please review the article:
- If satisfied with the result, enter ACCEPT to end the process
- Otherwise, enter modification suggestions to continue iterating
edges:
- from: Article Writer
to: Human Reviewer
- from: Human Reviewer
to: Article Writer
condition:
type: keyword
config:
none: [ACCEPT]
case_sensitive: false
```
### Multi-Stage Review
```yaml
nodes:
- id: Draft Generator
type: agent
config:
provider: openai
name: gpt-4o
- id: Content Review
type: human
config:
description: Please review content accuracy. Enter APPROVED or modification suggestions.
- id: Final Reviewer
type: human
config:
description: Final confirmation. Enter PUBLISH to publish or REJECT to reject.
edges:
- from: Draft Generator
to: Content Review
- from: Content Review
to: Final Reviewer
condition:
type: keyword
config:
any: [APPROVED]
```
## Best Practices
- Clearly explain expected operations and keywords in the `description`
- Use conditional edges with keywords to implement flow control
- Consider adding timeout mechanisms to avoid infinite workflow waiting
+140
View File
@@ -0,0 +1,140 @@
# Literal Node
The Literal node is used to output fixed text content. When the node is triggered, it ignores all inputs and directly outputs a predefined message.
## Configuration
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `content` | text | Yes | - | Fixed text content to output, cannot be empty |
| `role` | string | No | `user` | Message role: `user` or `assistant` |
## Core Concepts
### Fixed Output
Characteristics of the Literal node:
- **Ignores input**: Regardless of what upstream content is passed in, it does not affect the output
- **Fixed content**: Outputs the same `content` every time it executes
- **Role marking**: Output message carries the specified role identifier
### Message Role
- `user`: Indicates this is a message sent by the user
- `assistant`: Indicates this is a message sent by the assistant (AI)
The role setting affects how downstream nodes process the message.
## When to Use
- **Fixed prompt injection**: Inject fixed instructions or context into the workflow
- **Testing and debugging**: Use fixed input to test downstream nodes
- **Default responses**: Return fixed messages under specific conditions
- **Process initialization**: Serve as the starting point of a workflow to provide initial content
## Examples
### Basic Usage
```yaml
nodes:
- id: Welcome Message
type: literal
config:
content: |
Welcome to the intelligent assistant! Please describe your needs.
role: assistant
```
### Injecting Fixed Context
```yaml
nodes:
- id: Context Injector
type: literal
config:
content: |
Please note the following rules:
1. Answers must be concise and clear
2. Reply in English
3. If uncertain, please state so
role: user
- id: Assistant
type: agent
config:
provider: openai
name: gpt-4o
edges:
- from: Context Injector
to: Assistant
```
### Fixed Responses in Conditional Branches
```yaml
nodes:
- id: Classifier
type: agent
config:
provider: openai
name: gpt-4o
role: Determine user intent, reply with KNOWN or UNKNOWN
- id: Known Response
type: literal
config:
content: I can help you complete this task.
role: assistant
- id: Unknown Response
type: literal
config:
content: Sorry, I cannot understand your request. Please describe it in a different way.
role: assistant
edges:
- from: Classifier
to: Known Response
condition:
type: keyword
config:
any: [KNOWN]
- from: Classifier
to: Unknown Response
condition:
type: keyword
config:
any: [UNKNOWN]
```
### Testing Purposes
```yaml
nodes:
- id: Test Input
type: literal
config:
content: |
This is a test text for verifying downstream processing logic.
Contains multiple lines.
role: user
- id: Processor
type: python
config:
timeout_seconds: 30
edges:
- from: Test Input
to: Processor
start: [Test Input]
```
## Notes
- The `content` field cannot be an empty string
- Use YAML multi-line string syntax `|` for writing long text
- Choose the correct `role` to ensure downstream nodes process the message correctly
+153
View File
@@ -0,0 +1,153 @@
# Loop Counter Node
The Loop Counter node is a loop control node used to limit the number of iterations of a loop in a workflow. Through a counting mechanism, it suppresses output before reaching the preset limit, and only releases the message to trigger outgoing edges when the limit is reached, thereby terminating the loop.
## Configuration
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `max_iterations` | int | Yes | `10` | Maximum number of loop iterations, must be ≥ 1 |
| `reset_on_emit` | bool | No | `true` | Whether to reset the counter after reaching the limit |
| `message` | text | No | - | Message content to send to downstream when limit is reached |
## Core Concepts
### How It Works
The Loop Counter node maintains an internal counter with the following behavior:
1. **Each time it is triggered**: Counter +1
2. **Counter < `max_iterations`**: **No output is produced**, outgoing edges are not triggered
3. **Counter = `max_iterations`**: Output message is produced, triggering outgoing edges
This "suppress-release" mechanism allows the Loop Counter to precisely control when a loop terminates.
### Topological Structure Requirements
The Loop Counter node has special placement requirements in the graph structure:
```
┌──────────────────────────────────────┐
▼ │
Agent ──► Human ─────► Loop Counter ──┬──┘
▲ │ │
└─────────┘ ▼
End Node (outside loop)
```
> **Important**: Since Loop Counter **produces no output until the limit is reached**:
> - **Human must connect to both Agent and Loop Counter**: This way the "continue loop" edge is handled by Human → Agent, while Loop Counter only handles counting
> - **Loop Counter must connect to Agent (inside loop)**: So it's recognized as an in-loop node, avoiding premature loop termination
> - **Loop Counter must connect to End Node (outside loop)**: When the limit is reached, trigger the out-of-loop node to terminate the entire loop execution
### Counter State
- Counter state persists throughout the entire workflow execution
- When `reset_on_emit: true`, the counter resets to 0 after reaching the limit
- When `reset_on_emit: false`, the counter continues accumulating after reaching the limit, outputting on every subsequent trigger
## When to Use
- **Preventing infinite loops**: Set a safety limit for human-machine interaction loops
- **Iteration control**: Limit the maximum number of self-improvement iterations for an Agent
- **Timeout protection**: Serve as a "circuit breaker" for process execution
## Examples
### Basic Usage
```yaml
nodes:
- id: Iteration Guard
type: loop_counter
config:
max_iterations: 5
reset_on_emit: true
message: Maximum iteration count reached, process terminated.
```
### Human-Machine Interaction Loop Protection
This is the most typical use case for Loop Counter:
```yaml
graph:
id: review_loop
description: Review loop with iteration limit
nodes:
- id: Writer
type: agent
config:
provider: openai
name: gpt-4o
role: Improve articles based on user feedback
- id: Reviewer
type: human
config:
description: |
Review the article, enter ACCEPT to accept or provide modification suggestions.
- id: Loop Guard
type: loop_counter
config:
max_iterations: 3
message: Maximum modification count (3 times) reached, process automatically ended.
- id: Final Output
type: passthrough
config: {}
edges:
# Main loop: Writer -> Reviewer
- from: Writer
to: Reviewer
# Condition 1: User enters ACCEPT -> End
- from: Reviewer
to: Final Output
condition:
type: keyword
config:
any: [ACCEPT]
# Condition 2: User enters modification suggestions -> Trigger both Writer to continue loop AND Loop Guard to count
- from: Reviewer
to: Writer
condition:
type: keyword
config:
none: [ACCEPT]
- from: Reviewer
to: Loop Guard
condition:
type: keyword
config:
none: [ACCEPT]
# Loop Guard connects to Writer (keeps it inside the loop)
- from: Loop Guard
to: Writer
# When Loop Guard reaches limit: Trigger Final Output to end the process
- from: Loop Guard
to: Final Output
start: [Writer]
end: [Final Output]
```
**Execution Flow Explanation**:
1. User first enters modification suggestions → Triggers both Writer (continue loop) and Loop Guard (count 1, no output)
2. User enters modification suggestions again → Triggers both Writer (continue loop) and Loop Guard (count 2, no output)
3. User enters modification suggestions for the third time → Writer continues execution, Loop Guard count 3 reaches limit, outputs message triggering Final Output, terminating the loop
4. Or at any time user enters ACCEPT → Goes directly to Final Output to end
## Notes
- `max_iterations` must be a positive integer (≥ 1)
- Loop Counter **produces no output until the limit is reached**, outgoing edges will not trigger
- Ensure Loop Counter connects to both in-loop and out-of-loop nodes
- The `message` field is optional, default message is `"Loop limit reached (N)"`
+159
View File
@@ -0,0 +1,159 @@
# Loop Timer Node
The Loop Timer node is a loop control node used to limit the duration of a loop in a workflow. Through a time-tracking mechanism, it suppresses output before reaching the preset time limit, and only releases the message to trigger outgoing edges when the time limit is reached, thereby terminating the loop.
## Configuration
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `max_duration` | float | Yes | `60.0` | Maximum loop duration, must be > 0 |
| `duration_unit` | string | Yes | `"seconds"` | Time unit: "seconds", "minutes", or "hours" |
| `reset_on_emit` | bool | No | `true` | Whether to reset the timer after reaching the limit |
| `message` | text | No | - | Message content to send to downstream when time limit is reached |
## Core Concepts
### How It Works
The Loop Timer node maintains an internal timer with the following behavior:
1. **On first trigger**: Timer starts tracking elapsed time
2. **Elapsed time < `max_duration`**: **No output is produced**, outgoing edges are not triggered
3. **Elapsed time >= `max_duration`**: Output message is produced, triggering outgoing edges
This "suppress-release" mechanism allows the Loop Timer to precisely control when a loop terminates based on time rather than iteration count.
### Topological Structure Requirements
The Loop Timer node has special placement requirements in the graph structure:
```
┌──────────────────────────────────────┐
▼ │
Agent ──► Human ─────► Loop Timer ──┬──┘
▲ │ │
└─────────┘ ▼
End Node (outside loop)
```
> **Important**: Since Loop Timer **produces no output until the time limit is reached**:
> - **Human must connect to both Agent and Loop Timer**: This way the "continue loop" edge is handled by Human → Agent, while Loop Timer only handles time tracking
> - **Loop Timer must connect to Agent (inside loop)**: So it's recognized as an in-loop node, avoiding premature loop termination
> - **Loop Timer must connect to End Node (outside loop)**: When the time limit is reached, trigger the out-of-loop node to terminate the entire loop execution
### Timer State
- Timer state persists throughout the entire workflow execution
- Timer starts on the first trigger to the Loop Timer node
- When `reset_on_emit: true`, the timer resets after reaching the limit
- When `reset_on_emit: false`, the timer continues running after reaching the limit, outputting on every subsequent trigger
## When to Use
- **Time-based constraints**: Enforce time limits for loops (e.g., "review must complete within 5 minutes")
- **Timeout protection**: Serve as a "circuit breaker" to prevent runaway processes
- **Variable iteration time**: When each loop iteration takes unpredictable time, but total duration must be bounded
## Examples
### Basic Usage
```yaml
nodes:
- id: Time Guard
type: loop_timer
config:
max_duration: 5
duration_unit: minutes
reset_on_emit: true
message: Time limit reached (5 minutes), process terminated.
```
### Time-Limited Review Loop
This is the most typical use case for Loop Timer:
```yaml
graph:
id: timed_review_loop
description: Review loop with 5-minute time limit
nodes:
- id: Writer
type: agent
config:
provider: openai
name: gpt-4o
role: Improve articles based on user feedback
- id: Reviewer
type: human
config:
description: |
Review the article, enter ACCEPT to accept or provide modification suggestions.
- id: Loop Gate
type: loop_timer
config:
max_duration: 5
duration_unit: minutes
message: Time limit (5 minutes) reached, process automatically ended.
- id: Final Output
type: passthrough
config: {}
edges:
# Main loop: Writer -> Reviewer
- from: Writer
to: Reviewer
# Condition 1: User enters ACCEPT -> End
- from: Reviewer
to: Final Output
condition:
type: keyword
config:
any: [ACCEPT]
# Condition 2: User enters modification suggestions -> Trigger both Writer to continue loop AND Loop Gate to track time
- from: Reviewer
to: Writer
condition:
type: keyword
config:
none: [ACCEPT]
- from: Reviewer
to: Loop Gate
condition:
type: keyword
config:
none: [ACCEPT]
# Loop Gate connects to Writer (keeps it inside the loop)
- from: Loop Gate
to: Writer
# When Loop Gate reaches time limit: Trigger Final Output to end the process
- from: Loop Gate
to: Final Output
start: [Writer]
end: [Final Output]
```
**Execution Flow Explanation**:
1. User first enters modification suggestions → Triggers both Writer (continue loop) and Loop Gate (track time, no output)
2. User enters modification suggestions again → Triggers both Writer (continue loop) and Loop Gate (track time, no output)
3. After 5 minutes of elapsed time → Loop Gate outputs message triggering Final Output, terminating the loop
4. Or at any time user enters ACCEPT → Goes directly to Final Output to end
## Notes
- `max_duration` must be a positive number (> 0)
- `duration_unit` must be one of: "seconds", "minutes", "hours"
- Loop Timer **produces no output until the time limit is reached**, outgoing edges will not trigger
- Ensure Loop Timer connects to both in-loop and out-of-loop nodes
- The `message` field is optional, default message is `"Time limit reached (N units)"`
- Timer starts on the first trigger to the Loop Timer node
+206
View File
@@ -0,0 +1,206 @@
# Passthrough Node
The Passthrough node is the simplest node type. It performs no operations and passes received messages to downstream nodes. By default, it only passes the **last message**. It is primarily used for graph structure "wire management" optimization and context control.
## Configuration
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `only_last_message` | bool | No | `true` | Whether to pass only the last message. Set to `false` to pass all messages. |
### Basic Configuration
```yaml
config: {} # Uses default configuration, only passes the last message
```
### Pass All Messages
```yaml
config:
only_last_message: false # Pass all received messages
```
## Core Concepts
### Pass-through Behavior
- Receives all messages passed from upstream
- **By default, only passes the last message** (`only_last_message: true`)
- When `only_last_message: false`, passes all messages
- Does not perform any content processing or transformation
### Graph Structure Optimization
The core value of the Passthrough node is not in data processing, but in **graph structure optimization** ("wire management"):
- Makes complex edge connections clearer
- Centrally manages outgoing edge configurations (such as `keep_message`)
- Serves as a logical dividing point, improving workflow readability
## Key Uses
### 1. As an Entry Node to Preserve Initial Context
Using Passthrough as the workflow entry node, combined with the `keep_message: true` edge configuration, ensures that the user's initial task is always preserved in the context and won't be overwritten by subsequent node outputs:
```yaml
nodes:
- id: Task Keeper
type: passthrough
config: {}
- id: Worker A
type: agent
config:
provider: openai
name: gpt-4o
- id: Worker B
type: agent
config:
provider: openai
name: gpt-4o
edges:
# Distribute tasks from entry, preserving original message
- from: Task Keeper
to: Worker A
keep_message: true # Preserve initial task context
- from: Task Keeper
to: Worker B
keep_message: true
start: [Task Keeper]
```
**Effect**: Both Worker A and Worker B can see the user's original input, not just the output from the previous node.
### 2. Filtering Redundant Output in Loops
In workflows containing loops, nodes within the loop may produce a large amount of intermediate output. Passing all outputs to subsequent nodes would cause context bloat. Using a Passthrough node allows you to **pass only the final result of the loop**:
```yaml
nodes:
- id: Iterative Improver
type: agent
config:
provider: openai
name: gpt-4o
role: Continuously improve output based on feedback
- id: Evaluator
type: agent
config:
provider: openai
name: gpt-4o
role: |
Evaluate output quality, reply GOOD or provide improvement suggestions
- id: Result Filter
type: passthrough
config: {}
- id: Final Processor
type: agent
config:
provider: openai
name: gpt-4o
role: Post-process the final result
edges:
- from: Iterative Improver
to: Evaluator
# Loop: Return to improvement node when evaluation fails
- from: Evaluator
to: Iterative Improver
condition:
type: keyword
config:
none: [GOOD]
# Loop ends: Filter through Passthrough, only pass the last message
- from: Evaluator
to: Result Filter
condition:
type: keyword
config:
any: [GOOD]
- from: Result Filter
to: Final Processor
start: [Iterative Improver]
end: [Final Processor]
```
**Effect**: Regardless of how many loop iterations occur, `Final Processor` will only receive the last output from `Evaluator` (the one indicating quality passed), not all intermediate results.
## Other Uses
- **Placeholder**: Reserve node positions during design phase
- **Conditional branching**: Implement routing logic with conditional edges
- **Debugging observation point**: Insert nodes for easy observation in the workflow
## Examples
### Basic Usage
```yaml
nodes:
- id: Router
type: passthrough
config: {}
```
### Conditional Routing
```yaml
nodes:
- id: Classifier
type: agent
config:
provider: openai
name: gpt-4o
role: |
Classify input content, reply TECHNICAL or BUSINESS
- id: Router
type: passthrough
config: {}
- id: Tech Handler
type: agent
config:
provider: openai
name: gpt-4o
- id: Biz Handler
type: agent
config:
provider: openai
name: gpt-4o
edges:
- from: Classifier
to: Router
- from: Router
to: Tech Handler
condition:
type: keyword
config:
any: [TECHNICAL]
- from: Router
to: Biz Handler
condition:
type: keyword
config:
any: [BUSINESS]
```
## Best Practices
- Use meaningful node IDs that describe their topological role (e.g., `Task Keeper`, `Result Filter`)
- When used as an entry node, configure outgoing edges with `keep_message: true` to preserve context
- Use after loops to filter out redundant intermediate output
+89
View File
@@ -0,0 +1,89 @@
# Python Node
The Python node is used to execute Python scripts or inline code within workflows, implementing custom data processing, API calls, file operations, and other logic. Scripts are executed in the shared `code_workspace/` directory and can access workflow context data.
## Configuration
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `interpreter` | string | No | Current Python | Python interpreter path |
| `args` | list[str] | No | `[]` | Startup arguments appended to the interpreter |
| `env` | dict[str, str] | No | `{}` | Additional environment variables, overrides system defaults |
| `timeout_seconds` | int | No | `60` | Script execution timeout (seconds) |
| `encoding` | string | No | `utf-8` | Encoding for parsing stdout/stderr |
## Core Concepts
### Code Workspace
Python scripts are executed within the `code_workspace/` directory:
- Scripts can read and write files in this directory
- Multiple Python nodes share the same workspace
- The workspace persists for the duration of a single workflow execution
### Input/Output
- **Input**: Outputs from upstream nodes are passed as environment variables or standard input
- **Output**: The script's stdout output will be passed as a Message to downstream nodes
## When to Use
- **Data processing**: Parse JSON/XML, data transformation, formatting
- **API calls**: Call third-party services, fetch external data
- **File operations**: Read/write files, generate reports
- **Complex calculations**: Mathematical operations, algorithm implementations
- **Glue logic**: Custom logic connecting different nodes
## Examples
### Basic Configuration
```yaml
nodes:
- id: Data Processor
type: python
config:
timeout_seconds: 120
env:
key: value
```
### Specifying Interpreter and Arguments
```yaml
nodes:
- id: Script Runner
type: python
config:
interpreter: /usr/bin/python3.11
timeout_seconds: 300
encoding: utf-8
```
### Typical Workflow Example
```yaml
nodes:
- id: LLM Generator
type: agent
config:
provider: openai
name: gpt-4o
api_key: ${API_KEY}
role: You need to generate executable Python code based on user input. The code should be wrapped in ```python ```.
- id: Result Parser
type: python
config:
timeout_seconds: 30
edges:
- from: LLM Generator
to: Result Parser
```
## Notes
- Ensure script files are placed in the `code_workspace/` directory
- Long-running scripts should have an appropriately increased `timeout_seconds`
- Use `env` to pass additional environment variables, accessible in scripts via `os.getenv`
+138
View File
@@ -0,0 +1,138 @@
# Subgraph Node
The Subgraph node allows embedding another workflow graph into the current workflow, enabling process reuse and modular design. Subgraphs can come from external YAML files or be defined inline in the configuration.
## Configuration
| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `type` | string | Yes | - | Subgraph source type: `file` or `config` |
| `config` | object | Yes | - | Contains different configurations depending on `type` |
### File Type Configuration
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `path` | string | Yes | Subgraph file path (relative to `yaml_instance/` or absolute path) |
### Config Type Configuration
Inline definition of the complete subgraph structure, containing the same fields as the top-level `graph`:
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `id` | string | Yes | Subgraph identifier |
| `description` | string | No | Subgraph description |
| `log_level` | string | No | Log level (DEBUG/INFO) |
| `nodes` | list | Yes | Node list |
| `edges` | list | No | Edge list |
| `start` | list | No | Entry node list |
| `end` | list | No | Exit node list |
| `memory` | list | No | Memory definitions specific to the subgraph |
## Core Concepts
### Modular Reuse
Extract commonly used process fragments into independent YAML files that can be reused by multiple workflows:
- Example: Package an "article polishing" process as a subgraph
- Different main workflows can call this subgraph
### Variable Inheritance
Subgraphs inherit `vars` variable definitions from the parent graph, supporting cross-level variable passing.
### Execution Isolation
Subgraphs execute as independent units with their own:
- Node namespace
- Log level configuration
- Memory definitions (optional)
## When to Use
- **Process reuse**: Multiple workflows sharing the same sub-processes
- **Modular design**: Breaking complex processes into manageable smaller units
- **Team collaboration**: Different teams maintaining different subgraph modules
## Examples
### Referencing External File
```yaml
nodes:
- id: Review Process
type: subgraph
config:
type: file
config:
path: common/review_flow.yaml
```
### Inline Subgraph Definition
```yaml
nodes:
- id: Translation Unit
type: subgraph
config:
type: config
config:
id: translation_subgraph
description: Multi-language translation subprocess
nodes:
- id: Translator
type: agent
config:
provider: openai
name: gpt-4o
role: You are a professional translator who translates content to the target language.
- id: Proofreader
type: agent
config:
provider: openai
name: gpt-4o
role: You are a proofreading expert who checks and polishes translated content.
edges:
- from: Translator
to: Proofreader
start: [Translator]
end: [Proofreader]
```
### Combining Multiple Subgraphs
```yaml
nodes:
- id: Input Handler
type: agent
config:
provider: openai
name: gpt-4o
- id: Analysis Module
type: subgraph
config:
type: file
config:
path: modules/analysis.yaml
- id: Report Module
type: subgraph
config:
type: file
config:
path: modules/report_gen.yaml
edges:
- from: Input Handler
to: Analysis Module
- from: Analysis Module
to: Report Module
```
## Notes
- Subgraph file paths support relative paths (based on `yaml_instance/`) and absolute paths
- Avoid circular nesting (A references B, B references A)
- The subgraph's `start` and `end` nodes determine how data flows in and out, which decides how the subgraph processes messages from the parent graph and which node's final output is returned to the parent graph
+99
View File
@@ -0,0 +1,99 @@
# Frontend Web UI Quick Start Guide
This guide helps users quickly get started with the DevAll Web UI, covering main functional pages and operation workflows.
## 1. System Entry
After starting frontend and backend services, visit `http://localhost:5173` to access the Web UI.
## 2. Main Pages
### 2.1 Home
System homepage providing quick navigation links.
### 2.2 Workflow List
View and manage all available workflow YAML files.
**Features**:
- Browse workflows in `yaml_instance/` directory
- Preview YAML configuration content
- Select workflow to execute or edit
### 2.3 Launch View
The main interface for workflow execution, the most commonly used page.
**Operation Flow**:
1. **Select workflow**: Choose YAML file from the left panel
2. **Upload attachments** (optional): Click upload button to add files (CSV data, images, etc.)
3. **Enter task prompt**: Input instructions in the text box to guide workflow execution
4. **Click Launch**: Start workflow execution
**During Execution**:
- **Nodes View**: Observe node status changes (pending → running → success/failed)
- **Output Panel**: View real-time execution logs, node output context, and generated artifacts (all share the same panel)
**Human Input**:
- When execution reaches a `human` node, the interface displays an input prompt
- Fill in text content or upload attachments, then submit to continue execution
### 2.4 Workflow Workbench
Visual workflow editor.
**Features**:
- Drag-and-drop node editing
- Node configuration panel
- Edge connections and condition settings
- Export to YAML file
### 2.5 Tutorial
Built-in tutorial to help new users understand system features.
## 3. Common Operations
### 3.1 Running a Workflow
1. Go to **Launch View**
2. Select workflow from the left panel
3. Enter Task Prompt
4. Click **Launch** button
5. Monitor execution progress and wait for completion
### 3.2 Downloading Results
After execution completes:
1. Click the **Download** button on the right panel
2. Download the complete Session archive (includes context.json, attachments, logs, etc.)
### 3.3 Human Review Node Interaction
When workflow contains `human` nodes:
1. Execution pauses and displays prompt message
2. Read context content
3. Enter review comments or action instructions
4. Click submit to continue execution
## 4. Keyboard Shortcuts
| Shortcut | Function |
|----------|----------|
| `Ctrl/Cmd + Enter` | Submit input |
| `Esc` | Close popup/panel |
## 5. Troubleshooting
| Issue | Solution |
|-------|----------|
| Page won't load | Confirm frontend `npm run dev` is running |
| Cannot connect to backend | Confirm backend `uv run python server_main.py` is running |
| Empty workflow list | Check `yaml_instance/` directory for YAML files |
| Execution unresponsive | Check browser DevTools Network/Console logs |
| WebSocket disconnected | Refresh page to re-establish connection |
## 6. Related Documentation
- [Workflow Authoring](workflow_authoring.md) - YAML writing guide
+224
View File
@@ -0,0 +1,224 @@
# Workflow Authoring Guide
This guide covers YAML structure, node types, provider configuration, edge conditions, and design template export so you can build and debug DevAll DAGs efficiently. Content mirrors `docs/user_guide/workflow_authoring.md` with English copy for global contributors.
## 1. Prerequisites
- Know the layout of `yaml_instance/` and `yaml_template/`.
- Understand the core node types (`model`, `python`, `agent`, `human`, `subgraph`, `passthrough`, `literal`).
- Review `FIELD_SPECS` (see [field_specs.md](field_specs.md)) and the Schema API contract ([config_schema_contract.md](config_schema_contract.md)) if you rely on dynamic forms in the frontend/IDE.
## 2. YAML Top-level Structure
Every workflow file follows the `DesignConfig` root with only three keys: `version`, `vars`, and `graph`. The snippet below is adapted from `yaml_instance/net_example.yaml` and can run as-is:
```yaml
version: 0.4.0
vars:
BASE_URL: https://api.example.com/v1
API_KEY: ${API_KEY}
graph:
id: paper_gen
description: Article generation and refinement
log_level: INFO
is_majority_voting: false
initial_instruction: |
Provide a word or short phrase and the workflow will draft and polish an article.
start:
- Article Writer
end:
- Article Writer
nodes:
- id: Article Writer
type: agent
config:
provider: openai
base_url: ${BASE_URL}
api_key: ${API_KEY}
name: gpt-4o
params:
temperature: 0.1
- id: Human Reviewer
type: human
config:
description: Review the article. Type ACCEPT to finish; otherwise provide revision notes.
edges:
- from: Article Writer
to: Human Reviewer
- from: Human Reviewer
to: Article Writer
condition:
type: keyword
config:
none:
- ACCEPT
case_sensitive: false
```
- `version`: optional configuration version (defaults to `0.0.0`). Increment it whenever schema changes in `entity/configs/graph.py` require template or migration updates.
- `vars`: root-level key-value map. You can reference `${VAR}` anywhere in the file; fallback is the same-name environment variable. `GraphDefinition.from_dict` rejects nested `vars`, so keep this block at the top only.
**Environment Variables & `.env` File**
The system supports referencing variables in YAML configurations using `${VAR}` syntax. These variables can be used in any string field within the configuration. Common use cases include:
- **API keys**: `api_key: ${API_KEY}`
- **Service URLs**: `base_url: ${BASE_URL}`
- **Model names**: `name: ${MODEL_NAME}`
The system automatically loads the `.env` file from the project root (if present) when parsing configurations. Variable resolution follows this priority order:
| Priority | Source | Description |
| --- | --- | --- |
| 1 (highest) | Values defined in `vars` | Key-value pairs declared directly in the YAML file |
| 2 | System/shell environment variables | Values set via `export` or system config |
| 3 (lowest) | Values from `.env` file | Only applied if the variable doesn't already exist |
> [!TIP]
> The `.env` file does not override existing environment variables. This allows you to define defaults in `.env` while overriding them via `export` or deployment platform configurations.
> [!WARNING]
> If a placeholder references a variable that is not defined in any of the three sources above, a `ConfigError` will be raised during configuration parsing with the exact path indicated.
- `graph`: required block that maps to the `GraphDefinition` dataclass:
- **Metadata**: `id` (required), `description`, `log_level` (default `DEBUG`), `is_majority_voting`, `initial_instruction`, and optional `organization`.
- **Execution controls**: `start`/`end` entry lists (the system executes nodes listed in `start` at the beginning), plus `nodes` and `edges`. Provider/model/tooling settings now live inside each `node.config`; the legacy top-level `providers` table is deprecated. In the example the `keyword` condition on `Human Reviewer -> Article Writer` keeps looping unless the reviewer types `ACCEPT`.
- **Shared resources**: `memory` defines stores available to `node.config.memories`. The validator ensures every attachment points to a declared store.
- **Schema references**: `yaml_template/design.yaml` mirrors the latest `GraphDefinition` shape. After editing configs run `python -m tools.export_design_template` or hit the Schema API to validate.
Further reading: `docs/user_guide/en/field_specs.md` (field catalog), `docs/user_guide/en/runtime_ops.md` (runtime observability), and `yaml_template/design.yaml` (generated baseline template).
## 3. Node Type Cheatsheet
| Type | Description | Key fields | Detailed Docs |
| --- |------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| --- | --- |
| `agent` | Runs an LLM-backed agent with optional tools, memories, and thinking phases. | `provider`, `model`, `prompt_template`, `tooling`, `thinking`, `memories` | [agent.md](nodes/agent.md) |
| `python` | Executes Python scripts/commands sharing the `code_workspace/`. | `entry_script`, `inline_code`, `timeout`, `env` | [python.md](nodes/python.md) |
| `human` | Pauses in the Web UI awaiting human input. | `prompt`, `timeout`, `attachments` | [human.md](nodes/human.md) |
| `subgraph` | Embeds a child DAG to reuse complex flows. | `graph_path` or inline `graph` | [subgraph.md](nodes/subgraph.md) |
| `passthrough` | Pass-through node that forwards only the last message by default and can be configured to forward all messages; used for context filtering and graph structure optimization. | `only_last_message` | [passthrough.md](nodes/passthrough.md) |
| `literal` | Emits a fixed text payload whenever triggered and discards inputs. | `content`, `role` (`user`/`assistant`) | [literal.md](nodes/literal.md) |
| `loop_counter` | Guard node that limits loop iterations before releasing downstream edges. | `max_iterations`, `reset_on_emit`, `message` | [loop_counter.md](nodes/loop_counter.md) |
| `loop_timer` | Guard node that limits loop duration before releasing downstream edges. | `max_duration`, `duration_unit`, `reset_on_emit`, `message`, `passthrough` | [loop_timer.md](nodes/loop_timer.md) |
Fetch the full schema via `POST /api/config/schema` or inspect the dataclasses inside `entity/configs/`.
## 4. Providers & Agent Settings
- When a node omits `provider`, the engine uses `globals.default_provider` (e.g., `openai`).
- Fields such as `model`, `api_key`, and `base_url` accept `${VAR}` placeholders for environment portability.
- When combining multiple providers, define `globals` in the workflow root (`{ default_provider: ..., retry: {...} }`) if supported by the dataclass.
### 4.1 Gemini Provider Config Example
```yaml
model:
provider: gemini
base_url: https://generativelanguage.googleapis.com
api_key: ${GEMINI_API_KEY}
name: gemini-2.0-flash-001
input_mode: messages
params:
response_modalities: ["text", "image"]
safety_settings:
- category: HARM_CATEGORY_SEXUAL
threshold: BLOCK_LOWER
```
The Gemini Provider supports multi-modal input (images/video/audio are automatically converted to Parts) and supports `function_calling_config` to control tool execution behavior.
## 5. Edges & Conditions
- Basic edge:
```yaml
- source: plan
target: execute
```
- Conditional edge:
```yaml
edges:
- source: router
target: analyze
condition: should_analyze # functions/edge/should_analyze.py
```
- If a `condition` function raises, the scheduler marks the branch as failed and stops downstream execution.
### 5.1 Edge Payload Processors
- Add `process` to an edge when you want to transform or filter the payload after the condition is met (e.g., extract a verdict, keep only structured fields, or rewrite text).
- Structure mirrors `condition` (`type + config`). Built-ins include:
- `regex_extract`: Python regex with optional `group`, `mode` (`replace_content`, `metadata`, `data_block`), `multiple`, and `on_no_match` (`pass`, `default`, `drop`).
- `function`: calls helpers under `functions/edge_processor/`. The handler signature is `def foo(payload: Message, **kwargs) -> Message | None`. Note: The Processor interface is now standardized, and `kwargs` includes `context: ExecutionContext`, allowing access to the current execution context.
- Example:
```yaml
- from: reviewer
to: qa
process:
type: regex_extract
config:
pattern: "Score\\s*:\\s*(?P<score>\\d+)"
group: score
mode: metadata
metadata_key: quality_score
case_sensitive: false
on_no_match: default
default_value: "0"
```
## 6. Agent Node Advanced Features
- **Tooling**: Configure `AgentConfig.tooling`; see the [Tooling module](modules/tooling/README.md) (Chinese for now).
- **Thinking**: Enable staged reasoning via `AgentConfig.thinking` (e.g., chain-of-thought, reflection). Reference `entity/configs/thinking.py` for parameters.
- **Memories**: Attach `MemoryAttachmentConfig` through `AgentConfig.memories`; details live in the [Memory module](modules/memory.md).
## 7. Dynamic Execution (Map-Reduce/Tree)
Nodes support a sibling field `dynamic` to enable parallel processing or Map-Reduce patterns.
### 7.1 Core Concepts
- **Map Mode** (`type: map`): Fan-out. Splits list inputs into multiple units for parallel execution, outputting `List[Message]` (flattened results).
- **Tree Mode** (`type: tree`): Fan-out & Reduce. Splits inputs for parallel execution, then recursively reduces results in groups of `group_size` until a single result remains (e.g., "summary of summaries").
- **Split Strategy**: Defines how to partition the output of the previous node or current input into parallel units.
### 7.2 Configuration Structure
```yaml
nodes:
- id: Research Agents
type: agent
# Standard config (behaves as template for parallel units)
config:
provider: openai
model: gpt-4o
prompt_template: "Research this topic: {{content}}"
# Dynamic execution config
dynamic:
type: map
# Split strategy (first layer only)
split:
type: message # Options: message, regex, json_path
# pattern: "..." # Required for regex mode
# json_path: "$.items[*]" # Required for json_path mode
# Mode-specific config
config:
max_parallel: 5 # Concurrency limit
```
### 7.3 Tree Mode Example
Ideal for chunked summarization of long texts:
```yaml
dynamic:
type: tree
split:
type: regex
pattern: "(?s).{1,2000}(?:\\s|$)" # Split every ~2000 chars
config:
group_size: 3 # Reduce every 3 results into 1
max_parallel: 10
```
This mode automatically builds a multi-level execution tree until the result count is reduced to 1. Split config is the same as map mode.
## 8. Design Template Export
After editing configs or `FIELD_SPECS`, regenerate templates:
```bash
python -m tools.export_design_template \
--output yaml_template/design.yaml \
--mirror frontend/public/design_0.4.0.yaml
```
- The script scans registered nodes, memories, tooling, and `FIELD_SPECS` to emit YAML templates plus the frontend mirror file.
- Commit the generated files and notify frontend owners to refresh static assets.
## 9. CLI / API Execution Paths
- **Web UI**: Choose a YAML file, fill run parameters, start execution, and monitor in the dashboard. *Recommended path.*
- **HTTP**: `POST /api/workflow/execute` with `session_name`, `graph_path` or `graph_content`, `task_prompt`, optional `attachments`, and `log_level` (defaults to `INFO`, supports `INFO` or `DEBUG`).
- **CLI**: `python run.py --path yaml_instance/demo.yaml --name test_run`. Provide `TASK_PROMPT` via env var or respond to the CLI prompt.
## 10. Debugging Tips
- Use the Web UI context snapshots or `WareHouse/<session>/context.json` to inspect node I/O. Note that all node outputs are now standardized as `List[Message]`.
- Leverage the Schema API breadcrumbs ([config_schema_contract.md](config_schema_contract.md)) or run `python run.py --inspect-schema` to view field specs quickly.
- Missing YAML placeholders trigger `ConfigError` during parsing with a precise path surfaced in both UI and CLI logs.
+213
View File
@@ -0,0 +1,213 @@
# WebSocket Connection Lifecycle Analysis
## Scenario: Workflow Completed/Cancelled → File Change/Relaunch → Launch
### Initial State (Workflow Completed/Cancelled)
**Variables:**
- `status.value = 'Completed'` or `'Cancelled'`
- `isWorkflowRunning.value = false`
- `shouldGlow.value = true` (set by watch on status)
- `ws` = existing WebSocket connection (still open)
- `sessionId` = current session ID
- `isConnectionReady.value = true`
---
## Scenario 1: Another File is Chosen
### Step 1: File Selection Triggers Watch
**Function:** `watch(selectedFile, (newFile) => {...})` (line 879)
**Process:**
1. `taskPrompt.value = ''` - clears input
2. `fileSearchQuery.value = newFile || ''` - updates search query
3. `isFileSearchDirty.value = false` - resets search state
### Step 2: WebSocket Disconnection
**Function:** `resetConnectionState({ closeSocket: true })` (line 443)
**Called at:** line 891
**What happens:**
-**WebSocket DISCONNECTED** - `ws.close()` is called (line 446)
- `ws = null` (line 452)
- `sessionId = null` (line 453)
- `isConnectionReady.value = false` (line 454)
- `shouldGlow.value = false` (line 455)
- `isWorkflowRunning.value = false` (line 456)
- `activeNodes.value = []` (line 457)
- Clears attachment timeouts and uploaded attachments
**Status:** `status.value = 'Connecting...'` (line 892)
### Step 3: Load YAML
**Function:** `handleYAMLSelection(newFile)` (line 706)
**Called at:** line 893
**What happens:**
- Clears `chatMessages.value = []`
- Fetches YAML file content
- Parses YAML and stores in `workflowYaml.value`
- Displays `initial_instruction` as notification
- Loads VueFlow graph
### Step 4: WebSocket Reconnection
**Function:** `establishWebSocketConnection()` (line 807)
**Called at:** line 894
**What happens:**
1. **Double Reset:** Calls `resetConnectionState()` again (line 809) - ensures clean state
2. **New WebSocket Created:**
- `const socket = new WebSocket('ws://localhost:8000/ws')` (line 816)
- `ws = socket` (line 817)
3. **Connection Events:**
- `socket.onopen` (line 819): Logs "WebSocket connected"
- `socket.onmessage` (line 825):
- Receives `connection` message with `session_id`
- Sets `sessionId` (line 832)
- Sets `isConnectionReady.value = true` (line 842)
- Sets `shouldGlow.value = true` (line 843)
- Sets `status.value = 'Waiting for launch...'` (line 844)
- `socket.onerror` (line 854): Handles connection errors
- `socket.onclose` (line 864): Handles disconnection
**Status:** `status.value = 'Waiting for launch...'` (after connection message received)
---
## Scenario 2: Relaunch Button Clicked
### Step 1: Button Click Handler
**Function:** `handleButtonClick()` (line 740)
**Triggered:** When Launch button is clicked and status is 'Completed'/'Cancelled'
**Condition Check:**
```javascript
else if (status.value === 'Completed' || status.value === 'Cancelled')
```
### Step 2: WebSocket Disconnection
**Function:** `resetConnectionState()` (line 443)
**Called at:** line 754
**What happens:**
-**WebSocket DISCONNECTED** - `ws.close()` is called
- All state variables reset (same as Scenario 1, Step 2)
**Status:** `status.value = 'Connecting...'` (line 755)
### Step 3: Load YAML
**Function:** `handleYAMLSelection(selectedFile.value)` (line 706)
**Called at:** line 756
**What happens:** Same as Scenario 1, Step 3
### Step 4: WebSocket Reconnection
**Function:** `establishWebSocketConnection()` (line 807)
**Called at:** line 757
**What happens:** Same as Scenario 1, Step 4
**Status:** `status.value = 'Waiting for launch...'` (after connection message received)
---
## Step 5: Launch Button Clicked (After Reconnection)
### Launch Workflow
**Function:** `launchWorkflow()` (line 1124)
**Triggered:** When Launch button is clicked and status is 'Waiting for launch...'
**Prerequisites Check:**
- `selectedFile.value` must exist
- `taskPrompt.value.trim()` or `attachmentIds.length > 0` must exist
- `ws`, `isConnectionReady.value`, and `sessionId` must be valid
**What happens:**
1. Sets `shouldGlow.value = false` (line 1150)
2. Sets `status.value = 'Launching...'` (line 1151)
3. Sends POST request to `/api/workflow/execute` with:
- `yaml_file`: selected file name
- `task_prompt`: user input
- `session_id`: current session ID
- `attachments`: uploaded attachment IDs
4. On success:
- Clears uploaded attachments
- Adds user dialogue to chat
- Sets `status.value = 'Running...'` (line 1185)
- Sets `isWorkflowRunning.value = true` (line 1186)
**Status:** `status.value = 'Running...'`
---
## Key Variables and Their Roles
### WebSocket State Variables
| Variable | Type | Purpose | Changes When |
|----------|------|---------|--------------|
| `ws` | `WebSocket \| null` | Current WebSocket connection | Set to `null` on disconnect, new socket on connect |
| `sessionId` | `string \| null` | Current session identifier | Set from connection message, cleared on reset |
| `isConnectionReady` | `ref<boolean>` | Whether connection is ready for launch | `true` after connection message, `false` on reset |
### Status Variables
| Variable | Type | Purpose | Values |
|----------|------|---------|--------|
| `status` | `ref<string>` | Current workflow status | 'Completed', 'Cancelled', 'Connecting...', 'Waiting for launch...', 'Launching...', 'Running...' |
| `isWorkflowRunning` | `ref<boolean>` | Whether workflow is actively running | `true` during execution, `false` otherwise |
| `shouldGlow` | `ref<boolean>` | UI glow effect state | `true` when ready for input, `false` during execution |
### Workflow State Variables
| Variable | Type | Purpose |
|----------|------|---------|
| `selectedFile` | `ref<string>` | Currently selected YAML file |
| `workflowYaml` | `ref<object>` | Parsed YAML content |
| `activeNodes` | `ref<string[]>` | List of currently active node IDs |
| `chatMessages` | `ref<array>` | All chat messages and notifications |
---
## WebSocket Connection Timeline
### When Disconnected:
1. **File Change:** `watch(selectedFile)``resetConnectionState()``ws.close()`
2. **Relaunch:** `handleButtonClick()``resetConnectionState()``ws.close()`
3. **Error/Close Events:** `socket.onerror` or `socket.onclose``resetConnectionState({ closeSocket: false })`
### When Reconnected:
1. **File Change:** `watch(selectedFile)``establishWebSocketConnection()``new WebSocket()`
2. **Relaunch:** `handleButtonClick()``establishWebSocketConnection()``new WebSocket()`
### Connection States:
```
[Completed/Cancelled]
↓ (File Change or Relaunch)
[Disconnected] → resetConnectionState() closes ws
[Connecting...] → establishWebSocketConnection() creates new ws
[WebSocket.onopen] → socket opened
[WebSocket.onmessage: 'connection'] → sessionId received
[Waiting for launch...] → isConnectionReady = true
↓ (Launch clicked)
[Launching...] → POST /api/workflow/execute
[Running...] → isWorkflowRunning = true
```
---
## Important Notes
1. **Double Reset Issue:** `establishWebSocketConnection()` calls `resetConnectionState()` at the start (line 809), which means when called after `resetConnectionState()` in the caller, it's resetting twice. This is safe but redundant.
2. **Stale Socket Protection:** All WebSocket event handlers check `if (ws !== socket) return` to ignore events from old sockets that were replaced.
3. **Status Transitions:** The status goes through these states:
- `Completed/Cancelled``Connecting...``Waiting for launch...``Launching...``Running...`
4. **Connection Ready Check:** `launchWorkflow()` verifies `ws`, `isConnectionReady.value`, and `sessionId` before allowing launch.
5. **Automatic Reconnection:** Both file change and relaunch automatically establish a new WebSocket connection - no manual reconnection needed.