chore: import upstream snapshot with attribution
This commit is contained in:
@@ -0,0 +1,116 @@
|
||||
# s01: The Agent Loop
|
||||
|
||||
`[ s01 ] > s02 > s03 > s04 > s05 > s06 | s07 > s08 > s09 > s10 > s11 > s12`
|
||||
|
||||
> *"One loop & Bash is all you need"* -- one tool + one loop = an agent.
|
||||
>
|
||||
> **Harness layer**: The loop -- the model's first connection to the real world.
|
||||
|
||||
## Problem
|
||||
|
||||
A language model can reason about code, but it can't *touch* the real world -- can't read files, run tests, or check errors. Without a loop, every tool call requires you to manually copy-paste results back. You become the loop.
|
||||
|
||||
## Solution
|
||||
|
||||
```
|
||||
+--------+ +-------+ +---------+
|
||||
| User | ---> | LLM | ---> | Tool |
|
||||
| prompt | | | | execute |
|
||||
+--------+ +---+---+ +----+----+
|
||||
^ |
|
||||
| tool_result |
|
||||
+----------------+
|
||||
(loop until stop_reason != "tool_use")
|
||||
```
|
||||
|
||||
One exit condition controls the entire flow. The loop runs until the model stops calling tools.
|
||||
|
||||
## How It Works
|
||||
|
||||
1. User prompt becomes the first message.
|
||||
|
||||
```python
|
||||
messages.append({"role": "user", "content": query})
|
||||
```
|
||||
|
||||
2. Send messages + tool definitions to the LLM.
|
||||
|
||||
```python
|
||||
response = client.messages.create(
|
||||
model=MODEL, system=SYSTEM, messages=messages,
|
||||
tools=TOOLS, max_tokens=8000,
|
||||
)
|
||||
```
|
||||
|
||||
3. Append the assistant response. Check `stop_reason` -- if the model didn't call a tool, we're done.
|
||||
|
||||
```python
|
||||
messages.append({"role": "assistant", "content": response.content})
|
||||
if response.stop_reason != "tool_use":
|
||||
return
|
||||
```
|
||||
|
||||
4. Execute each tool call, collect results, append as a user message. Loop back to step 2.
|
||||
|
||||
```python
|
||||
results = []
|
||||
for block in response.content:
|
||||
if block.type == "tool_use":
|
||||
output = run_bash(block.input["command"])
|
||||
results.append({
|
||||
"type": "tool_result",
|
||||
"tool_use_id": block.id,
|
||||
"content": output,
|
||||
})
|
||||
messages.append({"role": "user", "content": results})
|
||||
```
|
||||
|
||||
Assembled into one function:
|
||||
|
||||
```python
|
||||
def agent_loop(query):
|
||||
messages = [{"role": "user", "content": query}]
|
||||
while True:
|
||||
response = client.messages.create(
|
||||
model=MODEL, system=SYSTEM, messages=messages,
|
||||
tools=TOOLS, max_tokens=8000,
|
||||
)
|
||||
messages.append({"role": "assistant", "content": response.content})
|
||||
|
||||
if response.stop_reason != "tool_use":
|
||||
return
|
||||
|
||||
results = []
|
||||
for block in response.content:
|
||||
if block.type == "tool_use":
|
||||
output = run_bash(block.input["command"])
|
||||
results.append({
|
||||
"type": "tool_result",
|
||||
"tool_use_id": block.id,
|
||||
"content": output,
|
||||
})
|
||||
messages.append({"role": "user", "content": results})
|
||||
```
|
||||
|
||||
That's the entire agent in under 30 lines. Everything else in this course layers on top -- without changing the loop.
|
||||
|
||||
## What Changed
|
||||
|
||||
| Component | Before | After |
|
||||
|---------------|------------|--------------------------------|
|
||||
| Agent loop | (none) | `while True` + stop_reason |
|
||||
| Tools | (none) | `bash` (one tool) |
|
||||
| Messages | (none) | Accumulating list |
|
||||
| Control flow | (none) | `stop_reason != "tool_use"` |
|
||||
|
||||
## Try It
|
||||
|
||||
```sh
|
||||
cd learn-claude-code
|
||||
python agents/s01_agent_loop.py
|
||||
```
|
||||
|
||||
1. `Create a file called hello.py that prints "Hello, World!"`
|
||||
2. `List all Python files in this directory`
|
||||
3. `What is the current git branch?`
|
||||
4. `Create a directory called test_output and write 3 files in it`
|
||||
@@ -0,0 +1,99 @@
|
||||
# s02: Tool Use
|
||||
|
||||
`s01 > [ s02 ] > s03 > s04 > s05 > s06 | s07 > s08 > s09 > s10 > s11 > s12`
|
||||
|
||||
> *"Adding a tool means adding one handler"* -- the loop stays the same; new tools register into the dispatch map.
|
||||
>
|
||||
> **Harness layer**: Tool dispatch -- expanding what the model can reach.
|
||||
|
||||
## Problem
|
||||
|
||||
With only `bash`, the agent shells out for everything. `cat` truncates unpredictably, `sed` fails on special characters, and every bash call is an unconstrained security surface. Dedicated tools like `read_file` and `write_file` let you enforce path sandboxing at the tool level.
|
||||
|
||||
The key insight: adding tools does not require changing the loop.
|
||||
|
||||
## Solution
|
||||
|
||||
```
|
||||
+--------+ +-------+ +------------------+
|
||||
| User | ---> | LLM | ---> | Tool Dispatch |
|
||||
| prompt | | | | { |
|
||||
+--------+ +---+---+ | bash: run_bash |
|
||||
^ | read: run_read |
|
||||
| | write: run_wr |
|
||||
+-----------+ edit: run_edit |
|
||||
tool_result | } |
|
||||
+------------------+
|
||||
|
||||
The dispatch map is a dict: {tool_name: handler_function}.
|
||||
One lookup replaces any if/elif chain.
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
1. Each tool gets a handler function. Path sandboxing prevents workspace escape.
|
||||
|
||||
```python
|
||||
def safe_path(p: str) -> Path:
|
||||
path = (WORKDIR / p).resolve()
|
||||
if not path.is_relative_to(WORKDIR):
|
||||
raise ValueError(f"Path escapes workspace: {p}")
|
||||
return path
|
||||
|
||||
def run_read(path: str, limit: int = None) -> str:
|
||||
text = safe_path(path).read_text()
|
||||
lines = text.splitlines()
|
||||
if limit and limit < len(lines):
|
||||
lines = lines[:limit]
|
||||
return "\n".join(lines)[:50000]
|
||||
```
|
||||
|
||||
2. The dispatch map links tool names to handlers.
|
||||
|
||||
```python
|
||||
TOOL_HANDLERS = {
|
||||
"bash": lambda **kw: run_bash(kw["command"]),
|
||||
"read_file": lambda **kw: run_read(kw["path"], kw.get("limit")),
|
||||
"write_file": lambda **kw: run_write(kw["path"], kw["content"]),
|
||||
"edit_file": lambda **kw: run_edit(kw["path"], kw["old_text"],
|
||||
kw["new_text"]),
|
||||
}
|
||||
```
|
||||
|
||||
3. In the loop, look up the handler by name. The loop body itself is unchanged from s01.
|
||||
|
||||
```python
|
||||
for block in response.content:
|
||||
if block.type == "tool_use":
|
||||
handler = TOOL_HANDLERS.get(block.name)
|
||||
output = handler(**block.input) if handler \
|
||||
else f"Unknown tool: {block.name}"
|
||||
results.append({
|
||||
"type": "tool_result",
|
||||
"tool_use_id": block.id,
|
||||
"content": output,
|
||||
})
|
||||
```
|
||||
|
||||
Add a tool = add a handler + add a schema entry. The loop never changes.
|
||||
|
||||
## What Changed From s01
|
||||
|
||||
| Component | Before (s01) | After (s02) |
|
||||
|----------------|--------------------|----------------------------|
|
||||
| Tools | 1 (bash only) | 4 (bash, read, write, edit)|
|
||||
| Dispatch | Hardcoded bash call | `TOOL_HANDLERS` dict |
|
||||
| Path safety | None | `safe_path()` sandbox |
|
||||
| Agent loop | Unchanged | Unchanged |
|
||||
|
||||
## Try It
|
||||
|
||||
```sh
|
||||
cd learn-claude-code
|
||||
python agents/s02_tool_use.py
|
||||
```
|
||||
|
||||
1. `Read the file requirements.txt`
|
||||
2. `Create a file called greet.py with a greet(name) function`
|
||||
3. `Edit greet.py to add a docstring to the function`
|
||||
4. `Read greet.py to verify the edit worked`
|
||||
@@ -0,0 +1,96 @@
|
||||
# s03: TodoWrite
|
||||
|
||||
`s01 > s02 > [ s03 ] > s04 > s05 > s06 | s07 > s08 > s09 > s10 > s11 > s12`
|
||||
|
||||
> *"An agent without a plan drifts"* -- list the steps first, then execute.
|
||||
>
|
||||
> **Harness layer**: Planning -- keeping the model on course without scripting the route.
|
||||
|
||||
## Problem
|
||||
|
||||
On multi-step tasks, the model loses track. It repeats work, skips steps, or wanders off. Long conversations make this worse -- the system prompt fades as tool results fill the context. A 10-step refactoring might complete steps 1-3, then the model starts improvising because it forgot steps 4-10.
|
||||
|
||||
## Solution
|
||||
|
||||
```
|
||||
+--------+ +-------+ +---------+
|
||||
| User | ---> | LLM | ---> | Tools |
|
||||
| prompt | | | | + todo |
|
||||
+--------+ +---+---+ +----+----+
|
||||
^ |
|
||||
| tool_result |
|
||||
+----------------+
|
||||
|
|
||||
+-----------+-----------+
|
||||
| TodoManager state |
|
||||
| [ ] task A |
|
||||
| [>] task B <- doing |
|
||||
| [x] task C |
|
||||
+-----------------------+
|
||||
|
|
||||
if rounds_since_todo >= 3:
|
||||
inject <reminder> into tool_result
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
1. TodoManager stores items with statuses. Only one item can be `in_progress` at a time.
|
||||
|
||||
```python
|
||||
class TodoManager:
|
||||
def update(self, items: list) -> str:
|
||||
validated, in_progress_count = [], 0
|
||||
for item in items:
|
||||
status = item.get("status", "pending")
|
||||
if status == "in_progress":
|
||||
in_progress_count += 1
|
||||
validated.append({"id": item["id"], "text": item["text"],
|
||||
"status": status})
|
||||
if in_progress_count > 1:
|
||||
raise ValueError("Only one task can be in_progress")
|
||||
self.items = validated
|
||||
return self.render()
|
||||
```
|
||||
|
||||
2. The `todo` tool goes into the dispatch map like any other tool.
|
||||
|
||||
```python
|
||||
TOOL_HANDLERS = {
|
||||
# ...base tools...
|
||||
"todo": lambda **kw: TODO.update(kw["items"]),
|
||||
}
|
||||
```
|
||||
|
||||
3. A nag reminder injects a nudge if the model goes 3+ rounds without calling `todo`.
|
||||
|
||||
```python
|
||||
if rounds_since_todo >= 3 and messages:
|
||||
last = messages[-1]
|
||||
if last["role"] == "user" and isinstance(last.get("content"), list):
|
||||
last["content"].insert(0, {
|
||||
"type": "text",
|
||||
"text": "<reminder>Update your todos.</reminder>",
|
||||
})
|
||||
```
|
||||
|
||||
The "one in_progress at a time" constraint forces sequential focus. The nag reminder creates accountability.
|
||||
|
||||
## What Changed From s02
|
||||
|
||||
| Component | Before (s02) | After (s03) |
|
||||
|----------------|------------------|----------------------------|
|
||||
| Tools | 4 | 5 (+todo) |
|
||||
| Planning | None | TodoManager with statuses |
|
||||
| Nag injection | None | `<reminder>` after 3 rounds|
|
||||
| Agent loop | Simple dispatch | + rounds_since_todo counter|
|
||||
|
||||
## Try It
|
||||
|
||||
```sh
|
||||
cd learn-claude-code
|
||||
python agents/s03_todo_write.py
|
||||
```
|
||||
|
||||
1. `Refactor the file hello.py: add type hints, docstrings, and a main guard`
|
||||
2. `Create a Python package with __init__.py, utils.py, and tests/test_utils.py`
|
||||
3. `Review all Python files and fix any style issues`
|
||||
@@ -0,0 +1,94 @@
|
||||
# s04: Subagents
|
||||
|
||||
`s01 > s02 > s03 > [ s04 ] > s05 > s06 | s07 > s08 > s09 > s10 > s11 > s12`
|
||||
|
||||
> *"Break big tasks down; each subtask gets a clean context"* -- subagents use independent messages[], keeping the main conversation clean.
|
||||
>
|
||||
> **Harness layer**: Context isolation -- protecting the model's clarity of thought.
|
||||
|
||||
## Problem
|
||||
|
||||
As the agent works, its messages array grows. Every file read, every bash output stays in context permanently. "What testing framework does this project use?" might require reading 5 files, but the parent only needs the answer: "pytest."
|
||||
|
||||
## Solution
|
||||
|
||||
```
|
||||
Parent agent Subagent
|
||||
+------------------+ +------------------+
|
||||
| messages=[...] | | messages=[] | <-- fresh
|
||||
| | dispatch | |
|
||||
| tool: task | ----------> | while tool_use: |
|
||||
| prompt="..." | | call tools |
|
||||
| | summary | append results |
|
||||
| result = "..." | <---------- | return last text |
|
||||
+------------------+ +------------------+
|
||||
|
||||
Parent context stays clean. Subagent context is discarded.
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
1. The parent gets a `task` tool. The child gets all base tools except `task` (no recursive spawning).
|
||||
|
||||
```python
|
||||
PARENT_TOOLS = CHILD_TOOLS + [
|
||||
{"name": "task",
|
||||
"description": "Spawn a subagent with fresh context.",
|
||||
"input_schema": {
|
||||
"type": "object",
|
||||
"properties": {"prompt": {"type": "string"}},
|
||||
"required": ["prompt"],
|
||||
}},
|
||||
]
|
||||
```
|
||||
|
||||
2. The subagent starts with `messages=[]` and runs its own loop. Only the final text returns to the parent.
|
||||
|
||||
```python
|
||||
def run_subagent(prompt: str) -> str:
|
||||
sub_messages = [{"role": "user", "content": prompt}]
|
||||
for _ in range(30): # safety limit
|
||||
response = client.messages.create(
|
||||
model=MODEL, system=SUBAGENT_SYSTEM,
|
||||
messages=sub_messages,
|
||||
tools=CHILD_TOOLS, max_tokens=8000,
|
||||
)
|
||||
sub_messages.append({"role": "assistant",
|
||||
"content": response.content})
|
||||
if response.stop_reason != "tool_use":
|
||||
break
|
||||
results = []
|
||||
for block in response.content:
|
||||
if block.type == "tool_use":
|
||||
handler = TOOL_HANDLERS.get(block.name)
|
||||
output = handler(**block.input)
|
||||
results.append({"type": "tool_result",
|
||||
"tool_use_id": block.id,
|
||||
"content": str(output)[:50000]})
|
||||
sub_messages.append({"role": "user", "content": results})
|
||||
return "".join(
|
||||
b.text for b in response.content if hasattr(b, "text")
|
||||
) or "(no summary)"
|
||||
```
|
||||
|
||||
The child's entire message history (possibly 30+ tool calls) is discarded. The parent receives a one-paragraph summary as a normal `tool_result`.
|
||||
|
||||
## What Changed From s03
|
||||
|
||||
| Component | Before (s03) | After (s04) |
|
||||
|----------------|------------------|---------------------------|
|
||||
| Tools | 5 | 5 (base) + task (parent) |
|
||||
| Context | Single shared | Parent + child isolation |
|
||||
| Subagent | None | `run_subagent()` function |
|
||||
| Return value | N/A | Summary text only |
|
||||
|
||||
## Try It
|
||||
|
||||
```sh
|
||||
cd learn-claude-code
|
||||
python agents/s04_subagent.py
|
||||
```
|
||||
|
||||
1. `Use a subtask to find what testing framework this project uses`
|
||||
2. `Delegate: read all .py files and summarize what each one does`
|
||||
3. `Use a task to create a new module, then verify it from here`
|
||||
@@ -0,0 +1,108 @@
|
||||
# s05: Skills
|
||||
|
||||
`s01 > s02 > s03 > s04 > [ s05 ] > s06 | s07 > s08 > s09 > s10 > s11 > s12`
|
||||
|
||||
> *"Load knowledge when you need it, not upfront"* -- inject via tool_result, not the system prompt.
|
||||
>
|
||||
> **Harness layer**: On-demand knowledge -- domain expertise, loaded when the model asks.
|
||||
|
||||
## Problem
|
||||
|
||||
You want the agent to follow domain-specific workflows: git conventions, testing patterns, code review checklists. Putting everything in the system prompt wastes tokens on unused skills. 10 skills at 2000 tokens each = 20,000 tokens, most of which are irrelevant to any given task.
|
||||
|
||||
## Solution
|
||||
|
||||
```
|
||||
System prompt (Layer 1 -- always present):
|
||||
+--------------------------------------+
|
||||
| You are a coding agent. |
|
||||
| Skills available: |
|
||||
| - git: Git workflow helpers | ~100 tokens/skill
|
||||
| - test: Testing best practices |
|
||||
+--------------------------------------+
|
||||
|
||||
When model calls load_skill("git"):
|
||||
+--------------------------------------+
|
||||
| tool_result (Layer 2 -- on demand): |
|
||||
| <skill name="git"> |
|
||||
| Full git workflow instructions... | ~2000 tokens
|
||||
| Step 1: ... |
|
||||
| </skill> |
|
||||
+--------------------------------------+
|
||||
```
|
||||
|
||||
Layer 1: skill *names* in system prompt (cheap). Layer 2: full *body* via tool_result (on demand).
|
||||
|
||||
## How It Works
|
||||
|
||||
1. Each skill is a directory containing a `SKILL.md` with YAML frontmatter.
|
||||
|
||||
```
|
||||
skills/
|
||||
pdf/
|
||||
SKILL.md # ---\n name: pdf\n description: Process PDF files\n ---\n ...
|
||||
code-review/
|
||||
SKILL.md # ---\n name: code-review\n description: Review code\n ---\n ...
|
||||
```
|
||||
|
||||
2. SkillLoader scans for `SKILL.md` files, uses the directory name as the skill identifier.
|
||||
|
||||
```python
|
||||
class SkillLoader:
|
||||
def __init__(self, skills_dir: Path):
|
||||
self.skills = {}
|
||||
for f in sorted(skills_dir.rglob("SKILL.md")):
|
||||
text = f.read_text()
|
||||
meta, body = self._parse_frontmatter(text)
|
||||
name = meta.get("name", f.parent.name)
|
||||
self.skills[name] = {"meta": meta, "body": body}
|
||||
|
||||
def get_descriptions(self) -> str:
|
||||
lines = []
|
||||
for name, skill in self.skills.items():
|
||||
desc = skill["meta"].get("description", "")
|
||||
lines.append(f" - {name}: {desc}")
|
||||
return "\n".join(lines)
|
||||
|
||||
def get_content(self, name: str) -> str:
|
||||
skill = self.skills.get(name)
|
||||
if not skill:
|
||||
return f"Error: Unknown skill '{name}'."
|
||||
return f"<skill name=\"{name}\">\n{skill['body']}\n</skill>"
|
||||
```
|
||||
|
||||
3. Layer 1 goes into the system prompt. Layer 2 is just another tool handler.
|
||||
|
||||
```python
|
||||
SYSTEM = f"""You are a coding agent at {WORKDIR}.
|
||||
Skills available:
|
||||
{SKILL_LOADER.get_descriptions()}"""
|
||||
|
||||
TOOL_HANDLERS = {
|
||||
# ...base tools...
|
||||
"load_skill": lambda **kw: SKILL_LOADER.get_content(kw["name"]),
|
||||
}
|
||||
```
|
||||
|
||||
The model learns what skills exist (cheap) and loads them when relevant (expensive).
|
||||
|
||||
## What Changed From s04
|
||||
|
||||
| Component | Before (s04) | After (s05) |
|
||||
|----------------|------------------|----------------------------|
|
||||
| Tools | 5 (base + task) | 5 (base + load_skill) |
|
||||
| System prompt | Static string | + skill descriptions |
|
||||
| Knowledge | None | skills/\*/SKILL.md files |
|
||||
| Injection | None | Two-layer (system + result)|
|
||||
|
||||
## Try It
|
||||
|
||||
```sh
|
||||
cd learn-claude-code
|
||||
python agents/s05_skill_loading.py
|
||||
```
|
||||
|
||||
1. `What skills are available?`
|
||||
2. `Load the agent-builder skill and follow its instructions`
|
||||
3. `I need to do a code review -- load the relevant skill first`
|
||||
4. `Build an MCP server using the mcp-builder skill`
|
||||
@@ -0,0 +1,124 @@
|
||||
# s06: Context Compact
|
||||
|
||||
`s01 > s02 > s03 > s04 > s05 > [ s06 ] | s07 > s08 > s09 > s10 > s11 > s12`
|
||||
|
||||
> *"Context will fill up; you need a way to make room"* -- three-layer compression strategy for infinite sessions.
|
||||
>
|
||||
> **Harness layer**: Compression -- clean memory for infinite sessions.
|
||||
|
||||
## Problem
|
||||
|
||||
The context window is finite. A single `read_file` on a 1000-line file costs ~4000 tokens. After reading 30 files and running 20 bash commands, you hit 100,000+ tokens. The agent cannot work on large codebases without compression.
|
||||
|
||||
## Solution
|
||||
|
||||
Three layers, increasing in aggressiveness:
|
||||
|
||||
```
|
||||
Every turn:
|
||||
+------------------+
|
||||
| Tool call result |
|
||||
+------------------+
|
||||
|
|
||||
v
|
||||
[Layer 1: micro_compact] (silent, every turn)
|
||||
Replace tool_result > 3 turns old
|
||||
with "[Previous: used {tool_name}]"
|
||||
|
|
||||
v
|
||||
[Check: tokens > 50000?]
|
||||
| |
|
||||
no yes
|
||||
| |
|
||||
v v
|
||||
continue [Layer 2: auto_compact]
|
||||
Save transcript to .transcripts/
|
||||
LLM summarizes conversation.
|
||||
Replace all messages with [summary].
|
||||
|
|
||||
v
|
||||
[Layer 3: compact tool]
|
||||
Model calls compact explicitly.
|
||||
Same summarization as auto_compact.
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
1. **Layer 1 -- micro_compact**: Before each LLM call, replace old tool results with placeholders.
|
||||
|
||||
```python
|
||||
def micro_compact(messages: list) -> list:
|
||||
tool_results = []
|
||||
for i, msg in enumerate(messages):
|
||||
if msg["role"] == "user" and isinstance(msg.get("content"), list):
|
||||
for j, part in enumerate(msg["content"]):
|
||||
if isinstance(part, dict) and part.get("type") == "tool_result":
|
||||
tool_results.append((i, j, part))
|
||||
if len(tool_results) <= KEEP_RECENT:
|
||||
return messages
|
||||
for _, _, part in tool_results[:-KEEP_RECENT]:
|
||||
if len(part.get("content", "")) > 100:
|
||||
part["content"] = f"[Previous: used {tool_name}]"
|
||||
return messages
|
||||
```
|
||||
|
||||
2. **Layer 2 -- auto_compact**: When tokens exceed threshold, save full transcript to disk, then ask the LLM to summarize.
|
||||
|
||||
```python
|
||||
def auto_compact(messages: list) -> list:
|
||||
# Save transcript for recovery
|
||||
transcript_path = TRANSCRIPT_DIR / f"transcript_{int(time.time())}.jsonl"
|
||||
with open(transcript_path, "w") as f:
|
||||
for msg in messages:
|
||||
f.write(json.dumps(msg, default=str) + "\n")
|
||||
# LLM summarizes
|
||||
response = client.messages.create(
|
||||
model=MODEL,
|
||||
messages=[{"role": "user", "content":
|
||||
"Summarize this conversation for continuity..."
|
||||
+ json.dumps(messages, default=str)[:80000]}],
|
||||
max_tokens=2000,
|
||||
)
|
||||
return [
|
||||
{"role": "user", "content": f"[Compressed]\n\n{response.content[0].text}"},
|
||||
]
|
||||
```
|
||||
|
||||
3. **Layer 3 -- manual compact**: The `compact` tool triggers the same summarization on demand.
|
||||
|
||||
4. The loop integrates all three:
|
||||
|
||||
```python
|
||||
def agent_loop(messages: list):
|
||||
while True:
|
||||
micro_compact(messages) # Layer 1
|
||||
if estimate_tokens(messages) > THRESHOLD:
|
||||
messages[:] = auto_compact(messages) # Layer 2
|
||||
response = client.messages.create(...)
|
||||
# ... tool execution ...
|
||||
if manual_compact:
|
||||
messages[:] = auto_compact(messages) # Layer 3
|
||||
```
|
||||
|
||||
Transcripts preserve full history on disk. Nothing is truly lost -- just moved out of active context.
|
||||
|
||||
## What Changed From s05
|
||||
|
||||
| Component | Before (s05) | After (s06) |
|
||||
|----------------|------------------|----------------------------|
|
||||
| Tools | 5 | 5 (base + compact) |
|
||||
| Context mgmt | None | Three-layer compression |
|
||||
| Micro-compact | None | Old results -> placeholders|
|
||||
| Auto-compact | None | Token threshold trigger |
|
||||
| Transcripts | None | Saved to .transcripts/ |
|
||||
|
||||
## Try It
|
||||
|
||||
```sh
|
||||
cd learn-claude-code
|
||||
python agents/s06_context_compact.py
|
||||
```
|
||||
|
||||
1. `Read every Python file in the agents/ directory one by one` (watch micro-compact replace old results)
|
||||
2. `Keep reading files until compression triggers automatically`
|
||||
3. `Use the compact tool to manually compress the conversation`
|
||||
@@ -0,0 +1,131 @@
|
||||
# s07: Task System
|
||||
|
||||
`s01 > s02 > s03 > s04 > s05 > s06 | [ s07 ] > s08 > s09 > s10 > s11 > s12`
|
||||
|
||||
> *"Break big goals into small tasks, order them, persist to disk"* -- a file-based task graph with dependencies, laying the foundation for multi-agent collaboration.
|
||||
>
|
||||
> **Harness layer**: Persistent tasks -- goals that outlive any single conversation.
|
||||
|
||||
## Problem
|
||||
|
||||
s03's TodoManager is a flat checklist in memory: no ordering, no dependencies, no status beyond done-or-not. Real goals have structure -- task B depends on task A, tasks C and D can run in parallel, task E waits for both C and D.
|
||||
|
||||
Without explicit relationships, the agent can't tell what's ready, what's blocked, or what can run concurrently. And because the list lives only in memory, context compression (s06) wipes it clean.
|
||||
|
||||
## Solution
|
||||
|
||||
Promote the checklist into a **task graph** persisted to disk. Each task is a JSON file with status, dependencies (`blockedBy`). The graph answers three questions at any moment:
|
||||
|
||||
- **What's ready?** -- tasks with `pending` status and empty `blockedBy`.
|
||||
- **What's blocked?** -- tasks waiting on unfinished dependencies.
|
||||
- **What's done?** -- `completed` tasks, whose completion automatically unblocks dependents.
|
||||
|
||||
```
|
||||
.tasks/
|
||||
task_1.json {"id":1, "status":"completed"}
|
||||
task_2.json {"id":2, "blockedBy":[1], "status":"pending"}
|
||||
task_3.json {"id":3, "blockedBy":[1], "status":"pending"}
|
||||
task_4.json {"id":4, "blockedBy":[2,3], "status":"pending"}
|
||||
|
||||
Task graph (DAG):
|
||||
+----------+
|
||||
+--> | task 2 | --+
|
||||
| | pending | |
|
||||
+----------+ +----------+ +--> +----------+
|
||||
| task 1 | | task 4 |
|
||||
| completed| --> +----------+ +--> | blocked |
|
||||
+----------+ | task 3 | --+ +----------+
|
||||
| pending |
|
||||
+----------+
|
||||
|
||||
Ordering: task 1 must finish before 2 and 3
|
||||
Parallelism: tasks 2 and 3 can run at the same time
|
||||
Dependencies: task 4 waits for both 2 and 3
|
||||
Status: pending -> in_progress -> completed
|
||||
```
|
||||
|
||||
This task graph becomes the coordination backbone for everything after s07: background execution (s08), multi-agent teams (s09+), and worktree isolation (s12) all read from and write to this same structure.
|
||||
|
||||
## How It Works
|
||||
|
||||
1. **TaskManager**: one JSON file per task, CRUD with dependency graph.
|
||||
|
||||
```python
|
||||
class TaskManager:
|
||||
def __init__(self, tasks_dir: Path):
|
||||
self.dir = tasks_dir
|
||||
self.dir.mkdir(exist_ok=True)
|
||||
self._next_id = self._max_id() + 1
|
||||
|
||||
def create(self, subject, description=""):
|
||||
task = {"id": self._next_id, "subject": subject,
|
||||
"status": "pending", "blockedBy": [],
|
||||
"owner": ""}
|
||||
self._save(task)
|
||||
self._next_id += 1
|
||||
return json.dumps(task, indent=2)
|
||||
```
|
||||
|
||||
2. **Dependency resolution**: completing a task clears its ID from every other task's `blockedBy` list, automatically unblocking dependents.
|
||||
|
||||
```python
|
||||
def _clear_dependency(self, completed_id):
|
||||
for f in self.dir.glob("task_*.json"):
|
||||
task = json.loads(f.read_text())
|
||||
if completed_id in task.get("blockedBy", []):
|
||||
task["blockedBy"].remove(completed_id)
|
||||
self._save(task)
|
||||
```
|
||||
|
||||
3. **Status + dependency wiring**: `update` handles transitions and dependency edges.
|
||||
|
||||
```python
|
||||
def update(self, task_id, status=None,
|
||||
add_blocked_by=None, remove_blocked_by=None):
|
||||
task = self._load(task_id)
|
||||
if status:
|
||||
task["status"] = status
|
||||
if status == "completed":
|
||||
self._clear_dependency(task_id)
|
||||
if add_blocked_by:
|
||||
task["blockedBy"] = list(set(task["blockedBy"] + add_blocked_by))
|
||||
if remove_blocked_by:
|
||||
task["blockedBy"] = [x for x in task["blockedBy"] if x not in remove_blocked_by]
|
||||
self._save(task)
|
||||
```
|
||||
|
||||
4. Four task tools go into the dispatch map.
|
||||
|
||||
```python
|
||||
TOOL_HANDLERS = {
|
||||
# ...base tools...
|
||||
"task_create": lambda **kw: TASKS.create(kw["subject"]),
|
||||
"task_update": lambda **kw: TASKS.update(kw["task_id"], kw.get("status")),
|
||||
"task_list": lambda **kw: TASKS.list_all(),
|
||||
"task_get": lambda **kw: TASKS.get(kw["task_id"]),
|
||||
}
|
||||
```
|
||||
|
||||
From s07 onward, the task graph is the default for multi-step work. s03's Todo remains for quick single-session checklists.
|
||||
|
||||
## What Changed From s06
|
||||
|
||||
| Component | Before (s06) | After (s07) |
|
||||
|---|---|---|
|
||||
| Tools | 5 | 8 (`task_create/update/list/get`) |
|
||||
| Planning model | Flat checklist (in-memory) | Task graph with dependencies (on disk) |
|
||||
| Relationships | None | `blockedBy` edges |
|
||||
| Status tracking | Done or not | `pending` -> `in_progress` -> `completed` |
|
||||
| Persistence | Lost on compression | Survives compression and restarts |
|
||||
|
||||
## Try It
|
||||
|
||||
```sh
|
||||
cd learn-claude-code
|
||||
python agents/s07_task_system.py
|
||||
```
|
||||
|
||||
1. `Create 3 tasks: "Setup project", "Write code", "Write tests". Make them depend on each other in order.`
|
||||
2. `List all tasks and show the dependency graph`
|
||||
3. `Complete task 1 and then list tasks to see task 2 unblocked`
|
||||
4. `Create a task board for refactoring: parse -> transform -> emit -> test, where transform and emit can run in parallel after parse`
|
||||
@@ -0,0 +1,107 @@
|
||||
# s08: Background Tasks
|
||||
|
||||
`s01 > s02 > s03 > s04 > s05 > s06 | s07 > [ s08 ] > s09 > s10 > s11 > s12`
|
||||
|
||||
> *"Run slow operations in the background; the agent keeps thinking"* -- daemon threads run commands, inject notifications on completion.
|
||||
>
|
||||
> **Harness layer**: Background execution -- the model thinks while the harness waits.
|
||||
|
||||
## Problem
|
||||
|
||||
Some commands take minutes: `npm install`, `pytest`, `docker build`. With a blocking loop, the model sits idle waiting. If the user asks "install dependencies and while that runs, create the config file," the agent does them sequentially, not in parallel.
|
||||
|
||||
## Solution
|
||||
|
||||
```
|
||||
Main thread Background thread
|
||||
+-----------------+ +-----------------+
|
||||
| agent loop | | subprocess runs |
|
||||
| ... | | ... |
|
||||
| [LLM call] <---+------- | enqueue(result) |
|
||||
| ^drain queue | +-----------------+
|
||||
+-----------------+
|
||||
|
||||
Timeline:
|
||||
Agent --[spawn A]--[spawn B]--[other work]----
|
||||
| |
|
||||
v v
|
||||
[A runs] [B runs] (parallel)
|
||||
| |
|
||||
+-- results injected before next LLM call --+
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
1. BackgroundManager tracks tasks with a thread-safe notification queue.
|
||||
|
||||
```python
|
||||
class BackgroundManager:
|
||||
def __init__(self):
|
||||
self.tasks = {}
|
||||
self._notification_queue = []
|
||||
self._lock = threading.Lock()
|
||||
```
|
||||
|
||||
2. `run()` starts a daemon thread and returns immediately.
|
||||
|
||||
```python
|
||||
def run(self, command: str) -> str:
|
||||
task_id = str(uuid.uuid4())[:8]
|
||||
self.tasks[task_id] = {"status": "running", "command": command}
|
||||
thread = threading.Thread(
|
||||
target=self._execute, args=(task_id, command), daemon=True)
|
||||
thread.start()
|
||||
return f"Background task {task_id} started"
|
||||
```
|
||||
|
||||
3. When the subprocess finishes, its result goes into the notification queue.
|
||||
|
||||
```python
|
||||
def _execute(self, task_id, command):
|
||||
try:
|
||||
r = subprocess.run(command, shell=True, cwd=WORKDIR,
|
||||
capture_output=True, text=True, timeout=300)
|
||||
output = (r.stdout + r.stderr).strip()[:50000]
|
||||
except subprocess.TimeoutExpired:
|
||||
output = "Error: Timeout (300s)"
|
||||
with self._lock:
|
||||
self._notification_queue.append({
|
||||
"task_id": task_id, "result": output[:500]})
|
||||
```
|
||||
|
||||
4. The agent loop drains notifications before each LLM call.
|
||||
|
||||
```python
|
||||
def agent_loop(messages: list):
|
||||
while True:
|
||||
notifs = BG.drain_notifications()
|
||||
if notifs:
|
||||
notif_text = "\n".join(
|
||||
f"[bg:{n['task_id']}] {n['result']}" for n in notifs)
|
||||
messages.append({"role": "user",
|
||||
"content": f"<background-results>\n{notif_text}\n"
|
||||
f"</background-results>"})
|
||||
response = client.messages.create(...)
|
||||
```
|
||||
|
||||
The loop stays single-threaded. Only subprocess I/O is parallelized.
|
||||
|
||||
## What Changed From s07
|
||||
|
||||
| Component | Before (s07) | After (s08) |
|
||||
|----------------|------------------|----------------------------|
|
||||
| Tools | 8 | 6 (base + background_run + check)|
|
||||
| Execution | Blocking only | Blocking + background threads|
|
||||
| Notification | None | Queue drained per loop |
|
||||
| Concurrency | None | Daemon threads |
|
||||
|
||||
## Try It
|
||||
|
||||
```sh
|
||||
cd learn-claude-code
|
||||
python agents/s08_background_tasks.py
|
||||
```
|
||||
|
||||
1. `Run "sleep 5 && echo done" in the background, then create a file while it runs`
|
||||
2. `Start 3 background tasks: "sleep 2", "sleep 4", "sleep 6". Check their status.`
|
||||
3. `Run pytest in the background and keep working on other things`
|
||||
@@ -0,0 +1,125 @@
|
||||
# s09: Agent Teams
|
||||
|
||||
`s01 > s02 > s03 > s04 > s05 > s06 | s07 > s08 > [ s09 ] > s10 > s11 > s12`
|
||||
|
||||
> *"When the task is too big for one, delegate to teammates"* -- persistent teammates + async mailboxes.
|
||||
>
|
||||
> **Harness layer**: Team mailboxes -- multiple models, coordinated through files.
|
||||
|
||||
## Problem
|
||||
|
||||
Subagents (s04) are disposable: spawn, work, return summary, die. No identity, no memory between invocations. Background tasks (s08) run shell commands but can't make LLM-guided decisions.
|
||||
|
||||
Real teamwork needs: (1) persistent agents that outlive a single prompt, (2) identity and lifecycle management, (3) a communication channel between agents.
|
||||
|
||||
## Solution
|
||||
|
||||
```
|
||||
Teammate lifecycle:
|
||||
spawn -> WORKING -> IDLE -> WORKING -> ... -> SHUTDOWN
|
||||
|
||||
Communication:
|
||||
.team/
|
||||
config.json <- team roster + statuses
|
||||
inbox/
|
||||
alice.jsonl <- append-only, drain-on-read
|
||||
bob.jsonl
|
||||
lead.jsonl
|
||||
|
||||
+--------+ send("alice","bob","...") +--------+
|
||||
| alice | -----------------------------> | bob |
|
||||
| loop | bob.jsonl << {json_line} | loop |
|
||||
+--------+ +--------+
|
||||
^ |
|
||||
| BUS.read_inbox("alice") |
|
||||
+---- alice.jsonl -> read + drain ---------+
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
1. TeammateManager maintains config.json with the team roster.
|
||||
|
||||
```python
|
||||
class TeammateManager:
|
||||
def __init__(self, team_dir: Path):
|
||||
self.dir = team_dir
|
||||
self.dir.mkdir(exist_ok=True)
|
||||
self.config_path = self.dir / "config.json"
|
||||
self.config = self._load_config()
|
||||
self.threads = {}
|
||||
```
|
||||
|
||||
2. `spawn()` creates a teammate and starts its agent loop in a thread.
|
||||
|
||||
```python
|
||||
def spawn(self, name: str, role: str, prompt: str) -> str:
|
||||
member = {"name": name, "role": role, "status": "working"}
|
||||
self.config["members"].append(member)
|
||||
self._save_config()
|
||||
thread = threading.Thread(
|
||||
target=self._teammate_loop,
|
||||
args=(name, role, prompt), daemon=True)
|
||||
thread.start()
|
||||
return f"Spawned teammate '{name}' (role: {role})"
|
||||
```
|
||||
|
||||
3. MessageBus: append-only JSONL inboxes. `send()` appends a JSON line; `read_inbox()` reads all and drains.
|
||||
|
||||
```python
|
||||
class MessageBus:
|
||||
def send(self, sender, to, content, msg_type="message", extra=None):
|
||||
msg = {"type": msg_type, "from": sender,
|
||||
"content": content, "timestamp": time.time()}
|
||||
if extra:
|
||||
msg.update(extra)
|
||||
with open(self.dir / f"{to}.jsonl", "a") as f:
|
||||
f.write(json.dumps(msg) + "\n")
|
||||
|
||||
def read_inbox(self, name):
|
||||
path = self.dir / f"{name}.jsonl"
|
||||
if not path.exists(): return "[]"
|
||||
msgs = [json.loads(l) for l in path.read_text().strip().splitlines() if l]
|
||||
path.write_text("") # drain
|
||||
return json.dumps(msgs, indent=2)
|
||||
```
|
||||
|
||||
4. Each teammate checks its inbox before every LLM call, injecting received messages into context.
|
||||
|
||||
```python
|
||||
def _teammate_loop(self, name, role, prompt):
|
||||
messages = [{"role": "user", "content": prompt}]
|
||||
for _ in range(50):
|
||||
inbox = BUS.read_inbox(name)
|
||||
if inbox != "[]":
|
||||
messages.append({"role": "user",
|
||||
"content": f"<inbox>{inbox}</inbox>"})
|
||||
response = client.messages.create(...)
|
||||
if response.stop_reason != "tool_use":
|
||||
break
|
||||
# execute tools, append results...
|
||||
self._find_member(name)["status"] = "idle"
|
||||
```
|
||||
|
||||
## What Changed From s08
|
||||
|
||||
| Component | Before (s08) | After (s09) |
|
||||
|----------------|------------------|----------------------------|
|
||||
| Tools | 6 | 9 (+spawn/send/read_inbox) |
|
||||
| Agents | Single | Lead + N teammates |
|
||||
| Persistence | None | config.json + JSONL inboxes|
|
||||
| Threads | Background cmds | Full agent loops per thread|
|
||||
| Lifecycle | Fire-and-forget | idle -> working -> idle |
|
||||
| Communication | None | message + broadcast |
|
||||
|
||||
## Try It
|
||||
|
||||
```sh
|
||||
cd learn-claude-code
|
||||
python agents/s09_agent_teams.py
|
||||
```
|
||||
|
||||
1. `Spawn alice (coder) and bob (tester). Have alice send bob a message.`
|
||||
2. `Broadcast "status update: phase 1 complete" to all teammates`
|
||||
3. `Check the lead inbox for any messages`
|
||||
4. Type `/team` to see the team roster with statuses
|
||||
5. Type `/inbox` to manually check the lead's inbox
|
||||
@@ -0,0 +1,106 @@
|
||||
# s10: Team Protocols
|
||||
|
||||
`s01 > s02 > s03 > s04 > s05 > s06 | s07 > s08 > s09 > [ s10 ] > s11 > s12`
|
||||
|
||||
> *"Teammates need shared communication rules"* -- one request-response pattern drives all negotiation.
|
||||
>
|
||||
> **Harness layer**: Protocols -- structured handshakes between models.
|
||||
|
||||
## Problem
|
||||
|
||||
In s09, teammates work and communicate but lack structured coordination:
|
||||
|
||||
**Shutdown**: Killing a thread leaves files half-written and config.json stale. You need a handshake: the lead requests, the teammate approves (finish and exit) or rejects (keep working).
|
||||
|
||||
**Plan approval**: When the lead says "refactor the auth module," the teammate starts immediately. For high-risk changes, the lead should review the plan first.
|
||||
|
||||
Both share the same structure: one side sends a request with a unique ID, the other responds referencing that ID.
|
||||
|
||||
## Solution
|
||||
|
||||
```
|
||||
Shutdown Protocol Plan Approval Protocol
|
||||
================== ======================
|
||||
|
||||
Lead Teammate Teammate Lead
|
||||
| | | |
|
||||
|--shutdown_req-->| |--plan_req------>|
|
||||
| {req_id:"abc"} | | {req_id:"xyz"} |
|
||||
| | | |
|
||||
|<--shutdown_resp-| |<--plan_resp-----|
|
||||
| {req_id:"abc", | | {req_id:"xyz", |
|
||||
| approve:true} | | approve:true} |
|
||||
|
||||
Shared FSM:
|
||||
[pending] --approve--> [approved]
|
||||
[pending] --reject---> [rejected]
|
||||
|
||||
Trackers:
|
||||
shutdown_requests = {req_id: {target, status}}
|
||||
plan_requests = {req_id: {from, plan, status}}
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
1. The lead initiates shutdown by generating a request_id and sending through the inbox.
|
||||
|
||||
```python
|
||||
shutdown_requests = {}
|
||||
|
||||
def handle_shutdown_request(teammate: str) -> str:
|
||||
req_id = str(uuid.uuid4())[:8]
|
||||
shutdown_requests[req_id] = {"target": teammate, "status": "pending"}
|
||||
BUS.send("lead", teammate, "Please shut down gracefully.",
|
||||
"shutdown_request", {"request_id": req_id})
|
||||
return f"Shutdown request {req_id} sent (status: pending)"
|
||||
```
|
||||
|
||||
2. The teammate receives the request and responds with approve/reject.
|
||||
|
||||
```python
|
||||
if tool_name == "shutdown_response":
|
||||
req_id = args["request_id"]
|
||||
approve = args["approve"]
|
||||
shutdown_requests[req_id]["status"] = "approved" if approve else "rejected"
|
||||
BUS.send(sender, "lead", args.get("reason", ""),
|
||||
"shutdown_response",
|
||||
{"request_id": req_id, "approve": approve})
|
||||
```
|
||||
|
||||
3. Plan approval follows the identical pattern. The teammate submits a plan (generating a request_id), the lead reviews (referencing the same request_id).
|
||||
|
||||
```python
|
||||
plan_requests = {}
|
||||
|
||||
def handle_plan_review(request_id, approve, feedback=""):
|
||||
req = plan_requests[request_id]
|
||||
req["status"] = "approved" if approve else "rejected"
|
||||
BUS.send("lead", req["from"], feedback,
|
||||
"plan_approval_response",
|
||||
{"request_id": request_id, "approve": approve})
|
||||
```
|
||||
|
||||
One FSM, two applications. The same `pending -> approved | rejected` state machine handles any request-response protocol.
|
||||
|
||||
## What Changed From s09
|
||||
|
||||
| Component | Before (s09) | After (s10) |
|
||||
|----------------|------------------|------------------------------|
|
||||
| Tools | 9 | 12 (+shutdown_req/resp +plan)|
|
||||
| Shutdown | Natural exit only| Request-response handshake |
|
||||
| Plan gating | None | Submit/review with approval |
|
||||
| Correlation | None | request_id per request |
|
||||
| FSM | None | pending -> approved/rejected |
|
||||
|
||||
## Try It
|
||||
|
||||
```sh
|
||||
cd learn-claude-code
|
||||
python agents/s10_team_protocols.py
|
||||
```
|
||||
|
||||
1. `Spawn alice as a coder. Then request her shutdown.`
|
||||
2. `List teammates to see alice's status after shutdown approval`
|
||||
3. `Spawn bob with a risky refactoring task. Review and reject his plan.`
|
||||
4. `Spawn charlie, have him submit a plan, then approve it.`
|
||||
5. Type `/team` to monitor statuses
|
||||
@@ -0,0 +1,142 @@
|
||||
# s11: Autonomous Agents
|
||||
|
||||
`s01 > s02 > s03 > s04 > s05 > s06 | s07 > s08 > s09 > s10 > [ s11 ] > s12`
|
||||
|
||||
> *"Teammates scan the board and claim tasks themselves"* -- no need for the lead to assign each one.
|
||||
>
|
||||
> **Harness layer**: Autonomy -- models that find work without being told.
|
||||
|
||||
## Problem
|
||||
|
||||
In s09-s10, teammates only work when explicitly told to. The lead must spawn each one with a specific prompt. 10 unclaimed tasks on the board? The lead assigns each one manually. Doesn't scale.
|
||||
|
||||
True autonomy: teammates scan the task board themselves, claim unclaimed tasks, work on them, then look for more.
|
||||
|
||||
One subtlety: after context compression (s06), the agent might forget who it is. Identity re-injection fixes this.
|
||||
|
||||
## Solution
|
||||
|
||||
```
|
||||
Teammate lifecycle with idle cycle:
|
||||
|
||||
+-------+
|
||||
| spawn |
|
||||
+---+---+
|
||||
|
|
||||
v
|
||||
+-------+ tool_use +-------+
|
||||
| WORK | <------------- | LLM |
|
||||
+---+---+ +-------+
|
||||
|
|
||||
| stop_reason != tool_use (or idle tool called)
|
||||
v
|
||||
+--------+
|
||||
| IDLE | poll every 5s for up to 60s
|
||||
+---+----+
|
||||
|
|
||||
+---> check inbox --> message? ----------> WORK
|
||||
|
|
||||
+---> scan .tasks/ --> unclaimed? -------> claim -> WORK
|
||||
|
|
||||
+---> 60s timeout ----------------------> SHUTDOWN
|
||||
|
||||
Identity re-injection after compression:
|
||||
if len(messages) <= 3:
|
||||
messages.insert(0, identity_block)
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
1. The teammate loop has two phases: WORK and IDLE. When the LLM stops calling tools (or calls `idle`), the teammate enters IDLE.
|
||||
|
||||
```python
|
||||
def _loop(self, name, role, prompt):
|
||||
while True:
|
||||
# -- WORK PHASE --
|
||||
messages = [{"role": "user", "content": prompt}]
|
||||
for _ in range(50):
|
||||
response = client.messages.create(...)
|
||||
if response.stop_reason != "tool_use":
|
||||
break
|
||||
# execute tools...
|
||||
if idle_requested:
|
||||
break
|
||||
|
||||
# -- IDLE PHASE --
|
||||
self._set_status(name, "idle")
|
||||
resume = self._idle_poll(name, messages)
|
||||
if not resume:
|
||||
self._set_status(name, "shutdown")
|
||||
return
|
||||
self._set_status(name, "working")
|
||||
```
|
||||
|
||||
2. The idle phase polls inbox and task board in a loop.
|
||||
|
||||
```python
|
||||
def _idle_poll(self, name, messages):
|
||||
for _ in range(IDLE_TIMEOUT // POLL_INTERVAL): # 60s / 5s = 12
|
||||
time.sleep(POLL_INTERVAL)
|
||||
inbox = BUS.read_inbox(name)
|
||||
if inbox:
|
||||
messages.append({"role": "user",
|
||||
"content": f"<inbox>{inbox}</inbox>"})
|
||||
return True
|
||||
unclaimed = scan_unclaimed_tasks()
|
||||
if unclaimed:
|
||||
claim_task(unclaimed[0]["id"], name)
|
||||
messages.append({"role": "user",
|
||||
"content": f"<auto-claimed>Task #{unclaimed[0]['id']}: "
|
||||
f"{unclaimed[0]['subject']}</auto-claimed>"})
|
||||
return True
|
||||
return False # timeout -> shutdown
|
||||
```
|
||||
|
||||
3. Task board scanning: find pending, unowned, unblocked tasks.
|
||||
|
||||
```python
|
||||
def scan_unclaimed_tasks() -> list:
|
||||
unclaimed = []
|
||||
for f in sorted(TASKS_DIR.glob("task_*.json")):
|
||||
task = json.loads(f.read_text())
|
||||
if (task.get("status") == "pending"
|
||||
and not task.get("owner")
|
||||
and not task.get("blockedBy")):
|
||||
unclaimed.append(task)
|
||||
return unclaimed
|
||||
```
|
||||
|
||||
4. Identity re-injection: when context is too short (compression happened), insert an identity block.
|
||||
|
||||
```python
|
||||
if len(messages) <= 3:
|
||||
messages.insert(0, {"role": "user",
|
||||
"content": f"<identity>You are '{name}', role: {role}, "
|
||||
f"team: {team_name}. Continue your work.</identity>"})
|
||||
messages.insert(1, {"role": "assistant",
|
||||
"content": f"I am {name}. Continuing."})
|
||||
```
|
||||
|
||||
## What Changed From s10
|
||||
|
||||
| Component | Before (s10) | After (s11) |
|
||||
|----------------|------------------|----------------------------|
|
||||
| Tools | 12 | 14 (+idle, +claim_task) |
|
||||
| Autonomy | Lead-directed | Self-organizing |
|
||||
| Idle phase | None | Poll inbox + task board |
|
||||
| Task claiming | Manual only | Auto-claim unclaimed tasks |
|
||||
| Identity | System prompt | + re-injection after compress|
|
||||
| Timeout | None | 60s idle -> auto shutdown |
|
||||
|
||||
## Try It
|
||||
|
||||
```sh
|
||||
cd learn-claude-code
|
||||
python agents/s11_autonomous_agents.py
|
||||
```
|
||||
|
||||
1. `Create 3 tasks on the board, then spawn alice and bob. Watch them auto-claim.`
|
||||
2. `Spawn a coder teammate and let it find work from the task board itself`
|
||||
3. `Create tasks with dependencies. Watch teammates respect the blocked order.`
|
||||
4. Type `/tasks` to see the task board with owners
|
||||
5. Type `/team` to monitor who is working vs idle
|
||||
@@ -0,0 +1,121 @@
|
||||
# s12: Worktree + Task Isolation
|
||||
|
||||
`s01 > s02 > s03 > s04 > s05 > s06 | s07 > s08 > s09 > s10 > s11 > [ s12 ]`
|
||||
|
||||
> *"Each works in its own directory, no interference"* -- tasks manage goals, worktrees manage directories, bound by ID.
|
||||
>
|
||||
> **Harness layer**: Directory isolation -- parallel execution lanes that never collide.
|
||||
|
||||
## Problem
|
||||
|
||||
By s11, agents can claim and complete tasks autonomously. But every task runs in one shared directory. Two agents refactoring different modules at the same time will collide: agent A edits `config.py`, agent B edits `config.py`, unstaged changes mix, and neither can roll back cleanly.
|
||||
|
||||
The task board tracks *what to do* but has no opinion about *where to do it*. The fix: give each task its own git worktree directory. Tasks manage goals, worktrees manage execution context. Bind them by task ID.
|
||||
|
||||
## Solution
|
||||
|
||||
```
|
||||
Control plane (.tasks/) Execution plane (.worktrees/)
|
||||
+------------------+ +------------------------+
|
||||
| task_1.json | | auth-refactor/ |
|
||||
| status: in_progress <------> branch: wt/auth-refactor
|
||||
| worktree: "auth-refactor" | task_id: 1 |
|
||||
+------------------+ +------------------------+
|
||||
| task_2.json | | ui-login/ |
|
||||
| status: pending <------> branch: wt/ui-login
|
||||
| worktree: "ui-login" | task_id: 2 |
|
||||
+------------------+ +------------------------+
|
||||
|
|
||||
index.json (worktree registry)
|
||||
events.jsonl (lifecycle log)
|
||||
|
||||
State machines:
|
||||
Task: pending -> in_progress -> completed
|
||||
Worktree: absent -> active -> removed | kept
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
1. **Create a task.** Persist the goal first.
|
||||
|
||||
```python
|
||||
TASKS.create("Implement auth refactor")
|
||||
# -> .tasks/task_1.json status=pending worktree=""
|
||||
```
|
||||
|
||||
2. **Create a worktree and bind to the task.** Passing `task_id` auto-advances the task to `in_progress`.
|
||||
|
||||
```python
|
||||
WORKTREES.create("auth-refactor", task_id=1)
|
||||
# -> git worktree add -b wt/auth-refactor .worktrees/auth-refactor HEAD
|
||||
# -> index.json gets new entry, task_1.json gets worktree="auth-refactor"
|
||||
```
|
||||
|
||||
The binding writes state to both sides:
|
||||
|
||||
```python
|
||||
def bind_worktree(self, task_id, worktree):
|
||||
task = self._load(task_id)
|
||||
task["worktree"] = worktree
|
||||
if task["status"] == "pending":
|
||||
task["status"] = "in_progress"
|
||||
self._save(task)
|
||||
```
|
||||
|
||||
3. **Run commands in the worktree.** `cwd` points to the isolated directory.
|
||||
|
||||
```python
|
||||
subprocess.run(command, shell=True, cwd=worktree_path,
|
||||
capture_output=True, text=True, timeout=300)
|
||||
```
|
||||
|
||||
4. **Close out.** Two choices:
|
||||
- `worktree_keep(name)` -- preserve the directory for later.
|
||||
- `worktree_remove(name, complete_task=True)` -- remove directory, complete the bound task, emit event. One call handles teardown + completion.
|
||||
|
||||
```python
|
||||
def remove(self, name, force=False, complete_task=False):
|
||||
self._run_git(["worktree", "remove", wt["path"]])
|
||||
if complete_task and wt.get("task_id") is not None:
|
||||
self.tasks.update(wt["task_id"], status="completed")
|
||||
self.tasks.unbind_worktree(wt["task_id"])
|
||||
self.events.emit("task.completed", ...)
|
||||
```
|
||||
|
||||
5. **Event stream.** Every lifecycle step emits to `.worktrees/events.jsonl`:
|
||||
|
||||
```json
|
||||
{
|
||||
"event": "worktree.remove.after",
|
||||
"task": {"id": 1, "status": "completed"},
|
||||
"worktree": {"name": "auth-refactor", "status": "removed"},
|
||||
"ts": 1730000000
|
||||
}
|
||||
```
|
||||
|
||||
Events emitted: `worktree.create.before/after/failed`, `worktree.remove.before/after/failed`, `worktree.keep`, `task.completed`.
|
||||
|
||||
After a crash, state reconstructs from `.tasks/` + `.worktrees/index.json` on disk. Conversation memory is volatile; file state is durable.
|
||||
|
||||
## What Changed From s11
|
||||
|
||||
| Component | Before (s11) | After (s12) |
|
||||
|--------------------|----------------------------|----------------------------------------------|
|
||||
| Coordination | Task board (owner/status) | Task board + explicit worktree binding |
|
||||
| Execution scope | Shared directory | Task-scoped isolated directory |
|
||||
| Recoverability | Task status only | Task status + worktree index |
|
||||
| Teardown | Task completion | Task completion + explicit keep/remove |
|
||||
| Lifecycle visibility | Implicit in logs | Explicit events in `.worktrees/events.jsonl` |
|
||||
|
||||
## Try It
|
||||
|
||||
```sh
|
||||
cd learn-claude-code
|
||||
python agents/s12_worktree_task_isolation.py
|
||||
```
|
||||
|
||||
1. `Create tasks for backend auth and frontend login page, then list tasks.`
|
||||
2. `Create worktree "auth-refactor" for task 1, then bind task 2 to a new worktree "ui-login".`
|
||||
3. `Run "git status --short" in worktree "auth-refactor".`
|
||||
4. `Keep worktree "ui-login", then list worktrees and inspect events.`
|
||||
5. `Remove worktree "auth-refactor" with complete_task=true, then list tasks/worktrees/events.`
|
||||
Reference in New Issue
Block a user