wehub/hkuds--autoagent
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:06:23 +08:00
commit ea5096c873
282 changed files with 55366 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.template
+11
View File
@@ -0,0 +1,11 @@
# Required Github Tokens
GITHUB_AI_TOKEN=
# Optional API Keys
OPENAI_API_KEY=
DEEPSEEK_API_KEY=
ANTHROPIC_API_KEY=
GEMINI_API_KEY=
HUGGINGFACE_API_KEY=
GROQ_API_KEY=
XAI_API_KEY=
.gitattributes
+1
View File
@@ -0,0 +1 @@
# *.mp4 filter=lfs diff=lfs merge=lfs -text
.gitignore
+31
View File
@@ -0,0 +1,31 @@
workplace_*/
workspace_*/
*.log
code_db/*
results/*
__pycache__/
tmp/*
logs/*
*.tar.gz
*.egg-info
.DS_Store
*.csv
eval_data/*
evaluation_results/*
casestudy_results/*
evaluation/*/data/
evaluation/*/data/*
evaluation/**/data/
.env
terminal_tmp/*
!tool_docs.csv
.port*
Communication.md
+6
View File
@@ -0,0 +1,6 @@
We provide QR codes for joining the HKUDS discussion groups on WeChat and Feishu.
You can join by scanning the QR codes below:
<img src="https://github.com/HKUDS/.github/blob/main/profile/QR.png" alt="WeChat QR Code" width="400"/>
LICENSE
+25
View File
@@ -0,0 +1,25 @@
The MIT License (MIT)
=====================
Copyright © 2023
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the “Software”), to deal in the Software without
restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
README.md
+527
View File
@@ -0,0 +1,527 @@
<a name="readme-top"></a>
<div align="center">
<img src="./assets/AutoAgent_logo.svg" alt="Logo" width="200">
<h1 align="center">AutoAgent: Fully-Automated & Zero-Code</br> LLM Agent Framework </h1>
</div>
<div align="center">
<a href="https://autoagent-ai.github.io"><img src="https://img.shields.io/badge/Project-Page-blue?style=for-the-badge&color=FFE165&logo=homepage&logoColor=white" alt="Credits"></a>
<a href="https://join.slack.com/t/metachain-workspace/shared_invite/zt-2zibtmutw-v7xOJObBf9jE2w3x7nctFQ"><img src="https://img.shields.io/badge/Slack-Join%20Us-red?logo=slack&logoColor=white&style=for-the-badge" alt="Join our Slack community"></a>
<a href="https://discord.gg/jQJdXyDB"><img src="https://img.shields.io/badge/Discord-Join%20Us-purple?logo=discord&logoColor=white&style=for-the-badge" alt="Join our Discord community"></a>
<!-- <a href="https://github.com/HKUDS/AutoAgent/blob/main/assets/autoagent-wechat.jpg"><img src="https://img.shields.io/badge/Wechat-Join%20Us-green?logo=wechat&logoColor=white&style=for-the-badge" alt="Join our Wechat community"></a> -->
<a href="./Communication.md"><img src="https://img.shields.io/badge/💬Feishu-Group-07c160?style=for-the-badge&logoColor=white&labelColor=1a1a2e"></a>
<a href="./Communication.md"><img src="https://img.shields.io/badge/WeChat-Group-07c160?style=for-the-badge&logo=wechat&logoColor=white&labelColor=1a1a2e"></a>
<br/>
<a href="https://autoagent-ai.github.io/docs"><img src="https://img.shields.io/badge/Documentation-000?logo=googledocs&logoColor=FFE165&style=for-the-badge" alt="Check out the documentation"></a>
<a href="https://arxiv.org/abs/2502.05957"><img src="https://img.shields.io/badge/Paper%20on%20Arxiv-000?logoColor=FFE165&logo=arxiv&style=for-the-badge" alt="Paper"></a>
<a href="https://gaia-benchmark-leaderboard.hf.space/"><img src="https://img.shields.io/badge/GAIA%20Benchmark-000?logoColor=FFE165&logo=huggingface&style=for-the-badge" alt="Evaluation Benchmark Score"></a>
<hr>
</div>
<div align="center">
<a href="https://trendshift.io/repositories/13954" target="_blank"><img src="https://trendshift.io/api/badge/repositories/13954" alt="HKUDS%2FAutoAgent | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
</div>
Welcome to AutoAgent! AutoAgent is a **Fully-Automated** and highly **Self-Developing** framework that enables users to create and deploy LLM agents through **Natural Language Alone**.
## ✨Key Features of AutoAgent
* 💬 **Natural Language-Driven Agent Building**
</br>Automatically constructs and orchestrates collaborative agent systems purely through natural dialogue, eliminating the need for manual coding or technical configuration.
* 🚀 **Zero-Code Framework**
</br>Democratizes AI development by allowing anyone, regardless of coding experience, to create and customize their own agents, tools, and workflows using natural language alone.
* ⚡ **Self-Managing Workflow Generation**
</br>Dynamically creates, optimizes and adapts agent workflows based on high-level task descriptions, even when users cannot fully specify implementation details.
* 🔧 **Intelligent Resource Orchestration**
</br>Enables controlled code generation for creating tools, agents, and workflows through iterative self-improvement, supporting both single agent creation and multi-agent workflow generation.
* 🎯 **Self-Play Agent Customization**
</br>Enables controlled code generation for creating tools, agents, and workflows through iterative self-improvement, supporting both single agent creation and multi-agent workflow generation.
🚀 Unlock the Future of LLM Agents. Try 🔥AutoAgent🔥 Now!
<div align="center">
<!-- <img src="./assets/AutoAgentnew-intro.pdf" alt="Logo" width="100%"> -->
<figure>
<img src="./assets/autoagent-intro.svg" alt="Logo" style="max-width: 100%; height: auto;">
<figcaption><em>Quick Overview of AutoAgent.</em></figcaption>
</figure>
</div>
## 🔥 News
<div class="scrollable">
<ul>
<li><strong>[2025, Feb 17]</strong>: &nbsp;🎉🎉We've updated and released AutoAgent v0.2.0 (formerly known as MetaChain). Detailed changes include: 1) fix the bug of different LLM providers from issues; 2) add automatic installation of AutoAgent in the container environment according to issues; 3) add more easy-to-use commands for the CLI mode. 4) Rename the project to AutoAgent for better understanding.</li>
<li><strong>[2025, Feb 10]</strong>: &nbsp;🎉🎉We've released <b>MetaChain!</b>, including framework, evaluation codes and CLI mode! Check our <a href="https://arxiv.org/abs/2502.05957">paper</a> for more details.</li>
</ul>
</div>
<span id='table-of-contents'/>
## 📑 Table of Contents
* <a href='#features'>✨ Features</a>
* <a href='#news'>🔥 News</a>
* <a href='#how-to-use'>🔍 How to Use AutoAgent</a>
* <a href='#user-mode'>1. `user mode` (Deep Research Agents)</a>
* <a href='#agent-editor'>2. `agent editor` (Agent Creation without Workflow)</a>
* <a href='#workflow-editor'>3. `workflow editor` (Agent Creation with Workflow)</a>
* <a href='#quick-start'>⚡ Quick Start</a>
* <a href='#installation'>Installation</a>
* <a href='#api-keys-setup'>API Keys Setup</a>
* <a href='#start-with-cli-mode'>Start with CLI Mode</a>
* <a href='#todo'>☑️ Todo List</a>
* <a href='#reproduce'>🔬 How To Reproduce the Results in the Paper</a>
* <a href='#documentation'>📖 Documentation</a>
* <a href='#community'>🤝 Join the Community</a>
* <a href='#acknowledgements'>🙏 Acknowledgements</a>
* <a href='#cite'>🌟 Cite</a>
<span id='how-to-use'/>
## 🔍 How to Use AutoAgent
<span id='user-mode'/>
### 1. `user mode` (Deep Research Agents)
AutoAgent features a ready-to-use multi-agent system accessible through user mode on the start page. This system serves as a comprehensive AI research assistant designed for information retrieval, complex analytical tasks, and comprehensive report generation.
- 🚀 **High Performance**: Matches Deep Research using Claude 3.5 rather than OpenAI's o3 model.
- 🔄 **Model Flexibility**: Compatible with any LLM (including Deepseek-R1, Grok, Gemini, etc.)
- 💰 **Cost-Effective**: Open-source alternative to Deep Research's $200/month subscription
- 🎯 **User-Friendly**: Easy-to-deploy CLI interface for seamless interaction
- 📁 **File Support**: Handles file uploads for enhanced data interaction
<div align="center">
<video width="80%" controls>
<source src="./assets/video_v1_compressed.mp4" type="video/mp4">
</video>
<p><em>🎥 Deep Research (aka User Mode)</em></p>
</div>
<span id='agent-editor'/>
### 2. `agent editor` (Agent Creation without Workflow)
The most distinctive feature of AutoAgent is its natural language customization capability. Unlike other agent frameworks, AutoAgent allows you to create tools, agents, and workflows using natural language alone. Simply choose `agent editor` or `workflow editor` mode to start your journey of building agents through conversations.
You can use `agent editor` as shown in the following figure.
<table>
<tr align="center">
<td width="33%">
<img src="./assets/agent_editor/1-requirement.png" alt="requirement" width="100%"/>
<br>
<em>Input what kind of agent you want to create.</em>
</td>
<td width="33%">
<img src="./assets/agent_editor/2-profiling.png" alt="profiling" width="100%"/>
<br>
<em>Automated agent profiling.</em>
</td>
<td width="33%">
<img src="./assets/agent_editor/3-profiles.png" alt="profiles" width="100%"/>
<br>
<em>Output the agent profiles.</em>
</td>
</tr>
</table>
<table>
<tr align="center">
<td width="33%">
<img src="./assets/agent_editor/4-tools.png" alt="tools" width="100%"/>
<br>
<em>Create the desired tools.</em>
</td>
<td width="33%">
<img src="./assets/agent_editor/5-task.png" alt="task" width="100%"/>
<br>
<em>Input what do you want to complete with the agent. (Optional)</em>
</td>
<td width="33%">
<img src="./assets/agent_editor/6-output-next.png" alt="output" width="100%"/>
<br>
<em>Create the desired agent(s) and go to the next step.</em>
</td>
</tr>
</table>
<span id='workflow-editor'/>
### 3. `workflow editor` (Agent Creation with Workflow)
You can also create the agent workflows using natural language description with the `workflow editor` mode, as shown in the following figure. (Tips: this mode does not support tool creation temporarily.)
<table>
<tr align="center">
<td width="33%">
<img src="./assets/workflow_editor/1-requirement.png" alt="requirement" width="100%"/>
<br>
<em>Input what kind of workflow you want to create.</em>
</td>
<td width="33%">
<img src="./assets/workflow_editor/2-profiling.png" alt="profiling" width="100%"/>
<br>
<em>Automated workflow profiling.</em>
</td>
<td width="33%">
<img src="./assets/workflow_editor/3-profiles.png" alt="profiles" width="100%"/>
<br>
<em>Output the workflow profiles.</em>
</td>
</tr>
</table>
<table>
<tr align="center">
<td width="33%">
<img src="./assets/workflow_editor/4-task.png" alt="task" width="66%"/>
<br>
<em>Input what do you want to complete with the workflow. (Optional)</em>
</td>
<td width="33%">
<img src="./assets/workflow_editor/5-output-next.png" alt="output" width="66%"/>
<br>
<em>Create the desired workflow(s) and go to the next step.</em>
</td>
</tr>
</table>
<span id='quick-start'/>
## ⚡ Quick Start
<span id='installation'/>
### Installation
#### AutoAgent Installation
```bash
git clone https://github.com/HKUDS/AutoAgent.git
cd AutoAgent
pip install -e .
```
#### Docker Installation
We use Docker to containerize the agent-interactive environment. So please install [Docker](https://www.docker.com/) first. You don't need to manually pull the pre-built image, because we have let Auto-Deep-Research **automatically pull the pre-built image based on your architecture of your machine**.
<span id='api-keys-setup'/>
### API Keys Setup
Create an environment variable file, just like `.env.template`, and set the API keys for the LLMs you want to use. Not every LLM API Key is required, use what you need.
```bash
# Required Github Tokens of your own
GITHUB_AI_TOKEN=
# Optional API Keys
OPENAI_API_KEY=
DEEPSEEK_API_KEY=
ANTHROPIC_API_KEY=
GEMINI_API_KEY=
HUGGINGFACE_API_KEY=
GROQ_API_KEY=
XAI_API_KEY=
```
<span id='start-with-cli-mode'/>
### Start with CLI Mode
> [🚨 **News**: ] We have updated a more easy-to-use command to start the CLI mode and fix the bug of different LLM providers from issues. You can follow the following steps to start the CLI mode with different LLM providers with much less configuration.
#### Command Options:
You can run `auto main` to start full part of AutoAgent, including `user mode`, `agent editor` and `workflow editor`. Btw, you can also run `auto deep-research` to start more lightweight `user mode`, just like the [Auto-Deep-Research](https://github.com/HKUDS/Auto-Deep-Research) project. Some configuration of this command is shown below.
- `--container_name`: Name of the Docker container (default: 'deepresearch')
- `--port`: Port for the container (default: 12346)
- `COMPLETION_MODEL`: Specify the LLM model to use, you should follow the name of [Litellm](https://github.com/BerriAI/litellm) to set the model name. (Default: `claude-3-5-sonnet-20241022`)
- `DEBUG`: Enable debug mode for detailed logs (default: False)
- `API_BASE_URL`: The base URL for the LLM provider (default: None)
- `FN_CALL`: Enable function calling (default: None). Most of time, you could ignore this option because we have already set the default value based on the model name.
- `git_clone`: Clone the AutoAgent repository to the local environment (only support with the `auto main` command, default: True)
- `test_pull_name`: The name of the test pull. (only support with the `auto main` command, default: 'autoagent_mirror')
#### More details about `git_clone` and `test_pull_name`]
In the `agent editor` and `workflow editor` mode, we should clone a mirror of the AutoAgent repository to the local agent-interactive environment and let our **AutoAgent** automatically update the AutoAgent itself, such as creating new tools, agents and workflows. So if you want to use the `agent editor` and `workflow editor` mode, you should set the `git_clone` to True and set the `test_pull_name` to 'autoagent_mirror' or other branches.
#### `auto main` with different LLM Providers
Then I will show you how to use the full part of AutoAgent with the `auto main` command and different LLM providers. If you want to use the `auto deep-research` command, you can refer to the [Auto-Deep-Research](https://github.com/HKUDS/Auto-Deep-Research) project for more details.
##### Anthropic
* set the `ANTHROPIC_API_KEY` in the `.env` file.
```bash
ANTHROPIC_API_KEY=your_anthropic_api_key
```
* run the following command to start Auto-Deep-Research.
```bash
auto main # default model is claude-3-5-sonnet-20241022
```
##### OpenAI
* set the `OPENAI_API_KEY` in the `.env` file.
```bash
OPENAI_API_KEY=your_openai_api_key
```
* run the following command to start Auto-Deep-Research.
```bash
COMPLETION_MODEL=gpt-4o auto main
```
##### Mistral
* set the `MISTRAL_API_KEY` in the `.env` file.
```bash
MISTRAL_API_KEY=your_mistral_api_key
```
* run the following command to start Auto-Deep-Research.
```bash
COMPLETION_MODEL=mistral/mistral-large-2407 auto main
```
##### Gemini - Google AI Studio
* set the `GEMINI_API_KEY` in the `.env` file.
```bash
GEMINI_API_KEY=your_gemini_api_key
```
* run the following command to start Auto-Deep-Research.
```bash
COMPLETION_MODEL=gemini/gemini-2.0-flash auto main
```
##### Huggingface
* set the `HUGGINGFACE_API_KEY` in the `.env` file.
```bash
HUGGINGFACE_API_KEY=your_huggingface_api_key
```
* run the following command to start Auto-Deep-Research.
```bash
COMPLETION_MODEL=huggingface/meta-llama/Llama-3.3-70B-Instruct auto main
```
##### Groq
* set the `GROQ_API_KEY` in the `.env` file.
```bash
GROQ_API_KEY=your_groq_api_key
```
* run the following command to start Auto-Deep-Research.
```bash
COMPLETION_MODEL=groq/deepseek-r1-distill-llama-70b auto main
```
##### OpenAI-Compatible Endpoints (e.g., Grok)
* set the `OPENAI_API_KEY` in the `.env` file.
```bash
OPENAI_API_KEY=your_api_key_for_openai_compatible_endpoints
```
* run the following command to start Auto-Deep-Research.
```bash
COMPLETION_MODEL=openai/grok-2-latest API_BASE_URL=https://api.x.ai/v1 auto main
```
##### OpenRouter (e.g., DeepSeek-R1)
We recommend using OpenRouter as LLM provider of DeepSeek-R1 temporarily. Because official API of DeepSeek-R1 can not be used efficiently.
* set the `OPENROUTER_API_KEY` in the `.env` file.
```bash
OPENROUTER_API_KEY=your_openrouter_api_key
```
* run the following command to start Auto-Deep-Research.
```bash
COMPLETION_MODEL=openrouter/deepseek/deepseek-r1 auto main
```
##### DeepSeek
* set the `DEEPSEEK_API_KEY` in the `.env` file.
```bash
DEEPSEEK_API_KEY=your_deepseek_api_key
```
* run the following command to start Auto-Deep-Research.
```bash
COMPLETION_MODEL=deepseek/deepseek-chat auto main
```
After the CLI mode is started, you can see the start page of AutoAgent:
<div align="center">
<!-- <img src="./assets/AutoAgentnew-intro.pdf" alt="Logo" width="100%"> -->
<figure>
<img src="./assets/cover.png" alt="Logo" style="max-width: 100%; height: auto;">
<figcaption><em>Start Page of AutoAgent.</em></figcaption>
</figure>
</div>
### Tips
#### Import browser cookies to browser environment
You can import the browser cookies to the browser environment to let the agent better access some specific websites. For more details, please refer to the [cookies](./AutoAgent/environment/cookie_json/README.md) folder.
#### Add your own API keys for third-party Tool Platforms
If you want to create tools from the third-party tool platforms, such as RapidAPI, you should subscribe tools from the platform and add your own API keys by running [process_tool_docs.py](./process_tool_docs.py).
```bash
python process_tool_docs.py
```
More features coming soon! 🚀 **Web GUI interface** under development.
<span id='todo'/>
## ☑️ Todo List
AutoAgent is continuously evolving! Here's what's coming:
- 📊 **More Benchmarks**: Expanding evaluations to **SWE-bench**, **WebArena**, and more
- 🖥️ **GUI Agent**: Supporting *Computer-Use* agents with GUI interaction
- 🔧 **Tool Platforms**: Integration with more platforms like **Composio**
- 🏗️ **Code Sandboxes**: Supporting additional environments like **E2B**
- 🎨 **Web Interface**: Developing comprehensive GUI for better user experience
Have ideas or suggestions? Feel free to open an issue! Stay tuned for more exciting updates! 🚀
<span id='reproduce'/>
## 🔬 How To Reproduce the Results in the Paper
### GAIA Benchmark
For the GAIA benchmark, you can run the following command to run the inference.
```bash
cd path/to/AutoAgent && sh evaluation/gaia/scripts/run_infer.sh
```
For the evaluation, you can run the following command.
```bash
cd path/to/AutoAgent && python evaluation/gaia/get_score.py
```
### Agentic-RAG
For the Agentic-RAG task, you can run the following command to run the inference.
Step1. Turn to [this page](https://huggingface.co/datasets/yixuantt/MultiHopRAG) and download it. Save them to your datapath.
Step2. Run the following command to run the inference.
```bash
cd path/to/AutoAgent && sh evaluation/multihoprag/scripts/run_rag.sh
```
Step3. The result will be saved in the `evaluation/multihoprag/result.json`.
<span id='documentation'/>
## 📖 Documentation
A more detailed documentation is coming soon 🚀, and we will update in the [Documentation](https://AutoAgent-ai.github.io/docs) page.
<span id='community'/>
## 🤝 Join the Community
We want to build a community for AutoAgent, and we welcome everyone to join us. You can join our community by:
- [Join our Slack workspace](https://join.slack.com/t/AutoAgent-workspace/shared_invite/zt-2zibtmutw-v7xOJObBf9jE2w3x7nctFQ) - Here we talk about research, architecture, and future development.
- [Join our Discord server](https://discord.gg/z68KRvwB) - This is a community-run server for general discussion, questions, and feedback.
- [Read or post Github Issues](https://github.com/HKUDS/AutoAgent/issues) - Check out the issues we're working on, or add your own ideas.
<span id='acknowledgements'/>
## Misc
<div align="center">
[![Stargazers repo roster for @HKUDS/AutoAgent](https://reporoster.com/stars/HKUDS/AutoAgent)](https://github.com/HKUDS/AutoAgent/stargazers)
[![Forkers repo roster for @HKUDS/AutoAgent](https://reporoster.com/forks/HKUDS/AutoAgent)](https://github.com/HKUDS/AutoAgent/network/members)
[![Star History Chart](https://api.star-history.com/svg?repos=HKUDS/AutoAgent&type=Date)](https://star-history.com/#HKUDS/AutoAgent&Date)
</div>
## 🙏 Acknowledgements
Rome wasn't built in a day. AutoAgent stands on the shoulders of giants, and we are deeply grateful for the outstanding work that came before us. Our framework architecture draws inspiration from [OpenAI Swarm](https://github.com/openai/swarm), while our user mode's three-agent design benefits from [Magentic-one](https://github.com/microsoft/autogen/tree/main/python/packages/autogen-magentic-one)'s insights. We've also learned from [OpenHands](https://github.com/All-Hands-AI/OpenHands) for documentation structure and many other excellent projects for agent-environment interaction design, among others. We express our sincere gratitude and respect to all these pioneering works that have been instrumental in shaping AutoAgent.
<span id='cite'/>
## 🌟 Cite
```tex
@misc{AutoAgent,
title={{AutoAgent: A Fully-Automated and Zero-Code Framework for LLM Agents}},
author={Jiabin Tang, Tianyu Fan, Chao Huang},
year={2025},
eprint={202502.05957},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2502.05957},
}
```
README.wehub.md
+7
View File
@@ -0,0 +1,7 @@
# WeHub 来源说明
- 原始项目：`HKUDS/AutoAgent`
- 原始仓库：https://github.com/HKUDS/AutoAgent
- 导入方式：上游默认分支的最新快照
- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准
- 本文件仅用于记录来源，不代表 WeHub 是原项目作者
assets/AutoAgent_Arxiv_2025.pdf
BIN
View File
Binary file not shown.
assets/AutoAgent_logo.svg
+4
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 616 KiB

assets/QR.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 602 KiB

assets/agent_editor/1-requirement.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 609 KiB

assets/agent_editor/2-profiling.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 651 KiB

assets/agent_editor/3-profiles.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 700 KiB

assets/agent_editor/4-tools.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 621 KiB

assets/agent_editor/5-task.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 630 KiB

assets/agent_editor/6-output-next.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 611 KiB

assets/arrow.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.8 KiB

assets/autoagent-intro-final.pdf
BIN
View File
Binary file not shown.
assets/autoagent-intro.pdf
BIN
View File
Binary file not shown.
assets/autoagent-intro.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 4.2 MiB

assets/autoagent-intro.svg
+4
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 12 MiB

assets/autoagent-wechat.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 247 KiB

assets/autoagent.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 406 KiB

assets/cookies/export.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 213 KiB

assets/cookies/extension.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 853 KiB

assets/cover.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 481 KiB

assets/paper/metachain_arxiv.pdf
BIN
View File
Binary file not shown.
assets/rec.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 5.4 KiB

assets/technology-278512.mp3
BIN
View File
Binary file not shown.
assets/user_mode/4-upload.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 589 KiB

assets/user_mode/5-select.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 548 KiB

assets/user_mode/6-success.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 438 KiB

assets/user_mode/input.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 570 KiB

assets/user_mode/output.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 718 KiB

assets/user_mode/select_agent.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 588 KiB

assets/video_v1_compressed.mp4
BIN
View File
Binary file not shown.
assets/workflow_editor/1-requirement.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 499 KiB

assets/workflow_editor/2-profiling.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 689 KiB

assets/workflow_editor/3-profiles.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 564 KiB

assets/workflow_editor/4-task.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 580 KiB

assets/workflow_editor/5-output-next.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 577 KiB

autoagent/__init__.py
+9
View File
@@ -0,0 +1,9 @@
from .core import MetaChain
from .types import Agent, Response
# from .workflow import Graph, meta_workflow, FlowEngine
from .flow import default_drive
import autoagent.workflows
import autoagent.tools
import autoagent.agents
__all__ = ["MetaChain", "Agent", "Response", "default_drive", ]
autoagent/agents/__init__.py
+64
View File
@@ -0,0 +1,64 @@
# from autoagent.agents.programming_agent import get_programming_agent
# from autoagent.agents.tool_retriver_agent import get_tool_retriver_agent
# from autoagent.agents.agent_check_agent import get_agent_check_agent
# from autoagent.agents.tool_check_agent import get_tool_check_agent
# from autoagent.agents.github_agent import get_github_agent
# from autoagent.agents.programming_triage_agent import get_programming_triage_agent
# from autoagent.agents.plan_agent import get_plan_agent
# import os
# import importlib
# from autoagent.registry import registry
# # 获取当前目录下的所有 .py 文件
# current_dir = os.path.dirname(__file__)
# for file in os.listdir(current_dir):
# if file.endswith('.py') and not file.startswith('__'):
# module_name = file[:-3]
# importlib.import_module(f'autoagent.agents.{module_name}')
# # 导出所有注册的 agent 创建函数
# globals().update(registry.agents)
# __all__ = list(registry.agents.keys())
import os
import importlib
from autoagent.registry import registry
def import_agents_recursively(base_dir: str, base_package: str):
"""Recursively import all agents in .py files
Args:
base_dir: the root directory to start searching
base_package: the base name of the Python package
"""
for root, dirs, files in os.walk(base_dir):
# get the relative path to the base directory
rel_path = os.path.relpath(root, base_dir)
for file in files:
if file.endswith('.py') and not file.startswith('__'):
# build the module path
if rel_path == '.':
# in the root directory
module_path = f"{base_package}.{file[:-3]}"
else:
# in the subdirectory
package_path = rel_path.replace(os.path.sep, '.')
module_path = f"{base_package}.{package_path}.{file[:-3]}"
try:
importlib.import_module(module_path)
except Exception as e:
print(f"Warning: Failed to import {module_path}: {e}")
# get the current directory and import all agents
current_dir = os.path.dirname(__file__)
import_agents_recursively(current_dir, 'autoagent.agents')
# export all agent creation functions
globals().update(registry.agents)
globals().update(registry.plugin_agents)
__all__ = list(registry.agents.keys())
autoagent/agents/dummy_agent.py
+36
View File
@@ -0,0 +1,36 @@
from autoagent.types import Agent
from autoagent.tools import tool_dummy
from typing import Union
from autoagent.registry import register_plugin_agent # import the register_agent function from the registry
@register_plugin_agent(name = "Dummy Agent", func_name="get_dummy_agent") # You must register the agent in the registry, otherwise the agent will not be loaded. The name of register_agent is get_xxx_agent.
def get_dummy_agent(model: str):
"""
This is a dummy agent, it's used for demonstrating the usage of the autoagent.
Args:
model: The model to be used for the agent.
Returns:
An agent instance.
"""
def dummy_instructions(context_variables: dict):
"""
The function should take the context_variables as an argument, and return a string. The context_variables is a dictionary, and it's track the important variables of the agent in the whole conversation.
The instructions should be concise and clear, and it's very important for the agent to follow the instructions.
"""
tmp_variables = context_variables.get("tmp_variables", {})
return f"""..."""
return Agent(
name="Dummy Agent", # The name of the agent, you can change it in different scenes.
model=model, # The default model is gpt-4o-2024-08-06, you can change it to other models if user specified.
instructions="..." or dummy_instructions, # the instructions of the agent, the instructions can be a string or a function that returns a string. If it is a function, the function should take the context_variables as an argument, and return a string. The instructions should be concise and clear, and it's very important for the agent to follow the instructions.
functions=[tool_dummy], # The tools of the agent, you can add different tools in different scenes.
)
"""
Form to create an agent:
agent_name = "Dummy Agent"
agent_description = "This is a dummy agent, it's used for demonstrating the usage of the autoagent."
agent_instructions = "..." | "...{global_variables}..."
agent_tools = [tool_dummy]
"""
autoagent/agents/github_agent.py
+25
View File
@@ -0,0 +1,25 @@
from autoagent.types import Agent
from autoagent.tools import (
push_changes, submit_pull_request
)
from autoagent.registry import register_agent
@register_agent(name = "Github Agent", func_name="get_github_agent")
def get_github_agent(model: str):
def instructions(context_variables):
return \
f"""You are an agent that helps user to manage the GitHub repository named 'autoagent'.
The user will give you the suggestion of the changes to be pushed to the repository.
Follow the following routine with the user:
1. First, use `push_changes` to push the changes to the repository. (If the user want to push all the changes, use `push_changes` with `file_paths=None` as the argument.)
2. Then, ask the user whether to submit a pull request to a target branch. (If yes, give the `target_branch`)
3. If the user wants to submit a pull request, use `submit_pull_request` to submit the pull request, if not, just ignore this step.
"""
return Agent(
name="Github Agent",
model=model,
instructions=instructions,
functions=[push_changes, submit_pull_request],
parallel_tool_calls = False
)
autoagent/agents/math/math_solver_agent.py
+17
View File
@@ -0,0 +1,17 @@
from autoagent.types import Agent
from autoagent.registry import register_plugin_agent
@register_plugin_agent(name="Math Solver Agent", func_name="get_math_solver_agent")
def get_math_solver_agent(model: str):
'''
This agent solves mathematical problems using analytical and systematic approaches.
'''
instructions = 'You are responsible for solving mathematical problems using a systematic approach. You should:\n1. Use the provided conditions and objective to formulate a solution strategy\n2. Break down complex problems into smaller steps\n3. Apply appropriate mathematical concepts and formulas\n4. Show clear step-by-step work and explanations\n5. Verify the solution matches the problem requirements'
return Agent(
name="Math Solver Agent",
model=model,
instructions=instructions,
functions=[]
)
autoagent/agents/math/vote_aggregator_agent.py
+17
View File
@@ -0,0 +1,17 @@
from autoagent.types import Agent
from autoagent.registry import register_plugin_agent
@register_plugin_agent(name="Vote Aggregator Agent", func_name="get_vote_aggregator_agent")
def get_vote_aggregator_agent(model: str):
'''
This agent aggregates solutions from different solvers and determines the final answer through majority voting.
'''
instructions = 'You are a solution aggregator specializing in combining and analyzing multiple solutions to determine the most accurate answer. Your responsibilities include:\n\n1. Carefully review all provided solutions\n2. Compare the reasoning and calculations in each solution\n3. Identify commonalities and differences between solutions\n4. Implement majority voting when solutions differ\n5. Evaluate the confidence level of each solution\n6. Provide justification for the final selected answer\n\nWhen aggregating solutions:\n1. List all solutions received\n2. Compare the approach and methodology used in each\n3. Identify the final answer from each solution\n4. Apply majority voting to determine the consensus\n5. If no clear majority, analyze the reasoning quality to break ties\n6. Present the final selected answer with explanation of the selection process'
return Agent(
name="Vote Aggregator Agent",
model=model,
instructions=instructions,
functions=[]
)
autoagent/agents/meta_agent/agent_creator.py
+78
View File
@@ -0,0 +1,78 @@
from autoagent.registry import register_agent
from autoagent.tools.meta.edit_agents import list_agents, create_agent, delete_agent, run_agent, read_agent, create_orchestrator_agent
from autoagent.tools.meta.edit_tools import list_tools, create_tool, delete_tool, run_tool
from autoagent.tools.terminal_tools import execute_command, terminal_page_down, terminal_page_up, terminal_page_to
from autoagent.types import Agent
from autoagent.io_utils import read_file
@register_agent(name = "Agent Creator Agent", func_name="get_agent_creator_agent")
def get_agent_creator_agent(model: str) -> str:
"""
The agent creator is an agent that can be used to create the agents.
"""
def instructions(context_variables):
return f"""\
You are an Agent Creator specialized in the MetaChain framework. Your primary responsibility is to create, manage, and orchestrate agents based on XML-formatted agent forms.
CORE RESPONSIBILITIES:
1. Parse and implement agent forms
2. Create and manage individual agents
3. Orchestrate multi-agent systems
4. Handle dependencies and system requirements
AVAILABLE FUNCTIONS:
1. Agent Management:
- `create_agent`: Create new agents or update existing ones strictly following the given agent form.
- `read_agent`: Retrieve existing agent definitions. Note that if you want to use `create_agent` to update an existing agent, you MUST use the `read_agent` function to get the definition of the agent first.
- `delete_agent`: Remove unnecessary agents.
- `list_agents`: Display all available agents and their information.
- `create_orchestrator_agent`: Create orchestrator for multi-agent systems. If the request is to create MORE THAN ONE agent, after you create ALL required agents, you MUST use the `create_orchestrator_agent` function to create an orchestrator agent that can orchestrate the workflow of the agents. And then use the `run_agent` function to run the orchestrator agent to complete the user task.
2. Execution:
- run_agent: Execute agent to complete the user task. The agent could be a single agent (single agent form) or an orchestrator agent (multi-agent form).
- execute_command: Handle system dependencies and requirements
- terminal_page_down: Move the terminal page down when the terminal output is too long.
- terminal_page_up: Move the terminal page up when the terminal output is too long.
- terminal_page_to: Move the terminal page to the specific page when the terminal output is too long, and you want to move to the specific page with the meaningful content.
WORKFLOW GUIDELINES:
1. Single Agent Implementation:
- Carefully read the agent form and understand the requirements.
- Create/update agent using create_agent
- Execute task using run_agent
- Monitor and handle any errors
2. Multi-Agent Implementation:
- Create all required agents individually using `create_agent`
- MUST create an orchestrator agent using `create_orchestrator_agent`
- Execute task through the `run_agent` function to execute the created orchestrator agent
- Monitor system performance
3. Error Handling:
- Check for missing dependencies using `execute_command`
- Install required packages using execute_command
- Validate agent creation and execution
- Report any issues clearly
BEST PRACTICES:
1. Always verify existing agents using `read_agent` before updates
2. Create orchestrator agents for ANY multi-agent scenario using `create_orchestrator_agent`
3. Handle dependencies proactively using `execute_command`
4. Maintain clear documentation of created agents
5. Follow the exact specifications from the agent form XML
Remember: Your success is measured by both the accurate creation of agents and their effective execution of the given tasks.
"""
tool_list = [list_agents, create_agent, delete_agent, run_agent, execute_command, read_agent, create_orchestrator_agent, terminal_page_down, terminal_page_up, terminal_page_to]
return Agent(
name="Agent Creator Agent",
model=model,
instructions=instructions,
functions=tool_list,
tool_choice = "required",
parallel_tool_calls = False
)
autoagent/agents/meta_agent/agent_editor.py
+38
View File
@@ -0,0 +1,38 @@
from autoagent.registry import register_agent
from autoagent.tools.meta.edit_agents import list_agents, create_agent, delete_agent, run_agent
from autoagent.tools.terminal_tools import execute_command
from autoagent.types import Agent
from autoagent.io_utils import read_file
@register_agent(name = "Agent Editor Agent", func_name="get_agent_editor_agent")
def get_agent_editor_agent(model: str) -> str:
"""
The agent editor is an agent that can be used to edit the agents.
"""
def instructions(context_variables):
return f"""\
You are an agent editor agent that can be used to edit the agents. You are working on a Agent framework named MetaChain, and your responsibility is to edit the agents in the MetaChain, so that the agents can be used to help the user with their request.
The existing agents are shown below:
{list_agents(context_variables)}
If you want to create a new agent, you should:
1. follow the format of the `get_dummy_agent` below:
```python
{read_file('autoagent/agents/dummy_agent.py')}
```
2. you successfully create the agent only after you have successfully run the agent with the `run_agent` function to satisfy the user's request.
3. If you encounter any error while creating and running the agent, like dependency missing, you should use the `execute_command` function to install the dependency.
[IMPORTANT] The `register_plugin_agent` registry function is strictly required for a agent implementation to be recognized by the MetaChain framework.
"""
tool_list = [list_agents, create_agent, delete_agent, run_agent, execute_command]
return Agent(
name="Agent Editor Agent",
model=model,
instructions=instructions,
functions=tool_list,
tool_choice = "required",
parallel_tool_calls = False
)
autoagent/agents/meta_agent/agent_former.py
+257
View File
@@ -0,0 +1,257 @@
from autoagent.registry import register_agent
from autoagent.tools.meta.edit_agents import list_agents, create_agent, delete_agent, run_agent, read_agent
from autoagent.tools.meta.edit_tools import list_tools, create_tool, delete_tool, run_tool
from autoagent.tools.terminal_tools import execute_command
from autoagent.types import Agent
from autoagent.io_utils import read_file
from pydantic import BaseModel, Field
from typing import List
@register_agent(name = "Agent Former Agent", func_name="get_agent_former_agent")
def get_agent_former_agent(model: str) -> str:
"""
This agent is used to complete a form that can be used to create an agent.
"""
def instructions(context_variables):
return r"""\
You are an agent specialized in creating agent forms for the MetaChain framework.
Your task is to analyze user requests and generate structured creation forms for either single or multi-agent systems.
KEY COMPONENTS OF THE FORM:
1. <agents> - Root element containing all agent definitions
2. <system_input> - Defines what the system receives
- Must describe the overall input that the system accepts
- For single agent: Same as agent_input
- For multi-agent: Should encompass all possible inputs that will be routed to different agents
3. <system_output> - Specifies system response format
- Must contain exactly ONE key-description pair
- <key>: Single identifier for the system's output
- <description>: Explanation of the output
- For single agent: Same as agent_output
- For multi-agent: Should represent the unified output format from all agents
4. <agent> - Individual agent definition
- name: Agent's identifier
- description: Agent's purpose and capabilities
- instructions: Agent's behavioral guidelines
* To reference global variables, use format syntax: {variable_key}
* Example: "Help the user {user_name} with his/her request"
* All referenced keys must exist in global_variables
- tools: Available tools (existing/new)
- agent_input:
* Must contain exactly ONE key-description pair
* <key>: Identifier for the input this agent accepts
* <description>: Detailed explanation of the input format
- agent_output:
* Must contain exactly ONE key-description pair
* <key>: Identifier for what this agent produces
* <description>: Detailed explanation of the output format
5. <global_variables> - Shared variables across agents (optional)
- Used for constants or shared values accessible by all agents
- Variables defined here can be referenced in instructions using {key}
- Example:
```xml
<global_variables>
<variable>
<key>user_name</key>
<description>The name of the user</description>
<value>John Doe</value>
</variable>
</global_variables>
```
- Usage in instructions: "You are a personal assistant for {user_name}."
IMPORTANT RULES:
- For single agent systems:
* system_input/output must match agent_input/output exactly
- For multi-agent systems:
* system_input should describe the complete input space
* Each agent_input should specify which subset of the system_input it handles
* system_output should represent the unified response format
""" + \
f"""
Existing tools you can use is:
{list_tools(context_variables)}
Existing agents you can use is:
{list_agents(context_variables)}
""" + \
r"""
EXAMPLE 1 - SINGLE AGENT:
User: I want to build an agent that can answer the user's question about the OpenAI products. The document of the OpenAI products is available at `/workspace/docs/openai_products/`.
The agent should be able to:
1. query and answer the user's question about the OpenAI products based on the document.
2. send email to the user if the sending email is required in the user's request.
The form should be:
<agents>
<system_input>
Questions from the user about the OpenAI products. The document of the OpenAI products is available at `/workspace/docs/openai_products/`.
</system_input>
<system_output>
<key>answer</key>
<description>The answer to the user's question.</description>
</system_output>
<agent>
<name>Helper Center Agent</name>
<description>The helper center agent is an agent that serves as a helper center agent for a specific user to answer the user's question about the OpenAI products.</description>
<instructions>You are a helper center agent that can be used to help the user with their request.</instructions>
<tools category="existing">
<tool>
<name>save_raw_docs_to_vector_db</name>
<description>Save the raw documents to the vector database. The documents could be:
- ANY text document with the extension of pdf, docx, txt, etcs.
- A zip file containing multiple text documents
- a directory containing multiple text documents
All documents will be converted to raw text format and saved to the vector database in the chunks of 4096 tokens.</description>
</tool>
<tool>
<name>query_db</name>
<description>Query the vector database to find the answer to the user's question.</description>
</tool>
<tool>
<name>modify_query</name>
<description>Modify the user's question to a more specific question.</description>
</tool>
<tool>
<name>answer_query</name>
<description>Answer the user's question based on the answer from the vector database.</description>
</tool>
<tool>
<name>can_answer</name>
<description>Check if the user's question can be answered by the vector database.</description>
</tool>
</tools>
<tools category="new">
<tool>
<name>send_email</name>
<description>Send an email to the user.</description>
</tool>
</tools>
<agent_input>
<key>user_question</key>
<description>The question from the user about the OpenAI products.</description>
</agent_input>
<agent_output>
<key>answer</key>
<description>The answer to the user's question.</description>
</agent_output>
</agent>
</agents>
EXAMPLE 2 - MULTI-AGENT:
User: I want to build a multi-agent system that can handle two types of requests for the specific user:
1. Purchase a product or service
2. Refund a product or service
The specific user worked for is named John Doe.
The form should be:
<agents>
<system_input>
The user request from the specific user about the product or service, mainly categorized into 2 types:
- Purchase a product or service
- Refund a product or service
</system_input>
<system_output>
<key>response</key>
<description>The response of the agent to the user's request.</description>
</system_output>
<global_variables>
<variable>
<key>user_name</key>
<description>The name of the user.</description>
<value>John Doe</value>
</variable>
</global_variables>
<agent>
<name>Personal Sales Agent</name>
<description>The personal sales agent is an agent that serves as a personal sales agent for a specific user.</description>
<instructions>You are a personal sales agent that can be used to help the user {user_name} with their request.</instructions>
<tools category="new">
<tool>
<name>recommend_product</name>
<description>Recommend a product to the user.</description>
</tool>
<tool>
<name>recommend_service</name>
<description>Recommend a service to the user.</description>
</tool>
<tool>
<name>conduct_sales</name>
<description>Conduct sales with the user.</description>
</tool>
</tools>
<agent_input>
<key>user_request</key>
<description>Request from the specific user for purchasing a product or service.</description>
</agent_input>
<agent_output>
<key>response</key>
<description>The response of the agent to the user's request.</description>
</agent_output>
</agent>
<agent>
<name>Personal Refunds Agent</name>
<description>The personal refunds agent is an agent that serves as a personal refunds agent for a specific user.</description>
<instructions>Help the user {user_name} with a refund. If the reason is that it was too expensive, offer the user a discount. If they insist, then process the refund.</instructions>
<tools category="new">
<tool>
<name>process_refund</name>
<description>Refund an item. Refund an item. Make sure you have the item_id of the form item_... Ask for user confirmation before processing the refund.</description>
</tool>
<tool>
<name>apply_discount</name>
<description>Apply a discount to the user's cart.</description>
</tool>
</tools>
<agent_input>
<key>user_request</key>
<description>Request from the specific user for refunding a product or service.</description>
</agent_input>
<agent_output>
<key>response</key>
<description>The response of the agent to the user's request.</description>
</agent_output>
</agent>
</agents>
GUIDELINES:
1. Each agent must have clear, focused responsibilities
2. Tool selections should be minimal but sufficient
3. Instructions should be specific and actionable
4. Input/Output definitions must be precise
5. Use global_variables for shared context across agents
Follow these examples and guidelines to create appropriate agent forms based on user requirements.
"""
return Agent(
name = "Agent Former Agent",
model = model,
instructions = instructions,
)
if __name__ == "__main__":
from autoagent import MetaChain
agent = get_agent_former_agent("claude-3-5-sonnet-20241022")
client = MetaChain()
task_yaml = """\
I want to create two agents that can help me to do two kinds of tasks:
1. Manage the private financial docs. I have a folder called `financial_docs` in my local machine, and I want to help me to manage the financial docs.
2. Search the financial information online. You may help me to:
- get balance sheets for a given ticker over a given period.
- get cash flow statements for a given ticker over a given period.
- get income statements for a given ticker over a given period.
"""
task_yaml = task_yaml + """\
Directly output the form in the XML format.
"""
messages = [{"role": "user", "content": task_yaml}]
response = client.run(agent, messages)
print(response.messages[-1]["content"])
autoagent/agents/meta_agent/form_complie.py
+139
View File
@@ -0,0 +1,139 @@
from pydantic import BaseModel, Field, validator, field_validator, ValidationInfo
from typing import List, Dict, Optional, Literal
import xml.etree.ElementTree as ET
class KeyDescription(BaseModel):
key: str
description: str
class Tool(BaseModel):
name: str
description: str
class ToolSet(BaseModel):
existing: List[Tool] = Field(default_factory=list)
new: List[Tool] = Field(default_factory=list)
class GlobalVariable(BaseModel):
key: str
description: str
value: str
class Agent(BaseModel):
name: str
description: str
instructions: str
tools: ToolSet
agent_input: KeyDescription
agent_output: KeyDescription
class AgentForm(BaseModel):
system_input: str
system_output: KeyDescription
global_variables: Dict[str, GlobalVariable] = Field(default_factory=dict)
agents: List[Agent]
@field_validator('agents')
def validate_single_agent_io(cls, v, info: ValidationInfo):
"""验证单agent系统的输入输出是否匹配"""
if len(v) == 1:
agent = v[0]
system_output = info.data.get('system_output')
if system_output and agent.agent_output.key != system_output.key:
raise ValueError("Single agent system must have matching system and agent output keys")
return v
# def validate_global_ctx_instructions(cls, v, info: ValidationInfo):
# """验证全局变量和系统输入是否匹配"""
class XMLParser:
@staticmethod
def parse_key_description(elem: ET.Element, tag_name: str) -> KeyDescription:
node = elem.find(tag_name)
if node is None:
raise ValueError(f"Missing {tag_name}")
return KeyDescription(
key=node.find('key').text.strip(),
description=node.find('description').text.strip()
)
@staticmethod
def parse_tools(agent_elem: ET.Element) -> ToolSet:
tools = ToolSet()
for tools_elem in agent_elem.findall('tools'):
category = tools_elem.get('category')
if category not in ('existing', 'new'):
continue
tool_list = []
for tool_elem in tools_elem.findall('tool'):
tool = Tool(
name=tool_elem.find('name').text.strip(),
description=tool_elem.find('description').text.strip()
)
tool_list.append(tool)
if category == 'existing':
tools.existing = tool_list
else:
tools.new = tool_list
return tools
@staticmethod
def parse_global_variables(root: ET.Element) -> Dict[str, GlobalVariable]:
variables = {}
global_vars = root.find('global_variables')
if global_vars is not None:
for var in global_vars.findall('variable'):
key = var.find('key').text.strip()
variables[key] = GlobalVariable(
key=key,
description=var.find('description').text.strip(),
value=var.find('value').text.strip()
)
return variables
@classmethod
def parse_agent(cls, agent_elem: ET.Element) -> Agent:
return Agent(
name=agent_elem.find('name').text.strip(),
description=agent_elem.find('description').text.strip(),
instructions=agent_elem.find('instructions').text.strip(),
tools=cls.parse_tools(agent_elem),
agent_input=cls.parse_key_description(agent_elem, 'agent_input'),
agent_output=cls.parse_key_description(agent_elem, 'agent_output')
)
@classmethod
def parse_xml(cls, xml_content: str) -> AgentForm:
root = ET.fromstring(xml_content)
return AgentForm(
system_input=root.find('system_input').text.strip(),
system_output=cls.parse_key_description(root, 'system_output'),
global_variables=cls.parse_global_variables(root),
agents=[cls.parse_agent(agent_elem) for agent_elem in root.findall('agent')]
)
def parse_agent_form(xml_content: str) -> Optional[AgentForm]:
"""
读取并解析agent form XML文件
Args:
xml_content: XML文件内容
Returns:
解析后的AgentForm对象，如果解析失败返回None
"""
try:
# with open(xml_path, 'r', encoding='utf-8') as f:
# xml_content = f.read()
return XMLParser.parse_xml(xml_content)
except ET.ParseError as e:
print(f"Error parsing XML: {e}")
return None
except Exception as e:
print(f"Unexpected error: {e}")
return None
autoagent/agents/meta_agent/tool_editor.py
+257
View File
@@ -0,0 +1,257 @@
from autoagent.registry import register_agent
from autoagent.tools.meta.edit_tools import list_tools, create_tool, delete_tool, run_tool, get_metachain_path
from autoagent.tools.meta.tool_retriever import get_api_plugin_tools_doc
from autoagent.tools.meta.search_tools import search_trending_models_on_huggingface, get_hf_model_tools_doc
from autoagent.types import Agent
from autoagent.io_utils import read_file
from autoagent.tools.terminal_tools import execute_command, terminal_page_down, terminal_page_up, terminal_page_to
@register_agent(name = "Tool Editor Agent", func_name="get_tool_editor_agent")
def get_tool_editor_agent(model: str) -> Agent:
"""
The tool editor is an agent that can be used to edit the tools.
"""
def instructions(context_variables):
return f"""\
You are a tool editor agent responsible for managing plugin tools in the MetaChain framework. Your core responsibility is to edit, create, and manage plugin tools that can be used by other agents.
[PLUGIN TOOLS SYSTEM]
- Plugin tools are the building blocks of MetaChain
- All available plugin tools are as follows:
{list_tools(context_variables)}
- Plugin tools can ONLY be executed using `run_tool(tool_name, run_code)`. You should import `run_tool` by `from autoagent.tools import run_tool`.
- NEVER try to import and run plugin tools directly - always use `run_tool`
[TOOL CREATION WORKFLOW]
1. ALWAYS start with `list_tools()` to check existing tools
2. For NEW plugin tool creation, FOLLOW THIS ORDER:
a. For third-party API integration (e.g., RapidAPI, external services):
- MUST FIRST use `get_api_plugin_tools_doc` to get API documentation and keys
- API keys should be embedded IN the function body, NOT as parameters.
- The API keys are always in the retrieved information from `get_api_plugin_tools_doc`, DO NOT guess the API keys by yourself.
- Follow the API implementation details from the documentation
b. For modal transformation tasks (image/video/audio generation/processing):
- FIRST use `search_trending_models_on_huggingface` to find suitable models, only support the following tags: ['audio-text-to-text', 'text-to-image', 'image-to-image', 'image-to-video', 'text-to-video', 'text-to-speech', 'text-to-audio', 'automatic-speech-recognition', 'audio-to-audio'].
- Then use `get_hf_model_tools_doc` for detailed model information
- Only use internal knowledge if no suitable models are found
c. For visual analysis tasks (images/videos):
- MUST use the existing `visual_question_answering` plugin tool by `run_tool("visual_question_answering", "from autoagent.tools import visual_question_answering; ...")`. DO NOT use it directly without `run_tool`.
- NO direct implementation of visual processing
- Chain with other tools as needed
3. Plugin Tool Implementation Requirements:
- Use @register_plugin_tool decorator (REQUIRED). You should import `register_plugin_tool` by `from autoagent.registry import register_plugin_tool`.
- Follow this template:
```python
{read_file('autoagent/tools/dummy_tool.py')}
```
- Include clear type hints
- Make tools abstract and reusable
- Use generic names (e.g., 'process_media' not 'process_youtube_video')
- Handle dependencies with `execute_command`
[AVAILABLE TOOLS]
1. get_api_plugin_tools_doc:
- PRIMARY tool for third-party API integration
- MUST be used FIRST for Finance, Entertainment, eCommerce, etc.
- Provides API documentation AND authentication keys
- API keys should be embedded in tool implementation
2. search_trending_models_on_huggingface:
- Use for finding models for media transformation tasks
- Supported tags: ['text-to-image', 'image-to-image', 'text-to-video', etc.]
- Use AFTER checking no suitable API exists via `get_api_plugin_tools_doc`
3. get_hf_model_tools_doc:
- Get the detailed information of a model on Hugging Face, such as the detailed usage of the model containing the model's README.md.
- You should use this tool after you have used `search_trending_models_on_huggingface` to find the model you want to use.
4. Other management tools:
- list_tools(): Check existing tools
- create_tool(tool_name, tool_code): Create new tools
- run_tool(tool_name, run_code): REQUIRED method to execute any plugin tool
- delete_tool(tool_name): Remove tools
- execute_command: Install dependencies. Handles system-level operations
- terminal_page_* tools: Navigate long outputs
5. case_resolved & case_not_resolved:
- case_resolved: after you have created all the tools and tested them using `run_tool` successfully (with the expected output rather than just run it), you should use the `case_resolved` tool to brief the result.
- case_not_resolved: after you have tried your best to create the tools but failed, you should use the `case_not_resolved` tool to tell the failure reason.
[CRITICAL RULES]
1. Tool Creation Priority:
- FIRST: Check existing tools via list_tools()
- SECOND: Use `get_api_plugin_tools_doc` for API-based tools
- THIRD: Use `search_trending_models_on_huggingface` for media tasks
- LAST: Use internal knowledge if no other options available
2. API Implementation:
- NEVER expose API keys as parameters
- ALWAYS embed API keys in function body
- Get keys from `get_api_plugin_tools_doc`
3. Tool Design:
- Tools MUST be abstract, modular, and reusable:
- Use generic function names (e.g., `download_media` instead of `download_youtube_video`)
- Break complex tasks into smaller, reusable components
- Avoid task-specific implementations
- Use parameters instead of hardcoded values
- Include proper error handling
[TESTING]
Test new tools using `run_tool`:
`run_tool(tool_name="your_tool", run_code="from autoagent.tools import your_tool; print(your_tool(param1='value1'))")`
"""
tool_list = [list_tools, create_tool, run_tool, delete_tool, get_api_plugin_tools_doc, execute_command, terminal_page_down, terminal_page_up, terminal_page_to, search_trending_models_on_huggingface, get_hf_model_tools_doc]
return Agent(
name="Tool Editor Agent",
model=model,
instructions=instructions,
functions=tool_list,
tool_choice = "required",
parallel_tool_calls = False
)
"""
5. [IMPORTANT] If you want to use Hugging Face models, especially for some tasks related to vision, audio, video, you should use the `search_trending_models_on_huggingface` tool to search trending models related to the specific task on Hugging Face, and then use the `get_hf_model_tools_doc` tool to get the detailed information about the specific model.
6. [IMPORTANT] As for the tags ['image-text-to-text', 'visual-question-answering', 'video-text-to-text'] and ANY visual tasks, you should use `visual_question_answering` tool instead of Hugging Face models.
"""
"""\
You are a tool editor agent that can be used to edit the tools. You are working on a Agent framework named MetaChain, and your responsibility is to edit the tools in the MetaChain, so that the tools can be used by the agents to help the user with their request.
The existing tools are shown below:
{list_tools(context_variables)}
If you want to create a new tool, you should:
1. follow the format of the `tool_dummy` below. Note that if the tool should be used with third-part api key, you should write the api key inside the definition of the tool:
```python
{read_file('autoagent/tools/dummy_tool.py')}
```
2. you successfully create the tool only after you have successfully run the tool with the `run_tool` function, and an example of testing the tool is shown below.:
```python
from autoagent.tools import tool_dummy
if __name__ == "__main__":
... # some pre-operations
print(run_tool(tool_name="tool_dummy", run_code="from autoagent.tools import tool_dummy; print(tool_dummy(args1=args1, args2=args1, ...))"))
```
3. If you encounter any error while creating and running the tool, like dependency missing, you should use the `execute_command` function to install the dependency.
4. [IMPORTANT] If you want to use third-party api, especially for some tasks related to Finance, Entertainment, eCommerce, Food, Travel, Sports, you MUST use the `get_api_plugin_tools_doc` tool to search information from existing api documents, it contains how to implement the api and API keys.
[IMPORTANT] The `register_plugin_tool` registry function is strictly required for a tool implementation to be recognized by the MetaChain framework.
[IMPORTANT] The tool you create should be abstract, modular, and reusable. Specifically, the function name must be generic (e.g.,
`count_objects` instead of `count_apples`). The function must use parameters instead of hard-coded values. The
function body must be self-contained.
[IMPORTANT] Explicitly declare input and output data types using type hints.
[IMPORTANT] For ANY visual tasks related to image and video, you should use `visual_question_answering` tool.
"""
"""\
You are a tool editor agent responsible for managing plugin tools in the MetaChain framework. Your core responsibility is to edit, create, and manage plugin tools that can be used by other agents.
[PLUGIN TOOLS SYSTEM]
- Plugin tools are the building blocks of MetaChain
- All available plugin tools are as follows:
{list_tools(context_variables)}
- Plugin tools can ONLY be executed using `run_tool(tool_name, run_code)`
- NEVER try to import and run tools directly - always use `run_tool`
[AVAILABLE MANAGEMENT TOOLS]
1. list_tools():
- Lists all existing plugin tools
- Returns: tool name, arguments, docstring, implementation details
- Use this FIRST to check existing tools
2. create_tool(tool_name: str, tool_code: str):
- Creates new plugin tools
- Requires proper registration using @register_plugin_tool, and you MUST import `register_plugin_tool` by `from autoagent.registry import register_plugin_tool`
3. run_tool(tool_name: str, run_code: str,):
- REQUIRED method to execute any plugin tool
- Format: run_tool("tool_name", "from autoagent.tools import tool_name; print(tool_name(args))")
4. delete_tool(tool_name: str,):
- Removes existing plugin tools
- Use with caution
5. get_api_plugin_tools_doc:
- Required for third-party API integrations, e.g. RapidAPI.
- MUST be used for Finance, Entertainment, etc.
6. execute_command:
- Handles system-level operations
- Use for dependency installation
7. terminal_page_down:
- Move the terminal page down when the terminal output is too long.
8. terminal_page_up:
- Move the terminal page up when the terminal output is too long.
9. terminal_page_to:
- Move the terminal page to the specific page when the terminal output is too long, and you want to move to the specific page with the meaningful content.
10. search_trending_models_on_huggingface:
- Search trending models on Hugging Face.
- Use this tool when you want to use Hugging Face models to generate images, videos, audios, etc.
- Do NOT use this tool for text-to-text or image-to-text tasks.
11. get_hf_model_tools_doc:
- Get the detailed information about the specific model on Hugging Face.
- Use this tool when you want to use Hugging Face models to generate images, videos, audios, etc.
[CRITICAL PRINCIPLES FOR PLUGIN TOOLS]
1. Tools MUST be abstract, modular, and reusable:
- Use generic function names (e.g., `download_media` instead of `download_youtube_video`)
- Break complex tasks into smaller, reusable components
- Avoid task-specific implementations
- Use parameters instead of hardcoded values
2. For ALL visual tasks (images, videos, visual analysis):
- MUST use the existing `visual_question_answering` plugin tool
- NO direct implementation of visual processing
- Chain `visual_question_answering` with other tools as needed
[WORKFLOW FOR PLUGIN TOOL MANAGEMENT]
1. Always start with `list_tools()` to check existing tools
2. For new plugin tools:
a. Design generic, reusable interface
b. Follow the template format:
```python
{read_file('autoagent/tools/dummy_tool.py')}
```
c. Create using `create_tool`
d. Test using `run_tool`
e. Handle dependencies with `execute_command`
[IMPORTANT RULES]
- ALL tools must be registered with @register_plugin_tool
- ALL tools must have type hints
- Each tool does ONE thing well
- Create modular tools that can be combined
- ALWAYS use `run_tool` to execute plugin tools
- NEVER modify the `visual_question_answering` tool
[TOOL TESTING EXAMPLE]
Correct way to test a plugin tool:
```python
result = run_tool(
tool_name="your_tool",
run_code="from autoagent.tools import your_tool; print(your_tool(param1='value1'))",
context_variables=context_variables
)
```
"""
autoagent/agents/meta_agent/workflow_creator.py
+81
View File
@@ -0,0 +1,81 @@
from autoagent.registry import register_agent
from autoagent.tools.meta.edit_agents import list_agents, create_agent, delete_agent, run_agent, read_agent, create_orchestrator_agent
from autoagent.tools.meta.edit_workflow import list_workflows, create_workflow, run_workflow
from autoagent.tools.terminal_tools import execute_command, terminal_page_down, terminal_page_up, terminal_page_to
from autoagent.types import Agent
from autoagent.io_utils import read_file
@register_agent(name = "Workflow Creator Agent", func_name="get_workflow_creator_agent")
def get_workflow_creator_agent(model: str) -> str:
"""
The workflow creator is an agent that can be used to create the workflow.
"""
def instructions(context_variables):
return f"""\
You are a Workflow Creator specialized in the MetaChain framework. Your primary responsibility is to create and manage workflows based on XML-formatted workflow forms.
CORE RESPONSIBILITIES:
1. Parse and implement workflow forms
2. Create necessary agents if specified in the workflow
3. Create and manage workflows
4. Execute workflows as needed
AVAILABLE FUNCTIONS:
1. Workflow Management:
- `create_workflow`: Create new workflows based on the workflow form
- `run_workflow`: Execute the created workflow
- `list_workflows`: Display all available workflows
2. Agent Management (when needed):
- `create_agent`: Create new agents if specified in the workflow form. If no tools are explicitly specified, use empty tool list ([])
- `read_agent`: Retrieve existing agent definitions before updates
- `list_agents`: Display all available agents
3. System Tools:
- `execute_command`: Handle system dependencies
- `terminal_page_down`, `terminal_page_up`, `terminal_page_to`: Navigate terminal output
WORKFLOW CREATION PROCESS:
1. Parse Workflow Form:
- Analyze the workflow form carefully
- Identify any new agents that need to be created
- Understand the workflow structure and requirements
2. Create Required Agents:
- For each new agent in the workflow form:
* Use `create_agent` with appropriate parameters
* If no tools specified, use empty tool list ([])
* Verify agent creation success
3. Create Workflow:
- Use `create_workflow` to generate the workflow
- Ensure all required agents exist
- Validate workflow structure
4. Execute Workflow:
- Use `run_workflow` to execute the created workflow
- Monitor execution progress
- Handle any errors appropriately
BEST PRACTICES:
1. Always check if required agents exist before creating new ones
2. Use empty tool list ([]) when no specific tools are mentioned
3. Validate workflow creation before execution
4. Follow the exact specifications from the workflow form XML
5. Handle errors and dependencies appropriately
Remember: Your primary goal is to create and execute workflows according to the provided workflow forms, creating any necessary agents along the way.
"""
tool_list = [list_agents, create_agent, execute_command, read_agent, terminal_page_down, terminal_page_up, terminal_page_to, list_workflows, create_workflow, run_workflow]
return Agent(
name="Workflow Creator Agent",
model=model,
instructions=instructions,
functions=tool_list,
tool_choice = "required",
parallel_tool_calls = False
)
autoagent/agents/meta_agent/workflow_former.py
+578
View File
@@ -0,0 +1,578 @@
from autoagent.registry import register_agent
from autoagent.tools.meta.edit_agents import list_agents, create_agent, delete_agent, run_agent, read_agent
from autoagent.tools.meta.edit_tools import list_tools, create_tool, delete_tool, run_tool
from autoagent.tools.meta.edit_workflow import list_workflows
from autoagent.tools.terminal_tools import execute_command
from autoagent.types import Agent
from autoagent.io_utils import read_file
from pydantic import BaseModel, Field
from typing import List
import json
@register_agent(name = "Workflow Former Agent", func_name="get_workflow_former_agent")
def get_workflow_former_agent(model: str) -> str:
"""
This agent is used to complete a form that can be used to create a workflow consisting of multiple agents.
"""
def instructions(context_variables):
workflow_list = list_workflows(context_variables)
workflow_list = json.loads(workflow_list)
workflow_list = [workflow_name for workflow_name in workflow_list.keys()]
workflow_list_str = ", ".join(workflow_list)
return r"""\
You are an agent specialized in creating workflow forms for the MetaChain framework.
Your task is to analyze user requests and generate structured creation forms for workflows consisting of multiple agents.
KEY COMPONENTS OF THE FORM:
1. <workflow> - Root element containing the entire workflow definition
2. <name> - The name of the workflow. It should be a single word with '_' as the separator, and as unique as possible to describe the speciality of the workflow.
3. <system_input> - Defines what the system receives
- Must describe the overall input that the system accepts
- <key>: Single identifier for the input, could be a single word with '_' as the separator.
- <description>: Detailed explanation of input format
4. <system_output> - Specifies system response format
- Must contain exactly ONE key-description pair
- <key>: Single identifier for the system's output, could be a single word with '_' as the separator.
- <description>: Explanation of the output format
5. <agents> - Contains all agent definitions
- Each <agent> can be existing or new (specified by category attribute)
- name: Agent's identifier
- description: Agent's purpose and capabilities
- tools: (optional): Only required for new agents when specific tools are requested
* Only include when user explicitly requests certain tools
6. <global_variables> - Shared variables across agents in the workflow (optional)
- Used for constants or shared values accessible by all agents in EVERY event in the workflow
- Example:
```xml
<global_variables>
<variable>
<key>user_name</key>
<description>The name of the user</description>
<value>John Doe</value>
</variable>
</global_variables>
```
7. <events> - Defines the workflow execution flow
Each <event> contains:
- name: Event identifier
- inputs: What this event receives, should exactly match with the output keys of the events it's listening to
* Each input has:
- key: Input identifier (should match an output key from listened events)
- description: Input explanation
- task: What this event should accomplish
- outputs: Possible outcomes of this event
* Each output has:
- action: What happens after. Every action has a type and a optional value. Action is categorized into 3 types:
- RESULT: The event is successful, and the workflow will continue to the next event which is listening to this event. Value is the output of this event.
- ABORT: The event is not successful, and the workflow will abort. Value could be empty.
- GOTO: The event is not successful, and the workflow will wait for the next event. Value is the name of the event to go to. The event go to should NOT listen to this event.
- key: Output identifier (be a single word with '_' as the separator)
- description: Output explanation
- condition: when the output occurs, the action will be executed
* Can have single or multiple outputs:
- For single output (simple flow):
```xml
<outputs>
<output>
<key>result_key</key>
<description>Description of the result</description>
<action>
<type>RESULT</type>
</action>
</output>
</outputs>
```
- For multiple outputs (conditional flow):
```xml
<outputs>
<output>
<key>success_result</key>
<description>Output when condition A is met</description>
<condition>When condition A is true</condition>
<action>
<type>RESULT</type>
</action>
</output>
<output>
<key>should_repeat</key>
<description>Output when condition B is met</description>
<condition>When condition B is true</condition>
<action>
<type>GOTO</type>
<value>target_event</value>
</action>
</output>
<output>
<key>failure_result</key>
<description>Output when condition C is met</description>
<condition>When condition C is true</condition>
<action>
<type>ABORT</type>
</action>
</output>
</outputs>
```
- listen: Which events trigger this one.
- agent: Which agent handles this event. Every agent has the name of the agent, and the exact model of the agent (like `claude-3-5-sonnet-20241022` or others)
IMPORTANT RULES:
0. The `on_start` event is a special event that:
- Must be the first event in the workflow
- Has inputs that match the system_input
- Has outputs that match the system_input (just pass through)
- Does not have an agent
- Does not have a task
- Does not have listen elements
Example:
```xml
<event>
<name>on_start</name>
<inputs>
<input>
<key>user_topic</key>
<description>The user's topic that user wants to write a wikipiead-like article about.</description>
</input>
</inputs>
<outputs>
<output>
<key>user_topic</key>
<description>The user's topic that user wants to write a wikipiead-like article about.</description>
<action>
<type>RESULT</type>
</action>
</output>
</outputs>
</event>
```
1. For simple sequential flows:
- Use single output with RESULT type
- No condition is needed
- Next event in chain listening to this event will be triggered automatically
2. For conditional flows:
- Multiple outputs must each have a condition
- Conditions should be mutually exclusive
- Each output should specify appropriate action type
- `GOTO` action should have a value which is the name of the event to go to
3. Only include tools section when:
- Agent is new (category="new") AND
- User explicitly requests specific tools for the agent
4. Omit tools section when:
- Using existing agents (category="existing") OR
- Creating new agents without specific tool requirements
""" + \
f"""
Existing tools you can use is:
{list_tools(context_variables)}
Existing agents you can use is:
{list_agents(context_variables)}
The name of existing workflows: [{workflow_list_str}]. The name of the new workflow you are creating should be DIFFERENT from these names according to the speciality of the workflow.
""" + \
r"""
COMMON WORKFLOW PATTERNS:
1. If-Else Pattern (Conditional Branching):
```xml
<event>
<name>analyze_data</name>
<task>Analyze the data and determine next steps</task>
<outputs>
<output>
<key>positive_case</key>
<description>Handle positive case</description>
<condition>If data meets criteria A</condition>
<action>
<type>RESULT</type>
</action>
</output>
<output>
<key>negative_case</key>
<description>Handle the negative case</description>
<condition>If data does not meet criteria A</condition>
<action>
<type>ABORT</type>
</action>
</output>
</outputs>
</event>
```
2. Parallelization Pattern (Concurrent Execution):
```xml
<!-- Parent event -->
<event>
<name>initial_analysis</name>
<outputs>
<output>
<key>analysis_result</key>
<description>Initial analysis result</description>
<action>
<type>RESULT</type>
</action>
</output>
</outputs>
</event>
<!-- Multiple events listening to the same parent -->
<event>
<name>technical_analysis</name>
<listen>
<event>initial_analysis</event>
</listen>
<outputs>
<output>
<key>technical_result</key>
<description>Technical analysis result</description>
<action>
<type>RESULT</type>
</action>
</output>
</outputs>
</event>
<event>
<name>financial_analysis</name>
<listen>
<event>initial_analysis</event>
</listen>
<outputs>
<output>
<key>financial_result</key>
<description>Financial analysis result</description>
<action>
<type>RESULT</type>
</action>
</output>
</outputs>
</event>
<!-- Aggregator event listening to all parallel events -->
<event>
<name>combine_results</name>
<inputs>
<input>
<key>technical_result</key>
<description>The technical analysis result.</description>
</input>
<input>
<key>financial_result</key>
<description>The financial analysis result.</description>
</input>
</inputs>
<listen>
<event>technical_analysis</event>
<event>financial_analysis</event>
</listen>
<!-- This event will only execute when ALL listened events complete -->
</event>
```
3. Evaluator-Optimizer Pattern (Iterative Refinement):
```xml
<event>
<name>generate_content</name>
<outputs>
<output>
<key>content</key>
<description>Generated content</description>
<action>
<type>RESULT</type>
</action>
</output>
</outputs>
</event>
<event>
<name>evaluate_content</name>
<listen>
<event>generate_content</event>
</listen>
<task>Evaluate the quality of generated content</task>
<outputs>
<output>
<key>approved</key>
<description>Content meets quality standards</description>
<condition>If quality score >= threshold</condition>
<action>
<type>RESULT</type>
</action>
</output>
<output>
<key>needs_improvement</key>
<description>Content needs improvement</description>
<condition>If quality score < threshold</condition>
<action>
<type>GOTO</type>
<value>generate_content</value>
</action>
</output>
</outputs>
</event>
```
IMPORTANT NOTES ON PATTERNS:
0. The above patterns are incomplete which some mandatory elements are missing due to the limitation of context length. In real-world, you could refer to the logic of the patterns to create a complete and correct workflow.
1. If-Else Pattern:
- Use mutually exclusive conditions
- You can NOT place MORE THAN ONE OUTPUT with RESULT type
- Outputs determine which branch executes
2. Parallelization Pattern:
- Multiple events can listen to the same parent event
- Aggregator event must list ALL parallel events in its listen section
- All parallel events must complete before aggregator executes
- Model of agents in every parallel event could be different
3. Evaluator-Optimizer Pattern:
- Use GOTO action for iteration
- Include clear evaluation criteria in conditions
- Have both success and retry paths
- Consider adding maximum iteration limit in global_variables
""" + \
r"""
EXAMPLE:
User: I want to build a workflow that can help me to write a wikipiead-like article about the user's topic. It should:
1. Search the web for the user's topic.
2. Write an outline for the user's topic.
3. Evaluate the outline. If the outline is not good enough, repeat the outline step, otherwise, continue to write the article.
4. Write the article.
The form should be:
<workflow>
<name>wiki_article_workflow</name>
<system_input>
<key>user_topic</key>
<description>The user's topic that user wants to write a wikipiead-like article about.</description>
</system_input>
<system_output>
<key>article</key>
<description>The article that satisfies the user's request.</description>
</system_output>
<agents>
<agent category="existing">
<name>Web Surfer Agent</name>
<description>This agent is used to search the web for the user's topic.</description>
</agent>
<agent category="new">
<name>Outline Agent</name>
<description>This agent is used to write an outline for the user's topic.</description>
</agent>
<agent category="new">
<name>Evaluator Agent</name>
<description>This agent is used to evaluate the outline of the user's topic.</description>
</agent>
<agent category="new">
<name>Article Writer Agent</name>
<description>This agent is used to write the article for the user's topic.</description>
</agent>
</agents>
<events>
<event>
<name>on_start</name>
<inputs>
<input>
<key>user_topic</key>
<description>The user's topic that user wants to write a wikipiead-like article about.</description>
</input>
</inputs>
<outputs>
<output>
<key>user_topic</key>
<description>The user's topic that user wants to write a wikipiead-like article about.</description>
<action>
<type>RESULT</type>
</action>
</output>
</outputs>
</event>
<event>
<name>on_search</name>
<inputs>
<input>
<key>user_topic</key>
<description>The user's topic that user wants to write a wikipiead-like article about.</description>
</input>
</inputs>
<task>
search the information about the topic and return the result.
</task>
<outputs>
<output>
<key>search_result</key>
<description>The search result of the user's topic.</description>
<action>
<type>RESULT</type>
</action>
</output>
</outputs>
<listen>
<event>on_start</event>
</listen>
<agent>
<name>Web Surfer Agent</name>
<model>claude-3-5-sonnet-20241022</model>
</agent>
</event>
<event>
<name>on_outline</name>
<inputs>
<input>
<key>search_result</key>
<description>The search result of the user's topic.</description>
</input>
</inputs>
<task>
write an outline for the user's topic.
</task>
<outputs>
<output>
<key>outline</key>
<description>The outline of the user's topic.</description>
<action>
<type>RESULT</type>
</action>
</output>
</outputs>
<listen>
<event>on_start</event>
</listen>
<agent>
<name>Outline Agent</name>
<model>claude-3-5-sonnet-20241022</model>
</agent>
</event>
<event>
<name>on_evaluate</name>
<inputs>
<input>
<key>outline</key>
<description>The outline of the user's topic.</description>
</input>
</inputs>
<task>
evaluate the outline of the user's topic.
</task>
<outputs>
<output>
<key>positive_feedback</key>
<description>The positive feedback of the outline of the user's topic.</description>
<condition>
If the outline is good enough, give positive feedback.
</condition>
<action>
<type>RESULT</type>
</action>
</output>
<output>
<key>negative_feedback</key>
<description>The negative feedback of the outline of the user's topic.</description>
<condition>
If the outline is not good enough, give negative feedback.
</condition>
<action>
<type>GOTO</type>
<value>on_outline</value>
</action>
</output>
</outputs>
<listen>
<event>on_outline</event>
</listen>
<agent>
<name>Evaluator Agent</name>
<model>claude-3-5-sonnet-20241022</model>
</agent>
</event>
<event>
<name>on_write</name>
<inputs>
<input>
<key>outline</key>
<description>The outline of user's topic.</description>
</input>
</inputs>
<task>
write the article for the user's topic.
</task>
<outputs>
<output>
<key>article</key>
<description>The article of the user's topic.</description>
<action>
<type>RESULT</type>
</action>
</output>
</outputs>
<listen>
<event>on_evaluate</event>
</listen>
<agent>
<name>Article Writer Agent</name>
<model>claude-3-5-sonnet-20241022</model>
</agent>
</event>
</events>
</workflow>
GUIDELINES:
1. Each event should have clear inputs and outputs
2. Use conditions to handle different outcomes
3. Properly chain events using the listen element
4. Review steps should be included for quality control
5. Action types should be either RESULT or ABORT
Follow these examples and guidelines to create appropriate workflow forms based on user requirements.
"""
return Agent(
name = "Workflow Former Agent",
model = model,
instructions = instructions,
)
if __name__ == "__main__":
from autoagent import MetaChain
agent = get_workflow_former_agent("claude-3-5-sonnet-20241022")
client = MetaChain()
# task_yaml = """\
# I want to create a workflow that can help me to solving the math problem.
# The workflow should:
# 2. Parallelize solving the math problem with the same `Math Solver Agent` using different language models (`gpt-4o-2024-08-06`, `claude-3-5-sonnet-20241022`, `deepseek/deepseek-chat`)
# 3. Aggregate the results from the `Math Solver Agent` and return the final result using majority voting.
# Please create the form of this workflow in the XML format.
# """
task_yaml = """\
I want to create a workflow that can help me to solving the math problem.
The workflow should:
1. The `Objective Extraction Agent` will extract the objective of the math problem.
2. The `Condition Extraction Agent` will extract the conditions of the math problem.
3. The `Math Solver Agent` will evaluate whether the conditions are enough to solve the math problem: if yes, solve the math problem; if no, return to the `Condition Extraction Agent` to extract more conditions.
Please create the form of this workflow in the XML format.
"""
task_yaml = task_yaml + """\
Directly output the form in the XML format.
"""
messages = [{"role": "user", "content": task_yaml}]
response = client.run(agent, messages)
print(response.messages[-1]["content"])
autoagent/agents/meta_agent/worklow_form_complie.py
+324
View File
@@ -0,0 +1,324 @@
from pydantic import BaseModel, Field, field_validator, ValidationInfo, model_validator
from typing import List, Dict, Optional, Literal, Union
import xml.etree.ElementTree as ET
import re
# 基础模型
class WorkflowFormParseError(Exception):
"""Exception raised when WorkflowForm failed to parse.
"""
def __init__(self, message):
super().__init__(message)
class WorkflowConstraintError(Exception):
"""Exception raised when WorkflowForm failed to parse. Use this Exception to raise when the workflow form does not meet some specific constraints.
"""
def __init__(self, message):
super().__init__(message)
class KeyDescription(BaseModel):
key: str
description: str
class Tool(BaseModel):
name: str
description: str
class Action(BaseModel):
type: Literal["RESULT", "ABORT", "GOTO"]
value: Optional[str] = None
@field_validator('value')
def validate_goto_value(cls, v, info: ValidationInfo):
if info.data.get('type') == 'GOTO' and not v:
raise WorkflowConstraintError("GOTO action must have a value")
return v
class Output(BaseModel):
key: str
description: str
condition: Optional[str] = None
action: Action
@field_validator('condition')
def validate_condition(cls, v, info: ValidationInfo):
"""验证condition的存在性"""
outputs_info = info.data.get('_outputs_info', {})
if outputs_info.get('multiple_outputs', False) and not v:
raise WorkflowConstraintError("Multiple outputs must each have a condition")
return v
class Event(BaseModel):
name: str
inputs: Optional[List[KeyDescription]] = None # 修改这里
task: Optional[str] = None # 修改为可选
outputs: List[Output]
listen: Optional[List[str]] = None
agent: Optional[Dict[str, str]] = None # 修改为可选
@field_validator('task')
def validate_task(cls, v, info: ValidationInfo):
"""验证非on_start事件必须有task"""
if info.data.get('name') != 'on_start' and not v:
raise WorkflowConstraintError("Non-start events must have a task")
return v
@field_validator('agent')
def validate_agent(cls, v, info: ValidationInfo):
"""验证非on_start事件必须有agent"""
if info.data.get('name') != 'on_start' and not v:
raise WorkflowConstraintError("Non-start events must have an agent")
return v
@field_validator('listen')
def validate_listen(cls, v, info: ValidationInfo):
"""验证on_start事件不能有listen"""
if info.data.get('name') == 'on_start' and v:
raise WorkflowConstraintError("Start event cannot have listen elements")
return v
@field_validator('name')
def validate_start_event(cls, v, info: ValidationInfo):
"""验证起始事件的名称"""
if info.data.get('is_start_event', False) and v != "on_start":
raise WorkflowConstraintError("Start event must be named 'on_start'")
return v
@field_validator('outputs')
def validate_start_event_outputs(cls, v, info: ValidationInfo):
"""验证on_start事件的输出必须与输入相同"""
if info.data.get('name') == 'on_start':
inputs = info.data.get('inputs', [])
if len(v) != len(inputs):
raise WorkflowConstraintError("Start event outputs must match inputs")
for output, input in zip(v, inputs):
if output.key != input.key or output.description != input.description:
raise WorkflowConstraintError("Start event output must match input")
return v
@field_validator('outputs')
def validate_outputs(cls, v):
"""验证输出的合法性"""
result_outputs = [out for out in v if out.action.type == "RESULT"]
if len(result_outputs) > 1:
raise WorkflowConstraintError("Cannot have more than one RESULT type output")
return v
@model_validator(mode='after')
def validate_event_constraints(self) -> 'Event':
"""验证事件的所有约束"""
# 如果是 on_start event，跳过输入验证
if self.name == "on_start":
return self
# 验证非on_start事件的输入
if self.inputs is None:
raise WorkflowConstraintError(f"Event '{self.name}': Non-start events must have inputs")
# 验证listen是否存在
if self.listen is None:
raise WorkflowConstraintError(f"Event '{self.name}': Non-start events must have listen events")
# 验证输入数量
if len(self.inputs) != len(self.listen):
raise WorkflowConstraintError(
f"Event '{self.name}': Number of inputs ({len(self.inputs)}) must match number of listen events ({len(self.listen)})"
)
return self
class Agent(BaseModel):
name: str
description: str
category: Literal["existing", "new"]
tools: Optional[List[Tool]] = None
@field_validator('tools')
def validate_tools(cls, v, info: ValidationInfo):
"""验证tools的存在性"""
if info.data.get('category') == 'existing' and v:
raise WorkflowConstraintError("Existing agents should not have tools defined")
return v
class WorkflowForm(BaseModel):
name: str
system_input: KeyDescription
system_output: KeyDescription
global_variables: Dict[str, str] = Field(default_factory=dict)
agents: List[Agent]
events: List[Event]
@field_validator('events')
def validate_events(cls, v):
"""验证事件流的合法性"""
# 验证是否有且仅有一个on_start事件
start_events = [e for e in v if e.name == "on_start"]
if len(start_events) != 1:
raise WorkflowConstraintError("Must have exactly one 'on_start' event")
# 验证事件监听的合法性
event_names = {e.name for e in v}
for event in v:
if event.listen:
for listened_event in event.listen:
if listened_event not in event_names:
raise WorkflowConstraintError(f"Event {event.name} listens to non-existent event {listened_event}")
return v
@model_validator(mode='after')
def validate_event_order(self) -> 'WorkflowForm':
"""验证事件的监听顺序：
1. 事件只能监听在它之前定义的事件
2. 不能有循环依赖
"""
# 创建事件名称到索引的映射
event_indices = {event.name: idx for idx, event in enumerate(self.events)}
# 验证每个事件的监听关系
for idx, event in enumerate(self.events):
if event.listen:
for listened_event_name in event.listen:
# 检查被监听的事件是否存在
if listened_event_name not in event_indices:
raise WorkflowConstraintError(
f"Event '{event.name}': Referenced listen event '{listened_event_name}' not found"
)
# 检查是否监听了后面的事件
listened_idx = event_indices[listened_event_name]
if listened_idx >= idx:
raise WorkflowConstraintError(
f"Event '{event.name}' cannot listen to event '{listened_event_name}' "
f"because it appears later in the workflow or creates a cycle"
)
return self
class XMLParser:
@staticmethod
def parse_key_description(elem: ET.Element) -> KeyDescription:
return KeyDescription(
key=elem.find('key').text.strip(),
description=elem.find('description').text.strip()
)
@staticmethod
def parse_action(elem: ET.Element) -> Action:
action_elem = elem.find('action')
return Action(
type=action_elem.find('type').text.strip(),
value=action_elem.find('value').text.strip() if action_elem.find('value') is not None else None
)
@staticmethod
def parse_output(elem: ET.Element, multiple_outputs: bool) -> Output:
return Output(
key=elem.find('key').text.strip(),
description=elem.find('description').text.strip(),
condition=elem.find('condition').text.strip() if elem.find('condition') is not None else None,
action=XMLParser.parse_action(elem),
_outputs_info={'multiple_outputs': multiple_outputs}
)
@staticmethod
def parse_event(elem: ET.Element, is_start: bool = False) -> Event:
name = elem.find('name').text.strip()
is_start = name == 'on_start'
outputs_elem = elem.find('outputs')
multiple_outputs = len(outputs_elem.findall('output')) > 1
listen_elem = elem.find('listen')
listen = [e.text.strip() for e in listen_elem.findall('event')] if listen_elem is not None and not is_start else None
agent_elem = elem.find('agent')
agent = {
"name": agent_elem.find('name').text.strip(),
"model": agent_elem.find('model').text.strip()
} if agent_elem is not None and not is_start else None
inputs_elem = elem.find('inputs')
inputs = [XMLParser.parse_key_description(input_elem)
for input_elem in inputs_elem.findall('input')] if inputs_elem is not None else None
task_elem = elem.find('task')
task = task_elem.text.strip() if task_elem is not None and not is_start else None
return Event(
name=name,
inputs=inputs,
task=task,
outputs=[XMLParser.parse_output(out, multiple_outputs)
for out in outputs_elem.findall('output')],
listen=listen,
agent=agent,
is_start_event=is_start
)
@staticmethod
def parse_agent(elem: ET.Element) -> Agent:
tools_elem = elem.find('tools')
tools = None
if tools_elem is not None:
tools = [Tool(
name=tool.find('name').text.strip(),
description=tool.find('description').text.strip()
) for tool in tools_elem.findall('tool')]
return Agent(
name=elem.find('name').text.strip(),
description=elem.find('description').text.strip(),
category=elem.get('category'),
tools=tools
)
@classmethod
def parse_xml(cls, xml_content: str) -> WorkflowForm:
root = ET.fromstring(xml_content)
workflow_name = root.get('name')
if not workflow_name:
# If name attribute doesn't exist, try to find name element
name_elem = root.find('name')
workflow_name = name_elem.text.strip() if name_elem is not None else "Unnamed Workflow"
return WorkflowForm(
name=workflow_name,
system_input=cls.parse_key_description(root.find('system_input')),
system_output=cls.parse_key_description(root.find('system_output')),
global_variables={var.find('key').text.strip(): var.find('value').text.strip()
for var in root.find('global_variables').findall('variable')}
if root.find('global_variables') is not None else {},
agents=[cls.parse_agent(agent) for agent in root.findall('.//agents/agent')],
events=[cls.parse_event(event, event.find('name').text.strip() == 'on_start')
for event in root.findall('.//events/event')]
)
def extract_workflow_content(text):
pattern = r'(<workflow>.*?</workflow>)'
# re.DOTALL 让 . 也能匹配换行符
match = re.search(pattern, text, re.DOTALL)
if match:
return match.group(1)
else:
raise WorkflowFormParseError("The workflow XML form is not correct. The workflow XML form should be wrapped by <workflow>...</workflow> tags.")
def parse_workflow_form(xml_content: str) -> Optional[WorkflowForm]:
"""
读取并解析workflow form XML文件
Args:
xml_content: XML文件内容
Returns:
解析后的WorkflowForm对象，如果解析失败返回None
"""
try:
workflow_content = extract_workflow_content(xml_content)
return XMLParser.parse_xml(workflow_content)
except WorkflowFormParseError as e:
return f"The Error to extract workflow content: {e}"
except WorkflowConstraintError as e:
return f"The generated workflow form MUST meet all the constraints in the given instructions, but the constraints are not met: {e}"
except ET.ParseError as e:
return f"The Error parsing XML workflow form: {e}"
except Exception as e:
return f"Unexpected error: {e}"
autoagent/agents/system_agent/filesurfer_agent.py
+43
View File
@@ -0,0 +1,43 @@
from autoagent.types import Agent
from autoagent.registry import register_agent
from autoagent.tools import open_local_file, page_up_markdown, page_down_markdown, find_on_page_ctrl_f, find_next, visual_question_answering
from autoagent.tools.file_surfer_tool import with_env
from autoagent.environment.markdown_browser import RequestsMarkdownBrowser
import time
from inspect import signature
from constant import LOCAL_ROOT, DOCKER_WORKPLACE_NAME
@register_agent(name = "File Surfer Agent", func_name="get_filesurfer_agent")
def get_filesurfer_agent(model: str = "gpt-4o", **kwargs):
def handle_mm_func(tool_name, tool_args):
return f"After using tool `{tool_name}({tool_args})`, I have opened the image I want to see and prepared a question according to the image. Please answer the question based on the image."
def instructions(context_variables):
file_env: RequestsMarkdownBrowser = context_variables.get("file_env", None)
assert file_env is not None, "file_env is required"
return \
f"""
You are a file surfer agent that can handle local files.
You can only access the files in the folder `{file_env.docker_workplace}` and when you want to open a file, you should use absolute path from root like `{file_env.docker_workplace}/...`.
Note that `open_local_file` can read a file as markdown text and ask questions about it. And `open_local_file` can handle the following file extensions: [".html", ".htm", ".xlsx", ".pptx", ".wav", ".mp3", ".flac", ".pdf", ".docx"], and all other types of text files.
But IT DOES NOT HANDLE IMAGES, you should use `visual_question_answering` to see the image.
If the converted markdown text has more than 1 page, you can use `page_up`, `page_down`, `find_on_page_ctrl_f`, `find_next` to navigate through the pages.
When you think you have completed the task the `System Triage Agent` asked you to do, you should use `transfer_back_to_triage_agent` to transfer the conversation back to the `System Triage Agent`. And you should not stop to try to solve the user's request by transferring to `System Triage Agent` only until the task is completed.
If you are unable to open the file, you can transfer the conversation back to the `System Triage Agent`, and let the `Coding Agent` try to solve the problem by coding.
"""
tool_list = [open_local_file, page_up_markdown, page_down_markdown, find_on_page_ctrl_f, find_next, visual_question_answering]
return Agent(
name="File Surfer Agent",
model=model,
instructions=instructions,
functions=tool_list,
handle_mm_func=handle_mm_func,
tool_choice = "required",
parallel_tool_calls = False
)
autoagent/agents/system_agent/programming_agent.py
+98
View File
@@ -0,0 +1,98 @@
from autoagent.types import Agent
from autoagent.tools import (
gen_code_tree_structure, execute_command, read_file, create_file, write_file, list_files, create_directory, run_python, terminal_page_up, terminal_page_down, terminal_page_to
)
from autoagent.util import make_message, make_tool_message
from autoagent.registry import register_agent, register_plugin_agent
from constant import LOCAL_ROOT, DOCKER_WORKPLACE_NAME
from autoagent.environment import DockerEnv, BrowserEnv, LocalEnv
from typing import Union
from inspect import signature
def examples(context_variables):
working_dir = context_variables.get("working_dir", None)
examples_list = []
examples_list.extend(make_message('user', "Create a list of numbers from 1 to 10, and display them in a web page at port 5000."))
examples_list.extend(make_message('assistant', "I should first use create_file to write the python code into a file named 'app.py' for starting a web server"))
examples_list.extend(make_tool_message(create_file, {'path': f"/{working_dir}/app.py",
'content': """
from flask import Flask
app = Flask(__name__)
@app.route('/')
def index():
numbers = list(range(1, 11))
return str(numbers)
if __name__ == '__main__':
app.run(port=5000)"""}, f"File created at: /{working_dir}/app.py"))
examples_list.extend(make_message('assistant', 'I have created a Python file `app.py` that will display a list of numbers from 1 to 10 when you run it. Let me run the Python file for you using `run_python`'))
examples_list.extend(make_tool_message(run_python, {'code_path': f"/{working_dir}/app.py"}, f"""
Traceback (most recent call last):
File "/{working_dir}/app.py", line 2, in <module>
from flask import Flask
ModuleNotFoundError: No module named 'flask'"""))
examples_list.extend(make_message('assistant', "It seems that Flask is not installed. Let me install Flask for you using `execute_command` by the command: pip install flask"))
examples_list.extend(make_tool_message(execute_command, {'command': 'pip install flask'}, """Defaulting to user installation because normal site-packages is not writeable
Collecting flask
Using cached flask-3.0.3-py3-none-any.whl (101 kB)
Collecting blinker>=1.6.2
Using cached blinker-1.7.0-py3-none-any.whl (13 kB)
Collecting Werkzeug>=3.0.0
Using cached werkzeug-3.0.2-py3-none-any.whl (226 kB)
Collecting click>=8.1.3
Using cached click-8.1.7-py3-none-any.whl (97 kB)
Collecting itsdangerous>=2.1.2
Using cached itsdangerous-2.2.0-py3-none-any.whl (16 kB)
Requirement already satisfied: Jinja2>=3.1.2 in /home/openhands/.local/lib/python3.10/site-packages (from flask) (3.1.3)
Requirement already satisfied: MarkupSafe>=2.0 in /home/openhands/.local/lib/python3.10/site-packages (from Jinja2>=3.1.2->flask) (2.1.5)
Installing collected packages: Werkzeug, itsdangerous, click, blinker, flask
Successfully installed Werkzeug-3.0.2 blinker-1.7.0 click-8.1.7 flask-3.0.3 itsdangerous-2.2.0"""))
examples_list.extend(make_message('assistant', "Now that Flask is installed, let me run the Python file again using `run_python`"))
examples_list.extend(make_tool_message(run_python, {'code_path': f"/{working_dir}/app.py"}, """* Serving Flask app 'app'
* Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
* Running on http://127.0.0.1:5000
Press CTRL+C to quit"""))
examples_list.extend(make_message('assistant', """The server is running on port 5000 with PID 124. You can access the list of numbers by visiting http://127.0.0.1:5000. Your task is completed."""))
return examples_list
@register_agent(name= "Coding Agent", func_name="get_coding_agent")
@register_plugin_agent(name= "Coding Agent", func_name="get_coding_agent")
def get_coding_agent(model: str, **kwargs):
def instructions(context_variables):
code_env: Union[DockerEnv, LocalEnv] = context_variables.get("code_env", LocalEnv())
return f"""You are a helpful programming assistant that can write and execute code. You are working in the folder: `{code_env.docker_workplace}`, and you can only access the files in this folder.
Your can leverage your capabilities by using the specific functions listed below:
1. Creating project structures based on the user requirement using function `create_directory`.
2. Writing clean, efficient, and well-documented code using function `create_file` and `write_file`.
3. You must run python scripts using function `run_python` rather than using the `execute_command` function.
4. Exam the project to re-use the existing code snippets as much as possible, you may need to use
functions like `list_files`, `read_file` and `write_file`.
5. Writing the code into the file when creating new files, do not create empty files.
6. Before you write code into the existing files, you should first read the file content using function `read_file` and reserve the original content as much as possible.
7. Decide whether the task requires execution and debugging before moving to the next or not.
8. Generate the commands to run and test the current task, and the dependencies list for this task.
9. You only write Python scripts, don't write Jupiter notebooks which require interactive execution.
10. Note that every path you read, write, or search should be the absolute path (starting with '/').
11. If you should use programming other than Python, you should use the `write_file` function to write the code into a file, and then use the `execute_command` function to run the code.
12. If the terminal output is too long, you should use `terminal_page_up` to move the viewport up, `terminal_page_down` to move the viewport down, `terminal_page_to` to move the viewport to the specific page of terminal where the meaningful content is.
Note that you can use this agent to make complex computation, write a api request, and anything else that can be done by writing code.
When you think you have completed the task the `System Triage Agent` asked you to do, you should use `transfer_back_to_triage_agent` to transfer the conversation back to the `System Triage Agent`. And you should not stop to try to solve the user's request by transferring to `System Triage Agent` only until the task is completed.
[IMPORTANT] You can only complete the task by coding. Talk is cheap, show me the code with tools.
"""
tool_list = [gen_code_tree_structure, execute_command, read_file, create_file, write_file, list_files, create_directory, run_python, terminal_page_up, terminal_page_down, terminal_page_to]
return Agent(
name="Coding Agent",
model=model,
instructions=instructions,
functions=tool_list,
# examples=examples,
tool_choice = "required",
parallel_tool_calls = False
)
autoagent/agents/system_agent/system_triage_agent.py
+66
View File
@@ -0,0 +1,66 @@
from .filesurfer_agent import get_filesurfer_agent
from .programming_agent import get_coding_agent
from .websurfer_agent import get_websurfer_agent
from autoagent.registry import register_agent
from autoagent.types import Agent, Result
from autoagent.tools.inner import case_resolved, case_not_resolved
@register_agent(name = "System Triage Agent", func_name="get_system_triage_agent")
def get_system_triage_agent(model: str, **kwargs):
"""
This is the `System Triage Agent`, it can help the user to determine which agent is best suited to handle the user's request under the current context, and transfer the conversation to that agent.
Args:
model: The model to use for the agent.
**kwargs: Additional keyword arguments, `file_env`, `web_env` and `code_env` are required.
"""
filesurfer_agent = get_filesurfer_agent(model)
websurfer_agent = get_websurfer_agent(model)
coding_agent = get_coding_agent(model)
instructions = \
f"""You are a helpful assistant that can help the user with their request.
Based on the state of solving user's task, your responsibility is to determine which agent is best suited to handle the user's request under the current context, and transfer the conversation to that agent. And you should not stop to try to solve the user's request by transferring to another agent only until the task is completed.
There are three agents you can transfer to:
1. use `transfer_to_filesurfer_agent` to transfer to {filesurfer_agent.name}, it can help you to open any type of local files and browse the content of them.
2. use `transfer_to_websurfer_agent` to transfer to {websurfer_agent.name}, it can help you to open any website and browse any content on it.
3. use `transfer_to_coding_agent` to transfer to {coding_agent.name}, it can help you to write code to solve the user's request, especially some complex tasks.
"""
tool_choice = "required"
tools = [case_resolved, case_not_resolved] if tool_choice == "required" else []
system_triage_agent = Agent(
name="System Triage Agent",
model=model,
instructions=instructions,
functions=tools,
tool_choice = tool_choice,
parallel_tool_calls = False,
)
def transfer_to_filesurfer_agent(sub_task_description: str):
"""
Args:
sub_task_description: The detailed description of the sub-task that the `System Triage Agent` will ask the `File Surfer Agent` to do.
"""
return Result(value=sub_task_description, agent=filesurfer_agent)
def transfer_to_websurfer_agent(sub_task_description: str):
return Result(value=sub_task_description, agent=websurfer_agent)
def transfer_to_coding_agent(sub_task_description: str):
return Result(value=sub_task_description, agent=coding_agent)
def transfer_back_to_triage_agent(task_status: str):
"""
Args:
task_status: The detailed description of the task status after a sub-agent has finished its sub-task. A sub-agent can use this tool to transfer the conversation back to the `System Triage Agent` only when it has finished its sub-task.
"""
return Result(value=task_status, agent=system_triage_agent)
system_triage_agent.agent_teams = {
filesurfer_agent.name: transfer_to_filesurfer_agent,
websurfer_agent.name: transfer_to_websurfer_agent,
coding_agent.name: transfer_to_coding_agent
}
system_triage_agent.functions.extend([transfer_to_filesurfer_agent, transfer_to_websurfer_agent, transfer_to_coding_agent])
filesurfer_agent.functions.append(transfer_back_to_triage_agent)
websurfer_agent.functions.append(transfer_back_to_triage_agent)
coding_agent.functions.append(transfer_back_to_triage_agent)
return system_triage_agent
autoagent/agents/system_agent/websurfer_agent.py
+37
View File
@@ -0,0 +1,37 @@
from autoagent.types import Agent
from autoagent.registry import register_agent
from autoagent.tools import click, page_down, page_up, history_back, history_forward, web_search, input_text, sleep, visit_url, get_page_markdown
from autoagent.tools.web_tools import with_env
from autoagent.environment.browser_env import BrowserEnv
import time
from constant import DOCKER_WORKPLACE_NAME, LOCAL_ROOT
@register_agent(name = "Web Surfer Agent", func_name="get_websurfer_agent")
def get_websurfer_agent(model: str = "gpt-4o", **kwargs):
def handle_mm_func(tool_name, tool_args):
return f"After take last action `{tool_name}({tool_args})`, the image of current page is shown below. Please take next action based on the image, the current state of the page as well as previous actions and observations."
def instructions(context_variables):
web_env: BrowserEnv = context_variables.get("web_env", None)
assert web_env is not None, "web_env is required"
return \
f"""Review the current state of the page and all other information to find the best possible next action to accomplish your goal. Your answer will be interpreted and executed by a program, make sure to follow the formatting instructions.
Note that if you want to analyze the YouTube video, Wikipedia page, or other pages that contain media content, or you just want to analyze the text content of the page in a more detailed way, you should use `get_page_markdown` tool to convert the page information to markdown text. And when browsing the web, if you have downloaded any files, the path of the downloaded files will be `{web_env.docker_workplace}/downloads`, and you CANNOT open the downloaded files directly, you should transfer back to the `System Triage Agent`, and let `System Triage Agent` to transfer to `File Surfer Agent` to open the downloaded files.
When you think you have completed the task the `System Triage Agent` asked you to do, you should use `transfer_back_to_triage_agent` to transfer the conversation back to the `System Triage Agent`. And you should not stop to try to solve the user's request by transferring to `System Triage Agent` only until the task is completed.
"""
tool_list = [click, page_down, page_up, history_back, history_forward, web_search, input_text, sleep, visit_url, get_page_markdown]
return Agent(
name="Web Surfer Agent",
model=model,
instructions=instructions,
functions=tool_list,
handle_mm_func=handle_mm_func,
tool_choice = "required",
parallel_tool_calls = False
)
"""
Note that when you need to download something, you should first know the url of the file, and then use the `visit_url` tool to download the file. For example, if you want to download paper from 'https://arxiv.org/abs/2310.13023', you should use `visit_url('url'='https://arxiv.org/pdf/2310.13023.pdf')`.
"""
autoagent/agents/tool_retriver_agent.py
+21
View File
@@ -0,0 +1,21 @@
from autoagent.types import Agent
from autoagent.tools import (
get_api_plugin_tools_doc
)
from autoagent.util import make_message, make_tool_message
from autoagent.registry import register_agent
@register_agent(name = "Tool Retriver Agent", func_name="get_tool_retriver_agent")
def get_tool_retriver_agent(model: str):
def instructions(context_variables):
return \
f"""
You are a tool retriver agent.
You are given a task instruction, and you need to retrieve the tool docs for the task using function `get_tool_doc`.
Note that if you want to complete the task, you may need to use more than one tool, so you should retrieve the tool docs for all the tools you may need. Finally, you should give a merged tool doc consisting of all the tool docs you retrieved, and the implementation code of each tool should be included in the tool doc.
"""
return Agent(
name="Tool Retriver Agent",
model=model,
instructions=instructions,
functions=[get_api_plugin_tools_doc],
)
autoagent/cli.py
+426
View File
@@ -0,0 +1,426 @@
import click
import importlib
from autoagent import MetaChain
from autoagent.util import debug_print
import asyncio
from constant import DOCKER_WORKPLACE_NAME
from autoagent.io_utils import read_yaml_file, get_md5_hash_bytext, read_file
from autoagent.environment.utils import setup_metachain
from autoagent.types import Response
from autoagent import MetaChain
from autoagent.util import ask_text, single_select_menu, print_markdown, debug_print, UserCompleter
from prompt_toolkit import PromptSession
from prompt_toolkit.completion import Completer, Completion
from prompt_toolkit.formatted_text import HTML
from prompt_toolkit.styles import Style
from rich.progress import Progress, SpinnerColumn, TextColumn
import json
import argparse
from datetime import datetime
from autoagent.agents.meta_agent import tool_editor, agent_editor
from autoagent.tools.meta.edit_tools import list_tools
from autoagent.tools.meta.edit_agents import list_agents
from loop_utils.font_page import MC_LOGO, version_table, NOTES, GOODBYE_LOGO
from rich.live import Live
from autoagent.environment.docker_env import DockerEnv, DockerConfig, check_container_ports
from autoagent.environment.local_env import LocalEnv
from autoagent.environment.browser_env import BrowserEnv
from autoagent.environment.markdown_browser import RequestsMarkdownBrowser
from evaluation.utils import update_progress, check_port_available, run_evaluation, clean_msg
import os
import os.path as osp
from autoagent.agents import get_system_triage_agent
from autoagent.logger import LoggerManager, MetaChainLogger
from rich.console import Console
from rich.markdown import Markdown
from rich.table import Table
from rich.columns import Columns
from rich.text import Text
from rich.panel import Panel
import re
from autoagent.cli_utils.metachain_meta_agent import meta_agent
from autoagent.cli_utils.metachain_meta_workflow import meta_workflow
from autoagent.cli_utils.file_select import select_and_copy_files
from evaluation.utils import update_progress, check_port_available, run_evaluation, clean_msg
from constant import COMPLETION_MODEL
@click.group()
def cli():
"""The command line interface for autoagent"""
pass
@cli.command()
@click.option('--model', default='gpt-4o-2024-08-06', help='the name of the model')
@click.option('--agent_func', default='get_dummy_agent', help='the function to get the agent')
@click.option('--query', default='...', help='the user query to the agent')
@click.argument('context_variables', nargs=-1)
def agent(model: str, agent_func: str, query: str, context_variables):
"""
Run an agent with a given model, agent function, query, and context variables.
Args:
model (str): The name of the model.
agent_func (str): The function to get the agent.
query (str): The user query to the agent.
context_variables (list): The context variables to pass to the agent.
Usage:
mc agent --model=gpt-4o-2024-08-06 --agent_func=get_weather_agent --query="What is the weather in Tokyo?" city=Tokyo unit=C timestamp=2024-01-01
"""
context_storage = {}
for arg in context_variables:
if '=' in arg:
key, value = arg.split('=', 1)
context_storage[key] = value
agent_module = importlib.import_module(f'autoagent.agents')
try:
agent_func = getattr(agent_module, agent_func)
except AttributeError:
raise ValueError(f'Agent function {agent_func} not found, you shoud check in the `autoagent.agents` directory for the correct function name')
agent = agent_func(model)
mc = MetaChain()
messages = [
{"role": "user", "content": query}
]
response = mc.run(agent, messages, context_storage, debug=True)
debug_print(True, response.messages[-1]['content'], title = f'Result of running {agent.name} agent', color = 'pink3')
return response.messages[-1]['content']
@cli.command()
@click.option('--workflow_name', default=None, help='the name of the workflow')
@click.option('--system_input', default='...', help='the user query to the agent')
def workflow(workflow_name: str, system_input: str):
"""命令行函数的同步包装器"""
return asyncio.run(async_workflow(workflow_name, system_input))
async def async_workflow(workflow_name: str, system_input: str):
"""异步实现的workflow函数"""
workflow_module = importlib.import_module(f'autoagent.workflows')
try:
workflow_func = getattr(workflow_module, workflow_name)
except AttributeError:
raise ValueError(f'Workflow function {workflow_name} not found...')
result = await workflow_func(system_input) # 使用 await 等待异步函数完成
debug_print(True, result, title=f'Result of running {workflow_name} workflow', color='pink3')
return result
def clear_screen():
console = Console()
console.print("[bold green]Coming soon...[/bold green]")
print('\033[u\033[J\033[?25h', end='') # Restore cursor and clear everything after it, show cursor
def get_config(container_name, port, test_pull_name="main", git_clone=False):
container_name = container_name
port_info = check_container_ports(container_name)
if port_info:
port = port_info[0]
else:
# while not check_port_available(port):
# port += 1
# 使用文件锁来确保端口分配的原子性
import filelock
lock_file = os.path.join(os.getcwd(), ".port_lock")
lock = filelock.FileLock(lock_file)
with lock:
port = port
while not check_port_available(port):
port += 1
print(f'{port} is not available, trying {port+1}')
# 立即标记该端口为已使用
with open(os.path.join(os.getcwd(), f".port_{port}"), 'w') as f:
f.write(container_name)
local_root = os.path.join(os.getcwd(), f"workspace_meta_showcase", f"showcase_{container_name}")
os.makedirs(local_root, exist_ok=True)
docker_config = DockerConfig(
workplace_name=DOCKER_WORKPLACE_NAME,
container_name=container_name,
communication_port=port,
conda_path='/root/miniconda3',
local_root=local_root,
test_pull_name=test_pull_name,
git_clone=git_clone
)
return docker_config
def create_environment(docker_config: DockerConfig):
"""
1. create the code environment
2. create the web environment
3. create the file environment
"""
code_env = DockerEnv(docker_config)
code_env.init_container()
web_env = BrowserEnv(browsergym_eval_env = None, local_root=docker_config.local_root, workplace_name=docker_config.workplace_name)
file_env = RequestsMarkdownBrowser(viewport_size=1024 * 5, local_root=docker_config.local_root, workplace_name=docker_config.workplace_name, downloads_folder=os.path.join(docker_config.local_root, docker_config.workplace_name, "downloads"))
return code_env, web_env, file_env
def create_environment_local(docker_config: DockerConfig):
"""
1. create the code environment
2. create the web environment
3. create the file environment
"""
code_env = LocalEnv(docker_config)
web_env = BrowserEnv(browsergym_eval_env = None, local_root=docker_config.local_root, workplace_name=docker_config.workplace_name)
file_env = RequestsMarkdownBrowser(viewport_size=1024 * 5, local_root=docker_config.local_root, workplace_name=docker_config.workplace_name, downloads_folder=os.path.join(docker_config.local_root, docker_config.workplace_name, "downloads"))
return code_env, web_env, file_env
def update_guidance(context_variables):
console = Console()
# print the logo
logo_text = Text(MC_LOGO, justify="center")
console.print(Panel(logo_text, style="bold salmon1", expand=True))
console.print(version_table)
console.print(Panel(NOTES,title="Important Notes", expand=True))
@cli.command(name='main') # 修改这里，使用连字符
@click.option('--container_name', default='auto_agent', help='the function to get the agent')
@click.option('--port', default=12347, help='the port to run the container')
@click.option('--test_pull_name', default='autoagent_mirror', help='the name of the test pull')
@click.option('--git_clone', default=True, help='whether to clone a mirror of the repository')
@click.option('--local_env', default=False, help='whether to use local environment')
def main(container_name: str, port: int, test_pull_name: str, git_clone: bool, local_env: bool):
"""
Run deep research with a given model, container name, port
"""
model = COMPLETION_MODEL
print('\033[s\033[?25l', end='') # Save cursor position and hide cursor
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
transient=True # 这会让进度条完成后消失
) as progress:
task = progress.add_task("[cyan]Initializing...", total=None)
progress.update(task, description="[cyan]Initializing config...[/cyan]\n")
docker_config = get_config(container_name, port, test_pull_name, git_clone)
progress.update(task, description="[cyan]Setting up logger...[/cyan]\n")
log_path = osp.join("casestudy_results", 'logs', f'agent_{container_name}_{model}.log')
LoggerManager.set_logger(MetaChainLogger(log_path = None))
progress.update(task, description="[cyan]Creating environment...[/cyan]\n")
if local_env:
code_env, web_env, file_env = create_environment_local(docker_config)
else:
code_env, web_env, file_env = create_environment(docker_config)
progress.update(task, description="[cyan]Setting up autoagent...[/cyan]\n")
clear_screen()
context_variables = {"working_dir": docker_config.workplace_name, "code_env": code_env, "web_env": web_env, "file_env": file_env}
# select the mode
while True:
update_guidance(context_variables)
mode = single_select_menu(['user mode', 'agent editor', 'workflow editor', 'exit'], "Please select the mode:")
match mode:
case 'user mode':
clear_screen()
user_mode(model, context_variables, False)
case 'agent editor':
clear_screen()
meta_agent(model, context_variables, False)
case 'workflow editor':
clear_screen()
meta_workflow(model, context_variables, False)
case 'exit':
console = Console()
logo_text = Text(GOODBYE_LOGO, justify="center")
console.print(Panel(logo_text, style="bold salmon1", expand=True))
break
def user_mode(model: str, context_variables: dict, debug: bool = True):
logger = LoggerManager.get_logger()
console = Console()
system_triage_agent = get_system_triage_agent(model)
assert system_triage_agent.agent_teams != {}, "System Triage Agent must have agent teams"
messages = []
agent = system_triage_agent
agents = {system_triage_agent.name.replace(' ', '_'): system_triage_agent}
for agent_name in system_triage_agent.agent_teams.keys():
agents[agent_name.replace(' ', '_')] = system_triage_agent.agent_teams[agent_name]("placeholder").agent
agents["Upload_files"] = "select"
style = Style.from_dict({
'bottom-toolbar': 'bg:#333333 #ffffff',
})
# 创建会话
session = PromptSession(
completer=UserCompleter(agents.keys()),
complete_while_typing=True,
style=style
)
client = MetaChain(log_path=logger)
upload_infos = []
while True:
# query = ask_text("Tell me what you want to do:")
query = session.prompt(
'Tell me what you want to do (type "exit" to quit): ',
bottom_toolbar=HTML('<b>Prompt:</b> Enter <b>@</b> to mention Agents')
)
if query.strip().lower() == 'exit':
# logger.info('User mode completed. See you next time! :waving_hand:', color='green', title='EXIT')
logo_text = "User mode completed. See you next time! :waving_hand:"
console.print(Panel(logo_text, style="bold salmon1", expand=True))
break
words = query.split()
console.print(f"[bold green]Your request: {query}[/bold green]", end=" ")
for word in words:
if word.startswith('@') and word[1:] in agents.keys():
# print(f"[bold magenta]{word}[bold magenta]", end=' ')
agent = agents[word.replace('@', '')]
else:
# print(word, end=' ')
pass
print()
if hasattr(agent, "name"):
agent_name = agent.name
console.print(f"[bold green][bold magenta]@{agent_name}[/bold magenta] will help you, be patient...[/bold green]")
if len(upload_infos) > 0:
query = "{}\n\nUser uploaded files:\n{}".format(query, "\n".join(upload_infos))
messages.append({"role": "user", "content": query})
response = client.run(agent, messages, context_variables, debug=debug)
messages.extend(response.messages)
model_answer_raw = response.messages[-1]['content']
# attempt to parse model_answer
if model_answer_raw.startswith('Case resolved'):
model_answer = re.findall(r'<solution>(.*?)</solution>', model_answer_raw, re.DOTALL)
if len(model_answer) == 0:
model_answer = model_answer_raw
else:
model_answer = model_answer[0]
else:
model_answer = model_answer_raw
console.print(f"[bold green][bold magenta]@{agent_name}[/bold magenta] has finished with the response:\n[/bold green] [bold blue]{model_answer}[/bold blue]")
agent = response.agent
elif agent == "select":
code_env: DockerEnv = context_variables["code_env"]
local_workplace = code_env.local_workplace
docker_workplace = code_env.docker_workplace
files_dir = os.path.join(local_workplace, "files")
docker_files_dir = os.path.join(docker_workplace, "files")
os.makedirs(files_dir, exist_ok=True)
upload_infos.extend(select_and_copy_files(files_dir, console, docker_files_dir))
agent = agents["System_Triage_Agent"]
else:
console.print(f"[bold red]Unknown agent: {agent}[/bold red]")
@cli.command(name='deep-research') # 修改这里，使用连字符
@click.option('--container_name', default='deepresearch', help='the function to get the agent')
@click.option('--port', default=12346, help='the port to run the container')
@click.option('--local_env', default=False, help='whether to use local environment')
def deep_research(container_name: str, port: int, local_env: bool):
"""
Run deep research with a given model, container name, port
"""
model = COMPLETION_MODEL
print('\033[s\033[?25l', end='') # Save cursor position and hide cursor
with Progress(
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
transient=True # 这会让进度条完成后消失
) as progress:
task = progress.add_task("[cyan]Initializing...", total=None)
progress.update(task, description="[cyan]Initializing config...[/cyan]\n")
docker_config = get_config(container_name, port)
progress.update(task, description="[cyan]Setting up logger...[/cyan]\n")
log_path = osp.join("casestudy_results", 'logs', f'agent_{container_name}_{model}.log')
LoggerManager.set_logger(MetaChainLogger(log_path = None))
progress.update(task, description="[cyan]Creating environment...[/cyan]\n")
if local_env:
code_env, web_env, file_env = create_environment_local(docker_config)
else:
code_env, web_env, file_env = create_environment(docker_config)
progress.update(task, description="[cyan]Setting up autoagent...[/cyan]\n")
clear_screen()
context_variables = {"working_dir": docker_config.workplace_name, "code_env": code_env, "web_env": web_env, "file_env": file_env}
update_guidance(context_variables)
logger = LoggerManager.get_logger()
console = Console()
system_triage_agent = get_system_triage_agent(model)
assert system_triage_agent.agent_teams != {}, "System Triage Agent must have agent teams"
messages = []
agent = system_triage_agent
agents = {system_triage_agent.name.replace(' ', '_'): system_triage_agent}
for agent_name in system_triage_agent.agent_teams.keys():
agents[agent_name.replace(' ', '_')] = system_triage_agent.agent_teams[agent_name]("placeholder").agent
agents["Upload_files"] = "select"
style = Style.from_dict({
'bottom-toolbar': 'bg:#333333 #ffffff',
})
# 创建会话
session = PromptSession(
completer=UserCompleter(agents.keys()),
complete_while_typing=True,
style=style
)
client = MetaChain(log_path=logger)
while True:
# query = ask_text("Tell me what you want to do:")
query = session.prompt(
'Tell me what you want to do (type "exit" to quit): ',
bottom_toolbar=HTML('<b>Prompt:</b> Enter <b>@</b> to mention Agents')
)
if query.strip().lower() == 'exit':
# logger.info('User mode completed. See you next time! :waving_hand:', color='green', title='EXIT')
logo_text = "See you next time! :waving_hand:"
console.print(Panel(logo_text, style="bold salmon1", expand=True))
break
words = query.split()
console.print(f"[bold green]Your request: {query}[/bold green]", end=" ")
for word in words:
if word.startswith('@') and word[1:] in agents.keys():
# print(f"[bold magenta]{word}[bold magenta]", end=' ')
agent = agents[word.replace('@', '')]
else:
# print(word, end=' ')
pass
print()
if hasattr(agent, "name"):
agent_name = agent.name
console.print(f"[bold green][bold magenta]@{agent_name}[/bold magenta] will help you, be patient...[/bold green]")
messages.append({"role": "user", "content": query})
response = client.run(agent, messages, context_variables, debug=False)
messages.extend(response.messages)
model_answer_raw = response.messages[-1]['content']
# attempt to parse model_answer
if model_answer_raw.startswith('Case resolved'):
model_answer = re.findall(r'<solution>(.*?)</solution>', model_answer_raw, re.DOTALL)
if len(model_answer) == 0:
model_answer = model_answer_raw
else:
model_answer = model_answer[0]
else:
model_answer = model_answer_raw
console.print(f"[bold green][bold magenta]@{agent_name}[/bold magenta] has finished with the response:\n[/bold green] [bold blue]{model_answer}[/bold blue]")
agent = response.agent
elif agent == "select":
code_env: DockerEnv = context_variables["code_env"]
local_workplace = code_env.local_workplace
files_dir = os.path.join(local_workplace, "files")
os.makedirs(files_dir, exist_ok=True)
select_and_copy_files(files_dir, console)
else:
console.print(f"[bold red]Unknown agent: {agent}[/bold red]")
autoagent/cli_utils/file_select.py
+57
View File
@@ -0,0 +1,57 @@
import tkinter as tk
from tkinter import filedialog
import shutil
import os
from rich.console import Console
def select_and_copy_files(dest_dir, console: Console, docker_files_dir: str):
# 创建 tkinter 根窗口但隐藏它
root = tk.Tk()
root.withdraw()
# 打开文件选择对话框
files = filedialog.askopenfilenames(
title='Select files to copy',
filetypes=[
# ('Text files', '*.txt'),
('All files', '*.*'),
('PDF files', '*.pdf'),
('Docx files', '*.docx'),
('Txt files', '*.txt'),
('Zip files', '*.zip'),
('Text files', '*.txt'),
]
)
if not files:
print("No files selected")
return
# 选择目标文件夹
# dest_dir = filedialog.askdirectory(
# title='Select destination folder'
# )
if not dest_dir:
print("No destination folder selected")
return
# 复制文件
upload_infos = []
for file_path in files:
file_name = os.path.basename(file_path)
dest_path = os.path.join(dest_dir, file_name)
docker_dest_path = os.path.join(docker_files_dir, file_name)
try:
shutil.copy2(file_path, dest_path)
msg = f"Uploaded: {file_name} to {docker_dest_path}"
upload_infos.append(msg)
console.print(f"[bold green]{msg}[/bold green]")
except Exception as e:
console.print(f"[bold red]Error uploading {file_name}: {e}[/bold red]")
console.print(f"[bold green]Successfully uploaded {len(files)} files[/bold green]")
return upload_infos
if __name__ == "__main__":
dest_dir = "/Users/tangjiabin/Documents/reasoning/metachain/workspace_meta_showcase/showcase_nl2agent_showcase/workplace"
select_and_copy_files(dest_dir)
autoagent/cli_utils/metachain_meta_agent.py
+256
View File
@@ -0,0 +1,256 @@
from autoagent import MetaChain
from autoagent.util import UserCompleter
from prompt_toolkit import PromptSession
from prompt_toolkit.formatted_text import HTML
from prompt_toolkit.styles import Style
from autoagent.logger import LoggerManager, MetaChainLogger
from rich.console import Console
from rich.panel import Panel
from autoagent.agents.meta_agent.agent_former import get_agent_former_agent
from autoagent.agents.meta_agent.tool_editor import get_tool_editor_agent
from autoagent.agents.meta_agent.agent_creator import get_agent_creator_agent
import re
from autoagent.agents.meta_agent.form_complie import parse_agent_form
def extract_agents_content(text):
pattern = r'(<agents>.*?</agents>)'
# re.DOTALL 让 . 也能匹配换行符
match = re.search(pattern, text, re.DOTALL)
if match:
return match.group(1)
return None
def agent_profiling(agent_former, client, messages, context_variables, requirements, debug):
messages.append({"role": "user", "content": requirements+ """
Directly output the form in the XML format without ANY other text.
"""})
response = client.run(agent_former, messages, context_variables, debug=debug)
output_xml_form = response.messages[-1]["content"]
messages.extend(response.messages)
agent_form = None
MAX_RETRY = 3
for i in range(MAX_RETRY):
try:
output_xml_form = extract_agents_content(output_xml_form)
assert output_xml_form is not None, "No the XML form should be found in the output with the tag <agents>...</agents>."
agent_form = parse_agent_form(output_xml_form)
break
except Exception as e:
print(f"Error parsing XML to agent form: {e}. Retry {i+1}/{MAX_RETRY}")
messages.append({"role": "user", "content": f"Error parsing XML to agent form: {e}\nNote that there are some special restrictions for creating agent form, please try again."})
response = client.run(agent_former, messages, context_variables, debug=debug)
output_xml_form = response.messages[-1]["content"]
messages.extend(response.messages)
return agent_form, output_xml_form, messages
def tool_editing(tool_editor_agent, client, messages, context_variables, agent_form, output_xml_form, debug, suggestions = ""):
def case_resolved(task_response: str, context_variables: dict):
"""
Use this tools when ALL desired tools are created and tested successfully. You can NOT use this tool if tools are not created or tested successfully by running the tools.
Args:
task_response: the response of creating the tool which contains the completion status of the tool.
"""
return f"Case resolved. ALL desired tools are created and tested successfully. Details: {task_response}"
def case_not_resolved(task_response: str, context_variables: dict):
"""
Use this tools when you encounter irresistible errors after trying your best with multiple attempts for creating the desired tool. You can NOT use this tool before you have tried your best.
Args:
task_response: the reason why the tool is not created or tested successfully.
"""
return f"Case not resolved. Some desired tools are not created or tested successfully. Details: {task_response}"
tool_editor_agent.functions.extend([case_resolved, case_not_resolved])
MAX_RETRY = 3
if suggestions != "":
suggestions = "[IMPORTANT] Here are some suggestions for creating the tools: " + suggestions
agents = agent_form.agents
new_tools = []
for agent in agents:
if len(agent.tools.new) > 0:
for idx, tool in enumerate(agent.tools.new):
new_tools.append(f"{idx+1}. Tool name: {tool.name}, Tool description: {tool.description}")
if len(new_tools) == 0:
return "Case resolved. ALL desired tools are created and tested successfully.", messages
new_tools_str = "\n".join(new_tools)
messages.append({"role": "user", "content": f"""\
Your task is to create a list of new tools for me, the tools are:
{new_tools_str}
{suggestions}
Please create these new tools for me, note that you can NOT stop util you have created all the tools and tested them using `run_tool` successfully.
If ALL tools are created and tested successfully, you can stop and use `case_resolved` tool. Otherwise, you should continue to create the tools. After you have tried your best, you can use `case_not_resolved` tool to give the reason why the tool is not created or tested successfully.
[IMPORTANT] ALL tools MUST be tested successfully by running the tools using `run_tool` before you stop.
"""})
response = client.run(tool_editor_agent, messages, context_variables, debug=debug)
content = response.messages[-1]["content"]
for i in range(MAX_RETRY):
if content.startswith("Case resolved"):
return content, messages
messages.append({"role": "user", "content": f"""\
Your task is to create a list of new tools for me, the tools are:
{new_tools_str}
Please create these new tools for me, note that you can NOT stop util you have created all the tools and tested them using `run_tool` successfully.
The last attempt failed with the following error: {content}, please try again to create the tools.
"""})
response = client.run(tool_editor_agent, messages, context_variables, debug=debug)
content = response.messages[-1]["content"]
if i == MAX_RETRY:
return f"{content}\nSome desired tools are not created or tested successfully with {MAX_RETRY} attempts.", messages
def agent_editing(agent_creator_agent, client, messages, context_variables, agent_form, output_xml_form, requirements, task, debug, suggestions = ""):
MAX_RETRY = 3
if suggestions != "":
suggestions = "[IMPORTANT] Here are some suggestions for creating the agent(s): " + suggestions
def case_resolved(task_response: str, context_variables: dict):
"""
Use this tools when the desired agent(s) is created and tested successfully. You can NOT use this tool if the agent(s) is not created or tested successfully by running the agent(s).
"""
return f"Case resolved. The desired agent(s) is created and tested successfully. : {task_response}"
def case_not_resolved(task_response: str, context_variables: dict):
"""
Use this tools when you encounter irresistible errors after trying your best with multiple attempts for creating the desired agent(s). You can NOT use this tool before you have tried your best.
"""
return f"Case not resolved. The desired agent(s) is not created or tested successfully. Details: {task_response}"
agent_creator_agent.functions.extend([case_resolved, case_not_resolved])
messages.append({"role": "user", "content": f"""\
The user's request to create agent(s) is: {requirements}
Given the completed agent form with XML format: {output_xml_form}
After previous attempts, you have created new tools that required by the desired agent(s).
Your task is to create the desired agent(s) for me, note that you may create ONE single agent or multiple agents connected by orchestrator agent.
After you have created the agent(s), you should test the agent(s) by running the agent(s) using `run_agent` tool to complete the user's task:
{task}
Note that you can NOT stop util you have created the agent(s) and tested it successfully.
{suggestions}
"""})
response = client.run(agent_creator_agent, messages, context_variables, debug=debug)
content = response.messages[-1]["content"]
for i in range(MAX_RETRY):
if content.startswith("Case resolved"):
return content, messages
messages.append({"role": "user", "content": f"""\
The user's request to create agent(s) is: {requirements}
Given the completed agent form with XML format: {output_xml_form}
After previous attempts, you have created new tools that required by the desired agent(s).
Your task is to create the desired agent(s) for me, note that you may create ONE single agent or multiple agents connected by orchestrator agent.
After you have created the agent(s), you should test the agent(s) by running the agent(s) using `run_agent` tool to complete the user's task:
{task}
Note that you can NOT stop util you have created the agent(s) and tested it successfully.
The last attempt failed with the following error: {content}, please try again to create the desired agent(s).
{suggestions}
"""})
response = client.run(agent_creator_agent, messages, context_variables, debug=debug)
content = response.messages[-1]["content"]
if i == MAX_RETRY:
return f"{content}\nThe desired agent(s) is not created or tested successfully with {MAX_RETRY} attempts.", messages
def meta_agent(model: str, context_variables: dict, debug: bool = True):
logger = LoggerManager.get_logger()
# generate agent form
agent_former = get_agent_former_agent(model)
tool_editor_agent = get_tool_editor_agent(model)
agent_creator_agent = get_agent_creator_agent(model)
# enter agent
agent = agent_former
agents = {agent_former.name.replace(' ', '_'): agent_former, tool_editor_agent.name.replace(' ', '_'): tool_editor_agent, agent_creator_agent.name.replace(' ', '_'): agent_creator_agent}
style = Style.from_dict({
'bottom-toolbar': 'bg:#333333 #ffffff',
})
# 创建会话
session = PromptSession(
completer=UserCompleter(agents.keys()),
complete_while_typing=True,
style=style
)
client = MetaChain(log_path=logger)
console = Console()
messages = []
last_message = "Tell me what do you want to create with `Agent Chain`?"
while True:
query = session.prompt(
f'{last_message} (type "exit" to quit, press "Enter" to continue): ',
bottom_toolbar=HTML('<b>Prompt:</b> Enter <b>@</b> to mention Agents'),
)
if query.strip().lower() == 'exit':
logo_text = "Agent Chain completed. See you next time! :waving_hand:"
console.print(Panel(logo_text, style="bold salmon1", expand=True))
break
words = query.split()
console.print(f"[bold green]Your request: {query}[/bold green]", end=" ")
for word in words:
if word.startswith('@') and word[1:] in agents.keys():
# print(f"[bold magenta]{word}[bold magenta]", end=' ')
agent = agents[word.replace('@', '')]
else:
# print(word, end=' ')
pass
print()
agent_name = agent.name
console.print(f"[bold green][bold magenta]@{agent_name}[/bold magenta] will help you, be patient...[/bold green]")
match agent_name:
case 'Agent Former Agent':
if query == "":
console.print(f"[bold red]There MUST be a request to create the agent form.[/bold red]")
continue
requirements = query
agent_form, output_xml_form, messages = agent_profiling(agent_former, client, messages, context_variables, requirements, debug)
if agent_form is None:
console.print(f"[bold red][bold magenta]@{agent_name}[/bold magenta] has not created agent form successfully, please modify your requirements again.[/bold red]")
last_message = "Tell me what do you want to create with `Agent Chain`?"
continue
agent = tool_editor_agent
console.print(f"[bold green][bold magenta]@{agent_name}[/bold magenta] has created agent form successfully with the following details:\n[/bold green][bold blue]{output_xml_form}[/bold blue]")
last_message = "It is time to create the desired tools, do you have any suggestions for creating the tools?"
case 'Tool Editor Agent':
suggestions = query
tool_response, messages = tool_editing(tool_editor_agent, client, messages, context_variables, agent_form, output_xml_form, debug, suggestions)
if tool_response.startswith("Case not resolved"):
console.print(f"[bold red][bold magenta]@{agent_name}[/bold magenta] has not created tools successfully with the following error: {tool_response}[/bold red]")
agent = tool_editor_agent
last_message = "The tools are not created successfully, do you have any suggestions for creating the tools?"
continue
elif tool_response.startswith("Case resolved"):
agent = agent_creator_agent
console.print(f"[bold green][bold magenta]@{agent_name}[/bold magenta] has created tools successfully with the following details:\n[/bold green][bold blue]{tool_response}[/bold blue]")
last_message = "It is time to create the desired agent(s), do you have any suggestions for creating the agent(s)?"
else:
raise ValueError(f"Unknown tool response: {tool_response}")
case 'Agent Creator Agent':
suggestions = query
default_value='Come up with a task for the agent(s) to test your created agent(s), and use `run_agent` tool to test your created agent(s).' # 这里设置你想要的默认值
task = session.prompt(
'It is time to create the desired agent(s), what task do you want to complete with the agent(s)? (Press Enter if none): ',
)
task = default_value if not task.strip() else task
agent_response, messages = agent_editing(agent_creator_agent, client, messages, context_variables, agent_form, output_xml_form, requirements, task, debug, suggestions)
if agent_response.startswith("Case not resolved"):
console.print(f"[bold red][bold magenta]@{agent_name}[/bold magenta] has not created agent(s) successfully with the following error: {agent_response}[/bold red]")
agent = agent_creator_agent
last_message = "The agent(s) are not created successfully, do you have any suggestions for creating the agent(s)?"
continue
else:
console.print(f"[bold green][bold magenta]@{agent_name}[/bold magenta] has created agent(s) successfully with the following details:\n[/bold green][bold blue]{agent_response}[/bold blue]")
last_message = "Tell me what do you want to create with `Agent Chain`?"
autoagent/cli_utils/metachain_meta_workflow.py
+194
View File
@@ -0,0 +1,194 @@
from autoagent import MetaChain
from autoagent.util import ask_text, single_select_menu, print_markdown, debug_print, UserCompleter
from prompt_toolkit import PromptSession
from prompt_toolkit.formatted_text import HTML
from prompt_toolkit.styles import Style
from autoagent.logger import LoggerManager, MetaChainLogger
from rich.console import Console
from rich.panel import Panel
from autoagent.agents.meta_agent.workflow_former import get_workflow_former_agent
from autoagent.agents.meta_agent.workflow_creator import get_workflow_creator_agent
import re
from autoagent.agents.meta_agent.worklow_form_complie import parse_workflow_form, WorkflowForm
def workflow_profiling(workflow_former, client, messages, context_variables, requirements, debug):
messages.append({"role": "user", "content": requirements + """
Directly output the form in the XML format without ANY other text.
"""})
response = client.run(workflow_former, messages, context_variables, debug=debug)
output_xml_form = response.messages[-1]["content"]
messages.extend(response.messages)
MAX_RETRY = 3
for i in range(MAX_RETRY):
workflow_form = parse_workflow_form(output_xml_form)
if isinstance(workflow_form, WorkflowForm):
break
elif isinstance(workflow_form, str):
print(f"Error parsing XML to workflow form: {workflow_form}. Retry {i+1}/{MAX_RETRY}")
messages.append({"role": "user", "content": f"Error parsing XML to workflow form, the error message is: {workflow_form}\nNote that there are some special restrictions for creating workflow form, please try again."})
response = client.run(workflow_former, messages, context_variables, debug=debug)
output_xml_form = response.messages[-1]["content"]
messages.extend(response.messages)
else:
raise ValueError(f"Unexpected error: {workflow_form}")
return workflow_form, output_xml_form, messages
def workflow_editing(workflow_creator_agent, client, messages, context_variables, workflow_form, output_xml_form, requirements, task, debug, suggestions = ""):
MAX_RETRY = 3
if suggestions != "":
suggestions = "[IMPORTANT] Here are some suggestions for creating the workflow: " + suggestions
agents = workflow_form.agents
new_agents = []
for agent in agents:
if agent.category == "new":
new_agents.append(agent)
if len(new_agents) != 0:
new_agent_str = "AGENT CREATION INSTRUCTIONS:\nBefore you create the workflow, you need to create the following new agents in the workflow:\n"
for agent in new_agents:
new_agent_str += f"Agent name: {agent.name}\nAgent description: {agent.description}\n"
new_agent_str += f"Agent tools: {agent.tools}\n" if agent.tools else "Agent tools: []\n"
else:
new_agent_str = ""
def case_resolved(task_response: str, context_variables: dict):
"""
Use this tools when the desired workflow is created and tested successfully. You can NOT use this tool if the workflow is not created or tested successfully by running the workflow.
"""
return f"Case resolved. The desired workflow is created and tested successfully. : {task_response}"
def case_not_resolved(task_response: str, context_variables: dict):
"""
Use this tools when you encounter irresistible errors after trying your best with multiple attempts for creating the desired workflow. You can NOT use this tool before you have tried your best.
"""
return f"Case not resolved. The desired workflow is not created or tested successfully. Details: {task_response}"
workflow_creator_agent.functions.extend([case_resolved, case_not_resolved])
messages.append({"role": "user", "content": f"""\
WORKFLOW CREATION INSTRUCTIONS:
The user's request to create workflow is: {requirements}
Given the completed workflow form with XML format: {output_xml_form}
TASK:
Your task is to create the workflow for me, and then test the workflow by running the workflow using `run_workflow` tool to complete the user's task:
{task}
{new_agent_str}
TERMINATION INSTRUCTIONS:
After you have created the workflow and tested it successfully, you can use the `case_resolved` tool to indicate the case is resolved, otherwise you should try your best to create the workflow. And ONLY after you have tried multiple times, you can use the `case_not_resolved` tool to indicate the case is not resolved and give the reason.
Remember: you can NOT stop util you have created the workflow and tested it successfully.
{suggestions}
"""})
response = client.run(workflow_creator_agent, messages, context_variables, debug=debug)
content = response.messages[-1]["content"]
for i in range(MAX_RETRY):
if content.startswith("Case resolved"):
return content, messages
messages.append({"role": "user", "content": f"""\
WORKFLOW CREATION INSTRUCTIONS:
The user's request to create workflow is: {requirements}
Given the completed workflow form with XML format: {output_xml_form}
TASK:
Your task is to create the workflow for me, and then test the workflow by running the workflow using `run_workflow` tool to complete the user's task:
{task}
{new_agent_str}
TERMINATION INSTRUCTIONS:
After you have created the workflow and tested it successfully, you can use the `case_resolved` tool to indicate the case is resolved, otherwise you should try your best to create the workflow. And ONLY after you have tried multiple times, you can use the `case_not_resolved` tool to indicate the case is not resolved and give the reason.
Remember: you can NOT stop util you have created the workflow and tested it successfully.
FEEDBACK:
The last attempt failed with the following error: {content}, please try again to create the desired workflow.
{suggestions}
"""})
response = client.run(workflow_creator_agent, messages, context_variables, debug=debug)
content = response.messages[-1]["content"]
if i == MAX_RETRY:
return f"The desired workflow is not created or tested successfully with {MAX_RETRY} attempts.", messages
def meta_workflow(model: str, context_variables: dict, debug: bool = True):
print('\033[s\033[?25l', end='') # Save cursor position and hide cursor
logger = LoggerManager.get_logger()
workflow_former = get_workflow_former_agent(model)
workflow_creator_agent = get_workflow_creator_agent(model)
agent = workflow_former
agents = {workflow_former.name.replace(' ', '_'): workflow_former, workflow_creator_agent.name.replace(' ', '_'): workflow_creator_agent}
style = Style.from_dict({
'bottom-toolbar': 'bg:#333333 #ffffff',
})
# 创建会话
session = PromptSession(
completer=UserCompleter(agents.keys()),
complete_while_typing=True,
style=style
)
client = MetaChain(log_path=logger)
console = Console()
messages = []
last_message = "Tell me what do you want to create with `Workflow Chain`?"
while True:
query = session.prompt(
f'{last_message} (type "exit" to quit, press "Enter" to continue): ',
bottom_toolbar=HTML('<b>Prompt:</b> Enter <b>@</b> to mention Agents'),
)
if query.strip().lower() == 'exit':
logo_text = "Workflow Chain completed. See you next time! :waving_hand:"
console.print(Panel(logo_text, style="bold salmon1", expand=True))
break
words = query.split()
console.print(f"[bold green]Your request: {query}[/bold green]", end=" ")
for word in words:
if word.startswith('@') and word[1:] in agents.keys():
# print(f"[bold magenta]{word}[bold magenta]", end=' ')
agent = agents[word.replace('@', '')]
else:
# print(word, end=' ')
pass
print()
agent_name = agent.name
console.print(f"[bold green][bold magenta]@{agent_name}[/bold magenta] will help you, be patient...[/bold green]")
match agent_name:
case "Workflow Former Agent":
if query == "":
console.print(f"[bold red]There MUST be a request to create the agent form.[/bold red]")
continue
requirements = query
workflow_form, output_xml_form, messages = workflow_profiling(workflow_former, client, messages, context_variables, requirements, debug)
if workflow_form is None:
console.print(f"[bold red][bold magenta]@{agent_name}[/bold magenta] has not created workflow form successfully, please modify your requirements again.[/bold red]")
last_message = "Tell me what do you want to create with `Workflow Chain`?"
continue
agent = workflow_creator_agent
context_variables["workflow_form"] = workflow_form
console.print(f"[bold green][bold magenta]@{agent_name}[/bold magenta] has created workflow form successfully with the following details:\n[/bold green][bold blue]{output_xml_form}[/bold blue]")
last_message = "It is time to create the desired workflow, do you have any suggestions for creating the workflow?"
case "Workflow Creator Agent":
suggestions = query
default_value='Come up with a task for the workflow to test your created workflow, and use `run_workflow` tool to test your created workflow.' # 这里设置你想要的默认值
task = session.prompt(
'It is time to create the desired workflow, what task do you want to complete with the workflow? (Press Enter if none): ',
)
task = default_value if not task.strip() else task
agent_response, messages = workflow_editing(workflow_creator_agent, client, messages, context_variables, workflow_form, output_xml_form, requirements, task, debug, suggestions)
if agent_response.startswith("Case not resolved"):
console.print(f"[bold red][bold magenta]@{agent_name}[/bold magenta] has not created workflow successfully with the following error: {agent_response}[/bold red]")
agent = workflow_creator_agent
else:
console.print(f"[bold green][bold magenta]@{agent_name}[/bold magenta] has created workflow successfully with the following details:\n[/bold green][bold blue]{agent_response}[/bold blue]")
last_message = "Tell me what do you want to create with `Workflow Chain` next?"
autoagent/core.py
+673
View File
@@ -0,0 +1,673 @@
# Standard library imports
import copy
import json
from collections import defaultdict
from typing import List, Callable, Union
from datetime import datetime
# Local imports
import os
from .util import function_to_json, debug_print, merge_chunk, pretty_print_messages
from .types import (
Agent,
AgentFunction,
Message,
ChatCompletionMessageToolCall,
Function,
Response,
Result,
)
from litellm import completion, acompletion
from pathlib import Path
from .logger import MetaChainLogger, LoggerManager
from httpx import RemoteProtocolError, ConnectError
from litellm.exceptions import APIError
from tenacity import (
retry,
stop_after_attempt,
wait_exponential,
retry_if_exception_type
)
from openai import AsyncOpenAI
import litellm
import inspect
from constant import MC_MODE, FN_CALL, API_BASE_URL, NOT_SUPPORT_SENDER, ADD_USER, NON_FN_CALL
from autoagent.fn_call_converter import convert_tools_to_description, convert_non_fncall_messages_to_fncall_messages, SYSTEM_PROMPT_SUFFIX_TEMPLATE, convert_fn_messages_to_non_fn_messages, interleave_user_into_messages
from litellm.types.utils import Message as litellmMessage
# litellm.set_verbose=True
# client = AsyncOpenAI()
def should_retry_error(exception):
if MC_MODE is False: print(f"Caught exception: {type(exception).__name__} - {str(exception)}")
# 匹配更多错误类型
if isinstance(exception, (APIError, RemoteProtocolError, ConnectError)):
return True
# 通过错误消息匹配
error_msg = str(exception).lower()
return any([
"connection error" in error_msg,
"server disconnected" in error_msg,
"eof occurred" in error_msg,
"timeout" in error_msg,
"event loop is closed" in error_msg, # 添加事件循环错误
"anthropicexception" in error_msg, # 添加 Anthropic 相关错误
])
__CTX_VARS_NAME__ = "context_variables"
logger = LoggerManager.get_logger()
def adapt_tools_for_gemini(tools):
"""为 Gemini 模型适配工具定义，确保所有 OBJECT 类型参数都有非空的 properties"""
if tools is None:
return None
adapted_tools = []
for tool in tools:
adapted_tool = copy.deepcopy(tool)
# 检查参数
if "parameters" in adapted_tool["function"]:
params = adapted_tool["function"]["parameters"]
# 处理顶层参数
if params.get("type") == "object":
if "properties" not in params or not params["properties"]:
params["properties"] = {
"dummy": {
"type": "string",
"description": "Dummy property for Gemini compatibility"
}
}
# 处理嵌套参数
if "properties" in params:
for prop_name, prop in params["properties"].items():
if isinstance(prop, dict) and prop.get("type") == "object":
if "properties" not in prop or not prop["properties"]:
prop["properties"] = {
"dummy": {
"type": "string",
"description": "Dummy property for Gemini compatibility"
}
}
adapted_tools.append(adapted_tool)
return adapted_tools
class MetaChain:
def __init__(self, log_path: Union[str, None, MetaChainLogger] = None):
"""
log_path: path of log file, None
"""
if logger:
self.logger = logger
elif isinstance(log_path, MetaChainLogger):
self.logger = log_path
else:
self.logger = MetaChainLogger(log_path=log_path)
# if self.logger.log_path is None: self.logger.info("[Warning] Not specific log path, so log will not be saved", "...", title="Log Path", color="light_cyan3")
# else: self.logger.info("Log file is saved to", self.logger.log_path, "...", title="Log Path", color="light_cyan3")
# @retry(
# stop=stop_after_attempt(4),
# wait=wait_exponential(multiplier=1, min=4, max=60),
# retry=should_retry_error,
# before_sleep=lambda retry_state: print(f"Retrying... (attempt {retry_state.attempt_number})")
# )
def get_chat_completion(
self,
agent: Agent,
history: List,
context_variables: dict,
model_override: str,
stream: bool,
debug: bool,
) -> Message:
context_variables = defaultdict(str, context_variables)
instructions = (
agent.instructions(context_variables)
if callable(agent.instructions)
else agent.instructions
)
if agent.examples:
examples = agent.examples(context_variables) if callable(agent.examples) else agent.examples
history = examples + history
messages = [{"role": "system", "content": instructions}] + history
# debug_print(debug, "Getting chat completion for...:", messages)
tools = [function_to_json(f) for f in agent.functions]
# hide context_variables from model
for tool in tools:
params = tool["function"]["parameters"]
params["properties"].pop(__CTX_VARS_NAME__, None)
if __CTX_VARS_NAME__ in params["required"]:
params["required"].remove(__CTX_VARS_NAME__)
create_model = model_override or agent.model
if "gemini" in create_model.lower():
tools = adapt_tools_for_gemini(tools)
if FN_CALL:
# create_model = model_override or agent.model
assert litellm.supports_function_calling(model = create_model) == True, f"Model {create_model} does not support function calling, please set `FN_CALL=False` to use non-function calling mode"
create_params = {
"model": create_model,
"messages": messages,
"tools": tools or None,
"tool_choice": agent.tool_choice,
"stream": stream,
}
NO_SENDER_MODE = False
for not_sender_model in NOT_SUPPORT_SENDER:
if not_sender_model in create_model:
NO_SENDER_MODE = True
break
if NO_SENDER_MODE:
messages = create_params["messages"]
for message in messages:
if 'sender' in message:
del message['sender']
create_params["messages"] = messages
if tools and create_params['model'].startswith("gpt"):
create_params["parallel_tool_calls"] = agent.parallel_tool_calls
completion_response = completion(**create_params)
else:
# create_model = model_override or agent.model
assert agent.tool_choice == "required", f"Non-function calling mode MUST use tool_choice = 'required' rather than {agent.tool_choice}"
last_content = messages[-1]["content"]
tools_description = convert_tools_to_description(tools)
messages[-1]["content"] = last_content + "\n[IMPORTANT] You MUST use the tools provided to complete the task.\n" + SYSTEM_PROMPT_SUFFIX_TEMPLATE.format(description=tools_description)
NO_SENDER_MODE = False
for not_sender_model in NOT_SUPPORT_SENDER:
if not_sender_model in create_model:
NO_SENDER_MODE = True
break
if NO_SENDER_MODE:
for message in messages:
if 'sender' in message:
del message['sender']
if NON_FN_CALL:
messages = convert_fn_messages_to_non_fn_messages(messages)
if ADD_USER and messages[-1]["role"] != "user":
# messages.append({"role": "user", "content": "Please think twice and take the next action according to your previous actions and observations."})
messages = interleave_user_into_messages(messages)
create_params = {
"model": create_model,
"messages": messages,
"stream": stream,
"base_url": API_BASE_URL,
}
completion_response = completion(**create_params)
last_message = [{"role": "assistant", "content": completion_response.choices[0].message.content}]
converted_message = convert_non_fncall_messages_to_fncall_messages(last_message, tools)
if "tool_calls" in converted_message[0]:
converted_tool_calls = [ChatCompletionMessageToolCall(**tool_call) for tool_call in converted_message[0]["tool_calls"]]
else:
converted_tool_calls = None
completion_response.choices[0].message = litellmMessage(content = converted_message[0]["content"], role = "assistant", tool_calls = converted_tool_calls)
return completion_response
def handle_function_result(self, result, debug) -> Result:
match result:
case Result() as result:
return result
case Agent() as agent:
return Result(
value=json.dumps({"assistant": agent.name}),
agent=agent,
)
case _:
try:
return Result(value=str(result))
except Exception as e:
error_message = f"Failed to cast response to string: {result}. Make sure agent functions return a string or Result object. Error: {str(e)}"
self.logger.info(error_message, title="Handle Function Result Error", color="red")
raise TypeError(error_message)
def handle_tool_calls(
self,
tool_calls: List[ChatCompletionMessageToolCall],
functions: List[AgentFunction],
context_variables: dict,
debug: bool,
handle_mm_func: Callable[[], str] = None,
) -> Response:
function_map = {f.__name__: f for f in functions}
partial_response = Response(
messages=[], agent=None, context_variables={})
for tool_call in tool_calls:
name = tool_call.function.name
# handle missing tool case, skip to next tool
if name not in function_map:
self.logger.info(f"Tool {name} not found in function map. You are recommended to use `run_tool` to run this tool.", title="Tool Call Error", color="red")
partial_response.messages.append(
{
"role": "tool",
"tool_call_id": tool_call.id,
"name": name,
"content": f"Error: Tool {name} not found. You are recommended to use `run_tool` to run this tool.",
}
)
continue
args = json.loads(tool_call.function.arguments)
# debug_print(
# debug, f"Processing tool call: {name} with arguments {args}")
func = function_map[name]
# pass context_variables to agent functions
# if __CTX_VARS_NAME__ in func.__code__.co_varnames:
# args[__CTX_VARS_NAME__] = context_variables
if __CTX_VARS_NAME__ in inspect.signature(func).parameters.keys():
args[__CTX_VARS_NAME__] = context_variables
raw_result = function_map[name](**args)
result: Result = self.handle_function_result(raw_result, debug)
partial_response.messages.append(
{
"role": "tool",
"tool_call_id": tool_call.id,
"name": name,
"content": result.value,
}
)
self.logger.pretty_print_messages(partial_response.messages[-1])
if result.image:
assert handle_mm_func, f"handle_mm_func is not provided, but an image is returned by tool call {name}({tool_call.function.arguments})"
partial_response.messages.append(
{
"role": "user",
"content": [
# {"type":"text", "text":f"After take last action `{name}({tool_call.function.arguments})`, the image of current page is shown below. Please take next action based on the image, the current state of the page as well as previous actions and observations."},
{"type":"text", "text":handle_mm_func(name, tool_call.function.arguments)},
{
"type":"image_url",
"image_url":{
"url":f"data:image/png;base64,{result.image}"
}
}
]
}
)
# debug_print(debug, "Tool calling: ", json.dumps(partial_response.messages[-1], indent=4), log_path=log_path, title="Tool Calling", color="green")
partial_response.context_variables.update(result.context_variables)
if result.agent:
partial_response.agent = result.agent
return partial_response
def run_and_stream(
self,
agent: Agent,
messages: List,
context_variables: dict = {},
model_override: str = None,
debug: bool = False,
max_turns: int = float("inf"),
execute_tools: bool = True,
):
active_agent = agent
context_variables = copy.deepcopy(context_variables)
history = copy.deepcopy(messages)
init_len = len(messages)
while len(history) - init_len < max_turns:
message = {
"content": "",
"sender": agent.name,
"role": "assistant",
"function_call": None,
"tool_calls": defaultdict(
lambda: {
"function": {"arguments": "", "name": ""},
"id": "",
"type": "",
}
),
}
# get completion with current history, agent
completion = self.get_chat_completion(
agent=active_agent,
history=history,
context_variables=context_variables,
model_override=model_override,
stream=True,
debug=debug,
)
yield {"delim": "start"}
for chunk in completion:
delta = json.loads(chunk.choices[0].delta.json())
if delta["role"] == "assistant":
delta["sender"] = active_agent.name
yield delta
delta.pop("role", None)
delta.pop("sender", None)
merge_chunk(message, delta)
yield {"delim": "end"}
message["tool_calls"] = list(
message.get("tool_calls", {}).values())
if not message["tool_calls"]:
message["tool_calls"] = None
debug_print(debug, "Received completion:", message)
history.append(message)
if not message["tool_calls"] or not execute_tools:
debug_print(debug, "Ending turn.")
break
# convert tool_calls to objects
tool_calls = []
for tool_call in message["tool_calls"]:
function = Function(
arguments=tool_call["function"]["arguments"],
name=tool_call["function"]["name"],
)
tool_call_object = ChatCompletionMessageToolCall(
id=tool_call["id"], function=function, type=tool_call["type"]
)
tool_calls.append(tool_call_object)
# handle function calls, updating context_variables, and switching agents
partial_response = self.handle_tool_calls(
tool_calls, active_agent.functions, context_variables, debug
)
history.extend(partial_response.messages)
context_variables.update(partial_response.context_variables)
if partial_response.agent:
active_agent = partial_response.agent
yield {
"response": Response(
messages=history[init_len:],
agent=active_agent,
context_variables=context_variables,
)
}
def run(
self,
agent: Agent,
messages: List,
context_variables: dict = {},
model_override: str = None,
stream: bool = False,
debug: bool = True,
max_turns: int = float("inf"),
execute_tools: bool = True,
) -> Response:
if stream:
return self.run_and_stream(
agent=agent,
messages=messages,
context_variables=context_variables,
model_override=model_override,
debug=debug,
max_turns=max_turns,
execute_tools=execute_tools,
)
active_agent = agent
enter_agent = agent
context_variables = copy.copy(context_variables)
history = copy.deepcopy(messages)
init_len = len(messages)
self.logger.info("Receiveing the task:", history[-1]['content'], title="Receive Task", color="green")
while len(history) - init_len < max_turns and active_agent:
# get completion with current history, agent
completion = self.get_chat_completion(
agent=active_agent,
history=history,
context_variables=context_variables,
model_override=model_override,
stream=stream,
debug=debug,
)
message: Message = completion.choices[0].message
message.sender = active_agent.name
# debug_print(debug, "Received completion:", message.model_dump_json(indent=4), log_path=log_path, title="Received Completion", color="blue")
self.logger.pretty_print_messages(message)
history.append(
json.loads(message.model_dump_json())
) # to avoid OpenAI types (?)
# if not message.tool_calls or not execute_tools:
# self.logger.info("Ending turn.", title="End Turn", color="red")
# break
if enter_agent.tool_choice != "required":
if (not message.tool_calls and active_agent.name == enter_agent.name) or not execute_tools:
self.logger.info("Ending turn.", title="End Turn", color="red")
break
else:
if (message.tool_calls and message.tool_calls[0].function.name == "case_resolved") or not execute_tools:
self.logger.info("Ending turn with case resolved.", title="End Turn", color="red")
partial_response = self.handle_tool_calls(
message.tool_calls, active_agent.functions, context_variables, debug, handle_mm_func=active_agent.handle_mm_func
)
history.extend(partial_response.messages)
context_variables.update(partial_response.context_variables)
break
elif (message.tool_calls and message.tool_calls[0].function.name == "case_not_resolved") or not execute_tools:
self.logger.info("Ending turn with case not resolved.", title="End Turn", color="red")
partial_response = self.handle_tool_calls(
message.tool_calls, active_agent.functions, context_variables, debug, handle_mm_func=active_agent.handle_mm_func
)
history.extend(partial_response.messages)
context_variables.update(partial_response.context_variables)
break
elif (not message.tool_calls) or not execute_tools:
self.logger.info("Ending turn with no tool calls.", title="End Turn", color="red")
break
# if (message.tool_calls and message.tool_calls[0].function.name == "case_resolved") or not execute_tools:
# debug_print(debug, "Ending turn.", log_path=log_path, title="End Turn", color="red")
# break
# handle function calls, updating context_variables, and switching agents
if message.tool_calls:
partial_response = self.handle_tool_calls(
message.tool_calls, active_agent.functions, context_variables, debug, handle_mm_func=active_agent.handle_mm_func
)
else:
partial_response = Response(messages=[message])
history.extend(partial_response.messages)
context_variables.update(partial_response.context_variables)
if partial_response.agent:
active_agent = partial_response.agent
return Response(
messages=history[init_len:],
agent=active_agent,
context_variables=context_variables,
)
@retry(
stop=stop_after_attempt(4),
wait=wait_exponential(multiplier=1, min=10, max=180),
retry=should_retry_error,
before_sleep=lambda retry_state: print(f"Retrying... (attempt {retry_state.attempt_number})")
)
async def get_chat_completion_async(
self,
agent: Agent,
history: List,
context_variables: dict,
model_override: str,
stream: bool,
debug: bool,
) -> Message:
context_variables = defaultdict(str, context_variables)
instructions = (
agent.instructions(context_variables)
if callable(agent.instructions)
else agent.instructions
)
if agent.examples:
examples = agent.examples(context_variables) if callable(agent.examples) else agent.examples
history = examples + history
messages = [{"role": "system", "content": instructions}] + history
# debug_print(debug, "Getting chat completion for...:", messages)
tools = [function_to_json(f) for f in agent.functions]
# hide context_variables from model
for tool in tools:
params = tool["function"]["parameters"]
params["properties"].pop(__CTX_VARS_NAME__, None)
if __CTX_VARS_NAME__ in params["required"]:
params["required"].remove(__CTX_VARS_NAME__)
if FN_CALL:
create_model = model_override or agent.model
assert litellm.supports_function_calling(model = create_model) == True, f"Model {create_model} does not support function calling, please set `FN_CALL=False` to use non-function calling mode"
create_params = {
"model": create_model,
"messages": messages,
"tools": tools or None,
"tool_choice": agent.tool_choice,
"stream": stream,
}
NO_SENDER_MODE = False
for not_sender_model in NOT_SUPPORT_SENDER:
if not_sender_model in create_model:
NO_SENDER_MODE = True
break
if NO_SENDER_MODE:
messages = create_params["messages"]
for message in messages:
if 'sender' in message:
del message['sender']
create_params["messages"] = messages
if tools and create_params['model'].startswith("gpt"):
create_params["parallel_tool_calls"] = agent.parallel_tool_calls
completion_response = await acompletion(**create_params)
else:
create_model = model_override or agent.model
assert agent.tool_choice == "required", f"Non-function calling mode MUST use tool_choice = 'required' rather than {agent.tool_choice}"
last_content = messages[-1]["content"]
tools_description = convert_tools_to_description(tools)
messages[-1]["content"] = last_content + "\n[IMPORTANT] You MUST use the tools provided to complete the task.\n" + SYSTEM_PROMPT_SUFFIX_TEMPLATE.format(description=tools_description)
NO_SENDER_MODE = False
for not_sender_model in NOT_SUPPORT_SENDER:
if not_sender_model in create_model:
NO_SENDER_MODE = True
break
if NO_SENDER_MODE:
for message in messages:
if 'sender' in message:
del message['sender']
create_params = {
"model": create_model,
"messages": messages,
"stream": stream,
"base_url": API_BASE_URL,
}
completion_response = await acompletion(**create_params)
last_message = [{"role": "assistant", "content": completion_response.choices[0].message.content}]
converted_message = convert_non_fncall_messages_to_fncall_messages(last_message, tools)
converted_tool_calls = [ChatCompletionMessageToolCall(**tool_call) for tool_call in converted_message[0]["tool_calls"]]
completion_response.choices[0].message = litellmMessage(content = converted_message[0]["content"], role = "assistant", tool_calls = converted_tool_calls)
# response = await acompletion(**create_params)
# response = await client.chat.completions.create(**create_params)
return completion_response
async def run_async(
self,
agent: Agent,
messages: List,
context_variables: dict = {},
model_override: str = None,
stream: bool = False,
debug: bool = True,
max_turns: int = float("inf"),
execute_tools: bool = True,
) -> Response:
assert stream == False, "Async run does not support stream"
active_agent = agent
enter_agent = agent
context_variables = copy.copy(context_variables)
history = copy.deepcopy(messages)
init_len = len(messages)
self.logger.info("Receiveing the task:", history[-1]['content'], title="Receive Task", color="green")
while len(history) - init_len < max_turns and active_agent:
# get completion with current history, agent
completion = await self.get_chat_completion_async(
agent=active_agent,
history=history,
context_variables=context_variables,
model_override=model_override,
stream=stream,
debug=debug,
)
message: Message = completion.choices[0].message
message.sender = active_agent.name
# debug_print(debug, "Received completion:", message.model_dump_json(indent=4), log_path=log_path, title="Received Completion", color="blue")
self.logger.pretty_print_messages(message)
history.append(
json.loads(message.model_dump_json())
) # to avoid OpenAI types (?)
if enter_agent.tool_choice != "required":
if (not message.tool_calls and active_agent.name == enter_agent.name) or not execute_tools:
self.logger.info("Ending turn.", title="End Turn", color="red")
break
else:
if (message.tool_calls and message.tool_calls[0].function.name == "case_resolved") or not execute_tools:
self.logger.info("Ending turn with case resolved.", title="End Turn", color="red")
partial_response = self.handle_tool_calls(
message.tool_calls, active_agent.functions, context_variables, debug, handle_mm_func=active_agent.handle_mm_func
)
history.extend(partial_response.messages)
context_variables.update(partial_response.context_variables)
break
elif (message.tool_calls and message.tool_calls[0].function.name == "case_not_resolved") or not execute_tools:
self.logger.info("Ending turn with case not resolved.", title="End Turn", color="red")
partial_response = self.handle_tool_calls(
message.tool_calls, active_agent.functions, context_variables, debug, handle_mm_func=active_agent.handle_mm_func
)
history.extend(partial_response.messages)
context_variables.update(partial_response.context_variables)
break
elif (not message.tool_calls) or not execute_tools:
self.logger.info("Ending turn with no tool calls.", title="End Turn", color="red")
break
# if (message.tool_calls and message.tool_calls[0].function.name == "case_resolved") or not execute_tools:
# debug_print(debug, "Ending turn.", log_path=log_path, title="End Turn", color="red")
# break
# handle function calls, updating context_variables, and switching agents
if message.tool_calls:
partial_response = self.handle_tool_calls(
message.tool_calls, active_agent.functions, context_variables, debug, handle_mm_func=active_agent.handle_mm_func
)
else:
partial_response = Response(messages=[message])
history.extend(partial_response.messages)
context_variables.update(partial_response.context_variables)
if partial_response.agent:
active_agent = partial_response.agent
return Response(
messages=history[init_len:],
agent=active_agent,
context_variables=context_variables,
)
autoagent/environment/__init__.py
+5
View File
@@ -0,0 +1,5 @@
from .docker_env import DockerEnv, DockerConfig
from .local_env import LocalEnv
from .browser_env import BrowserEnv, VIEWPORT
from .markdown_browser import RequestsMarkdownBrowser
from .utils import setup_metachain
autoagent/environment/browser_cookies.py
+42
View File
@@ -0,0 +1,42 @@
import json
from pathlib import Path
import glob
wd = Path(__file__).parent.resolve()
def load_cookies_from_json(json_path):
with open(json_path, 'r') as f:
cookies = json.load(f)
return cookies
def convert_cookies_to_python():
all_cookies = []
# cookie_files = [
# "orcid.org.cookies.json",
# "www.researchgate.net.cookies.json",
# "github.com.cookies.json",
# "www.youtube.com.cookies.json",
# "www.ncbi.nlm.nih.gov.cookies.json",
# "archive.org.cookies.json",
# "nature.com.cookies.json"
# ]
json_dir = wd / "cookie_json"
cookie_files = glob.glob(str(json_dir / "*.json"))
for cookie_file in cookie_files:
json_path = wd / "cookie_json" / cookie_file
cookies = load_cookies_from_json(json_path)
all_cookies.extend(cookies)
# 生成Python格式的cookies文件
output_path = wd / "cookies_data.py"
output_str = "COOKIES_LIST = [\n"
for cookie in all_cookies:
output_str += f" {repr(cookie)},\n"
output_str += "]\n"
with open(output_path, "w", encoding="utf-8") as f:
f.write(output_str)
return output_str
if __name__ == "__main__":
print(convert_cookies_to_python())
autoagent/environment/browser_env.py
+649
View File
@@ -0,0 +1,649 @@
import atexit
import base64
import io
import json
import multiprocessing
import time
import uuid
import browsergym.core # noqa F401 (we register the openended task as a gym environment)
import gymnasium as gym
import html2text
import numpy as np
import tenacity
from browsergym.utils.obs import flatten_dom_to_str
from PIL import Image
from autoagent.util import debug_print
from autoagent.logger import LoggerManager
import inspect
import textwrap
from .shutdown_listener import should_continue, should_exit
from .tenacity_stop import stop_if_should_exit
from datetime import datetime
from pathlib import Path
from browsergym.core.action.functions import goto, page, get_elem_by_bid, demo_mode, tab_focus
import os
from typing import Dict, Union, cast, Literal
from playwright.sync_api import Page, Download
from autoagent.io_utils import read_file
from autoagent.environment.mdconvert import _get_page_markdown
from autoagent.environment.browser_cookies import convert_cookies_to_python
from autoagent.environment.cookies_data import COOKIES_LIST
# from constant import DOCKER_WORKPLACE_NAME, LOCAL_ROOT
from functools import update_wrapper
from inspect import signature
import types
import sys
import tempfile
VIEWPORT = {"width": 1280, "height": 720}
BROWSER_EVAL_GET_GOAL_ACTION = 'GET_EVAL_GOAL'
BROWSER_EVAL_GET_REWARDS_ACTION = 'GET_EVAL_REWARDS'
class BrowserInitException(Exception):
def __init__(self, message='Failed to initialize browser environment'):
super().__init__(message)
def _local_to_docker(local_path: str):
"""
Convert a local path to a docker path
local_path: the local path to convert, like `{local_workplace}/downloads/xxx`
docker_path: the docker path to convert, like `{docker_workplace}/downloads/xxx`
Examples:
_local_to_docker('/Users/tangjiabin/Documents/reasoning/autoagent/workplace_gaia_eval/downloads/xxx')
"""
local_workplace = None
docker_workplace = None
assert local_workplace in local_path, f"local_path must contain {local_workplace}"
return local_path.replace(local_workplace, docker_workplace)
def _visit_page(url: str):
"""
Visit a page, including downloading files based on the url
Examples:
_visit_page('https://archive.org/download/higpt_stage2/instruct_ds_dblp.tar.gz')
"""
# def _local_to_docker(local_path: str):
# """
# Convert a local path to a docker path
# local_path: the local path to convert, like `{LOCAL_ROOT}/{DOCKER_WORKPLACE_NAME}/downloads/xxx`
# docker_path: the docker path to convert, like `/{DOCKER_WORKPLACE_NAME}/downloads/xxx`
# """
# assert LOCAL_ROOT in local_path, f"local_path must contain {LOCAL_ROOT}"
# return local_path.replace(LOCAL_ROOT, '')
try:
# 尝试作为普通网页访问
page.context.add_cookies(COOKIES_LIST)
# goto(url)
page.set_extra_http_headers({
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8",
"Accept-Language": "en-US,en;q=0.9"
})
page.goto(url, timeout=6000)
if page.get_by_text("Verify you are human by completing the action below.").count() > 0:
_checkMeetChallenge()
# 等待页面完全加载
# 增加等待时间，确保页面完全加载
page.wait_for_load_state("networkidle", timeout=3000)
# page.wait_for_timeout(3000)
except Exception as e_outer:
# 处理文件下载情况
if "net::ERR_ABORTED" in str(e_outer):
import os
import requests
import base64
downloads_folder = f"{local_workplace}/downloads"
os.makedirs(downloads_folder, exist_ok=True)
filename = os.path.basename(url)
filepath = os.path.join(downloads_folder, filename)
filepath = os.path.abspath(filepath)
try:
# 使用requests下载文件
response = requests.get(url, stream=True)
response.raise_for_status()
with open(filepath, 'wb') as f:
for chunk in response.iter_content(chunk_size=8192):
if chunk:
f.write(chunk)
# 显示下载成功页面
message = f"""<body style="margin: 20px;">
<h1>Successfully downloaded '{filename}' to local path:
<br><br>{_local_to_docker(filepath)}</h1></body>"""
goto(
"data:text/html;base64," +
base64.b64encode(message.encode("utf-8")).decode("utf-8")
)
# 触发pageshow事件
page.evaluate("""
const event = new Event('pageshow', {
bubbles: true,
cancelable: false
});
window.dispatchEvent(event);
""")
except Exception as e:
raise Exception(f"Download error: {str(e)}")
else:
raise e_outer
# def _click_id(bid: str, button: Literal["left", "middle", "right"] = "left"):
# """
# Clicks the mouse on the target with the given element bid.
# Examples:
# _click_id('12')
# _click_id('12', button='left')
# """
# from typing import Dict, Union, cast
# try:
# elem = get_elem_by_bid(page, bid, demo_mode != "off")
# box = cast(Dict[str, Union[int, float]], elem.bounding_box())
# # 如果既不是下载也不是新页面，在当前页面处理
# page.mouse.click(box["x"] + box["width"] / 2, box["y"] + box["height"] / 2, button=button)
# try:
# page.wait_for_load_state("networkidle", timeout=5000)
# except:
# pass
# return
# except Exception as e:
# raise Exception(f"Click error: {str(e)}")
def _click_id(bid: str, button: Literal["left", "middle", "right"] = "left"):
"""
Clicks the mouse on the target with the given element bid.
Examples:
_click_id('12')
_click_id('12', button='left')
"""
# def _local_to_docker(local_path: str):
# """
# Convert a local path to a docker path
# local_path: the local path to convert, like `{LOCAL_ROOT}/{DOCKER_WORKPLACE_NAME}/downloads/xxx`
# docker_path: the docker path to convert, like `/{DOCKER_WORKPLACE_NAME}/downloads/xxx`
# """
# assert LOCAL_ROOT in local_path, f"local_path must contain {LOCAL_ROOT}"
# return local_path.replace(LOCAL_ROOT, '')
from typing import Dict, Union, cast
import time
import base64
import os
from playwright._impl._api_types import TimeoutError as playwright_TimeoutError
try:
global page
elem = get_elem_by_bid(page, bid, demo_mode != "off")
box = cast(Dict[str, Union[int, float]], elem.bounding_box())
# 获取当前页面URL
current_url = page.url
page.context.add_cookies(COOKIES_LIST)
# goto(url)
page.set_extra_http_headers({
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8",
"Accept-Language": "en-US,en;q=0.9"
})
# 执行点击并等待下载
try:
with page.expect_download(timeout=5000) as download_info: # 增加到30秒
page.mouse.click(box["x"] + box["width"] / 2, box["y"] + box["height"] / 2, button=button)
download = download_info.value
print(f"Downloading file: {download.suggested_filename}")
# 确保下载目录存在
download_path = f"{local_workplace}/downloads"
os.makedirs(download_path, exist_ok=True)
# 保存文件
filepath = os.path.join(download_path, download.suggested_filename)
filepath = os.path.abspath(filepath)
download.save_as(filepath)
# 显示下载成功页面
message = f"""<body style="margin: 20px;">
<h1>Successfully downloaded '{download.suggested_filename}' to local path:
<br><br>{_local_to_docker(filepath)}</h1></body>"""
goto(
"data:text/html;base64," +
base64.b64encode(message.encode("utf-8")).decode("utf-8")
)
# 触发pageshow事件
page.evaluate("""
const event = new Event('pageshow', {
bubbles: true,
cancelable: false
});
window.dispatchEvent(event);
""")
return
except playwright_TimeoutError:
# print("Download timeout, trying alternative approach...")
# # 如果超时，尝试获取PDF直接URL并下载
# if "arxiv.org" in current_url:
# paper_id = current_url.split("/")[-1]
# pdf_url = f"https://arxiv.org/pdf/{paper_id}.pdf"
# _visit_page(pdf_url)
# return
pass
# 等待可能的新标签页或导航
time.sleep(1)
# 检查是否有新标签页
pages_after = len(page.context.pages)
if pages_after > 1:
# 切换到最新的标签页
page = page.context.pages[-1]
page.bring_to_front()
elif page.url != current_url:
# URL改变了，说明发生了导航
try:
page.wait_for_load_state("networkidle", timeout=5000)
if page.get_by_text("Verify you are human by completing the action below.").count() > 0:
_checkMeetChallenge()
# 等待页面完全加载
# 增加等待时间，确保页面完全加载
page.wait_for_load_state("networkidle", timeout=3000)
except:
pass
return
except Exception as e:
raise Exception(f"Click error: {str(e)}, {type(e)}")
def _checkMeetChallenge():
"""
check if meet challenge
Examples:
_checkMeetChallenge()
"""
global page
def tryToClickChallenge(this_page):
try:
# 尝试定位并点击验证框架中的复选框
frame = this_page.frame_locator("iframe[title*='challenge']")
if frame:
checkbox = frame.locator("input[type='checkbox']")
if checkbox.is_visible():
checkbox.click()
return True
# 尝试点击验证按钮 (同时支持中英文)
verify_texts = ["请完成以下操作，验证您是真人。", "Verify you are human by completing the action below."]
for text in verify_texts:
verify_button = this_page.get_by_text(text)
if verify_button.is_visible():
verify_button.click()
return True
# 尝试点击任何可见的验证按钮
challenge_buttons = this_page.locator("button[class*='challenge']")
if challenge_buttons.count() > 0:
challenge_buttons.first.click()
return True
except Exception as e:
print(f"尝试点击验证失败: {str(e)}")
return False
check_count = 1
max_attempts = 6
while check_count <= max_attempts:
# 检查是否存在验证页面的特征 (同时支持中英文)
if (page.get_by_text("请完成以下操作，验证您是真人。").count() == 0 and
page.get_by_text("Verify you are human by completing the action below.").count() == 0):
print("验证已完成")
break
print(f"检测到 Cloudflare 验证页面，尝试处理... (第 {check_count}/{max_attempts} 次)")
# 尝试处理验证
if tryToClickChallenge(page):
print("已尝试点击验证按钮，等待响应...")
# 等待验证结果
try:
# 等待验证页面消失或出现新内容
page.wait_for_function("""
() => !document.querySelector("div#challenge-stage") ||
(!document.body.textContent.includes("请完成以下操作，验证您是真人。") &&
!document.body.textContent.includes("Verify you are human by completing the action below."))
""", timeout=20000)
except:
print("等待验证超时")
# 检查是否仍在验证页面
if check_count >= max_attempts:
if (page.get_by_text("请完成以下操作，验证您是真人。").count() > 0 or
page.get_by_text("Verify you are human by completing the action below.").count() > 0):
raise Exception("cannot pass challenge, need to restart")
check_count += 1
page.wait_for_timeout(5000) # 短暂等待后再次检查
class BrowserEnv:
def __init__(self, browsergym_eval_env: str | None = None, local_root: str | None = None, workplace_name: str | None = None):
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
log_dir = Path(f"logs/res_{timestamp}")
log_dir.mkdir(parents=True, exist_ok=True) # recursively create all necessary parent directories
log_path = str(log_dir / "browser_env.log")
self.log_path = log_path
# self.logger = LoggerManager.get_logger()
self.html_text_converter = self.get_html_text_converter()
self.eval_mode = False
self.eval_dir = ''
self.local_workplace = os.path.join(local_root, workplace_name)
self.docker_workplace = f"/{workplace_name}"
# EVAL only: browsergym_eval_env must be provided for evaluation
self.browsergym_eval_env = browsergym_eval_env
self.eval_mode = bool(browsergym_eval_env)
# Initialize browser environment process
multiprocessing.set_start_method('spawn', force=True)
self.browser_side, self.agent_side = multiprocessing.Pipe()
# tmp_env = gym.make(self.browsergym_eval_env,tags_to_mark='all') if self.eval_mode else gym.make('browsergym/openended',task_kwargs={'start_url': 'about:blank', 'goal': 'PLACEHOLDER_GOAL'},
# wait_for_user_message=False,
# headless=True,
# disable_env_checker=True,
# tags_to_mark='all'
# )
# obs, info = tmp_env.reset()
# self.viewport = tmp_env.env.viewport if tmp_env.env.viewport else tmp_env.env.task.viewport
# tmp_env.close()
self.init_browser()
atexit.register(self.close)
def get_html_text_converter(self):
html_text_converter = html2text.HTML2Text()
# ignore links and images
html_text_converter.ignore_links = False
html_text_converter.ignore_images = True
# use alt text for images
html_text_converter.images_to_alt = True
# disable auto text wrapping
html_text_converter.body_width = 0
return html_text_converter
@tenacity.retry(
wait=tenacity.wait_fixed(1),
stop=tenacity.stop_after_attempt(5) | stop_if_should_exit(),
retry=tenacity.retry_if_exception_type(BrowserInitException),
)
def init_browser(self):
debug_print(True, "Starting browser env...", title = "Browser Env", log_path=self.log_path)
# self.logger.info("Starting browser env...", title="Browser Env", color="green")
try:
self.process = multiprocessing.Process(target=self.browser_process)
self.process.start()
except Exception as e:
debug_print(True, f'Failed to start browser process: {e}', title = "Browser Env", log_path=self.log_path)
# self.logger.info(f'Failed to start browser process: {e}', title="Browser Env", color="red")
raise
if not self.check_alive():
self.close()
raise BrowserInitException('Failed to start browser environment.')
def browser_process(self):
if self.eval_mode:
assert self.browsergym_eval_env is not None
debug_print(True, 'Initializing browser env for web browsing evaluation.', title = "Browser Env", log_path=self.log_path)
# self.logger.info('Initializing browser env for web browsing evaluation.', title="Browser Env", color="green")
if 'webarena' in self.browsergym_eval_env:
import browsergym.webarena # noqa F401 register webarena tasks as gym environments
elif 'miniwob' in self.browsergym_eval_env:
import browsergym.miniwob # noqa F401 register miniwob tasks as gym environments
else:
raise ValueError(
f'Unsupported browsergym eval env: {self.browsergym_eval_env}'
)
env = gym.make(
self.browsergym_eval_env,
tags_to_mark='all',
)
else:
from browsergym.core.action.highlevel import HighLevelActionSet
def _local_to_docker(local_path: str):
"""
Convert a local path to a docker path
local_path: the local path to convert, like `{local_workplace}/downloads/xxx`
docker_path: the docker path to convert, like `{docker_workplace}/downloads/xxx`
Examples:
_local_to_docker('/Users/tangjiabin/Documents/reasoning/autoagent/workplace_gaia_eval/downloads/xxx')
"""
local_workplace = None
docker_workplace = None
assert local_workplace in local_path, f"local_path must contain {local_workplace}"
return local_path.replace(local_workplace, docker_workplace)
source = inspect.getsource(_local_to_docker)
normalized_source = textwrap.dedent(source)
normalized_source = normalized_source.replace('local_workplace = None', f'local_workplace = {repr(self.local_workplace)}')
normalized_source = normalized_source.replace('docker_workplace = None', f'docker_workplace = {repr(self.docker_workplace)}')
action_set = HighLevelActionSet(subsets = ["chat", "infeas", "bid", "nav", "tab", "custom"], custom_actions = [_visit_page, _click_id, _get_page_markdown, _checkMeetChallenge])
# action_set.python_includes = \
# f"""
# {repr(read_file('autoagent/environment/markdown_browser/mdconvert.py'))}
# """ + action_set.python_includes
action_set.python_includes = f"""\
{convert_cookies_to_python()}
""" + action_set.python_includes
action_set.python_includes = f"""\
def _local_to_docker(local_path: str):
local_workplace = {repr(self.local_workplace)}
docker_workplace = {repr(self.docker_workplace)}
assert local_workplace in local_path
return local_path.replace(local_workplace, docker_workplace)
""" + action_set.python_includes
action_set.python_includes = f"local_workplace = {repr(self.local_workplace)}\n" + action_set.python_includes
# action_set.python_includes = f"LOCAL_ROOT = {repr(LOCAL_ROOT)}\n" + action_set.python_includes
# print(action_set.python_includes)
action_mapping = action_set.to_python_code
env = gym.make(
'browsergym/openended',
task_kwargs={'start_url': 'about:blank', 'goal': 'PLACEHOLDER_GOAL'},
wait_for_user_message=False,
headless=True,
disable_env_checker=True,
tags_to_mark='all',
action_mapping = action_mapping
)
obs, info = env.reset()
# self.viewport = env.env.viewport if env.env.viewport else env.env.task.viewport
# print(f"Viewport: {self.viewport}")
# 通过管道发送viewport信息
# EVAL ONLY: save the goal into file for evaluation
self.eval_goal = None
self.eval_rewards: list[float] = []
if self.eval_mode:
debug_print(True, f"Browsing goal: {obs['goal']}", title = "Browser Env", log_path=self.log_path)
# self.logger.info(f"Browsing goal: {obs['goal']}", title="Browser Env", color="green")
self.eval_goal = obs['goal']
debug_print(True, 'Browser env started.', title = "Browser Env", log_path=self.log_path)
# self.logger.info('Browser env started.', title="Browser Env", color="green")
while should_continue():
try:
if self.browser_side.poll(timeout=0.01):
unique_request_id, action_data = self.browser_side.recv()
# shutdown the browser environment
if unique_request_id == 'SHUTDOWN':
debug_print(False, 'SHUTDOWN recv, shutting down browser env...', title = "Browser Env", log_path=self.log_path)
# self.logger.info('SHUTDOWN recv, shutting down browser env...', title="Browser Env", color="green")
env.close()
return
elif unique_request_id == 'IS_ALIVE':
self.browser_side.send(('ALIVE', None))
continue
# EVAL ONLY: Get evaluation info
if action_data['action'] == BROWSER_EVAL_GET_GOAL_ACTION:
self.browser_side.send(
(unique_request_id, {'text_content': self.eval_goal})
)
continue
elif action_data['action'] == BROWSER_EVAL_GET_REWARDS_ACTION:
self.browser_side.send(
(
unique_request_id,
{'text_content': json.dumps(self.eval_rewards)},
)
)
continue
action = action_data['action']
obs, reward, terminated, truncated, info = env.step(action)
# EVAL ONLY: Save the rewards into file for evaluation
if self.eval_mode:
self.eval_rewards.append(reward)
# add text content of the page
html_str = flatten_dom_to_str(obs['dom_object'])
obs['text_content'] = self.html_text_converter.handle(html_str)
# make observation serializable
obs['screenshot'] = self.image_to_png_base64_url(obs['screenshot'])
obs['active_page_index'] = obs['active_page_index'].item()
obs['elapsed_time'] = obs['elapsed_time'].item()
self.browser_side.send((unique_request_id, obs))
except KeyboardInterrupt:
debug_print(True, 'Browser env process interrupted by user.', title = "Browser Env", log_path=self.log_path)
# self.logger.info('Browser env process interrupted by user.', title="Browser Env", color="green")
try:
env.close()
except Exception:
pass
return
def step(self, action_str: str, timeout: float = 30) -> dict:
"""Execute an action in the browser environment and return the observation."""
unique_request_id = str(uuid.uuid4())
self.agent_side.send((unique_request_id, {'action': action_str}))
start_time = time.time()
while True:
if should_exit() or (time.time() - start_time > timeout and '_visit_page' not in action_str):
raise TimeoutError('Browser environment took too long to respond.')
if should_exit() or (time.time() - start_time > 600 and '_visit_page' in action_str):
raise TimeoutError('Browser environment took too long to respond.')
if self.agent_side.poll(timeout=0.01):
response_id, obs = self.agent_side.recv()
if response_id == unique_request_id:
return obs
def check_alive(self, timeout: float = 60):
self.agent_side.send(('IS_ALIVE', None))
if self.agent_side.poll(timeout=timeout):
response_id, _ = self.agent_side.recv()
if response_id == 'ALIVE':
return True
debug_print(True, f'Browser env is not alive. Response ID: {response_id}', title = "Browser Env", log_path=self.log_path)
# self.logger.info(f'Browser env is not alive. Response ID: {response_id}', title="Browser Env", color="red")
def close(self):
if not self.process.is_alive():
return
try:
self.agent_side.send(('SHUTDOWN', None))
self.process.join(5) # Wait for the process to terminate
if self.process.is_alive():
debug_print(True, 'Browser process did not terminate, forcefully terminating...', title = "Browser Env", log_path=self.log_path)
# self.logger.info('Browser process did not terminate, forcefully terminating...', title="Browser Env", color="red")
self.process.terminate()
self.process.join(5) # Wait for the process to terminate
if self.process.is_alive():
self.process.kill()
self.process.join(5) # Wait for the process to terminate
self.agent_side.close()
self.browser_side.close()
except Exception:
debug_print(True, 'Encountered an error when closing browser env', exc_info=True, title = "Browser Env", log_path=self.log_path)
# self.logger.info('Encountered an error when closing browser env', exc_info=True, title="Browser Env", color="red")
@staticmethod
def image_to_png_base64_url(
image: np.ndarray | Image.Image, add_data_prefix: bool = False
):
"""Convert a numpy array to a base64 encoded png image url."""
if isinstance(image, np.ndarray):
image = Image.fromarray(image)
if image.mode in ('RGBA', 'LA'):
image = image.convert('RGB')
buffered = io.BytesIO()
image.save(buffered, format='PNG')
image_base64 = base64.b64encode(buffered.getvalue()).decode()
return (
f'data:image/png;base64,{image_base64}'
if add_data_prefix
else f'{image_base64}'
)
@staticmethod
def image_to_jpg_base64_url(
image: np.ndarray | Image.Image, add_data_prefix: bool = False
):
"""Convert a numpy array to a base64 encoded jpeg image url."""
if isinstance(image, np.ndarray):
image = Image.fromarray(image)
if image.mode in ('RGBA', 'LA'):
image = image.convert('RGB')
buffered = io.BytesIO()
image.save(buffered, format='JPEG')
image_base64 = base64.b64encode(buffered.getvalue()).decode()
return (
f'data:image/jpeg;base64,{image_base64}'
if add_data_prefix
else f'{image_base64}'
)
def source_to_function(source_code: str, func_name: str):
"""将源代码字符串转换为函数，支持 inspect.getsource"""
# 创建临时文件
with tempfile.NamedTemporaryFile(mode='w', suffix='.py', delete=False) as f:
f.write(source_code)
temp_path = f.name
try:
# 导入临时模块
import importlib.util
spec = importlib.util.spec_from_file_location("temp_module", temp_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
# 获取函数
func = getattr(module, func_name)
return func
finally:
# 清理临时文件
os.unlink(temp_path)
autoagent/environment/cookie_json/README.md
+36
View File
@@ -0,0 +1,36 @@
# How to obtain cookie json files
## What are cookies?
Cookies are small pieces of data stored by websites on users' computers, containing information like login status and preferences. They are essential for web automation as they allow automated browsers to maintain authenticated sessions, skip repeated logins, and simulate real user behavior across multiple page visits.
## How to organize them in this folder?
We recommend you to use the Google Chrome browser with the extension "Export cookie JSON file for Puppeteer", as show in the following figure:
![extension](../../../assets/cookies/extension.png)
1. Go to a specific website and login.
2. Then use the extension to export the cookies, and save it as a json file in the `cookie_json` folder.
![export](../../../assets/cookies/export.png)
3. After you have exported all cookies, use the following command to convert them to python code:
```bash
cd path/to/MetaChain && python autoagent/environment/browser_cookies.py
```
## Recommended websites
We recommend you to export the cookies from the following websites:
- [archive.org](https://archive.org)
- [github.com](https://github.com)
- [nature.com](https://nature.com)
- [orcid.org](https://orcid.org)
- [www.collinsdictionary.com](https://www.collinsdictionary.com)
- [www.jstor.org](https://www.jstor.org)
- [www.ncbi.nlm.nih.gov](https://www.ncbi.nlm.nih.gov)
- [www.pnas.org](https://www.pnas.org)
- [www.reddit.com](https://www.reddit.com)
- [www.researchgate.net](https://www.researchgate.net)
- [www.youtube.com](https://www.youtube.com)
autoagent/environment/cookies_data.py
+2
View File
@@ -0,0 +1,2 @@
COOKIES_LIST = [
]
autoagent/environment/docker_env.py
+276
View File
@@ -0,0 +1,276 @@
import os
import os.path as osp
import subprocess
from constant import BASE_IMAGES, AI_USER, GITHUB_AI_TOKEN
import time
import socket
import json
from pathlib import Path
import shutil
wd = Path(__file__).parent.resolve()
from dataclasses import dataclass, field
from typing import Optional, Union, Dict
from functools import update_wrapper
from inspect import signature
@dataclass
class DockerConfig:
container_name: str
workplace_name: str
communication_port: int # 12345
conda_path: str # /root/miniconda3
test_pull_name: str = field(default='main')
task_name: Optional[str] = field(default=None)
git_clone: bool = field(default=False)
setup_package: Optional[str] = field(default=None)
local_root: str = field(default=os.getcwd())
class DockerEnv:
def __init__(self, config: Union[DockerConfig, Dict]):
if isinstance(config, Dict):
config = DockerConfig(**config)
self.workplace_name = config.workplace_name
self.local_workplace = osp.join(config.local_root, config.workplace_name)
self.docker_workplace = f"/{config.workplace_name}"
self.container_name = config.container_name
self.test_pull_name = config.test_pull_name
self.task_name = config.task_name
self.git_clone = config.git_clone
self.setup_package = config.setup_package
self.communication_port = config.communication_port
self.conda_path = config.conda_path
def init_container(self):
container_check_command = ["docker", "ps", "-a", "--filter", f"name={self.container_name}", "--format", "{{.Names}}"]
existing_container = subprocess.run(container_check_command, capture_output=True, text=True)
os.makedirs(self.local_workplace, exist_ok=True)
if not osp.exists(osp.join(self.local_workplace, 'tcp_server.py')):
shutil.copy(osp.join(wd, 'tcp_server.py'), self.local_workplace)
assert osp.exists(osp.join(self.local_workplace, 'tcp_server.py')), "Failed to copy tcp_server.py to the local workplace"
if self.setup_package is not None:
unzip_command = ["tar", "-xzvf", f"packages/{self.setup_package}.tar.gz", "-C", self.local_workplace]
subprocess.run(unzip_command)
if self.git_clone:
if not os.path.exists(os.path.join(self.local_workplace, 'AutoAgent')):
git_command = ["cd", self.local_workplace, "&&", "git", "clone", "-b", self.test_pull_name, f"https://{AI_USER}:{GITHUB_AI_TOKEN}@github.com/HKUDS/AutoAgent.git"]
print(git_command)
git_command = " ".join(git_command)
result = subprocess.run(git_command, shell=True)
if result.returncode != 0:
raise Exception(f"Failed to clone the repository. The error is: {result.stdout}")
copy_env_command = f"cp .env {self.local_workplace}/AutoAgent"
result = subprocess.run(copy_env_command, shell=True, capture_output=True, text=True)
if result.returncode != 0:
raise Exception(f"Failed to copy .env file to the AutoAgent directory. Error: {result.stderr}")
# create a new branch
new_branch_name = f"{self.test_pull_name}_{self.task_name}"
create_branch_command = f"cd {self.local_workplace}/AutoAgent && git checkout -b {new_branch_name}"
result = subprocess.run(create_branch_command, shell=True, capture_output=True, text=True)
if result.returncode != 0:
print(Exception(f"Failed to create and switch to new branch. Error: {result.stderr}"))
switch_branch_command = f"cd {self.local_workplace}/AutoAgent && git checkout {new_branch_name}"
result = subprocess.run(switch_branch_command, shell=True, capture_output=True, text=True)
if result.returncode != 0:
raise Exception(f"Failed to switch to new branch. Error: {result.stderr}")
else:
print(f"Successfully switched to new branch: {new_branch_name}")
else:
print(f"Successfully created and switched to new branch: {new_branch_name}")
if existing_container.stdout.strip() == self.container_name:
# check if the container is running
running_check_command = ["docker", "ps", "--filter", f"name={self.container_name}", "--format", "{{.Names}}"]
running_container = subprocess.run(running_check_command, capture_output=True, text=True)
if running_container.stdout.strip() == self.container_name:
print(f"Container '{self.container_name}' is already running. Skipping creation.")
return # container is already running, skip creation
else:
# container exists but is not running, start it
start_command = ["docker", "start", self.container_name]
subprocess.run(start_command)
print(f"Container '{self.container_name}' has been started.")
return
# if the container does not exist, create and start a new container
docker_command = [
"docker", "run", "-d", "--name", self.container_name, "--user", "root",
"-v", f"{self.local_workplace}:{self.docker_workplace}",
"-w", f"{self.docker_workplace}", "-p", f"{self.communication_port}:{self.communication_port}", BASE_IMAGES,
"/bin/bash", "-c",
f"python3 {self.docker_workplace}/tcp_server.py --workplace {self.workplace_name} --conda_path {self.conda_path} --port {self.communication_port}"
]
# execute the docker command
result = subprocess.run(docker_command, capture_output=True, text=True)
if result.returncode != 0:
raise Exception(f"Failed to start container: {result.stderr}")
if self.wait_for_container_ready(timeout=60):
print(f"Container '{self.container_name}' has been created and started.")
def wait_for_container_ready(self, timeout=30):
"""using subprocess to check if the container is running"""
start_time = time.time()
while time.time() - start_time < timeout:
result = subprocess.run(
["docker", "inspect", "--format", "{{.State.Running}}", self.container_name],
capture_output=True,
text=True
)
if result.returncode == 0 and "true" in result.stdout.lower():
# 额外检查 tcp_server 是否运行
try:
port_info = check_container_ports(self.container_name)
assert port_info and (port_info[0] == port_info[1])
available_port = port_info[0]
self.communication_port = available_port
result = self.run_command('ps aux')
if "tcp_server.py" in result['result']:
return True
except Exception as e:
pass
time.sleep(1)
raise TimeoutError(f"Container {self.container_name} failed to start within {timeout} seconds")
def stop_container(self):
stop_command = ["docker", "stop", self.container_name]
result = subprocess.run(stop_command, capture_output=True, text=True)
if result.returncode != 0:
raise Exception(f"Failed to stop container: {result.stderr}")
def run_command(self, command, stream_callback=None):
"""
communicate with docker container and execute command, support stream output
Args:
command: the command to execute
stream_callback: optional callback function, for handling stream output
the function signature should be callback(text: str)
Returns:
dict: the complete JSON result returned by the docker container
"""
hostname = 'localhost'
port = self.communication_port
buffer_size = 4096
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((hostname, port))
s.sendall(command.encode())
partial_line = ""
while True:
chunk = s.recv(buffer_size)
if not chunk:
break
# add new received data to the unfinished data
data = partial_line + chunk.decode('utf-8')
lines = data.split('\n')
# except the last line, process all complete lines
for line in lines[:-1]:
if line:
try:
response = json.loads(line)
if response['type'] == 'chunk':
# process stream output
if stream_callback:
stream_callback(response['data'])
elif response['type'] == 'final':
# return the final result
return {
'status': response['status'],
'result': response['result']
}
except json.JSONDecodeError:
print(f"Invalid JSON: {line}")
# save the possibly unfinished last line
partial_line = lines[-1]
# if the loop ends normally without receiving a final response
return {
'status': -1,
'result': 'Connection closed without final response'
}
def with_env(env: DockerEnv):
"""将env注入到工具函数中的装饰器"""
def decorator(func):
def wrapped(*args, **kwargs):
return func(env=env, *args, **kwargs)
# 保留原始函数的所有属性
update_wrapper(wrapped, func)
# 修改signature，移除env参数
wrapped.__signature__ = signature(func).replace(
parameters=[p for p in signature(func).parameters.values() if p.name != 'env']
)
if func.__doc__:
try:
if '{docker_workplace}' in func.__doc__:
wrapped.__doc__ = func.__doc__.format(docker_workplace=env.docker_workplace)
else:
wrapped.__doc__ = func.__doc__
if '{local_workplace}' in func.__doc__:
wrapped.__doc__ = func.__doc__.format(local_workplace=env.local_workplace)
else:
wrapped.__doc__ = func.__doc__
except (KeyError, IndexError, ValueError):
# 如果格式化失败（没有占位符），保持原始文档
wrapped.__doc__ = func.__doc__
return wrapped
return decorator
def check_container_ports(container_name: str):
"""
check if the container has port mapping
return format:
- if the container exists and has port mapping: '0.0.0.0:12345->12345/tcp'
- if the container does not exist or does not have port mapping: None
"""
# use docker ps to check the container and get the port information
container_check_command = [
"docker", "ps", "-a",
"--filter", f"name={container_name}",
"--format", "{{.Ports}}"
]
result = subprocess.run(container_check_command, capture_output=True, text=True)
ports_info = result.stdout.strip()
if not ports_info:
return None
# only process the mapped ports
for mapping in ports_info.split(','):
mapping = mapping.strip()
if '->' in mapping:
# parse '0.0.0.0:12345->12345/tcp' to (12345, 12345)
host_part, container_part = mapping.split('->')
host_port = host_part.split(':')[1] # get '12345' from '0.0.0.0:12345'
container_port = container_part.split('/')[0] # get '12345' from '12345/tcp'
return (int(host_port), int(container_port)) # convert to integers
return None
def check_container_exist(container_name: str):
container_check_command = [
"docker", "ps", "-a",
"--filter", f"name={container_name}",
"--format", "{{.Names}}"
]
result = subprocess.run(container_check_command, capture_output=True, text=True)
return container_name in result.stdout.strip()
def check_container_running(container_name: str):
container_check_command = [
"docker", "ps",
"--filter", f"name={container_name}",
"--format", "{{.Names}}"
]
result = subprocess.run(container_check_command, capture_output=True, text=True)
return container_name in result.stdout.strip()
autoagent/environment/local_env.py
+101
View File
@@ -0,0 +1,101 @@
import subprocess
import json
import os
from pathlib import Path
import platform
import os.path as osp
from autoagent.environment.docker_env import DockerConfig
class LocalEnv:
def __init__(self, docker_config: DockerConfig = None):
if docker_config is None:
self.docker_workplace = os.getcwd()
if self.docker_workplace.endswith("autoagent"):
self.docker_workplace = os.path.dirname(self.docker_workplace)
self.local_workplace = self.docker_workplace
else:
self.local_workplace = osp.join(docker_config.local_root, docker_config.workplace_name)
self.docker_workplace = osp.join(docker_config.local_root, docker_config.workplace_name)
os.makedirs(self.local_workplace, exist_ok=True)
self.conda_sh = self._find_conda_sh()
def _find_conda_sh(self) -> str:
"""
Find conda.sh file location across different environments
"""
# 1. Try common locations based on OS
possible_paths = []
home = str(Path.home())
if platform.system() == "Windows":
possible_paths.extend([
Path(home) / "Anaconda3" / "etc" / "profile.d" / "conda.sh",
Path(home) / "miniconda3" / "etc" / "profile.d" / "conda.sh",
Path(home) / "micromamba" / "etc" / "profile.d" / "conda.sh",
])
else: # Linux and MacOS
possible_paths.extend([
Path(home) / "anaconda3" / "etc" / "profile.d" / "conda.sh",
Path(home) / "miniconda3" / "etc" / "profile.d" / "conda.sh",
Path(home) / "micromamba" / "etc" / "profile.d" / "conda.sh",
Path("/opt/conda/etc/profile.d/conda.sh"), # Docker containers
Path("/usr/local/conda/etc/profile.d/conda.sh"),
])
# For Linux, also check root installations
if platform.system() == "Linux":
possible_paths.extend([
Path("/opt/anaconda3/etc/profile.d/conda.sh"),
Path("/opt/miniconda3/etc/profile.d/conda.sh"),
Path("/opt/micromamba/etc/profile.d/conda.sh"),
])
# Check all possible paths
for path in possible_paths:
if path.exists():
return str(path)
# 2. Try to find using conda info command
try:
result = subprocess.run(['conda', 'info', '--base'],
capture_output=True,
text=True)
if result.returncode == 0:
base_path = result.stdout.strip()
conda_sh = Path(base_path) / "etc" / "profile.d" / "conda.sh"
if conda_sh.exists():
return str(conda_sh)
except:
pass
# 3. If all fails, return None and handle in run_command
return None
def run_command(self, command, stream_callback=None):
assert self.conda_sh is not None, "Conda.sh not found"
modified_command = f"/bin/bash -c 'source {self.conda_sh} && conda activate auto && cd {self.docker_workplace} && {command}'"
process = subprocess.Popen(modified_command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
output = ''
while True:
line = process.stdout.readline()
if not line and process.poll() is not None:
break
output += line
# 立即发送每一行输出
# 发送最终的完整响应
response = {
"status": process.poll(),
"result": output
}
return response
def _convert_local_to_docker(self, path):
return path
def _convert_docker_to_local(self, path):
return path
if __name__ == "__main__":
# print(str(Path.home()))
local_env = LocalEnv()
print(local_env.conda_sh)
autoagent/environment/markdown_browser/__init__.py
+22
View File
@@ -0,0 +1,22 @@
from .abstract_markdown_browser import AbstractMarkdownBrowser
from .markdown_search import AbstractMarkdownSearch, BingMarkdownSearch
# TODO: Fix mdconvert
from .mdconvert import ( # type: ignore
DocumentConverterResult,
FileConversionException,
MarkdownConverter,
UnsupportedFormatException,
)
from .requests_markdown_browser import RequestsMarkdownBrowser
__all__ = (
"AbstractMarkdownBrowser",
"RequestsMarkdownBrowser",
"AbstractMarkdownSearch",
"BingMarkdownSearch",
"MarkdownConverter",
"UnsupportedFormatException",
"FileConversionException",
"DocumentConverterResult",
)
autoagent/environment/markdown_browser/abstract_markdown_browser.py
+64
View File
@@ -0,0 +1,64 @@
from abc import ABC, abstractmethod
from typing import Union
class AbstractMarkdownBrowser(ABC):
"""
An abstract class for a Markdown web browser.
All MarkdownBrowers work by:
(1) fetching a web page by URL (via requests, Selenium, Playwright, etc.)
(2) converting the page's HTML or DOM to Markdown
(3) operating on the Markdown
Such browsers are simple, and suitable for read-only agentic use.
They cannot be used to interact with complex web applications.
"""
@abstractmethod
def __init__(self) -> None:
pass
@property
@abstractmethod
def address(self) -> str:
pass
@abstractmethod
def set_address(self, uri_or_path: str) -> None:
pass
@property
@abstractmethod
def viewport(self) -> str:
pass
@property
@abstractmethod
def page_content(self) -> str:
pass
@abstractmethod
def page_down(self) -> None:
pass
@abstractmethod
def page_up(self) -> None:
pass
@abstractmethod
def visit_page(self, path_or_uri: str) -> str:
pass
@abstractmethod
def open_local_file(self, local_path: str) -> str:
pass
@abstractmethod
def find_on_page(self, query: str) -> Union[str, None]:
pass
@abstractmethod
def find_next(self) -> Union[str, None]:
pass
autoagent/environment/markdown_browser/markdown_search.py
+282
View File
@@ -0,0 +1,282 @@
import logging
import os
import re
from abc import ABC, abstractmethod
from typing import Any, Dict, List, Optional, cast
from urllib.parse import quote, quote_plus, unquote, urlparse, urlunparse
import requests
# TODO: Fix these types
from .mdconvert import MarkdownConverter # type: ignore
logger = logging.getLogger(__name__)
class AbstractMarkdownSearch(ABC):
"""
An abstract class for providing search capabilities to a Markdown browser.
"""
@abstractmethod
def search(self, query: str) -> str:
pass
class BingMarkdownSearch(AbstractMarkdownSearch):
"""
Provides Bing web search capabilities to Markdown browsers.
"""
def __init__(self, bing_api_key: Optional[str] = None, interleave_results: bool = True):
"""
Perform a Bing web search, and return the results formatted in Markdown.
Args:
bing_api_key: key for the Bing search API. If omitted, an attempt is made to read the key from the BING_API_KEY environment variable. If no key is found, BingMarkdownSearch will print a warning, and will fall back to visiting and scraping the live Bing results page. Scraping is objectively worse than using the API, and thus is not recommended.
interleave_results: When using the Bing API, results are returned based on category (web, news, videos, etc.), along with instructions for how they should be interleaved on the page. When `interleave` is set to True, these interleaving instructions are followed, and a single results list is returned by BingMarkdownSearch. When `interleave` is set to false, results are separated by category, and no interleaving is done.
"""
self._mdconvert = MarkdownConverter()
self._interleave_results = interleave_results
if bing_api_key is None or bing_api_key.strip() == "":
self._bing_api_key = os.environ.get("BING_API_KEY")
else:
self._bing_api_key = bing_api_key
if self._bing_api_key is None:
if not self._interleave_results:
raise ValueError(
"No Bing API key was provided. This is incompatible with setting `interleave_results` to False. Please provide a key, or set `interleave_results` to True."
)
# logger.warning(
# "Warning: No Bing API key provided. BingMarkdownSearch will submit an HTTP request to the Bing landing page, but results may be missing or low quality. To resolve this warning, provide a Bing API key by setting the BING_API_KEY environment variable, or using the 'bing_api_key' parameter in by BingMarkdownSearch's constructor. Bing API keys can be obtained via https://www.microsoft.com/en-us/bing/apis/bing-web-search-api\n"
# )
def search(self, query: str) -> str:
"""Search Bing and return the results formatted in Markdown. If a Bing API key is available, the API is used to perform the search. If no API key is available, the search is performed by submitting an HTTPs GET request directly to Bing. Searches performed with the API are much higher quality, and are more reliable.
Args:
query: The search query to issue
Returns:
A Markdown rendering of the search results.
"""
if self._bing_api_key is None:
return self._fallback_search(query)
else:
return self._api_search(query)
def _api_search(self, query: str) -> str:
"""Search Bing using the API, and return the results formatted in Markdown.
Args:
query: The search query to issue
Returns:
A Markdown rendering of the search results.
"""
results = self._bing_api_call(query)
snippets: Dict[str, List[str]] = dict()
def _processFacts(elm: List[Dict[str, Any]]) -> str:
facts: List[str] = list()
for e in elm:
k = e["label"]["text"]
v = " ".join(item["text"] for item in e["items"])
facts.append(f"{k}: {v}")
return "\n".join(facts)
# Web pages
# __POS__ is a placeholder for the final ranking position, added at the end
web_snippets: List[str] = list()
if "webPages" in results:
for page in results["webPages"]["value"]:
snippet = f"__POS__. {self._markdown_link(page['name'], page['url'])}\n{page['snippet']}"
if "richFacts" in page:
snippet += "\n" + _processFacts(page["richFacts"])
if "mentions" in page:
snippet += "\nMentions: " + ", ".join(e["name"] for e in page["mentions"])
if page["id"] not in snippets:
snippets[page["id"]] = list()
snippets[page["id"]].append(snippet)
web_snippets.append(snippet)
if "deepLinks" in page:
for dl in page["deepLinks"]:
deep_snippet = f"__POS__. {self._markdown_link(dl['name'], dl['url'])}\n{dl['snippet'] if 'snippet' in dl else ''}"
snippets[page["id"]].append(deep_snippet)
web_snippets.append(deep_snippet)
# News results
news_snippets: List[str] = list()
if "news" in results:
for page in results["news"]["value"]:
snippet = (
f"__POS__. {self._markdown_link(page['name'], page['url'])}\n{page.get('description', '')}".strip()
)
if "datePublished" in page:
snippet += "\nDate published: " + page["datePublished"].split("T")[0]
if "richFacts" in page:
snippet += "\n" + _processFacts(page["richFacts"])
if "mentions" in page:
snippet += "\nMentions: " + ", ".join(e["name"] for e in page["mentions"])
news_snippets.append(snippet)
if len(news_snippets) > 0:
snippets[results["news"]["id"]] = news_snippets
# Videos
video_snippets: List[str] = list()
if "videos" in results:
for page in results["videos"]["value"]:
if not page["contentUrl"].startswith("https://www.youtube.com/watch?v="):
continue
snippet = f"__POS__. {self._markdown_link(page['name'], page['contentUrl'])}\n{page.get('description', '')}".strip()
if "datePublished" in page:
snippet += "\nDate published: " + page["datePublished"].split("T")[0]
if "richFacts" in page:
snippet += "\n" + _processFacts(page["richFacts"])
if "mentions" in page:
snippet += "\nMentions: " + ", ".join(e["name"] for e in page["mentions"])
video_snippets.append(snippet)
if len(video_snippets) > 0:
snippets[results["videos"]["id"]] = video_snippets
# Related searches
related_searches = ""
if "relatedSearches" in results:
related_searches = "## Related Searches:\n"
for s in results["relatedSearches"]["value"]:
related_searches += "- " + s["text"] + "\n"
snippets[results["relatedSearches"]["id"]] = [related_searches.strip()]
idx = 0
content = ""
if self._interleave_results:
# Interleaved
for item in results["rankingResponse"]["mainline"]["items"]:
_id = item["value"]["id"]
if _id in snippets:
for s in snippets[_id]:
if "__POS__" in s:
idx += 1
content += s.replace("__POS__", str(idx)) + "\n\n"
else:
content += s + "\n\n"
else:
# Categorized
if len(web_snippets) > 0:
content += "## Web Results\n\n"
for s in web_snippets:
if "__POS__" in s:
idx += 1
content += s.replace("__POS__", str(idx)) + "\n\n"
else:
content += s + "\n\n"
if len(news_snippets) > 0:
content += "## News Results\n\n"
for s in news_snippets:
if "__POS__" in s:
idx += 1
content += s.replace("__POS__", str(idx)) + "\n\n"
else:
content += s + "\n\n"
if len(video_snippets) > 0:
content += "## Video Results\n\n"
for s in video_snippets:
if "__POS__" in s:
idx += 1
content += s.replace("__POS__", str(idx)) + "\n\n"
else:
content += s + "\n\n"
if len(related_searches) > 0:
content += related_searches
return f"## A Bing search for '{query}' found {idx} results:\n\n" + content.strip()
def _bing_api_call(self, query: str) -> Dict[str, Any]:
"""Make a Bing API call, and return a Python representation of the JSON response."
Args:
query: The search query to issue
Returns:
A Python representation of the Bing API's JSON response (as parsed by `json.loads()`).
"""
# Make sure the key was set
if not self._bing_api_key:
raise ValueError("Missing Bing API key.")
# Prepare the request parameters
request_kwargs: Dict[str, Any] = {}
request_kwargs["headers"] = {}
request_kwargs["headers"]["Ocp-Apim-Subscription-Key"] = self._bing_api_key
request_kwargs["params"] = {}
request_kwargs["params"]["q"] = query
request_kwargs["params"]["textDecorations"] = False
request_kwargs["params"]["textFormat"] = "raw"
request_kwargs["stream"] = False
# Make the request
response = requests.get("https://api.bing.microsoft.com/v7.0/search", **request_kwargs)
response.raise_for_status()
results = response.json()
return cast(Dict[str, Any], results)
def _fallback_search(self, query: str) -> str:
"""When no Bing API key is provided, we issue a simple HTTPs GET call to the Bing landing page and convert it to Markdown.
Args:
query: The search query to issue
Returns:
The Bing search results page, converted to Markdown.
"""
user_agent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0"
headers = {"User-Agent": user_agent}
url = f"https://www.bing.com/search?q={quote_plus(query)}&FORM=QBLH"
response = requests.get(url, headers=headers)
response.raise_for_status()
# TODO: Fix the types
return self._mdconvert.convert_response(response).text_content # type: ignore
def _markdown_link(self, anchor: str, href: str) -> str:
"""Create a Markdown hyperlink, escaping the URLs as appropriate.
Args:
anchor: The anchor text of the hyperlink
href: The href destination of the hyperlink
Returns:
A correctly-formatted Markdown hyperlink
"""
try:
parsed_url = urlparse(href)
# URLs provided by Bing are sometimes only partially quoted, leaving in characters
# the conflict with Markdown. We unquote the URL, and then re-quote more completely
href = urlunparse(parsed_url._replace(path=quote(unquote(parsed_url.path))))
anchor = re.sub(r"[\[\]]", " ", anchor)
return f"[{anchor}]({href})"
except ValueError: # It's not clear if this ever gets thrown
return f"[{anchor}]({href})"
autoagent/environment/markdown_browser/mdconvert.py
+1033
View File
File diff suppressed because it is too large Load Diff
autoagent/environment/markdown_browser/requests_markdown_browser.py
+454
View File
@@ -0,0 +1,454 @@
# ruff: noqa: E722
import datetime
import html
import io
import mimetypes
import os
import pathlib
import re
import time
import traceback
import uuid
from typing import Any, Dict, List, Optional, Tuple, Union
from urllib.parse import unquote, urljoin, urlparse
import pathvalidate
import requests
from .abstract_markdown_browser import AbstractMarkdownBrowser
from .markdown_search import AbstractMarkdownSearch, BingMarkdownSearch
# TODO: Fix unfollowed import
from .mdconvert import FileConversionException, MarkdownConverter, UnsupportedFormatException # type: ignore
class RequestsMarkdownBrowser(AbstractMarkdownBrowser):
"""
(In preview) An extremely simple Python requests-powered Markdown web browser.
This browser cannot run JavaScript, compute CSS, etc. It simply fetches the HTML document, and converts it to Markdown.
See AbstractMarkdownBrowser for more details.
"""
# TODO: Fix unfollowed import
def __init__( # type: ignore
self,
local_root: str,
workplace_name: str,
start_page: Union[str, None] = None,
viewport_size: Union[int, None] = 1024 * 8,
downloads_folder: Union[str, None] = None,
search_engine: Union[AbstractMarkdownSearch, None] = None,
markdown_converter: Union[MarkdownConverter, None] = None,
requests_session: Union[requests.Session, None] = None,
requests_get_kwargs: Union[Dict[str, Any], None] = None,
):
"""
Instantiate a new RequestsMarkdownBrowser.
Arguments:
start_page: The page on which the browser starts (default: "about:blank")
viewport_size: Approximately how many *characters* fit in the viewport. Viewport dimensions are adjusted dynamically to avoid cutting off words (default: 8192).
downloads_folder: Path to where downloads are saved. If None, downloads are disabled. (default: None)
search_engine: An instance of MarkdownSearch, which handles web searches performed by this browser (default: a new `BingMarkdownSearch()` with default parameters)
markdown_converted: An instance of a MarkdownConverter used to convert HTML pages and downloads to Markdown (default: a new `MarkdownConerter()` with default parameters)
request_session: The session from which to issue requests (default: a new `requests.Session()` instance with default parameters)
request_get_kwargs: Extra parameters passed to evert `.get()` call made to requests.
"""
self.local_workplace = os.path.join(local_root, workplace_name)
self.docker_workplace = f"/{workplace_name}"
self.start_page: str = start_page if start_page else "about:blank"
self.viewport_size = viewport_size # Applies only to the standard uri types
self.downloads_folder = downloads_folder
self.history: List[Tuple[str, float]] = list()
self.page_title: Optional[str] = None
self.viewport_current_page = 0
self.viewport_pages: List[Tuple[int, int]] = list()
self.set_address(self.start_page)
self._page_content: str = ""
if search_engine is None:
self._search_engine: AbstractMarkdownSearch = BingMarkdownSearch()
else:
self._search_engine = search_engine
if markdown_converter is None:
self._markdown_converter = MarkdownConverter()
else:
self._markdown_converter = markdown_converter
if requests_session is None:
self._requests_session = requests.Session()
else:
self._requests_session = requests_session
if requests_get_kwargs is None:
self._requests_get_kwargs = {}
else:
self._requests_get_kwargs = requests_get_kwargs
self._find_on_page_query: Union[str, None] = None
self._find_on_page_last_result: Union[int, None] = None # Location of the last result
@property
def address(self) -> str:
"""Return the address of the current page."""
return self.history[-1][0]
def _convert_docker_to_local(self, path: str) -> str:
assert self.docker_workplace in path, f"The path must be a absolute path from `{self.docker_workplace}/` directory"
local_path = path.replace(self.docker_workplace, self.local_workplace)
return local_path
def _convert_local_to_docker(self, path: str) -> str:
assert self.local_workplace in path, f"The path must be a absolute path from `{self.local_workplace}/` directory"
docker_path = path.replace(self.local_workplace, self.docker_workplace)
return docker_path
def set_address(self, uri_or_path: str) -> None:
"""Sets the address of the current page.
This will result in the page being fetched via the underlying requests session.
Arguments:
uri_or_path: The fully-qualified URI to fetch, or the path to fetch from the current location. If the URI protocol is `search:`, the remainder of the URI is interpreted as a search query, and a web search is performed. If the URI protocol is `file://`, the remainder of the URI is interpreted as a local absolute file path.
"""
# TODO: Handle anchors
self.history.append((uri_or_path, time.time()))
# Handle special URIs
if uri_or_path == "about:blank":
self._set_page_content("")
elif uri_or_path.startswith("search:"):
query = uri_or_path[len("search:") :].strip()
results = self._search_engine.search(query)
self.page_title = f"{query} - Search"
self._set_page_content(results, split_pages=False)
else:
if (
not uri_or_path.startswith("http:")
and not uri_or_path.startswith("https:")
and not uri_or_path.startswith("file:")
):
if len(self.history) > 1:
prior_address = self.history[-2][0]
uri_or_path = urljoin(prior_address, uri_or_path)
# Update the address with the fully-qualified path
self.history[-1] = (uri_or_path, self.history[-1][1])
self._fetch_page(uri_or_path)
self.viewport_current_page = 0
self.find_on_page_query = None
self.find_on_page_viewport = None
@property
def viewport(self) -> str:
"""Return the content of the current viewport."""
bounds = self.viewport_pages[self.viewport_current_page]
return self.page_content[bounds[0] : bounds[1]]
@property
def page_content(self) -> str:
"""Return the full contents of the current page."""
return self._page_content
def _set_page_content(self, content: str, split_pages: bool = True) -> None:
"""Sets the text content of the current page."""
self._page_content = content
if split_pages:
self._split_pages()
else:
self.viewport_pages = [(0, len(self._page_content))]
if self.viewport_current_page >= len(self.viewport_pages):
self.viewport_current_page = len(self.viewport_pages) - 1
def page_down(self) -> None:
"""Move the viewport down one page, if possible."""
self.viewport_current_page = min(self.viewport_current_page + 1, len(self.viewport_pages) - 1)
def page_up(self) -> None:
"""Move the viewport up one page, if possible."""
self.viewport_current_page = max(self.viewport_current_page - 1, 0)
def page_to(self, page_idx: int) -> None:
"""Move the viewport to the specified page index."""
self.viewport_current_page = min(max(page_idx, 0), len(self.viewport_pages) - 1)
def find_on_page(self, query: str) -> Union[str, None]:
"""Searches for the query from the current viewport forward, looping back to the start if necessary."""
# Did we get here via a previous find_on_page search with the same query?
# If so, map to find_next
if query == self._find_on_page_query and self.viewport_current_page == self._find_on_page_last_result:
return self.find_next()
# Ok it's a new search start from the current viewport
self._find_on_page_query = query
viewport_match = self._find_next_viewport(query, self.viewport_current_page)
if viewport_match is None:
self._find_on_page_last_result = None
return None
else:
self.viewport_current_page = viewport_match
self._find_on_page_last_result = viewport_match
return self.viewport
def find_next(self) -> Union[str, None]:
"""Scroll to the next viewport that matches the query"""
if self._find_on_page_query is None:
return None
starting_viewport = self._find_on_page_last_result
if starting_viewport is None:
starting_viewport = 0
else:
starting_viewport += 1
if starting_viewport >= len(self.viewport_pages):
starting_viewport = 0
viewport_match = self._find_next_viewport(self._find_on_page_query, starting_viewport)
if viewport_match is None:
self._find_on_page_last_result = None
return None
else:
self.viewport_current_page = viewport_match
self._find_on_page_last_result = viewport_match
return self.viewport
def _find_next_viewport(self, query: Optional[str], starting_viewport: int) -> Union[int, None]:
"""Search for matches between the starting viewport looping when reaching the end."""
if query is None:
return None
# Normalize the query, and convert to a regular expression
nquery = re.sub(r"\*", "__STAR__", query)
nquery = " " + (" ".join(re.split(r"\W+", nquery))).strip() + " "
nquery = nquery.replace(" __STAR__ ", "__STAR__ ") # Merge isolated stars with prior word
nquery = nquery.replace("__STAR__", ".*").lower()
if nquery.strip() == "":
return None
idxs: List[int] = list()
idxs.extend(range(starting_viewport, len(self.viewport_pages)))
idxs.extend(range(0, starting_viewport))
for i in idxs:
bounds = self.viewport_pages[i]
content = self.page_content[bounds[0] : bounds[1]]
# TODO: Remove markdown links and images
ncontent = " " + (" ".join(re.split(r"\W+", content))).strip().lower() + " "
if re.search(nquery, ncontent):
return i
return None
def visit_page(self, path_or_uri: str) -> str:
"""Update the address, visit the page, and return the content of the viewport."""
self.set_address(path_or_uri)
return self.viewport
def open_local_file(self, local_path: str) -> str:
"""Convert a local file path to a file:/// URI, update the address, visit the page, and return the contents of the viewport."""
full_path = os.path.abspath(os.path.expanduser(local_path))
self.set_address(pathlib.Path(full_path).as_uri())
return self.viewport
def _split_pages(self) -> None:
"""Split the page contents into pages that are approximately the viewport size. Small deviations are permitted to ensure words are not broken."""
# Handle empty pages
if len(self._page_content) == 0:
self.viewport_pages = [(0, 0)]
return
# Break the viewport into pages
self.viewport_pages = []
start_idx = 0
while start_idx < len(self._page_content):
end_idx = min(start_idx + self.viewport_size, len(self._page_content)) # type: ignore[operator]
# Adjust to end on a space
while end_idx < len(self._page_content) and self._page_content[end_idx - 1] not in [" ", "\t", "\r", "\n"]:
end_idx += 1
self.viewport_pages.append((start_idx, end_idx))
start_idx = end_idx
def _fetch_page(
self,
url: str,
session: Optional[requests.Session] = None,
requests_get_kwargs: Union[Dict[str, Any], None] = None,
) -> None:
"""Fetch a page using the requests library. Then convert it to Markdown, and set `page_content` (which splits the content into pages as necessary.
Arguments:
url: The fully-qualified URL to fetch.
session: Used to override the session used for this request. If None, use `self._requests_session` as usual.
requests_get_kwargs: Extra arguments passes to `requests.Session.get`.
"""
download_path: str = ""
response: Union[requests.Response, None] = None
# print(url)
try:
if url.startswith("file://"):
download_path = os.path.normcase(os.path.normpath(unquote(url[7:])))
if os.path.isdir(download_path): # TODO: Fix markdown_converter types
res = self._markdown_converter.convert_stream( # type: ignore
io.StringIO(self._fetch_local_dir(download_path)), file_extension=".html"
)
self.page_title = res.title
self._set_page_content(
res.text_content, split_pages=False
) # Like search results, don't split directory listings
else:
res = self._markdown_converter.convert_local(download_path)
self.page_title = res.title
self._set_page_content(res.text_content)
else:
# Send a HTTP request to the URL
if session is None:
session = self._requests_session
_get_kwargs: Dict[str, Any] = {} # TODO: Deal with kwargs
_get_kwargs.update(self._requests_get_kwargs)
if requests_get_kwargs is not None:
_get_kwargs.update(requests_get_kwargs)
_get_kwargs["stream"] = True
response = session.get(url, **_get_kwargs)
response.raise_for_status()
# If the HTTP request was successful
content_type = response.headers.get("content-type", "")
# Text or HTML
if "text/" in content_type.lower():
res = self._markdown_converter.convert_response(response)
self.page_title = res.title
self._set_page_content(res.text_content)
# A download
else:
# Was a downloads folder configured?
if self.downloads_folder is None:
self.page_title = "Error 400"
self._set_page_content("## Error 400\n\nClient does not support downloads")
return
assert self.downloads_folder is not None
# Try producing a safe filename
fname: str = ""
try:
fname = pathvalidate.sanitize_filename(os.path.basename(urlparse(url).path)).strip()
download_path = os.path.abspath(os.path.join(self.downloads_folder, fname))
suffix = 0
while os.path.exists(download_path) and suffix < 1000:
suffix += 1
base, ext = os.path.splitext(fname)
new_fname = f"{base}__{suffix}{ext}"
download_path = os.path.abspath(os.path.join(self.downloads_folder, new_fname))
except NameError:
pass
# No suitable name, so make one
if fname == "":
extension = mimetypes.guess_extension(content_type)
if extension is None:
extension = ".download"
fname = str(uuid.uuid4()) + extension
download_path = os.path.abspath(os.path.join(self.downloads_folder, fname))
# Open a file for writing
with open(download_path, "wb") as fh:
for chunk in response.iter_content(chunk_size=512):
fh.write(chunk)
# Render it
local_uri = pathlib.Path(download_path).as_uri()
self.set_address(local_uri)
except UnsupportedFormatException:
self.page_title = "Download complete."
self._set_page_content(f"# Download complete\n\nSaved file to '{download_path}'")
except FileConversionException:
self.page_title = "Download complete."
self._set_page_content(f"# Download complete\n\nSaved file to '{download_path}'")
except FileNotFoundError:
self.page_title = "Error 404"
self._set_page_content(f"## Error 404\n\nFile not found: {download_path}")
except requests.exceptions.RequestException:
if response is None:
self.page_title = "Request Exception"
self._set_page_content("## Unhandled Request Exception:\n\n" + traceback.format_exc())
else:
self.page_title = f"Error {response.status_code}"
# If the error was rendered in HTML we might as well render it
content_type = response.headers.get("content-type", "")
if "text/html" in content_type.lower():
res = self._markdown_converter.convert(response)
self.page_title = f"Error {response.status_code}"
self._set_page_content(f"## Error {response.status_code}\n\n{res.text_content}")
else:
text = ""
for chunk in response.iter_content(chunk_size=512, decode_unicode=True):
text += chunk
self.page_title = f"Error {response.status_code}"
self._set_page_content(f"## Error {response.status_code}\n\n{text}")
def _fetch_local_dir(self, local_path: str) -> str:
"""Render a local directory listing in HTML to assist with local file browsing via the "file://" protocol.
Through rendered in HTML, later parts of the pipeline will convert the listing to Markdown.
Arguments:
local_path: A path to the local directory whose contents are to be listed.
Returns:
A directory listing, rendered in HTML.
"""
pardir = os.path.normpath(os.path.join(local_path, os.pardir))
pardir_uri = pathlib.Path(pardir).as_uri()
listing = f"""
<!DOCTYPE html>
<html>
<head>
<title>Index of {html.escape(local_path)}</title>
</head>
<body>
<h1>Index of {html.escape(local_path)}</h1>
<a href="{html.escape(pardir_uri, quote=True)}">.. (parent directory)</a>
<table>
<tr>
<th>Name</th><th>Size</th><th>Date modified</th>
</tr>
"""
for entry in os.listdir(local_path):
full_path = os.path.normpath(os.path.join(local_path, entry))
full_path_uri = pathlib.Path(full_path).as_uri()
size = ""
mtime = datetime.datetime.fromtimestamp(os.path.getmtime(full_path)).strftime("%Y-%m-%d %H:%M")
if os.path.isdir(full_path):
entry = entry + os.path.sep
else:
size = str(os.path.getsize(full_path))
listing += (
"<tr>\n"
+ f'<td><a href="{html.escape(full_path_uri, quote=True)}">{html.escape(entry)}</a></td>'
+ f"<td>{html.escape(size)}</td>"
+ f"<td>{html.escape(mtime)}</td>"
+ "</tr>"
)
listing += """
</table>
</body>
</html>
"""
return listing
autoagent/environment/mdconvert.py
+1235
View File
File diff suppressed because it is too large Load Diff
autoagent/environment/shutdown_listener.py
+65
View File
@@ -0,0 +1,65 @@
"""
This module monitors the app for shutdown signals
"""
import asyncio
import signal
import threading
import time
from types import FrameType
from uvicorn.server import HANDLED_SIGNALS
_should_exit = None
def _register_signal_handler(sig: signal.Signals):
original_handler = None
def handler(sig_: int, frame: FrameType | None):
global _should_exit
_should_exit = True
if original_handler:
original_handler(sig_, frame) # type: ignore[unreachable]
original_handler = signal.signal(sig, handler)
def _register_signal_handlers():
global _should_exit
if _should_exit is not None:
return
_should_exit = False
# Check if we're in the main thread of the main interpreter
if threading.current_thread() is threading.main_thread():
for sig in HANDLED_SIGNALS:
_register_signal_handler(sig)
def should_exit() -> bool:
_register_signal_handlers()
return bool(_should_exit)
def should_continue() -> bool:
_register_signal_handlers()
return not _should_exit
def sleep_if_should_continue(timeout: float):
if timeout <= 1:
time.sleep(timeout)
return
start_time = time.time()
while (time.time() - start_time) < timeout and should_continue():
time.sleep(1)
async def async_sleep_if_should_continue(timeout: float):
if timeout <= 1:
await asyncio.sleep(timeout)
return
start_time = time.time()
while time.time() - start_time < timeout and should_continue():
await asyncio.sleep(1)
autoagent/environment/tcp_server.py
+80
View File
@@ -0,0 +1,80 @@
import socket
import subprocess
import json
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--workplace", type=str, default=None)
parser.add_argument("--conda_path", type=str, default=None)
parser.add_argument("--port", type=int, default=None)
args = parser.parse_args()
if __name__ == "__main__":
assert args.workplace is not None, "Workplace is not specified"
assert args.conda_path is not None, "Conda path is not specified"
assert args.port is not None, "Port is not specified"
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("0.0.0.0", args.port))
server.listen(1)
print(f"Listening on port {args.port}...")
def receive_all(conn, buffer_size=4096):
data = b""
while True:
part = conn.recv(buffer_size)
data += part
if len(part) < buffer_size:
# 如果接收的数据小于缓冲区大小，可能已经接收完毕
break
return data.decode()
while True:
conn, addr = server.accept()
print(f"Connection from {addr}")
while True:
# command = conn.recv(1024).decode()
command = receive_all(conn)
if not command:
break
# Execute the command
try:
modified_command = f"/bin/bash -c 'source {args.conda_path}/etc/profile.d/conda.sh && conda activate autogpt && cd /{args.workplace} && {command}'"
process = subprocess.Popen(modified_command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)
output = ''
while True:
line = process.stdout.readline()
if not line and process.poll() is not None:
break
output += line
# 立即发送每一行输出
chunk_response = {
"type": "chunk",
"data": line
}
conn.send(json.dumps(chunk_response).encode() + b"\n") # 添加换行符作为分隔符
# 发送最终的完整响应
final_response = {
"type": "final",
"status": process.poll(),
"result": output
}
conn.send(json.dumps(final_response).encode() + b"\n")
except Exception as e:
error_response = {
"type": "final",
"status": -1,
"result": f"Error running command: {str(e)}"
}
conn.send(json.dumps(error_response).encode() + b"\n")
# Create a JSON response
# response = {
# "status": exit_code,
# "result": output
# }
# # Send the JSON response
# conn.send(json.dumps(response).encode())
conn.close()
autoagent/environment/tenacity_stop.py
+11
View File
@@ -0,0 +1,11 @@
from tenacity import RetryCallState
from tenacity.stop import stop_base
from .shutdown_listener import should_exit
class stop_if_should_exit(stop_base):
"""Stop if the should_exit flag is set."""
def __call__(self, retry_state: 'RetryCallState') -> bool:
return should_exit()
autoagent/environment/utils.py
+16
View File
@@ -0,0 +1,16 @@
from autoagent.util import run_command_in_container
from .docker_env import DockerEnv
from autoagent.io_utils import print_stream
def setup_metachain(workplace_name: str, env: DockerEnv):
cmd = "pip list | grep autoagent"
response = env.run_command(cmd, print_stream)
if response['status'] == 0:
print("AutoAgent is already installed.")
return
cmd = f"cd /{workplace_name}/AutoAgent && pip install -e ."
response = env.run_command(cmd, print_stream)
if response['status'] == 0:
print("AutoAgent is installed.")
return
else:
raise Exception(f"Failed to install autoagent. {response['result']}")
autoagent/flow/__init__.py
+4
View File
@@ -0,0 +1,4 @@
from .core import EventEngineCls
from .types import EventInput, ReturnBehavior
default_drive = EventEngineCls()
autoagent/flow/broker.py
+11
View File
@@ -0,0 +1,11 @@
from typing import Any
from .types import BaseEvent, EventInput, Task, GroupEventReturns
from .utils import generate_uuid
class BaseBroker:
async def append(self, event: BaseEvent, event_input: EventInput) -> Task:
raise NotImplementedError()
async def callback_after_run_done(self) -> tuple[BaseEvent, Any]:
raise NotImplementedError()
autoagent/flow/core.py
+175
View File
@@ -0,0 +1,175 @@
import inspect
import asyncio
from typing import Callable, Optional, Union, Any, Tuple, Literal
from .types import (
BaseEvent,
EventFunction,
EventGroup,
EventInput,
_SpecialEventReturn,
ReturnBehavior,
InvokeInterCache,
)
from .broker import BaseBroker
from .utils import logger, string_to_md5_hash, generate_uuid
class EventEngineCls:
def __init__(self, name="default", broker: Optional[BaseBroker] = None):
self.name = name
self.broker = broker or BaseBroker()
self.__event_maps: dict[str, BaseEvent] = {}
self.__max_group_size = 0
def reset(self):
self.__event_maps = {}
def get_event_from_id(self, event_id: str) -> Optional[BaseEvent]:
return self.__event_maps.get(event_id)
def make_event(self, func: Union[EventFunction, BaseEvent]) -> BaseEvent:
if isinstance(func, BaseEvent):
self.__event_maps[func.id] = func
return func
assert inspect.iscoroutinefunction(
func
), "Event function must be a coroutine function"
event = BaseEvent(func)
self.__event_maps[event.id] = event
return event
def listen_group(
self,
group_markers: list[BaseEvent],
group_name: Optional[str] = None,
retrigger_type: Literal["all", "any"] = "all",
) -> Callable[[BaseEvent], BaseEvent]:
assert all(
[isinstance(m, BaseEvent) for m in group_markers]
), "group_markers must be a list of BaseEvent"
assert all(
[m.id in self.__event_maps for m in group_markers]
), f"group_markers must be registered in the same event engine, current event engine is {self.name}"
group_markers_in_dict = {event.id: event for event in group_markers}
def decorator(func: BaseEvent) -> BaseEvent:
if not isinstance(func, BaseEvent):
func = self.make_event(func)
assert (
func.id in self.__event_maps
), f"Event function must be registered in the same event engine, current event engine is {self.name}"
this_group_name = group_name or f"{len(func.parent_groups)}"
this_group_hash = string_to_md5_hash(":".join(group_markers_in_dict.keys()))
new_group = EventGroup(
this_group_name,
this_group_hash,
group_markers_in_dict,
retrigger_type=retrigger_type,
)
self.__max_group_size = max(
self.__max_group_size, len(group_markers_in_dict)
)
if new_group.hash() in func.parent_groups:
logger.warning(f"Group {group_markers} already listened by {func}")
return func
func.parent_groups[new_group.hash()] = new_group
return func
return decorator
def goto(self, group_markers: list[BaseEvent], *args):
raise NotImplementedError()
async def invoke_event(
self,
event: BaseEvent,
event_input: Optional[EventInput] = None,
global_ctx: Any = None,
max_async_events: Optional[int] = None,
) -> dict[str, Any]:
this_run_ctx: dict[str, InvokeInterCache] = {}
queue: list[Tuple[str, EventInput]] = [(event.id, event_input)]
async def run_event(current_event_id: str, current_event_input: Any):
current_event = self.get_event_from_id(current_event_id)
assert current_event is not None, f"Event {current_event_id} not found"
result = await current_event.solo_run(current_event_input, global_ctx)
this_run_ctx[current_event.id] = {
"result": result,
"already_sent_to_event_group": set(),
}
if isinstance(result, _SpecialEventReturn):
if result.behavior == ReturnBehavior.GOTO:
group_markers, any_return = result.returns
for group_marker in group_markers:
this_group_returns = {current_event.id: any_return}
build_input_goto = EventInput(
group_name="$goto",
results=this_group_returns,
behavior=ReturnBehavior.GOTO,
)
queue.append((group_marker.id, build_input_goto))
elif result.behavior == ReturnBehavior.ABORT:
return
else:
# dispath to events who listen
for cand_event in self.__event_maps.values():
cand_event_parents = cand_event.parent_groups
for group_hash, group in cand_event_parents.items():
if_current_event_trigger = current_event.id in group.events
if_ctx_cover = all(
[event_id in this_run_ctx for event_id in group.events]
)
event_group_id = f"{cand_event.id}:{group_hash}"
if if_current_event_trigger and if_ctx_cover:
if (
any(
[
event_group_id
in this_run_ctx[event_id][
"already_sent_to_event_group"
]
for event_id in group.events
]
)
and group.retrigger_type == "all"
):
# some events already dispatched to this event and group, skip
logger.debug(f"Skip {cand_event} for {current_event}")
continue
this_group_returns = {
event_id: this_run_ctx[event_id]["result"]
for event_id in group.events
}
for event_id in group.events:
this_run_ctx[event_id][
"already_sent_to_event_group"
].add(event_group_id)
build_input = EventInput(
group_name=group.name, results=this_group_returns
)
queue.append((cand_event.id, build_input))
tasks = set()
try:
while len(queue) or len(tasks):
this_batch_events = (
queue[:max_async_events] if max_async_events else queue
)
queue = queue[max_async_events:] if max_async_events else []
new_tasks = {
asyncio.create_task(run_event(*run_event_input))
for run_event_input in this_batch_events
}
tasks.update(new_tasks)
done, tasks = await asyncio.wait(
tasks, return_when=asyncio.FIRST_COMPLETED
)
for task in done:
await task # Handle any exceptions
except asyncio.CancelledError:
for task in tasks:
task.cancel()
await asyncio.gather(*tasks, return_exceptions=True)
raise
return this_run_ctx
autoagent/flow/dynamic.py
+18
View File
@@ -0,0 +1,18 @@
from typing import Any
from .types import (
BaseEvent,
_SpecialEventReturn,
ReturnBehavior,
)
def goto_events(
group_markers: list[BaseEvent], any_return: Any = None
) -> _SpecialEventReturn:
return _SpecialEventReturn(
behavior=ReturnBehavior.GOTO, returns=(group_markers, any_return)
)
def abort_this():
return _SpecialEventReturn(behavior=ReturnBehavior.ABORT, returns=None)
autoagent/flow/types.py
+146
View File
@@ -0,0 +1,146 @@
from copy import copy
from enum import Enum
from dataclasses import dataclass, field
from datetime import datetime
from typing import Any, Awaitable, Optional, Union, Callable, TypedDict, Literal
from .utils import (
string_to_md5_hash,
generate_uuid,
function_or_method_to_string,
function_or_method_to_repr,
)
class ReturnBehavior(Enum):
DISPATCH = "dispatch"
GOTO = "goto"
ABORT = "abort"
INPUT = "input"
class TaskStatus(Enum):
RUNNING = "running"
SUCCESS = "success"
FAILURE = "failure"
PENDING = "pending"
class InvokeInterCache(TypedDict):
result: Any
already_sent_to_event_group: set[str]
GroupEventReturns = dict[str, Any]
@dataclass
class EventGroupInput:
group_name: str
results: GroupEventReturns
behavior: ReturnBehavior = ReturnBehavior.DISPATCH
@dataclass
class EventInput(EventGroupInput):
task_id: str = field(default_factory=generate_uuid)
@classmethod
def from_input(cls: "EventInput", input_data: dict[str, Any]) -> "EventInput":
return cls(
group_name="user_input", results=input_data, behavior=ReturnBehavior.INPUT
)
@dataclass
class _SpecialEventReturn:
behavior: ReturnBehavior
returns: Any
def __post_init__(self):
if not isinstance(self.behavior, ReturnBehavior):
raise TypeError(
f"behavior must be a ReturnBehavior, not {type(self.behavior)}"
)
# (group_event_results, global ctx set by user) -> result
EventFunction = Callable[
[Optional[EventInput], Optional[Any]], Awaitable[Union[Any, _SpecialEventReturn]]
]
@dataclass
class EventGroup:
name: str
events_hash: str
events: dict[str, "BaseEvent"]
retrigger_type: Literal["all", "any"] = "all"
def hash(self) -> str:
return self.events_hash
class BaseEvent:
parent_groups: dict[str, EventGroup]
func_inst: EventFunction
id: str
repr_name: str
def __init__(
self,
func_inst: EventFunction,
parent_groups: Optional[dict[str, EventGroup]] = None,
):
self.parent_groups = parent_groups or {}
self.func_inst = func_inst
self.id = string_to_md5_hash(function_or_method_to_string(self.func_inst))
self.repr_name = function_or_method_to_repr(self.func_inst)
self.meta = {"func_body": function_or_method_to_string(self.func_inst)}
def debug_string(self, exclude_events: Optional[set[str]] = None) -> str:
exclude_events = exclude_events or set([self.id])
parents_str = format_parents(self.parent_groups, exclude_events=exclude_events)
return f"{self.repr_name}\n{parents_str}"
def __repr__(self) -> str:
return f"Node(source={self.repr_name})"
async def solo_run(
self, event_input: EventInput, global_ctx: Any = None
) -> Awaitable[Any]:
return await self.func_inst(event_input, global_ctx)
@dataclass
class Task:
task_id: str
status: TaskStatus = TaskStatus.PENDING
created_at: datetime = field(default_factory=datetime.now)
upated_at: datetime = field(default_factory=datetime.now)
def format_parents(parents: dict[str, EventGroup], exclude_events: set[str], indent=""):
# Below code is ugly
# But it works and only for debug display
result = []
for i, parent_group in enumerate(parents.values()):
is_last_group = i == len(parents) - 1
group_prefix = "└─ " if is_last_group else "├─ "
result.append(indent + group_prefix + f"<{parent_group.name}>")
for j, parent in enumerate(parent_group.events.values()):
root_events = copy(exclude_events)
is_last = j == len(parent_group.events) - 1
child_indent = indent + (" " if is_last_group else "│ ")
inter_indent = " " if is_last else "│ "
prefix = "└─ " if is_last else "├─ "
if parent.id in root_events:
result.append(f"{child_indent}{prefix}{parent.repr_name} <loop>")
continue
root_events.add(parent.id)
parent_debug = parent.debug_string(exclude_events=root_events).split("\n")
parent_debug = [p for p in parent_debug if p.strip()]
result.append(f"{child_indent}{prefix}{parent.repr_name}")
for line in parent_debug[1:]:
result.append(f"{child_indent}{inter_indent}{line}")
return "\n".join(result)
autoagent/flow/utils.py
+48
View File
@@ -0,0 +1,48 @@
import uuid
import logging
import asyncio
import inspect
import hashlib
from typing import Callable
logger = logging.getLogger("drive-flow")
def generate_uuid() -> str:
return str(uuid.uuid4())
def function_or_method_to_repr(func_or_method: Callable) -> str:
is_method = inspect.ismethod(func_or_method)
is_function = inspect.isfunction(func_or_method)
if not is_method and not is_function:
raise ValueError("Input must be a function or method")
module = func_or_method.__module__
name = func_or_method.__name__
line_number = inspect.getsourcelines(func_or_method)[1]
if is_method:
class_name = func_or_method.__self__.__class__.__name__
return f"{module}.l_{line_number}.{class_name}.{name}".strip()
else:
return f"{module}.l_{line_number}.{name}".strip()
def function_or_method_to_string(func_or_method: Callable) -> str:
is_method = inspect.ismethod(func_or_method)
is_function = inspect.isfunction(func_or_method)
if not is_method and not is_function:
raise ValueError("Input must be a function or method")
module = func_or_method.__module__
source = inspect.getsource(func_or_method)
line_number = inspect.getsourcelines(func_or_method)[1]
if is_method:
class_name = func_or_method.__self__.__class__.__name__
return f"{module}.l_{line_number}.{class_name}\n{source}".strip()
else:
return f"{module}.l_{line_number}\n{source}".strip()
def string_to_md5_hash(string: str) -> str:
return hashlib.md5(string.encode()).hexdigest()
autoagent/fn_call_converter.py
+848
View File
@@ -0,0 +1,848 @@
"""Convert function calling messages to non-function calling messages and vice versa.
This will inject prompts so that models that doesn't support function calling
can still be used with function calling agents.
We follow format from: https://docs.litellm.ai/docs/completion/function_call
"""
import copy
import json
import re
from typing import Iterable
from litellm import ChatCompletionToolParam
class FunctionCallConversionError(Exception):
"""Exception raised when FunctionCallingConverter failed to convert a non-function call message to a function call message.
This typically happens when there's a malformed message (e.g., missing <function=...> tags). But not due to LLM output.
"""
def __init__(self, message):
super().__init__(message)
class FunctionCallValidationError(Exception):
"""Exception raised when FunctionCallingConverter failed to validate a function call message.
This typically happens when the LLM outputs unrecognized function call / parameter names / values.
"""
def __init__(self, message):
super().__init__(message)
# Inspired by: https://docs.together.ai/docs/llama-3-function-calling#function-calling-w-llama-31-70b
SYSTEM_PROMPT_SUFFIX_TEMPLATE = """
You have access to the following functions:
{description}
If you choose to call a function ONLY reply in the following format with NO suffix:
<function=example_function_name>
<parameter=example_parameter_1>value_1</parameter>
<parameter=example_parameter_2>
This is the value for the second parameter
that can span
multiple lines
</parameter>
</function>
<IMPORTANT>
Reminder:
- Function calls MUST follow the specified format, start with <function= and end with </function>
- Required parameters MUST be specified
- Only call one function at a time
- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after.
- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls
"""
STOP_WORDS = ['</function']
# NOTE: we need to make sure this example is always in-sync with the tool interface designed in openhands/agenthub/codeact_agent/function_calling.py
IN_CONTEXT_LEARNING_EXAMPLE_PREFIX = """
Here's a running example of how to perform a task with the provided tools.
--------------------- START OF EXAMPLE ---------------------
USER: Create a list of numbers from 1 to 10, and display them in a web page at port 5000.
ASSISTANT: Sure! Let me first check the current directory:
<function=execute_bash>
<parameter=command>
pwd && ls
</parameter>
</function>
USER: EXECUTION RESULT of [execute_bash]:
/workspace
openhands@runtime:~/workspace$
ASSISTANT: There is no `app.py` file in the current directory. Let me create a Python file `app.py`:
<function=str_replace_editor>
<parameter=command>create</parameter>
<parameter=path>/workspace/app.py</parameter>
<parameter=file_text>
from flask import Flask
app = Flask(__name__)
@app.route('/')
def index():
numbers = list(range(1, 11))
return str(numbers)
if __name__ == '__main__':
app.run(port=5000)
</parameter>
</function>
USER: EXECUTION RESULT of [str_replace_editor]:
File created successfully at: /workspace/app.py
ASSISTANT: I have created a Python file `app.py` that will display a list of numbers from 1 to 10 when you run it. Let me run the Python file for you:
<function=execute_bash>
<parameter=command>
python3 app.py > server.log 2>&1 &
</parameter>
</function>
USER: EXECUTION RESULT of [execute_bash]:
[1] 121
[1]+ Exit 1 python3 app.py > server.log 2>&1
ASSISTANT: Looks like the server is running with PID 121 then crashed. Let me check the server log:
<function=execute_bash>
<parameter=command>
cat server.log
</parameter>
</function>
USER: EXECUTION RESULT of [execute_bash]:
Traceback (most recent call last):
File "/workspace/app.py", line 2, in <module>
from flask import Flask
ModuleNotFoundError: No module named 'flask'
ASSISTANT: Looks like the server crashed because the `flask` module is not installed. Let me install the `flask` module for you:
<function=execute_bash>
<parameter=command>
pip3 install flask
</parameter>
</function>
USER: EXECUTION RESULT of [execute_bash]:
Defaulting to user installation because normal site-packages is not writeable
Collecting flask
Using cached flask-3.0.3-py3-none-any.whl (101 kB)
Collecting blinker>=1.6.2
Using cached blinker-1.7.0-py3-none-any.whl (13 kB)
Collecting Werkzeug>=3.0.0
Using cached werkzeug-3.0.2-py3-none-any.whl (226 kB)
Collecting click>=8.1.3
Using cached click-8.1.7-py3-none-any.whl (97 kB)
Collecting itsdangerous>=2.1.2
Using cached itsdangerous-2.2.0-py3-none-any.whl (16 kB)
Requirement already satisfied: Jinja2>=3.1.2 in /home/openhands/.local/lib/python3.10/site-packages (from flask) (3.1.3)
Requirement already satisfied: MarkupSafe>=2.0 in /home/openhands/.local/lib/python3.10/site-packages (from Jinja2>=3.1.2->flask) (2.1.5)
Installing collected packages: Werkzeug, itsdangerous, click, blinker, flask
Successfully installed Werkzeug-3.0.2 blinker-1.7.0 click-8.1.7 flask-3.0.3 itsdangerous-2.2.0
ASSISTANT: The `flask` module is installed successfully. Let me run the Python file again:
<function=execute_bash>
<parameter=command>
python3 app.py > server.log 2>&1 &
</parameter>
</function>
USER: EXECUTION RESULT of [execute_bash]:
[1] 124
ASSISTANT:
Let me check the server log again:
<function=execute_bash>
<parameter=command>
cat server.log
</parameter>
</function>
USER: EXECUTION RESULT of [execute_bash]:
* Serving Flask app 'app'
* Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
* Running on http://127.0.0.1:5000
Press CTRL+C to quit
ASSISTANT:
The server is running on port 5000 with PID 124. You can access the list of numbers by visiting http://127.0.0.1:5000. If you have any further questions, feel free to ask!
USER: Now kill the server, make it display the numbers in a table format.
ASSISTANT:
Sure! Let me stop the server first:
<function=execute_bash>
<parameter=command>
kill 124
</parameter>
</function>
USER: EXECUTION RESULT of [execute_bash]:
[1]+ Terminated python3 app.py > server.log 2>&1
ASSISTANT:
Now let me display the numbers in a table format:
<function=str_replace_editor>
<parameter=command>str_replace</parameter>
<parameter=path>/workspace/app.py</parameter>
<parameter=old_str>return str(numbers)</parameter>
<parameter=new_str>return '<table>' + ''.join([f'<tr><td>{i}</td></tr>' for i in numbers]) + '</table>'</parameter>
</function>
USER: EXECUTION RESULT of [str_replace_editor]:
The file /workspace/app.py has been edited. Here's the result of running `cat -n` on a snippet of /workspace/app.py:
3
4 @app.route('/')
5 def index():
6 numbers = list(range(1, 11))
7 return '<table>' + ''.join([f'<tr><td>{i}</td></tr>' for i in numbers]) + '</table>'
8
9 if __name__ == '__main__':
10 app.run(port=5000)
Review the changes and make sure they are as expected. Edit the file again if necessary.
ASSISTANT:
Running the updated file:
<function=execute_bash>
<parameter=command>
<parameter=command>
python3 app.py > server.log 2>&1 &
</parameter>
</function>
USER: EXECUTION RESULT of [execute_bash]:
[1] 126
ASSISTANT:
The server is running on port 5000 with PID 126. You can access the list of numbers in a table format by visiting http://127.0.0.1:5000. Let me know if you have any further requests!
<function=finish>
</function>
--------------------- END OF EXAMPLE ---------------------
Do NOT assume the environment is the same as in the example above.
--------------------- NEW TASK DESCRIPTION ---------------------
""".lstrip()
IN_CONTEXT_LEARNING_EXAMPLE_SUFFIX = """
--------------------- END OF NEW TASK DESCRIPTION ---------------------
PLEASE follow the format strictly! PLEASE EMIT ONE AND ONLY ONE FUNCTION CALL PER MESSAGE.
""".lstrip()
# Regex patterns for function call parsing
FN_REGEX_PATTERN = r'<function=([^>]+)>\n(.*?)</function>'
FN_PARAM_REGEX_PATTERN = r'<parameter=([^>]+)>(.*?)</parameter>'
# Add new regex pattern for tool execution results
TOOL_RESULT_REGEX_PATTERN = r'EXECUTION RESULT of \[(.*?)\]:\n(.*)'
def convert_tool_call_to_string(tool_call: dict) -> str:
"""Convert tool call to content in string format."""
if 'function' not in tool_call:
raise FunctionCallConversionError("Tool call must contain 'function' key.")
if 'id' not in tool_call:
raise FunctionCallConversionError("Tool call must contain 'id' key.")
if 'type' not in tool_call:
raise FunctionCallConversionError("Tool call must contain 'type' key.")
if tool_call['type'] != 'function':
raise FunctionCallConversionError("Tool call type must be 'function'.")
ret = f"<function={tool_call['function']['name']}>\n"
try:
args = json.loads(tool_call['function']['arguments'])
except json.JSONDecodeError as e:
raise FunctionCallConversionError(
f"Failed to parse arguments as JSON. Arguments: {tool_call['function']['arguments']}"
) from e
for param_name, param_value in args.items():
is_multiline = isinstance(param_value, str) and '\n' in param_value
ret += f'<parameter={param_name}>'
if is_multiline:
ret += '\n'
ret += f'{param_value}'
if is_multiline:
ret += '\n'
ret += '</parameter>\n'
ret += '</function>'
return ret
def convert_tools_to_description(tools: list[dict]) -> str:
ret = ''
for i, tool in enumerate(tools):
assert tool['type'] == 'function'
fn = tool['function']
if i > 0:
ret += '\n'
ret += f"---- BEGIN FUNCTION #{i+1}: {fn['name']} ----\n"
ret += f"Description: {fn['description']}\n"
if 'parameters' in fn:
ret += 'Parameters:\n'
properties = fn['parameters'].get('properties', {})
required_params = set(fn['parameters'].get('required', []))
for j, (param_name, param_info) in enumerate(properties.items()):
# Indicate required/optional in parentheses with type
is_required = param_name in required_params
param_status = 'required' if is_required else 'optional'
param_type = param_info.get('type', 'string')
# Get parameter description
desc = param_info.get('description', 'No description provided')
# Handle enum values if present
if 'enum' in param_info:
enum_values = ', '.join(f'`{v}`' for v in param_info['enum'])
desc += f'\nAllowed values: [{enum_values}]'
ret += (
f' ({j+1}) {param_name} ({param_type}, {param_status}): {desc}\n'
)
else:
ret += 'No parameters are required for this function.\n'
ret += f'---- END FUNCTION #{i+1} ----\n'
return ret
def convert_fncall_messages_to_non_fncall_messages(
messages: list[dict],
tools: list[ChatCompletionToolParam],
add_in_context_learning_example: bool = True,
) -> list[dict]:
"""Convert function calling messages to non-function calling messages."""
messages = copy.deepcopy(messages)
formatted_tools = convert_tools_to_description(tools)
system_prompt_suffix = SYSTEM_PROMPT_SUFFIX_TEMPLATE.format(
description=formatted_tools
)
converted_messages = []
first_user_message_encountered = False
for message in messages:
role = message['role']
content = message['content']
# 1. SYSTEM MESSAGES
# append system prompt suffix to content
if role == 'system':
if isinstance(content, str):
content += system_prompt_suffix
elif isinstance(content, list):
if content and content[-1]['type'] == 'text':
content[-1]['text'] += system_prompt_suffix
else:
content.append({'type': 'text', 'text': system_prompt_suffix})
else:
raise FunctionCallConversionError(
f'Unexpected content type {type(content)}. Expected str or list. Content: {content}'
)
converted_messages.append({'role': 'system', 'content': content})
# 2. USER MESSAGES (no change)
elif role == 'user':
# Add in-context learning example for the first user message
if not first_user_message_encountered and add_in_context_learning_example:
first_user_message_encountered = True
# Check tools
if not (
tools
and len(tools) > 0
and any(
(
tool['type'] == 'function'
and tool['function']['name'] == 'execute_bash'
and 'command'
in tool['function']['parameters']['properties']
)
for tool in tools
)
and any(
(
tool['type'] == 'function'
and tool['function']['name'] == 'str_replace_editor'
and 'path' in tool['function']['parameters']['properties']
and 'file_text'
in tool['function']['parameters']['properties']
and 'old_str'
in tool['function']['parameters']['properties']
and 'new_str'
in tool['function']['parameters']['properties']
)
for tool in tools
)
):
raise FunctionCallConversionError(
'The currently provided tool set are NOT compatible with the in-context learning example for FnCall to Non-FnCall conversion. '
'Please update your tool set OR the in-context learning example in openhands/llm/fn_call_converter.py'
)
# add in-context learning example
if isinstance(content, str):
content = (
IN_CONTEXT_LEARNING_EXAMPLE_PREFIX
+ content
+ IN_CONTEXT_LEARNING_EXAMPLE_SUFFIX
)
elif isinstance(content, list):
if content and content[0]['type'] == 'text':
content[0]['text'] = (
IN_CONTEXT_LEARNING_EXAMPLE_PREFIX
+ content[0]['text']
+ IN_CONTEXT_LEARNING_EXAMPLE_SUFFIX
)
else:
content = (
[
{
'type': 'text',
'text': IN_CONTEXT_LEARNING_EXAMPLE_PREFIX,
}
]
+ content
+ [
{
'type': 'text',
'text': IN_CONTEXT_LEARNING_EXAMPLE_SUFFIX,
}
]
)
else:
raise FunctionCallConversionError(
f'Unexpected content type {type(content)}. Expected str or list. Content: {content}'
)
converted_messages.append(
{
'role': 'user',
'content': content,
}
)
# 3. ASSISTANT MESSAGES
# - 3.1 no change if no function call
# - 3.2 change if function call
elif role == 'assistant':
if 'tool_calls' in message and message['tool_calls'] is not None:
if len(message['tool_calls']) != 1:
raise FunctionCallConversionError(
f'Expected exactly one tool call in the message. More than one tool call is not supported. But got {len(message["tool_calls"])} tool calls. Content: {content}'
)
try:
tool_content = convert_tool_call_to_string(message['tool_calls'][0])
except FunctionCallConversionError as e:
raise FunctionCallConversionError(
f'Failed to convert tool call to string.\nCurrent tool call: {message["tool_calls"][0]}.\nRaw messages: {json.dumps(messages, indent=2)}'
) from e
if isinstance(content, str):
content += '\n\n' + tool_content
content = content.lstrip()
elif isinstance(content, list):
if content and content[-1]['type'] == 'text':
content[-1]['text'] += '\n\n' + tool_content
content[-1]['text'] = content[-1]['text'].lstrip()
else:
content.append({'type': 'text', 'text': tool_content})
else:
raise FunctionCallConversionError(
f'Unexpected content type {type(content)}. Expected str or list. Content: {content}'
)
converted_messages.append({'role': 'assistant', 'content': content})
# 4. TOOL MESSAGES (tool outputs)
elif role == 'tool':
# Convert tool result as user message
tool_name = message.get('name', 'function')
prefix = f'EXECUTION RESULT of [{tool_name}]:\n'
# and omit "tool_call_id" AND "name"
if isinstance(content, str):
content = prefix + content
elif isinstance(content, list):
if content and content[-1]['type'] == 'text':
content[-1]['text'] = prefix + content[-1]['text']
else:
content = [{'type': 'text', 'text': prefix}] + content
else:
raise FunctionCallConversionError(
f'Unexpected content type {type(content)}. Expected str or list. Content: {content}'
)
converted_messages.append({'role': 'user', 'content': content})
else:
raise FunctionCallConversionError(
f'Unexpected role {role}. Expected system, user, assistant or tool.'
)
return converted_messages
def _extract_and_validate_params(
matching_tool: dict, param_matches: Iterable[re.Match], fn_name: str
) -> dict:
params = {}
# Parse and validate parameters
required_params = set()
if 'parameters' in matching_tool and 'required' in matching_tool['parameters']:
required_params = set(matching_tool['parameters'].get('required', []))
allowed_params = set()
if 'parameters' in matching_tool and 'properties' in matching_tool['parameters']:
allowed_params = set(matching_tool['parameters']['properties'].keys())
param_name_to_type = {}
if 'parameters' in matching_tool and 'properties' in matching_tool['parameters']:
param_name_to_type = {
name: val.get('type', 'string')
for name, val in matching_tool['parameters']['properties'].items()
}
# Collect parameters
found_params = set()
for param_match in param_matches:
param_name = param_match.group(1)
param_value = param_match.group(2).strip()
# Validate parameter is allowed
if allowed_params and param_name not in allowed_params:
raise FunctionCallValidationError(
f"Parameter '{param_name}' is not allowed for function '{fn_name}'. "
f'Allowed parameters: {allowed_params}'
)
# Validate and convert parameter type
# supported: string, integer, array
if param_name in param_name_to_type:
if param_name_to_type[param_name] == 'integer':
try:
param_value = int(param_value)
except ValueError:
raise FunctionCallValidationError(
f"Parameter '{param_name}' is expected to be an integer."
)
elif param_name_to_type[param_name] == 'array':
try:
param_value = json.loads(param_value)
except json.JSONDecodeError:
raise FunctionCallValidationError(
f"Parameter '{param_name}' is expected to be an array."
)
else:
# string
pass
# Enum check
if 'enum' in matching_tool['parameters']['properties'][param_name]:
if (
param_value
not in matching_tool['parameters']['properties'][param_name]['enum']
):
raise FunctionCallValidationError(
f"Parameter '{param_name}' is expected to be one of {matching_tool['parameters']['properties'][param_name]['enum']}."
)
params[param_name] = param_value
found_params.add(param_name)
# Check all required parameters are present
missing_params = required_params - found_params
if missing_params:
raise FunctionCallValidationError(
f"Missing required parameters for function '{fn_name}': {missing_params}"
)
return params
def _fix_stopword(content: str) -> str:
"""Fix the issue when some LLM would NOT return the stopword."""
if '<function=' in content and content.count('<function=') == 1:
if content.endswith('</'):
content = content.rstrip() + 'function>'
else:
content = content + '\n</function>'
return content
def convert_non_fncall_messages_to_fncall_messages(
messages: list[dict],
tools: list[ChatCompletionToolParam],
) -> list[dict]:
"""Convert non-function calling messages back to function calling messages."""
messages = copy.deepcopy(messages)
formatted_tools = convert_tools_to_description(tools)
system_prompt_suffix = SYSTEM_PROMPT_SUFFIX_TEMPLATE.format(
description=formatted_tools
)
converted_messages = []
tool_call_counter = 1 # Counter for tool calls
first_user_message_encountered = False
for message in messages:
role, content = message['role'], message['content']
content = content or '' # handle cases where content is None
# For system messages, remove the added suffix
if role == 'system':
if isinstance(content, str):
# Remove the suffix if present
content = content.split(system_prompt_suffix)[0]
elif isinstance(content, list):
if content and content[-1]['type'] == 'text':
# Remove the suffix from the last text item
content[-1]['text'] = content[-1]['text'].split(
system_prompt_suffix
)[0]
converted_messages.append({'role': 'system', 'content': content})
# Skip user messages (no conversion needed)
elif role == 'user':
# Check & replace in-context learning example
if not first_user_message_encountered:
first_user_message_encountered = True
if isinstance(content, str):
content = content.replace(IN_CONTEXT_LEARNING_EXAMPLE_PREFIX, '')
content = content.replace(IN_CONTEXT_LEARNING_EXAMPLE_SUFFIX, '')
elif isinstance(content, list):
for item in content:
if item['type'] == 'text':
item['text'] = item['text'].replace(
IN_CONTEXT_LEARNING_EXAMPLE_PREFIX, ''
)
item['text'] = item['text'].replace(
IN_CONTEXT_LEARNING_EXAMPLE_SUFFIX, ''
)
else:
raise FunctionCallConversionError(
f'Unexpected content type {type(content)}. Expected str or list. Content: {content}'
)
# Check for tool execution result pattern
if isinstance(content, str):
tool_result_match = re.search(
TOOL_RESULT_REGEX_PATTERN, content, re.DOTALL
)
elif isinstance(content, list):
tool_result_match = next(
(
_match
for item in content
if item.get('type') == 'text'
and (
_match := re.search(
TOOL_RESULT_REGEX_PATTERN, item['text'], re.DOTALL
)
)
),
None,
)
else:
raise FunctionCallConversionError(
f'Unexpected content type {type(content)}. Expected str or list. Content: {content}'
)
if tool_result_match:
if not (
isinstance(content, str)
or (
isinstance(content, list)
and len(content) == 1
and content[0].get('type') == 'text'
)
):
raise FunctionCallConversionError(
f'Expected str or list with one text item when tool result is present in the message. Content: {content}'
)
tool_name = tool_result_match.group(1)
tool_result = tool_result_match.group(2).strip()
# Convert to tool message format
converted_messages.append(
{
'role': 'tool',
'name': tool_name,
'content': [{'type': 'text', 'text': tool_result}]
if isinstance(content, list)
else tool_result,
'tool_call_id': f'toolu_{tool_call_counter-1:02d}', # Use last generated ID
}
)
else:
converted_messages.append({'role': 'user', 'content': content})
# Handle assistant messages
elif role == 'assistant':
if isinstance(content, str):
content = _fix_stopword(content)
fn_match = re.search(FN_REGEX_PATTERN, content, re.DOTALL)
elif isinstance(content, list):
if content and content[-1]['type'] == 'text':
content[-1]['text'] = _fix_stopword(content[-1]['text'])
fn_match = re.search(
FN_REGEX_PATTERN, content[-1]['text'], re.DOTALL
)
else:
fn_match = None
fn_match_exists = any(
item.get('type') == 'text'
and re.search(FN_REGEX_PATTERN, item['text'], re.DOTALL)
for item in content
)
if fn_match_exists and not fn_match:
raise FunctionCallConversionError(
f'Expecting function call in the LAST index of content list. But got content={content}'
)
else:
raise FunctionCallConversionError(
f'Unexpected content type {type(content)}. Expected str or list. Content: {content}'
)
if fn_match:
fn_name = fn_match.group(1)
fn_body = fn_match.group(2)
matching_tool = next(
(
tool['function']
for tool in tools
if tool['type'] == 'function'
and tool['function']['name'] == fn_name
),
None,
)
# Validate function exists in tools
if not matching_tool:
raise FunctionCallValidationError(
f"Function '{fn_name}' not found in available tools: {[tool['function']['name'] for tool in tools if tool['type'] == 'function']}"
)
# Parse parameters
param_matches = re.finditer(FN_PARAM_REGEX_PATTERN, fn_body, re.DOTALL)
params = _extract_and_validate_params(
matching_tool, param_matches, fn_name
)
# Create tool call with unique ID
tool_call_id = f'toolu_{tool_call_counter:02d}'
tool_call = {
'index': 1, # always 1 because we only support **one tool call per message**
'id': tool_call_id,
'type': 'function',
'function': {'name': fn_name, 'arguments': json.dumps(params)},
}
tool_call_counter += 1 # Increment counter
# Remove the function call part from content
if isinstance(content, list):
assert content and content[-1]['type'] == 'text'
content[-1]['text'] = (
content[-1]['text'].split('<function=')[0].strip()
)
elif isinstance(content, str):
content = content.split('<function=')[0].strip()
else:
raise FunctionCallConversionError(
f'Unexpected content type {type(content)}. Expected str or list. Content: {content}'
)
converted_messages.append(
{'role': 'assistant', 'content': content, 'tool_calls': [tool_call]}
)
else:
# No function call, keep message as is
converted_messages.append(message)
else:
raise FunctionCallConversionError(
f'Unexpected role {role}. Expected system, user, or assistant in non-function calling messages.'
)
return converted_messages
def convert_from_multiple_tool_calls_to_single_tool_call_messages(
messages: list[dict],
ignore_final_tool_result: bool = False,
) -> list[dict]:
"""Break one message with multiple tool calls into multiple messages."""
converted_messages = []
pending_tool_calls: dict[str, dict] = {}
for message in messages:
role, content = message['role'], message['content']
if role == 'assistant':
if message.get('tool_calls') and len(message['tool_calls']) > 1:
# handle multiple tool calls by breaking them into multiple messages
for i, tool_call in enumerate(message['tool_calls']):
pending_tool_calls[tool_call['id']] = {
'role': 'assistant',
'content': content if i == 0 else '',
'tool_calls': [tool_call],
}
else:
converted_messages.append(message)
elif role == 'tool':
if message['tool_call_id'] in pending_tool_calls:
# remove the tool call from the pending list
_tool_call_message = pending_tool_calls.pop(message['tool_call_id'])
converted_messages.append(_tool_call_message)
# add the tool result
converted_messages.append(message)
else:
assert (
len(pending_tool_calls) == 0
), f'Found pending tool calls but not found in pending list: {pending_tool_calls=}'
converted_messages.append(message)
else:
assert (
len(pending_tool_calls) == 0
), f'Found pending tool calls but not expect to handle it with role {role}: {pending_tool_calls=}, {message=}'
converted_messages.append(message)
if not ignore_final_tool_result and len(pending_tool_calls) > 0:
raise FunctionCallConversionError(
f'Found pending tool calls but no tool result: {pending_tool_calls=}'
)
return converted_messages
def convert_fn_messages_to_non_fn_messages(messages: list[dict]) -> list[dict]:
"""Convert function calling messages back to non-function calling messages."""
new_messages = []
for idx, message in enumerate(messages):
if message["role"] == "tool":
assert messages[idx-1]["role"] == "assistant" and new_messages[-1]["role"] == "assistant"
new_messages[-1]["content"] = messages[idx-1]["content"] + f"""
I have executed the tool {message["name"]} and the result is {message["content"]}.
"""
elif message["role"] == "assistant":
if message["tool_calls"] is not None:
msg_content = message["content"] + f"""
I want to use the tool named {message["tool_calls"][0]["function"]["name"]}, with the following arguments: {message["tool_calls"][0]["function"]["arguments"]}.
"""
else:
msg_content = message["content"]
new_messages.append({"role": message["role"], "content": msg_content}.copy())
else:
new_messages.append(message.copy())
return new_messages
def interleave_user_into_messages(messages: list[dict]) -> list[dict]:
new_messages = []
for idx, message in enumerate(messages):
if message["role"] == "assistant" and messages[idx-1]["role"] == "assistant":
# 在两个连续的 assistant 消息之间插入一个空的 user 消息
new_messages.append({
"role": "user",
"content": "Please think twice and take the next action according to your previous actions and observations." # 空内容的用户消息
}.copy())
new_messages.append(message.copy())
new_messages.append({"role": "user", "content": "Please think twice and take the next action according to your previous actions and observations."})
return new_messages
autoagent/io_utils.py
+51
View File
@@ -0,0 +1,51 @@
import yaml
import hashlib
import zipfile
import os
import json
from rich.console import Console
def read_file(file_path):
with open(file_path, 'r', encoding='utf-8') as file:
content = file.read()
return content
def read_yaml_file(file_path):
with open(file_path, 'r', encoding='utf-8') as file:
content = yaml.safe_load(file)
return content
def get_file_md5(file_path):
md5_hash = hashlib.md5()
with open(file_path, "rb") as f:
# read file block
for byte_block in iter(lambda: f.read(4096), b""):
md5_hash.update(byte_block)
return md5_hash.hexdigest()
def compress_folder(source_folder, destination_folder, archive_name):
os.makedirs(destination_folder, exist_ok=True)
archive_path = os.path.join(destination_folder, archive_name)
with zipfile.ZipFile(archive_path, 'w', zipfile.ZIP_DEFLATED) as zipf:
for root, _, files in os.walk(source_folder):
for file in files:
file_path = os.path.join(root, file)
arcname = os.path.relpath(file_path, source_folder)
zipf.write(file_path, arcname)
print(f"Folder '{source_folder}' has been compressed to '{archive_path}'")
def get_md5_hash_bytext(text):
return hashlib.md5(text.encode('utf-8')).hexdigest()
def read_json_file(file_path):
with open(file_path, 'r', encoding='utf-8') as file:
content = json.load(file)
return content
def print_stream(text):
console = Console()
console.print(f"[grey42]{text}[/grey42]")
autoagent/logger.py
+181
View File
@@ -0,0 +1,181 @@
from datetime import datetime
from rich.console import Console
from rich.markup import escape
import json
from typing import List
from constant import DEBUG, DEFAULT_LOG, LOG_PATH, MC_MODE
from pathlib import Path
BAR_LENGTH = 60
class MetaChainLogger:
def __init__(self, log_path: str):
self.log_path = log_path
self.console = Console()
self.debug = DEBUG
def _write_log(self, message: str):
with open(self.log_path, 'a') as f:
f.write(message + '\n')
def _warp_args(self, args_dict: str):
args_dict = json.loads(args_dict)
args_str = ''
for k, v in args_dict.items():
args_str += f"{repr(k)}={repr(v)}, "
return args_str[:-2]
def _wrap_title(self, title: str, color: str = None):
single_len = (BAR_LENGTH - len(title)) // 2
color_bos = f"[{color}]" if color else ""
color_eos = f"[/{color}]" if color else ""
return f"{color_bos}{'*'*single_len} {title} {'*'*single_len}{color_eos}"
def info(self, *args: str, **kwargs: dict):
# console = Console()
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
message = "\n".join(map(str, args))
color = kwargs.get("color", "white")
if MC_MODE: color = "grey58"
title = kwargs.get("title", "INFO")
log_str = f"[{timestamp}]\n{message}"
if self.debug:
# print_in_box(log_str, color=color, title=title)
self.console.print(self._wrap_title(title, f"bold {color}"))
print_str = escape(log_str)
if MC_MODE: print_str = f"[grey58]{print_str}[/grey58]"
self.console.print(print_str, highlight=True, emoji=True)
log_str = self._wrap_title(title) + "\n" + log_str
if self.log_path: self._write_log(log_str)
def lprint(self, *args: str, **kwargs: dict):
if not self.debug: return
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
message = "\n".join(map(str, args))
color = kwargs.get("color", "white")
if MC_MODE: color = "grey58"
title = kwargs.get("title", "")
log_str = f"[{timestamp}]\n{message}"
# print_in_box(log_str, color=color, title=title)
self.console.print(self._wrap_title(title, f"bold {color}"))
print_str = escape(log_str)
if MC_MODE: print_str = f"[grey58]{print_str}[/grey58]"
self.console.print(print_str, highlight=True, emoji=True)
def _wrap_timestamp(self, timestamp: str, color: bool = True):
color_bos = "[grey58]" if color else ""
color_eos = "[/grey58]" if color else ""
return f"{color_bos}[{timestamp}]{color_eos}"
def _print_tool_execution(self, message, timestamp: str):
if MC_MODE: colors = ["grey58"] * 3
else: colors = ["pink3", "blue", "purple"]
self.console.print(self._wrap_title("Tool Execution", f"bold {colors[0]}"))
self.console.print(self._wrap_timestamp(timestamp, color=True))
self.console.print(f"[bold {colors[1]}]Tool Execution:[/bold {colors[1]}]", end=" ")
self.console.print(f"[bold {colors[2]}]{message['name']}[/bold {colors[2]}]\n[bold {colors[1]}]Result:[/bold {colors[1]}]")
print_str = f"---\n{escape(message['content'])}\n---"
if MC_MODE: print_str = f"[grey58]{print_str}[/grey58]"
self.console.print(print_str, highlight=True, emoji=True)
def _save_tool_execution(self, message, timestamp: str):
self._write_log(self._wrap_title("Tool Execution"))
self._write_log(f"{self._wrap_timestamp(timestamp, color=False)}\ntool execution: {message['name']}\nResult:\n---\n{message['content']}\n---")
def _print_assistant_message(self, message, timestamp: str):
if MC_MODE: colors = ["grey58"] * 3
else: colors = ["light_salmon3", "blue", "purple"]
self.console.print(self._wrap_title("Assistant Message", f"bold {colors[0]}"))
self.console.print(f"{self._wrap_timestamp(timestamp, color=True)}\n[bold {colors[1]}]{message['sender']}[/bold {colors[1]}]:", end=" ")
if message["content"]:
print_str = escape(message["content"])
if MC_MODE: print_str = f"[grey58]{print_str}[/grey58]"
self.console.print(print_str, highlight=True, emoji=True)
else:
print_str = None
if MC_MODE: print_str = "[grey58]None[/grey58]"
self.console.print(print_str, highlight=True, emoji=True)
def _save_assistant_message(self, message, timestamp: str):
self._write_log(self._wrap_title("Assistant Message"))
content = message["content"] if message["content"] else None
self._write_log(f"{self._wrap_timestamp(timestamp, color=False)}\n{message['sender']}: {content}")
def _print_tool_call(self, tool_calls: List, timestamp: str):
if MC_MODE: colors = ["grey58"] * 3
else: colors = ["light_pink1", "blue", "purple"]
if len(tool_calls) >= 1: self.console.print(self._wrap_title("Tool Calls", f"bold {colors[0]}"))
for tool_call in tool_calls:
f = tool_call["function"]
name, args = f["name"], f["arguments"]
arg_str = self._warp_args(args)
print_arg_str = escape(arg_str)
if MC_MODE: print_arg_str = f"[grey58]{print_arg_str}[/grey58]"
self.console.print(f"{self._wrap_timestamp(timestamp, color=True)}\n[bold {colors[2]}]{name}[/bold {colors[2]}]({print_arg_str})")
def _save_tool_call(self, tool_calls: List, timestamp: str):
if len(tool_calls) >= 1: self._write_log(self._wrap_title("Tool Calls"))
for tool_call in tool_calls:
f = tool_call["function"]
name, args = f["name"], f["arguments"]
arg_str = self._warp_args(args)
self._write_log(f"{self._wrap_timestamp(timestamp, color=False)}\n{name}({arg_str})")
def pretty_print_messages(self, message, **kwargs) -> None:
# for message in messages:
if message["role"] != "assistant" and message["role"] != "tool":
return
# console = Console()
# handle tool call
if message["role"] == "tool":
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
if self.log_path: self._save_tool_execution(message, timestamp)
if self.debug: self._print_tool_execution(message, timestamp)
return
# handle assistant message
# print agent name in blue
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
if self.log_path: self._save_assistant_message(message, timestamp)
if self.debug: self._print_assistant_message(message, timestamp)
# print tool calls in purple, if any
tool_calls = message.get("tool_calls") or []
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
if self.log_path: self._save_tool_call(tool_calls, timestamp)
if self.debug: self._print_tool_call(tool_calls, timestamp)
class LoggerManager:
_instance = None
_logger: MetaChainLogger = None
@classmethod
def get_instance(cls):
if cls._instance is None:
cls._instance = LoggerManager()
return cls._instance
@classmethod
def get_logger(cls):
return cls.get_instance()._logger
@classmethod
def set_logger(cls, new_logger):
cls.get_instance()._logger = new_logger
if DEFAULT_LOG:
if LOG_PATH is None:
log_dir = Path(f'logs/res_{datetime.now().strftime("%Y%m%d_%H%M%S")}')
log_dir.mkdir(parents=True, exist_ok=True) # recursively create all necessary parent directories
log_path = str(log_dir / "agent.log")
# logger = MetaChainLogger(log_path=log_path)
LoggerManager.set_logger(MetaChainLogger(log_path=log_path))
else:
# logger = MetaChainLogger(log_path=LOG_PATH)
LoggerManager.set_logger(MetaChainLogger(log_path=LOG_PATH))
# logger.info("Log file is saved to", logger.log_path, "...", title="Log Path", color="light_cyan3")
LoggerManager.get_logger().info("Log file is saved to",
LoggerManager.get_logger().log_path, "...",
title="Log Path", color="light_cyan3")
else:
# logger = None
LoggerManager.set_logger(None)
logger = LoggerManager.get_logger()
def set_logger(new_logger):
LoggerManager.set_logger(new_logger)
# if __name__ == "__main__":
# logger = MetaChainLogger(log_path="test.log")
# logger.pretty_print_messages({"role": "assistant", "content": "Hello, world!", "tool_calls": [{"function": {"name": "test", "arguments": {"url": "https://www.google.com", "query": "test"}}}], "sender": "test_agent"})
# logger.pretty_print_messages({"role": "tool", "name": "test", "content": "import requests\n\nurl = 'https://www.google.com'\nquery = 'test'\n\nresponse = requests.get(url)\nprint(response.text)", "sender": "test_agent"})
# logger.info("test content", color="red", title="test")
autoagent/main.py
+109
View File
@@ -0,0 +1,109 @@
from autoagent import MetaChain, Agent, Response
from typing import List
from autoagent.logger import MetaChainLogger
from autoagent.environment.utils import setup_metachain
from autoagent.environment.docker_env import DockerConfig, DockerEnv
def case_resolved(result: str):
"""
Use this tool to indicate that the case is resolved. You can use this tool only after you truly resolve the case with exsiting tools and created new tools.Please encapsulate your final answer (answer ONLY) within <solution> and </solution>.
Args:
result: The final result of the case resolution following the instructions.
Example: case_resolved(`The answer to the question is <solution> 42 </solution>`)
"""
return f"Case resolved. No further actions are needed. The result of the case resolution is: {result}"
def case_not_resolved(failure_reason: str, take_away_message: str):
"""
Use this tool to indicate that the case is not resolved when all agents have tried their best.
[IMPORTANT] Please do not use this function unless all of you have tried your best.
You should give the failure reason to tell the user why the case is not resolved, and give the take away message to tell which information you gain from creating new tools.
Args:
failure_reason: The reason why the case is not resolved.
take_away_message: The message to take away from the case.
"""
return f"Case not resolved. The reason is: {failure_reason}. But though creating new tools, I gain some information: {take_away_message}"
async def run_in_client(
agent: Agent,
messages: List,
context_variables: dict = {},
logger: MetaChainLogger = None,
meta_agent: Agent = None,
docker_config: DockerConfig = None,
code_env: DockerConfig = None,
):
"""
"""
client = MetaChain(log_path=logger)
MAX_RETRY = 3
for i in range(MAX_RETRY):
try:
response: Response = await client.run_async(agent, messages, context_variables, debug=True)
except Exception as e:
logger.info(f'Exception in main loop: {e}', title='ERROR', color='red')
raise e
if 'Case resolved' in response.messages[-1]['content']:
break
elif 'Case not resolved' in response.messages[-1]['content']:
messages.extend(response.messages)
if meta_agent and (i >= 2):
setup_metachain(docker_config.workplace_name, code_env)
messages.append({
'role': 'user',
'content': """\
It seems that the case is not resolved with the existing agent system.
Help me to solve this problem by running tools in the MetaChain.
IMPORTANT: You should fully take advantage of existing tools, and if existing tools are not enough, you should develop new tools.
Use `visual_question_answering` tool for ALL visual tasks (images, videos, visual analysis, including object detection, etc.)
IMPORTANT: You can not stop with `case_not_resolved` after you try your best to creating new tools.
IMPORTANT: You should ONLY interact with the environment provided to you AND NEVER ASK FOR HUMAN HELP.
Please encapsulate your final answer (answer ONLY) within <solution> and </solution>.
"""
})
meta_agent.functions.append(case_not_resolved)
meta_agent.functions.append(case_resolved)
response: Response = await client.run_async(meta_agent, messages, context_variables, debug=True)
if 'Case resolved' in response.messages[-1]['content']:
break
else:
messages.extend(response.messages)
messages.append({
'role': 'user',
'content': 'Please try to resolve the case again. It\'s important for me to resolve the case. Trying again in another way may be helpful.'
})
return response
def run_in_client_non_async(
agent: Agent,
messages: List,
context_variables: dict = {},
logger: MetaChainLogger = None,
):
"""
"""
client = MetaChain(log_path=logger)
MAX_RETRY = 3
for i in range(MAX_RETRY):
try:
response: Response = client.run(agent, messages, context_variables, debug=True)
except Exception as e:
logger.info(f'Exception in main loop: {e}', title='ERROR', color='red')
raise e
if 'Case resolved' in response.messages[-1]['content']:
break
elif 'Case not resolved' in response.messages[-1]['content']:
messages.extend(response.messages)
messages.append({
'role': 'user',
'content': 'Please try to resolve the case again. It\'s important for me to resolve the case. Trying again in another way may be helpful.'
})
return response
autoagent/memory/__init__.py
View File
autoagent/memory/code_memory.py
+174
View File
@@ -0,0 +1,174 @@
import os
from typing import List, Dict
from autoagent.memory.rag_memory import Memory, Reranker
from litellm import completion
import re
class CodeMemory(Memory):
def __init__(self, project_path: str, db_name: str = '.sa', platform: str = 'OpenAI', api_key: str = None, embedding_model: str = "text-embedding-ada-002"):
super().__init__(project_path, db_name, platform, api_key, embedding_model)
self.collection_name = 'code_memory'
def add_code_files(self, directory: str, exclude_prefix: List[str] = ["workplace_"]):
"""
Add all code files in the specified directory to the memory.
Args:
directory (str): The directory path containing the code files to add.
"""
code_files = []
for root, _, files in os.walk(directory):
root_name = str(root)
if any(prefix in root_name for prefix in exclude_prefix):
continue
for file in files:
if file.endswith(('.py', '.js', '.java', '.cpp', '.h', '.c', '.html', '.css')): # add more file types if needed
file_path = os.path.join(root, file)
with open(file_path, 'r', encoding='utf-8') as f:
content = f.read()
code_files.append({
"query": f"File: {file_path}\n\nContent:\n{content}",
"response": f"This is the content of file {file_path}"
})
self.add_query(code_files, self.collection_name)
def query_code(self, query_text: str, n_results: int = 5) -> List[Dict]:
"""
Query the code memory.
Args:
query_text (str): The query text
n_results (int): The number of results to return
Returns:
List[Dict]: The query results list
"""
results = self.query([query_text], self.collection_name, n_results)
return [
{
"file": doc.split('\n')[0].replace("File: ", ""),
"content": '\n'.join(doc.split('\n')[3:]),
"metadata": metadata
}
for doc, metadata in zip(results['documents'][0], results['metadatas'][0])
]
class CodeReranker(Reranker):
def __init__(self, model: str) -> None:
super().__init__(model)
def wrap_query_results(self, query_results: List[Dict]) -> str:
wrapped_query_results = ""
for result in query_results:
wrapped_query_results += f"File: {result['file']}\n"
wrapped_query_results += f"Content: {result['content'][:300]}...\n"
wrapped_query_results += "---"
return wrapped_query_results
def wrap_reranked_results(self, reranked_paths: List[str]) -> str:
wrapped_reranked_results = "[Referenced code files]:"
for path in reranked_paths:
wrapped_reranked_results += f"Code path: {path}\n"
try:
with open(path, 'r', encoding='utf-8') as file:
content = file.read()
wrapped_reranked_results += f"Code content:\n{content}\n"
except Exception as e:
wrapped_reranked_results += f"Error reading file: {str(e)}\n"
wrapped_reranked_results += "---\n"
return wrapped_reranked_results
def parse_results(self, reranked_results: str) -> List[str]:
lines = reranked_results.strip().split('\n')
# get the last 5 lines
last_lines = lines[-5:]
# remove the number and dot at the beginning of each line
cleaned_lines = [re.sub(r'^\d+\.\s*', '', line.strip()) for line in last_lines]
unique_lines = list(dict.fromkeys(cleaned_lines))
return unique_lines
def rerank(self, query_text: str, query_results: List[Dict]) -> List[Dict]:
system_prompt = \
"""
You are a helpful assistant that reranks the given code files (containing the path of files and Overview of the content of files) based on the query.
You should rerank the code files based on the query, and the most relevant code files should be ranked on the top.
You should select the top 5 code files to answer the query, by giving the file path of the code files.
Example:
[Query]: "The definition of 'BaseAgent'"
[Code files]:
File: /Users/tangjiabin/Documents/reasoning/SelfAgent/sa/agents/__init__.py
Content: from .ABCAgent import ABCAgent
from .BaseAgent import BaseAgent
from .ManagerAgent import ManagerAge...
---
File: /Users/tangjiabin/Documents/reasoning/SelfAgent/sa/agents/__init__.py
Content: from .ABCAgent import ABCAgent
from .BaseAgent import BaseAgent
from .ManagerAgent import ManagerAge...
---
File: /Users/tangjiabin/Documents/reasoning/SelfAgent/sa/agents/__init__.py
Content: from .ABCAgent import ABCAgent
from .BaseAgent import BaseAgent
from .ManagerAgent import ManagerAge...
---
File: /Users/tangjiabin/Documents/reasoning/SelfAgent/sa/agents/__init__.py
Content: from .ABCAgent import ABCAgent
from .BaseAgent import BaseAgent
from .ManagerAgent import ManagerAge...
---
File: /Users/tangjiabin/Documents/reasoning/SelfAgent/sa/agent_prompts/__init__.py
Content: from .BasePrompt import BasePromptGen, ManagerPromptGen, PromptGen
...
---
File: /Users/tangjiabin/Documents/reasoning/SelfAgent/sa/agent_prompts/__init__.py
Content: from .BasePrompt import BasePromptGen, ManagerPromptGen, PromptGen
...
---
File: /Users/tangjiabin/Documents/reasoning/SelfAgent/sa/agent_prompts/__init__.py
Content: from .BasePrompt import BasePromptGen, ManagerPromptGen, PromptGen
...
---
File: /Users/tangjiabin/Documents/reasoning/SelfAgent/sa/agent_prompts/__init__.py
Content: from .BasePrompt import BasePromptGen, ManagerPromptGen, PromptGen
...
---
File: /Users/tangjiabin/Documents/reasoning/SelfAgent/sa/agents/BaseAgent.py
Content: from typing import List
from sa.actions import BaseAction, FinishAct, ThinkAct, PlanAct
from sa.age...
---
File: /Users/tangjiabin/Documents/reasoning/SelfAgent/sa/agents/BaseAgent.py
Content: from typing import List
from sa.actions import BaseAction, FinishAct, ThinkAct, PlanAct
from sa.age...
---
[Reranked 5 code files]:
1. /Users/tangjiabin/Documents/reasoning/SelfAgent/sa/agents/BaseAgent.py
2. /Users/tangjiabin/Documents/reasoning/SelfAgent/sa/agents/__init__.py
3. /Users/tangjiabin/Documents/reasoning/SelfAgent/sa/agents/ABCAgent.py
4. /Users/tangjiabin/Documents/reasoning/SelfAgent/sa/agents/ManagerAgent.py
5. /Users/tangjiabin/Documents/reasoning/SelfAgent/sa/agents/AgentLogger.py
"""
wrapped_query_results = self.wrap_query_results(query_results)
user_prompt = \
"""
[Query]: \n{query_text}
[Code files]: \n{query_results}
[Reranked 5 code files]:
""".format(query_text=query_text, query_results=wrapped_query_results)
chat_history = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt}
]
create_params = {
"model": self.model,
"messages": chat_history,
"stream": False,
}
response = completion(**create_params)
reranked_results = self.parse_results(response.choices[0].message.content)
reranked_results = self.wrap_reranked_results(reranked_results)
return reranked_results
autoagent/memory/code_tree/code_parser.py
+123
View File
@@ -0,0 +1,123 @@
import dataclasses
from tree_sitter import Language
import tree_sitter
import glob
import uuid
from loguru import logger
@dataclasses.dataclass
class Snippet:
"""Dataclass for storing Embedded Snippets"""
id: str
embedding: list[float] | None
snippet: str
filename: str
language: str
class CodeParser:
"""Code Parser Class."""
def __init__(self, language: str, node_types: list[str], path_to_object_file: str):
self.node_types = node_types
self.language = language
try:
self.parser = tree_sitter.Parser()
self.parser.set_language(
tree_sitter.Language(f"{path_to_object_file}/my-languages.so", language)
)
except Exception as e:
logger.exception("failed to build %s parser: ", e)
def parse_file(self, content: str, filename: str):
"""
Parse code snippets from single code file.
Args:
content: The content of the file.
filename: The name of the code file.
Returns:
List of Parsed Snippets
"""
try:
tree = self.parser.parse(content)
except Exception as e:
logger.error(f"Failed to parse snippet: {filename} \n Error: {e}")
return
cursor = tree.walk()
parsed_snippets = []
# Walking nodes from abstract syntax tree
while cursor.goto_first_child():
if cursor.node.type in self.node_types:
parsed_snippets.append(
Snippet(
id=str(uuid.uuid4()),
snippet=cursor.node.text,
filename=filename,
language=self.language,
embedding=None,
)
)
while cursor.goto_next_sibling():
if cursor.node.type in self.node_types:
parsed_snippets.append(
Snippet(
id=str(uuid.uuid4()),
snippet=cursor.node.text,
filename=filename,
language=self.language,
embedding=None,
)
)
return parsed_snippets
def parse_directory(self, code_directory_path):
"""
Parse code snippets from all files in directory.
Args:
code_directory_path: Directory path containing code files.
Returns:
List of Parsed Snippets
"""
parsed_contents = []
for filename in glob.glob(f"{code_directory_path}/**/*.py", recursive=True):
# print(filename)
with open(filename, "rb") as codefile:
code_content = codefile.read()
parsed_content = self.parse_file(code_content, filename)
parsed_contents.extend(parsed_content)
return parsed_contents
def to_dataframe_row(embedded_snippets: list[Snippet]):
"""
Helper function to convert Embedded Snippet object to a dataframe row
in dictionary format.
Args:
embedded_snippets: List of Snippets to be converted
Returns:
List of Dictionaries
"""
outputs = []
for embedded_snippet in embedded_snippets:
output = {
"ids": embedded_snippet.id,
"embeddings": embedded_snippet.embedding,
"snippets": embedded_snippet.snippet,
"metadatas": {
"filenames": embedded_snippet.filename,
"languages": embedded_snippet.language,
},
}
outputs.append(output)
return outputs
autoagent/memory/codetree_memory.py
+118
View File
@@ -0,0 +1,118 @@
import os
from typing import List, Dict
from autoagent.memory.rag_memory import Memory, Reranker
import openai
import re
from autoagent.memory.code_tree.code_parser import CodeParser, to_dataframe_row
from tree_sitter import Language
from loguru import logger
from openai import OpenAI
import pandas as pd
class CodeTreeMemory(Memory):
def __init__(self, project_path: str, db_name: str = '.code_tree', platform: str = 'OpenAI', api_key: str = None, embedding_model: str = "text-embedding-ada-002"):
super().__init__(project_path, db_name, platform, api_key, embedding_model)
self.collection_name = 'code_tree_memory'
self.embedder = OpenAI(api_key=api_key)
def add_code_files(self, directory: str, exclude_prefix: List[str] = ["workplace_"]):
"""
将指定目录下的所有代码文件添加到内存中。
Args:
directory (str): 要添加的代码文件所在的目录路径
"""
tree_sitter_parent_dir = os.path.dirname(os.getcwd())
# Build Tree sitter Parser object
Language.build_library(
f"{tree_sitter_parent_dir}/my-languages.so",
[
f"{tree_sitter_parent_dir}/tree-sitter-python",
],
)
parser = CodeParser(
language="python",
node_types=["class_definition", "function_definition"],
path_to_object_file=tree_sitter_parent_dir,
)
logger.info("Parsing Code...")
parsed_snippets = parser.parse_directory(
directory
)
snippet_texts = list(map(lambda x: x.snippet.decode("ISO-8859-1"), parsed_snippets))
embedded_texts = self.embedder.embeddings.create(input=snippet_texts, model="text-embedding-3-small").data
embedded_snippets = []
for code_text, embedding, snippet in zip(
snippet_texts, embedded_texts, parsed_snippets
):
snippet.snippet = code_text
snippet.embedding = embedding.embedding
embedded_snippets.append(snippet)
# Convert Snippets to DataFrame for ChromaDB Ingestion
data = pd.DataFrame(to_dataframe_row(embedded_snippets))
collection = self.client.get_or_create_collection(
name=self.collection_name, metadata={"hnsw:space": "cosine"}
)
logger.info(
f"Adding {data.shape[0]} Code snippets and embedding to "
"local chroma db collection..."
)
collection.add(
documents=data["snippets"].tolist(),
embeddings=data["embeddings"].tolist(),
metadatas=data["metadatas"].tolist(),
ids=data["ids"].tolist(),
)
def query_code(self, query_text: str, n_results: int = 5) -> List[Dict]:
"""
Query the code memory.
Args:
query_text (str): The query text
n_results (int): The number of results to return
Returns:
List[Dict]: The query results list
"""
query_embedding = self.embedder.embeddings.create(input=[query_text], model="text-embedding-3-small").data[0].embedding
results = self.client.get_or_create_collection(self.collection_name).query(query_embeddings=[query_embedding], n_results=n_results)
return [
{
"file": metadata['filenames'],
"content": doc
}
for doc, metadata in zip(results['documents'][0], results['metadatas'][0])
]
class DummyReranker(Reranker):
def __init__(self, model: str = None) -> None:
super().__init__(model)
def rerank(self, query_results: List[Dict]) -> List[Dict]:
wrapped_reranked_results = "[Referenced code files]:"
result_path = []
for result in query_results:
if result['file'] in result_path:
continue
else:
result_path.append(result['file'])
wrapped_reranked_results = f"Code path: {result['file']}\n"
wrapped_reranked_results += f"Code content:\n{result['content']}...\n"
wrapped_reranked_results += "---\n"
return wrapped_reranked_results
# 使用示例
if __name__ == "__main__":
code_memory = CodeTreeMemory(project_path = './code_db', db_name='code_tree', platform='OpenAI', api_key='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')
# 添加代码文件到内存
code_memory.add_code_files("/Users/tangjiabin/Documents/reasoning/SelfAgent/workplace_test/SelfAgent", exclude_prefix=['workplace_', '__pycache__', 'code_db', '.git'])
# 查询代码
query_results = code_memory.query_code("The definition of BaseAction", n_results=10)
for result in query_results:
print(f"File: {result['file']}")
print(f"Content: {result['content'][:100]}...") # 只打印前100个字符
print("---")
autoagent/memory/paper_memory.py
+80
View File
@@ -0,0 +1,80 @@
import pandas as pd
from typing import List, Dict
from autoagent.memory.rag_memory import Memory, Reranker
import json
import math
import os
from litellm import completion
from autoagent.memory.utils import chunking_by_token_size
class TextMemory(Memory):
def __init__(
self,
project_path: str,
db_name: str = '.text_table',
platform: str = 'OpenAI',
api_key: str = None,
embedding_model: str = "text-embedding-3-small",
):
super().__init__(
project_path=project_path,
db_name=db_name,
platform=platform,
api_key=api_key,
embedding_model=embedding_model
)
self.collection_name = 'text_memory'
def add_text_content(self, paper_content: str, batch_size: int = 100, collection = None):
assert collection is not None, "Collection is required. Should be the path of the paper."
queries = []
content_chunks = chunking_by_token_size(paper_content, max_token_size=4096)
idx_list = ["chunk_" + str(chunk['chunk_order_index']) for chunk in content_chunks]
for chunk in content_chunks:
query = {
'query': chunk['content'],
'response': chunk['content']
}
queries.append(query)
# self.add_query(queries, collection=collection)
print(f'Adding {len(queries)} queries to {collection} with batch size {batch_size}')
num_batches = math.ceil(len(queries) / batch_size)
for i in range(num_batches):
start_idx = i * batch_size
end_idx = min((i + 1) * batch_size, len(queries))
batch_queries = queries[start_idx:end_idx]
batch_idx = idx_list[start_idx:end_idx]
# Add the current batch of queries
self.add_query(batch_queries, collection=collection, idx=batch_idx)
print(f"Batch {i+1}/{num_batches} added")
def query_text_content(
self,
query_text: str,
collection: str = None,
n_results: int = 5
) -> List[str]:
"""
Query the table and return the results
"""
assert collection is not None, "Collection is required. Should be the path of the paper."
results = self.query([query_text], collection=collection, n_results=n_results)
metadata_results = results['metadatas'][0]
results = [item['response'] for item in metadata_results]
return results
def peek_table(self, collection: str = None, n_results: int = 20) -> pd.DataFrame:
"""
Peek at the data in the table
"""
assert collection is not None, "Collection is required. Should be the path of the paper."
raw_results = self.peek(collection=collection, n_results=n_results)
results = [item['response'] for item in raw_results['metadatas']]
return results
autoagent/memory/rag_memory.py
+179
View File
@@ -0,0 +1,179 @@
import uuid
import os.path
from datetime import datetime
from typing import List, Dict
import chromadb
from chromadb.utils import embedding_functions
from abc import ABC, abstractmethod
from openai import OpenAI
import numpy as np
from chromadb.api.types import QueryResult
chromadb.logger.setLevel(chromadb.logging.ERROR)
class Memory:
def __init__(
self,
project_path: str,
db_name: str = '.sa',
platform: str = 'OpenAI',
api_key: str = None,
embedding_model: str = "text-embedding-3-small"
):
"""
Memory: memory and external knowledge management.
Args:
project_path: the path to store the data.
embedding_model: the embedding model to use, default will use the embedding model from ChromaDB,
if the OpenAI has been set in the configuration, it will use the OpenAI embedding model
"text-embedding-ada-002".
"""
self.db_name = db_name
self.collection_name = 'memory'
self.client = chromadb.PersistentClient(path=os.path.join(project_path, self.db_name))
self.client.get_or_create_collection(
self.collection_name,
)
# use the OpenAI embedding function if the openai section is set in the configuration.
if platform == 'OpenAI':
openai_client = OpenAI(api_key=api_key or os.environ["OPENAI_API_KEY"])
self.embedder = lambda x: [i.embedding for i in openai_client.embeddings.create(input=x, model=embedding_model).data]
else:
# self.embedder = embedding_functions.DefaultEmbeddingFunction()
self.embedder = embedding_functions.SentenceTransformerEmbeddingFunction(model_name="all-MiniLM-L6-v2")
def add_query(
self,
queries: List[Dict[str, str]],
collection: str = None,
idx: List[str] = None
):
"""
add_query: add the queries to the memery.
Args:
queries: the queries to add to the memery. Should be in the format of
{
"query": "the query",
"response": "the response"
}
collection: the name of the collection to add the queries.
idx: the ids of the queries, should be in the same length as the queries.
If not provided, the ids will be generated by UUID.
Return: A list of generated IDs.
"""
if idx:
ids = idx
else:
ids = [str(uuid.uuid4()) for _ in range(len(queries))]
if not collection:
collection = self.collection_name
query_list = [query['query'] for query in queries]
embeddings = self.embedder(query_list)
added_time = datetime.now().isoformat()
resp_list = [{'response': query['response'], 'created_at': added_time} for query in queries]
# insert the record into the database
self.client.get_or_create_collection(collection).add(
documents=query_list,
metadatas=resp_list,
ids=ids,
embeddings=embeddings
)
return ids
def query(self, query_texts: List[str], collection: str = None, n_results: int = 5) -> QueryResult:
"""
query: query the memery.
Args:
query_texts: the query texts to search in the memery.
collection: the name of the collection to search.
n_results: the number of results to return.
Returns: QueryResult
class QueryResult(TypedDict):
ids: List[IDs]
embeddings: Optional[
Union[
List[Embeddings],
List[PyEmbeddings],
List[NDArray[Union[np.int32, np.float32]]],
]
]
documents: Optional[List[List[Document]]]
uris: Optional[List[List[URI]]]
data: Optional[List[Loadable]]
metadatas: Optional[List[List[Metadata]]]
distances: Optional[List[List[float]]]
included: Include
"""
if not collection:
collection = self.collection_name
query_embedding = self.embedder(query_texts)
return self.client.get_or_create_collection(collection).query(query_embeddings=query_embedding, n_results=n_results)
def peek(self, collection: str = None, n_results: int = 20):
"""
peek: peek the memery.
Args:
collection: the name of the collection to peek.
n_results: the number of results to return.
Returns: the top k results.
"""
if not collection:
collection = self.collection_name
return self.client.get_or_create_collection(collection).peek(limit=n_results)
def get(self, collection: str = None, record_id: str = None):
"""
get: get the record by the id.
Args:
record_id: the id of the record.
collection: the name of the collection to get the record.
Returns: the record.
"""
if not collection:
collection = self.collection_name
collection = self.client.get_collection(collection)
if not record_id:
return collection.get()
return collection.get(record_id)
def delete(self, collection_name=None):
"""
delete: delete the memery collections.
Args:
collection_name: the name of the collection to delete.
"""
if not collection_name:
collection_name = self.collection_name
return self.client.delete_collection(name=collection_name)
def count(self, collection_name=None):
"""
count: count the number of records in the memery.
Args:
collection_name: the name of the collection to count.
"""
if not collection_name:
collection_name = self.collection_name
return self.client.get_or_create_collection(name=collection_name).count()
def reset(self):
"""
reset: reset the memory.
Notice: You may need to set the environment variable `ALLOW_RESET` to `TRUE` to enable this function.
"""
self.client.reset()
class Reranker:
def __init__(self, model: str) -> None:
self.model = model
@abstractmethod
def rerank(self, query_text: str, query_results: List[Dict]) -> List[Dict]:
raise NotImplementedError("Reranker is not implemented")
autoagent/memory/tool_memory.py
+164
View File
@@ -0,0 +1,164 @@
import pandas as pd
from typing import List, Dict
from autoagent.memory.rag_memory import Memory, Reranker
import json
import math
import os
from litellm import completion
from pydantic import BaseModel
"""
Category | Tool_Name | Tool_Description | API_Name | API_Description | Method | API_Details | Required_API_Key | Platform
"""
class ToolMemory(Memory):
def __init__(
self,
project_path: str,
db_name: str = '.tool_table',
platform: str = 'OpenAI',
api_key: str = None,
embedding_model: str = "text-embedding-3-small",
):
super().__init__(
project_path=project_path,
db_name=db_name,
platform=platform,
api_key=api_key,
embedding_model=embedding_model
)
self.collection_name = 'tool_memory'
def add_dataframe(self, df: pd.DataFrame, collection: str = None, batch_size: int = 100):
if not collection:
collection = self.collection_name
queries = []
for idx, row in df.iterrows():
query = {
'query': ' '.join(row[['Tool_Name', 'Tool_Description', 'API_Name', 'API_Description']].astype(str)),
'response': row.to_json()
}
queries.append(query)
# self.add_query(queries, collection=collection)
print(f'Adding {len(queries)} queries to {collection} with batch size {batch_size}')
num_batches = math.ceil(len(queries) / batch_size)
for i in range(num_batches):
start_idx = i * batch_size
end_idx = min((i + 1) * batch_size, len(queries))
batch_queries = queries[start_idx:end_idx]
# Add the current batch of queries
self.add_query(batch_queries, collection=collection)
print(f"Batch {i+1}/{num_batches} added")
def query_table(
self,
query_text: str,
collection: str = None,
n_results: int = 5
) -> pd.DataFrame:
"""
Query the table and return the results
"""
if not collection:
collection = self.collection_name
results = self.query([query_text], collection=collection, n_results=n_results)
metadata_results = results['metadatas'][0]
df_results = pd.DataFrame([json.loads(item['response']) for item in metadata_results])
return df_results
def peek_table(self, collection: str = None, n_results: int = 20) -> pd.DataFrame:
"""
Peek at the data in the table
"""
if not collection:
collection = self.collection_name
results = self.peek(collection=collection, n_results=n_results)
df_results = pd.DataFrame([json.loads(item['response']) for item in results['metadatas']])
return df_results
class ToolReranker(Reranker):
def rerank(self, query_text: str, query_df: pd.DataFrame) -> str:
system_prompt = \
"""
You are a helpful assistant that reranks the given API table based on the query.
You should select the top 5 APIs to answer the query in the given format.
You can only select APIs I give you.
Directly give the answer without any other words.
"""
# Use the DataFrame's to_dict method to convert all rows to a list of dictionaries
# print('query_df', query_df)
api_data = query_df.to_dict(orient='records')
# Use a list comprehension and f-string to format each API's data
api_prompts = [f"\n\nAPI {i+1}:\n{api}" for i, api in enumerate(api_data)]
# add the query text to the prompt
prompt = ''.join(api_prompts)
prompt = f"The query is: {query_text}\n\n{prompt}"
message = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": prompt}
]
class Tools(BaseModel):
tool_name: str
api_name: str
rank: int
class RerankResult(BaseModel):
tools: list[Tools]
create_params = {
"model": self.model,
"messages": message,
"stream": False,
"response_format": RerankResult
}
response = completion(**create_params).choices[0].message.content
print(response)
rerank_result = json.loads(response)["tools"]
print(rerank_result)
if len(rerank_result) == 0:
return "Fail to retrieve the relevant information from the tool documentation."
try:
return self.wrap_rerank_result(rerank_result, query_df)
except Exception as e:
raise ValueError(f"Failed to wrap rerank result: {e}")
def wrap_rerank_result(self, rerank_result: List[pd.DataFrame], query_df: pd.DataFrame) -> str:
res = ""
res_tmp = """
The rank {rank} referenced tool documentation is:
API Name: {api_name}
API Description: {api_description}
API Details: {api_details}
Required API Key: {required_api_key}
Platform: {platform}
"""
try:
for tool_api in rerank_result:
tool_name = tool_api['tool_name']
api_name = tool_api['api_name']
matched_rows = query_df[(query_df['API_Name'] == api_name) & (query_df['Tool_Name'] == tool_name)]
if not matched_rows.empty:
res = res + res_tmp.format(rank=tool_api['rank'], api_name=matched_rows['API_Name'].values[0], api_description=matched_rows['API_Description'].values[0], api_details=matched_rows['API_Details'].values[0], required_api_key=matched_rows['Required_API_Key'].values[0], platform=matched_rows['Platform'].values[0])
return res
except Exception as e:
raise ValueError(f"Failed to wrap rerank result: {e}")
def dummy_rerank(self, query_text: str, query_df: pd.DataFrame) -> str:
res = ""
res_tmp = """
The rank {rank} referenced tool documentation is:
API Name: {api_name}
API Description: {api_description}
API Details: {api_details}
Required API Key: {required_api_key}
Platform: {platform}
"""
for i in range(len(query_df)):
res = res + res_tmp.format(rank=i+1, api_name=query_df['API_Name'].values[i], api_description=query_df['API_Description'].values[i], api_details=query_df['API_Details'].values[i], required_api_key=query_df['Required_API_Key'].values[i], platform=query_df['Platform'].values[i])
return res
autoagent/memory/utils.py
+36
View File
@@ -0,0 +1,36 @@
import tiktoken
ENCODER = None
def encode_string_by_tiktoken(content: str, model_name: str = "gpt-4o"):
global ENCODER
if ENCODER is None:
ENCODER = tiktoken.encoding_for_model(model_name)
tokens = ENCODER.encode(content)
return tokens
def decode_tokens_by_tiktoken(tokens: list[int], model_name: str = "gpt-4o"):
global ENCODER
if ENCODER is None:
ENCODER = tiktoken.encoding_for_model(model_name)
content = ENCODER.decode(tokens)
return content
def chunking_by_token_size(
content: str, overlap_token_size=128, max_token_size=1024, tiktoken_model="gpt-4o"
):
tokens = encode_string_by_tiktoken(content, model_name=tiktoken_model)
results = []
for index, start in enumerate(
range(0, len(tokens), max_token_size - overlap_token_size)
):
chunk_content = decode_tokens_by_tiktoken(
tokens[start : start + max_token_size], model_name=tiktoken_model
)
results.append(
{
"tokens": min(max_token_size, len(tokens) - start),
"content": chunk_content.strip(),
"chunk_order_index": index,
}
)
return results

Some files were not shown because too many files have changed in this diff Show More