wehub-resource-sync
Scaling Agents via Continual Pre-training 🚀
This work is the first to bring Agentic Continual Pretraining (Agentic CPT) into the training pipeline of Deep Research Agents, resulting in the powerful agentic model AgentFounder-30B ⚡️.
🌟 Beyond standard post-training, we propose a systematic agentic training data synthesis method for Agentic CPT, and design a two-stage continual pre-training strategy (as illustrated below):
✨ Features
🧩 Agentic Training Pipeline
We redesign the training pipeline of deep research agents by introducing continual pre-training with context lengths of 32K and 128K. This design ensures both training efficiency and improved performance, enabling the agent to effectively handle increasingly complex research tasks.
🧠 Scaling Training Contexts based on Open-World Memory
We transform continuously updated data streams into an open-world memory, enabling the synthesis of diverse QA styles.
📝 FAS — Planning Action Synthesis
Building on the strong correlation between initial planning and trajectory's accuracy, we generate a large number of reasoning–action data from diverse QA instances to strengthen the agent’s planning capability.
💡 FAS — Reasoning Action Synthesis
By combining questions with their knowledge sources, we emulate the process of deriving final answers through logical inference under fully informed conditions, strengthening the agent’s reasoning capability.
🔗 HAS — Decision-Making Action Synthesis
We reformulate the agent trajectories as multi-step decision-making processes, fully exploring the reasoning–action space at each step. HAS expands the agent’s capacity to explore the action–answer space while enhancing its decision-making abilities.
🏆 High-light Performance
General Web Search Benchmarks
| Backbone | BrowseComp-en | BrowseComp-zh | GAIA(text) | xbench-DeepSearch | WebwalkerQA |
|---|---|---|---|---|---|
| General LLMs with tools | |||||
| Qwen3-30B-A3B | 0.5 | 13.5 | 35.9 | 32.0 | 46.9 |
| Qwen3-235B-A22B | 2.3 | 29.4 | 45.6 | 46.0 | 59.6 |
| DeepSeek-R1 | 8.9† | 35.7† | – | 55.0† | – |
| Claude-4-Sonnet | 12.2† | 29.1† | 68.3† | 64.6† | 61.7† |
| Commercial Deep Research Agents | |||||
| Kimi-Researcher | – | – | – | 69.0† | – |
| OpenAI-o3 | 49.7† | 58.1† | 70.5† | 66.7† | 71.7† |
| OpenAI Deep Research | 51.5† | – | 67.0† | – | – |
| Open-source Deep Research Agents | |||||
| WebThinker-32B-RL | 2.8† | 7.3† | 48.5† | 24.0† | 46.5† |
| ASearcher-Web-QwQ | 5.2† | 15.6† | 52.8† | 42.1† | 34.3† |
| WebSailor-72B | 12.0† | 30.1† | 55.4† | 55.0† | – |
| WebShaper-72B | – | – | 60.1† | – | 52.2† |
| AFM-32B-RL | 11.1† | – | 55.3† | 63.0† | – |
| MiroThinker-32B-DPOv0.2 | 17.2† | 29.4† | 64.1† | 56.0† | 53.6† |
| DeepDiver-V2-38B | 13.4† | 34.6† | – | 53.0† | – |
| WebExplorer-8B | 15.7† | 32.0† | 50.0† | 53.7† | 62.7† |
| DeepDive-32B | 14.8† | 25.6† | – | 50.5† | – |
| Kimi-K2 | 14.1† | 28.8† | 57.3† | 50.0† | 63.0† |
| GLM-4.5 | 26.4† | 37.5† | 66.0† | 70.0† | 65.6† |
| DeepSeek-V3.1 | 30.0† | 49.2† | 63.1† | 71.2† | 61.2† |
| Ours | |||||
| AgentFounder-30B | 40.0 | 43.3 | 72.8 | 73.0 | 71.9 |
Scenario-targeted Web Search Benchmarks
| Backbone | HLE Pass@1 |
DeepResearch Bench RACE Overall |
Frames Pass@1 |
SEAL-0 Pass@1 |
AcademicBrowse Pass@1 |
|---|---|---|---|---|---|
| General LLMs with tools | |||||
| Qwen3-30B-A3B | 13.2 | 40.2 | 56.4 | 9.9 | 41.3 |
| Qwen3-235B-A22B | 20.0 | 44.8 | – | 14.4 | 50.7 |
| DeepSeek-R1 | 24.8† | – | 82.0† | 29.7† | – |
| Claude-4-Sonnet | 20.3† | – | 80.7† | – | – |
| Commercial Deep Research Agents | |||||
| Grok Deeper Search | – | 38.2† | – | – | – |
| Perplexity Deep Research | 21.1† | 40.5† | – | – | – |
| Gemini Deep Research | 26.9† | 49.7† | – | – | – |
| Kimi-Researcher | 26.9† | 44.6† | 78.8† | 36.0† | – |
| OpenAI-o3 | 20.2† | – | 84.0† | – | – |
| OpenAI Deep Research | 26.6† | 46.5† | – | – | – |
| Open-source Deep Research Agents | |||||
| ASearcher-Web-QwQ | 12.5† | – | 70.9† | – | – |
| DeepDive-32B | – | – | 76.1† | 29.3† | – |
| MiroThinker-32B-DPOv0.2 | 17.8† | – | 74.8† | – | – |
| WebExplorer-8B | 17.3† | – | 75.7† | – | – |
| Kimi-K2 | 18.1† | 25.4 | 72.0† | 25.2 | 48.7 |
| GLM-4.5 | 21.2† | 39.2 | 78.9† | 34.2 | 55.6 |
| DeepSeek-V3.1 | 29.8† | 35.4 | 83.7† | 42.6† | 65.0 |
| Ours | |||||
| AgentFounder-30B | 31.5 | 48.9 | 89.6 | 43.9 | 75.3 |
Data Scaling
We are excited to observe that as the training data increases, AgentFounder-30B achieves consistent improvements in average performance across multiple benchmarks, exhibiting characteristics of a potential scaling law.
📚 Citation
If you find our work inspiring, please kindly cite as:
@article{su2025agentfounder,
title={Scaling Agents via Continual Pre-training},
author={Liangcai Su and Zhen Zhang and Guangyu Li and Zhuo Chen and Chenxi Wang and Maojia Song and Xinyu Wang and Kuan Li and Jialong Wu and Xuanzhong Chen and Zile Qiao and Zhongwang Zhang and Huifeng Yin and Shihao Cai and Runnan Fang and Zhengwei Tao and Wenbiao Yin and Chenxiong Qian and Yong Jiang and Pengjun Xie and Fei Huang and Jingren Zhou},
year={2025},
journal={arXiv preprint arXiv:2509.13310},
}