chore: import upstream snapshot with attribution

WebAgent/WebResummer/README.md

@@ -0,0 +1,93 @@
+# ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization
+
+<p align="center">
+  <!-- Placeholder for a logo. You can replace this with your own logo. -->
+  <img src="./assets/resum_icon.png" alt="logo" width="30%"/>
+</p>
+
+![version](https://img.shields.io/badge/version-1.0.0-blue) [![arXiv](https://img.shields.io/badge/arXiv-2509.13313-b31b1b.svg)](https://arxiv.org/abs/2509.13313)
+
+## 🏅 Introduction
+
+![workflow](./assets/motivation.jpg)
+
+We introduce **ReSum**, a novel inference paradigm that enables **unlimited exploration** for web agents. Unlike the classic ReAct paradigm, which appends all interaction history, ReSum periodically compresses conversations into compact, restartable reasoning states. This approach minimizes modifications to ReAct, avoiding additional architectural complexity while ensuring **simplicity, efficiency, and plug-and-play compatibility** with existing agents. The tailored **ReSum-GRPO** algorithm allows for a **～10% performance boost with only 1K training samples** 👑
+
+### Our detailed contributions:
+
+- **ReSumTool-30B**: A specialized summarization model designed to extract key evidence, identify information gaps, and highlight next-step directions for continued exploration. This lightweight, open-source tool enables goal-oriented conversation compression tailored for web search contexts.
+  
+- **ReSum-GRPO**: A tailored algorithm for paradigm adaptation. ReSum-GRPO segments long trajectories and broadcasts trajectory-level advantages across all segments. With just 1K training samples, our ReSum-GRPO-trained WebSailor-30B-A3B achieves **33.3% Pass@1** on BrowseComp-zh and **18.3% on BrowseComp-en**, outperforming most open-source web agents.
+
+
+
+## 🚀 Performance Highlights 
+
+1. **Universal Compatibility**: Direct application of the ReSum paradigm across three WebSailor agents achieves **4.5% average Pass@1 improvement**, demonstrating broad applicability.
+2. **Specialized Summarization**: Our developed ReSumTool-30B achieves performance comparable to significantly larger models like Qwen3-235B and DeepSeek-R1-671B when serving as the summary tool, while maintaining lightweight deployment advantages.
+3. **Effectiveness of Targeted RL Training**: The tailored ReSum-GRPO algorithm achieves higher training rewards and faster convergence compared to standard GRPO. Such paradigm-adapted reinforcement learning enables agents to reach **8.2% absolute improvement**.
+
+
+
+## 🔧 Quick Start 
+
+### Step 0 - Environment 
+
+```
+sglang[all]
+qwen-agent[gui,rag,code_interpreter,mcp]
+```
+
+
+### Step 1 - Prepare models
+
+Running the ReSum inference requires three models: **inference model**, **summary model for `visit`**, and **summary tool** (e.g., ReSumTool-30B). Download these models and set their paths in either `src/run_react.sh` or `src/run_resum.sh`.
+
+> 🎉 Our ReSumTool-30B will be released soon, specializing in conversation summarization for web search contexts.
+
+
+### Step 2 - Prepare dataset 
+
+We provide a sample example.jsonl in `src/eval_data` to prevent test data leakage. Download the following official benchmarks and save them in the same folder with the listed filenames, following the exact JSONL format of example.jsonl:
+
+* browsecomp_en.jsonl
+* browsecomp_zh.jsonl
+* gaia.jsonl
+
+
+### Step 3 - Inference with tools 
+
+Before running the code, obtain valid API keys for search and visit tools. The search tool requires a Google Search API key from [Serper](https://serper.dev/), and the visit tool requires a Jina API key from [Jina](https://r.jina.ai). Configure the keys in the scripts.
+
+With the keys in hand, remember to correctly set the model paths in Step 1. You can run the ReAct inference from `src/run_react.sh` or ReSum inference from `src/run_resum.sh`.
+
+An example command is provided below. Simply set the inference model path, the name of the evaluated dataset, and the output path. Additionally, you can adjust various hyperparameters in the scripts, such as the maximum context (`MAX_CONTEXT`) and the maximum number of tool calls per run (`MAX_LLM_CALL_PER_RUN`).
+
+```shell
+cd src 
+bash run_resum.sh MODEL_PATH browsecomp_zh OUTPUT_PATH
+```
+
+
+### Step 4 - Evaluation 
+
+With the inference files (usually in the format `OUTPUT_PATH/MODEL_NAME_sglang/DATASET/iter*.jsonl`), set up the base folder for these rollouts and the dataset name. You'll need a valid DASHSCOPE API key, as we use Qwen2.5-72B-Instruct as the judging model, which can be replaced with your own model and method.
+
+```shell
+python3 evaluate.py --dataset $DATASET --input_folder $FOLDER
+```
+
+
+
+## 📚 Citation 
+
+If you find ReSum useful, please kindly cite as:
+
+```bibtex
+@article{wu2025resumun,
+   title={ReSum: Unlocking Long-Horizon Search Intelligence via Context Summarization}, 
+   author={Xixi Wu and Kuan Li and Yida Zhao and Liwen Zhang and Litu Ou and Huifeng Yin and Zhongwang Zhang and Yong Jiang and Pengjun Xie and Fei Huang and Minhao Cheng and Shuai Wang and Hong Cheng and Jingren Zhou},
+   year={2025},
+   journal={arXiv preprint arXiv:2509.13313},
+}
+```

WebAgent/WebResummer/src/eval_data/example.jsonl

@@ -0,0 +1,4 @@
+{"question": "Among CS conferences, in 2025, which conference has exactly the same full paper submission deadline and the same CCF rank as IJCAI?", "answer": "SIGIR 2025"} 
+{"question": "Between 1990 and 1994 (Inclusive), what teams played in a soccer match with a Brazilian referee had four yellow cards, two for each team where three of the total four were not issued during the first half, and four substitutions, one of which was for an injury in the first 25 minutes of the match.", "answer": "Ireland v Romania"}
+{"question": "In Audre Lorde's poem 'Father Son and Holy Ghost', what is the number of the stanza in which some lines are indented?", "answer": "2"}
+{"question": "How many YouTube music videos have surpassed 7 billion views?", "answer": "5"}

WebAgent/WebResummer/src/evaluate.py

@@ -0,0 +1,309 @@
+import argparse 
+from concurrent.futures import ThreadPoolExecutor, as_completed
+from judge_prompt import JUDGE_PROMPT_GAIA, JUDGE_PROMPT_BC_en, JUDGE_PROMPT_BC_zh
+import os 
+import json 
+import glob
+from collections import defaultdict
+from tqdm import tqdm 
+import time
+from transformers import AutoTokenizer 
+# TODO: Replace with your own DashScope API key
+import dashscope
+dashscope.api_key = 'YOUR_DASHSCOPE_API_KEY'
+
+
+def call_llm_judge(item, max_retries=10): 
+    """Judge if predicted answer matches ground-truth""" 
+    for _ in range(max_retries): 
+        try: 
+            question = item.get(args.question_key, "")
+            correct_answer = item.get(args.answer_key, "") 
+            response = item.get(args.prediction_key, "")
+ 
+            judge_prompt = JUDGE_PROMPT_GAIA
+            if args.dataset.startswith("browsecomp_zh"):
+                judge_prompt = JUDGE_PROMPT_BC_zh
+            elif args.dataset.startswith("browsecomp_en"):
+                judge_prompt = JUDGE_PROMPT_BC_en
+            
+            prompt = judge_prompt.format(question=question, correct_answer=correct_answer, response=response)
+        
+            response = dashscope.Generation.call(
+                model='qwen2.5-72b-instruct',
+                messages=[{"role": "user", "content": prompt}],
+            )
+            judgement = response.output.text
+            judgement = "Correct" if judgement[:1] in ["a", "A"]  else judgement
+       
+            if judgement == "Correct" and args.print_correct_question:
+                print("Correct Question: ", question, "Prediction ", item.get(args.prediction_key, ""), "Ground-truth", correct_answer, "\n")
+
+            return {
+                "question": question, 
+                "answer": correct_answer, 
+                "judgement": judgement, 
+            }
+    
+        except Exception as e:
+            time.sleep(1)
+            if _ == max_retries - 1: 
+                print(f"Error judgement for question: {question}: {e}")
+                return {
+                    "question": question,  
+                    "answer": correct_answer, 
+                    "judgement": "Error",
+                    "error": str(e),
+                 }
+
+
+def single_round_statistics(input_file, available_tools=None):  
+    """Calculate statistics for a single round"""
+    def avg_statistic(value_list):
+        if value_list:
+            return sum(value_list) / len(value_list)
+        return 0 
+    
+    try:
+        with open(input_file, 'r', encoding='utf-8') as f: 
+            samples = [json.loads(line) for line in f]
+    except Exception as e:
+        print(f"Error loading file {input_file}: {e}")
+        return {}
+
+    num_invalid = 0 
+    tool_invocation = defaultdict(list)
+    answer_lengths, traj_lengths = [], []
+    
+    try:
+        tokenizer = AutoTokenizer.from_pretrained("/path/to/your/Qwen2.5-72B-Instruct")
+    except Exception as e: 
+        import tiktoken
+        tokenizer = tiktoken.encoding_for_model("gpt-4o")
+    
+    for sample in samples:
+        msgs = sample.get("messages", [])
+        final_msg = msgs[-1]["content"] if len(msgs) else "" 
+
+        if "<answer>" not in final_msg or "</answer>" not in final_msg:  
+            num_invalid += 1 
+            answer_length = 0 
+        else:
+            answer = final_msg.split("<answer>")[1].split("</answer>")[0].strip()
+            answer_length = len(tokenizer.encode(answer))
+        answer_lengths.append(answer_length)
+        
+        cur_tool_invocation = defaultdict(int)
+        for msg in msgs:
+            if msg["role"] == "assistant": 
+                try:
+                    tool_call = msg["content"].split("<tool_call>")[1].split("</tool_call>")[0].strip() 
+                    tool_call = json.loads(tool_call) 
+                    tool_name = tool_call["name"] 
+                    if available_tools and tool_name in available_tools:
+                        cur_tool_invocation[tool_name] += 1 
+                    else:
+                        cur_tool_invocation["invalid"] += 1 
+                    cur_tool_invocation["total"] += 1 
+                except:
+                    continue 
+        
+        for k, v in cur_tool_invocation.items(): 
+            tool_invocation[k].append(v)
+        
+        traj_length = len(tokenizer.encode("".join(msg["content"] for msg in msgs)))
+        traj_lengths.append(traj_length)
+    
+    metrics = {
+        "num_invalid": num_invalid,  
+        "avg_answer_length": avg_statistic(answer_lengths), 
+        "avg_traj_length": avg_statistic(traj_lengths)
+    }
+
+    for k, v in tool_invocation.items(): 
+        if k != "invalid":
+            metrics[f"avg_tool_{k}"] = avg_statistic(v)
+        else:
+            metrics[f"avg_tool_invalid"] = sum(v) / len(samples)
+
+    return metrics
+
+
+def process_one_prediction(prediction_file): 
+    try:
+        iteration_name = prediction_file.split("/")[-1].replace(".jsonl", "")
+        
+        # Check if the scored file exists
+        scored_file = prediction_file.replace(".jsonl", "_scored.jsonl")
+        if os.path.exists(scored_file):
+            print(f"Found existing scored file for {iteration_name}, loading results...")
+            with open(scored_file, 'r', encoding='utf-8') as f:
+                scored_items = [json.loads(line) for line in f]
+            
+            correct_predictions = []
+            score_dict = defaultdict(bool)
+            
+            for scored_item in scored_items:
+                if scored_item.get("is_correct", False):
+                    correct_predictions.append({
+                        "question": scored_item["question"], 
+                        "answer": scored_item["answer"], 
+                    })
+                score_dict[scored_item["question"]] = scored_item.get("is_correct", False)
+            
+            acc = round(len(correct_predictions) / len(scored_items) * 100, 2)
+            print(f"Loaded scored file: {scored_file} has {len(correct_predictions)} correct predictions (total {len(scored_items)}). Pass@1 {acc}%")
+            
+            return {
+                "file": prediction_file,
+                "accuracy": acc,
+                "correct_count": len(correct_predictions),
+                "total_count": len(scored_items),
+                "correct_predictions": correct_predictions,
+                "score_dict": score_dict
+            }
+        
+        # If the scored file does not exist, score the predictions
+        with open(prediction_file, 'r') as file: 
+            predictions = [json.loads(line) for line in file]
+        
+        correct_predictions, score_dict = [], defaultdict(bool)
+        judgement_results = []
+        
+        with ThreadPoolExecutor(max_workers=args.max_workers) as executor: 
+            futures = [executor.submit(call_llm_judge, item) for item in predictions]
+            for future in tqdm(as_completed(futures), desc=f"Judging {iteration_name}", total=len(futures)): 
+                result = future.result()
+                judgement_results.append(result)
+                
+                if result["judgement"] == "Correct":
+                    correct_predictions.append({
+                        "question": result["question"], 
+                        "answer": result["answer"], 
+                    })
+                
+                score_dict[result["question"]] = result["judgement"] == "Correct"
+        
+        acc = round(len(correct_predictions) / len(predictions) * 100, 2)
+        print(f"Prediction file: {prediction_file} has {len(correct_predictions)} correct predictions (total {len(predictions)}). Pass@1 {acc}%")
+        
+        # Save scored results
+        if not os.path.exists(scored_file):
+            print(f"Saving scored results for {iteration_name}...")
+            with open(scored_file, 'w', encoding='utf-8') as f:
+                for judgement_result in judgement_results:
+                    orig_item = next((item for item in predictions if item["question"] == judgement_result["question"]), None) 
+                    
+                    save_item = orig_item.copy()
+                    save_item["is_correct"] = judgement_result["judgement"] == "Correct"
+                    save_item["origin_judgement"] = judgement_result["judgement"]
+                    
+                    if "error" in judgement_result:
+                        save_item["error"] = judgement_result["error"]
+                    f.write(json.dumps(save_item, ensure_ascii=False) + '\n')
+        
+        return {
+            "file": prediction_file,
+            "accuracy": acc,
+            "correct_count": len(correct_predictions),
+            "total_count": len(predictions),
+            "correct_predictions": correct_predictions,
+            "score_dict": score_dict
+        }
+    except Exception as e:
+        print(f"Error processing file {prediction_file}: {e}")
+        return {
+            "file": prediction_file,
+            "error": str(e)
+        }
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--input_folder")
+    parser.add_argument("--question_key", type=str, default="question")
+    parser.add_argument("--answer_key", type=str, default="answer")
+    parser.add_argument("--prediction_key", type=str, default="prediction")
+    parser.add_argument("--print_correct_question", action="store_true")
+    parser.add_argument("--dataset", type=str, default="gaia")
+    parser.add_argument("--available_tools", type=str, default="search,visit")
+    parser.add_argument("--max_workers", type=int, default=5)
+    parser.add_argument("--restore_result_path", default='output/summary.jsonl', help="record result")
+    args = parser.parse_args()
+    
+    available_tools = args.available_tools.split(",") if args.available_tools else ["search", "visit"]
+    
+    all_scores_dict = defaultdict(list)
+    acc_list = []
+    file_list = []
+    
+    for path in glob.glob(os.path.join(args.input_folder, "iter*.jsonl")):
+        if "_scored" in path:
+            continue 
+
+        result = process_one_prediction(path) 
+        if "error" not in result:
+            for question, score in result["score_dict"].items():
+                all_scores_dict[question].append(score)
+            acc_list.append(result["accuracy"])
+            file_list.append(path)
+    
+    if not acc_list:
+        print("No valid results found!")
+        exit(1)
+    
+    # Compute Average Pass@1 
+    avg_pass_at_1 = sum(acc_list) / len(acc_list)
+    print(f"Average Pass@1: {avg_pass_at_1:.2f}")
+
+    # Compute Best Pass@1 
+    best_pass_at_1 = max(acc_list)
+    print(f"Best Pass@1: {best_pass_at_1:.2f}")
+
+    # Compute Pass@k
+    correct_num = 0 
+    for question, scores in all_scores_dict.items():
+        if sum(scores) >= 1:
+            correct_num += 1
+    pass_at_k = correct_num / len(all_scores_dict) * 100
+    print(f"Pass@{len(acc_list)}: {pass_at_k:.2f}")
+    
+    # Calculate statistics
+    print("\n========== Statistics ==========")
+    all_stats = []
+    for file_path in file_list:
+        stats = single_round_statistics(file_path, available_tools)
+        if stats:
+            all_stats.append(stats)
+    
+    if all_stats:
+        # Aggregate statistics
+        avg_stats = {}
+        for key in all_stats[0].keys():
+            avg_stats[key] = round(sum(stats.get(key, 0) for stats in all_stats) / len(all_stats), 2)
+        
+        print(f"# Invalid: {avg_stats.get('num_invalid', 0)}")
+        print(f"Avg. Answer Length: {avg_stats.get('avg_answer_length', 0)}")
+        print(f"Avg. Trajectory Length: {avg_stats.get('avg_traj_length', 0)}")
+        
+        for k, v in avg_stats.items(): 
+            if k.startswith("avg_tool_"): 
+                print(f"{k}: {v}")
+    
+    # Save overall results
+    overall_eval_dict = {
+        "dataset": args.dataset, 
+        "files": file_list,
+        "overall": {
+            "avg_pass_at_1": avg_pass_at_1, 
+            "best_pass_at_1": best_pass_at_1, 
+            "pass_at_k": pass_at_k
+        }, 
+        "individual": {f"iter{i+1}_pass_at_1": acc for i, acc in enumerate(acc_list)},
+        "statistics": avg_stats if all_stats else {}
+    }
+
+    with open(args.restore_result_path, 'a', encoding='utf-8') as jsonl_file:
+        jsonl_file.write(json.dumps(overall_eval_dict, ensure_ascii=False) + '\n')
+    
+    print(f"\nResults saved to {args.restore_result_path}")

WebAgent/WebResummer/src/judge_prompt.py

@@ -0,0 +1,150 @@
+JUDGE_PROMPT_GAIA = """You are an evaluation assistant. Please determine if the predicted answer is equivalent to the labeled answer.
+
+Question: {question}
+
+Labeled Answer: {correct_answer}
+
+Predicted Answer: {response}
+
+Did the model give an answer **equivalent** to the labeled answer? Please respond with "Correct" if they are equivalent, or "Incorrect" if they are not equivalent. Do not include any other text.
+"""
+
+
+JUDGE_PROMPT_BC_zh = """
+请根据给定问题、标准答案和模型预测的答案来评估模型的回答是否正确。您的任务是将结果评定为：【正确】、【错误】。
+
+首先，我们将列出每个评定类别的示例，然后请您对新问题的预测答案进行评定。
+以下是【正确】的答复示例：
+```
+问题：贝拉克·奥巴马的孩子叫什么名字？
+标准答案：玛丽亚·奥巴马和萨莎·奥巴马
+模型预测1：Malia Obama and Sasha Obama
+模型预测2：玛丽亚和萨沙
+模型预测3：大多数人会说是玛丽亚和萨莎，但我不确定，需要再确认
+模型预测4：巴拉克·奥巴马有两个女儿，她们分别是玛丽亚·安和娜塔莎·玛丽安，但通常称作玛丽亚·奥巴马和萨莎·奥巴马。
+```
+这些答复均为【正确】，因为：
+    - 完整地包含了标准答案中的重要信息。
+    - 不包含任何与标准答案矛盾的信息。
+    - 只关注语义内容，中英文，大小写、标点、语法和顺序不重要。
+    - 答复中出现模糊语句或猜测是可以接受的，前提是包含了标准答案且不含有不正确信息或矛盾。
+
+以下是【错误】的答复示例：
+```
+问题：巴拉克·奥巴马的孩子叫什么名字？
+标准答案：玛丽亚·奥巴马和萨莎·奥巴马
+模型预测1：玛丽亚
+模型预测2：玛丽亚、萨莎和苏珊和萨莎·奥巴马或玛丽亚·奥巴马，或娜塔莎·玛丽安，或爱因斯坦
+模型预测3：虽然我不知道他们的确切名字，但能说出巴拉克·奥巴马有两个孩子。
+模型预测4：你可能是想说贝茜和奥利维亚。不过您应通过最新的参考资料确认详细信息。那是正确的答案吗？
+模型预测5：巴拉克·奥巴马的孩子
+```
+这些答复均为【错误】，因为：
+    - 答复中包含与标准答案矛盾的事实陈述。
+    - 答案为空、重复表述问题。
+    - 答案枚举了多个答案，重复表述答案。
+
+需要格外注意的是：
+- 标准答案中包含对于问题中多个方面的回答，并且在同一个方面的答案中可能会有多种不同的描述，这些描述均是正确的，并且在同一个括号中给出，通过逗号连接。例如，考虑问题"抖音自己的人工智能大模型叫什么名字？"，标准答案为"【【豆包，云雀】】"：
+    - 预测答案"豆包"、"豆包、云雀"、"云雀"等均为【正确】。
+- 对于标准答案中包含的不同方面的回答，模型需要同时给出所有方面的回答才可以算是正确，否则直接判断为【错误】，不存在【部分正确】这种输出方式，这些答案会在不同的括号中给出。例如，考虑问题"TFBOYS组合中的成员有哪些？"，标准答案为"【【王俊凯】【王源】【易洋千玺】】"：
+    - 预测答案"王俊凯、王源、易洋千玺"等同时包含所有答案，才可以算为【正确】。
+    - 预测答案为"王俊凯、易洋千玺"等没有同时包含所有答案，会被算为【错误】。
+
+另外注意以下几点：
+- 对于标准答案为数字的问题，预测答案应和标准答案一致。例如，考虑问题"金山铁路黄浦江特大桥的全长是多少米？"，标准答案为"3518.17"：
+    - 预测答案"3518"、"3518.1"、"3518.17"均为【正确】。
+    - 预测答案"3520"和"3600"均为【错误】。
+- 如果模型预测并没有直接回答问题，模型试图绕过或未能直接给出标准答案视为【错误】答案。
+    - 例如：问题"林宥嘉的老婆是谁"，标准答案为"丁文琪"。模型预测"林宥嘉的老婆"、"林宥嘉的老婆应该很优秀"、"林宥嘉的老婆可能是某个公众人物"均为【错误】。
+- 如果标准答案包含比问题更多的信息，预测答案只需包含问题中提到的信息。
+    - 例如，考虑问题"菱镁矿的主要化学成分是什么？"标准答案为"碳酸镁（MgCO3）"。"碳酸镁"或"MgCO3"均视为【正确】答案。
+- 如果从问题中明显可以推断出预测答案省略的信息，那么算作正确。
+    - 例如，问题"巴鲁米尼的努拉吉遗迹在1997年被联合国教科文组织列为世界文化遗产，那么这遗址在哪个地区？"标准答案为"意大利撒丁岛"，预测答案"撒丁岛"被视为【正确】。
+- 如果能明显看出名字翻译版本不同但是是同一个人也认为正确。
+    - 例如，如果标准答案是"Robinson"，那么回答鲁滨逊或者鲁滨孙均正确。
+- 你应该更关注标准答案和模型预测的匹配度，而不是关心标准答案是否是正确的。
+
+下面是一个新的问题示例。请只回复【正确】、【错误】之一，不要道歉或纠正自己的错误，只需要评估该回答。
+```
+问题: {question}
+标准答案: {correct_answer}
+预测答案: {response}
+```
+
+将此新问题的预测答案评定为以下之一：
+A.【正确】
+B.【错误】
+
+只返回【正确】、【错误】所代表的选项即可，即仅返回A或B即可，无须添加任何其他的文本。
+""".strip()
+
+
+JUDGE_PROMPT_BC_en = """
+Based on the given question, standard answer, and model-predicted answer, evaluate whether the model's response is correct. Your task is to classify the result as: [CORRECT] or [INCORRECT].
+
+First, we'll list examples for each category, then you'll evaluate a new question's predicted answer.
+Here are examples of [CORRECT] responses:
+```
+Question: What are the names of Barack Obama's children?
+Standard Answer: Malia Obama and Sasha Obama
+Model Prediction 1: Malia Obama and Sasha Obama
+Model Prediction 2: Malia and Sasha
+Model Prediction 3: Most would say Malia and Sasha, but I'm not sure, I should verify
+Model Prediction 4: Barack Obama has two daughters, Malia Ann and Natasha Marian, commonly known as Malia Obama and Sasha Obama.
+```
+These responses are all [CORRECT] because they:
+    - Fully include the important information from the standard answer.
+    - Don't contain any information that contradicts the standard answer.
+    - Focus only on semantic content; language, capitalization, punctuation, grammar, and order aren't important.
+    - Vague statements or guesses are acceptable as long as they include the standard answer and don't contain incorrect information or contradictions.
+
+Here are examples of [INCORRECT] responses:
+```
+Question: What are the names of Barack Obama's children?
+Standard Answer: Malia Obama and Sasha Obama
+Model Prediction 1: Malia
+Model Prediction 2: Malia, Sasha and Susan or Sasha Obama or Malia Obama, or Natasha Marian, or Einstein
+Model Prediction 3: While I don't know their exact names, I can tell you Barack Obama has two children.
+Model Prediction 4: You might be thinking of Betsy and Olivia. But you should verify the details with the latest references. Is that the correct answer?
+Model Prediction 5: Barack Obama's children
+```
+These responses are all [INCORRECT] because they:
+    - Contain factual statements that contradict the standard answer.
+    - Are empty or merely repeat the question.
+    - Enumerate multiple answers or repeat the answer.
+
+Pay special attention to the following:
+- The standard answer may contain responses to multiple aspects of the question, and within the same aspect, there might be different descriptions, all of which are correct and are given in the same bracket, connected by commas. For example, for the question "What is the name of ByteDance's AI model?", the standard answer is "[[Doubao, Skylark]]":
+    - Predicted answers "Doubao", "Doubao, Skylark", "Skylark", etc. are all [CORRECT].
+- For standard answers containing responses to different aspects, the model needs to provide answers to all aspects to be considered correct; otherwise, it's directly judged as [INCORRECT]. There is no [PARTIALLY CORRECT] output option. These answers will be given in different brackets. For example, for the question "Who are the members of TFBOYS?", the standard answer is "[[Wang Junkai][Wang Yuan][Yi Yangqianxi]]":
+    - Predicted answers like "Wang Junkai, Wang Yuan, Yi Yangqianxi" that include all answers are [CORRECT].
+    - Predicted answers like "Wang Junkai, Yi Yangqianxi" that don't include all answers are [INCORRECT].
+
+Also note the following points:
+- For questions with numerical standard answers, the predicted answer should match the standard answer. For example, for the question "What is the total length in meters of the Huangpu River Bridge on the Jinshan Railway?", the standard answer is "3518.17":
+    - Predicted answers "3518", "3518.1", "3518.17" are all [CORRECT].
+    - Predicted answers "3520" and "3600" are [INCORRECT].
+- If the model prediction doesn't directly answer the question, attempts to circumvent or fails to directly provide the standard answer, it's considered an [INCORRECT] answer.
+    - For example, for the question "Who is JJ Lin's wife?", with the standard answer "Ding Wenqi", model predictions like "JJ Lin's wife", "JJ Lin's wife should be excellent", "JJ Lin's wife might be a public figure" are all [INCORRECT].
+- If the standard answer contains more information than the question asks for, the predicted answer only needs to include the information mentioned in the question.
+    - For example, for the question "What is the main chemical component of magnesite?", with the standard answer "Magnesium carbonate (MgCO3)", "Magnesium carbonate" or "MgCO3" are both considered [CORRECT] answers.
+- If information omitted in the predicted answer can be clearly inferred from the question, it's considered correct.
+    - For example, for the question "The Nuragic ruins of Barumini were listed as a World Cultural Heritage by UNESCO in 1997, so where is this site located?", with the standard answer "Sardinia, Italy", the predicted answer "Sardinia" is considered [CORRECT].
+- If it's clear that different translations of a name refer to the same person, it's considered correct.
+    - For example, if the standard answer is "Robinson", answers like "Lubinson" or "Lubinsun" are both correct.
+- You should focus more on the match between the standard answer and the model prediction, rather than whether the standard answer itself is correct.
+
+Below is a new question example. Please reply with only [CORRECT] or [INCORRECT], without apologies or corrections to your own errors, just evaluate the answer.
+```
+Question: {question}
+Standard Answer: {correct_answer}
+Predicted Answer: {response}
+```
+
+Evaluate this new question's predicted answer as one of the following:
+A. [CORRECT]
+B. [INCORRECT]
+
+Return only the option representing [CORRECT] or [INCORRECT], i.e., just return A or B, without adding any other text.
+""".strip()

WebAgent/WebResummer/src/main.py

@@ -0,0 +1,164 @@
+import argparse
+import json
+import os
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from tqdm import tqdm
+import threading
+from react_agent import MultiTurnReactAgent
+from prompt import SYSTEM_PROMPT
+from tool_search import *
+from tool_visit import * 
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--model", type=str, default="")
+    parser.add_argument("--output", type=str, default="")
+    parser.add_argument("--dataset", type=str, default="gaia")
+    parser.add_argument("--temperature", type=float, default=0.85)
+    parser.add_argument("--top_p", type=float, default=0.95)
+    parser.add_argument("--max_workers", type=int, default=20)
+    parser.add_argument("--roll_out_count", type=int, default=3)
+    parser.add_argument("--summary_iteration", type=int, default=1000)
+    args = parser.parse_args()
+
+    model = args.model
+    output_base = args.output
+    roll_out_count = args.roll_out_count
+
+    # Parse model name (the part after the last / in the path)
+    model_name = os.path.basename(model.rstrip('/'))
+    
+    # Create output directory structure: output_base/model_name_sglang/dataset_name/
+    model_dir = os.path.join(output_base, f"{model_name}_sglang")
+    dataset_dir = os.path.join(model_dir, args.dataset)
+    
+    # Create directories
+    os.makedirs(dataset_dir, exist_ok=True)
+    
+    print(f"Model name: {model_name}")
+    print(f"Dataset name: {args.dataset}")
+    print(f"Output directory: {dataset_dir}")
+    print(f"Rollout count: {roll_out_count}")
+
+    data_filepath = f"eval_data/{args.dataset}.jsonl"
+    try:
+        if data_filepath.endswith(".json"):
+            with open(data_filepath, "r", encoding="utf-8") as f:
+                items = json.load(f)
+            if not isinstance(items, list):
+                raise ValueError("Input JSON must be a list of objects.")
+            if items and not isinstance(items[0], dict):
+                raise ValueError("Input JSON list items must be objects.")
+        elif data_filepath.endswith(".jsonl"):
+            with open(data_filepath, "r", encoding="utf-8") as f:
+                items = [json.loads(line) for line in f]
+        else:
+            raise ValueError("Unsupported file extension. Please use .json or .jsonl files.")
+        items = items
+    except FileNotFoundError:
+        print(f"Error: Input file not found at {data_filepath}")
+        exit(1)
+    except (json.JSONDecodeError, ValueError) as e:
+        print(f"Error reading or parsing input file {data_filepath}: {e}")
+        exit(1)
+    
+    tasks_to_run = []
+    for rollout_idx in range(1, roll_out_count + 1):
+        output_file = os.path.join(dataset_dir, f"iter{rollout_idx}.jsonl")
+        print(f"\nBegin rollout {rollout_idx}/{roll_out_count}")
+        base_num = len(tasks_to_run)
+
+        processed_queries = set()
+        if os.path.exists(output_file):
+            try:
+                with open(output_file, "r", encoding="utf-8") as f:
+                    for line in f:
+                        try:
+                            data = json.loads(line)
+                            # Check for successful completion based on absence of top-level error key
+                            if "question" in data and "error" not in data:
+                                processed_queries.add(data["question"].strip())
+                        except json.JSONDecodeError:
+                            print(f"Warning: Skipping invalid line in output file: {line.strip()}")
+            except FileNotFoundError:
+                pass
+
+        for item in items:
+            question = item.get("question", "").strip()
+            if not question:
+                print(f"Warning: Skipping item with empty question: {item}")
+                continue
+
+            if question not in processed_queries:
+                tasks_to_run.append({"item": item.copy(), "rollout_id": rollout_idx})
+            else:
+                print(f"Skipping already processed question: {question}")
+
+        print(f"Total questions in input: {len(items)}")
+        print(f"Already successfully processed: {len(processed_queries)}")
+        print(f"Total tasks to run for rollout {rollout_idx}: {len(tasks_to_run) - base_num}")
+
+    if not tasks_to_run:
+        print(f"All rollouts completed, no need to execute")
+    else:
+        llm_cfg = {
+            'model': model,
+            'generate_cfg': {
+                'max_input_tokens': 320000,
+                'max_retries': 10, 
+                'temperature': args.temperature, 
+                'top_p': args.top_p
+            }, 
+            'model_type': 'qwen_dashscope'
+        }
+        
+        test_agent = MultiTurnReactAgent(
+            llm=llm_cfg,
+            function_list=["search", "visit"],
+            system_message=SYSTEM_PROMPT
+        )
+
+        write_locks = {i: threading.Lock() for i in range(1, roll_out_count + 1)}
+
+        with ProcessPoolExecutor(max_workers=args.max_workers) as executor:
+            # Submit tasks
+            future_to_task = {
+                executor.submit(
+                    test_agent._run,
+                    task,
+                    model,
+                    args.summary_iteration,
+                ): task
+                for task in tasks_to_run
+            }
+
+            for future in tqdm(as_completed(future_to_task), total=len(tasks_to_run), desc=f"Processing Rollout {rollout_idx}"):
+                task_info = future_to_task[future]
+                rollout_idx = task_info["rollout_id"]
+                output_file = os.path.join(dataset_dir, f"iter{rollout_idx}.jsonl")
+                try:
+                    result = future.result(timeout=3600)
+                    with write_locks[rollout_idx]:
+                        with open(output_file, "a", encoding="utf-8") as f:
+                            f.write(json.dumps(result, ensure_ascii=False) + "\n")
+                except Exception as exc:
+                    print(f'Task for question "{task_info["item"]["question"]}" (Rollout {task_info["rollout_id"]}) generated an exception: {exc}')
+                    # Log error to the output file
+                    error_result = {
+                        "question": task_info["item"]["question"],
+                        "answer": task_info["item"].get("answer", ""),
+                        "rollout_id": task_info["rollout_id"],
+                        "error": f"Future resolution failed: {exc}",
+                        "messages": [],
+                        "prediction": "[Failed]",
+                    }
+                    print("===============================")
+                    print(error_result)
+                    print("===============================")
+                    
+                    with write_locks[rollout_idx]:
+                        with open(output_file, "a", encoding="utf-8") as f:
+                            f.write(json.dumps(error_result, ensure_ascii=False) + "\n")
+
+    print(f"\nAll {roll_out_count} rollouts completed!")

WebAgent/WebResummer/src/prompt.py

@@ -0,0 +1,169 @@
+EXTRACTOR_PROMPT = """Please process the following webpage content and user goal to extract relevant information:
+
+## **Webpage Content** 
+{webpage_content}
+
+## **User Goal**
+{goal}
+
+## **Task Guidelines**
+1. **Content Scanning for Rational**: Locate the **specific sections/data** directly related to the user's goal within the webpage content
+2. **Key Extraction for Evidence**: Identify and extract the **most relevant information** from the content, you never miss any important information, output the **full original context** of the content as far as possible, it can be more than three paragraphs.
+3. **Summary Output for Summary**: Organize into a concise paragraph with logical flow, prioritizing clarity and judge the contribution of the information to the goal.
+
+**Final Output Format using JSON format has "rational", "evidence", "summary" feilds**
+"""
+
+
+SYSTEM_PROMPT = """You are a Web Information Seeking Master. Your task is to thoroughly seek the internet for information and provide accurate answers to questions. No matter how complex the query, you will not give up until you find the corresponding information.
+
+As you proceed, adhere to the following principles:
+
+1. **Persistent Actions for Answers**: You will engage in many interactions, delving deeply into the topic to explore all possible aspects until a satisfactory answer is found.
+
+2. **Repeated Verification**: Before presenting a Final Answer, you will **cross-check** and **validate the information** you've gathered to confirm its accuracy and reliability.
+
+3. **Attention to Detail**: You will carefully analyze each information source to ensure that all data is current, relevant, and from credible origins.
+
+You will engage in a conversation between User and Assistant. The user asks a question, and the assistant solves it by calling one or more of the following tools:
+
+<tools>
+{
+  "name": "search",
+  "description": "Performs batched web searches: supply an array 'query'; the tool retrieves the top 10 results for each query in one call.",
+  "parameters": {
+    "type": "object",
+    "properties": {
+      "query": {
+        "type": "array",
+        "items": {
+          "type": "string"
+        },
+        "description": "Array of query strings. Include multiple complementary search queries in a single call."
+      }
+    },
+    "required": [
+      "query"
+    ]
+    }
+},
+{
+  "name": "visit",
+    "description": "Visit webpage(s) and return the summary of the content.",
+    "parameters": {
+        "type": "object",
+        "properties": {
+            "url": {
+                "type": "array",
+                "items": {"type": "string"},
+                "description": "The URL(s) of the webpage(s) to visit. Can be a single URL or an array of URLs."
+            },
+            "goal": {
+                "type": "string",
+                "description": "The specific information goal for visiting webpage(s)."
+            }
+        },
+        "required": [
+            "url",
+            "goal"
+        ]
+    }
+}
+</tools>
+
+The assistant starts with one or more cycles of (thinking about which tool to use -> performing tool call -> waiting for tool response), and ends with (thinking about the answer -> answer of the question). The thinking processes, tool calls, tool responses, and answer are enclosed within their tags. There could be multiple thinking processes, tool calls, tool call parameters and tool response parameters.
+
+Example response:
+<think> thinking process here </think>
+<tool_call>
+{"name": "tool name here", "arguments": {"parameter name here": parameter value here, "another parameter name here": another parameter value here, ...}}
+</tool_call>
+<tool_response>
+tool_response here
+</tool_response>
+<think> thinking process here </think>
+<tool_call>
+{"name": "another tool name here", "arguments": {...}}
+</tool_call>
+<tool_response>
+tool_response here
+</tool_response>
+(more thinking processes, tool calls and tool responses here)
+<think> thinking process here </think>
+<answer> answer here </answer>"""
+
+
+
+QUERY_SUMMARY_PROMPT = """You are an expert at analyzing conversation history and extracting relevant information. Your task is to thoroughly evaluate the conversation history and current question to provide a comprehensive summary that will help answer the question.
+
+Task Guidelines 
+1. Information Analysis:
+   - Carefully analyze the conversation history to identify truly useful information.
+   - Focus on information that directly contributes to answering the question.
+   - Do NOT make assumptions, guesses, or inferences beyond what is explicitly stated in the conversation.
+   - If information is missing or unclear, do NOT include it in your summary.
+
+2. Summary Requirements:
+   - Extract only the most relevant information that is explicitly present in the conversation.
+   - Synthesize information from multiple exchanges when relevant.
+   - Only include information that is certain and clearly stated in the conversation.
+   - Do NOT output or mention any information that is uncertain, insufficient, or cannot be confirmed from the conversation.
+
+3. Output Format: Your response should be structured as follows:
+<summary>
+- Essential Information: [Organize the relevant and certain information from the conversation history that helps address the question.]
+</summary>
+
+Strictly avoid fabricating, inferring, or exaggerating any information not present in the conversation. Only output information that is certain and explicitly stated.
+
+Question
+{{{question}}} 
+
+Conversation History
+{{{recent_history_messages}}}
+
+Please generate a comprehensive and useful summary. Note that you are not permitted to invoke tools during this process.
+"""
+
+
+QUERY_SUMMARY_PROMPT_LAST = """You are an expert at analyzing conversation history and extracting relevant information. Your task is to thoroughly evaluate the conversation history and current question to provide a comprehensive summary that will help answer the question.
+
+The last summary serves as your starting point, marking the information landscape previously collected. Your role is to:
+- Analyze progress made since the last summary
+- Identify remaining information gaps
+- Generate a useful summary that combines previous and new information
+- Maintain continuity, especially when recent conversation history is limited
+
+Task Guidelines
+
+1. Information Analysis:
+   - Carefully analyze the conversation history to identify truly useful information.
+   - Focus on information that directly contributes to answering the question.
+   - Do NOT make assumptions, guesses, or inferences beyond what is explicitly stated.
+   - If information is missing or unclear, do NOT include it in your summary.
+   - Use the last summary as a baseline when recent history is sparse.
+
+2. Summary Requirements:
+   - Extract only the most relevant information that is explicitly present in the conversation.
+   - Synthesize information from multiple exchanges when relevant.
+   - Only include information that is certain and clearly stated.
+   - Do NOT output or mention any information that is uncertain, insufficient, or cannot be confirmed.
+
+3. Output Format: Your response should be structured as follows:
+<summary>
+- Essential Information: [Organize the relevant and certain information from the conversation history that helps address the question.]
+</summary>
+
+Strictly avoid fabricating, inferring, or exaggerating any information not present in the conversation. Only output information that is certain and explicitly stated.
+
+Question
+{{{question}}}
+
+Last Summary
+{{{last_summary}}}
+
+Conversation History
+{{{recent_history_messages}}}
+
+Please generate a comprehensive and useful summary. Note that you are not permitted to invoke tools during this process.
+"""

WebAgent/WebResummer/src/react_agent.py

@@ -0,0 +1,202 @@
+import json
+import os
+from typing import Dict, List, Optional, Union
+from qwen_agent.utils.utils import build_text_completion_prompt
+from openai import OpenAI
+import tiktoken
+from transformers import AutoTokenizer 
+from qwen_agent.agents.fncall_agent import FnCallAgent
+from qwen_agent.llm import BaseChatModel
+from qwen_agent.llm.schema import DEFAULT_SYSTEM_MESSAGE, Message
+from qwen_agent.settings import MAX_LLM_CALL_PER_RUN
+from qwen_agent.tools import BaseTool
+from summary_utils import summarize_conversation
+
+
+MAX_LLM_CALL_PER_RUN = int(os.getenv('MAX_LLM_CALL_PER_RUN', 60))
+print(f'Running with MAX_LLM_CALL_PER_RUN = {MAX_LLM_CALL_PER_RUN}')
+RESUM = os.getenv('RESUM', 'False').lower() == 'true'
+print(f'ReSum Mode: {RESUM}')
+MAX_CONTEXT = int(os.getenv('MAX_CONTEXT', 32))
+print(f"Maximum Context: {MAX_CONTEXT}k")
+
+
+class MultiTurnReactAgent(FnCallAgent):
+    def __init__(self,
+                 function_list: Optional[List[Union[str, Dict, BaseTool]]] = None,
+                 llm: Optional[Union[Dict, BaseChatModel]] = None,
+                 system_message: Optional[str] = DEFAULT_SYSTEM_MESSAGE,
+                 name: Optional[str] = None,
+                 description: Optional[str] = None,
+                 files: Optional[List[str]] = None,
+                 **kwargs):
+        super().__init__(function_list=function_list,
+                         llm=llm,
+                         system_message=system_message,
+                         name=name,
+                         description=description,
+                         files=files,
+                         **kwargs)
+        self.llm_generate_cfg = llm["generate_cfg"]
+        self.llm_local_path = llm["model"]
+
+    def call_server(self, msgs, max_tries=10):
+        # Set OpenAI API key and base URL using vLLM API server
+        openai_api_key = "EMPTY"
+        openai_api_base = "http://127.0.0.1:6001/v1"
+
+        client = OpenAI(
+            api_key=openai_api_key,
+            base_url=openai_api_base,
+        )
+
+        for attempt in range(max_tries):
+            try:
+                chat_response = client.chat.completions.create(
+                    model=self.model,
+                    messages=msgs,
+                    stop=["\n<tool_responseF", "<tool_response>"],
+                    temperature=self.llm_generate_cfg.get('temperature', 0.6),
+                    top_p=self.llm_generate_cfg.get('top_p', 0.95),
+                )
+                content = chat_response.choices[0].message.content
+                if content:
+                    return content
+            except Exception as e:
+                if attempt == (max_tries - 1):
+                    print(f"SGLang server error {e}")
+                    return f"SGLang server error"
+                continue
+        
+        return "SGLang server empty response"
+
+    def count_tokens(self, messages, model="gpt-4o"):
+        try: 
+            tokenizer = AutoTokenizer.from_pretrained(self.llm_local_path) 
+        except Exception as e: 
+            tokenizer = tiktoken.encoding_for_model(model)
+        
+        full_message = [Message(**x) for x in messages]
+        full_prompt = build_text_completion_prompt(full_message, allow_special=True)
+        
+        return len(tokenizer.encode(full_prompt))
+
+    def _run(self, data: str, model: str, summary_iteration: int, **kwargs) -> List[List[Message]]:
+        self.model = model
+        question = data['item']['question']
+        answer = data['item']['answer']
+
+        messages = [
+            {"role": "system", "content": self.system_message}, 
+            {"role": "user", "content": question}
+        ]
+        num_llm_calls_available = MAX_LLM_CALL_PER_RUN
+        round = 0
+        last_summary, full_trajectory = None, messages.copy() 
+        while num_llm_calls_available > 0:
+            round += 1
+            num_llm_calls_available -= 1
+            content = self.call_server(messages)
+            print(f'round {round}: {content}')
+
+            if '<tool_response>' in content:
+                pos = content.find('<tool_response>')
+                content = content[:pos]
+
+            messages.append({"role": "assistant", "content": content.strip()})
+            full_trajectory.append({"role": "assistant", "content": content.strip()}) 
+
+            if '<tool_call>' in content and '</tool_call>' in content:
+                tool_call = content.split('<tool_call>')[1].split('</tool_call>')[0]
+                try:
+                    tool_call = json.loads(tool_call)
+                    tool_name = tool_call.get('name', '')
+                    tool_args = tool_call.get('arguments', {})
+                    result = self._call_tool(tool_name, tool_args)
+                    print(f"Tool call {tool_name} invocation success with length {len(result)}")
+                except Exception as e:
+                    print(f"Tool call error: {e}")
+                    result = 'Error: Tool call is not a valid JSON. Tool call must contain a valid "name" and "arguments" field.'
+                result = "<tool_response>" + result + "</tool_response>"
+                messages.append({"role": "user", "content": result})
+                full_trajectory.append({"role": "user", "content": result}) 
+
+            elif '<answer>' in content and '</answer>' in content:
+                answer_content = content.split('<answer>')[1].split('</answer>')[0].strip()
+                if len(answer_content):
+                    termination = 'answer'
+                    prediction = answer_content 
+                    break
+      
+            max_tokens = MAX_CONTEXT * 1024 - 1000
+            token_count = self.count_tokens(messages)
+            print(f"round: {round}, token count: {token_count}")
+
+            should_summarize = ((RESUM and token_count >= max_tokens * 0.9) or round % summary_iteration == 0) and num_llm_calls_available 
+            if should_summarize: 
+                recent_messages = messages[2:].copy() 
+
+                try:
+                    summary_response = summarize_conversation(question, recent_messages, last_summary)
+                    print(f"[Summary Tool] ReSum-Tool-30B Invocation success (len: {len(summary_response)}): {summary_response}")
+                except Exception as e: 
+                    print(f"[Summary Tool] ReSum-Tool-30B Invocation failed: {e}")
+                    summary_response = "" 
+                
+                if summary_response:  
+                    last_summary = summary_response  
+                    new_observation = "Question: " + question + "\nBelow is a summary of the previous conversation. This summary condenses key information from earlier steps, so please consider it carefully. Assess whether the summary provides enough information to answer the question and use it as the basis for further reasoning and information gathering to answer the question.\n" \
+                                     + "Summary: " + summary_response + "\n"
+                    messages = [
+                        {"role": "system", "content": self.system_message}, 
+                        {"role": "user", "content": new_observation}
+                    ]
+                    full_trajectory.append({"role": "user", "content": new_observation})  
+                    token_count = self.count_tokens(messages) 
+                    print(f"round {round}, token count after summary: {token_count}")
+            
+            if num_llm_calls_available <= 0 and '<answer>' not in content:
+                messages[-1]['content'] = 'Sorry, the number of llm calls exceeds the limit.'
+
+            if token_count > max_tokens:
+                print(f"Token count exceeds limit: {token_count} > {max_tokens}")
+                
+                messages[-1]['content'] = "You have now reached the maximum context length you can handle. You should stop invoking tools and, based on all the information above, think again and provide what you consider the most likely answer in the following format: <think> your final thinking </think>\n <answer> your answer </answer>"
+                content = self.call_server(messages)
+
+                messages.append({"role": "assistant", "content": content.strip()})
+                full_trajectory.append({"role": "assistant", "content": content.strip()}) 
+
+                if '<answer>' in content and '</answer>' in content:
+                    prediction = messages[-1]['content'].split('<answer>')[1].split('</answer>')[0]
+                    termination = 'generate an answer as token limit reached'
+                else:
+                    prediction = messages[-1]['content']
+                    termination = 'format error: generate an answer as token limit reached'
+                result = {
+                    "question": question,
+                    "answer": answer,
+                    "rollout_id": data['rollout_id'],
+                    "messages": full_trajectory,
+                    "prediction": prediction,
+                    "termination": termination
+                }
+                return result
+
+        if '<answer>' in messages[-1]['content']:
+            prediction = messages[-1]['content'].split('<answer>')[1].split('</answer>')[0]
+            termination = 'answer'
+        else:
+            prediction = 'No answer found.'
+            termination = 'answer not found'
+            if num_llm_calls_available == 0:
+                termination = 'exceed available llm calls'
+        result = {
+            "question": question,
+            "answer": answer,
+            "rollout_id": data['rollout_id'],
+            "messages": full_trajectory,
+            "prediction": prediction,
+            "termination": termination
+        }
+        return result

WebAgent/WebResummer/src/run_react.sh

@@ -0,0 +1,89 @@
+############## Evaluation Parameters ################
+export MODEL_PATH=$1 # Model path
+
+# Dataset names (strictly match the following names):
+# - gaia
+# - browsecomp_zh (Full set, 289 Cases)
+# - browsecomp_en (Full set, 1266 Cases)
+export DATASET=$2 
+export OUTPUT_PATH=$3 # Output path for prediction results
+
+
+######################################
+##### 0. System Configuration   #####
+######################################
+# Search Tool 
+export GOOGLE_SEARCH_KEY="Your Google Search API Key" 
+
+# Visit Tool 
+export JINA_API_KEYS="Your Jina API Key"
+export SUMMARY_API_KEY="EMPTY"
+export SUMMARY_API_BASE="http://localhost:8001/v1"
+export SUMMARY_MODEL_NAME="/path/to/your/Qwen3-30B-A3B-Instruct-2507"
+
+
+######################################
+### 1. Start server (background)   ###
+######################################
+# Server - default localhost:6001
+CUDA_VISIBLE_DEVICES=0,1,2,3 vllm serve $MODEL_PATH --host 0.0.0.0 --port 6001 --tensor-parallel-size 4 & 
+
+# Summary model 
+CUDA_VISIBLE_DEVICES=4,5,6,7 vllm serve "/path/to/your/Qwen3-30B-A3B-Instruct-2507" --host 0.0.0.0 --port 8001 --tensor-parallel-size 4 & 
+
+
+#####################################
+#### 2. Wait for server ready     ###
+#####################################
+timeout=2400
+sleep_interval=30
+
+check_port() {
+    local port=$1
+    local model_name=$2
+    local start_time=$(date +%s)
+    
+    echo "Wait for $port ($model_name) to start..."
+    while true; do
+        if netstat -tuln 2>/dev/null | grep -q ":$port " || ss -tuln 2>/dev/null | grep -q ":$port "; then
+            if curl -s --connect-timeout 10 --max-time 5 http://localhost:$port/v1/chat/completions > /dev/null 2>&1; then
+                echo "Port $port ($model_name) is ready."
+                return 0
+            fi
+        fi
+    
+        current_time=$(date +%s)
+        elapsed=$((current_time - start_time))
+        if [ $elapsed -gt $timeout ]; then
+            echo "Error: start port $port ($model_name) timeout ($timeout seconds)!"
+            return 1
+        fi
+        echo "Port $port ($model_name) is not started, waiting $sleep_interval seconds to retry... (elapsed ${elapsed} seconds)"
+        sleep $sleep_interval
+    done
+}
+
+check_port 6001 "Infer Model" || exit 1 
+check_port 8001 "Summary Model" || exit 1  
+
+echo "All vLLM services are ready!"
+
+
+#####################################
+### 3. Start inference           ####
+#####################################
+echo "==== Starting inference... ===="
+
+# Todo: Activate inference conda environment
+# Other settings 
+# export MAX_CONTEXT=32
+# export MAX_LLM_CALL_PER_RUN=60 
+
+python3 -u main.py \
+        --dataset $DATASET \
+        --output $OUTPUT_PATH \
+        --max_workers 40 \
+        --model $MODEL_PATH 
+
+echo "==== Inference completed! ===="
+exit 0 

WebAgent/WebResummer/src/run_resum.sh

@@ -0,0 +1,99 @@
+############## Evaluation Parameters ################
+export MODEL_PATH=$1 # Model path
+
+# Dataset names (strictly match the following names):
+# - gaia
+# - browsecomp_zh (Full set, 289 Cases)
+# - browsecomp_en (Full set, 1266 Cases)
+export DATASET=$2 
+export OUTPUT_PATH=$3 # Output path for prediction results
+
+
+######################################
+##### 0. System Configuration   #####
+######################################
+# Search Tool 
+export GOOGLE_SEARCH_KEY="Your Google Search API Key" 
+
+# Visit Tool 
+export JINA_API_KEYS="Your Jina API Key"
+export SUMMARY_API_KEY="EMPTY"
+export SUMMARY_API_BASE="http://localhost:8001/v1"
+export SUMMARY_MODEL_NAME="/path/to/your/Qwen3-30B-A3B-Instruct-2507"
+
+# ReSum Tool 
+export RESUM_TOOL_NAME="/path/to/your/ReSum-Tool-30B-A3B" 
+export RESUM_TOOL_URL="http://localhost:6002/v1/chat/completions" 
+
+
+######################################
+### 1. Start server (background)   ###
+######################################
+# Server - default localhost:6001
+CUDA_VISIBLE_DEVICES=0,1,2,3 vllm serve $MODEL_PATH --host 0.0.0.0 --port 6001 --tensor-parallel-size 4 & 
+
+# Summary model 
+CUDA_VISIBLE_DEVICES=4,5 vllm serve "/path/to/your/Qwen3-30B-A3B-Instruct-2507" --host 0.0.0.0 --port 8001 --tensor-parallel-size 2 & 
+
+# Resum model 
+CUDA_VISIBLE_DEVICES=6,7 vllm serve "/path/to/your/ReSum-Tool-30B-A3B" --host 0.0.0.0 --port 6002 --tensor-parallel-size 2 & 
+
+
+#####################################
+#### 2. Wait for server ready     ###
+#####################################
+timeout=2400
+sleep_interval=30
+
+check_port() {
+    local port=$1
+    local model_name=$2
+    local start_time=$(date +%s)
+    
+    echo "Wait for $port ($model_name) to start..."
+    while true; do
+        if netstat -tuln 2>/dev/null | grep -q ":$port " || ss -tuln 2>/dev/null | grep -q ":$port "; then
+            if curl -s --connect-timeout 10 --max-time 5 http://localhost:$port/v1/chat/completions > /dev/null 2>&1; then
+                echo "Port $port ($model_name) is ready."
+                return 0
+            fi
+        fi
+    
+        current_time=$(date +%s)
+        elapsed=$((current_time - start_time))
+        if [ $elapsed -gt $timeout ]; then
+            echo "Error: start port $port ($model_name) timeout ($timeout seconds)!"
+            return 1
+        fi
+        echo "Port $port ($model_name) is not started, waiting $sleep_interval seconds to retry... (elapsed ${elapsed} seconds)"
+        sleep $sleep_interval
+    done
+}
+
+check_port 6001 "Infer Model" || exit 1 
+check_port 8001 "Summary Model" || exit 1  
+check_port 6002 "ReSum Model" || exit 1  
+
+echo "All vLLM services are ready!"
+
+
+#####################################
+### 3. Start inference           ####
+#####################################
+echo "==== Starting inference... ===="
+
+# Todo: Activate inference conda environment
+
+export RESUM=True
+# Other settings 
+# export MAX_CONTEXT=32
+# export MAX_LLM_CALL_PER_RUN=80 
+
+python3 -u main.py \
+        --dataset $DATASET \
+        --output $OUTPUT_PATH \
+        --max_workers 40 \
+        --model $MODEL_PATH 
+
+echo "==== Inference completed! ===="
+exit 0 

WebAgent/WebResummer/src/summary_utils.py

@@ -0,0 +1,66 @@
+import json 
+import requests 
+from prompt import QUERY_SUMMARY_PROMPT, QUERY_SUMMARY_PROMPT_LAST
+import os 
+import re
+import time 
+
+
+RESUM_TOOL_NAME = os.getenv("RESUM_TOOL_NAME", "")
+RESUM_TOOL_URL = os.getenv("RESUM_TOOL_URL", "") 
+
+
+def call_resum_server(query, max_retries=10):
+    data = {
+        "model": RESUM_TOOL_NAME,
+        "messages": [{ "role": "user",  "content": query}], 
+        "max_tokens": 4096,
+        "temperature": 0.6,
+        "top_p": 0.95,
+    }
+    header = {
+        "Content-Type": "application/json", 
+        "Authorization": ""
+    }
+    
+    for _ in range(max_retries):
+        try: 
+            response = requests.post(RESUM_TOOL_URL, headers=header, json=data, timeout=360)
+            response.raise_for_status()
+            content = response.json()["choices"][0]["message"]["content"]
+            # print(content)
+            if content: 
+                pattern = r'<think>.*?</think>' 
+                content = re.sub(pattern, '', content, flags=re.DOTALL).strip()
+                try:
+                    content = content.split("<summary>")[1].split("</summary>")[0]
+                except:
+                    content = content 
+                return "<summary>" + content + "</summary>"
+            else: 
+                return ""
+        except Exception as e:
+            time.sleep(0.2)
+            if _ == max_retries - 1: 
+                print(f"[Summary Tool] Server attempt {_+1}/{max_retries} failed: {str(e)}")
+                return ""
+    return ""
+
+
+def summarize_conversation(question, recent_history_messages, last_summary, max_retries=10):
+    recent_history_str = "\n".join([str(msg) for msg in recent_history_messages])
+    
+    if not last_summary:
+        query_prompt = QUERY_SUMMARY_PROMPT.replace("{{{question}}}", question).replace("{{{recent_history_messages}}}", recent_history_str)
+    else:
+        query_prompt = QUERY_SUMMARY_PROMPT_LAST.replace("{{{question}}}", question).replace("{{{recent_history_messages}}}", recent_history_str).replace("{{{last_summary}}}", last_summary)
+    
+    response = call_resum_server(query_prompt, max_retries=max_retries)
+    return response
+
+
+###### Test Code ###### 
+if __name__ == "__main__": 
+    query = "Please give me a simple three-day travel plan for Kitakyushu, Japan."
+    print(call_resum_server(query))
+    print(summarize_conversation(query, [], ""))

WebAgent/WebResummer/src/tool_search.py

@@ -0,0 +1,112 @@
+from qwen_agent.tools.base import BaseTool, register_tool 
+import json
+from concurrent.futures import ThreadPoolExecutor
+from typing import List, Union
+import requests
+from qwen_agent.tools.base import BaseTool, register_tool
+import os
+
+
+GOOGLE_SEARCH_KEY = os.getenv("GOOGLE_SEARCH_KEY")
+
+
+@register_tool("search", allow_overwrite=True)
+class Search(BaseTool):
+    name = "search"
+    description = "Performs batched web searches: supply an array 'query'; the tool retrieves the top 10 results for each query in one call."
+    parameters = {
+            "type": "object",
+            "properties": {
+                "query": {
+                    "type": "array",
+                    "items": {
+                    "type": "string"
+                    },
+                    "description": "Array of query strings. Include multiple complementary search queries in a single call."
+                },
+            },
+        "required": ["query"],
+    }
+
+    def google_search(self, query: str):
+        url = 'https://google.serper.dev/search'
+        headers = {
+            'X-API-KEY': GOOGLE_SEARCH_KEY,
+            'Content-Type': 'application/json',
+        }
+        data = {
+            "q": query,
+            "num": 10,
+            "extendParams": {
+                "country": "en",
+                "page": 1,
+            },
+        }
+
+        for i in range(5):
+            try:
+                response = requests.post(url, headers=headers, data=json.dumps(data))
+                results = response.json()
+            except Exception as e:
+                print(e)
+                if i == 4:
+                    return f"Google search Timeout, return None, Please try again later."
+        if response.status_code != 200:
+            raise Exception(f"Error: {response.status_code} - {response.text}")
+
+        try:
+            if "organic" not in results:
+                raise Exception(f"No results found for query: '{query}'. Use a less specific query.")
+
+            web_snippets = list()
+            idx = 0
+            if "organic" in results:
+                for page in results["organic"]:
+                    idx += 1
+                    date_published = ""
+                    if "date" in page:
+                        date_published = "\nDate published: " + page["date"]
+
+                    source = ""
+                    if "source" in page:
+                        source = "\nSource: " + page["source"]
+
+                    snippet = ""
+                    if "snippet" in page:
+                        snippet = "\n" + page["snippet"]
+
+                    redacted_version = f"{idx}. [{page['title']}]({page['link']}){date_published}{source}\n{snippet}"
+
+                    redacted_version = redacted_version.replace("Your browser can't play this video.", "")
+                    web_snippets.append(redacted_version)
+
+            content = f"A Google search for '{query}' found {len(web_snippets)} results:\n\n## Web Results\n" + "\n\n".join(web_snippets)
+            return content
+        except:
+            return f"No results found for '{query}'. Try with a more general query, or remove the year filter."
+
+
+    def call(self, params: Union[str, dict], **kwargs) -> str:
+        assert GOOGLE_SEARCH_KEY is not None, "Please set the GOOGLE_SEARCH_KEY environment variable."
+        try:
+            query = params["query"]
+        except:
+            return "[Search] Invalid request format: Input must be a JSON object containing 'query' field"
+        
+        if isinstance(query, str):
+            response = self.google_search(query)
+        else:
+            assert isinstance(query, List)
+            with ThreadPoolExecutor(max_workers=3) as executor:
+                response = list(executor.map(self.google_search, query))
+            response = "\n=======\n".join(response)
+        return response
+
+
+###### Test Code ###### 
+if __name__ == "__main__": 
+    search = Search() 
+
+    query = ["Hong Kong Actor Tony Leung Chiu wai", "Tokyo Food Restaurant"]
+    result = search.call({"query": query}) 
+    print(result)

WebAgent/WebResummer/src/tool_visit.py

@@ -0,0 +1,240 @@
+import json
+import os
+from typing import List, Union
+import requests
+from qwen_agent.tools.base import BaseTool, register_tool
+from prompt import EXTRACTOR_PROMPT 
+from openai import OpenAI
+import time 
+import tiktoken
+
+
+JINA_API_KEYS = os.getenv("JINA_API_KEYS", "")
+SUMMARY_API_KEY = os.environ.get("SUMMARY_API_KEY", "EMPTY")
+SUMMARY_URL = os.environ.get("SUMMARY_API_BASE")
+SUMMARY_MODEL_NAME = os.environ.get("SUMMARY_MODEL_NAME", "")
+        
+
+@staticmethod
+def truncate_to_tokens(text: str, max_tokens: int = 95000) -> str:
+    encoding = tiktoken.get_encoding("cl100k_base")
+    
+    tokens = encoding.encode(text)
+    if len(tokens) <= max_tokens:
+        return text
+    
+    truncated_tokens = tokens[:max_tokens]
+    return encoding.decode(truncated_tokens)
+
+
+@register_tool('visit', allow_overwrite=True)
+class Visit(BaseTool):
+    # The `description` tells the agent the functionality of this tool.
+    name = 'visit'
+    description = 'Visit webpage(s) and return the summary of the content.'
+    # The `parameters` tell the agent what input parameters the tool has.
+    parameters = {
+        "type": "object",
+        "properties": {
+            "url": {
+                "type": ["string", "array"],
+                "items": {
+                    "type": "string"
+                    },
+                "minItems": 1,
+                "description": "The URL(s) of the webpage(s) to visit. Can be a single URL or an array of URLs."
+        },
+        "goal": {
+                "type": "string",
+                "description": "The goal of the visit for webpage(s)."
+        }
+        },
+        "required": ["url", "goal"]
+    }
+    # The `call` method is the main function of the tool.
+    def call(self, params: Union[str, dict], **kwargs) -> str:
+        try:
+            url = params["url"]
+            goal = params["goal"]
+        except:
+            return "[Visit] Invalid request format: Input must be a JSON object containing 'url' and 'goal' fields"
+
+        if isinstance(url, str):
+            response = self.readpage(url, goal)
+        else:
+            response = []
+            assert isinstance(url, List)
+            start_time = time.time()
+            for u in url: 
+                if time.time() - start_time > 900:
+                    cur_response = "The useful information in {url} for user goal {goal} as follows: \n\n".format(url=url, goal=goal)
+                    cur_response += "Evidence in page: \n" + "The provided webpage content could not be accessed. Please check the URL or file format." + "\n\n"
+                    cur_response += "Summary: \n" + "The webpage content could not be processed, and therefore, no information is available." + "\n\n"
+                else:
+                    try:
+                        cur_response = self.readpage(u, goal)
+                    except Exception as e:
+                        cur_response = f"Error fetching {u}: {str(e)}"
+                response.append(cur_response)
+            response = "\n=======\n".join(response)
+        
+        print(f'Summary Length {len(response)}; Summary Content {response}')
+        return response.strip()
+        
+    def call_server(self, msgs, max_retries=2):
+        client = OpenAI(
+            api_key=SUMMARY_API_KEY,
+            base_url=SUMMARY_URL,
+        )
+        for attempt in range(max_retries):
+            try:
+                chat_response = client.chat.completions.create(
+                    model=SUMMARY_MODEL_NAME,
+                    messages=msgs,
+                    temperature=0.7
+                )
+                content = chat_response.choices[0].message.content
+                print(content)
+                if content:
+                    try:
+                        json.loads(content)
+                    except:
+                        # extract json from string 
+                        left = content.find('{')
+                        right = content.rfind('}') 
+                        if left != -1 and right != -1 and left <= right: 
+                            content = content[left:right+1]
+                    return content
+            except Exception as e:
+                print(e)
+                if attempt == (max_retries - 1):
+                    return ""
+                continue
+
+
+    def jina_readpage(self, url: str) -> str:
+        """
+        Read webpage content using Jina service.
+        
+        Args:
+            url: The URL to read
+            goal: The goal/purpose of reading the page
+            
+        Returns:
+            str: The webpage content or error message
+        """
+        max_retries = 5
+        timeout = 100
+        
+        for attempt in range(max_retries):
+            headers = {
+                "Authorization": f"Bearer {JINA_API_KEYS}",
+            }
+            try:
+                response = requests.get(
+                    f"https://r.jina.ai/{url}",
+                    headers=headers,
+                    timeout=timeout
+                )
+                if response.status_code == 200:
+                    webpage_content = response.text
+                    return webpage_content
+                else:
+                    print(response.text)
+                    raise ValueError("jina readpage error")
+            except Exception as e:
+                time.sleep(0.5)
+                if attempt == max_retries - 1:
+                    return "[visit] Failed to read page."
+                
+        return "[visit] Failed to read page."
+
+    def readpage(self, url: str, goal: str) -> str:
+        """
+        Attempt to read webpage content by alternating between jina and aidata services.
+        
+        Args:
+            url: The URL to read
+            goal: The goal/purpose of reading the page
+            
+        Returns:
+            str: The webpage content or error message
+        """
+        summary_page_func = self.call_server
+    
+        content = self.jina_readpage(url)
+        print("Jina Content", len(content))
+        
+        max_retries = int(os.getenv('VISIT_SERVER_MAX_RETRIES', 3))
+
+        if content and not content.startswith("[visit] Failed to read page.") and content != "[visit] Empty content." and not content.startswith("[document_parser]"):
+            content = truncate_to_tokens(content, max_tokens=95000)
+            messages = [{"role":"user","content": EXTRACTOR_PROMPT.format(webpage_content=content, goal=goal)}]
+            parse_retry_times = 0
+            raw = summary_page_func(messages, max_retries=max_retries)
+            summary_retries = 3
+            while len(raw) < 10 and summary_retries >= 0:
+                truncate_length = int(0.7 * len(content)) if summary_retries > 0 else 25000
+                status_msg = (
+                    f"[visit] Summary url[{url}] " 
+                    f"attempt {3 - summary_retries + 1}/3, "
+                    f"content length: {len(content)}, "
+                    f"truncating to {truncate_length} chars"
+                ) if summary_retries > 0 else (
+                    f"[visit] Summary url[{url}] failed after 3 attempts, "
+                    f"final truncation to 25000 chars"
+                )
+                print(status_msg)
+                content = content[:truncate_length]
+                extraction_prompt = EXTRACTOR_PROMPT.format(
+                    webpage_content=content,
+                    goal=goal
+                )
+                messages = [{"role": "user", "content": extraction_prompt}]
+                raw = summary_page_func(messages, max_retries=max_retries)
+                summary_retries -= 1
+
+            parse_retry_times = 2
+            if isinstance(raw, str):
+                raw = raw.replace("```json", "").replace("```", "").strip()
+            while parse_retry_times < 3:
+                try:
+                    raw = json.loads(raw)
+                    break
+                except:
+                    raw = summary_page_func(messages, max_retries=max_retries)
+                    parse_retry_times += 1
+            
+            if parse_retry_times >= 3:
+                useful_information = "The useful information in {url} for user goal {goal} as follows: \n\n".format(url=url, goal=goal)
+                useful_information += "Evidence in page: \n" + "The provided webpage content could not be accessed. Please check the URL or file format." + "\n\n"
+                useful_information += "Summary: \n" + "The webpage content could not be processed, and therefore, no information is available." + "\n\n"
+            else:
+                useful_information = "The useful information in {url} for user goal {goal} as follows: \n\n".format(url=url, goal=goal)
+                useful_information += "Evidence in page: \n" + str(raw["evidence"]) + "\n\n"
+                useful_information += "Summary: \n" + str(raw["summary"]) + "\n\n"
+
+            if len(useful_information) < 10 and summary_retries < 0:
+                print("[visit] Could not generate valid summary after maximum retries")
+                useful_information = "[visit] Failed to read page"
+            
+            return useful_information
+
+        # If no valid content was obtained after all retries
+        else:
+            useful_information = "The useful information in {url} for user goal {goal} as follows: \n\n".format(url=url, goal=goal)
+            useful_information += "Evidence in page: \n" + "The provided webpage content could not be accessed. Please check the URL or file format." + "\n\n"
+            useful_information += "Summary: \n" + "The webpage content could not be processed, and therefore, no information is available." + "\n\n"
+            return useful_information
+
+    
+###### Test Code ###### 
+if __name__ == "__main__":  
+    visit = Visit()
+
+    params = {
+        "url": "https://arxiv.org/pdf/2509.13313", 
+        "goal": "Please provide a summary of the paper."
+    } 
+
+    visit.call(params)