microsoft--ai-agents-for-beginners/12-context-engineering/code_samples/12-chat_summarization.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lesson 12 - Chat History Reduction with Agent Scratchpad\n",
    "\n",
    "This notebook demonstrates how to manage context in long conversations using Microsoft Agent Framework. As conversations grow, the token count increases — eventually exceeding the model's context window. We address this with a **context summarization pattern** and an **agent scratchpad** for persistent memory.\n",
    "\n",
    "## What You'll Learn:\n",
    "1. **Why Context Management Matters**: Understanding token limits and context windows\n",
    "2. **Context-Aware Agents**: Building agents that manage their own conversation context\n",
    "3. **Context Summarization Pattern**: Using tools to condense conversation history\n",
    "4. **Agent Scratchpad**: Persistent memory that survives context reduction\n",
    "\n",
    "## Prerequisites:\n",
    "- Azure OpenAI setup with environment variables configured\n",
    "- Understanding of basic agent concepts from previous lessons"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install agent-framework azure-ai-projects azure-identity python-dotenv --quiet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import asyncio\n",
    "import dotenv\n",
    "from datetime import datetime\n",
    "from pathlib import Path\n",
    "\n",
    "from agent_framework import tool\n",
    "from agent_framework.foundry import FoundryChatClient\n",
    "from azure.identity import DefaultAzureCredential"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dotenv.load_dotenv()\n",
    "\n",
    "endpoint = os.getenv(\"AZURE_AI_PROJECT_ENDPOINT\")\n",
    "deployment_name = os.getenv(\"AZURE_AI_MODEL_DEPLOYMENT_NAME\")\n",
    "\n",
    "missing = [k for k, v in {\n",
    "    \"AZURE_AI_PROJECT_ENDPOINT\": endpoint,\n",
    "    \"AZURE_AI_MODEL_DEPLOYMENT_NAME\": deployment_name\n",
    "}.items() if not v]\n",
    "\n",
    "if missing:\n",
    "    raise ValueError(\n",
    "        f\"Missing required environment variables: {', '.join(missing)}. \"\n",
    "        \"Please set them as environment variables (e.g., in your .env file or shell environment).\"\n",
    "    )\n",
    "\n",
    "# Create the Microsoft Foundry client\n",
    "client = FoundryChatClient(\n",
    "    project_endpoint=endpoint,\n",
    "    model=deployment_name,\n",
    "    credential=DefaultAzureCredential()\n",
    ")\n",
    "\n",
    "print(\"Microsoft Foundry client configured\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Why Context Management Matters\n",
    "\n",
    "Every LLM has a finite **context window** — the maximum number of tokens it can process in a single request. As a multi-turn conversation progresses:\n",
    "\n",
    "- **Token count grows linearly** with each user message and assistant reply.\n",
    "- **Prompt tokens dominate cost** because the entire history is re-sent every turn.\n",
    "- Eventually the conversation **exceeds the context window** and the model either truncates or errors.\n",
    "\n",
    "### Strategies for Managing Context\n",
    "\n",
    "| Strategy | How It Works | Trade-off |\n",
    "|---|---|---|\n",
    "| **Truncation** | Drop oldest messages | Loses early context |\n",
    "| **Summarization** | Condense older messages into a summary | Some detail lost, but key points retained |\n",
    "| **Scratchpad / External Memory** | Store key facts outside the conversation | Requires tool calls, but survives any reduction |\n",
    "\n",
    "In this notebook we combine **summarization** with a **scratchpad tool** so the agent can maintain continuity even when conversation history is condensed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a Context-Aware Agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "agent = client.as_agent(\n",
    "    name=\"ContextAwareAgent\",\n",
    "    instructions=\"\"\"You are a helpful travel planning assistant with excellent memory management.\n",
    "When conversations get long:\n",
    "1. Summarize previous context into key points\n",
    "2. Track user preferences mentioned earlier\n",
    "3. Reference previous decisions without repeating full details\n",
    "Always maintain continuity while being concise.\"\"\",\n",
    ")\n",
    "\n",
    "print(\"Context-aware travel planning agent created\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simulating a Long Conversation\n",
    "\n",
    "Let's walk through a multi-turn conversation to see how context accumulates. The agent should retain key details (preferences, budget, travel dates) across turns and demonstrate continuity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "session = agent.create_session()\n",
    "\n",
    "# Turn 1 - Initial preferences\n",
    "response = await agent.run(\"I'm planning a trip to Japan. I love sushi, temples, and photography.\", session=session)\n",
    "print(f\"Turn 1: {response}\\n\")\n",
    "\n",
    "# Turn 2 - More details\n",
    "response = await agent.run(\"My budget is $3000 and I'll be traveling solo for 10 days in April.\", session=session)\n",
    "print(f\"Turn 2: {response}\\n\")\n",
    "\n",
    "# Turn 3 - Test context retention\n",
    "response = await agent.run(\"Based on everything I've told you so far, what's the one thing you'd recommend I not miss?\", session=session)\n",
    "print(f\"Turn 3: {response}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how the agent retains context from earlier turns — it knows about Japan, sushi, temples, photography, the $3000 budget, solo travel, and the April timeframe. In a short conversation this works well, but as the conversation grows the full history becomes expensive to re-send.\n",
    "\n",
    "Let's continue the conversation with more turns to see context accumulation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Turn 4 - Expand the conversation\n",
    "response = await agent.run(\"What about accommodation? I prefer traditional Japanese inns.\", session=session)\n",
    "print(f\"Turn 4: {response}\\n\")\n",
    "\n",
    "# Turn 5 - Change of plans\n",
    "response = await agent.run(\"Actually, I've changed my mind about the dates. I'll go in October instead for the autumn colors.\", session=session)\n",
    "print(f\"Turn 5: {response}\\n\")\n",
    "\n",
    "# Turn 6 - Test retention after change\n",
    "response = await agent.run(\"Summarize my complete travel plan so far — destination, budget, duration, interests, accommodation, and timing.\", session=session)\n",
    "print(f\"Turn 6: {response}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Context Summarization Pattern\n",
    "\n",
    "As the conversation grows, we can use a **summarization tool** to condense accumulated context into a compact format. The agent calls this tool to record key preferences so that even if older messages are dropped, the essential information is preserved.\n",
    "\n",
    "This pattern is the building block for more sophisticated history reduction:\n",
    "1. The agent identifies key facts from the conversation\n",
    "2. It calls the summarization tool to persist them\n",
    "3. Older messages can be safely removed because the summary captures what matters\n",
    "\n",
    "Below we define a `summarize_preferences` tool that the agent can call to record a compact summary of what it has learned."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@tool(approval_mode=\"never_require\")\n",
    "def summarize_preferences(conversation_notes: str) -> str:\n",
    "    \"\"\"Summarize accumulated user preferences into a compact format.\"\"\"\n",
    "    return f\"[SUMMARY] User preferences recorded: {conversation_notes}\"\n",
    "\n",
    "\n",
    "# Create an enhanced agent with the summarization tool\n",
    "summarizing_agent = client.as_agent(\n",
    "    name=\"SummarizingTravelAgent\",\n",
    "    instructions=\"\"\"You are a helpful travel planning assistant that actively manages conversation context.\n",
    "\n",
    "CONTEXT MANAGEMENT RULES:\n",
    "1. After gathering several user preferences, call summarize_preferences() to record a compact summary\n",
    "2. When the user asks you to recall details, reference your recorded summaries\n",
    "3. Keep responses concise — avoid restating the entire history\n",
    "\n",
    "PLANNING PROCESS:\n",
    "1. Gather user preferences (destination, budget, dates, interests)\n",
    "2. Summarize preferences using the tool\n",
    "3. Create recommendations based on the summary\n",
    "4. Update the summary when preferences change\"\"\",\n",
    "    tools=[summarize_preferences],\n",
    ")\n",
    "\n",
    "print(\"Summarizing travel agent created with context tools\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Demonstrate the summarization pattern\n",
    "summary_session = summarizing_agent.create_session()\n",
    "\n",
    "# Provide a batch of preferences\n",
    "response = await summarizing_agent.run(\n",
    "    \"I want to visit Greece. I love seafood, history, and island hopping. \"\n",
    "    \"Budget is $4000 for two weeks. Traveling with my partner in June. \"\n",
    "    \"Please record these preferences using your summarization tool.\",\n",
    "    session=summary_session,\n",
    ")\n",
    "print(f\"Agent: {response}\\n\")\n",
    "\n",
    "# Ask the agent to use the recorded context\n",
    "response = await summarizing_agent.run(\n",
    "    \"Now, based on what you've recorded, suggest the top 3 islands we should visit.\",\n",
    "    session=summary_session,\n",
    ")\n",
    "print(f\"Agent: {response}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "In this lesson you learned how to manage context in long-running agent conversations using Microsoft Agent Framework:\n",
    "\n",
    "### Key Concepts\n",
    "- **Context windows are finite** — every token in the conversation history costs money and counts toward the limit.\n",
    "- **Summarization tools** let the agent condense accumulated context into compact summaries, reducing token usage while preserving essential information.\n",
    "- **Agent scratchpads** provide persistent external memory that survives any conversation reduction.\n",
    "\n",
    "### What You Built\n",
    "- A **context-aware agent** that maintains continuity across multi-turn conversations\n",
    "- A **summarization tool** (`summarize_preferences`) that records key user details in a compact format\n",
    "- A **multi-turn conversation** demonstrating context retention and change handling\n",
    "\n",
    "### Real-World Applications\n",
    "- **Customer Service Bots**: Remember preferences across long support sessions\n",
    "- **Personal Assistants**: Track ongoing projects without re-explaining context\n",
    "- **Educational Tutors**: Maintain student progress across many interactions\n",
    "\n",
    "### Next Steps\n",
    "- Implement a full scratchpad tool with file-based persistence\n",
    "- Add automatic history truncation after summarization\n",
    "- Combine with vector databases for semantic memory search\n",
    "- Build agents that can resume conversations days later with full context"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}