chore: import upstream snapshot with attribution
dotnet-build-and-test / dotnet-test-functions (push) Has been cancelled
dotnet-build-and-test / paths-filter (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Debug, windows-latest, net9.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net8.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-foundry-hosted-it (push) Has been cancelled
dotnet-build-and-test / dotnet-build-and-test-check (push) Has been cancelled
dotnet-build-and-test / Integration Test Report (push) Has been cancelled
CodeQL / Analyze (csharp) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
dotnet-build-and-test / dotnet-test-functions (push) Has been cancelled
dotnet-build-and-test / paths-filter (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Debug, windows-latest, net9.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, ubuntu-latest, net8.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-build (Release, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, ubuntu-latest, net10.0) (push) Has been cancelled
dotnet-build-and-test / dotnet-test (Release, integration, true, windows-latest, net472) (push) Has been cancelled
dotnet-build-and-test / dotnet-foundry-hosted-it (push) Has been cancelled
dotnet-build-and-test / dotnet-build-and-test-check (push) Has been cancelled
dotnet-build-and-test / Integration Test Report (push) Has been cancelled
CodeQL / Analyze (csharp) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,33 @@
|
||||
# Agent Framework hosting helper samples
|
||||
|
||||
End-to-end samples for exposing Agent Framework targets through app-owned
|
||||
hosting routes.
|
||||
|
||||
The helper-first hosting packages provide protocol conversion and optional
|
||||
execution state. The application still owns the web framework, native SDK
|
||||
clients, authentication, response construction, and deployment shape.
|
||||
|
||||
| Sample | What it shows | Packaging |
|
||||
|---|---|---|
|
||||
| [`local_responses/`](./local_responses) | One agent + one `@tool` + native FastAPI route + Responses helper functions + `AgentState` / `SessionStore`. | **Local only.** Start here to learn the helper seam. |
|
||||
| [`local_responses_workflow/`](./local_responses_workflow) | A workflow target behind a native FastAPI route using Responses helper functions, `WorkflowState`, explicit `CheckpointStorage`, and an app-owned checkpoint cursor. | **Local only.** |
|
||||
|
||||
Each sample is self-contained with its own `pyproject.toml`, server `app.py`,
|
||||
calling script(s), and `storage/` directory. Samples use `[tool.uv.sources]`
|
||||
to wire unreleased hosting packages to the upstream repo while those packages
|
||||
are still pre-PyPI. Once those packages publish, drop the `[tool.uv.sources]`
|
||||
block and let the declared dependencies resolve from PyPI.
|
||||
|
||||
## Relationship to `../foundry-hosted-agents/`
|
||||
|
||||
The sibling [`../foundry-hosted-agents/`](../foundry-hosted-agents) directory
|
||||
contains samples for agents that run inside the Foundry Hosted Agents platform.
|
||||
Those samples use the Foundry-managed protocol surface with no
|
||||
`agent-framework-hosting` package involved.
|
||||
|
||||
| Aspect | `af-hosting/` (this directory) | `foundry-hosted-agents/` |
|
||||
|---|---|---|
|
||||
| Server stack | App-owned FastAPI + hosting protocol helpers | Foundry Hosted Agents runtime |
|
||||
| Protocol surface | The app exposes the route and calls helpers | The platform exposes Responses + Invocations |
|
||||
| Run target | Local Hypercorn (`local_responses/`, `local_responses_workflow/`) | Hosted Agents or local container targeting the Hosted Agents contract |
|
||||
| When to pick this | You need custom hosting code or want to learn the helper seam | You want the Foundry-managed hosting surface |
|
||||
@@ -0,0 +1,90 @@
|
||||
# local_responses — Responses helpers with native FastAPI routes
|
||||
|
||||
The smallest end-to-end Responses hosting shape: one Foundry agent with a
|
||||
`@tool`, one native FastAPI route, a small `SessionStore`, and the Responses
|
||||
helper functions:
|
||||
|
||||
- `responses_to_run(...)`
|
||||
- `responses_session_id(...)`
|
||||
- `create_response_id(...)`
|
||||
- `responses_from_run(...)`
|
||||
|
||||
The sample demonstrates the lighter hosting direction. Agent Framework provides
|
||||
the run conversion and session-state pieces; FastAPI owns route registration,
|
||||
request bodies, response objects, and server startup.
|
||||
|
||||
What the route demonstrates:
|
||||
|
||||
- Uses an explicit request-option allowlist. This sample only allows
|
||||
`max_tokens` and then overrides `reasoning`; all other caller-supplied
|
||||
options, including `model`, `temperature`, `store`, `tools`, and
|
||||
`tool_choice`, are denied by default. Your app decides the exact allowed,
|
||||
altered, and denied options.
|
||||
- **Forces** a `reasoning` preset (`effort=medium`, `summary=auto`) on every
|
||||
turn.
|
||||
- Produces the AF messages, options, and session id that the route passes to
|
||||
`agent.run(...)`.
|
||||
- **Stores** each newly minted response id for the session it was just
|
||||
resolved from, via `state.set_session(response_id, session)` after
|
||||
`agent.run(...)` has updated the session.
|
||||
OpenAI's `previous_response_id` rotates every turn *by design* — it lets a
|
||||
caller continue from any earlier response, not just the latest one — so
|
||||
every response id needs to stay independently resolvable, not just the
|
||||
most recent.
|
||||
- Treats an unknown `conversation_id` as a request to create a new local
|
||||
session. Your app can choose a stricter policy, such as requiring a separate
|
||||
API to create new conversations before callers can continue them.
|
||||
|
||||
`app:app` is a module-level FastAPI ASGI app; recommended local launch is
|
||||
Hypercorn.
|
||||
|
||||
## Production readiness
|
||||
|
||||
This is not a full-fledged production deployment. Before exposing this pattern
|
||||
to callers, add authentication and authorization at the infrastructure layer,
|
||||
the FastAPI app layer, or inside the route body.
|
||||
|
||||
Session continuation deserves particular care: treat `previous_response_id` and
|
||||
`conversation_id` as untrusted request values, authorize the caller before
|
||||
loading or storing a session for those ids, and partition any durable session
|
||||
store by tenant/user as appropriate for your application.
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
export FOUNDRY_PROJECT_ENDPOINT=https://<your-project>.services.ai.azure.com
|
||||
export FOUNDRY_MODEL=gpt-5-nano
|
||||
az login
|
||||
|
||||
uv sync
|
||||
uv run hypercorn app:app --bind 0.0.0.0:8000
|
||||
```
|
||||
|
||||
Single-process for quick iteration:
|
||||
|
||||
```bash
|
||||
uv run python app.py
|
||||
```
|
||||
|
||||
## Call locally
|
||||
|
||||
```bash
|
||||
uv sync --group dev
|
||||
|
||||
# Plain OpenAI SDK call:
|
||||
uv run python call_server.py
|
||||
|
||||
# The client intentionally omits `model`; the app chooses the backing deployment
|
||||
# from FOUNDRY_MODEL.
|
||||
|
||||
# The script then sends two more turns, each continuing from the previous
|
||||
# turn's `response.id` as `previous_response_id`. The third turn asks about
|
||||
# the first turn's city, so it only succeeds if the server still remembers
|
||||
# that far back in the chain.
|
||||
|
||||
# Same three-turn interaction through an Agent Framework Agent backed by
|
||||
# OpenAIChatClient:
|
||||
uv run python call_server_af.py
|
||||
```
|
||||
|
||||
> This sample is **local-only** — no Dockerfile, no Foundry packaging.
|
||||
@@ -0,0 +1,195 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Minimal Responses-only hosting sample with native FastAPI routes.
|
||||
|
||||
This sample demonstrates the helper-first hosting shape:
|
||||
|
||||
1. ``agent-framework-hosting-responses`` converts Responses request/response
|
||||
payloads to and from Agent Framework run values.
|
||||
2. ``agent-framework-hosting`` owns shared execution state via
|
||||
``AgentState`` and ``SessionStore``.
|
||||
3. FastAPI owns the route, request parsing, policy decisions, and response
|
||||
object.
|
||||
|
||||
Production readiness
|
||||
---
|
||||
This sample is not a full-fledged production deployment. Before exposing this
|
||||
route to callers, add authentication and authorization at the infrastructure
|
||||
layer, the FastAPI app layer, or inside the route body.
|
||||
|
||||
Session continuation deserves particular care: treat ``previous_response_id``
|
||||
and ``conversation_id`` as untrusted request values, authorize the caller
|
||||
before loading or storing a session for those ids, and partition durable session
|
||||
storage by tenant/user as appropriate for your application. See
|
||||
``README.md#production-readiness``.
|
||||
|
||||
Unknown ``conversation_id`` values create a new local session in this sample.
|
||||
Your app can choose a different policy, such as requiring a separate API to
|
||||
create new conversations before callers can continue them.
|
||||
|
||||
Run
|
||||
---
|
||||
``app`` is a module-level FastAPI ASGI app. Recommended local launch::
|
||||
|
||||
uv sync
|
||||
az login
|
||||
export FOUNDRY_PROJECT_ENDPOINT=https://<your-project>.services.ai.azure.com
|
||||
export FOUNDRY_MODEL=gpt-5-nano
|
||||
uv run hypercorn app:app --bind 0.0.0.0:8000
|
||||
|
||||
Or use the ``__main__`` block (single-process Hypercorn) for quick
|
||||
iteration::
|
||||
|
||||
uv run python app.py
|
||||
|
||||
Then call it::
|
||||
|
||||
uv run python call_server.py "What is the weather in Tokyo?"
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
from collections.abc import AsyncIterator
|
||||
from pathlib import Path
|
||||
from typing import Annotated, Any, cast
|
||||
|
||||
from agent_framework import Agent, FileHistoryProvider, ResponseStream, tool
|
||||
from agent_framework_foundry import FoundryChatClient
|
||||
from agent_framework_hosting import AgentState
|
||||
from agent_framework_hosting_responses import (
|
||||
create_response_id,
|
||||
responses_from_run,
|
||||
responses_from_streaming_run,
|
||||
responses_session_id,
|
||||
responses_to_run,
|
||||
)
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
from fastapi import Body, FastAPI, HTTPException
|
||||
from fastapi.responses import JSONResponse, StreamingResponse
|
||||
from hypercorn.asyncio import serve
|
||||
from hypercorn.config import Config
|
||||
|
||||
SESSIONS_DIR = Path(__file__).resolve().parent / "storage" / "sessions"
|
||||
SESSIONS_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
@tool(approval_mode="never_require")
|
||||
def lookup_weather(
|
||||
location: Annotated[str, "The city to look up weather for."],
|
||||
) -> str:
|
||||
"""Return a deterministic weather report for a city."""
|
||||
high_temp = 5 + (sum(location.encode("utf-8")) % 21)
|
||||
reports = {
|
||||
"Seattle": f"Seattle is rainy with a high of {high_temp}°C.",
|
||||
"Amsterdam": f"Amsterdam is cloudy with a high of {high_temp}°C.",
|
||||
"Tokyo": f"Tokyo is clear with a high of {high_temp}°C.",
|
||||
}
|
||||
return reports.get(location, f"{location} is sunny with a high of {high_temp}°C.")
|
||||
|
||||
|
||||
def create_agent() -> Agent:
|
||||
"""Create the sample weather agent."""
|
||||
return Agent(
|
||||
client=FoundryChatClient(credential=DefaultAzureCredential()),
|
||||
name="WeatherAgent",
|
||||
instructions=(
|
||||
"You are a friendly weather assistant. Use the lookup_weather tool "
|
||||
"for any weather question and answer in one short sentence."
|
||||
),
|
||||
tools=[lookup_weather],
|
||||
context_providers=[FileHistoryProvider(SESSIONS_DIR)],
|
||||
default_options={"store": False},
|
||||
)
|
||||
|
||||
|
||||
app = FastAPI()
|
||||
state = AgentState(create_agent)
|
||||
|
||||
ALLOWED_REQUEST_OPTIONS = frozenset({"max_tokens", "reasoning"})
|
||||
|
||||
|
||||
@app.post("/responses", response_model=None)
|
||||
async def responses(body: dict[str, Any] = Body(...)) -> JSONResponse | StreamingResponse: # noqa: B008
|
||||
"""Handle one OpenAI Responses-shaped request."""
|
||||
try:
|
||||
run = responses_to_run(body)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
session_id = responses_session_id(body)
|
||||
response_id = create_response_id()
|
||||
|
||||
# App-specific policy: allow only the request options this route is willing
|
||||
# to honor. This denies tools, tool_choice, deployment/persistence fields,
|
||||
# and all other caller-supplied options by default. Your app decides which
|
||||
# options are allowed, altered, or denied.
|
||||
options = {key: value for key, value in run["options"].items() if key in ALLOWED_REQUEST_OPTIONS}
|
||||
options["reasoning"] = {"effort": "medium", "summary": "auto"}
|
||||
options_for_run = cast(Any, options)
|
||||
|
||||
target = await state.get_target()
|
||||
lookup_id = session_id or response_id
|
||||
# An unknown `conversation_id` becomes a new session here. Production apps
|
||||
# can choose to require a separate "create conversation" API instead.
|
||||
session = await state.get_or_create_session(lookup_id)
|
||||
if run["stream"]:
|
||||
stream = target.run(
|
||||
run["messages"],
|
||||
stream=True,
|
||||
session=session,
|
||||
options=options_for_run,
|
||||
)
|
||||
if not isinstance(stream, ResponseStream):
|
||||
raise HTTPException(status_code=500, detail="agent did not return a response stream")
|
||||
|
||||
async def stream_events() -> AsyncIterator[str]:
|
||||
async for event in responses_from_streaming_run(
|
||||
stream,
|
||||
response_id=response_id,
|
||||
session_id=session_id,
|
||||
):
|
||||
yield event
|
||||
# `agent.run(..., stream=True)` updates the session while the stream
|
||||
# is consumed/finalized. Store it under the newly minted response id
|
||||
# after finalization so a later `previous_response_id` can restore
|
||||
# this exact continuation point.
|
||||
await state.set_session(response_id, session)
|
||||
|
||||
return StreamingResponse(
|
||||
stream_events(),
|
||||
media_type="text/event-stream",
|
||||
)
|
||||
|
||||
result = await target.run(
|
||||
run["messages"],
|
||||
session=session,
|
||||
options=options_for_run,
|
||||
)
|
||||
# `agent.run(...)` updates the session. Store it under the newly minted
|
||||
# response id after the run so `previous_response_id=response_id` continues
|
||||
# from this exact point.
|
||||
await state.set_session(response_id, session)
|
||||
return JSONResponse(
|
||||
responses_from_run(
|
||||
result,
|
||||
response_id=response_id,
|
||||
session_id=session_id,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
"""Run the sample with Hypercorn for local development."""
|
||||
config = Config()
|
||||
config.bind = [f"0.0.0.0:{int(os.environ.get('PORT', '8000'))}"]
|
||||
await serve(cast(Any, app), config)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
|
||||
# Sample output:
|
||||
# User: What is the weather in Tokyo?
|
||||
# Agent: Tokyo is clear with a high of 18°C.
|
||||
# Response ID: resp_...
|
||||
@@ -0,0 +1,63 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Local client for the local_responses sample.
|
||||
|
||||
Posts to ``/responses`` using the standard ``openai`` SDK.
|
||||
|
||||
Pass ``--previous-response-id <id>`` to continue a conversation by its
|
||||
``response.id`` (returned in the prior response).
|
||||
|
||||
Start the server first (in another shell)::
|
||||
|
||||
uv run python app.py
|
||||
|
||||
Then::
|
||||
|
||||
uv run python call_server.py
|
||||
|
||||
The script sends two follow-up turns, each continuing from the previous
|
||||
turn's ``response.id`` as ``previous_response_id``. The third turn asks about
|
||||
information from the *first* turn only, so it also exercises session
|
||||
continuity across a rotating response id chain, not just a single hop.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from openai import OpenAI
|
||||
|
||||
BASE_URL = "http://127.0.0.1:8000"
|
||||
PROMPT = "What is the weather in Tokyo?"
|
||||
FOLLOW_UP_PROMPT = "And what about Amsterdam?"
|
||||
THIRD_PROMPT = "Which of the two cities we just discussed is warmer?"
|
||||
|
||||
|
||||
def main() -> None:
|
||||
client = OpenAI(base_url=BASE_URL, api_key="not-needed")
|
||||
response = client.responses.create(
|
||||
input=PROMPT,
|
||||
)
|
||||
print(f"User: {PROMPT}")
|
||||
print(f"Agent: {response.output_text}")
|
||||
print(f"Response ID: {response.id}")
|
||||
|
||||
follow_up = client.responses.create(
|
||||
input=FOLLOW_UP_PROMPT,
|
||||
previous_response_id=response.id,
|
||||
)
|
||||
print()
|
||||
print(f"User: {FOLLOW_UP_PROMPT}")
|
||||
print(f"Agent: {follow_up.output_text}")
|
||||
print(f"Response ID: {follow_up.id}")
|
||||
|
||||
third = client.responses.create(
|
||||
input=THIRD_PROMPT,
|
||||
previous_response_id=follow_up.id,
|
||||
)
|
||||
print()
|
||||
print(f"User: {THIRD_PROMPT}")
|
||||
print(f"Agent: {third.output_text}")
|
||||
print(f"Response ID: {third.id}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,59 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Agent Framework agent client for the local_responses sample.
|
||||
|
||||
Creates a local :class:`agent_framework.Agent` backed by
|
||||
:class:`agent_framework.openai.OpenAIChatClient` and points that client at the
|
||||
hosted ``/responses`` endpoint for all turns:
|
||||
|
||||
1. ``What is the weather in Tokyo?``
|
||||
2. ``And what about Amsterdam?``
|
||||
3. ``Which of the two cities we just discussed is warmer?``
|
||||
|
||||
All turns use the same :class:`agent_framework.AgentSession`; the first turn
|
||||
binds the hosted response id to the session, and later turns continue through
|
||||
that session via a chain of rotating ``previous_response_id`` values. The
|
||||
third turn only makes sense if the server still remembers the first turn, so
|
||||
it also exercises session continuity across that whole chain, not just a
|
||||
single hop.
|
||||
|
||||
Start the server first (in another shell)::
|
||||
|
||||
uv run python app.py
|
||||
|
||||
Then::
|
||||
|
||||
uv run python call_server_af.py
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
|
||||
from agent_framework import Agent
|
||||
from agent_framework.openai import OpenAIChatClient
|
||||
|
||||
BASE_URL = "http://127.0.0.1:8000"
|
||||
PROMPTS = [
|
||||
"What is the weather in Tokyo?",
|
||||
"And what about Amsterdam?",
|
||||
"Which of the two cities we just discussed is warmer?",
|
||||
]
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
agent = Agent(
|
||||
client=OpenAIChatClient(base_url=BASE_URL, api_key="not-needed"),
|
||||
name="HostedWeatherClient",
|
||||
)
|
||||
session = agent.create_session()
|
||||
|
||||
for prompt in PROMPTS:
|
||||
print(f"User: {prompt}")
|
||||
response = await agent.run(prompt, session=session)
|
||||
print(f"Agent: {response.text}\n")
|
||||
print(f"Response ID: {response.response_id}\n")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,28 @@
|
||||
[project]
|
||||
name = "agent-framework-hosting-sample-local-responses"
|
||||
version = "0.0.1"
|
||||
description = "Minimal Responses-only local hosting sample with native FastAPI routes."
|
||||
requires-python = ">=3.10"
|
||||
dependencies = [
|
||||
"agent-framework-foundry",
|
||||
"agent-framework-hosting",
|
||||
"agent-framework-hosting-responses",
|
||||
"azure-identity",
|
||||
"aiohttp>=3.13.5",
|
||||
"fastapi>=0.115.0,<0.138.1",
|
||||
"hypercorn>=0.17",
|
||||
]
|
||||
|
||||
[dependency-groups]
|
||||
dev = [
|
||||
"agent-framework-openai",
|
||||
"openai>=1.99",
|
||||
]
|
||||
|
||||
[tool.uv]
|
||||
package = false
|
||||
|
||||
[tool.uv.sources]
|
||||
agent-framework-hosting = { git = "https://github.com/microsoft/agent-framework.git", branch = "main", subdirectory = "python/packages/hosting" }
|
||||
agent-framework-hosting-responses = { git = "https://github.com/microsoft/agent-framework.git", branch = "main", subdirectory = "python/packages/hosting-responses" }
|
||||
agent-framework-openai = { git = "https://github.com/microsoft/agent-framework.git", branch = "main", subdirectory = "python/packages/openai" }
|
||||
@@ -0,0 +1,2 @@
|
||||
*
|
||||
!.gitignore
|
||||
@@ -0,0 +1,65 @@
|
||||
# local_responses_workflow — Responses helpers with a workflow target
|
||||
|
||||
This sample shows the helper-first hosting shape for a local workflow:
|
||||
|
||||
- `responses_to_run(...)` parses the Responses request body.
|
||||
- `WorkflowState` resolves the workflow target.
|
||||
- FastAPI owns the route and response construction.
|
||||
- The app owns file-based checkpoint storage and the
|
||||
`response_id -> checkpoint_id` cursor used to continue from a previous
|
||||
response.
|
||||
- Continuation is intentionally limited to `previous_response_id`; this sample
|
||||
rejects `conversation_id` continuity with HTTP 400.
|
||||
|
||||
The workflow writes a slogan with one Foundry-backed writer agent and a small
|
||||
deterministic formatter executor. That keeps the sample focused on native
|
||||
FastAPI routing, Responses helpers, `WorkflowState`, and app-owned checkpoint
|
||||
cursor storage. Both workflow checkpoints and the checkpoint cursor file are
|
||||
stored under the sample's local `storage/` root. Checkpoints are scoped into
|
||||
per-continuation buckets so a "latest checkpoint" lookup cannot cross
|
||||
conversations.
|
||||
|
||||
## Production readiness
|
||||
|
||||
This is not a full-fledged production deployment. Before exposing this pattern
|
||||
to callers, add authentication and authorization at the infrastructure layer,
|
||||
the FastAPI app layer, or inside the route body.
|
||||
|
||||
Session continuation deserves particular care: treat `previous_response_id` as
|
||||
an untrusted request value, authorize the caller before restoring or storing a
|
||||
checkpoint cursor for that id, and partition durable checkpoint/cursor storage
|
||||
by tenant/user as appropriate for your application.
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
export FOUNDRY_PROJECT_ENDPOINT=https://<your-project>.services.ai.azure.com
|
||||
export FOUNDRY_MODEL=gpt-5-nano
|
||||
az login
|
||||
|
||||
uv sync
|
||||
uv run hypercorn app:app --bind 0.0.0.0:8000
|
||||
```
|
||||
|
||||
Single-process for quick iteration:
|
||||
|
||||
```bash
|
||||
uv run python app.py
|
||||
```
|
||||
|
||||
## Call locally
|
||||
|
||||
```bash
|
||||
uv sync --group dev
|
||||
uv run python call_server.py '{"topic": "electric SUV", "style": "playful", "audience": "young families"}'
|
||||
```
|
||||
|
||||
The script sends a follow-up using the first response id as
|
||||
`previous_response_id`, so the workflow restores the prior checkpoint before
|
||||
running the next turn. It deliberately does not send `conversation_id`, because
|
||||
this sample rejects `conversation_id` continuation.
|
||||
|
||||
> This sample uses local file storage under `storage/` for both workflow
|
||||
> checkpoints and checkpoint cursors. The checkpoint bucket names are hashed
|
||||
> from the continuation id before they are used as directory names. Replace this
|
||||
> with production-grade durable storage for multi-replica or transient hosting.
|
||||
@@ -0,0 +1,288 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Responses helper sample with a local workflow target and native FastAPI route.
|
||||
|
||||
This sample demonstrates the helper-first hosting shape for workflows:
|
||||
|
||||
1. ``agent-framework-hosting-responses`` converts the Responses request body to
|
||||
Agent Framework run values and renders the final response payload.
|
||||
2. ``agent-framework-hosting`` resolves the workflow target via ``WorkflowState``.
|
||||
3. FastAPI owns the route, request parsing, policy decisions, response object,
|
||||
and file-backed checkpoint cursor.
|
||||
|
||||
Production readiness
|
||||
---
|
||||
This sample is not a full-fledged production deployment. Before exposing this
|
||||
route to callers, add authentication and authorization at the infrastructure
|
||||
layer, the FastAPI app layer, or inside the route body.
|
||||
|
||||
This sample demonstrates continuation with ``previous_response_id`` only. It
|
||||
rejects ``conversation_id`` continuity with HTTP 400. Treat every
|
||||
``previous_response_id`` as an untrusted request value, authorize the caller
|
||||
before restoring or storing a checkpoint cursor for that id, and partition
|
||||
durable checkpoint/cursor storage by tenant/user as appropriate for your
|
||||
application. See ``README.md#production-readiness``.
|
||||
|
||||
Run
|
||||
---
|
||||
``app`` is a module-level FastAPI ASGI app. Recommended local launch::
|
||||
|
||||
uv sync
|
||||
az login
|
||||
export FOUNDRY_PROJECT_ENDPOINT=https://<your-project>.services.ai.azure.com
|
||||
export FOUNDRY_MODEL=gpt-5-nano
|
||||
uv run hypercorn app:app --bind 0.0.0.0:8000
|
||||
|
||||
Or use the ``__main__`` block (single-process Hypercorn) for quick iteration::
|
||||
|
||||
uv run python app.py
|
||||
|
||||
Then call it with a structured brief::
|
||||
|
||||
uv run python call_server.py \
|
||||
'{"topic": "electric SUV", "style": "playful", "audience": "young families"}'
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from collections.abc import Mapping
|
||||
from pathlib import Path
|
||||
from typing import Any, TypedDict, cast
|
||||
|
||||
from agent_framework import (
|
||||
Agent,
|
||||
AgentExecutor,
|
||||
AgentExecutorResponse,
|
||||
AgentResponse,
|
||||
Content,
|
||||
Executor,
|
||||
FileCheckpointStorage,
|
||||
Message,
|
||||
WorkflowBuilder,
|
||||
WorkflowContext,
|
||||
handler,
|
||||
)
|
||||
from agent_framework_foundry import FoundryChatClient
|
||||
from agent_framework_hosting import WorkflowState
|
||||
from agent_framework_hosting_responses import (
|
||||
create_response_id,
|
||||
responses_from_run,
|
||||
responses_session_id,
|
||||
responses_to_run,
|
||||
)
|
||||
from azure.identity.aio import DefaultAzureCredential
|
||||
from fastapi import Body, FastAPI, HTTPException
|
||||
from fastapi.responses import JSONResponse
|
||||
from hypercorn.asyncio import serve
|
||||
from hypercorn.config import Config
|
||||
|
||||
STORAGE_ROOT = Path(__file__).resolve().parent / "storage"
|
||||
CHECKPOINTS_ROOT = STORAGE_ROOT / "checkpoints"
|
||||
CHECKPOINT_CURSOR_PATH = STORAGE_ROOT / "checkpoint_cursors.json"
|
||||
CHECKPOINTS_ROOT.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
class CheckpointCursor(TypedDict):
|
||||
"""Stored pointer to a workflow checkpoint and its storage bucket."""
|
||||
|
||||
checkpoint_id: str
|
||||
storage_id: str
|
||||
|
||||
|
||||
class CheckpointCursorStore:
|
||||
"""File-backed mapping from Responses ids to workflow checkpoint ids."""
|
||||
|
||||
def __init__(self, path: Path) -> None:
|
||||
"""Create a cursor store at the given path.
|
||||
|
||||
Args:
|
||||
path: JSON file containing response-id to checkpoint-id mappings.
|
||||
"""
|
||||
self._path = path
|
||||
|
||||
def get(self, key: str) -> CheckpointCursor | None:
|
||||
"""Return the checkpoint cursor for a previous response id."""
|
||||
return self._load().get(key)
|
||||
|
||||
def set_many(self, cursors: Mapping[str, CheckpointCursor]) -> None:
|
||||
"""Persist one or more checkpoint cursors."""
|
||||
data = self._load()
|
||||
data.update(cursors)
|
||||
self._path.parent.mkdir(parents=True, exist_ok=True)
|
||||
self._path.write_text(json.dumps(data, indent=2, sort_keys=True) + "\n", encoding="utf-8")
|
||||
|
||||
def _load(self) -> dict[str, CheckpointCursor]:
|
||||
if not self._path.exists():
|
||||
return {}
|
||||
|
||||
raw = json.loads(self._path.read_text(encoding="utf-8"))
|
||||
if not isinstance(raw, dict):
|
||||
raise ValueError("Checkpoint cursor file must contain a JSON object.")
|
||||
|
||||
data: dict[str, CheckpointCursor] = {}
|
||||
for key, value in raw.items():
|
||||
if not isinstance(key, str) or not isinstance(value, Mapping):
|
||||
raise ValueError("Checkpoint cursor file must map string ids to checkpoint cursor objects.")
|
||||
checkpoint_id = value.get("checkpoint_id")
|
||||
storage_id = value.get("storage_id")
|
||||
if not isinstance(checkpoint_id, str) or not isinstance(storage_id, str):
|
||||
raise ValueError("Checkpoint cursor objects must contain string checkpoint_id and storage_id fields.")
|
||||
data[key] = CheckpointCursor(checkpoint_id=checkpoint_id, storage_id=storage_id)
|
||||
return data
|
||||
|
||||
|
||||
checkpoint_cursor_store = CheckpointCursorStore(CHECKPOINT_CURSOR_PATH)
|
||||
|
||||
|
||||
def checkpoint_storage_for(storage_id: str) -> FileCheckpointStorage:
|
||||
"""Return file checkpoint storage scoped to a single continuation bucket."""
|
||||
storage_key = hashlib.sha256(storage_id.encode("utf-8")).hexdigest()
|
||||
return FileCheckpointStorage(str(CHECKPOINTS_ROOT / storage_key))
|
||||
|
||||
|
||||
def workflow_prompt_from_messages(messages: Any) -> str:
|
||||
"""Prepare the workflow's initial writer prompt from Responses input."""
|
||||
|
||||
def extract_text(value: object) -> str:
|
||||
if isinstance(value, str):
|
||||
return value
|
||||
if isinstance(value, Message):
|
||||
return value.text
|
||||
if isinstance(value, list):
|
||||
return "\n".join(extract_text(item) for item in value)
|
||||
return ""
|
||||
|
||||
text = extract_text(messages).strip()
|
||||
topic = text or "a generic product"
|
||||
style = "modern"
|
||||
audience = "general"
|
||||
if topic.startswith("{"):
|
||||
try:
|
||||
data = json.loads(topic)
|
||||
except json.JSONDecodeError:
|
||||
data = None
|
||||
if isinstance(data, dict) and "topic" in data:
|
||||
topic = str(data["topic"])
|
||||
style = str(data.get("style", style))
|
||||
audience = str(data.get("audience", audience))
|
||||
|
||||
return (
|
||||
f"Topic: {topic}\n"
|
||||
f"Style: {style}\n"
|
||||
f"Audience: {audience}\n\n"
|
||||
"Write a single short slogan that fits the topic, style, and audience."
|
||||
)
|
||||
|
||||
|
||||
def response_from_workflow_result(result: Any) -> AgentResponse[Any]:
|
||||
"""Collapse workflow outputs to one assistant response for Responses rendering."""
|
||||
outputs = result.get_outputs() if hasattr(result, "get_outputs") else []
|
||||
output = outputs[-1] if outputs else "(no workflow output)"
|
||||
text = output.text if isinstance(output, AgentResponse) else str(output)
|
||||
return AgentResponse(messages=Message(role="assistant", contents=[Content.from_text(text=text)]))
|
||||
|
||||
|
||||
class TerminalFormatter(Executor):
|
||||
"""Format the writer's output as the workflow's final response."""
|
||||
|
||||
@handler
|
||||
async def handle(self, response: AgentExecutorResponse, ctx: WorkflowContext[Any, str]) -> None:
|
||||
"""Yield one terminal-friendly slogan string.
|
||||
|
||||
Args:
|
||||
response: The writer agent's response.
|
||||
ctx: Workflow context used to yield the final output.
|
||||
"""
|
||||
slogan = response.agent_response.text.strip().strip('"')
|
||||
await ctx.yield_output(f'Slogan: "{slogan}"')
|
||||
|
||||
|
||||
client = FoundryChatClient(credential=DefaultAzureCredential())
|
||||
writer = Agent(
|
||||
client=client,
|
||||
name="writer",
|
||||
instructions="You are an excellent slogan writer. Create one short slogan from the given brief.",
|
||||
)
|
||||
writer_ex = AgentExecutor(writer, context_mode="last_agent")
|
||||
formatter_ex = TerminalFormatter(id="terminal_formatter")
|
||||
|
||||
workflow_builder = WorkflowBuilder(
|
||||
name="local_responses_slogan_workflow",
|
||||
start_executor=writer_ex,
|
||||
output_from=[formatter_ex],
|
||||
).add_edge(writer_ex, formatter_ex)
|
||||
|
||||
|
||||
app = FastAPI()
|
||||
state = WorkflowState(workflow_builder, cache_target=False)
|
||||
|
||||
|
||||
@app.post("/responses", response_model=None)
|
||||
async def responses(body: dict[str, Any] = Body(...)) -> JSONResponse: # noqa: B008
|
||||
"""Handle one OpenAI Responses-shaped request for the workflow."""
|
||||
try:
|
||||
run = responses_to_run(body)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
|
||||
# This sample demonstrates only Responses `previous_response_id`
|
||||
# continuation. `responses_session_id` also returns `conversation_id`, so
|
||||
# reject that shape here instead of treating it as a checkpoint cursor.
|
||||
previous_response_id = responses_session_id(body)
|
||||
if previous_response_id and not previous_response_id.startswith("resp_"):
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="This server supports previous_response_id continuation only; conversation_id is not implemented.",
|
||||
)
|
||||
response_id = create_response_id()
|
||||
|
||||
target = await state.get_target()
|
||||
if previous_response_id and (checkpoint_cursor := checkpoint_cursor_store.get(previous_response_id)) is not None:
|
||||
# Restore first. Workflow.run does not allow `message` and
|
||||
# `checkpoint_id` in the same call.
|
||||
await target.run(
|
||||
checkpoint_id=checkpoint_cursor["checkpoint_id"],
|
||||
checkpoint_storage=checkpoint_storage_for(checkpoint_cursor["storage_id"]),
|
||||
)
|
||||
|
||||
storage_id = response_id
|
||||
checkpoint_storage = checkpoint_storage_for(storage_id)
|
||||
result = await target.run(
|
||||
message=workflow_prompt_from_messages(run["messages"]),
|
||||
checkpoint_storage=checkpoint_storage,
|
||||
)
|
||||
|
||||
latest = await checkpoint_storage.get_latest(workflow_name=target.name)
|
||||
if latest is not None:
|
||||
# Responses `previous_response_id` can point to any response id. Store
|
||||
# the current response id as the cursor for this workflow continuation.
|
||||
cursor = CheckpointCursor(checkpoint_id=latest.checkpoint_id, storage_id=storage_id)
|
||||
checkpoint_cursor_store.set_many({response_id: cursor})
|
||||
|
||||
return JSONResponse(
|
||||
responses_from_run(
|
||||
response_from_workflow_result(result),
|
||||
response_id=response_id,
|
||||
session_id=previous_response_id,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
"""Run the sample with Hypercorn for local development."""
|
||||
config = Config()
|
||||
config.bind = [f"0.0.0.0:{int(os.environ.get('PORT', '8000'))}"]
|
||||
await serve(cast(Any, app), config)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
|
||||
# Sample output:
|
||||
# User: {"topic": "electric SUV", "style": "playful", "audience": "young families"}
|
||||
# Assistant: Slogan: "Big Adventures. Tiny Emissions."
|
||||
# Response ID: resp_...
|
||||
@@ -0,0 +1,53 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Local client for the local_responses_workflow sample.
|
||||
|
||||
Posts to ``/responses`` using the standard ``openai`` SDK. This client
|
||||
demonstrates the sample's only supported continuation mode:
|
||||
``previous_response_id``. It deliberately does not send ``conversation_id``,
|
||||
which the sample server rejects.
|
||||
|
||||
Start the server first (in another shell)::
|
||||
|
||||
uv run python app.py
|
||||
|
||||
Then::
|
||||
|
||||
uv run python call_server.py '{"topic": "electric SUV", "style": "playful", "audience": "young families"}'
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
|
||||
from openai import OpenAI
|
||||
|
||||
BASE_URL = "http://127.0.0.1:8000"
|
||||
DEFAULT_BRIEF = '{"topic": "electric SUV", "style": "playful", "audience": "young families"}'
|
||||
FOLLOW_UP = "Make it a little more premium, but still family friendly."
|
||||
|
||||
|
||||
def main() -> None:
|
||||
"""Send a two-turn workflow conversation using ``previous_response_id``."""
|
||||
client = OpenAI(base_url=BASE_URL, api_key="not-needed")
|
||||
brief = sys.argv[1] if len(sys.argv) > 1 else DEFAULT_BRIEF
|
||||
|
||||
response = client.responses.create(input=brief)
|
||||
print(f"User: {brief}")
|
||||
print(f"Workflow: {response.output_text}")
|
||||
print(f"Response ID: {response.id}")
|
||||
|
||||
# Continue with the returned response id. The server sample rejects
|
||||
# `conversation_id` continuity.
|
||||
follow_up = client.responses.create(
|
||||
input=FOLLOW_UP,
|
||||
previous_response_id=response.id,
|
||||
)
|
||||
print()
|
||||
print(f"User: {FOLLOW_UP}")
|
||||
print(f"Workflow: {follow_up.output_text}")
|
||||
print(f"Response ID: {follow_up.id}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,26 @@
|
||||
[project]
|
||||
name = "agent-framework-hosting-sample-local-responses-workflow"
|
||||
version = "0.0.1"
|
||||
description = "Minimal Responses-only local hosting sample with a workflow target and native FastAPI routes."
|
||||
requires-python = ">=3.10"
|
||||
dependencies = [
|
||||
"agent-framework-foundry",
|
||||
"agent-framework-hosting",
|
||||
"agent-framework-hosting-responses",
|
||||
"azure-identity",
|
||||
"aiohttp>=3.13.5",
|
||||
"fastapi>=0.115.0,<0.138.1",
|
||||
"hypercorn>=0.17",
|
||||
]
|
||||
|
||||
[dependency-groups]
|
||||
dev = [
|
||||
"openai>=1.99",
|
||||
]
|
||||
|
||||
[tool.uv]
|
||||
package = false
|
||||
|
||||
[tool.uv.sources]
|
||||
agent-framework-hosting = { git = "https://github.com/microsoft/agent-framework.git", branch = "main", subdirectory = "python/packages/hosting" }
|
||||
agent-framework-hosting-responses = { git = "https://github.com/microsoft/agent-framework.git", branch = "main", subdirectory = "python/packages/hosting-responses" }
|
||||
@@ -0,0 +1,2 @@
|
||||
*
|
||||
!.gitignore
|
||||
Reference in New Issue
Block a user