chore: import upstream snapshot with attribution
tools_continuous_delivery / Private PyPI non-main branch release (push) Has been skipped
tools_continuous_delivery / Private PyPI main branch release (push) Failing after 2m42s
Publish Promptflow Doc / Build (push) Has been cancelled
Publish Promptflow Doc / Deploy (push) Has been cancelled
Flake8 Lint / flake8 (push) Has been cancelled
Spell check CI / Spell_Check (push) Has been cancelled
tools_continuous_delivery / Private PyPI non-main branch release (push) Has been skipped
tools_continuous_delivery / Private PyPI main branch release (push) Failing after 2m42s
Publish Promptflow Doc / Build (push) Has been cancelled
Publish Promptflow Doc / Deploy (push) Has been cancelled
Flake8 Lint / flake8 (push) Has been cancelled
Spell check CI / Spell_Check (push) Has been cancelled
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,26 @@
|
||||
# Copy this file to .env and fill in your values.
|
||||
# cp .env.example .env
|
||||
# Never commit .env to source control — it is in .gitignore.
|
||||
|
||||
# ── Microsoft Foundry ─────────────────────────────────────────────────────────
|
||||
# All phase-2 samples and the deployment use FoundryChatClient with these variables.
|
||||
# Endpoint format: https://<resource>.services.ai.azure.com
|
||||
FOUNDRY_PROJECT_ENDPOINT=https://<resource>.services.ai.azure.com
|
||||
FOUNDRY_MODEL=
|
||||
|
||||
# ── Azure OpenAI (parity evaluation only) ────────────────────────────────────
|
||||
# Used by the SimilarityEvaluator in phase-3-validate/.
|
||||
AZURE_OPENAI_API_KEY=
|
||||
AZURE_OPENAI_ENDPOINT=https://<resource>.openai.azure.com/
|
||||
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME=
|
||||
|
||||
# ── Azure AI Search ───────────────────────────────────────────────────────────
|
||||
# Required for RAG sample only (phase-2-rebuild/05_rag_flow.py).
|
||||
AZURE_AI_SEARCH_ENDPOINT=https://<search-resource>.search.windows.net
|
||||
AZURE_AI_SEARCH_INDEX_NAME=
|
||||
AZURE_AI_SEARCH_API_KEY=
|
||||
|
||||
# ── Application Insights (optional) ──────────────────────────────────────────
|
||||
# Tracing is optional — phases 1–3 work without it.
|
||||
# Required only for phase-4-migrate-ops/4a-tracing/ and the deployment score.py.
|
||||
# APPLICATIONINSIGHTS_CONNECTION_STRING=
|
||||
@@ -0,0 +1,44 @@
|
||||
---
|
||||
name: Migration issue
|
||||
about: Report a problem encountered while migrating from Prompt Flow to MAF
|
||||
labels: bug
|
||||
---
|
||||
|
||||
## MAF version
|
||||
|
||||
pip show agent-framework | grep Version
|
||||
|
||||
Paste output here.
|
||||
|
||||
## Prompt Flow node types involved
|
||||
|
||||
List the node types from your `flow.dag.yaml` that are relevant to the issue
|
||||
(e.g. `llm`, `python`, `prompt`, `embed_text`, `vector_db_lookup`).
|
||||
|
||||
## Phase
|
||||
|
||||
Which phase were you on when the issue occurred?
|
||||
|
||||
- [ ] Phase 1 — Audit & Map
|
||||
- [ ] Phase 2 — Rebuild in MAF
|
||||
- [ ] Phase 3 — Validate Parity
|
||||
- [ ] Phase 4 — Migrate Ops
|
||||
- [ ] Phase 5 — Cut Over
|
||||
|
||||
## Sample file
|
||||
|
||||
Which sample file (if any) does this relate to?
|
||||
|
||||
## What happened
|
||||
|
||||
Describe what you expected and what actually occurred.
|
||||
|
||||
## Full error output
|
||||
|
||||
Paste the complete traceback or error message here.
|
||||
|
||||
## Environment
|
||||
|
||||
- OS:
|
||||
- Python version:
|
||||
- Azure region:
|
||||
@@ -0,0 +1,54 @@
|
||||
# Migrated Flow Examples
|
||||
|
||||
We have generated migrated flows under the [`examples/flows/`](../../examples/flows/) folder for your reference. Each original Prompt Flow (PF) flow has a corresponding Microsoft Agent Framework (MAF) version (suffixed with `-maf`). The MAF workflows were created by AI using the [migration skill](../../.github/skills/promptflow-to-maf/).
|
||||
|
||||
---
|
||||
|
||||
## Chat Flows
|
||||
|
||||
| Example | PF Version | MAF Version | Description |
|
||||
|---|---|---|---|
|
||||
| Chat Basic | [chat-basic](../../examples/flows/chat/chat-basic/) | [chat-basic-maf](../../examples/flows/chat/chat-basic-maf/) | Chatbot that remembers previous interactions and generates responses using conversation history |
|
||||
| Chat Math Variant | [chat-math-variant](../../examples/flows/chat/chat-math-variant/) | [chat-math-variant-maf](../../examples/flows/chat/chat-math-variant-maf/) | Prompt tuning example with three variants for testing math question answering approaches |
|
||||
| Chat with Image | [chat-with-image](../../examples/flows/chat/chat-with-image/) | [chat-with-image-maf](../../examples/flows/chat/chat-with-image-maf/) | Chatbot that accepts both image and text as input using GPT-4V |
|
||||
| Chat with PDF | [chat-with-pdf](../../examples/flows/chat/chat-with-pdf/) | [chat-with-pdf-maf](../../examples/flows/chat/chat-with-pdf-maf/) | Ask questions about PDF content using retrieval-augmented generation |
|
||||
| Chat with Wikipedia | [chat-with-wikipedia](../../examples/flows/chat/chat-with-wikipedia/) | [chat-with-wikipedia-maf](../../examples/flows/chat/chat-with-wikipedia-maf/) | Chatbot with conversation history that searches Wikipedia for current information |
|
||||
| Promptflow Copilot | [promptflow-copilot](../../examples/flows/chat/promptflow-copilot/) | [promptflow-copilot-maf](../../examples/flows/chat/promptflow-copilot-maf/) | Chat flow for building a copilot assistant for Promptflow |
|
||||
| Use Functions with Chat Models | [use_functions_with_chat_models](../../examples/flows/chat/use_functions_with_chat_models/) | [use_functions_with_chat_models-maf](../../examples/flows/chat/use_functions_with_chat_models-maf/) | Extend LLM capabilities with external function specifications for chat models |
|
||||
|
||||
## Standard Flows
|
||||
|
||||
| Example | PF Version | MAF Version | Description |
|
||||
|---|---|---|---|
|
||||
| Autonomous Agent | [autonomous-agent](../../examples/flows/standard/autonomous-agent/) | [autonomous-agent-maf](../../examples/flows/standard/autonomous-agent-maf/) | AutoGPT agent that autonomously figures out how to apply functions to solve goals |
|
||||
| Basic | [basic](../../examples/flows/standard/basic/) | [basic-maf](../../examples/flows/standard/basic-maf/) | Basic flow using custom Python tool calling Azure OpenAI with environment variables |
|
||||
| Basic with Built-in LLM | [basic-with-builtin-llm](../../examples/flows/standard/basic-with-builtin-llm/) | [basic-with-builtin-llm-maf](../../examples/flows/standard/basic-with-builtin-llm-maf/) | Basic flow calling Azure OpenAI using the built-in LLM tool |
|
||||
| Basic with Connection | [basic-with-connection](../../examples/flows/standard/basic-with-connection/) | [basic-with-connection-maf](../../examples/flows/standard/basic-with-connection-maf/) | Basic flow using custom Python tool with connection info stored in custom connection |
|
||||
| Conditional Flow (If-Else) | [conditional-flow-for-if-else](../../examples/flows/standard/conditional-flow-for-if-else/) | [conditional-flow-for-if-else-maf](../../examples/flows/standard/conditional-flow-for-if-else-maf/) | Conditional flow that checks content safety and routes accordingly |
|
||||
| Conditional Flow (Switch) | [conditional-flow-for-switch](../../examples/flows/standard/conditional-flow-for-switch/) | [conditional-flow-for-switch-maf](../../examples/flows/standard/conditional-flow-for-switch-maf/) | Conditional flow that dynamically routes by classifying user intent |
|
||||
| Customer Intent Extraction | [customer-intent-extraction](../../examples/flows/standard/customer-intent-extraction/) | [customer-intent-extraction-maf](../../examples/flows/standard/customer-intent-extraction-maf/) | Identify customer intent from customer questions using LLM |
|
||||
| Describe Image | [describe-image](../../examples/flows/standard/describe-image/) | [describe-image-maf](../../examples/flows/standard/describe-image-maf/) | Flow that flips an image horizontally and describes it using GPT-4V |
|
||||
| Flow with Additional Includes | [flow-with-additional-includes](../../examples/flows/standard/flow-with-additional-includes/) | [flow-with-additional-includes-maf](../../examples/flows/standard/flow-with-additional-includes-maf/) | Demonstrates how to reference common files using additional_includes |
|
||||
| Flow with Symlinks | [flow-with-symlinks](../../examples/flows/standard/flow-with-symlinks/) | [flow-with-symlinks-maf](../../examples/flows/standard/flow-with-symlinks-maf/) | Flow that uses symbolic links to reference common files across flows |
|
||||
| Generate Docstring | [gen-docstring](../../examples/flows/standard/gen-docstring/) | [gen-docstring-maf](../../examples/flows/standard/gen-docstring-maf/) | Automatically generate Python docstrings for code blocks using LLM |
|
||||
| Maths to Code | [maths-to-code](../../examples/flows/standard/maths-to-code/) | [maths-to-code-maf](../../examples/flows/standard/maths-to-code-maf/) | Generate executable code from math problems and execute it for answers |
|
||||
| Named Entity Recognition | [named-entity-recognition](../../examples/flows/standard/named-entity-recognition/) | [named-entity-recognition-maf](../../examples/flows/standard/named-entity-recognition-maf/) | Perform NER task identifying and classifying named entities using GPT-4 |
|
||||
| Question Simulation | [question-simulation](../../examples/flows/standard/question-simulation/) | [question-simulation-maf](../../examples/flows/standard/question-simulation-maf/) | Generate suggestions for next questions based on chat history context |
|
||||
| Web Classification | [web-classification](../../examples/flows/standard/web-classification/) | [web-classification-maf](../../examples/flows/standard/web-classification-maf/) | Multi-class website classification from URLs using few-shot learning |
|
||||
|
||||
## Evaluation Flows
|
||||
|
||||
| Example | PF Version | MAF Version | Description |
|
||||
|---|---|---|---|
|
||||
| Eval Accuracy (Maths to Code) | [eval-accuracy-maths-to-code](../../examples/flows/evaluation/eval-accuracy-maths-to-code/) | [eval-accuracy-maths-to-code-maf](../../examples/flows/evaluation/eval-accuracy-maths-to-code-maf/) | Evaluates mathematical accuracy by comparing predictions to ground truth |
|
||||
| Eval Basic | [eval-basic](../../examples/flows/evaluation/eval-basic/) | [eval-basic-maf](../../examples/flows/evaluation/eval-basic-maf/) | Basic evaluation flow showing how to calculate point-wise metrics |
|
||||
| Eval Chat Math | [eval-chat-math](../../examples/flows/evaluation/eval-chat-math/) | [eval-chat-math-maf](../../examples/flows/evaluation/eval-chat-math-maf/) | Evaluate math question answers by comparing results numerically |
|
||||
| Eval Classification Accuracy | [eval-classification-accuracy](../../examples/flows/evaluation/eval-classification-accuracy/) | [eval-classification-accuracy-maf](../../examples/flows/evaluation/eval-classification-accuracy-maf/) | Evaluate classification system performance with accuracy metrics |
|
||||
| Eval Entity Match Rate | [eval-entity-match-rate](../../examples/flows/evaluation/eval-entity-match-rate/) | [eval-entity-match-rate-maf](../../examples/flows/evaluation/eval-entity-match-rate-maf/) | Evaluate how well extracted entities match expected entities |
|
||||
| Eval Groundedness | [eval-groundedness](../../examples/flows/evaluation/eval-groundedness/) | [eval-groundedness-maf](../../examples/flows/evaluation/eval-groundedness-maf/) | Evaluate if answers are grounded in provided context using LLM |
|
||||
| Eval Multi-Turn Metrics | [eval-multi-turn-metrics](../../examples/flows/evaluation/eval-multi-turn-metrics/) | [eval-multi-turn-metrics-maf](../../examples/flows/evaluation/eval-multi-turn-metrics-maf/) | Evaluate multi-turn conversations on quality, coherence, and engagement |
|
||||
| Eval Perceived Intelligence | [eval-perceived-intelligence](../../examples/flows/evaluation/eval-perceived-intelligence/) | [eval-perceived-intelligence-maf](../../examples/flows/evaluation/eval-perceived-intelligence-maf/) | Evaluate bot perceived intelligence, creativity, and originality |
|
||||
| Eval QnA Non-RAG | [eval-qna-non-rag](../../examples/flows/evaluation/eval-qna-non-rag/) | [eval-qna-non-rag-maf](../../examples/flows/evaluation/eval-qna-non-rag-maf/) | Q&A system evaluation with LLM-assisted coherence, relevance, and similarity metrics |
|
||||
| Eval QnA RAG Metrics | [eval-qna-rag-metrics](../../examples/flows/evaluation/eval-qna-rag-metrics/) | [eval-qna-rag-metrics-maf](../../examples/flows/evaluation/eval-qna-rag-metrics-maf/) | RAG Q&A system evaluation with retrieval, groundedness, and relevance metrics |
|
||||
| Eval Single-Turn Metrics | [eval-single-turn-metrics](../../examples/flows/evaluation/eval-single-turn-metrics/) | [eval-single-turn-metrics-maf](../../examples/flows/evaluation/eval-single-turn-metrics-maf/) | Single-turn Q&A evaluation on grounding, relevance, and answer quality |
|
||||
| Eval Summarization | [eval-summarization](../../examples/flows/evaluation/eval-summarization/) | [eval-summarization-maf](../../examples/flows/evaluation/eval-summarization-maf/) | Reference-free abstractive summarization evaluation on fluency, coherence, consistency, relevance |
|
||||
@@ -0,0 +1,68 @@
|
||||
# Migrating from Prompt Flow to Microsoft Agent Framework (MAF)
|
||||
|
||||
Prompt Flow is being retired. Feature development ended **20 April 2026**, with full retirement on **20 April 2027**. This repo is a hands-on migration guide with runnable code samples for every step of the move to
|
||||
[Microsoft Agent Framework 1.0](https://devblogs.microsoft.com/agent-framework/microsoft-agent-framework-version-1-0/)
|
||||
(GA as of 2 April 2026, Python & .NET).
|
||||
|
||||
---
|
||||
|
||||
## Who this is for
|
||||
|
||||
Teams running Prompt Flow workloads on Microsoft Foundry or Azure Machine Learning who want a structured, low-risk path to MAF.
|
||||
|
||||
---
|
||||
|
||||
## The 5-Phase Migration Plan
|
||||
|
||||
| Phase | What you do | Folder |
|
||||
|---|---|---|
|
||||
| **1 — Audit & Map** | Export your PF flow YAML; map every node to its MAF equivalent | [`phase-1-audit/`](./phase-1-audit/) |
|
||||
| **2 — Rebuild in MAF** | Re-implement the workflow using `WorkflowBuilder` and `Executor` | [`phase-2-rebuild/`](./phase-2-rebuild/) |
|
||||
| **3 — Validate Parity** | Run PF and MAF side-by-side; score similarity with Azure AI Evaluation SDK | [`phase-3-validate/`](./phase-3-validate/) |
|
||||
| **4 — Migrate Ops** | Wire up tracing (OTel), deploy as managed online endpoint (Azure ML / AI Foundry), add CI/CD quality gate | [`phase-4-migrate-ops/`](./phase-4-migrate-ops/) |
|
||||
| **5 — Cut Over** | Switch traffic to MAF; decommission PF endpoints and connections | [`phase-5-cutover/`](./phase-5-cutover/) |
|
||||
|
||||
Work through the phases in order. Each folder has its own README with context, prerequisites, and expected outputs.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.10+
|
||||
- An Azure subscription with:
|
||||
- Azure OpenAI resource (or Microsoft Foundry project)
|
||||
- Azure AI Search index (for RAG samples only)
|
||||
- Application Insights instance (for tracing samples)
|
||||
- Azure CLI (`az login` completed)
|
||||
|
||||
---
|
||||
|
||||
## Setup
|
||||
|
||||
```bash
|
||||
git clone https://github.com/shshubhe/promptflow-migration-guide
|
||||
cd promptflow-migration-guide/migration-guide/PromptFlow-to-MAF
|
||||
|
||||
# GA packages
|
||||
pip install agent-framework>=1.0.1 agent-framework-foundry>=1.0.1
|
||||
|
||||
# Preview packages (orchestrations for multi-agent, Azure AI Search for RAG)
|
||||
pip install agent-framework-orchestrations agent-framework-azure-ai-search --pre
|
||||
|
||||
# Remaining dependencies (evaluation, deployment, tracing)
|
||||
pip install -r requirements.txt
|
||||
|
||||
cp .env.example .env # then fill in your values
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Migrated Examples
|
||||
|
||||
See [EXAMPLES.md](./EXAMPLES.md) for a full catalog of migrated flows with links to both the original Prompt Flow and MAF versions.
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
Start with [Phase 1 — Audit & Map](./phase-1-audit/) to export and map your existing Prompt Flow.
|
||||
@@ -0,0 +1,381 @@
|
||||
# Troubleshooting
|
||||
|
||||
Common issues encountered when migrating from Prompt Flow to MAF, grounded
|
||||
in the MAF 1.0 GA release (2 Apr 2026) and the official API documentation.
|
||||
If your issue is not listed here, open a GitHub issue using the [migration issue template](../.github/ISSUE_TEMPLATE/migration-issue.md).
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
### `ModuleNotFoundError: No module named 'agent_framework'`
|
||||
|
||||
The package was not installed, or an old RC install is conflicting.
|
||||
|
||||
Uninstall any previous version first, then reinstall cleanly:
|
||||
|
||||
pip uninstall agent-framework agent-framework-core agent-framework-foundry -y
|
||||
pip install agent-framework>=1.0.1
|
||||
|
||||
MAF core packages are GA (1.0.1) — `--pre` is no longer needed for
|
||||
`agent-framework` and `agent-framework-foundry`. However,
|
||||
`agent-framework-orchestrations` and `agent-framework-azure-ai-search` are
|
||||
still in preview and require `--pre`.
|
||||
|
||||
---
|
||||
|
||||
### I'm getting dependency conflicts after upgrading from an RC version
|
||||
|
||||
RC installs (`1.0.0rc*` or `1.0.0b*`) are incompatible with the GA release.
|
||||
The GA packages enforce `>=1.0.1,<2` dependency floors.
|
||||
|
||||
Clean install:
|
||||
|
||||
pip uninstall agent-framework agent-framework-core agent-framework-openai agent-framework-foundry -y
|
||||
pip install agent-framework>=1.0.1 agent-framework-foundry>=1.0.1
|
||||
|
||||
Preview packages (orchestrations, Azure AI Search, and connectors like
|
||||
`agent-framework-copilotstudio`) still require `--pre` on a separate
|
||||
install command:
|
||||
|
||||
pip install agent-framework-orchestrations agent-framework-azure-ai-search --pre
|
||||
|
||||
Do not mix `--pre` and non-`--pre` packages in a single install command.
|
||||
|
||||
---
|
||||
|
||||
### `agent-framework-azure-ai` cannot be found
|
||||
|
||||
There is no package called `agent-framework-azure-ai`. For Foundry project
|
||||
endpoints, use `agent-framework-foundry`:
|
||||
|
||||
pip install agent-framework-foundry
|
||||
|
||||
Import from `agent_framework.foundry`.
|
||||
|
||||
---
|
||||
|
||||
## Authentication & connections
|
||||
|
||||
### `401 Unauthorized` when calling Azure OpenAI
|
||||
|
||||
Your API key is missing, empty, or pointing at the wrong resource. Check:
|
||||
|
||||
1. Your `.env` file exists at the project root and is populated.
|
||||
2. `load_dotenv()` is called before any client is instantiated.
|
||||
3. `AZURE_OPENAI_ENDPOINT` ends with `.openai.azure.com/` (trailing slash matters).
|
||||
4. `AZURE_OPENAI_CHAT_DEPLOYMENT_NAME` matches the exact deployment name in
|
||||
Azure Portal → your OpenAI resource → Model deployments (case-sensitive).
|
||||
|
||||
Verify your key length is non-zero:
|
||||
|
||||
echo "Key length: $(echo $AZURE_OPENAI_API_KEY | wc -c)"
|
||||
|
||||
---
|
||||
|
||||
### `FoundryChatClient` is not connecting to my Foundry project
|
||||
|
||||
Verify your `.env` contains the correct Foundry project endpoint. The endpoint
|
||||
format is `https://<resource>.services.ai.azure.com`:
|
||||
|
||||
from agent_framework.foundry import FoundryChatClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
client = FoundryChatClient(
|
||||
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
|
||||
model=os.environ["FOUNDRY_MODEL"],
|
||||
credential=DefaultAzureCredential(),
|
||||
)
|
||||
|
||||
If using `DefaultAzureCredential`, ensure you are logged in (`az login`) or
|
||||
that a managed identity is assigned to your compute.
|
||||
|
||||
---
|
||||
|
||||
### I want to use managed identity instead of `DefaultAzureCredential`
|
||||
|
||||
See `phase-4-migrate-ops/4b-deployment/managed_identity.md` for step-by-step
|
||||
instructions. The short version: use `ManagedIdentityCredential()` as the
|
||||
`credential=` argument to `FoundryChatClient()` when deploying to Azure.
|
||||
|
||||
---
|
||||
|
||||
## Workflows & executors
|
||||
|
||||
### `workflow.run()` returns a result but `get_outputs()` is an empty list
|
||||
|
||||
The terminal executor is not yielding a workflow output. Check three things:
|
||||
|
||||
**1. The context annotation includes a workflow output type.**
|
||||
|
||||
Incorrect — sends a message but yields nothing:
|
||||
|
||||
async def handle(self, text: str, ctx: WorkflowContext[str]) -> None:
|
||||
await ctx.send_message(text)
|
||||
|
||||
Correct — yields a final workflow output:
|
||||
|
||||
async def handle(self, text: str, ctx: WorkflowContext[Never, str]) -> None:
|
||||
await ctx.yield_output(text)
|
||||
|
||||
**2. `ctx.yield_output()` is actually called.**
|
||||
Check for early returns or unhandled exceptions that might skip the call.
|
||||
|
||||
**3. The executor is connected to the workflow graph.**
|
||||
An executor that is not reachable from the `start_executor` via `add_edge()`
|
||||
will never run. Double-check your edge definitions.
|
||||
|
||||
---
|
||||
|
||||
### `TypeError` or `AttributeError` on `Message(text=...)`
|
||||
|
||||
The `text=` parameter on the `Message` constructor was removed in 1.0.
|
||||
Build messages using `contents=[...]` instead:
|
||||
|
||||
Incorrect:
|
||||
|
||||
message = Message(role="user", text="Hello")
|
||||
|
||||
Correct:
|
||||
|
||||
message = Message(role="user", contents=["Hello"])
|
||||
|
||||
Plain strings inside `contents=[]` are automatically normalised into text
|
||||
content, so `contents=["Hello"]` is the simplest form.
|
||||
|
||||
---
|
||||
|
||||
### My `add_edge()` condition function never fires
|
||||
|
||||
Condition functions receive the exact message passed to `ctx.send_message()`.
|
||||
Make sure the value and the condition logic match precisely:
|
||||
|
||||
# Executor sends:
|
||||
await ctx.send_message("safe")
|
||||
|
||||
# Condition must match that exact string:
|
||||
def is_safe(message: str) -> bool:
|
||||
return message == "safe"
|
||||
|
||||
A common mistake is sending a tagged string (e.g. `"billing||<question>"`)
|
||||
but writing the condition as if the message is a plain label. See
|
||||
`phase-2-rebuild/07_multi_agent.py` for a worked example of the tagged
|
||||
string pattern.
|
||||
|
||||
---
|
||||
|
||||
### My workflow runs but hangs and never completes
|
||||
|
||||
The most common cause is a circular edge definition — executor A sends to B,
|
||||
and B sends back to A, creating an infinite loop. MAF uses a superstep
|
||||
execution model and will keep iterating until it reaches the `max_iterations`
|
||||
limit (default: 100), then raise an error.
|
||||
|
||||
Check your `add_edge()` calls for cycles. If you need a loop intentionally,
|
||||
set `max_iterations` explicitly on `WorkflowBuilder`:
|
||||
|
||||
WorkflowBuilder(name="MyWorkflow", max_iterations=10)
|
||||
|
||||
---
|
||||
|
||||
### `WorkflowBuilder` raises a validation error at `.build()`
|
||||
|
||||
MAF validates the workflow graph at build time. Common causes:
|
||||
|
||||
- **No start executor set** — you must pass `start_executor=` to
|
||||
`WorkflowBuilder(...)`.
|
||||
- **Type mismatch on an edge** — the output type of executor A does not match
|
||||
the input type of executor B. Check that `WorkflowContext[T_Out]` in the
|
||||
upstream executor matches the handler parameter type in the downstream one.
|
||||
- **Duplicate executor ID** — each executor must have a unique `id=` value.
|
||||
- **Unreachable executor** — an executor passed to `add_edge()` but not
|
||||
connected to the graph via any edge path from the start executor.
|
||||
|
||||
---
|
||||
|
||||
### Fan-in aggregation is not waiting for all parallel branches
|
||||
|
||||
`add_fan_in_edges()` waits for all listed sources to complete before firing.
|
||||
Make sure every parallel executor in the fan-out is also listed in the fan-in:
|
||||
|
||||
.add_fan_out_edges(dispatch, [path_a, path_b])
|
||||
.add_fan_in_edges([path_a, path_b], aggregate) # both must be listed
|
||||
|
||||
If one branch is missing from `add_fan_in_edges()`, the aggregator may fire
|
||||
early with a partial result.
|
||||
|
||||
---
|
||||
|
||||
## Parity validation
|
||||
|
||||
### `TypeError: SimilarityEvaluator() missing required argument: 'model_config'`
|
||||
|
||||
`model_config` became a required argument in azure-ai-evaluation GA (1.16+).
|
||||
|
||||
Incorrect:
|
||||
|
||||
evaluator = SimilarityEvaluator()
|
||||
|
||||
Correct:
|
||||
|
||||
model_config = {
|
||||
"azure_endpoint": os.environ["AZURE_OPENAI_ENDPOINT"],
|
||||
"api_key": os.environ["AZURE_OPENAI_API_KEY"],
|
||||
"azure_deployment": os.environ["AZURE_OPENAI_CHAT_DEPLOYMENT_NAME"],
|
||||
}
|
||||
evaluator = SimilarityEvaluator(model_config=model_config, threshold=3)
|
||||
|
||||
---
|
||||
|
||||
### Similarity scores are unexpectedly low (< 2.0)
|
||||
|
||||
Check the keyword arguments passed to the evaluator. Using the wrong kwargs
|
||||
causes the evaluator to compare the wrong fields and score near zero:
|
||||
|
||||
Incorrect:
|
||||
|
||||
evaluator(answer=maf_answer, ground_truth=pf_answer)
|
||||
|
||||
Correct:
|
||||
|
||||
evaluator(query=question, response=maf_answer, ground_truth=pf_answer)
|
||||
|
||||
Also verify that the `pf_output` column in your CSV contains the actual
|
||||
text output from your PF app — not the input question.
|
||||
|
||||
---
|
||||
|
||||
### The parity check script is very slow
|
||||
|
||||
Use `parity_check_batch.py` instead of `parity_check.py`. The batch version
|
||||
runs all rows concurrently with `asyncio.gather()` and is significantly
|
||||
faster for test suites with 20+ rows.
|
||||
|
||||
---
|
||||
|
||||
## Tracing & deployment
|
||||
|
||||
### No traces appearing in Application Insights
|
||||
|
||||
Make sure **both** `configure_azure_monitor()` and `configure_otel_providers()`
|
||||
are called **before** any `workflow.run()` call — not after, and not inside a
|
||||
handler. They must run at application startup:
|
||||
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
from agent_framework.observability import configure_otel_providers
|
||||
|
||||
configure_azure_monitor(
|
||||
connection_string=os.environ["APPLICATIONINSIGHTS_CONNECTION_STRING"]
|
||||
)
|
||||
configure_otel_providers()
|
||||
# workflow.run() calls go after this
|
||||
|
||||
`configure_azure_monitor()` sets up the Application Insights exporter.
|
||||
`configure_otel_providers()` enables MAF's built-in instrumentation so that
|
||||
executor transitions, agent calls, and LLM requests produce workflow-level
|
||||
spans. Without it, you will see Application Insights metadata but no
|
||||
workflow-specific trace data.
|
||||
|
||||
Also verify the connection string is correct: Azure Portal → your Application
|
||||
Insights resource → Overview → Connection String.
|
||||
|
||||
---
|
||||
|
||||
### Traces show Application Insights data but no workflow steps
|
||||
|
||||
You called `configure_azure_monitor()` but not `configure_otel_providers()`.
|
||||
The Azure Monitor function only handles transport, but MAF workflows require
|
||||
`configure_otel_providers()` from `agent_framework.observability` to generate
|
||||
executor-level spans. See the entry above for the correct two-step setup.
|
||||
|
||||
---
|
||||
|
||||
### Online endpoint deployment stays in `Updating` state or fails
|
||||
|
||||
Check the deployment logs for the actual error:
|
||||
|
||||
az ml online-deployment get-logs \
|
||||
--endpoint-name maf-endpoint --name blue \
|
||||
--resource-group <rg> --workspace-name <ws>
|
||||
|
||||
Common causes:
|
||||
|
||||
1. **`score.py` import error** — a missing dependency or an incorrect
|
||||
`sys.path` in `init()`. Make sure every package used by `score.py` and
|
||||
the workflow file is listed in `conda.yml`.
|
||||
2. **`init()` raises an exception** — the managed online endpoint calls
|
||||
`init()` once at container startup. If it throws, the container is
|
||||
marked unhealthy and the deployment fails. Run your workflow locally
|
||||
first to rule out runtime errors.
|
||||
3. **Quota exceeded** — the subscription does not have enough quota for the
|
||||
requested instance type. Check regional quota in the Azure Portal or
|
||||
run `az ml online-deployment list --endpoint-name <name>`.
|
||||
|
||||
---
|
||||
|
||||
### `az ml online-endpoint invoke` returns a scoring error
|
||||
|
||||
The endpoint is running but `run()` in `score.py` raised an exception.
|
||||
Retrieve the full traceback from the deployment logs (see above).
|
||||
|
||||
Frequent causes:
|
||||
|
||||
- **Empty or malformed request body** — the `run(raw_data)` function expects
|
||||
a JSON string with a `"question"` key. Verify your request file:
|
||||
|
||||
echo '{"question": "What is MAF?"}' > request.json
|
||||
az ml online-endpoint invoke \
|
||||
--name maf-endpoint --request-file request.json \
|
||||
--resource-group <rg> --workspace-name <ws>
|
||||
|
||||
- **Wrong workflow file** — `score.py` defaults to
|
||||
`phase-2-rebuild/01_linear_flow.py`. Override via the
|
||||
`MAF_WORKFLOW_FILE` environment variable in `deployment.yml`.
|
||||
- **Workflow produced no output** — see the
|
||||
"`workflow.run()` returns a result but `get_outputs()` is an empty list"
|
||||
section above.
|
||||
|
||||
---
|
||||
|
||||
### Managed identity authentication fails on the online endpoint
|
||||
|
||||
If `score.py` uses `DefaultAzureCredential` or `ManagedIdentityCredential`
|
||||
to call Foundry or other Azure services, ensure:
|
||||
|
||||
1. A system-assigned or user-assigned managed identity is enabled on the
|
||||
online endpoint.
|
||||
2. The identity has the required role assignments (e.g.
|
||||
`Cognitive Services User` on the Foundry resource).
|
||||
|
||||
See `phase-4-migrate-ops/4b-deployment/managed_identity.md` for full
|
||||
instructions.
|
||||
|
||||
---
|
||||
|
||||
### How do I update an existing deployment without downtime?
|
||||
|
||||
Use `az ml online-deployment update` with the same deployment name. The
|
||||
managed endpoint performs a rolling update. To do a blue/green swap instead,
|
||||
create a second deployment and shift traffic:
|
||||
|
||||
az ml online-endpoint update --name maf-endpoint \
|
||||
--traffic "blue=0 green=100" \
|
||||
--resource-group <rg> --workspace-name <ws>
|
||||
|
||||
---
|
||||
|
||||
## Reference links
|
||||
|
||||
- [Azure ML managed online endpoints overview](https://learn.microsoft.com/en-us/azure/machine-learning/concept-endpoints-online)
|
||||
- [Deploy and score a model with a managed online endpoint](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-deploy-online-endpoints)
|
||||
- [Troubleshoot online endpoint deployments](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-troubleshoot-online-endpoints)
|
||||
- [Managed online endpoint VM SKU list](https://learn.microsoft.com/en-us/azure/machine-learning/reference-managed-online-endpoints-vm-sku-list)
|
||||
|
||||
---
|
||||
|
||||
## Still stuck?
|
||||
|
||||
- [MAF GitHub Issues](https://github.com/microsoft/agent-framework/issues/)
|
||||
- [MAF Workflows documentation](https://learn.microsoft.com/en-us/agent-framework/workflows/executors)
|
||||
- [Azure AI Evaluation SDK reference](https://learn.microsoft.com/en-us/python/api/overview/azure/ai-evaluation-readme?view=azure-python)
|
||||
- Open an issue in this repo using the [migration issue template](../.github/ISSUE_TEMPLATE/migration-issue.md)
|
||||
@@ -0,0 +1,203 @@
|
||||
# Replacing Prompt Flow Connections
|
||||
|
||||
Prompt Flow connections (`AzureOpenAIConnection`, `OpenAIConnection`,
|
||||
`CognitiveSearchConnection`, `CustomConnection`, etc.) are **no longer
|
||||
supported** for migrated Microsoft Agent Framework workflows. Prompt Flow itself
|
||||
is being retired (feature freeze 20 April 2026, full retirement 20 April 2027),
|
||||
so the connection registry it relied on is going away too.
|
||||
|
||||
This affects both places where connections are commonly used:
|
||||
|
||||
- SDK clients in migrated workflows, such as `FoundryChatClient` or Azure AI
|
||||
Search clients.
|
||||
- Your own Python code that used to receive a Prompt Flow connection object,
|
||||
such as a tool function that read `connection.api_key`, `connection.configs`,
|
||||
or `connection.secrets`.
|
||||
|
||||
In migrated code, pass credentials and configuration explicitly. Pick one of
|
||||
the alternatives below.
|
||||
|
||||
---
|
||||
|
||||
## 1. Identity-based authentication (recommended)
|
||||
|
||||
No secret to store, rotate, or leak. Use Microsoft Entra ID and grant the
|
||||
right RBAC role to your user (local) or managed identity (production).
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
from agent_framework.foundry import FoundryChatClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
client = FoundryChatClient(
|
||||
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
|
||||
model=os.environ["FOUNDRY_MODEL"],
|
||||
credential=DefaultAzureCredential(),
|
||||
)
|
||||
```
|
||||
|
||||
`DefaultAzureCredential` works locally after `az login` and in Azure when the
|
||||
hosting resource has a managed identity. Swap to `ManagedIdentityCredential()`
|
||||
when you want to be explicit in production. See
|
||||
[phase-4-migrate-ops/4b-deployment/managed_identity.md](./phase-4-migrate-ops/4b-deployment/managed_identity.md).
|
||||
|
||||
Required roles (assign to the user / managed identity):
|
||||
|
||||
| Service | Role |
|
||||
|---|---|
|
||||
| Azure OpenAI | `Cognitive Services OpenAI User` |
|
||||
| Microsoft Foundry project | `Azure AI Developer` |
|
||||
| Azure AI Search | `Search Index Data Reader` (+ `Search Service Contributor` if you create indexes) |
|
||||
|
||||
Docs:
|
||||
- [DefaultAzureCredential overview](https://learn.microsoft.com/azure/developer/python/sdk/authentication/credential-chains#defaultazurecredential-overview)
|
||||
- [Managed identities for Azure resources](https://learn.microsoft.com/entra/identity/managed-identities-azure-resources/overview)
|
||||
- [Authenticate to Azure OpenAI with Entra ID](https://learn.microsoft.com/azure/ai-services/openai/how-to/managed-identity)
|
||||
|
||||
---
|
||||
|
||||
## 2. Workspace / project connections
|
||||
|
||||
If you already have connections in a Microsoft Foundry project or an Azure
|
||||
ML workspace and want to keep them, read them at runtime instead of
|
||||
re-creating them as PF objects.
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
from azure.ai.projects import AIProjectClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
project = AIProjectClient(
|
||||
endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
|
||||
credential=DefaultAzureCredential(),
|
||||
)
|
||||
conn = project.connections.get(name="my-connection", include_credentials=True)
|
||||
|
||||
# Connection is a mapping-like object. Common fields include:
|
||||
# conn.name, conn.type, conn.target, conn.credentials, conn.metadata
|
||||
target = conn.target
|
||||
credentials = conn.credentials
|
||||
|
||||
# Pass the values your code needs into the client or function of your choice.
|
||||
```
|
||||
|
||||
Use this option when you want a short-term bridge from existing workspace or
|
||||
project connections. For new production code, identity-based authentication is
|
||||
usually cleaner.
|
||||
|
||||
Docs:
|
||||
- [Foundry project connections](https://learn.microsoft.com/azure/ai-foundry/how-to/connections-add)
|
||||
- [`azure-ai-projects` Python SDK](https://learn.microsoft.com/python/api/overview/azure/ai-projects-readme)
|
||||
- [`ConnectionsOperations.get`](https://learn.microsoft.com/python/api/azure-ai-projects/azure.ai.projects.operations.connectionsoperations?view=azure-python#azure-ai-projects-operations-connectionsoperations-get)
|
||||
- [Azure ML workspace connections](https://learn.microsoft.com/azure/machine-learning/how-to-connection)
|
||||
|
||||
---
|
||||
|
||||
## 3. Environment variables and `.env` files
|
||||
|
||||
Simplest path for local development and CI. The repo uses this for every
|
||||
phase-2 sample — see [.env.example](./.env.example).
|
||||
|
||||
```bash
|
||||
# .env (do not commit)
|
||||
FOUNDRY_PROJECT_ENDPOINT=https://<resource>.services.ai.azure.com
|
||||
FOUNDRY_MODEL=gpt-4o
|
||||
```
|
||||
|
||||
```python
|
||||
import os
|
||||
from dotenv import load_dotenv
|
||||
|
||||
load_dotenv()
|
||||
endpoint = os.environ["FOUNDRY_PROJECT_ENDPOINT"]
|
||||
```
|
||||
|
||||
Docs:
|
||||
- [`python-dotenv`](https://pypi.org/project/python-dotenv/)
|
||||
- [App Service app settings](https://learn.microsoft.com/azure/app-service/configure-common#configure-app-settings)
|
||||
- [Azure Container Apps environment variables](https://learn.microsoft.com/azure/container-apps/environment-variables)
|
||||
|
||||
> Keep `.env` in `.gitignore`. Use this for **non-secret** config; combine
|
||||
> with option 1 or 4 for anything sensitive.
|
||||
|
||||
---
|
||||
|
||||
## 4. Azure Key Vault
|
||||
|
||||
When you must keep an API key (third-party model provider, legacy system),
|
||||
store it in Key Vault and read it at startup.
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.keyvault.secrets import SecretClient
|
||||
|
||||
kv = SecretClient(
|
||||
vault_url=os.environ["KEY_VAULT_URL"],
|
||||
credential=DefaultAzureCredential(),
|
||||
)
|
||||
api_key = kv.get_secret("openai-api-key").value
|
||||
```
|
||||
|
||||
Docs:
|
||||
- [Key Vault overview](https://learn.microsoft.com/azure/key-vault/general/overview)
|
||||
- [`azure-keyvault-secrets` Python SDK](https://learn.microsoft.com/python/api/overview/azure/keyvault-secrets-readme)
|
||||
|
||||
If your app runs on App Service or Azure Functions, you can also expose a Key
|
||||
Vault secret as an app setting and keep your Python code using
|
||||
`os.environ["SETTING_NAME"]`. For Azure Container Apps, use Container Apps
|
||||
secrets or environment variables.
|
||||
|
||||
---
|
||||
|
||||
## Migrating user code that used a connection
|
||||
|
||||
Old Prompt Flow tools often accepted a connection object directly:
|
||||
|
||||
```python
|
||||
def call_service(question: str, connection) -> str:
|
||||
endpoint = connection.configs["endpoint"]
|
||||
api_key = connection.secrets["api_key"]
|
||||
return call_api(endpoint=endpoint, api_key=api_key, question=question)
|
||||
```
|
||||
|
||||
In migrated code, make the inputs explicit. For non-secret config, read from
|
||||
environment variables. For secrets, prefer identity-based authentication or read
|
||||
from Key Vault.
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
from azure.identity import DefaultAzureCredential
|
||||
from azure.keyvault.secrets import SecretClient
|
||||
|
||||
def load_service_settings() -> tuple[str, str]:
|
||||
endpoint = os.environ["SERVICE_ENDPOINT"]
|
||||
kv = SecretClient(
|
||||
vault_url=os.environ["KEY_VAULT_URL"],
|
||||
credential=DefaultAzureCredential(),
|
||||
)
|
||||
api_key = kv.get_secret("service-api-key").value
|
||||
return endpoint, api_key
|
||||
|
||||
|
||||
def call_service(question: str) -> str:
|
||||
endpoint, api_key = load_service_settings()
|
||||
return call_api(endpoint=endpoint, api_key=api_key, question=question)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick mapping
|
||||
|
||||
| Prompt Flow connection | Recommended replacement |
|
||||
|---|---|
|
||||
| `AzureOpenAIConnection` (key) | Option 1 (Entra ID) — fall back to Option 4 |
|
||||
| `AzureOpenAIConnection` already in workspace/project | Option 2 |
|
||||
| `OpenAIConnection` (third-party key) | Option 4 |
|
||||
| `CognitiveSearchConnection` | Option 1 |
|
||||
| `CustomConnection.configs` | Option 3 |
|
||||
| `CustomConnection.secrets` | Option 4 |
|
||||
@@ -0,0 +1,36 @@
|
||||
# Phase 1 — Audit & Map
|
||||
|
||||
Before writing any MAF code, understand what you have. This phase is diagnostic — no code changes, no deployments.
|
||||
|
||||
## Step 1a: Export your flow structure
|
||||
|
||||
Run this from the Prompt Flow CLI to get a full YAML representation of your flow's nodes, types, and wiring:
|
||||
|
||||
```bash
|
||||
# Exports flow.dag.yaml into ./flow_export/
|
||||
pf flow export --source <your-flow-directory> --output ./flow_export
|
||||
|
||||
```
|
||||
|
||||
Open flow_export/flow.dag.yaml. It lists every node with:
|
||||
|
||||
- type: llm, python, or prompt
|
||||
- inputs: what data each node receives
|
||||
- outputs: what it passes downstream
|
||||
|
||||
This YAML is your migration blueprint. Keep it open while working through Phase 2.
|
||||
|
||||
## Step 1b: Map each node to its MAF equivalent
|
||||
|
||||
See [node-mapping.md](./node-mapping.md) for the full table.
|
||||
|
||||
The core mental model:
|
||||
- Every node → one Executor class with a @handler method
|
||||
- The flow graph → a WorkflowBuilder chain with .add_edge() calls
|
||||
- Connections (credentials) → environment variables in .env
|
||||
|
||||
## Checklist before moving to Phase 2
|
||||
- [ ] flow.dag.yaml exported and reviewed
|
||||
- [ ] Every node has a mapped MAF equivalent (see [node-mapping.md](./node-mapping.md))
|
||||
- [ ] .env file populated (copy [.env.example](../.env.example) from repo root)
|
||||
- [ ] You know which samples in [phase-2-rebuild](../phase-2-rebuild/) match your flow's patterns
|
||||
@@ -0,0 +1,32 @@
|
||||
# Prompt Flow → MAF Node Mapping
|
||||
|
||||
| Prompt Flow Concept | MAF Equivalent | Sample |
|
||||
|---|---|---|
|
||||
| Flow (YAML / visual graph) | `Workflow` built with `WorkflowBuilder` | All samples |
|
||||
| Node (any step) | `Executor` class with a `@handler` method | All samples |
|
||||
| LLM node | `Agent(client=FoundryChatClient(...), instructions=...)` inside an `Executor` | `01_linear_flow.py` |
|
||||
| Python node | Plain Python logic inside an `Executor @handler` | `02_python_node.py` |
|
||||
| Prompt node | String formatting inside an `Executor @handler` | `02_python_node.py` |
|
||||
| Embed Text + Vector Lookup nodes | `AzureAISearchContextProvider` via `context_providers=[...]` | `05_rag_flow.py` |
|
||||
| If / conditional node | `.add_edge(source, target, condition=fn)` | `03_conditional_flow.py` |
|
||||
| Parallel nodes (no dependencies) | `.add_fan_out_edges(source, [targetA, targetB])` | `04_parallel_flow.py` |
|
||||
| Merge / aggregate node | `.add_fan_in_edges([sourceA, sourceB], target)` | `04_parallel_flow.py` |
|
||||
| Flow Inputs | Type annotation on the start `Executor`'s `@handler` parameter | All samples |
|
||||
| Flow Outputs | `await ctx.yield_output(value)` in the terminal `Executor` | All samples |
|
||||
| Connections (credentials) | Identity-based auth, workspace connections, env vars, or Key Vault | [`connections-alternatives.md`](../connections-alternatives.md) |
|
||||
| Evaluation flow | `SimilarityEvaluator` from Azure AI Evaluation SDK | `phase-3-validate/` |
|
||||
| Multi-step specialist routing | `HandoffBuilder` from `agent-framework-orchestrations` | `07_multi_agent.py` |
|
||||
| Managed Online Endpoint | AML managed online endpoint via `score.py` init()/run() pattern | `phase-4-migrate-ops/4b-deployment/` |
|
||||
|
||||
---
|
||||
|
||||
## WorkflowContext type parameters
|
||||
|
||||
| Annotation | Behaviour |
|
||||
|---|---|
|
||||
| `WorkflowContext` | Side effects only — no output sent |
|
||||
| `WorkflowContext[str]` | Sends a `str` downstream via `ctx.send_message()` |
|
||||
| `WorkflowContext[Never, str]` | Yields a `str` as the final workflow output via `ctx.yield_output()` |
|
||||
| `WorkflowContext[str, str]` | Both sends a message downstream AND yields a workflow output |
|
||||
|
||||
`Never` is imported from `typing_extensions`.
|
||||
@@ -0,0 +1,80 @@
|
||||
"""
|
||||
Migrates the simplest Prompt Flow pattern: Input node → LLM node.
|
||||
|
||||
Prompt Flow equivalent:
|
||||
[Input node] --> [LLM node]
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from typing_extensions import Never
|
||||
|
||||
from agent_framework import Agent, Executor, WorkflowBuilder, WorkflowContext, handler
|
||||
from agent_framework.foundry import FoundryChatClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
load_dotenv()
|
||||
|
||||
|
||||
class InputExecutor(Executor):
|
||||
"""Replaces the Prompt Flow Input node.
|
||||
|
||||
WorkflowContext[str] sends a str to the next executor via ctx.send_message().
|
||||
"""
|
||||
|
||||
@handler
|
||||
async def receive(self, question: str, ctx: WorkflowContext[str]) -> None:
|
||||
await ctx.send_message(question.strip())
|
||||
|
||||
|
||||
class LLMExecutor(Executor):
|
||||
"""Replaces the Prompt Flow LLM node.
|
||||
|
||||
WorkflowContext[Never, str]:
|
||||
- Never → does not send messages to downstream executors
|
||||
- str → yields a str as the final workflow output via ctx.yield_output()
|
||||
|
||||
FoundryChatClient connects to a Microsoft Foundry project endpoint.
|
||||
It reads FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_MODEL from the environment.
|
||||
|
||||
The 'instructions' parameter replaces the system prompt template in the PF LLM node.
|
||||
"""
|
||||
|
||||
def __init__(self, **kwargs):
|
||||
super().__init__(**kwargs)
|
||||
client = FoundryChatClient(
|
||||
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
|
||||
model=os.environ["FOUNDRY_MODEL"],
|
||||
credential=DefaultAzureCredential(),
|
||||
)
|
||||
self._agent = Agent(
|
||||
client=client,
|
||||
name="QAAgent",
|
||||
instructions="You are a helpful assistant. Answer concisely.",
|
||||
)
|
||||
|
||||
@handler
|
||||
async def call_llm(self, question: str, ctx: WorkflowContext[Never, str]) -> None:
|
||||
result = await self._agent.run(question)
|
||||
await ctx.yield_output(result)
|
||||
|
||||
|
||||
_input = InputExecutor(id="input")
|
||||
_llm = LLMExecutor(id="llm")
|
||||
|
||||
workflow = (
|
||||
WorkflowBuilder(name="LinearWorkflow", start_executor=_input)
|
||||
.add_edge(_input, _llm)
|
||||
.build()
|
||||
)
|
||||
|
||||
|
||||
async def main():
|
||||
result = await workflow.run("What is retrieval-augmented generation?")
|
||||
print(result.get_outputs()[0])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,59 @@
|
||||
"""
|
||||
Migrates a Prompt Flow Python code node.
|
||||
|
||||
In MAF, custom logic lives directly inside a @handler method — no separate
|
||||
YAML snippet, no per-node file registration required.
|
||||
|
||||
Prompt Flow equivalent:
|
||||
def clean_text(text: str) -> str:
|
||||
return text.strip().upper()
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from typing_extensions import Never
|
||||
|
||||
from agent_framework import Executor, WorkflowBuilder, WorkflowContext, handler
|
||||
|
||||
load_dotenv()
|
||||
|
||||
|
||||
class TextCleanerExecutor(Executor):
|
||||
"""Replaces the Prompt Flow Python node.
|
||||
|
||||
Any Python logic — library calls, data transforms, API calls — goes
|
||||
inside the @handler method. WorkflowContext[str] sends the result downstream.
|
||||
"""
|
||||
|
||||
@handler
|
||||
async def clean(self, text: str, ctx: WorkflowContext[str]) -> None:
|
||||
cleaned = text.strip().upper()
|
||||
await ctx.send_message(cleaned)
|
||||
|
||||
|
||||
class OutputExecutor(Executor):
|
||||
"""Terminal executor — yields the final workflow output."""
|
||||
|
||||
@handler
|
||||
async def output(self, text: str, ctx: WorkflowContext[Never, str]) -> None:
|
||||
await ctx.yield_output(text)
|
||||
|
||||
|
||||
_cleaner = TextCleanerExecutor(id="cleaner")
|
||||
_output = OutputExecutor(id="output")
|
||||
|
||||
workflow = (
|
||||
WorkflowBuilder(name="PythonNodeWorkflow", start_executor=_cleaner)
|
||||
.add_edge(_cleaner, _output)
|
||||
.build()
|
||||
)
|
||||
|
||||
|
||||
async def main():
|
||||
result = await workflow.run(" hello from prompt flow ")
|
||||
print(result.get_outputs()[0]) # → "HELLO FROM PROMPT FLOW"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,93 @@
|
||||
"""
|
||||
Migrates a Prompt Flow conditional (If) node.
|
||||
|
||||
In MAF, branching is expressed by passing a condition callable to add_edge().
|
||||
An edge fires only when the condition returns True for the outgoing message.
|
||||
|
||||
Prompt Flow equivalent:
|
||||
activate_config: ${classify.output} == "safe"
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from typing import TypedDict
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from typing_extensions import Never
|
||||
|
||||
from agent_framework import Executor, WorkflowBuilder, WorkflowContext, handler
|
||||
|
||||
load_dotenv()
|
||||
|
||||
|
||||
class ClassifiedMessage(TypedDict):
|
||||
label: str
|
||||
text: str
|
||||
|
||||
|
||||
class ClassifyExecutor(Executor):
|
||||
"""Classifies input and sends a structured payload downstream.
|
||||
|
||||
Replace the body of classify() with your real classification logic
|
||||
(e.g. an LLM call, a rules engine, a model inference call).
|
||||
"""
|
||||
|
||||
@handler
|
||||
async def classify(self, text: str, ctx: WorkflowContext[ClassifiedMessage]) -> None:
|
||||
label = "unsafe" if "bad_word" in text.lower() else "safe"
|
||||
await ctx.send_message({"label": label, "text": text})
|
||||
|
||||
|
||||
class SafeHandlerExecutor(Executor):
|
||||
"""Handles input that passed the safety check."""
|
||||
|
||||
@handler
|
||||
async def handle_safe(self, message: ClassifiedMessage, ctx: WorkflowContext[Never, str]) -> None:
|
||||
await ctx.yield_output(f"Processed: {message['text']}")
|
||||
|
||||
|
||||
class FlaggedHandlerExecutor(Executor):
|
||||
"""Handles input that failed the safety check."""
|
||||
|
||||
@handler
|
||||
async def handle_flagged(self, message: ClassifiedMessage, ctx: WorkflowContext[Never, str]) -> None:
|
||||
await ctx.yield_output(f"Flagged for review: {message['text']}")
|
||||
|
||||
|
||||
def is_safe(message: ClassifiedMessage) -> bool:
|
||||
"""Condition function passed to add_edge().
|
||||
|
||||
Receives the outgoing message from ClassifyExecutor.
|
||||
Returns True to fire the edge, False to suppress it.
|
||||
"""
|
||||
return message["label"] == "safe"
|
||||
|
||||
|
||||
def is_unsafe(message: ClassifiedMessage) -> bool:
|
||||
"""Explicit negation of is_safe — clearer and more testable than a lambda."""
|
||||
return message["label"] == "unsafe"
|
||||
|
||||
|
||||
# Two edges leave ClassifyExecutor — only one fires per run.
|
||||
# This is the MAF equivalent of the PF If node.
|
||||
_classify = ClassifyExecutor(id="classify")
|
||||
_safe_handler = SafeHandlerExecutor(id="safe")
|
||||
_flagged_handler = FlaggedHandlerExecutor(id="flagged")
|
||||
|
||||
workflow = (
|
||||
WorkflowBuilder(name="ConditionalWorkflow", start_executor=_classify)
|
||||
.add_edge(_classify, _safe_handler, condition=is_safe)
|
||||
.add_edge(_classify, _flagged_handler, condition=is_unsafe)
|
||||
.build()
|
||||
)
|
||||
|
||||
|
||||
async def main():
|
||||
result_safe = await workflow.run("this is fine")
|
||||
print(result_safe.get_outputs()[0])
|
||||
|
||||
result_flagged = await workflow.run("contains bad_word")
|
||||
print(result_flagged.get_outputs()[0])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,82 @@
|
||||
"""
|
||||
Migrates Prompt Flow parallel nodes — two nodes with no shared dependencies
|
||||
that both feed into a merge/aggregate node.
|
||||
|
||||
In MAF:
|
||||
- add_fan_out_edges() broadcasts one message to multiple executors concurrently
|
||||
- add_fan_in_edges() waits for ALL upstream executors before proceeding
|
||||
|
||||
Prompt Flow equivalent:
|
||||
[Dispatch] --> [NodeA] (no deps) --> [Merge]
|
||||
--> [NodeB] (no deps) --> [Merge]
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from typing_extensions import Never
|
||||
|
||||
from agent_framework import Executor, WorkflowBuilder, WorkflowContext, handler
|
||||
|
||||
load_dotenv()
|
||||
|
||||
|
||||
class DispatchExecutor(Executor):
|
||||
"""Entry point — broadcasts the input to both parallel paths."""
|
||||
|
||||
@handler
|
||||
async def dispatch(self, text: str, ctx: WorkflowContext[str]) -> None:
|
||||
await ctx.send_message(text)
|
||||
|
||||
|
||||
class PathAExecutor(Executor):
|
||||
"""First parallel branch. Replace with your actual processing logic."""
|
||||
|
||||
@handler
|
||||
async def process_a(self, text: str, ctx: WorkflowContext[str]) -> None:
|
||||
await ctx.send_message(f"PathA: {text.upper()}")
|
||||
|
||||
|
||||
class PathBExecutor(Executor):
|
||||
"""Second parallel branch. Replace with your actual processing logic."""
|
||||
|
||||
@handler
|
||||
async def process_b(self, text: str, ctx: WorkflowContext[str]) -> None:
|
||||
await ctx.send_message(f"PathB: {text[::-1]}")
|
||||
|
||||
|
||||
class AggregatorExecutor(Executor):
|
||||
"""Merge node — runs only after both PathA and PathB have completed.
|
||||
|
||||
fan-in delivers all upstream results as list[str].
|
||||
The order of results matches the order executors were declared in
|
||||
add_fan_in_edges() — [PathA_result, PathB_result] — so it is safe to
|
||||
rely on positional indexing if needed.
|
||||
"""
|
||||
|
||||
@handler
|
||||
async def aggregate(self, results: list[str], ctx: WorkflowContext[Never, str]) -> None:
|
||||
combined = " | ".join(results)
|
||||
await ctx.yield_output(combined)
|
||||
|
||||
|
||||
_dispatch = DispatchExecutor(id="dispatch")
|
||||
_path_a = PathAExecutor(id="path_a")
|
||||
_path_b = PathBExecutor(id="path_b")
|
||||
_aggregate = AggregatorExecutor(id="aggregate")
|
||||
|
||||
workflow = (
|
||||
WorkflowBuilder(name="ParallelWorkflow", start_executor=_dispatch)
|
||||
.add_fan_out_edges(_dispatch, [_path_a, _path_b])
|
||||
.add_fan_in_edges([_path_a, _path_b], _aggregate)
|
||||
.build()
|
||||
)
|
||||
|
||||
|
||||
async def main():
|
||||
result = await workflow.run("hello")
|
||||
print(result.get_outputs()[0]) # → "PathA: HELLO | PathB: olleh" (order matches add_fan_in_edges declaration)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,84 @@
|
||||
"""
|
||||
Migrates a Prompt Flow RAG pipeline.
|
||||
|
||||
AzureAISearchContextProvider replaces all three PF RAG nodes — it handles
|
||||
embedding generation and vector similarity search automatically before
|
||||
every agent.run() call.
|
||||
|
||||
Prompt Flow equivalent (3 separate nodes):
|
||||
[Embed Text] --> [Vector DB Lookup] --> [LLM node]
|
||||
|
||||
Prerequisites:
|
||||
pip install agent-framework-azure-ai-search agent-framework-foundry azure-identity
|
||||
Set in .env: FOUNDRY_PROJECT_ENDPOINT, FOUNDRY_MODEL,
|
||||
AZURE_AI_SEARCH_ENDPOINT, AZURE_AI_SEARCH_INDEX_NAME,
|
||||
AZURE_AI_SEARCH_API_KEY
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from typing_extensions import Never
|
||||
|
||||
from agent_framework import Agent, Executor, WorkflowBuilder, WorkflowContext, handler
|
||||
from agent_framework.foundry import FoundryChatClient
|
||||
from agent_framework_azure_ai_search import AzureAISearchContextProvider
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
load_dotenv()
|
||||
|
||||
|
||||
# Replaces: Embed Text node + Vector DB Lookup node.
|
||||
# Queries Azure AI Search and injects retrieved documents into the agent's
|
||||
# context automatically before every agent.run() call.
|
||||
search_provider = AzureAISearchContextProvider(
|
||||
endpoint=os.environ["AZURE_AI_SEARCH_ENDPOINT"],
|
||||
index_name=os.environ["AZURE_AI_SEARCH_INDEX_NAME"],
|
||||
api_key=os.environ["AZURE_AI_SEARCH_API_KEY"],
|
||||
)
|
||||
|
||||
|
||||
class RAGExecutor(Executor):
|
||||
"""Replaces the Embed Text, Vector DB Lookup, and LLM nodes combined.
|
||||
|
||||
Attaching context_providers=[search_provider] means retrieval runs
|
||||
automatically — no separate executor needed for embedding or search.
|
||||
"""
|
||||
|
||||
def __init__(self, **kwargs):
|
||||
super().__init__(**kwargs)
|
||||
client = FoundryChatClient(
|
||||
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
|
||||
model=os.environ["FOUNDRY_MODEL"],
|
||||
credential=DefaultAzureCredential(),
|
||||
)
|
||||
self._agent = Agent(
|
||||
client=client,
|
||||
name="DocQAAgent",
|
||||
instructions=(
|
||||
"You are a precise document Q&A assistant. "
|
||||
"Answer using ONLY the retrieved context provided. "
|
||||
"If the answer is not in the context, say 'I don't know'."
|
||||
),
|
||||
context_providers=[search_provider],
|
||||
)
|
||||
|
||||
@handler
|
||||
async def answer(self, question: str, ctx: WorkflowContext[Never, str]) -> None:
|
||||
result = await self._agent.run(question)
|
||||
await ctx.yield_output(result)
|
||||
|
||||
|
||||
_rag = RAGExecutor(id="rag")
|
||||
|
||||
workflow = WorkflowBuilder(name="RAGWorkflow", start_executor=_rag).build()
|
||||
|
||||
|
||||
async def main():
|
||||
result = await workflow.run("What is the refund policy?")
|
||||
print(result.get_outputs()[0])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,107 @@
|
||||
"""
|
||||
Migrates a Prompt Flow custom tool node or tool integration.
|
||||
|
||||
In MAF, Python functions are registered as agent tools by passing them to
|
||||
tools=[] on the agent. The agent calls them autonomously during a run based
|
||||
on its instructions and the user input.
|
||||
|
||||
Prompt Flow equivalent:
|
||||
[LLM node] --> [Python tool node] (e.g. a custom API call or lookup)
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from typing_extensions import Never
|
||||
|
||||
from agent_framework import Agent, Executor, WorkflowBuilder, WorkflowContext, handler
|
||||
from agent_framework.foundry import FoundryChatClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
load_dotenv()
|
||||
|
||||
|
||||
# --- Tool functions ----------------------------------------------------------
|
||||
# These replace Prompt Flow Python tool nodes.
|
||||
# Any plain Python function can be passed to tools=[].
|
||||
# The docstring is used by the agent to decide when and how to call the function.
|
||||
|
||||
def get_order_status(order_id: str) -> str:
|
||||
"""Look up the status of a customer order by order ID.
|
||||
|
||||
Args:
|
||||
order_id: The unique order identifier.
|
||||
|
||||
Returns:
|
||||
A string describing the current order status.
|
||||
"""
|
||||
# Replace with your real data source (database, API call, etc.)
|
||||
mock_orders = {
|
||||
"ORD-001": "Shipped — expected delivery 9 Apr 2026",
|
||||
"ORD-002": "Processing — not yet dispatched",
|
||||
"ORD-003": "Delivered — 3 Apr 2026",
|
||||
}
|
||||
return mock_orders.get(order_id, f"Order {order_id} not found.")
|
||||
|
||||
|
||||
def get_refund_policy() -> str:
|
||||
"""Return the company refund policy.
|
||||
|
||||
Returns:
|
||||
A string describing the refund policy.
|
||||
"""
|
||||
return "Refunds are accepted within 30 days of purchase with proof of receipt."
|
||||
|
||||
|
||||
# --- Executor ----------------------------------------------------------------
|
||||
|
||||
class ToolAgentExecutor(Executor):
|
||||
"""Replaces an LLM node wired to one or more Python tool nodes.
|
||||
|
||||
The agent decides autonomously which tools to call based on the user
|
||||
question and its instructions. tools=[] accepts plain Python callables.
|
||||
"""
|
||||
|
||||
def __init__(self, **kwargs):
|
||||
super().__init__(**kwargs)
|
||||
client = FoundryChatClient(
|
||||
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
|
||||
model=os.environ["FOUNDRY_MODEL"],
|
||||
credential=DefaultAzureCredential(),
|
||||
)
|
||||
self._agent = Agent(
|
||||
client=client,
|
||||
name="SupportAgent",
|
||||
instructions=(
|
||||
"You are a customer support assistant. "
|
||||
"Use the available tools to answer questions about orders and refunds. "
|
||||
"Always use a tool if the answer can be looked up — do not guess."
|
||||
),
|
||||
tools=[get_order_status, get_refund_policy],
|
||||
)
|
||||
|
||||
@handler
|
||||
async def run(self, question: str, ctx: WorkflowContext[Never, str]) -> None:
|
||||
result = await self._agent.run(question)
|
||||
await ctx.yield_output(result)
|
||||
|
||||
|
||||
_tool_agent = ToolAgentExecutor(id="tool_agent")
|
||||
|
||||
workflow = WorkflowBuilder(name="FunctionToolsWorkflow", start_executor=_tool_agent).build()
|
||||
|
||||
|
||||
async def main():
|
||||
questions = [
|
||||
"What is the status of order ORD-002?",
|
||||
"Can I get a refund on something I bought 2 weeks ago?",
|
||||
]
|
||||
for q in questions:
|
||||
result = await workflow.run(q)
|
||||
print(f"Q: {q}")
|
||||
print(f"A: {result.get_outputs()[0]}\n")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,109 @@
|
||||
"""
|
||||
Migrates a multi-step Prompt Flow that requires specialist handling based
|
||||
on the nature of the input — implemented in MAF as a multi-agent handoff
|
||||
using the HandoffBuilder from agent-framework-orchestrations.
|
||||
|
||||
A triage agent classifies the input and uses tool-based handoffs to route
|
||||
to specialist agents. The HandoffBuilder automatically generates handoff
|
||||
tools for each participant.
|
||||
|
||||
Prompt Flow equivalent:
|
||||
[Classify node] --> [SpecialistA LLM node] (if billing)
|
||||
--> [SpecialistB LLM node] (if technical)
|
||||
|
||||
Prerequisites:
|
||||
pip install agent-framework-orchestrations --pre
|
||||
Set in .env: FOUNDRY_PROJECT_ENDPOINT, FOUNDRY_MODEL
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from agent_framework import Agent
|
||||
from agent_framework.foundry import FoundryChatClient
|
||||
from agent_framework.orchestrations import HandoffBuilder
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
load_dotenv()
|
||||
|
||||
# One shared client is the recommended production pattern — instantiating a
|
||||
# separate client per agent is unnecessary and wastes connection resources.
|
||||
_client = FoundryChatClient(
|
||||
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
|
||||
model=os.environ["FOUNDRY_MODEL"],
|
||||
credential=DefaultAzureCredential(),
|
||||
)
|
||||
|
||||
triage_agent = Agent(
|
||||
client=_client,
|
||||
name="triage_agent",
|
||||
instructions=(
|
||||
"You are frontline support triage. Route customer issues to the "
|
||||
"appropriate specialist agent based on the problem described. "
|
||||
"Use the handoff tools to transfer to billing_agent or technical_agent."
|
||||
),
|
||||
)
|
||||
|
||||
billing_agent = Agent(
|
||||
client=_client,
|
||||
name="billing_agent",
|
||||
instructions=(
|
||||
"You are a billing support specialist. "
|
||||
"Answer questions about invoices, payments, and subscriptions concisely."
|
||||
),
|
||||
)
|
||||
|
||||
technical_agent = Agent(
|
||||
client=_client,
|
||||
name="technical_agent",
|
||||
instructions=(
|
||||
"You are a technical support specialist. "
|
||||
"Answer questions about product features, errors, and configuration concisely."
|
||||
),
|
||||
)
|
||||
|
||||
# HandoffBuilder automatically creates handoff tools for each participant,
|
||||
# allowing agents to transfer control by invoking a tool call (e.g.
|
||||
# handoff_to_billing_agent). This replaces the manual tagged-string
|
||||
# routing pattern with condition functions.
|
||||
workflow = (
|
||||
HandoffBuilder(
|
||||
name="MultiAgentHandoffWorkflow",
|
||||
participants=[triage_agent, billing_agent, technical_agent],
|
||||
# Stop when a specialist has responded (conversation has > 2 messages
|
||||
# and the last message is not from triage).
|
||||
termination_condition=lambda conversation: (
|
||||
len(conversation) > 2
|
||||
and conversation[-1].author_name != "triage_agent"
|
||||
),
|
||||
)
|
||||
.with_start_agent(triage_agent)
|
||||
.build()
|
||||
)
|
||||
|
||||
|
||||
async def main():
|
||||
questions = [
|
||||
"My invoice from last month looks wrong.",
|
||||
"The app keeps crashing when I open the settings page.",
|
||||
]
|
||||
for q in questions:
|
||||
result = await workflow.run(q)
|
||||
outputs = result.get_outputs()
|
||||
print(f"Q: {q}")
|
||||
if outputs:
|
||||
# The handoff workflow yields the full conversation as output.
|
||||
# The last message is typically the specialist's response.
|
||||
conversation = outputs[0]
|
||||
if isinstance(conversation, list) and conversation:
|
||||
last_msg = conversation[-1]
|
||||
speaker = getattr(last_msg, "author_name", None) or "assistant"
|
||||
print(f"A ({speaker}): {last_msg.text}\n")
|
||||
else:
|
||||
print(f"A: {conversation}\n")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -0,0 +1,58 @@
|
||||
# Phase 2 — Rebuild in MAF
|
||||
|
||||
Re-implement your Prompt Flow application using MAF's `WorkflowBuilder`
|
||||
and `Executor` pattern. Work through the samples in numbered order.
|
||||
|
||||
## Samples
|
||||
|
||||
| File | Prompt Flow pattern it replaces |
|
||||
|---|---|
|
||||
| [01_linear_flow.py](01_linear_flow.py) | Input node → LLM node |
|
||||
| [02_python_node.py](02_python_node.py) | Python code node with custom logic |
|
||||
| [03_conditional_flow.py](03_conditional_flow.py) | If / conditional node |
|
||||
| [04_parallel_flow.py](04_parallel_flow.py)| Parallel nodes with no shared dependencies |
|
||||
| [05_rag_flow.py](05_rag_flow.py) | Embed Text + Vector Lookup + LLM nodes (full RAG pipeline) |
|
||||
| [06_function_tools.py](06_function_tools.py) | Python tool node → function tools |
|
||||
| [07_multi_agent.py](07_multi_agent.py) | Multi-step specialist routing → `HandoffBuilder` multi-agent handoff |
|
||||
|
||||
## Run any sample
|
||||
|
||||
```bash
|
||||
cd phase-2-rebuild
|
||||
python 01_linear_flow.py
|
||||
```
|
||||
|
||||
## Visualize a workflow in DevUI
|
||||
|
||||
[`agent-framework-devui`](https://github.com/microsoft/agent-framework/tree/main/python/packages/devui)
|
||||
is a lightweight web app for inspecting the executor graph of a MAF workflow
|
||||
and running it interactively. [visualize_workflow.py](visualize_workflow.py)
|
||||
loads the module-level `workflow` object from one or more samples in this
|
||||
folder and hands them to DevUI's `serve()`.
|
||||
|
||||
```bash
|
||||
# One-time install (preview package)
|
||||
pip install agent-framework-devui --pre
|
||||
|
||||
cd phase-2-rebuild
|
||||
|
||||
# Visualize all 7 samples — pick one from the entity dropdown in the UI
|
||||
python visualize_workflow.py
|
||||
|
||||
# Or visualize a single sample
|
||||
python visualize_workflow.py --file 03_conditional_flow.py
|
||||
|
||||
```
|
||||
|
||||
By default the UI opens at <http://127.0.0.1:8080>. Pass `--port`, `--host`,
|
||||
or `--no-open` to override. Make sure your `.env` is filled in (see
|
||||
[.env.example](../.env.example)) — the samples that use `FoundryChatClient`
|
||||
or Azure AI Search will fail to load otherwise.
|
||||
|
||||
## The pattern every sample follows
|
||||
|
||||
- Define Executors — one class per logical step, each with a @handler method
|
||||
- Build the Workflow — connect executors with WorkflowBuilder and .add_edge()
|
||||
- Run — await workflow.run(input), read output from result.get_outputs()
|
||||
|
||||
See [node-mapping](../phase-1-audit/node-mapping.md) for the full PF → MAF concept mapping.
|
||||
@@ -0,0 +1,118 @@
|
||||
"""Visualize Phase 2 workflows in agent-framework-devui.
|
||||
|
||||
Loads one or more module-level ``workflow`` objects from this folder and
|
||||
launches the DevUI web app so you can inspect the executor graph and run
|
||||
the workflow interactively.
|
||||
|
||||
Examples:
|
||||
# Visualize all phase-2-rebuild samples (default)
|
||||
python visualize_workflow.py
|
||||
|
||||
# Visualize a single sample
|
||||
python visualize_workflow.py --file 03_conditional_flow.py
|
||||
|
||||
# Visualize multiple specific samples
|
||||
python visualize_workflow.py --file 01_linear_flow.py --file 04_parallel_flow.py
|
||||
|
||||
# Custom port, don't auto-open browser
|
||||
python visualize_workflow.py --port 9000 --no-open
|
||||
|
||||
Prerequisites:
|
||||
pip install agent-framework-devui --pre
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
GUIDE_ROOT = SCRIPT_DIR.parent
|
||||
|
||||
if str(GUIDE_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(GUIDE_ROOT))
|
||||
|
||||
from workflow_loader import load_workflow # noqa: E402
|
||||
|
||||
# Default sample set — every numbered workflow file in this folder.
|
||||
DEFAULT_SAMPLES = sorted(
|
||||
p.name for p in SCRIPT_DIR.glob("[0-9][0-9]_*.py")
|
||||
)
|
||||
|
||||
|
||||
def _load_one(file_name: str):
|
||||
"""Load a single workflow file by re-using ``workflow_loader.load_workflow``.
|
||||
|
||||
``load_workflow()`` reads the ``MAF_WORKFLOW_FILE`` env var, so we set it
|
||||
per file and restore the previous value afterwards.
|
||||
"""
|
||||
workflow_path = SCRIPT_DIR / file_name
|
||||
if not workflow_path.exists():
|
||||
raise FileNotFoundError(f"Workflow file not found: {workflow_path}")
|
||||
|
||||
previous = os.environ.get("MAF_WORKFLOW_FILE")
|
||||
os.environ["MAF_WORKFLOW_FILE"] = str(workflow_path)
|
||||
try:
|
||||
return load_workflow()
|
||||
finally:
|
||||
if previous is None:
|
||||
os.environ.pop("MAF_WORKFLOW_FILE", None)
|
||||
else:
|
||||
os.environ["MAF_WORKFLOW_FILE"] = previous
|
||||
|
||||
|
||||
def main() -> None:
|
||||
parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0])
|
||||
parser.add_argument(
|
||||
"--file",
|
||||
action="append",
|
||||
default=None,
|
||||
help=(
|
||||
"Workflow file under phase-2-rebuild/ to load. May be passed "
|
||||
"multiple times. Defaults to all numbered samples."
|
||||
),
|
||||
)
|
||||
parser.add_argument("--port", type=int, default=8080, help="Server port (default: 8080).")
|
||||
parser.add_argument("--host", default="127.0.0.1", help="Server host (default: 127.0.0.1).")
|
||||
parser.add_argument(
|
||||
"--no-open",
|
||||
action="store_true",
|
||||
help="Do not auto-open the browser when the server starts.",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
try:
|
||||
from agent_framework.devui import serve
|
||||
except ImportError:
|
||||
try:
|
||||
from agent_framework_devui import serve # type: ignore[no-redef]
|
||||
except ImportError as exc:
|
||||
raise SystemExit(
|
||||
"agent-framework-devui is not installed.\n"
|
||||
"Install it with: pip install agent-framework-devui --pre"
|
||||
) from exc
|
||||
|
||||
files = args.file or DEFAULT_SAMPLES
|
||||
workflows = []
|
||||
for file_name in files:
|
||||
print(f"Loading {file_name} ...")
|
||||
workflow = _load_one(file_name)
|
||||
workflows.append(workflow)
|
||||
name = getattr(workflow, "name", workflow.__class__.__name__)
|
||||
print(f" -> {name}")
|
||||
|
||||
print(f"\nLaunching DevUI on http://{args.host}:{args.port} with {len(workflows)} workflow(s)")
|
||||
print("Pick a workflow from the entity dropdown to inspect its graph and run it.\n")
|
||||
|
||||
serve(
|
||||
entities=workflows,
|
||||
port=args.port,
|
||||
host=args.host,
|
||||
auto_open=not args.no_open,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,29 @@
|
||||
# Phase 3 — Validate Output Parity
|
||||
|
||||
Run your captured PF outputs and the new MAF workflow against the same test inputs, then score semantic similarity using the Azure AI Evaluation SDK.
|
||||
Similarity scores are 1–5 (5 = most similar).
|
||||
|
||||
## Setup
|
||||
|
||||
1. Capture 20–30 real queries from your PF app and save them as a CSV with columns `question` and `pf_output`. See [test_inputs.csv.example](test_inputs.csv.example).
|
||||
|
||||
2. If you are validating a workflow other than the default sample, set `MAF_WORKFLOW_FILE`
|
||||
to the Python file that defines your module-level `workflow` object
|
||||
(example: `phase-2-rebuild/01_linear_flow.py`).
|
||||
|
||||
## Run
|
||||
|
||||
cd phase-3-validate
|
||||
python parity_check.py
|
||||
|
||||
Outputs `parity_results.csv`. Rows below the threshold are printed to stdout.
|
||||
|
||||
## Interpreting scores
|
||||
|
||||
| Score | Meaning |
|
||||
|---|---|
|
||||
| < 3.5 | Outputs diverge --> check for missing prompt context or unmigrated nodes |
|
||||
| 3.5 – 4.5 | Minor phrasing differences --> generally acceptable |
|
||||
| > 4.5 | Strong semantic match --> safe to proceed to Phase 4 |
|
||||
|
||||
Do not proceed to Phase 4 until mean similarity is consistently ≥ 3.5.
|
||||
@@ -0,0 +1,104 @@
|
||||
"""
|
||||
Compares captured Prompt Flow outputs against the new MAF workflow using
|
||||
SimilarityEvaluator from the Azure AI Evaluation SDK.
|
||||
|
||||
Scores are 1–5 (5 = most similar). Rows below SIMILARITY_THRESHOLD are
|
||||
flagged for manual review and the full results are saved to parity_results.csv.
|
||||
|
||||
Usage:
|
||||
python parity_check.py
|
||||
|
||||
Prerequisites:
|
||||
pip install azure-ai-evaluation pandas
|
||||
CSV format: columns 'question' and 'pf_output' (see test_inputs.csv.example)
|
||||
Optional: set MAF_WORKFLOW_FILE to your workflow file path
|
||||
(default: phase-2-rebuild/01_linear_flow.py).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
import pandas as pd
|
||||
from dotenv import load_dotenv
|
||||
from azure.ai.evaluation import SimilarityEvaluator
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
GUIDE_ROOT = SCRIPT_DIR.parent
|
||||
INPUT_CSV_PATH = SCRIPT_DIR / "test_inputs.csv"
|
||||
OUTPUT_CSV_PATH = SCRIPT_DIR / "parity_results.csv"
|
||||
ENV_PATH = GUIDE_ROOT / ".env"
|
||||
SIMILARITY_THRESHOLD = 3.5 # Scores below this are flagged for review (scale: 1–5)
|
||||
|
||||
if str(GUIDE_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(GUIDE_ROOT))
|
||||
|
||||
from workflow_loader import load_workflow # noqa: E402
|
||||
|
||||
|
||||
async def run_parity_check():
|
||||
load_dotenv(dotenv_path=ENV_PATH)
|
||||
|
||||
workflow = load_workflow()
|
||||
|
||||
# SimilarityEvaluator requires model_config in GA (1.16+).
|
||||
model_config = {
|
||||
"azure_endpoint": os.environ["AZURE_OPENAI_ENDPOINT"],
|
||||
"api_key": os.environ["AZURE_OPENAI_API_KEY"],
|
||||
"azure_deployment": os.environ["AZURE_OPENAI_CHAT_DEPLOYMENT_NAME"],
|
||||
}
|
||||
|
||||
evaluator = SimilarityEvaluator(model_config=model_config, threshold=3)
|
||||
|
||||
if not INPUT_CSV_PATH.exists():
|
||||
raise FileNotFoundError(
|
||||
f"Missing input file: {INPUT_CSV_PATH}\n"
|
||||
"Copy test_inputs.csv.example to test_inputs.csv and replace it with your "
|
||||
"captured Prompt Flow outputs before running parity_check.py."
|
||||
)
|
||||
|
||||
test_data = pd.read_csv(INPUT_CSV_PATH)
|
||||
results = []
|
||||
|
||||
for _, row in test_data.iterrows():
|
||||
question = row["question"]
|
||||
pf_answer = row["pf_output"]
|
||||
|
||||
maf_result = await workflow.run(question)
|
||||
maf_answer = maf_result.get_outputs()[0]
|
||||
|
||||
# evaluator() is a synchronous callable that makes network requests.
|
||||
# Wrap in asyncio.to_thread() to avoid blocking the event loop.
|
||||
# evaluator() returns {"similarity": float, "gpt_similarity": float}.
|
||||
# Use "similarity" — "gpt_similarity" is deprecated in GA.
|
||||
score_dict = await asyncio.to_thread(
|
||||
evaluator,
|
||||
query=question,
|
||||
response=maf_answer,
|
||||
ground_truth=pf_answer,
|
||||
)
|
||||
results.append({
|
||||
"question": question,
|
||||
"pf_output": pf_answer,
|
||||
"maf_output": maf_answer,
|
||||
"similarity": score_dict["similarity"],
|
||||
})
|
||||
|
||||
df = pd.DataFrame(results)
|
||||
mean_score = df["similarity"].mean()
|
||||
print(f"\nMean similarity: {mean_score:.2f} / 5.0")
|
||||
|
||||
regressions = df[df["similarity"] < SIMILARITY_THRESHOLD]
|
||||
if regressions.empty:
|
||||
print("All outputs meet the quality threshold. Ready for Phase 4.")
|
||||
else:
|
||||
print(f"\n{len(regressions)} answer(s) to review:")
|
||||
print(regressions[["question", "similarity"]].to_string(index=False))
|
||||
|
||||
df.to_csv(OUTPUT_CSV_PATH, index=False)
|
||||
print(f"\nFull results saved to {OUTPUT_CSV_PATH}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(run_parity_check())
|
||||
@@ -0,0 +1,133 @@
|
||||
"""
|
||||
Batched version of parity_check.py.
|
||||
|
||||
Runs all evaluations concurrently using asyncio.gather() instead of
|
||||
sequentially. Significantly faster for test suites with 20+ rows.
|
||||
|
||||
Prerequisites:
|
||||
pip install azure-ai-evaluation pandas
|
||||
CSV format: columns 'question' and 'pf_output' (see test_inputs.csv.example)
|
||||
Optional: set MAF_WORKFLOW_FILE to your workflow file path
|
||||
(default: phase-2-rebuild/01_linear_flow.py).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
import pandas as pd
|
||||
from dotenv import load_dotenv
|
||||
from azure.ai.evaluation import SimilarityEvaluator
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
GUIDE_ROOT = SCRIPT_DIR.parent
|
||||
INPUT_CSV_PATH = SCRIPT_DIR / "test_inputs.csv"
|
||||
OUTPUT_CSV_PATH = SCRIPT_DIR / "parity_results.csv"
|
||||
ENV_PATH = GUIDE_ROOT / ".env"
|
||||
SIMILARITY_THRESHOLD = 3.5 # Scale: 1–5. Rows below this are flagged for review.
|
||||
# Max simultaneous Azure OpenAI calls; prevents 429 rate-limit errors.
|
||||
# Adjust based on your Azure OpenAI quota (tokens-per-minute limit).
|
||||
CONCURRENCY_LIMIT = 5
|
||||
|
||||
if str(GUIDE_ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(GUIDE_ROOT))
|
||||
|
||||
from workflow_loader import load_workflow # noqa: E402
|
||||
|
||||
|
||||
async def evaluate_row(
|
||||
semaphore: asyncio.Semaphore,
|
||||
workflow,
|
||||
evaluator,
|
||||
question: str,
|
||||
pf_answer: str,
|
||||
) -> dict:
|
||||
"""Runs one MAF workflow call and scores it against the PF baseline."""
|
||||
async with semaphore:
|
||||
maf_result = await workflow.run(question)
|
||||
maf_answer = maf_result.get_outputs()[0]
|
||||
|
||||
# Keep evaluator calls inside the same concurrency bound because they also
|
||||
# make model-backed requests and can trigger the same rate limits.
|
||||
# evaluator() returns {"similarity": float, "gpt_similarity": float}.
|
||||
# Use "similarity" — "gpt_similarity" is deprecated in GA.
|
||||
score_dict = await asyncio.to_thread(
|
||||
evaluator,
|
||||
query=question,
|
||||
response=maf_answer,
|
||||
ground_truth=pf_answer,
|
||||
)
|
||||
return {
|
||||
"question": question,
|
||||
"pf_output": pf_answer,
|
||||
"maf_output": maf_answer,
|
||||
"similarity": score_dict["similarity"],
|
||||
}
|
||||
|
||||
|
||||
async def run_parity_check():
|
||||
load_dotenv(dotenv_path=ENV_PATH)
|
||||
|
||||
workflow = load_workflow()
|
||||
|
||||
model_config = {
|
||||
"azure_endpoint": os.environ["AZURE_OPENAI_ENDPOINT"],
|
||||
"api_key": os.environ["AZURE_OPENAI_API_KEY"],
|
||||
"azure_deployment": os.environ["AZURE_OPENAI_CHAT_DEPLOYMENT_NAME"],
|
||||
}
|
||||
|
||||
evaluator = SimilarityEvaluator(model_config=model_config, threshold=3)
|
||||
|
||||
if not INPUT_CSV_PATH.exists():
|
||||
raise FileNotFoundError(
|
||||
f"Missing input file: {INPUT_CSV_PATH}\n"
|
||||
"Copy test_inputs.csv.example to test_inputs.csv and replace it with your "
|
||||
"captured Prompt Flow outputs before running parity_check_batch.py."
|
||||
)
|
||||
|
||||
test_data = pd.read_csv(INPUT_CSV_PATH)
|
||||
|
||||
semaphore = asyncio.Semaphore(CONCURRENCY_LIMIT)
|
||||
tasks = [
|
||||
evaluate_row(semaphore, workflow, evaluator, row["question"], row["pf_output"])
|
||||
for _, row in test_data.iterrows()
|
||||
]
|
||||
|
||||
# Run rows concurrently, capped at CONCURRENCY_LIMIT to avoid rate-limit errors.
|
||||
# return_exceptions=True prevents one transient failure from losing all results.
|
||||
raw_results = await asyncio.gather(*tasks, return_exceptions=True)
|
||||
|
||||
results = []
|
||||
errors = []
|
||||
for i, r in enumerate(raw_results):
|
||||
if isinstance(r, Exception):
|
||||
errors.append((i, r))
|
||||
else:
|
||||
results.append(r)
|
||||
|
||||
if errors:
|
||||
print(f"\nWARNING: {len(errors)} row(s) failed during evaluation:")
|
||||
for idx, err in errors:
|
||||
print(f" Row {idx}: {err}")
|
||||
|
||||
if not results:
|
||||
raise RuntimeError("All evaluation rows failed. Check credentials and network connectivity.")
|
||||
|
||||
df = pd.DataFrame(results)
|
||||
mean_score = df["similarity"].mean()
|
||||
print(f"\nMean similarity: {mean_score:.2f} / 5.0")
|
||||
|
||||
regressions = df[df["similarity"] < SIMILARITY_THRESHOLD]
|
||||
if regressions.empty:
|
||||
print("All outputs meet the quality threshold. Ready for Phase 4.")
|
||||
else:
|
||||
print(f"\n{len(regressions)} answer(s) to review:")
|
||||
print(regressions[["question", "similarity"]].to_string(index=False))
|
||||
|
||||
df.to_csv(OUTPUT_CSV_PATH, index=False)
|
||||
print(f"\nFull results saved to {OUTPUT_CSV_PATH}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(run_parity_check())
|
||||
@@ -0,0 +1,4 @@
|
||||
question,pf_output
|
||||
"What is the refund policy?","You can request a refund within 30 days of purchase."
|
||||
"How do I reset my password?","Go to the login page and click 'Forgot password'."
|
||||
"What are your support hours?","Our support team is available Monday to Friday, 9am–6pm."
|
||||
@@ -0,0 +1,35 @@
|
||||
"""
|
||||
Wires MAF's built-in OpenTelemetry instrumentation to Azure Application Insights.
|
||||
|
||||
Replaces: Prompt Flow's built-in run viewer (step-by-step trace per node).
|
||||
|
||||
MAF automatically emits OTel spans for every executor invocation, agent call,
|
||||
and LLM request. Call configure_otel_providers() once at application startup,
|
||||
before any workflow.run() calls. No instrumentation changes needed in Executor classes.
|
||||
|
||||
View traces: Azure Portal → Application Insights → Transaction Search
|
||||
|
||||
Prerequisites:
|
||||
pip install azure-monitor-opentelemetry
|
||||
Set in .env: APPLICATIONINSIGHTS_CONNECTION_STRING
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
from agent_framework.observability import configure_otel_providers
|
||||
|
||||
load_dotenv()
|
||||
|
||||
# Step 1: Configure Azure Monitor as the export destination for OTel spans.
|
||||
# This sets up the Application Insights exporter for traces, metrics, and logs.
|
||||
configure_azure_monitor(
|
||||
connection_string=os.environ["APPLICATIONINSIGHTS_CONNECTION_STRING"]
|
||||
)
|
||||
|
||||
# Step 2: Enable MAF's built-in instrumentation so that executor transitions,
|
||||
# agent calls, and LLM requests produce workflow-level spans.
|
||||
# This must be called AFTER the Azure Monitor exporter is configured and
|
||||
# BEFORE any workflow.run() calls.
|
||||
configure_otel_providers()
|
||||
@@ -0,0 +1,188 @@
|
||||
# Deploying a MAF Workflow as an Azure ML Managed Online Endpoint
|
||||
|
||||
This guide walks through deploying a Microsoft Agent Framework (MAF) workflow
|
||||
to an **Azure Machine Learning Managed Online Endpoint**, replacing the
|
||||
Prompt Flow Managed Online Endpoint pattern.
|
||||
|
||||
> **Tip for AI agents / Copilot users:** Use the [`maf-online-endpoint`](../../../.github/skills/maf-online-endpoint/SKILL.md)
|
||||
> skill to scaffold this deployment automatically. The skill wraps any MAF
|
||||
> workflow into an `init()` / `run()` scoring script and generates the
|
||||
> `endpoint.yml`, `deployment.yml`, `conda.yml`, `score.py`, and `deploy.sh`
|
||||
> files described below, plus the required RBAC assignments. Trigger it with
|
||||
> prompts like *"deploy this MAF workflow as a managed online endpoint"* or
|
||||
> *"create an online endpoint for my agent-framework workflow"*. The manual
|
||||
> steps below remain useful for understanding or customizing the output.
|
||||
|
||||
## Files Overview
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `endpoint.yml` | Endpoint definition (name, auth mode) |
|
||||
| `deployment.yml` | Deployment template (environment, code, instance config) — uses `${VAR}` placeholders |
|
||||
| `score.py` | Scoring script with `init()` / `run()` entry points |
|
||||
| `conda.yml` | Conda environment with pip dependencies |
|
||||
| `deploy.sh` | End-to-end deployment script |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Azure CLI** with the `ml` extension installed:
|
||||
```bash
|
||||
az extension add --name ml --yes
|
||||
```
|
||||
- An existing **Azure ML workspace**
|
||||
- A **Foundry project** with a deployed model
|
||||
- `envsubst` available (part of `gettext`; used by `deploy.sh`)
|
||||
- Logged in: `az login`
|
||||
|
||||
## Step 1 — Set Environment Variables
|
||||
|
||||
Required:
|
||||
|
||||
```bash
|
||||
export SUBSCRIPTION_ID="<your-subscription-id>"
|
||||
export RESOURCE_GROUP="<your-resource-group>"
|
||||
export WORKSPACE_NAME="<your-workspace>"
|
||||
export FOUNDRY_PROJECT_ENDPOINT="https://<account>.services.ai.azure.com/api/projects/<project>"
|
||||
export FOUNDRY_MODEL="gpt-4o"
|
||||
```
|
||||
|
||||
Optional:
|
||||
|
||||
```bash
|
||||
export MAF_WORKFLOW_FILE="phase-2-rebuild/01_linear_flow.py" # default
|
||||
export INSTANCE_TYPE="Standard_DS3_v2" # default
|
||||
export INSTANCE_COUNT="1" # default
|
||||
export AZURE_AI_SEARCH_ENDPOINT="https://..." # for RAG workflows
|
||||
export AZURE_AI_SEARCH_INDEX_NAME="my-index"
|
||||
export AZURE_AI_SEARCH_API_KEY="..."
|
||||
export APPLICATIONINSIGHTS_CONNECTION_STRING="InstrumentationKey=..." # for tracing
|
||||
```
|
||||
|
||||
## Step 2 — Create the Online Endpoint
|
||||
|
||||
```bash
|
||||
az ml online-endpoint create \
|
||||
--subscription "$SUBSCRIPTION_ID" \
|
||||
--resource-group "$RESOURCE_GROUP" \
|
||||
--workspace-name "$WORKSPACE_NAME" \
|
||||
--file phase-4-migrate-ops/4b-deployment/endpoint.yml
|
||||
```
|
||||
|
||||
This creates an endpoint named `maf-endpoint` with key-based auth. The
|
||||
endpoint has a system-assigned managed identity.
|
||||
|
||||
## Step 3 — Assign RBAC to the Endpoint Identity
|
||||
|
||||
The endpoint's managed identity needs permission to call the Foundry model.
|
||||
Get the identity's principal ID:
|
||||
|
||||
```bash
|
||||
az ml online-endpoint show \
|
||||
--subscription "$SUBSCRIPTION_ID" \
|
||||
--name maf-endpoint \
|
||||
--resource-group "$RESOURCE_GROUP" \
|
||||
--workspace-name "$WORKSPACE_NAME" \
|
||||
--query identity.principal_id -o tsv
|
||||
```
|
||||
|
||||
Assign the **Cognitive Services User** role on the Foundry resource:
|
||||
|
||||
```bash
|
||||
az role assignment create \
|
||||
--assignee-object-id <principal-id> \
|
||||
--assignee-principal-type ServicePrincipal \
|
||||
--role "Cognitive Services User" \
|
||||
--scope "/subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.CognitiveServices/accounts/<account>"
|
||||
```
|
||||
|
||||
> **Note:** The `Azure AI Developer` role does _not_ include the
|
||||
> `Microsoft.CognitiveServices/accounts/AIServices/agents/write` data action
|
||||
> required by Foundry. Use `Cognitive Services User` (which has the wildcard
|
||||
> `Microsoft.CognitiveServices/*`) or `Azure AI Owner`.
|
||||
|
||||
Allow **5–10 minutes** for RBAC data plane propagation before testing.
|
||||
|
||||
## Step 4 — Render and Create the Deployment
|
||||
|
||||
`deployment.yml` is a template with `${VAR}` placeholders. The deploy script
|
||||
renders it with `envsubst` and then creates the deployment:
|
||||
|
||||
```bash
|
||||
# Render
|
||||
envsubst '$FOUNDRY_PROJECT_ENDPOINT $FOUNDRY_MODEL ...' \
|
||||
< deployment.yml > deployment-rendered.yml
|
||||
|
||||
# Create
|
||||
az ml online-deployment create \
|
||||
--subscription "$SUBSCRIPTION_ID" \
|
||||
--resource-group "$RESOURCE_GROUP" \
|
||||
--workspace-name "$WORKSPACE_NAME" \
|
||||
--file deployment-rendered.yml \
|
||||
--all-traffic
|
||||
```
|
||||
|
||||
Key deployment settings:
|
||||
- **Base image:** `mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu22.04:latest`
|
||||
- **Conda environment:** installs `agent-framework`, `azureml-inference-server-http`, etc.
|
||||
- **Request timeout:** 60 000 ms (LLM calls need more than the 5 s default)
|
||||
- **Code root:** the `PromptFlow-to-MAF` directory (so the scoring script can
|
||||
import `workflow_loader` and the workflow files)
|
||||
|
||||
## Step 5 — Smoke Test
|
||||
|
||||
```bash
|
||||
az ml online-endpoint invoke \
|
||||
--subscription "$SUBSCRIPTION_ID" \
|
||||
--name maf-endpoint \
|
||||
--resource-group "$RESOURCE_GROUP" \
|
||||
--workspace-name "$WORKSPACE_NAME" \
|
||||
--request-file <(echo '{"question": "What is the refund policy?"}')
|
||||
```
|
||||
|
||||
Expected response:
|
||||
|
||||
```json
|
||||
{"answer": "..."}
|
||||
```
|
||||
|
||||
## One-Command Deploy
|
||||
|
||||
`deploy.sh` automates all of the above (Steps 2–5):
|
||||
|
||||
```bash
|
||||
export SUBSCRIPTION_ID=... RESOURCE_GROUP=... WORKSPACE_NAME=... FOUNDRY_PROJECT_ENDPOINT=... FOUNDRY_MODEL=...
|
||||
bash phase-4-migrate-ops/4b-deployment/deploy.sh
|
||||
```
|
||||
|
||||
## Scoring Script Pattern
|
||||
|
||||
The scoring script (`score.py`) follows the AML `init()` / `run()` convention:
|
||||
|
||||
- **`init()`** — called once at container startup. Loads the MAF workflow via
|
||||
`workflow_loader` and optionally configures Application Insights tracing.
|
||||
- **`run(raw_data)`** — called per request. Parses JSON `{"question": "..."}`,
|
||||
runs the workflow, and returns `{"answer": "..."}`.
|
||||
|
||||
`AgentResponse` objects returned by the workflow are not JSON-serializable, so
|
||||
the script extracts `.text` before returning.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Fix |
|
||||
|---------|-------|-----|
|
||||
| `401 PermissionDenied` / `AIServices/agents/write` | Endpoint identity missing RBAC | Assign `Cognitive Services User` on the Foundry resource |
|
||||
| `upstream request timeout` | Default 5 s timeout too short for LLM | Set `request_timeout_ms: 60000` in deployment YAML |
|
||||
| `AgentResponse is not JSON serializable` | `run()` returns non-serializable object | Extract `.text` from the response |
|
||||
| `pip_requirements` validation error | Not a valid field for inline environment | Use `conda_file` with `name` + `version` instead |
|
||||
| Image build fails on package version | Package version not on PyPI | Remove version constraints from `conda.yml` |
|
||||
|
||||
## Cleanup
|
||||
|
||||
```bash
|
||||
az ml online-endpoint delete \
|
||||
--subscription "$SUBSCRIPTION_ID" \
|
||||
--name maf-endpoint \
|
||||
--resource-group "$RESOURCE_GROUP" \
|
||||
--workspace-name "$WORKSPACE_NAME" \
|
||||
--yes
|
||||
```
|
||||
@@ -0,0 +1,13 @@
|
||||
name: maf-env
|
||||
channels:
|
||||
- defaults
|
||||
dependencies:
|
||||
- python=3.11
|
||||
- pip
|
||||
- pip:
|
||||
- agent-framework
|
||||
- agent-framework-azure-ai-search
|
||||
- python-dotenv
|
||||
- azure-monitor-opentelemetry
|
||||
- azure-identity
|
||||
- azureml-inference-server-http
|
||||
@@ -0,0 +1,98 @@
|
||||
#!/bin/bash
|
||||
# Deploys the MAF workflow as an Azure Machine Learning Managed Online Endpoint
|
||||
# using the standard scoring-script pattern (init/run).
|
||||
#
|
||||
# Replaces: Prompt Flow Managed Online Endpoint.
|
||||
#
|
||||
# Prerequisites:
|
||||
# - az login completed
|
||||
# - Azure ML workspace already exists
|
||||
# - The ml CLI extension is installed: az extension add -n ml
|
||||
# - envsubst is available (part of gettext)
|
||||
# - For keyless auth to Foundry, assign a managed identity with the
|
||||
# appropriate role — see managed_identity.md for details.
|
||||
#
|
||||
# Required environment variables:
|
||||
# SUBSCRIPTION_ID - Azure subscription ID
|
||||
# RESOURCE_GROUP - Resource group containing the AML workspace
|
||||
# WORKSPACE_NAME - Azure ML workspace name
|
||||
# FOUNDRY_PROJECT_ENDPOINT - Foundry project endpoint URL
|
||||
# FOUNDRY_MODEL - Model name (e.g. gpt-4o)
|
||||
#
|
||||
# Optional environment variables:
|
||||
# MAF_WORKFLOW_FILE - Workflow file path (default: phase-2-rebuild/01_linear_flow.py)
|
||||
# INSTANCE_TYPE - VM SKU (default: Standard_DS3_v2)
|
||||
# INSTANCE_COUNT - Number of instances (default: 1)
|
||||
# AZURE_AI_SEARCH_ENDPOINT - AI Search endpoint (for RAG workflows)
|
||||
# AZURE_AI_SEARCH_INDEX_NAME - AI Search index name
|
||||
# AZURE_AI_SEARCH_API_KEY - AI Search API key
|
||||
# APPLICATIONINSIGHTS_CONNECTION_STRING - App Insights connection string (enables tracing)
|
||||
#
|
||||
# Usage: bash deploy.sh
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
|
||||
GUIDE_DIR=$(cd "${SCRIPT_DIR}/../.." && pwd)
|
||||
cd "$GUIDE_DIR"
|
||||
|
||||
SUBSCRIPTION_ID="${SUBSCRIPTION_ID:?Set SUBSCRIPTION_ID}"
|
||||
RESOURCE_GROUP="${RESOURCE_GROUP:?Set RESOURCE_GROUP}"
|
||||
WORKSPACE_NAME="${WORKSPACE_NAME:?Set WORKSPACE_NAME}"
|
||||
|
||||
# ── Export variables that deployment.yml references via envsubst ──────
|
||||
export FOUNDRY_PROJECT_ENDPOINT="${FOUNDRY_PROJECT_ENDPOINT:?Set FOUNDRY_PROJECT_ENDPOINT}"
|
||||
export FOUNDRY_MODEL="${FOUNDRY_MODEL:?Set FOUNDRY_MODEL}"
|
||||
export MAF_WORKFLOW_FILE="${MAF_WORKFLOW_FILE:-phase-2-rebuild/01_linear_flow.py}"
|
||||
export INSTANCE_TYPE="${INSTANCE_TYPE:-Standard_DS3_v2}"
|
||||
export INSTANCE_COUNT="${INSTANCE_COUNT:-1}"
|
||||
export AZURE_AI_SEARCH_ENDPOINT="${AZURE_AI_SEARCH_ENDPOINT:-}"
|
||||
export AZURE_AI_SEARCH_INDEX_NAME="${AZURE_AI_SEARCH_INDEX_NAME:-}"
|
||||
export AZURE_AI_SEARCH_API_KEY="${AZURE_AI_SEARCH_API_KEY:-}"
|
||||
export APPLICATIONINSIGHTS_CONNECTION_STRING="${APPLICATIONINSIGHTS_CONNECTION_STRING:-}"
|
||||
|
||||
# ── Render deployment YAML with current environment variables ────────
|
||||
RENDERED_DEPLOYMENT=$(mktemp --suffix=.yml)
|
||||
SUBST_VARS='${FOUNDRY_PROJECT_ENDPOINT} ${FOUNDRY_MODEL} ${MAF_WORKFLOW_FILE} ${INSTANCE_TYPE} ${INSTANCE_COUNT} ${AZURE_AI_SEARCH_ENDPOINT} ${AZURE_AI_SEARCH_INDEX_NAME} ${AZURE_AI_SEARCH_API_KEY} ${APPLICATIONINSIGHTS_CONNECTION_STRING}'
|
||||
envsubst "$SUBST_VARS" < "${SCRIPT_DIR}/deployment.yml" > "$RENDERED_DEPLOYMENT"
|
||||
|
||||
# ── Create managed online endpoint ──────────────────────────────────
|
||||
echo "Creating online endpoint..."
|
||||
az ml online-endpoint create \
|
||||
--subscription "$SUBSCRIPTION_ID" \
|
||||
--resource-group "$RESOURCE_GROUP" \
|
||||
--workspace-name "$WORKSPACE_NAME" \
|
||||
--file "${SCRIPT_DIR}/endpoint.yml" \
|
||||
2>/dev/null || echo "Endpoint already exists, continuing..."
|
||||
|
||||
# ── Create deployment under the endpoint ─────────────────────────────
|
||||
echo "Creating deployment..."
|
||||
az ml online-deployment create \
|
||||
--subscription "$SUBSCRIPTION_ID" \
|
||||
--resource-group "$RESOURCE_GROUP" \
|
||||
--workspace-name "$WORKSPACE_NAME" \
|
||||
--file "$RENDERED_DEPLOYMENT" \
|
||||
--all-traffic
|
||||
|
||||
rm -f "$RENDERED_DEPLOYMENT"
|
||||
|
||||
# ── Smoke test ───────────────────────────────────────────────────────
|
||||
ENDPOINT_NAME=$(grep '^name:' "${SCRIPT_DIR}/endpoint.yml" | awk '{print $2}')
|
||||
|
||||
echo "Running smoke test..."
|
||||
az ml online-endpoint invoke \
|
||||
--subscription "$SUBSCRIPTION_ID" \
|
||||
--resource-group "$RESOURCE_GROUP" \
|
||||
--workspace-name "$WORKSPACE_NAME" \
|
||||
--name "$ENDPOINT_NAME" \
|
||||
--request-file <(echo '{"question": "What is the refund policy?"}')
|
||||
|
||||
echo ""
|
||||
echo "Endpoint deployed successfully."
|
||||
SCORING_URI=$(az ml online-endpoint show \
|
||||
--subscription "$SUBSCRIPTION_ID" \
|
||||
--resource-group "$RESOURCE_GROUP" \
|
||||
--workspace-name "$WORKSPACE_NAME" \
|
||||
--name "$ENDPOINT_NAME" \
|
||||
--query "scoring_uri" -o tsv)
|
||||
echo "Scoring URI: ${SCORING_URI}"
|
||||
@@ -0,0 +1,24 @@
|
||||
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineDeployment.schema.json
|
||||
name: blue
|
||||
endpoint_name: maf-endpoint
|
||||
environment:
|
||||
name: maf-env
|
||||
version: "2"
|
||||
image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu22.04:latest
|
||||
conda_file: phase-4-migrate-ops/4b-deployment/conda.yml
|
||||
code_configuration:
|
||||
code: .
|
||||
scoring_script: phase-4-migrate-ops/4b-deployment/score.py
|
||||
environment_variables:
|
||||
FOUNDRY_PROJECT_ENDPOINT: ${FOUNDRY_PROJECT_ENDPOINT}
|
||||
FOUNDRY_MODEL: ${FOUNDRY_MODEL}
|
||||
MAF_WORKFLOW_FILE: ${MAF_WORKFLOW_FILE}
|
||||
AZURE_AI_SEARCH_ENDPOINT: ${AZURE_AI_SEARCH_ENDPOINT}
|
||||
AZURE_AI_SEARCH_INDEX_NAME: ${AZURE_AI_SEARCH_INDEX_NAME}
|
||||
AZURE_AI_SEARCH_API_KEY: ${AZURE_AI_SEARCH_API_KEY}
|
||||
APPLICATIONINSIGHTS_CONNECTION_STRING: ${APPLICATIONINSIGHTS_CONNECTION_STRING}
|
||||
instance_type: ${INSTANCE_TYPE}
|
||||
instance_count: ${INSTANCE_COUNT}
|
||||
request_settings:
|
||||
request_timeout_ms: 60000
|
||||
max_concurrent_requests_per_instance: 5
|
||||
@@ -0,0 +1,3 @@
|
||||
$schema: https://azuremlschemas.azureedge.net/latest/managedOnlineEndpoint.schema.json
|
||||
name: maf-endpoint
|
||||
auth_mode: key
|
||||
@@ -0,0 +1,48 @@
|
||||
"""
|
||||
Alternative client for teams using Microsoft Foundry as their model hub.
|
||||
|
||||
NOTE: This file demonstrates a **standalone agent test** — it uses Agent
|
||||
directly without the Executor + WorkflowBuilder pattern. This is intentional
|
||||
for quickly verifying Foundry connectivity. For production workflows, see the
|
||||
samples in phase-2-rebuild/ which show the full Executor/WorkflowBuilder
|
||||
pattern.
|
||||
|
||||
Use FoundryChatClient (from agent_framework.foundry) when connecting to a
|
||||
Foundry project endpoint.
|
||||
|
||||
Prerequisites:
|
||||
pip install agent-framework-foundry azure-identity
|
||||
Set in .env: FOUNDRY_PROJECT_ENDPOINT, FOUNDRY_MODEL
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
|
||||
from dotenv import load_dotenv
|
||||
from agent_framework import Agent
|
||||
from agent_framework.foundry import FoundryChatClient
|
||||
from azure.identity import DefaultAzureCredential
|
||||
|
||||
load_dotenv()
|
||||
|
||||
# Endpoint format: https://<resource>.services.ai.azure.com
|
||||
client = FoundryChatClient(
|
||||
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
|
||||
model=os.environ["FOUNDRY_MODEL"],
|
||||
credential=DefaultAzureCredential(),
|
||||
)
|
||||
|
||||
agent = Agent(
|
||||
client=client,
|
||||
name="DocQAAgent",
|
||||
instructions="You are a precise document Q&A assistant.",
|
||||
)
|
||||
|
||||
|
||||
async def main():
|
||||
result = await agent.run("What is the refund policy?")
|
||||
print(result)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
+50
@@ -0,0 +1,50 @@
|
||||
# Using Managed Identity Instead of API Keys
|
||||
|
||||
For production deployments, use managed identity with `FoundryChatClient`.
|
||||
This removes the need to store API keys in your online endpoint environment
|
||||
variables entirely.
|
||||
|
||||
Azure ML managed online endpoints automatically have a **system-assigned
|
||||
managed identity** — no extra step is needed to enable it.
|
||||
|
||||
## Step 1: Get the endpoint's managed identity principal ID
|
||||
|
||||
az ml online-endpoint show \
|
||||
--name maf-endpoint \
|
||||
--resource-group <your-rg> \
|
||||
--workspace-name <your-workspace> \
|
||||
--query identity.principal_id -o tsv
|
||||
|
||||
## Step 2: Grant the identity access to the Foundry project
|
||||
|
||||
az role assignment create \
|
||||
--role "Cognitive Services User" \
|
||||
--assignee-object-id <principal-id-from-step-1> \
|
||||
--assignee-principal-type ServicePrincipal \
|
||||
--scope /subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.CognitiveServices/accounts/<foundry-resource>
|
||||
|
||||
> **Note:** Use `Cognitive Services User` (not `Azure AI Developer`).
|
||||
> The `Azure AI Developer` role does not include the
|
||||
> `Microsoft.CognitiveServices/accounts/AIServices/agents/write` data action
|
||||
> required by Foundry. Allow 5–10 minutes for RBAC data plane propagation.
|
||||
|
||||
## Step 3: Update your client code
|
||||
|
||||
Use `ManagedIdentityCredential()` as the credential:
|
||||
|
||||
from agent_framework import Agent
|
||||
from agent_framework.foundry import FoundryChatClient
|
||||
from azure.identity import ManagedIdentityCredential
|
||||
|
||||
client = FoundryChatClient(
|
||||
project_endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
|
||||
model=os.environ["FOUNDRY_MODEL"],
|
||||
credential=ManagedIdentityCredential(),
|
||||
)
|
||||
agent = Agent(client=client, name="MyAgent", instructions="...")
|
||||
|
||||
## Step 4: Verify
|
||||
|
||||
`DefaultAzureCredential` also works and will automatically use managed
|
||||
identity when running in Azure, and Azure CLI credentials when running locally —
|
||||
making it a good choice for code that needs to work in both environments.
|
||||
@@ -0,0 +1,79 @@
|
||||
"""
|
||||
Azure ML managed online endpoint scoring script.
|
||||
|
||||
Replaces: Prompt Flow Managed Online Endpoint.
|
||||
|
||||
init() is called once when the container starts.
|
||||
run(raw_data) is called for each request; raw_data is the JSON request body.
|
||||
|
||||
Deploy:
|
||||
bash deploy.sh
|
||||
|
||||
Optional: set MAF_WORKFLOW_FILE to your workflow file path
|
||||
(default: phase-2-rebuild/01_linear_flow.py).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
workflow = None
|
||||
|
||||
|
||||
def init():
|
||||
"""Called once when the endpoint container starts."""
|
||||
global workflow
|
||||
|
||||
guide_root = Path(__file__).resolve().parents[2]
|
||||
if str(guide_root) not in sys.path:
|
||||
sys.path.insert(0, str(guide_root))
|
||||
|
||||
# Optional tracing — only when the connection string is present.
|
||||
appinsights_conn = os.getenv("APPLICATIONINSIGHTS_CONNECTION_STRING")
|
||||
if appinsights_conn:
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
from agent_framework.observability import configure_otel_providers
|
||||
|
||||
configure_azure_monitor(connection_string=appinsights_conn)
|
||||
configure_otel_providers()
|
||||
|
||||
from workflow_loader import load_workflow
|
||||
|
||||
workflow = load_workflow()
|
||||
logger.info("Workflow loaded successfully.")
|
||||
|
||||
|
||||
def run(raw_data):
|
||||
"""Called for each scoring request.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
raw_data : str
|
||||
JSON string with the request body, e.g. '{"question": "..."}'
|
||||
|
||||
Returns
|
||||
-------
|
||||
dict
|
||||
{"answer": str}
|
||||
"""
|
||||
data = json.loads(raw_data)
|
||||
question = data.get("question", "").strip()
|
||||
if not question:
|
||||
raise ValueError("Question must not be empty.")
|
||||
|
||||
result = asyncio.get_event_loop().run_until_complete(workflow.run(question))
|
||||
outputs = result.get_outputs()
|
||||
|
||||
if not outputs:
|
||||
raise RuntimeError("Workflow produced no output.")
|
||||
|
||||
output = outputs[0]
|
||||
# AgentResponse objects are not JSON-serializable; extract the text.
|
||||
if hasattr(output, "text"):
|
||||
return {"answer": output.text}
|
||||
return {"answer": str(output)}
|
||||
@@ -0,0 +1,62 @@
|
||||
# Runs parity_check.py on every push and fails the pipeline if mean
|
||||
# similarity drops below the quality threshold (3.5 / 5.0).
|
||||
#
|
||||
# Replaces: Manual evaluation runs in the Prompt Flow UI.
|
||||
#
|
||||
# Required repository secrets:
|
||||
# AZURE_OPENAI_API_KEY
|
||||
# AZURE_OPENAI_ENDPOINT
|
||||
# AZURE_OPENAI_CHAT_DEPLOYMENT_NAME
|
||||
# APPINSIGHTS_CONN (optional) — mapped to APPLICATIONINSIGHTS_CONNECTION_STRING in the workflow
|
||||
|
||||
name: Evaluate on Deploy
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
|
||||
defaults:
|
||||
run:
|
||||
working-directory: migration-guide/PromptFlow-to-MAF
|
||||
|
||||
jobs:
|
||||
evaluate:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.11"
|
||||
cache: pip
|
||||
|
||||
- name: Install dependencies
|
||||
run: pip install -r requirements.txt
|
||||
|
||||
- name: Check parity input file exists
|
||||
run: |
|
||||
if [ ! -f phase-3-validate/test_inputs.csv ]; then
|
||||
echo "ERROR: phase-3-validate/test_inputs.csv is missing." >&2
|
||||
echo "Copy test_inputs.csv.example to test_inputs.csv and populate it with real production queries before enabling the quality gate." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Run parity evaluation
|
||||
run: python phase-3-validate/parity_check.py
|
||||
env:
|
||||
AZURE_OPENAI_API_KEY: ${{ secrets.AZURE_OPENAI_API_KEY }}
|
||||
AZURE_OPENAI_ENDPOINT: ${{ secrets.AZURE_OPENAI_ENDPOINT }}
|
||||
AZURE_OPENAI_CHAT_DEPLOYMENT_NAME: ${{ secrets.AZURE_OPENAI_CHAT_DEPLOYMENT_NAME }}
|
||||
APPLICATIONINSIGHTS_CONNECTION_STRING: ${{ secrets.APPINSIGHTS_CONN }}
|
||||
MAF_WORKFLOW_FILE: phase-2-rebuild/01_linear_flow.py
|
||||
|
||||
- name: Enforce quality gate
|
||||
run: |
|
||||
python -c "
|
||||
import pandas as pd, sys
|
||||
df = pd.read_csv('phase-3-validate/parity_results.csv')
|
||||
score = df['similarity'].mean()
|
||||
print(f'Mean similarity: {score:.2f} / 5.0')
|
||||
sys.exit(0 if score >= 3.5 else 1)
|
||||
"
|
||||
@@ -0,0 +1,225 @@
|
||||
# Wrapping a MAF Workflow as an Azure ML Parallel Component
|
||||
|
||||
This sample wraps [`phase-2-rebuild/01_linear_flow.py`](../../phase-2-rebuild/01_linear_flow.py)
|
||||
as an **Azure ML Parallel Component** (PRS — Parallel Run Step), so you can
|
||||
batch-run the workflow over a JSONL input file and get one JSONL output row
|
||||
per input row.
|
||||
|
||||
This is the MAF equivalent of what `load_component(flow.dag.yaml)`
|
||||
auto-generated for Prompt Flow PRS jobs.
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| [`component.yaml`](component.yaml) | Parallel component spec (replaces `flow.dag.yaml`) |
|
||||
| [`env/conda.yml`](env/conda.yml) | Runtime conda env (MAF + AML PRS deps) |
|
||||
| [`src/entry.py`](src/entry.py) | PRS `init` / `run(mini_batch, context)` / `shutdown` wrapper |
|
||||
| [`src/hooks.py`](src/hooks.py) | Per-workflow customisation (row → input, output → JSON) |
|
||||
| [`src/maf_prs/`](src/maf_prs/) | Generic plumbing — config, processor, executor |
|
||||
| [`data/sample.jsonl`](data/sample.jsonl) | Sample input rows |
|
||||
| [`submit_pipeline.py`](submit_pipeline.py) | `MLClient` + `@pipeline` DSL submission driver |
|
||||
|
||||
## How it wraps `01_linear_flow.py`
|
||||
|
||||
`01_linear_flow.py` exports a module-level `workflow` object whose start
|
||||
executor's handler signature is `receive(self, question: str, ...)`.
|
||||
|
||||
The PRS wrapper plumbs this together as follows:
|
||||
|
||||
1. **Per-row factory.** `executor.py` calls
|
||||
[`workflow_loader.load_workflow()`](../../workflow_loader.py) for every
|
||||
row. `load_workflow()` re-imports the workflow file via `importlib.util`,
|
||||
producing a fresh `workflow` instance per row — required because MAF
|
||||
workflows do not support concurrent `run()` on the same instance.
|
||||
2. **Row → workflow input.** [`src/hooks.py::build_workflow_input`](src/hooks.py)
|
||||
maps each `{"question": "..."}` row to the string `question` the start
|
||||
executor expects.
|
||||
3. **Workflow output → JSON.** [`src/hooks.py::serialize_output`](src/hooks.py)
|
||||
unwraps the `AgentRunResponse` returned by the LLM executor to its
|
||||
`.text`, so PRS can append it as a JSONL line to `flow_outputs`.
|
||||
|
||||
To wrap a different workflow, change `--maf_workflow_file` in
|
||||
[`component.yaml`](component.yaml) and edit
|
||||
[`src/hooks.py`](src/hooks.py).
|
||||
|
||||
## Generating this scaffolding for your own flow (`maf-prs-job` skill)
|
||||
|
||||
This sample was hand-built for the linear flow. For a real Prompt Flow PRS
|
||||
job, the [`maf-prs-job`](../../../../.github/skills/maf-prs-job/SKILL.md)
|
||||
skill regenerates the same scaffolding (component spec, conda env, entry
|
||||
script, processor/executor, hooks, submission driver) tailored to your
|
||||
flow — you only edit `src/hooks.py` afterwards.
|
||||
|
||||
### When to use the skill
|
||||
|
||||
Use it when you have **either**:
|
||||
|
||||
- An existing PF PRS submission script (a `.py` or notebook cell that
|
||||
calls `load_component("flow.dag.yaml")` and wires the component into a
|
||||
pipeline), and you want to keep the pipeline plumbing but swap the
|
||||
per-row task to a MAF workflow.
|
||||
- An already-migrated MAF workflow folder that exports
|
||||
`create_workflow()` (or a single-file workflow like
|
||||
[`01_linear_flow.py`](../../phase-2-rebuild/01_linear_flow.py)) and you
|
||||
want it submittable as a parallel job on AML.
|
||||
|
||||
If your flow has not been migrated yet, run the
|
||||
[`promptflow-to-maf`](../../../../.github/skills/promptflow-to-maf/SKILL.md)
|
||||
skill **first** to produce the workflow file, then this one.
|
||||
|
||||
### How to invoke it
|
||||
|
||||
In Copilot Chat, point at both sides of the migration so the skill can
|
||||
auto-derive the row-mapping and pipeline settings:
|
||||
|
||||
```
|
||||
Use the maf-prs-job skill to wrap
|
||||
examples/flows/standard/web-classification-maf/workflow.py
|
||||
into a parallel pipeline. The original PF submission is at
|
||||
examples/tutorials/run-flow-with-pipeline/pipeline.py
|
||||
```
|
||||
|
||||
The skill will:
|
||||
|
||||
1. **Audit** — parse the PF submission and print the run-settings table
|
||||
it lifted (`compute`, `mini_batch_size`, `retry_settings`, `connections`,
|
||||
…) so you can confirm before any files are written.
|
||||
2. **Decide** — run its 10 auto-derive checks (A–J) and show a verdict
|
||||
table marking which fields it can fill from the source vs. which it
|
||||
leaves as `# TODO` stubs in `src/hooks.py`.
|
||||
3. **Generate** — drop the same files this sample contains
|
||||
(`component.yaml`, `env/conda.yml`, `src/entry.py`, `src/hooks.py`,
|
||||
`src/maf_prs/`, `submit_pipeline.py`) into your MAF workflow folder
|
||||
(consolidated layout, default) or into a sibling `<folder>-prs/`
|
||||
(opt-in). `workflow.py` and `requirements.txt` are not touched.
|
||||
4. **Validate** — if a local jsonl sample is available, run the same
|
||||
`dryrun.py`-style smoke test before handing back the
|
||||
`python submit_pipeline.py` command.
|
||||
|
||||
### What the skill produces vs. what's in this sample
|
||||
|
||||
| File | This sample (hand-built) | Skill output |
|
||||
|---|---|---|
|
||||
| `component.yaml` | Hard-codes `--maf_workflow_file phase-2-rebuild/01_linear_flow.py` | Surfaces real `connections=` from PF as typed component inputs (e.g. `model_endpoint`, `model_deployment`) |
|
||||
| `src/hooks.py` | Maps the single `question` field → `str` input | Auto-fills `build_workflow_input` from the PF input mapping + your start handler signature; leaves a `# TODO` stub when the source is ambiguous |
|
||||
| `submit_pipeline.py` | Loads `data/sample.jsonl` from this folder | Preserves the source `Input(path=..., type=..., mode=...)` verbatim — same data asset, no invented sample |
|
||||
| `src/maf_prs/`, `src/entry.py` | Identical | Identical (vendored verbatim) |
|
||||
|
||||
The generic plumbing (`src/maf_prs/{config,executor,processor}.py` and
|
||||
`src/entry.py`) is exactly the same in both — the skill never touches it
|
||||
unless your workflow needs an extra component input.
|
||||
|
||||
### Skill references worth reading
|
||||
|
||||
- [`SKILL.md`](../../../../.github/skills/maf-prs-job/SKILL.md) — the
|
||||
five-step generation loop and the file-action table.
|
||||
- [`references/pf-vs-maf-prs.md`](../../../../.github/skills/maf-prs-job/references/pf-vs-maf-prs.md) —
|
||||
side-by-side mapping of every PF PRS knob to its AML equivalent (the
|
||||
source for the run-settings table further down this README).
|
||||
- [`references/auto-derive-checks.md`](../../../../.github/skills/maf-prs-job/references/auto-derive-checks.md) —
|
||||
what the skill can and cannot infer automatically.
|
||||
- [`references/gotchas.md`](../../../../.github/skills/maf-prs-job/references/gotchas.md) —
|
||||
the 16 issues you're likely to hit (most relevant: #12 `uri_file`
|
||||
compat flags, #14 `setuptools<80`, #15 `resolution-too-deep`,
|
||||
#16 `$[[]]` for optional inputs).
|
||||
|
||||
## Local dry-run (no AML compute)
|
||||
|
||||
Activate the workspace `.venv-af` and install the small extra packages PRS
|
||||
expects (only needed locally — the conda env on AML already has them):
|
||||
|
||||
```powershell
|
||||
# from the repo root
|
||||
.\.venv-af\Scripts\Activate.ps1
|
||||
pip install pandas
|
||||
```
|
||||
|
||||
Set Foundry credentials so the workflow can call the model:
|
||||
|
||||
```powershell
|
||||
$env:FOUNDRY_PROJECT_ENDPOINT = "https://<account>.services.ai.azure.com/api/projects/<project>"
|
||||
$env:FOUNDRY_MODEL = "gpt-4o"
|
||||
```
|
||||
|
||||
Run the dry-run from this folder:
|
||||
|
||||
```powershell
|
||||
cd migration-guide/PromptFlow-to-MAF/phase-4-migrate-ops/4d-pipelines
|
||||
python dryrun.py
|
||||
```
|
||||
|
||||
You should see one JSONL line per input row, each containing the
|
||||
`line_number`, the original `input`, and the model's `output`. The script
|
||||
uses [`dryrun.py`](dryrun.py), which exercises the same
|
||||
`init` / `run(mini_batch, context)` / `shutdown` contract that AML PRS
|
||||
calls on [`src/entry.py`](src/entry.py).
|
||||
|
||||
## Submit to Azure ML
|
||||
|
||||
Prerequisites:
|
||||
|
||||
- An Azure ML workspace and a CPU compute cluster (default name
|
||||
`cpu-cluster`; override with `$env:AML_COMPUTE`).
|
||||
- The compute identity has the `Cognitive Services User` role on the
|
||||
Foundry resource (see [`../4b-deployment/managed_identity.md`](../4b-deployment/managed_identity.md)).
|
||||
- `azure-ai-ml` installed locally (the parity-check venv has it; otherwise
|
||||
`pip install azure-ai-ml`).
|
||||
|
||||
Submit:
|
||||
|
||||
```powershell
|
||||
$env:SUBSCRIPTION_ID = "<sub>"
|
||||
$env:RESOURCE_GROUP = "<rg>"
|
||||
$env:WORKSPACE_NAME = "<ws>"
|
||||
$env:FOUNDRY_PROJECT_ENDPOINT = "https://<account>.services.ai.azure.com/api/projects/<project>"
|
||||
$env:FOUNDRY_MODEL = "gpt-4o"
|
||||
$env:AML_COMPUTE = "cpu-cluster"
|
||||
|
||||
python submit_pipeline.py
|
||||
```
|
||||
|
||||
The submission script streams the run log. When it finishes, the appended
|
||||
output file (`outputs.flow_outputs/parallel_run_step.jsonl`) has one JSON
|
||||
line per input row, in the same order as `data/sample.jsonl`.
|
||||
|
||||
## How the run-settings map to Prompt Flow PRS
|
||||
|
||||
| Prompt Flow PRS setting | Equivalent in this sample | Where |
|
||||
|---|---|---|
|
||||
| `flow_node.compute` | `workflow_node.compute` | [`submit_pipeline.py`](submit_pipeline.py) |
|
||||
| `flow_node.resources` | `workflow_node.resources` | [`submit_pipeline.py`](submit_pipeline.py) |
|
||||
| `flow_node.mini_batch_size` | `workflow_node.mini_batch_size` / `mini_batch_size` in YAML | [`component.yaml`](component.yaml) |
|
||||
| `flow_node.max_concurrency_per_instance` | same key, both files | [`component.yaml`](component.yaml) |
|
||||
| `flow_node.retry_settings` | `retry_settings` | [`component.yaml`](component.yaml) |
|
||||
| `flow_node.error_threshold` / `mini_batch_error_threshold` | same keys | [`component.yaml`](component.yaml) |
|
||||
| `flow_node.logging_level` | `logging_level` | [`component.yaml`](component.yaml) |
|
||||
| `flow_node(connections={...})` | env vars on the component node | [`submit_pipeline.py`](submit_pipeline.py) |
|
||||
| `flow.dag.yaml` `inputs:` | `inputs:` block + `hooks.build_workflow_input` | [`component.yaml`](component.yaml), [`src/hooks.py`](src/hooks.py) |
|
||||
| `flow.dag.yaml` `outputs:` | `flow_outputs` (jsonl) + `debug_info` (folder) | [`component.yaml`](component.yaml) |
|
||||
|
||||
## Why the four `--amlbi_pf_*` flags?
|
||||
|
||||
PRS's public schema only declares `mltable` and `uri_folder` as supported
|
||||
input types, but its runtime also accepts `uri_file` when the four flags
|
||||
|
||||
```
|
||||
--amlbi_pf_enabled True
|
||||
--amlbi_pf_run_mode component
|
||||
--amlbi_file_format jsonl
|
||||
--amlbi_mini_batch_rows 1
|
||||
```
|
||||
|
||||
are passed in `program_arguments`. Prompt Flow's runtime emits the same
|
||||
flag set when `load_component(flow.dag.yaml)` builds the parallel
|
||||
component; we replicate it so callers can pass a plain `.jsonl` file with
|
||||
no MLTable spec required.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Fix |
|
||||
|---|---|---|
|
||||
| `RequireMLTablePackageException` on every mini-batch | `pkg_resources` import fails because setuptools 80+ removed it | Keep `setuptools<80` in [`env/conda.yml`](env/conda.yml) |
|
||||
| `MAF workflow file not found` | `--maf_workflow_file` path is relative to the AML code snapshot root | Use a path relative to the migration-guide root, e.g. `phase-2-rebuild/01_linear_flow.py` |
|
||||
| `RuntimeError: This workflow is already running` | Cached workflow instance reused across rows | Don't cache — `executor.py` calls `load_workflow()` per row |
|
||||
| `401 PermissionDenied` from Foundry | Compute identity missing RBAC | Assign `Cognitive Services User` on the Foundry resource |
|
||||
@@ -0,0 +1,88 @@
|
||||
# Azure ML Parallel Component that wraps `phase-2-rebuild/01_linear_flow.py`
|
||||
# as the per-row task. This is the MAF equivalent of what
|
||||
# `load_component(flow.dag.yaml)` auto-generated for Prompt Flow PRS jobs.
|
||||
#
|
||||
# Settings here mirror Prompt Flow PRS settings 1:1 — see README.md.
|
||||
$schema: https://azuremlschemas.azureedge.net/latest/parallelComponent.schema.json
|
||||
type: parallel
|
||||
name: maf_linear_flow_prs
|
||||
display_name: MAF Linear Flow as PRS
|
||||
version: 1
|
||||
description: >-
|
||||
Run the phase-2-rebuild/01_linear_flow.py MAF workflow over a JSONL input
|
||||
as a parallel run step. Each row becomes one workflow invocation.
|
||||
|
||||
inputs:
|
||||
# PRS only declares `mltable` and `uri_folder` as supported types in its
|
||||
# public schema, but it ALSO accepts `uri_file` when the PF compatibility
|
||||
# flags below are present in `program_arguments`. PF's runtime emits the
|
||||
# same flag set when `load_component(flow.dag.yaml)` generates the parallel
|
||||
# component; we replicate that here so callers can pass a plain `.jsonl`
|
||||
# file with no MLTable spec required.
|
||||
data:
|
||||
type: uri_file
|
||||
description: 'Single jsonl file with one {"question": "..."} object per line.'
|
||||
# The workflow file (relative to the AML code snapshot root) wrapped by
|
||||
# this component. Defaults to the linear flow sample but you can swap in
|
||||
# any other phase-2-rebuild file.
|
||||
maf_workflow_file:
|
||||
type: string
|
||||
default: phase-2-rebuild/01_linear_flow.py
|
||||
|
||||
outputs:
|
||||
# Equivalent of PF's auto-generated `flow_outputs` port — PRS appends one
|
||||
# JSONL row per input row to this file.
|
||||
flow_outputs:
|
||||
type: uri_file
|
||||
# Equivalent of PF's auto-generated `debug_info` port — free-form folder
|
||||
# for the workflow to write intermediate artefacts to.
|
||||
debug_info:
|
||||
type: uri_folder
|
||||
|
||||
# Tell PRS which input is the dataset to be split into mini-batches.
|
||||
input_data: ${{inputs.data}}
|
||||
|
||||
# === PRS run settings (carry over from PF flow_node.* settings) =============
|
||||
mini_batch_size: "10"
|
||||
mini_batch_error_threshold: -1
|
||||
error_threshold: -1
|
||||
logging_level: DEBUG
|
||||
resources:
|
||||
instance_count: 1
|
||||
max_concurrency_per_instance: 2
|
||||
retry_settings:
|
||||
max_retries: 1
|
||||
timeout: 1200
|
||||
# ============================================================================
|
||||
|
||||
task:
|
||||
type: run_function
|
||||
# `code: ../..` ships the migration-guide root so workflow_loader.py and
|
||||
# phase-2-rebuild/01_linear_flow.py are uploaded together with src/entry.py.
|
||||
code: ../..
|
||||
entry_script: phase-4-migrate-ops/4d-pipelines/src/entry.py
|
||||
environment:
|
||||
name: maf-prs-env
|
||||
version: 1
|
||||
image: mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu22.04:latest
|
||||
conda_file: env/conda.yml
|
||||
program_arguments: >-
|
||||
--amlbi_pf_enabled True
|
||||
--amlbi_pf_run_mode component
|
||||
--amlbi_file_format jsonl
|
||||
--amlbi_mini_batch_rows 1
|
||||
--maf_workflow_file ${{inputs.maf_workflow_file}}
|
||||
--output_dir ${{outputs.debug_info}}
|
||||
# === PF compatibility flag set ============================================
|
||||
# `--amlbi_pf_enabled True` flips PRS's ArgValidator gate so that
|
||||
# `type: uri_file` is accepted (the public
|
||||
# schema only lists mltable/uri_folder).
|
||||
# `--amlbi_pf_run_mode component` PF "I am a flow component" signal.
|
||||
# `--amlbi_file_format jsonl` tells PRS to parse the input file as
|
||||
# jsonl and dispatch row dicts.
|
||||
# `--amlbi_mini_batch_rows 1` switch from file-count to row-count
|
||||
# batching (paired with --amlbi_file_format).
|
||||
# ==========================================================================
|
||||
# Equivalent of PF's `parallel_run_step.jsonl` — PRS appends each entry of
|
||||
# the list returned by run() as a JSONL line to this output.
|
||||
append_row_to: ${{outputs.flow_outputs}}
|
||||
@@ -0,0 +1,50 @@
|
||||
{"question": "What is retrieval-augmented generation?"}
|
||||
{"question": "Explain vector embeddings in one sentence."}
|
||||
{"question": "What is a transformer model?"}
|
||||
{"question": "Define Microsoft Agent Framework in one sentence."}
|
||||
{"question": "What is the difference between an agent and a workflow?"}
|
||||
{"question": "What is prompt engineering?"}
|
||||
{"question": "Explain the attention mechanism in plain English."}
|
||||
{"question": "What is fine-tuning of a language model?"}
|
||||
{"question": "Define context window for an LLM."}
|
||||
{"question": "What is a system prompt?"}
|
||||
{"question": "What is a chat completion?"}
|
||||
{"question": "What is a token in the context of LLMs?"}
|
||||
{"question": "Explain temperature and top-p sampling."}
|
||||
{"question": "What is hallucination in LLM outputs?"}
|
||||
{"question": "What is a vector database?"}
|
||||
{"question": "What is semantic search?"}
|
||||
{"question": "What is a chunking strategy in RAG?"}
|
||||
{"question": "Explain the difference between RAG and fine-tuning."}
|
||||
{"question": "What is an embedding model?"}
|
||||
{"question": "What is cosine similarity?"}
|
||||
{"question": "What is a function-calling LLM?"}
|
||||
{"question": "What is a tool in agent frameworks?"}
|
||||
{"question": "What is multi-agent orchestration?"}
|
||||
{"question": "What is a workflow executor in MAF?"}
|
||||
{"question": "What is a handler decorator in MAF?"}
|
||||
{"question": "What is the role of WorkflowContext in MAF?"}
|
||||
{"question": "What is yield_output used for?"}
|
||||
{"question": "What is send_message used for?"}
|
||||
{"question": "What is Azure AI Foundry?"}
|
||||
{"question": "What is Azure OpenAI Service?"}
|
||||
{"question": "What is the difference between Azure OpenAI and OpenAI?"}
|
||||
{"question": "What is a managed online endpoint?"}
|
||||
{"question": "What is Azure ML Parallel Run Step?"}
|
||||
{"question": "What is a parallel component in Azure ML?"}
|
||||
{"question": "What is mini_batch_size in PRS?"}
|
||||
{"question": "What is max_concurrency_per_instance?"}
|
||||
{"question": "What is OpenTelemetry?"}
|
||||
{"question": "What is Application Insights used for?"}
|
||||
{"question": "What is structured output in LLMs?"}
|
||||
{"question": "What is JSON mode in chat completions?"}
|
||||
{"question": "What is a guardrail for LLM applications?"}
|
||||
{"question": "What is content safety filtering?"}
|
||||
{"question": "What is a moderation model?"}
|
||||
{"question": "What is grounding in LLM responses?"}
|
||||
{"question": "Explain the difference between zero-shot and few-shot prompting."}
|
||||
{"question": "What is chain-of-thought prompting?"}
|
||||
{"question": "What is ReAct prompting?"}
|
||||
{"question": "What is an agentic loop?"}
|
||||
{"question": "What is Microsoft Foundry SDK?"}
|
||||
{"question": "What is Prompt Flow used for?"}
|
||||
@@ -0,0 +1,37 @@
|
||||
"""
|
||||
Local dry-run for the MAF Linear Flow PRS sample. Skips AML and runs the
|
||||
PRS init / run / shutdown contract in-process using the sample input file.
|
||||
|
||||
Run from this folder:
|
||||
|
||||
python dryrun.py
|
||||
|
||||
Requires FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_MODEL in the environment so
|
||||
that 01_linear_flow.py can construct its FoundryChatClient.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pandas as pd
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
sys.path.insert(0, str(HERE / "src"))
|
||||
sys.argv = [
|
||||
"entry",
|
||||
"--maf_workflow_file",
|
||||
"phase-2-rebuild/01_linear_flow.py",
|
||||
]
|
||||
|
||||
from entry import init, run, shutdown # noqa: E402
|
||||
|
||||
init()
|
||||
try:
|
||||
df = pd.read_json(HERE / "data" / "sample.jsonl", lines=True)
|
||||
ctx = SimpleNamespace(minibatch_index=0, global_row_index_lower_bound=0)
|
||||
for line in run(df, ctx):
|
||||
print(line)
|
||||
finally:
|
||||
shutdown()
|
||||
@@ -0,0 +1,48 @@
|
||||
"""
|
||||
Azure ML Parallel Run Step (PRS) entry script for the
|
||||
`phase-2-rebuild/01_linear_flow.py` MAF workflow.
|
||||
|
||||
PRS calls three top-level functions on this module per worker process:
|
||||
|
||||
init() -- once at boot
|
||||
run(mini_batch, context) -- once per mini-batch; returns list[str]
|
||||
shutdown() -- once before the worker exits
|
||||
|
||||
Keep this file thin. All real logic lives in `maf_prs/`.
|
||||
"""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# `code: ../..` in component.yaml uploads the migration-guide root, so the
|
||||
# entry script ends up at <root>/phase-4-migrate-ops/4d-pipelines/src/entry.py.
|
||||
# Prepend this directory so `import maf_prs.*` resolves cleanly regardless of
|
||||
# how PRS happens to load the entry module.
|
||||
_HERE = Path(__file__).resolve().parent
|
||||
if str(_HERE) not in sys.path:
|
||||
sys.path.insert(0, str(_HERE))
|
||||
|
||||
from maf_prs.processor import create_processor # noqa: E402
|
||||
|
||||
# Migration-guide root: <root>/phase-4-migrate-ops/4d-pipelines/src/entry.py
|
||||
# ^^^^ parents[3]
|
||||
_GUIDE_ROOT = Path(__file__).resolve().parents[3]
|
||||
|
||||
_processor = None
|
||||
|
||||
|
||||
def init():
|
||||
"""Called once per worker process at boot."""
|
||||
global _processor
|
||||
_processor = create_processor(_GUIDE_ROOT)
|
||||
_processor.init()
|
||||
|
||||
|
||||
def run(mini_batch, context):
|
||||
"""Called once per mini-batch. Returns one JSON string per input row."""
|
||||
return _processor.process(mini_batch, context)
|
||||
|
||||
|
||||
def shutdown():
|
||||
"""Called once before the worker exits. Flushes tracing and closes the loop."""
|
||||
if _processor is not None:
|
||||
_processor.finalize()
|
||||
@@ -0,0 +1,50 @@
|
||||
"""
|
||||
Per-workflow customisation hooks for the MAF PRS sample wrapping
|
||||
`phase-2-rebuild/01_linear_flow.py`.
|
||||
|
||||
This is the only file you typically need to edit when adapting the sample to
|
||||
a different workflow. The other files in `maf_prs/` are generic plumbing.
|
||||
|
||||
Three hooks are exposed:
|
||||
|
||||
setup(config) -- one-time worker setup (env vars, etc.)
|
||||
build_workflow_input(row) -- map one input row (dict) to the input the
|
||||
workflow's start executor expects.
|
||||
serialize_output(output) -- convert the workflow's terminal output to
|
||||
a JSON-serialisable value.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
|
||||
def setup(config) -> None:
|
||||
"""One-time worker setup. No-op for the linear flow sample."""
|
||||
return None
|
||||
|
||||
|
||||
def build_workflow_input(row: dict) -> Any:
|
||||
"""Map an input row to the workflow's start executor input.
|
||||
|
||||
The linear flow's `InputExecutor.receive` handler signature is::
|
||||
|
||||
async def receive(self, question: str, ctx: WorkflowContext[str]) -> None
|
||||
|
||||
so we extract the `question` field from the row.
|
||||
"""
|
||||
return row["question"]
|
||||
|
||||
|
||||
def serialize_output(output: Any) -> Any:
|
||||
"""Convert the workflow's terminal output to a JSON-serialisable value.
|
||||
|
||||
The linear flow's `LLMExecutor` yields an `AgentRunResponse` (it has a
|
||||
`.text` attribute). We unwrap it to a plain string for JSONL output.
|
||||
"""
|
||||
if output is None:
|
||||
return None
|
||||
if isinstance(output, (dict, list, str, int, float, bool)):
|
||||
return output
|
||||
if hasattr(output, "text"):
|
||||
return output.text
|
||||
return str(output)
|
||||
+1
@@ -0,0 +1 @@
|
||||
"""MAF PRS plumbing — wraps a MAF workflow as an Azure ML parallel component."""
|
||||
+31
@@ -0,0 +1,31 @@
|
||||
"""
|
||||
PRS argument parsing — produces a typed config object for the worker.
|
||||
|
||||
`program_arguments` in component.yaml are forwarded here. We use
|
||||
`parse_known_args` so PRS-injected flags (logging, telemetry, PF compat
|
||||
flags) don't crash us.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from argparse import ArgumentParser
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import List, Optional
|
||||
|
||||
|
||||
@dataclass
|
||||
class MafPrsConfig:
|
||||
"""Resolved PRS config for one worker."""
|
||||
maf_workflow_file: Optional[str] = None
|
||||
debug_output_dir: Optional[Path] = None
|
||||
|
||||
|
||||
def parse_args(argv: Optional[List[str]] = None) -> MafPrsConfig:
|
||||
parser = ArgumentParser()
|
||||
# Path (relative to the AML code snapshot root) to the workflow file.
|
||||
# For this sample: phase-2-rebuild/01_linear_flow.py
|
||||
parser.add_argument("--maf_workflow_file")
|
||||
# Bound to outputs.debug_info — workflow may write intermediate artefacts here.
|
||||
parser.add_argument("--output_dir", type=Path, dest="debug_output_dir")
|
||||
parsed, _unknown = parser.parse_known_args(argv)
|
||||
return MafPrsConfig(**vars(parsed))
|
||||
+111
@@ -0,0 +1,111 @@
|
||||
"""
|
||||
Per-row execution of a MAF workflow.
|
||||
|
||||
`workflow_loader.load_workflow()` (from the migration guide root) is used as
|
||||
the per-row factory. Each call re-imports the workflow file via
|
||||
`importlib.util`, producing a fresh module-level `workflow` object — required
|
||||
because MAF workflows do not support concurrent `run()` on the same instance.
|
||||
|
||||
All per-workflow customisation lives in `src/hooks.py`; this driver is generic.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable, Optional
|
||||
|
||||
from .config import MafPrsConfig
|
||||
|
||||
logger = logging.getLogger("maf-prs.executor")
|
||||
|
||||
|
||||
class MafWorkflowExecutor:
|
||||
"""Drives a MAF workflow once per input row."""
|
||||
|
||||
def __init__(self, working_dir: Path, config: MafPrsConfig):
|
||||
self._working_dir = working_dir
|
||||
self._config = config
|
||||
# Factory only — never cache a workflow instance across rows.
|
||||
self._create_workflow: Optional[Callable[[], Any]] = None
|
||||
self._hooks: Any = None
|
||||
|
||||
# ---- init ---------------------------------------------------------
|
||||
def init(self) -> None:
|
||||
# Make the migration guide root (which contains workflow_loader.py and
|
||||
# the wrapped workflow file) importable.
|
||||
if str(self._working_dir) not in sys.path:
|
||||
sys.path.insert(0, str(self._working_dir))
|
||||
|
||||
# `hooks` is imported *after* sys.path is wired so it can perform
|
||||
# workflow-relative imports at module load.
|
||||
import hooks # noqa: E402
|
||||
|
||||
hooks.setup(self._config)
|
||||
self._hooks = hooks
|
||||
|
||||
# Tell workflow_loader which file to import. Each `load_workflow()`
|
||||
# call re-execs the module, giving a fresh `workflow` per row.
|
||||
if self._config.maf_workflow_file:
|
||||
os.environ["MAF_WORKFLOW_FILE"] = self._config.maf_workflow_file
|
||||
|
||||
from workflow_loader import load_workflow # noqa: E402
|
||||
|
||||
def _factory() -> Any:
|
||||
return load_workflow()
|
||||
|
||||
self._create_workflow = _factory
|
||||
self._setup_tracing()
|
||||
|
||||
# ---- per-row ------------------------------------------------------
|
||||
async def execute(self, row: dict, row_number: int) -> dict:
|
||||
"""Run the workflow for one row. Per-row error isolation so a single
|
||||
bad row does not poison the mini-batch."""
|
||||
assert self._create_workflow is not None, "init() was not called"
|
||||
try:
|
||||
# Build a fresh workflow per row (re-imports the module so the
|
||||
# start executor is re-instantiated). Inside the try/except so
|
||||
# construction errors (e.g. missing env vars) are captured as
|
||||
# per-row errors rather than failing the whole mini-batch.
|
||||
workflow = self._create_workflow()
|
||||
wf_input = self._hooks.build_workflow_input(row)
|
||||
result = await workflow.run(wf_input)
|
||||
outputs = result.get_outputs() or [None]
|
||||
return {
|
||||
"line_number": row_number,
|
||||
"input": row,
|
||||
"output": self._hooks.serialize_output(outputs[0]),
|
||||
"error": None,
|
||||
}
|
||||
except Exception as exc: # noqa: BLE001 — per-row isolation
|
||||
logger.exception("Row %d failed", row_number)
|
||||
return {
|
||||
"line_number": row_number,
|
||||
"input": row,
|
||||
"output": None,
|
||||
"error": f"{type(exc).__name__}: {exc}",
|
||||
}
|
||||
|
||||
# ---- finalize -----------------------------------------------------
|
||||
def finalize(self) -> None:
|
||||
return None
|
||||
|
||||
# ---- internal -----------------------------------------------------
|
||||
def _setup_tracing(self) -> None:
|
||||
"""Optional Application Insights tracing (Phase 4a)."""
|
||||
conn = os.getenv("APPLICATIONINSIGHTS_CONNECTION_STRING")
|
||||
if not conn:
|
||||
return
|
||||
try:
|
||||
from azure.monitor.opentelemetry import configure_azure_monitor
|
||||
from agent_framework.observability import configure_otel_providers
|
||||
|
||||
configure_azure_monitor(connection_string=conn)
|
||||
configure_otel_providers()
|
||||
logger.info("Application Insights tracing enabled.")
|
||||
except ImportError:
|
||||
logger.warning(
|
||||
"APPLICATIONINSIGHTS_CONNECTION_STRING is set but "
|
||||
"azure-monitor-opentelemetry is not installed; skipping."
|
||||
)
|
||||
+105
@@ -0,0 +1,105 @@
|
||||
"""
|
||||
Mini-batch orchestration. Equivalent of the Prompt Flow PRS processor:
|
||||
|
||||
PF MAF (this file)
|
||||
---------------------------------- ----------------------------------
|
||||
load_component(flow.dag.yaml) create_processor(working_dir, args)
|
||||
Row.from_dict(data, row_number=base+i) executor.execute(row, base+i)
|
||||
json.dumps(result, cls=DataClassEncoder) json.dumps(result, default=str)
|
||||
|
||||
The processor:
|
||||
* holds a single asyncio event loop (created in init(), reused across all
|
||||
run() calls — calling asyncio.run() per row leaks Azure SDK transports);
|
||||
* dispatches rows to the executor with `asyncio.gather` for in-process row
|
||||
concurrency (bounded by `max_concurrency_per_instance` in component.yaml);
|
||||
* preserves PRS row order so appended JSONL lines match the input file.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Any, Iterable, List, Optional
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from .config import MafPrsConfig, parse_args
|
||||
from .executor import MafWorkflowExecutor
|
||||
|
||||
logger = logging.getLogger("maf-prs.processor")
|
||||
|
||||
|
||||
class MafWorkflowProcessor:
|
||||
"""PRS processor — drives one MafWorkflowExecutor per worker process."""
|
||||
|
||||
def __init__(self, working_dir: Path, args: Optional[List[str]] = None):
|
||||
self._working_dir = working_dir
|
||||
self._args = args
|
||||
self._cfg: Optional[MafPrsConfig] = None
|
||||
self._executor: Optional[MafWorkflowExecutor] = None
|
||||
self._loop: Optional[asyncio.AbstractEventLoop] = None
|
||||
|
||||
# ---- PRS: init ----------------------------------------------------
|
||||
def init(self) -> None:
|
||||
self._cfg = parse_args(self._args)
|
||||
self._loop = asyncio.new_event_loop()
|
||||
asyncio.set_event_loop(self._loop)
|
||||
self._executor = MafWorkflowExecutor(self._working_dir, self._cfg)
|
||||
self._executor.init()
|
||||
if self._cfg.debug_output_dir:
|
||||
self._cfg.debug_output_dir.mkdir(parents=True, exist_ok=True)
|
||||
logger.info("MAF PRS processor initialised: cfg=%s", self._cfg)
|
||||
|
||||
# ---- PRS: process(mini_batch, context) ----------------------------
|
||||
def process(self, mini_batch: Any, context: Any) -> List[str]:
|
||||
"""Returns one JSON string per input row. PRS appends each as a
|
||||
line to outputs.flow_outputs (AML equivalent of PF's
|
||||
`parallel_run_step.jsonl`)."""
|
||||
assert self._executor is not None and self._loop is not None, "init() was not called"
|
||||
base = getattr(context, "global_row_index_lower_bound", 0)
|
||||
minibatch_id = getattr(context, "minibatch_index", "?")
|
||||
rows = list(self._iter_rows(mini_batch))
|
||||
logger.info("mini_batch %s: %d rows (base row_number=%d)", minibatch_id, len(rows), base)
|
||||
|
||||
async def _run_all() -> List[dict]:
|
||||
return await asyncio.gather(
|
||||
*(self._executor.execute(row, base + i) for i, row in enumerate(rows))
|
||||
)
|
||||
|
||||
results = self._loop.run_until_complete(_run_all())
|
||||
return [json.dumps(r, default=str) for r in results]
|
||||
|
||||
# ---- PRS: shutdown / finalize ------------------------------------
|
||||
def finalize(self) -> None:
|
||||
try:
|
||||
if self._executor is not None:
|
||||
self._executor.finalize()
|
||||
finally:
|
||||
if self._loop is not None and not self._loop.is_closed():
|
||||
self._loop.run_until_complete(self._loop.shutdown_asyncgens())
|
||||
self._loop.close()
|
||||
|
||||
# ---- helpers ------------------------------------------------------
|
||||
@staticmethod
|
||||
def _iter_rows(mini_batch: Any) -> Iterable[dict]:
|
||||
"""Normalise the PRS mini_batch into a stream of row dicts.
|
||||
|
||||
PRS dispatches one of three shapes:
|
||||
* `pandas.DataFrame` when input is `mltable` (tabular).
|
||||
* `list[dict]` when input is `uri_file` + `--amlbi_file_format jsonl`
|
||||
(PRS parses the jsonl and hands each row as a dict).
|
||||
* `list[str]` of file paths when input is `uri_folder` of opaque files.
|
||||
"""
|
||||
if isinstance(mini_batch, pd.DataFrame):
|
||||
yield from mini_batch.to_dict(orient="records")
|
||||
return
|
||||
for item in mini_batch:
|
||||
if isinstance(item, dict):
|
||||
yield item
|
||||
else:
|
||||
yield {"path": str(item)}
|
||||
|
||||
|
||||
def create_processor(working_dir: Path, args: Optional[List[str]] = None) -> MafWorkflowProcessor:
|
||||
return MafWorkflowProcessor(working_dir, args)
|
||||
+131
@@ -0,0 +1,131 @@
|
||||
"""
|
||||
Submit the MAF Linear Flow PRS pipeline to Azure ML.
|
||||
|
||||
This is the MAF-side analogue of the Prompt Flow PRS submission shown in
|
||||
`examples/tutorials/run-flow-with-pipeline/pipeline.ipynb` section 3.2.1.
|
||||
|
||||
PF MAF (this file)
|
||||
---------------------------------- -----------------------------------
|
||||
load_component("flow.dag.yaml") --> load_component("component.yaml")
|
||||
flow_node(question="${data.q}") --> workflow_component(data=...)
|
||||
+ hooks.build_workflow_input
|
||||
maps the row to the input
|
||||
flow_node.compute / .resources / --> identical assignments on the
|
||||
.mini_batch_size / .retry_settings component instance returned by
|
||||
/ .logging_level / etc. the @pipeline DSL
|
||||
|
||||
Run:
|
||||
python submit_pipeline.py
|
||||
"""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from azure.ai.ml import Input, MLClient, Output, load_component
|
||||
from azure.ai.ml.constants import AssetTypes
|
||||
from azure.ai.ml.dsl import pipeline
|
||||
from azure.identity import DefaultAzureCredential, InteractiveBrowserCredential
|
||||
from dotenv import load_dotenv
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 0. Load .env from the migration-guide root (same file the rest of the
|
||||
# samples use). Variables already present in the environment win.
|
||||
# ---------------------------------------------------------------------------
|
||||
HERE = Path(__file__).parent
|
||||
load_dotenv(HERE.parents[1] / ".env", override=False)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. Workspace handle. Set the SUBSCRIPTION_ID, RESOURCE_GROUP, and
|
||||
# WORKSPACE_NAME env vars (or run `az ml workspace show` to populate
|
||||
# config.json next to this script).
|
||||
# ---------------------------------------------------------------------------
|
||||
try:
|
||||
credential = DefaultAzureCredential()
|
||||
credential.get_token("https://management.azure.com/.default")
|
||||
except Exception: # noqa: BLE001
|
||||
credential = InteractiveBrowserCredential()
|
||||
|
||||
if {"SUBSCRIPTION_ID", "RESOURCE_GROUP", "WORKSPACE_NAME"} <= set(os.environ):
|
||||
ml_client = MLClient(
|
||||
credential=credential,
|
||||
subscription_id=os.environ["SUBSCRIPTION_ID"],
|
||||
resource_group_name=os.environ["RESOURCE_GROUP"],
|
||||
workspace_name=os.environ["WORKSPACE_NAME"],
|
||||
)
|
||||
else:
|
||||
ml_client = MLClient.from_config(credential=credential)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. Load the parallel component (replaces `load_component(flow.dag.yaml)`).
|
||||
# ---------------------------------------------------------------------------
|
||||
linear_flow_component = load_component(str(HERE / "component.yaml"))
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. Pipeline-level input — sample data shipped with this folder.
|
||||
# ---------------------------------------------------------------------------
|
||||
data_input = Input(
|
||||
path=str(HERE / "data" / "sample.jsonl"),
|
||||
type=AssetTypes.URI_FILE,
|
||||
mode="mount",
|
||||
)
|
||||
|
||||
pipeline_output = Output(
|
||||
type=AssetTypes.URI_FOLDER,
|
||||
mode="rw_mount",
|
||||
)
|
||||
|
||||
# CUSTOMISE: pick the cluster you want to run on. Must already exist in the
|
||||
# workspace (see `az ml compute create`).
|
||||
COMPUTE_NAME = os.environ.get("AML_COMPUTE", "cpu-cluster")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. Pipeline definition (mirrors the @pipeline() function in the PF script).
|
||||
# ---------------------------------------------------------------------------
|
||||
@pipeline()
|
||||
def linear_flow_prs_pipeline(
|
||||
pipeline_input_data: Input,
|
||||
parallel_node_count: int = 1,
|
||||
):
|
||||
workflow_node = linear_flow_component(data=pipeline_input_data)
|
||||
|
||||
# === Carry over PF run settings 1:1 =====================================
|
||||
workflow_node.environment_variables = {
|
||||
# The Foundry chat client inside 01_linear_flow.py reads these at
|
||||
# workflow construction time. Make sure the AML compute identity has
|
||||
# `Cognitive Services User` on the Foundry resource.
|
||||
"FOUNDRY_PROJECT_ENDPOINT": os.environ.get("FOUNDRY_PROJECT_ENDPOINT", ""),
|
||||
"FOUNDRY_MODEL": os.environ.get("FOUNDRY_MODEL", "gpt-4o"),
|
||||
}
|
||||
workflow_node.compute = COMPUTE_NAME
|
||||
workflow_node.resources = {"instance_count": parallel_node_count}
|
||||
workflow_node.mini_batch_size = 5
|
||||
workflow_node.max_concurrency_per_instance = 2
|
||||
workflow_node.retry_settings = {"max_retries": 1, "timeout": 1200}
|
||||
workflow_node.error_threshold = -1
|
||||
workflow_node.mini_batch_error_threshold = -1
|
||||
workflow_node.logging_level = "DEBUG"
|
||||
|
||||
# When instance_count > 1, both PRS outputs must use mount mode (same
|
||||
# rule as the PF flow component).
|
||||
workflow_node.outputs.flow_outputs.mode = "mount"
|
||||
workflow_node.outputs.debug_info.mode = "mount"
|
||||
# ========================================================================
|
||||
|
||||
return {"flow_result_folder": workflow_node.outputs.flow_outputs}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. Submit.
|
||||
# ---------------------------------------------------------------------------
|
||||
pipeline_job_def = linear_flow_prs_pipeline(pipeline_input_data=data_input)
|
||||
pipeline_job_def.outputs.flow_result_folder = pipeline_output
|
||||
|
||||
if __name__ == "__main__":
|
||||
pipeline_job_run = ml_client.jobs.create_or_update(
|
||||
pipeline_job_def,
|
||||
experiment_name="maf_linear_flow_prs",
|
||||
)
|
||||
print(f"Submitted job: {pipeline_job_run.name}")
|
||||
print(f"Studio URL: {pipeline_job_run.studio_url}")
|
||||
ml_client.jobs.stream(pipeline_job_run.name)
|
||||
@@ -0,0 +1,17 @@
|
||||
# Phase 4 — Migrate Operations
|
||||
|
||||
Replace the operational infrastructure that Prompt Flow managed automatically.
|
||||
|
||||
| Sub-phase | What it replaces | Folder |
|
||||
|---|---|---|
|
||||
| **4a — Tracing** | Prompt Flow's built-in run viewer | [4a-tracing](4a-tracing) |
|
||||
| **4b — Deployment** | Prompt Flow Managed Online Endpoint | [4b-deployment](4b-deployment) |
|
||||
| **4c — CI/CD** | Manual evaluation runs in the PF UI | [4c-cicd](4c-cicd) |
|
||||
| **4d — Batch (PRS)** | Prompt Flow Parallel Run Step (`load_component(flow.dag.yaml)`) | [4d-pipelines](4d-pipelines) |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Application Insights instance (`4a`)
|
||||
- Azure Container Registry + Container Apps environment (`4b`)
|
||||
- GitHub repository secrets configured (`4c`) — see `4c-cicd/evaluate.yml`
|
||||
- Azure ML workspace + CPU cluster (`4d`)
|
||||
@@ -0,0 +1,21 @@
|
||||
# Phase 5 — Cut Over
|
||||
|
||||
Switch production traffic to MAF and decommission Prompt Flow resources.
|
||||
|
||||
## Prerequisites before running cutover.sh
|
||||
|
||||
- [ ] Mean parity score ≥ 3.5 across the full test suite (`parity_results.csv`)
|
||||
- [ ] MAF Container App healthy (`az containerapp show`)
|
||||
- [ ] Tracing confirmed in Application Insights
|
||||
- [ ] CI/CD quality gate passing on main branch
|
||||
- [ ] API gateway or client config already updated to point at the MAF endpoint
|
||||
|
||||
## Run
|
||||
|
||||
bash phase-5-cutover/cutover.sh # execute for real
|
||||
bash phase-5-cutover/cutover.sh --dry-run # preview commands without executing them
|
||||
|
||||
## After cutover
|
||||
|
||||
- Monitor Application Insights for error spikes in the first 24 hours
|
||||
- Keep the archived flow YAML for 30 days before deleting
|
||||
@@ -0,0 +1,69 @@
|
||||
#!/bin/bash
|
||||
# Retires Prompt Flow resources after MAF is confirmed stable in production.
|
||||
#
|
||||
# Replaces: Prompt Flow managed online endpoint and connections.
|
||||
#
|
||||
# Prerequisites: Traffic already rerouted to MAF. az login completed.
|
||||
# Usage:
|
||||
# bash cutover.sh # execute for real
|
||||
# bash cutover.sh --dry-run # print commands without executing them
|
||||
# bash cutover.sh --yes # skip confirmation prompt
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
DRY_RUN=false
|
||||
SKIP_CONFIRM=false
|
||||
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--dry-run)
|
||||
DRY_RUN=true
|
||||
;;
|
||||
--yes|--force)
|
||||
SKIP_CONFIRM=true
|
||||
;;
|
||||
*)
|
||||
echo "Unknown argument: $arg" >&2
|
||||
echo "Usage: bash cutover.sh [--dry-run] [--yes]" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Wrapper: prints the command in dry-run mode, otherwise executes it.
|
||||
run() {
|
||||
if $DRY_RUN; then
|
||||
echo "[DRY RUN] $*"
|
||||
else
|
||||
"$@"
|
||||
fi
|
||||
}
|
||||
|
||||
PF_ENDPOINT="<your-pf-endpoint>"
|
||||
PF_CONNECTION="<your-pf-connection>"
|
||||
RESOURCE_GROUP="<your-rg>"
|
||||
WORKSPACE="<your-ws>"
|
||||
FLOW_DIR="<your-flow-directory>"
|
||||
|
||||
if ! $DRY_RUN && ! $SKIP_CONFIRM; then
|
||||
read -p "Confirm traffic has been rerouted to the MAF endpoint (y/n): " confirm
|
||||
[[ "$confirm" == "y" ]] || { echo "Aborting."; exit 1; }
|
||||
fi
|
||||
|
||||
echo "Archiving flow YAML to ./archived-flow/..."
|
||||
run cp -r "$FLOW_DIR" ./archived-flow/
|
||||
|
||||
echo "Deleting PF managed online endpoint..."
|
||||
run az ml online-endpoint delete \
|
||||
--name "$PF_ENDPOINT" \
|
||||
--resource-group "$RESOURCE_GROUP" \
|
||||
--workspace-name "$WORKSPACE" \
|
||||
--yes
|
||||
|
||||
echo "Deleting PF connection..."
|
||||
run az ml connection delete \
|
||||
--name "$PF_CONNECTION" \
|
||||
--resource-group "$RESOURCE_GROUP" \
|
||||
--workspace-name "$WORKSPACE"
|
||||
|
||||
echo "Done. Keep the archived flow YAML for at least 30 days before deleting."
|
||||
@@ -0,0 +1,32 @@
|
||||
# Root requirements — covers all samples in this repo.
|
||||
# MAF core and foundry are GA (1.0.1). Orchestrations and Azure AI Search are
|
||||
# still in preview and require --pre on a separate pip install command.
|
||||
|
||||
# GA packages
|
||||
agent-framework>=1.0.1
|
||||
agent-framework-foundry>=1.0.1
|
||||
|
||||
# Preview packages — install separately with --pre:
|
||||
# pip install agent-framework-orchestrations --pre
|
||||
# pip install agent-framework-azure-ai-search --pre
|
||||
# pip install agent-framework-devui --pre (Phase 2 visualization only)
|
||||
agent-framework-orchestrations>=1.0.0b260409
|
||||
agent-framework-azure-ai-search>=1.0.0b260409
|
||||
agent-framework-devui>=1.0.0b260409
|
||||
|
||||
# Evaluation (Phase 3)
|
||||
azure-ai-evaluation>=1.16.0
|
||||
|
||||
# Deployment wrapper (Phase 4b)
|
||||
fastapi>=0.115.0
|
||||
uvicorn>=0.34.0
|
||||
python-dotenv>=1.0.0
|
||||
|
||||
# Tracing (Phase 4a)
|
||||
azure-monitor-opentelemetry>=1.6.4
|
||||
|
||||
# Azure identity — used by FoundryChatClient for authentication
|
||||
azure-identity>=1.19.0
|
||||
|
||||
# Parity check (Phase 3)
|
||||
pandas>=2.2.0
|
||||
@@ -0,0 +1,430 @@
|
||||
# Skills — PromptFlow-to-MAF Migration Guide
|
||||
|
||||
Instructions for AI coding agents working on the Prompt Flow → Microsoft Agent Framework migration guide.
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
Prompt Flow is being retired. This folder contains a 5-phase, hands-on migration guide with runnable Python samples that move a Prompt Flow workload to **Microsoft Agent Framework (MAF) 1.0 GA**.
|
||||
|
||||
Target audience: teams running Prompt Flow on Microsoft Foundry or Azure Machine Learning.
|
||||
|
||||
---
|
||||
|
||||
## AI-Assisted Migration with the Copilot Skill
|
||||
|
||||
This repository includes a **Copilot skill file** at [`.github/skills/promptflow-to-maf/SKILL.md`](../../.github/skills/promptflow-to-maf/SKILL.md) that enables AI coding agents (such as GitHub Copilot in VS Code) to **automatically convert** your Prompt Flow `flow.dag.yaml` into a runnable Microsoft Agent Framework project.
|
||||
|
||||
### What the Skill Does
|
||||
|
||||
The skill teaches the AI agent how to:
|
||||
|
||||
1. **Parse your `flow.dag.yaml`** and all referenced source files (`.jinja2` templates, `.py` nodes, `requirements.txt`).
|
||||
2. **Map every Prompt Flow node** to its MAF equivalent (`Executor`, `Agent`, `WorkflowBuilder`, etc.) using a built-in conversion table.
|
||||
3. **Generate a complete MAF project** in a sibling `<your-flow>-maf/` folder — including workflow code, `.env.example`, `requirements.txt`, and a runnable test script.
|
||||
4. **Handle advanced patterns** — chat history, multimodal inputs, fan-out/fan-in, conditional branching, evaluation flows with aggregation, and multi-agent handoffs.
|
||||
5. **Preserve prompts verbatim** — system prompts, Jinja2 templates, and LLM parameters (`temperature`, `max_tokens`, etc.) are carried over exactly.
|
||||
|
||||
### How to Use It
|
||||
|
||||
#### Prerequisites
|
||||
|
||||
- **VS Code** with the [GitHub Copilot Chat](https://marketplace.visualstudio.com/items?itemName=GitHub.copilot-chat) extension installed.
|
||||
- This repository cloned locally and opened as your workspace (the skill is auto-discovered from `.github/skills/`).
|
||||
|
||||
#### Step-by-Step
|
||||
|
||||
1. **Open your Prompt Flow folder** in VS Code — navigate to the directory containing your `flow.dag.yaml`.
|
||||
|
||||
2. **Open GitHub Copilot Chat** (Ctrl+Shift+I or the Copilot icon in the sidebar).
|
||||
|
||||
3. **Ask Copilot to convert your flow.** Use a prompt like:
|
||||
|
||||
```
|
||||
Convert this Prompt Flow to Microsoft Agent Framework
|
||||
```
|
||||
|
||||
or be more specific:
|
||||
|
||||
```
|
||||
Migrate the flow in examples/flows/chat/chat-basic to MAF
|
||||
```
|
||||
|
||||
The skill activates automatically when it detects migration-related intent (e.g., "convert promptflow", "migrate flow.dag.yaml", "PF to agent-framework").
|
||||
|
||||
4. **Copilot reads your flow**, maps each node, and generates the MAF project files in a new `<flow-name>-maf/` folder alongside your original flow.
|
||||
|
||||
5. **Review the generated code.** The output includes:
|
||||
- `workflow.py` (or numbered sample files) — Executor classes and `WorkflowBuilder` wiring
|
||||
- `requirements.txt` — only the needed `agent-framework-*` packages
|
||||
- `.env.example` — environment variable template for your credentials
|
||||
- `test_<name>.py` — runnable script to verify the workflow
|
||||
|
||||
6. **Set up and run:**
|
||||
|
||||
```bash
|
||||
cd <flow-name>-maf/
|
||||
pip install -r requirements.txt
|
||||
cp .env.example .env # fill in your credentials
|
||||
python test_<name>.py
|
||||
```
|
||||
|
||||
### What the Skill Covers
|
||||
|
||||
| Prompt Flow Pattern | Skill Handles It? |
|
||||
|---|---|
|
||||
| Linear LLM chains | Yes |
|
||||
| Chat flows with history | Yes |
|
||||
| Conditional branching (`activate_config`) | Yes |
|
||||
| Parallel execution (fan-out / fan-in) | Yes |
|
||||
| RAG (Embed + Vector Lookup + LLM) | Yes |
|
||||
| Python tool nodes | Yes |
|
||||
| Multimodal inputs (images) | Yes |
|
||||
| Evaluation flows (`aggregation: true`) | Yes |
|
||||
| Multi-agent handoffs | Yes |
|
||||
| Custom Python packages imported by nodes | Yes — copied into output folder |
|
||||
|
||||
### Tips
|
||||
|
||||
- **Attach your flow files** — if Copilot doesn't read your flow automatically, attach `flow.dag.yaml` and key source files to the chat for context.
|
||||
- **Iterate** — you can ask follow-up questions like "add error handling to the LLM executor" or "switch from API key auth to managed identity".
|
||||
- **The original flow is never modified** — all generated files go into the new `-maf/` folder.
|
||||
- **Evaluation flows** are automatically split into a per-row workflow, an aggregation function, and an `EvalRunner` orchestrator.
|
||||
|
||||
> **Note:** The skill file is designed for AI coding agents. You do not need to read or edit `SKILL.md` yourself — it is consumed by Copilot automatically when the workspace is loaded.
|
||||
|
||||
---
|
||||
|
||||
## AI-Assisted Online Endpoint Deployment with the Copilot Skill
|
||||
|
||||
A second Copilot skill at [`.github/skills/maf-online-endpoint/SKILL.md`](../../.github/skills/maf-online-endpoint/SKILL.md) enables AI coding agents to **automatically generate deployment configuration files** and **deploy a MAF workflow** as an Azure ML managed online endpoint — to either an Azure Machine Learning workspace or an Azure AI Foundry hub-based project.
|
||||
|
||||
### What the Skill Does
|
||||
|
||||
The skill teaches the AI agent how to:
|
||||
|
||||
1. **Inspect your workflow file** — read the `workflow.py` (or equivalent) to discover imports, environment variables, and credential patterns (API key vs. managed identity).
|
||||
2. **Gather deployment parameters** — interactively ask for subscription ID, resource group, workspace/project name, endpoint name, VM SKU, and workflow-specific environment variables.
|
||||
3. **Generate a complete `online-deployment/` directory** containing all files needed for a managed online endpoint:
|
||||
- `score.py` — scoring script with `init()`/`run()` pattern, importing the workflow factory
|
||||
- `conda.yml` — conda environment with Python 3.11, `agent-framework`, and workflow-specific packages
|
||||
- `endpoint.yml` — endpoint configuration (name, auth mode)
|
||||
- `deployment.yml` — deployment template with `${VAR}` placeholders for environment variables
|
||||
- `deploy.sh` — Bash deploy script (Linux/macOS); on Windows, the agent runs `az` CLI commands directly in PowerShell
|
||||
- `.gitignore` — prevents rendered YAML files containing secrets from being committed
|
||||
4. **Render and deploy** — substitute placeholders with actual values, create the endpoint, create the deployment, and run a smoke test.
|
||||
5. **Assign RBAC** (when needed) — for managed-identity workflows (Foundry/`DefaultAzureCredential`), assign `Cognitive Services User` on the AI Services resource.
|
||||
|
||||
### Deployment Targets
|
||||
|
||||
| Target | Description |
|
||||
|--------|-------------|
|
||||
| **Azure Machine Learning workspace** | Standalone AML workspace — provide subscription, resource group, and workspace name |
|
||||
| **Azure AI Foundry project** | Hub-based AI project — the project name is used as the workspace name for `az ml` commands |
|
||||
|
||||
Both targets produce identical generated files and use the same `az ml` CLI commands.
|
||||
|
||||
### How to Use It
|
||||
|
||||
#### Prerequisites
|
||||
|
||||
- **VS Code** with the [GitHub Copilot Chat](https://marketplace.visualstudio.com/items?itemName=GitHub.copilot-chat) extension installed.
|
||||
- This repository cloned locally and opened as your workspace.
|
||||
- **Azure CLI** installed with the `ml` extension (`az extension add -n ml`).
|
||||
- An existing MAF workflow (e.g., generated by the conversion skill above).
|
||||
|
||||
#### Step-by-Step
|
||||
|
||||
1. **Open your MAF workflow project** in VS Code — navigate to the directory containing your `workflow.py`.
|
||||
|
||||
2. **Open GitHub Copilot Chat** (Ctrl+Shift+I or the Copilot icon in the sidebar).
|
||||
|
||||
3. **Ask Copilot to deploy your workflow.** Use a prompt like:
|
||||
|
||||
```
|
||||
Deploy this workflow as an online endpoint
|
||||
```
|
||||
|
||||
or be more specific:
|
||||
|
||||
```
|
||||
Create a managed online endpoint for examples/flows/standard/describe-image-maf
|
||||
```
|
||||
|
||||
The skill activates automatically when it detects deployment-related intent (e.g., "deploy MAF workflow", "create online endpoint", "deploy agent as endpoint").
|
||||
|
||||
4. **Copilot asks for deployment details** — it will interactively prompt you for:
|
||||
- Deployment target (AML workspace or AI Foundry project)
|
||||
- Subscription ID, resource group, workspace/project name
|
||||
- Endpoint name and VM SKU
|
||||
- Workflow-specific credentials (API keys, endpoints, model deployment names)
|
||||
|
||||
5. **Copilot generates all deployment files** in an `online-deployment/` subdirectory:
|
||||
|
||||
```
|
||||
<your-workflow>/
|
||||
workflow.py
|
||||
online-deployment/
|
||||
score.py
|
||||
conda.yml
|
||||
endpoint.yml
|
||||
deployment.yml
|
||||
deploy.sh
|
||||
.gitignore
|
||||
```
|
||||
|
||||
6. **Copilot renders and deploys** — it substitutes placeholders, runs `az ml online-endpoint create` and `az ml online-deployment create`, then invokes the endpoint with a smoke test.
|
||||
|
||||
7. **Review the results.** Copilot reports the scoring URI and endpoint status.
|
||||
|
||||
### Generated Files Reference
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `score.py` | Scoring script — `init()` imports the workflow factory; `run()` creates a fresh workflow per request to avoid concurrency errors |
|
||||
| `conda.yml` | Conda environment — Python 3.11 with only the packages your workflow needs |
|
||||
| `endpoint.yml` | Endpoint name and auth mode (`key` by default) |
|
||||
| `deployment.yml` | Deployment template with `${VAR}` placeholders for environment variables, instance type, and request settings |
|
||||
| `deploy.sh` | Bash deploy script for Linux/macOS (on Windows, the agent runs commands directly in PowerShell) |
|
||||
| `.gitignore` | Excludes `deployment-rendered.yml` which may contain secrets |
|
||||
|
||||
### Key Design Decisions
|
||||
|
||||
- **One workflow per request** — `score.py` calls the `create_workflow()` factory on every request, avoiding `RuntimeError: Workflow is already running` on concurrent requests.
|
||||
- **Path resolution** — since `deployment.yml` lives in `online-deployment/`, it uses `conda_file: conda.yml` (same directory) and `code: ..` (parent = project root). The scoring script path is `online-deployment/score.py` relative to the code root.
|
||||
- **Request timeout** — set to 60 seconds (vs. the 5-second AML default) to accommodate LLM call latency.
|
||||
- **Security** — rendered YAML files with substituted secrets are `.gitignore`d. API keys are injected as deployment environment variables, not baked into code.
|
||||
|
||||
### Credential Patterns
|
||||
|
||||
| Pattern | Env Vars | RBAC Needed? |
|
||||
|---------|----------|--------------|
|
||||
| **Azure OpenAI (API key)** | `AZURE_OPENAI_ENDPOINT`, `AZURE_OPENAI_DEPLOYMENT`, `AZURE_OPENAI_API_KEY` | No |
|
||||
| **Foundry (managed identity)** | `FOUNDRY_PROJECT_ENDPOINT`, `FOUNDRY_MODEL` | Yes — `Cognitive Services User` on the AI Services resource |
|
||||
| **RAG (AI Search)** | Above + `AZURE_AI_SEARCH_ENDPOINT`, `AZURE_AI_SEARCH_INDEX_NAME`, `AZURE_AI_SEARCH_API_KEY` | Depends on LLM auth pattern |
|
||||
|
||||
### Example Prompts
|
||||
|
||||
```
|
||||
Deploy this MAF workflow as an online endpoint to my AI Foundry project
|
||||
```
|
||||
|
||||
```
|
||||
Create an online endpoint deployment for the describe-image workflow
|
||||
```
|
||||
|
||||
```
|
||||
I need to deploy my agent-framework workflow to Azure ML
|
||||
```
|
||||
|
||||
### Tips
|
||||
|
||||
- **Run from the project root** — the `az ml online-deployment create` command must be run from the directory containing `workflow.py`, not from inside `online-deployment/`.
|
||||
- **Windows users** — `deploy.sh` requires Bash; Copilot automatically uses PowerShell string replacement on Windows instead of `envsubst`.
|
||||
- **Check deployment logs** — if the endpoint returns errors, run `az ml online-deployment get-logs --name blue --endpoint-name <name>` to view container logs.
|
||||
- **RBAC propagation** — after assigning `Cognitive Services User` for managed-identity workflows, wait 5–10 minutes before invoking the endpoint.
|
||||
- **Iterate** — you can ask follow-up questions like "switch to managed identity auth" or "add Application Insights tracing to the endpoint".
|
||||
|
||||
> **Note:** The skill file is designed for AI coding agents. You do not need to read or edit `SKILL.md` yourself — it is consumed by Copilot automatically when the workspace is loaded.
|
||||
|
||||
---
|
||||
|
||||
## Repository Layout
|
||||
|
||||
```
|
||||
migration-guide/PromptFlow-to-MAF/
|
||||
├── README.md # Top-level overview and setup instructions
|
||||
├── TROUBLESHOOTING.md # Common migration errors and fixes
|
||||
├── requirements.txt # Python dependencies (MAF 1.0 GA, eval SDK, etc.)
|
||||
├── .env.example # Environment variable template
|
||||
├── .github/ISSUE_TEMPLATE/ # Issue template for migration problems
|
||||
├── phase-1-audit/ # Export PF flow YAML; map nodes to MAF equivalents
|
||||
│ ├── README.md
|
||||
│ └── node-mapping.md # Full PF → MAF concept mapping table
|
||||
├── phase-2-rebuild/ # Re-implement flows using WorkflowBuilder + Executor
|
||||
│ ├── README.md
|
||||
│ └── 01–07_*.py # Progressive samples (linear → multi-agent)
|
||||
├── phase-3-validate/ # Side-by-side parity scoring with Azure AI Eval SDK
|
||||
│ ├── README.md
|
||||
│ ├── parity_check.py # Single-row parity scorer
|
||||
│ └── parity_check_batch.py # Concurrent batch parity scorer
|
||||
├── phase-4-migrate-ops/ # Tracing, deployment, CI/CD
|
||||
│ ├── 4a-tracing/ # OpenTelemetry + Application Insights setup
|
||||
│ ├── 4b-deployment/ # AML managed online endpoint (score.py, conda.yml)
|
||||
│ └── 4c-cicd/ # GitHub Actions quality gate (evaluate.yml)
|
||||
└── phase-5-cutover/ # Traffic switch + PF decommissioning script
|
||||
├── README.md
|
||||
└── cutover.sh # Automated (or dry-run) PF retirement
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Migration Phases — Quick Reference
|
||||
|
||||
| Phase | Goal | Key Output |
|
||||
|-------|------|------------|
|
||||
| **1 — Audit & Map** | Understand and document the existing PF flow | Exported `flow.dag.yaml`, completed node-mapping table |
|
||||
| **2 — Rebuild** | Re-implement in MAF using `WorkflowBuilder` + `Executor` | Working `.py` files mirroring PF behaviour |
|
||||
| **3 — Validate** | Confirm semantic parity with `SimilarityEvaluator` | `parity_results.csv` with mean score ≥ 3.5 |
|
||||
| **4 — Migrate Ops** | Replace PF operational infra (tracing, hosting, CI/CD) | App Insights traces, Container App, GitHub Actions gate |
|
||||
| **5 — Cut Over** | Route traffic to MAF; retire PF endpoints | `cutover.sh` executed; PF connections deleted |
|
||||
|
||||
Always work through phases in order. Do not skip ahead.
|
||||
|
||||
---
|
||||
|
||||
## Core MAF Concepts
|
||||
|
||||
These are the foundational abstractions agents should understand when generating or modifying code in this guide:
|
||||
|
||||
| Concept | Description |
|
||||
|---------|-------------|
|
||||
| **Executor** | A class with a `@handler` method that performs one logical step (replaces a PF "node"). |
|
||||
| **WorkflowBuilder** | Fluent builder that registers executors and wires them with `.add_edge()`, `.add_fan_out_edges()`, `.add_fan_in_edges()`, then `.build()`. |
|
||||
| **WorkflowContext** | Type-parameterised context passed to handlers: `WorkflowContext[SendType]` to send downstream, `WorkflowContext[Never, YieldType]` to yield final output, `WorkflowContext[SendType, YieldType]` for both. |
|
||||
| **Agent** | Created via `Agent(client=FoundryChatClient(...), name=..., instructions=...)`. Replaces PF LLM nodes. |
|
||||
| **Context Provider** | E.g. `AzureAISearchContextProvider` — injects RAG context into an agent. Replaces PF Embed Text + Vector Lookup nodes. |
|
||||
| **SimilarityEvaluator** | From `azure-ai-evaluation`. Scores semantic similarity 1–5 between PF and MAF outputs. |
|
||||
|
||||
### Import Paths (MAF 1.0 GA)
|
||||
|
||||
```python
|
||||
from agent_framework import Agent, Executor, WorkflowBuilder, WorkflowContext, handler
|
||||
from agent_framework.foundry import FoundryChatClient
|
||||
from agent_framework.orchestrations import HandoffBuilder # multi-agent handoff
|
||||
from agent_framework_azure_ai_search import AzureAISearchContextProvider
|
||||
from azure.identity import DefaultAzureCredential
|
||||
```
|
||||
|
||||
> **Package versions**: `agent-framework` and `agent-framework-foundry` are GA (1.0.1). `agent-framework-orchestrations` and `agent-framework-azure-ai-search` are still in preview (1.0.0b260409) and require `--pre` for pip install.
|
||||
|
||||
---
|
||||
|
||||
## Code Patterns
|
||||
|
||||
### Every sample follows this structure
|
||||
|
||||
1. **Define Executors** — one class per logical step, each with a `@handler` method.
|
||||
2. **Build the Workflow** — connect executors via `WorkflowBuilder` and `.add_edge()`.
|
||||
3. **Run** — `await workflow.run(input)`, read output from `result.get_outputs()`.
|
||||
|
||||
### Naming Conventions
|
||||
|
||||
- Executor class names: `<Purpose>Executor` (e.g. `InputExecutor`, `LLMExecutor`, `RouterExecutor`).
|
||||
- Workflow names: descriptive PascalCase string (e.g. `"LinearWorkflow"`, `"RAGPipeline"`).
|
||||
- Sample files: `NN_<pattern>.py` numbered by complexity (01–07).
|
||||
|
||||
### Message Construction
|
||||
|
||||
```python
|
||||
# Correct (MAF 1.0 GA):
|
||||
message = Message(role="user", contents=["Hello"])
|
||||
|
||||
# Incorrect (removed in 1.0):
|
||||
message = Message(role="user", text="Hello") # TypeError
|
||||
```
|
||||
|
||||
### Workflow Output
|
||||
|
||||
Terminal executors must call `ctx.yield_output()`, not just `ctx.send_message()`:
|
||||
|
||||
```python
|
||||
# Correct — yields a workflow output:
|
||||
async def handle(self, text: str, ctx: WorkflowContext[Never, str]) -> None:
|
||||
await ctx.yield_output(text)
|
||||
```
|
||||
|
||||
### Environment Variables
|
||||
|
||||
All credentials are read from `.env` via `load_dotenv()`. Never hard-code secrets. See `.env.example` for the full list:
|
||||
|
||||
- `FOUNDRY_PROJECT_ENDPOINT`, `FOUNDRY_MODEL` (for all phase-2 samples and deployment)
|
||||
- `AZURE_OPENAI_API_KEY`, `AZURE_OPENAI_ENDPOINT`, `AZURE_OPENAI_CHAT_DEPLOYMENT_NAME` (for parity evaluation only)
|
||||
- `AZURE_AI_SEARCH_ENDPOINT`, `AZURE_AI_SEARCH_INDEX_NAME`, `AZURE_AI_SEARCH_API_KEY`
|
||||
- `APPLICATIONINSIGHTS_CONNECTION_STRING` (tracing, Phase 4+)
|
||||
|
||||
---
|
||||
|
||||
## Modifying or Adding Samples
|
||||
|
||||
When adding a new sample to `phase-2-rebuild/`:
|
||||
|
||||
1. Number it sequentially after the last file (e.g. `08_<pattern>.py`).
|
||||
2. Start with a docstring that names the Prompt Flow pattern being replaced.
|
||||
3. Follow the three-step structure (Executors → Builder → Run).
|
||||
4. Add the sample to the table in `phase-2-rebuild/README.md`.
|
||||
5. If it introduces a new PF concept, add a row to `phase-1-audit/node-mapping.md`.
|
||||
|
||||
When editing existing samples:
|
||||
|
||||
- Keep the `load_dotenv()` call at the top, before any client instantiation.
|
||||
- Preserve the `if __name__ == "__main__"` block so samples stay independently runnable.
|
||||
- Use `asyncio.run(main())` as the entry point.
|
||||
|
||||
---
|
||||
|
||||
## Validation & Parity Checks
|
||||
|
||||
- **Single-row**: `python phase-3-validate/parity_check.py`
|
||||
- **Batch (concurrent)**: `python phase-3-validate/parity_check_batch.py`
|
||||
- Parity threshold: mean similarity ≥ **3.5** before proceeding to Phase 4.
|
||||
- `SimilarityEvaluator` requires `model_config` with `azure_endpoint`, `api_key`, and `azure_deployment`.
|
||||
- Correct kwargs: `evaluator(query=question, response=maf_answer, ground_truth=pf_answer)`.
|
||||
|
||||
---
|
||||
|
||||
## Deployment
|
||||
|
||||
- **Deploy script**: `phase-4-migrate-ops/4b-deployment/deploy.sh` (Azure ML Online Endpoints)
|
||||
- **CI/CD quality gate**: `phase-4-migrate-ops/4c-cicd/evaluate.yml` (GitHub Actions)
|
||||
- **Tracing**: Both `configure_azure_monitor()` and `configure_otel_providers()` must be called **before** any `workflow.run()`.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting Quick Reference
|
||||
|
||||
| Symptom | Likely Cause | Fix |
|
||||
|---------|-------------|-----|
|
||||
| `ModuleNotFoundError: agent_framework` | Package not installed or RC conflict | `pip uninstall ... -y && pip install agent-framework>=1.0.1` |
|
||||
| `401 Unauthorized` on Azure OpenAI | Missing/wrong API key or endpoint | Check `.env`; ensure endpoint ends with `/` |
|
||||
| `workflow.run()` returns empty outputs | Terminal executor not calling `ctx.yield_output()` | Use `WorkflowContext[Never, T]` and call `ctx.yield_output()` |
|
||||
| `TypeError` on `Message(text=...)` | Removed in 1.0 | Use `Message(role=..., contents=[...])` |
|
||||
| Workflow hangs | Circular edge definition | Check `add_edge()` calls for cycles; set `max_iterations` |
|
||||
| Low parity scores (< 2.0) | Wrong evaluator kwargs | Use `query=`, `response=`, `ground_truth=` |
|
||||
| No traces in App Insights | Missing `configure_otel_providers()` or `configure_azure_monitor()` | Call both at startup, before `workflow.run()` |
|
||||
| `WorkflowBuilder.build()` validation error | Missing start executor, type mismatch, duplicate IDs, or unreachable executor | Check `start_executor=`, edge types, and executor `id=` values |
|
||||
| `/ask` returns 500 | `MAF_WORKFLOW_FILE` points at the wrong file, or the file does not define `workflow` | Point `MAF_WORKFLOW_FILE` at a valid workflow sample/module |
|
||||
| Container App image pull error | ACR auth or tag mismatch | Verify `--registry-server`, `AcrPull` role, and image tag |
|
||||
|
||||
For the full list, see [TROUBLESHOOTING.md](./TROUBLESHOOTING.md).
|
||||
|
||||
---
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
1. **Mixing `--pre` and non-`--pre` installs** — Core MAF packages are GA; preview connectors (e.g. `agent-framework-copilotstudio`) still need `--pre` on a separate `pip install`. Never combine them in a single command.
|
||||
2. **Foundry project endpoints require `FoundryChatClient`** — Foundry project endpoints (`*.services.ai.azure.com`) require `FoundryChatClient` from `agent_framework.foundry`.
|
||||
3. **Fan-in missing a branch** — Every executor in `add_fan_out_edges()` must also appear in `add_fan_in_edges()`, or the aggregator fires early with a partial result.
|
||||
4. **Fan-in handler receives `list[T]`, not `T`** — The aggregator executor's `@handler` parameter must be typed as `list[str]` (or `list[T]`), not a single `str`. The order matches the declaration order in `add_fan_in_edges()`.
|
||||
5. **Condition functions receiving unexpected types** — Conditions receive the exact value passed to `ctx.send_message()`. Match on that value, not a transformed version. Use named functions, not lambdas, for readability and testability.
|
||||
6. **Skipping Phase 3** — Always validate parity before migrating ops. Low-scoring outputs indicate unmigrated logic.
|
||||
7. **Instantiating one client per agent** — Share a single `FoundryChatClient()` instance across multiple agents. Creating separate clients wastes connection resources. See `07_multi_agent.py` for the pattern.
|
||||
8. **Forgetting `start_executor=`** — `WorkflowBuilder(...)` requires a `start_executor=` keyword argument. Also check for duplicate executor IDs, type mismatches on edges, and unreachable executors.
|
||||
9. **Each executor needs a unique `id`** — The `id=` kwarg passed to the executor constructor must be unique within the workflow. Duplicates cause silent overwrites or runtime errors.
|
||||
10. **Tool function docstrings drive agent behaviour** — When registering Python functions as agent tools via `tools=[fn]`, the agent uses the function's docstring to decide when and how to call it. Missing or vague docstrings lead to unreliable tool use.
|
||||
11. **Use `HandoffBuilder` for multi-agent routing** — `07_multi_agent.py` uses `HandoffBuilder` from `agent-framework-orchestrations` which automatically generates handoff tools for each participant. This is cleaner than manual tagged-string routing with condition functions.
|
||||
12. **Using `gpt_similarity` instead of `similarity`** — `SimilarityEvaluator` returns both keys. `gpt_similarity` is deprecated; always read from `similarity`.
|
||||
13. **API keys in production Container Apps** — Use managed identity (`ManagedIdentityCredential`) and Key Vault secret references (`secretref:kv-*`) instead of inline API keys. See `phase-4-migrate-ops/4b-deployment/managed_identity.md`.
|
||||
14. **`DefaultAzureCredential` for local + cloud portability** — Use `DefaultAzureCredential()` when code must run both locally (Azure CLI auth) and in Azure (managed identity). Avoid it in production-only paths where `ManagedIdentityCredential` is more predictable.
|
||||
|
||||
---
|
||||
|
||||
## External References
|
||||
|
||||
- [MAF 1.0 GA announcement](https://devblogs.microsoft.com/agent-framework/microsoft-agent-framework-version-1-0/)
|
||||
- [MAF Python API docs](https://learn.microsoft.com/en-us/agent-framework/)
|
||||
- [MAF Workflows documentation](https://learn.microsoft.com/en-us/agent-framework/workflows/executors)
|
||||
- [Azure AI Evaluation SDK](https://learn.microsoft.com/en-us/python/api/overview/azure/ai-evaluation-readme)
|
||||
- [MAF GitHub repository](https://github.com/microsoft/agent-framework)
|
||||
- [MAF Discord community](https://discord.gg/b5zjErwbQM)
|
||||
@@ -0,0 +1,59 @@
|
||||
"""Helpers for loading a workflow object from a sample file path."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
|
||||
GUIDE_ROOT = Path(__file__).resolve().parent
|
||||
DEFAULT_WORKFLOW_FILE = GUIDE_ROOT / "phase-2-rebuild" / "01_linear_flow.py"
|
||||
|
||||
|
||||
def load_workflow(env_var: str = "MAF_WORKFLOW_FILE"):
|
||||
"""Load a module-level ``workflow`` object from a Python file.
|
||||
|
||||
By default this uses the Phase 2 linear sample so that the migration guide's
|
||||
parity-check and deployment examples remain runnable out of the box. Set
|
||||
``MAF_WORKFLOW_FILE`` to a different file when validating or deploying your
|
||||
own migrated workflow.
|
||||
|
||||
Returns:
|
||||
The loaded module-level workflow object.
|
||||
"""
|
||||
|
||||
workflow_file = os.getenv(env_var)
|
||||
if workflow_file:
|
||||
workflow_path = Path(workflow_file)
|
||||
if not workflow_path.is_absolute():
|
||||
workflow_path = GUIDE_ROOT / workflow_path
|
||||
else:
|
||||
workflow_path = DEFAULT_WORKFLOW_FILE
|
||||
|
||||
if not workflow_path.exists():
|
||||
raise FileNotFoundError(
|
||||
f"Workflow file not found: {workflow_path}\n"
|
||||
f"Set {env_var} to a Python file that defines a module-level "
|
||||
f"'workflow' object. Example: phase-2-rebuild/01_linear_flow.py"
|
||||
)
|
||||
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
f"maf_workflow_{workflow_path.stem.replace('-', '_')}",
|
||||
workflow_path,
|
||||
)
|
||||
if spec is None or spec.loader is None:
|
||||
raise ImportError(f"Unable to load workflow module from {workflow_path}")
|
||||
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = module # register before exec to handle re-imports correctly
|
||||
spec.loader.exec_module(module)
|
||||
|
||||
if not hasattr(module, "workflow"):
|
||||
raise AttributeError(
|
||||
f"{workflow_path} does not define a module-level 'workflow' object.\n"
|
||||
"Expected pattern: workflow = WorkflowBuilder(...).build()"
|
||||
)
|
||||
|
||||
return module.workflow
|
||||
Reference in New Issue
Block a user