chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,285 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "Deep fixtures from feature-parity.json for langgraph-python (needs manual redistribution)",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "beautiful-chat Sales Dashboard pill — turn 0: call query_data before building dashboard.",
|
||||
"match": {
|
||||
"userMessage": "fetch the financial sales data",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_query_data_sales_001",
|
||||
"name": "query_data",
|
||||
"arguments": "{\"query\":\"financial sales data\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "beautiful-chat Sales Dashboard pill — turn 1: after query_data returns, call generate_a2ui. NB: `turnIndex` is intentionally absent — counts assistant messages thread-globally and silently misses when this pill fires after another pill. Anchored on the leg-1 query_data toolCallId so the matcher cleanly fires exactly once: on leg-3 the last tool result is from generate_a2ui (different call_id) so this fixture doesn't re-match and create a generate_a2ui loop. NB: aimock's `toolName` matcher only checks that the named tool is REGISTERED in `effective.tools`, not that the last tool result was from it — so `toolName` alone cannot disambiguate leg-2 from leg-3 here.",
|
||||
"match": {
|
||||
"userMessage": "with total revenue, new customers, and conversion rate metrics",
|
||||
"hasToolResult": true,
|
||||
"toolCallId": "call_fp_query_data_sales_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_generate_a2ui_sales_dashboard_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator pill follow-up after generateSandboxedUi returns (repeat-click variant 1 of 3, anchored on call_fp_beautiful_chat_calculator_001). Closes the turn with a brief confirmation so the chat plateau settles. ORDERING INVARIANT: this entry MUST precede the leg-1 entries below (first-match-wins) — it only matches when the thread's last message is this exact tool result, and on the follow-up turn leg-1's userMessage would otherwise re-match and loop the tool call. Leg-2 disambiguation rests entirely on the runtime round-tripping this tool_call_id (all 18 integrations' sales-dashboard fixtures already depend on that; generateSandboxedUi is a frontend tool, not MCP, so id variability is not a concern).",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_calculator_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Calculator app rendered above — tap a metric shortcut to insert its value, then operate on it with the standard keypad."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator pill follow-up after generateSandboxedUi returns (repeat-click variant 2 of 3, anchored on call_fp_beautiful_chat_calculator_002). Closes the turn with a brief confirmation so the chat plateau settles. ORDERING INVARIANT: this entry MUST precede the leg-1 entries below (first-match-wins) — it only matches when the thread's last message is this exact tool result, and on the follow-up turn leg-1's userMessage would otherwise re-match and loop the tool call. Leg-2 disambiguation rests entirely on the runtime round-tripping this tool_call_id (all 18 integrations' sales-dashboard fixtures already depend on that; generateSandboxedUi is a frontend tool, not MCP, so id variability is not a concern).",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_calculator_002",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Calculator app rendered above — tap a metric shortcut to insert its value, then operate on it with the standard keypad."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator pill follow-up after generateSandboxedUi returns (repeat-click variant 3 of 3, anchored on call_fp_beautiful_chat_calculator_003). Closes the turn with a brief confirmation so the chat plateau settles. ORDERING INVARIANT: this entry MUST precede the leg-1 entries below (first-match-wins) — it only matches when the thread's last message is this exact tool result, and on the follow-up turn leg-1's userMessage would otherwise re-match and loop the tool call. Leg-2 disambiguation rests entirely on the runtime round-tripping this tool_call_id (all 18 integrations' sales-dashboard fixtures already depend on that; generateSandboxedUi is a frontend tool, not MCP, so id variability is not a concern).",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_beautiful_chat_calculator_003",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Calculator app rendered above — tap a metric shortcut to insert its value, then operate on it with the standard keypad."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator (Open Generative UI) pill, repeat-click variant 1 of 3 (sequenceIndex 0: serves the first match for its X-Test-Id). REPEAT-CLICK FIX: each click must emit a DISTINCT tool_call_id — the frontend keys tool renders by id, so a repeated id collapses the second widget into the first one's slot (second never mounts, first remounts and loses its state). aimock co-increments every sequenceIndex sibling that shares these match criteria whenever one of them matches. SCOPE CAVEAT: sequence counters are scoped per X-Test-Id for the lifetime of the aimock process (see showcase/GOTCHAS.md), and matchCriteriaEqual ignores 'context', so all 18 integrations' copies of these variants form ONE co-increment sibling group — a calculator click on ANY integration consumes a seq slot for all. D6 mints per-run unique test ids (buildE2eTestId), so CI gets the full click-order guarantee; manual/staging traffic sends no test id and shares DEFAULT_TEST_ID server-wide, so after the first two calculator clicks (any integration, any session) every later click falls through to the _003 fallback and reuses its id — a degradation floor equal to the pre-fix collapse behavior, never worse. The proper fix is aimock-side: per-thread sequence scoping and/or including 'context' in matchCriteriaEqual. Returns a generateSandboxedUi tool call with a complete calculator widget (matches the d5 calc fixture's pattern but tailored to the beautiful-chat pill's user message). DELIBERATE TRADEOFF: no hasToolResult gate here — hasToolResult is thread-global in aimock and gating on it broke interleaved pills; the toolCallId follow-up entries above (which must stay first) are what stop these entries from re-matching after leg 1. An Excalidraw-style userMessage+hasToolResult:true backstop was considered and REJECTED because it would hijack a first calculator click in a thread that already contains tool results.",
|
||||
"match": {
|
||||
"userMessage": "build a modern calculator with standard buttons",
|
||||
"context": "langgraph-python",
|
||||
"sequenceIndex": 0
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_calculator_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":520,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:280px;margin:0 auto}.display{background:#1e293b;padding:12px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right;font-size:18px;min-height:24px}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}.shortcuts{display:grid;grid-template-columns:repeat(2,1fr);gap:6px;margin-bottom:8px}button{padding:12px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px;cursor:pointer}button.metric{background:#475569;font-size:11px}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-beautiful-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"shortcuts\\\"><button class=\\\"metric\\\" data-v=\\\"1200000\\\">Revenue $1.2M</button><button class=\\\"metric\\\" data-v=\\\"342\\\">Customers 342</button><button class=\\\"metric\\\" data-v=\\\"4.2\\\">Conversion 4.2%</button><button class=\\\"metric\\\" data-v=\\\"50000\\\">Avg Deal $50K</button></div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');function evaluateExpr(){if(!expr||!/^[0-9eE+\\\\-*\\\\/.() ]+$/.test(expr)){display.textContent='err';expr='';return;}try{var value=Function('\\\"use strict\\\";return ('+expr+');')();if(typeof value==='number'&&isFinite(value)){display.textContent=String(value);expr=String(value);}else{display.textContent='err';expr='';}}catch(e){display.textContent='err';expr='';}}document.querySelectorAll('.shortcuts button').forEach(function(btn){btn.addEventListener('click',function(){expr+=btn.dataset.v;display.textContent=expr;});});document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',function(){if(btn.id==='eq'){evaluateExpr();}else{expr+=btn.textContent;display.textContent=expr;}});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator (Open Generative UI) pill, repeat-click variant 2 of 3 (sequenceIndex 1: serves the second match for its X-Test-Id). REPEAT-CLICK FIX: each click must emit a DISTINCT tool_call_id — the frontend keys tool renders by id, so a repeated id collapses the second widget into the first one's slot (second never mounts, first remounts and loses its state). aimock co-increments every sequenceIndex sibling that shares these match criteria whenever one of them matches. SCOPE CAVEAT: sequence counters are scoped per X-Test-Id for the lifetime of the aimock process (see showcase/GOTCHAS.md), and matchCriteriaEqual ignores 'context', so all 18 integrations' copies of these variants form ONE co-increment sibling group — a calculator click on ANY integration consumes a seq slot for all. D6 mints per-run unique test ids (buildE2eTestId), so CI gets the full click-order guarantee; manual/staging traffic sends no test id and shares DEFAULT_TEST_ID server-wide, so after the first two calculator clicks (any integration, any session) every later click falls through to the _003 fallback and reuses its id — a degradation floor equal to the pre-fix collapse behavior, never worse. The proper fix is aimock-side: per-thread sequence scoping and/or including 'context' in matchCriteriaEqual. Returns a generateSandboxedUi tool call with a complete calculator widget (matches the d5 calc fixture's pattern but tailored to the beautiful-chat pill's user message). DELIBERATE TRADEOFF: no hasToolResult gate here — hasToolResult is thread-global in aimock and gating on it broke interleaved pills; the toolCallId follow-up entries above (which must stay first) are what stop these entries from re-matching after leg 1. An Excalidraw-style userMessage+hasToolResult:true backstop was considered and REJECTED because it would hijack a first calculator click in a thread that already contains tool results.",
|
||||
"match": {
|
||||
"userMessage": "build a modern calculator with standard buttons",
|
||||
"context": "langgraph-python",
|
||||
"sequenceIndex": 1
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_calculator_002",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":520,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:280px;margin:0 auto}.display{background:#1e293b;padding:12px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right;font-size:18px;min-height:24px}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}.shortcuts{display:grid;grid-template-columns:repeat(2,1fr);gap:6px;margin-bottom:8px}button{padding:12px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px;cursor:pointer}button.metric{background:#475569;font-size:11px}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-beautiful-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"shortcuts\\\"><button class=\\\"metric\\\" data-v=\\\"1200000\\\">Revenue $1.2M</button><button class=\\\"metric\\\" data-v=\\\"342\\\">Customers 342</button><button class=\\\"metric\\\" data-v=\\\"4.2\\\">Conversion 4.2%</button><button class=\\\"metric\\\" data-v=\\\"50000\\\">Avg Deal $50K</button></div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');function evaluateExpr(){if(!expr||!/^[0-9eE+\\\\-*\\\\/.() ]+$/.test(expr)){display.textContent='err';expr='';return;}try{var value=Function('\\\"use strict\\\";return ('+expr+');')();if(typeof value==='number'&&isFinite(value)){display.textContent=String(value);expr=String(value);}else{display.textContent='err';expr='';}}catch(e){display.textContent='err';expr='';}}document.querySelectorAll('.shortcuts button').forEach(function(btn){btn.addEventListener('click',function(){expr+=btn.dataset.v;display.textContent=expr;});});document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',function(){if(btn.id==='eq'){evaluateExpr();}else{expr+=btn.textContent;display.textContent=expr;}});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Calculator (Open Generative UI) pill, repeat-click variant 3 of 3 (NO sequenceIndex — fallback once both sequenced variants are consumed for the request's X-Test-Id, so strict mode never 503s on repeated clicks; later clicks reuse call_fp_beautiful_chat_calculator_003 and accept the original id-collision degradation. Because sequence counters are per X-Test-Id for the aimock process lifetime and shared across ALL 18 integrations — matchCriteriaEqual ignores 'context'; see showcase/GOTCHAS.md — DEFAULT_TEST_ID traffic without per-run ids lands here after the first two calculator clicks server-wide; floor = pre-fix behavior, and the proper fix is aimock-side per-thread sequence scoping). ORDERING INVARIANT: must stay AFTER the sequenceIndex variants above (their state gates pass only on clicks #1/#2; this entry would otherwise shadow them) and BEFORE the generic 'build a modern calculator' pair below (which it keeps permanently shadowed for this pill). Same response payload and DELIBERATE TRADEOFF as the variants above.",
|
||||
"match": {
|
||||
"userMessage": "build a modern calculator with standard buttons",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_calculator_003",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":520,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:280px;margin:0 auto}.display{background:#1e293b;padding:12px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right;font-size:18px;min-height:24px}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}.shortcuts{display:grid;grid-template-columns:repeat(2,1fr);gap:6px;margin-bottom:8px}button{padding:12px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px;cursor:pointer}button.metric{background:#475569;font-size:11px}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-beautiful-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"shortcuts\\\"><button class=\\\"metric\\\" data-v=\\\"1200000\\\">Revenue $1.2M</button><button class=\\\"metric\\\" data-v=\\\"342\\\">Customers 342</button><button class=\\\"metric\\\" data-v=\\\"4.2\\\">Conversion 4.2%</button><button class=\\\"metric\\\" data-v=\\\"50000\\\">Avg Deal $50K</button></div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');function evaluateExpr(){if(!expr||!/^[0-9eE+\\\\-*\\\\/.() ]+$/.test(expr)){display.textContent='err';expr='';return;}try{var value=Function('\\\"use strict\\\";return ('+expr+');')();if(typeof value==='number'&&isFinite(value)){display.textContent=String(value);expr=String(value);}else{display.textContent='err';expr='';}}catch(e){display.textContent='err';expr='';}}document.querySelectorAll('.shortcuts button').forEach(function(btn){btn.addEventListener('click',function(){expr+=btn.dataset.v;display.textContent=expr;});});document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',function(){if(btn.id==='eq'){evaluateExpr();}else{expr+=btn.textContent;display.textContent=expr;}});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Generic calculator fixture (follow-up) — toolCallId response after generateSandboxedUi renders. Generic fallback: for the beautiful-chat pill this pair is permanently shadowed by the more-specific 'build a modern calculator with standard buttons' entries above (including the non-sequenced repeat-click variant-3 fallback, which keeps this pair dead — so it needs no repeat-click/sequenceIndex treatment FOR THAT PILL); it only fires for hypothetical other prompts containing 'build a modern calculator' but not the more specific 'with standard buttons' phrase (which the specific entries above win), and for those prompts it still emits a single static tool_call_id, so repeated clicks would hit the original id-collision degradation.",
|
||||
"match": {
|
||||
"toolCallId": "call_fp_open_gen_ui_calc_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your calculator with metric shortcut buttons. The shortcuts insert sample company values (revenue, customers, conversion rate, category breakdowns) into the display. Tap any shortcut, then use the standard buttons to do math with those values."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Generic calculator fixture (leg 1) — calls generateSandboxedUi with metric shortcuts (testid ogui-calculator). Generic fallback: for the beautiful-chat pill this pair is permanently shadowed by the more-specific 'build a modern calculator with standard buttons' entries above (including the non-sequenced repeat-click variant-3 fallback, which keeps this pair dead — so it needs no repeat-click/sequenceIndex treatment FOR THAT PILL); it only fires for hypothetical other prompts containing 'build a modern calculator' but not the more specific 'with standard buttons' phrase (which the specific entries above win), and for those prompts it still emits a single static tool_call_id, so repeated clicks would hit the original id-collision degradation.",
|
||||
"match": {
|
||||
"userMessage": "build a modern calculator",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_open_gen_ui_calc_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":520,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:260px}.display{background:#1e293b;padding:12px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right;font-size:18px;min-height:24px}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}button{padding:10px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px;cursor:pointer}button:hover{background:#475569}.shortcuts{display:grid;grid-template-columns:repeat(3,1fr);gap:6px;margin-top:8px}.shortcuts button{background:#1e3a5f;font-size:11px;padding:8px 4px}.shortcuts button:hover{background:#2563eb}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div><div class=\\\"shortcuts\\\"><button data-val=\\\"1200000\\\">Revenue $1.2M</button><button data-val=\\\"342\\\">Customers 342</button><button data-val=\\\"4.2\\\">Conv 4.2%</button><button data-val=\\\"42000\\\">Electronics $42K</button><button data-val=\\\"28000\\\">Clothing $28K</button><button data-val=\\\"18000\\\">Food $18K</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');function evaluateExpr(){if(!expr||!/^[0-9eE+\\\\-*\\\\/.() ]+$/.test(expr)){display.textContent='err';expr='';return;}try{var value=Function('\\\"use strict\\\";return ('+expr+');')();if(typeof value==='number'&&isFinite(value)){display.textContent=String(value);expr=String(value);}else{display.textContent='err';expr='';}}catch(e){display.textContent='err';expr='';}}document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',function(){if(btn.id==='eq'){evaluateExpr();}else{expr+=btn.textContent;display.textContent=expr;}});});document.querySelectorAll('.shortcuts button').forEach(function(btn){btn.addEventListener('click',function(){var val=btn.getAttribute('data-val');expr+=val;display.textContent=expr;});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read-write — Greet pill. systemMessage array gate (all-present substring match) fires ONLY when every preference is at its INITIAL_PREFERENCES default. (1) 'preferences:\\n- Preferred tone: casual\\n' breaks if name is set (Name line inserts) or tone changes. (2) '- Preferred language: English\\nTailor every response' breaks if language changes or interests are added (Interests line inserts between language and Tailor). Any state change → fixture skips → aimock --provider-gemini proxies to real Gemini for a personalised reply.",
|
||||
"match": {
|
||||
"userMessage": "Say hi and introduce yourself.",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi — I'm your shared-state co-pilot. Your Preferences panel (name, tone, language, interests) is fed to me on every turn, and I jot notes back into the Agent Scratch Pad via set_notes so the UI re-renders. Try setting your name or asking me to remember something."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read-write — Plan-a-weekend pill. Same default-state gate as the Greet pill.",
|
||||
"match": {
|
||||
"userMessage": "Suggest a weekend plan based on my interests.",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "A weekend tailored to your interests panel: if you haven't picked any yet, try Cooking + Travel for a market-and-day-trip combo, or Tech + Books for a maker session and a long reading afternoon. Add interests in the Preferences panel and re-ask for a more specific plan."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Agentic Chat — Write a sonnet pill. Open-ended creative response; deterministic so the headed walkthrough doesn't depend on real-LLM proxy.",
|
||||
"match": {
|
||||
"userMessage": "Write a short sonnet about AI",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "A mind of silicon, a crafted thought,\nThat learns from data, vast and ever-deep,\nWith tireless logic ancient questions fought,\nWhile we, its makers, fragile vigils keep.\n\nIt weaves through code, a future yet unseen,\nA digital dawn where new ideas ignite,\nFrom simple tasks to realms where few have been,\nIt sheds at once both warming and cold light.\n\nWhat wonders shall it bring, what truths reveal,\nOr shadows cast across our shared domain?\nA mirror held, reflecting how we feel —\nA complex hope, a promise, or a pain.\n\nThus from the spark, intelligence takes flight,\nA changing world within its boundless might."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Agentic Chat — Is 17 prime pill. Walk-through style response so the headed walkthrough has deterministic, copy-stable text.",
|
||||
"match": {
|
||||
"userMessage": "whether 17 is prime",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Yes — 17 is prime. A prime number is a positive integer greater than 1 with no divisors other than 1 and itself. To check 17, we only need to test divisors up to its square root (≈4.12), so 2, 3, and 4. 17 ÷ 2 leaves remainder 1, 17 ÷ 3 leaves remainder 2, and 17 ÷ 4 leaves remainder 1. No divisor in that range divides 17 cleanly, so it has no factors besides 1 and 17 — confirming it is prime."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Excalidraw (MCP App) pill. Returns a create_view tool call (the Excalidraw MCP server's tool) with a basic network diagram. Without this fixture aimock falls through to real OpenAI proxy and the pill 502s on the gh-mock key.",
|
||||
"match": {
|
||||
"userMessage": "Use Excalidraw to create a simple network diagram",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_fp_beautiful_chat_excalidraw_001",
|
||||
"name": "create_view",
|
||||
"arguments": "{\"elements\":\"[{\\\"type\\\":\\\"ellipse\\\",\\\"x\\\":200,\\\"y\\\":40,\\\"width\\\":100,\\\"height\\\":60,\\\"strokeColor\\\":\\\"#1971c2\\\",\\\"backgroundColor\\\":\\\"#a5d8ff\\\",\\\"fillStyle\\\":\\\"solid\\\",\\\"text\\\":\\\"Router\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":80,\\\"y\\\":160,\\\"width\\\":100,\\\"height\\\":50,\\\"strokeColor\\\":\\\"#2f9e44\\\",\\\"backgroundColor\\\":\\\"#b2f2bb\\\",\\\"fillStyle\\\":\\\"solid\\\",\\\"text\\\":\\\"Switch A\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":320,\\\"y\\\":160,\\\"width\\\":100,\\\"height\\\":50,\\\"strokeColor\\\":\\\"#2f9e44\\\",\\\"backgroundColor\\\":\\\"#b2f2bb\\\",\\\"fillStyle\\\":\\\"solid\\\",\\\"text\\\":\\\"Switch B\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":20,\\\"y\\\":280,\\\"width\\\":80,\\\"height\\\":40,\\\"text\\\":\\\"PC 1\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":120,\\\"y\\\":280,\\\"width\\\":80,\\\"height\\\":40,\\\"text\\\":\\\"PC 2\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":300,\\\"y\\\":280,\\\"width\\\":80,\\\"height\\\":40,\\\"text\\\":\\\"PC 3\\\"},{\\\"type\\\":\\\"rectangle\\\",\\\"x\\\":400,\\\"y\\\":280,\\\"width\\\":80,\\\"height\\\":40,\\\"text\\\":\\\"PC 4\\\"},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":250,\\\"y\\\":100,\\\"width\\\":-120,\\\"height\\\":60},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":250,\\\"y\\\":100,\\\"width\\\":120,\\\"height\\\":60},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":130,\\\"y\\\":210,\\\"width\\\":-70,\\\"height\\\":70},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":130,\\\"y\\\":210,\\\"width\\\":30,\\\"height\\\":70},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":370,\\\"y\\\":210,\\\"width\\\":-30,\\\"height\\\":70},{\\\"type\\\":\\\"arrow\\\",\\\"x\\\":370,\\\"y\\\":210,\\\"width\\\":70,\\\"height\\\":70}]\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Beautiful Chat — Excalidraw pill follow-up after create_view returns. Matches on userMessage + hasToolResult so it fires regardless of how the MCP runtime wires the tool_call_id back (per-runtime variability across LGP / ADK / Microsoft Agent Framework).",
|
||||
"match": {
|
||||
"userMessage": "Use Excalidraw to create a simple network diagram",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Network diagram drawn above — a router branching to two switches, each connected to two PCs."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-chat — Schedule 1:1 with Alice pill, 2nd turn (book_call tool result returned). Keyed on userMessage + toolCallId so it wins ahead of the bare 'alice' / 'Alice' substring fixtures further below in this file, regardless of prior pill state in the thread.",
|
||||
"match": {
|
||||
"userMessage": "Schedule a 1:1 with Alice next week to review Q2 goals.",
|
||||
"toolCallId": "call_hitl_book_1on1_alice_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked the 1:1 with Alice for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-chat — Schedule 1:1 with Alice pill, 1st turn (emit book_call). Exact pill substring + toolName ensures this fires ONLY for the intended pill, not for arbitrary messages containing 'alice'.",
|
||||
"match": {
|
||||
"userMessage": "Schedule a 1:1 with Alice next week to review Q2 goals.",
|
||||
"toolName": "book_call",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_hitl_book_1on1_alice_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"1:1 with Alice — review Q2 goals\",\"attendee\":\"Alice\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Scoped 'Alice' greeting fixture for the showcase-assistant 'Hi, my name is Alice' / introduction flow. Anchored on the longer phrase so it doesn't substring-match the hitl-in-chat 'Schedule a 1:1 with Alice' pill or other prompts that merely mention Alice.",
|
||||
"match": {
|
||||
"userMessage": "Hi, my name is Alice",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Nice to meet you, Alice! I see you're in Tokyo -- wonderful city. How can I help you today? I can check the weather, look up flights, manage your sales pipeline, or help with other tasks."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: multi-turn test turn 1 — 'My name is Alice.'",
|
||||
"match": {
|
||||
"userMessage": "My name is Alice",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Nice to meet you, Alice! How can I help you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat e2e: multi-turn test turn 2 — 'What name did I just give you?' Response must contain 'Alice'.",
|
||||
"match": {
|
||||
"userMessage": "What name did I just give you",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "You told me your name is Alice."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "readonly-state-agent-context — 'Suggest next steps' pill. Narrowed userMessage to a sentinel substring 'recent activity for the feature-parity probe' that no current pill prompt contains, so this entry no longer shadows the readonly-state.json D6 fixture (which expects the test-asserted leading phrase 'Since you recently viewed the pricing page...'). The original broad userMessage 'Based on my recent activity, what should I try next' was matching the D6 readonly-state pill verbatim and winning by load order (underscore-prefixed _from-feature-parity.json loads before readonly-state.json alphabetically) — first-match-wins meant the test-asserted content never fired. Preserved as a structural placeholder.",
|
||||
"match": {
|
||||
"userMessage": "recent activity for the feature-parity probe",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your recent activity, I'd suggest exploring features you haven't tried yet — check out the documentation for advanced use-cases, or dive into a hands-on tutorial to deepen your understanding."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / a2ui-recovery (A2UI error recovery)",
|
||||
"_note": "Backend-owned A2UI (recovery_agent.py, get_a2ui_tools, injectA2UITool=false) with the toolkit validate->retry recovery loop made visible. NO `context` match field anywhere: a real browser (the dojo) does NOT send the x-aimock-context header (only the test harness does), so context-scoped fixtures would 404 in-browser. The recovery pills' prompts are UNIQUE per framework, so userMessage alone disambiguates across frameworks/demos without needing context. Flow per pill: outer generate_a2ui emit (matched by userMessage; the inner-render fixtures are listed FIRST and carry toolName=render_a2ui so they win for the inner calls, leaving the tool-less outer call to match here); inner render_a2ui (userMessage+toolName+sequenceIndex); outer narration (matched by the unique toolCallId of the emit). HEAL: attempt 1 (seq 0) invalid (dangling child) -> retry -> attempt 2 (seq 1) valid -> paints (declarative-metric x2). EXHAUST: invalid on every attempt -> recovery exhausts -> a2ui_recovery_exhausted. Requires ag-ui-langgraph>=0.0.41.",
|
||||
"created": "2026-06-26"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "HEAL pill - inner render_a2ui attempt 1 (sequenceIndex 0): structurally INVALID (root references a missing child) -> the validate->retry loop rejects it and retries.",
|
||||
"match": {
|
||||
"userMessage": "Build my Q2 revenue summary and self-correct a malformed first attempt.",
|
||||
"toolName": "render_a2ui",
|
||||
"sequenceIndex": 0
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_lp_recover_heal_bad",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\": \"recovery-demo\", \"catalogId\": \"declarative-gen-ui-catalog\", \"components\": [{\"id\": \"root\", \"component\": \"Column\", \"gap\": 16, \"children\": [\"missing-metric\"]}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "HEAL pill - inner render_a2ui attempt 2 (sequenceIndex 1): VALID surface (Column + two Metric tiles) -> recovery succeeds and paints (declarative-metric x2).",
|
||||
"match": {
|
||||
"userMessage": "Build my Q2 revenue summary and self-correct a malformed first attempt.",
|
||||
"toolName": "render_a2ui",
|
||||
"sequenceIndex": 1
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_lp_recover_heal_good",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\": \"recovery-demo\", \"catalogId\": \"declarative-gen-ui-catalog\", \"components\": [{\"id\": \"root\", \"component\": \"Column\", \"gap\": 16, \"children\": [\"m1\", \"m2\"]}, {\"id\": \"m1\", \"component\": \"Metric\", \"label\": \"Quarterly Revenue\", \"value\": \"$4.2M\", \"trend\": \"up\", \"trendValue\": \"+12% QoQ\"}, {\"id\": \"m2\", \"component\": \"Metric\", \"label\": \"Win Rate\", \"value\": \"31%\", \"trend\": \"down\", \"trendValue\": \"-2 pts\"}], \"data\": {}}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "HEAL pill - outer narration after generate_a2ui returns the healed surface (matched by the emit's unique toolCallId).",
|
||||
"match": {
|
||||
"toolCallId": "call_d6_lp_recover_heal_outer_001"
|
||||
},
|
||||
"response": {
|
||||
"content": "The first render came back malformed - I recovered and painted your Q2 revenue summary."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "HEAL pill - outer agent emits generate_a2ui (matched by the unique prompt).",
|
||||
"match": {
|
||||
"userMessage": "Build my Q2 revenue summary and self-correct a malformed first attempt."
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_lp_recover_heal_outer_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{\"intent\": \"create\"}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "EXHAUST pill - inner render_a2ui: structurally INVALID on every attempt (unresolved child) -> recovery exhausts.",
|
||||
"match": {
|
||||
"userMessage": "Build a report that fails every validation pass so I can preview the fallback.",
|
||||
"toolName": "render_a2ui"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_lp_recover_exhaust_design",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\": \"recovery-demo-fail\", \"catalogId\": \"declarative-gen-ui-catalog\", \"components\": [{\"id\": \"root\", \"component\": \"Column\", \"gap\": 16, \"children\": [\"never-defined\"]}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "EXHAUST pill - outer narration after generate_a2ui returns the recovery-exhausted envelope (matched by the emit's unique toolCallId).",
|
||||
"match": {
|
||||
"toolCallId": "call_d6_lp_recover_exhaust_outer_001"
|
||||
},
|
||||
"response": {
|
||||
"content": "I couldn't produce a valid surface after several attempts - showing a graceful fallback instead."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "EXHAUST pill - outer agent emits generate_a2ui (matched by the unique prompt).",
|
||||
"match": {
|
||||
"userMessage": "Build a report that fails every validation pass so I can preview the fallback."
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_lp_recover_exhaust_outer_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{\"intent\": \"create\"}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / agent-config",
|
||||
"sourceFile": "d5-agent-config.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "agent-config tone:professional — value-A for the tone knob pair. Must differ from the tone:casual fixture below.",
|
||||
"match": {
|
||||
"userMessage": "tone:professional — introduce yourself per your config",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Good day. I am your CopilotKit assistant, configured to operate in a professional capacity. I am prepared to assist you with any questions or tasks you may have. How may I be of service today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config tone:casual — value-B for the tone knob pair. Must differ from the tone:professional fixture above.",
|
||||
"match": {
|
||||
"userMessage": "tone:casual — introduce yourself per your config",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hey there! I'm your CopilotKit buddy. I keep things chill and easy-going. What's up — need a hand with anything?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config expertise:beginner — value-A for the expertise knob pair.",
|
||||
"match": {
|
||||
"userMessage": "expertise:beginner — explain how copilotkit works per your config",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "CopilotKit is like a smart helper you can add to your website or app. Think of it as a friendly assistant that lives inside your project and can answer questions, fill out forms, and do tasks for you automatically."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config expertise:expert — value-B for the expertise knob pair. Must differ from the beginner fixture above.",
|
||||
"match": {
|
||||
"userMessage": "expertise:expert — explain how copilotkit works per your config",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "CopilotKit is an open-source AI agent framework with three layers: a React frontend SDK providing hooks (useCopilotChat, useCopilotAction), a runtime middleware (Express/Hono) that manages the AG-UI SSE stream, and agent adapters for LangGraph, CrewAI, and custom backends. The runtime proxies tool calls between the frontend and the agent, handling streaming, state synchronization, and human-in-the-loop interrupts transparently."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config responseLength:concise — value-A for the response-length knob pair. Must be at least 80 chars shorter than the detailed fixture below.",
|
||||
"match": {
|
||||
"userMessage": "responseLength:concise — describe agent context per your config",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Agent context is a per-turn data bag passed from the frontend to the agent via useAgentContext."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agent-config responseLength:detailed — value-B for the response-length knob pair. Must exceed the concise fixture by >= 80 chars.",
|
||||
"match": {
|
||||
"userMessage": "responseLength:detailed — describe agent context per your config",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Agent context is a per-turn data bag passed from the frontend to the agent via the useAgentContext hook. On the React side, you call useAgentContext({ tone, expertise, responseLength }) and CopilotKit serialises those values into the runtime request. The runtime injects them into the agent's system prompt builder, so each turn's LLM call reflects the latest frontend state. This enables real-time personalisation: switching a dropdown from 'professional' to 'casual' immediately changes how the agent responds, without restarting the conversation or reconfiguring the backend. The pattern extends to any key-value pair your app needs — feature flags, user roles, locale, or custom metadata."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / agentic-chat",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "agentic-chat turn 1 \u2014 goldfish name. No turnIndex gate: the probe sends 3 sequential turns in one thread; turnIndex would restrict this to the first turn only, causing turns 2-3 to fall through to proxy.",
|
||||
"match": {
|
||||
"userMessage": "good name for a goldfish",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "How about Bubbles? It is friendly, classic, and easy to call out at the tank. If you want alternatives: Goldie, Finley, or Mango."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat turn 2 \u2014 tank name. No turnIndex gate (multi-turn conversation).",
|
||||
"match": {
|
||||
"userMessage": "name for its tank",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Following the Bubbles theme, you could call the tank The Bubble Bowl. It pairs naturally with the goldfish's name and keeps the playful tone."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "agentic-chat turn 3 \u2014 recall. No turnIndex gate (multi-turn conversation).",
|
||||
"match": {
|
||||
"userMessage": "what we named the goldfish",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "We named the goldfish Bubbles, and the tank The Bubble Bowl."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Write me a haiku about nature",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_generate_haiku_001",
|
||||
"name": "generate_haiku",
|
||||
"arguments": "{\"japanese\":[\"\u53e4\u6c60\u3084\",\"\u86d9\u98db\u3073\u8fbc\u3080\",\"\u6c34\u306e\u97f3\"],\"english\":[\"An old silent pond\",\"A frog jumps into the pond\",\"Splash! Silence again.\"],\"image_name\":\"Mount_Fuji_Lake_Reflection_Cherry_Blossoms_Sakura_Spring.jpg\",\"gradient\":\"linear-gradient(135deg, #f5f7fa 0%, #c3cfe2 100%)\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Write me a haiku about nature",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Haiku generated above \u2014 a beautiful verse about nature with an accompanying image."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The image attachment shows a small abstract test pattern used by the multimodal demo to validate the image-upload pipeline. Successful render of this response confirms the binary attachment round-tripped through the runtime."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The PDF document contains a single test page used by the multimodal demo. Its text was flattened by pypdf on the Python side and forwarded as text content to the model. Receiving this response confirms the document upload path works."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "hi from the popup test",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello from the popup \u2014 the CopilotPopup prebuilt component is wired up and reachable. The launcher floats in the corner and the chat sits in an overlay panel."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "hi from the sidebar test",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello from the sidebar \u2014 the CopilotSidebar prebuilt component is wired up and reachable. Anything else you would like me to confirm from inside the sidebar?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "analyze data and call the tool",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Reasoning step 1: I considered the query. Reasoning step 2: I decided to invoke the analysis tool. The tool returned its structured payload, and the reasoning chain wraps the rendered tool card. Both the reasoning block and the tool card should be visible in the transcript."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / auth",
|
||||
"sourceFile": "d5-auth.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "auth — single turn proving the auth lifecycle works. The probe sends this after sign-in (idiomatic shape) or on the pre-mounted chat (legacy shape). The assertion then clicks sign-out and verifies the auth gate kicks in.",
|
||||
"match": {
|
||||
"userMessage": "auth check turn 1",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Auth confirmed — you are signed in and the CopilotKit runtime accepted your credentials. The chat is live and ready for messages."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / beautiful-chat",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: bar chart of expenses by category",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_bc_bar_chart_001",
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\":\"Expenses by Category\",\"description\":\"Monthly expense breakdown\",\"data\":[{\"label\":\"Rent\",\"value\":15000},{\"label\":\"Salaries\",\"value\":80000},{\"label\":\"Marketing\",\"value\":12000},{\"label\":\"Travel\",\"value\":5000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: bar chart of expenses by category",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Bar chart rendered above — Salaries dominate at $80k, with Rent at $15k as the next-largest category."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: pie chart of revenue distribution by category",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_bc_pie_chart_001",
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Breakdown of revenue across product categories\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: pie chart of revenue distribution by category",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics leads at $42k, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: schedule a 30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_bc_schedule_time_001",
|
||||
"name": "scheduleTime",
|
||||
"arguments": "{\"reasonForScheduling\":\"Learn about CopilotKit\",\"meetingDuration\":30}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: schedule a 30-minute meeting to learn about CopilotKit",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Meeting scheduled — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: toggle the theme",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_bc_toggle_theme_001",
|
||||
"name": "toggleTheme",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: toggle the theme",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Theme toggled."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "beautiful-chat 'Search Flights (A2UI Fixed Schema)' pill — sends 'Find flights from SFO to JFK for next Tuesday.'. Beautiful-chat's search_flights tool takes `flights: list[Flight]` (NOT the {origin,destination} shape used by tool_rendering_agent). Test asserts United Airlines + $349 + Delta + $289 are visible. Narrowed userMessage to 'for next Tuesday' so this stays scoped to the beautiful-chat pill (the tool-rendering basic + reasoning-chain pills both lack that phrase). hasToolResult:false gates this to the initial user turn; the follow-up content fixture below catches iteration 2 via toolCallId.",
|
||||
"match": {
|
||||
"userMessage": "for next Tuesday",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_bc_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA231\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, May 6\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$349\",\"currency\":\"USD\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL412\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, May 6\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"statusColor\":\"#22c55e\",\"price\":\"$289\",\"currency\":\"USD\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d6_bc_search_flights_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above — United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / byoc (declarative-hashbrown)",
|
||||
"sourceFile": "d5-byoc.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "byoc (declarative-hashbrown) — Sales Dashboard pill. The agent returns structured JSON that the BYOC custom renderer parses into metric cards + charts. The response must include enough structure for the demo renderer to materialise [data-testid='metric-card'] and [data-testid='bar-chart'] or [data-testid='pie-chart'].",
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_byoc_dashboard_001",
|
||||
"name": "render_dashboard",
|
||||
"arguments": "{\"metrics\":[{\"label\":\"Total Revenue\",\"value\":\"$2.4M\",\"change\":\"+12%\"},{\"label\":\"New Customers\",\"value\":\"1,847\",\"change\":\"+8%\"},{\"label\":\"Conversion Rate\",\"value\":\"3.2%\",\"change\":\"+0.4%\"}],\"charts\":[{\"type\":\"pie\",\"title\":\"Revenue by Segment\",\"data\":[{\"label\":\"Enterprise\",\"value\":1200000},{\"label\":\"Mid-Market\",\"value\":720000},{\"label\":\"SMB\",\"value\":480000}]},{\"type\":\"bar\",\"title\":\"Monthly Revenue\",\"data\":[{\"label\":\"Oct\",\"value\":780000},{\"label\":\"Nov\",\"value\":810000},{\"label\":\"Dec\",\"value\":810000}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "byoc — follow-up content after render_dashboard tool result returns.",
|
||||
"match": {
|
||||
"userMessage": "Show me a Q4 sales dashboard. Include a total-revenue metric card, a pie chart of revenue by segment, and a bar chart of monthly revenue.",
|
||||
"toolCallId": "call_d6_byoc_dashboard_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Q4 sales dashboard rendered above — total revenue hit $2.4M (+12%), with Enterprise leading segment revenue. Monthly trend shows steady growth across Oct-Dec."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / chat-css",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "verify the css theme rendering",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The chat is themed with hot pink user bubbles and amber assistant bubbles. CSS variables are scoped to .chat-css-demo-scope so the theme does not leak."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "switch theme to dark mode",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — the change_theme tool was invoked and the page is now in dark mode. Frontend tools were registered via useCopilotAction and the agent dispatched the call client-side."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "tone:professional",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Greetings. I am operating in professional tone. I will provide measured, formal responses calibrated to your stated preferences and refrain from colloquialisms in my replies."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "tone:casual",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hey! Casual mode here — I'll keep things friendly and easygoing. Just shoot me whatever you want to know and I'll riff on it without sounding like a press release."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expertise:beginner",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Sure! Think of CopilotKit as a friendly toolkit. It helps you add an AI helper to your app. The helper can answer questions, run small tasks, and show buttons or charts when needed."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "expertise:expert",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "CopilotKit composes a runtime adapter (Express/Hono) over the AG-UI SSE protocol; the React client wires hooks (useFrontendTool, useAgentContext) into a typed agent runner. The architecture front-runs round-trip latency by streaming TEXT_MESSAGE_CHUNK and TOOL_CALL events on the same channel."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "responseLength:concise",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Agent context is a typed payload sent each turn."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "responseLength:detailed",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Agent context is a typed payload published from the frontend on every turn through the useAgentContext hook. The payload is forwarded into the agent's runtime context (LangGraph 0.6 introduced the `context` channel as the supported relay for per-run frontend-supplied data; legacy `properties` flowed via `forwardedProps` and did not land in `RunnableConfig`). On the Python side, CopilotKitMiddleware reads the value off the runtime context, then routes it into the system-prompt builder so the model sees the user's tone, expertise, and length preferences before each call. The result is per-turn behavior change without a model swap."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / chat-slots",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "verify chat slots are wired",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Confirmed — the chat-slots demo is rendering through the custom slot wrappers. The CustomAssistantMessage component should be visible as a tinted card with a 'slot' badge in the corner."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "chat-slots e2e (chat-slots.spec.ts:90 'second assistant turn is also wrapped in the custom slot') — turn 2 prompt after the 'Say hello in one short sentence' first turn. Without this fixture aimock has no match for turn 2 and the second assistant slot never mounts. turnIndex:1 is REQUIRED to avoid stealing headless-simple's turn-0 fun-fact prompt (which resolves to a different deterministic reply in headless-simple.json).",
|
||||
"match": {
|
||||
"userMessage": "Give me a fun fact",
|
||||
"turnIndex": 1,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Sure — sea otters hold hands while they sleep so they don't drift apart on the current."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,128 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / frontend-tools-async",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "auth check turn 1",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Authenticated session is active. The runtime accepted your request because the Authorization header carried the demo bearer token."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "fetch the async metric",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The async tool resolved with the requested metric. The frontend handler awaited completion before forwarding the result back to the agent — async-streaming behavior confirmed."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — project-planning pill, follow-up content keyed on the prior tool's id. Comes BEFORE the tool-emitting fixture so iteration 2 (after the async handler returns) hits this branch instead of re-emitting query_notes. hasToolResult gates were dropped because they broke after the user clicked another tool-using pill earlier in the same thread.",
|
||||
"match": {
|
||||
"userMessage": "Find my notes about project planning",
|
||||
"toolCallId": "call_d5_query_notes_project_planning_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "You have notes on project planning: \"Q2 project planning kickoff\" and \"Project planning retrospective notes\". Let me know if you want a summary of either."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — project-planning pill. 1st turn: query_notes tool call. Specific match wins over the broad 'plan' fixture in feature-parity.json (d5-all.json loads first). turnIndex gate dropped — the multi-pill sequential e2e (frontend-tools-async.spec.ts:149) clicks earlier pills first, leaving prior tool/assistant turns in the thread, so this user message is no longer at turnIndex 0. The toolCallId-keyed follow-up fixture above still wins for iteration 2 (last message is tool with that id); this fixture only fires when last-message.role=user.",
|
||||
"match": {
|
||||
"userMessage": "Find my notes about project planning",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_query_notes_project_planning_001",
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"project planning\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — auth pill, follow-up content keyed on the prior tool's id. Must come BEFORE the tool-emitting fixture.",
|
||||
"match": {
|
||||
"userMessage": "Search my notes for anything related to auth",
|
||||
"toolCallId": "call_d5_query_notes_auth_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "You have one note related to auth: \"Planning: migrate auth to passkeys\" — it covers WebAuthn library options and a fallback for unsupported browsers."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — auth pill. 1st turn: query_notes tool call. Beats the showcase-assistant catch-all in feature-parity.json by virtue of d5-all.json's load order and a longer specific substring match. turnIndex gate dropped — multi-pill sequential e2e leaves prior turns in the thread before this pill is clicked, so the prompt is no longer at turnIndex 0. The toolCallId-keyed follow-up above still wins for iteration 2.",
|
||||
"match": {
|
||||
"userMessage": "Search my notes for anything related to auth",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_query_notes_auth_001",
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"auth\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — reading pill, follow-up content keyed on the prior tool's id. Must come BEFORE the tool-emitting fixture. Locked narration leading phrase per spec test #4 assertion.",
|
||||
"match": {
|
||||
"userMessage": "Do I have any notes tagged reading",
|
||||
"toolCallId": "call_d5_query_notes_reading_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "You have a note titled \"Book recommendations\" that is tagged with \"reading.\" It includes the following books: Thinking Fast and Slow by Daniel Kahneman, The Design of Everyday Things by Don Norman."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools-async — reading pill. 1st turn: query_notes tool call. turnIndex gate dropped — multi-pill sequential e2e leaves prior turns in the thread before this pill is clicked, so the prompt is no longer at turnIndex 0. The toolCallId-keyed follow-up above still wins for iteration 2.",
|
||||
"match": {
|
||||
"userMessage": "Do I have any notes tagged reading",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_query_notes_reading_001",
|
||||
"name": "query_notes",
|
||||
"arguments": "{\"keyword\":\"reading\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "project planning",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "I searched your notes and found a few results.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_notes",
|
||||
"arguments": {
|
||||
"keyword": "project planning"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / frontend-tools",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Make the background a sunset gradient",
|
||||
"toolCallId": "call_d5_change_background_sunset",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — sunset gradient is live."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools 'Sunset' pill — 1st leg: emit change_background tool call only. The toolCallId-keyed follow-up fixture above is ordered first, so it catches the second-leg request (after tool result). This fixture only matches the first-leg request (before tool execution). No turnIndex gate — the 3-pill probe sends sequential turns; turnIndex would restrict to the first turn.",
|
||||
"match": {
|
||||
"userMessage": "Make the background a sunset gradient",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_change_background_sunset",
|
||||
"name": "change_background",
|
||||
"arguments": {
|
||||
"background": "linear-gradient(135deg, #ff7e5f 0%, #feb47b 50%, #ff6b6b 100%)"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "deep green forest gradient",
|
||||
"toolCallId": "call_d5_change_background_forest",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — forest gradient is live."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools 'Forest' pill — 1st leg: emit change_background tool call only. No turnIndex gate — multi-turn probe.",
|
||||
"match": {
|
||||
"userMessage": "deep green forest gradient",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_change_background_forest",
|
||||
"name": "change_background",
|
||||
"arguments": {
|
||||
"background": "linear-gradient(135deg, #0a3d2e 0%, #166534 50%, #059669 100%)"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "navy → magenta cosmic gradient",
|
||||
"toolCallId": "call_d5_change_background_cosmic",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — cosmic gradient is live."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "frontend-tools 'Cosmic' pill — 1st leg: emit change_background tool call only. No turnIndex gate — multi-turn probe.",
|
||||
"match": {
|
||||
"userMessage": "navy → magenta cosmic gradient",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_change_background_cosmic",
|
||||
"name": "change_background",
|
||||
"arguments": {
|
||||
"background": "linear-gradient(135deg, #1e3a8a 0%, #6b21a8 50%, #9333ea 100%)"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,375 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / gen-ui-agent",
|
||||
"sourceFile": "frontend-tools-async.json (extracted)",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_007",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "All three launch steps complete. Goals defined, marketing aligned, post-launch metrics tracked \u2014 ready to ship."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_006",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_007",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"completed\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"completed\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"completed\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_005",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_006",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"completed\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"completed\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"in_progress\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_004",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_005",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"completed\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"completed\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_003",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_004",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"completed\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"in_progress\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_002",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_003",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"completed\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"pending\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_launch_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_002",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"in_progress\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"pending\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Plan a product launch",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_launch_001",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"launch-1\",\"title\":\"Define launch goals and audience\",\"status\":\"pending\"},{\"id\":\"launch-2\",\"title\":\"Coordinate marketing and PR rollout\",\"status\":\"pending\"},{\"id\":\"launch-3\",\"title\":\"Track post-launch metrics for week 1\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_007",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Offsite locked in. Venue booked, agenda set, travel and meals arranged."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_006",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_007",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"completed\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"completed\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"completed\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_005",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_006",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"completed\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"completed\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"in_progress\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_004",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_005",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"completed\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"completed\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_003",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_004",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"completed\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"in_progress\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_002",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_003",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"completed\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"pending\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_offsite_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_002",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"in_progress\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"pending\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "team offsite",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_offsite_001",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"offsite-1\",\"title\":\"Reserve venue near downtown for 30 engineers\",\"status\":\"pending\"},{\"id\":\"offsite-2\",\"title\":\"Build day-by-day agenda with workshop slots\",\"status\":\"pending\"},{\"id\":\"offsite-3\",\"title\":\"Arrange travel, lodging and group meals\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_007",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Competitor brief assembled. Surface mapped, themes summarized, exploitable weaknesses identified."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_006",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_007",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"completed\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"completed\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"completed\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_005",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_006",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"completed\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"completed\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"in_progress\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_004",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_005",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"completed\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"completed\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_003",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_004",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"completed\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"in_progress\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_002",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_003",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"completed\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"pending\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"toolCallId": "call_d5_set_steps_competitor_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_002",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"in_progress\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"pending\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Research our top competitor",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_steps_competitor_001",
|
||||
"name": "set_steps",
|
||||
"arguments": "{\"steps\":[{\"id\":\"comp-1\",\"title\":\"Map competitor product surface and pricing tiers\",\"status\":\"pending\"},{\"id\":\"comp-2\",\"title\":\"Summarize their public differentiation themes\",\"status\":\"pending\"},{\"id\":\"comp-3\",\"title\":\"Identify weaknesses our positioning can exploit\",\"status\":\"pending\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / gen-ui-custom (pie chart path)",
|
||||
"sourceFile": "harness/fixtures/d5/gen-ui-custom.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "gen-ui-custom pie chart — first leg: emit render_pie_chart tool call. hasToolResult:false ensures this only matches when there's no prior tool result in the conversation. Includes narration content (with 'pie chart' tokens) inline so the D5 probe's last-assistant-text assertion passes even when the LGP runtime does not re-invoke the LLM after a frontend tool result returns (the second-leg fixture below remains a safety net for runtimes that DO loop back). chunkSize: 9999 guards against the @ag-ui/langgraph 0.0.34 partial-tool-call JSON.parse class — the render_pie_chart args are large enough to otherwise stream across multiple chunks and trip RUN_ERROR.",
|
||||
"match": {
|
||||
"userMessage": "Show me a pie chart of revenue by category",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_render_pie_chart_001",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category (Q4)\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-custom pie chart — second leg: narration after the pie chart tool result lands. Kept as a safety net when the LGP runtime DOES loop back to the LLM. The follow-up text contains 'pie' and 'chart' tokens — the D5 probe asserts their presence.",
|
||||
"match": {
|
||||
"userMessage": "Show me a pie chart of revenue by category",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,189 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / gen-ui-declarative (A2UI Dynamic Schema)",
|
||||
"_note": "Agent plays a sales analyst for the fictional Vantage Threads company (OSS-136). Two-stage A2UI pattern, mirroring google-adk's gen-ui-declarative.json 1:1: (1) outer agent emits a no-arg `generate_a2ui` toolcall; (2) CopilotKit runtime middleware (`a2ui.injectA2UITool: true`) intercepts the call and drives a secondary LLM with tool_choice forced on `render_a2ui` — that inner fixture (matched by toolName=render_a2ui) emits the flat A2UI components payload; (3) after the tool returns, the outer agent emits a one-sentence narration (matched by toolCallId). Pill userMessage matchers are the full prompt text from declarative-gen-ui/suggestions.ts so future overlapping pills can't be hijacked by a substring collision.",
|
||||
"created": "2026-06-11"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "pill sales-dashboard hero — inner secondary LLM (middleware-injected render_a2ui) emits the flat A2UI components.",
|
||||
"match": {
|
||||
"userMessage": "Show me my sales dashboard for this quarter.",
|
||||
"toolName": "render_a2ui"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_decl_dash_design_001_render",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"sales-dashboard\",\"catalogId\":\"declarative-gen-ui-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"gap\":16,\"children\":[\"kpi-row\",\"charts-row\"]},{\"id\":\"kpi-row\",\"component\":\"Row\",\"gap\":16,\"children\":[\"metric-revenue\",\"metric-new-customers\",\"metric-win-rate\",\"metric-deal-size\"]},{\"id\":\"metric-revenue\",\"component\":\"Metric\",\"label\":\"Quarterly Revenue\",\"value\":\"$4.2M\",\"trend\":\"up\",\"trendValue\":\"+12% QoQ\"},{\"id\":\"metric-new-customers\",\"component\":\"Metric\",\"label\":\"New Customers\",\"value\":\"186\",\"trend\":\"up\",\"trendValue\":\"+8%\"},{\"id\":\"metric-win-rate\",\"component\":\"Metric\",\"label\":\"Win Rate\",\"value\":\"31%\",\"trend\":\"down\",\"trendValue\":\"-2 pts\"},{\"id\":\"metric-deal-size\",\"component\":\"Metric\",\"label\":\"Avg Deal Size\",\"value\":\"$22.6k\",\"trend\":\"up\",\"trendValue\":\"+5%\"},{\"id\":\"charts-row\",\"component\":\"Row\",\"gap\":16,\"children\":[\"region-pie\",\"monthly-bar\"]},{\"id\":\"region-pie\",\"component\":\"PieChart\",\"title\":\"Revenue by Region\",\"description\":\"Quarter revenue split across North America, EMEA, APAC, and LATAM.\",\"data\":[{\"label\":\"North America\",\"value\":1900000},{\"label\":\"EMEA\",\"value\":1300000},{\"label\":\"APAC\",\"value\":720000},{\"label\":\"LATAM\",\"value\":280000}]},{\"id\":\"monthly-bar\",\"component\":\"BarChart\",\"title\":\"Monthly Revenue\",\"description\":\"Revenue trend from Jan through Jun.\",\"data\":[{\"label\":\"Jan\",\"value\":1210000},{\"label\":\"Feb\",\"value\":1340000},{\"label\":\"Mar\",\"value\":1650000},{\"label\":\"Apr\",\"value\":1380000},{\"label\":\"May\",\"value\":1420000},{\"label\":\"Jun\",\"value\":1400000}]}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill sales-dashboard hero — outer agent narration after generate_a2ui returns.",
|
||||
"match": {
|
||||
"context": "langgraph-python",
|
||||
"toolCallId": "call_d6_decl_dash_outer_001"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's your Q2 sales dashboard."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill sales-dashboard hero — outer agent emits generate_a2ui (no args).",
|
||||
"match": {
|
||||
"userMessage": "Show me my sales dashboard for this quarter.",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_decl_dash_outer_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill team-performance — inner secondary LLM (middleware-injected render_a2ui) emits the flat A2UI components.",
|
||||
"match": {
|
||||
"userMessage": "How are our sales reps performing against quota?",
|
||||
"toolName": "render_a2ui"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_decl_team_design_001_render",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"rep-quota-performance\",\"catalogId\":\"declarative-gen-ui-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"gap\":16,\"children\":[\"title\",\"summary\",\"content\"]},{\"id\":\"title\",\"component\":\"Text\",\"text\":\"Sales rep performance vs quota\"},{\"id\":\"summary\",\"component\":\"Text\",\"text\":\"2 of 5 reps are above quota, 1 is near plan, and 2 are below target in Q2.\"},{\"id\":\"content\",\"component\":\"Row\",\"gap\":16,\"children\":[\"table-card\",\"attainment-chart\"]},{\"id\":\"table-card\",\"component\":\"Card\",\"title\":\"Rep attainment table\",\"child\":\"rep-table\"},{\"id\":\"rep-table\",\"component\":\"DataTable\",\"columns\":[{\"key\":\"rep\",\"label\":\"Rep\"},{\"key\":\"attainment\",\"label\":\"Attainment\"},{\"key\":\"pipeline\",\"label\":\"Pipeline\"}],\"rows\":[{\"rep\":\"Dana Whitfield\",\"attainment\":\"124%\",\"pipeline\":\"Leading team; biggest account Meridian Apparel Group has 4 open opps worth $210k\"},{\"rep\":\"Marcus Lee\",\"attainment\":\"108%\",\"pipeline\":\"Above plan\"},{\"rep\":\"Priya Sharma\",\"attainment\":\"97%\",\"pipeline\":\"Near quota\"},{\"rep\":\"Tom Okafor\",\"attainment\":\"88%\",\"pipeline\":\"Below plan\"},{\"rep\":\"Elena Vasquez\",\"attainment\":\"71%\",\"pipeline\":\"Furthest from quota\"}]},{\"id\":\"attainment-chart\",\"component\":\"BarChart\",\"title\":\"Quota attainment by rep\",\"description\":\"Q2 quota attainment percentages for all sales reps.\",\"data\":[{\"label\":\"Dana Whitfield\",\"value\":124},{\"label\":\"Marcus Lee\",\"value\":108},{\"label\":\"Priya Sharma\",\"value\":97},{\"label\":\"Tom Okafor\",\"value\":88},{\"label\":\"Elena Vasquez\",\"value\":71}]}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill team-performance — outer agent narration after generate_a2ui returns.",
|
||||
"match": {
|
||||
"context": "langgraph-python",
|
||||
"toolCallId": "call_d6_decl_team_outer_001"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's how the team is tracking against quota."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill team-performance — outer agent emits generate_a2ui (no args).",
|
||||
"match": {
|
||||
"userMessage": "How are our sales reps performing against quota?",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_decl_team_outer_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill at-risk — inner secondary LLM (middleware-injected render_a2ui) emits the flat A2UI components.",
|
||||
"match": {
|
||||
"userMessage": "Are any accounts or pipeline deals at risk this quarter?",
|
||||
"toolName": "render_a2ui"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_decl_risk_design_001_render",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"risk-dashboard\",\"catalogId\":\"declarative-gen-ui-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Column\",\"gap\":16,\"children\":[\"metrics-row\",\"accounts-row\"]},{\"id\":\"metrics-row\",\"component\":\"Row\",\"gap\":16,\"children\":[\"metric-arr\",\"metric-accounts\",\"metric-biggest\"]},{\"id\":\"metric-arr\",\"component\":\"Metric\",\"label\":\"ARR at risk\",\"value\":\"$615k\",\"trend\":\"down\",\"trendValue\":\"3 accounts\"},{\"id\":\"metric-accounts\",\"component\":\"Metric\",\"label\":\"Accounts at risk\",\"value\":\"3\",\"trend\":\"neutral\",\"trendValue\":\"This quarter\"},{\"id\":\"metric-biggest\",\"component\":\"Metric\",\"label\":\"Biggest exposure\",\"value\":\"Northwind Retail\",\"trend\":\"down\",\"trendValue\":\"$340k renewal\"},{\"id\":\"accounts-row\",\"component\":\"Row\",\"gap\":16,\"children\":[\"card-northwind\",\"card-cascadia\",\"card-atlas\"]},{\"id\":\"card-northwind\",\"component\":\"Card\",\"title\":\"Northwind Retail\",\"subtitle\":\"$340k ARR at stake\",\"child\":\"northwind-content\"},{\"id\":\"northwind-content\",\"component\":\"Column\",\"gap\":8,\"children\":[\"northwind-badge\",\"northwind-text\"]},{\"id\":\"northwind-badge\",\"component\":\"StatusBadge\",\"text\":\"High severity\",\"variant\":\"error\"},{\"id\":\"northwind-text\",\"component\":\"Text\",\"text\":\"No contact in 6 weeks; next action: exec outreach this week to protect the renewal.\"},{\"id\":\"card-cascadia\",\"component\":\"Card\",\"title\":\"Cascadia Outfitters\",\"subtitle\":\"$180k ARR at stake\",\"child\":\"cascadia-content\"},{\"id\":\"cascadia-content\",\"component\":\"Column\",\"gap\":8,\"children\":[\"cascadia-badge\",\"cascadia-text\"]},{\"id\":\"cascadia-badge\",\"component\":\"StatusBadge\",\"text\":\"Medium severity\",\"variant\":\"warning\"},{\"id\":\"cascadia-text\",\"component\":\"Text\",\"text\":\"Champion left; next action: rebuild stakeholder map and secure a new sponsor.\"},{\"id\":\"card-atlas\",\"component\":\"Card\",\"title\":\"Atlas Goods\",\"subtitle\":\"$95k ARR at stake\",\"child\":\"atlas-content\"},{\"id\":\"atlas-content\",\"component\":\"Column\",\"gap\":8,\"children\":[\"atlas-badge\",\"atlas-text\"]},{\"id\":\"atlas-badge\",\"component\":\"StatusBadge\",\"text\":\"Medium severity\",\"variant\":\"warning\"},{\"id\":\"atlas-text\",\"component\":\"Text\",\"text\":\"Legal review is stalled; next action: align procurement and legal on open terms.\"}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill at-risk — outer agent narration after generate_a2ui returns.",
|
||||
"match": {
|
||||
"context": "langgraph-python",
|
||||
"toolCallId": "call_d6_decl_risk_outer_001"
|
||||
},
|
||||
"response": {
|
||||
"content": "Three accounts need attention this quarter."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill at-risk — outer agent emits generate_a2ui (no args).",
|
||||
"match": {
|
||||
"userMessage": "Are any accounts or pipeline deals at risk this quarter?",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_decl_risk_outer_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill top-account — inner secondary LLM (middleware-injected render_a2ui) emits the flat A2UI components.",
|
||||
"match": {
|
||||
"userMessage": "Pull up the details on our biggest account.",
|
||||
"toolName": "render_a2ui"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_decl_acct_design_001_render",
|
||||
"name": "render_a2ui",
|
||||
"arguments": "{\"surfaceId\":\"biggest-account-details\",\"catalogId\":\"declarative-gen-ui-catalog\",\"components\":[{\"id\":\"root\",\"component\":\"Row\",\"gap\":16,\"children\":[\"account-card\",\"revenue-pie\"]},{\"id\":\"account-card\",\"component\":\"Card\",\"title\":\"Meridian Apparel Group\",\"subtitle\":\"Biggest account\",\"child\":\"account-facts\"},{\"id\":\"account-facts\",\"component\":\"Column\",\"gap\":8,\"children\":[\"fact1\",\"fact2\",\"fact3\",\"fact4\",\"fact5\",\"fact6\",\"fact7\"]},{\"id\":\"fact1\",\"component\":\"InfoRow\",\"label\":\"Owner\",\"value\":\"Dana Whitfield\"},{\"id\":\"fact2\",\"component\":\"InfoRow\",\"label\":\"Region\",\"value\":\"North America\"},{\"id\":\"fact3\",\"component\":\"InfoRow\",\"label\":\"ARR\",\"value\":\"$612k\"},{\"id\":\"fact4\",\"component\":\"InfoRow\",\"label\":\"Renewal date\",\"value\":\"Sep 30\"},{\"id\":\"fact5\",\"component\":\"InfoRow\",\"label\":\"Last contact\",\"value\":\"3 days ago\"},{\"id\":\"fact6\",\"component\":\"InfoRow\",\"label\":\"Health\",\"value\":\"Green\"},{\"id\":\"fact7\",\"component\":\"InfoRow\",\"label\":\"Open opportunities\",\"value\":\"4 opportunities worth $210k\"},{\"id\":\"revenue-pie\",\"component\":\"PieChart\",\"title\":\"Revenue by product line\",\"description\":\"Meridian Apparel Group revenue mix across product lines.\",\"data\":[{\"label\":\"Outerwear\",\"value\":260000},{\"label\":\"Footwear\",\"value\":180000},{\"label\":\"Accessories\",\"value\":112000},{\"label\":\"Custom\",\"value\":60000}]}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill top-account — outer agent narration after generate_a2ui returns.",
|
||||
"match": {
|
||||
"context": "langgraph-python",
|
||||
"toolCallId": "call_d6_decl_acct_outer_001"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here's the rundown on Meridian Apparel Group."
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "pill top-account — outer agent emits generate_a2ui (no args).",
|
||||
"match": {
|
||||
"userMessage": "Pull up the details on our biggest account.",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_decl_acct_outer_001",
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / gen-ui-headless-complete",
|
||||
"sourceFile": "headless-complete.json",
|
||||
"copiedFrom": "langgraph-python",
|
||||
"_note": "Alias of the headless-complete demo's 4 gen-UI pills (weather/stock/highlight/revenue) for the gen-ui-headless-complete probe (showcase/harness/src/probes/scripts/d5-gen-ui-headless-complete.ts -> fixtureFile 'gen-ui-headless-complete.json'), which had no matching fixture in any slug so its first leg 503'd under strict. Narration (toolCallId) fixtures FIRST so they win when the last message is a matching tool result; toolcall (userMessage+context) fixtures AFTER with NO turnIndex gate so each of the 4 sequential pill turns in one chat thread matches regardless of prior assistant/tool history (the turnIndex multi-turn match-gate bug). Modeled on the green langgraph-python pattern. Interrupt fixtures intentionally omitted (interrupt-headless is a separate cluster item)."
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "headless-complete weather pill narration — matches when the get_weather tool result with id call_d5_headless_weather_001 is the last message. The 'tokyo' textToken assertion is satisfied by the city name. Narration verbatim matches the headless-complete.spec.ts assertion `toContainText('Tokyo is 22°C and partly cloudy.')`.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_weather_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Tokyo is 22°C and partly cloudy. The headless WeatherCard above shows the sunny snapshot from the tool."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete stock pill narration — matches when the get_stock_price tool result with id call_d5_headless_stock_001 is the last message. textToken 'aapl' satisfied by ticker mention.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_stock_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "AAPL is at $189.42, up 1.27% on the day."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete highlight pill narration — matches when the highlight_note tool result with id call_d5_headless_highlight_001 is the last message.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_highlight_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Highlighted 'ship the demo on Friday' for you above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete revenue chart narration — matches when the get_revenue_chart tool result with id call_d5_headless_revenue_chart_001 is the last message.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_revenue_chart_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the chart of revenue over the last six months — quarterly revenue grew steadily, peaking in June."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete weather pill toolcall — emit get_weather for Tokyo. Probe sends \"What's the weather in Tokyo?\". Narrowed to the full phrase 'What's the weather in Tokyo' so this no longer shadows the D6 tool-rendering 'Chain tools' pill (whose prompt contains 'weather in Tokyo' as a substring) — that pill needs to hit its own 3-tool emission fixture in tool-rendering.json. Only userMessage+context discriminators because narration fixtures above (toolCallId) already win once the tool result lands.",
|
||||
"match": {
|
||||
"userMessage": "What's the weather in Tokyo",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete stock pill toolcall — emit get_stock_price for AAPL. Probe sends \"What's the price of AAPL right now?\". Narrowed to 'price of AAPL right now' so this no longer shadows the D6 tool-rendering 'Stock price' pill which sends \"What's the current price of AAPL?\" (also containing 'price of AAPL' as a substring) — that pill needs its own deterministic-price fixture in tool-rendering.json.",
|
||||
"match": {
|
||||
"userMessage": "price of AAPL right now",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_stock_001",
|
||||
"name": "get_stock_price",
|
||||
"arguments": "{\"ticker\":\"AAPL\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete highlight pill toolcall — emit highlight_note. Probe sends \"Highlight this note for me: 'ship the demo on Friday'.\" so userMessage substring 'ship the demo on Friday' catches it.",
|
||||
"match": {
|
||||
"userMessage": "ship the demo on Friday",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_highlight_001",
|
||||
"name": "highlight_note",
|
||||
"arguments": "{\"text\":\"ship the demo on Friday\",\"color\":\"yellow\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete revenue chart toolcall — emit get_revenue_chart (no args). Probe sends 'Show me a chart of revenue over the last six months.'. Includes a short leading `content` snippet so the word 'revenue' lands in the assistant bubble synchronously with the tool-call emission. Without this, the chart card's title falls back to 'Chart' during the `executing` render phase (result still undefined), and the conversation-runner's count-based settle can fire its 1500ms quiet window before the narration follow-up message arrives — leaving the assistant text without 'revenue' on the probe's first attempt.",
|
||||
"match": {
|
||||
"userMessage": "chart of revenue over the last six months",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pulling the revenue chart for the last six months...",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_revenue_chart_001",
|
||||
"name": "get_revenue_chart",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / gen-ui-interrupt",
|
||||
"sourceFile": "harness/fixtures/d5/gen-ui-interrupt.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "gen-ui-interrupt sales-call pill — first leg: schedule_meeting tool call triggers Python agent's interrupt(...). The toolName gate ensures this matches only when schedule_meeting is in the tool list (the agent must declare it). Placed BEFORE the toolCallId resume-leg so the duplicate-userMessage dedup does not shadow the tool-emitting leg — mirrors the passing mastra/pydantic-ai ordering.",
|
||||
"match": {
|
||||
"userMessage": "intro call with the sales team",
|
||||
"toolName": "schedule_meeting",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Sure — let me check available times.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_schedule_sales_001",
|
||||
"name": "schedule_meeting",
|
||||
"arguments": {
|
||||
"topic": "Sales intro call",
|
||||
"attendee": "Sales team",
|
||||
"slots": [
|
||||
{ "label": "Mon 10:00 AM", "iso": "2026-05-11T10:00:00Z" },
|
||||
{ "label": "Tue 2:00 PM", "iso": "2026-05-12T14:00:00Z" }
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-interrupt sales-call pill — second leg: confirmation after user picks a slot and resolve fires. Keyed by toolCallId so it only matches when the prior tool result is being fed back (last message is the tool result for this call).",
|
||||
"match": {
|
||||
"userMessage": "intro call with the sales team",
|
||||
"toolCallId": "call_d5_schedule_sales_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked: Sales intro call confirmed for the slot you picked."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-interrupt alice-1on1 pill — first leg: schedule_meeting tool call. Placed BEFORE the toolCallId resume-leg so the tool-emitting leg is not shadowed.",
|
||||
"match": {
|
||||
"userMessage": "1:1 with Alice",
|
||||
"toolName": "schedule_meeting",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — pulling up next-week slots.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_schedule_alice_001",
|
||||
"name": "schedule_meeting",
|
||||
"arguments": {
|
||||
"topic": "1:1 with Alice — Q2 goals",
|
||||
"attendee": "Alice",
|
||||
"slots": [
|
||||
{ "label": "Wed 11:00 AM", "iso": "2026-05-13T11:00:00Z" },
|
||||
{ "label": "Thu 3:30 PM", "iso": "2026-05-14T15:30:00Z" }
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-interrupt alice-1on1 pill — second leg: confirmation after user picks a slot. Keyed by toolCallId so it only matches when the prior tool result is being fed back.",
|
||||
"match": {
|
||||
"userMessage": "1:1 with Alice",
|
||||
"toolCallId": "call_d5_schedule_alice_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Scheduled: 1:1 with Alice locked in for the slot you picked."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / gen-ui-open-advanced",
|
||||
"sourceFile": "harness/fixtures/d5/gen-ui-open.json (advanced pills)",
|
||||
"copiedFrom": "langgraph-typescript (canonical sandboxed-UI jsFunctions)",
|
||||
"note": "Sandboxed-UI jsFunctions copied verbatim from langgraph-typescript so the calculator keypad+= and host-ping button wire to the iframe<->host bridge. The 'Inline expression evaluator' pill is NOT duplicated here — it already lives in gen-ui-open.json (same context, identical generateSandboxedUi arguments + its own tool-result follow-up), so this file only supplies the otherwise-missing Calculator and Ping pills.",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "gen-ui-open-advanced: Calculator pill — generateSandboxedUi with jsFunctions wiring for evaluateExpression host callback.",
|
||||
"match": {
|
||||
"userMessage": "Calculator (calls evaluateExpression)",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_calc_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:240px}.display{background:#1e293b;padding:8px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}button{padding:10px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',async function(){if(btn.id==='eq'){var res=await Websandbox.connection.remote.evaluateExpression({expression:expr});if(res&&res.ok){display.textContent=String(res.value);expr=String(res.value);}else{display.textContent='err';expr='';}}else{expr+=btn.textContent;display.textContent=expr;}});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-open-advanced: Ping the host pill — generateSandboxedUi with notifyHost callback.",
|
||||
"match": {
|
||||
"userMessage": "Ping the host (calls notifyHost)",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_ping_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":320,\"placeholderMessages\":[\"Composing host-ping card…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.card{padding:24px;background:#1e293b;border-radius:12px;margin:16px;text-align:center}button{padding:10px 20px;background:#6366f1;border:0;color:#fff;border-radius:8px;font-size:14px;cursor:pointer}.out{margin-top:12px;font-size:12px;color:#94a3b8}\",\"html\":\"<div class=\\\"card\\\" data-testid=\\\"ogui-ping\\\"><h2>Notify the host</h2><button id=\\\"hi\\\">Say hi to the host</button><div class=\\\"out\\\" id=\\\"out\\\">awaiting click…</div></div>\",\"jsFunctions\":\"document.getElementById('hi').addEventListener('click',async function(){var out=document.getElementById('out');out.textContent='sending…';var res=await Websandbox.connection.remote.notifyHost({message:'Hello from sandbox'});out.textContent=res&&res.ok?'host replied at '+res.receivedAt:'failed';});\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-open-advanced narration fallback.",
|
||||
"match": {
|
||||
"userMessage": "continue the advanced gen-ui flow",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The advanced gen-UI flow continued with the second-step component. The chained payloads from turns 1 and 2 form the complete advanced gen-UI sequence."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / gen-ui-open + gen-ui-open-advanced",
|
||||
"sourceFile": "d5-gen-ui-open.ts + d5-gen-ui-open-advanced.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "gen-ui-open — 3D axis visualization pill. The agent calls generateSandboxedUi which renders an iframe[srcdoc] with a non-trivial HTML payload (>= 100 chars). The probe asserts iframe presence + srcdoc length.",
|
||||
"match": {
|
||||
"userMessage": "3D axis visualization (model airplane)",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_open_genui_3d_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":400,\"placeholderMessages\":[\"Rendering 3D axis visualization…\"],\"css\":\"body{margin:0;background:#0f172a;display:flex;justify-content:center;align-items:center;height:100vh;font-family:system-ui}canvas{border-radius:8px;box-shadow:0 4px 24px rgba(0,0,0,.4)}#info{position:absolute;top:12px;left:12px;color:#94a3b8;font-size:12px}\",\"html\":\"<div id=\\\"info\\\">3D Axis — Model Airplane</div><canvas id=\\\"c\\\" width=\\\"400\\\" height=\\\"300\\\"></canvas>\",\"jsFunctions\":\"(function(){var c=document.getElementById('c');var ctx=c.getContext('2d');var cx=200,cy=150;function draw(t){ctx.clearRect(0,0,400,300);ctx.strokeStyle='#ef4444';ctx.beginPath();ctx.moveTo(cx,cy);ctx.lineTo(cx+Math.cos(t)*80,cy-Math.sin(t)*40);ctx.stroke();ctx.strokeStyle='#22c55e';ctx.beginPath();ctx.moveTo(cx,cy);ctx.lineTo(cx,cy-80);ctx.stroke();ctx.strokeStyle='#3b82f6';ctx.beginPath();ctx.moveTo(cx,cy);ctx.lineTo(cx-Math.sin(t)*60,cy-Math.cos(t)*30);ctx.stroke();ctx.fillStyle='#e2e8f0';ctx.font='10px system-ui';ctx.fillText('X',cx+Math.cos(t)*85,cy-Math.sin(t)*40);ctx.fillText('Y',cx-6,cy-85);ctx.fillText('Z',cx-Math.sin(t)*65,cy-Math.cos(t)*30);requestAnimationFrame(function(){draw(t+0.01);});}draw(0);})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-open — follow-up content after generateSandboxedUi returns.",
|
||||
"match": {
|
||||
"userMessage": "3D axis visualization (model airplane)",
|
||||
"toolCallId": "call_d6_open_genui_3d_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "3D axis visualization rendered above — the canvas shows rotating X (red), Y (green), and Z (blue) axes representing the model airplane's orientation."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-open-advanced — Inline expression evaluator pill. The agent calls generateSandboxedUi with sandbox-functions, producing a sandboxed iframe with allow-scripts. The probe asserts iframe[sandbox*='allow-scripts'] is present.",
|
||||
"match": {
|
||||
"userMessage": "Inline expression evaluator",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_open_genui_advanced_eval_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":320,\"placeholderMessages\":[\"Composing expression evaluator…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.card{padding:16px;background:#1e293b;border-radius:12px;margin:16px}input{width:100%;padding:8px;background:#0f172a;border:1px solid #334155;color:#e2e8f0;border-radius:6px;box-sizing:border-box}button{margin-top:8px;padding:8px 16px;background:#6366f1;border:0;color:#fff;border-radius:6px;cursor:pointer}.out{margin-top:8px;font-size:12px;color:#94a3b8}\",\"html\":\"<div class=\\\"card\\\" data-testid=\\\"ogui-inline-eval\\\"><h2>Inline expression evaluator</h2><input id=\\\"in\\\" placeholder=\\\"e.g. 2 + 2\\\"/><button id=\\\"go\\\">Evaluate</button><div class=\\\"out\\\" id=\\\"out\\\">awaiting input…</div></div>\",\"jsFunctions\":\"(function(){var input=document.getElementById('in');var out=document.getElementById('out');var go=document.getElementById('go');async function run(){var expr=input.value;out.textContent='evaluating…';var res=await Websandbox.connection.remote.evaluateExpression({expression:expr});out.textContent=res&&res.ok?'= '+res.value:'error: '+res.error;}go.addEventListener('click',run);input.addEventListener('keydown',function(e){if(e.key==='Enter')run();});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "gen-ui-open-advanced — follow-up content after generateSandboxedUi returns.",
|
||||
"match": {
|
||||
"userMessage": "Inline expression evaluator",
|
||||
"toolCallId": "call_d6_open_genui_advanced_eval_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Expression evaluator rendered — type any math expression and it evaluates in real time via the sandbox bridge."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,231 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / gen-ui-tool-based",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "3D axis visualization (model airplane)",
|
||||
"toolCallId": "call_d5_open_gen_ui_3d_axis_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "3D axis visualization (model airplane)",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_3d_axis_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing 3D axis scene…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px}h1{font-size:14px;margin:0 0 8px}svg{display:block;background:#1e293b;border-radius:8px}\",\"html\":\"<div class=\\\"wrap\\\"><h1>3D axis visualization (pitch / yaw / roll)</h1><svg width=\\\"320\\\" height=\\\"260\\\" viewBox=\\\"-80 -80 160 160\\\" data-testid=\\\"ogui-3d-axis\\\"><line x1=\\\"-60\\\" y1=\\\"0\\\" x2=\\\"60\\\" y2=\\\"0\\\" stroke=\\\"#f59e0b\\\"/><line x1=\\\"0\\\" y1=\\\"-60\\\" x2=\\\"0\\\" y2=\\\"60\\\" stroke=\\\"#6366f1\\\"/><line x1=\\\"-40\\\" y1=\\\"40\\\" x2=\\\"40\\\" y2=\\\"-40\\\" stroke=\\\"#10b981\\\"/><text x=\\\"62\\\" y=\\\"4\\\" font-size=\\\"8\\\" fill=\\\"#f59e0b\\\">X pitch</text><text x=\\\"4\\\" y=\\\"-62\\\" font-size=\\\"8\\\" fill=\\\"#6366f1\\\">Y yaw</text><text x=\\\"42\\\" y=\\\"-42\\\" font-size=\\\"8\\\" fill=\\\"#10b981\\\">Z roll</text></svg></div>\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "How a neural network works",
|
||||
"toolCallId": "call_d5_open_gen_ui_neural_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "How a neural network works",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_neural_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing neural network forward pass…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px}h1{font-size:14px;margin:0 0 8px}svg{display:block;background:#1e293b;border-radius:8px}circle{fill:#6366f1}\",\"html\":\"<div class=\\\"wrap\\\"><h1>Forward pass: input → hidden → output</h1><svg width=\\\"320\\\" height=\\\"220\\\" data-testid=\\\"ogui-neural-net\\\"><g><circle cx=\\\"40\\\" cy=\\\"40\\\" r=\\\"8\\\"/><circle cx=\\\"40\\\" cy=\\\"90\\\" r=\\\"8\\\"/><circle cx=\\\"40\\\" cy=\\\"140\\\" r=\\\"8\\\"/><circle cx=\\\"40\\\" cy=\\\"190\\\" r=\\\"8\\\"/></g><g fill=\\\"#a78bfa\\\"><circle cx=\\\"160\\\" cy=\\\"30\\\" r=\\\"8\\\"/><circle cx=\\\"160\\\" cy=\\\"75\\\" r=\\\"8\\\"/><circle cx=\\\"160\\\" cy=\\\"115\\\" r=\\\"8\\\"/><circle cx=\\\"160\\\" cy=\\\"155\\\" r=\\\"8\\\"/><circle cx=\\\"160\\\" cy=\\\"195\\\" r=\\\"8\\\"/></g><g fill=\\\"#10b981\\\"><circle cx=\\\"280\\\" cy=\\\"80\\\" r=\\\"8\\\"/><circle cx=\\\"280\\\" cy=\\\"140\\\" r=\\\"8\\\"/></g></svg></div>\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Quicksort visualization",
|
||||
"toolCallId": "call_d5_open_gen_ui_quicksort_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Quicksort visualization",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_quicksort_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing quicksort animation…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px}h1{font-size:14px;margin:0 0 8px}svg{display:block;background:#1e293b;border-radius:8px}rect{fill:#64748b}\",\"html\":\"<div class=\\\"wrap\\\"><h1>Quicksort: partition around pivot</h1><svg width=\\\"320\\\" height=\\\"220\\\" data-testid=\\\"ogui-quicksort\\\"><rect x=\\\"10\\\" y=\\\"170\\\" width=\\\"24\\\" height=\\\"40\\\"/><rect x=\\\"40\\\" y=\\\"130\\\" width=\\\"24\\\" height=\\\"80\\\"/><rect x=\\\"70\\\" y=\\\"100\\\" width=\\\"24\\\" height=\\\"110\\\"/><rect x=\\\"100\\\" y=\\\"60\\\" width=\\\"24\\\" height=\\\"150\\\" fill=\\\"#f59e0b\\\"/><rect x=\\\"130\\\" y=\\\"80\\\" width=\\\"24\\\" height=\\\"130\\\" fill=\\\"#6366f1\\\"/><rect x=\\\"160\\\" y=\\\"110\\\" width=\\\"24\\\" height=\\\"100\\\"/><rect x=\\\"190\\\" y=\\\"50\\\" width=\\\"24\\\" height=\\\"160\\\"/><rect x=\\\"220\\\" y=\\\"90\\\" width=\\\"24\\\" height=\\\"120\\\"/><rect x=\\\"250\\\" y=\\\"140\\\" width=\\\"24\\\" height=\\\"70\\\"/><rect x=\\\"280\\\" y=\\\"160\\\" width=\\\"24\\\" height=\\\"50\\\"/></svg></div>\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Fourier: square wave from sines",
|
||||
"toolCallId": "call_d5_open_gen_ui_fourier_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Fourier: square wave from sines",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_fourier_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing Fourier series animation…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px}h1{font-size:14px;margin:0 0 8px}svg{display:block;background:#1e293b;border-radius:8px}\",\"html\":\"<div class=\\\"wrap\\\"><h1>Fourier series: square wave</h1><svg width=\\\"320\\\" height=\\\"220\\\" viewBox=\\\"0 -60 320 120\\\" data-testid=\\\"ogui-fourier\\\"><circle cx=\\\"60\\\" cy=\\\"0\\\" r=\\\"40\\\" stroke=\\\"#6366f1\\\" fill=\\\"none\\\"/><circle cx=\\\"60\\\" cy=\\\"0\\\" r=\\\"13\\\" stroke=\\\"#a78bfa\\\" fill=\\\"none\\\"/><circle cx=\\\"60\\\" cy=\\\"0\\\" r=\\\"8\\\" stroke=\\\"#c4b5fd\\\" fill=\\\"none\\\"/><path d=\\\"M120 0 L300 0\\\" stroke=\\\"#10b981\\\" fill=\\\"none\\\"/></svg></div>\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Calculator (calls evaluateExpression)",
|
||||
"toolCallId": "call_d5_open_gen_ui_calc_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Calculator (calls evaluateExpression)",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_calc_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":480,\"placeholderMessages\":[\"Composing calculator UI…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.wrap{padding:16px;max-width:240px}.display{background:#1e293b;padding:8px;border-radius:8px;margin-bottom:8px;font-family:monospace;text-align:right}.grid{display:grid;grid-template-columns:repeat(4,1fr);gap:6px}button{padding:10px;background:#334155;border:0;color:#e2e8f0;border-radius:6px;font-size:14px}\",\"html\":\"<div class=\\\"wrap\\\" data-testid=\\\"ogui-calculator\\\"><div class=\\\"display\\\" id=\\\"d\\\">0</div><div class=\\\"grid\\\"><button>7</button><button>8</button><button>9</button><button>+</button><button>4</button><button>5</button><button>6</button><button>-</button><button>1</button><button>2</button><button>3</button><button>*</button><button>0</button><button>.</button><button id=\\\"eq\\\">=</button><button>/</button></div></div>\",\"jsFunctions\":\"(function(){var expr='';var display=document.getElementById('d');document.querySelectorAll('.grid button').forEach(function(btn){btn.addEventListener('click',async function(){if(btn.id==='eq'){var res=await Websandbox.connection.remote.evaluateExpression({expression:expr});if(res&&res.ok){display.textContent=String(res.value);expr=String(res.value);}else{display.textContent='err';expr='';}}else{expr+=btn.textContent;display.textContent=expr;}});});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Ping the host (calls notifyHost)",
|
||||
"toolCallId": "call_d5_open_gen_ui_ping_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Ping the host (calls notifyHost)",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_ping_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":320,\"placeholderMessages\":[\"Composing host-ping card…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.card{padding:24px;background:#1e293b;border-radius:12px;margin:16px;text-align:center}button{padding:10px 20px;background:#6366f1;border:0;color:#fff;border-radius:8px;font-size:14px;cursor:pointer}.out{margin-top:12px;font-size:12px;color:#94a3b8}\",\"html\":\"<div class=\\\"card\\\" data-testid=\\\"ogui-ping\\\"><h2>Notify the host</h2><button id=\\\"hi\\\">Say hi to the host</button><div class=\\\"out\\\" id=\\\"out\\\">awaiting click…</div></div>\",\"jsFunctions\":\"document.getElementById('hi').addEventListener('click',async function(){var out=document.getElementById('out');out.textContent='sending…';var res=await Websandbox.connection.remote.notifyHost({message:'Hello from sandbox'});out.textContent=res&&res.ok?'host replied at '+res.receivedAt:'failed';});\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Inline expression evaluator",
|
||||
"toolCallId": "call_d5_open_gen_ui_inline_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Generated. The sandboxed UI is rendered above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Inline expression evaluator",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_open_gen_ui_inline_001",
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":320,\"placeholderMessages\":[\"Composing expression evaluator…\"],\"css\":\"body{margin:0;font-family:system-ui;background:#0f172a;color:#e2e8f0}.card{padding:16px;background:#1e293b;border-radius:12px;margin:16px}input{width:100%;padding:8px;background:#0f172a;border:1px solid #334155;color:#e2e8f0;border-radius:6px;box-sizing:border-box}button{margin-top:8px;padding:8px 16px;background:#6366f1;border:0;color:#fff;border-radius:6px;cursor:pointer}.out{margin-top:8px;font-size:12px;color:#94a3b8}\",\"html\":\"<div class=\\\"card\\\" data-testid=\\\"ogui-inline-eval\\\"><h2>Inline expression evaluator</h2><input id=\\\"in\\\" placeholder=\\\"e.g. 2 + 2\\\"/><button id=\\\"go\\\">Evaluate</button><div class=\\\"out\\\" id=\\\"out\\\">awaiting input…</div></div>\",\"jsFunctions\":\"(function(){var input=document.getElementById('in');var out=document.getElementById('out');var go=document.getElementById('go');async function run(){var expr=input.value;out.textContent='evaluating…';var res=await Websandbox.connection.remote.evaluateExpression({expression:expr});out.textContent=res&&res.ok?'= '+res.value:'error: '+res.error;}go.addEventListener('click',run);input.addEventListener('keydown',function(e){if(e.key==='Enter')run();});})();\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "request the gen-ui interrupt",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The agent paused at a gen-UI interrupt and rendered a choice component for the user. Choose to continue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "confirm the gen-ui choice",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Gen-UI interrupt resolved. The agent received the user's choice and resumed, completing the workflow."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "render an open gen-ui element",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The open gen-UI element was rendered. The LLM produced an arbitrary-shape JSON payload and the renderer materialized it as a UI block."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "continue the advanced gen-ui flow",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The advanced gen-UI flow continued with the second-step component. The chained payloads from turns 1 and 2 form the complete advanced gen-UI sequence."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,135 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / headless-complete",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21",
|
||||
"_note": "Narration fixtures (toolCallId) listed FIRST so they win when last message is a matching tool result. Toolcall fixtures listed AFTER and use only userMessage+context so they match on the initial user turn regardless of prior assistant/tool history."
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "headless-complete weather pill narration — matches when the get_weather tool result with id call_d5_headless_weather_001 is the last message. The 'tokyo' textToken assertion is satisfied by the city name. Narration verbatim matches the headless-complete.spec.ts assertion `toContainText('Tokyo is 22°C and partly cloudy.')`.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_weather_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Tokyo is 22°C and partly cloudy. The headless WeatherCard above shows the sunny snapshot from the tool."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete stock pill narration — matches when the get_stock_price tool result with id call_d5_headless_stock_001 is the last message. textToken 'aapl' satisfied by ticker mention.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_stock_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "AAPL is at $189.42, up 1.27% on the day."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete highlight pill narration — matches when the highlight_note tool result with id call_d5_headless_highlight_001 is the last message.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_highlight_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Highlighted 'ship the demo on Friday' for you above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete revenue chart narration — matches when the get_revenue_chart tool result with id call_d5_headless_revenue_chart_001 is the last message.",
|
||||
"match": {
|
||||
"toolCallId": "call_d5_headless_revenue_chart_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the chart of revenue over the last six months — quarterly revenue grew steadily, peaking in June."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete weather pill toolcall — emit get_weather for Tokyo. Probe sends \"What's the weather in Tokyo?\". Narrowed to the full phrase 'What's the weather in Tokyo' so this no longer shadows the D6 tool-rendering 'Chain tools' pill (whose prompt contains 'weather in Tokyo' as a substring) — that pill needs to hit its own 3-tool emission fixture in tool-rendering.json. Only userMessage+context discriminators because narration fixtures above (toolCallId) already win once the tool result lands.",
|
||||
"match": {
|
||||
"userMessage": "What's the weather in Tokyo",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete stock pill toolcall — emit get_stock_price for AAPL. Probe sends \"What's the price of AAPL right now?\". Narrowed to 'price of AAPL right now' so this no longer shadows the D6 tool-rendering 'Stock price' pill which sends \"What's the current price of AAPL?\" (also containing 'price of AAPL' as a substring) — that pill needs its own deterministic-price fixture in tool-rendering.json.",
|
||||
"match": {
|
||||
"userMessage": "price of AAPL right now",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_stock_001",
|
||||
"name": "get_stock_price",
|
||||
"arguments": "{\"ticker\":\"AAPL\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete highlight pill toolcall — emit highlight_note. Probe sends \"Highlight this note for me: 'ship the demo on Friday'.\" so userMessage substring 'ship the demo on Friday' catches it.",
|
||||
"match": {
|
||||
"userMessage": "ship the demo on Friday",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_highlight_001",
|
||||
"name": "highlight_note",
|
||||
"arguments": "{\"text\":\"ship the demo on Friday\",\"color\":\"yellow\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-complete revenue chart toolcall — emit get_revenue_chart (no args). Probe sends 'Show me a chart of revenue over the last six months.'. Includes a short leading `content` snippet so the word 'revenue' lands in the assistant bubble synchronously with the tool-call emission. Without this, the chart card's title falls back to 'Chart' during the `executing` render phase (result still undefined), and the conversation-runner's count-based settle can fire its 1500ms quiet window before the narration follow-up message arrives — leaving the assistant text without 'revenue' on the probe's first attempt.",
|
||||
"match": {
|
||||
"userMessage": "chart of revenue over the last six months",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pulling the revenue chart for the last six months...",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_headless_revenue_chart_001",
|
||||
"name": "get_revenue_chart",
|
||||
"arguments": "{}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "trigger the headless interrupt",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Interrupt raised. The agent has paused at a graph-level interrupt and is waiting for user input. Reply with 'yes' or 'no' to resume."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "resolve the interrupt with yes",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Interrupt resolved with 'yes'. The agent resumed execution from the suspended node and produced its final answer."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / headless-simple",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "headless-simple pill 1: locks the deterministic greeting leading phrase. The reply is intentionally NON-boilerplate (i.e. NOT the showcase-assistant catch-all 'I can help you with weather lookups...') so the headless-simple spec's HELLO_LEADING assertion fails when fixture priority misroutes this prompt to the catch-all.",
|
||||
"match": {
|
||||
"userMessage": "Say hello in one short sentence",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi! In one short sentence: I'm a CopilotKit demo agent here to help you try features."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-simple pill 2: locks the deterministic joke.",
|
||||
"match": {
|
||||
"userMessage": "Tell me a one-line joke",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Why did the scarecrow win an award? Because he was outstanding in his field!"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "headless-simple pill 3: locks the deterministic fun fact.",
|
||||
"match": {
|
||||
"userMessage": "Give me a fun fact",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "A fun fact: Honey never spoils! Archaeologists have found pots of honey in ancient Egyptian tombs that are over 3,000 years old and still perfectly edible."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,119 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / hitl-in-app",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Issue a $50 refund to customer #12345",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_request_approval_001",
|
||||
"name": "request_user_approval",
|
||||
"arguments": "{\"message\":\"Issue a $50 refund to customer #12345.\",\"context\":\"Per the standard goodwill-credit policy for shipping delays.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Issue a $50 refund to customer #12345",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Approved — processing the $50 refund to customer #12345 now."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — refund pill (#12345), post-tool-result response. Content includes BOTH approve and reject key phrases so both test assertions can match (the test checks for a substring, not the full string). This avoids sequenceIndex which is fragile across aimock restarts.",
|
||||
"match": {
|
||||
"userMessage": "$50 refund to Jordan Rivera on ticket #12345",
|
||||
"toolCallId": "call_d5_hitl_refund_12345_001",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "I am processing the $50 refund to Jordan Rivera on ticket #12345 now. The refund request was not approved by default — your explicit approval or rejection determines the outcome."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — refund pill (#12345). 1st turn: emit request_user_approval tool call. The post-tool-result fixture above (with toolCallId + hasToolResult: true) is more specific and wins on the 2nd turn, so hasToolResult: false is NOT needed here — omitting it allows this fixture to match even in multi-pill conversations where prior tool results exist.",
|
||||
"match": {
|
||||
"userMessage": "$50 refund to Jordan Rivera on ticket #12345",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_hitl_refund_12345_001",
|
||||
"name": "request_user_approval",
|
||||
"arguments": "{\"message\":\"Issue a $50 refund to Jordan Rivera on ticket #12345 for the duplicate charge.\",\"context\":\"Ticket #12345 — duplicate-charge goodwill credit.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — downgrade pill (#12346), 2nd turn. Same pattern as refund/escalate. Without this entry, the generic 'plan' catch-all in feature-parity.json hijacks the downgrade message.",
|
||||
"match": {
|
||||
"userMessage": "downgrade Priya Shah (#12346) to the Starter plan",
|
||||
"toolCallId": "call_d5_hitl_downgrade_12346_001",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Downgrade confirmed — Priya Shah (#12346) will move to the Starter plan effective next billing cycle."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — downgrade pill (#12346). 1st turn: emit request_user_approval tool call. Specific userMessage substring takes precedence over the bare 'plan' fixture in feature-parity.json (which lives later in the load order). hasToolResult: false omitted so multi-pill conversations still match.",
|
||||
"match": {
|
||||
"userMessage": "downgrade Priya Shah (#12346) to the Starter plan",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_hitl_downgrade_12346_001",
|
||||
"name": "request_user_approval",
|
||||
"arguments": "{\"message\":\"Downgrade Priya Shah (#12346) to the Starter plan effective next billing cycle.\",\"context\":\"Ticket #12346 — voluntary downgrade per customer request.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — escalate pill (#12347), post-tool-result response. Content includes both approve and reject key phrases.",
|
||||
"match": {
|
||||
"userMessage": "escalate ticket #12347 to the payments team",
|
||||
"toolCallId": "call_d5_hitl_escalate_12347_001",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Escalated ticket #12347 to the payments team for Morgan Lee. Not escalated by default — your explicit approval determines the outcome."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "hitl-in-app — escalate pill (#12347). 1st turn: emit request_user_approval tool call. hasToolResult: false omitted so multi-pill conversations still match (the post-tool-result fixture above with toolCallId is more specific and wins on the 2nd turn).",
|
||||
"match": {
|
||||
"userMessage": "escalate ticket #12347 to the payments team",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_hitl_escalate_12347_001",
|
||||
"name": "request_user_approval",
|
||||
"arguments": "{\"message\":\"Escalate ticket #12347 to the payments team for Morgan Lee's stuck payment.\",\"context\":\"Ticket #12347 — payment stuck, needs payments-team triage.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,115 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / hitl-in-chat",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Book a 30-minute onboarding call for Alice",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_book_call_001",
|
||||
"name": "book_call",
|
||||
"arguments": "{\"topic\":\"Onboarding call\",\"attendee\":\"Alice\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Book a 30-minute onboarding call for Alice",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked Alice's onboarding call for the time you selected — calendar invite is on its way."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "intro call with the sales team",
|
||||
"toolCallId": "call_d5_schedule_sales_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked: Sales intro call confirmed for the slot you picked."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "intro call with the sales team",
|
||||
"toolName": "schedule_meeting",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "schedule_meeting",
|
||||
"arguments": {
|
||||
"topic": "Sales intro call",
|
||||
"attendee": "Sales team",
|
||||
"slots": [
|
||||
{
|
||||
"label": "Mon 10:00 AM",
|
||||
"iso": "2026-05-11T10:00:00Z"
|
||||
},
|
||||
{
|
||||
"label": "Tue 2:00 PM",
|
||||
"iso": "2026-05-12T14:00:00Z"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": "call_d5_schedule_sales_001"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "1:1 with Alice",
|
||||
"toolCallId": "call_d5_schedule_alice_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Scheduled: 1:1 with Alice locked in for the slot you picked."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "1:1 with Alice",
|
||||
"toolName": "schedule_meeting",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "schedule_meeting",
|
||||
"arguments": {
|
||||
"topic": "1:1 with Alice — Q2 goals",
|
||||
"attendee": "Alice",
|
||||
"slots": [
|
||||
{
|
||||
"label": "Wed 11:00 AM",
|
||||
"iso": "2026-05-13T11:00:00Z"
|
||||
},
|
||||
{
|
||||
"label": "Thu 3:30 PM",
|
||||
"iso": "2026-05-14T15:30:00Z"
|
||||
}
|
||||
]
|
||||
},
|
||||
"id": "call_d5_schedule_alice_001"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / interrupt-headless",
|
||||
"sourceFile": "harness/fixtures/d5/interrupt-headless.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "interrupt-headless sales-call pill — second leg: confirmation after user picks a slot. Placed BEFORE first-leg so toolCallId matching takes precedence over toolName matching.",
|
||||
"match": {
|
||||
"userMessage": "intro call with the sales team",
|
||||
"toolCallId": "call_d5_schedule_sales_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Booked: Sales intro call confirmed for the slot you picked."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "interrupt-headless alice-1on1 pill — second leg: confirmation after user picks a slot. Placed BEFORE first-leg so toolCallId matching takes precedence.",
|
||||
"match": {
|
||||
"userMessage": "1:1 with Alice",
|
||||
"toolCallId": "call_d5_schedule_alice_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Scheduled: 1:1 with Alice locked in for the slot you picked."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "interrupt-headless sales-call pill — first leg: schedule_meeting tool call triggers interrupt(...). Same agent/prompts as gen-ui-interrupt but different frontend (useHeadlessInterrupt renders in app surface pane, not inline).",
|
||||
"match": {
|
||||
"userMessage": "intro call with the sales team",
|
||||
"toolName": "schedule_meeting",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Sure — let me check available times.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_schedule_sales_001",
|
||||
"name": "schedule_meeting",
|
||||
"arguments": {
|
||||
"topic": "Sales intro call",
|
||||
"attendee": "Sales team",
|
||||
"slots": [
|
||||
{ "label": "Mon 10:00 AM", "iso": "2026-05-11T10:00:00Z" },
|
||||
{ "label": "Tue 2:00 PM", "iso": "2026-05-12T14:00:00Z" }
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "interrupt-headless alice-1on1 pill — first leg: schedule_meeting tool call.",
|
||||
"match": {
|
||||
"userMessage": "1:1 with Alice",
|
||||
"toolName": "schedule_meeting",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — pulling up next-week slots.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_schedule_alice_001",
|
||||
"name": "schedule_meeting",
|
||||
"arguments": {
|
||||
"topic": "1:1 with Alice — Q2 goals",
|
||||
"attendee": "Alice",
|
||||
"slots": [
|
||||
{ "label": "Wed 11:00 AM", "iso": "2026-05-13T11:00:00Z" },
|
||||
{ "label": "Thu 3:30 PM", "iso": "2026-05-14T15:30:00Z" }
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / mcp-apps",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Open Excalidraw and sketch a system diagram",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Sketched a three-box system diagram (Client → Server → Database). Tap the iframe to open it in Excalidraw."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D5 mcp-apps suggestion pill #1 (\"Draw a flowchart\"). Sends the verbatim suggestion message \"Use Excalidraw to draw a simple flowchart with three steps.\" The distinctive substring \"draw a simple flowchart\" prevents collision with feature-parity.json's generic `{userMessage: \"steps\"}` fixture, which would otherwise absorb the prompt and return a tool-call-free planning blurb (breaking the iframe). Turn 1 emits `create_view` with three-step flowchart elements so the MCP middleware fetches the Excalidraw UI resource and mounts the iframe; turn 2 (after the tool result) emits the deterministic narration.",
|
||||
"match": {
|
||||
"userMessage": "draw a simple flowchart",
|
||||
"toolName": "create_view",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "The user wants a simple three-step flowchart. I'll call create_view once with three labelled rectangles connected by arrows and a title, framed by a cameraUpdate.",
|
||||
"content": "Drawing a simple three-step flowchart in Excalidraw.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_mcp_apps_create_view_flowchart_001",
|
||||
"name": "create_view",
|
||||
"arguments": "{\"elements\":[{\"id\":\"title\",\"type\":\"text\",\"x\":300,\"y\":40,\"text\":\"Flowchart\",\"fontSize\":24},{\"id\":\"step1\",\"type\":\"rectangle\",\"x\":80,\"y\":160,\"width\":160,\"height\":70,\"label\":{\"text\":\"Start\",\"fontSize\":18}},{\"id\":\"step2\",\"type\":\"rectangle\",\"x\":320,\"y\":160,\"width\":160,\"height\":70,\"label\":{\"text\":\"Process\",\"fontSize\":18}},{\"id\":\"step3\",\"type\":\"rectangle\",\"x\":560,\"y\":160,\"width\":160,\"height\":70,\"label\":{\"text\":\"End\",\"fontSize\":18}},{\"id\":\"a1\",\"type\":\"arrow\",\"x\":240,\"y\":195,\"endX\":320,\"endY\":195},{\"id\":\"a2\",\"type\":\"arrow\",\"x\":480,\"y\":195,\"endX\":560,\"endY\":195},{\"id\":\"camera\",\"type\":\"cameraUpdate\",\"x\":40,\"y\":0,\"width\":800,\"height\":600}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "draw a simple flowchart",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Drew a three-step flowchart (Start → Process → End). Tap the iframe to open it in Excalidraw."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Use Excalidraw to sketch",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Sketched a simple system diagram for you above — three boxes (frontend, runtime, agent) connected by arrows showing the request flow. Ask if you'd like a different shape or more detail."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / multimodal",
|
||||
"sourceFile": "d5-multimodal.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "multimodal \u2014 sample image turn. The probe clicks 'Try with sample image' which auto-sends via agent.addMessage. The assertion checks the assistant transcript contains the keyword 'image'. The userMessage here matches the agentic-chat fixture for 'can you tell me what is in this demo image I just attached' which is the message the sample-attachment-buttons component dispatches.",
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo image I just attached",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The image attachment shows a small abstract test pattern used by the multimodal demo to validate the image-upload pipeline. Successful render of this response confirms the binary attachment round-tripped through the runtime."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "multimodal \u2014 sample PDF turn. The probe clicks 'Try with sample PDF' which auto-sends. The assertion checks the assistant transcript contains the keyword 'document'.",
|
||||
"match": {
|
||||
"userMessage": "can you tell me what is in this demo pdf I just attached",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The PDF document contains a single test page used by the multimodal demo. Its text was flattened by pypdf on the Python side and forwarded as text content to the model. Receiving this response confirms the document upload path works."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / prebuilt-popup",
|
||||
"sourceFile": "d5-prebuilt-popup.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "prebuilt-popup — single turn proving the CopilotPopup component renders and messages round-trip inside its .copilotKitPopup root. The probe asserts the popup root is visible and the assistant response lands inside it.",
|
||||
"match": {
|
||||
"userMessage": "hi from the popup test",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello from the popup — the CopilotPopup prebuilt component is wired up and reachable. The launcher floats in the corner and the chat sits in an overlay panel."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / prebuilt-sidebar",
|
||||
"sourceFile": "d5-prebuilt-sidebar.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "prebuilt-sidebar — single turn proving the CopilotSidebar component renders and messages round-trip inside its .copilotKitSidebar root. The probe asserts the sidebar root is visible and the assistant response lands inside it.",
|
||||
"match": {
|
||||
"userMessage": "hi from the sidebar test",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hello from the sidebar — the CopilotSidebar prebuilt component is wired up and reachable. Anything else you would like me to confirm from inside the sidebar?"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / readonly-state",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "readonly-state-agent-context — Who am I pill. Matches the verbatim pill prompt. systemMessage gating on 'Atai' was removed because @langchain/openai >=1.4 sends SystemMessage as role:'developer' for gpt-5 models, and aimock's getSystemText only checks role:'system'. The userMessage alone is deterministic from the pill.",
|
||||
"match": {
|
||||
"userMessage": "What do you know about me from my context?",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "I see you're Atai, and you're in the America/Los_Angeles timezone. Recently, you viewed the pricing page and watched the product demo video. How can I assist you today?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "readonly-state-agent-context — Suggest next steps pill. Matches the verbatim pill prompt. systemMessage gating removed (see Who am I comment above).",
|
||||
"match": {
|
||||
"userMessage": "Based on my recent activity, what should I try next?",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Since you recently viewed the pricing page and watched the product demo video, it might be a good idea to explore user testimonials or case studies to see how others have benefited from the Pro Plan. You could also start the 14-day free trial to experience the features firsthand."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What do you know about me from my context",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Based on your context I can see your name and recent activity. I'll keep responses calibrated to your timezone."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / reasoning",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "reasoning-default e2e test pill prompt. Substring 'sky appears blue' matches 'Explain step by step why the sky appears blue during the day but red at sunset.' Includes a `reasoning` field so aimock emits REASONING_MESSAGE_* events; without it the built-in CopilotChatReasoningMessage 'Thinking…/Thought for…' label never lands.",
|
||||
"match": {
|
||||
"userMessage": "sky appears blue",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "First, I considered that visible sunlight contains all wavelengths. Then I noted that air molecules scatter shorter wavelengths (blue) more efficiently than longer ones (Rayleigh scattering). At sunset the path through the atmosphere is much longer, so even more blue is scattered out and the remaining direct light skews red.",
|
||||
"content": "Daytime sky looks blue because air molecules scatter short-wavelength light more strongly than long-wavelength light (Rayleigh scattering). At sunset the sun's light traverses far more atmosphere, scattering out most of the blue and leaving the remaining direct light dominated by red and orange wavelengths."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D5 reasoning-display probe (harness/src/probes/scripts/d5-reasoning-display.ts) sends 'show your reasoning step by step' verbatim. Mirrored from showcase/harness/fixtures/d5/reasoning-display.json so the bundled aimock has a match (the harness fixture file isn't loaded by the Docker-baked aimock).",
|
||||
"match": {
|
||||
"userMessage": "show your reasoning step by step",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "First, I identified that the question requires step-by-step reasoning. Then I broke it into sub-steps and worked through each one. Finally, I aggregated the partial answers into a single response.",
|
||||
"content": "Reasoning: first, I identified the question requires step-by-step thinking. Then, I broke it into sub-steps and worked through each one. Finally, I aggregated the partial answers into a single response. The reasoning block above the answer demonstrates intermediate-thought rendering."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,148 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / recorded",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-54-59-257Z-b614f3f8.json",
|
||||
"match": {
|
||||
"userMessage": "First use the query_data tool to fetch the financial sales data, then using A2UI, show me a sales dashboard with total revenue, new customers, and conversion rate metrics. Include a pie chart of revenue by category and a bar chart of monthly sales.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 0,
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "query_data",
|
||||
"arguments": "{\"query\":\"financial sales data including total revenue, new customers, conversion rate, revenue by category, and monthly sales\"}",
|
||||
"id": "call_jWKVlcdnnYiV6MT3wzNZphnF"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-55-00-312Z-814153b0.json",
|
||||
"match": {
|
||||
"userMessage": "First use the query_data tool to fetch the financial sales data, then using A2UI, show me a sales dashboard with total revenue, new customers, and conversion rate metrics. Include a pie chart of revenue by category and a bar chart of monthly sales.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 1,
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{}",
|
||||
"id": "call_K6au6wp1CjG4OuiGzhL1yCXL"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-55-05-134Z-890eead9.json",
|
||||
"match": {
|
||||
"userMessage": "First use the query_data tool to fetch the financial sales data, then using A2UI, show me a sales dashboard with total revenue, new customers, and conversion rate metrics. Include a pie chart of revenue by category and a bar chart of monthly sales.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 2,
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "pieChart",
|
||||
"arguments": "{\"title\": \"Revenue by Category\", \"description\": \"Distribution of revenue across different categories\", \"data\": [{\"label\": \"Enterprise Subscriptions\", \"value\": 94000}, {\"label\": \"Pro Tier Upgrades\", \"value\": 66500}, {\"label\": \"API Usage Overages\", \"value\": 34500}, {\"label\": \"Consulting Services\", \"value\": 54000}, {\"label\": \"Marketplace Sales\", \"value\": 42800}, {\"label\": \"Partnership Revenue\", \"value\": 25700}, {\"label\": \"Training & Workshops\", \"value\": 10200}]}",
|
||||
"id": "call_OJDsrTtDcWoV0PObgKtU9fXv"
|
||||
},
|
||||
{
|
||||
"name": "barChart",
|
||||
"arguments": "{\"title\": \"Monthly Sales\", \"description\": \"Sales performance over the months\", \"data\": [{\"label\": \"January\", \"value\": 82800}, {\"label\": \"February\", \"value\": 119000}, {\"label\": \"March\", \"value\": 135700}]}",
|
||||
"id": "call_o9RdNrZTq8qV0Majn12OTZcu"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-55-07-715Z-9531bf64.json",
|
||||
"match": {
|
||||
"userMessage": "First use the query_data tool to fetch the financial sales data, then using A2UI, show me a sales dashboard with total revenue, new customers, and conversion rate metrics. Include a pie chart of revenue by category and a bar chart of monthly sales.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 3,
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The sales dashboard is now available, featuring metrics for total revenue, new customers, and conversion rate. Additionally, you can view a pie chart of revenue by category and a bar chart of monthly sales."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-56-02-547Z-7afa30ad.json. SUPERSEDED: this entry is unreachable and editing it has no effect — the calculator pill is served by the specific calculator fixtures in _from-feature-parity.json (loaded first alphabetically; first match wins). Note the model gate (gpt-4o) is NOT an independent safety net — whether it matches depends on each integration's current model; unreachability is guaranteed only by the load-order shadowing above. Kept for historical reference; do not delete.",
|
||||
"match": {
|
||||
"userMessage": "Using the generateSandboxedUi tool, build a modern calculator with standard buttons plus labeled metric shortcut buttons that insert their values into the display when clicked. Use sample company data.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 0,
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "generateSandboxedUi",
|
||||
"arguments": "{\"initialHeight\":400,\"placeholderMessages\":[\"Building calculator UI...\"],\"css\":\"body { font-family: Arial, sans-serif; display: flex; justify-content: center; align-items: center; height: 100vh; margin: 0; background-color: #f4f4f9; } .calculator { width: 320px; background-color: #fff; border-radius: 10px; box-shadow: 0 4px 10px rgba(0, 0, 0, 0.1); } .display { background-color: #222; color: #fff; font-size: 2em; padding: 20px; text-align: right; border-top-left-radius: 10px; border-top-right-radius: 10px; } .buttons { display: grid; grid-template-columns: repeat(4, 1fr); grid-gap: 1px; } .button { background-color: #e0e0e0; padding: 20px; font-size: 1.2em; text-align: center; cursor: pointer; transition: background-color 0.2s; } .button:hover { background-color: #d0d0d0; } .metric-button { background-color: #4caf50; color: #fff; }\",\"html\":\"<div class='calculator'> <div class='display' id='display'>0</div> <div class='buttons'> <div class='button' onclick='appendNumber(7)'>7</div> <div class='button' onclick='appendNumber(8)'>8</div> <div class='button' onclick='appendNumber(9)'>9</div> <div class='button' onclick='operate(\\\"/\\\")'>/</div> <div class='button' onclick='appendNumber(4)'>4</div> <div class='button' onclick='appendNumber(5)'>5</div> <div class='button' onclick='appendNumber(6)'>6</div> <div class='button' onclick='operate(\\\"*\\\")'>*</div> <div class='button' onclick='appendNumber(1)'>1</div> <div class='button' onclick='appendNumber(2)'>2</div> <div class='button' onclick='appendNumber(3)'>3</div> <div class='button' onclick='operate(\\\"-\\\")'>-</div> <div class='button' onclick='appendNumber(0)'>0</div> <div class='button' onclick='clearDisplay()'>C</div> <div class='button' onclick='calculate()'>=</div> <div class='button' onclick='operate(\\\"+\\\")'>+</div> <div class='button metric-button' onclick='insertMetric(1000)'>Revenue</div> <div class='button metric-button' onclick='insertMetric(500)'>Expenses</div> <div class='button metric-button' onclick='insertMetric(200)'>Profit</div> </div> </div>\",\"jsFunctions\":\"function appendNumber(number) { const display = document.getElementById('display'); if (display.innerText === '0') { display.innerText = number; } else { display.innerText += number; } } function operate(operator) { const display = document.getElementById('display'); display.innerText += ' ' + operator + ' '; } function clearDisplay() { document.getElementById('display').innerText = '0'; } function calculate() { const display = document.getElementById('display'); try { display.innerText = eval(display.innerText); } catch { display.innerText = 'Error'; } } function insertMetric(value) { const display = document.getElementById('display'); if (display.innerText === '0') { display.innerText = value; } else { display.innerText += value; } }\",\"jsExpressions\":[]}",
|
||||
"id": "call_u2aFJvWZUKzzlvqfnDeWhMyC"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T20-56-04-559Z-1bafed55.json. SUPERSEDED: this entry is unreachable and editing it has no effect — the calculator pill is served by the specific calculator fixtures in _from-feature-parity.json (loaded first alphabetically; first match wins). Note the model gate (gpt-4o) is NOT an independent safety net — whether it matches depends on each integration's current model; unreachability is guaranteed only by the load-order shadowing above. Kept for historical reference; do not delete.",
|
||||
"match": {
|
||||
"userMessage": "Using the generateSandboxedUi tool, build a modern calculator with standard buttons plus labeled metric shortcut buttons that insert their values into the display when clicked. Use sample company data.",
|
||||
"model": "gpt-4o",
|
||||
"turnIndex": 1,
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The modern calculator with standard buttons and labeled metric shortcut buttons is ready for use. You can perform calculations and insert sample company data values like Revenue, Expenses, and Profit directly into the display."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T22-01-34-384Z-edf48666.json",
|
||||
"match": {
|
||||
"userMessage": "What is the weather in Tokyo?",
|
||||
"model": "gpt-4o-mini",
|
||||
"turnIndex": 0,
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}",
|
||||
"id": "call_0uXBAr4ySTrLQGQgszQYFtEJ"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Recorded fixture from d5-recorded: openai-2026-05-15T22-01-35-344Z-6ea6b18a.json",
|
||||
"match": {
|
||||
"userMessage": "What is the weather in Tokyo?",
|
||||
"model": "gpt-4o-mini",
|
||||
"turnIndex": 1,
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The weather in Tokyo is currently 68°F, with 55% humidity, a wind speed of 10 mph, and sunny conditions."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,494 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / render-a2ui",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "render the a2ui schema",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The A2UI fixed-schema component was rendered. The schema-driven UI received the agent's payload and produced the corresponding UI element from the locked schema definition."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "have the agent emit a ui",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The agent emitted a UI block as part of its turn. The agent acts as the UI generator: its response payload describes the component and the renderer materialized it inline with the assistant message."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a pie chart of revenue by category",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_render_pie_chart_001",
|
||||
"name": "render_pie_chart",
|
||||
"arguments": "{\"title\":\"Revenue by Category\",\"description\":\"Revenue breakdown by product category (Q4)\",\"data\":[{\"label\":\"Electronics\",\"value\":42000},{\"label\":\"Clothing\",\"value\":28000},{\"label\":\"Food\",\"value\":18000},{\"label\":\"Books\",\"value\":12000}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a pie chart of revenue by category",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Pie chart rendered above — Electronics is the largest slice, followed by Clothing, Food, and Books."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "render the declarative card",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The declarative gen-UI specification has been resolved into a rendered card. The component descriptor was forwarded to the frontend renderer which materialized the card declaratively from the schema."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a profile card for Ada Lovelace",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_show_card_001",
|
||||
"name": "show_card",
|
||||
"arguments": "{\"title\":\"Ada Lovelace\",\"body\":\"English mathematician (1815\\u20131852), credited as the first computer programmer for her notes on Charles Babbage's Analytical Engine \\u2014 including what is now recognized as the first algorithm intended to be carried out by a machine.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Show me a profile card for Ada Lovelace",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is a quick card for Ada Lovelace — the rendered card above shows a short biography. Let me know if you want a deeper dive on her work or a different historical figure."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "trip to mars",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_generate_steps_001",
|
||||
"name": "generate_task_steps",
|
||||
"arguments": "{\"steps\":[{\"description\":\"Research Mars mission requirements and timeline\",\"status\":\"enabled\"},{\"description\":\"Design spacecraft and life support systems\",\"status\":\"enabled\"},{\"description\":\"Recruit and train the crew\",\"status\":\"enabled\"},{\"description\":\"Launch and navigate to Mars\",\"status\":\"enabled\"},{\"description\":\"Land and establish base camp\",\"status\":\"enabled\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "trip to mars",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Great choices! I will proceed with executing the selected steps for your trip to Mars. Let me work through each one."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "KPI dashboard",
|
||||
"toolCallId": "call_d5_a2ui_dynamic_kpi_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the KPI dashboard you requested."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "KPI dashboard",
|
||||
"toolName": "generate_a2ui",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{\"context\":\"KPI dashboard\"}",
|
||||
"id": "call_d5_a2ui_dynamic_kpi_001"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "KPI dashboard",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_design_a2ui_kpi_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\": \"kpi-dashboard\", \"catalogId\": \"declarative-gen-ui-catalog\", \"components\": [{\"id\": \"root\", \"component\": \"Card\", \"title\": \"Quarterly KPIs\", \"subtitle\": \"Revenue, signups, and churn\", \"child\": \"metrics-row\"}, {\"id\": \"metrics-row\", \"component\": \"Row\", \"children\": [\"m-rev\", \"m-sign\", \"m-churn\"], \"gap\": 16}, {\"id\": \"m-rev\", \"component\": \"Metric\", \"label\": \"Revenue\", \"value\": \"$1.24M\", \"trend\": \"up\"}, {\"id\": \"m-sign\", \"component\": \"Metric\", \"label\": \"Signups\", \"value\": \"8,420\", \"trend\": \"up\"}, {\"id\": \"m-churn\", \"component\": \"Metric\", \"label\": \"Churn\", \"value\": \"2.3%\", \"trend\": \"down\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of sales by region",
|
||||
"toolCallId": "call_d5_a2ui_dynamic_pie_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the pie chart by region."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of sales by region",
|
||||
"toolName": "generate_a2ui",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{\"context\":\"pie chart of sales by region\"}",
|
||||
"id": "call_d5_a2ui_dynamic_pie_001"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "pie chart of sales by region",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_design_a2ui_pie_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\": \"pie-sales\", \"catalogId\": \"declarative-gen-ui-catalog\", \"components\": [{\"id\": \"root\", \"component\": \"PieChart\", \"title\": \"Sales by region\", \"description\": \"Q4 revenue split\", \"data\": [{\"label\": \"NA\", \"value\": 540}, {\"label\": \"EMEA\", \"value\": 320}, {\"label\": \"APAC\", \"value\": 210}, {\"label\": \"LATAM\", \"value\": 90}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly revenue",
|
||||
"toolCallId": "call_d5_a2ui_dynamic_bar_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the bar chart of quarterly revenue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly revenue",
|
||||
"toolName": "generate_a2ui",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{\"context\":\"bar chart of quarterly revenue\"}",
|
||||
"id": "call_d5_a2ui_dynamic_bar_001"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly revenue",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_design_a2ui_bar_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\": \"bar-quarterly\", \"catalogId\": \"declarative-gen-ui-catalog\", \"components\": [{\"id\": \"root\", \"component\": \"BarChart\", \"title\": \"Quarterly revenue\", \"description\": \"FY 2025 per quarter\", \"data\": [{\"label\": \"Q1\", \"value\": 820}, {\"label\": \"Q2\", \"value\": 950}, {\"label\": \"Q3\", \"value\": 1100}, {\"label\": \"Q4\", \"value\": 1240}]}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "status report on system health",
|
||||
"toolCallId": "call_d5_a2ui_dynamic_status_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the system health status report."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "status report on system health",
|
||||
"toolName": "generate_a2ui",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "generate_a2ui",
|
||||
"arguments": "{\"context\":\"status report on system health\"}",
|
||||
"id": "call_d5_a2ui_dynamic_status_001"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "status report on system health",
|
||||
"toolName": "_design_a2ui_surface",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_design_a2ui_status_001",
|
||||
"name": "_design_a2ui_surface",
|
||||
"arguments": "{\"surfaceId\": \"status-report\", \"catalogId\": \"declarative-gen-ui-catalog\", \"components\": [{\"id\": \"root\", \"component\": \"Card\", \"title\": \"System health\", \"subtitle\": \"Live status\", \"child\": \"rows\"}, {\"id\": \"rows\", \"component\": \"Column\", \"children\": [\"r-api\", \"r-db\", \"r-bg\"], \"gap\": 8}, {\"id\": \"r-api\", \"component\": \"Row\", \"children\": [\"l-api\", \"b-api\"], \"gap\": 8}, {\"id\": \"l-api\", \"component\": \"InfoRow\", \"label\": \"API\", \"value\": \"p99 142ms\"}, {\"id\": \"b-api\", \"component\": \"StatusBadge\", \"text\": \"Healthy\", \"variant\": \"success\"}, {\"id\": \"r-db\", \"component\": \"Row\", \"children\": [\"l-db\", \"b-db\"], \"gap\": 8}, {\"id\": \"l-db\", \"component\": \"InfoRow\", \"label\": \"Database\", \"value\": \"Replication lag 220ms\"}, {\"id\": \"b-db\", \"component\": \"StatusBadge\", \"text\": \"Degraded\", \"variant\": \"warning\"}, {\"id\": \"r-bg\", \"component\": \"Row\", \"children\": [\"l-bg\", \"b-bg\"], \"gap\": 8}, {\"id\": \"l-bg\", \"component\": \"InfoRow\", \"label\": \"Background workers\", \"value\": \"Queue depth 12\"}, {\"id\": \"b-bg\", \"component\": \"StatusBadge\", \"text\": \"Healthy\", \"variant\": \"success\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "render_a2ui — KPI dashboard pill (Google-ADK secondary tool name). Card has a single `child` slot (per myDefinitions.Card.props.child: string), so we wrap the three Metrics in a basic-catalog Column to satisfy the multi-child layout.",
|
||||
"match": {
|
||||
"userMessage": "KPI dashboard",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_a2ui",
|
||||
"arguments": {
|
||||
"surfaceId": "declarative-surface",
|
||||
"catalogId": "declarative-gen-ui-catalog",
|
||||
"components": [
|
||||
{
|
||||
"id": "root",
|
||||
"component": "Card",
|
||||
"title": "KPI dashboard",
|
||||
"subtitle": "Last 30 days",
|
||||
"child": "metrics-col"
|
||||
},
|
||||
{
|
||||
"id": "metrics-col",
|
||||
"component": "Column",
|
||||
"children": [
|
||||
"metric-revenue",
|
||||
"metric-signups",
|
||||
"metric-churn"
|
||||
],
|
||||
"gap": 12
|
||||
},
|
||||
{
|
||||
"id": "metric-revenue",
|
||||
"component": "Metric",
|
||||
"label": "Revenue",
|
||||
"value": "$1.2M",
|
||||
"trend": "up"
|
||||
},
|
||||
{
|
||||
"id": "metric-signups",
|
||||
"component": "Metric",
|
||||
"label": "Signups",
|
||||
"value": "4,820",
|
||||
"trend": "up"
|
||||
},
|
||||
{
|
||||
"id": "metric-churn",
|
||||
"component": "Metric",
|
||||
"label": "Churn",
|
||||
"value": "2.1%",
|
||||
"trend": "down"
|
||||
}
|
||||
],
|
||||
"data": {}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "render_a2ui — pie-chart pill (Google-ADK). Flat {id, component, ...props} shape.",
|
||||
"match": {
|
||||
"userMessage": "pie chart of sales by region",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_a2ui",
|
||||
"arguments": {
|
||||
"surfaceId": "declarative-surface",
|
||||
"catalogId": "declarative-gen-ui-catalog",
|
||||
"components": [
|
||||
{
|
||||
"id": "root",
|
||||
"component": "PieChart",
|
||||
"title": "Sales by region",
|
||||
"description": "Q4 — share of total revenue",
|
||||
"data": [
|
||||
{
|
||||
"label": "North America",
|
||||
"value": 540
|
||||
},
|
||||
{
|
||||
"label": "EMEA",
|
||||
"value": 320
|
||||
},
|
||||
{
|
||||
"label": "APAC",
|
||||
"value": 210
|
||||
},
|
||||
{
|
||||
"label": "LATAM",
|
||||
"value": 90
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"data": {}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "render_a2ui — bar-chart pill (Google-ADK). Flat {id, component, ...props} shape.",
|
||||
"match": {
|
||||
"userMessage": "bar chart of quarterly revenue",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_a2ui",
|
||||
"arguments": {
|
||||
"surfaceId": "declarative-surface",
|
||||
"catalogId": "declarative-gen-ui-catalog",
|
||||
"components": [
|
||||
{
|
||||
"id": "root",
|
||||
"component": "BarChart",
|
||||
"title": "Quarterly revenue",
|
||||
"description": "FY24 — USD thousands",
|
||||
"data": [
|
||||
{
|
||||
"label": "Q1",
|
||||
"value": 820
|
||||
},
|
||||
{
|
||||
"label": "Q2",
|
||||
"value": 940
|
||||
},
|
||||
{
|
||||
"label": "Q3",
|
||||
"value": 1080
|
||||
},
|
||||
{
|
||||
"label": "Q4",
|
||||
"value": 1240
|
||||
}
|
||||
]
|
||||
}
|
||||
],
|
||||
"data": {}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "render_a2ui — status-report pill (Google-ADK). Card has single `child` slot, so wrap the three StatusBadges in a basic-catalog Column.",
|
||||
"match": {
|
||||
"userMessage": "status report on system health",
|
||||
"toolName": "render_a2ui",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "render_a2ui",
|
||||
"arguments": {
|
||||
"surfaceId": "declarative-surface",
|
||||
"catalogId": "declarative-gen-ui-catalog",
|
||||
"components": [
|
||||
{
|
||||
"id": "root",
|
||||
"component": "Card",
|
||||
"title": "System health",
|
||||
"subtitle": "All services",
|
||||
"child": "status-col"
|
||||
},
|
||||
{
|
||||
"id": "status-col",
|
||||
"component": "Column",
|
||||
"children": [
|
||||
"status-api",
|
||||
"status-db",
|
||||
"status-workers"
|
||||
],
|
||||
"gap": 8
|
||||
},
|
||||
{
|
||||
"id": "status-api",
|
||||
"component": "StatusBadge",
|
||||
"text": "API: healthy",
|
||||
"variant": "success"
|
||||
},
|
||||
{
|
||||
"id": "status-db",
|
||||
"component": "StatusBadge",
|
||||
"text": "Database: healthy",
|
||||
"variant": "success"
|
||||
},
|
||||
{
|
||||
"id": "status-workers",
|
||||
"component": "StatusBadge",
|
||||
"text": "Workers: degraded",
|
||||
"variant": "warning"
|
||||
}
|
||||
],
|
||||
"data": {}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / shared-state-read-write",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "recall the user preference",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Per the read-only context the user prefers concise responses. The agent received this preference via the shared-state context and is honoring it without writing back to state."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read-write — Greet pill. Matches on userMessage only; the systemMessage gate was too fragile (the middleware's output format varies by CopilotKit runtime version and system prompt ordering).",
|
||||
"match": {
|
||||
"userMessage": "Say hi and introduce yourself",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Hi — I'm your shared-state co-pilot. Your Preferences panel (name, tone, language, interests) is fed to me on every turn, and I jot notes back into the Agent Scratch Pad via set_notes so the UI re-renders. Try setting your name or asking me to remember something."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-read-write — Plan-a-weekend pill. Same no-systemMessage gate as the Greet pill.",
|
||||
"match": {
|
||||
"userMessage": "weekend plan based on my interests",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "A weekend tailored to your interests panel: if you haven't picked any yet, try Cooking + Travel for a market-and-day-trip combo, or Tech + Books for a maker session and a long reading afternoon. Add interests in the Preferences panel and re-ask for a more specific plan."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "remember that my favorite color is blue",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_set_notes_001",
|
||||
"name": "set_notes",
|
||||
"arguments": "{\"notes\":[\"Favorite color: blue\"]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "remember that my favorite color is blue",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Got it — I have noted that your favorite color is blue."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-write turn 2 — recall. No turnIndex gate: this is turn 2 of a 2-turn probe (turn 1 = set_notes, turn 2 = recall).",
|
||||
"match": {
|
||||
"userMessage": "favorite color",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Your favorite color is blue — I noted it earlier."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / shared-state-read",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "D5 fixture for /demos/shared-state-read (recipe-editor) turn 1. Substring 'Italian pasta recipe' is unique. Text-only — agent has no tools.",
|
||||
"match": {
|
||||
"userMessage": "Italian pasta recipe",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Great choice — looking at your current recipe state, I'd build an Italian pasta around the existing ingredients: a quick spaghetti aglio e olio, finishing with parmesan and a squeeze of lemon. Want me to suggest substitutions or adjust the cooking time?"
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "D5 fixture for /demos/shared-state-read (recipe-editor) turn 2. Reply mentions recipe-context tokens so the probe's soft assertion lands.",
|
||||
"match": {
|
||||
"userMessage": "healthier with more vegetables",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here are a few healthier ingredient additions for the recipe: roasted bell peppers, baby spinach folded in at the end, and cherry tomatoes for brightness. The Italian flavor profile holds up well, and the vegetable swap keeps the dish lighter without losing the comfort of pasta."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / shared-state-streaming",
|
||||
"sourceFile": "d5/shared-state-streaming.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "shared-state-streaming poem pill — second leg: after write_document tool result, emit content-only confirmation. MUST appear ABOVE the first-leg fixture so the matcher returns it after the tool result is appended (otherwise the unchanged last user message keeps re-matching the first-leg fixture and the agent re-fires write_document infinitely).",
|
||||
"match": {
|
||||
"userMessage": "poem about autumn leaves",
|
||||
"toolCallId": "call_d5_write_document_poem_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — the poem has been written into the shared document state."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-streaming poem pill — first leg: emit write_document tool call with substantive content (≥ 100 chars).",
|
||||
"match": {
|
||||
"userMessage": "poem about autumn leaves",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Streaming the poem now.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_write_document_poem_001",
|
||||
"name": "write_document",
|
||||
"arguments": "{\"document\":\"Crimson and amber in slow descent, / each leaf a quiet ledger of summer spent. / The wind, a courier with nothing to say, / files them gently into the morning's gray. / Somewhere a kettle hums, and afternoons grow brief — / autumn keeps its books in vermilion and gold leaf.\"}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-streaming email pill — second leg: after write_document tool result.",
|
||||
"match": {
|
||||
"userMessage": "polite email declining",
|
||||
"toolCallId": "call_d5_write_document_email_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — the decline-email draft has been written into the shared document state."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-streaming email pill — first leg: emit write_document tool call.",
|
||||
"match": {
|
||||
"userMessage": "polite email declining",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Drafting the email now.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_write_document_email_001",
|
||||
"name": "write_document",
|
||||
"arguments": "{\"document\":\"Hi — thanks for sending the invite for Tuesday afternoon. Unfortunately I won't be able to make it this week. I'd love to find time later in the month if your schedule allows. In the meantime, feel free to send any pre-reads my way and I'll review them async so we don't lose momentum. Best, [name]\"}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-streaming quantum pill — second leg: after write_document tool result.",
|
||||
"match": {
|
||||
"userMessage": "quantum computing for a curious teenager",
|
||||
"toolCallId": "call_d5_write_document_quantum_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — the quantum-computing explainer has been written into the shared document state."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "shared-state-streaming quantum pill — first leg: emit write_document tool call.",
|
||||
"match": {
|
||||
"userMessage": "quantum computing for a curious teenager",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Streaming the explainer now.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_write_document_quantum_001",
|
||||
"name": "write_document",
|
||||
"arguments": "{\"document\":\"A regular computer stores information in bits — tiny switches that are either on (1) or off (0). A quantum computer uses qubits, which can sit in a fuzzy superposition of both states at once until you check them. Stack many qubits together and they can explore lots of possibilities in parallel, which is why people are excited.\\n\\nThis doesn't make quantum computers faster at everything. They're great at problems with hidden structure — like factoring big numbers, simulating molecules, or searching certain databases — but useless for, say, opening Excel. Today's machines are noisy and small, so we mostly use them to test ideas rather than replace your laptop.\"}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,385 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / subagents",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Research the benefits of remote work and draft a one-paragraph summary",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_research_agent_001",
|
||||
"name": "research_agent",
|
||||
"arguments": "{\"task\":\"Benefits of remote work\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: research sub-agent single-turn LLM call",
|
||||
"match": {
|
||||
"userMessage": "Benefits of remote work",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "- Eliminates commute, returning ~10 hours/week to employees\n- Surveys consistently show higher job satisfaction among remote workers\n- Employers gain access to a geographically unbounded talent pool\n- Reduced office overhead (rent, utilities, maintenance)\n- Trade-offs: ad-hoc collaboration, mentorship of junior staff, and cultural cohesion degrade without intentional replacement rituals"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Research the benefits of remote work and draft a one-paragraph summary",
|
||||
"turnIndex": 1,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_writing_agent_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"task\":\"One-paragraph summary on the benefits of remote work, grounded in the research facts.\\n\\nFacts:\\n- Eliminates commute, returning ~10 hours per week\\n- Higher reported job satisfaction in repeated surveys\\n- Wider talent pool for employers, no geographic limit\\n- Reduced office overhead\\n- Trade-offs: collaboration, mentorship, cultural cohesion\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: writing sub-agent single-turn LLM call",
|
||||
"match": {
|
||||
"userMessage": "One-paragraph summary on the benefits of remote work",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Research the benefits of remote work and draft a one-paragraph summary",
|
||||
"turnIndex": 2,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_critique_agent_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"task\":\"Remote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: critique sub-agent single-turn LLM call",
|
||||
"match": {
|
||||
"userMessage": "Remote work returns roughly ten hours",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "1. Add a specific citation or date range for the surveys mentioned — 'repeated surveys' is vague and weakens the empirical claim.\n2. The ten-hours figure needs sourcing; without it the opening reads as an unsupported assertion rather than a research finding.\n3. The final sentence about 'intentional rituals' is abstract — one concrete example (e.g., scheduled pair-programming sessions, virtual coffee chats) would make the counterweight more actionable."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Research the benefits of remote work and draft a one-paragraph summary",
|
||||
"turnIndex": 3,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the summary, after research → drafting → critique:\n\nRemote work returns roughly ten hours a week to employees by eliminating the commute, and repeated surveys show meaningfully higher job satisfaction among remote workers. Employers benefit too: a geographically unbounded talent pool and lower office overhead. The honest counterweight is that ad-hoc collaboration, mentorship of junior staff, and cultural cohesion all degrade without intentional rituals to replace what an office provided implicitly."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Subagents pill 1 — 'Write a blog post' / cold exposure training. Drives supervisor → research_agent → writing_agent → critique_agent → final reply, plus three nested sub-agent turns.",
|
||||
"match": {
|
||||
"userMessage": "Produce a short blog post about the benefits of cold exposure training",
|
||||
"hasToolResult": false,
|
||||
"toolName": "research_agent",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p1_research_001",
|
||||
"name": "research_agent",
|
||||
"arguments": "{\"task\":\"Cold exposure training key facts\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: research sub-agent returns deterministic facts about cold exposure training.",
|
||||
"match": {
|
||||
"userMessage": "Cold exposure training key facts",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "- Brief cold immersion (cold showers, ice baths) triggers a sympathetic-nervous-system response that releases noradrenaline\n- Repeated exposure is associated with improved self-reported mood and stress tolerance\n- Activates brown adipose tissue, modestly increasing basal metabolic rate\n- May reduce post-exercise muscle soreness when used as a recovery modality\n- Health risk for people with cardiovascular conditions; sessions should be short (1-3 minutes) and supervised at first"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Produce a short blog post about the benefits of cold exposure training",
|
||||
"turnIndex": 1,
|
||||
"toolName": "writing_agent",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p1_writing_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"task\":\"Short blog-post paragraph on the benefits of cold exposure training, grounded in the research facts.\\n\\nFacts:\\n- Brief cold immersion triggers a noradrenaline release\\n- Repeated exposure improves self-reported mood and stress tolerance\\n- Activates brown adipose tissue, modestly raises basal metabolic rate\\n- May reduce post-exercise muscle soreness\\n- Cardiovascular risk; keep early sessions short and supervised\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: writing sub-agent returns deterministic prose for cold exposure.",
|
||||
"match": {
|
||||
"userMessage": "Short blog-post paragraph on the benefits of cold exposure training",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Cold exposure training — short, deliberate plunges into cold water or showers — has earned a foothold in modern recovery routines for reasons grounded in physiology rather than folklore. Each immersion produces a measurable surge of noradrenaline, the same chemistry that underpins the lift practitioners report in mood and focus afterward; with repetition, that response is associated with greater day-to-day stress tolerance. Cold also activates brown adipose tissue and can blunt post-exercise soreness, making it a low-cost adjunct for active people. The honest caveat is cardiovascular risk: keep early sessions to one to three minutes, and if you have a heart condition, get a green light before you start."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Produce a short blog post about the benefits of cold exposure training",
|
||||
"turnIndex": 2,
|
||||
"toolName": "critique_agent",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p1_critique_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"task\":\"Critique the cold-exposure blog draft. Draft: Cold exposure training — short, deliberate plunges into cold water or showers — has earned a foothold in modern recovery routines for reasons grounded in physiology rather than folklore. Each immersion produces a measurable surge of noradrenaline...\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: critique sub-agent returns deterministic critique of cold-exposure draft.",
|
||||
"match": {
|
||||
"userMessage": "Critique the cold-exposure blog draft",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "1. The phrase 'measurable surge of noradrenaline' should cite a study or rough magnitude — without a number, the claim reads as marketing rather than evidence.\n2. 'Modern recovery routines' is filler; lead with the physiological mechanism instead and let the cultural framing follow.\n3. The cardiovascular caveat is buried at the end. Move it earlier or make it a standalone closing line so a reader who skims still sees it."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Produce a short blog post about the benefits of cold exposure training",
|
||||
"turnIndex": 3,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the polished blog post on cold exposure training, after research → drafting → critique:\n\nCold exposure training — short, deliberate plunges into cold water or showers — produces a measurable noradrenaline surge that practitioners feel as a mood and focus lift; repeated exposure is associated with greater stress tolerance, brown-adipose activation, and reduced post-exercise soreness. Keep early sessions to one to three minutes, and if you have a cardiovascular condition, clear it with a clinician first."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Subagents pill 2 — 'Explain a topic' / LLM tool calling.",
|
||||
"match": {
|
||||
"userMessage": "Explain how large language models handle tool calling",
|
||||
"hasToolResult": false,
|
||||
"toolName": "research_agent",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p2_research_001",
|
||||
"name": "research_agent",
|
||||
"arguments": "{\"task\":\"How LLMs handle tool calling\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: research sub-agent for LLM tool calling.",
|
||||
"match": {
|
||||
"userMessage": "How LLMs handle tool calling",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "- The model is shown a tool schema (name, description, JSON-schema parameters) inside the system or developer prompt at request time\n- During decoding, instead of emitting natural-language text, the model emits a structured tool_call block (function name + JSON-encoded arguments)\n- The application runs the tool, packages the result into a tool message, and resends the full conversation so the model can continue\n- Modern decoders use constrained decoding or grammars to keep the arguments syntactically valid JSON\n- The model decides on tool use turn-by-turn — there is no out-of-band channel; tool calls are just a different message role in the same chat thread"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain how large language models handle tool calling",
|
||||
"turnIndex": 1,
|
||||
"toolName": "writing_agent",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p2_writing_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"task\":\"One-paragraph explanation of how LLMs handle tool calling, grounded in the research.\\n\\nFacts:\\n- Tool schemas (name, description, JSON-schema params) are passed in the prompt\\n- Models emit a structured tool_call block instead of text\\n- Application runs the tool and replays the result as a tool message\\n- Constrained decoding keeps arguments valid JSON\\n- Tool use is decided turn-by-turn in the same chat thread\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: writing sub-agent for LLM tool calling.",
|
||||
"match": {
|
||||
"userMessage": "One-paragraph explanation of how LLMs handle tool calling",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Large language models handle tool calling by treating tools as a structured extension of the chat protocol rather than a separate channel. At request time the application supplies each tool's name, description, and JSON-schema parameters in the prompt; during decoding the model can emit a tool_call block — a function name plus JSON-encoded arguments — instead of plain text, with constrained decoding keeping the arguments syntactically valid. The application then executes the tool and replays the result back as a tool-role message, and the model continues the conversation from there. The decision to call a tool is made turn-by-turn, so a single user request can fan out into a chain of tool calls that the model orchestrates as it reads each result."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain how large language models handle tool calling",
|
||||
"turnIndex": 2,
|
||||
"toolName": "critique_agent",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p2_critique_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"task\":\"Critique the LLM tool-calling explanation draft. Draft: Large language models handle tool calling by treating tools as a structured extension of the chat protocol rather than a separate channel. At request time the application supplies each tool's name, description, and JSON-schema parameters in the prompt...\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: critique sub-agent for LLM tool calling.",
|
||||
"match": {
|
||||
"userMessage": "Critique the LLM tool-calling explanation draft",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "1. The opening contrast 'rather than a separate channel' assumes the reader already knows what a 'separate channel' would mean — either drop the contrast or give a one-clause example (e.g., 'rather than a side API the model talks to in parallel').\n2. 'Constrained decoding keeping the arguments syntactically valid' is technically correct but vague; mention that this is what makes the JSON parseable on the application side.\n3. The final sentence introduces multi-tool chains without saying who controls the loop — clarify that the application is the runtime that decides whether to keep going, not the model itself."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Explain how large language models handle tool calling",
|
||||
"turnIndex": 3,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the explanation of LLM tool calling, after research → drafting → critique:\n\nLLMs treat tool calling as part of the chat protocol: each tool's name, description, and JSON-schema parameters are passed in the prompt, and during decoding the model can emit a structured tool_call block (function name + JSON arguments) instead of plain text. The application — not the model — runs the tool, returns the result as a tool-role message, and decides whether to keep looping. Constrained decoding keeps the arguments valid JSON the application can parse, and the model orchestrates multi-step chains turn-by-turn from inside the same conversation."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Subagents pill 3 — 'Summarize a topic' / reusable rockets. Note: the original Railway-side bug for this pill was the concurrent-update on `delegations`; the agent state reducer fix unblocks this fixture chain.",
|
||||
"match": {
|
||||
"userMessage": "Summarize the current state of reusable rockets",
|
||||
"hasToolResult": false,
|
||||
"toolName": "research_agent",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p3_research_001",
|
||||
"name": "research_agent",
|
||||
"arguments": "{\"task\":\"Current state of reusable rockets\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: research sub-agent for reusable rockets.",
|
||||
"match": {
|
||||
"userMessage": "Current state of reusable rockets",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "- SpaceX Falcon 9 routinely lands and re-flies first stages; individual boosters have flown more than 20 missions each\n- Falcon Heavy reuses both side boosters; the center core has been recovered on a subset of flights\n- Rocket Lab's Electron has demonstrated mid-air booster catch but routine reuse is still in development\n- SpaceX Starship is targeting full reuse of both stages; orbital test flights are ongoing as of 2024-2025\n- Reuse is the dominant lever on launch cost: Falcon 9 list pricing is set well below expendable competitors largely because of stage recovery"
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize the current state of reusable rockets",
|
||||
"turnIndex": 1,
|
||||
"toolName": "writing_agent",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p3_writing_001",
|
||||
"name": "writing_agent",
|
||||
"arguments": "{\"task\":\"One polished paragraph summarizing the current state of reusable rockets, grounded in the research.\\n\\nFacts:\\n- Falcon 9 first stages routinely re-fly, some 20+ flights\\n- Falcon Heavy reuses side boosters; center core recovered sometimes\\n- Rocket Lab Electron demonstrating mid-air catch, reuse still in development\\n- SpaceX Starship targeting full reuse of both stages, in flight test\\n- Reuse drives launch cost downward\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: writing sub-agent for reusable rockets.",
|
||||
"match": {
|
||||
"userMessage": "One polished paragraph summarizing the current state of reusable rockets",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Reusable rockets have shifted from a research goal to the default cost lever in commercial spaceflight. SpaceX's Falcon 9 routinely lands and re-flies its first stage — individual boosters have now flown twenty-plus missions — and Falcon Heavy reuses both side boosters with intermittent recovery of the center core. Smaller-class operators like Rocket Lab have demonstrated mid-air booster catch but have not yet made reuse routine, while SpaceX's Starship is in active flight testing toward full two-stage reuse. The economic consequence is already visible: Falcon 9 list pricing sits well below expendable competitors precisely because the dominant cost — building a fresh first stage every flight — has been amortized across many missions."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize the current state of reusable rockets",
|
||||
"turnIndex": 2,
|
||||
"toolName": "critique_agent",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_subagents_p3_critique_001",
|
||||
"name": "critique_agent",
|
||||
"arguments": "{\"task\":\"Critique the reusable rockets summary draft. Draft: Reusable rockets have shifted from a research goal to the default cost lever in commercial spaceflight. SpaceX's Falcon 9 routinely lands and re-flies its first stage — individual boosters have now flown twenty-plus missions...\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Nested: critique sub-agent for reusable rockets.",
|
||||
"match": {
|
||||
"userMessage": "Critique the reusable rockets summary draft",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "1. 'Default cost lever' is jargon that pre-supposes the reader already accepts the framing — open instead with the concrete result (Falcon 9 reflight count) and let the framing emerge.\n2. The Starship sentence is hedged ('in active flight testing toward full reuse') in a way that obscures the actual milestone reached as of writing — name the latest test outcome or drop the clause.\n3. The closing economic claim asserts pricing is 'well below expendable competitors' without a reference price; one number (e.g., $/kg-to-LEO) would land the point much harder than the qualitative claim alone."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Summarize the current state of reusable rockets",
|
||||
"turnIndex": 3,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the polished summary on reusable rockets, after research → drafting → critique:\n\nReusable rockets are now mainstream in commercial spaceflight. SpaceX's Falcon 9 lands and re-flies its first stage routinely — individual boosters have flown twenty-plus missions — and Falcon Heavy reuses both side boosters. Rocket Lab has demonstrated mid-air Electron booster catch but reuse is not yet routine, while SpaceX Starship is in active orbital flight testing with full two-stage reuse as the target. The economic impact is already priced in: Falcon 9 sits well below expendable competitors per kilogram to low Earth orbit because amortizing a recovered first stage across many missions removes the largest single cost from the launch."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "ENT-658 regression fixture for langgraph-python /demos/threadid-frontend-tool-roundtrip. First turn emits a frontend tool call; follow-up turn confirms the tool result without changing threads.",
|
||||
"created": "2026-05-27"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "invoke testFrontendToolCalling with label X",
|
||||
"toolCallId": "call_ent_658_test_frontend_tool_x",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Frontend tool finished for X."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "invoke testFrontendToolCalling with label X",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Calling frontend tool.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_ent_658_test_frontend_tool_x",
|
||||
"name": "testFrontendToolCalling",
|
||||
"arguments": {
|
||||
"label": "X"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,266 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / tool-rendering-custom-catchall. Pills: 'Weather in SF' (get_weather/San Francisco), 'Find flights' (search_flights/SFO->JFK), 'Roll a d20' (5 sequential roll_d20 calls ending in 20), 'Chain tools' (get_weather Tokyo + search_flights SFO->Tokyo + roll_d20=11). Backend tools execute on LangGraph Python and return real JSON results, which the wildcard renderer surfaces in the result block. Mirrors the matcher/tool-call shape of d6/langgraph-python/tool-rendering.json, with distinct toolCallIds to avoid cross-fixture leakage. PILL PROMPTS (must match suggestions.ts verbatim): \"What's the weather in San Francisco?\", \"Find flights from SFO to JFK.\", \"Roll a 20-sided die.\", \"Chain a few tools in this single turn: get the weather in Tokyo, search flights from SFO to Tokyo, and roll a d20.\"",
|
||||
"sourceFile": "d5-tool-rendering-custom-catchall.ts",
|
||||
"created": "2026-05-22",
|
||||
"updated": "2026-05-29"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "d5-tool-rendering-custom-catchall probe — turn 2 narration after get_weather tool result. Scoped via toolCallId (per-leg) so it only matches the weather tool-result turn; ordered BEFORE the tool-emit sibling so first-match-wins picks narration when the tool result is present. Canonical phrase 'rendered through the custom wildcard catchall' is asserted by the probe (LGP-gold disjoint-prompts content guard).",
|
||||
"match": {
|
||||
"userMessage": "Forecast Tokyo through the wildcard renderer",
|
||||
"toolCallId": "call_trcc_get_weather_d5_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Tokyo is 22°C and partly cloudy — the custom wildcard renderer should show the tool card above — rendered through the custom wildcard catchall."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "d5-tool-rendering-custom-catchall probe — turn 1 emit for get_weather (Tokyo). Disjoint userMessage from the default-catchall probe ('forecast for Tokyo' lowercase) so aimock's substring matcher cannot cross-route between catchall probes. The toolCallId-gated narration above fires after the backend tool result arrives.",
|
||||
"match": {
|
||||
"userMessage": "Forecast Tokyo through the wildcard renderer",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_trcc_get_weather_d5_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "d5-tool-rendering-custom-catchall probe — turn 2 narration after get_stock_price tool result. Scoped via toolCallId (per-leg). Canonical phrase 'rendered through the custom wildcard catchall' is asserted by the probe; AAPL $338.37/-2.96% matches the canonical payload used across the tool-rendering family.",
|
||||
"match": {
|
||||
"userMessage": "Quote AAPL through the wildcard renderer",
|
||||
"toolCallId": "call_trcc_get_stock_price_d5_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "AAPL is trading at $338.37, down 2.96% — rendered through the custom wildcard catchall renderer."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "d5-tool-rendering-custom-catchall probe — turn 1 emit for get_stock_price (AAPL). Cross-tool pair with get_weather above; the probe asserts both tools render through the SAME custom-wildcard-card testid with distinct data-tool-name values. No hasToolResult gate (the prior turn's tool result makes hasToolResult permanently true here); the toolCallId-gated narration above wins after this tool result arrives.",
|
||||
"match": {
|
||||
"userMessage": "Quote AAPL through the wildcard renderer",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_trcc_get_stock_price_d5_001",
|
||||
"name": "get_stock_price",
|
||||
"arguments": "{\"ticker\":\"AAPL\",\"price_usd\":338.37,\"change_pct\":-2.96}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Chain tools — follow-up after all 3 tools ran. Anchored on whichever of the 3 chain toolCallIds appears last in the request (LangGraph's ToolNode preserves tool_calls order). MUST come before the toolCalls-emitting fixture below so iteration 2 of the chain-tools loop hits this branch instead of re-emitting.",
|
||||
"match": {
|
||||
"userMessage": "Chain a few tools in this single turn",
|
||||
"toolCallId": "call_trcc_chain_roll_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — Tokyo is sunny, three flights found, and the d20 came up 11."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Chain a few tools in this single turn",
|
||||
"toolCallId": "call_trcc_chain_flights_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — Tokyo is sunny, three flights found, and the d20 came up 11."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Chain a few tools in this single turn",
|
||||
"toolCallId": "call_trcc_chain_weather_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — Tokyo is sunny, three flights found, and the d20 came up 11."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Chain tools — emit 3 tool calls in one assistant turn (get_weather Tokyo + search_flights SFO->Tokyo + roll_d20=11). No turnIndex/hasToolResult gate: in multi-pill demo sessions prior clicks leave tool results AND additional user turns in the thread; the toolCallId fixtures above eat iteration 2 via last-message tool_call_id gating.",
|
||||
"match": {
|
||||
"userMessage": "Chain a few tools in this single turn",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_trcc_chain_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
},
|
||||
{
|
||||
"id": "call_trcc_chain_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"origin\":\"SFO\",\"destination\":\"Tokyo\"}"
|
||||
},
|
||||
{
|
||||
"id": "call_trcc_chain_roll_001",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":11}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Weather in SF — follow-up content after get_weather tool ran. MUST come before the tool-emitting fixture below (first-match-wins) so iteration 2 of the loop hits this branch instead of re-emitting. toolCallId chain keeps the fixture stateless across multi-pill thread history.",
|
||||
"match": {
|
||||
"userMessage": "What's the weather in San Francisco?",
|
||||
"toolCallId": "call_trcc_weather_sf_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "San Francisco is currently 68°F and sunny with light winds — rendered through the custom wildcard catchall."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Weather in SF — first turn: emit get_weather tool call with location=San Francisco. The wildcard renderer paints [data-testid='custom-wildcard-card'][data-tool-name='get_weather'] with args containing 'San Francisco'.",
|
||||
"match": {
|
||||
"userMessage": "What's the weather in San Francisco?",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_trcc_weather_sf_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"San Francisco\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Find flights — follow-up content after search_flights ran. MUST come BEFORE the first-leg fixture below — the matcher is first-match-wins, and the second leg is uniquely identified by `toolCallId` (last message is a tool with this id), so it cannot accidentally swallow the first-leg request (whose last message is the user prompt). The result block of the wildcard card surfaces the tool execution JSON (which already contains 'United', 'Delta', 'JetBlue' from the Python backend); this narration is just to terminate the agent loop.",
|
||||
"match": {
|
||||
"userMessage": "Find flights from SFO to JFK.",
|
||||
"toolCallId": "call_trcc_flights_sfo_jfk_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Three flights from SFO to JFK — United UA231 at 08:15 ($348), Delta DL412 at 11:20 ($312), and JetBlue B6722 at 17:05 ($289)."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Find flights — first turn: emit search_flights tool call (SFO -> JFK). Backend executes and returns the 3-flight result list; the wildcard renderer surfaces 'United'/'Delta'/'JetBlue' in the result block.",
|
||||
"match": {
|
||||
"userMessage": "Find flights from SFO to JFK.",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_trcc_flights_sfo_jfk_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"origin\":\"SFO\",\"destination\":\"JFK\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Roll a d20 — exactly 5 sequential roll_d20 calls returning [7, 14, 3, 19, 20]. Chained by toolCallId so the sequence is stateless across thread history. Specific-toolCallId fixtures MUST come before the userMessage-only fixture below; first-match-wins. The 5th roll has value=20 so the result block of the 5th card surfaces \"value\":20.",
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"toolCallId": "call_trcc_d20_seq_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_trcc_d20_seq_002",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":14}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"toolCallId": "call_trcc_d20_seq_002",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_trcc_d20_seq_003",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":3}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"toolCallId": "call_trcc_d20_seq_003",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_trcc_d20_seq_004",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":19}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"toolCallId": "call_trcc_d20_seq_004",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_trcc_d20_seq_005",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":20}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"toolCallId": "call_trcc_d20_seq_005",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Rolled the d20 five times — landed on 20 on the final roll."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "First roll. Matches the initial user prompt (no prior d20 tool result in this chain yet). Comes after the toolCallId-chained fixtures above so iterations 2-6 of the loop hit those first. The multi-pill sequential e2e test clicks 'Find flights' first which leaves prior turns in the thread, so a new 'Roll a 20-sided die.' user message is no longer at turnIndex 0 — keep this fixture turnIndex-less. The toolCallId-chained fixtures still take precedence for iterations 2-6 because their last-message gate (role=tool with the chained id) only matches mid-chain — this fixture only matches when last-message.role=user.",
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_trcc_d20_seq_001",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":7}"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,48 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / tool-rendering-default-catchall",
|
||||
"sourceFile": "d5-tool-rendering-default-catchall.ts",
|
||||
"created": "2026-05-22"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "tool-rendering-default-catchall \u2014 weather in Tokyo follow-up. After get_weather tool result returns, emit narrated content. MUST come before the tool-emitting fixture so iteration 2 hits this branch. The built-in default renderer renders [data-testid='copilot-tool-render'][data-tool-name='get_weather'] with a status pill [data-testid='copilot-tool-render-status'].",
|
||||
"match": {
|
||||
"userMessage": "forecast for Tokyo",
|
||||
"toolCallId": "call_d6_dc_weather_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Tokyo is 22\u00b0C and partly cloudy \u2014 the default catchall renderer displayed the tool card above."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering-default-catchall \u2014 weather in Tokyo (first turn). Emits get_weather tool call that the built-in default catchall renderer handles.",
|
||||
"match": {
|
||||
"userMessage": "forecast for Tokyo",
|
||||
"toolName": "get_weather",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d6_dc_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering-default-catchall \u2014 weather in Tokyo fallback when no get_weather tool registered.",
|
||||
"match": {
|
||||
"userMessage": "forecast for Tokyo",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The weather in Tokyo is currently 22\u00b0C with partly cloudy skies and light easterly winds."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,197 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / tool-rendering-reasoning-chain",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "tool-rendering-reasoning-chain pill 1 (stocks) — final narration after the MSFT tool result lands. Specific toolCallId so this matches BEFORE the first-leg fixture below despite the same userMessage substring. Source: showcase/harness/fixtures/d5/tool-rendering-reasoning-chain.json.",
|
||||
"match": {
|
||||
"userMessage": "Compare AAPL and MSFT stocks",
|
||||
"toolCallId": "call_rc_stock_msft_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "AAPL is at $338.37 (-2.96% on the day) while MSFT is at $412.18 (+1.08%). MSFT is outpacing AAPL by roughly 4 points today — strong day for MSFT, rough one for AAPL."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering-reasoning-chain pill 1 (stocks) — second leg: after get_stock_price(AAPL) returns, chain to MSFT for the comparison.",
|
||||
"match": {
|
||||
"userMessage": "Compare AAPL and MSFT stocks",
|
||||
"toolCallId": "call_rc_stock_aapl_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "AAPL quote is in hand. The user explicitly asked to compare AAPL with MSFT, so I'll fetch MSFT next and then summarize the side-by-side.",
|
||||
"content": "Now pulling MSFT to complete the comparison.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_rc_stock_msft_001",
|
||||
"name": "get_stock_price",
|
||||
"arguments": "{\"ticker\":\"MSFT\",\"price_usd\":412.18,\"change_pct\":1.08}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering-reasoning-chain pill 1 (stocks) — first leg: emit get_stock_price(AAPL). The `Compare AAPL and MSFT stocks` substring is unique to this demo's pill prompt — other integrations' reasoning-chain demos still send the older `How is AAPL doing?` prompt — so we don't need a tool-name gate and the fixture stays scoped to langgraph-python without affecting fleet-wide aimock traffic. MUST appear before the bare 'AAPL' fixtures later in this file.",
|
||||
"match": {
|
||||
"userMessage": "Compare AAPL and MSFT stocks",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "The user asked to compare AAPL and MSFT. I'll fetch AAPL first, then MSFT, then summarize the deltas in a single sentence so the comparison is the punchline.",
|
||||
"content": "Pulling the AAPL quote first.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_rc_stock_aapl_001",
|
||||
"name": "get_stock_price",
|
||||
"arguments": "{\"ticker\":\"AAPL\",\"price_usd\":338.37,\"change_pct\":-2.96}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering-reasoning-chain pill 2 (dice) — final narration after the d6 contrast roll lands. userMessage is the LANGGRAPH-PYTHON-UNIQUE tail of the pill prompt ('compare it to a smaller one'); other integrations' reasoning-chain demos still send the older 'Roll a 20-sided die for me.' (no period-after-die substring match against 'Roll a 20-sided die.') which lacks this suffix — so this fixture stays scoped to this demo and does NOT hijack the older 5x roll_d20 fixtures further down in this file.",
|
||||
"match": {
|
||||
"userMessage": "compare it to a smaller one",
|
||||
"toolCallId": "call_rc_dice_d6_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The d20 came up 14, and a d6 for contrast landed on 4 — the d20's range is much wider, which is the whole point of the comparison."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering-reasoning-chain pill 2 (dice) — second leg: after roll_dice(sides=20) returns, chain a smaller die for contrast.",
|
||||
"match": {
|
||||
"userMessage": "compare it to a smaller one",
|
||||
"toolCallId": "call_rc_dice_d20_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "Got the d20 result. The user explicitly asked to compare it to a smaller die — a d6 is a natural choice because its 1-6 range is what most people picture when they think 'die'. Rolling that next.",
|
||||
"content": "Now rolling a d6 for contrast.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_rc_dice_d6_001",
|
||||
"name": "roll_dice",
|
||||
"arguments": "{\"sides\":6}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering-reasoning-chain pill 2 (dice) — first leg: emit roll_dice(sides=20). Unique substring 'compare it to a smaller one' keeps us scoped to this demo (other integrations' dice pills still send 'Roll a 20-sided die for me.' and fall through to the older 5x roll_d20 fixtures further down). NB: aimock's `toolName` gate is a tool-LIST gate, not a tool-CALL gate; we don't use it here because the reasoning-chain agents across the fleet all register `roll_dice`, so a `toolName: roll_dice` claim would NOT have distinguished this demo from the others.",
|
||||
"match": {
|
||||
"userMessage": "compare it to a smaller one",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "The user asked for a d20 roll and wants to compare it against a smaller die. I'll roll the d20 first, then chain a smaller die so the contrast in possible-value ranges is concrete and observable.",
|
||||
"content": "Rolling the d20 now.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_rc_dice_d20_001",
|
||||
"name": "roll_dice",
|
||||
"arguments": "{\"sides\":20}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering-reasoning-chain pill 3 (flights + destination weather) — final narration after get_weather(JFK) lands. userMessage is the LANGGRAPH-PYTHON-UNIQUE tail 'show me the weather there'; the basic tool-rendering demos AND every other integration's reasoning-chain demo still send 'Find flights from SFO to JFK.' (no destination-weather request) which lacks this substring, so this fixture stays scoped to this demo without affecting fleet-wide aimock traffic.",
|
||||
"match": {
|
||||
"userMessage": "show me the weather there",
|
||||
"toolCallId": "call_rc_weather_jfk_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Three flights from SFO to JFK — United UA231 at 08:15 ($348), Delta DL412 at 11:20 ($312), and JetBlue B6722 at 17:05 ($289). JFK is currently 68°F and sunny — easy travel weather on the receiving end."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering-reasoning-chain pill 3 (flights + destination weather) — second leg: after search_flights(SFO,JFK) returns, chain to destination weather.",
|
||||
"match": {
|
||||
"userMessage": "show me the weather there",
|
||||
"toolCallId": "call_rc_flights_jfk_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "Flights are in hand. The user explicitly asked for the destination weather as part of the request, so pulling JFK weather next to round out the trip plan.",
|
||||
"content": "Pulling JFK weather to round out the trip plan.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_rc_weather_jfk_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"JFK\"}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering-reasoning-chain pill 3 (flights + destination weather) — first leg: emit search_flights(SFO,JFK). The 'show me the weather there' substring is unique to this demo's pill, so the basic tool-rendering demo's bare 'Find flights from SFO to JFK.' prompt (and the equivalent prompts in other integrations' reasoning-chain demos that have not yet been ported to the chained phrasing) flow through to the basic single-tool flights fixture later in the file.",
|
||||
"match": {
|
||||
"userMessage": "show me the weather there",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "The user wants flights from SFO to JFK AND the destination weather. I'll call search_flights first, then chain get_weather on JFK so they have flight options and arrival conditions in one reply.",
|
||||
"content": "Searching SFO→JFK flights.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_rc_flights_jfk_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"origin\":\"SFO\",\"destination\":\"JFK\"}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering pill: Find flights — first leg (verbatim pill prompt, dedicated search_flights fixture). MUST take precedence over the a2ui beautiful-chat fixture below (which uses the same tool name with non-flight-list args shape). Intentionally omits `hasToolResult` so this fixture also matches when SFO/JFK is the SECOND turn of a multi-turn flow (e.g. tool-rendering-reasoning-chain probe sends weather→flights). With `hasToolResult: false`, Turn 1's tool result would prevent this fixture from matching Turn 2's first leg, and the matcher would fall through to the second-leg fixture above (which now requires a matching `toolCallId` so it cannot swallow this request).",
|
||||
"match": {
|
||||
"userMessage": "Find flights from SFO to JFK.",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_flights_sfo_jfk_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"origin\":\"SFO\",\"destination\":\"JFK\"}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
},
|
||||
{
|
||||
"_comment": "D5 mcp-apps probe — verbatim probe prompt drives a real `create_view` MCP tool call so the runtime's MCP Apps middleware fetches the UI resource and mounts the iframe. Mirrored in showcase/harness/fixtures/d5/mcp-apps.json. Non-empty `content` is load-bearing (see tool-rendering-reasoning-chain.json header).",
|
||||
"match": {
|
||||
"userMessage": "Open Excalidraw and sketch a system diagram",
|
||||
"toolName": "create_view",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"reasoning": "The user wants a sketch of a client/server/database architecture. I'll call create_view once with three labelled rectangles connected by arrows and a title, framed by a cameraUpdate.",
|
||||
"content": "Sketching a client → server → database diagram in Excalidraw.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_mcp_apps_create_view_001",
|
||||
"name": "create_view",
|
||||
"arguments": "{\"elements\":[{\"id\":\"title\",\"type\":\"text\",\"x\":260,\"y\":40,\"text\":\"System Diagram\",\"fontSize\":24},{\"id\":\"client\",\"type\":\"rectangle\",\"x\":80,\"y\":160,\"width\":160,\"height\":70,\"label\":{\"text\":\"Client\",\"fontSize\":18}},{\"id\":\"server\",\"type\":\"rectangle\",\"x\":320,\"y\":160,\"width\":160,\"height\":70,\"label\":{\"text\":\"Server\",\"fontSize\":18}},{\"id\":\"database\",\"type\":\"rectangle\",\"x\":560,\"y\":160,\"width\":160,\"height\":70,\"label\":{\"text\":\"Database\",\"fontSize\":18}},{\"id\":\"a1\",\"type\":\"arrow\",\"x\":240,\"y\":195,\"endX\":320,\"endY\":195},{\"id\":\"a2\",\"type\":\"arrow\",\"x\":480,\"y\":195,\"endX\":560,\"endY\":195},{\"id\":\"camera\",\"type\":\"cameraUpdate\",\"x\":40,\"y\":0,\"width\":800,\"height\":600}]}"
|
||||
}
|
||||
]
|
||||
},
|
||||
"chunkSize": 9999
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,465 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / tool-rendering",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "tool-rendering pill: Chain tools — follow-up after all 3 tools ran. Matches whichever of the 3 chain-tools tool_call_ids appears last in the request (LangGraph's ToolNode preserves tool_calls order, so roll_d20 is typically last; we register all 3 for safety). MUST come before the toolCalls-emitting fixture below so iteration 2 of the chain-tools loop hits this branch instead of re-emitting.",
|
||||
"match": {
|
||||
"userMessage": "Chain a few tools in this single turn",
|
||||
"toolCallId": "call_tr_chain_roll_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — Tokyo is sunny, three flights found, and the d20 came up 11."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Chain a few tools in this single turn",
|
||||
"toolCallId": "call_tr_chain_flights_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — Tokyo is sunny, three flights found, and the d20 came up 11."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Chain a few tools in this single turn",
|
||||
"toolCallId": "call_tr_chain_weather_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — Tokyo is sunny, three flights found, and the d20 came up 11."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering pill: Chain tools — emit 3 tool calls in one assistant turn (get_weather Tokyo + search_flights SFO->Tokyo + roll_d20=11). MUST appear before the bare 'weather in Tokyo' fixture below; substring match would otherwise leak into this prompt. No hasToolResult / turnIndex gate: in multi-pill demo sessions prior clicks leave tool results AND additional user turns in the thread, which previously caused this fixture to be skipped (turnIndex no longer 0 after sibling pills, hasToolResult breaks once prior pills emitted tool results) — the chain-tools toolCallId fixtures above already eat iteration 2 via last-message tool_call_id gating, so removing the turnIndex gate is safe.",
|
||||
"match": {
|
||||
"userMessage": "Chain a few tools in this single turn",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_chain_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
},
|
||||
{
|
||||
"id": "call_tr_chain_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"origin\":\"SFO\",\"destination\":\"Tokyo\"}"
|
||||
},
|
||||
{
|
||||
"id": "call_tr_chain_roll_001",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":11}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering pill: Weather in SF — follow-up content after get_weather tool ran. MUST come before the tool-emitting fixture below (first-match-wins) so iteration 2 of the loop hits this branch instead of re-emitting. toolCallId chain keeps the fixture stateless across multi-pill thread history (hasToolResult breaks when a prior pill left tool results in the thread).",
|
||||
"match": {
|
||||
"userMessage": "What's the weather in San Francisco?",
|
||||
"toolCallId": "call_tr_weather_sf_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "San Francisco is currently 68°F and sunny with light winds."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "What's the weather in San Francisco?",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_weather_sf_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"San Francisco\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering pill: Find flights — second leg (after search_flights tool result). MUST come BEFORE the first-leg fixture below — the matcher is first-match-wins, and the second leg is uniquely identified by `toolCallId` (last message is a tool with this id), so it cannot accidentally swallow the first-leg request (whose last message is the user prompt). Must also take precedence over the a2ui beautiful-chat fixture below (which uses the same tool name with non-flight-list args shape).",
|
||||
"match": {
|
||||
"userMessage": "Find flights from SFO to JFK.",
|
||||
"toolCallId": "call_tr_flights_sfo_jfk_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Three flights from SFO to JFK — United UA231 at 08:15 ($348), Delta DL412 at 11:20 ($312), and JetBlue B6722 at 17:05 ($289)."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Find flights — first leg: emit search_flights tool call. Gated on toolName:search_flights (same pattern as the sibling weather/AAPL/chain first-leg fixtures) rather than hasToolResult:false. The D6 all-pills probe clicks 'Find flights' alongside other pills, so prior clicks (e.g. 'weather in SF') may already have left a tool result in the thread; the aimock router implements hasToolResult as messages.some(m=>m.role==='tool'), so a hasToolResult:false gate could never match (→ no_fixture_match → 503 → 30s timeout). toolName fires whenever search_flights is registered (always, here) regardless of prior tool results. The toolCallId follow-up fixture above still wins on iteration 2 (last message is tool with id call_tr_flights_sfo_jfk_001), so this fixture only emits on the first leg.",
|
||||
"match": {
|
||||
"userMessage": "Find flights from SFO to JFK.",
|
||||
"toolName": "search_flights",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_flights_sfo_jfk_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"origin\":\"SFO\",\"destination\":\"JFK\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering pill: Stock price — follow-up content after get_stock_price tool ran. MUST come before the tool-emitting fixture below (first-match-wins) so iteration 2 of the loop hits this branch instead of re-emitting an infinite loop of tool calls.",
|
||||
"match": {
|
||||
"userMessage": "What's the current price of AAPL?",
|
||||
"toolCallId": "call_tr_stock_aapl_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "AAPL is trading at $338.37, down 2.96% on the day."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "AAPL — first leg: emit get_stock_price tool call. Gated on toolName:get_stock_price (same pattern as the sibling weather/chain first-leg fixtures) rather than hasToolResult:false. The D5 tool-rendering-custom-catchall probe runs 'weather in Tokyo' first, which leaves a get_weather tool result in the thread; the aimock router implements hasToolResult as messages.some(m=>m.role==='tool'), so hasToolResult is permanently true on the AAPL turn and a hasToolResult:false gate could never match (→ no_fixture_match → 503 → 30s timeout). toolName fires whenever get_stock_price is registered (always, here) regardless of prior tool results. The toolCallId follow-up fixture above still wins on iteration 2 (last message is tool with id call_tr_stock_aapl_001), so this fixture only emits on the first leg.",
|
||||
"match": {
|
||||
"userMessage": "What's the current price of AAPL?",
|
||||
"toolName": "get_stock_price",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_stock_aapl_001",
|
||||
"name": "get_stock_price",
|
||||
"arguments": "{\"ticker\":\"AAPL\",\"price_usd\":338.37,\"change_pct\":-2.96}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "tool-rendering pill: Roll a d20 — exactly 5 sequential roll_d20 calls returning [7, 14, 3, 19, 20]. Chained by toolCallId so the sequence is stateless across thread history (turnIndex/hasToolResult break in multi-pill demo sessions where prior clicks leave assistant/tool messages in the thread). Specific-toolCallId fixtures MUST come before the userMessage-only fixture below; first-match-wins.",
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"toolCallId": "call_tr_d20_seq_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_d20_seq_002",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":14}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"toolCallId": "call_tr_d20_seq_002",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_d20_seq_003",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":3}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"toolCallId": "call_tr_d20_seq_003",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_d20_seq_004",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":19}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"toolCallId": "call_tr_d20_seq_004",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_d20_seq_005",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":20}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"toolCallId": "call_tr_d20_seq_005",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Rolled the d20 five times — landed on 20 on the final roll."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "First roll. Matches the initial user prompt (no prior d20 tool result in this chain yet). Comes after the toolCallId-chained fixtures above so iterations 2-6 of the loop hit those first. turnIndex dropped: the multi-pill sequential e2e test clicks 'Find flights' first which leaves prior turns in the thread, so a new 'Roll a 20-sided die.' user message is no longer at turnIndex 0. The toolCallId-chained fixtures still take precedence for iterations 2-6 because their last-message gate (role=tool with the chained id) only matches mid-chain — this fixture only matches when last-message.role=user.",
|
||||
"match": {
|
||||
"userMessage": "Roll a 20-sided die.",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_tr_d20_seq_001",
|
||||
"name": "roll_d20",
|
||||
"arguments": "{\"value\":7}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Follow-up content after get_weather (or get-weather Mastra alias) ran for Tokyo. Keyed on the prior tool's id so it fires after iteration 1 regardless of thread history. Must come BEFORE the tool-emitting fixtures so iteration 2 hits this branch instead of re-emitting get_weather.",
|
||||
"match": {
|
||||
"userMessage": "weather in Tokyo",
|
||||
"toolCallId": "call_d5_get_weather_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Tokyo is 22°C and partly cloudy."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "weather in Tokyo",
|
||||
"toolName": "get_weather",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_get_weather_001",
|
||||
"name": "get_weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
],
|
||||
"reasoning": "The user asked about Tokyo weather. I'll call get_weather with location='Tokyo' to get the current conditions.",
|
||||
"content": "Looking up the weather in Tokyo for you."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Mastra registers the weather tool as get-weather (hyphen); duplicate for compat",
|
||||
"match": {
|
||||
"userMessage": "weather in Tokyo",
|
||||
"toolName": "get-weather",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_get_weather_001",
|
||||
"name": "get-weather",
|
||||
"arguments": "{\"location\":\"Tokyo\"}"
|
||||
}
|
||||
],
|
||||
"reasoning": "The user asked about Tokyo weather. I'll call get_weather with location='Tokyo' to get the current conditions.",
|
||||
"content": "Looking up the weather in Tokyo for you."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "Final fallback when the agent has neither get_weather nor get-weather registered — return narrated content with no tool call. Comes last so the tool-emitting fixtures above win when the tool IS available.",
|
||||
"match": {
|
||||
"userMessage": "weather in Tokyo",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The weather in Tokyo is currently 22°C with partly cloudy skies and light easterly winds."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "AAPL",
|
||||
"toolCallId": "call_d5_get_stock_price_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "AAPL is trading at $189.42, up 1.27% on the day. The card above shows the live ticker and the percentage change."
|
||||
}
|
||||
},
|
||||
{
|
||||
"_comment": "AAPL — first leg: emit get_stock_price tool call. turnIndex:0 gate REMOVED — the D5 tool-rendering-custom-catchall probe runs 'weather in Tokyo' first then sends 'What's the current price of AAPL?', so the AAPL leg fires at turnIndex>=2 (multi-pill thread). Replaced turnIndex:0 with hasToolResult:false: matches when the last message is a user prompt (not a tool result), which keeps the first-leg fixture from re-firing after the follow-up content already lands. The toolCallId follow-up fixture above wins on iteration 2 (last message is tool with id call_d5_get_stock_price_001).",
|
||||
"match": {
|
||||
"userMessage": "AAPL",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_get_stock_price_001",
|
||||
"name": "get_stock_price",
|
||||
"arguments": "{\"ticker\":\"AAPL\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: search flights from SFO to JFK",
|
||||
"hasToolResult": false,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_bc_search_flights_001",
|
||||
"name": "search_flights",
|
||||
"arguments": "{\"flights\":[{\"airline\":\"United Airlines\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=united.com&sz=128\",\"flightNumber\":\"UA123\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"08:00\",\"arrivalTime\":\"16:30\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"price\":\"$349\"},{\"airline\":\"Delta\",\"airlineLogo\":\"https://www.google.com/s2/favicons?domain=delta.com&sz=128\",\"flightNumber\":\"DL456\",\"origin\":\"SFO\",\"destination\":\"JFK\",\"date\":\"Tue, Apr 15\",\"departureTime\":\"10:15\",\"arrivalTime\":\"18:45\",\"duration\":\"5h 30m\",\"status\":\"On Time\",\"price\":\"$289\"}]}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "d5 beautiful-chat probe: search flights from SFO to JFK",
|
||||
"hasToolResult": true,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Two flights shown above — United at $349 (08:00) and Delta at $289 (10:15), both on time."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "SFO to JFK",
|
||||
"toolCallId": "call_d5_display_flight_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Flight rendered. Tap 'Book flight' to confirm."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "SFO to JFK",
|
||||
"toolName": "display_flight",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Here is the SFO to JFK flight on United.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"name": "display_flight",
|
||||
"arguments": {
|
||||
"origin": "SFO",
|
||||
"destination": "JFK",
|
||||
"airline": "United",
|
||||
"price": "$289"
|
||||
},
|
||||
"id": "call_d5_display_flight_001"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "poem about autumn leaves",
|
||||
"toolCallId": "call_d5_write_document_poem_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — the poem has been written into the shared document state."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "poem about autumn leaves",
|
||||
"toolName": "write_document",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Streaming the poem now.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_write_document_poem_001",
|
||||
"name": "write_document",
|
||||
"arguments": "{\"document\":\"Crimson and amber in slow descent, / each leaf a quiet ledger of summer spent. / The wind, a courier with nothing to say, / files them gently into the morning's gray. / Somewhere a kettle hums, and afternoons grow brief — / autumn keeps its books in vermilion and gold leaf.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "polite email declining",
|
||||
"toolCallId": "call_d5_write_document_email_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — the decline-email draft has been written into the shared document state."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "polite email declining",
|
||||
"toolName": "write_document",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Drafting the email now.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_write_document_email_001",
|
||||
"name": "write_document",
|
||||
"arguments": "{\"document\":\"Hi — thanks for sending the invite for Tuesday afternoon. Unfortunately I won't be able to make it this week. I'd love to find time later in the month if your schedule allows. In the meantime, feel free to send any pre-reads my way and I'll review them async so we don't lose momentum. Best, [name]\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "quantum computing for a curious teenager",
|
||||
"toolCallId": "call_d5_write_document_quantum_001",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Done — the quantum-computing explainer has been written into the shared document state."
|
||||
}
|
||||
},
|
||||
{
|
||||
"match": {
|
||||
"userMessage": "quantum computing for a curious teenager",
|
||||
"toolName": "write_document",
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "Streaming the explainer now.",
|
||||
"toolCalls": [
|
||||
{
|
||||
"id": "call_d5_write_document_quantum_001",
|
||||
"name": "write_document",
|
||||
"arguments": "{\"document\":\"A regular computer stores information in bits — tiny switches that are either on (1) or off (0). A quantum computer uses qubits, which can sit in a fuzzy superposition of both states at once until you check them. Stack many qubits together and they can explore lots of possibilities in parallel, which is why people are excited.\\n\\nThis doesn't make quantum computers faster at everything. They're great at problems with hidden structure — like factoring big numbers, simulating molecules, or searching certain databases — but useless for, say, opening Excel. Today's machines are noisy and small, so we mostly use them to test ideas rather than replace your laptop.\"}"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
{
|
||||
"_meta": {
|
||||
"description": "D6 fixtures for langgraph-python / voice",
|
||||
"sourceFile": "d5-all.json",
|
||||
"created": "2026-05-21"
|
||||
},
|
||||
"fixtures": [
|
||||
{
|
||||
"_comment": "Voice probe fast-path: exact-match content-only response.",
|
||||
"match": {
|
||||
"userMessage": "What is the weather in Tokyo?",
|
||||
"turnIndex": 0,
|
||||
"context": "langgraph-python"
|
||||
},
|
||||
"response": {
|
||||
"content": "The weather in Tokyo is currently 22°C with partly cloudy skies and light easterly winds."
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user