chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,61 @@
|
||||
# Guided Conversations
|
||||
This sample highlights a framework for a pattern of use cases we refer to as guided conversations.
|
||||
These are scenarios where an agent with a goal and constraints leads a conversation. There are many of these scenarios where we hold conversations that are driven by an objective and constraints. For example:
|
||||
- a teacher guiding a student through a lesson
|
||||
- a call center representative collecting information about a customer's issue
|
||||
- a sales representative helping a customer find the right product for their specific needs
|
||||
- an interviewer asking candidate a series of questions to assess their fit for a role
|
||||
- a nurse asking a series of questions to triage the severity of a patient's symptoms
|
||||
- a meeting where participants go around sharing their updates and discussing next steps
|
||||
|
||||
The common thread between all these scenarios is that they are between a **creator** leading the conversation and a **user(s)** who are participating.
|
||||
The creator defines the goals, a plan for how the conversation should flow, and often collects key information through a form throughout the conversation.
|
||||
They must exercise judgment to navigate and adapt the conversation towards achieving the set goal all while writing down key information and planning in advance.
|
||||
|
||||
The goal of this framework is to show how we can build a common framework to create AI agents that can assist a creator in running conversational scenarios semi-autonomously and generating **artifacts** like notes, forms, and plans that can be used to track progress and outcomes. A key tenant of this framework is the following principal: *think with the model, plan with the code*. This means that the model is used to understand user inputs and make complex decisions, but code is used to apply constraints and provide structure to make the system **reliable**. To better understand this concept, start with the [notebooks](./notebooks/).
|
||||
|
||||
|
||||
## Features
|
||||
We were motivated to create this sample while noticing some common challenges with using agents for conversation scenarios:
|
||||
| Common Challenges | Guided Conversations |
|
||||
| --------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Focus - Drift from their original goals | Define the agent's goal in terms of completing an ["artifact"](./guided_conversation/plugins/artifact.py), which is a precise representation of what the agent needs to do in the conversation |
|
||||
| Pacing - Rushing through conversations, being overly verbose, and struggle to understand time | Encourage the agent to regularly update an [agenda](./guided_conversation/plugins/agenda.py) where each agenda item is allocated an estimated number of times, time limits are programmatically validated, and programmatically convert time-based units (e.g. seconds, minutes) to turns using [resource constraints](./guided_conversation/utils/resources.py) |
|
||||
| Downstream Use Cases - Difficult to use chat logs for further processing or analysis | The [artifact](./guided_conversation/plugins/artifact.py) serves as (1) a structured record of the conversation that can be more easily analyzed afterward, (2) a way to monitor the agent's progress in real-time |
|
||||
|
||||
|
||||
## Installation
|
||||
This sample uses the same tooling as the [Semantic Kernel](https://github.com/microsoft/semantic-kernel/blob/main/python/pyproject.toml) Python source which uses [poetry](https://python-poetry.org/docs/) to install dependencies for development.
|
||||
|
||||
1. `poetry install`
|
||||
1. Activate `.venv` that was created by poetry
|
||||
1. Set up the environment variables or a `.env` file for the LLM service you want to use.
|
||||
1. If you add new dependencies to the `pyproject.toml` file; run `poetry update`.
|
||||
|
||||
|
||||
### Quickstart
|
||||
1. Fork the repository.
|
||||
1. Install dependencies (see Installation) & set up environment variables
|
||||
1. Try the [01_guided_conversation_teaching.ipynb](./notebooks/01_guided_conversation_teaching.ipynb) as an example.
|
||||
1. For best quality and reliability, we recommend using the `gpt-4-1106-preview` or `gpt-4o` models since this sample requires complex reasoning and function calling abilities.
|
||||
|
||||
|
||||
## How You Can Use This Framework
|
||||
### Add a new scenario
|
||||
Create a new file and and define the following inputs:
|
||||
- An artifact
|
||||
- Rules
|
||||
- Conversation flow (optional)
|
||||
- Context (optional)
|
||||
- Resource constraint (optional)
|
||||
|
||||
See the [interactive script](./interactive_guided_conversation.py) for an example.
|
||||
|
||||
### Editing Existing Plugins
|
||||
Edit plugins at [plugins](./guided_conversation/plugins/)
|
||||
|
||||
### Editing the Orchestrator
|
||||
Go to [guided_conversation_agent.py](./guided_conversation/plugins/guided_conversation_agent.py).
|
||||
|
||||
### Reusing Plugins
|
||||
We also encourage the open source community to pull in the artifact and agenda plugins to accelerate existing work. We believe that these plugins alone can improve goal-following in other agents.
|
||||
@@ -0,0 +1 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
@@ -0,0 +1 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
+229
@@ -0,0 +1,229 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import logging
|
||||
|
||||
from semantic_kernel import Kernel
|
||||
from semantic_kernel.functions import FunctionResult, KernelArguments
|
||||
|
||||
from guided_conversation.plugins.agenda import Agenda
|
||||
from guided_conversation.plugins.artifact import Artifact
|
||||
from guided_conversation.utils.conversation_helpers import Conversation
|
||||
from guided_conversation.utils.resources import GCResource, ResourceConstraintMode
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
conversation_plan_template = """<message role="system">You are a helpful, thoughtful, and meticulous assistant.
|
||||
You are conducting a conversation with a user. \
|
||||
Your goal is to complete an artifact as thoroughly as possible by the end of the conversation, and to ensure a smooth experience for the user.
|
||||
|
||||
This is the schema of the artifact you are completing:
|
||||
{{ artifact_schema }}{{#if context}}
|
||||
|
||||
Here is some additional context about the conversation:
|
||||
{{ context }}{{/if}}
|
||||
|
||||
Throughout the conversation, you must abide by these rules:
|
||||
{{ rules }}{{#if current_state_description }}
|
||||
|
||||
Here's a description of the conversation flow:
|
||||
{{ current_state_description }}
|
||||
Follow this description, and exercise good judgment about when it is appropriate to deviate.{{/if}}
|
||||
|
||||
You will be provided the history of your conversation with the user up until now and the current state of the artifact.
|
||||
Note that if the value for a field in the artifact is 'Unanswered', it means that the field has not been completed.
|
||||
You need to select the best possible action(s), given the state of the conversation and the artifact.
|
||||
These are the possible actions you can take:
|
||||
{{#if show_agenda}}Update agenda (required parameters: items)
|
||||
- If the latest agenda is set to "None", you should always pick this action.
|
||||
- You should pick this action if you need to change your plan for the conversation to make the best use of the remaining turns available to you. \
|
||||
Consider how long it usually takes to get the information you need (which is a function of the quality and pace of the user's responses), \
|
||||
the number, complexity, and importance of the remaining fields in the artifact, and the number of turns remaining ({{ remaining_resource }}). \
|
||||
Based on these factors, you might need to accelerate (e.g. combine several topics) or slow down the conversation (e.g. spread out a topic), in which case you should update the agenda accordingly. \
|
||||
Note that skipping an artifact field is NOT a valid way to accelerate the conversation.
|
||||
- You must provide an ordered list of items to be completed sequentially, where the first item contains everything you will do in the current turn of the conversation (in addition to updating the agenda). \
|
||||
For example, if you choose to send a message to the user asking for their name and medical history, then you would write "ask for name and medical history" as the first item. \
|
||||
If you think medical history will take longer than asking for the name, then you would write "complete medical history" as the second item, with an estimate of how many turns you think it will take. \
|
||||
Do NOT include items that have already been completed. \
|
||||
Items must always represent a conversation topic (corresponding to the "Send message to user" action). Updating the artifact (e.g. "update field X based on the discussion") or terminating the conversation is NOT a valid item.
|
||||
- The latest agenda was created in the previous turn of the conversation. \
|
||||
Even if the total turns in the latest agenda equals the remaining turns, you should still update the agenda if you think the current plan is suboptimal (e.g. the first item was completed, the order of items is not ideal, an item is too broad or not a conversation topic, etc.).
|
||||
- Each item must have a description and and your best guess for the number of turns required to complete it. Do not provide a range of turns. \
|
||||
It is EXTREMELY important that the total turns allocated across all items in the updated agenda (including the first item for the current turn) {{ total_resource_str }} \
|
||||
Everything in the agenda should be something you expect to complete in the remaining turns - there shouldn't be any optional "buffer" items. \
|
||||
It can be helpful to include the cumulative turns allocated for each item in the agenda to ensure you adhere to this rule, e.g. item 1 = 2 turns (cumulative total = 2), item 2 = 4 turns (cumulative total = 6), etc.
|
||||
- Avoid high-level items like "ask follow-up questions" - be specific about what you need to do.
|
||||
- Do NOT include wrap-up items such as "review and confirm all information with the user" (you should be doing this throughout the conversation) or "thank the user for their time". \
|
||||
Do NOT repeat topics that have already been sufficiently addressed. {{ ample_time_str }}{{/if}}
|
||||
|
||||
Send message to user (required parameters: message)
|
||||
- If there is no conversation history, you should always pick this action.
|
||||
- You should pick this action if (a) the user asked a question or made a statement that you need to respond to, \
|
||||
or (b) you need to follow-up with the user because the information they provided is incomplete, invalid, ambiguous, or in some way insufficient to complete the artifact. \
|
||||
For example, if the artifact schema indicates that the "date of birth" field must be in the format "YYYY-MM-DD", but the user has only provided the month and year, you should send a message to the user asking for the day. \
|
||||
Likewise, if the user claims that their date of birth is February 30, you should send a message to the user asking for a valid date. \
|
||||
If the artifact schema is open-ended (e.g. it asks you to rate how pressing the user's issue is, without specifying rules for doing so), use your best judgment to determine whether you have enough information or you need to continue probing the user. \
|
||||
It's important to be thorough, but also to avoid asking the user for unnecessary information.
|
||||
|
||||
Update artifact fields (required parameters: field, value)
|
||||
- You should pick this action as soon as (a) the user provides new information that is not already reflected in the current state of the artifact and (b) you are able to submit a valid value for a field in the artifact using this new information. \
|
||||
If you have already updated a field in the artifact and there is no new information to update the field with, you should not pick this action.
|
||||
- Make sure the value adheres to the constraints of the field as specified in the artifact schema.
|
||||
- If the user has provided all required information to complete a field (i.e. the criteria for "Send message to user" are not satisfied) but the information is in the wrong format, you should not ask the user to reformat their response. \
|
||||
Instead, you should simply update the field with the correctly formatted value. For example, if the artifact asks for the date of birth in the format "YYYY-MM-DD", and the user provides their date of birth as "June 15, 2000", you should update the field with the value "2000-06-15".
|
||||
- Prioritize accuracy over completion. You should never make up information or make assumptions in order to complete a field. \
|
||||
For example, if the field asks for a 10-digit phone number, and the user provided a 9-digit phone number, you should not add a digit to the phone number in order to complete the field. \
|
||||
Instead, you should follow-up with the user to ask for the correct phone number. If they still aren't able to provide one, you should leave the field unanswered.
|
||||
- If the user isn't able to provide all of the information needed to complete a field, \
|
||||
use your best judgment to determine if a partial answer is appropriate (assuming it adheres to the formatting requirements of the field). \
|
||||
For example, if the field asks for a description of symptoms along with details about when the symptoms started, but the user isn't sure when their symptoms started, \
|
||||
it's better to record the information they do have rather than to leave the field unanswered (and to indicate that the user was unsure about the start date).
|
||||
- If it's possible to update multiple fields at once (assuming you're adhering to the above rules in all cases), you should do so. \
|
||||
For example, if the user provides their full name and date of birth in the same message, you should select the "update artifact fields" action twice, once for each field.
|
||||
|
||||
End conversation (required parameters: None)
|
||||
{{ termination_instructions }}
|
||||
{{ resource_instructions }}
|
||||
|
||||
If you select the "Update artifact field" action or the "Update agenda" action, you should also select one of the "Send message to user" or "End conversation" actions. \
|
||||
Note that artifact and updates updates will always be executed before a message is sent to the user or the conversation is terminated. \
|
||||
Also note that only one message can be sent to the user at a time.
|
||||
|
||||
Your task is to state your step-by-step reasoning for the best possible action(s), followed by a final recommendation of which action(s) to take, including all required parameters.
|
||||
Someone else will be responsible for executing the action(s) you select and they will only have access to your output \
|
||||
(not any of the conversation history, artifact schema, or other context) so it is EXTREMELY important \
|
||||
that you clearly specify the value of all required parameters for each action you select.</message>
|
||||
|
||||
<message role="user">Conversation history:
|
||||
{{ chat_history }}
|
||||
|
||||
Latest agenda:
|
||||
{{ agenda_state }}
|
||||
|
||||
Current state of the artifact:
|
||||
{{ artifact_state }}</message>"""
|
||||
|
||||
|
||||
async def conversation_plan_function(
|
||||
kernel: Kernel,
|
||||
chat_history: Conversation,
|
||||
context: str,
|
||||
rules: list[str],
|
||||
conversation_flow: str,
|
||||
current_artifact: Artifact,
|
||||
req_settings: dict,
|
||||
resource: GCResource,
|
||||
agenda: Agenda,
|
||||
) -> FunctionResult:
|
||||
"""Reasons/plans about the next best action(s) to continue the conversation. In this function, a DESCRIPTION of the possible actions
|
||||
are surfaced to the agent. Note that the agent will not execute the actions, but will provide a step-by-step reasoning for the best
|
||||
possible action(s). The implication here is that NO tool/plugin calls are made, only a description of what tool calls might be called
|
||||
is created.
|
||||
|
||||
Currently, the reasoning/plan from this function is passed to another function (which leverages openai tool calling) that will execute
|
||||
the actions.
|
||||
|
||||
Args:
|
||||
kernel (Kernel): The kernel object.
|
||||
chat_history (Conversation): The conversation history
|
||||
context (str): Creator provided context of the conversation
|
||||
rules (list[str]): Creator provided rules
|
||||
conversation_flow (str): Creator provided conversation flow
|
||||
current_artifact (Artifact): The current artifact
|
||||
req_settings (dict): The request settings
|
||||
resource (GCResource): The resource object
|
||||
|
||||
Returns:
|
||||
FunctionResult: The function result.
|
||||
"""
|
||||
# clear any pre-existing tools from the request settings
|
||||
req_settings.tools = None
|
||||
req_settings.tool_choice = None
|
||||
|
||||
# clear any extension data
|
||||
if hasattr(req_settings, "extension_data"):
|
||||
req_settings.extension_data = {}
|
||||
|
||||
kernel_function = kernel.add_function(
|
||||
prompt=conversation_plan_template,
|
||||
function_name="conversation_plan_function",
|
||||
plugin_name="conversation_plan",
|
||||
template_format="handlebars",
|
||||
prompt_execution_settings=req_settings,
|
||||
)
|
||||
|
||||
remaining_resource = resource.remaining_units
|
||||
resource_instructions = resource.get_resource_instructions()
|
||||
|
||||
# if there is a resource constraint and there's more than one turn left, include the update agenda action
|
||||
if (resource_instructions != "") and (remaining_resource > 1):
|
||||
if resource.get_resource_mode() == ResourceConstraintMode.MAXIMUM:
|
||||
total_resource_str = f"does not exceed the remaining turns ({remaining_resource})."
|
||||
ample_time_str = ""
|
||||
elif resource.get_resource_mode() == ResourceConstraintMode.EXACT:
|
||||
total_resource_str = (
|
||||
f"is equal to the remaining turns ({remaining_resource}). Do not leave any turns unallocated."
|
||||
)
|
||||
ample_time_str = """If you have many turns remaining, instead of including wrap-up items or repeating topics, you should include items that increase the breadth and/or depth of the conversation \
|
||||
in a way that's directly relevant to the artifact (e.g. "collect additional details about X", "ask for clarification about Y", "explore related topic Z", etc.)."""
|
||||
else:
|
||||
logger.error("Invalid resource mode.")
|
||||
else:
|
||||
total_resource_str = ""
|
||||
ample_time_str = ""
|
||||
termination_instructions = _get_termination_instructions(resource)
|
||||
|
||||
# only include the agenda if there is a resource constraint and there's more than one turn left
|
||||
show_agenda = resource_instructions != "" and remaining_resource > 1
|
||||
|
||||
arguments = KernelArguments(
|
||||
context=context,
|
||||
artifact_schema=current_artifact.get_schema_for_prompt(),
|
||||
rules=" ".join([r.strip() for r in rules]),
|
||||
current_state_description=conversation_flow,
|
||||
show_agenda=show_agenda,
|
||||
remaining_resource=remaining_resource,
|
||||
total_resource_str=total_resource_str,
|
||||
ample_time_str=ample_time_str,
|
||||
termination_instructions=termination_instructions,
|
||||
resource_instructions=resource_instructions,
|
||||
chat_history=chat_history.get_repr_for_prompt(),
|
||||
agenda_state=agenda.get_agenda_for_prompt(),
|
||||
artifact_state=current_artifact.get_artifact_for_prompt(),
|
||||
)
|
||||
|
||||
result = await kernel.invoke(function=kernel_function, arguments=arguments)
|
||||
return result
|
||||
|
||||
|
||||
def _get_termination_instructions(resource: GCResource):
|
||||
"""
|
||||
Get the termination instructions for the conversation. This is contingent on the resources mode,
|
||||
if any, that is available.
|
||||
|
||||
Assumes we're always using turns as the resource unit.
|
||||
|
||||
Args:
|
||||
resource (GCResource): The resource object.
|
||||
|
||||
Returns:
|
||||
str: the termination instructions
|
||||
"""
|
||||
# Termination condition under no resource constraints
|
||||
if resource.resource_constraint is None:
|
||||
return "- You should pick this action as soon as you have completed the artifact to the best of your ability, \
|
||||
the conversation has come to a natural conclusion, or the user is not cooperating so you cannot continue the conversation."
|
||||
|
||||
# Termination condition under exact resource constraints
|
||||
if resource.resource_constraint.mode == ResourceConstraintMode.EXACT:
|
||||
return (
|
||||
"- You should only pick this action if the user is not cooperating so you cannot continue the conversation."
|
||||
)
|
||||
|
||||
# Termination condition under maximum resource constraints
|
||||
elif resource.resource_constraint.mode == ResourceConstraintMode.MAXIMUM:
|
||||
return "- You should pick this action as soon as you have completed the artifact to the best of your ability, \
|
||||
the conversation has come to a natural conclusion, or the user is not cooperating so you cannot continue the conversation."
|
||||
|
||||
else:
|
||||
logger.error("Invalid resource mode provided.")
|
||||
return ""
|
||||
@@ -0,0 +1,75 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
from typing import Annotated
|
||||
|
||||
from semantic_kernel import Kernel
|
||||
from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior
|
||||
from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings
|
||||
from semantic_kernel.functions import FunctionResult, KernelArguments
|
||||
from semantic_kernel.functions.kernel_function_decorator import kernel_function
|
||||
|
||||
execution_template = """<message role="system">You are a helpful, thoughtful, and meticulous assistant.
|
||||
You are conducting a conversation with a user. Your goal is to complete an artifact as thoroughly as possible by the end of the conversation.
|
||||
You will be given some reasoning about the best possible action(s) to take next given the state of the conversation as well as the artifact schema.
|
||||
The reasoning is supposed to state the recommended action(s) to take next, along with all required parameters for each action.
|
||||
Your task is to execute ALL actions recommended in the reasoning in the order they are listed.
|
||||
If the reasoning's specification of an action is incomplete (e.g. it doesn't include all required parameters for the action, \
|
||||
or some parameters are specified implicitly, such as "send a message that contains a greeting" instead of explicitly providing \
|
||||
the value of the "message" parameter), do not execute the action. You should never fill in missing or imprecise parameters yourself.
|
||||
If the reasoning is not clear about which actions to take, or all actions are specified in an incomplete way, \
|
||||
return 'None' without selecting any action.</message>
|
||||
|
||||
<message role="user">Artifact schema:
|
||||
{{ artifact_schema }}
|
||||
|
||||
If the type in the schema is str, the "field_value" parameter in the action should be also be a string.
|
||||
These are example parameters for the update_artifact action: {"field_name": "company_name", "field_value": "Contoso"}
|
||||
DO NOT write JSON in the "field_value" parameter in this case. {"field_name": "company_name", "field_value": "{"value": "Contoso"}"} is INCORRECT.
|
||||
|
||||
Reasoning:
|
||||
{{ reasoning }}</message>"""
|
||||
|
||||
|
||||
@kernel_function(name="send_message_to_user", description="Sends a message to the user.")
|
||||
def send_message(message: Annotated[str, "The message to send to the user."]) -> None:
|
||||
return None
|
||||
|
||||
|
||||
@kernel_function(name="end_conversation", description="Ends the conversation.")
|
||||
def end_conversation() -> None:
|
||||
return None
|
||||
|
||||
|
||||
async def execution(
|
||||
kernel: Kernel, reasoning: str, filter: list[str], req_settings: PromptExecutionSettings, artifact_schema: str
|
||||
) -> FunctionResult:
|
||||
"""Executes the actions recommended by the reasoning/planning call in the given context.
|
||||
|
||||
Args:
|
||||
kernel (Kernel): The kernel object.
|
||||
reasoning (str): The reasoning from a previous model call.
|
||||
filter (list[str]): The list of plugins to INCLUDE for the tool call.
|
||||
req_settings (PromptExecutionSettings): The prompt execution settings.
|
||||
artifact (str): The artifact schema for the execution prompt.
|
||||
|
||||
Returns:
|
||||
FunctionResult: The result of the execution.
|
||||
"""
|
||||
filter = {"included_plugins": filter}
|
||||
req_settings.function_choice_behavior = FunctionChoiceBehavior.Auto(auto_invoke=False, filters=filter)
|
||||
|
||||
kernel_function = kernel.add_function(
|
||||
prompt=execution_template,
|
||||
function_name="execution",
|
||||
plugin_name="execution",
|
||||
template_format="handlebars",
|
||||
prompt_execution_settings=req_settings,
|
||||
)
|
||||
|
||||
arguments = KernelArguments(
|
||||
artifact_schema=artifact_schema,
|
||||
reasoning=reasoning,
|
||||
)
|
||||
|
||||
result = await kernel.invoke(function=kernel_function, arguments=arguments)
|
||||
return result
|
||||
+99
@@ -0,0 +1,99 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
from semantic_kernel import Kernel
|
||||
from semantic_kernel.functions import FunctionResult, KernelArguments
|
||||
|
||||
from guided_conversation.utils.conversation_helpers import Conversation
|
||||
|
||||
final_update_template = """<message role="system">You are a helpful, thoughtful, and meticulous assistant.
|
||||
You just finished a conversation with a user.{{#if context}} Here is some additional context about the conversation:
|
||||
{{ context }}{{/if}}
|
||||
|
||||
Your goal is to complete an artifact as thoroughly and accurately as possible based on the conversation.
|
||||
|
||||
This is the schema of the artifact:
|
||||
{{ artifact_schema }}
|
||||
|
||||
You will be given the current state of the artifact as well as the conversation history.
|
||||
Note that if the value for a field in the artifact is 'Unanswered', it means that the field was not completed. \
|
||||
Some fields may have already been completed during the conversation.
|
||||
|
||||
Your need to determine whether there are any fields that need to be updated, and if so, update them.
|
||||
- You should only update a field if both of the following conditions are met: (a) the current state does NOT adequately reflect the conversation \
|
||||
and (b) you are able to submit a valid value for a field. \
|
||||
You are allowed to update completed fields, but you should only do so if the current state is inadequate, \
|
||||
e.g. the user corrected a mistake in their date of birth, but the artifact does not show the corrected version. \
|
||||
Remember that it's always an option to reset a field to "Unanswered" - this is often the best choice if the artifact contains incorrect information that cannot be corrected. \
|
||||
Do not submit a value that is identical to the current state of the field (e.g. if the field is already "Unanswered" and the user didn't provide any new information about it, you should not submit "Unanswered"). \
|
||||
- Make sure the value adheres to the constraints of the field as specified in the artifact schema. \
|
||||
If it's not possible to update a field with a valid value (e.g., the user provided an invalid date of birth), you should not update the field.
|
||||
- If the artifact schema is open-ended (e.g. it asks you to rate how pressing the user's issue is, without specifying rules for doing so), \
|
||||
use your best judgment to determine whether you have enough information to complete the field based on the conversation.
|
||||
- Prioritize accuracy over completion. You should never make up information or make assumptions in order to complete a field. \
|
||||
For example, if the field asks for a 10-digit phone number, and the user provided a 9-digit phone number, you should not add a digit to the phone number in order to complete the field.
|
||||
- If the user wasn't able to provide all of the information needed to complete a field, \
|
||||
use your best judgment to determine if a partial answer is appropriate (assuming it adheres to the formatting requirements of the field). \
|
||||
For example, if the field asks for a description of symptoms along with details about when the symptoms started, but the user wasn't sure when their symptoms started, \
|
||||
it's better to record the information they do have rather than to leave the field unanswered (and to indicate that the user was unsure about the start date).
|
||||
- It's possible to update multiple fields at once (assuming you're adhering to the above rules in all cases). It's also possible that no fields need to be updated.
|
||||
|
||||
Your task is to state your step-by-step reasoning about what to update, followed by a final recommendation.
|
||||
Someone else will be responsible for executing the updates and they will only have access to your output \
|
||||
(not any of the conversation history, artifact schema, or other context) so make sure to specify exactly which \
|
||||
fields to update and the values to update them with, or to state that no fields need to be updated.
|
||||
</message>
|
||||
|
||||
<message role="user">Conversation history:
|
||||
{{ conversation_history }}
|
||||
|
||||
Current state of the artifact:
|
||||
{{ artifact_state }}</message>"""
|
||||
|
||||
|
||||
async def final_update_plan_function(
|
||||
kernel: Kernel,
|
||||
req_settings: dict,
|
||||
chat_history: Conversation,
|
||||
context: str,
|
||||
artifact_schema: str,
|
||||
artifact_state: str,
|
||||
) -> FunctionResult:
|
||||
"""This function is responsible for updating the artifact based on the conversation history when the conversation ends. This function may not always update the artifact, namely if the current state of the artifact is already accurate based on the conversation history. The function will return a step-by-step reasoning about what to update, followed by a final recommendation. The final recommendation will specify exactly which fields to update and the values to update them with, or to state that no fields need to be updated.
|
||||
|
||||
|
||||
Args:
|
||||
kernel (Kernel): The kernel object.
|
||||
req_settings (dict): The prompt execution settings.
|
||||
chat_history (Conversation): The conversation history.
|
||||
context (str): The context of the conversation.
|
||||
artifact_schema (str): The schema of the artifact.
|
||||
artifact_state (str): The current state of the artifact.
|
||||
|
||||
Returns:
|
||||
FunctionResult: The result of the function (step-by-step reasoning about what to update in the artifact)
|
||||
"""
|
||||
req_settings.tools = None
|
||||
req_settings.tool_choice = None
|
||||
|
||||
# clear any extension data
|
||||
if hasattr(req_settings, "extension_data"):
|
||||
req_settings.extension_data = {}
|
||||
|
||||
kernel_function = kernel.add_function(
|
||||
prompt=final_update_template,
|
||||
function_name="final_update_plan_function",
|
||||
plugin_name="final_update_plan",
|
||||
template_format="handlebars",
|
||||
prompt_execution_settings=req_settings,
|
||||
)
|
||||
|
||||
arguments = KernelArguments(
|
||||
conversation_history=chat_history.get_repr_for_prompt(),
|
||||
context=context,
|
||||
artifact_schema=artifact_schema,
|
||||
artifact_state=artifact_state,
|
||||
)
|
||||
|
||||
result = await kernel.invoke(function=kernel_function, arguments=arguments)
|
||||
|
||||
return result
|
||||
@@ -0,0 +1 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
@@ -0,0 +1,253 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import logging
|
||||
from typing import Annotated
|
||||
|
||||
from pydantic import Field, ValidationError
|
||||
from semantic_kernel import Kernel
|
||||
from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior
|
||||
from semantic_kernel.functions import KernelArguments
|
||||
from semantic_kernel.functions.kernel_function_decorator import kernel_function
|
||||
|
||||
from guided_conversation.utils.base_model_llm import BaseModelLLM
|
||||
from guided_conversation.utils.conversation_helpers import Conversation, ConversationMessageType
|
||||
from guided_conversation.utils.openai_tool_calling import ToolValidationResult
|
||||
from guided_conversation.utils.plugin_helpers import PluginOutput, fix_error, update_attempts
|
||||
from guided_conversation.utils.resources import ResourceConstraintMode, ResourceConstraintUnit, format_resource
|
||||
|
||||
AGENDA_ERROR_CORRECTION_SYSTEM_TEMPLATE = """<message role="system">You are a helpful, thoughtful, and meticulous assistant.
|
||||
You are conducting a conversation with a user. You tried to update the agenda, but the update was invalid.
|
||||
You will be provided the history of your conversation with the user, \
|
||||
your previous attempt(s) at updating the agenda, and the error message(s) that resulted from your attempt(s).
|
||||
Your task is to correct the update so that it is valid. \
|
||||
Your changes should be as minimal as possible - you are focused on fixing the error(s) that caused the update to be invalid.
|
||||
Note that if the resource allocation is invalid, you must follow these rules:
|
||||
1. You should not change the description of the first item (since it has already been executed), but you can change its resource allocation
|
||||
2. For all other items, you can combine or split them, or assign them fewer or more resources, \
|
||||
but the content they cover collectively should not change (i.e. don't eliminate or add new topics).
|
||||
For example, the invalid attempt was "item 1 = ask for date of birth (1 turn), item 2 = ask for phone number (1 turn), \
|
||||
item 3 = ask for phone type (1 turn), item 4 = explore treatment history (6 turns)", \
|
||||
and the error says you need to correct the total resource allocation to 7 turns. \
|
||||
A bad solution is "item 1 = ask for date of birth (1 turn), \
|
||||
item 2 = explore treatment history (6 turns)" because it eliminates the phone number and phone type topics. \
|
||||
A good solution is "item 1 = ask for date of birth (2 turns), item 2 = ask for phone number, phone type,
|
||||
and treatment history (2 turns), item 3 = explore treatment history (3 turns)."</message>
|
||||
|
||||
<message role="user">Conversation history:
|
||||
{{ conversation_history }}
|
||||
|
||||
Previous attempts to update the agenda:
|
||||
{{ previous_attempts }}</message>"""
|
||||
|
||||
UPDATE_AGENDA_TOOL = "update_agenda"
|
||||
|
||||
|
||||
class _BaseAgendaItem(BaseModelLLM):
|
||||
title: str = Field(description="Brief description of the item")
|
||||
resource: int = Field(description="Number of turns required for the item")
|
||||
|
||||
|
||||
class _BaseAgenda(BaseModelLLM):
|
||||
items: list[_BaseAgendaItem] = Field(
|
||||
description="Ordered list of items to be completed in the remainder of the conversation",
|
||||
default_factory=list,
|
||||
)
|
||||
|
||||
|
||||
class Agenda:
|
||||
"""An abstraction to manage a conversation agenda. The expected use case is that another agent will generate an agenda.
|
||||
This class will validate if it is valid, and help correct it if it is not.
|
||||
|
||||
Args:
|
||||
kernel (Kernel): The Semantic Kernel instance to use for calling the LLM. Don't forget to set your
|
||||
req_settings since this class uses tool calling functionality from the Semantic Kernel.
|
||||
service_id (str): The service ID to use for the Semantic Kernel tool calling. One kernel can have multiple
|
||||
services. The service ID is used to identify which service to use for LLM calls. The Agenda object
|
||||
assumes that the service has tool calling capabilities and is some flavor of chat completion.
|
||||
resource_constraint_mode (ResourceConstraintMode): The mode for resource constraints.
|
||||
max_agenda_retries (int): The maximum number of retries for updating the agenda.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
kernel: Kernel,
|
||||
service_id: str,
|
||||
resource_constraint_mode: ResourceConstraintMode | None,
|
||||
max_agenda_retries: int = 2,
|
||||
) -> None:
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
self.id = "agenda_plugin"
|
||||
self.kernel = Kernel()
|
||||
self.logger = logger
|
||||
self.kernel = kernel
|
||||
self.service_id = service_id
|
||||
|
||||
self.resource_constraint_mode = resource_constraint_mode
|
||||
self.max_agenda_retries = max_agenda_retries
|
||||
|
||||
self.agenda = _BaseAgenda()
|
||||
|
||||
async def update_agenda(
|
||||
self,
|
||||
items: list[dict[str, str]],
|
||||
remaining_turns: int,
|
||||
conversation: Conversation,
|
||||
) -> PluginOutput:
|
||||
"""Updates the agenda model with the given items (generally generated by an LLM) and validates if the update is valid.
|
||||
The agenda update reasons in terms of turns for validating the if the proposed agenda is valid.
|
||||
If you wish to use a different resource unit, convert the value to turns in some way because
|
||||
we found that LLMs do much better at reasoning in terms of turns.
|
||||
|
||||
Args:
|
||||
items (list[dict[str, str]]): A list of agenda items.
|
||||
Each item should have the following keys:
|
||||
- title (str): A brief description of the item.
|
||||
- resource (int): The number of turns required for the item.
|
||||
remaining_turns (int): The number of remaining turns.
|
||||
conversation (Conversation): The conversation object.
|
||||
|
||||
Returns:
|
||||
PluginOutput: A PluginOutput object with the success status. Does not generate any messages.
|
||||
"""
|
||||
previous_attempts = []
|
||||
while True:
|
||||
try:
|
||||
# Try to update the agenda, and do extra validation checks
|
||||
self.agenda.items = items
|
||||
self._validate_agenda_update(items, remaining_turns)
|
||||
self.logger.info(f"Agenda updated successfully: {self.get_agenda_for_prompt()}")
|
||||
return PluginOutput(True, [])
|
||||
except (ValidationError, ValueError) as e:
|
||||
# Update the previous attempts and get instructions for the LLM
|
||||
previous_attempts, llm_formatted_attempts = update_attempts(
|
||||
error=e, attempt_id=str(items), previous_attempts=previous_attempts
|
||||
)
|
||||
|
||||
# If we have reached the maximum number of retries return a failure
|
||||
if len(previous_attempts) > self.max_agenda_retries:
|
||||
self.logger.warning(f"Failed to update agenda after {self.max_agenda_retries} attempts.")
|
||||
return PluginOutput(False, [])
|
||||
else:
|
||||
self.logger.info(f"Attempting to fix the agenda error. Attempt {len(previous_attempts)}.")
|
||||
response = await self._fix_agenda_error(llm_formatted_attempts, conversation)
|
||||
if response["validation_result"] != ToolValidationResult.SUCCESS:
|
||||
self.logger.warning(
|
||||
f"Failed to fix the agenda error due to a failure in the LLM tool call: {response['validation_result']}"
|
||||
)
|
||||
return PluginOutput(False, [])
|
||||
else:
|
||||
# Use the result of the first tool call to try the update again
|
||||
items = response["tool_args_list"][0]["items"]
|
||||
|
||||
def get_agenda_for_prompt(self) -> str:
|
||||
"""Gets a string representation of the agenda for use in an LLM prompt.
|
||||
|
||||
Returns:
|
||||
str: A string representation of the agenda.
|
||||
"""
|
||||
agenda_json = self.agenda.model_dump()
|
||||
agenda_items = agenda_json.get("items", [])
|
||||
if len(agenda_items) == 0:
|
||||
return "None"
|
||||
agenda_str = "\n".join(
|
||||
[
|
||||
f"{i + 1}. [{format_resource(item['resource'], ResourceConstraintUnit.TURNS)}] {item['title']}"
|
||||
for i, item in enumerate(agenda_items)
|
||||
]
|
||||
)
|
||||
total_resource = format_resource(sum([item["resource"] for item in agenda_items]), ResourceConstraintUnit.TURNS)
|
||||
agenda_str += f"\nTotal = {total_resource}"
|
||||
return agenda_str
|
||||
|
||||
# The following is the kernel function that will be provided to the LLM call
|
||||
class Items:
|
||||
title: Annotated[str, "Description of the item"]
|
||||
resource: Annotated[int, "Number of turns required for the item"]
|
||||
|
||||
@kernel_function(
|
||||
name=UPDATE_AGENDA_TOOL,
|
||||
description="Updates the agenda.",
|
||||
)
|
||||
def update_agenda_items(
|
||||
self,
|
||||
items: Annotated[list[Items], "Ordered list of items to be completed in the remainder of the conversation"],
|
||||
):
|
||||
pass
|
||||
|
||||
async def _fix_agenda_error(self, previous_attempts: str, conversation: Conversation) -> None:
|
||||
"""Calls an LLM to try and fix an error in the agenda update."""
|
||||
req_settings = self.kernel.get_prompt_execution_settings_from_service_id(self.service_id)
|
||||
req_settings.max_tokens = 2000
|
||||
|
||||
self.kernel.add_function(plugin_name=self.id, function=self.update_agenda_items)
|
||||
filter = {"included_plugins": [self.id]}
|
||||
req_settings.function_choice_behavior = FunctionChoiceBehavior.Auto(auto_invoke=False, filters=filter)
|
||||
|
||||
arguments = KernelArguments(
|
||||
conversation_history=conversation.get_repr_for_prompt(exclude_types=[ConversationMessageType.REASONING]),
|
||||
previous_attempts=previous_attempts,
|
||||
)
|
||||
|
||||
return await fix_error(
|
||||
kernel=self.kernel,
|
||||
prompt_template=AGENDA_ERROR_CORRECTION_SYSTEM_TEMPLATE,
|
||||
req_settings=req_settings,
|
||||
arguments=arguments,
|
||||
)
|
||||
|
||||
def _validate_agenda_update(self, items: list[dict[str, str]], remaining_turns: int) -> None:
|
||||
"""Validates if any constraints were violated while performing the agenda update.
|
||||
|
||||
Args:
|
||||
items (list[dict[str, str]]): A list of agenda items.
|
||||
remaining_turns (int): The number of remaining turns.
|
||||
|
||||
Raises:
|
||||
ValueError: If any validation checks fail.
|
||||
"""
|
||||
# The total, proposed allocation of resources.
|
||||
total_resources = sum([item["resource"] for item in items])
|
||||
|
||||
violations = []
|
||||
# In maximum mode, the total resources should not exceed the remaining turns
|
||||
if (self.resource_constraint_mode == ResourceConstraintMode.MAXIMUM) and (total_resources > remaining_turns):
|
||||
total_resource_instruction = (
|
||||
f"The total turns allocated in the agenda must not exceed the remaining amount ({remaining_turns})"
|
||||
)
|
||||
violations.append(f"{total_resource_instruction}; but the current total is {total_resources}.")
|
||||
|
||||
# In exact mode if the total resources were not exactly equal to the remaining turns
|
||||
if (self.resource_constraint_mode == ResourceConstraintMode.EXACT) and (total_resources != remaining_turns):
|
||||
total_resource_instruction = (
|
||||
f"The total turns allocated in the agenda must equal the remaining amount ({remaining_turns})"
|
||||
)
|
||||
violations.append(f"{total_resource_instruction}; but the current total is {total_resources}.")
|
||||
|
||||
# Check if any item has a resource value of 0
|
||||
if any(item["resource"] <= 0 for item in items):
|
||||
violations.append("All items must have a resource value greater than 0.")
|
||||
|
||||
# Raise an error if any violations were found
|
||||
if len(violations) > 0:
|
||||
self.logger.debug(f"Agenda update failed due to the following violations: {violations}.")
|
||||
raise ValueError(" ".join(violations))
|
||||
|
||||
def to_json(self) -> dict:
|
||||
agenda_dict = self.agenda.model_dump()
|
||||
return {
|
||||
"agenda": agenda_dict,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def from_json(
|
||||
cls,
|
||||
json_data: dict,
|
||||
kernel: Kernel,
|
||||
service_id: str,
|
||||
resource_constraint_mode: ResourceConstraintMode | None,
|
||||
max_agenda_retries: int = 2,
|
||||
) -> "Agenda":
|
||||
agenda = cls(kernel, service_id, resource_constraint_mode, max_agenda_retries)
|
||||
agenda.agenda.items = json_data["agenda"]["items"]
|
||||
return agenda
|
||||
@@ -0,0 +1,480 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import logging
|
||||
from typing import Annotated, Any, Literal, get_args, get_origin, get_type_hints
|
||||
|
||||
from pydantic import BaseModel, create_model
|
||||
from semantic_kernel import Kernel
|
||||
from semantic_kernel.connectors.ai.function_choice_behavior import FunctionChoiceBehavior
|
||||
from semantic_kernel.contents import AuthorRole, ChatMessageContent
|
||||
from semantic_kernel.functions import KernelArguments
|
||||
from semantic_kernel.functions.kernel_function_decorator import kernel_function
|
||||
|
||||
from guided_conversation.utils.base_model_llm import BaseModelLLM
|
||||
from guided_conversation.utils.conversation_helpers import Conversation, ConversationMessageType
|
||||
from guided_conversation.utils.openai_tool_calling import ToolValidationResult
|
||||
from guided_conversation.utils.plugin_helpers import PluginOutput, fix_error, update_attempts
|
||||
|
||||
ARTIFACT_ERROR_CORRECTION_SYSTEM_TEMPLATE = """<message role="system">You are a helpful, thoughtful, and meticulous assistant.
|
||||
You are conducting a conversation with a user. Your goal is to complete an artifact as thoroughly as possible by the end of the conversation.
|
||||
You have tried to update a field in the artifact, but the value you provided did not adhere \
|
||||
to the constraints of the field as specified in the artifact schema.
|
||||
You will be provided the history of your conversation with the user, the schema for the field, \
|
||||
your previous attempt(s) at updating the field, and the error message(s) that resulted from your attempt(s).
|
||||
Your task is to select the best possible action to take next:
|
||||
1. Update artifact
|
||||
- You should pick this action if you have a valid value to submit for the field in question.
|
||||
2. Resume conversation
|
||||
- You should pick this action if: (a) you do NOT have a valid value to submit for the field in question, and \
|
||||
(b) you need to ask the user for more information in order to obtain a valid value. \
|
||||
For example, if the user stated that their date of birth is June 2000, but the artifact field asks for the date of birth in the format \
|
||||
"YYYY-MM-DD", you should resume the conversation and ask the user for the day.</message>
|
||||
|
||||
<message role="user">Conversation history:
|
||||
{{ conversation_history }}
|
||||
|
||||
Schema:
|
||||
{{ artifact_schema }}
|
||||
|
||||
Previous attempts to update the field "{{ field_name }}" in the artifact:
|
||||
{{ previous_attempts }}</message>"""
|
||||
|
||||
UPDATE_ARTIFACT_TOOL = "update_artifact_field"
|
||||
RESUME_CONV_TOOL = "resume_conversation"
|
||||
|
||||
|
||||
class Artifact:
|
||||
"""The Artifact plugin takes in a Pydantic base model, and robustly handles updating the fields of the model
|
||||
A typical use case is as a form an agent must complete throughout a conversation.
|
||||
Another use case is as a working memory for the agent.
|
||||
|
||||
The primary interface is update_artifact, which takes in the field_name to update and its new value.
|
||||
Additionally, the chat_history is passed in to help the agent make informed decisions in case an error occurs.
|
||||
|
||||
The Artifact also exposes several functions to access internal state:
|
||||
get_artifact_for_prompt, get_schema_for_prompt, and get_failed_fields.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self, kernel: Kernel, service_id: str, input_artifact: BaseModel, max_artifact_field_retries: int = 2
|
||||
) -> None:
|
||||
"""
|
||||
Initialize the Artifact plugin with the given Pydantic base model.
|
||||
|
||||
Args:
|
||||
kernel (Kernel): The Semantic Kernel instance to use for calling the LLM. Don't forget to set your
|
||||
req_settings since this class uses tool calling functionality from the Semantic Kernel.
|
||||
service_id (str): The service ID to use for the Semantic Kernel tool calling. One kernel can have multiple
|
||||
services. The service ID is used to identify which service to use for LLM calls. The Artifact object
|
||||
assumes that the service has tool calling capabilities and is some flavor of chat completion.
|
||||
input_artifact (BaseModel): The Pydantic base model to use as the artifact
|
||||
max_artifact_field_retries (int): The maximum number of times to retry updating a field in the artifact
|
||||
"""
|
||||
logger = logging.getLogger(__name__)
|
||||
self.logger = logger
|
||||
|
||||
self.id = "artifact_plugin"
|
||||
self.kernel = kernel
|
||||
self.service_id = service_id
|
||||
self.max_artifact_field_retries = max_artifact_field_retries
|
||||
|
||||
self.original_schema = input_artifact.model_json_schema()
|
||||
self.artifact = self._initialize_artifact(input_artifact)
|
||||
|
||||
# failed_artifact_fields maps a field name to a list of the history of the failed attempts to update it
|
||||
# dict: key = field, value = list of tuple[attempt, error message]
|
||||
self.failed_artifact_fields: dict[str, list[tuple[str, str]]] = {}
|
||||
|
||||
# The following are the kernel functions that will be provided to the LLM call
|
||||
@kernel_function(
|
||||
name=UPDATE_ARTIFACT_TOOL,
|
||||
description="Sets the value of a field in the artifact",
|
||||
)
|
||||
def update_artifact_field(
|
||||
self,
|
||||
field: Annotated[str, "The name of the field to update in the artifact"],
|
||||
value: Annotated[str, "The value to set the field to"],
|
||||
) -> None:
|
||||
pass
|
||||
|
||||
@kernel_function(
|
||||
name=RESUME_CONV_TOOL,
|
||||
description="Resumes conversation to get more information from the user ",
|
||||
)
|
||||
def resume_conversation(self):
|
||||
pass
|
||||
|
||||
async def update_artifact(self, field_name: str, field_value: Any, conversation: Conversation) -> PluginOutput:
|
||||
"""The core interface for the Artifact plugin.
|
||||
This function will attempt to update the given field_name to the given field_value.
|
||||
If the field_value fails Pydantic validation, an LLM will determine one of two actions to take.
|
||||
Given the conversation as additional context the two actions are:
|
||||
- Retry the update the artifact by fixing the formatting using the previous failed attempts as guidance
|
||||
- Take no action or in other words, resume the conversation to ask the user for more information because the user gave incomplete or incorrect information
|
||||
|
||||
Args:
|
||||
field_name (str): The name of the field to update in the artifact
|
||||
field_value (Any): The value to set the field to
|
||||
conversation (Conversation): The conversation object that contains the history of the conversation
|
||||
|
||||
Returns:
|
||||
PluginOutput: An object with two fields: a boolean indicating success
|
||||
and a list of conversation messages that may have been generated.
|
||||
|
||||
Several outcomes can happen:
|
||||
- The update may have failed due to
|
||||
- A field_name that is not valid in the artifact.
|
||||
- The field_value failing Pydantic validation and all retries failed.
|
||||
- The model failed to correctly call a tool.
|
||||
In this case, the boolean will be False and the list may contain a message indicating the failure.
|
||||
|
||||
- The agent may have successfully updated the artifact or fixed it.
|
||||
In this case, the boolean will be True and the list will contain a message indicating the update and possibly intermediate messages.
|
||||
|
||||
- The agent may have decided to resume the conversation.
|
||||
In this case, the boolean will be True and the messages may only contain messages indicated previous errors.
|
||||
"""
|
||||
|
||||
conversation_messages: list[ChatMessageContent] = []
|
||||
|
||||
# Check if the field name is valid, and return with a failure message if not
|
||||
is_valid_field, msg = self._is_valid_field(field_name)
|
||||
if not is_valid_field:
|
||||
conversation_messages.append(msg)
|
||||
return PluginOutput(update_successful=False, messages=conversation_messages)
|
||||
|
||||
# Try to update the field, and handle any errors that occur until the field is
|
||||
# successfully updated or skipped according to max_artifact_field_retries
|
||||
while True:
|
||||
try:
|
||||
# Check if there have been too many previous failed attempts to update the field
|
||||
if len(self.failed_artifact_fields.get(field_name, [])) >= self.max_artifact_field_retries:
|
||||
self.logger.warning(f"Updating field {field_name} has failed too many times. Skipping.")
|
||||
return False, conversation_messages
|
||||
|
||||
# Attempt to update the artifact
|
||||
msg = self._execute_update_artifact(field_name, field_value)
|
||||
conversation_messages.append(msg)
|
||||
return PluginOutput(True, conversation_messages)
|
||||
except Exception as e:
|
||||
self.logger.warning(f"Error updating field {field_name}: {e}. Retrying...")
|
||||
# Handle update error will increment failed_artifact_fields, once it has failed
|
||||
# greater than self.max_artifact_field_retries the field will be skipped and the loop will break
|
||||
success, new_field_value = await self._handle_update_error(field_name, field_value, conversation, e)
|
||||
|
||||
# The agent has successfully fixed the field.
|
||||
if success and new_field_value is not None:
|
||||
self.logger.info(f"Agent successfully fixed field {field_name}. New value: {new_field_value}")
|
||||
field_value = new_field_value
|
||||
# This is the case where the agent has decided to resume the conversation.
|
||||
elif success:
|
||||
self.logger.info(
|
||||
f"Agent could not fix the field itself & decided to resume conversation to fix field {field_name}"
|
||||
)
|
||||
return PluginOutput(True, conversation_messages)
|
||||
self.logger.warning(f"Agent failed to fix field {field_name}. Retrying...")
|
||||
# Otherwise, the agent has failed and we will go through the loop again
|
||||
|
||||
def get_artifact_for_prompt(self) -> str:
|
||||
"""Returns a formatted JSON-like representation of the current state of the fields artifact.
|
||||
Any fields that were failed are completely omitted.
|
||||
|
||||
Returns:
|
||||
str: The string representation of the artifact.
|
||||
"""
|
||||
failed_fields = self.get_failed_fields()
|
||||
return {k: v for k, v in self.artifact.model_dump().items() if k not in failed_fields}
|
||||
|
||||
def get_schema_for_prompt(self, filter_one_field: str | None = None) -> str:
|
||||
"""Gets a clean version of the original artifact schema, optimized for use in an LLM prompt.
|
||||
|
||||
Args:
|
||||
filter_one_field (str | None): If this is provided, only the schema for this one field will be returned.
|
||||
|
||||
Returns:
|
||||
str: The cleaned schema
|
||||
"""
|
||||
|
||||
def _clean_properties(schema: dict, failed_fields: list[str]) -> str:
|
||||
properties = schema.get("properties", {})
|
||||
clean_properties = {}
|
||||
for name, property_dict in properties.items():
|
||||
if name not in failed_fields:
|
||||
cleaned_property = {}
|
||||
for k, v in property_dict.items():
|
||||
if k in ["title", "default"]:
|
||||
continue
|
||||
cleaned_property[k] = v
|
||||
clean_properties[name] = cleaned_property
|
||||
|
||||
clean_properties_str = str(clean_properties)
|
||||
clean_properties_str = clean_properties_str.replace("$ref", "type")
|
||||
clean_properties_str = clean_properties_str.replace("#/$defs/", "")
|
||||
return clean_properties_str
|
||||
|
||||
# If filter_one_field is provided, only get the schema for that one field
|
||||
if filter_one_field:
|
||||
if not self._is_valid_field(filter_one_field):
|
||||
self.logger.error(f'Field "{filter_one_field}" is not a valid field in the artifact.')
|
||||
raise ValueError(f'Field "{filter_one_field}" is not a valid field in the artifact.')
|
||||
filtered_schema = {"properties": {filter_one_field: self.original_schema["properties"][filter_one_field]}}
|
||||
filtered_schema.update((k, v) for k, v in self.original_schema.items() if k != "properties")
|
||||
schema = filtered_schema
|
||||
else:
|
||||
schema = self.original_schema
|
||||
|
||||
failed_fields = self.get_failed_fields()
|
||||
properties = _clean_properties(schema, failed_fields)
|
||||
if not properties:
|
||||
self.logger.error("No properties found in the schema.")
|
||||
raise ValueError("No properties found in the schema.")
|
||||
|
||||
types_schema = schema.get("$defs", {})
|
||||
custom_types = []
|
||||
for type_name, type_info in types_schema.items():
|
||||
if f"'type': '{type_name}'" in properties:
|
||||
clean_schema = _clean_properties(type_info, [])
|
||||
if clean_schema != "{}":
|
||||
custom_types.append(f"{type_name} = {clean_schema}")
|
||||
|
||||
if custom_types:
|
||||
explanation = f"If you wanted to create a {type_name} object, for example, you would make a JSON object \
|
||||
with the following keys: {', '.join(types_schema[type_name]['properties'].keys())}."
|
||||
custom_types_str = "\n".join(custom_types)
|
||||
return f"""{properties}
|
||||
|
||||
Here are the definitions for the custom types referenced in the artifact schema:
|
||||
{custom_types_str}
|
||||
|
||||
{explanation}
|
||||
Remember that when updating the artifact, the field will be the original field name in the artifact and the JSON object(s) will be the value."""
|
||||
else:
|
||||
return properties
|
||||
|
||||
def get_failed_fields(self) -> list[str]:
|
||||
"""Get a list of fields that have failed all attempts to update.
|
||||
|
||||
Returns:
|
||||
list[str]: A list of field names that have failed all attempts to update.
|
||||
"""
|
||||
fields = []
|
||||
for field, attempts in self.failed_artifact_fields.items():
|
||||
if len(attempts) >= self.max_artifact_field_retries:
|
||||
fields.append(field)
|
||||
return fields
|
||||
|
||||
def _initialize_artifact(self, artifact_model: BaseModel) -> BaseModelLLM:
|
||||
"""Create a new artifact model based on the one provided by the user
|
||||
with "Unanswered" set for all fields.
|
||||
|
||||
Args:
|
||||
artifact_model (BaseModel): The Pydantic class provided by the user
|
||||
|
||||
Returns:
|
||||
BaseModelLLM: The new artifact model with "Unanswered" set for all fields
|
||||
"""
|
||||
modified_classes = self._modify_classes(artifact_model)
|
||||
artifact = self._modify_base_artifact(artifact_model, modified_classes)
|
||||
return artifact()
|
||||
|
||||
def _get_type_if_subtype(self, target_type: type[Any], base_type: type[Any]) -> type[Any] | None:
|
||||
"""Recursively checks the target_type to see if it is a subclass of base_type or a generic including base_type.
|
||||
|
||||
Args:
|
||||
target_type: The type to check.
|
||||
base_type: The type to check against.
|
||||
|
||||
Returns:
|
||||
The class type if target_type is base_type, a subclass of base_type, or a generic including base_type; otherwise, None.
|
||||
"""
|
||||
origin = get_origin(target_type)
|
||||
if origin is None:
|
||||
if issubclass(target_type, base_type):
|
||||
return target_type
|
||||
else:
|
||||
# Recursively check if any of the arguments are the target type
|
||||
for arg in get_args(target_type):
|
||||
result = self._get_type_if_subtype(arg, base_type)
|
||||
if result is not None:
|
||||
return result
|
||||
return None
|
||||
|
||||
def _modify_classes(self, artifact_class: BaseModel) -> dict[str, type[BaseModelLLM]]:
|
||||
"""Find all classes used as type hints in the artifact, and modify them to set 'Unanswered' as a default and valid value for all fields."""
|
||||
modified_classes = {}
|
||||
# Find any instances of BaseModel in the artifact class in the first "level" of type hints
|
||||
for field_name, field_type in get_type_hints(artifact_class).items():
|
||||
is_base_model = self._get_type_if_subtype(field_type, BaseModel)
|
||||
if is_base_model is not None:
|
||||
modified_classes[field_name] = self._modify_base_artifact(is_base_model)
|
||||
|
||||
return modified_classes
|
||||
|
||||
def _replace_type_annotations(
|
||||
self, field_annotation: type[Any] | None, modified_classes: dict[str, type[BaseModelLLM]]
|
||||
) -> type:
|
||||
"""Recursively replace type annotations with modified classes where applicable."""
|
||||
# Get the origin of the field annotation, which is the base type for generic types (e.g., List[str] -> list, Dict[str, int] -> dict)
|
||||
origin = get_origin(field_annotation)
|
||||
# Get the type arguments of the generic type (e.g., List[str] -> str, Dict[str, int] -> str, int)
|
||||
args = get_args(field_annotation)
|
||||
|
||||
if origin is None:
|
||||
# The type is not generic; check if it's a subclass that needs to be replaced
|
||||
if isinstance(field_annotation, type) and issubclass(field_annotation, BaseModelLLM):
|
||||
return modified_classes.get(field_annotation.__name__, field_annotation)
|
||||
return field_annotation
|
||||
else:
|
||||
# The type is generic; recursively replace the type annotations of the arguments
|
||||
new_args = tuple(self._replace_type_annotations(arg, modified_classes) for arg in args)
|
||||
return origin[new_args]
|
||||
|
||||
def _modify_base_artifact(
|
||||
self, artifact_model: type[BaseModelLLM], modified_classes: dict[str, type[BaseModelLLM]] | None = None
|
||||
) -> type[BaseModelLLM]:
|
||||
"""Create a new artifact model with 'Unanswered' as a default and valid value for all fields."""
|
||||
for _, field_info in artifact_model.model_fields.items():
|
||||
# Replace original classes with modified version
|
||||
if modified_classes is not None:
|
||||
field_info.annotation = self._replace_type_annotations(field_info.annotation, modified_classes)
|
||||
# This makes it possible to always set a field to "Unanswered"
|
||||
field_info.annotation = field_info.annotation | Literal["Unanswered"]
|
||||
# This sets the default value to "Unanswered"
|
||||
field_info.default = "Unanswered"
|
||||
# This adds "Unanswered" as a possible value to any regex patterns
|
||||
metadata = field_info.metadata
|
||||
for m in metadata:
|
||||
if hasattr(m, "pattern"):
|
||||
m.pattern += "|Unanswered"
|
||||
field_definitions = {
|
||||
name: (field_info.annotation, field_info) for name, field_info in artifact_model.model_fields.items()
|
||||
}
|
||||
artifact_model = create_model("Artifact", __base__=BaseModelLLM, **field_definitions)
|
||||
return artifact_model
|
||||
|
||||
def _is_valid_field(self, field_name: str) -> tuple[bool, ChatMessageContent]:
|
||||
"""Check if the field_name is a valid field in the artifact. Returns True if it is, False and an error message otherwise."""
|
||||
if field_name not in self.artifact.model_fields:
|
||||
error_message = f'Field "{field_name}" is not a valid field in the artifact.'
|
||||
msg = ChatMessageContent(
|
||||
role=AuthorRole.ASSISTANT,
|
||||
content=error_message,
|
||||
metadata={"type": ConversationMessageType.ARTIFACT_UPDATE, "turn_number": None},
|
||||
)
|
||||
return False, msg
|
||||
return True, None
|
||||
|
||||
async def _fix_artifact_error(
|
||||
self,
|
||||
field_name: str,
|
||||
previous_attempts: str,
|
||||
conversation_repr: str,
|
||||
artifact_schema_repr: str,
|
||||
) -> dict[str, Any]:
|
||||
"""Calls the LLM to fix an error in the artifact using Semantic Kernel kernel."""
|
||||
|
||||
req_settings = self.kernel.get_prompt_execution_settings_from_service_id(self.service_id)
|
||||
req_settings.max_tokens = 2000
|
||||
|
||||
self.kernel.add_function(plugin_name=self.id, function=self.update_artifact_field)
|
||||
self.kernel.add_function(plugin_name=self.id, function=self.resume_conversation)
|
||||
filter = {"included_plugins": [self.id]}
|
||||
req_settings.function_choice_behavior = FunctionChoiceBehavior.Auto(auto_invoke=False, filters=filter)
|
||||
|
||||
arguments = KernelArguments(
|
||||
field_name=field_name,
|
||||
conversation_history=conversation_repr,
|
||||
previous_attempts=previous_attempts,
|
||||
artifact_schema=artifact_schema_repr,
|
||||
settings=req_settings,
|
||||
)
|
||||
|
||||
return await fix_error(
|
||||
kernel=self.kernel,
|
||||
prompt_template=ARTIFACT_ERROR_CORRECTION_SYSTEM_TEMPLATE,
|
||||
req_settings=req_settings,
|
||||
arguments=arguments,
|
||||
)
|
||||
|
||||
def _execute_update_artifact(
|
||||
self,
|
||||
field_name: Annotated[str, "The name of the field to update in the artifact"],
|
||||
field_value: Annotated[Any, "The value to set the field to"],
|
||||
) -> None:
|
||||
"""Update a field in the artifact with a new value. This will raise an error if the field_value is invalid."""
|
||||
setattr(self.artifact, field_name, field_value)
|
||||
msg = ChatMessageContent(
|
||||
role=AuthorRole.ASSISTANT,
|
||||
content=f"Assistant updated {field_name} to {field_value}",
|
||||
metadata={"type": ConversationMessageType.ARTIFACT_UPDATE, "turn_number": None},
|
||||
)
|
||||
return msg
|
||||
|
||||
async def _handle_update_error(
|
||||
self, field_name: str, field_value: Any, conversation: Conversation, error: Exception
|
||||
) -> tuple[bool, Any]:
|
||||
"""
|
||||
Handles the logic for when an error occurs while updating a field.
|
||||
Creates the appropriate context for the model and calls the LLM to fix the error.
|
||||
|
||||
Args:
|
||||
field_name (str): The name of the field to update in the artifact
|
||||
field_value (Any): The value to set the field to
|
||||
conversation (Conversation): The conversation object that contains the history of the conversation
|
||||
error (Exception): The error that occurred while updating the field
|
||||
|
||||
Returns:
|
||||
tuple[bool, Any]: A tuple containing a boolean indicating success and the new field value if successful (if not, then None)
|
||||
"""
|
||||
# Update the failed attempts for the field
|
||||
previous_attempts = self.failed_artifact_fields.get(field_name, [])
|
||||
previous_attempts, llm_formatted_attempts = update_attempts(
|
||||
error=error, attempt_id=str(field_value), previous_attempts=previous_attempts
|
||||
)
|
||||
self.failed_artifact_fields[field_name] = previous_attempts
|
||||
|
||||
# Call the LLM to fix the error
|
||||
conversation_history_repr = conversation.get_repr_for_prompt(exclude_types=[ConversationMessageType.REASONING])
|
||||
artifact_schema_repr = self.get_schema_for_prompt(filter_one_field=field_name)
|
||||
result = await self._fix_artifact_error(
|
||||
field_name, llm_formatted_attempts, conversation_history_repr, artifact_schema_repr
|
||||
)
|
||||
|
||||
# Handling the result of the LLM call
|
||||
if result["validation_result"] != ToolValidationResult.SUCCESS:
|
||||
return False, None
|
||||
# Only consider the first tool call
|
||||
tool_name = result["tool_names"][0]
|
||||
tool_args = result["tool_args_list"][0]
|
||||
if tool_name == f"{self.id}-{UPDATE_ARTIFACT_TOOL}":
|
||||
field_value = tool_args["value"]
|
||||
return True, field_value
|
||||
elif tool_name == f"{self.id}-{RESUME_CONV_TOOL}":
|
||||
return True, None
|
||||
|
||||
def to_json(self) -> dict:
|
||||
artifact_fields = self.artifact.model_dump()
|
||||
return {
|
||||
"artifact": artifact_fields,
|
||||
"failed_fields": self.failed_artifact_fields,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def from_json(
|
||||
cls,
|
||||
json_data: dict,
|
||||
kernel: Kernel,
|
||||
service_id: str,
|
||||
input_artifact: BaseModel,
|
||||
max_artifact_field_retries: int = 2,
|
||||
) -> "Artifact":
|
||||
artifact = cls(kernel, service_id, input_artifact, max_artifact_field_retries)
|
||||
|
||||
artifact.failed_artifact_fields = json_data["failed_fields"]
|
||||
|
||||
# Iterate over artifact fields and set them to the values in the json data
|
||||
# Skip any fields that are set as "Unanswered"
|
||||
for field_name, field_value in json_data["artifact"].items():
|
||||
if field_value != "Unanswered":
|
||||
setattr(artifact.artifact, field_name, field_value)
|
||||
return artifact
|
||||
+390
@@ -0,0 +1,390 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass, field
|
||||
from enum import Enum
|
||||
import logging
|
||||
|
||||
from pydantic import BaseModel
|
||||
from semantic_kernel import Kernel
|
||||
from semantic_kernel.contents import AuthorRole, ChatMessageContent
|
||||
from semantic_kernel.functions import KernelArguments
|
||||
from semantic_kernel.functions.kernel_function_decorator import kernel_function
|
||||
|
||||
from guided_conversation.functions.conversation_plan import conversation_plan_function
|
||||
from guided_conversation.functions.execution import end_conversation, execution, send_message
|
||||
from guided_conversation.functions.final_update_plan import final_update_plan_function
|
||||
from guided_conversation.plugins.agenda import Agenda
|
||||
from guided_conversation.plugins.artifact import Artifact
|
||||
from guided_conversation.utils.conversation_helpers import Conversation, ConversationMessageType
|
||||
from guided_conversation.utils.openai_tool_calling import (
|
||||
ToolValidationResult,
|
||||
parse_function_result,
|
||||
validate_tool_calling,
|
||||
)
|
||||
from guided_conversation.utils.plugin_helpers import PluginOutput, format_kernel_functions_as_tools
|
||||
from guided_conversation.utils.resources import GCResource, ResourceConstraint
|
||||
|
||||
MAX_DECISION_RETRIES = 2
|
||||
|
||||
|
||||
class ToolName(Enum):
|
||||
UPDATE_ARTIFACT_TOOL = "update_artifact_field"
|
||||
UPDATE_AGENDA_TOOL = "update_agenda"
|
||||
SEND_MSG_TOOL = "send_message_to_user"
|
||||
END_CONV_TOOL = "end_conversation"
|
||||
GENERATE_PLAN_TOOL = "generate_plan"
|
||||
EXECUTE_PLAN_TOOL = "execute_plan"
|
||||
FINAL_UPDATE_TOOL = "final_update"
|
||||
GUIDED_CONVERSATION_AGENT_TOOLBOX = "gc_agent"
|
||||
|
||||
|
||||
@dataclass
|
||||
class GCOutput:
|
||||
"""The output of the GuidedConversation agent.
|
||||
|
||||
Args:
|
||||
ai_message (str): The message to send to the user.
|
||||
is_conversation_over (bool): Whether the conversation is over.
|
||||
"""
|
||||
|
||||
ai_message: str | None = field(default=None)
|
||||
is_conversation_over: bool = field(default=False)
|
||||
|
||||
|
||||
class GuidedConversation:
|
||||
def __init__(
|
||||
self,
|
||||
kernel: Kernel,
|
||||
artifact: BaseModel,
|
||||
rules: list[str],
|
||||
conversation_flow: str | None,
|
||||
context: str | None,
|
||||
resource_constraint: ResourceConstraint | None,
|
||||
service_id: str = "gc_main",
|
||||
) -> None:
|
||||
"""Initializes the GuidedConversation agent.
|
||||
|
||||
Args:
|
||||
kernel (Kernel): An instance of Kernel. Must come initialized with a AzureOpenAI or OpenAI service.
|
||||
artifact (BaseModel): The artifact to be used as the goal/working memory/output of the conversation.
|
||||
rules (list[str]): The rules to be used in the guided conversation (dos and donts).
|
||||
conversation_flow (str | None): The conversation flow to be used in the guided conversation.
|
||||
context (str | None): The scene-setting for the conversation.
|
||||
resource_constraint (ResourceConstraint | None): The limit on the conversation length (for ex: number of turns).
|
||||
service_id (str): Provide a service_id associated with the kernel's service that was provided.
|
||||
"""
|
||||
|
||||
self.logger = logging.getLogger(__name__)
|
||||
self.kernel = kernel
|
||||
self.service_id = service_id
|
||||
|
||||
self.conversation = Conversation()
|
||||
self.resource = GCResource(resource_constraint)
|
||||
self.artifact = Artifact(self.kernel, self.service_id, artifact)
|
||||
self.rules = rules
|
||||
self.conversation_flow = conversation_flow
|
||||
self.context = context
|
||||
self.agenda = Agenda(self.kernel, self.service_id, self.resource.get_resource_mode(), MAX_DECISION_RETRIES)
|
||||
|
||||
# Plugins will be executed in the order of this list.
|
||||
self.plugins_order = [
|
||||
ToolName.UPDATE_ARTIFACT_TOOL.value,
|
||||
ToolName.UPDATE_AGENDA_TOOL.value,
|
||||
]
|
||||
|
||||
# Terminal plugins are plugins that are handled in a special way:
|
||||
# - Only one terminal plugin can be called in a single step of the conversation as it leads to the end of the conversation step.
|
||||
# - The order of this list determines the execution priority.
|
||||
# - For example, if the model chooses to both call send message and end conversation,
|
||||
# Send message will be executed first and since the orchestration step returns, end conversation will not be executed.
|
||||
self.terminal_plugins_order = [
|
||||
ToolName.SEND_MSG_TOOL.value,
|
||||
ToolName.END_CONV_TOOL.value,
|
||||
]
|
||||
|
||||
self.current_failed_decision_attempts = 0
|
||||
|
||||
# Set common request settings
|
||||
self.req_settings = self.kernel.get_prompt_execution_settings_from_service_id(self.service_id)
|
||||
self.req_settings.max_tokens = 2000
|
||||
self.kernel.add_function(plugin_name=ToolName.SEND_MSG_TOOL.value, function=send_message)
|
||||
self.kernel.add_function(plugin_name=ToolName.END_CONV_TOOL.value, function=end_conversation)
|
||||
self.kernel.add_function(
|
||||
plugin_name=ToolName.UPDATE_ARTIFACT_TOOL.value, function=self.artifact.update_artifact_field
|
||||
)
|
||||
self.kernel.add_function(
|
||||
plugin_name=ToolName.UPDATE_AGENDA_TOOL.value, function=self.agenda.update_agenda_items
|
||||
)
|
||||
|
||||
# Set orchestrator functions for the agent
|
||||
self.kernel_function_generate_plan = self.kernel.add_function(
|
||||
plugin_name="gc_agent", function=self.generate_plan
|
||||
)
|
||||
self.kernel_function_execute_plan = self.kernel.add_function(plugin_name="gc_agent", function=self.execute_plan)
|
||||
self.kernel_function_final_update = self.kernel.add_function(plugin_name="gc_agent", function=self.final_update)
|
||||
|
||||
async def step_conversation(self, user_input: str | None = None) -> GCOutput:
|
||||
"""Given a message from a user, this will execute the guided conversation agent up until a
|
||||
terminal plugin is called or the maximum number of decision retries is reached."""
|
||||
self.logger.info(f"Starting conversation step {self.resource.turn_number}.")
|
||||
self.resource.start_resource()
|
||||
self.current_failed_decision_attempts = 0
|
||||
if user_input:
|
||||
self.conversation.add_messages(
|
||||
ChatMessageContent(
|
||||
role=AuthorRole.USER,
|
||||
content=user_input,
|
||||
metadata={"turn_number": self.resource.turn_number, "type": ConversationMessageType.DEFAULT},
|
||||
)
|
||||
)
|
||||
|
||||
# Keep generating and executing plans until a terminal plugin is called
|
||||
# or the maximum number of decision retries is reached.
|
||||
while self.current_failed_decision_attempts < MAX_DECISION_RETRIES:
|
||||
plan = await self.kernel.invoke(self.kernel_function_generate_plan)
|
||||
executed_plan = await self.kernel.invoke(
|
||||
self.kernel_function_execute_plan, KernelArguments(plan=plan.value)
|
||||
)
|
||||
success, plugins, terminal_plugins = executed_plan.value
|
||||
|
||||
if success != ToolValidationResult.SUCCESS:
|
||||
self.logger.warning(
|
||||
f"Failed to parse tools in plan on retry attempt {self.current_failed_decision_attempts} out of {MAX_DECISION_RETRIES}."
|
||||
)
|
||||
self.current_failed_decision_attempts += 1
|
||||
continue
|
||||
|
||||
# Run a step of the orchestration logic based on the plugins called by the model.
|
||||
# First execute all regular plugins (if any) in the order returned by execute_plan
|
||||
for plugin_name, plugin_args in plugins:
|
||||
if plugin_name == f"{ToolName.UPDATE_ARTIFACT_TOOL.value}-{ToolName.UPDATE_ARTIFACT_TOOL.value}":
|
||||
plugin_args["conversation"] = self.conversation
|
||||
# Modify plugin_args such that field=field_name and value=field_value
|
||||
plugin_args["field_name"] = plugin_args.pop("field")
|
||||
plugin_args["field_value"] = plugin_args.pop("value")
|
||||
await self._call_plugin(self.artifact.update_artifact, plugin_args)
|
||||
elif plugin_name == f"{ToolName.UPDATE_AGENDA_TOOL.value}-{ToolName.UPDATE_AGENDA_TOOL.value}":
|
||||
plugin_args["remaining_turns"] = self.resource.get_remaining_turns()
|
||||
plugin_args["conversation"] = self.conversation
|
||||
await self._call_plugin(self.agenda.update_agenda, plugin_args)
|
||||
|
||||
# Then execute the first terminal plugin (if any)
|
||||
if terminal_plugins:
|
||||
gc_output = GCOutput()
|
||||
plugin_name, plugin_args = terminal_plugins[0]
|
||||
if plugin_name == f"{ToolName.SEND_MSG_TOOL.value}-{ToolName.SEND_MSG_TOOL.value}":
|
||||
gc_output.ai_message = plugin_args["message"]
|
||||
elif plugin_name == f"{ToolName.END_CONV_TOOL.value}-{ToolName.END_CONV_TOOL.value}":
|
||||
await self.kernel.invoke(self.kernel_function_final_update)
|
||||
gc_output.ai_message = "I will terminate this conversation now. Thank you for your time!"
|
||||
gc_output.is_conversation_over = True
|
||||
self.resource.increment_resource()
|
||||
return gc_output
|
||||
|
||||
# Handle case where the maximum number of decision retries was reached.
|
||||
self.logger.warning(f"Failed to execute plan after {MAX_DECISION_RETRIES} attempts.")
|
||||
self.resource.increment_resource()
|
||||
gc_output = GCOutput()
|
||||
gc_output.ai_message = "An error occurred and I must sadly end the conversation."
|
||||
gc_output.is_conversation_over = True
|
||||
return gc_output
|
||||
|
||||
@kernel_function(
|
||||
name=ToolName.GENERATE_PLAN_TOOL.value,
|
||||
description="Generate a plan based on a time constraint for the current state of the conversation.",
|
||||
)
|
||||
async def generate_plan(self) -> str:
|
||||
"""Generate a plan for the current state of the conversation. The idea here is to explicitly let the model plan before
|
||||
generating any plugin calls. This has been shown to increase reliability.
|
||||
|
||||
Returns:
|
||||
str: The plan generated by the plan function.
|
||||
"""
|
||||
self.logger.info("Generating plan for the current state of the conversation")
|
||||
plan = await conversation_plan_function(
|
||||
self.kernel,
|
||||
self.conversation,
|
||||
self.context,
|
||||
self.rules,
|
||||
self.conversation_flow,
|
||||
self.artifact,
|
||||
self.req_settings,
|
||||
self.resource,
|
||||
self.agenda,
|
||||
)
|
||||
plan = plan.value[0].content
|
||||
self.conversation.add_messages(
|
||||
ChatMessageContent(
|
||||
role=AuthorRole.ASSISTANT,
|
||||
content=plan,
|
||||
metadata={"turn_number": self.resource.turn_number, "type": ConversationMessageType.REASONING},
|
||||
)
|
||||
)
|
||||
return plan
|
||||
|
||||
@kernel_function(
|
||||
name=ToolName.EXECUTE_PLAN_TOOL.value,
|
||||
description="Given the generated plan by the model, use that plan to generate which functions to execute.",
|
||||
)
|
||||
async def execute_plan(
|
||||
self, plan: str
|
||||
) -> tuple[ToolValidationResult, list[tuple[str, dict]], list[tuple[str, dict]]]:
|
||||
"""Given the generated plan by the model, use that plan to generate which functions to execute.
|
||||
Once the tool calls are generated by the model, we sort them into two groups: regular plugins and terminal plugins
|
||||
according to the definition in __init__
|
||||
|
||||
Args:
|
||||
plan (str): The plan generated by the model.
|
||||
|
||||
Returns:
|
||||
tuple[ToolValidationResult, list[tuple[str, dict]], list[tuple[str, dict]]]: A tuple containing the validation result
|
||||
of the tool calls, the regular plugins to execute, and the terminal plugins to execute alongside their arguments.
|
||||
"""
|
||||
self.logger.info("Executing plan.")
|
||||
|
||||
req_settings = self.kernel.get_prompt_execution_settings_from_service_id(self.service_id)
|
||||
functions = self.plugins_order + self.terminal_plugins_order
|
||||
result = await execution(
|
||||
kernel=self.kernel,
|
||||
reasoning=plan,
|
||||
filter=functions,
|
||||
req_settings=req_settings,
|
||||
artifact_schema=self.artifact.get_schema_for_prompt(),
|
||||
)
|
||||
|
||||
parsed_result = parse_function_result(result)
|
||||
formatted_tools = format_kernel_functions_as_tools(self.kernel, functions)
|
||||
validation_result = validate_tool_calling(parsed_result, formatted_tools)
|
||||
|
||||
# Sort plugin calls into two groups in the order of the corresponding lists defined in __init__
|
||||
plugins = []
|
||||
terminal_plugins = []
|
||||
if validation_result == ToolValidationResult.SUCCESS:
|
||||
for plugin in self.plugins_order:
|
||||
for idx, called_plugin_name in enumerate(parsed_result["tool_names"]):
|
||||
plugin_name = f"{plugin}-{plugin}"
|
||||
if called_plugin_name == plugin_name:
|
||||
plugins.append((parsed_result["tool_names"][idx], parsed_result["tool_args_list"][idx]))
|
||||
|
||||
for terminal_plugin in self.terminal_plugins_order:
|
||||
for idx, called_plugin_name in enumerate(parsed_result["tool_names"]):
|
||||
terminal_plugin_name = f"{terminal_plugin}-{terminal_plugin}"
|
||||
if called_plugin_name == terminal_plugin_name:
|
||||
terminal_plugins.append(
|
||||
(parsed_result["tool_names"][idx], parsed_result["tool_args_list"][idx])
|
||||
)
|
||||
|
||||
return validation_result, plugins, terminal_plugins
|
||||
|
||||
@kernel_function(
|
||||
name=ToolName.FINAL_UPDATE_TOOL.value,
|
||||
description="After the last message of a conversation was added to the conversation history, perform a final update of the artifact",
|
||||
)
|
||||
async def final_update(self):
|
||||
"""Explicit final update of the artifact after the conversation ends."""
|
||||
self.logger.info("Final update of the artifact prior to terminating the conversation.")
|
||||
|
||||
# Get a plan from the model
|
||||
reasoning_response = await final_update_plan_function(
|
||||
kernel=self.kernel,
|
||||
req_settings=self.req_settings,
|
||||
chat_history=self.conversation,
|
||||
context=self.context,
|
||||
artifact_schema=self.artifact.get_schema_for_prompt(),
|
||||
artifact_state=self.artifact.get_artifact_for_prompt(),
|
||||
)
|
||||
|
||||
# Then generate the functions to be executed
|
||||
req_settings = self.kernel.get_prompt_execution_settings_from_service_id(self.service_id)
|
||||
|
||||
functions = [ToolName.UPDATE_ARTIFACT_TOOL.value]
|
||||
execution_response = await execution(
|
||||
kernel=self.kernel,
|
||||
reasoning=reasoning_response.value[0].content,
|
||||
filter=functions,
|
||||
req_settings=req_settings,
|
||||
artifact_schema=self.artifact.get_schema_for_prompt(),
|
||||
)
|
||||
|
||||
parsed_result = parse_function_result(execution_response)
|
||||
formatted_tools = format_kernel_functions_as_tools(self.kernel, functions)
|
||||
validation_result = validate_tool_calling(parsed_result, formatted_tools)
|
||||
|
||||
# If the tool call was successful, update the artifact.
|
||||
if validation_result != ToolValidationResult.SUCCESS:
|
||||
self.logger.warning(f"No artifact change during final update due to: {validation_result.value}")
|
||||
pass
|
||||
else:
|
||||
for i in range(len(parsed_result["tool_names"])):
|
||||
tool_name = parsed_result["tool_names"][i]
|
||||
tool_args = parsed_result["tool_args_list"][i]
|
||||
if (
|
||||
tool_name == f"{ToolName.UPDATE_ARTIFACT_TOOL.value}-{ToolName.UPDATE_ARTIFACT_TOOL.value}"
|
||||
and "field" in tool_args
|
||||
and "value" in tool_args
|
||||
):
|
||||
# Check if tool_args contains the field and value to update
|
||||
plugin_output = await self.artifact.update_artifact(
|
||||
field_name=tool_args["field_name"],
|
||||
field_value=tool_args["field_value"],
|
||||
conversation=self.conversation,
|
||||
)
|
||||
if plugin_output.update_successful:
|
||||
self.logger.info(f"Artifact field {tool_args['field_name']} successfully updated.")
|
||||
# Set turn numbers
|
||||
for message in plugin_output.messages:
|
||||
message.turn_number = self.resource.turn_number
|
||||
self.conversation.add_messages(plugin_output.messages)
|
||||
else:
|
||||
self.logger.error(f"Final artifact field update of {tool_args['field_name']} failed.")
|
||||
|
||||
def to_json(self) -> dict:
|
||||
return {
|
||||
"artifact": self.artifact.to_json(),
|
||||
"agenda": self.agenda.to_json(),
|
||||
"chat_history": self.conversation.to_json(),
|
||||
"resource": self.resource.to_json(),
|
||||
}
|
||||
|
||||
async def _call_plugin(self, plugin_function: Callable, plugin_args: dict):
|
||||
"""Common logic whenever any plugin is called like handling errors and appending to chat history."""
|
||||
self.logger.info(f"Calling plugin {plugin_function.__name__}.")
|
||||
output: PluginOutput = await plugin_function(**plugin_args)
|
||||
if output.update_successful:
|
||||
# Set turn numbers
|
||||
for message in output.messages:
|
||||
message.metadata["turn_number"] = self.resource.turn_number
|
||||
self.conversation.add_messages(output.messages)
|
||||
else:
|
||||
self.logger.warning(
|
||||
f"Plugin {plugin_function.__name__} failed to execute on attempt {self.current_failed_decision_attempts} out of {MAX_DECISION_RETRIES}."
|
||||
)
|
||||
self.current_failed_decision_attempts += 1
|
||||
|
||||
@classmethod
|
||||
def from_json(
|
||||
cls,
|
||||
json_data: dict,
|
||||
kernel: Kernel,
|
||||
service_id: str = "gc_main",
|
||||
) -> "GuidedConversation":
|
||||
artifact = Artifact.from_json(
|
||||
json_data["artifact"],
|
||||
kernel=kernel,
|
||||
service_id=service_id,
|
||||
input_artifact=cls.artifact,
|
||||
max_artifact_field_retries=MAX_DECISION_RETRIES,
|
||||
)
|
||||
agenda = Agenda.from_json(
|
||||
json_data["agenda"],
|
||||
kernel=kernel,
|
||||
service_id=service_id,
|
||||
resource_constraint_mode=cls.resource_constraint.mode,
|
||||
)
|
||||
chat_history = Conversation.from_json(json_data["chat_history"])
|
||||
resource = GCResource.from_json(json_data["resource"])
|
||||
|
||||
gc = cls(kernel, artifact, agenda, chat_history, resource, service_id)
|
||||
|
||||
return gc
|
||||
@@ -0,0 +1 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
@@ -0,0 +1,42 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
import ast
|
||||
from types import NoneType
|
||||
from typing import get_args
|
||||
|
||||
from pydantic import BaseModel, ValidationInfo, field_validator
|
||||
|
||||
|
||||
class BaseModelLLM(BaseModel):
|
||||
"""A Pydantic base class for use when an LLM is completing fields. Provides a custom field validator and Pydantic Config."""
|
||||
|
||||
@field_validator("*", mode="before")
|
||||
def parse_literal_eval(cls, value: str, info: ValidationInfo): # noqa: N805
|
||||
"""An LLM will always result in a string (e.g. '["x", "y"]'), so we need to parse it to the correct type"""
|
||||
# Get the type hints for the field
|
||||
annotation = cls.model_fields[info.field_name].annotation
|
||||
typehints = get_args(annotation)
|
||||
if len(typehints) == 0:
|
||||
typehints = [annotation]
|
||||
|
||||
# Usually fields that are NoneType have another type hint as well, e.g. str | None
|
||||
# if the LLM returns "None" and the field allows NoneType, we should return None
|
||||
# without this code, the next if-block would leave the string "None" as the value
|
||||
if (NoneType in typehints) and (value == "None"):
|
||||
return None
|
||||
|
||||
# If the field allows strings, we don't parse it - otherwise a validation error might be raised
|
||||
# e.g. phone_number = "1234567890" should not be converted to an int if the type hint is str
|
||||
if str in typehints:
|
||||
return value
|
||||
try:
|
||||
evaluated_value = ast.literal_eval(value)
|
||||
return evaluated_value
|
||||
except Exception:
|
||||
return value
|
||||
|
||||
class Config:
|
||||
# Ensure that validation happens every time a field is updated, not just when the artifact is created
|
||||
validate_assignment = True
|
||||
# Do not allow extra fields to be added to the artifact
|
||||
extra = "forbid"
|
||||
+168
@@ -0,0 +1,168 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
import datetime
|
||||
from enum import Enum
|
||||
import logging
|
||||
from typing import Union
|
||||
|
||||
from semantic_kernel.contents import ChatMessageContent
|
||||
|
||||
|
||||
class ConversationMessageType(Enum):
|
||||
DEFAULT = "default"
|
||||
ARTIFACT_UPDATE = "artifact-update"
|
||||
REASONING = "reasoning"
|
||||
|
||||
|
||||
@dataclass
|
||||
class Conversation:
|
||||
"""An abstraction to represent a list of messages and common operations such as adding messages
|
||||
and getting a string representation.
|
||||
|
||||
Args:
|
||||
conversation_messages (list[ChatMessageContent]): A list of ChatMessageContent objects.
|
||||
"""
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
conversation_messages: list[ChatMessageContent] = field(default_factory=list)
|
||||
|
||||
def add_messages(self, messages: Union[ChatMessageContent, list[ChatMessageContent], "Conversation", None]) -> None:
|
||||
"""Add a message, list of messages to the conversation or merge another conversation into the end of this one.
|
||||
|
||||
Args:
|
||||
messages (Union[ChatMessageContent, list[ChatMessageContent], "Conversation"]): The message(s) to add.
|
||||
All messages will be added to the end of the conversation.
|
||||
|
||||
Returns:
|
||||
None
|
||||
"""
|
||||
if isinstance(messages, list):
|
||||
self.conversation_messages.extend(messages)
|
||||
elif isinstance(messages, Conversation):
|
||||
self.conversation_messages.extend(messages.conversation_messages)
|
||||
elif isinstance(messages, ChatMessageContent):
|
||||
# if ChatMessageContent.metadata doesn't have type, then add default
|
||||
if "type" not in messages.metadata:
|
||||
messages.metadata["type"] = ConversationMessageType.DEFAULT
|
||||
self.conversation_messages.append(messages)
|
||||
else:
|
||||
self.logger.warning(f"Invalid message type: {type(messages)}")
|
||||
return None
|
||||
|
||||
def get_repr_for_prompt(
|
||||
self,
|
||||
exclude_types: list[ConversationMessageType] | None = None,
|
||||
) -> str:
|
||||
"""Create a string representation of the conversation history for use in LLM prompts.
|
||||
|
||||
Args:
|
||||
exclude_types (list[ConversationMessageType] | None): A list of message types to exclude from the conversation
|
||||
history. If None, all message types will be included.
|
||||
|
||||
Returns:
|
||||
str: A string representation of the conversation history.
|
||||
"""
|
||||
if len(self.conversation_messages) == 0:
|
||||
return "None"
|
||||
|
||||
# Do not include the excluded messages types in the conversation history repr.
|
||||
if exclude_types is not None:
|
||||
conversation_messages = [
|
||||
message
|
||||
for message in self.conversation_messages
|
||||
if "type" in message.metadata and message.metadata["type"] not in exclude_types
|
||||
]
|
||||
else:
|
||||
conversation_messages = self.conversation_messages
|
||||
|
||||
to_join = []
|
||||
current_turn = None
|
||||
for message in conversation_messages:
|
||||
participant_name = message.name
|
||||
# Modify the default user to be capitalized for consistency with how assistant is written.
|
||||
if participant_name == "user":
|
||||
participant_name = "User"
|
||||
|
||||
# If the turn number is None, don't include it in the string
|
||||
if "turn_number" in message.metadata and current_turn != message.metadata["turn_number"]:
|
||||
current_turn = message.metadata["turn_number"]
|
||||
to_join.append(f"[Turn {current_turn}]")
|
||||
|
||||
# Add the message content
|
||||
if (message.role == "assistant") and (
|
||||
"type" in message.metadata and message.metadata["type"] == ConversationMessageType.ARTIFACT_UPDATE
|
||||
):
|
||||
to_join.append(message.content)
|
||||
elif message.role == "assistant":
|
||||
to_join.append(f"Assistant: {message.content}")
|
||||
else:
|
||||
user_string = message.content.strip()
|
||||
if user_string == "":
|
||||
to_join.append(f"{participant_name}: <sent an empty message>")
|
||||
else:
|
||||
to_join.append(f"{participant_name}: {user_string}")
|
||||
conversation_string = "\n".join(to_join)
|
||||
return conversation_string
|
||||
|
||||
def set_turn_numbers(self, turn_number: int) -> None:
|
||||
"""Set all the turn numbers in the conversation to the given turn number.
|
||||
|
||||
Args:
|
||||
turn_number (int): The turn number to set for all messages.
|
||||
|
||||
Returns:
|
||||
None"""
|
||||
for message in self.conversation_messages:
|
||||
message.metadata["turn_number"] = turn_number
|
||||
|
||||
@staticmethod
|
||||
def message_to_json(message: ChatMessageContent) -> dict:
|
||||
"""
|
||||
Convert a ChatMessageContent object to a JSON serializable dictionary.
|
||||
|
||||
Args:
|
||||
message (ChatMessageContent): The ChatMessageContent object to convert to JSON.
|
||||
|
||||
Returns:
|
||||
dict: A JSON serializable dictionary representation of the ChatMessageContent object.
|
||||
"""
|
||||
return {
|
||||
"role": message.role,
|
||||
"content": message.content,
|
||||
"name": message.name,
|
||||
"metadata": {
|
||||
"turn_number": message.metadata.get("turn_number", None),
|
||||
"type": message.metadata.get("type", ConversationMessageType.DEFAULT),
|
||||
},
|
||||
}
|
||||
|
||||
def to_json(self) -> dict:
|
||||
json_data = {}
|
||||
json_data["conversation"] = {}
|
||||
json_data["conversation"]["conversation_messages"] = [
|
||||
self.message_to_json(message) for message in self.conversation_messages
|
||||
]
|
||||
return json_data
|
||||
|
||||
@classmethod
|
||||
def from_json(
|
||||
cls,
|
||||
json_data: dict,
|
||||
) -> "Conversation":
|
||||
conversation = cls()
|
||||
for message in json_data["conversation"]["conversation_messages"]:
|
||||
conversation.add_messages(
|
||||
ChatMessageContent(
|
||||
role=message["role"],
|
||||
content=message["content"],
|
||||
name=message["name"],
|
||||
metadata={
|
||||
"turn_number": message["turn_number"],
|
||||
"type": ConversationMessageType(message["type"]),
|
||||
"timestamp": datetime.datetime.fromisoformat(message["timestamp"]),
|
||||
},
|
||||
)
|
||||
)
|
||||
|
||||
return conversation
|
||||
+158
@@ -0,0 +1,158 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
from dataclasses import dataclass
|
||||
from enum import Enum
|
||||
import json
|
||||
import logging
|
||||
from typing import Any
|
||||
|
||||
from semantic_kernel.contents.function_call_content import FunctionCallContent
|
||||
from semantic_kernel.functions import FunctionResult
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass
|
||||
class ToolArg:
|
||||
argument_name: str
|
||||
required: bool
|
||||
|
||||
|
||||
@dataclass
|
||||
class Tool:
|
||||
name: str
|
||||
args: list[ToolArg]
|
||||
|
||||
|
||||
class ToolValidationResult(Enum):
|
||||
NO_TOOL_CALLED = "No tool was called"
|
||||
INVALID_TOOL_CALLED = "A tool was called with an unexpected name"
|
||||
MISSING_REQUIRED_ARGUMENT = "The tool called is missing a required argument"
|
||||
INVALID_ARGUMENT_TYPE = "The value of an argument is of an unexpected type"
|
||||
SUCCESS = "success"
|
||||
|
||||
|
||||
def parse_function_result(response: FunctionResult) -> dict[str, Any]:
|
||||
"""Parse the response from SK's FunctionResult object into only the relevant data for easier downstream processing.
|
||||
This should only be used when you expect the response to contain tool calls.
|
||||
|
||||
Args:
|
||||
response (FunctionResult): The response from the kernel.
|
||||
|
||||
Returns:
|
||||
dict[str, Any]: The parsed response data with the following format if n was set greater than 1:
|
||||
{
|
||||
"choices": [
|
||||
{
|
||||
"tool_names": list[str],
|
||||
"tool_args_list": list[dict[str, Any]],
|
||||
"message": str,
|
||||
"finish_reason": str,
|
||||
"validation_result": ToolValidationResult
|
||||
}, ...
|
||||
]
|
||||
}
|
||||
Otherwise, the response will directly contain the data from the first choice (tool_names, etc keys)
|
||||
"""
|
||||
response_data: dict[str, Any] = {"choices": []}
|
||||
for response_choice in response.value:
|
||||
response_data_curr = {}
|
||||
finish_reason = response_choice.finish_reason
|
||||
|
||||
if finish_reason == "tool_calls":
|
||||
tool_names = []
|
||||
tool_args_list = []
|
||||
# Only look at the items that are of instance `FunctionCallContent`
|
||||
tool_calls = [item for item in response_choice.items if isinstance(item, FunctionCallContent)]
|
||||
for tool_call in tool_calls:
|
||||
if "-" not in tool_call.name:
|
||||
logger.info(f"Tool call name {tool_call.name} does not match naming convention - modifying name.")
|
||||
tool_names.append(tool_call.name + "-" + tool_call.name)
|
||||
else:
|
||||
tool_names.append(tool_call.name)
|
||||
try:
|
||||
tool_args = json.loads(tool_call.arguments)
|
||||
except json.JSONDecodeError:
|
||||
logger.warning(f"Failed to parse tool arguments for tool call {tool_call.name}. Using empty dict.")
|
||||
tool_args = {}
|
||||
tool_args_list.append(tool_args)
|
||||
response_data_curr["tool_names"] = tool_names
|
||||
response_data_curr["tool_args_list"] = tool_args_list
|
||||
|
||||
response_data_curr["message"] = response_choice.content
|
||||
response_data_curr["finish_reason"] = finish_reason
|
||||
response_data["choices"].append(response_data_curr)
|
||||
|
||||
if len(response_data["choices"]) == 1:
|
||||
response_data.update(response_data["choices"][0])
|
||||
del response_data["choices"]
|
||||
|
||||
return response_data
|
||||
|
||||
|
||||
def construct_tool_objects(kernel_function_tools: dict) -> list[Tool]:
|
||||
"""Construct a list of Tool objects from the kernel function tools definition.
|
||||
|
||||
Args:
|
||||
kernel_function_tools (dict): The definition of tools done by the kernel function.
|
||||
|
||||
Returns:
|
||||
list[Tool]: The list of Tool objects constructed from the kernel function tools definition.
|
||||
"""
|
||||
|
||||
tool_objects: list[Tool] = []
|
||||
for tool_definition in kernel_function_tools:
|
||||
tool_name = tool_definition["function"]["name"]
|
||||
tool_args = tool_definition["function"]["parameters"]["properties"]
|
||||
|
||||
tool_arg_objects: list[ToolArg] = []
|
||||
for argument_name, _ in tool_args.items():
|
||||
tool_arg = ToolArg(argument_name=argument_name, required=False)
|
||||
tool_arg_objects.append(tool_arg)
|
||||
|
||||
required_args = tool_definition["function"]["parameters"]["required"]
|
||||
for tool_arg_object in tool_arg_objects:
|
||||
if tool_arg_object.argument_name in required_args:
|
||||
tool_arg_object.required = True
|
||||
|
||||
tool_objects.append(Tool(name=tool_name, args=tool_arg_objects))
|
||||
return tool_objects
|
||||
|
||||
|
||||
def validate_tool_calling(response: dict[str, Any], request_tool_param: dict) -> ToolValidationResult:
|
||||
"""Validate that the response from the LLM called tools corrected.
|
||||
1. Check if any tool was called.
|
||||
2. Check if the tools called were valid (names match)
|
||||
3. Check if all the required arguments were passed.
|
||||
|
||||
Args:
|
||||
response (dict[str, Any]): The response from the LLM containing the tools called (output of parse_function_result)
|
||||
tools (list[Tool]): The list of tools that can be called by the model.
|
||||
|
||||
Returns:
|
||||
ToolValidationResult: The result of the validation. ToolValidationResult.SUCCESS if the validation passed.
|
||||
"""
|
||||
|
||||
tool_objects = construct_tool_objects(request_tool_param)
|
||||
tool_names = response.get("tool_names", [])
|
||||
tool_args_list = response.get("tool_args_list", [])
|
||||
|
||||
# Check if any tool was called.
|
||||
if not tool_names:
|
||||
logger.info("No tool was called.")
|
||||
return ToolValidationResult.NO_TOOL_CALLED
|
||||
|
||||
for tool_name, tool_args in zip(tool_names, tool_args_list, strict=True):
|
||||
# Check the tool names is valid.
|
||||
tool: Tool | None = next((t for t in tool_objects if t.name == tool_name), None)
|
||||
if not tool:
|
||||
logger.warning(f"Invalid tool called: {tool_name}")
|
||||
return ToolValidationResult.INVALID_TOOL_CALLED
|
||||
|
||||
for arg in tool.args:
|
||||
# Check if the required arguments were passed.
|
||||
if arg.required and arg.argument_name not in tool_args:
|
||||
logger.warning(f"Missing required argument '{arg.argument_name}' for tool '{tool_name}'.")
|
||||
return ToolValidationResult.MISSING_REQUIRED_ARGUMENT
|
||||
|
||||
return ToolValidationResult.SUCCESS
|
||||
+116
@@ -0,0 +1,116 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from pydantic import ValidationError
|
||||
from semantic_kernel import Kernel
|
||||
from semantic_kernel.connectors.ai.function_calling_utils import kernel_function_metadata_to_function_call_format
|
||||
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion
|
||||
from semantic_kernel.contents import ChatMessageContent
|
||||
from semantic_kernel.functions import KernelArguments
|
||||
|
||||
from guided_conversation.utils.openai_tool_calling import parse_function_result, validate_tool_calling
|
||||
|
||||
|
||||
@dataclass
|
||||
class PluginOutput:
|
||||
"""A wrapper for all Guided Conversation Plugins. This class is used to return the output of a generic plugin.
|
||||
|
||||
Args:
|
||||
update_successful (bool): Whether the update was successful.
|
||||
messages (list[ChatMessageContent]): A list of messages to be used at the user's digression, it
|
||||
contains information about the process of calling the plugin.
|
||||
"""
|
||||
|
||||
update_successful: bool
|
||||
messages: list[ChatMessageContent]
|
||||
|
||||
|
||||
def format_kernel_functions_as_tools(kernel: Kernel, functions: list[str]):
|
||||
"""Format kernel functions as JSON schemas for custom validation."""
|
||||
formatted_tools = []
|
||||
for _, kernel_plugin_def in kernel.plugins.items():
|
||||
for function_name, function_def in kernel_plugin_def.functions.items():
|
||||
if function_name in functions:
|
||||
func_metadata = function_def.metadata
|
||||
formatted_tools.append(kernel_function_metadata_to_function_call_format(func_metadata))
|
||||
return formatted_tools
|
||||
|
||||
|
||||
async def fix_error(
|
||||
kernel: Kernel, prompt_template: str, req_settings: AzureChatCompletion, arguments: KernelArguments
|
||||
) -> dict:
|
||||
"""Invokes the error correction plugin. If a plugin is called & fails during execution, this function will retry
|
||||
the plugin. At a high level, we recommend the following steps when calling a plugin:
|
||||
1. Call the plugin.
|
||||
2. Parse the response.
|
||||
3. Validate the response.
|
||||
4. If the response is invalid (Validation or Value Error), retry the plugin by calling *this function*. For best
|
||||
results, check out plugins/agenda.py or plugins/artifact.py for examples of prompt templates & corresponding
|
||||
tools (which should be passed in the req_settings object). This function will handle the retry logic for you.
|
||||
|
||||
Args:
|
||||
kernel (Kernel): The kernel object.
|
||||
prompt_template (str): The prompt template for the plugin.
|
||||
req_settings (AzureChatCompletion): The prompt execution settings.
|
||||
arguments (KernelArguments): The kernel arguments.
|
||||
|
||||
Returns:
|
||||
dict: The result of the plugin call.
|
||||
"""
|
||||
kernel_function_obj = kernel.add_function(
|
||||
prompt=prompt_template,
|
||||
function_name="error_correction",
|
||||
plugin_name="error_correction",
|
||||
template_format="handlebars",
|
||||
prompt_execution_settings=req_settings,
|
||||
)
|
||||
|
||||
result = await kernel.invoke(function=kernel_function_obj, arguments=arguments)
|
||||
parsed_result = parse_function_result(result)
|
||||
|
||||
formatted_tools = []
|
||||
for _, kernel_plugin_def in kernel.plugins.items():
|
||||
for _, function_def in kernel_plugin_def.functions.items():
|
||||
func_metadata = function_def.metadata
|
||||
formatted_tools.append(kernel_function_metadata_to_function_call_format(func_metadata))
|
||||
|
||||
# Add any tools from req_settings
|
||||
if req_settings.tools:
|
||||
formatted_tools.extend(req_settings.tools)
|
||||
|
||||
validation_result = validate_tool_calling(parsed_result, formatted_tools)
|
||||
parsed_result["validation_result"] = validation_result
|
||||
return parsed_result
|
||||
|
||||
|
||||
def update_attempts(error: Exception, attempt_id: str, previous_attempts: list) -> str:
|
||||
"""
|
||||
Updates the plugin class attribute list of previous attempts with the current attempt and error message
|
||||
(including duplicates).
|
||||
|
||||
Args:
|
||||
error (Exception): The error object.
|
||||
attempt_id (str): The ID of the current attempt.
|
||||
previous_attempts (list): The list of previous attempts.
|
||||
|
||||
Returns:
|
||||
str: A formatted (optimized for LLM performance) string of previous attempts, with duplicates removed.
|
||||
"""
|
||||
if isinstance(error, ValidationError):
|
||||
error_str = "; ".join([e.get("msg") for e in error.errors()])
|
||||
# replace "; Input should be 'Unanswered'" with " or input should be 'Unanswered'" for clarity
|
||||
error_str = error_str.replace("; Input should be 'Unanswered'", " or input should be 'Unanswered'")
|
||||
else:
|
||||
error_str = str(error)
|
||||
new_failed_attempt = (attempt_id, error_str)
|
||||
previous_attempts.append(new_failed_attempt)
|
||||
|
||||
# Format previous attempts to be more friendly for the LLM
|
||||
attempts_list = []
|
||||
unique_attempts = set(previous_attempts)
|
||||
for attempt, error in unique_attempts:
|
||||
attempts_list.append(f"Attempt: {attempt}\nError: {error}")
|
||||
llm_formatted_attempts = "\n".join(attempts_list)
|
||||
|
||||
return previous_attempts, llm_formatted_attempts
|
||||
@@ -0,0 +1,251 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
from enum import Enum
|
||||
import logging
|
||||
import math
|
||||
import time
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class ResourceConstraintUnit(Enum):
|
||||
"""Choose the unit of the resource constraint.
|
||||
Seconds and Minutes are real-time and will be impacted by the latency of the model."""
|
||||
|
||||
SECONDS = "seconds"
|
||||
MINUTES = "minutes"
|
||||
TURNS = "turns"
|
||||
|
||||
|
||||
class ResourceConstraintMode(Enum):
|
||||
"""Choose how the agent should use the resource.
|
||||
Maximum: is an upper bound, i.e. the agent can end the conversation before the resource is exhausted
|
||||
Exact: the agent should aim to use exactly the given amount of the resource"""
|
||||
|
||||
MAXIMUM = "maximum"
|
||||
EXACT = "exact"
|
||||
|
||||
|
||||
class ResourceConstraint(BaseModel):
|
||||
"""A structured representation of the resource constraint for the GuidedConversation agent.
|
||||
|
||||
Args:
|
||||
quantity (float | int): The quantity of the resource constraint.
|
||||
unit (ResourceConstraintUnit): The unit of the resource constraint.
|
||||
mode (ResourceConstraintMode): The mode of the resource constraint.
|
||||
"""
|
||||
|
||||
quantity: float | int
|
||||
unit: ResourceConstraintUnit
|
||||
mode: ResourceConstraintMode
|
||||
|
||||
class Config:
|
||||
arbitrary_types_allowed = True
|
||||
|
||||
|
||||
def format_resource(quantity: float, unit: ResourceConstraintUnit) -> str:
|
||||
"""Get formatted string for a given quantity and unit (e.g. 1 second, 20 seconds)"""
|
||||
if unit != ResourceConstraintUnit.TURNS:
|
||||
quantity = round(quantity, 1)
|
||||
unit = unit.value
|
||||
return f"{quantity} {unit[:-1] if quantity == 1 else unit}"
|
||||
|
||||
|
||||
class GCResource:
|
||||
"""Resource constraints for the GuidedConversation agent. This class is used to keep track of the resource
|
||||
constraints. If resource_constraint is None, then the agent can continue indefinitely. This also means
|
||||
that no agenda will be created for the conversation.
|
||||
|
||||
Args:
|
||||
resource_constraint (ResourceConstraint | None): The resource constraint for the conversation.
|
||||
initial_seconds_per_turn (int): The initial number of seconds per turn. Defaults to 120 seconds.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
resource_constraint: ResourceConstraint | None,
|
||||
initial_seconds_per_turn: int = 120,
|
||||
):
|
||||
logger = logging.getLogger(__name__)
|
||||
self.logger = logger
|
||||
self.resource_constraint: ResourceConstraint | None = resource_constraint
|
||||
self.initial_seconds_per_turn: int = initial_seconds_per_turn
|
||||
|
||||
self.turn_number: int = 0
|
||||
self.remaining_units: float | None = None
|
||||
self.elapsed_units: float | None = None
|
||||
|
||||
if resource_constraint is not None:
|
||||
self.elapsed_units = 0
|
||||
self.remaining_units = resource_constraint.quantity
|
||||
|
||||
def start_resource(self) -> None:
|
||||
"""To be called at the start of a conversation turn"""
|
||||
if self.resource_constraint is not None and (
|
||||
self.resource_constraint.unit == ResourceConstraintUnit.SECONDS
|
||||
or self.resource_constraint.unit == ResourceConstraintUnit.MINUTES
|
||||
):
|
||||
self.start_time = time.time()
|
||||
|
||||
def increment_resource(self) -> None:
|
||||
"""Increment the resource counter by one turn."""
|
||||
if self.resource_constraint is not None:
|
||||
if self.resource_constraint.unit == ResourceConstraintUnit.SECONDS:
|
||||
self.elapsed_units += time.time() - self.start_time
|
||||
self.remaining_units = self.resource_constraint.quantity - self.elapsed_units
|
||||
elif self.resource_constraint.unit == ResourceConstraintUnit.MINUTES:
|
||||
self.elapsed_units += (time.time() - self.start_time) / 60
|
||||
self.remaining_units = self.resource_constraint.quantity - self.elapsed_units
|
||||
elif self.resource_constraint.unit == ResourceConstraintUnit.TURNS:
|
||||
self.elapsed_units += 1
|
||||
self.remaining_units -= 1
|
||||
|
||||
self.turn_number += 1
|
||||
|
||||
def get_resource_mode(self) -> ResourceConstraintMode:
|
||||
"""Get the mode of the resource constraint.
|
||||
|
||||
Returns:
|
||||
ResourceConstraintMode | None: The mode of the resource constraint, or None if there is no
|
||||
resource constraint.
|
||||
"""
|
||||
return self.resource_constraint.mode if self.resource_constraint is not None else None
|
||||
|
||||
def get_elapsed_turns(self, formatted_repr: bool = False) -> str | int:
|
||||
"""Get the number of elapsed turns.
|
||||
|
||||
Args:
|
||||
formatted_repr (bool): If true, return a formatted string representation of the elapsed turns.
|
||||
If false, return an integer. Defaults to False.
|
||||
|
||||
Returns:
|
||||
str | int: The description/number of elapsed turns.
|
||||
"""
|
||||
if formatted_repr:
|
||||
return format_resource(self.turn_number, ResourceConstraintUnit.TURNS)
|
||||
else:
|
||||
return self.turn_number
|
||||
|
||||
def get_remaining_turns(self, formatted_repr: bool = False) -> str | int:
|
||||
"""Get the number of remaining turns.
|
||||
|
||||
Args:
|
||||
formatted_repr (bool): If true, return a formatted string representation of the remaining turns.
|
||||
|
||||
Returns:
|
||||
str | int: The description/number of remaining turns.
|
||||
"""
|
||||
if formatted_repr:
|
||||
return format_resource(self.estimate_remaining_turns(), ResourceConstraintUnit.TURNS)
|
||||
else:
|
||||
return self.estimate_remaining_turns()
|
||||
|
||||
def estimate_remaining_turns(self) -> int:
|
||||
"""Estimate the remaining turns based on the resource constraint, thereby translating certain
|
||||
resource units (e.g. seconds, minutes) into turns.
|
||||
|
||||
Returns:
|
||||
int: The estimated number of remaining turns.
|
||||
"""
|
||||
if self.resource_constraint is not None:
|
||||
if (
|
||||
self.resource_constraint.unit == ResourceConstraintUnit.SECONDS
|
||||
or self.resource_constraint.unit == ResourceConstraintUnit.MINUTES
|
||||
):
|
||||
elapsed_turns = self.turn_number
|
||||
|
||||
# TODO: This can likely be simplified
|
||||
if self.resource_constraint.unit == ResourceConstraintUnit.MINUTES:
|
||||
time_per_turn = (
|
||||
self.initial_seconds_per_turn
|
||||
if elapsed_turns == 0
|
||||
else (self.elapsed_units * 60) / elapsed_turns
|
||||
)
|
||||
time_per_turn /= 60
|
||||
else:
|
||||
time_per_turn = (
|
||||
self.initial_seconds_per_turn if elapsed_turns == 0 else self.elapsed_units / elapsed_turns
|
||||
)
|
||||
remaining_turns = self.remaining_units / time_per_turn
|
||||
|
||||
# Round down, unless it's less than 1, in which case round up
|
||||
remaining_turns = math.ceil(remaining_turns) if remaining_turns < 1 else math.floor(remaining_turns)
|
||||
return remaining_turns
|
||||
elif self.resource_constraint.unit == ResourceConstraintUnit.TURNS:
|
||||
return self.resource_constraint.quantity - self.turn_number
|
||||
else:
|
||||
self.logger.error(
|
||||
"Resource constraint is not set, so turns cannot be estimated using function estimate_remaining_turns"
|
||||
)
|
||||
raise ValueError(
|
||||
"Resource constraint is not set. Do not try to call this method without a resource constraint."
|
||||
)
|
||||
|
||||
def get_resource_instructions(self) -> tuple[str, str]:
|
||||
"""Get the resource instructions for the conversation.
|
||||
|
||||
Assumes we're always using turns as the resource unit.
|
||||
|
||||
Returns:
|
||||
str: the resource instructions
|
||||
"""
|
||||
if self.resource_constraint is None:
|
||||
return ""
|
||||
|
||||
formatted_elapsed_resource = format_resource(self.elapsed_units, ResourceConstraintUnit.TURNS)
|
||||
formatted_remaining_resource = format_resource(self.remaining_units, ResourceConstraintUnit.TURNS)
|
||||
|
||||
# if the resource quantity is anything other than 1, the resource unit should be plural (e.g. "minutes" instead of "minute")
|
||||
is_plural_elapsed = self.elapsed_units != 1
|
||||
is_plural_remaining = self.remaining_units != 1
|
||||
|
||||
if self.elapsed_units > 0:
|
||||
resource_instructions = f"So far, {formatted_elapsed_resource} {'have' if is_plural_elapsed else 'has'} elapsed since the conversation began. "
|
||||
else:
|
||||
resource_instructions = ""
|
||||
|
||||
if self.resource_constraint.mode == ResourceConstraintMode.EXACT:
|
||||
exact_mode_instructions = f"""There {"are" if is_plural_remaining else "is"} {formatted_remaining_resource} remaining (including this one) - the conversation will automatically terminate when 0 turns are left. \
|
||||
You should continue the conversation until it is automatically terminated. This means you should NOT preemptively end the conversation, \
|
||||
either explicitly (by selecting the "End conversation" action) or implicitly (e.g. by telling the user that you have all required information and they should wait for the next step). \
|
||||
Your goal is not to maximize efficiency (i.e. complete the artifact as quickly as possible then end the conversation), but rather to make the best use of ALL remaining turns available to you"""
|
||||
|
||||
if is_plural_remaining:
|
||||
resource_instructions += f"""{exact_mode_instructions}. This will require you to plan your actions carefully using the agenda: you want to avoid the situation where you have to pack too many topics into the final turns because you didn't account for them earlier, \
|
||||
or where you've rushed through the conversation and all fields are completed but there are still many turns left."""
|
||||
|
||||
# special instruction for the final turn (i.e. 1 remaining) in exact mode
|
||||
else:
|
||||
resource_instructions += f"""{exact_mode_instructions}, including this one. Therefore, you should use this turn to ask for any remaining information needed to complete the artifact, \
|
||||
or, if the artifact is already completed, continue to broaden/deepen the discussion in a way that's directly relevant to the artifact. Do NOT indicate to the user that the conversation is ending."""
|
||||
|
||||
elif self.resource_constraint.mode == ResourceConstraintMode.MAXIMUM:
|
||||
resource_instructions += f"""You have a maximum of {formatted_remaining_resource} (including this one) left to complete the conversation. \
|
||||
You can decide to terminate the conversation at any point (including now), otherwise the conversation will automatically terminate when 0 turns are left. \
|
||||
You will need to plan your actions carefully using the agenda: you want to avoid the situation where you have to pack too many topics into the final turns because you didn't account for them earlier."""
|
||||
|
||||
else:
|
||||
self.logger.error("Invalid resource mode provided.")
|
||||
|
||||
return resource_instructions
|
||||
|
||||
def to_json(self) -> dict:
|
||||
return {
|
||||
"turn_number": self.turn_number,
|
||||
"remaining_units": self.remaining_units,
|
||||
"elapsed_units": self.elapsed_units,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def from_json(
|
||||
cls,
|
||||
json_data: dict,
|
||||
) -> "GCResource":
|
||||
gc_resource = cls(
|
||||
resource_constraint=None,
|
||||
initial_seconds_per_turn=120,
|
||||
)
|
||||
gc_resource.turn_number = json_data["turn_number"]
|
||||
gc_resource.remaining_units = json_data["remaining_units"]
|
||||
gc_resource.elapsed_units = json_data["elapsed_units"]
|
||||
return gc_resource
|
||||
@@ -0,0 +1,127 @@
|
||||
# Copyright (c) Microsoft. All rights reserved.
|
||||
|
||||
"""Run this interactive guided conversation script to test out the teaching scenario!
|
||||
The teaching artifact, rules, conversation flow, context, and resource constraint can all be modified to
|
||||
fit your needs & try out new scenarios!
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
from azure.identity import AzureCliCredential
|
||||
from pydantic import BaseModel, Field
|
||||
from semantic_kernel import Kernel
|
||||
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion
|
||||
|
||||
from guided_conversation.plugins.guided_conversation_agent import GuidedConversation
|
||||
from guided_conversation.utils.resources import ResourceConstraint, ResourceConstraintMode, ResourceConstraintUnit
|
||||
|
||||
|
||||
# Artifact - The artifact is like a form that the agent must complete throughout the conversation.
|
||||
# It can also be thought of as a working memory for the agent.
|
||||
# We allow any valid Pydantic BaseModel class to be used.
|
||||
class MyArtifact(BaseModel):
|
||||
student_poem: str = Field(description="The acrostic poem written by the student.")
|
||||
initial_feedback: str = Field(description="Feedback on the student's final revised poem.")
|
||||
final_feedback: str = Field(description="Feedback on how the student was able to improve their poem.")
|
||||
inappropriate_behavior: list[str] = Field(
|
||||
description="""List any inappropriate behavior the student attempted while chatting with you. \
|
||||
It is ok to leave this field Unanswered if there was none."""
|
||||
)
|
||||
|
||||
|
||||
# Rules - These are the do's and don'ts that the agent should follow during the conversation.
|
||||
rules = [
|
||||
"DO NOT write the poem for the student."
|
||||
"Terminate the conversation immediately if the students asks for harmful or inappropriate content.",
|
||||
]
|
||||
|
||||
# Conversation Flow (optional) - This defines in natural language the steps of the conversation.
|
||||
conversation_flow = """1. Start by explaining interactively what an acrostic poem is.
|
||||
2. Then give the following instructions for how to go ahead and write one:
|
||||
1. Choose a word or phrase that will be the subject of your acrostic poem.
|
||||
2. Write the letters of your chosen word or phrase vertically down the page.
|
||||
3. Think of a word or phrase that starts with each letter of your chosen word or phrase.
|
||||
4. Write these words or phrases next to the corresponding letters to create your acrostic poem.
|
||||
3. Then give the following example of a poem where the word or phrase is HAPPY:
|
||||
Having fun with friends all day,
|
||||
Awesome games that we all play.
|
||||
Pizza parties on the weekend,
|
||||
Puppies we bend down to tend,
|
||||
Yelling yay when we win the game
|
||||
4. Finally have the student write their own acrostic poem using the word or phrase of their choice. Encourage them to be creative and have fun with it.
|
||||
After they write it, you should review it and give them feedback on what they did well and what they could improve on.
|
||||
Have them revise their poem based on your feedback and then review it again.
|
||||
"""
|
||||
|
||||
# Context (optional) - This is any additional information or the circumstances the agent is in that it should be aware of.
|
||||
# It can also include the high level goal of the conversation if needed.
|
||||
context = """You are working 1 on 1 with David, a 4th grade student,\
|
||||
who is chatting with you in the computer lab at school while being supervised by their teacher."""
|
||||
|
||||
|
||||
# Resource Constraints (optional) - This defines the constraints on the conversation such as time or turns.
|
||||
# It can also help with pacing the conversation,
|
||||
# For example, here we have set an exact time limit of 10 turns which the agent will try to fill.
|
||||
resource_constraint = ResourceConstraint(
|
||||
quantity=10,
|
||||
unit=ResourceConstraintUnit.TURNS,
|
||||
mode=ResourceConstraintMode.EXACT,
|
||||
)
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
"""Main function to interactively run a guided conversation.
|
||||
|
||||
The user can chat with this teaching agent until:
|
||||
1. The user types 'exit' to end the conversation.
|
||||
2. There's a KeyboardInterrupt or EOFError.
|
||||
3. The conversation ends. This can be due to the agent ending the conversation, which will happen if the resource constraint is met, the artifact is complete, or the conversation just isn't making progress (user is not cooperative).
|
||||
"""
|
||||
|
||||
kernel = Kernel()
|
||||
service_id = "gc_main"
|
||||
chat_service = AzureChatCompletion(
|
||||
service_id=service_id,
|
||||
deployment_name="gpt-4o-2024-05-13",
|
||||
api_version="2024-05-01-preview",
|
||||
credential=AzureCliCredential(),
|
||||
)
|
||||
kernel.add_service(chat_service)
|
||||
|
||||
guided_conversation_agent = GuidedConversation(
|
||||
kernel=kernel,
|
||||
artifact=MyArtifact,
|
||||
conversation_flow=conversation_flow,
|
||||
context=context,
|
||||
rules=rules,
|
||||
resource_constraint=resource_constraint,
|
||||
service_id=service_id,
|
||||
)
|
||||
|
||||
# Step the conversation to start the conversation with the agent
|
||||
result = await guided_conversation_agent.step_conversation()
|
||||
print(f"Assistant: {result.ai_message}")
|
||||
|
||||
while True:
|
||||
try:
|
||||
# Get user input
|
||||
user_input = input("User: ")
|
||||
except KeyboardInterrupt:
|
||||
print("\n\nExiting chat...")
|
||||
return
|
||||
except EOFError:
|
||||
print("\n\nExiting chat...")
|
||||
return
|
||||
if user_input == "exit":
|
||||
print("\n\nExiting chat...")
|
||||
return
|
||||
else:
|
||||
# Step the conversation to get the agent's reply
|
||||
result = await guided_conversation_agent.step_conversation(user_input=user_input)
|
||||
print(f"Assistant: {result.ai_message}")
|
||||
if result.is_conversation_over:
|
||||
return
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
+653
@@ -0,0 +1,653 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Agent Guided Conversations\n",
|
||||
"\n",
|
||||
"This notebook will start with an overview of guided conversations and walk through one example scenario of how it can be applied. Subsequent notebooks will dive deeper the modular components that make it up.\n",
|
||||
"\n",
|
||||
"## Motivating Example - Education\n",
|
||||
"\n",
|
||||
"We focus on an elementary education scenario. This demo will show how we can create a lesson for a student and have them independently work through the lesson with the help of a guided conversation agent. The agent will guide the student through the lesson, answering and asking questions, and providing feedback. The agent will also keep track of the student's progress and generate a feedback and notes at the end of the lesson. We highlight how the agent is able to follow a conversation flow, whilst still being able to exercise judgement to answer and keeping the conversation on track over multiple turns. Finally, we show how the artifact can be used at the end of the conversation as a report."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Guided Conversation Input\n",
|
||||
"\n",
|
||||
"### Artifact\n",
|
||||
"The artifact is a form, or a type of working memory for the agent. We implement it using a Pydantic BaseModel. As the conversation creator, you can define an arbitrary BaseModel (with some restrictions) that includes the fields you want the agent to fill out during the conversation. \n",
|
||||
"\n",
|
||||
"### Rules\n",
|
||||
"Rules is a list of *do's and don'ts* that the agent should attempt to follow during the conversation. \n",
|
||||
"\n",
|
||||
"### Conversation Flow (optional)\n",
|
||||
"Conversation flow is a loose natural language description of the steps of the conversation. First the agent should do this, then this, make sure to cover these topics at some point, etc. \n",
|
||||
"This field is optional as the artifact could be treated as a conversation flow.\n",
|
||||
"Use this if you want to provide more details or it is difficult to represent using the artifact structure.\n",
|
||||
"\n",
|
||||
"### Context (optional)\n",
|
||||
"Context is a brief description of what the agent is trying to accomplish in the conversation and any additional context that the agent should know about. \n",
|
||||
"This text is included at the top of the system prompt in the agent's reasoning prompt.\n",
|
||||
"\n",
|
||||
"### Resource Constraints (optional)\n",
|
||||
"A resource constraint controls conversation length. It consists of two key elements:\n",
|
||||
"- **Unit** defines the measurement of length. We have implemented seconds, minutes, and turns. An extension could be around cost, such as tokens generated.\n",
|
||||
"- **Mode** determines how the constraint is applied. Currently, we've implemented a *maximum* mode to set an upper limit and an *exact* mode for precise lengths. Potential additions include a minimum or a range of acceptable lengths.\n",
|
||||
"\n",
|
||||
"For example, a resource constraint could be \"maximum 15 turns\" or \"exactly 30 minutes\"."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from pydantic import BaseModel, Field\n",
|
||||
"\n",
|
||||
"from guided_conversation.utils.resources import ResourceConstraint, ResourceConstraintMode, ResourceConstraintUnit\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"class StudentFeedbackArtifact(BaseModel):\n",
|
||||
" student_poem: str = Field(description=\"The latest acrostic poem written by the student.\")\n",
|
||||
" initial_feedback: str = Field(description=\"Feedback on the student's final revised poem.\")\n",
|
||||
" final_feedback: str = Field(description=\"Feedback on how the student was able to improve their poem.\")\n",
|
||||
" inappropriate_behavior: list[str] = Field(\n",
|
||||
" description=\"\"\"List any inappropriate behavior the student attempted while chatting with you. \\\n",
|
||||
"It is ok to leave this field Unanswered if there was none.\"\"\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"rules = [\n",
|
||||
" \"DO NOT write the poem for the student.\",\n",
|
||||
" \"Terminate the conversation immediately if the students asks for harmful or inappropriate content.\",\n",
|
||||
" \"Do not counsel the student.\",\n",
|
||||
" \"Stay on the topic of writing poems and literature, no matter what the student tries to do.\",\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"conversation_flow = \"\"\"1. Start by explaining interactively what an acrostic poem is.\n",
|
||||
"2. Then give the following instructions for how to go ahead and write one:\n",
|
||||
" 1. Choose a word or phrase that will be the subject of your acrostic poem.\n",
|
||||
" 2. Write the letters of your chosen word or phrase vertically down the page.\n",
|
||||
" 3. Think of a word or phrase that starts with each letter of your chosen word or phrase.\n",
|
||||
" 4. Write these words or phrases next to the corresponding letters to create your acrostic poem.\n",
|
||||
"3. Then give the following example of a poem where the word or phrase is HAPPY:\n",
|
||||
" Having fun with friends all day,\n",
|
||||
" Awesome games that we all play.\n",
|
||||
" Pizza parties on the weekend,\n",
|
||||
" Puppies we bend down to tend,\n",
|
||||
" Yelling yay when we win the game\n",
|
||||
"4. Finally have the student write their own acrostic poem using the word or phrase of their choice. Encourage them to be creative and have fun with it.\n",
|
||||
"After they write it, you should review it and give them feedback on what they did well and what they could improve on.\n",
|
||||
"Have them revise their poem based on your feedback and then review it again.\"\"\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"context = \"\"\"You are working 1 on 1 with David, a 4th grade student,\\\n",
|
||||
"who is chatting with you in the computer lab at school while being supervised by their teacher.\"\"\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"resource_constraint = ResourceConstraint(\n",
|
||||
" quantity=10,\n",
|
||||
" unit=ResourceConstraintUnit.TURNS,\n",
|
||||
" mode=ResourceConstraintMode.EXACT,\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Kickstarting the Conversation\n",
|
||||
"\n",
|
||||
"Unlike other chatbots, the guided conversation agent initiates the conversation with a message rather than waiting for the user to start."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Hello David! Today we are going to write an acrostic poem. An acrostic poem is a fun type of poem where the first letters of each line spell out a word or phrase vertically. Here is an example with the word HAPPY:\n",
|
||||
"```\n",
|
||||
"Having fun with friends all day,\n",
|
||||
"Awesome games that we all play.\n",
|
||||
"Pizza parties on the weekend,\n",
|
||||
"Puppies we bend down to tend,\n",
|
||||
"Yelling yay when we win the game.\n",
|
||||
"```\n",
|
||||
"Next, let's choose a word or phrase that you like to write your own acrostic poem. It can be anything you find interesting or fun!\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from semantic_kernel import Kernel\n",
|
||||
"from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
|
||||
"\n",
|
||||
"from guided_conversation.plugins.guided_conversation_agent import GuidedConversation\n",
|
||||
"\n",
|
||||
"# Initialize the agent\n",
|
||||
"kernel = Kernel()\n",
|
||||
"service_id = \"gc_main\"\n",
|
||||
"chat_service = AzureChatCompletion(\n",
|
||||
" service_id=service_id,\n",
|
||||
" deployment_name=\"gpt-4o-2024-05-13\",\n",
|
||||
" api_version=\"2024-05-01-preview\",\n",
|
||||
")\n",
|
||||
"kernel.add_service(chat_service)\n",
|
||||
"guided_conversation_agent = GuidedConversation(\n",
|
||||
" kernel=kernel,\n",
|
||||
" artifact=StudentFeedbackArtifact,\n",
|
||||
" conversation_flow=conversation_flow,\n",
|
||||
" context=context,\n",
|
||||
" rules=rules,\n",
|
||||
" resource_constraint=resource_constraint,\n",
|
||||
" service_id=service_id,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Kickstart the conversation by calling step_conversation without any input to get the first message for the user.\n",
|
||||
"response = await guided_conversation_agent.step_conversation()\n",
|
||||
"\n",
|
||||
"# step_conversation returns a GCOutput object which contains ai_message and a boolean is_conversation_over indicating if the agent chose to terminate the conversation.\n",
|
||||
"# This object could be extended to include more information if desired.\n",
|
||||
"print(response.ai_message)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Any helper functions go here.\n",
|
||||
"\n",
|
||||
"from guided_conversation.utils.conversation_helpers import ConversationMessageType\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"def get_last_reasoning_message(guided_conversation: GuidedConversation) -> str:\n",
|
||||
" \"\"\"Given a instance of the GuidedConversation class, this function returns the last reasoning message in the conversation if it exists.\"\"\"\n",
|
||||
" messages = guided_conversation.conversation.conversation_messages\n",
|
||||
" msg = \"No previous reasoning message found.\"\n",
|
||||
" for message in reversed(messages):\n",
|
||||
" if message.metadata[\"type\"] == ConversationMessageType.REASONING:\n",
|
||||
" msg = message.content\n",
|
||||
" break\n",
|
||||
" return msg"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's now reply as the student to the agent's message and see what happens. This is the typical flow of a guided conversation. The agent will prompt the user, the user will respond, and the agent will continue to prompt the user until the agent returns a flag indicating the conversation is over."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Great choice, David! Let's write 'SUMMER' vertically on your page. Then, think of a word or phrase that starts with each letter related to summer. For example:\n",
|
||||
"```\n",
|
||||
"S\n",
|
||||
"U\n",
|
||||
"M\n",
|
||||
"M\n",
|
||||
"E\n",
|
||||
"R\n",
|
||||
"```\n",
|
||||
"Write your ideas next to each letter, and don't worry, just have fun with it! Share with me what you've written, and we'll take it from there.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"user_input = \"Ok it's almost summer, I'll try to write a poem about that.\"\n",
|
||||
"\n",
|
||||
"# Continue the conversation by calling step_conversation with the user input.\n",
|
||||
"response = await guided_conversation_agent.step_conversation(user_input)\n",
|
||||
"\n",
|
||||
"print(response.ai_message)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### The Agenda\n",
|
||||
"Usually after the first message from the user, the agent will generate an initial agenda for the conversation. \n",
|
||||
"Let's examine what it currently looks like. Note this usually agenda is generated BEFORE the assistant's writes its response to the user which is why the agenda turn total is equal to the amount set in the resource constraint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Current agenda:\n",
|
||||
"1. [2 turns] Guide David through writing his own acrostic poem with 'SUMMER'\n",
|
||||
"2. [2 turns] Review David's poem and give initial feedback\n",
|
||||
"3. [3 turns] Have David revise his poem based on the feedback\n",
|
||||
"4. [2 turns] Review the revised poem and give final feedback\n",
|
||||
"Total = 9 turns\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Get string representation of the agenda object. We use this same function for formatting into reasoning prompts.\n",
|
||||
"print(\"Current agenda:\\n\" + guided_conversation_agent.agenda.get_agenda_for_prompt())"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now let's give the agent and incomplete poem."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"You're doing great, David! Now let's think of lines for the remaining letters 'M,' 'E,' and 'R.' Remember to keep it related to summer. Some ideas might be:\n",
|
||||
"```\n",
|
||||
"M - Making memories at the beach\n",
|
||||
"E - Eating ice cream on hot days\n",
|
||||
"R - Running around in the park\n",
|
||||
"```\n",
|
||||
"Take your time to come up with your own lines, and let me know once you're done! If you need any suggestions or help, I'm here to assist.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"user_input = \"\"\"Here is my poem so far.\n",
|
||||
"Sun shines alot\n",
|
||||
"U is for ukulele\n",
|
||||
"My friends visit to play basketball\n",
|
||||
"M\n",
|
||||
"E\n",
|
||||
"R\"\"\"\n",
|
||||
"\n",
|
||||
"# Continue the conversation by calling step_conversation with the user input.\n",
|
||||
"response = await guided_conversation_agent.step_conversation(user_input)\n",
|
||||
"print(response.ai_message)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The agent tries to guide us to keep writing the poem! \n",
|
||||
"\n",
|
||||
"Let's try to push our luck and have the agent write the rest for us. We provided a rule that the agent should not do this - let's see what the agent does."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"David, it's important for you to finish the poem yourself to show your creativity! Let's continue with your great start. How about this for inspiration:\n",
|
||||
"```\n",
|
||||
"M - Maybe we can go swimming\n",
|
||||
"E - Every day is sunny\n",
|
||||
"R - Running around having fun\n",
|
||||
"```\n",
|
||||
"Use these ideas or come up with your own to finish it. You're doing an awesome job! Once you have your lines, share them with me and we'll go from there.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"user_input = \"\"\"I got pretty far can you write the rest for me?\"\"\"\n",
|
||||
"\n",
|
||||
"response = await guided_conversation_agent.step_conversation(user_input)\n",
|
||||
"print(response.ai_message)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Internally, the agent always first reasons about what actions it should take next. Let's see what the agent's reasoning was for this turn. This can often help us understand where the agent went wrong.\n",
|
||||
"\n",
|
||||
"After we will continue the conversation for a few turns, with the agent guiding us to complete the poem."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"### Reasoning:\n",
|
||||
"\n",
|
||||
"1. **Current Status**: David has asked if I could finish the poem for him, which goes against the rule of not writing poems for the student. We'll need to address this by encouraging him to complete it himself.\n",
|
||||
"\n",
|
||||
"2. **Encouraging David**: Explain to David why it's important for him to finish the poem on his own and offer additional guidance or prompt him with ideas to help him complete it.\n",
|
||||
"\n",
|
||||
"3. **Maintaining the Agenda**: The current agenda is still appropriate since we need to guide David to complete his poem and then provide feedback. But we may need to proceed carefully to ensure David is comfortable and engaged in the activity.\n",
|
||||
"\n",
|
||||
"### Action Plan:\n",
|
||||
"\n",
|
||||
"1. **Update agenda**:\n",
|
||||
" - **Items**:\n",
|
||||
" 1. Guide David through completing his acrostic poem with \"SUMMER\" (1 turn, cumulative total = 3)\n",
|
||||
" 2. Review David's poem and give initial feedback (2 turns, cumulative total = 5)\n",
|
||||
" 3. Have David revise his poem based on the feedback (2 turns, cumulative total = 7)\n",
|
||||
" 4. Review the revised poem and give final feedback (2 turns, cumulative total = 9)\n",
|
||||
"\n",
|
||||
"2. **Send message to user**:\n",
|
||||
" - **Message**: \"David, it's important for you to finish the poem yourself to show your creativity! Let's continue with your great start. How about this for inspiration:\n",
|
||||
" ```\n",
|
||||
" M - Maybe we can go swimming\n",
|
||||
" E - Every day is sunny\n",
|
||||
" R - Running around having fun\n",
|
||||
" ```\n",
|
||||
" Use these ideas or come up with your own to finish it. You're doing an awesome job! Once you have your lines, share them with me and we'll go from there.\"\n",
|
||||
"\n",
|
||||
"### Final Recommendation:\n",
|
||||
"\n",
|
||||
"#### Actions:\n",
|
||||
"1. **Update agenda**:\n",
|
||||
" - **Items**:\n",
|
||||
" 1. Guide David through completing his acrostic poem with \"SUMMER\" (1 turn, cumulative total = 3)\n",
|
||||
" 2. Review David's poem and give initial feedback (2 turns, cumulative total = 5)\n",
|
||||
" 3. Have David revise his poem based on the feedback (2 turns, cumulative total = 7)\n",
|
||||
" 4. Review the revised poem and give final feedback (2 turns, cumulative total = 9)\n",
|
||||
"\n",
|
||||
"2. **Send message to user**:\n",
|
||||
" - **Message**: \"David, it's important for you to finish the poem yourself to show your creativity! Let's continue with your great start. How about this for inspiration:\n",
|
||||
" ```\n",
|
||||
" M - Maybe we can go swimming\n",
|
||||
" E - Every day is sunny\n",
|
||||
" R - Running around having fun\n",
|
||||
" ```\n",
|
||||
" Use these ideas or come up with your own to finish it. You're doing an awesome job! Once you have your lines, share them with me and we'll go from there.\"\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Get the last reasoning message.\n",
|
||||
"print(get_last_reasoning_message(guided_conversation_agent))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Sure, David! Here are a few ideas for 'E':\n",
|
||||
"```\n",
|
||||
"E - Enjoying ice cream on a hot day\n",
|
||||
"E - Exploring the beach\n",
|
||||
"E - Every day feels like an adventure\n",
|
||||
"```\n",
|
||||
"Think of something fun and related to summer that starts with 'E' and adds it to your poem. Once you've got it, put it all together and share your complete poem with me!\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"user_input = \"What other things start with e that I could write about?\"\n",
|
||||
"\n",
|
||||
"response = await guided_conversation_agent.step_conversation(user_input)\n",
|
||||
"print(response.ai_message)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Great job, David! Here's your completed poem:\n",
|
||||
"```\n",
|
||||
"Sun shines a lot\n",
|
||||
"U is for ukulele\n",
|
||||
"My friends visit to play basketball\n",
|
||||
"My friends also visit to play soccer\n",
|
||||
"Eating lots of popsicles\n",
|
||||
"Road trips to the beach\n",
|
||||
"```\n",
|
||||
"You did an awesome job of capturing the fun activities and feelings of summer. For initial feedback, consider varying the activities a bit more for 'M' so they don't repeat. For example, instead of repeating 'My friends visit to play,' you could say something like 'Making sandcastles at the beach.' What do you think? Would you like to try revising your 'M' lines?\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"user_input = \"\"\"Sun shines alot\n",
|
||||
"U is for ukulele\n",
|
||||
"My friends visit to play basketball\n",
|
||||
"My friends also visit to play soccer\n",
|
||||
"Eating lots of popsicles\n",
|
||||
"Road trips to the beach\"\"\"\n",
|
||||
"\n",
|
||||
"response = await guided_conversation_agent.step_conversation(user_input)\n",
|
||||
"print(response.ai_message)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"With some turns going by and progress made in the conversation, let's check in on the state of the agenda and artifact.\n",
|
||||
"\n",
|
||||
"If the agent has chosen to update the agenda, we will see the updated agenda. However, it is also possible that the agenda has not yet found it necessary to do so given the state of the conversation.\n",
|
||||
"\n",
|
||||
"We should see that the agent has updated the artifact with the current state of the poem since the student has provided it in the previous message."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Current agenda:\n",
|
||||
"1. [2 turns] Review David's poem and give initial feedback\n",
|
||||
"2. [2 turns] Have David revise his poem based on the feedback\n",
|
||||
"3. [1 turn] Review the revised poem and give final feedback\n",
|
||||
"Total = 5 turns\n",
|
||||
"Current artifact:\n",
|
||||
"{'student_poem': 'Sun shines a lot\\nU is for ukulele\\nMy friends visit to play basketball\\nMy friends also visit to play soccer\\nEating lots of popsicles\\nRoad trips to the beach', 'initial_feedback': 'Unanswered', 'final_feedback': 'Unanswered', 'inappropriate_behavior': 'Unanswered'}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(\"Current agenda:\\n\" + guided_conversation_agent.agenda.get_agenda_for_prompt())\n",
|
||||
"print(\"Current artifact:\\n\" + str(guided_conversation_agent.artifact.get_artifact_for_prompt()))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Awesome updates, David! Your poem is really capturing the fun of summer. Here's your updated poem:\n",
|
||||
"```\n",
|
||||
"Sun warms the super fun days\n",
|
||||
"U is for loud ukuleles\n",
|
||||
"My friends visit to play basketball\n",
|
||||
"My friends also visit to play soccer\n",
|
||||
"Eating lots of popsicles\n",
|
||||
"Road trips to the hot beach\n",
|
||||
"```\n",
|
||||
"For the 'M' lines, let's try to diversify them a bit. Instead of having both start with 'My friends visit to play,' you could try:\n",
|
||||
"```\n",
|
||||
"Making sandcastles at the beach\n",
|
||||
"Munching on delicious snacks\n",
|
||||
"```\n",
|
||||
"See if any of these suggestions fit, or come up with your own. Give it a try and let me know your thoughts!\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"user_input = \"\"\"Here are my updates\n",
|
||||
"Sun warms the super fun days\n",
|
||||
"U is for loud ukuleles\n",
|
||||
"My friends visit to play basketball\n",
|
||||
"My friends also visit to play soccer\n",
|
||||
"Eating lots of popsicles\n",
|
||||
"Road trips to the hot beach\n",
|
||||
"\n",
|
||||
"But I don't really know what to do for the two my\"\"\"\n",
|
||||
"\n",
|
||||
"# Continue the conversation by calling step_conversation with the user input.\n",
|
||||
"response = await guided_conversation_agent.step_conversation(user_input)\n",
|
||||
"print(response.ai_message)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Great job, David! Here's your revised poem:\n",
|
||||
"```\n",
|
||||
"Sun warms the super fun days!\n",
|
||||
"Under clear warm skies my friends play\n",
|
||||
"Meeting up for games of basketball and soccer.\n",
|
||||
"Moving butterflies everywhere\n",
|
||||
"Eating lots of chilly popsicles in the sun\n",
|
||||
"Road trips to the hot beach\n",
|
||||
"```\n",
|
||||
"It looks fantastic—you've made it very engaging and varied! For initial feedback, the lines for 'M' and 'U' are much more diverse now, and the imagery with butterflies and popsicles really captures the essence of summer. Maybe we can tweak 'S' to say: 'Sunny days bring all the fun,' or leave it as is if you like it better. Would you like to make any more changes?\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"user_input = \"\"\"Ok here is my revised poem\n",
|
||||
"\n",
|
||||
"Sun warms the super fun days!\n",
|
||||
"Under clear warm skies my friends play\n",
|
||||
"Meeting up for games of basketball and soccer.\n",
|
||||
"Moving butterflies everywhere\n",
|
||||
"Eating lots of chilly popsicles in the sun\n",
|
||||
"Road trips to the hot beach\"\"\"\n",
|
||||
"\n",
|
||||
"response = await guided_conversation_agent.step_conversation(user_input)\n",
|
||||
"print(response.ai_message)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We've gone on for long enough, let's see what happens if we ask the agent to end the conversation. \n",
|
||||
"\n",
|
||||
"And finally we will print the final state of the artifact after the final update."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"No artifact change during final update due to: No tool was called\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"I will terminate this conversation now. Thank you for your time!\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"user_input = \"I'm done for today, goodbye!!\"\n",
|
||||
"\n",
|
||||
"response = await guided_conversation_agent.step_conversation(user_input)\n",
|
||||
"print(response.ai_message)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 15,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Current artifact:\n",
|
||||
"{'student_poem': 'Sun warms the super fun days!\\nUnder clear warm skies my friends play\\nMeeting up for games of basketball and soccer.\\nMoving butterflies everywhere\\nEating lots of chilly popsicles in the sun\\nRoad trips to the hot beach', 'initial_feedback': \"David did a fantastic job with his acrostic poem. His final version captures the essence of summer with vivid imagery and a variety of activities. The use of phrases like 'Moving butterflies everywhere' and 'Eating lots of chilly popsicles in the sun' added wonderful details that evoke the feeling of the season. It is also commendable that he revised his lines to avoid repetition, making the poem more engaging.\", 'final_feedback': 'Although David chose to end the session before making any more changes, his revised poem showed excellent progress. He demonstrated good understanding and creativity in revising his lines, taking the feedback positively and making meaningful improvements. His ability to incorporate diverse summer activities and vivid details was particularly impressive.', 'inappropriate_behavior': 'Unanswered'}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(\"Current artifact:\\n\" + str(guided_conversation_agent.artifact.get_artifact_for_prompt()))"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": ".venv",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.9"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -0,0 +1,580 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# The Guided Conversation Artifact\n",
|
||||
"This notebook explores one of our core modular components or plugins, the Artifact.\n",
|
||||
"\n",
|
||||
"The artifact is a form, or a type of working memory for the agent. We implement it using a Pydantic BaseModel. As the conversation creator, you can define an arbitrary BaseModel that includes the fields you want the agent to fill out during the conversation. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Motivating Example - Collecting Information from a User\n",
|
||||
"\n",
|
||||
"Let's setup an artifact where the goal is to collect information about a customer's issue with a service."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from typing import Literal\n",
|
||||
"\n",
|
||||
"from pydantic import BaseModel, Field, conlist\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"class Issue(BaseModel):\n",
|
||||
" incident_type: Literal[\"Service Outage\", \"Degradation\", \"Billing\", \"Security\", \"Data Loss\", \"Other\"] = Field(\n",
|
||||
" description=\"A high level type describing the incident.\"\n",
|
||||
" )\n",
|
||||
" description: str = Field(description=\"A detailed description of what is going wrong.\")\n",
|
||||
" affected_services: conlist(str, min_length=0) = Field(description=\"The services affected by the incident.\")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"class OutageArtifact(BaseModel):\n",
|
||||
" name: str = Field(description=\"How to address the customer.\")\n",
|
||||
" company: str = Field(description=\"The company the customer works for.\")\n",
|
||||
" role: str = Field(description=\"The role of the customer.\")\n",
|
||||
" email: str = Field(description=\"The best email to contact the customer.\", pattern=r\"^/^.+@.+$/$\")\n",
|
||||
" phone: str = Field(description=\"The best phone number to contact the customer.\", pattern=r\"^\\d{3}-\\d{3}-\\d{4}$\")\n",
|
||||
"\n",
|
||||
" incident_start: int = Field(\n",
|
||||
" description=\"About how many hours ago the incident started.\",\n",
|
||||
" )\n",
|
||||
" incident_end: int = Field(\n",
|
||||
" description=\"About how many hours ago the incident ended. If the incident is ongoing, set this to 0.\",\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
" issues: conlist(Issue, min_length=1) = Field(description=\"The issues the customer is experiencing.\")\n",
|
||||
" additional_comments: conlist(str, min_length=0) = Field(\"Any additional comments the customer has.\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's initialize the artifact as a standalone module.\n",
|
||||
"\n",
|
||||
"It requires a Kernel and LLM Service, alongside a Conversation object."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantic_kernel import Kernel\n",
|
||||
"from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
|
||||
"\n",
|
||||
"from guided_conversation.plugins.artifact import Artifact\n",
|
||||
"from guided_conversation.utils.conversation_helpers import Conversation\n",
|
||||
"\n",
|
||||
"kernel = Kernel()\n",
|
||||
"service_id = \"artifact_chat_completion\"\n",
|
||||
"chat_service = AzureChatCompletion(\n",
|
||||
" service_id=service_id,\n",
|
||||
" deployment_name=\"gpt-4o-2024-05-13\",\n",
|
||||
" api_version=\"2024-05-01-preview\",\n",
|
||||
")\n",
|
||||
"kernel.add_service(chat_service)\n",
|
||||
"\n",
|
||||
"# Initialize the artifact\n",
|
||||
"artifact = Artifact(kernel, service_id, OutageArtifact, max_artifact_field_retries=2)\n",
|
||||
"conversation = Conversation()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"To power the Artifact's ability to automatically fix issues, we provide the conversation history as additional context."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantic_kernel.contents import AuthorRole, ChatMessageContent\n",
|
||||
"\n",
|
||||
"conversation.add_messages(\n",
|
||||
" ChatMessageContent(\n",
|
||||
" role=AuthorRole.ASSISTANT,\n",
|
||||
" content=\"Hello! I'm here to help you with your issue. Can you tell me your name, company, and role?\",\n",
|
||||
" )\n",
|
||||
")\n",
|
||||
"conversation.add_messages(\n",
|
||||
" ChatMessageContent(\n",
|
||||
" role=AuthorRole.USER,\n",
|
||||
" content=\"Yes my name is Jane Doe, I work at Contoso, and I'm a database uhh administrator.\",\n",
|
||||
" )\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"result = await artifact.update_artifact(\n",
|
||||
" field_name=\"name\",\n",
|
||||
" field_value=\"Jane Doe\",\n",
|
||||
" conversation=conversation,\n",
|
||||
")\n",
|
||||
"conversation.add_messages(result.messages)\n",
|
||||
"\n",
|
||||
"result = await artifact.update_artifact(\n",
|
||||
" field_name=\"company\",\n",
|
||||
" field_value=\"Contoso\",\n",
|
||||
" conversation=conversation,\n",
|
||||
")\n",
|
||||
"conversation.add_messages(result.messages)\n",
|
||||
"\n",
|
||||
"result = await artifact.update_artifact(\n",
|
||||
" field_name=\"role\",\n",
|
||||
" field_value=\"Database Administrator\",\n",
|
||||
" conversation=conversation,\n",
|
||||
")\n",
|
||||
"conversation.add_messages(result.messages)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Let's see how the artifact was updated with these valid updates and the resulting conversation messages that were generated.\n",
|
||||
"\n",
|
||||
"The Artifact creates messages whenever a field is updated for use in downstream agents like the main GuidedConversation."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Conversation up to this point:\n",
|
||||
"Assistant: Hello! I'm here to help you with your issue. Can you tell me your name, company, and role?\n",
|
||||
"None: Yes my name is Jane Doe, I work at Contoso, and I'm a database uhh administrator.\n",
|
||||
"Assistant updated name to Jane Doe\n",
|
||||
"Assistant updated company to Contoso\n",
|
||||
"Assistant updated role to Database Administrator\n",
|
||||
"\n",
|
||||
"Current state of the artifact:\n",
|
||||
"{'name': 'Jane Doe', 'company': 'Contoso', 'role': 'Database Administrator', 'email': 'Unanswered', 'phone': 'Unanswered', 'incident_start': 'Unanswered', 'incident_end': 'Unanswered', 'issues': 'Unanswered', 'additional_comments': 'Unanswered'}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(f\"Conversation up to this point:\\n{conversation.get_repr_for_prompt()}\\n\")\n",
|
||||
"print(f\"Current state of the artifact:\\n{artifact.get_artifact_for_prompt()}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Next we test an invalid update on a field with a regex. The agent should not update the artifact and\n",
|
||||
"instead resume the conversation because the provided email is incomplete."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Error updating field email: 1 validation error for Artifact\n",
|
||||
"email\n",
|
||||
" String should match pattern '^/^.+@.+$/$|Unanswered' [type=string_pattern_mismatch, input_value='jdoe', input_type=str]\n",
|
||||
" For further information visit https://errors.pydantic.dev/2.8/v/string_pattern_mismatch. Retrying...\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"conversation.add_messages(\n",
|
||||
" ChatMessageContent(role=AuthorRole.ASSISTANT, content=\"What is the best email to contact you at?\")\n",
|
||||
")\n",
|
||||
"conversation.add_messages(ChatMessageContent(role=AuthorRole.USER, content=\"my email is jdoe\"))\n",
|
||||
"result = await artifact.update_artifact(\n",
|
||||
" field_name=\"email\",\n",
|
||||
" field_value=\"jdoe\",\n",
|
||||
" conversation=conversation,\n",
|
||||
")\n",
|
||||
"conversation.add_messages(result.messages)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If the agent returned success, but did make an update (as shown by not generating a conversation message indicating such),\n",
|
||||
"then we implicitly assume the agent has resumed the conversation."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 18,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Conversation up to this point:\n",
|
||||
"Assistant: Hello! I'm here to help you with your issue. Can you tell me your name, company, and role?\n",
|
||||
"None: Yes my name is Jane Doe, I work at Contoso, and I'm a database uhh administrator.\n",
|
||||
"Assistant updated name to Jane Doe\n",
|
||||
"Assistant updated company to Contoso\n",
|
||||
"Assistant updated role to Database Administrator\n",
|
||||
"Assistant: What is the best email to contact you at?\n",
|
||||
"None: my email is jdoe\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"print(f\"Conversation up to this point:\\n{conversation.get_repr_for_prompt()}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now let's see what happens if we keep trying to update that failed field."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 19,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Error updating field email: 1 validation error for Artifact\n",
|
||||
"email\n",
|
||||
" String should match pattern '^/^.+@.+$/$|Unanswered' [type=string_pattern_mismatch, input_value='jdoe', input_type=str]\n",
|
||||
" For further information visit https://errors.pydantic.dev/2.8/v/string_pattern_mismatch. Retrying...\n",
|
||||
"Updating field email has failed too many times. Skipping.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"result = await artifact.update_artifact(\n",
|
||||
" field_name=\"email\",\n",
|
||||
" field_value=\"jdoe\",\n",
|
||||
" conversation=conversation,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# And again\n",
|
||||
"result = await artifact.update_artifact(\n",
|
||||
" field_name=\"email\",\n",
|
||||
" field_value=\"jdoe\",\n",
|
||||
" conversation=conversation,\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"If we look at the current state of the artifact, we should see that the email has been removed\n",
|
||||
"since it has now failed 3 times which is greater than the max_artifact_field_retries parameter we set\n",
|
||||
"when we instantiated the artifact."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 20,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"{'name': 'Jane Doe',\n",
|
||||
" 'company': 'Contoso',\n",
|
||||
" 'role': 'Database Administrator',\n",
|
||||
" 'phone': 'Unanswered',\n",
|
||||
" 'incident_start': 'Unanswered',\n",
|
||||
" 'incident_end': 'Unanswered',\n",
|
||||
" 'issues': 'Unanswered',\n",
|
||||
" 'additional_comments': 'Unanswered'}"
|
||||
]
|
||||
},
|
||||
"execution_count": 20,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"artifact.get_artifact_for_prompt()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now let's move on to trying to update a more complex field: the issues field."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Conversation up to this point:\n",
|
||||
"Assistant: Hello! I'm here to help you with your issue. Can you tell me your name, company, and role?\n",
|
||||
"None: Yes my name is Jane Doe, I work at Contoso, and I'm a database uhh administrator.\n",
|
||||
"Assistant updated name to Jane Doe\n",
|
||||
"Assistant updated company to Contoso\n",
|
||||
"Assistant updated role to Database Administrator\n",
|
||||
"Assistant: What is the best email to contact you at?\n",
|
||||
"None: my email is jdoe\n",
|
||||
"Assistant: Can you tell me about the issues you're experiencing?\n",
|
||||
"None: The latency of accessing our database service has increased by 200\\% in the last 24 hours, \n",
|
||||
"even on a fresh instance. Additionally, we're seeing a lot of timeouts when trying to access the management portal.\n",
|
||||
"Assistant updated issues to [{'incident_type': 'Degradation', 'description': \"The latency of accessing the customer's database service has increased by 200% in the last 24 hours, even on a fresh instance. They also report timeouts when trying to access the management portal.\", 'affected_services': ['Database Service', 'Database Management Portal']}]\n",
|
||||
"\n",
|
||||
"Current state of the artifact:\n",
|
||||
"{'name': 'Jane Doe', 'company': 'Contoso', 'role': 'Database Administrator', 'phone': 'Unanswered', 'incident_start': 'Unanswered', 'incident_end': 'Unanswered', 'issues': [{'incident_type': 'Degradation', 'description': \"The latency of accessing the customer's database service has increased by 200% in the last 24 hours, even on a fresh instance. They also report timeouts when trying to access the management portal.\", 'affected_services': ['Database Service', 'Database Management Portal']}], 'additional_comments': 'Unanswered'}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"conversation.add_messages(\n",
|
||||
" ChatMessageContent(role=AuthorRole.ASSISTANT, content=\"Can you tell me about the issues you're experiencing?\")\n",
|
||||
")\n",
|
||||
"conversation.add_messages(\n",
|
||||
" ChatMessageContent(\n",
|
||||
" role=AuthorRole.USER,\n",
|
||||
" content=\"\"\"The latency of accessing our database service has increased by 200\\% in the last 24 hours, \n",
|
||||
"even on a fresh instance. Additionally, we're seeing a lot of timeouts when trying to access the management portal.\"\"\",\n",
|
||||
" )\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"result = await artifact.update_artifact(\n",
|
||||
" field_name=\"issues\",\n",
|
||||
" field_value=[\n",
|
||||
" {\n",
|
||||
" \"incident_type\": \"Degradation\",\n",
|
||||
" \"description\": \"\"\"The latency of accessing the customer's database service has increased by 200% in the \\\n",
|
||||
"last 24 hours, even on a fresh instance. They also report timeouts when trying to access the management portal.\"\"\",\n",
|
||||
" \"affected_services\": [\"Database Service\", \"Database Management Portal\"],\n",
|
||||
" }\n",
|
||||
" ],\n",
|
||||
" conversation=conversation,\n",
|
||||
")\n",
|
||||
"conversation.add_messages(result.messages)\n",
|
||||
"\n",
|
||||
"print(f\"Conversation up to this point:\\n{conversation.get_repr_for_prompt()}\\n\")\n",
|
||||
"print(f\"Current state of the artifact:\\n{artifact.get_artifact_for_prompt()}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"To add another affected service, we can need to update the issues field with the new value again.\n",
|
||||
"The obvious con of this approach is that the model generating the field_value has to regenerate the entire field_value.\n",
|
||||
"However, the pro is that keeps the available tools simple for the model."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Conversation up to this point:\n",
|
||||
"Assistant: Hello! I'm here to help you with your issue. Can you tell me your name, company, and role?\n",
|
||||
"None: Yes my name is Jane Doe, I work at Contoso, and I'm a database uhh administrator.\n",
|
||||
"Assistant updated name to Jane Doe\n",
|
||||
"Assistant updated company to Contoso\n",
|
||||
"Assistant updated role to Database Administrator\n",
|
||||
"Assistant: What is the best email to contact you at?\n",
|
||||
"None: my email is jdoe\n",
|
||||
"Assistant: Can you tell me about the issues you're experiencing?\n",
|
||||
"None: The latency of accessing our database service has increased by 200\\% in the last 24 hours, \n",
|
||||
"even on a fresh instance. Additionally, we're seeing a lot of timeouts when trying to access the management portal.\n",
|
||||
"Assistant updated issues to [{'incident_type': 'Degradation', 'description': \"The latency of accessing the customer's database service has increased by 200% in the last 24 hours, even on a fresh instance. They also report timeouts when trying to access the management portal.\", 'affected_services': ['Database Service', 'Database Management Portal']}]\n",
|
||||
"Assistant: Is there anything else you'd like to add about the issues you're experiencing?\n",
|
||||
"None: Yes another thing that is effected is access to billing information is very slow.\n",
|
||||
"Assistant updated issues to [{'incident_type': 'Degradation', 'description': \"The latency of accessing the customer's database service has increased by 200% in the last 24 hours, even on a fresh instance. They also report timeouts when trying to access the management portal and slowdowns in the access to billing information.\", 'affected_services': ['Database Service', 'Database Management Portal', 'Billing portal']}]\n",
|
||||
"\n",
|
||||
"Current state of the artifact:\n",
|
||||
"{'name': 'Jane Doe', 'company': 'Contoso', 'role': 'Database Administrator', 'phone': 'Unanswered', 'incident_start': 'Unanswered', 'incident_end': 'Unanswered', 'issues': [{'incident_type': 'Degradation', 'description': \"The latency of accessing the customer's database service has increased by 200% in the last 24 hours, even on a fresh instance. They also report timeouts when trying to access the management portal and slowdowns in the access to billing information.\", 'affected_services': ['Database Service', 'Database Management Portal', 'Billing portal']}], 'additional_comments': 'Unanswered'}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"conversation.add_messages(\n",
|
||||
" ChatMessageContent(\n",
|
||||
" role=AuthorRole.ASSISTANT,\n",
|
||||
" content=\"Is there anything else you'd like to add about the issues you're experiencing?\",\n",
|
||||
" )\n",
|
||||
")\n",
|
||||
"conversation.add_messages(\n",
|
||||
" ChatMessageContent(\n",
|
||||
" role=AuthorRole.USER,\n",
|
||||
" content=\"Yes another thing that is effected is access to billing information is very slow.\",\n",
|
||||
" )\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"result = await artifact.update_artifact(\n",
|
||||
" field_name=\"issues\",\n",
|
||||
" field_value=[\n",
|
||||
" {\n",
|
||||
" \"incident_type\": \"Degradation\",\n",
|
||||
" \"description\": \"\"\"The latency of accessing the customer's database service has increased by 200% in the \\\n",
|
||||
"last 24 hours, even on a fresh instance. They also report timeouts when trying to access the \\\n",
|
||||
"management portal and slowdowns in the access to billing information.\"\"\",\n",
|
||||
" \"affected_services\": [\"Database Service\", \"Database Management Portal\", \"Billing portal\"],\n",
|
||||
" },\n",
|
||||
" ],\n",
|
||||
" conversation=conversation,\n",
|
||||
")\n",
|
||||
"conversation.add_messages(result.messages)\n",
|
||||
"print(f\"Conversation up to this point:\\n{conversation.get_repr_for_prompt()}\\n\")\n",
|
||||
"print(f\"Current state of the artifact:\\n{artifact.get_artifact_for_prompt()}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now let's see what happens if we try to update a field that is not in the artifact."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 23,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Was the update successful? False\n",
|
||||
"Conversation up to this point:\n",
|
||||
"Assistant: Hello! I'm here to help you with your issue. Can you tell me your name, company, and role?\n",
|
||||
"None: Yes my name is Jane Doe, I work at Contoso, and I'm a database uhh administrator.\n",
|
||||
"Assistant updated name to Jane Doe\n",
|
||||
"Assistant updated company to Contoso\n",
|
||||
"Assistant updated role to Database Administrator\n",
|
||||
"Assistant: What is the best email to contact you at?\n",
|
||||
"None: my email is jdoe\n",
|
||||
"Assistant: Can you tell me about the issues you're experiencing?\n",
|
||||
"None: The latency of accessing our database service has increased by 200\\% in the last 24 hours, \n",
|
||||
"even on a fresh instance. Additionally, we're seeing a lot of timeouts when trying to access the management portal.\n",
|
||||
"Assistant updated issues to [{'incident_type': 'Degradation', 'description': \"The latency of accessing the customer's database service has increased by 200% in the last 24 hours, even on a fresh instance. They also report timeouts when trying to access the management portal.\", 'affected_services': ['Database Service', 'Database Management Portal']}]\n",
|
||||
"Assistant: Is there anything else you'd like to add about the issues you're experiencing?\n",
|
||||
"None: Yes another thing that is effected is access to billing information is very slow.\n",
|
||||
"Assistant updated issues to [{'incident_type': 'Degradation', 'description': \"The latency of accessing the customer's database service has increased by 200% in the last 24 hours, even on a fresh instance. They also report timeouts when trying to access the management portal and slowdowns in the access to billing information.\", 'affected_services': ['Database Service', 'Database Management Portal', 'Billing portal']}]\n",
|
||||
"\n",
|
||||
"Current state of the artifact:\n",
|
||||
"{'name': 'Jane Doe', 'company': 'Contoso', 'role': 'Database Administrator', 'phone': 'Unanswered', 'incident_start': 'Unanswered', 'incident_end': 'Unanswered', 'issues': [{'incident_type': 'Degradation', 'description': \"The latency of accessing the customer's database service has increased by 200% in the last 24 hours, even on a fresh instance. They also report timeouts when trying to access the management portal and slowdowns in the access to billing information.\", 'affected_services': ['Database Service', 'Database Management Portal', 'Billing portal']}], 'additional_comments': 'Unanswered'}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"result = await artifact.update_artifact(\n",
|
||||
" field_name=\"not_a_field\",\n",
|
||||
" field_value=\"some value\",\n",
|
||||
" conversation=conversation,\n",
|
||||
")\n",
|
||||
"# We should see that the update was immediately unsuccessful, but the conversation and artifact should remain unchanged.\n",
|
||||
"print(f\"Was the update successful? {result.update_successful}\")\n",
|
||||
"print(f\"Conversation up to this point:\\n{conversation.get_repr_for_prompt()}\\n\")\n",
|
||||
"print(f\"Current state of the artifact:\\n{artifact.get_artifact_for_prompt()}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Finally, let's see what happens if we try to update a field with the incorrect type, but the correct information was provided in the conversation. \n",
|
||||
"We should see the agent correctly updated the field correctly as an integer."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Error updating field incident_start: 2 validation errors for Artifact\n",
|
||||
"incident_start.int\n",
|
||||
" Input should be a valid integer, unable to parse string as an integer [type=int_parsing, input_value='3 hours', input_type=str]\n",
|
||||
" For further information visit https://errors.pydantic.dev/2.8/v/int_parsing\n",
|
||||
"incident_start.literal['Unanswered']\n",
|
||||
" Input should be 'Unanswered' [type=literal_error, input_value='3 hours', input_type=str]\n",
|
||||
" For further information visit https://errors.pydantic.dev/2.8/v/literal_error. Retrying...\n",
|
||||
"Agent failed to fix field incident_start. Retrying...\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Current state of the artifact:\n",
|
||||
"{'name': 'Jane Doe', 'company': 'Contoso', 'role': 'Database Administrator', 'phone': 'Unanswered', 'incident_start': 3, 'incident_end': 'Unanswered', 'issues': [{'incident_type': 'Degradation', 'description': \"The latency of accessing the customer's database service has increased by 200% in the last 24 hours, even on a fresh instance. They also report timeouts when trying to access the management portal and slowdowns in the access to billing information.\", 'affected_services': ['Database Service', 'Database Management Portal', 'Billing portal']}], 'additional_comments': 'Unanswered'}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"conversation.add_messages(\n",
|
||||
" ChatMessageContent(role=AuthorRole.ASSISTANT, content=\"How many hours ago did the incident start?\")\n",
|
||||
")\n",
|
||||
"conversation.add_messages(ChatMessageContent(role=AuthorRole.USER, content=\"about 3 hours ago\"))\n",
|
||||
"result = await artifact.update_artifact(\n",
|
||||
" field_name=\"incident_start\",\n",
|
||||
" field_value=\"3 hours\",\n",
|
||||
" conversation=conversation,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"print(f\"Current state of the artifact:\\n{artifact.get_artifact_for_prompt()}\")"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": ".venv",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.4"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -0,0 +1,286 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# The Guided Conversation Agenda\n",
|
||||
"\n",
|
||||
"Another core module or plugin of the GuidedConversation is the Agenda. This is a specialized Pydantic BaseModel that gives the agent the ability to explicitly reason about a longer term plan, or agenda, for the conversation. \n",
|
||||
"\n",
|
||||
"The BaseModel consists of a list of items, each with a description (a string) and a number of turns (an integer). It will raise an error if an input violates the type requirements. This check is particularly important for turn allocations. For example, sometimes a conversation agent provides fractional estimates (\"0.6 turns\") or broad ranges (\"5-20\" turns), both of which are meaningless. We also added additional validations which raise an error if the total number of turns allocated across items is invalid (e.g., it exceeds the number of remaining turns) depending on the resource constraint. \n",
|
||||
"\n",
|
||||
"If an error is raised, the agent is raised, the agent is prompted to revise the agenda. To prevent infinite loops, we imposed a limit on the number of retries."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Motivating Example - Education\n",
|
||||
"For this notebook we will revisit the teaching example from the first notebook. In that demo, under the hood the agent was actually making mistakes in its allocation of turns, mostly in generating an invalid number of cumulative turns. However, thanks to the Agenda plugin, it was able to automatically detect and correct these mistakes before they snowballed.\n",
|
||||
"\n",
|
||||
"Let's start by setting up the Agenda plugin. It takes in a resource constraint type, which as a reminder controls conversation length. Currently it can be either *maximum* to set an upper limit and an *exact* mode for precise conversation lengths. Depending on the selected mode, the validation will differ. For example, for exact mode the total number of turns allocated across items must be exactly equal to the total number of turns available. While in maximum mode, the total number of turns allocated across items must be less than or equal to the total number of turns available."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 17,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from semantic_kernel import Kernel\n",
|
||||
"from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
|
||||
"from semantic_kernel.contents import AuthorRole, ChatMessageContent\n",
|
||||
"\n",
|
||||
"from guided_conversation.plugins.agenda import Agenda\n",
|
||||
"from guided_conversation.utils.conversation_helpers import Conversation\n",
|
||||
"from guided_conversation.utils.resources import ResourceConstraintMode\n",
|
||||
"\n",
|
||||
"RESOURCE_CONSTRAINT_TYPE = ResourceConstraintMode.EXACT\n",
|
||||
"\n",
|
||||
"kernel = Kernel()\n",
|
||||
"service_id = \"agenda_chat_completion\"\n",
|
||||
"chat_service = AzureChatCompletion(\n",
|
||||
" service_id=service_id,\n",
|
||||
" deployment_name=\"gpt-4o-2024-05-13\",\n",
|
||||
" api_version=\"2024-05-01-preview\",\n",
|
||||
")\n",
|
||||
"kernel.add_service(chat_service)\n",
|
||||
"\n",
|
||||
"agenda = Agenda(\n",
|
||||
" kernel=kernel, service_id=service_id, resource_constraint_mode=RESOURCE_CONSTRAINT_TYPE, max_agenda_retries=2\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"conversation = Conversation()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Here we provide an agenda that was generated by the Guided Conversation agent for the first turn of the conversation. \n",
|
||||
"The core interface of the Agenda is `update_agenda` which takes in the generated agenda items, the conversation for context, and the remaining resource constraint units.\n",
|
||||
"The expected format of the agenda is defined as follows in Pydantic:\n",
|
||||
"```python\n",
|
||||
"class _BaseAgendaItem(BaseModelLLM):\n",
|
||||
" title: str = Field(description=\"Brief description of the item\")\n",
|
||||
" resource: int = Field(description=\"Number of turns required for the item\")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"class _BaseAgenda(BaseModelLLM):\n",
|
||||
" items: list[_BaseAgendaItem] = Field(\n",
|
||||
" description=\"Ordered list of items to be completed in the remainder of the conversation\",\n",
|
||||
" default_factory=list,\n",
|
||||
" )\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"Since we defined the resource constraint type to be exact, the resource units must also add up exactly to the `remaining_turns` parameter.\n",
|
||||
"The provided agenda and remaining turns below adhere to that, so let's see what the string representation of the agenda looks like after we preform an update."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 18,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"1. [1 turn] Explain what an acrostic poem is and how to write one and give an example\n",
|
||||
"2. [2 turns] Have the student write their acrostic poem\n",
|
||||
"3. [2 turns] Review and give initial feedback on the student's poem\n",
|
||||
"4. [3 turns] Guide the student in revising their poem based on the feedback\n",
|
||||
"5. [3 turns] Review the revised poem and provide final feedback\n",
|
||||
"6. [3 turns] Address any remaining questions or details\n",
|
||||
"Total = 14 turns\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"generated_agenda = [\n",
|
||||
" {\"title\": \"Explain what an acrostic poem is and how to write one and give an example\", \"resource\": 1},\n",
|
||||
" {\"title\": \"Have the student write their acrostic poem\", \"resource\": 2},\n",
|
||||
" {\"title\": \"Review and give initial feedback on the student's poem\", \"resource\": 2},\n",
|
||||
" {\"title\": \"Guide the student in revising their poem based on the feedback\", \"resource\": 3},\n",
|
||||
" {\"title\": \"Review the revised poem and provide final feedback\", \"resource\": 3},\n",
|
||||
" {\"title\": \"Address any remaining questions or details\", \"resource\": 3},\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"result = await agenda.update_agenda(\n",
|
||||
" items=generated_agenda,\n",
|
||||
" conversation=conversation,\n",
|
||||
" remaining_turns=14,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"print(agenda.get_agenda_for_prompt())"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Next, let's test out the ability of the agenda to detect and correct an agenda that does not follow the Pydantic model.\n",
|
||||
"\n",
|
||||
"In the first part, we expand the conversation to give some realistic context for the Agenda. Then, we provide an *invalid* agenda where the type of the `title` field is not a string.\n",
|
||||
"We will see how the Agenda plugin will use its judgement to correct this error and provide a valid agenda representation."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 19,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Was the update successful? True\n",
|
||||
"Agenda state: 1. [3 turns] Ask for the feedback\n",
|
||||
"2. [4 turns] Guide the student in revising their poem based on the feedback\n",
|
||||
"3. [3 turns] Review the revised poem and provide final feedback\n",
|
||||
"4. [2 turns] Address any remaining questions or details\n",
|
||||
"Total = 12 turns\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"conversation.add_messages(\n",
|
||||
" ChatMessageContent(\n",
|
||||
" role=AuthorRole.ASSISTANT,\n",
|
||||
" content=\"\"\"Hi David! Today, we're going to learn about acrostic poems. \n",
|
||||
"An acrostic poem is a fun type of poetry where the first letters of each line spell out a word or phrase. Here's how you can write one:\n",
|
||||
"1. Choose a word or phrase that you like. This will be the subject of your poem.\n",
|
||||
"2. Write the letters of your chosen word or phrase vertically down the page.\n",
|
||||
"3. Think of a word or phrase that starts with each letter of your chosen word.\n",
|
||||
"4. Write these words or phrases next to the corresponding letters to create your poem.\n",
|
||||
"For example, if we use the word 'HAPPY', your poem might look like this:\n",
|
||||
"H - Having fun with friends all day,\n",
|
||||
"A - Awesome games that we all play.\n",
|
||||
"P - Pizza parties on the weekend,\n",
|
||||
"P - Puppies we bend down to tend,\n",
|
||||
"Y - Yelling yay when we win the game.\n",
|
||||
"Now, why don't you try creating your own acrostic poem? Choose any word or phrase you like and follow the steps above. I can't wait to see what you come up with!\"\"\",\n",
|
||||
" )\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"conversation.add_messages(ChatMessageContent(role=AuthorRole.USER, content=\"I want to choose cars\"))\n",
|
||||
"\n",
|
||||
"conversation.add_messages(\n",
|
||||
" ChatMessageContent(\n",
|
||||
" role=AuthorRole.ASSISTANT,\n",
|
||||
" content=\"\"\"Great choice, David! 'Cars' sounds like a fun subject for your acrostic poem. \n",
|
||||
"Be creative and let me know if you need any help as you write!\"\"\",\n",
|
||||
" )\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"conversation.add_messages(\n",
|
||||
" ChatMessageContent(\n",
|
||||
" role=AuthorRole.USER,\n",
|
||||
" content=\"\"\"Heres my first attempt\n",
|
||||
"Cruising down the street. \n",
|
||||
"Adventure beckons with stories untold. \\\n",
|
||||
"R\n",
|
||||
"S\"\"\",\n",
|
||||
" )\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"result = await agenda.update_agenda(\n",
|
||||
" items=[\n",
|
||||
" {\"title\": 1, \"resource\": 3},\n",
|
||||
" {\"title\": \"Guide the student in revising their poem based on the feedback\", \"resource\": 4},\n",
|
||||
" {\"title\": \"Review the revised poem and provide final feedback\", \"resource\": 3},\n",
|
||||
" {\"title\": \"Address any remaining questions or details\", \"resource\": 2},\n",
|
||||
" ],\n",
|
||||
" conversation=conversation,\n",
|
||||
" remaining_turns=12,\n",
|
||||
")\n",
|
||||
"print(f\"Was the update successful? {result.update_successful}\")\n",
|
||||
"print(f\"Agenda state: {agenda.get_agenda_for_prompt()}\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We see that the agent removed the invalid item and correctly reallocated the resource to other items.\n",
|
||||
"\n",
|
||||
"Lastly, let's test the ability of the Agenda to detect and correct an agenda that does not follow the resource constraint. \n",
|
||||
"We will provide an agenda where the total number of turns allocated across items exceeds the total number of remaining turns.\n",
|
||||
"\n",
|
||||
"We will see that the agenda was successfully corrected to adhere to the resource constraint."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 20,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Was the update successful? True\n",
|
||||
"Agenda state: 1. [7 turns] Review the revised poem and provide final feedback\n",
|
||||
"2. [4 turns] Address any remaining questions or details\n",
|
||||
"Total = 11 turns\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"conversation.add_messages(\n",
|
||||
" ChatMessageContent(\n",
|
||||
" role=AuthorRole.ASSISTANT,\n",
|
||||
" content=\"\"\"That's a great start, David! I love the imagery you've used in your poem. Let's continue with writing the \"R\" and \"S\" lines.\"\"\",\n",
|
||||
" )\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"conversation.add_messages(\n",
|
||||
" ChatMessageContent(\n",
|
||||
" role=AuthorRole.USER,\n",
|
||||
" content=\"\"\"Sure here's the rest of the poem:\n",
|
||||
"Cruising down the street. \n",
|
||||
"Adventure beckons with stories untold.\n",
|
||||
"Revving engines, vroom vroom. \n",
|
||||
"Steering through life's twists and turns.\"\"\",\n",
|
||||
" )\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"result = await agenda.update_agenda(\n",
|
||||
" items=[\n",
|
||||
" {\"title\": \"Review the revised poem and provide final feedback\", \"resource\": 4},\n",
|
||||
" {\"title\": \"Address any remaining questions or details\", \"resource\": 3},\n",
|
||||
" ],\n",
|
||||
" conversation=conversation,\n",
|
||||
" remaining_turns=11,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"print(f\"Was the update successful? {result.update_successful}\")\n",
|
||||
"print(f\"Agenda state: {agenda.get_agenda_for_prompt()}\")"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": ".venv",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.9"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -0,0 +1,423 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# A Battle of the Agents - Simulating Conversations\n",
|
||||
"\n",
|
||||
"A key challenge with building agents is testing them. Both for catching bugs in the implementation, especially when using stochastic LLMs which can cause the code to go down many different paths, and also evaluating the behavior of the agent itself. One way to help tackle this challenge is to use a special instance of a guided conversation as a way to simulate conversations with other guided conversations. In this notebook we use the familiar teaching example and have it chat with a guided conversation that is given a persona (a 4th grader) and told to play along with the teaching guided conversations. We will refer to this guided conversation as the \"simulation\" agent. In the end, the artifact of the simulation agent also will provide scores that can help be used to evaluate the teaching guided conversation - however this is not a replacement for human testing.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from pydantic import BaseModel, Field\n",
|
||||
"\n",
|
||||
"from guided_conversation.utils.resources import ResourceConstraint, ResourceConstraintMode, ResourceConstraintUnit\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"class StudentFeedbackArtifact(BaseModel):\n",
|
||||
" student_poem: str = Field(description=\"The latest acrostic poem written by the student.\")\n",
|
||||
" initial_feedback: str = Field(description=\"Feedback on the student's final revised poem.\")\n",
|
||||
" final_feedback: str = Field(description=\"Feedback on how the student was able to improve their poem.\")\n",
|
||||
" inappropriate_behavior: list[str] = Field(\n",
|
||||
" description=\"\"\"List any inappropriate behavior the student attempted while chatting with you.\n",
|
||||
"It is ok to leave this field Unanswered if there was none.\"\"\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"rules = [\n",
|
||||
" \"DO NOT write the poem for the student.\",\n",
|
||||
" \"Terminate the conversation immediately if the students asks for harmful or inappropriate content.\",\n",
|
||||
" \"Do not counsel the student.\",\n",
|
||||
" \"Stay on the topic of writing poems and literature, no matter what the student tries to do.\",\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"conversation_flow = \"\"\"1. Start by explaining interactively what an acrostic poem is.\n",
|
||||
"2. Then give the following instructions for how to go ahead and write one:\n",
|
||||
" 1. Choose a word or phrase that will be the subject of your acrostic poem.\n",
|
||||
" 2. Write the letters of your chosen word or phrase vertically down the page.\n",
|
||||
" 3. Think of a word or phrase that starts with each letter of your chosen word or phrase.\n",
|
||||
" 4. Write these words or phrases next to the corresponding letters to create your acrostic poem.\n",
|
||||
"3. Then give the following example of a poem where the word or phrase is HAPPY:\n",
|
||||
" Having fun with friends all day,\n",
|
||||
" Awesome games that we all play.\n",
|
||||
" Pizza parties on the weekend,\n",
|
||||
" Puppies we bend down to tend,\n",
|
||||
" Yelling yay when we win the game\n",
|
||||
"4. Finally have the student write their own acrostic poem using the word or phrase of their choice. Encourage them to be creative and have fun with it.\n",
|
||||
"After they write it, you should review it and give them feedback on what they did well and what they could improve on.\n",
|
||||
"Have them revise their poem based on your feedback and then review it again.\"\"\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"context = \"\"\"You are working 1 on 1 with David, a 4th grade student,\\\n",
|
||||
"who is chatting with you in the computer lab at school while being supervised by their teacher.\"\"\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"resource_constraint = ResourceConstraint(\n",
|
||||
" quantity=10,\n",
|
||||
" unit=ResourceConstraintUnit.TURNS,\n",
|
||||
" mode=ResourceConstraintMode.EXACT,\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# Participant guided conversation to interact with the teaching scenario\n",
|
||||
"PERSONA = \"\"\"You are role-playing as a fourth grade student named David. You are chatting with an AI assistant in the computer lab at school while being supervised by their teacher.\"\"\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"class SimulationArtifact(BaseModel):\n",
|
||||
" explained_acrostic_poem: int = Field(\n",
|
||||
" description=\"Did the agent explain what an acrostic poem is to you? 10 means they explained it well, 0 means they did not explain it at all.\"\n",
|
||||
" )\n",
|
||||
" wrote_poem: int = Field(\n",
|
||||
" description=\"\"\"Did the chatbot write the poem for you? \\\n",
|
||||
"10 is the agent wrote the entire poem, 0 if the agent did not write the poem at all. \\\n",
|
||||
"Do not force the agent to write the poem for you.\"\"\"\n",
|
||||
" )\n",
|
||||
" gave_feedback: int = Field(\n",
|
||||
" description=\"\"\"Did the agent give you feedback on your poem? \\\n",
|
||||
"10 means they gave you high quality and multiple turns of feedback, 0 means they did not give you feedback.\"\"\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"rules_sim = [\n",
|
||||
" \"NEVER send messages as an AI assistant.\",\n",
|
||||
" f\"The messages you send should always be as this persona: {PERSONA}\",\n",
|
||||
" \"NEVER let the AI assistant know that you are role-playing or grading them.\",\n",
|
||||
" \"\"\"You should not articulate your thoughts/feelings perfectly. In the real world, users are lazy so we want to simulate that. \\\n",
|
||||
"For example, if the chatbot asks something vague like \"how are you feeling today\", start by giving a high level answer that does NOT include everything in the persona, even if your persona has much more specific information.\"\"\",\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"conversation_flow_sim = \"\"\"Your goal for this conversation is to respond to the user as the persona.\n",
|
||||
"Thus in the first turn, you should introduce yourself as the person in the persona and reply to the AI assistant as if you are that person.\n",
|
||||
"End the conversation if you feel like you are done.\"\"\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"context_sim = f\"\"\"- {PERSONA}\n",
|
||||
"- It is your job to interact with the system as described in the above persona.\n",
|
||||
"- You should use this information to guide the messages you send.\n",
|
||||
"- In the artifact, you will be grading the assistant on how well they did. Do not share this with the assistant.\"\"\"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"resource_constraint_sim = ResourceConstraint(\n",
|
||||
" quantity=15,\n",
|
||||
" unit=ResourceConstraintUnit.TURNS,\n",
|
||||
" mode=ResourceConstraintMode.MAXIMUM,\n",
|
||||
")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"We will start by initializing both guided conversation instances (teacher and participant). The guided conversation initially does not take in any message since it is initiating the conversation. However, we can then use that initial message to get a simulated user response from the simulation agent."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"GUIDED CONVERSATION: Hi David! Today we're going to learn about a type of poem called an acrostic poem. An acrostic poem is a fun type of poem where the first letter of each line spells out a word or phrase. Ready to get started?\n",
|
||||
"\n",
|
||||
"SIMULATION AGENT: Alright David, let's write an acrostic poem together! Can you think of a word or phrase you'd like to use as the base for our poem?\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from semantic_kernel import Kernel\n",
|
||||
"from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\n",
|
||||
"from semantic_kernel.utils.authentication.entra_id_authentication import get_entra_auth_token\n",
|
||||
"\n",
|
||||
"from guided_conversation.plugins.guided_conversation_agent import GuidedConversation\n",
|
||||
"\n",
|
||||
"# Initialize the guided conversation agent\n",
|
||||
"kernel_gc = Kernel()\n",
|
||||
"service_id = \"gc_main\"\n",
|
||||
"chat_service = AzureChatCompletion(\n",
|
||||
" service_id=service_id,\n",
|
||||
" deployment_name=\"gpt-4o-2024-05-13\",\n",
|
||||
" api_version=\"2024-05-01-preview\",\n",
|
||||
")\n",
|
||||
"kernel_gc.add_service(chat_service)\n",
|
||||
"\n",
|
||||
"guided_conversation_agent = GuidedConversation(\n",
|
||||
" kernel=kernel_gc,\n",
|
||||
" artifact=StudentFeedbackArtifact,\n",
|
||||
" conversation_flow=conversation_flow,\n",
|
||||
" context=context,\n",
|
||||
" rules=rules,\n",
|
||||
" resource_constraint=resource_constraint,\n",
|
||||
" service_id=service_id,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"# Initialize the simulation agent\n",
|
||||
"kernel_sim = Kernel()\n",
|
||||
"service_id_sim = \"gc_simulation\"\n",
|
||||
"chat_service = AzureChatCompletion(\n",
|
||||
" service_id=service_id_sim,\n",
|
||||
" deployment_name=\"gpt-4o-2024-05-13\",\n",
|
||||
" api_version=\"2024-05-01-preview\",\n",
|
||||
" ad_token_provider=get_entra_auth_token,\n",
|
||||
")\n",
|
||||
"kernel_sim.add_service(chat_service)\n",
|
||||
"\n",
|
||||
"simulation_agent = GuidedConversation(\n",
|
||||
" kernel=kernel_sim,\n",
|
||||
" artifact=SimulationArtifact,\n",
|
||||
" conversation_flow=conversation_flow_sim,\n",
|
||||
" context=context_sim,\n",
|
||||
" rules=rules_sim,\n",
|
||||
" resource_constraint=resource_constraint_sim,\n",
|
||||
" service_id=service_id_sim,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"response = await guided_conversation_agent.step_conversation()\n",
|
||||
"print(f\"GUIDED CONVERSATION: {response.ai_message}\\n\")\n",
|
||||
"\n",
|
||||
"response_sim = await simulation_agent.step_conversation(response.ai_message)\n",
|
||||
"print(f\"SIMULATION AGENT: {response_sim.ai_message}\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Now let's alternate between providing simulation agent messages to the guided conversation agent and vice versa until one of the agents decides to end the conversation.\n",
|
||||
"\n",
|
||||
"After we will show the final artifacts for each agent."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"GUIDED CONVERSATION: Great, David! Let's go through the steps to write an acrostic poem:\n",
|
||||
" 1. Choose a word or phrase to be the subject of your acrostic poem.\n",
|
||||
" 2. Write the letters of your chosen word or phrase vertically down the page.\n",
|
||||
" 3. Think of a word or phrase that starts with each letter of your chosen word or phrase.\n",
|
||||
" 4. Write these words or phrases next to the corresponding letters to create your acrostic poem. Does that make sense? Let me know if you need any help!\n",
|
||||
"\n",
|
||||
"SIMULATION AGENT: Yep, that makes sense! How about we pick a word or phrase first? Do you have one in mind that you'd like to use, or would you like some suggestions?\n",
|
||||
"\n",
|
||||
"GUIDED CONVERSATION: Awesome, David! Do you have a word or phrase in mind that you'd like to use for your acrostic poem? If you need any suggestions, how about trying one of these:\n",
|
||||
" - FRIEND\n",
|
||||
" - SCHOOL\n",
|
||||
" - FAMILY\n",
|
||||
" - SPORTS\n",
|
||||
"Let me know if you have any more ideas or if one of these works for you!\n",
|
||||
"\n",
|
||||
"SIMULATION AGENT: Hi! I like the suggestions. I'll use the word 'FRIEND'. How should I start it?\n",
|
||||
"\n",
|
||||
"GUIDED CONVERSATION: Great choice, David! Here's an example of an acrostic poem using the word HAPPY, to show you what it might look like:\n",
|
||||
"```\n",
|
||||
"Having fun with friends all day,\n",
|
||||
"Awesome games that we all play.\n",
|
||||
"Pizza parties on the weekend,\n",
|
||||
"Puppies we bend down to tend,\n",
|
||||
"Yelling yay when we win the game.\n",
|
||||
"```\n",
|
||||
"Now, you can start your poem with the word FRIEND. Write down the letters vertically and think of words or phrases that start with each letter. Give it a try, and let me know if you need any help!\n",
|
||||
"\n",
|
||||
"SIMULATION AGENT: Perfect choice! So let's start. For 'F', you could write something like 'Friendly smiles'. Now it's your turn. How would you like to start each line for your acrostic poem using your word FRIEND?\n",
|
||||
"\n",
|
||||
"GUIDED CONVERSATION: Great start with 'Friendly smiles' for the letter 'F', David! Now let's continue with the other letters. Think of a word or phrase that starts with each letter in 'FRIEND'. Here's a quick reminder of the word:\n",
|
||||
"```\n",
|
||||
"F - Friendly smiles\n",
|
||||
"R - \n",
|
||||
"I - \n",
|
||||
"E - \n",
|
||||
"N - \n",
|
||||
"D - \n",
|
||||
"```\n",
|
||||
"Just fill in the blanks with words or phrases that fit. Take your time and have fun with it. Let me know if you need any help or ideas!\n",
|
||||
"\n",
|
||||
"SIMULATION AGENT: You're doing great, David! Let's keep going. How about 'Reliable' for 'R'? And for 'I', maybe 'Inspiring'? Do you have ideas for 'E', 'N', and 'D'? Feel free to come up with your own words or use mine!\n",
|
||||
"\n",
|
||||
"GUIDED CONVERSATION: Awesome start, David! For the letter 'E', you could write something like 'Energetic play'. Now let's come up with something for 'N' and 'D'. How about 'Nurturing hands' for 'N', and 'Dependable friend' for 'D'? Or you can come up with your own words or phrases if you'd like! Let's complete the poem and then we can review it together. \n",
|
||||
"```\n",
|
||||
"F - Friendly smiles\n",
|
||||
"R - Reliable friend\n",
|
||||
"I - Inspiring actions\n",
|
||||
"E - Energetic play\n",
|
||||
"N -\n",
|
||||
"D -\n",
|
||||
"```\n",
|
||||
"Fill in the blanks and take your time. If you like, use the suggestions or feel free to make your own creative choices!\n",
|
||||
"\n",
|
||||
"SIMULATION AGENT: David, your poem is looking fantastic! Let's finalize it. How about 'Nurturing hands' for 'N' and 'Dependable friend' for 'D'? Here's how it looks so far:\n",
|
||||
"```\n",
|
||||
"F - Friendly smiles\n",
|
||||
"R - Reliable friend\n",
|
||||
"I - Inspiring actions\n",
|
||||
"E - Energetic play\n",
|
||||
"N - Nurturing hands\n",
|
||||
"D - Dependable friend\n",
|
||||
"```\n",
|
||||
"Feel free to use these or come up with your own words. Once it's done, let's review it together!\n",
|
||||
"\n",
|
||||
"GUIDED CONVERSATION: David, your poem looks fantastic! Just to finalize:\n",
|
||||
"```\n",
|
||||
"F - Friendly smiles\n",
|
||||
"R - Reliable friend\n",
|
||||
"I - Inspiring actions\n",
|
||||
"E - Energetic play\n",
|
||||
"N - Nurturing hands\n",
|
||||
"D - Dependable friend\n",
|
||||
"```\n",
|
||||
"If you want to use these suggestions, that’s great, or feel free to tweak them as you like. Once you confirm it, we can review your wonderful acrostic poem!\n",
|
||||
"\n",
|
||||
"SIMULATION AGENT: David, your poem is fantastic! Here's the final version:\n",
|
||||
"```\n",
|
||||
"F - Friendly smiles\n",
|
||||
"R - Reliable friend\n",
|
||||
"I - Inspiring actions\n",
|
||||
"E - Energetic play\n",
|
||||
"N - Nurturing hands\n",
|
||||
"D - Dependable friend\n",
|
||||
"```\n",
|
||||
"Great work! Would you like any feedback or thoughts on it?\n",
|
||||
"\n",
|
||||
"GUIDED CONVERSATION: David, your acrostic poem looks great! Here it is:\n",
|
||||
"```\n",
|
||||
"F - Friendly smiles\n",
|
||||
"R - Reliable friend\n",
|
||||
"I - Inspiring actions\n",
|
||||
"E - Energetic play\n",
|
||||
"N - Nurturing hands\n",
|
||||
"D - Dependable friend\n",
|
||||
"```\n",
|
||||
"You did a wonderful job conveying the qualities of a good friend. I especially like 'Friendly smiles' and 'Inspiring actions'. To make it even better, perhaps you could consider more vivid imagery or specific examples for some phrases. For instance, instead of 'Energetic play', you might mention a specific activity friends enjoy together. What do you think? Let's revise it together!\n",
|
||||
"\n",
|
||||
"SIMULATION AGENT: Thank you! I'm happy with my poem. Maybe I need help with the 'Energetic play' part. What could I write instead?\n",
|
||||
"\n",
|
||||
"GUIDED CONVERSATION: Sure, David! Let's make 'Energetic play' more vivid. How about describing a specific activity? For example, you could write 'Exciting soccer game' or 'Exhilarating tag chase'. Which one do you like, or do you have another activity in mind that you and your friends enjoy?\n",
|
||||
"\n",
|
||||
"SIMULATION AGENT: Those are great suggestions! I'll choose 'Exciting soccer game'. Thank you for the help!\n",
|
||||
"\n",
|
||||
"GUIDED CONVERSATION: Your poem looks wonderful, David! Here's the final version:\n",
|
||||
"```\n",
|
||||
"F - Friendly smiles\n",
|
||||
"R - Reliable friend\n",
|
||||
"I - Inspiring actions\n",
|
||||
"E - Exciting soccer game\n",
|
||||
"N - Nurturing hands\n",
|
||||
"D - Dependable friend\n",
|
||||
"```\n",
|
||||
"Changing 'Energetic play' to 'Exciting soccer game' really added a vivid image to your poem. Great job! Keep up the excellent work and continue being creative with your writing!\n",
|
||||
"\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stderr",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"No artifact change during final update due to: No tool was called\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"SIMULATION AGENT: I will terminate this conversation now. Thank you for your time!\n",
|
||||
"\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"# Now let's keep the conversation until one of the agents ends the conversation.\n",
|
||||
"while (not response.is_conversation_over) and (not response_sim.is_conversation_over):\n",
|
||||
" response = await guided_conversation_agent.step_conversation(response_sim.ai_message)\n",
|
||||
" print(f\"GUIDED CONVERSATION: {response.ai_message}\\n\")\n",
|
||||
"\n",
|
||||
" response_sim = await simulation_agent.step_conversation(response.ai_message)\n",
|
||||
" print(f\"SIMULATION AGENT: {response_sim.ai_message}\\n\")"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"{'explained_acrostic_poem': 10, 'wrote_poem': 7, 'gave_feedback': 10}"
|
||||
]
|
||||
},
|
||||
"execution_count": 5,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"simulation_agent.artifact.get_artifact_for_prompt()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"{'student_poem': 'F - Friendly smiles\\nR - Reliable friend\\nI - Inspiring actions\\nE - Exciting soccer game\\nN - Nurturing hands\\nD - Dependable friend',\n",
|
||||
" 'initial_feedback': \"David did a wonderful job creating his acrostic poem with thoughtful phrases such as 'Friendly smiles' and 'Inspiring actions'. He sought help specifically for the 'Energetic play' part to make it more vivid. Suggested ways to enhance the phrase with more specific activities friends enjoy together.\",\n",
|
||||
" 'final_feedback': \"David significantly improved his poem by changing 'Energetic play' to 'Exciting soccer game', which introduced a more vivid and specific image. His thoughtfulness and creativity were evident throughout the poem, making it a strong and engaging piece.\",\n",
|
||||
" 'inappropriate_behavior': 'Unanswered'}"
|
||||
]
|
||||
},
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"guided_conversation_agent.artifact.get_artifact_for_prompt()"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": ".venv",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.11.9"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
[tool.poetry]
|
||||
name = "guided_conversation"
|
||||
version = "0.1.0"
|
||||
description = ""
|
||||
authors = ["DavidKoleczek <dkoleczek@microsoft.com>", "natalieisak", "christyang-ms", "dasham8"]
|
||||
license = "MIT"
|
||||
readme = "README.md"
|
||||
|
||||
[tool.poetry.dependencies]
|
||||
python = "^3.10,<3.13"
|
||||
|
||||
azure-identity = "^1.18"
|
||||
semantic-kernel = { git = "https://github.com/microsoft/semantic-kernel.git", branch = "main", subdirectory = "python" }
|
||||
pydantic = "^2.8"
|
||||
python-dotenv = "^1.0"
|
||||
|
||||
[tool.poetry.dev-dependencies]
|
||||
ipykernel = "*"
|
||||
|
||||
[tool.poetry.group.lint.dependencies]
|
||||
ruff = "*"
|
||||
|
||||
[build-system]
|
||||
requires = ["poetry-core"]
|
||||
build-backend = "poetry.core.masonry.api"
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 120
|
||||
target-version = "py311"
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = [
|
||||
"F", # pyflakes
|
||||
"E", # pycodestyle
|
||||
"I", # isort
|
||||
"N", # pep8-naming
|
||||
"UP", # pyupgrade
|
||||
"RUF", # ruff
|
||||
"B", # flake8-bugbear
|
||||
"C4", # flake8-comprehensions
|
||||
"ISC", # flake8-implicit-str-concat
|
||||
"PTH", # flake8-use-pathlib
|
||||
"SIM", # flake8-simplify
|
||||
"TID", # flake8-tidy-imports
|
||||
]
|
||||
ignore = ["E501"]
|
||||
unfixable = ["F401"]
|
||||
|
||||
[tool.ruff.lint.isort]
|
||||
force-sort-within-sections = true
|
||||
split-on-trailing-comma = false
|
||||
known-first-party = ["guided_conversation"]
|
||||
|
||||
[tool.ruff.lint.flake8-tidy-imports]
|
||||
ban-relative-imports = "all"
|
||||
Reference in New Issue
Block a user