wehub/microsoft--promptflow
1
0
Fork 0
Code Issues Pull Requests Actions 11 Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution
tools_continuous_delivery / Private PyPI non-main branch release (push) Has been skipped
Details
tools_continuous_delivery / Private PyPI main branch release (push) Failing after 2m42s
Details
Publish Promptflow Doc / Build (push) Has been cancelled
Details
Publish Promptflow Doc / Deploy (push) Has been cancelled
Details
Flake8 Lint / flake8 (push) Has been cancelled
Details
Spell check CI / Spell_Check (push) Has been cancelled
Details

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:39:52 +08:00
commit e768098d0e
4004 changed files with 2804145 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/reference/flow-yaml-schema-reference.md
+80
View File
@@ -0,0 +1,80 @@
# Flow YAML Schema
:::{admonition} Experimental feature
This is an experimental feature, and may change at any time. Learn [more](../how-to-guides/faq.md#stable-vs-experimental).
:::
The source JSON schema can be found at [Flow.schema.json](https://azuremlschemas.azureedge.net/promptflow/latest/Flow.schema.json)
## YAML syntax
| Key | Type | Description |
|-------------------------|---------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `$schema` | string | The YAML schema. If you use the prompt flow VS Code extension to author the YAML file, including `$schema` at the top of your file enables you to invoke schema and resource completions. |
| `inputs` | object | Dictionary of flow inputs. The key is a name for the input within the context of the flow and the value is the flow input definition. |
| `inputs.<input_name>` | object | The flow input definition. See [Flow input](#flow-input) for the set of configurable properties. |
| `outputs` | object | Dictionary of flow outputs. The key is a name for the output within the context of the flow and the value is the flow output definition. |
| `outputs.<output_name>` | object | The component output definition. See [Flow output](#flow-output) for the set of configurable properties. |
| `nodes` | array | Sets of dictionary of individual nodes to run as steps within the flow. Node can use built-in tool or third-party tool. See [Nodes](#nodes) for more information. |
| `node_variants` | object | Dictionary of nodes with variants. The key is the node name and value contains variants definition and `default_variant_id`. See [Node variants](#node-variants) for more information. |
| `environment` | object | The environment to use for the flow. The key can be `image` or `python_requirements_txt` and the value can be either a image or a python requirements text file. |
| `environment_variables` | object/string | Environment variables to set by specifying a property path and value. Example: `{"key1"="${my_connection.api_key}"}`. The value reference to connection keys will be resolved to the actual value, and all environment variables specified will be set into os.environ. |
| `additional_includes` | array | Additional includes is a list of files that can be shared among flows. Users can specify additional files and folders used by flow, and prompt flow will help copy them all to the snapshot during flow creation. |
### Flow input
| Key | Type | Description | Allowed values |
|-------------------|------------------------------------------------|------------------------------------------------------|--------------------------------------------------------------|
| `type` | string | The type of flow input. | `int`, `double`, `bool`, `string`, `list`, `object`, `image` |
| `description` | string | Description of the input. | |
| `default` | int, double, bool, string, list, object, image | The default value for the input. | |
| `is_chat_input` | boolean | Whether the input is the chat flow input. | |
| `is_chat_history` | boolean | Whether the input is the chat history for chat flow. | |
### Flow output
| Key | Type | Description | Allowed values |
|------------------|---------|-------------------------------------------------------------------------------|-----------------------------------------------------|
| `type` | string | The type of flow output. | `int`, `double`, `bool`, `string`, `list`, `object` |
| `description` | string | Description of the output. | |
| `reference` | string | A reference to the node output, e.g. ${<node_name>.output.<node_output_name>} | |
| `is_chat_output` | boolean | Whether the output is the chat flow output. | |
### Nodes
Nodes is a set of node which is a dictionary with following fields. Below, we only show the common fields of a single node using built-in tool.
| Key | Type | Description | Allowed values |
|----------------|--------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------|
| `name` | string | The name of the node. | |
| `type` | string | The type of the node. | Type of built-in tool like `Python`, `Prompt`, `LLM` and third-party tool like `Vector Search`, etc. |
| `inputs` | object | Dictionary of node inputs. The key is the input name and the value can be primitive value or a reference to the flow input or the node output, e.g. `${inputs.<flow_input_name>}`, `${<node_name>.output}` or `${<node_name>.output.<node_output_name>}` | |
| `source` | object | Dictionary of tool source used by the node. The key contains `type`, `path` and `tool`. The type can be `code`, `package` and `package_with_prompt`. | |
| `provider` | string | It indicates the provider of the tool. Used when the `type` is LLM. | `AzureOpenAI` or `OpenAI` |
| `connection` | string | The connection name which has been created before. Used when the `type` is LLM. | |
| `api` | string | The api name of the provider. Used when the `type` is LLM. | |
| `module` | string | The module name of the tool using by the node. Used when the `type` is LLM. | |
| `use_variants` | bool | Whether the node has variants. | |
### Node variants
Node variants is a dictionary containing variants definition for nodes with variants with their respective node names as dictionary keys.
Below, we explore the variants for a single node.
| Key | Type | Description | Allowed values |
|----------------------|--------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------|
| `<node_name>` | string | The name of the node. | |
| `default_variant_id` | string | Default variant id. | |
| `variants ` | object | This dictionary contains all node variations, with the variant id serving as the key and a node definition dictionary as the corresponding value. Within the node definition dictionary, the key labeled 'node' should contain a variant definition similar to [Nodes](#nodes), excluding the 'name' field. | |
## Examples
Flow examples are available in the [GitHub repository](https://github.com/microsoft/promptflow/tree/main/examples/flows).
- [basic](https://github.com/microsoft/promptflow/tree/main/examples/flows/standard/basic)
- [web-classification](https://github.com/microsoft/promptflow/tree/main/examples/flows/standard/web-classification)
- [chat-basic](https://github.com/microsoft/promptflow/tree/main/examples/flows/chat/chat-basic)
- [chat-with-pdf](https://github.com/microsoft/promptflow/tree/main/examples/flows/chat/chat-with-pdf)
- [eval-basic](https://github.com/microsoft/promptflow/tree/main/examples/flows/evaluation/eval-basic)
docs/reference/index.md
+87
View File
@@ -0,0 +1,87 @@
# Reference
**Current stable version:**
- [promptflow](https://pypi.org/project/promptflow):
[![PyPI version](https://badge.fury.io/py/promptflow.svg)](https://badge.fury.io/py/promptflow)
[![PyPI - Downloads](https://img.shields.io/pypi/dm/promptflow)](https://pypi.org/project/promptflow/)
- [promptflow-tracing](https://pypi.org/project/promptflow-tracing):
[![PyPI version](https://badge.fury.io/py/promptflow-tracing.svg)](https://badge.fury.io/py/promptflow-tracing)
[![PyPI - Downloads](https://img.shields.io/pypi/dm/promptflow-tracing)](https://pypi.org/project/promptflow-tracing/)
- [promptflow-core](https://pypi.org/project/promptflow-core):
[![PyPI version](https://badge.fury.io/py/promptflow-core.svg)](https://badge.fury.io/py/promptflow-core)
[![PyPI - Downloads](https://img.shields.io/pypi/dm/promptflow-core)](https://pypi.org/project/promptflow-core/)
- [promptflow-devkit](https://pypi.org/project/promptflow-devkit):
[![PyPI version](https://badge.fury.io/py/promptflow-devkit.svg)](https://badge.fury.io/py/promptflow-devkit)
[![PyPI - Downloads](https://img.shields.io/pypi/dm/promptflow-devkit)](https://pypi.org/project/promptflow-devkit/)
- [promptflow-azure](https://pypi.org/project/promptflow-azure):
[![PyPI version](https://badge.fury.io/py/promptflow-azure.svg)](https://badge.fury.io/py/promptflow-azure)
[![PyPI - Downloads](https://img.shields.io/pypi/dm/promptflow-azure)](https://pypi.org/project/promptflow-azure/)
- [promptflow-tools](https://pypi.org/project/promptflow-tools/):
[![PyPI version](https://badge.fury.io/py/promptflow-tools.svg)](https://badge.fury.io/py/promptflow-tools)
[![PyPI - Downloads](https://img.shields.io/pypi/dm/promptflow-tools)](https://pypi.org/project/promptflow-tools/)
```{toctree}
:caption: Command Line Interface
:maxdepth: 1
pf-command-reference.md
pfazure-command-reference.md
```
```{toctree}
:caption: Python Library ChangeLog
:maxdepth: 1
changelog/promptflow
```
```{toctree}
:caption: Python Library Reference
:maxdepth: 1
python-library-reference/promptflow-tracing/promptflow
python-library-reference/promptflow-core/promptflow
python-library-reference/promptflow-devkit/promptflow
python-library-reference/promptflow-azure/promptflow
python-library-reference/promptflow-rag/promptflow
python-library-reference/promptflow-evals/promptflow
```
```{toctree}
:caption: Tool Reference
:maxdepth: 1
tools-reference/llm-tool
tools-reference/llm-vision-tool
tools-reference/prompt-tool
tools-reference/python-tool
tools-reference/serp-api-tool
tools-reference/faiss_index_lookup_tool
tools-reference/vector_db_lookup_tool
tools-reference/embedding_tool
tools-reference/open_model_llm_tool
tools-reference/openai-gpt-4v-tool
tools-reference/contentsafety_text_tool
tools-reference/aoai-gpt4-turbo-vision
tools-reference/rerank-tool
```
```{toctree}
:caption: YAML Schema
:maxdepth: 1
flow-yaml-schema-reference.md
run-yaml-schema-reference.md
```
```{toctree}
:caption: Trace Specification
:maxdepth: 1
trace-span-spec-reference.md
```
docs/reference/pf-command-reference.md
+1104
View File
File diff suppressed because it is too large Load Diff
docs/reference/pfazure-command-reference.md
+617
View File
@@ -0,0 +1,617 @@
# pfazure
:::{admonition} Experimental feature
This is an experimental feature, and may change at any time. Learn [more](../how-to-guides/faq.md#stable-vs-experimental).
:::
Manage prompt flow resources on Azure with the prompt flow CLI.
| Command | Description |
| --- | --- |
| [pfazure flow](#pfazure-flow) | Manage flows. |
| [pfazure run](#pfazure-run) | Manage runs. |
## pfazure flow
Manage flows.
| Command | Description |
| --- | --- |
| [pfazure flow create](#pfazure-flow-create) | Create a flow. |
| [pfazure flow update](#pfazure-flow-update) | Update a flow. |
| [pfazure flow list](#pfazure-flow-list) | List flows in a workspace. |
### pfazure flow create
Create a flow in Azure AI from a local flow folder.
```bash
pfazure flow create [--flow]
[--set]
[--subscription]
[--resource-group]
[--workspace-name]
```
#### Parameters
`--flow`
Local path to the flow directory.
`--set`
Update an object by specifying a property path and value to set.
- `display_name`: Flow display name that will be created in remote. Default to be flow folder name + timestamp if not specified. e.g. "--set display_name=\<display_name\>".
- `type`: Flow type. Default to be "standard" if not specified. Available types are: "standard", "evaluation", "chat". e.g. "--set type=\<type\>".
- `description`: Flow description. e.g. "--set description=\<description\>."
- `tags`: Flow tags. e.g. "--set tags.key1=value1 tags.key2=value2."
`--subscription`
Subscription id, required when there is no default value from `az configure`.
`--resource-group -g`
Resource group name, required when there is no default value from `az configure`.
`--workspace-name -w`
Workspace name, required when there is no default value from `az configure`.
### pfazure flow update
Update a flow's metadata, such as `display name`, `description` and `tags`.
```bash
pfazure flow update --flow
[--set]
[--subscription]
[--resource-group]
[--workspace-name]
```
#### Parameters
`--flow`
The flow name on azure. It's a guid that can be found from 2 ways:
- After creating a flow to azure, it can be found in the printed message in "name" attribute.
- Open a flow in azure portal, the guid is in the url. e.g. https://ml.azure.com/prompts/flow/{workspace-id}/{flow-name}/xxx
`--set`
Update an object by specifying a property path and value to set.
- `display_name`: Flow display name. e.g. "--set display_name=\<display_name\>".
- `description`: Flow description. e.g. "--set description=\<description\>."
- `tags`: Flow tags. e.g. "--set tags.key1=value1 tags.key2=value2."
`--subscription`
Subscription id, required when there is no default value from `az configure`.
`--resource-group -g`
Resource group name, required when there is no default value from `az configure`.
`--workspace-name -w`
Workspace name, required when there is no default value from `az configure`.
### pfazure flow list
List remote flows on Azure AI.
```bash
pfazure flow list [--max-results]
[--include-others]
[--type]
[--output]
[--archived-only]
[--include-archived]
[--subscription]
[--resource-group]
[--workspace-name]
[--output]
```
#### Parameters
`--max-results -r`
Max number of results to return. Default is 50, upper bound is 100.
`--include-others`
Include flows created by other owners. By default only flows created by the current user are returned.
`--type`
Filter flows by type. Available types are: "standard", "evaluation", "chat".
`--archived-only`
List archived flows only.
`--include-archived`
List archived flows and active flows.
`--output -o`
Output format. Allowed values: `json`, `table`. Default: `json`.
`--subscription`
Subscription id, required when there is no default value from `az configure`.
`--resource-group -g`
Resource group name, required when there is no default value from `az configure`.
`--workspace-name -w`
Workspace name, required when there is no default value from `az configure`.
## pfazure run
Manage prompt flow runs.
| Command | Description |
| --- | --- |
| [pfazure run create](#pfazure-run-create) | Create a run. |
| [pfazure run list](#pfazure-run-list) | List runs in a workspace. |
| [pfazure run show](#pfazure-run-show) | Show details for a run. |
| [pfazure run stream](#pfazure-run-stream) | Stream run logs to the console. |
| [pfazure run show-details](#pfazure-run-show-details) | Show a run details. |
| [pfazure run show-metrics](#pfazure-run-show-metrics) | Show run metrics. |
| [pfazure run visualize](#pfazure-run-visualize) | Visualize a run. |
| [pfazure run archive](#pfazure-run-archive) | Archive a run. |
| [pfazure run restore](#pfazure-run-restore) | Restore a run. |
| [pfazure run update](#pfazure-run-update) | Update a run. |
| [pfazure run download](#pfazure-run-download) | Download a run. |
### pfazure run create
Create a run.
```bash
pfazure run create [--file]
[--flow]
[--data]
[--column-mapping]
[--run]
[--variant]
[--stream]
[--environment-variables]
[--connections]
[--resume-from] # require promptflow>=1.8.0
[--set]
[--subscription]
[--resource-group]
[--workspace-name]
```
#### Parameters
`--file -f`
Local path to the YAML file containing the prompt flow run specification; can be overwritten by other parameters. Reference [here](https://azuremlschemas.azureedge.net/promptflow/latest/Run.schema.json) for YAML schema.
`--flow`
The flow source to create the run. It could be:
- Local path to the flow directory.
```bash
pfazure run create --flow <path-to-flow-directory> --data <path-to-data-file> --column-mapping <key-value-pair>
```
- The flow name on azure with a prefix `azureml:`. Flow name is a guid that can be found from 2 ways:
- After creating a flow to azure, it can be found in the printed message in "name" attribute.
- Open a flow in azure portal, the guid is in the url. e.g. https://ml.azure.com/prompts/flow/{workspace-id}/{flow-name}/xxx
```bash
pfazure run create --flow azureml:<flow-name> --data <path-to-data-file> --column-mapping <key-value-pair>
```
`--data`
Local path to the data file or remote data. e.g. azureml:name:version.
`--column-mapping`
Inputs column mapping, use `${data.xx}` to refer to data columns, use `${run.inputs.xx}` to refer to referenced run's data columns, and `${run.outputs.xx}` to refer to run outputs columns.
`--run`
Referenced flow run name. For example, you can run an evaluation flow against an existing run. For example, "pfazure run create --flow evaluation_flow_dir --run existing_bulk_run --column-mapping url='${data.url}'".
`--variant`
Node & variant name in format of `${node_name.variant_name}`.
`--stream -s`
Indicates whether to stream the run's logs to the console.
default value: False
`--environment-variables`
Environment variables to set by specifying a property path and value. Example:
`--environment-variable key1='${my_connection.api_key}' key2='value2'`. The value reference
to connection keys will be resolved to the actual value, and all environment variables
specified will be set into os.environ.
`--connections`
Overwrite node level connections with provided value.
Example: `--connections node1.connection=test_llm_connection node1.deployment_name=gpt-35-turbo`
`--resume-from`
Create a run resume from an existing run. (Require promptflow>=1.8.0)
Example: `--resume-from <run_name>`
`--set`
Update an object by specifying a property path and value to set.
Example: `--set property1.property2=<value>`.
`--subscription`
Subscription id, required when there is no default value from `az configure`.
`--resource-group -g`
Resource group name, required when there is no default value from `az configure`.
`--workspace-name -w`
Workspace name, required when there is no default value from `az configure`.
### pfazure run list
List runs in a workspace.
```bash
pfazure run list [--archived-only]
[--include-archived]
[--max-results]
[--subscription]
[--resource-group]
[--workspace-name]
```
#### Parameters
`--archived-only`
List archived runs only.
default value: False
`--include-archived`
List archived runs and active runs.
default value: False
`--max-results -r`
Max number of results to return. Default is 50, upper bound is 100.
default value: 50
`--subscription`
Subscription id, required when there is no default value from `az configure`.
`--resource-group -g`
Resource group name, required when there is no default value from `az configure`.
`--workspace-name -w`
Workspace name, required when there is no default value from `az configure`.
### pfazure run show
Show details for a run.
```bash
pfazure run show --name
[--subscription]
[--resource-group]
[--workspace-name]
```
#### Parameters
`--name -n`
Name of the run.
`--subscription`
Subscription id, required when there is no default value from `az configure`.
`--resource-group -g`
Resource group name, required when there is no default value from `az configure`.
`--workspace-name -w`
Workspace name, required when there is no default value from `az configure`.
### pfazure run stream
Stream run logs to the console.
```bash
pfazure run stream --name
[--timeout]
[--subscription]
[--resource-group]
[--workspace-name]
```
#### Parameters
`--name -n`
Name of the run.
`--timeout`
Timeout in seconds. If the run stays in the same status and produce no new logs in a period longer than the timeout value, the stream operation will abort. Default value is 600 seconds
`--subscription`
Subscription id, required when there is no default value from `az configure`.
`--resource-group -g`
Resource group name, required when there is no default value from `az configure`.
`--workspace-name -w`
Workspace name, required when there is no default value from `az configure`.
### pfazure run show-details
Show a run details.
```bash
pfazure run show-details --name
[--subscription]
[--resource-group]
[--workspace-name]
```
#### Parameters
`--name -n`
Name of the run.
`--subscription`
Subscription id, required when there is no default value from `az configure`.
`--resource-group -g`
Resource group name, required when there is no default value from `az configure`.
`--workspace-name -w`
Workspace name, required when there is no default value from `az configure`.
### pfazure run show-metrics
Show run metrics.
```bash
pfazure run show-metrics --name
[--subscription]
[--resource-group]
[--workspace-name]
```
#### Parameters
`--name -n`
Name of the run.
`--subscription`
Subscription id, required when there is no default value from `az configure`.
`--resource-group -g`
Resource group name, required when there is no default value from `az configure`.
`--workspace-name -w`
Workspace name, required when there is no default value from `az configure`.
### pfazure run visualize
Visualize a run.
```bash
pfazure run visualize --name
[--subscription]
[--resource-group]
[--workspace-name]
```
#### Parameters
`--name -n`
Name of the run.
`--subscription`
Subscription id, required when there is no default value from `az configure`.
`--resource-group -g`
Resource group name, required when there is no default value from `az configure`.
`--workspace-name -w`
Workspace name, required when there is no default value from `az configure`.
### pfazure run archive
Archive a run.
```bash
pfazure run archive --name
[--subscription]
[--resource-group]
[--workspace-name]
```
#### Parameters
`--name -n`
Name of the run.
`--subscription`
Subscription id, required when there is no default value from `az configure`.
`--resource-group -g`
Resource group name, required when there is no default value from `az configure`.
`--workspace-name -w`
Workspace name, required when there is no default value from `az configure`.
### pfazure run restore
Restore a run.
```bash
pfazure run restore --name
[--subscription]
[--resource-group]
[--workspace-name]
```
#### Parameters
`--name -n`
Name of the run.
`--subscription`
Subscription id, required when there is no default value from `az configure`.
`--resource-group -g`
Resource group name, required when there is no default value from `az configure`.
`--workspace-name -w`
Workspace name, required when there is no default value from `az configure`.
### pfazure run update
Update a run's metadata, such as `display name`, `description` and `tags`.
```bash
pfazure run update --name
[--set display_name="<value>" description="<value>" tags.key="<value>"]
[--subscription]
[--resource-group]
[--workspace-name]
```
#### Examples
Set `display name`, `description` and `tags`:
```bash
pfazure run update --name <run_name> --set display_name="<value>" description="<value>" tags.key="<value>"
```
#### Parameters
`--name -n`
Name of the run.
`--set`
Set meta information of the run, like `display_name`, `description` or `tags`. Example: --set <key>=<value>.
`--subscription`
Subscription id, required when there is no default value from `az configure`.
`--resource-group -g`
Resource group name, required when there is no default value from `az configure`.
`--workspace-name -w`
Workspace name, required when there is no default value from `az configure`.
### pfazure run download
Download a run's metadata, such as `input`, `output`, `snapshot` and `artifact`. After the download is finished, you can use `pf run create --source <run-info-local-folder>` to register this run as a local run record, then you can use commands like `pf run show/visualize` to inspect the run just like a run that was created from local flow.
```bash
pfazure run download --name
[--output]
[--overwrite]
[--subscription]
[--resource-group]
[--workspace-name]
```
#### Examples
Download a run data to local:
```bash
pfazure run download --name <name> --output <output-folder-path>
```
#### Parameters
`--name -n`
Name of the run.
`--output -o`
Output folder path to store the downloaded run data. Default to be `~/.promptflow/.runs` if not specified
`--overwrite`
Overwrite the existing run data if the output folder already exists. Default to be `False` if not specified
`--subscription`
Subscription id, required when there is no default value from `az configure`.
`--resource-group -g`
Resource group name, required when there is no default value from `az configure`.
`--workspace-name -w`
Workspace name, required when there is no default value from `az configure`.
docs/reference/python-library-reference/promptflow.md
+1
View File
@@ -0,0 +1 @@
# PLACEHOLDER
docs/reference/run-yaml-schema-reference.md
+69
View File
@@ -0,0 +1,69 @@
# Run YAML Schema
:::{admonition} Experimental feature
This is an experimental feature, and may change at any time. Learn [more](../how-to-guides/faq.md#stable-vs-experimental).
:::
The source JSON schema can be found at [Run.schema.json](https://azuremlschemas.azureedge.net/promptflow/latest/Run.schema.json)
## YAML syntax
| Key | Type | Description |
|-------------------------|---------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `$schema` | string | The YAML schema. If you use the prompt flow VS Code extension to author the YAML file, including $schema at the top of your file enables you to invoke schema and resource completions. |
| `name` | string | The name of the run. |
| `flow` | string | Path of the flow directory. |
| `description` | string | Description of the run. |
| `display_name` | string | Display name of the run. |
| `data` | string | Input data for the run. Local path or remote uri(starts with azureml: or public URL) are supported. Note: remote uri is only supported for cloud run. |
| `run` | string | Referenced flow run name. For example, you can run an evaluation flow against an existing run. |
| `column_mapping` | object | Inputs column mapping, use `${data.xx}` to refer to data columns, use `${run.inputs.xx}` to refer to referenced run's data columns, and `${run.outputs.xx}` to refer to run outputs columns. |
| `connections` | object | Overwrite node level connections with provided value. Example: --connections node1.connection=test_llm_connection node1.deployment_name=gpt-35-turbo |
| `environment_variables` | object/string | Environment variables to set by specifying a property path and value. Example: `{"key1"="${my_connection.api_key}"}`. The value reference to connection keys will be resolved to the actual value, and all environment variables specified will be set into os.environ. |
| `properties` | object | Dictionary of properties of the run. |
| `tags` | object | Dictionary of tags of the run. |
| `resources` | object | Dictionary of resources used for compute session. Only supported for cloud run. See [Resources Schema](#resources-schema) for the set of configurable properties. |
| `variant` | string | The variant for the run. |
| `status` | string | The status of the run. Only available for when getting an existing run. Won't take affect if set when creating a run. |
|`identity`| object | Dictionary of identity configuration for compute session. Only supported for cloud run. See [Identity Schema](#identity-schema) for the set of configurable properties.
### Resources Schema
| Key | Type | Description |
|-----------------|--------|-----------------------------------------------------|
| `instance_type` | string | The instance type for compute session of the run. |
| `compute` | string | The compute instance for compute session session. |
### Identity Schema
| Key | Type | Description |
|-------------|--------|----------------------------------------------------------------------|
| `type` | string | Identity type, currently only support `managed` and `user_identity`. |
| `client_id` | string | Client id for managed identity, only avaible on managed identity. |
## Examples
Run examples are available in the [GitHub repository](https://github.com/microsoft/promptflow/tree/main/examples/flows).
- [basic](https://github.com/microsoft/promptflow/tree/main/examples/flows/standard/basic/run.yml)
- [web-classification](https://github.com/microsoft/promptflow/tree/main/examples/flows/standard/web-classification/run.yml)
- [flow-with-additional-includes](https://github.com/microsoft/promptflow/tree/main/examples/flows/standard/flow-with-additional-includes/run.yml)
### Run with identity examples
```yaml
# default value
identity:
type: user_identity
# use workspace primary UAI
identity:
type: managed
# use specified client_id's UAI
identity:
type: managed
client_id: xxx
```
docs/reference/tools-reference/aoai-gpt4-turbo-vision.md
+45
View File
@@ -0,0 +1,45 @@
# Azure OpenAI GPT-4 Turbo with Vision
## Introduction
Azure OpenAI GPT-4 Turbo with Vision tool enables you to leverage your AzureOpenAI GPT-4 Turbo with Vision model deployment to analyze images and provide textual responses to questions about them.
## Prerequisites
- Create AzureOpenAI resources
Create Azure OpenAI resources with [instruction](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal)
- Create a GPT-4 Turbo with Vision deployment
Browse to [Azure OpenAI Studio](https://oai.azure.com/) and sign in with the credentials associated with your Azure OpenAI resource. During or after the sign-in workflow, select the appropriate directory, Azure subscription, and Azure OpenAI resource.
Under Management select Deployments and Create a GPT-4 Turbo with Vision deployment by selecting model name: `gpt-4` and model version `vision-preview`.
## Connection
Setup connections to provisioned resources in prompt flow.
| Type | Name | API KEY | API Type | API Version |
|-------------|----------|----------|----------|-------------|
| AzureOpenAI | Required | Required | Required | Required |
## Inputs
| Name | Type | Description | Required |
|------------------------|-------------|------------------------------------------------------------------------------------------------|----------|
| connection | AzureOpenAI | the AzureOpenAI connection to be used in the tool | Yes |
| deployment\_name | string | the language model to use | Yes |
| prompt | string | The text prompt that the language model will use to generate it's response. | Yes |
| max\_tokens | integer | the maximum number of tokens to generate in the response. Default is 512. | No |
| temperature | float | the randomness of the generated text. Default is 1. | No |
| stop | list | the stopping sequence for the generated text. Default is null. | No |
| top_p | float | the probability of using the top choice from the generated tokens. Default is 1. | No |
| presence\_penalty | float | value that controls the model's behavior with regards to repeating phrases. Default is 0. | No |
| frequency\_penalty | float | value that controls the model's behavior with regards to generating rare phrases. Default is 0. | No |
| detail | string | configure how the model interprets and processes images, default is "auto". [Read more](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/gpt-with-vision?tabs=rest%2Csystem-assigned%2Cresource#detail-parameter-settings-in-image-processing-low-high-auto) | No |
## Outputs
| Return Type | Description |
|-------------|------------------------------------------|
| string | The text of one response of conversation |
docs/reference/tools-reference/contentsafety_text_tool.md
+55
View File
@@ -0,0 +1,55 @@
# Content Safety (Text)
Azure Content Safety is a content moderation service developed by Microsoft that help users detect harmful content from different modalities and languages. This tool is a wrapper for the Azure Content Safety Text API, which allows you to detect text content and get moderation results. See the [Azure Content Safety](https://aka.ms/acs-doc) for more information.
## Requirements
- For AzureML users, the tool is installed in default image, you can use the tool without extra installation.
- For local users,
`pip install promptflow-tools`
> [!NOTE]
> Content Safety (Text) tool is now incorporated into the latest `promptflow-tools` package. If you have previously installed the package `promptflow-contentsafety`, please uninstall it to avoid the duplication in your local tool list.
## Prerequisites
- Create an [Azure Content Safety](https://aka.ms/acs-create) resource.
- Add "Azure Content Safety" connection in prompt flow. Fill "API key" field with "Primary key" from "Keys and Endpoint" section of created resource.
## Inputs
You can use the following parameters as inputs for this tool:
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| text | string | The text that need to be moderated. | Yes |
| hate_category | string | The moderation sensitivity for Hate category. You can choose from four options: *disable*, *low_sensitivity*, *medium_sensitivity*, or *high_sensitivity*. The *disable* option means no moderation for hate category. The other three options mean different degrees of strictness in filtering out hate content. The default option is *medium_sensitivity*. | Yes |
| sexual_category | string | The moderation sensitivity for Sexual category. You can choose from four options: *disable*, *low_sensitivity*, *medium_sensitivity*, or *high_sensitivity*. The *disable* option means no moderation for sexual category. The other three options mean different degrees of strictness in filtering out sexual content. The default option is *medium_sensitivity*. | Yes |
| self_harm_category | string | The moderation sensitivity for Self-harm category. You can choose from four options: *disable*, *low_sensitivity*, *medium_sensitivity*, or *high_sensitivity*. The *disable* option means no moderation for self-harm category. The other three options mean different degrees of strictness in filtering out self_harm content. The default option is *medium_sensitivity*. | Yes |
| violence_category | string | The moderation sensitivity for Violence category. You can choose from four options: *disable*, *low_sensitivity*, *medium_sensitivity*, or *high_sensitivity*. The *disable* option means no moderation for violence category. The other three options mean different degrees of strictness in filtering out violence content. The default option is *medium_sensitivity*. | Yes |
For more information, please refer to [Azure Content Safety](https://aka.ms/acs-doc)
## Outputs
The following is an example JSON format response returned by the tool:
<details>
<summary>Output</summary>
```json
{
"action_by_category": {
"Hate": "Accept",
"SelfHarm": "Accept",
"Sexual": "Accept",
"Violence": "Accept"
},
"suggested_action": "Accept"
}
```
</details>
The `action_by_category` field gives you a binary value for each category: *Accept* or *Reject*. This value shows if the text meets the sensitivity level that you set in the request parameters for that category.
The `suggested_action` field gives you an overall recommendation based on the four categories. If any category has a *Reject* value, the `suggested_action` will be *Reject* as well.
docs/reference/tools-reference/embedding_tool.md
+62
View File
@@ -0,0 +1,62 @@
# Embedding
## Introduction
OpenAI's embedding models convert text into dense vector representations for various NLP tasks. See the [OpenAI Embeddings API](https://platform.openai.com/docs/api-reference/embeddings) for more information.
## Prerequisite
Create OpenAI resources:
- **OpenAI**
Sign up account [OpenAI website](https://openai.com/)
Login and [Find personal API key](https://platform.openai.com/account/api-keys)
- **Azure OpenAI (AOAI)**
Create Azure OpenAI resources with [instruction](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal)
## **Connections**
Setup connections to provide resources in embedding tool.
| Type | Name | API KEY | API Type | API Version |
|-------------|----------|----------|----------|-------------|
| OpenAI | Required | Required | - | - |
| AzureOpenAI | Required | Required | Required | Required |
## Inputs
| Name | Type | Description | Required |
|------------------------|-------------|-----------------------------------------------------------------------|----------|
| input | string | the input text to embed | Yes |
| connection | string | the connection for the embedding tool use to provide resources | Yes |
| model/deployment_name | string | instance of the text-embedding engine to use. Fill in model name if you use OpenAI connection, or deployment name if use Azure OpenAI connection. | Yes |
## Outputs
| Return Type | Description |
|-------------|------------------------------------------|
| list | The vector representations for inputs |
The following is an example response returned by the embedding tool:
<details>
<summary>Output</summary>
```
[-0.005744616035372019,
-0.007096089422702789,
-0.00563855143263936,
-0.005272455979138613,
-0.02355326898396015,
0.03955197334289551,
-0.014260607771575451,
-0.011810848489403725,
-0.023170066997408867,
-0.014739611186087132,
...]
```
</details>
docs/reference/tools-reference/faiss_index_lookup_tool.md
+87
View File
@@ -0,0 +1,87 @@
# Faiss Index Lookup
Faiss Index Lookup is a tool tailored for querying within a user-provided Faiss-based vector store. In combination with our Large Language Model (LLM) tool, it empowers users to extract contextually relevant information from a domain knowledge base.
## Requirements
- For AzureML users, the tool is installed in default image, you can use the tool without extra installation.
- For local users, if your index is stored in local path,
`pip install promptflow-vectordb`
if your index is stored in Azure storage,
`pip install promptflow-vectordb[azure]`
## Prerequisites
### For AzureML users,
- step 1. Prepare an accessible path on Azure Blob Storage. Here's the guide if a new storage account needs to be created: [Azure Storage Account](https://learn.microsoft.com/en-us/azure/storage/common/storage-account-create?tabs=azure-portal).
- step 2. Create related Faiss-based index files on Azure Blob Storage. We support the LangChain format (index.faiss + index.pkl) for the index files, which can be prepared either by employing our promptflow-vectordb SDK or following the quick guide from [LangChain documentation](https://python.langchain.com/docs/integrations/vectorstores/faiss). Please refer to the instructions of <a href="https://aka.ms/pf-sample-build-faiss-index" target="_blank">An example code for creating Faiss index</a> for building index using promptflow-vectordb SDK.
- step 3. Based on where you put your own index files, the identity used by the promptflow runtime should be granted with certain roles. Please refer to [Steps to assign an Azure role](https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-steps):
| Location | Role |
| ---- | ---- |
| workspace datastores or workspace default blob | AzureML Data Scientist |
| other blobs | Storage Blob Data Reader |
### For local users,
- Create Faiss-based index files in local path by only doing step 2 above.
## Inputs
The tool accepts the following inputs:
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| path | string | URL or path for the vector store.<br><br>local path (for local users):<br>`<local_path_to_the_index_folder>`<br><br> Azure blob URL format (with [azure] extra installed):<br>https://`<account_name>`.blob.core.windows.net/`<container_name>`/`<path_and_folder_name>`.<br><br>AML datastore URL format (with [azure] extra installed):<br>azureml://subscriptions/`<your_subscription>`/resourcegroups/`<your_resource_group>`/workspaces/`<your_workspace>`/data/`<data_path>`<br><br>public http/https URL (for public demonstration):<br>http(s)://`<path_and_folder_name>` | Yes |
| vector | list[float] | The target vector to be queried, which can be generated by the LLM tool. | Yes |
| top_k | integer | The count of top-scored entities to return. Default value is 3. | No |
## Outputs
The following is an example for JSON format response returned by the tool, which includes the top-k scored entities. The entity follows a generic schema of vector search result provided by our promptflow-vectordb SDK. For the Faiss Index Search, the following fields are populated:
| Field Name | Type | Description |
| ---- | ---- | ----------- |
| text | string | Text of the entity |
| score | float | Distance between the entity and the query vector |
| metadata | dict | Customized key-value pairs provided by user when create the index |
<details>
<summary>Output</summary>
```json
[
{
"metadata": {
"link": "http://sample_link_0",
"title": "title0"
},
"original_entity": null,
"score": 0,
"text": "sample text #0",
"vector": null
},
{
"metadata": {
"link": "http://sample_link_1",
"title": "title1"
},
"original_entity": null,
"score": 0.05000000447034836,
"text": "sample text #1",
"vector": null
},
{
"metadata": {
"link": "http://sample_link_2",
"title": "title2"
},
"original_entity": null,
"score": 0.20000001788139343,
"text": "sample text #2",
"vector": null
}
]
```
</details>
docs/reference/tools-reference/llm-tool.md
+225
View File
@@ -0,0 +1,225 @@
# LLM
## Introduction
Prompt flow LLM tool enables you to leverage widely used large language models like [OpenAI](https://platform.openai.com/), [Azure OpenAI (AOAI)](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/overview), and models in [Azure AI Studio model catalog](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/model-catalog) for natural language processing.
> [!NOTE]
> The previous version of the LLM tool is now being deprecated. Please upgrade to latest [promptflow-tools](https://pypi.org/project/promptflow-tools/) package to consume new llm tools.
Prompt flow provides a few different LLM APIs:
- **[Completion](https://platform.openai.com/docs/api-reference/completions)**: OpenAI's completion models generate text based on provided prompts.
- **[Chat](https://platform.openai.com/docs/api-reference/chat)**: OpenAI's chat models facilitate interactive conversations with text-based inputs and responses.
## Prerequisite
Create OpenAI resources, Azure OpenAI resources or MaaS deployment with the LLM models (e.g.: llama2, mistral, cohere etc.) in Azure AI Studio model catalog:
- **OpenAI**
Sign up account [OpenAI website](https://openai.com/)
Login and [Find personal API key](https://platform.openai.com/account/api-keys)
- **Azure OpenAI (AOAI)**
Create Azure OpenAI resources with [instruction](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal)
- **MaaS deployment**
Create MaaS deployment for models in Azure AI Studio model catalog with [instruction](https://learn.microsoft.com/azure/ai-studio/concepts/deployments-overview)
You can create serverless connection to use this MaaS deployment.
## **Connections**
Setup connections to provisioned resources in prompt flow.
| Type | Name | API KEY | API BASE | API Type | API Version |
|-------------|----------|----------|----------|-----------|-------------|
| OpenAI | Required | Required | - | - | - |
| AzureOpenAI | Required | Required | Required | Required | Required |
| Serverless | Required | Required | Required | - | - |
## Inputs
### Text Completion
| Name | Type | Description | Required |
|------------------------|-------------|-----------------------------------------------------------------------------------------|----------|
| prompt | string | text prompt that the language model will complete | Yes |
| model, deployment_name | string | the language model to use | Yes |
| max\_tokens | integer | the maximum number of tokens to generate in the completion. Default is 16. | No |
| temperature | float | the randomness of the generated text. Default is 1. | No |
| stop | list | the stopping sequence for the generated text. Default is null. | No |
| suffix | string | text appended to the end of the completion | No |
| top_p | float | the probability of using the top choice from the generated tokens. Default is 1. | No |
| logprobs | integer | the number of log probabilities to generate. Default is null. | No |
| echo | boolean | value that indicates whether to echo back the prompt in the response. Default is false. | No |
| presence\_penalty | float | value that controls the model's behavior with regards to repeating phrases. Default is 0. | No |
| frequency\_penalty | float | value that controls the model's behavior with regards to generating rare phrases. Default is 0. | No |
| best\_of | integer | the number of best completions to generate. Default is 1. | No |
| logit\_bias | dictionary | the logit bias for the language model. Default is empty dictionary. | No |
### Chat
| Name | Type | Description | Required |
|------------------------|-------------|------------------------------------------------------------------------------------------------|----------|
| prompt | string | text prompt that the language model will response | Yes |
| model, deployment_name | string | the language model to use | Yes |
| max\_tokens | integer | the maximum number of tokens to generate in the response. Default is inf. | No |
| temperature | float | the randomness of the generated text. Default is 1. | No |
| stop | list | the stopping sequence for the generated text. Default is null. | No |
| top_p | float | the probability of using the top choice from the generated tokens. Default is 1. | No |
| presence\_penalty | float | value that controls the model's behavior with regards to repeating phrases. Default is 0. | No |
| frequency\_penalty | float | value that controls the model's behavior with regards to generating rare phrases. Default is 0.| No |
| logit\_bias | dictionary | the logit bias for the language model. Default is empty dictionary. | No |
| tool\_choice | object | value that controls which tool is called by the model. Default is null. | No |
| tools | list | a list of tools the model may generate JSON inputs for. Default is null. | No |
| response_format | object | an object specifying the format that the model must output. Default is null. | No |
## Outputs
| Return Type | Description |
|-------------|----------------------------------------------------------------------|
| string | The text of one predicted completion or response of conversation |
## How to use LLM Tool?
1. Setup and select the connections to OpenAI resources
2. Configure LLM model api and its parameters
3. Prepare the Prompt with [guidance](./prompt-tool.md#how-to-write-prompt).
## How to write a chat prompt?
_To grasp the fundamentals of creating a chat prompt, begin with [this section](./prompt-tool.md#how-to-write-prompt) for an introductory understanding of jinja._
We offer a method to distinguish between different roles in a chat prompt, such as "system", "user", "assistant" and "tool". The "system", "user", "assistant" roles can have "name" and "content" properties. The "tool" role, however, should have "tool_call_id" and "content" properties. For an example of a tool chat prompt, please refer to [Sample 3](#sample-3).
### Sample 1
```jinja
# system:
You are a helpful assistant.
{% for item in chat_history %}
# user:
{{item.inputs.question}}
# assistant:
{{item.outputs.answer}}
{% endfor %}
# user:
{{question}}
```
In LLM tool, the prompt is transformed to match the [openai messages](https://platform.openai.com/docs/api-reference/chat/create#chat-create-messages) structure before sending to openai chat API.
```
[
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "<question-of-chat-history-round-1>"
},
{
"role": "assistant",
"content": "<answer-of-chat-history-round-1>"
},
...
{
"role": "user",
"content": "<question>"
}
]
```
### Sample 2
```jinja
# system:
{# For role naming customization, the following syntax is used #}
## name:
Alice
## content:
You are a bot can tell good jokes.
```
In LLM tool, the prompt is transformed to match the [openai messages](https://platform.openai.com/docs/api-reference/chat/create#chat-create-messages) structure before sending to openai chat API.
```
[
{
"role": "system",
"name": "Alice",
"content": "You are a bot can tell good jokes."
}
]
```
### Sample 3
This sample illustrates how to write a tool chat prompt.
```jinja
# system:
You are a helpful assistant.
# user:
What is the current weather like in Boston?
# assistant:
{# The assistant message with 'tool_calls' must be followed by messages with role 'tool'. #}
## tool_calls:
{{llm_output.tool_calls}}
# tool:
{#
Messages with role 'tool' must be a response to a preceding message with 'tool_calls'.
Additionally, 'tool_call_id's should match ids of assistant message 'tool_calls'.
#}
## tool_call_id:
{{llm_output.tool_calls[0].id}}
## content:
{{tool-answer-of-last-question}}
# user:
{{question}}
```
In LLM tool, the prompt is transformed to match the [openai messages](https://platform.openai.com/docs/api-reference/chat/create#chat-create-messages) structure before sending to openai chat API.
```
[
{
"role": "system",
"content": "You are a helpful assistant."
},
{
"role": "user",
"content": "What is the current weather like in Boston?"
},
{
"role": "assistant",
"content": null,
"function_call": null,
"tool_calls": [
{
"id": "<tool-call-id-of-last-question>",
"type": "function",
"function": "<function-to-call-of-last-question>"
}
]
},
{
"role": "tool",
"tool_call_id": "<tool-call-id-of-last-question>",
"content": "<tool-answer-of-last-question>"
}
...
{
"role": "user",
"content": "<question>"
}
]
```
docs/reference/tools-reference/llm-vision-tool.md
+51
View File
@@ -0,0 +1,51 @@
# LLM Vision
## Introduction
Prompt flow LLM vision tool enables you to leverage your AzureOpenAI GPT-4 Turbo or OpenAI's GPT-4 with vision to analyze images and provide textual responses to questions about them.
## Prerequisites
Create OpenAI or Azure OpenAI resources:
- **OpenAI**
Sign up account [OpenAI website](https://openai.com/)
Login and [Find personal API key](https://platform.openai.com/account/api-keys)
- **Azure OpenAI (AOAI)**
Create Azure OpenAI resources with [instruction](https://learn.microsoft.com/en-us/azure/cognitive-services/openai/how-to/create-resource?pivots=web-portal)
Browse to [Azure OpenAI Studio](https://oai.azure.com/) and sign in with the credentials associated with your Azure OpenAI resource. During or after the sign-in workflow, select the appropriate directory, Azure subscription, and Azure OpenAI resource.
Under Management select Deployments and Create a GPT-4 Turbo with Vision deployment by selecting model name: `gpt-4` and model version `vision-preview`.
## **Connections**
Setup connections to provisioned resources in prompt flow.
| Type | Name | API KEY | API Type | API Version |
|-------------|----------|----------|----------|-------------|
| OpenAI | Required | Required | - | - |
| AzureOpenAI | Required | Required | Required | Required |
## Inputs
| Name | Type | Description | Required |
|-------------------------|-------------|------------------------------------------------------------------------------------------------|----------|
| model, deployment\_name | string | the language model to use | Yes |
| prompt | string | The text prompt that the language model will use to generate it's response. | Yes |
| max\_tokens | integer | the maximum number of tokens to generate in the response. Default is 512. | No |
| temperature | float | the randomness of the generated text. Default is 1. | No |
| stop | list | the stopping sequence for the generated text. Default is null. | No |
| top_p | float | the probability of using the top choice from the generated tokens. Default is 1. | No |
| presence\_penalty | float | value that controls the model's behavior with regards to repeating phrases. Default is 0. | No |
| frequency\_penalty | float | value that controls the model's behavior with regards to generating rare phrases. Default is 0.| No |
## Outputs
| Return Type | Description |
|-------------|------------------------------------------|
| string | The text of one response of conversation |
docs/reference/tools-reference/open_model_llm_tool.md
+89
View File
@@ -0,0 +1,89 @@
# Open Model LLM
## Introduction
The Open Model LLM tool enables the utilization of a variety of Open Model and Foundational Models, such as [Falcon](https://ml.azure.com/models/tiiuae-falcon-7b/version/4/catalog/registry/azureml) and [Llama 2](https://ml.azure.com/models/Llama-2-7b-chat/version/14/catalog/registry/azureml-meta), for natural language processing in Azure ML Prompt Flow.
Here's how it looks in action on the Visual Studio Code prompt flow extension. In this example, the tool is being used to call a LlaMa-2 chat endpoint and asking "What is CI?".
![Screenshot of the Open Model LLM On VScode Prompt Flow extension](../../media/reference/tools-reference/open_model_llm_on_vscode_promptflow.png)
This prompt flow tool supports two different LLM API types:
- **Chat**: Shown in the example above. The chat API type facilitates interactive conversations with text-based inputs and responses.
- **Completion**: The Completion API type is used to generate single response text completions based on provided prompt input.
## Quick Overview: How do I use Open Model LLM Tool?
1. Choose a Model from the AzureML Model Catalog and get it deployed.
2. Connect to the model deployment.
3. Configure the open model llm tool settings.
4. Prepare the Prompt with [guidance](./prompt-tool.md#how-to-write-prompt).
5. Run the flow.
## Prerequisites: Model Deployment
1. Pick the model which matched your scenario from the [Azure Machine Learning model catalog](https://ml.azure.com/model/catalog).
2. Use the "Deploy" button to deploy the model to a AzureML Online Inference endpoint.
2.1. Use one of the Pay as you go deployment options.
More detailed instructions can be found here [Deploying foundation models to endpoints for inferencing.](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-foundation-models?view=azureml-api-2#deploying-foundation-models-to-endpoints-for-inferencing)
## Prerequisites: Connect to the Model
In order for prompt flow to use your deployed model, you will need to connect to it. There are several ways to connect.
### 1. Endpoint Connections
Once associated to a AzureML or Azure AI Studio workspace, the Open Model LLM tool can use the endpoints on that workspace.
1. **Using AzureML or Azure AI Studio workspaces**: If you are using prompt flow in one of the web page based browsers workspaces, the online endpoints available on that workspace will automatically who up.
2. **Using VScode or Code First**: If you are using prompt flow in VScode or one of the Code First offerings, you will need to connect to the workspace. The Open Model LLM tool uses the azure.identity DefaultAzureCredential client for authorization. One way is through [setting environment credential values](https://learn.microsoft.com/en-us/python/api/azure-identity/azure.identity.environmentcredential?view=azure-python).
### 2. Custom Connections
The Open Model LLM tool uses the CustomConnection. Prompt flow supports two types of connections:
1. **Workspace Connections** - These are connections which are stored as secrets on an Azure Machine Learning workspace. While these can be used, in many places, the are commonly created and maintained in the Studio UI.
2. **Local Connections** - These are connections which are stored locally on your machine. These connections are not available in the Studio UX's, but can be used with the VScode extension.
Instructions on how to create a workspace or local Custom Connection [can be found here.](../../how-to-guides/manage-connections.md#create-a-connection)
The required keys to set are:
1. **endpoint_url**
- This value can be found at the previously created Inferencing endpoint.
2. **endpoint_api_key**
- Ensure to set this as a secret value.
- This value can be found at the previously created Inferencing endpoint.
3. **model_family**
- Supported values: LLAMA, DOLLY, GPT2, or FALCON
- This value is dependent on the type of deployment you are targeting.
## Running the Tool: Inputs
The Open Model LLM tool has a number of parameters, some of which are required. Please see the below table for details, you can match these to the screen shot above for visual clarity.
| Name | Type | Description | Required |
|------|------|-------------|----------|
| api | string | This is the API mode and will depend on the model used and the scenario selected. *Supported values: (Completion \| Chat)* | Yes |
| endpoint_name | string | Name of an Online Inferencing Endpoint with a supported model deployed on it. Takes priority over connection. | No |
| temperature | float | The randomness of the generated text. Default is 1. | No |
| max_new_tokens | integer | The maximum number of tokens to generate in the completion. Default is 500. | No |
| top_p | float | The probability of using the top choice from the generated tokens. Default is 1. | No |
| model_kwargs | dictionary | This input is used to provide configuration specific to the model used. For example, the Llama-02 model may use {\"temperature\":0.4}. *Default: {}* | No |
| deployment_name | string | The name of the deployment to target on the Online Inferencing endpoint. If no value is passed, the Inferencing load balancer traffic settings will be used. | No |
| prompt | string | The text prompt that the language model will use to generate it's response. | Yes |
## Outputs
| API | Return Type | Description |
|------------|-------------|------------------------------------------|
| Completion | string | The text of one predicted completion |
| Chat | string | The text of one response int the conversation |
## Deploying to an Online Endpoint
When deploying a flow containing the Open Model LLM tool to an online endpoint, there is an additional step to setup permissions. During deployment through the web pages, there is a choice between System-assigned and User-assigned Identity types. Either way, using the Azure Portal (or a similar functionality), add the "Reader" Job function role to the identity on the Azure Machine Learning workspace or Ai Studio project which is hosting the endpoint. The prompt flow deployment may need to be refreshed.
docs/reference/tools-reference/openai-gpt-4v-tool.md
+44
View File
@@ -0,0 +1,44 @@
# OpenAI GPT-4V
## Introduction
OpenAI GPT-4V tool enables you to leverage OpenAI's GPT-4 with vision, also referred to as GPT-4V or gpt-4-vision-preview in the API, to take images as input and answer questions about them.
## Prerequisites
- Create OpenAI resources
Sign up account [OpenAI website](https://openai.com/)
Login and [Find personal API key](https://platform.openai.com/account/api-keys)
- Get Access to GPT-4 API
To use GPT-4 with vision, you need access to GPT-4 API. Learn more about [How to get access to GPT-4 API](https://help.openai.com/en/articles/7102672-how-can-i-access-gpt-4)
## Connection
Setup connections to provisioned resources in prompt flow.
| Type | Name | API KEY |
|-------------|----------|----------|
| OpenAI | Required | Required |
## Inputs
| Name | Type | Description | Required |
|------------------------|-------------|------------------------------------------------------------------------------------------------|----------|
| connection | OpenAI | the OpenAI connection to be used in the tool | Yes |
| model | string | the language model to use, currently only support gpt-4-vision-preview | Yes |
| prompt | string | The text prompt that the language model will use to generate it's response. | Yes |
| max\_tokens | integer | the maximum number of tokens to generate in the response. Default is 512. | No |
| temperature | float | the randomness of the generated text. Default is 1. | No |
| stop | list | the stopping sequence for the generated text. Default is null. | No |
| top_p | float | the probability of using the top choice from the generated tokens. Default is 1. | No |
| presence\_penalty | float | value that controls the model's behavior with regards to repeating phrases. Default is 0. | No |
| frequency\_penalty | float | value that controls the model's behavior with regards to generating rare phrases. Default is 0. | No |
| detail | string | control over how the model processes the image and generates its textual understanding, default is "auto". [Read more](https://platform.openai.com/docs/guides/vision/low-or-high-fidelity-image-understanding) | No |
## Outputs
| Return Type | Description |
|-------------|------------------------------------------|
| string | The text of one response of conversation |
docs/reference/tools-reference/prompt-tool.md
+73
View File
@@ -0,0 +1,73 @@
# Prompt
## Introduction
The Prompt Tool in PromptFlow offers a collection of textual templates that serve as a starting point for creating prompts.
These templates, based on the Jinja2 template engine, facilitate the definition of prompts. The tool proves useful
when prompt tuning is required prior to feeding the prompts into the Language Model (LLM) model in PromptFlow.
## Inputs
| Name | Type | Description | Required |
|--------------------|--------|----------------------------------------------------------|----------|
| prompt | string | The prompt template in Jinja | Yes |
| Inputs | - | List of variables of prompt template and its assignments | - |
## Outputs
The prompt text parsed from the prompt + Inputs
## How to write Prompt?
1. Prepare jinja template. Learn more about [Jinja](https://jinja.palletsprojects.com/en/3.1.x/)
_In below example, the prompt incorporates Jinja templating syntax to dynamically generate the welcome message and personalize it based on the user's name. It also presents a menu of options for the user to choose from. Depending on whether the user_name variable is provided, it either addresses the user by name or uses a generic greeting._
```jinja
Welcome to {{ website_name }}!
{% if user_name %}
Hello, {{ user_name }}!
{% else %}
Hello there!
{% endif %}
Please select an option from the menu below:
1. View your account
2. Update personal information
3. Browse available products
4. Contact customer support
```
2. Assign value for the variables.
_In above example, two variables would be automatically detected and listed in '**Inputs**' section. Please assign values._
### Sample 1
Inputs
| Variable | Type | Sample Value |
|---------------|--------|--------------|
| website_name | string | "Microsoft" |
| user_name | string | "Jane" |
Outputs
```
Welcome to Microsoft! Hello, Jane! Please select an option from the menu below: 1. View your account 2. Update personal information 3. Browse available products 4. Contact customer support
```
### Sample 2
Inputs
| Variable | Type | Sample Value |
|--------------|--------|----------------|
| website_name | string | "Bing" |
| user_name | string | " |
Outputs
```
Welcome to Bing! Hello there! Please select an option from the menu below: 1. View your account 2. Update personal information 3. Browse available products 4. Contact customer support
```
docs/reference/tools-reference/python-tool.md
+112
View File
@@ -0,0 +1,112 @@
# Python
## Introduction
Users are empowered by the Python Tool to offer customized code snippets as self-contained executable nodes in PromptFlow.
Users can effortlessly create Python tools, edit code, and verify results with ease.
## Inputs
| Name | Type | Description | Required |
|--------|--------|------------------------------------------------------|---------|
| Code | string | Python code snippet | Yes |
| Inputs | - | List of tool function parameters and its assignments | - |
### Types
| Type | Python example | Description |
|-----------------------------------------------------|---------------------------------|--------------------------------------------|
| int | param: int | Integer type |
| bool | param: bool | Boolean type |
| string | param: str | String type |
| double | param: float | Double type |
| list | param: list or param: List[T] | List type |
| object | param: dict or param: Dict[K, V] | Object type |
| [Connection](../../concepts/concept-connections.md) | param: CustomConnection | Connection type, will be handled specially |
Parameters with `Connection` type annotation will be treated as connection inputs, which means:
- Promptflow extension will show a selector to select the connection.
- During execution time, promptflow will try to find the connection with the name same from parameter value passed in.
Note that `Union[...]` type annotation is supported **ONLY** for connection type,
for example, `param: Union[CustomConnection, OpenAIConnection]`.
## Outputs
The return of the python tool function.
## How to write Python Tool?
### Guidelines
1. Python Tool Code should consist of a complete Python code, including any necessary module imports.
2. Python Tool Code must contain a function decorated with @tool (tool function), serving as the entry point for execution. The @tool decorator should be applied only once within the snippet.
_Below sample defines python tool "my_python_tool", decorated with @tool_
3. Python tool function parameters must be assigned in 'Inputs' section
_Below sample defines inputs "message" and assign with "world"_
4. Python tool function shall have return
_Below sample returns a concatenated string_
### Code
The snippet below shows the basic structure of a tool function. Promptflow will read the function and extract inputs
from function parameters and type annotations.
```python
from promptflow.core import tool
from promptflow.connections import CustomConnection
# The inputs section will change based on the arguments of the tool function, after you save the code
# Adding type to arguments and return value will help the system show the types properly
# Please update the function name/signature per need
@tool
def my_python_tool(message: str, my_conn: CustomConnection) -> str:
my_conn_dict = dict(my_conn)
# Do some function call with my_conn_dict...
return 'hello ' + message
```
### Inputs
| Name | Type | Sample Value in Flow Yaml | Value passed to function|
|---------|--------|-------------------------| ------------------------|
| message | string | "world" | "world" |
| my_conn | CustomConnection | "my_conn" | CustomConnection object |
Promptflow will try to find the connection named 'my_conn' during execution time.
### outputs
```python
"hello world"
```
### Keyword Arguments Support
Starting from version 1.0.0 of PromptFlow and version 1.4.0 of [Prompt flow for VS Code](https://marketplace.visualstudio.com/items?itemName=prompt-flow.prompt-flow),
we have introduced support for keyword arguments (kwargs) in the Python tool.
```python
from promptflow.core import tool
@tool
def print_test(normal_input: str, **kwargs):
for key, value in kwargs.items():
print(f"Key {key}'s value is {value}")
return len(kwargs)
```
When you add `kwargs` in your python tool like above code, you can insert variable number of inputs by the `+Add input` button.
![Screenshot of the kwargs On VScode Prompt Flow extension](../../media/reference/tools-reference/python_tool_kwargs.png)
docs/reference/tools-reference/rerank-tool.md
+69
View File
@@ -0,0 +1,69 @@
# Rerank
## Introduction
Rerank is a semantic search tool that improves search quality with a semantic-based reranking system which can contextualize the meaning of a user's query beyond keyword relevance. This tool works best with look up tool as a ranker after the initial retrieval. The list of current supported ranking method is as follows.
| Name | Description |
| --- | --- |
| BM25 | BM25 is an open source ranking algorithm to measure the relevance of documents to a given query |
| Scaled Score Fusion | Scaled Score Fusion calculates a scaled relevance score. |
| Cohere Rerank | Cohere Rerank is the market’s leading reranking model used for semantic search and retrieval-augmented generation (RAG). |
## Requirements
- For AzureML users, the tool is installed in default image, you can use the tool without extra installation.
- For local users,
`pip install promptflow-vectordb`
## Prerequisites
BM25 and Scaled Score Fusion are included as default reranking methods. To use cohere rerank model, you should create serverless deployment to the model, and establish connection between the tool and the resource as follows.
- Add Serverless Model connection. Fill "API base" and "API key" field to your serverless deployment.
## Inputs
| Name | Type | Description | Required |
|------------------------|-------------|-----------------------------------------------------------------------|----------|
| query | string | the question relevant to your input documents | Yes |
| ranker_parameters | string | the type of ranking methods to use | Yes |
| result_groups | object | the list of document chunks to rerank. Normally this is output from lookup | Yes |
| top_k | int | the maximum number of relevant documents to return | No |
## Outputs
| Return Type | Description |
|-------------|------------------------------------------|
| text | text of the entity |
| metadata | metadata like file path and url |
| additional_fields | metadata and rerank score |
<details>
<summary>Output</summary>
```json
[
{
"text": "sample text",
"metadata":
{
"filepath": "sample_file_path",
"metadata_json_string": "meta_json_string"
"title": "",
"url": ""
},
"additional_fields":
{
"filepath": "sample_file_path",
"metadata_json_string": "meta_json_string"
"title": "",
"url": "",
"@promptflow_vectordb.reranker_score": 0.013795365
}
}
]
```
</details>
docs/reference/tools-reference/serp-api-tool.md
+45
View File
@@ -0,0 +1,45 @@
# SerpAPI
## Introduction
The SerpAPI API is a Python tool that provides a wrapper to the [SerpAPI Google Search Engine Results API](https://serpapi.com/search-api) and [SerpApi Bing Search Engine Results API
](https://serpapi.com/bing-search-api).
We could use the tool to retrieve search results from a number of different search engines, including Google and Bing, and you can specify a range of search parameters, such as the search query, location, device type, and more.
## Prerequisite
Sign up at [SERP API homepage](https://serpapi.com/)
## Connection
Connection is the model used to establish connections with Serp API.
| Type | Name | API KEY |
|-------------|----------|----------|
| Serp | Required | Required |
_**API Key** is on SerpAPI account dashboard_
## Inputs
The **serp api** tool supports following parameters:
| Name | Type | Description | Required |
|----------|---------|---------------------------------------------------------------|----------|
| query | string | The search query to be executed. | Yes |
| engine | string | The search engine to use for the search. Default is 'google'. | Yes |
| num | integer | The number of search results to return.Default is 10. | No |
| location | string | The geographic location to execute the search from. | No |
| safe | string | The safe search mode to use for the search. Default is 'off'. | No |
## Outputs
The json representation from serpapi query.
| Engine | Return Type | Output |
|----------|-------------|-------------------------------------------------------|
| google | json | [Sample](https://serpapi.com/search-api#api-examples) |
| bing | json | [Sample](https://serpapi.com/bing-search-api) |
docs/reference/tools-reference/vector_db_lookup_tool.md
+187
View File
@@ -0,0 +1,187 @@
# Vector DB Lookup
Vector DB Lookup is a vector search tool that allows users to search top k similar vectors from vector database. This tool is a wrapper for multiple third-party vector databases. The list of current supported databases is as follows.
| Name | Description |
| --- | --- |
| Azure Cognitive Search | Microsoft's cloud search service with built-in AI capabilities that enrich all types of information to help identify and explore relevant content at scale. |
| Qdrant | Qdrant is a vector similarity search engine that provides a production-ready service with a convenient API to store, search and manage points (i.e. vectors) with an additional payload. |
| Weaviate | Weaviate is an open source vector database that stores both objects and vectors. This allows for combining vector search with structured filtering. |
This tool will support more vector databases.
## Requirements
- For AzureML users, the tool is installed in default image, you can use the tool without extra installation.
- For local users,
`pip install promptflow-vectordb`
## Prerequisites
The tool searches data from a third-party vector database. To use it, you should create resources in advance and establish connection between the tool and the resource.
- **Azure Cognitive Search:**
- Create resource [Azure Cognitive Search](https://learn.microsoft.com/en-us/azure/search/search-create-service-portal).
- Add "Cognitive search" connection. Fill "API key" field with "Primary admin key" from "Keys" section of created resource, and fill "API base" field with the URL, the URL format is `https://{your_serive_name}.search.windows.net`.
- **Qdrant:**
- Follow the [installation](https://qdrant.tech/documentation/quick-start/) to deploy Qdrant to a self-maintained cloud server.
- Add "Qdrant" connection. Fill "API base" with your self-maintained cloud server address and fill "API key" field.
- **Weaviate:**
- Follow the [installation](https://weaviate.io/developers/weaviate/installation) to deploy Weaviate to a self-maintained instance.
- Add "Weaviate" connection. Fill "API base" with your self-maintained instance address and fill "API key" field.
## Inputs
The tool accepts the following inputs:
- **Azure Cognitive Search:**
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| connection | CognitiveSearchConnection | The created connection for accessing to Cognitive Search endpoint. | Yes |
| index_name | string | The index name created in Cognitive Search resource. | Yes |
| text_field | string | The text field name. The returned text field will populate the text of output. | No |
| vector_field | string | The vector field name. The target vector is searched in this vector field. | Yes |
| search_params | dict | The search parameters. It's key-value pairs. Except for parameters in the tool input list mentioned above, additional search parameters can be formed into a JSON object as search_params. For example, use `{"select": ""}` as search_params to select the returned fields, use `{"search": ""}` to perform a [hybrid search](https://learn.microsoft.com/en-us/azure/search/search-get-started-vector#hybrid-search). | No |
| search_filters | dict | The search filters. It's key-value pairs, the input format is like `{"filter": ""}` | No |
| vector | list | The target vector to be queried, which can be generated by Embedding tool. | Yes |
| top_k | int | The count of top-scored entities to return. Default value is 3 | No |
- **Qdrant:**
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| connection | QdrantConnection | The created connection for accessing to Qdrant server. | Yes |
| collection_name | string | The collection name created in self-maintained cloud server. | Yes |
| text_field | string | The text field name. The returned text field will populate the text of output. | No |
| search_params | dict | The search parameters can be formed into a JSON object as search_params. For example, use `{"params": {"hnsw_ef": 0, "exact": false, "quantization": null}}` to set search_params. | No |
| search_filters | dict | The search filters. It's key-value pairs, the input format is like `{"filter": {"should": [{"key": "", "match": {"value": ""}}]}}` | No |
| vector | list | The target vector to be queried, which can be generated by Embedding tool. | Yes |
| top_k | int | The count of top-scored entities to return. Default value is 3 | No |
- **Weaviate:**
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| connection | WeaviateConnection | The created connection for accessing to Weaviate. | Yes |
| class_name | string | The class name. | Yes |
| text_field | string | The text field name. The returned text field will populate the text of output. | No |
| vector | list | The target vector to be queried, which can be generated by Embedding tool. | Yes |
| top_k | int | The count of top-scored entities to return. Default value is 3 | No |
## Outputs
The following is an example JSON format response returned by the tool, which includes the top-k scored entities. The entity follows a generic schema of vector search result provided by promptflow-vectordb SDK.
- **Azure Cognitive Search:**
For Azure Cognitive Search, the following fields are populated:
| Field Name | Type | Description |
| ---- | ---- | ----------- |
| original_entity | dict | the original response json from search REST API|
| score | float | @search.score from the original entity, which evaluates the similarity between the entity and the query vector |
| text | string | text of the entity|
| vector | list | vector of the entity|
<details>
<summary>Output</summary>
```json
[
{
"metadata": null,
"original_entity": {
"@search.score": 0.5099789,
"id": "",
"your_text_filed_name": "sample text1",
"your_vector_filed_name": [-0.40517663431890405, 0.5856996257406859, -0.1593078462266455, -0.9776269170785785, -0.6145604369828972],
"your_additional_field_name": ""
},
"score": 0.5099789,
"text": "sample text1",
"vector": [-0.40517663431890405, 0.5856996257406859, -0.1593078462266455, -0.9776269170785785, -0.6145604369828972]
}
]
```
</details>
- **Qdrant:**
For Qdrant, the following fields are populated:
| Field Name | Type | Description |
| ---- | ---- | ----------- |
| original_entity | dict | the original response json from search REST API|
| metadata | dict | payload from the original entity|
| score | float | score from the original entity, which evaluates the similarity between the entity and the query vector|
| text | string | text of the payload|
| vector | list | vector of the entity|
<details>
<summary>Output</summary>
```json
[
{
"metadata": {
"text": "sample text1"
},
"original_entity": {
"id": 1,
"payload": {
"text": "sample text1"
},
"score": 1,
"vector": [0.18257418, 0.36514837, 0.5477226, 0.73029673],
"version": 0
},
"score": 1,
"text": "sample text1",
"vector": [0.18257418, 0.36514837, 0.5477226, 0.73029673]
}
]
```
</details>
- **Weaviate:**
For Weaviate, the following fields are populated:
| Field Name | Type | Description |
| ---- | ---- | ----------- |
| original_entity | dict | the original response json from search REST API|
| score | float | certainty from the original entity, which evaluates the similarity between the entity and the query vector|
| text | string | text in the original entity|
| vector | list | vector of the entity|
<details>
<summary>Output</summary>
```json
[
{
"metadata": null,
"original_entity": {
"_additional": {
"certainty": 1,
"distance": 0,
"vector": [
0.58,
0.59,
0.6,
0.61,
0.62
]
},
"text": "sample text1."
},
"score": 1,
"text": "sample text1.",
"vector": [
0.58,
0.59,
0.6,
0.61,
0.62
]
}
]
```
</details>
docs/reference/trace-span-spec-reference.md
+150
View File
@@ -0,0 +1,150 @@
# Trace Span Specification
:::{admonition} Experimental feature
This is an experimental feature, and may change at any time. Learn [more](../how-to-guides/faq.md#stable-vs-experimental).
:::
This document outlines the design of Prompt flow spans, detailing what information is traced and how it is structured.
## Introduction
The span, as you may know, is the fundamental unit of the trace system, representing a unit that captures execution information in the Prompt flow system. Spans are nested together in a parent-child relationship and paired together by link relationships, providing developers and users with a comprehensive view of the application’s execution process.
By adhering to these specifications, we ensure transparency and consistency in our tracing system.
The UI interprets the captured spans and presents them in a user-friendly manner. Understanding the fields and contracts defined within the spans is essential for effectively utilizing Prompt flow or integrating its components.
## OpenTelemetry Span Basics
A typical span object contains below information:
| Field | Description |
|---|---|
| name | Name of span |
| parent_id | Parent span ID (empty for root spans) |
| context | [Span Context](https://opentelemetry.io/docs/concepts/signals/traces/#span-context) |
| start_time | Start time of the span |
| end_time | End time of the span |
| status | [Span Status](https://opentelemetry.io/docs/concepts/signals/traces/#span-status) |
| attributes | [Attributes](https://opentelemetry.io/docs/concepts/signals/traces/#attributes) |
| events | [Span Events](https://opentelemetry.io/docs/concepts/signals/traces/#span-events) |
| links | [Span Links](https://opentelemetry.io/docs/concepts/signals/traces/#span-links) |
## Span in Prompt flow
In Prompt flow, we define several span types, and the system automatically creates spans with execution information in designated attributes and events.
These span types share common attributes and events, which we refer to as standard attributes and events. Let’s explore these common elements before diving into the specifics of each span type.
### Common Attributes and Events
#### Attributes
Each span in Prompt flow is enriched with a set of standard attributes that provide essential information about the span's context and purpose. The following table outlines these attributes:
| Attribute | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) |
|---|---|---|---|---|
| framework | string | This attribute specifies the framework in which the trace was recorded. For our project, this value is consistently set to promptflow. | promptflow | `Required` |
| node_name | string | Denotes the name of the flow node. | chat | `Conditionally Required` if the flow is a Directed Acyclic Graph ([DAG](../concepts/concept-flows.md#dag-flow)) flow. |
| span_type | string | Specifies the type of span, such as LLM or Flow. See [this](#span-types-specification) for details. | LLM | `Required` |
| line_run_id | string | Unique identifier for the execution run within Prompt flow. | d23159d5-cae0-4de6-a175-295c715ce251 | `Required` |
| function | string | The function associated with the span. | search | `Recommended` |
| session_id | string | Unique identifier for chat sessions. | 4ea1a462-7617-439f-a40c-12a8b93f51fb | `Opt-In` |
| referenced.line_run_id | string | Represents the line run ID that is the source of the evaluation run. | f747f7b8-983c-4bf2-95db-0ec3e33d4fd1 | `Conditionally Required` only used in evaluation runs - runs on [evaluation flow](../concepts/concept-flows.md#flow-types).|
| batch_run_id | string | The batch run ID when in batch mode. | 61daff70-80d5-4e79-a50b-11b38bb3d344 | `Conditionally Required` only used in batch runs. |
| referenced.batch_run_id | string | Notes the batch run ID against which an evaluation flow ran. | 851b32cb-545c-421d-8e51-0a3ea66f0075 | `Conditionally Required` only used in evaluation runs. |
| line_number | int | The line number within a batch run, starting from 0. | `1` | `Conditionally Required` only used in batch runs. |
| \_\_computed\_\_.cumulative_token_count.prompt | int | Cumulative token count of child nodes for prompts. [1] | `200` | `Recommended` |
| \_\_computed\_\_.cumulative_token_count.completion | int | Cumulative token count of child nodes for completion responses. [1] | `80` | `Recommended` |
| \_\_computed\_\_.cumulative_token_count.total | int | Total cumulative token count for both prompts and completions. [1] | `120` | `Recommended` |
**[1]:** Cumulative token counts are propagated up the span hierarchy, ensuring each span reflects the total token count of all LLM executions within its scope.
#### Events
In Prompt flow, events emitted by the Prompt flow framework follow the format below
- event MUST has attributes
- event attributes MUST contain a key named `payload`, which refers to the data carried within an event.
- event attributes payload MUST be a JSON string that represent an object.
| Event | Payload Description | Payload Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) |
|---|---|---|---|
| promptflow.function.inputs | Input of a function call | ```{"chat_history":[],"question":"What is ChatGPT?"}``` | `Required` |
| promptflow.function.output | Output of a function call | ```{"answer":"ChatGPT is a conversational AI model developed by OpenAI."}``` | `Required` |
### Span Types Specification
Within the Prompt flow system, we have delineated several distinct span types to cater to various execution units. Each span type is designed to capture specific execution information, complementing the standard attributes and events. Currently, our system includes the following span types: `LLM`, `Function`, `LangChain`, `Flow`, `Embedding` and `Retrieval`.
Beyond the standard attributes and events, each span type possesses designated fields to store pertinent information unique to its role within the system. These specialized attributes and events ensure that all relevant data is meticulously traced and available for analysis.
#### LLM
The LLM (Large Language Model) span captures detailed execution information from calls to large language models.
| Attribute | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) |
|---|---|---|---|---|
| span_type | string | Identifies the span as an LLM type. | LLM | `Required` |
| llm.usage.total_tokens | int | Total number of tokens used, including both prompt and response. | `180` | `Required` |
| llm.usage.prompt_tokens | int | Number of tokens used in the LLM prompt. | `100` | `Required` |
| llm.usage.completion_tokens | int | Number of tokens used in the LLM response (completion). | `80` | `Required` |
| llm.response.model | string | Specifies the LLM that generated the response. | gpt-4 | `Required` |
| Event | Payload Description | Payload Examples | Requirement Level |
|---|---|---|---|
| promptflow.llm.generated_message | Captures the output message from an LLM call. | ```{"content":"ChatGPT is a conversational AI model developed by OpenAI.","role":"assistant","function_call":null,"tool_calls":null}``` | `Required` |
>Note: OpenTelemetry currently defines several LLM-related span attributes and events as semantic conventions. We plan to align with these conventions in the future. For more information, visit [Semantic Conventions for GenAI operations](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-spans.md).
#### Function
The Function span is a versatile default span within Prompt flow, designed to capture a wide range of general function execution information.
| Attribute | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) |
|---|---|---|---|---|
| span_type | string | Identifies the span as a Function type. | Function | `Required` |
| Event | Payload Description | Payload Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) |
|---|---|---|---|
| promptflow.prompt.template | Details the prompt template and variable information. | ```{"prompt.template":"# system:\nYou are a helpful assistant.\n\n# user:\n{{question}}","prompt.variables":"{\n "question": "What is ChatGPT?"\n}"}``` | `Conditionally Required` if the function contains prompt template formating. [1] |
**[1]**: Template formatting is a process by resolving prompt template into prompt message, this process can happen within a function that invokes LLM call.
#### Flow
The Flow span encapsulates the execution details of a flow within Prompt flow.
| Attribute | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) |
|---|---|---|---|---|
| span_type | string | Designates the span as a Flow type. | Flow | `Required` |
#### Embedding
The Embedding span is dedicated to recording the details of embedding calls within Prompt flow.
| Attribute | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) |
|---|---|---|---|---|
| span_type | string | Denotes the span as an Embedding type. | Embedding | `Required` |
| llm.usage.total_tokens | int | Total number of tokens used, sum of prompt and response tokens. | `180` | `Required` |
| llm.usage.prompt_tokens | int | Number of tokens used in the prompt for the embedding call. | `100` | `Required` |
| llm.usage.completion_tokens | int | Number of tokens used in the response from the embedding call. | `80` | `Required` |
| llm.response.model | string | Identifies the LLM model used for generating the embedding. | text-embedding-ada-002 | `Required` |
| Event | Payload Description | Payload Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) |
|---|---|---|---|
| promptflow.embedding.embeddings | Details the embeddings generated by a call. | ```[{"embedding.vector":"","embedding.text":"When does a pipeline job reuse a previous job's results in Azure Machine Learning?"}]``` | `Required` |
#### Retrieval
The Retrieval span type is specifically designed to encapsulate the execution details of a retrieval task within Prompt flow.
| Attribute | Type | Description | Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) |
|---|---|---|---|---|
| span_type | string | Labels the span as a Retrieval type. | Retrieval | `Required` |
| Event | Payload Description | Payload Examples | [Requirement Level](https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/) |
|---|---|---|---|
| promptflow.retrieval.query | Captures the text of the retrieval query. | ```"When does a pipeline job reuse a previous job's results in Azure Machine Learning?"``` | `Required` |
| promptflow.retrieval.documents | Details the list of retrieved documents relevant to the query. | ```[{"document.id":"https://componentsdk.azurewebsites.net/howto/caching-reuse.html","document.score":2.677619457244873,"document.content":"# Component concepts &..."},{"document.id":"https://learn.microsoft.com/en-us/azure/machine-learning/v1/reference-pipeline-yaml","document.score":2.563112735748291,"document.content":"etc. \|\r\n\| runconfig \| T..."}]``` | `Required` |