chore: import upstream snapshot with attribution
tools_continuous_delivery / Private PyPI non-main branch release (push) Has been skipped
tools_continuous_delivery / Private PyPI main branch release (push) Failing after 2m42s
Publish Promptflow Doc / Build (push) Has been cancelled
Publish Promptflow Doc / Deploy (push) Has been cancelled
Flake8 Lint / flake8 (push) Has been cancelled
Spell check CI / Spell_Check (push) Has been cancelled
tools_continuous_delivery / Private PyPI non-main branch release (push) Has been skipped
tools_continuous_delivery / Private PyPI main branch release (push) Failing after 2m42s
Publish Promptflow Doc / Build (push) Has been cancelled
Publish Promptflow Doc / Deploy (push) Has been cancelled
Flake8 Lint / flake8 (push) Has been cancelled
Spell check CI / Spell_Check (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,5 @@
|
||||
promptflow
|
||||
langchain>=0.1.5
|
||||
langchain_community
|
||||
opentelemetry-instrumentation-langchain
|
||||
python-dotenv
|
||||
@@ -0,0 +1,181 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Tracing with LangChain apps"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The tracing capability provided by Prompt flow is built on top of [OpenTelemetry](https://opentelemetry.io/) that gives you complete observability over your LLM applications. \n",
|
||||
"And there is already a rich set of OpenTelemetry [instrumentation packages](https://opentelemetry.io/ecosystem/registry/?language=python&component=instrumentation) available in OpenTelemetry Eco System. \n",
|
||||
"\n",
|
||||
"In this example we will demo how to use [opentelemetry-instrumentation-langchain](https://github.com/traceloop/openllmetry/tree/main/packages/opentelemetry-instrumentation-langchain) package provided by [Traceloop](https://www.traceloop.com/) to instrument [LangChain](https://python.langchain.com/docs/tutorials/) apps.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"**Learning Objectives** - Upon completing this tutorial, you should be able to:\n",
|
||||
"\n",
|
||||
"- Trace `LangChain` applications and visualize the trace of your application in prompt flow.\n",
|
||||
"\n",
|
||||
"## Requirements\n",
|
||||
"\n",
|
||||
"To run this notebook example, please install required dependencies:"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%capture --no-stderr\n",
|
||||
"%pip install -r ./requirements.txt"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Start tracing LangChain using promptflow\n",
|
||||
"\n",
|
||||
"Start trace using `promptflow.start_trace`, click the printed url to view the trace ui."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from promptflow.tracing import start_trace\n",
|
||||
"\n",
|
||||
"# start a trace session, and print a url for user to check trace\n",
|
||||
"start_trace()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"By default, `opentelemetry-instrumentation-langchain` instrumentation logs prompts, completions, and embeddings to span attributes. This gives you a clear visibility into how your LLM application is working, and can make it easy to debug and evaluate the quality of the outputs."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# enable langchain instrumentation\n",
|
||||
"from opentelemetry.instrumentation.langchain import LangchainInstrumentor\n",
|
||||
"\n",
|
||||
"instrumentor = LangchainInstrumentor()\n",
|
||||
"if not instrumentor.is_instrumented_by_opentelemetry:\n",
|
||||
" instrumentor.instrument()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Run a simple LangChain\n",
|
||||
"\n",
|
||||
"Below is an example targeting an AzureOpenAI resource. Please configure you `API_KEY` using an `.env` file, see `../.env.example`."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"\n",
|
||||
"from langchain.chat_models import AzureChatOpenAI\n",
|
||||
"from langchain.prompts.chat import ChatPromptTemplate\n",
|
||||
"from langchain.chains import LLMChain\n",
|
||||
"from dotenv import load_dotenv\n",
|
||||
"\n",
|
||||
"if \"AZURE_OPENAI_API_KEY\" not in os.environ:\n",
|
||||
" # load environment variables from .env file\n",
|
||||
" load_dotenv()\n",
|
||||
"\n",
|
||||
"llm = AzureChatOpenAI(\n",
|
||||
" deployment_name=os.environ[\"CHAT_DEPLOYMENT_NAME\"],\n",
|
||||
" openai_api_key=os.environ[\"AZURE_OPENAI_API_KEY\"],\n",
|
||||
" azure_endpoint=os.environ[\"AZURE_OPENAI_ENDPOINT\"],\n",
|
||||
" openai_api_type=\"azure\",\n",
|
||||
" openai_api_version=\"2023-07-01-preview\",\n",
|
||||
" temperature=0,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"prompt = ChatPromptTemplate.from_messages(\n",
|
||||
" [\n",
|
||||
" (\"system\", \"You are world class technical documentation writer.\"),\n",
|
||||
" (\"user\", \"{input}\"),\n",
|
||||
" ]\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"chain = LLMChain(llm=llm, prompt=prompt, output_key=\"metrics\")\n",
|
||||
"chain({\"input\": \"What is ChatGPT?\"})"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"You should be able to see traces of the chain in promptflow UI now. Check the cell with `start_trace` on the trace UI url."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Next steps\n",
|
||||
"\n",
|
||||
"By now you've successfully tracing LLM calls in your app using prompt flow.\n",
|
||||
"\n",
|
||||
"You can check out more examples:\n",
|
||||
"- [Trace your flow](https://github.com/microsoft/promptflow/blob/main/examples/flex-flows/basic/flex-flow-quickstart.ipynb): using promptflow @trace to structurally tracing your app and do evaluation on it with batch run."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"build_doc": {
|
||||
"author": [
|
||||
"zhengfeiwang@github.com",
|
||||
"wangchao1230@github.com"
|
||||
],
|
||||
"category": "local",
|
||||
"section": "Tracing",
|
||||
"weight": 30
|
||||
},
|
||||
"description": "Tracing LLM calls in langchain application",
|
||||
"kernelspec": {
|
||||
"display_name": "prompt_flow",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.9.17"
|
||||
},
|
||||
"resources": ""
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
Reference in New Issue
Block a user