wehub-resource-sync
440 lines
13 KiB
Plaintext
440 lines
13 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "vwELCooy4ljr"
|
|
},
|
|
"source": [
|
|
"# Prompt templates and task chains\n",
|
|
"\n",
|
|
"txtai has long had support for workflows. Workflows connect the input and outputs of machine learning models together to create powerful transformation and processing functions.\n",
|
|
"\n",
|
|
"There has been a recent surge in interest in \"model prompting\", which is the process of building a natural language description of a task and passing it to a large language model (LLM). txtai has recently improved support for task templating, which builds string outputs from a set of parameters.\n",
|
|
"\n",
|
|
"This notebook demonstrates how txtai workflows can be used to apply prompt templates and chain those tasks together."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "ew7orE2O441o"
|
|
},
|
|
"source": [
|
|
"# Install dependencies\n",
|
|
"\n",
|
|
"Install `txtai` and all dependencies."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "LPQTb25tASIG"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%capture\n",
|
|
"!pip install git+https://github.com/neuml/txtai#egg=txtai[api]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "_YnqorRKAbLu"
|
|
},
|
|
"source": [
|
|
"# Prompt workflow\n",
|
|
"\n",
|
|
"First, we'll look at building a workflow with a series of model prompts. This workflow creates a conditional translation using a statement and target language. Another task reads that output text and detects the language.\n",
|
|
"\n",
|
|
"This workflow uses a LLM pipeline. The LLM pipeline loads a local model for inference, in this case [Qwen3-4B-Instruct-2507](https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507). The [LLM pipeline](https://neuml.github.io/txtai/pipeline/llm/llm) supports local transformers models, llama.cpp models and LLM APIs such as Ollama, vLLM, OpenAI, Claude etc. \n",
|
|
"\n",
|
|
"It's important to note that a pipeline is simply a callable function. It can easily be replaced with a call to an external API."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"id": "OUc9gqTyAYnm",
|
|
"outputId": "83300311-736c-47c8-bc16-ec0303274054"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"['French', 'German']\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from txtai import LLM, Workflow\n",
|
|
"from txtai.workflow import TemplateTask\n",
|
|
"\n",
|
|
"# Create LLM\n",
|
|
"llm = LLM(\"Qwen/Qwen3-4B-Instruct-2507\")\n",
|
|
"\n",
|
|
"# Define workflow or chaining of tasks together.\n",
|
|
"workflow = Workflow([\n",
|
|
" TemplateTask(\n",
|
|
" template=\"Translate text '{statement}' to {language} if the text is English, otherwise keep the original text\",\n",
|
|
" action=llm\n",
|
|
" ),\n",
|
|
" TemplateTask(\n",
|
|
" template=\"What language is the following text. Only print the answer? {text}\",\n",
|
|
" action=llm\n",
|
|
" )\n",
|
|
"])\n",
|
|
"\n",
|
|
"inputs = [\n",
|
|
" {\"statement\": \"Hello, how are you\", \"language\": \"French\"},\n",
|
|
" {\"statement\": \"Hallo, wie geht's dir\", \"language\": \"French\"}\n",
|
|
"]\n",
|
|
"\n",
|
|
"print(list(workflow(inputs)))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "_zz4Do8BV-Lk"
|
|
},
|
|
"source": [
|
|
"Let's recap what happened here. The first workflow task conditionally translates text to a language if it's English.\n",
|
|
"\n",
|
|
"The first statement is `Hello, how are you` with a target language of French. So the statement is translated to French.\n",
|
|
"\n",
|
|
"The second statement is German, so it's not converted to French.\n",
|
|
"\n",
|
|
"The next step asks the model what the language is and it correctly prints `French` and `German`."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "iXDAKP4CX0W9"
|
|
},
|
|
"source": [
|
|
"# Prompt Workflow as YAML\n",
|
|
"\n",
|
|
"The same workflow above can be created with YAML configuration."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"id": "GwV5A9xRYtYs",
|
|
"outputId": "ffe6ee65-95a7-46c6-e6b9-5324eab26ca8"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Writing workflow.yml\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"%%writefile workflow.yml\n",
|
|
"\n",
|
|
"llm:\n",
|
|
" path: Qwen/Qwen3-4B-Instruct-2507\n",
|
|
"\n",
|
|
"workflow:\n",
|
|
" chain:\n",
|
|
" tasks:\n",
|
|
" - task: template\n",
|
|
" template: Translate text '{statement}' to {language} if the text is English, otherwise keep the original text\n",
|
|
" action: llm\n",
|
|
" - task: template\n",
|
|
" template: What language is the following text. Only print the answer? {text}\n",
|
|
" action: llm"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"id": "dr7Lv5S5X98e",
|
|
"outputId": "d6ac0427-671d-4525-aa21-664430109af3"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"['French', 'German']\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from txtai import Application\n",
|
|
"\n",
|
|
"app = Application(\"workflow.yml\")\n",
|
|
"print(list(app.workflow(\"chain\", inputs)))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "EGqiV45fYVse"
|
|
},
|
|
"source": [
|
|
"As expected, the same result! This is a matter of preference on how you want to create a workflow. One advantage of YAML workflows is that an API can easily be created from the workflow file."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "9PqMU0bNYinf"
|
|
},
|
|
"source": [
|
|
"# Prompt Workflow via an API call\n",
|
|
"\n",
|
|
"Let's say you want the workflow to be available via an API call. Well good news, txtai has a built in API mechanism using FastAPI. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"id": "vDxQj1ZIYsz3"
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Start an API service\n",
|
|
"!CONFIG=workflow.yml nohup uvicorn \"txtai.api:app\" &> api.log &\n",
|
|
"!sleep 60"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"id": "R1o08SVtZW7h",
|
|
"outputId": "99875acd-18a8-4c2c-ead3-cb6975a4b2d2"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"['French', 'German']"
|
|
]
|
|
},
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"import requests\n",
|
|
"\n",
|
|
"# Run API request\n",
|
|
"requests.post(\"http://localhost:8000/workflow\", json={\"name\": \"chain\", \"elements\": inputs}).json()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "B88mCrGFl5W-"
|
|
},
|
|
"source": [
|
|
"Just like the previous steps, except through an API call. Let's run via cURL for good measure."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"id": "hRUyh0cQl_P2",
|
|
"outputId": "9db8481d-0b6e-4a31-bdf6-5443df5f768a"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[\"French\",\"German\"]"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"%%bash\n",
|
|
"\n",
|
|
"curl -s -X POST \"http://localhost:8000/workflow\" \\\n",
|
|
" -H \"Content-Type: application/json\" \\\n",
|
|
" --data @- << EOF\n",
|
|
"{\n",
|
|
" \"name\": \"chain\",\n",
|
|
" \"elements\": [\n",
|
|
" {\"statement\": \"Hello, how are you\", \"language\": \"French\"},\n",
|
|
" {\"statement\": \"Hallo, wie geht's dir\", \"language\": \"French\"}\n",
|
|
" ]\n",
|
|
"}\n",
|
|
"EOF"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "W0zL93WPoaCo"
|
|
},
|
|
"source": [
|
|
"One last time, the same output is shown.\n",
|
|
"\n",
|
|
"If your primary development environment isn't Python, txtai does have API bindings for [JavaScript](https://github.com/neuml/txtai.js), [Rust](https://github.com/neuml/txtai.rs), [Go](https://github.com/neuml/txtai.go) and [Java](https://github.com/neuml/txtai.java).\n",
|
|
"\n",
|
|
"More information on the API is available [here](https://neuml.github.io/txtai/api/)."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "q9WiFG6fpzw5"
|
|
},
|
|
"source": [
|
|
"# Chat with your data\n",
|
|
"\n",
|
|
"\"Chat with your data\" is a popular entry point into the AI space. Let's run an example."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"id": "rM3Y551LqF-J",
|
|
"outputId": "85623785-c15f-4996-9460-0644f69cf5bf"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"Writing search.yml\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"%%writefile search.yml\n",
|
|
"\n",
|
|
"writable: false\n",
|
|
"cloud:\n",
|
|
" provider: huggingface-hub\n",
|
|
" container: neuml/txtai-intro\n",
|
|
"\n",
|
|
"rag:\n",
|
|
" path: Qwen/Qwen3-4B-Instruct-2507\n",
|
|
" output: reference\n",
|
|
" template: |\n",
|
|
" Answer the following question using only the context below.\n",
|
|
"\n",
|
|
" Question: {question}\n",
|
|
" Context: {context}\n",
|
|
"\n",
|
|
"workflow:\n",
|
|
" search:\n",
|
|
" tasks:\n",
|
|
" - action: rag\n",
|
|
" - task: template\n",
|
|
" template: \"{answer}\\n\\nReference: {reference}\"\n",
|
|
" rules:\n",
|
|
" answer: I don't have data on that"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"colab": {
|
|
"base_uri": "https://localhost:8080/"
|
|
},
|
|
"id": "1Elb8JANqpwX",
|
|
"outputId": "b1f1ffa1-6c47-4d90-b6f1-8098d4dc45f8"
|
|
},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[\"Canada's last fully intact ice shelf has suddenly collapsed, forming a Manhattan-sized iceberg.\\n\\nReference: 1\"]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"app = Application(\"search.yml\")\n",
|
|
"print(list(app.workflow(\"search\", [\"Find something about North America\"])))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "4r49V4c9s5nf"
|
|
},
|
|
"source": [
|
|
"The first thing the code above does is run an embeddings search to build a conversational context. That context is then used to build a prompt and inference is run against the LLM. \n",
|
|
"\n",
|
|
"The next task formats the outputs with a reference to the best matching record. In this case, it's only an id of 1. But this can be much more useful if the id is a URL or there is logic to format the id back to a unique reference string."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {
|
|
"id": "KqfvCXp2B3li"
|
|
},
|
|
"source": [
|
|
"# Wrapping up\n",
|
|
"\n",
|
|
"This notebook covered how to build prompt templates and task chains through a series of results. txtai has long had a robust and efficient workflow framework for connecting models together. This can be small and simple models and/or prompting with large models. Go ahead and give it a try!"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"accelerator": "GPU",
|
|
"colab": {
|
|
"provenance": []
|
|
},
|
|
"gpuClass": "standard",
|
|
"kernelspec": {
|
|
"display_name": "local",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.19"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 0
|
|
}
|