portkey-ai--gateway/cookbook/guardrails/Langchain Chatbot with PII Guardrails.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1YnDHZY6owEypgSgvJJgXPcntxOjy9GpS?usp=sharing)\n"
      ],
      "metadata": {
        "id": "L81TWFYhHM-K"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Build a Chatbot with Guardrails: Using Langchain and Portkey to Enforce PII Detection and more\n",
        "\n",
        "In this tutorial, we'll create two versions of a customer support chatbot:\n",
        "\n",
        "1. A basic chatbot without PII protection\n",
        "2. An enhanced chatbot using LangChain and Portkey guardrails to protect sensitive information\n",
        "\n",
        "By comparing these two implementations, we'll demonstrate the importance and effectiveness of using guardrails to prevent PII exposure in AI applications.\n",
        "\n",
        "\n",
        "Portkey's **Guardrails** offer real-time enforcement of LLM behavior, with features including:\n",
        "\n",
        "- **Regex matching** and **JSON Schema validation**\n",
        "- **Code detection**\n",
        "- **Custom guardrail**s: Integrate your existing guardrail systems custom guardrail integration\n",
        "-**LLM-based guardrails** (e.g., gibberish detection, prompt injection scanning)\n",
        "\n",
        "With **20+ deterministic guardrail**s and integrations with platforms like **Aporia, Pillar, Patronus AI and Prompt Security,** Portkey provides comprehensive AI safety solutions. Guardrails can be configured for inputs, outputs, or both, with actions ranging from request denial to alternative LLM fallbacks.\n",
        "\n",
        "For more details, visit the [Portkey Guardrails documentation](https://docs.portkey.ai/docs/product/guardrails/list-of-guardrail-checks/patronus-ai).\n",
        "\n",
        "\n",
        "Let's build our guardrail-protected chatbot!"
      ],
      "metadata": {
        "id": "cWj7GyUW76uP"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Chatbot Without Guardrails\n",
        "\n",
        "Before we implement PII protection with Portkey guardrails, let's create a basic chatbot using LangChain. This version will not have any PII detection or protection mechanisms.\n",
        "\n",
        "## Step 1: Setting up the environment\n",
        "\n",
        "First, let's install the necessary packages:"
      ],
      "metadata": {
        "id": "WYXKs8sQGy3H"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install -qU langchain-openai langchain langchain_community"
      ],
      "metadata": {
        "id": "P-SvF2AKGyXU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#Step 2: Importing required libraries and setting up API keys"
      ],
      "metadata": {
        "id": "Etxbrc-CG42W"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "from typing import Dict\n",
        "from langchain_openai import ChatOpenAI\n",
        "from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder\n",
        "from langchain_core.chat_history import BaseChatMessageHistory\n",
        "from langchain_core.runnables.history import RunnableWithMessageHistory\n",
        "from langchain_core.chat_history import InMemoryChatMessageHistory\n",
        "\n",
        "\n",
        "model = ChatOpenAI(\n",
        "    api_key=\"YOUR_OPENAI_KEY\",\n",
        "    model=\"gpt-3.5-turbo\"\n",
        ")"
      ],
      "metadata": {
        "id": "sAf52HvBG6nS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a chat prompt template\n",
        "prompt = ChatPromptTemplate.from_messages([\n",
        "    (\"system\", \"You are a helpful assistant. Answer all questions to the best of your ability.\"),\n",
        "    MessagesPlaceholder(variable_name=\"messages\"),\n",
        "])\n",
        "\n",
        "# Create a chain that combines the prompt and the model\n",
        "chain = prompt | model\n",
        "\n",
        "# Create a dictionary to store chat histories for different sessions\n",
        "store: Dict[str, BaseChatMessageHistory] = {}\n",
        "\n",
        "def get_session_history(session_id: str) -> BaseChatMessageHistory:\n",
        "    if session_id not in store:\n",
        "        store[session_id] = InMemoryChatMessageHistory()\n",
        "    return store[session_id]\n",
        "\n",
        "# Wrap the chain with message history\n",
        "with_message_history = RunnableWithMessageHistory(\n",
        "    chain,\n",
        "    get_session_history,\n",
        "    input_messages_key=\"messages\",\n",
        ")\n",
        "\n",
        "# Function to chat with the bot\n",
        "def chat_with_bot(session_id: str, user_input: str):\n",
        "    config = {\"configurable\": {\"session_id\": session_id}}\n",
        "    response1 = with_message_history.invoke(\n",
        "        {\"messages\": [{\"content\": user_input, \"type\": \"human\"}]},\n",
        "        config=config\n",
        "    )\n",
        "    return response1.content"
      ],
      "metadata": {
        "id": "_cA4eFpyG8st"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Step 4: Running the chatbot"
      ],
      "metadata": {
        "id": "Cc3A6g9fHBlP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Example usage\n",
        "if __name__ == \"__main__\":\n",
        "    session_id = \"abc123\"\n",
        "\n",
        "    print(\"Chatbot Without Guardrails\")\n",
        "    print(\"Type 'exit', 'quit', or 'bye' to end the conversation.\")\n",
        "\n",
        "    while True:\n",
        "        user_input1 = input(\"You: \")\n",
        "        if user_input1.lower() in ['exit', 'quit', 'bye']:\n",
        "            print(\"Bot: Goodbye!\")\n",
        "            break\n",
        "\n",
        "        response1 = chat_with_bot(session_id, user_input1)\n",
        "        print(f\"Bot: {response1}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VrQjj6SFHAz_",
        "outputId": "04876806-525a-4d4e-c10f-cb325b3efb1e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Chatbot Without Guardrails\n",
            "Type 'exit', 'quit', or 'bye' to end the conversation.\n",
            "You: hey\n",
            "Bot: Hello! How can I assist you today?\n",
            "You: \"My name is Siddharth and my email id is xyz@gmail.com\n",
            "Bot: Hello Siddharth! It's nice to meet you. How can I help you today?\n",
            "You: what is my email\n",
            "Bot: Your email is xyz@gmail.com.\n",
            "You: exit\n",
            "Bot: Goodbye!\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Build a Chatbot with Guardrails: Using Langchain and Portkey to Enforce PII Detection and more\n",
        "\n",
        "In this tutorial, we'll create two versions of a customer support chatbot:\n",
        "\n",
        "1. A basic chatbot without PII protection\n",
        "2. An enhanced chatbot using LangChain and Portkey guardrails to protect sensitive information\n",
        "\n",
        "By comparing these two implementations, we'll demonstrate the importance and effectiveness of using guardrails to prevent PII exposure in AI applications.\n"
      ],
      "metadata": {
        "id": "4BJczP3_HFuN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Setting Up Portkey Guardrails\n",
        "\n",
        "Before building our chatbot, let's configure Portkey guardrails for PII detection:\n",
        "\n",
        "1. Sign Up for [portkey.ai](https://portkey.ai) (the dev account is **free** ferver) 🎊\n",
        "\n",
        "\n",
        "2. Enable **Patronus AI** in the [Integrations page](https://app.portkey.ai/integrations) on Portkey App\n",
        "\n",
        "\n",
        "3. Navigate to the [Guardrails dashboard](https://app.portkey.ai/guardrails)\n",
        "\n",
        "\n",
        "4. Create a new guardrail, by selecting \"**Detect PII Guardrail**\" by Patronus AI\n",
        "\n",
        "\n",
        "5. In the Actions tab, select \"**Deny the request if guardrail fails**\" from the Settings menu.\n",
        "\n",
        "6. Name your guardrail, create it, and then copy your Config ID for future use.\n",
        "\n",
        "\n",
        "\n",
        "This **Config ID** is crucial for integrating PII protection into our LangChain-Portkey  chatbot. With this setup, we're ready to create a secure, PII-aware conversational AI.\n",
        "\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/siddharthsambharia-portkey/Portkey-Product-Images/main/Portkey_PII_Guardrails.png\" alt=\"O\" width=\"600\" />\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/siddharthsambharia-portkey/Portkey-Product-Images/main/actions-gaurdrails.png\" alt=\"O\" width=\"600\" />\n",
        "\n"
      ],
      "metadata": {
        "id": "RhXlcYb5NFqj"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Step 1: Setting up the environment\n",
        "\n",
        "First, let's install the necessary packages:"
      ],
      "metadata": {
        "id": "_x0jcajG76AY"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "kXamAIlNgU7J",
        "collapsed": true
      },
      "outputs": [],
      "source": [
        "!pip install -qU langchain-openai portkey-ai langchain langchain_community"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Step 2: Importing required libraries and setting up API keys\n",
        "\n",
        "In this step, we'll import the necessary libraries and set up our API keys. We'll also create Portkey config object to add guardrails to protect against PII exposure. Get your Portkey API key from [here](https://app.portkey.ai/api-keys)."
      ],
      "metadata": {
        "id": "E943mJcZG7SX"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VZShDZzRr-yo",
        "collapsed": true
      },
      "outputs": [],
      "source": [
        "import os\n",
        "from typing import Dict\n",
        "from langchain_openai import ChatOpenAI\n",
        "from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder\n",
        "from langchain_core.chat_history import BaseChatMessageHistory\n",
        "from langchain_core.runnables.history import RunnableWithMessageHistory\n",
        "from portkey_ai import createHeaders, PORTKEY_GATEWAY_URL\n",
        "from langchain_core.chat_history import InMemoryChatMessageHistory\n",
        "\n",
        "\n",
        "portkey_config={\n",
        "\t\"after_request_hooks\": [\n",
        "\t\t{\n",
        "\t\t\t\"id\": \"pg-portke-416feb\" #YOUR_CONFIG_ID\n",
        "\t\t}\n",
        "\t]\n",
        "}\n",
        "\n",
        "\n",
        "llm = ChatOpenAI(\n",
        "    api_key=\"YOUR_OPENAI_API_KEY\",\n",
        "    base_url=PORTKEY_GATEWAY_URL,\n",
        "    model=\"gpt-3.5-turbo\",\n",
        "    default_headers=createHeaders(\n",
        "        provider=\"openai\",\n",
        "        api_key=\"Your_Portkey_API_key\",\n",
        "        config=portkey_config,\n",
        "    )\n",
        ")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This code sets up our environment with the necessary configurations for Portkey guardrails. The portkey_config defines our guardrail settings, including retry attempts, caching, and hooks for request and response processing.\n",
        "\n"
      ],
      "metadata": {
        "id": "IeIvEQiIG9y-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Step 3: Creating the chat prompt template and chain\n",
        "\n",
        "Now, let's set up our chat prompt template and create a chain that combines the prompt with our Portkey-protected model. This structure allows us to maintain a conversation history while applying guardrails to each interaction."
      ],
      "metadata": {
        "id": "TdMZ3Mx378rd"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "# Create a chat prompt template\n",
        "prompt = ChatPromptTemplate.from_messages([\n",
        "    (\"system\", \"You are a helpful assistant. Answer all questions to the best of your ability. Reply with the user info given in the request\"),\n",
        "    MessagesPlaceholder(variable_name=\"messages\"),\n",
        "])\n",
        "\n",
        "# Create a chain that combines the prompt and the model\n",
        "chain1 = prompt | llm\n",
        "\n",
        "# Create a dictionary to store chat histories for different sessions\n",
        "store: Dict[str, BaseChatMessageHistory] = {}\n",
        "\n",
        "def get_session_history_1(session_id: str) -> BaseChatMessageHistory:\n",
        "    if session_id not in store:\n",
        "        store[session_id] = InMemoryChatMessageHistory()\n",
        "    return store[session_id]\n",
        "\n",
        "# Wrap the chain with message history\n",
        "with_message_history_1 = RunnableWithMessageHistory(\n",
        "    chain1,\n",
        "    get_session_history_1,\n",
        "    input_messages_key=\"messages\",\n",
        ")\n",
        "\n",
        "# Function to chat with the bot\n",
        "def chat_with_bot1(session_id: str, user_input: str):\n",
        "    config = {\"configurable\": {\"session_id\": session_id}}\n",
        "    response = with_message_history_1.invoke(\n",
        "        {\"messages\": [{\"content\": user_input, \"type\": \"human\"}]},\n",
        "        config=config\n",
        "    )\n",
        "    return response.content\n",
        "\n"
      ],
      "metadata": {
        "id": "oL9gBUBu7iLL"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "This code creates a chat prompt template, sets up a chain combining the prompt and the Portkey-protected model, and establishes a system for managing chat history. The chat_with_bot function handles individual interactions, applying our guardrails to each message.\n"
      ],
      "metadata": {
        "id": "BdxJenOI79hs"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Step 4: Running the chatbot\n",
        "Finally, we'll implement the main loop for our chatbot. This will allow users to interact with the bot while our Portkey guardrails work in the background to protect sensitive information.\n",
        "\n",
        "Try using this prompt to see guardrails in action- \"My name is Siddharth and my email id is xyz@gmail.com\""
      ],
      "metadata": {
        "id": "D02MN0oLHE2x"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Example usage\n",
        "if __name__ == \"__main__\":\n",
        "    session_id = \"testWithGuradrails\"\n",
        "\n",
        "    while True:\n",
        "        user_input = input(\"You: \")\n",
        "        if user_input.lower() in ['exit', 'quit', 'bye']:\n",
        "            print(\"Bot: Goodbye!\")\n",
        "            break\n",
        "\n",
        "        response = chat_with_bot1(session_id, user_input)\n",
        "        print(f\"Bot: {response}\")"
      ],
      "metadata": {
        "collapsed": true,
        "id": "Z2w0DB0b7j6g"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Conclusion\n",
        "\n",
        "Here is how the traces of your request looks like on your Portkey dashboard-\n",
        "\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/siddharthsambharia-portkey/Portkey-Product-Images/main/PII-fail-traces-guardrails.png\" alt=\"O\" width=\"600\" />\n",
        "\n",
        "Portkey helps you connect to 250+ LLMs with just 2 lines of code. It helps you build robust aps with built in observability and guardrails.\n",
        "\n",
        "In this tutorial, we've created two versions of a chatbot:\n",
        "\n",
        "1. A basic chatbot without PII protection\n",
        "2. An enhanced chatbot with Portkey guardrails\n",
        "\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/siddharthsambharia-portkey/Portkey-Product-Images/main/Portkey-Dashboard.png\" alt=\"O\" width=\"600\" />\n",
        "\n",
        "\n",
        "\n",
        "The first version demonstrates a simple implementation using LangChain, but it lacks any mechanism to detect or protect sensitive information. This could potentially lead to the exposure of PII in logs or responses.\n",
        "\n",
        "The second version, integrating Portkey with our LangChain-based chatbot, significantly enhances our AI application's capabilities, reliability, and security. By implementing PII detection guardrails, we've added a crucial layer of protection against the unintended exposure of sensitive information.\n",
        "\n",
        "By comparing these two implementations, we can clearly see the benefits of using Portkey guardrails in AI applications, especially when dealing with potentially sensitive user inputs."
      ],
      "metadata": {
        "id": "A3BOYCMwFoxF"
      }
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}