confident-ai--deepeval/docs/content/tutorials/medical-chatbot/development.mdx

---
id: development
title: Building Your Chatbot
sidebar_label: Building Your Chatbot
---

In this section, we are going to create a **multi-turn** chatbot that can use various tools to diagnose and schedule appointments for users based on their symptoms.
We will be using `langchain` and `qdrant` to build our chatbot, with functionalies including a:

- **RAG pipeline** to retrieve medical knowledge to diagnose patients
- **Custom tools** to create new appointments based on patient symptoms
- **Memory system** to keep track of chat histories

We'll also implement our chatbot with an independent **model and system prompt** variable - which we'll be evaluating in the next section.

:::tip
If you already have a multi-turn chatbot that you want to evaluate, feel free to skip to the [**evaluation section of this tutorial**](/tutorials/medical-chatbot/evaluation).
:::

## Setup Your Model

First create a `MedicalChatbot` class and use `langchain`'s chat models to call `OpenAI`:

```python title="main.py"
from langchain_openai import ChatOpenAI

class MedicalChatbot:
    def __init__(self, model: str):
        self.model = ChatOpenAI(model=model)
        # Choose the LLM that will drive the agent
        # Only certain models support this so ensure your model supports it as well
```

:::note
You can also use other interfaces to call OpenAI, or any other model.
:::

Try prompting it with a messages array:

```python title="main.py"
chatbot = MedicalChatbot(model="gpt-4o-mini")
chatbot.model.invoke([{"user": "Hi!"}])
```

Which should let you see something like this:

```text
AIMessage(
    content="Hey, how can I help you today?",
    additional_kwargs={},
    response_metadata={
        'prompt_feedback': {'block_reason': 0, 'safety_ratings': []},
        'finish_reason': 'STOP',
        'model_name': 'gpt-4o-mini',
        'safety_ratings': []
    },
    id='run--c2786aa1-75c4-4644-ae59-9327a2e8c153-0',
    usage_metadata={'input_tokens': 23, 'output_tokens': 417, 'total_tokens': 440, 'input_token_details': {'cache_read': 0}}
)
```

✅ Done. Now let's create some tools for the chatbot to start booking appointments.

## Create RAG Pipeline For Diagnosis

Since OpenAI models weren't specifically trained on medical knowledge, we'll need to leverage RAG to provide additional context at runtime to diagnose patients that are grounded in context.

:::info
We'll be using a text version of [The Gale Encyclopedia of Alternative Medicine](https://dl.icdst.org/pdfs/files/03cb46934164321f675385fb74ac1bed.pdf) as our knowledge base in this example. You will need to download it locally and convert it to a `.txt` file.
:::

### Index medical knowledge

We'll ingest "The Gale Encyclopedia of Alternative Medicine" to Qdrant, a popular vector database choice for fast and accurate retrievals:

```python title="main.py"
from qdrant_client import models, QdrantClient
from sentence_transformers import SentenceTransformer
from langchain_openai import ChatOpenAI

class MedicalChatbot:
    def __init__(self, model: str):
        self.model = ChatOpenAI(model=model)
        # For RAG engine
        self.encoder = SentenceTransformer("all-MiniLM-L6-v2")
        self.client = QdrantClient(":memory:")

    def index_knowledge(self, document_path: str):
        with open(document_path) as file:
            documents = file.readlines()

        # Create namespace in qdrant
        self.client.create_collection(
            collection_name="gale_encyclopedia",
            vectors_config=models.VectorParams(size=self.encoder.get_sentence_embedding_dimension(), distance=models.Distance.COSINE),
        )

        # Vectorize and index into qdrant
        self.client.upload_points(
            collection_name="gale_encyclopedia",
            points=[models.PointStruct(id=idx, vector=self.encoder.encode(doc).tolist(), payload={"content": doc}) for idx, doc in enumerate(documents)],
        )
```

Then, simply run your `index_knowledge` method using the encyclopedia you've downloaded as `.txt`:

```python title="main.py"
chatbot = MedicalChatbot()
chatbot.index_knowledge("path-to-your-encyclopedia.txt")
```

✅ Done. Now let's try querying it to sanity check yourself.

:::note
You only have the run `index_knowledge` once.
:::

### Query your knowledge base

Simply implement a **TOOL** to query from qdrant. in this case `retrieve_knowledge`:

```python title="main.py" {14}
from qdrant_client import models, QdrantClient
from sentence_transformers import SentenceTransformer
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool

class MedicalChatbot:
    def __init__(self, model: str):
        self.model = ChatOpenAI(model=model)
        # For RAG engine
        self.encoder = SentenceTransformer("all-MiniLM-L6-v2")
        self.client = QdrantClient(":memory:")

    @tool
    def retrieve_knowledge(self, query: str) -> str:
        """"A tool to retrieve data on various diagnosis methods from gale encyclopedia"""
        hits = self.client.query_points(collection_name="gale_encyclopedia", query=self.encoder.encode(query).tolist(), limit=3).points

        contexts = [hit.payload['content'] for hit in hits]
        return "\n".join(contexts)

    def index_knowledge(self, document_path: str):
        # Same as above
        pass
```

:::info
The `@tool` decorator tells `langchain` that the `retrieve_knowledge` method can be called as a function call and will come in handy in later sections.
:::

Now try calling it:

```python title="main.py"
chatbot = MedicalChatbot()
chatbot.retrieve_knowledge("Cough, fever, and diarrhea.")
```

Great! Now that we have the essentials for making a diagnosis, time to move on to implementing a way to book appointments after a diagnosis.

## Create Tool To Book Appointments

Since we need a way for our chatbot to book appointments based on the diagnosis at hand, this section will focus on creating the tools required to do so. There's only one tool for booking appointments for the sake of simplicity:

- `create_appointment`: Creates a new appointment **in memory** (you can also use something like SQLite for persistent storage)

First, let's create a simple data model for appointments:

```python title="main.py"
from pydantic import BaseModel, Field
from typing import Optional, List
from datetime import date

class Appointment(BaseModel):
    id: str
    name: str
    email: str
    date: date
    symptoms: Optional[List[str]] = Field(default=None)
    diagnosis: Optional[str] = Field(default=None)
```

Now let's implement the `create_appointment` tool:

```python title="main.py" {14}
import uuid

...

class MedicalChatbot:
    def __init__(self, model: str):
        self.model = ChatOpenAI(model=model)
        # For RAG engine
        self.encoder = SentenceTransformer("all-MiniLM-L6-v2")
        self.client = QdrantClient(":memory:")
        # For managing appointments
        self.appointments: List[Appointment] = []

    @tool
    def create_appointment(self, name: str, email: str, date: str) -> str:
        """Create a new appointment with the given ID, name, email, and date"""
        try:
            appointment = Appointment(
                id=str(uuid.uuid4()),
                name=name,
                email=email,
                date=date.fromisoformat(date)
            )
            self.appointments.append(appointment)
            return f"Created new appointment with ID: {appointment.id} for {name} on {date}."
        except ValueError:
            return f"Invalid date format. Please use YYYY-MM-DD format."

    @tool
    def retrieve_knowledge(self, query: str) -> str:
        # Same as above
        pass

    def index_knowledge(self, document_path: str):
        # Same as above
        pass
```

Great! Now let's glue everything together using LangChain.

## Implementing Chat Histories

First create a helper method that retrieves conversation histories, which would be required for our LLM:

```python title"main.py"
from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.chat_history import BaseChatMessageHistory

# Simple in-memory store for chat histories
chat_store = {}
def get_session_history(session_id: str) -> BaseChatMessageHistory:
    if session_id not in chat_store:
        chat_store[session_id] = ChatMessageHistory()
    return chat_store[session_id]
```

Then we'll combine the agent setup and memory functionality into one clean implementation, including the `retrieve_knowledge` and `create_appointment` tools in our agent:

```python title="main.py" {20,28-29,33}
from langchain.agents import create_tool_calling_agent, AgentExecutor
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_core.tools import StructuredTool
...

class MedicalChatbot:
    def __init__(self, model: str, system_prompt: str):
        self.model = ChatOpenAI(model=model)
        self.system_prompt = system_prompt
        # For RAG engine
        self.encoder = SentenceTransformer("all-MiniLM-L6-v2")
        self.client = QdrantClient(":memory:")
        # For managing appointments
        self.appointments: List[Appointment] = []

        # Setup agent with memory
        self.setup_agent()

    def setup_agent(self):
        """Setup the agent with tools and memory"""

        # Create prompt messages
        prompt = ChatPromptTemplate.from_messages([("system", self.system_prompt), MessagesPlaceholder(variable_name="chat_history"), ("human", "{input}")])

        # Create agent
        tools = [
            StructuredTool.from_function(func=self.retrieve_knowledge),
            StructuredTool.from_function(func=self.create_appointment)
        ]
        agent = create_tool_calling_agent(self.model, tools, prompt)
        agent_executor = AgentExecutor(agent=agent, tools=tools)
        self.agent_with_memory = RunnableWithMessageHistory(
            agent_executor,
            get_session_history,
            input_messages_key="input",
            history_messages_key="chat_history",
        )

    # Other methods from above goes here
    ...
```

🎉🥳 Congratulations! You've just created a fully functional medical chatbot with memory, the ability to diagnose users, and book appointments when needed.

## Eyeball Your First Output

Now that you have your chatbot, its time to query it to see if it lives up to your expectations. Create a method so you can interact with it in the CLI, and **supply your model and choice and system prompt**:

```python title="main.py" {22,23,28}
def start_session(session_id: Optional[str] = None):
    """Start an interactive session with the chatbot"""
    print("Hello! I am Baymax, your personal healthcare companion.")
    print("How are you feeling today? (type 'exit' to quit."))

    while True:
        if session_id is None:
            session_id = str(uuid.uuid4())

        user_input = input("Your query: ")
        if user_input.lower() == 'exit':
            break

        response = chatbot.agent_with_memory.invoke(
            {"input": user_input},
            config={"configurable": {"session_id": session_id}}
        )
        print("Baymax:", response["output"])


# These parameters will be evaluated later
MODEL = "gpt-4o-mini"
SYSTEM_PROMPT = """"""

# Initialize chatbot and start session
chatbot = MedicalChatbot(model=MODEL, system_prompt=SYSTEM_PROMPT)
chatbot.index_knowledge("path-to-your-encyclopedia.txt")
start_session()
```

:::tip
The model and system prompt are the variables you'll want to be improving in the next section.
:::

Then simply execute the file in the CLI:

```bash
python main.py
```

Here's an example conversation:

```text
Hello! I am Baymax, your personal healthcare companion.
How are you feeling today? (type 'exit' to quit.")

Your query: Hello Baymax, I've been feeling unwell. I have a fever and a sore throat.
Baymax: I'm sorry to hear you're not feeling well. A fever and sore throat are common symptoms of upper respiratory infections. Based on medical knowledge, rest, fluids, and over-the-counter pain relievers can help. If your symptoms are severe or persistent, it's best to consult a doctor. Would you like me to help you schedule an appointment?

Your query: Yes, please book an appointment for tomorrow at 2 PM.
Baymax: I'll help you create an appointment. I need your name and email to proceed. Could you please provide those details?

Your query: My name is John Doe and my email is john@example.com
Baymax: Created new appointment with ID: 550e8400-e29b-41d4-a716-446655440000 for John Doe on 2024-01-16.
```

Was this what you really wanted? Is this diagnosis a good one? Was the appointment booked appropriately? The process of making these judgements yourself are known as **eyeballing** LLM outputs. It works, but isn't very scalable nor reliable - especially when conversations get long and you find yourself skimming instead of evaluating.