chore: import upstream snapshot with attribution
Rebuild Cookbook Website / deploy (push) Has been cancelled
Rebuild Cookbook Website / deploy (push) Has been cancelled
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,136 @@
|
||||
# LangChain, OpenAI, and Oracle AI Database for Vector Search
|
||||
|
||||
This example shows how to build a semantic search workflow using:
|
||||
|
||||
- **OpenAI embeddings** for turning text into vector representations
|
||||
- **LangChain's Oracle vector store integration** for working with vectors through a familiar framework abstraction
|
||||
- **Oracle AI Database Vector Search** for storing embeddings and retrieving relevant content with similarity search
|
||||
|
||||
If you are looking for a practical starting point for **LangChain + Oracle AI Database**, **OpenAI embeddings with Oracle Vector Search**, or a developer-friendly example of semantic retrieval that integrates directly with data stored in Oracle AI Database, this notebook is designed to be easy to read, run, and adapt.
|
||||
|
||||
## Why this stack works well
|
||||
|
||||
This combination is useful because each component solves a different part of the workflow:
|
||||
|
||||
- **OpenAI embeddings** convert text into semantic vectors that capture meaning beyond keyword matching
|
||||
- **LangChain** provides a developer-friendly abstraction for vector stores and retrieval workflows
|
||||
- **Oracle AI Database** stores vectors alongside application and relational data, making it easier to build retrieval-enabled applications without introducing a separate vector database
|
||||
|
||||
For developers evaluating semantic search patterns, this creates a practical setup that is easy to prototype locally and extend later into larger RAG or agent-based workflows.
|
||||
|
||||
## Why use Oracle AI Database with LangChain
|
||||
|
||||
Oracle AI Database is a strong fit for LangChain-based vector workflows when you want:
|
||||
|
||||
- **vector search and application data in one place**
|
||||
- **less infrastructure complexity**, especially for teams already using Oracle
|
||||
- **native SQL access to vector operations**
|
||||
- **a smoother path from notebook prototype to production architecture**
|
||||
- compatibility with **LangChain retrieval patterns**, while keeping Oracle AI Database as the retrieval backbone
|
||||
|
||||
This is especially helpful when vector search is not a standalone demo, but one part of a broader enterprise or application stack.
|
||||
|
||||
## What you will learn
|
||||
|
||||
By working through the notebook, you will learn how to:
|
||||
|
||||
- connect to Oracle AI Database from Python
|
||||
- configure OpenAI embeddings for semantic search
|
||||
- initialize LangChain's Oracle vector store integration
|
||||
- store sample documents and embeddings in Oracle AI Database
|
||||
- run vector similarity search against indexed content
|
||||
- inspect how LangChain retrieval maps to Oracle AI Database Vector Search
|
||||
- use the same pattern as a baseline for larger retrieval or RAG workflows
|
||||
|
||||
## Architecture at a glance
|
||||
|
||||
The example follows a simple semantic retrieval flow:
|
||||
|
||||
1. Load configuration and environment variables
|
||||
2. Initialize OpenAI embeddings
|
||||
3. Connect to Oracle AI Database
|
||||
4. Initialize the LangChain Oracle vector store
|
||||
5. Insert sample text documents
|
||||
6. Convert the query into an embedding
|
||||
7. Run vector similarity search
|
||||
8. Return the most relevant matching content
|
||||
|
||||
In a larger application, this same pattern can become the retrieval layer behind a RAG pipeline, internal search assistant, or AI agent workflow.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
You will need:
|
||||
|
||||
- **Oracle AI Database** with Vector Search enabled
|
||||
- **OpenAI API key**
|
||||
- **Python 3.10+**
|
||||
- Notebook dependencies installed for this example
|
||||
|
||||
## Environment variables
|
||||
|
||||
Create a local `.env` file and provide values similar to the following:
|
||||
|
||||
```env
|
||||
OPENAI_API_KEY=
|
||||
ORACLE_USER=PDBADMIN
|
||||
ORACLE_PASSWORD=YourPassword
|
||||
ORACLE_DSN=localhost:1521/FREEPDB1
|
||||
```
|
||||
|
||||
## Python dependencies
|
||||
Install the required packages using pip:
|
||||
|
||||
```bash
|
||||
pip install langchain
|
||||
pip install langchain-openai
|
||||
pip install langchain-oracledb
|
||||
pip install oracledb
|
||||
pip install python-dotenv
|
||||
```
|
||||
|
||||
|
||||
## Files in this example
|
||||
|
||||
- `oracle_vector_search_langchain.ipynb` - the main notebook demonstrating semantic search with LangChain, OpenAI, and Oracle AI Database
|
||||
- `.env` - local configuration file for OpenAI and Oracle connection settings
|
||||
|
||||
|
||||
|
||||
## How to run
|
||||
|
||||
1. Set your `OPENAI_API_KEY` in the local `.env` file
|
||||
2. Make sure your Oracle AI Database instance is running and the connection values are correct
|
||||
3. Open `oracle_vector_search_langchain.ipynb`
|
||||
4. Run the notebook cells in order
|
||||
|
||||
The notebook walks through dependency setup, embedding initialization, Oracle database connection, vector store creation, sample document ingestion, and semantic similarity search.
|
||||
|
||||
|
||||
## Expected outcome
|
||||
|
||||
By the end of this notebook, you will have a working semantic search example where:
|
||||
|
||||
- text is converted into embeddings using OpenAI
|
||||
- vectors are stored in Oracle AI Database
|
||||
- LangChain interacts with Oracle AI Database through its vector store integration
|
||||
- similarity search returns the most relevant documents for a given query
|
||||
|
||||
This can serve as a baseline for larger retrieval, RAG, or agent-based applications.
|
||||
|
||||
## Next steps
|
||||
|
||||
Once this example is working, you can extend it by:
|
||||
|
||||
- replacing the sample documents with your own data
|
||||
- adding metadata and document chunking
|
||||
- using the same retrieval flow in a larger RAG pipeline
|
||||
- integrating the notebook logic into an API or application
|
||||
|
||||
|
||||
## Documentation
|
||||
For additional information, please refer to the documentation:
|
||||
|
||||
[](https://docs.langchain.com/oss/python/integrations/vectorstores/oracle)
|
||||
[](https://docs.langchain.com/oss/python/integrations/text_embedding/openai)
|
||||
[](https://docs.oracle.com/en/database/oracle/oracle-database/26/vecse/)
|
||||
|
||||
@@ -0,0 +1,592 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "intro",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Building Semantic Search with OpenAI Embeddings, LangChain, and Oracle AI Database\n",
|
||||
"\n",
|
||||
"This cookbook shows how to build a semantic search workflow using:\n",
|
||||
"\n",
|
||||
"- **OpenAI embeddings** to turn text into vector representations\n",
|
||||
"- **LangChain's Oracle vector store integration** to write and query vectors through a familiar Python interface\n",
|
||||
"- **Oracle AI Database Vector Search** to store embeddings alongside relational/application data\n",
|
||||
"\n",
|
||||
"OpenAI is used for embedding generation. LangChain provides the vector store abstraction, and Oracle AI Database provides vector storage and similarity search. This keeps the OpenAI role explicit without implying that OpenAI provides a separate managed vector search feature.\n",
|
||||
"\n",
|
||||
"This pattern is useful for retrieval-augmented generation (RAG), internal semantic search, and applications where source data already lives in Oracle. It lets you add semantic retrieval without introducing a separate vector database, while still keeping the LangChain retrieval interface available for larger application workflows.\n",
|
||||
"\n",
|
||||
"## Architecture at a glance\n",
|
||||
"\n",
|
||||
"1. Text documents are embedded with `text-embedding-3-small`.\n",
|
||||
"2. LangChain writes the text, metadata, and vectors into an Oracle AI Database table.\n",
|
||||
"3. A natural-language query is embedded with the same OpenAI model.\n",
|
||||
"4. Oracle AI Database Vector Search returns the nearest stored documents.\n",
|
||||
"5. The retrieved text can be passed to a generation model in a larger RAG application.\n",
|
||||
"\n",
|
||||
"## What this notebook demonstrates\n",
|
||||
"\n",
|
||||
"1. Load credentials from environment variables or a local `.env` file.\n",
|
||||
"2. Connect to Oracle AI Database with the Python `oracledb` driver.\n",
|
||||
"3. Initialize OpenAI embeddings and LangChain's Oracle vector store.\n",
|
||||
"4. Insert a small rerunnable sample dataset without accumulating duplicates.\n",
|
||||
"5. Run semantic similarity search with `similarity_search(..., k=3)`.\n",
|
||||
"\n",
|
||||
"## Requirements\n",
|
||||
"\n",
|
||||
"- Python 3.10+\n",
|
||||
"- OpenAI API key with embeddings access\n",
|
||||
"- Oracle AI Database with Vector Search enabled\n",
|
||||
"- Python packages: `langchain`, `langchain-openai`, `langchain-oracledb`, `oracledb`, `python-dotenv`, `numpy`\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "install-deps-md",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Install dependencies\n",
|
||||
"\n",
|
||||
"If these packages are already installed in your environment, this cell will not make any changes.\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"id": "install-deps",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%%capture\n",
|
||||
"%pip install -q langchain langchain-oracledb langchain-openai oracledb python-dotenv numpy\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "config-md",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Configure credentials and Oracle connectivity\n",
|
||||
"\n",
|
||||
"The notebook reads configuration from environment variables. You can set them in your shell, in your notebook environment, or in a local `.env` file next to this notebook:\n",
|
||||
"\n",
|
||||
"```env\n",
|
||||
"OPENAI_API_KEY=your-openai-api-key\n",
|
||||
"ORACLE_USER=your-oracle-user\n",
|
||||
"ORACLE_PASSWORD=your-oracle-password\n",
|
||||
"ORACLE_DSN=host:port/service_name\n",
|
||||
"```\n",
|
||||
"\n",
|
||||
"For local experiments, one common option is an Oracle Database Free container with port `1521` exposed and a pluggable database service such as `FREEPDB1`, making `ORACLE_DSN` look like `localhost:1521/FREEPDB1`. You can also use an Oracle Autonomous Database or another Oracle AI Database instance that has Vector Search enabled.\n",
|
||||
"\n",
|
||||
"Oracle's Python driver can run in **Thin mode**, which needs no separate Oracle client libraries. If Oracle Instant Client is installed, the same driver can optionally use **Thick mode** for compatibility with some advanced client features. The setup cell below tries to enable Thick mode when available and otherwise continues in Thin mode.\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "oracle-setup-options",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Oracle setup options\n",
|
||||
"\n",
|
||||
"This notebook assumes you already have access to an Oracle AI Database endpoint. Any of these setup paths can work:\n",
|
||||
"\n",
|
||||
"- **Local development:** Oracle Database Free in a container or local install, with a service name such as `FREEPDB1`.\n",
|
||||
"- **Managed cloud:** Oracle Autonomous Database with Vector Search enabled.\n",
|
||||
"- **Existing environment:** an internal Oracle AI Database instance provided by your team.\n",
|
||||
"\n",
|
||||
"The notebook only needs a SQL connection string through `ORACLE_DSN`. It does not create a database instance for you.\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"id": "config-code",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Oracle Client initialized in Thick mode\n",
|
||||
"Configuration loaded\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"import warnings\n",
|
||||
"\n",
|
||||
"warnings.filterwarnings(\"ignore\", category=DeprecationWarning)\n",
|
||||
"warnings.filterwarnings(\n",
|
||||
" \"ignore\",\n",
|
||||
" message=\"Core Pydantic V1 functionality isn't compatible with Python 3.14 or greater.*\",\n",
|
||||
" category=UserWarning,\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"import numpy as np\n",
|
||||
"import oracledb\n",
|
||||
"from dotenv import load_dotenv\n",
|
||||
"from langchain_openai import OpenAIEmbeddings\n",
|
||||
"from langchain_oracledb.vectorstores import OracleVS\n",
|
||||
"from langchain_oracledb.vectorstores.oraclevs import DistanceStrategy\n",
|
||||
"\n",
|
||||
"load_dotenv()\n",
|
||||
"\n",
|
||||
"# Optional: uncomment and fill these in if you are not using environment variables\n",
|
||||
"# or a local .env file.\n",
|
||||
"# os.environ[\"OPENAI_API_KEY\"] = \"your-openai-api-key\"\n",
|
||||
"# os.environ[\"ORACLE_USER\"] = \"your-oracle-user\"\n",
|
||||
"# os.environ[\"ORACLE_PASSWORD\"] = \"your-oracle-password\"\n",
|
||||
"# os.environ[\"ORACLE_DSN\"] = \"host:port/service_name\"\n",
|
||||
"\n",
|
||||
"required_env_vars = [\n",
|
||||
" \"OPENAI_API_KEY\",\n",
|
||||
" \"ORACLE_USER\",\n",
|
||||
" \"ORACLE_PASSWORD\",\n",
|
||||
" \"ORACLE_DSN\",\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"missing = [name for name in required_env_vars if not os.getenv(name)]\n",
|
||||
"if missing:\n",
|
||||
" raise EnvironmentError(\n",
|
||||
" \"Missing required environment variables: \"\n",
|
||||
" f\"{missing}. Set them in your environment or in a local .env file.\"\n",
|
||||
" )\n",
|
||||
"\n",
|
||||
"try:\n",
|
||||
" oracledb.init_oracle_client()\n",
|
||||
" print(\"Oracle Client initialized in Thick mode\")\n",
|
||||
"except Exception:\n",
|
||||
" print(\"Oracle Client running in Thin mode\")\n",
|
||||
"\n",
|
||||
"print(\"Configuration loaded\")\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "connect-md",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Connect to Oracle AI Database\n",
|
||||
"\n",
|
||||
"This connection is used by LangChain's Oracle vector store to create or reuse the demo table, insert embeddings, and run similarity search.\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"id": "connect-code",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Connected to Oracle AI Database\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"conn = oracledb.connect(\n",
|
||||
" user=os.getenv(\"ORACLE_USER\"),\n",
|
||||
" password=os.getenv(\"ORACLE_PASSWORD\"),\n",
|
||||
" dsn=os.getenv(\"ORACLE_DSN\"),\n",
|
||||
")\n",
|
||||
"\n",
|
||||
"print(\"Connected to Oracle AI Database\")\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "table-distance-md",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Choose a demo table and distance metric\n",
|
||||
"\n",
|
||||
"The demo uses one table, `LANGCHAIN_DEMO_VECTORS`, and cosine distance. Cosine distance is a common default for text embeddings because it compares vector direction rather than raw magnitude.\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "vector-store-md",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Initialize embeddings and the Oracle vector store\n",
|
||||
"\n",
|
||||
"We initialize the OpenAI embedding model close to the vector store setup, because LangChain uses the embedding function when documents are inserted and when natural language queries are searched.\n",
|
||||
"\n",
|
||||
"If an incompatible demo table already exists from an earlier version of the notebook, it is dropped so LangChain can recreate it with the columns it expects: `ID`, `TEXT`, `METADATA`, and `EMBEDDING`.\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"id": "vector-store-code",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Oracle Vector Store initialized\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"table_name = \"LANGCHAIN_DEMO_VECTORS\"\n",
|
||||
"distance_strategy = DistanceStrategy.COSINE\n",
|
||||
"\n",
|
||||
"embeddings = OpenAIEmbeddings(model=\"text-embedding-3-small\")\n",
|
||||
"\n",
|
||||
"required_columns = {\"ID\", \"TEXT\", \"METADATA\", \"EMBEDDING\"}\n",
|
||||
"\n",
|
||||
"with conn.cursor() as cursor:\n",
|
||||
" cursor.execute(\n",
|
||||
" \"\"\"\n",
|
||||
" SELECT column_name\n",
|
||||
" FROM user_tab_columns\n",
|
||||
" WHERE table_name = :table_name\n",
|
||||
" \"\"\",\n",
|
||||
" table_name=table_name.upper(),\n",
|
||||
" )\n",
|
||||
" existing_columns = {row[0] for row in cursor.fetchall()}\n",
|
||||
"\n",
|
||||
"if existing_columns and not required_columns.issubset(existing_columns):\n",
|
||||
" with conn.cursor() as cursor:\n",
|
||||
" cursor.execute(f\"DROP TABLE {table_name} PURGE\")\n",
|
||||
" conn.commit()\n",
|
||||
" print(f\"Dropped incompatible existing demo table: {table_name}\")\n",
|
||||
"\n",
|
||||
"try:\n",
|
||||
" oracle_vs = OracleVS(\n",
|
||||
" client=conn,\n",
|
||||
" embedding_function=embeddings,\n",
|
||||
" table_name=table_name,\n",
|
||||
" distance_strategy=distance_strategy,\n",
|
||||
" )\n",
|
||||
" print(\"Oracle Vector Store initialized\")\n",
|
||||
"except Exception as e:\n",
|
||||
" oracle_vs = None\n",
|
||||
" print(\"Oracle Vector Store not initialized.\")\n",
|
||||
" print(\"This can happen if embedding generation is unavailable or configuration is incomplete.\")\n",
|
||||
" print(e)\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "sample-data-md",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Insert rerunnable sample data\n",
|
||||
"\n",
|
||||
"The sample data is intentionally small so the retrieval behavior is easy to inspect. Before inserting, the cell clears the demo table. This keeps the notebook idempotent: repeated runs do not accumulate duplicate rows or change retrieval results.\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "idempotency-note",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The `DELETE FROM` in the next cell is intentional. It resets only the demo table's sample rows before insertion, making repeated notebook runs deterministic. Without that reset, rerunning the notebook would append the same documents again and could change the top-k retrieval results.\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"id": "sample-data-code",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Cleared existing sample documents\n",
|
||||
"Sample documents embedded and stored\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"texts = [\n",
|
||||
" \"Oracle AI Database provides native vector search.\",\n",
|
||||
" \"LangChain integrates Oracle AI Database using the langchain-oracledb plugin.\",\n",
|
||||
" \"Vector search enables semantic similarity over unstructured text.\",\n",
|
||||
" \"Relational AI databases can now store and search embeddings efficiently.\",\n",
|
||||
"]\n",
|
||||
"\n",
|
||||
"if oracle_vs:\n",
|
||||
" try:\n",
|
||||
" with conn.cursor() as cursor:\n",
|
||||
" cursor.execute(f\"DELETE FROM {table_name}\")\n",
|
||||
" conn.commit()\n",
|
||||
" print(\"Cleared existing sample documents\")\n",
|
||||
"\n",
|
||||
" oracle_vs.add_texts(texts)\n",
|
||||
" print(\"Sample documents embedded and stored\")\n",
|
||||
" except Exception as e:\n",
|
||||
" oracle_vs = None\n",
|
||||
" print(\"Skipping document embedding because vector store population failed.\")\n",
|
||||
" print(e)\n",
|
||||
"else:\n",
|
||||
" print(\"Skipping document embedding (vector store not available)\")\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "inspect-md",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Inspect embedding and table details\n",
|
||||
"\n",
|
||||
"These checks are optional and are guarded so the notebook can continue gracefully if embedding quota is unavailable. The table inspection does not call the OpenAI API.\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"id": "inspect-code",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Embedding statistics:\n",
|
||||
"- Embedding dimension: 1536\n",
|
||||
"- Number of documents embedded: 4\n",
|
||||
"- Oracle embedding table: LANGCHAIN_DEMO_VECTORS\n",
|
||||
"Storage size estimation:\n",
|
||||
"- Approx. bytes per embedding: 12288\n",
|
||||
"- Approx. total bytes stored: 49152\n",
|
||||
"Vector table LANGCHAIN_DEMO_VECTORS columns: ['ID', 'TEXT', 'METADATA', 'EMBEDDING']\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"sample_embedding = None\n",
|
||||
"embedding_dim = None\n",
|
||||
"\n",
|
||||
"if oracle_vs:\n",
|
||||
" try:\n",
|
||||
" sample_embedding = embeddings.embed_query(\"test\")\n",
|
||||
" embedding_dim = len(sample_embedding)\n",
|
||||
"\n",
|
||||
" print(\"Embedding statistics:\")\n",
|
||||
" print(f\"- Embedding dimension: {embedding_dim}\")\n",
|
||||
" print(f\"- Number of documents embedded: {len(texts)}\")\n",
|
||||
" print(f\"- Oracle embedding table: {table_name}\")\n",
|
||||
"\n",
|
||||
" embedding_bytes = np.array(sample_embedding).nbytes\n",
|
||||
" print(\"Storage size estimation:\")\n",
|
||||
" print(f\"- Approx. bytes per embedding: {embedding_bytes}\")\n",
|
||||
" print(f\"- Approx. total bytes stored: {embedding_bytes * len(texts)}\")\n",
|
||||
" except Exception as e:\n",
|
||||
" print(\"Skipping embedding statistics because embedding generation is unavailable.\")\n",
|
||||
" print(e)\n",
|
||||
"else:\n",
|
||||
" print(\"Skipping embedding statistics (vector store not available)\")\n",
|
||||
"\n",
|
||||
"with conn.cursor() as cursor:\n",
|
||||
" cursor.execute(\n",
|
||||
" \"\"\"\n",
|
||||
" SELECT column_name\n",
|
||||
" FROM user_tab_columns\n",
|
||||
" WHERE table_name = :table_name\n",
|
||||
" ORDER BY column_id\n",
|
||||
" \"\"\",\n",
|
||||
" table_name=table_name.upper(),\n",
|
||||
" )\n",
|
||||
" columns = [row[0] for row in cursor.fetchall()]\n",
|
||||
"\n",
|
||||
"print(f\"Vector table {table_name} columns: {columns}\")\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "search-md",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Run semantic similarity search\n",
|
||||
"\n",
|
||||
"LangChain embeds the natural language query and searches the Oracle vector store for the closest stored documents. The example uses `similarity_search(..., k=3)`, which returns the top matching documents.\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"id": "search-code",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"LangChain similarity search returned 3 results\n",
|
||||
"1. Oracle AI Database provides native vector search.\n",
|
||||
"2. Vector search enables semantic similarity over unstructured text.\n",
|
||||
"3. Relational AI databases can now store and search embeddings efficiently.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"query = \"How does Oracle support vector search?\"\n",
|
||||
"\n",
|
||||
"if oracle_vs:\n",
|
||||
" try:\n",
|
||||
" results = oracle_vs.similarity_search(query, k=3)\n",
|
||||
" print(f\"LangChain similarity search returned {len(results)} results\")\n",
|
||||
"\n",
|
||||
" for i, doc in enumerate(results, start=1):\n",
|
||||
" print(f\"{i}. {doc.page_content}\")\n",
|
||||
" except Exception as e:\n",
|
||||
" print(\"Skipping similarity search because query embedding or retrieval failed.\")\n",
|
||||
" print(e)\n",
|
||||
"else:\n",
|
||||
" print(\"Skipping similarity search (vector store not available)\")\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "inspect-results-md",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Inspect returned documents\n",
|
||||
"\n",
|
||||
"LangChain returns `Document` objects. In this simple example the important field is `page_content`, but in larger workflows metadata can be used to track source IDs, URLs, document sections, or other application-specific attributes.\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"id": "inspect-results-code",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Result 1\n",
|
||||
"- content: Oracle AI Database provides native vector search.\n",
|
||||
"- metadata: {}\n",
|
||||
"Result 2\n",
|
||||
"- content: Vector search enables semantic similarity over unstructured text.\n",
|
||||
"- metadata: {}\n",
|
||||
"Result 3\n",
|
||||
"- content: Relational AI databases can now store and search embeddings efficiently.\n",
|
||||
"- metadata: {}\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"if oracle_vs and \"results\" in globals():\n",
|
||||
" for i, doc in enumerate(results, start=1):\n",
|
||||
" print(f\"Result {i}\")\n",
|
||||
" print(f\"- content: {doc.page_content}\")\n",
|
||||
" print(f\"- metadata: {doc.metadata}\")\n",
|
||||
"else:\n",
|
||||
" print(\"Skipping document inspection because no search results are available.\")\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "second-query-md",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Try another query\n",
|
||||
"\n",
|
||||
"The same vector store can be reused for additional questions without reinitializing the embedding model.\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"id": "second-query-code",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Oracle AI Database vector search results:\n",
|
||||
"Number of results: 3\n",
|
||||
"1. Oracle AI Database provides native vector search.\n",
|
||||
"2. LangChain integrates Oracle AI Database using the langchain-oracledb plugin.\n",
|
||||
"3. Vector search enables semantic similarity over unstructured text.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"if oracle_vs:\n",
|
||||
" try:\n",
|
||||
" print(\"Oracle AI Database vector search results:\")\n",
|
||||
" results = oracle_vs.similarity_search(\"What is Oracle Search?\", k=3)\n",
|
||||
"\n",
|
||||
" print(f\"Number of results: {len(results)}\")\n",
|
||||
" for i, doc in enumerate(results, start=1):\n",
|
||||
" print(f\"{i}. {doc.page_content}\")\n",
|
||||
" except Exception as e:\n",
|
||||
" print(\"Skipping final vector search because query embedding or retrieval failed.\")\n",
|
||||
" print(e)\n",
|
||||
"else:\n",
|
||||
" print(\"Skipping final vector search because the Oracle vector store was not initialized.\")\n",
|
||||
" "
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "conclusion",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"### Conclusion\n",
|
||||
"\n",
|
||||
"This notebook implemented semantic vector search with OpenAI embeddings, LangChain, and Oracle AI Database. OpenAI converts text and queries into embeddings, LangChain provides the vector store API, and Oracle AI Database stores and searches the vectors alongside application data.\n",
|
||||
"\n",
|
||||
"The same pattern can be extended into a RAG workflow by passing retrieved documents to a generation model, or into an application search layer where Oracle remains the source of truth for both relational and vector data.\n",
|
||||
" "
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.14.2"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
Reference in New Issue
Block a user