wehub-resource-sync
a0c8464e58
Build Package / build (ubuntu-latest) (push) Failing after 1s
CodeQL / Analyze (python) (push) Failing after 1s
Core Typecheck / core-typecheck (push) Failing after 1s
Linting / lint (push) Failing after 1s
llama-dev tests / test-llama-dev (push) Failing after 1s
Publish Sub-Package to PyPI if Needed / publish_subpackage_if_needed (push) Has been skipped
Sync Docs to Developer Hub / sync-docs (push) Failing after 0s
Build Package / build (windows-latest) (push) Has been cancelled
393 lines
9.6 KiB
Plaintext
393 lines
9.6 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9dced1fd-2db9-4b4e-88f5-bd65563ec1a6",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Vertex AI\n",
|
|
"\n",
|
|
"**NOTE:** Vertex has largely been replaced by Google GenAI, which supports the same functionality from Vertex using the `google-genai` package. Visit the [Google GenAI page](https://docs.llamaindex.ai/en/stable/examples/llm/google_genai/) for the latest examples and documentation.\n",
|
|
"\n",
|
|
"## Installing Vertex AI \n",
|
|
"To Install Vertex AI you need to follow the following steps\n",
|
|
"* Install Vertex Cloud SDK (https://googleapis.dev/python/aiplatform/latest/index.html)\n",
|
|
"* Setup your Default Project, credentials, region\n",
|
|
"# Basic auth example for service account"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "e25b1ac2",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"%pip install llama-index-llms-vertex"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "3d42f4996210bdc7",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from llama_index.llms.vertex import Vertex\n",
|
|
"from google.oauth2 import service_account\n",
|
|
"\n",
|
|
"filename = \"vertex-407108-37495ce6c303.json\"\n",
|
|
"credentials: service_account.Credentials = (\n",
|
|
" service_account.Credentials.from_service_account_file(filename)\n",
|
|
")\n",
|
|
"Vertex(\n",
|
|
" model=\"text-bison\", project=credentials.project_id, credentials=credentials\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "119bbfb7d84a593d",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Basic Usage\n",
|
|
"Basic call to the text-bison model"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "bf7deb10-28fe-41f2-abda-283162e9f35b",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"' ```\\nHello this is a sample text\\n```'"
|
|
]
|
|
},
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"from llama_index.llms.vertex import Vertex\n",
|
|
"from llama_index.core.llms import ChatMessage, MessageRole\n",
|
|
"\n",
|
|
"llm = Vertex(model=\"text-bison\", temperature=0, additional_kwargs={})\n",
|
|
"llm.complete(\"Hello this is a sample text\").text"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "c3afe813-a5cb-4175-bf9c-5484b2da0a9b",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Async Usage\n",
|
|
"### Async "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "7916602b-4f97-43bb-85a9-ac683bc962f3",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"' Hello! How can I help you?'"
|
|
]
|
|
},
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"(await llm.acomplete(\"hello\")).text"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2cfd0b78-b779-4390-96f7-99ff60087786",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Streaming Usage \n",
|
|
"### Streaming "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "ef4765c5-ec6a-4109-8453-c9f76396d572",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"' Hello! How can I help you?'"
|
|
]
|
|
},
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"list(llm.stream_complete(\"hello\"))[-1].text"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "ce5d9453-6465-489e-b205-a25a3da37dde",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Chat Usage\n",
|
|
"### chat generation"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "2b8861d4-a151-4a5d-8a6f-d24e284009c3",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"chat = Vertex(model=\"chat-bison\")\n",
|
|
"messages = [\n",
|
|
" ChatMessage(role=MessageRole.SYSTEM, content=\"Reply everything in french\"),\n",
|
|
" ChatMessage(role=MessageRole.USER, content=\"Hello\"),\n",
|
|
"]"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "f2286016-9d58-468d-a9a2-571ad90066ad",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"' Bonjour! Comment vas-tu?'"
|
|
]
|
|
},
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"chat.chat(messages=messages).message.content"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "b830d59e-12f6-4ef7-b1ce-e71408ae3cbb",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Async Chat\n",
|
|
"### Asynchronous chat response"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "2920968a-9566-4964-b468-9a5899f42e61",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"' Bonjour! Comment vas-tu?'"
|
|
]
|
|
},
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"(await chat.achat(messages=messages)).message.content"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "632585b4-f146-492c-bb16-30e8b64e2d52",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Streaming Chat\n",
|
|
"### streaming chat response"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "9ee22376-317a-4bda-a20e-5d54f88503f7",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"' Bonjour! Comment vas-tu?'"
|
|
]
|
|
},
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
"list(chat.stream_chat(messages=messages))[-1].message.content"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "e24abcf3278f5ee6",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Gemini Models\n",
|
|
"Calling Google Gemini Models using Vertex AI is fully supported.\n",
|
|
"### Gemini Pro"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "2b900308342b19be",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"llm = Vertex(\n",
|
|
" model=\"gemini-pro\",\n",
|
|
" project=credentials.project_id,\n",
|
|
" credentials=credentials,\n",
|
|
" context_window=100000,\n",
|
|
")\n",
|
|
"llm.complete(\"Hello Gemini\").text"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "b840065e0312fd46",
|
|
"metadata": {},
|
|
"source": [
|
|
"### Gemini Vision Models\n",
|
|
"Gemini vision-capable models now support `TextBlock` and `ImageBlock` for structured multi-modal inputs, replacing the older dictionary-based `content` format. Use `blocks` to include text and images via file paths or URLs."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "510f79b7",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Example with Image Path:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "1a9f12769bdeca6f",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"The image shows a man in handcuffs being escorted by police officers. The man is wearing a black jacket and a black hooded sweatshirt. He is being escorted by two police officers who are wearing black balaclavas and bulletproof vests. The officers are from the Romanian Gendarmerie. The image is likely from a news report or a documentary about a crime.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from llama_index.llms.vertex import Vertex\n",
|
|
"from llama_index.core.llms import ChatMessage, TextBlock, ImageBlock\n",
|
|
"\n",
|
|
"history = [\n",
|
|
" ChatMessage(\n",
|
|
" role=\"user\",\n",
|
|
" blocks=[\n",
|
|
" ImageBlock(path=\"sample.jpg\"),\n",
|
|
" TextBlock(text=\"What is in this image?\"),\n",
|
|
" ],\n",
|
|
" ),\n",
|
|
"]\n",
|
|
"llm = Vertex(\n",
|
|
" model=\"gemini-1.5-flash\",\n",
|
|
" project=credentials.project_id,\n",
|
|
" credentials=credentials,\n",
|
|
" context_window=100000,\n",
|
|
")\n",
|
|
"print(llm.chat(history).message.content)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "dc49ff72",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Example with Image URL:**"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "ba0c94de",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"A close-up of a tiger's face. The tiger is looking directly at the camera with a serious expression. The fur is a mix of orange, black, and white. The background is blurred, making the tiger the main focus of the image.\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"from llama_index.llms.vertex import Vertex\n",
|
|
"from llama_index.core.llms import ChatMessage, TextBlock, ImageBlock\n",
|
|
"\n",
|
|
"history = [\n",
|
|
" ChatMessage(\n",
|
|
" role=\"user\",\n",
|
|
" blocks=[\n",
|
|
" ImageBlock(\n",
|
|
" url=\"https://upload.wikimedia.org/wikipedia/commons/7/71/Sibirischer_tiger_de_edit02.jpg\"\n",
|
|
" ),\n",
|
|
" TextBlock(text=\"What is in this image?\"),\n",
|
|
" ],\n",
|
|
" ),\n",
|
|
"]\n",
|
|
"llm = Vertex(\n",
|
|
" model=\"gemini-1.5-flash\",\n",
|
|
" project=credentials.project_id,\n",
|
|
" credentials=credentials,\n",
|
|
" context_window=100000,\n",
|
|
")\n",
|
|
"print(llm.chat(history).message.content)"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "base",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|