chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,20 @@
|
||||
# Distributed embeddings clusters
|
||||
|
||||
The API supports combining multiple API instances into a single logical embeddings index. An example configuration is shown below.
|
||||
|
||||
```yaml
|
||||
cluster:
|
||||
shards:
|
||||
- http://127.0.0.1:8002
|
||||
- http://127.0.0.1:8003
|
||||
```
|
||||
|
||||
This configuration aggregates the API instances above as index shards. Data is evenly split among each of the shards at index time. Queries are run in parallel against each shard and the results are joined together. This method allows horizontal scaling and supports very large index clusters.
|
||||
|
||||
This method is only recommended for data sets in the 1 billion+ records. The ANN libraries can easily support smaller data sizes and this method is not worth the additional complexity. At this time, new shards can not be added after building the initial index.
|
||||
|
||||
See the link below for a detailed example covering distributed embeddings clusters.
|
||||
|
||||
| Notebook | Description | |
|
||||
|:----------|:-------------|------:|
|
||||
| [Distributed embeddings cluster](https://github.com/neuml/txtai/blob/master/examples/15_Distributed_embeddings_cluster.ipynb) | Distribute an embeddings index across multiple data nodes | [](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/15_Distributed_embeddings_cluster.ipynb) |
|
||||
@@ -0,0 +1,165 @@
|
||||
# Configuration
|
||||
|
||||
Configuration is set through YAML. In most cases, YAML keys map to fields names in Python. The [example in the previous section](../) gave a full-featured example covering a wide array of configuration options.
|
||||
|
||||
Each section below describes the available configuration settings.
|
||||
|
||||
## Embeddings
|
||||
|
||||
The configuration parser expects a top level `embeddings` key to be present in the YAML. All [embeddings configuration](../../embeddings/configuration) is supported.
|
||||
|
||||
The following example defines an embeddings index.
|
||||
|
||||
```yaml
|
||||
path: index path
|
||||
writable: true
|
||||
|
||||
embeddings:
|
||||
path: vector model
|
||||
content: true
|
||||
```
|
||||
|
||||
Three top level settings are available to control where indexes are saved and if an index is a read-only index.
|
||||
|
||||
### path
|
||||
```yaml
|
||||
path: string
|
||||
```
|
||||
|
||||
Path to save and load the embeddings index. Each API instance can only access a single index at a time.
|
||||
|
||||
### writable
|
||||
```yaml
|
||||
writable: boolean
|
||||
```
|
||||
|
||||
Sets this embeddings index as writable (true) or read-only (false). This allows serving a read-only index. Defaults to read-only.
|
||||
|
||||
### reindex
|
||||
```yaml
|
||||
reindex: boolean
|
||||
```
|
||||
|
||||
Enables re-indexing for this embeddings index (defaults to disabled).
|
||||
|
||||
Re-indexing accepts new configuration and should only be enabled when the configuration comes from trusted sources.
|
||||
|
||||
### cloud
|
||||
[Cloud storage settings](../../embeddings/configuration/cloud) can be set under a `cloud` top level configuration group.
|
||||
|
||||
## Agent
|
||||
|
||||
Agents are defined under a top level `agent` key. Each key under the `agent` key is the name of the agent. Constructor parameters can be passed under this key.
|
||||
|
||||
The following example defines an agent.
|
||||
|
||||
```yaml
|
||||
agent:
|
||||
researcher:
|
||||
tools:
|
||||
- websearch
|
||||
|
||||
llm:
|
||||
path: Qwen/Qwen3-4B-Instruct-2507
|
||||
```
|
||||
|
||||
## Pipeline
|
||||
|
||||
Pipelines are loaded as top level configuration parameters. Pipeline names are automatically detected in the YAML configuration and created upon startup. All [pipelines](../../pipeline) are supported.
|
||||
|
||||
The following example defines a series of pipelines. Note that entries below are the lower-case names of the pipeline class.
|
||||
|
||||
```yaml
|
||||
caption:
|
||||
|
||||
extractor:
|
||||
path: model path
|
||||
|
||||
labels:
|
||||
|
||||
summary:
|
||||
|
||||
tabular:
|
||||
|
||||
translation:
|
||||
```
|
||||
|
||||
Under each pipeline name, configuration settings for the pipeline can be set.
|
||||
|
||||
## Workflow
|
||||
|
||||
Workflows are defined under a top level `workflow` key. Each key under the `workflow` key is the name of the workflow. Under that is a `tasks` key with each task definition.
|
||||
|
||||
The following example defines a workflow.
|
||||
|
||||
```yaml
|
||||
workflow:
|
||||
sumtranslate:
|
||||
tasks:
|
||||
- action: summary
|
||||
- action: translation
|
||||
```
|
||||
|
||||
### schedule
|
||||
|
||||
Schedules a workflow using a [cron expression](../../workflow/schedule).
|
||||
|
||||
```yaml
|
||||
workflow:
|
||||
index:
|
||||
schedule:
|
||||
cron: 0/10 * * * * *
|
||||
elements: ["api params"]
|
||||
tasks:
|
||||
- task: service
|
||||
url: api url
|
||||
- action: index
|
||||
```
|
||||
|
||||
### tasks
|
||||
```yaml
|
||||
tasks: list
|
||||
```
|
||||
|
||||
Expects a list of workflow tasks. Each element defines a single workflow task. All [task configuration](../../workflow/task) is supported.
|
||||
|
||||
A shorthand syntax for creating tasks is supported. This syntax will automatically map task strings to an `action:value` pair.
|
||||
|
||||
Example below.
|
||||
|
||||
```yaml
|
||||
workflow:
|
||||
index:
|
||||
tasks:
|
||||
- action1
|
||||
- action2
|
||||
```
|
||||
|
||||
Each task element supports the following additional arguments.
|
||||
|
||||
#### action
|
||||
```yaml
|
||||
action: string|list
|
||||
```
|
||||
|
||||
Both single and multi-action tasks are supported.
|
||||
|
||||
The action parameter works slightly different when passed via configuration. The parameter(s) needs to be converted into callable method(s). If action is a pipeline that has been defined in the current configuration, it will use that pipeline as the action.
|
||||
|
||||
There are three special action names `index`, `upsert` and `search`. If `index` or `upsert` are used as the action, the task will collect workflow data elements and load them into defined the embeddings index. If `search` is used, the task will execute embeddings queries for each input data element.
|
||||
|
||||
Otherwise, the action must be a path to a callable object or function. The configuration parser will resolve the function name and use that as the task action.
|
||||
|
||||
#### task
|
||||
```yaml
|
||||
task: string
|
||||
```
|
||||
|
||||
Optionally sets the type of task to create. For example, this could be a `file` task or a `retrieve` task. If this is not specified, a generic task is created. [The list of workflow tasks can be found here](../../workflow).
|
||||
|
||||
#### args
|
||||
```yaml
|
||||
args: list
|
||||
```
|
||||
|
||||
Optional list of static arguments to pass to the workflow task. These are combined with workflow data to pass to each `__call__`.
|
||||
@@ -0,0 +1,25 @@
|
||||
# Customization
|
||||
|
||||
The txtai API has a number of features out of the box that are designed to help get started quickly. API services can also be augmented with custom code and functionality. The two main ways to do this are with extensions and dependencies.
|
||||
|
||||
Extensions add a custom endpoint. Dependencies add middleware that executes with each request. See the sections below for more.
|
||||
|
||||
## Extensions
|
||||
|
||||
While the API is extremely flexible and complex logic can be executed through YAML-driven workflows, some may prefer to create an endpoint in Python. API extensions define custom Python endpoints that interact with txtai applications.
|
||||
|
||||
See the link below for a detailed example.
|
||||
|
||||
| Notebook | Description | |
|
||||
|:----------|:-------------|------:|
|
||||
| [Custom API Endpoints](https://github.com/neuml/txtai/blob/master/examples/51_Custom_API_Endpoints.ipynb) | Extend the API with custom endpoints | [](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/51_Custom_API_Endpoints.ipynb) |
|
||||
|
||||
## Dependencies
|
||||
|
||||
txtai has a default API token authorization method that works well in many cases. Dependencies can also add custom logic with each request. This could be an additional authorization step and/or an authentication method.
|
||||
|
||||
See the link below for a detailed example.
|
||||
|
||||
| Notebook | Description | |
|
||||
|:----------|:-------------|------:|
|
||||
| [API Authorization and Authentication](https://github.com/neuml/txtai/blob/master/examples/54_API_Authorization_and_Authentication.ipynb) | Add authorization, authentication and middleware dependencies to the API | [](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/54_API_Authorization_and_Authentication.ipynb) |
|
||||
@@ -0,0 +1,134 @@
|
||||
# API
|
||||
|
||||
|
||||
|
||||
|
||||
txtai has a full-featured API, backed by [FastAPI](https://github.com/tiangolo/fastapi), that can optionally be enabled for any txtai process. All functionality found in txtai can be accessed via the API.
|
||||
|
||||
The following is an example configuration and startup script for the API.
|
||||
|
||||
Note: This configuration file enables all functionality. For memory-bound systems, splitting pipelines into multiple instances is a best practice.
|
||||
|
||||
```yaml
|
||||
# Index file path
|
||||
path: /tmp/index
|
||||
|
||||
# Allow indexing of documents
|
||||
writable: True
|
||||
|
||||
# Enbeddings index
|
||||
embeddings:
|
||||
path: sentence-transformers/nli-mpnet-base-v2
|
||||
|
||||
# Extractive QA
|
||||
extractor:
|
||||
path: distilbert-base-cased-distilled-squad
|
||||
|
||||
# Zero-shot labeling
|
||||
labels:
|
||||
|
||||
# Similarity
|
||||
similarity:
|
||||
|
||||
# Text segmentation
|
||||
segmentation:
|
||||
sentences: true
|
||||
|
||||
# Text summarization
|
||||
summary:
|
||||
|
||||
# Text extraction
|
||||
textractor:
|
||||
paragraphs: true
|
||||
minlength: 100
|
||||
join: true
|
||||
|
||||
# Transcribe audio to text
|
||||
transcription:
|
||||
|
||||
# Translate text between languages
|
||||
translation:
|
||||
|
||||
# Workflow definitions
|
||||
workflow:
|
||||
sumfrench:
|
||||
tasks:
|
||||
- action: textractor
|
||||
task: url
|
||||
- action: summary
|
||||
- action: translation
|
||||
args: ["fr"]
|
||||
sumspanish:
|
||||
tasks:
|
||||
- action: textractor
|
||||
task: url
|
||||
- action: summary
|
||||
- action: translation
|
||||
args: ["es"]
|
||||
```
|
||||
|
||||
Assuming this YAML content is stored in a file named config.yml, the following command starts the API process.
|
||||
|
||||
```bash
|
||||
CONFIG=config.yml uvicorn "txtai.api:app"
|
||||
```
|
||||
|
||||
Uvicorn is a full-featured production-ready server. See the [Uvicorn deployment guide](https://www.uvicorn.org/deployment/) for more on configuration options.
|
||||
|
||||
## Connect to API
|
||||
|
||||
The default port for the API is 8000. See the uvicorn link above to change this.
|
||||
|
||||
txtai has a number of language bindings which abstract the API (see links below). Alternatively, code can be written to connect directly to the API. Documentation for a live running instance can be found at the `/docs` url (i.e. http://localhost:8000/docs). The following example runs a workflow using cURL.
|
||||
|
||||
```bash
|
||||
curl \
|
||||
-X POST "http://localhost:8000/workflow" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"name":"sumfrench", "elements": ["https://github.com/neuml/txtai"]}'
|
||||
```
|
||||
|
||||
## Local instance
|
||||
|
||||
A local instance can be instantiated. In this case, a txtai application runs internally, without any network connections, providing the same consolidated functionality. This enables running txtai in Python with configuration.
|
||||
|
||||
The configuration above can be run in Python with:
|
||||
|
||||
```python
|
||||
from txtai import Application
|
||||
|
||||
# Load and run workflow
|
||||
app = Application(config.yml)
|
||||
app.workflow("sumfrench", ["https://github.com/neuml/txtai"])
|
||||
```
|
||||
|
||||
See this [link for a full list of methods](./methods).
|
||||
|
||||
## Run with containers
|
||||
|
||||
The API can be containerized and run. This will bring up an API instance without having to install Python, txtai or any dependencies on your machine!
|
||||
|
||||
[See this section for more information](../cloud/#api).
|
||||
|
||||
## Supported language bindings
|
||||
|
||||
The following programming languages have bindings with the txtai API:
|
||||
|
||||
- [Python](https://github.com/neuml/txtai.py)
|
||||
- [JavaScript](https://github.com/neuml/txtai.js)
|
||||
- [Java](https://github.com/neuml/txtai.java)
|
||||
- [Rust](https://github.com/neuml/txtai.rs)
|
||||
- [Go](https://github.com/neuml/txtai.go)
|
||||
|
||||
The API also supports hosting [OpenAI-compatible](./openai) and [Model Context Protocol (MCP)](./mcp) endpoints.
|
||||
|
||||
See the links below for detailed examples covering the API.
|
||||
|
||||
| Notebook | Description | |
|
||||
|:----------|:-------------|------:|
|
||||
| [API Gallery](https://github.com/neuml/txtai/blob/master/examples/08_API_Gallery.ipynb) | Using txtai in JavaScript, Java, Rust and Go | [](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/08_API_Gallery.ipynb) |
|
||||
| [Distributed embeddings cluster](https://github.com/neuml/txtai/blob/master/examples/15_Distributed_embeddings_cluster.ipynb) | Distribute an embeddings index across multiple data nodes | [](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/15_Distributed_embeddings_cluster.ipynb) |
|
||||
| [Embeddings in the Cloud](https://github.com/neuml/txtai/blob/master/examples/43_Embeddings_in_the_Cloud.ipynb) | Load and use an embeddings index from the Hugging Face Hub | [](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/43_Embeddings_in_the_Cloud.ipynb) |
|
||||
| [Custom API Endpoints](https://github.com/neuml/txtai/blob/master/examples/51_Custom_API_Endpoints.ipynb) | Extend the API with custom endpoints | [](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/51_Custom_API_Endpoints.ipynb) |
|
||||
| [API Authorization and Authentication](https://github.com/neuml/txtai/blob/master/examples/54_API_Authorization_and_Authentication.ipynb) | Add authorization, authentication and middleware dependencies to the API | [](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/54_API_Authorization_and_Authentication.ipynb) |
|
||||
| [OpenAI Compatible API](https://github.com/neuml/txtai/blob/master/examples/74_OpenAI_Compatible_API.ipynb) | Connect to txtai with a standard OpenAI client library | [](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/74_OpenAI_Compatible_API.ipynb) |
|
||||
@@ -0,0 +1,34 @@
|
||||
# Model Context Protocol
|
||||
|
||||
The [Model Context Protocol (MCP)](https://modelcontextprotocol.io/introduction) is an open standard that enables developers to build secure, two-way connections between their data sources and AI-powered tools.
|
||||
|
||||
The API can be configured to handle MCP requests. All enabled endpoints set in the API configuration are automatically added as MCP tools.
|
||||
|
||||
```yaml
|
||||
mcp: boolean|dict
|
||||
```
|
||||
|
||||
When `mcp` is a boolean, default arguments are used. When `mcp` is a dictionary it supports the following options.
|
||||
|
||||
```yaml
|
||||
mcp:
|
||||
clientargs: http client options (dict)
|
||||
mcpargs: FastApiMCP options (dict)
|
||||
```
|
||||
|
||||
See the following links for details on the options.
|
||||
|
||||
- [HTTP Client Arguments](https://www.python-httpx.org/api/#asyncclient)
|
||||
- [FastApiMCP Arguments](https://github.com/tadata-org/fastapi_mcp/blob/main/fastapi_mcp/server.py#L22)
|
||||
|
||||
Once this configuration option is added, a new route is added to the application `/mcp`.
|
||||
|
||||
The [Model Context Protocol Inspector tool](https://www.npmjs.com/package/@modelcontextprotocol/inspector) is a quick way to explore how the MCP tools are exported through this interface.
|
||||
|
||||
Run the following and go to the local URL specified.
|
||||
|
||||
```
|
||||
npx @modelcontextprotocol/inspector node build/index.js
|
||||
```
|
||||
|
||||
Enter `http://localhost:8000/mcp` to see the full list of tools available.
|
||||
@@ -0,0 +1,15 @@
|
||||
# Methods
|
||||
|
||||
::: txtai.api.API
|
||||
options:
|
||||
inherited_members: true
|
||||
filters:
|
||||
- "!__del__"
|
||||
- "!flows"
|
||||
- "!function"
|
||||
- "!indexes"
|
||||
- "!limit"
|
||||
- "!pipes"
|
||||
- "!read"
|
||||
- "!resolve"
|
||||
- "!weights"
|
||||
@@ -0,0 +1,13 @@
|
||||
# OpenAI-compatible API
|
||||
|
||||
The API can be configured to serve an OpenAI-compatible API as shown below.
|
||||
|
||||
```yaml
|
||||
openai: True
|
||||
```
|
||||
|
||||
See the link below for a detailed example.
|
||||
|
||||
| Notebook | Description | |
|
||||
|:----------|:-------------|------:|
|
||||
| [OpenAI Compatible API](https://github.com/neuml/txtai/blob/master/examples/74_OpenAI_Compatible_API.ipynb) | Connect to txtai with a standard OpenAI client library | [](https://colab.research.google.com/github/neuml/txtai/blob/master/examples/74_OpenAI_Compatible_API.ipynb) |
|
||||
@@ -0,0 +1,33 @@
|
||||
# Security
|
||||
|
||||
The default implementation of an API service runs via HTTP and is fully open. If the service is being run as a prototype on an internal network, that may be fine. In most scenarios, the connection should at least be encrypted. Authorization is another built-in feature that requires a valid API token with each request. See below for more.
|
||||
|
||||
## HTTPS
|
||||
|
||||
The default API service command starts a Uvicorn server as a HTTP service on port 8000. To run a HTTPS service, consider the following options.
|
||||
|
||||
- [TLS Proxy Server](https://fastapi.tiangolo.com/deployment/https/). *Recommended choice*. With this configuration, the txtai API service runs as a HTTP service only accessible on the localhost/local network. The proxy server handles all encryption and redirects requests to local services. See this [example configuration](https://www.uvicorn.org/deployment/#running-behind-nginx) for more.
|
||||
|
||||
- [Uvicorn SSL Certificate](https://www.uvicorn.org/deployment/). Another option is setting the SSL certificate on the Uvicorn service. This works in simple situations but gets complex when hosting multiple txtai or other related services.
|
||||
|
||||
## Authorization
|
||||
|
||||
Authorization requires a valid API token with each API request. This token is sent as a HTTP `Authorization` header.
|
||||
|
||||
*Server*
|
||||
```bash
|
||||
CONFIG=config.yml TOKEN=<sha256 encoded token> uvicorn "txtai.api:app"
|
||||
```
|
||||
|
||||
*Client*
|
||||
```bash
|
||||
curl \
|
||||
-X POST "http://localhost:8000/workflow" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer <token>" \
|
||||
-d '{"name":"sumfrench", "elements": ["https://github.com/neuml/txtai"]}'
|
||||
```
|
||||
|
||||
It's important to note that HTTPS **must** be enabled using one of the methods mentioned above. Otherwise, tokens will be exchanged as clear text.
|
||||
|
||||
Authentication and Authorization can be fully customized. See the [dependencies](../customization#dependencies) section for more.
|
||||
Reference in New Issue
Block a user