light-heart-labs--dreamserver/ods/docs/INTEGRATION-GUIDE.md

ODS Integration Guide

You've got ODS running. Now what? This guide shows how to connect your apps.

---

1. OpenAI SDK Compatibility

ODS exposes an OpenAI-compatible API. Just point your SDK at localhost.

Python

pip install openai

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",  # ODS llama-server
    api_key="not-needed"  # Local, no auth required
)

response = client.chat.completions.create(
    model="qwen2.5-32b-instruct",  # Your running model
    messages=[
        {"role": "user", "content": "Hello!"}
    ]
)
print(response.choices[0].message.content)

Node.js / TypeScript

npm install openai

import OpenAI from 'openai';

const openai = new OpenAI({
  baseURL: 'http://localhost:8080/v1',
  apiKey: 'not-needed',
});

const response = await openai.chat.completions.create({
  model: 'qwen2.5-32b-instruct',
  messages: [{ role: 'user', content: 'Hello!' }],
});

console.log(response.choices[0].message.content);

curl

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen2.5-32b-instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

---

2. LangChain Integration

pip install langchain langchain-openai

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="http://localhost:8080/v1",
    api_key="not-needed",
    model="qwen2.5-32b-instruct",
    temperature=0.7,
)

response = llm.invoke("Explain quantum computing in one sentence.")
print(response.content)

With RAG (Qdrant + Embeddings)

from langchain_openai import OpenAIEmbeddings
from langchain_qdrant import Qdrant

embeddings = OpenAIEmbeddings(
    base_url="http://localhost:8090/v1",  # Embeddings service
    api_key="not-needed",
)

# Connect to ODS's Qdrant
qdrant = Qdrant.from_existing_collection(
    embeddings=embeddings,
    collection_name="documents",
    url="http://localhost:6333",
)

# Query documents
results = qdrant.similarity_search("What is the policy on refunds?")

---

3. Continue.dev Setup

Continue is an open-source AI code assistant that works in VS Code.

1. Install Continue extension in VS Code
2. Edit ~/.continue/config.json:

{
  "models": [
    {
      "title": "ODS",
      "provider": "openai",
      "model": "qwen2.5-32b-instruct",
      "apiBase": "http://localhost:8080/v1",
      "apiKey": "not-needed"
    }
  ]
}

3. Restart VS Code, select "ODS" in Continue panel

---

4. Cursor IDE Integration

Cursor supports custom API endpoints.

1. Open Cursor Settings → Models
2. Add custom model:
   • API Base: http://localhost:8080/v1
   • API Key: not-needed
   • Model: qwen2.5-32b-instruct

---

5. n8n Workflow Examples

ODS includes n8n for workflow automation. Access at http://localhost:5678.

Creating Workflows

1. Open n8n at http://localhost:5678
2. Log in with the credentials from your .env (N8N_USER / N8N_PASS)
3. Create a new workflow or import from the n8n template library
4. Use the "HTTP Request" node pointed at http://llama-server:8080/v1/chat/completions (Docker-internal URL)

Example Workflow Ideas

Workflow	Description
Chat Endpoint	HTTP webhook → LLM → response
Document Q&A	File upload → embeddings → Qdrant → LLM
Voice Transcription	Audio → Whisper STT → text
TTS API	Text → Kokoro TTS → audio
Voice-to-Voice	STT → LLM → TTS pipeline

---

6. REST API Reference

Endpoints

Endpoint	Description
POST /v1/chat/completions	Chat (OpenAI compatible)
POST /v1/completions	Text completion
GET /v1/models	List available models
GET /health	Health check

Streaming

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="x")

stream = client.chat.completions.create(
    model="qwen2.5-32b-instruct",
    messages=[{"role": "user", "content": "Write a poem"}],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

---

7. Environment Variables

Key variables in .env (see .env.example for the full list):

Variable	Default	Description
OLLAMA_PORT	8080	llama-server external port (maps to internal 8080)
WEBUI_PORT	3000	Open WebUI port
N8N_PORT	5678	n8n workflows port
LLM_MODEL	(tier-dependent)	Model name for OpenClaw/dashboard
CTX_SIZE	16384	Context window size (tokens)
GGUF_FILE	(tier-dependent)	GGUF model filename in data/models/

---

8. Authentication Options

No Auth (Default)

Local-only, no auth required. Good for development.

API Key Auth

Set in .env:

LLM_API_KEY=your-secret-key

Then include in requests:

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="your-secret-key"
)

Open WebUI Auth

WebUI has built-in user management:

1. First user becomes admin
2. Configure auth in WebUI settings
3. Set WEBUI_AUTH=true in .env

---

Common Issues

"Model not found"

Check running model name:

curl http://localhost:8080/v1/models

Use the exact model name in your requests.

Connection refused

Ensure services are running:

docker compose ps

Check llama-server is ready:

docker compose logs llama-server | tail -20

Slow first response

First request after start triggers model warm-up. Wait 30-60 seconds.

---

Built by The Collective