wehub-resource-sync
378 lines
9.9 KiB
Plaintext
378 lines
9.9 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f95d2373",
|
|
"metadata": {},
|
|
"source": [
|
|
"# OpenCode as a txtai LLM\n",
|
|
"\n",
|
|
"[OpenCode](https://github.com/anomalyco/opencode) is an open source AI coding agent. It's rapidly growing in popularity as an open alternative to [Claude Code](https://code.claude.com/docs/en/overview). \n",
|
|
"\n",
|
|
"OpenCode shows the power of permissive open source. It's rapidly undergoing mass adoption and as of January 2026 sits at `81K+` GitHub ⭐'s. OpenCode supports running as a [local server](https://opencode.ai/docs/server/) via `opencode serve`.\n",
|
|
"\n",
|
|
"This notebook will explore how TxtAI integrates this as a LLM pipeline."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "a8719b92",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Install dependencies\n",
|
|
"\n",
|
|
"Install `txtai` and all dependencies."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "e00c85bc",
|
|
"metadata": {
|
|
"vscode": {
|
|
"languageId": "shellscript"
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%%capture\n",
|
|
"!pip install git+https://github.com/neuml/txtai#egg=txtai[api,pipeline-llm]\n",
|
|
"\n",
|
|
"# Install OpenCode if not already installed and run serve\n",
|
|
"!curl -fsSL https://opencode.ai/install | bash\n",
|
|
"!opencode serve &"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "73ffe60e",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Create a OpenCode LLM\n",
|
|
"\n",
|
|
"Now that everything is installed, let's test it out!"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"id": "0e35a304",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/markdown": [
|
|
"```python\n",
|
|
"print(\"Hello, World!\")\n",
|
|
"```"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Markdown object>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"from IPython.display import Markdown, display\n",
|
|
"\n",
|
|
"from txtai import LLM\n",
|
|
"\n",
|
|
"def generate(request):\n",
|
|
" display(Markdown(llm(request)))\n",
|
|
"\n",
|
|
"# Connect to OpenCode server and use default LLM provider\n",
|
|
"llm = LLM(\"opencode\")\n",
|
|
"\n",
|
|
"generate(\"Show a Python Hello World example\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "55dd0fad",
|
|
"metadata": {},
|
|
"source": [
|
|
"OK, now let's try this for a more advanced use case."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 8,
|
|
"id": "b3080841",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/markdown": [
|
|
"```python\n",
|
|
"from txtai import Embeddings, LLM\n",
|
|
"\n",
|
|
"# Create embeddings model\n",
|
|
"embeddings = Embeddings(path=\"sentence-transformers/all-MiniLM-L6-v2\")\n",
|
|
"\n",
|
|
"# Create LLM\n",
|
|
"llm = LLM(\"huggingface/Qwen/Qwen2.5-1.5B-Instruct\")\n",
|
|
"\n",
|
|
"# Sample documents\n",
|
|
"documents = [\n",
|
|
" \"Python is a programming language created by Guido van Rossum\",\n",
|
|
" \"Machine learning is a subset of artificial intelligence\",\n",
|
|
" \"Deep learning uses neural networks with multiple layers\"\n",
|
|
"]\n",
|
|
"\n",
|
|
"# Build the index\n",
|
|
"embeddings.index([(i, doc) for i, doc in enumerate(documents)])\n",
|
|
"\n",
|
|
"# RAG function\n",
|
|
"def rag_query(question):\n",
|
|
" # Search for relevant documents\n",
|
|
" results = embeddings.search(question, 2)\n",
|
|
" \n",
|
|
" # Build context from retrieved documents\n",
|
|
" context = \" \".join([documents[result[0]] for result in results])\n",
|
|
" \n",
|
|
" # Generate response using LLM with context\n",
|
|
" prompt = f\"Context: {context}\\n\\nQuestion: {question}\\n\\nAnswer:\"\n",
|
|
" return llm(prompt)\n",
|
|
"\n",
|
|
"# Example usage\n",
|
|
"question = \"What is Python?\"\n",
|
|
"answer = rag_query(question)\n",
|
|
"print(f\"Question: {question}\")\n",
|
|
"print(f\"Answer: {answer}\")\n",
|
|
"```"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Markdown object>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"generate(\"Show a simple and basic TxtAI RAG example\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "b44e0309",
|
|
"metadata": {},
|
|
"source": [
|
|
"Interesting! It's not limited to Python though."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 9,
|
|
"id": "0a2ec1b4",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/markdown": [
|
|
"```assembly\n",
|
|
"section .data\n",
|
|
" hello db 'Hello, World!', 10 ; String with newline\n",
|
|
" hello_len equ $ - hello ; Length of string\n",
|
|
"\n",
|
|
"section .text\n",
|
|
" global _start\n",
|
|
"\n",
|
|
"_start:\n",
|
|
" ; sys_write system call\n",
|
|
" mov eax, 4 ; sys_write\n",
|
|
" mov ebx, 1 ; stdout\n",
|
|
" mov ecx, hello ; message\n",
|
|
" mov edx, hello_len ; message length\n",
|
|
" int 0x80 ; kernel interrupt\n",
|
|
"\n",
|
|
" ; sys_exit system call\n",
|
|
" mov eax, 1 ; sys_exit\n",
|
|
" mov ebx, 0 ; exit code 0\n",
|
|
" int 0x80 ; kernel interrupt\n",
|
|
"```\n",
|
|
"\n",
|
|
"Compile and run:\n",
|
|
"```bash\n",
|
|
"nasm -f elf32 hello.asm -o hello.o\n",
|
|
"ld -m elf_i386 hello.o -o hello\n",
|
|
"./hello\n",
|
|
"```"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Markdown object>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"generate(\"Show a x86 assembly hello world\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 11,
|
|
"id": "4e28d5c4",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/markdown": [
|
|
"```python\n",
|
|
"a = None\n",
|
|
"if a is not None:\n",
|
|
" a.startswith(\"hello\")\n",
|
|
"else:\n",
|
|
" print(\"a is None, cannot call startswith\")\n",
|
|
"```\n",
|
|
"\n",
|
|
"**What's wrong:** The variable `a` is `None`, which is a `NoneType` object. The `startswith()` method only exists for strings, not for `NoneType`. You need to check if `a` is not `None` before calling string methods on it."
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Markdown object>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"generate(\"\"\"\n",
|
|
"Fix the following error in the code below and explain what's wrong.\n",
|
|
"\n",
|
|
"Error:\n",
|
|
"AttributeError: 'NoneType' object has no attribute 'startswith'\n",
|
|
"\n",
|
|
"Code:\n",
|
|
"a = None\n",
|
|
"a.startswith(\"hello\")\n",
|
|
"\"\"\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "156122f0",
|
|
"metadata": {},
|
|
"source": [
|
|
"# OpenAI-compatible Endpoint\n",
|
|
"\n",
|
|
"Now that OpenCode is integrated into the TxtAI ecosystem, it opens a lot of different opportunities. Let's say we want to host a local OpenAI compatible endpoint, no problem!\n",
|
|
"\n",
|
|
"Write the following file.\n",
|
|
"\n",
|
|
"## config.yml\n",
|
|
"\n",
|
|
"```yaml\n",
|
|
"# Enable OpenAI compat endpoint\n",
|
|
"openai: True\n",
|
|
"\n",
|
|
"llm:\n",
|
|
" path: opencode\n",
|
|
"```\n",
|
|
"\n",
|
|
"and start the following process.\n",
|
|
"\n",
|
|
"```\n",
|
|
"CONFIG=config.yml uvicorn \"txtai.api:app\"\n",
|
|
"```"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 16,
|
|
"id": "33057f59",
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"data": {
|
|
"text/markdown": [
|
|
"```python\n",
|
|
"def factorial(n):\n",
|
|
" if n < 0:\n",
|
|
" raise ValueError(\"Factorial is not defined for negative numbers\")\n",
|
|
" if n == 0 or n == 1:\n",
|
|
" return 1\n",
|
|
" result = 1\n",
|
|
" for i in range(2, n + 1):\n",
|
|
" result *= i\n",
|
|
" return result\n",
|
|
"\n",
|
|
"# Example usage:\n",
|
|
"# print(factorial(5)) # Output: 120\n",
|
|
"```"
|
|
],
|
|
"text/plain": [
|
|
"<IPython.core.display.Markdown object>"
|
|
]
|
|
},
|
|
"metadata": {},
|
|
"output_type": "display_data"
|
|
}
|
|
],
|
|
"source": [
|
|
"from IPython.display import Markdown, display\n",
|
|
"\n",
|
|
"from openai import OpenAI\n",
|
|
"\n",
|
|
"client = OpenAI(\n",
|
|
" base_url=\"http://localhost:8000/v1\",\n",
|
|
" api_key=\"api-key\",\n",
|
|
")\n",
|
|
"\n",
|
|
"response = client.chat.completions.create(\n",
|
|
" messages=[{\n",
|
|
" \"role\": \"user\",\n",
|
|
" \"content\": \"Show a Python factorial function\",\n",
|
|
" }],\n",
|
|
" model=\"opencode\",\n",
|
|
")\n",
|
|
"\n",
|
|
"display(Markdown((response.choices[0].message.content)))"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "58a9169b",
|
|
"metadata": {},
|
|
"source": [
|
|
"Just like before we get an answer! This time via the OpenAI client. Fun times."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "47f9dbbd",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Wrapping up\n",
|
|
"\n",
|
|
"This notebook showed how to connect TxtAI and OpenCode. This will lead to some interesting integrations. One such example is [ncoder](https://github.com/neuml/ncoder), an AI coding agent that integrates with Jupyter Notebooks. Stay tuned for more!"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "local",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.19"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|