wehub/pathwaycom--llm-app
1
0
Fork 0
Code Issues Pull Requests Actions 1 Deployments Models Agent Notes Packages Projects Releases Wiki Activity

docs: make Chinese README the default
lint PR / linter (push) Waiting to run
Details

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 10:12:05 +00:00
parent 8b83f8468e
commit 4b7747cc63
1 changed files with 47 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+47 -42
View File
@@ -1,89 +1,94 @@
<!-- WEHUB_ZH_README -->
> [!NOTE]
> 本文档由 WeHub 基于上游 README 翻译整理，属于社区翻译，非官方中文文档。
> [English](./README.en.md) · [原始项目](https://github.com/pathwaycom/llm-app) · [上游 README](https://github.com/pathwaycom/llm-app/blob/HEAD/README.md)
> 原作者、版权与许可证归属以原始项目及本仓库 LICENSE 文件为准。
<div align="center">
# Pathway Live Data Framework AI Pipelines
# Pathway Live Data Framework AI 流水线
<a href="https://trendshift.io/repositories/4400" target="_blank"><img src="https://trendshift.io/api/badge/repositories/4400" alt="pathwaycom%2Fllm-app | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
![Linux](https://img.shields.io/badge/Linux-FCC624?style=for-the-badge&logo=linux&logoColor=black)
![macOS](https://img.shields.io/badge/mac%20os-000000?style=for-the-badge&logo=apple&logoColor=white)
[![chat on Discord](https://img.shields.io/badge/Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white)](https://discord.gg/pathway)
[![follow on X]( https://img.shields.io/badge/X-000000?style=for-the-badge&logo=x&logoColor=white)](https://x.com/intent/follow?screen_name=pathway_com)
[![在 Discord 上聊天](https://img.shields.io/badge/Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white)](https://discord.gg/pathway)
[![在 X 上关注]( https://img.shields.io/badge/X-000000?style=for-the-badge&logo=x&logoColor=white)](https://x.com/intent/follow?screen_name=pathway_com)
</div>
The Pathway Live Data Framework's **AI Pipelines** allow you to quickly put in production AI applications that offer **high-accuracy RAG and AI enterprise search at scale** using the most **up-to-date knowledge** available in your data sources. It provides you ready-to-deploy **LLM (Large Language Model) App Templates**. You can test them on your own machine and deploy on-cloud (GCP, AWS, Azure, Render,...) or on-premises.
Pathway Live Data Framework 的 **AI Pipelines（AI 流水线）** 可让你快速将 AI 应用投入生产；这些应用利用数据源中最**最新的知识**，在规模化场景下提供**高精度的 RAG（Retrieval-Augmented Generation，检索增强生成）与企业级 AI 搜索**。它为你提供开箱即用的 **LLM（Large Language Model，大语言模型）应用模板**。你可以在本地机器上测试它们，并部署到云端（GCP、AWS、Azure、Render 等）或本地环境。
The apps connect and sync (all new data additions, deletions, updates) with data sources on your **file system, Google Drive, Sharepoint, S3, Kafka, PostgreSQL, real-time data APIs**. They come with no infrastructure dependencies that would need a separate setup. They include **built-in data indexing** enabling vector search, hybrid search, and full-text search - all done in-memory, with cache.
这些应用会连接并与数据源同步（包括所有新增、删除、更新），数据源涵盖**文件系统、Google Drive、Sharepoint、S3、Kafka、PostgreSQL、实时数据 API**。它们不依赖需要单独搭建的基础设施。它们内置**数据索引**，支持向量搜索、混合搜索和全文搜索——全部在内存中完成，并带有缓存。
## Application Templates
## 应用模板
The application templates provided in this repo scale up to **millions of pages of documents**. Some of them are optimized for simplicity, some are optimized for amazing accuracy. Pick the one that suits you best. You can use it out of the box, or change some steps of the pipeline - for example, if you would like to add a new data source, or change a Vector Index into a Hybrid Index, it's just a one-line change.
本仓库提供的应用模板可扩展至**数百万页文档**。其中一些为简洁性而优化，另一些为出色精度而优化。选择最适合你的那一款。你可以开箱即用，也可以修改流水线的某些步骤——例如，如果你想添加新数据源，或将 Vector Index 改为 Hybrid Index，只需一行代码即可。
| Application (template) | Description |
| --------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [`Question-Answering RAG App`](templates/question_answering_rag/) | Basic end-to-end RAG app. A question-answering pipeline that uses the GPT model of choice to provide answers to queries to your documents (PDF, DOCX,...) on a live connected data source (files, Google Drive, Sharepoint,...). You can also try out a [demo REST endpoint](https://pathway.com/solutions/rag-pipelines#try-it-out). |
| [`Live Document Indexing (Vector Store / Retriever)`](templates/document_indexing/) | A real-time document indexing pipeline for RAG that acts as a vector store service. It performs live indexing on your documents (PDF, DOCX,...) from a connected data source (files, Google Drive, Sharepoint,...). It can be used with any frontend, or integrated as a retriever backend for a [Langchain](https://pathway.com/blog/langchain-integration) or [Llamaindex](https://pathway.com/blog/llamaindex-pathway) application. You can also try out a [demo REST endpoint](https://pathway.com/solutions/ai-contract-management#try-it-out). |
| [`Multimodal RAG pipeline with GPT4o`](templates/multimodal_rag/) | Multimodal RAG using GPT-4o in the parsing stage to index PDFs and other documents from a connected data source files, Google Drive, Sharepoint,...). It is perfect for extracting information from unstructured financial documents in your folders (including charts and tables), updating results as documents change or new ones arrive.|
| [`Unstructured-to-SQL pipeline + SQL question-answering`](templates/unstructured_to_sql_on_the_fly/) | A RAG example which connects to unstructured financial data sources (financial report PDFs), structures the data into SQL, and loads it into a PostgreSQL table. It also answers natural language user queries to these financial documents by translating them into SQL using an LLM and executing the query on the PostgreSQL table. |
| [`Adaptive RAG App`](templates/adaptive_rag/) | A RAG application using Adaptive RAG, a technique developed by Pathway to reduce token cost in RAG up to 4x while maintaining accuracy. |
| [`Private RAG App with Mistral and Ollama`](templates/private_rag/) | A fully private (local) version of the `question_answering_rag` RAG pipeline using Pathway Live Data Framework, Mistral, and Ollama. |
| [`Slides AI Search App`](templates/slides_ai_search/) | An indexing pipeline for retrieving slides. It performs multi-modal of PowerPoint and PDF and maintains live index of your slides."|
| [`Video RAG with TwelveLabs`](templates/video_rag_twelvelabs/) | A RAG pipeline over **video**. It uses [TwelveLabs](https://twelvelabs.io) Pegasus to turn videos into rich text descriptions and Marengo multimodal embeddings to index them, so you can ask questions about your videos on a live connected data source (files, Google Drive,...). |
| [`Question-Answering RAG App`](templates/question_answering_rag/) | 基本端到端 RAG 应用。一个问答流水线，使用你选择的 GPT 模型，针对连接到实时数据源（文件、Google Drive、Sharepoint 等）的文档（PDF、DOCX 等）中的查询提供答案。你也可以试用 [demo REST endpoint](https://pathway.com/solutions/rag-pipelines#try-it-out). |
| [`Live Document Indexing (Vector Store / Retriever)`](templates/document_indexing/) | 用于 RAG 的实时文档索引流水线，可作为向量存储服务。它对来自已连接数据源（文件、Google Drive、Sharepoint 等）的文档（PDF、DOCX 等）进行实时索引。可与任何前端配合使用，或作为 [Langchain](https://pathway.com/blog/langchain-integration) 或 [Llamaindex](https://pathway.com/blog/llamaindex-pathway) 应用的检索后端集成。你也可以试用 [demo REST endpoint](https://pathway.com/solutions/ai-contract-management#try-it-out). |
| [`Multimodal RAG pipeline with GPT4o`](templates/multimodal_rag/) | 多模态 RAG，在解析阶段使用 GPT-4o，从已连接的数据源（文件、Google Drive、Sharepoint 等）索引 PDF 及其他文档。非常适合从文件夹中的非结构化金融文档（包括图表和表格）中提取信息，并在文档变更或新增时更新结果。|
| [`Unstructured-to-SQL pipeline + SQL question-answering`](templates/unstructured_to_sql_on_the_fly/) | 一个 RAG 示例，连接非结构化金融数据源（金融报告 PDF），将数据结构化并存入 SQL，加载到 PostgreSQL 表中。它还通过使用 LLM 将自然语言用户查询翻译为 SQL，并在 PostgreSQL 表上执行查询，从而回答有关这些金融文档的问题。 |
| [`Adaptive RAG App`](templates/adaptive_rag/) | 使用 Adaptive RAG 的 RAG 应用，这是 Pathway 开发的一种技术，可在保持精度的同时将 RAG 中的 token 成本降低多达 4 倍。 |
| [`Private RAG App with Mistral and Ollama`](templates/private_rag/) | `question_answering_rag` RAG 流水线的完全私有（本地）版本，使用 Pathway Live Data Framework、Mistral 和 Ollama。 |
| [`Slides AI Search App`](templates/slides_ai_search/) | 用于检索幻灯片的索引流水线。它对 PowerPoint 和 PDF 进行多模态处理，并维护幻灯片的实时索引。"|
| [`Video RAG with TwelveLabs`](templates/video_rag_twelvelabs/) | 基于**视频**的 RAG 流水线。它使用 [TwelveLabs](https://twelvelabs.io) Pegasus 将视频转换为丰富的文本描述，并使用 Marengo 多模态嵌入进行索引，这样你就可以针对连接到实时数据源（文件、Google Drive 等）的视频提问。 |
## How do these AI Pipelines work?
## 这些 AI 流水线如何工作？
The apps can be run as **Docker containers**, and expose an **HTTP API** to connect the frontend. To allow quick testing and demos, some app templates also include an optional Streamlit UI which connects to this API.
这些应用可作为 **Docker 容器**运行，并对外暴露 **HTTP API** 以连接前端。为便于快速测试和演示，部分应用模板还包含可选的 Streamlit UI，用于连接该 API。
The apps rely on the [Pathway Live Data Framework](https://github.com/pathwaycom/pathway) for data source synchronization and for serving API requests (Pathway is a standalone Python library with a Rust engine built into it). They bring you a **simple and unified application logic** for back-end, embedding, retrieval, LLM tech stack. There is no need to integrate and maintain separate modules for your Gen AI app: ~Vector Database (e.g. Pinecone/Weaviate/Qdrant) + Cache (e.g. Redis) + API Framework (e.g. Fast API)~. Pathway's default choice of **built-in vector index** is based on the lightning-fast [usearch](https://github.com/unum-cloud/usearch) library, and **hybrid full-text indexes** make use of [Tantivy](https://github.com/quickwit-oss/tantivy) library. Everything works out of the box.
这些应用依赖 [Pathway Live Data Framework](https://github.com/pathwaycom/pathway) 进行数据源同步并处理 API 请求（Pathway 是一个独立的 Python 库，内置 Rust 引擎）。它们为你提供后端、嵌入、检索、LLM 技术栈的**简单统一的应用逻辑**。无需为 Gen AI 应用集成和维护独立模块：~向量数据库（如 Pinecone/Weaviate/Qdrant）+ 缓存（如 Redis）+ API 框架（如 Fast API）~。Pathway 默认选择的**内置向量索引**基于极速 [usearch](https://github.com/unum-cloud/usearch) 库，**混合全文索引**则使用 [Tantivy](https://github.com/quickwit-oss/tantivy) 库。一切开箱即用。
## Getting started
## 入门
Each of the [App templates](templates/) in this repo contains a README.md with instructions on how to run it.
本仓库中的每个[应用模板](templates/)都包含 README.md，其中说明了如何运行。
You can also find [more ready-to-run code templates](https://pathway.com/developers/templates/) on the Pathway website.
你也可以在 Pathway 网站上找到[更多开箱即用的代码模板](https://pathway.com/developers/templates/)。
## Some visual highlights
## 部分视觉亮点
Effortlessly extract and organize table and chart data from PDFs, docs, and more with multimodal RAG - in real-time:
使用多模态 RAG，实时轻松提取并整理 PDF、文档等中的表格和图表数据：
![Effortlessly extract and organize table and chart data from PDFs, docs, and more with multimodal RAG - in real-time](https://github.com/pathwaycom/llm-app/blob/main/templates/multimodal_rag/gpt4o_with_pathway_comparison.gif)
![使用多模态 RAG，实时轻松提取并整理 PDF、文档等中的表格和图表数据](https://github.com/pathwaycom/llm-app/blob/main/templates/multimodal_rag/gpt4o_with_pathway_comparison.gif)
(Check out [`Multimodal RAG pipeline with GPT4o`](templates/multimodal_rag/) to see the whole pipeline in the works. You may also check out the [`Unstructured-to-SQL pipeline`](templates/unstructured_to_sql_on_the_fly/) for a minimal example that works with non-multimodal models as well.)
（查看 [`Multimodal RAG pipeline with GPT4o`](templates/multimodal_rag/) 了解完整流水线的运行方式。你也可以查看 [`Unstructured-to-SQL pipeline`](templates/unstructured_to_sql_on_the_fly/)，这是一个可与非多模态模型配合使用的精简示例。）
Automated real-time knowledge mining and alerting:
自动化实时知识挖掘与告警：
![Automated real-time knowledge mining and alerting](templates/drive_alert/drive_alert_demo.gif)
![自动化实时知识挖掘与告警](templates/drive_alert/drive_alert_demo.gif)
(Check out the [`Alerting when answers change on Google Drive`](https://github.com/pathwaycom/llm-app/tree/main/templates/drive_alert) app example.)
（查看 [`Alerting when answers change on Google Drive`](https://github.com/pathwaycom/llm-app/tree/main/templates/drive_alert) 应用示例。）
### 动手实践视频
▶️ [使用 Pathway Live Data Framework 构建 LLM 应用简介](https://www.youtube.com/watch?v=kcrJSk00duw) - by [Jan Chorowski](https://scholar.google.com/citations?user=Yc94070AAAAJ)
▶️ [11 分钟搭建真实场景 LLM 应用](https://www.youtube.com/watch?v=k1XGo7ts4tI) - by [Pau Labarta Bajo](https://substack.com/@paulabartabajo)
### Do-it-Yourself Videos
## 故障排除
▶️ [An introduction to building LLM apps with Pathway Live Data Framework](https://www.youtube.com/watch?v=kcrJSk00duw) - by [Jan Chorowski](https://scholar.google.com/citations?user=Yc94070AAAAJ)
如需提供反馈或报告 bug，请在[我们的 issue 跟踪器上提交 issue](https://github.com/pathwaycom/pathway/issues).
▶️ [Let's build a real-world LLM app in 11 minutes](https://www.youtube.com/watch?v=k1XGo7ts4tI) - by [Pau Labarta Bajo](https://substack.com/@paulabartabajo)
## 贡献
我们非常欢迎任何人为本项目贡献力量，无论是文档、功能、bug 修复、代码清理、测试还是代码审查。如果这是你第一次向 GitHub 项目做贡献，可以参考这份[入门指南](https://docs.github.com/en/get-started/quickstart/contributing-to-projects).
## Troubleshooting
如果你想贡献的内容需要更多协作，欢迎在 [Pathway Discord 服务器](https://discord.com/invite/pathway) (#get-help) 中举手示意，并告诉我们你的计划！
To provide feedback or report a bug, please [raise an issue on our issue tracker](https://github.com/pathwaycom/pathway/issues).
## Contributing
Anyone who wishes to contribute to this project, whether documentation, features, bug fixes, code cleanup, testing, or code reviews, is very much encouraged to do so. If this is your first contribution to a GitHub project, here is a [Get Started Guide](https://docs.github.com/en/get-started/quickstart/contributing-to-projects).
If you'd like to make a contribution that needs some more work, just raise your hand on the [Pathway Discord server](https://discord.com/invite/pathway) (#get-help) and let us know what you are planning!
## Supported and maintained by
## 支持与维护方
<p align="center">
<a href="https://github.com/pathwaycom/"><img src="https://pathway.com/logo-light.svg" alt="Pathway"/></a>
</p>
<p align="center">
<a href="https://pathway.com/solutions/llm-app">
<img src="https://img.shields.io/badge/See%20Pathway's%20offering%20for%20AI%20applications-0000FF" alt="See Pathway's offering for AI applications"/>
<img src="https://img.shields.io/badge/See%20Pathway's%20offering%20for%20AI%20applications-0000FF" alt="了解 Pathway 的 AI 应用产品与服务"/>
</a>
</p>