wehub/llmware-ai--llmware
1
0
Fork 0
Code Issues Pull Requests Actions 1 Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution
Documentation / build (push) Waiting to run
Details
Documentation / deploy (push) Blocked by required conditions
Details

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 13:34:55 +08:00
commit 86db9aae8e
563 changed files with 137401 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/pages.yml
+86
View File
@@ -0,0 +1,86 @@
# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.
# Sample workflow for building and deploying a Jekyll site to GitHub Pages
name: Documentation
on:
push:
branches: ["main"]
paths: ["docs/**"] # only changes in the docs directory triger the workflow
pull_request:
branches: ["main"]
types: ["closed"]
paths: ["docs/**"]
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write
# Allow one concurrent deployment
concurrency:
group: "pages"
cancel-in-progress: true
jobs:
# Build job
build:
runs-on: ubuntu-latest
defaults:
run:
working-directory: docs
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Setup Ruby
uses: ruby/setup-ruby@v1
with:
ruby-version: '3.3' # Not needed with a .ruby-version file
bundler-cache: true # runs 'bundle install' and caches installed gems automatically
cache-version: 0 # Increment this number if you need to re-download cached gems
working-directory: '${{ github.workspace }}/docs'
- name: Setup Pages
id: pages
uses: actions/configure-pages@v3
- name: Build with Jekyll
# Outputs to the './_site' directory by default
run: bundle exec jekyll build --baseurl "${{ steps.pages.outputs.base_path }}"
env:
JEKYLL_ENV: production
- name: Upload artifact
# Automatically uploads an artifact from the './_site' directory by default
uses: actions/upload-pages-artifact@v3
with:
path: "docs/_site/"
# Deployment job
deploy:
permissions:
contents: read
pages: write
id-token: write
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
needs: build
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
.gitignore
+19
View File
@@ -0,0 +1,19 @@
# Backup fiels
~*
# Python build / packaging
build/
dist/
__pycache__/
llmware.egg-info/
venv/
# Jekyll
docs/_site/
# Mac
.DS_Store/
# Testing
.env
.python-version
+1
View File
@@ -0,0 +1 @@
3.12.0
LICENSE
+201
View File
@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright 2023-2026 by AI Bloks for LLMWare
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
NOTICE
+670
View File
@@ -0,0 +1,670 @@
Copyright 2023-2026 by llmware
NOTICE - Overview
LLMWare Models. This software contains links to the LLMWare public model repository currently hosted on Huggingface (www.huggingface.co/llmware). LLMWare models in this repository are generally licensed under a separate Apache License 2.0 license, e.g., https://www.apache.org/licenses/LICENSE-2.0. Where there are any exceptions to Apache 2.0, it is noted on the Model Card, e.g., Llama 2 Community License or CC-BY-SA-4.0, and is following licensing conditions of the underlying base models. All of the LLMWare models that are listed in the ModelCatalog in this repository are permissively licensed and may be used for commercial purposes in conjunction with llmware, subject to the license terms found with the model cards.
3rd Party Models. The llmware software package also provides the ability to use other 3rd party models in conjunction with the llmware software. Use of any third party models is subject to the licensing terms of those models.
Sample Testing Files. This software includes sample files that are made available for testing, including documents and audio files. These files have either been derived from open public domain sources, or produced as original materials by the llmware team to be used as test samples and examples. These files are made available solely for purposes of testing and illustrating the use of key llmware functions, and should not be used, distributed or published in any other manner. The audio files, in particular, may be subject to copyright protections that preclude any other form of publishing beyond limited 'fair use' as testing files.
Compiled Enabling Components. This software consists primarily of Python source code, but also includes back-end enabling components consisting of prebuilt compiled C/C++ shared libraries and supporting 3rd party dependencies:
* libpdf_llmware - this is a PDF parser written in C developed and copyrighted separately by llmware, which implements the ISO-32000 specification. The compiled parser is provided as a shared library as part of the llmware software package, under an Apache 2.0 license, but the source code has not been provided in the llmware project currently. The interfaces to the shared library are in the Python source code in the Parser module, which enable the user to access, configure and control the behavior of the PDF parser as part of building applications and other solutions with llmware.
* liboffice_llmware - this is an Office XML parser written in C developed and copyrighted by llmware, which implements the ISO-29500 Office Open XML specification for documents that conform with Word, Powerpoint and Excel formats. The compiled parser is provided as a shared library as part of the llmware software package, under an Apache 2.0 license, but the source code has not been provided in the llmware project currently. The interfaces to the shared library are in the Python source code in the Parser module, which enable the user to access, configure and control the behavior of the Office parser as part of building applications and other solutions with llmware.
* libgraph_llmware - this module provides a set of NLP-based utility functions, written in C developed and copyrighted by llmware, and exposed in the Python code in the graph module. The compiled code is provided as a shared library as part of the llmware software package, under an Apache 2.0 license, but the source code has not been released in the llmware project currently.
* llama.cpp / GGUF - this a prebuilt shared library implementation of Llama.CPP and associated GGML middleware components, licensed under a MIT license with the source code and all other information in the repository: www.github.com/ggerganov/llama.cpp.git. The llmware implementation generally follows the core Python interface for Llama.CPP as provided in the repository: www.github.com/abetlen/llama-cpp-python.git, although there will be differences from time-to-time, and llmware may have additional features not found in other Python interfaces and vice versa. The llmware shared library implementation is regularly updated and re-compiled, and tested for integration into llmware, but may depart from the current source code of llama.cpp.
* whisper.cpp / GGML - this is a prebuilt shared library implementation of Whisper.CPP and associated GGML middleware components, licensed under a MIT license with the source code and all other information in the repository: www.github.com/ggerganov/whisper.cpp.git. The llmware Python interface for Whisper.CPP was developed originally, but took inspiration from www.github.com/carloscdias/whisper-cpp-python.git, which is provided under a MIT license. The llmware shared library implementation is regularly updated and recompiled, and tested for integration into llmware, but may depart from the current source code of whisper.cpp.
Why shared libraries rather than source code? The objective of including these prebuilt shared libraries in this project is to enable a complete LLM-based pipeline development framework that works "out of the box" on multiple platforms, and also enables a developer to rapidly and intuitively scale to large, complex solutions. We view these shared libraries as enabling components with the core of the llmware project consisting of the higher-level classes and functions exposed in the Python source code. In the future, we may revisit the decision to publish the source code for the llmware parsers (e.g., libpdf_llmware, liboffice_llmware, libgraph_llmware), and/or provide under a separate project repository focused on lower-level components and in C/C++. Generally, these parsers are complex, standalone, low-level code that implements the arcane rules of various ISO standards, and for the foreseeable future, we do not have the bandwidth to manage a full open source documentation, testing, and lifecycle around that source code base - without it overwhelming the main objective of llmware.
Vector Databases. This software provides integration to a wide variety of vector database resources, using the open source Python drivers and associated SDKs provided by the vector database vendor with the licensing details outlined below. Users need to install any vector databases separately and independently, subject to the licensing terms from those vendors. llmware does not provide any licenses to vector databases, and llmware does not require the use of a vector database for a wide range of use cases. In several Fast Start examples, we demonstrate how to use "no-install" vector databases, such as FAISS (MIT license), ChromaDB (Apache 2.0 license - www.github.com/chroma-core/chroma.git) and LanceDB (Apache 2.0 license - www.github.com/lancedb/lancedb.git), but no license is provided to those separate resources.
General Purpose Databases. This software provides integration to three core database resources - Postgres, MongoDB and SQLite. llmware connects to these resources using open source Python and C drivers. Users need to install these databases separately and indendently, subject to the licensing terms from those vendors. llmware does not provide any licenses to databases, and llmware does not require the use of a database for a wide range of use cases.
Huggingface Integration. This software provides features and functions that enable access to models, tokenizers and datasets hosted in Huggingface repositories and accessible via the Huggingface transformers, tokenizers, datasets and huggingface-hub libraries. In providing interfaces to these resources, llmware uses transformers, in particular, which is provided by Huggingface under an Apache 2.0 license (see www.github.com/huggingface/transformers.git). In addition, related to these interfaces, llmware provides code that was influenced, derived, and in a few limited cases, copied, from transformers source code to facilitate the integration. Some of this transformers code, especially the underlying model class code, is subject to additional copyright notices from HuggingFace, Google AI, EleutherAI, NVIDIA and potentially others for specific models. For use of any individual third party model, accessed through a Huggingface repository, we recommend reviewing the individual model card to confirm the licensing terms and other associated copyrights.
=================================================================================================
Open Source Dependencies and Optional Components
Please note that the list below includes 'first-level' dependencies but does not include potential 'second-level' dependencies included within the components outlined below.
3rd Party Open Source libraries, drivers and tools in Python (pip install) and C/C++ dependencies used in conjunction for the llmware software package:
3-clause BSD License (https://opensource.org/license/bsd-3-clause/)
* Software: libzip (https://libzip.org/) [C library]
* Software: numpy (https://github.com/numpy/numpy) [Python pip install]
* Software: torch (https://github.com/pytorch/pytorch) [Python pip install]
* Software: colorama (https://github.com/tartley/colorama) [Python pip install]
* Software - Optional - lxml (https://github.com/lxml/lxml) [Optional / Python pip install]
Apache License 2.0 (https://www.apache.org/licenses/LICENSE-2.0)
* Software: boto3 (https://github.com/boto/boto3) [Python pip install]
* Software: mongo-c-driver (https://github.com/mongodb/mongo-c-driver) [C library]
* Software: pymongo (http://github.com/mongodb/mongo-python-driver) [Python pip install]
* Software: tokenizers (https://github.com/huggingface/tokenizers) [Python pip install]
* Software: transformers (https://github.com/huggingface/transformers) [Python pip install]
* Software: huggingface-hub (https://github.com/huggingface/huggingface-hub) [Python pip install]
* Software: requests (https://github.com/psf/requests) [Python pip install]
* Software - Optional - pymilvus (https://github.com/milvus-io/pymilvus) [Optional / Python pip install]
* Software - Optional - pytesseract (https://github.com/madmaze/pytesseract) [Optional / Python pip install]
* Software - Optional - datasets (https://github.com/huggingface/datasets) [Optional / Python pip install]
* Software - Optional - sentence-transformers (https://github.com/UKPLabs/sentence-transformers) [Optional / Python pip install]
* Software - Optional - yfinance (https://github.com/ranaroussi/yfinance) [Optional / Python pip install]
* Software - Optional - chromadb (https://github.com/chroma-core/chroma) [Optional / Python pip install]
* Optional - Optional - neo4j-python-driver (https://github.com/neo4j/neo4j-python-driver) [Optional / Python pip install]
* Optional - Optional - Tiny-Llama base model (https://github.com/jzhang38/TinyLlama) [Separate Download and install]
* Optional - Optional - tesseract (https://github.com/tesseract-ocr/tesseract) [Optional / not included / C library]
ICS License (https://opensource.org/license/isc-license-txt)
* Software: librosa - source code and license at: https://github.com/librosa/librosa [Python pip install]
Database Drivers used in llmware
* Mongo C driver - libmongoc / libbson - Apache 2.0 (https://www.github.com/mongodb/mongo-c-driver) [C library]
* PostgreSQL C driver - libpq - PostgresSQL Global Development Group (https://www.github.com/postgres/postgres/blob/master/copyright) [C library]
* PostgreSQL psycopg - unmodified and linked dynamically from pypi release - GNU Lesser General Public License (https://www.github.com/psyocopg/psycopg/LICENSE.txt) [Python pip install]
* SQLite C driver - libsqlite3 - (https://www.github.com/LuaDist/libsqlite3) [C library]
* psycopg2 & psycopg2-binary - GNU Lesser General Public License - no modifications - it is provided through pip install as standard python library
PNG Reference Library License
* libpng: source code and license (https://github.com/pnggroup/libpng) [C library]
libtiff License (https://spdx.org/licenses/libtiff.html)
* Software: libtiff (http://www.libtiff.org/) [C library]
MIT License (https://opensource.org/license/mit/)
* Software: libxml2 (https://github/com/GNOME/libxml2) [C library]
* Software: llama.cpp (https://github.com/ggerganov/llama.cpp) [C/C++ library]
* Software: whisper.cpp (https://github.com/ggerganov/whisper.cpp) [C/C++ library]
* Software: openai (https://github.com/openai/openai-python) [Python pip install]
* Software: Wikipedia-API (https://github.com/martin-majlis/Wikipedia-API) [Python pip install]
* Software: einops (https://github.com/arogozhinov/einops) [Python pip install]
* Software - Optional - beautifulsoup4 (https://pypi.org/project/beautifulsoup4/) [Optional / Python pip install]
* Software - Optional - Whisper models - openai (https://github.com/openai/whisper) [Separate download and install]
* Software - Optional - anthropic (https://github.com/anthropics/anthropic-sdk-python) [Optional / Python pip install]
* Software - Optional - faise-cpu (https://github.com/kyamagu/faiss-wheels) [Optional / Python pip install]
* Software - Optional - cohere (https://github.com/cohere-ai/cohere-python) [Optional / Python pip install]
* Software - Optional - tabulate (https://github.com/astanin/python-tabulate) [Optional / Python pip install]
* Software - Optional - pdf2image (https://github.com/Belval/pdf2image) [Optional / Python pip install]
* Software - Optional - word2number (https://github.com/akshaynagpal/w2n) [Optional / Python pip install]
zlib License (https://github.com/madler/zlib/blob/develop/LICENSE)
* Software: zlib (https://www.zlib.net/) [C library]
Databases - separate and optional components that may be used in conjunction with llmware and require user to license and install independently:
* Postgres: source code for Postgres database provided at: https://github.com/postgres. Copyright is by PostgreSQL Global Development Group and Regents of the University of California - (https://github.com/postgres/postgres/blob/master/copyright) - with license terms outlined below in the Copyright Notices section. [Separate download and install]
* MongoDB: source code for Community edition provided at: https://github.com/mongodb/mongo - "MongoDB is free and the source is available" subject to the Server Side Public License (SSPL) v1 - (https://github.com/mongodb/mongo/blob/master/LICENSE-Community.txt). [Separate download and install]
* SQLite: source code provided at: https://github.com/sqlite/sqlite - not subject to copyright - "May you share freely, never taking more than you give." [Separate download and install]
Vector Databases - separate and optional open source components that may be used in conjunction with llmware and require user to license and install independently:
* Milvus: Apache 2.0 license - source code and license at: https://github.com/milvus-io/milvus
* Qdrant: Apache 2.0 license - source code and license at: https://github.com/qdrant/qdrant
* ChromaDB: Apache 2.0 license - source code and license at: https://github.com/chroma-core/chroma
* PGVector: Postgres Development Group license - source code and license at: https://github.com/pgvector/pgvector
* Neo4J: GPL-3.0 license - source and license at: https://github.com/neo4j/neo4j
* FAISS: MIT license - source code and license at: https://github.com/facebookresearch/faiss
* LanceDB: Apache 2.0 license - source code and license at: www.github.com/lancedb/lancedb.git
=================================================================================================
Required Copyright Notices Per Open Source Licenses for Components Distributed Directly with a Full Clone of the Repository - does not include components installed through a separate pip install process, or additional components used in conjunction with LLMWare, but installed independently from this repository.
Name: llama.cpp
License: MIT
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Name: whisper.cpp
License: MIT
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Name: libxml2
License: MIT
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Name: einops
License: MIT
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Name: wikipedia-api
License: MIT
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Name: openai (python sdk)
License: MIT
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Name: mongo-c-driver (libmongoc and libbson)
License: Apache 2.0 Copyright Notice
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Name: pymongo
License: Apache 2.0 Copyright Notice
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Name: boto3
License: Apache 2.0 Copyright Notice
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Name: requests
License: Apache 2.0 Copyright Notice
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Name: transformers (Huggingface)
License: Apache 2.0 Copyright Notice
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Name: tokenizers (Huggingface)
License: Apache 2.0 Copyright Notice
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Name: huggingface_hub (HuggingFace)
License: Apache 2.0 Copyright Notice
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Name: librosa
License: ISC License
Copyright (c) 2013--2023, librosa development team.
Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
Name: psycopg [no modifications - it is provided through pip install as standard python library]
License: Lesser GPL
psycopg2 is free software: you can redistribute it and/or modify it
under the terms of the GNU Lesser General Public License as published
by the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
psycopg2 is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public
License for more details.
In addition, as a special exception, the copyright holders give
permission to link this program with the OpenSSL library (or with
modified versions of OpenSSL that use the same license as OpenSSL),
and distribute linked combinations including the two.
You must obey the GNU Lesser General Public License in all respects for
all of the code used other than OpenSSL. If you modify file(s) with this
exception, you may extend this exception to your version of the file(s),
but you are not obligated to do so. If you do not wish to do so, delete
this exception statement from your version. If you delete this exception
statement from all source files in the program, then also delete it here.
You should have received a copy of the GNU Lesser General Public License
along with psycopg2 (see the doc/ directory.)
If not, see <https://www.gnu.org/licenses/>.
Name: numpy
License: BSD-3-Clause
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Name: torch
License: BSD-3-Clause
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Name: colorama
License: BSD-3-Clause
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Name: libzip
License: BSD-3-Clause
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Name: pgvector
License: Postgres License
Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
Portions Copyright (c) 1994, The Regents of the University of California
Permission to use, copy, modify, and distribute this software and its
documentation for any purpose, without fee, and without a written agreement
is hereby granted, provided that the above copyright notice and this
paragraph and the following two paragraphs appear in all copies.
IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR
DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING
LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS
DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS
ON AN "AS IS" BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATIONS TO
PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
Name: libpq
License: GNU LESSER GENERAL PUBLIC LICENSE & Postgres License
PostgreSQL License
PostgreSQL is released under the PostgreSQL License, a liberal Open Source license, similar to the BSD or MIT licenses.
PostgreSQL Database Management System
(formerly known as Postgres, then as Postgres95)
Portions Copyright © 1996-2024, The PostgreSQL Global Development Group
Portions Copyright © 1994, The Regents of the University of California
Permission to use, copy, modify, and distribute this software and its documentation for any purpose, without fee, and without a written agreement is hereby granted, provided that the above copyright notice and this paragraph and the following two paragraphs appear in all copies.
IN NO EVENT SHALL THE UNIVERSITY OF CALIFORNIA BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES, INCLUDING LOST PROFITS, ARISING OUT OF THE USE OF THIS SOFTWARE AND ITS DOCUMENTATION, EVEN IF THE UNIVERSITY OF CALIFORNIA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
THE UNIVERSITY OF CALIFORNIA SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE SOFTWARE PROVIDED HEREUNDER IS ON AN "AS IS" BASIS, AND THE UNIVERSITY OF CALIFORNIA HAS NO OBLIGATIONS TO PROVIDE MAINTENANCE, SUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.
GNU Lesser General Public License - Version 3, 29 June 2007
Copyright © 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
This version of the GNU Lesser General Public License incorporates the terms and conditions of version 3 of the GNU General Public License, supplemented by the additional permissions listed below.
0. Additional Definitions.
As used herein, “this License” refers to version 3 of the GNU Lesser General Public License, and the “GNU GPL” refers to version 3 of the GNU General Public License.
“The Library” refers to a covered work governed by this License, other than an Application or a Combined Work as defined below.
An “Application” is any work that makes use of an interface provided by the Library, but which is not otherwise based on the Library. Defining a subclass of a class defined by the Library is deemed a mode of using an interface provided by the Library.
A “Combined Work” is a work produced by combining or linking an Application with the Library. The particular version of the Library with which the Combined Work was made is also called the “Linked Version”.
The “Minimal Corresponding Source” for a Combined Work means the Corresponding Source for the Combined Work, excluding any source code for portions of the Combined Work that, considered in isolation, are based on the Application, and not on the Linked Version.
The “Corresponding Application Code” for a Combined Work means the object code and/or source code for the Application, including any data and utility programs needed for reproducing the Combined Work from the Application, but excluding the System Libraries of the Combined Work.
1. Exception to Section 3 of the GNU GPL.
You may convey a covered work under sections 3 and 4 of this License without being bound by section 3 of the GNU GPL.
2. Conveying Modified Versions.
If you modify a copy of the Library, and, in your modifications, a facility refers to a function or data to be supplied by an Application that uses the facility (other than as an argument passed when the facility is invoked), then you may convey a copy of the modified version:
a) under this License, provided that you make a good faith effort to ensure that, in the event an Application does not supply the function or data, the facility still operates, and performs whatever part of its purpose remains meaningful, or
b) under the GNU GPL, with none of the additional permissions of this License applicable to that copy.
3. Object Code Incorporating Material from Library Header Files.
The object code form of an Application may incorporate material from a header file that is part of the Library. You may convey such object code under terms of your choice, provided that, if the incorporated material is not limited to numerical parameters, data structure layouts and accessors, or small macros, inline functions and templates (ten or fewer lines in length), you do both of the following:
a) Give prominent notice with each copy of the object code that the Library is used in it and that the Library and its use are covered by this License.
b) Accompany the object code with a copy of the GNU GPL and this license document.
4. Combined Works.
You may convey a Combined Work under terms of your choice that, taken together, effectively do not restrict modification of the portions of the Library contained in the Combined Work and reverse engineering for debugging such modifications, if you also do each of the following:
a) Give prominent notice with each copy of the Combined Work that the Library is used in it and that the Library and its use are covered by this License.
b) Accompany the Combined Work with a copy of the GNU GPL and this license document.
c) For a Combined Work that displays copyright notices during execution, include the copyright notice for the Library among these notices, as well as a reference directing the user to the copies of the GNU GPL and this license document.
d) Do one of the following:
0) Convey the Minimal Corresponding Source under the terms of this License, and the Corresponding Application Code in a form suitable for, and under terms that permit, the user to recombine or relink the Application with a modified version of the Linked Version to produce a modified Combined Work, in the manner specified by section 6 of the GNU GPL for conveying Corresponding Source.
1) Use a suitable shared library mechanism for linking with the Library. A suitable mechanism is one that (a) uses at run time a copy of the Library already present on the user's computer system, and (b) will operate properly with a modified version of the Library that is interface-compatible with the Linked Version.
e) Provide Installation Information, but only if you would otherwise be required to provide such information under section 6 of the GNU GPL, and only to the extent that such information is necessary to install and execute a modified version of the Combined Work produced by recombining or relinking the Application with a modified version of the Linked Version. (If you use option 4d0, the Installation Information must accompany the Minimal Corresponding Source and Corresponding Application Code. If you use option 4d1, you must provide the Installation Information in the manner specified by section 6 of the GNU GPL for conveying Corresponding Source.)
5. Combined Libraries.
You may place library facilities that are a work based on the Library side by side in a single library together with other library facilities that are not Applications and are not covered by this License, and convey such a combined library under terms of your choice, if you do both of the following:
a) Accompany the combined library with a copy of the same work based on the Library, uncombined with any other library facilities, conveyed under the terms of this License.
b) Give prominent notice with the combined library that part of it is a work based on the Library, and explaining where to find the accompanying uncombined form of the same work.
6. Revised Versions of the GNU Lesser General Public License.
The Free Software Foundation may publish revised and/or new versions of the GNU Lesser General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the Library as you received it specifies that a certain numbered version of the GNU Lesser General Public License “or any later version” applies to it, you have the option of following the terms and conditions either of that published version or of any later version published by the Free Software Foundation. If the Library as you received it does not specify a version number of the GNU Lesser General Public License, you may choose any version of the GNU Lesser General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide whether future versions of the GNU Lesser General Public License shall apply, that proxy's public statement of acceptance of any version is permanent authorization for you to choose that version for the Library.
Name: libpng
License: PNG Reference Library Copyright Noticee and License version 2
* Copyright (c) 1995-2024 The PNG Reference Library Authors.
* Copyright (c) 2018-2024 Cosmin Truta.
* Copyright (c) 2000-2002, 2004, 2006-2018 Glenn Randers-Pehrson.
* Copyright (c) 1996-1997 Andreas Dilger.
* Copyright (c) 1995-1996 Guy Eric Schalnat, Group 42, Inc.
The software is supplied "as is", without warranty of any kind,
express or implied, including, without limitation, the warranties
of merchantability, fitness for a particular purpose, title, and
non-infringement. In no event shall the Copyright owners, or
anyone distributing the software, be liable for any damages or
other liability, whether in contract, tort or otherwise, arising
from, out of, or in connection with the software, or the use or
other dealings in the software, even if advised of the possibility
of such damage.
Permission is hereby granted to use, copy, modify, and distribute
this software, or portions hereof, for any purpose, without fee,
subject to the following restrictions:
1. The origin of this software must not be misrepresented; you
must not claim that you wrote the original software. If you
use this software in a product, an acknowledgment in the product
documentation would be appreciated, but is not required.
2. Altered source versions must be plainly marked as such, and must
not be misrepresented as being the original software.
3. This Copyright notice may not be removed or altered from any
source or altered source distribution.
Name: libtiff
License: Custom (https://github.com/libsdl-org/libtiff)
Silicon Graphics has seen fit to allow us to give this work away. It is free. There is no support or guarantee of any sort as to its operations, correctness, or whatever. If you do anything useful with all or parts of it you need to honor the copyright notices. I would also be interested in knowing about it and, hopefully, be acknowledged.
The legal way of saying that is:
Copyright (c) 1988-1997 Sam Leffler Copyright (c) 1991-1997 Silicon Graphics, Inc.
Permission to use, copy, modify, distribute, and sell this software and its documentation for any purpose is hereby granted without fee, provided that (i) the above copyright notices and this permission notice appear in all copies of the software and related documentation, and (ii) the names of Sam Leffler and Silicon Graphics may not be used in any advertising or publicity relating to the software without the specific, prior written permission of Sam Leffler and Silicon Graphics.
THE SOFTWARE IS PROVIDED "AS-IS" AND WITHOUT WARRANTY OF ANY KIND, EXPRESS, IMPLIED OR OTHERWISE, INCLUDING WITHOUT LIMITATION, ANY WARRANTY OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
IN NO EVENT SHALL SAM LEFFLER OR SILICON GRAPHICS BE LIABLE FOR ANY SPECIAL, INCIDENTAL, INDIRECT OR CONSEQUENTIAL DAMAGES OF ANY KIND, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER OR NOT ADVISED OF THE POSSIBILITY OF DAMAGE, AND ON ANY THEORY OF LIABILITY, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
Name: zlib
License: Custom (https://github.com/madler/zlib?tab=License-1-ov-file)
Copyright notice:
(C) 1995-2024 Jean-loup Gailly and Mark Adler
This software is provided 'as-is', without any express or implied
warranty. In no event will the authors be held liable for any damages
arising from the use of this software.
Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions:
1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software
in a product, an acknowledgment in the product documentation would be
appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.
Jean-loup Gailly Mark Adler
jloup@gzip.org madler@alumni.caltech.edu
=================================================================================================
Citations for Open Source Software, Models and Research used in the development of llmware:
Transformers - https://github.com/huggingface/transformers
@inproceedings{wolf-etal-2020-transformers,
title = "Transformers: State-of-the-Art Natural Language Processing",
author = "Thomas Wolf and Lysandre Debut and Victor Sanh and Julien Chaumond and Clement Delangue and Anthony Moi and Pierric Cistac and Tim Rault and Rémi Louf and Morgan Funtowicz and Joe Davison and Sam Shleifer and Patrick von Platen and Clara Ma and Yacine Jernite and Julien Plu and Canwen Xu and Teven Le Scao and Sylvain Gugger and Mariama Drame and Quentin Lhoest and Alexander M. Rush",
booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
month = oct,
year = "2020",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "https://www.aclweb.org/anthology/2020.emnlp-demos.6",
pages = "38--45"
}
Sentence Transformers - https://github.com/UKPLab/sentence-transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
Pytorch - https://github.com/pytorch/blob/main/CITATION.cff (with full and updated contributor list)
title: PyTorch
authors: PyTorch Team
url: https://pytorch.org
type: conference-paper
title: "PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation"
collection-title: "29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2 (ASPLOS '24)"
collection-type: proceedings
month: 4
year: 2024
publisher:
name: ACM
url: "https://pytorch.org/assets/pytorch2-2.pdf"
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., & Chintala, S. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library [Conference paper]. Advances in Neural Information Processing Systems 32, 8024–8035. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf
Whisper - OpenAI - https://github.com/openai/whisper & www.huggingface.co/openai/whisper
@misc{radford2022whisper,
doi = {10.48550/ARXIV.2212.04356},
url = {https://arxiv.org/abs/2212.04356},
author = {Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
title = {Robust Speech Recognition via Large-Scale Weak Supervision},
publisher = {arXiv},
year = {2022},
copyright = {arXiv.org perpetual, non-exclusive license}
}
BERT
Turc, Iulia; Chang, Ming-Wei; Lee, Kenton; Toutanova, Kristina. Well-Read Students Learn Better: On the Importance of Pre-training Compact Models, arXiv preprint arXiv:1908.08962v2 (2019).
Devlin, Jacob; Chang, Ming-Wei; Lee, Kenton; Toutanova, Kristina. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, arXiv preprint arXiv:1810.04805 (2018).
GPT2
Radford, Alec; Wu, Jeff; Child, Rewon; Luan, David; Amodei, Dario; Sutskever, Ilya. Language Models are Unsupervised Multitask Learners. (2019)
Tiny Llama
@misc{zhang2024tinyllama,
title={TinyLlama: An Open-Source Small Language Model},
author={Peiyuan Zhang and Guangtao Zeng and Tianduo Wang and Wei Lu},
year={2024},
eprint={2401.02385},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
StableLM-3b-4e1t Model (StabilityAI)
@misc{StableLM-3B-4E1T,
url={[https://huggingface.co/stabilityai/stablelm-3b-4e1t](https://huggingface.co/stabilityai/stablelm-3b-4e1t)},
title={StableLM 3B 4E1T},
author={Tow, Jonathan and Bellagente, Marco and Mahan, Dakota and Riquelme, Carlos}
}
Sheared Llama
@article{xia2023sheared, title={Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning}, author={Xia, Mengzhou and Gao, Tianyu, and Zeng Zhiyuan, and Chen Danqi}, year={2023} }
FAISS
@article{douze2024faiss,
title={The Faiss library},
author={Matthijs Douze and Alexandr Guzhva and Chengqi Deng and Jeff Johnson and Gergely Szilvasy and Pierre-Emmanuel Mazaré and Maria Lomeli and Lucas Hosseini and Hervé Jégou},
year={2024},
eprint={2401.08281},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
README.md
+1123
View File
File diff suppressed because it is too large Load Diff
README.wehub.md
+7
View File
@@ -0,0 +1,7 @@
# WeHub 来源说明
- 原始项目：`llmware-ai/llmware`
- 原始仓库：https://github.com/llmware-ai/llmware
- 导入方式：上游默认分支的最新快照
- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准
- 本文件仅用于记录来源，不代表 WeHub 是原项目作者
docs/1.0.4
+94
View File
@@ -0,0 +1,94 @@
Fetching gem metadata from https://rubygems.org/..........
Resolving dependencies...
Resolving dependencies...
Fetching rake 13.2.1
Installing rake 13.2.1
Fetching public_suffix 6.0.1
Fetching bigdecimal 3.1.8
Fetching colorator 1.1.0
Fetching concurrent-ruby 1.3.4
Fetching eventmachine 1.2.7
Fetching uri 1.1.1 (was 1.0.1)
Fetching http_parser.rb 0.8.0
Fetching json 2.8.1
Installing colorator 1.1.0
Fetching logger 1.6.1
Installing uri 1.1.1 (was 1.0.1)
Fetching ffi 1.17.0
Installing bigdecimal 3.1.8 with native extensions
Installing public_suffix 6.0.1
Fetching forwardable-extended 2.6.0
Installing logger 1.6.1
Fetching rb-fsevent 0.11.2
Installing json 2.8.1 with native extensions
Installing eventmachine 1.2.7 with native extensions
Installing forwardable-extended 2.6.0
Fetching rexml 3.3.9
Installing rb-fsevent 0.11.2
Fetching liquid 4.0.4
Installing liquid 4.0.4
Installing rexml 3.3.9
Installing http_parser.rb 0.8.0 with native extensions
Fetching mercenary 0.4.0
Fetching rouge 4.4.0
Installing concurrent-ruby 1.3.4
Installing mercenary 0.4.0
Fetching safe_yaml 1.0.5
Installing safe_yaml 1.0.5
Fetching unicode-display_width 2.6.0
Fetching webrick 1.9.0
Installing unicode-display_width 2.6.0
Fetching net-http 0.5.0
Installing ffi 1.17.0 with native extensions
Installing webrick 1.9.0
Installing net-http 0.5.0
Fetching addressable 2.8.7
Installing rouge 4.4.0
Fetching pathutil 0.16.2
Installing pathutil 0.16.2
Installing addressable 2.8.7
Fetching kramdown 2.4.0
Fetching i18n 1.14.6
Installing kramdown 2.4.0
Installing i18n 1.14.6
Fetching terminal-table 3.0.2
Installing terminal-table 3.0.2
Fetching faraday-net_http 3.3.0
Installing faraday-net_http 3.3.0
Fetching kramdown-parser-gfm 1.1.0
Installing kramdown-parser-gfm 1.1.0
Fetching faraday 2.12.0
Installing faraday 2.12.0
Fetching sawyer 0.9.2
Installing sawyer 0.9.2
Fetching octokit 6.1.1
Installing octokit 6.1.1
Fetching google-protobuf 4.28.3
Installing google-protobuf 4.28.3 with native extensions
Fetching sass-embedded 1.80.6 (x64-mingw-ucrt)
Fetching em-websocket 0.5.3
Installing em-websocket 0.5.3
Installing sass-embedded 1.80.6 (x64-mingw-ucrt)
Fetching jekyll-sass-converter 3.0.0
Installing jekyll-sass-converter 3.0.0
Fetching rb-inotify 0.11.1
Installing rb-inotify 0.11.1
Fetching listen 3.9.0
Installing listen 3.9.0
Fetching jekyll-watch 2.2.1
Installing jekyll-watch 2.2.1
Fetching jekyll 4.3.4
Installing jekyll 4.3.4
Fetching jekyll-default-layout 0.1.5
Fetching jekyll-github-metadata 2.16.1
Fetching jekyll-seo-tag 2.8.0
Fetching jekyll-include-cache 0.2.1
Installing jekyll-include-cache 0.2.1
Installing jekyll-github-metadata 2.16.1
Installing jekyll-seo-tag 2.8.0
Installing jekyll-default-layout 0.1.5
Fetching just-the-docs 0.7.0
Installing just-the-docs 0.7.0
Bundle updated!
Post-install message from i18n:
PSA: I18n will be dropping support for Ruby < 3.2 in the next major release (April 2025), due to Ruby's end of life for 3.1 and below (https://endoflife.date/ruby). Please upgrade to Ruby 3.2 or newer by April 2025 to continue using future versions of this gem.
docs/3.4.2
+5
View File
@@ -0,0 +1,5 @@
Fetching gem metadata from https://rubygems.org/..........
Resolving dependencies...
Resolving dependencies...
Using rexml 3.4.4 (was 3.3.9)
Bundle updated!
docs/Gemfile
+26
View File
@@ -0,0 +1,26 @@
source 'https://rubygems.org'
#
# Gems that are locked to a version.
#
ruby ">=2.7"
gem "jekyll", "~> 4.3.2" # installed by `gem jekyll`
gem "just-the-docs", "0.7.0" # pinned to the current release
gem "rexml", ">= 3.3.9" # pinned to that version to fix security alert
gem "google-protobuf", ">= 3.25.5"
#
# Gems that are only loaded if they are configured correctly.
#
gem 'jekyll-default-layout'
#
# Gems that loaded irrespective of site configuration.
#
gem 'jekyll-seo-tag', group: :jekyll_plugins
gem 'jekyll-include-cache', group: :jekyll_plugins
gem 'jekyll-github-metadata', ">= 2.15", group: :jekyll_plugins
docs/Gemfile.lock
+212
View File
@@ -0,0 +1,212 @@
GEM
remote: https://rubygems.org/
specs:
addressable (2.8.7)
public_suffix (>= 2.0.2, < 7.0)
bigdecimal (3.1.8)
colorator (1.1.0)
concurrent-ruby (1.3.4)
em-websocket (0.5.3)
eventmachine (>= 0.12.9)
http_parser.rb (~> 0)
eventmachine (1.2.7)
faraday (2.12.0)
faraday-net_http (>= 2.0, < 3.4)
json
logger
faraday-net_http (3.3.0)
net-http
ffi (1.17.0)
ffi (1.17.0-aarch64-linux-gnu)
ffi (1.17.0-aarch64-linux-musl)
ffi (1.17.0-arm-linux-gnu)
ffi (1.17.0-arm-linux-musl)
ffi (1.17.0-arm64-darwin)
ffi (1.17.0-x86-linux-gnu)
ffi (1.17.0-x86-linux-musl)
ffi (1.17.0-x86_64-darwin)
ffi (1.17.0-x86_64-linux-gnu)
ffi (1.17.0-x86_64-linux-musl)
forwardable-extended (2.6.0)
google-protobuf (4.28.3)
bigdecimal
rake (>= 13)
google-protobuf (4.28.3-aarch64-linux)
bigdecimal
rake (>= 13)
google-protobuf (4.28.3-arm64-darwin)
bigdecimal
rake (>= 13)
google-protobuf (4.28.3-x86-linux)
bigdecimal
rake (>= 13)
google-protobuf (4.28.3-x86_64-darwin)
bigdecimal
rake (>= 13)
google-protobuf (4.28.3-x86_64-linux)
bigdecimal
rake (>= 13)
http_parser.rb (0.8.0)
i18n (1.14.6)
concurrent-ruby (~> 1.0)
jekyll (4.3.4)
addressable (~> 2.4)
colorator (~> 1.0)
em-websocket (~> 0.5)
i18n (~> 1.0)
jekyll-sass-converter (>= 2.0, < 4.0)
jekyll-watch (~> 2.0)
kramdown (~> 2.3, >= 2.3.1)
kramdown-parser-gfm (~> 1.0)
liquid (~> 4.0)
mercenary (>= 0.3.6, < 0.5)
pathutil (~> 0.9)
rouge (>= 3.0, < 5.0)
safe_yaml (~> 1.0)
terminal-table (>= 1.8, < 4.0)
webrick (~> 1.7)
jekyll-default-layout (0.1.5)
jekyll (>= 3.0, < 5.0)
jekyll-github-metadata (2.16.1)
jekyll (>= 3.4, < 5.0)
octokit (>= 4, < 7, != 4.4.0)
jekyll-include-cache (0.2.1)
jekyll (>= 3.7, < 5.0)
jekyll-sass-converter (3.0.0)
sass-embedded (~> 1.54)
jekyll-seo-tag (2.8.0)
jekyll (>= 3.8, < 5.0)
jekyll-watch (2.2.1)
listen (~> 3.0)
json (2.8.1)
just-the-docs (0.7.0)
jekyll (>= 3.8.5)
jekyll-include-cache
jekyll-seo-tag (>= 2.0)
rake (>= 12.3.1)
kramdown (2.4.0)
rexml
kramdown-parser-gfm (1.1.0)
kramdown (~> 2.0)
liquid (4.0.4)
listen (3.9.0)
rb-fsevent (~> 0.10, >= 0.10.3)
rb-inotify (~> 0.9, >= 0.9.10)
logger (1.6.1)
mercenary (0.4.0)
net-http (0.5.0)
uri
octokit (6.1.1)
faraday (>= 1, < 3)
sawyer (~> 0.9)
pathutil (0.16.2)
forwardable-extended (~> 2.6)
public_suffix (6.0.1)
rake (13.2.1)
rb-fsevent (0.11.2)
rb-inotify (0.11.1)
ffi (~> 1.0)
rexml (3.4.4)
rouge (4.4.0)
safe_yaml (1.0.5)
sass-embedded (1.80.6)
google-protobuf (~> 4.28)
rake (>= 13)
sass-embedded (1.80.6-aarch64-linux-android)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-aarch64-linux-gnu)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-aarch64-linux-musl)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-aarch64-mingw-ucrt)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-arm-linux-androideabi)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-arm-linux-gnueabihf)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-arm-linux-musleabihf)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-arm64-darwin)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-riscv64-linux-android)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-riscv64-linux-gnu)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-riscv64-linux-musl)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-x64-mingw-ucrt)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-x86-cygwin)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-x86-linux-android)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-x86-linux-gnu)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-x86-linux-musl)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-x86-mingw-ucrt)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-x86_64-cygwin)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-x86_64-darwin)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-x86_64-linux-android)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-x86_64-linux-gnu)
google-protobuf (~> 4.28)
sass-embedded (1.80.6-x86_64-linux-musl)
google-protobuf (~> 4.28)
sawyer (0.9.2)
addressable (>= 2.3.5)
faraday (>= 0.17.3, < 3)
terminal-table (3.0.2)
unicode-display_width (>= 1.1.1, < 3)
unicode-display_width (2.6.0)
uri (1.1.1)
webrick (1.9.0)
PLATFORMS
aarch64-linux
aarch64-linux-android
aarch64-linux-gnu
aarch64-linux-musl
aarch64-mingw-ucrt
arm-linux-androideabi
arm-linux-gnu
arm-linux-gnueabihf
arm-linux-musl
arm-linux-musleabihf
arm64-darwin
riscv64-linux-android
riscv64-linux-gnu
riscv64-linux-musl
ruby
x64-mingw-ucrt
x86-cygwin
x86-linux
x86-linux-android
x86-linux-gnu
x86-linux-musl
x86-mingw-ucrt
x86_64-cygwin
x86_64-darwin
x86_64-linux
x86_64-linux-android
x86_64-linux-gnu
x86_64-linux-musl
DEPENDENCIES
google-protobuf (>= 3.25.5)
jekyll (~> 4.3.2)
jekyll-default-layout
jekyll-github-metadata (>= 2.15)
jekyll-include-cache
jekyll-seo-tag
just-the-docs (= 0.7.0)
rexml (>= 3.3.9)
RUBY VERSION
ruby 3.3.6p108
BUNDLED WITH
4.0.3
docs/_config.yml
+54
View File
@@ -0,0 +1,54 @@
# Site settings
# These are used to personalize your new site. If you look in the HTML files,
# you will see them accessed via {{ site.title }}, {{ site.github_repo }}, and so on.
# You can create any custom variable you would like, and they will be accessible
# in the templates via {{ site.myvariable }}.
title: llmware
description: llmware is an enterprise-grade LLM-based development framework, including tools and fine-tuned models.
theme: just-the-docs
url: https://llmware-ai.github.io/llmware/
baseurl: "" # the subpath of your site
favicon_ico: "/assets/images/favicon.ico"
repository: llmware-ai/llmware # for github-metadata
# Enable or disable heading anchors
heading_anchors: true
# Aux links for the upper right navigation
aux_links:
llmware repository: https://github.com/llmware-ai/llmware
lsi: false
safe: true
highlighter: rouge
gist:
noscript: false
kramdown:
math_engine: mathjax
syntax_highlighter: rouge
callouts_level: quiet # or loud
callouts:
highlight:
color: yellow
important:
title: Important
color: blue
new:
title: New
color: green
note:
title: Note
color: purple
warning:
title: Warning
color: red
plugins:
- jekyll-default-layout
- jekyll-seo-tag
- jekyll-github-metadata
- jekyll-include-cache
docs/_includes/footer_custom.html
+43
View File
@@ -0,0 +1,43 @@
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://llmware.ai/">llmware</a>
</li>
<li class="d-inline-block mr-1">
by
</li>
<li class="d-inline-block mr-1">
<a href="https://www.aibloks.com/home">AI Bloks</a>
</li>
<li class="d-inline-block mr-1">
|
</li>
<li class="d-inline-block mr-1 hugging-face-li">
<a href="https://huggingface.co/llmware">
<span>
{% for static_file in site.static_files %}
{% if static_file.basename == "hf-logo"%}
{% assign hf_logo = static_file %}
{% endif %}
{% endfor %}
<img src="{{ site.url }}{{ hf_logo.path }}" alt="Hugging Face" class="hugging-face-logo"/>
</span>
</a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><i class="fa-brands fa-youtube"></i></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><i class="fa-brands fa-linkedin"></i></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><i class="fa-brands fa-square-x-twitter"></i></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><i class="fa-brands fa-instagram"></i></a>
</li>
</ul>
<p>
&copy; 2023-{{ "now" | date: "%Y" }} <a href="https://www.aibloks.com/home">AI Bloks</a>
</p>
docs/_includes/head_custom.html
+65
View File
@@ -0,0 +1,65 @@
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.5.1/css/all.min.css" integrity="sha512-DTOQO9RWCH3ppGqcWaEA1BIZOC6xxalwEsw9c2QQeAIftl+Vegovlnee1c9QX4TctnWMn13TZye+giMm8e2LwA==" crossorigin="anonymous" referrerpolicy="no-referrer" />
<style>
/* Updated font size for better readability */
ul li span {
font-size: 2em;
}
/* Added transition for smoother hover effects */
.fa-youtube:hover {
color: red;
transition: color 0.3s;
}
/*
Code taken from
https://stackoverflow.com/questions/38507734/fontawesome-instagram-icon-colorized
*/
.fa-instagram:hover {
color: transparent;
background: radial-gradient(circle at 30% 107%, #fdf497 0%, #fdf497 5%, #fd5949 45%, #d6249f 60%, #285AEB 90%);
background: -webkit-radial-gradient(circle at 30% 107%, #fdf497 0%, #fdf497 5%, #fd5949 45%, #d6249f 60%, #285AEB 90%);
background-clip: text;
-webkit-background-clip: text;
transition: background 0.3s;
}
.fa-square-x-twitter:hover {
color: black;
transition: color 0.3s;
}
.fa-linkedin:hover {
color: #0077B5;
transition: color 0.3s;
}
.fa-discord:hover {
color: white;
background-color: #5865F2;
border-radius: 5px;
transition: background-color 0.3s;
}
.hugging-face-logo {
height: 60px;
width: 60px;
padding-top: 5px;
filter: hue-rotate(210deg);
}
footer ul li .hugging-face-logo {
height: 20px;
width: 20px;
padding-top: 0px;
padding: 0px;
filter: hue-rotate(210deg);
}
ul li .hugging-face-logo:hover {
filter: hue-rotate(0deg);
transition: filter 0.3s;
}
</style>
docs/assets/images/favicon.ico
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 9.4 KiB

docs/assets/images/hf-logo.svg
+8
View File
File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 34 KiB

docs/assets/images/llmware_logo_color_cropped.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 27 KiB

docs/community/community.md
+105
View File
@@ -0,0 +1,105 @@
---
layout: default
title: Community
nav_order: 6
has_children: true
description: community resources, getting help and sharing ideas
permalink: /community
---
# Community
Welcome to the llmware community! We are on a mission to pioneer the use of small language models as transformational tools
in the enterprise to automate workflows and knowledge-based processes cost-effectively, securely and with high quality. We believe that the
secret is increasing out that small models can be extremely effective, but require a lot of attention to detail in building scalable data pipelines
and fine-tuning both models and end-to-end workflows.
We are open to both the most advanced machine learning researchers and the beginning developer just learning python.
We publish a wide range of examples, use cases and tutorial videos, and are always looking for feedback, new ideas and contributors.
{: .note}
> The contributions to `llmware` are governed by our [Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md).
{: .warning}
> Have you found a security issue? Then please jump to [Security Vulnerabilities](#security-vulnerabilities).
On this page, we provide information ``llmware`` contributions.
There are **two ways** on how you can contribute.
The first is by making **code contributions**, and the second by making contributions to the **documentation**.
Please look at our [contribution suggestions](#how-can-you-contribute) if you need inspiration, or take a look at [open issues](#open-issues).
Contributions to `llmware` are welcome from everyone.
Our goal is to make the process simple, transparent, and straightforward.
We are happy to receive suggestions on how the process can be improved.
## How can you contribute?
{: .note}
> If you have never contributed before look for issues with the tag [``good first issue``](https://github.com/llmware-ai/llmware/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22).
The most usual ways to contribute is to add new features, fix bugs, add tests, or add documentation.
You can visit the [issues](https://github.com/llmware-ai/llmware/issues) site of the project and search for tags such as
``bug``, ``enhancement``, ``documentation``, or ``test``.
Here is a non exhaustive list of contributions you can make.
1. Code refactoring
2. Add new text data bases
3. Add new vector data bases
4. Fix bugs
5. Add usage examples (see for example the issues [jupyter notebook - more examples and better support](https://github.com/llmware-ai/llmware/issues/508) and [google colab examples and start up scripts](https://github.com/llmware-ai/llmware/issues/507))
6. Add experimental features
7. Improve code quality
8. Improve documentation in the docs (what you are reading right now)
9. Improve documentation by adding or updating docstrings in modules, classes, methods, or functions (see for example [Add docstrings](https://github.com/llmware-ai/llmware/issues/219))
10. Improve test coverage
11. Answer questions in our [Discord channel](https://discord.gg/MhZn5Nc39h), especially in the [technical support forum](https://discord.com/channels/1179245642770559067/1218498778915672194)
12. Post projects in which you use ``llmware`` in our Discord forum [made with llmware](https://discord.com/channels/1179245642770559067/1218567269471486012), ideally with a link to a public GitHub repository
## Open Issues
If you're interested in existing issues, you can
- Look for issues, if you are a new to the project, look for issues with the `good first issue` label.
- Provide answers for questions in our [GitHub discussions](https://github.com/llmware-ai/llmware/discussions)
- Provide help for bug or enhancement issues.
- Ask questions, reproduce the issues, or provide solutions.
- Pull a request to fix the issue.
## Security Vulnerabilities
**If you believe you've found a security vulnerability, then please _do not_ submit an issue ticket or pull request or otherwise publicly disclose the issue.**
Please follow the process at [Reporting a Vulnerability](https://github.com/llmware-ai/llmware/blob/main/Security.md)
## GitHub workflow
We follow the [``fork-and-pull``](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork) Git workflow.
1. [Fork](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) the repository on GitHub.
2. Clone your fork to your local machine with `git clone git@github.com:<yourname>/llmware.git`.
3. Create a branch with `git checkout -b my-topic-branch`.
4. Run the test suite by navigating to the tests/ folder and running ```./run-tests.py -s``` to ensure there are no failures
5. [Commit](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/committing-changes-to-a-pull-request-branch-created-from-a-fork) changes to your own branch, then push to GitHub with `git push origin my-topic-branch`.
6. Submit a [pull request](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests) so that we can review your changes.
Remember to [synchronize your forked repository](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo#keep-your-fork-synced) _before_ submitting proposed changes upstream. If you have an existing local repository, please update it before you start, to minimize the chance of merge conflicts.
```shell
git remote add upstream git@github.com:llmware-ai/llmware.git
git fetch upstream
git checkout upstream/main -b my-topic-branch
```
## Community
Questions and discussions are welcome in any shape or form.
Please fell free to join our community on our discord channel, on which we are active daily.
You are also welcome if you just want to post an idea!
- [Discord Channel](https://discord.gg/MhZn5Nc39h)
- [GitHub discussions](https://github.com/llmware-ai/llmware/discussions)
docs/community/faq.md
+191
View File
@@ -0,0 +1,191 @@
---
layout: default
title: FAQ
parent: Community
nav_order: 1
description: overview of the major modules and classes of LLMWare
permalink: /community/faq
---
# Frequently Asked Questions (FAQ)
### How can I set the chunk size?
#### "I want to parse my documents into smaller chunks"
You can set the chunk size with the ``chunk_size`` parameter of the ``add_files`` method.
The ``add_files`` method from the ``Library`` class has a ``chunk_size`` parameter that controls the chunk size.
The method in addition has a parameter to control the maximum chunk size with ``max_chunk_size``.
These two parameters are passed on to the ``Parser`` class.
In the following example, we add the same files with different chunk sizes to the library ``chunk_size_example``.
```python
from pathlib import Path
from llmware.library import Library
path_to_my_library_files = Path('~/llmware_data/sample_files/Agreements')
my_library = Library().create_new_library(library_name='chunk_size_example')
my_library.add_files(input_folder_path=path_to_my_library_files, chunk_size=400)
my_library.add_files(input_folder_path=path_to_my_library_files, chunk_size=600)
```
### How can I set the embedding store?
#### "I want to use a specific embedding store"
You can set the embedding store with the ``vector_db`` parameter of the ``install_new_embedding`` method, which you call on a ``Library`` object each time you want to create an embedding for a *library*.
The ``install_new_embedding`` method from the ``Library`` class has a ``vector_db`` parameter that sets the embedding store.
At the moment of this writing, *LLMWare* supports the embedding stores [chromadb](https://github.com/chroma-core/chroma), [neo4j](https://github.com/neo4j/neo4j), [milvus](https://github.com/milvus-io/milvus), [pg_vector](https://github.com/pgvector/pgvector), [postgres](https://github.com/postgres/postgres), [redis](https://github.com/redis/redis), [pinecone](https://www.pinecone.io/), [faiss](https://github.com/facebookresearch/faiss), [qdrant](https://github.com/qdrant/qdrant), [mongo atlas](https://www.mongodb.com/products/platform/atlas-database), and [lancedb](https://github.com/lancedb/lancedb).
In the following example, we create the same embeddings three times for the same library, but store them in three different embedding stores.
```python
import logging
from pathlib import Path
from llmware.configs import LLMWareConfig
from llmware.library import Library
logging.info(f'Currently supported embedding stores: {LLMWareConfig().get_supported_vector_db()}')
library = Library().create_new_library(library_name='embedding_store_example')
library.add_files(input_foler_path=Path('~/llmware_data/sample_files/Agreements'))
library.install_new_embedding(vector_db="pg_vector")
library.install_new_embedding(vector_db="milvus")
library.install_new_embedding(vector_db="faiss")
```
### How can I set the collection store?
#### "I want to use a specific collection store"
You can set the collection store with the ``set_active_db`` method of the ``LLMWareConfig`` class.
The collection store is set using the ``LLMWareConfig`` class with the ``set_active_db`` method.
At the time of writing, **LLMWare** supports the three collection stores *MongoDB*, *Postgres*, and *SQLite* - which is the default.
You can retrieve the supported collection store with the method ``get_supported_collection_db``.
In the example below, we first print the currently active collection store, then we retrieve the supported collection stores, before we switch to *Postgres*.
```python
import logging
from llmware.configs import LLMWareConfig
logging.info(f'Currently active collection store: {LLMWareConfig.get_active_db()}')
logging.info(f'Currently supported collection stores: {LLMWareConfig().get_supported_collection_db()}')
LLMWareConfig.set_active_db("postgres")
logging.info(f'Currently active collection store: {LLMWareConfig.get_active_db()}')
```
### How can I retrieve more context?
#### "I want to retrieve more context from a query"
One way to retrieve more context is to set the ``result_count`` parameter of the ``query``, ``text_query``, and ``semantic_query`` methods from the ``Query`` class.
By increasing ``result_count``, the number of retrieved results is increased which increases the context size.
The ``Query`` class has the methods ``query``, ``text_query``, and ``semantic_query`` methods which allow to set the number of retrieved results with ``result_count``.
On a side note, ``query`` is a wrapper function for ``text_query`` and ``semantic_query``.
The value of ``result_count`` is passed on to the queried embedding store to control the number of retrieved results.
For example, for *pgvector* ``result_count`` is passed on to the value after the ``LIMIT`` keyword.
In the ``SQL`` example below, you can see the resulting ``SQL`` query of ``LLMWare`` if ``result_count=10``, the name of the collection being ``agreements``, and the query vector being ``[1, 2, 3]``.
```sql
SELECT
id,
block_mongo_id,
embedding <-> '[1, 2, 3]' AS distance,
text
FROM agreements
ORDER BY distance
LIMIT 10;
```
In the following example, we execute the same query against a library twice but change the number of retrieved results from ``3`` to ``6``.
```python
import logging
from pathlib import Path
from llmware.configs import LLMWareConfig
from llmware.library import Library
from llmware.retrieval import Query
logging.info(f'Currently supported embedding stores: {LLMWareConfig().get_supported_vector_db()}')
library = Library().create_new_library(library_name='context_size_example')
library.add_files(input_foler_path=Path('~/llmware_data/sample_files/Agreements'))
library.install_new_embedding(vector_db="pg_vector")
query = Query(library)
query_results = query.semantic_query(query='salary', result_count=3, results_only=True)
logging.info(f'Number of results: {len(query_results)}')
query_results = query.semantic_query(query='salary', result_count=6, results_only=True)
logging.info(f'Number of results: {len(query_results)}')
```
### How can I set the Large Language Model?
#### "I want to use a different LLM"
You can set the Large Language Model (LLM) with the ``gen_model`` parameter of the ``load_model`` method from the ``Prompt`` class.
The ``Prompt`` class has the method ``load_model`` with the ``gen_model`` parameter which sets the LLM.
The ``gen_model`` parameter is passed on to the ``ModelCatalog`` class, which loads the LLM either from HuggingFace or from another source.
The ``ModelCatalog`` allows you to **list all available models** with the method ``list_generative_models``, or just the local models ``list_generative_local_models``, or just the open source models ``list_open_source_models``.
In the example below, we log all available LLMs, including the ones that are available locally and the open source ones, and also create the prompters.
Each prompter uses a different LLM from our [BLING model series](https://llmware.ai/about), which you can also find on [HuggingFace](https://huggingface.co/collections/llmware/bling-models-6553c718f51185088be4c91a).
```python
import logging
from llmware.models import ModelCatalog
from llmware.prompts import Prompt
llm_gen = ModelCatalog().list_generative_models()
logging.info(f'List of all LLMs: {llm_gen}')
llm_gen_local = ModelCatalog().list_generative_local_models()
logging.info(f'List of all local LLMs: {llm_local}')
llm_gen_open_source = ModelCatalog().list_open_source_models()
logging.info(f'List of all open source LLMs: {llm_gen_open_source}')
prompter_bling_1b = Prompt().load_model(gen_model='llmware/bling-1b-0.1')
prompter_bling_tiny_llama = Prompt().load_model(gen_model='llmware/bling-tiny-llama-v0')
prompter_bling_falcon_1b = Prompt().load_model(gen_model='llmware/bling-falcon-1b-0.1')
```
### How can I set the embedding model?
#### "I want to use a different embedding model"
You can set the embedding model with the ``embedding_model_name`` parameter of the ``install_new_embedding`` method from the ``Library`` class.
The ``Library`` class has the method ``install_new_embedding`` with the ``embedding_model_name`` parameter which sets the embedding model.
The ``ModelCatalog`` allows you to **list all available embedding models** with the ``list_embedding_models`` method.
In the following example, we list all available embedding models, and then we create a library with the name ``embedding_models_example``, which we embed two times with embedding models ``'mini-lm-sber'`` and ``'industry-bert-contracts'``.
```python
import logging
from llmware.models import ModelCatalog
from llmware.library import Library
embedding_models = ModelCatalog().list_generative_models()
logging.info(f'List of embedding models: {embedding_models}')
library = Library().create_new_library(library_name='embedding_models_example')
library.add_files(input_foler_path=Path('~/llmware_data/sample_files/Agreements'))
library.install_new_embedding(embedding_model_name='mini-lm-sber')
library.install_new_embedding(embedding_model_name='industry-bert-contracts')
```
### Why is the model running slowly in Google Colab?
#### "I want to improve the performance of my model on Google Colab"
Our models are designed to run on at least 16GB of RAM. By default Google Colab provides ~13GB of RAM, which significantly slows computational speed. To ensure the best performance when using our models, we highly recommend enabling the T4 GPU in Colab. This will provide the notebook with additional resources, including 16GB of RAM, allowing our models to run smoothly and efficiently.
Steps to enabling T4 GPU in Colab:
1. In your Colab notebook, click on the "Runtime" tab
2. Select "Change runtime type"
3. Under "Hardware Accelerator", select T4 GPU
NOTE: There is a weekly usage limit on using T4 for free.
docs/community/join_our_community.md
+71
View File
@@ -0,0 +1,71 @@
---
layout: default
title: Join Our Community
parent: Community
nav_order: 4
description: overview of the major modules and classes of LLMWare
permalink: /community/join_our_community
---
# Join the LLMWare Community
___
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discord channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/community/need_help.md
+71
View File
@@ -0,0 +1,71 @@
---
layout: default
title: Need Hep
parent: Community
nav_order: 3
description: overview of the major modules and classes of LLMWare
permalink: /community/need_help
---
# Need Help
___
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discord channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/community/troubleshooting.md
+166
View File
@@ -0,0 +1,166 @@
---
layout: default
title: Troubleshooting
parent: Community
nav_order: 2
description: overview of the major modules and classes of LLMWare
permalink: /community/troubleshooting
---
# Common Troubleshooting Issues
___
1. **Can not install the pip package**
-- Check your Python version. If using Python 3.9-3.11, then almost any version of llmware should work. If using an older Python (before 3.9), then it is likely that dependencies will fail in the pip process. If you are using Python 3.12, then you need to use llmware>=0.2.12.
-- Dependency constraint error. If you receive a specific error around a dependency version constraint, then please raise an issue and include details about your OS, Python version, any unique elements in your virtual environment, and specific error.
2. **Parser module not found**
-- Check your OS and confirm that you are using a [supported platform](platforms.md/#platform-support).
-- If you cloned the repository, please confirm that the /lib folder has been copied into your local path.
3. **Pytorch Model not loading**
-- Confirm the obvious stuff - correct model name, model exists in Huggingface repository, connected to the Internet with open ports for HTTPS connection, etc.
-- Check Pytorch version - update Pytorch to >2.0, which is required for many recent models released in the last 6 months, and in some cases, may require other dependencies not included in the llmware package.
--note: we have seen some compatibility issues with Pytorch==2.3 on Wintel platforms - if you run into these issues, we recommend using a back-level Pytorch==2.1, which we have seen fixing the issue.
4. **GGUF Model not loading**
-- Confirm that you are using llmware>=0.2.11 for the latest GGUF support.
-- Confirm that you are using a [supported platform](platforms.md/#platform-support). We provide pre-built binaries for llama.cpp as a back-end GGUF engine on the following platforms:
- Mac M1/M2/M3 - OS version 14 - "with accelerate framework"
- Mac M1/M2/M3 - OS older versions - "without accelerate framework"
- Windows - x86
- Windows with CUDA
- Linux - x86 (Ubuntu 20+)
- Linux with CUDA (Ubuntu 20+)
If you are using a different OS platform, you have the option to "bring your own llama.cpp" lib as follows:
```python
from llmware.gguf_configs import GGUFConfigs
GGUFConfigs().set_config("custom_lib_path", "/path/to/your/libllama_binary")
```
If you have any trouble, feel free to raise an Issue and we can provide you with instructions and/or help compiling llama.cpp for your platform.
-- Specific GGUF model - if you are successfully using other GGUF models, and only having problems with a specific model, then please raise an Issue, and share the specific model and architecture.
5. **Example not working as expected** - please raise an issue, so we can evaluate and fix any bugs in the example code. Also, pull requests are always especially welcomed with a fix or improvement in an example.
6. **Model not leveraging CUDA available in environment.**
-- **Check CUDA drivers installed correctly** - easy check of the NVIDIA CUDA drivers is to use `nvidia-smi` and `nvcc --version` from the command line. Both commands should respond positively with details on the versions and implementations. Any errors indicates that either the driver or CUDA toolkit are not installed or recognized. It can be complicated at times to debug the environment, usually with some trial and error. See extensive [Nvidia Developer documentation](https://docs.nvidia.com) for trouble-shooting steps, specific to your environment.
-- **Check CUDA drivers are up to date** - we build to CUDA 12.1, which translates to a minimum of 525.60 on Linux, and 528.33 on Windows.
-- **Pytorch model** - check that Pytorch is finding CUDA, e.g., `torch.cuda.is_available()` == True. We have seen issues on Windows, in particular, to confirm that your Pytorch version has been compiled with CUDA drivers. For Windows, in particular, we have found that you may need to compile a CUDA-specific version of Pytorch, using the following command:
```pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121```
-- **GGUF model** - logs will be displayed on the screen confirming that CUDA is being used, or whether 'fall-back' to CPU drivers. We run a custom CUDA install check, which you can run on your system with:
```gpu_status = ModelCatalog().gpu_available```
If you are confirming CUDA present, but fall-back to CPU is being used, you can set the GGUFConfigs to force to CUDA:
```GGUFConfigs().set_config("force_gpu", True)```
If you are looking to use specific optimizations, you can bring your own llama.cpp lib as follows:
```GGUFConfigs().set_config("custom_lib_path", "/path/to/your/custom/llama_cpp_backend")```
-- If you can not debug after these steps, then please raise an Issue. We are happy to dig in and work with you to run FAST local inference.
7. **Model result inconsistent**
-- when loading the model, set `temperature=0.0` and `sample=False` -> this will give a deterministic output for better testing and debugging.
-- usually the issue will be related to the retrieval step and formation of the Prompt, and as always, good pipelines and a little experimentation usually help !
8. **Newly added examples not working as intended**
-- If you run a recently added example and it does not run as intended, it is possible that the feature being used in the example has not yet been added to the latest pip install.
-- To fix this, move the example file to the outer-most directory of the repository, so that the example file you are trying to run is in the same directory as the `llmware` source code directory.
-- This will let you run the example using the latest source code!
9. **Git permission denied error**
-- If you are using SSH to clone the repository and you get an error that looks similar to `git@github.com: Permission denied (publickey)`, then you might not have configured your SSH key correctly.
-- If you don't already have one, you will need to create a new SSH key on your local machine. For instructions on how to do this, check out this page: https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent.
-- You then need to add the SSH key to your GitHub account. For instructions on how to do this, check out this page: https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account.
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discord channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/components/agent_inference_server.md
+194
View File
@@ -0,0 +1,194 @@
---
layout: default
title: Agent Inference Server
parent: Components
nav_order: 12
description: overview of the major modules and classes of LLMWare
permalink: /components/agent_inference_server
---
# Agent Inference Server
---
LLMWare supports multiple deployment options, including the use of REST APIs to implement most model invocations.
To set up an inference server for Agent processes:
```python
""" This example shows how to set up an inference server that can be used in conjunction with agent-based workflows.
This script covers both the server-side deployment, as well as the steps taken on the client-side to deploy
in an Agent example.
Note: this example will build off two other examples:
1. "examples/Models/launch_llmware_inference_server.py"
2. "examples/SLIM-Agents/agent-llmfx-getting-started.py"
"""
from llmware.models import ModelCatalog, LLMWareInferenceServer
# *** SERVER SIDE SCRIPT ***
base_model = "llmware/bling-tiny-llama-v0"
LLMWareInferenceServer(base_model,
model_catalog=ModelCatalog(),
secret_api_key="demo-test",
home_path="/home/ubuntu/",
verbose=True).start()
# this will start Flask-based server, which will display the launched IP address and port, e.g.,
# "Running on " ip_address = "http://127.0.0.1:8080"
# *** CLIENT SIDE AGENT PROCESS ***
from llmware.agents import LLMfx
def create_multistep_report_over_api_endpoint():
""" This is derived from the script in the example agent-llmfx-getting-started.py. """
customer_transcript = "My name is Michael Jones, and I am a long-time customer. " \
"The Mixco product is not working currently, and it is having a negative impact " \
"on my business, as we can not deliver our products while it is down. " \
"This is the fourth time that I have called. My account number is 93203, and " \
"my user name is mjones. Our company is based in Tampa, Florida."
# create an agent using LLMfx class
agent = LLMfx()
# copy the ip address from the Flask launch readout
ip_address = "http://127.0.0.1:8080"
# inserting this line below into the agent process sets the 'api endpoint' execution to "ON"
# all agent function calls will be deployed over the API endpoint on the remote inference server
# to "switch back" to local execution, comment out this line
agent.register_api_endpoint(api_endpoint=ip_address,
api_key="demo-test",
endpoint_on=True)
# to explicitly turn the api endpoint "on" or "off"
# agent.switch_endpoint_on()
# agent.switch_endpoint_off()
agent.load_work(customer_transcript)
# load tools individually
agent.load_tool("sentiment")
agent.load_tool("ner")
# load multiple tools
agent.load_tool_list(["emotions", "topics", "intent", "tags", "ratings", "answer"])
# start deploying tools and running various analytics
# first conduct three 'soft skills' initial assessment using 3 different models
agent.sentiment()
agent.emotions()
agent.intent()
# alternative way to execute a tool, passing the tool name as a string
agent.exec_function_call("ratings")
# call multiple tools concurrently
agent.exec_multitool_function_call(["ner","topics","tags"])
# the 'answer' tool is a quantized question-answering model - ask an 'inline' question
# the optional 'key' assigns the output to a dictionary key for easy consolidation
agent.answer("What is a short summary?",key="summary")
# prompting tool to ask a quick question as part of the analytics
response = agent.answer("What is the customer's account number and user name?", key="customer_info")
# you can 'unload_tool' to release it from memory
agent.unload_tool("ner")
agent.unload_tool("topics")
# at end of processing, show the report that was automatically aggregated by key
report = agent.show_report()
# displays a summary of the activity in the process
activity_summary = agent.activity_summary()
# list of the responses gathered
for i, entries in enumerate(agent.response_list):
print("update: response analysis: ", i, entries)
output = {"report": report, "activity_summary": activity_summary, "journal": agent.journal}
return output
```
Need help or have questions?
============================
Check out the [llmware videos](https://www.youtube.com/@llmware) and [GitHub repository](https://github.com/llmware-ai/llmware).
Reach out to us on [GitHub Discussions](https://github.com/llmware-ai/llmware/discussions).
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discord channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/components/agents.md
+136
View File
@@ -0,0 +1,136 @@
---
layout: default
title: Agents
parent: Components
nav_order: 4
description: overview of the major modules and classes of LLMWare
permalink: /components/agents
---
# Agents
---
Agents with Function Calls and SLIM Models 🔥
llmware has been designed to enable Agent and LLM-based function calls using small language models designed for local and private
deployment and the ability to leverage open source models to conduct complex RAG and knowledge-based workflow automation.
The key elements in llmware:
- **SLIM models** - 18 function-calling small language models, optimized for a specific extraction, classification, generation, or
summarization activity, and generate python dictionaries and lists as output.
- **LLMfx class** - enables a wide range of agent-based processes.
Here is an example to get started:
```python
from llmware.agents import LLMfx
text = ("Tesla stock fell 8% in premarket trading after reporting fourth-quarter revenue and profit that "
"missed analysts’ estimates. The electric vehicle company also warned that vehicle volume growth in "
"2024 'may be notably lower' than last year’s growth rate. Automotive revenue, meanwhile, increased "
"just 1% from a year earlier, partly because the EVs were selling for less than they had in the past. "
"Tesla implemented steep price cuts in the second half of the year around the world. In a Wednesday "
"presentation, the company warned investors that it’s 'currently between two major growth waves.'")
# create an agent using LLMfx class
agent = LLMfx()
# load text to process
agent.load_work(text)
# load 'models' as 'tools' to be used in analysis process
agent.load_tool("sentiment")
agent.load_tool("extract")
agent.load_tool("topics")
agent.load_tool("boolean")
# run function calls using different tools
agent.sentiment()
agent.topics()
agent.extract(params=["company"])
agent.extract(params=["automotive revenue growth"])
agent.xsum()
agent.boolean(params=["is 2024 growth expected to be strong? (explain)"])
# at end of processing, show the report that was automatically aggregated by key
report = agent.show_report()
# displays a summary of the activity in the process
activity_summary = agent.activity_summary()
# list of the responses gathered
for i, entries in enumerate(agent.response_list):
print("update: response analysis: ", i, entries)
output = {"report": report, "activity_summary": activity_summary, "journal": agent.journal}
```
Need help or have questions?
============================
Check out the [llmware videos](https://www.youtube.com/@llmware) and [GitHub repository](https://github.com/llmware-ai/llmware).
Reach out to us on [GitHub Discussions](https://github.com/llmware-ai/llmware/discussions).
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discord channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/components/components.md
+164
View File
@@ -0,0 +1,164 @@
---
layout: default
title: Components
nav_order: 3
has_children: true
description: llmware key architectural components, modules and classes
permalink: /components
---
# LLMWare Architecture
---
llmware is characterized by a logically integrated set of data pipelines involved in building LLM-based workflows, centered on two main sub-pipelines with high-level interfaces intended to provide an abstraction layer over individual 'end point' components to promote code re-use and the ability to easily 'swap' different components with minimal, if any, code change:
**1. Knowledge Ingestion** - "creating Gen Ai Food" - ingesting and organizing unstructured information from a wide range of data sources, including each of the major steps:
- Extracting and Parsing
- Text Chunking
- Indexing, Organizing and Storing
- Embedding
- Retrieval
- Analytics and Reuse of Content
- Combining with SQL Table and Other Structured Content
**Core LLMWare classes**: **Library**, **Query** (retrieval module), **Parser**, **EmbeddingHandler** (embeddings module), **Graph**, **CustomTables** (resources module) and **Datasets** dataset_tools module).
In many cases, it is easy to get things done in LLMWare using only **Library** and **Query** - which provide convenient interfaces into parsing and embedding such that most use cases will not require calling those classes directly.
Supported document file types: pdf, pptx, docx, xlsx, txt, csv, html, jsonl, json, tsv, jpg, jpeg, png, wav, zip, md, mp3, mp4, m4a
Key methods to know:
- Ingest anything - `Library().add_files(input_folder_path="path/to/docs")`
- Embed library - `Library().install_new_embedding(embedding_model_name="your embedding model", vector_db="your vector db")`
- Run Query - `Query(library).query(query, query_type="semantic", result_count=20)`
Top examples to get started:
- [Parsing examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing) - ~14 stand-alone parsing examples for all common document types, including options for parsing in memory, outputting to JSON, parsing custom configured CSV and JSON files, running OCR on embedded images found in documents, table extraction, image extraction, text chunking, zip files, and web sources.
- [Embedding examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/Embedding) - ~15 stand-alone embedding examples to show how to use ~10 different vector databases and wide range of leading open source embedding models (including sentence transformers).
- [Retrieval examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/Retrieval) - ~10 stand-alone examples illustrating different query and retrieval techniques - semantic queries, text queries, document filters, page filters, 'hybrid' queries, author search, using query state, and generating bibliographies.
- [Dataset examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/Datasets) - ~5 stand-alone examples to show 'next steps' of how to leverage a Library to re-package content into various datasets and automated NLP analytics.
- [Fast start example #1-Parsing](https://www.github.com/llmware-ai/llmware/tree/main/fast_start/example-1-create_first_library.py) - shows the basics of parsing.
- [Fast start example #2-Embedding](https://www.github.com/llmware-ai/llmware/tree/main/fast_start/example-2-build_embeddings.py) - shows the basics of building embeddings.
- [CustomTable examples](https://www.github.com/llmware-ai/llmware/tree/main/Structured_Tables) - ~5 examples to start building structured tables that can be used in conjunction with LLM-based workflows.
**2. Model Prompting** - "Fun with LLMs" - the lifecycle of discovering, instantiating, and configuring an LLM-based model to execute an inference, including the ability to seamlessly prepare and integrate knowledge retrieval, and post-processing steps to validate accuracy, including:
- ModelCatalog - discover, load and manage configuration
- Inference
- Function Calls
- Prompts
- Prompt with Sources
- Fact Checking methods
- Agent-based multi-step processes
- Prompt History
Core LLMWare classes: **ModelCatalog** (models module), **Prompt**, **LLMfx** (agents module).
Key methods to know:
- Discover Models - `ModelCatalog().list_all_models()`
- Load Model - `model = ModelCatalog().load_model(model_name)`
- Inference - `response = model.inference(prompt, add_context=context)`
- Prompt - wraps the model class to provide easy source/retrieval management
- LLMfx - wraps the model class for function-calling SLIM models for agent processes
While ~17 individual model classes are exposed in the models module, for most use cases, we recommend working through the higher-level interface of ModelCatalog, as it promotes code re-use and the easy ability to swap models. In many pipelines, even ModelCatalog is not required to be called directly, as the Prompt class (knowledge retrieval) and LLMfx (agents and function calls) class provide seamless workflow capabilities and are built on top of the ModelCatalog.
Top examples to get started:
- [Models examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/Models) - ~20 examples showing a wide range of different model inferences and use cases, including the ability to integrate Ollama models, OpenChat (e.g., LMStudio) models, using LLama-3 and Phi-3, bringing your own models into the ModelCatalog, and configuring sampling settings.
- [Prompts examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/Prompts) - ~5 examples that illustrate how to use Prompt as an integrated workflow for integrating knowledge sources, managing prompt history, and applying fact-checking.
- [SLIM-Agents examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/SLIM-Agents) - ~20 examples showing how to build multi-model, multi-step Agent processes using locally-running SLIM function calling models.
- [Fast start example #3-Prompts and Models](https://www.github.com/llmware-ai/llmware/tree/main/fast_start/example-3-prompts_and_models.py) - getting started with model inference.
In addition, to support these two key pipelines, LLMWare has a set of supporting and enabling classes and methods, including:
- resource module: CollectionRetrieval, CollectionWriter, PromptState, QueryState, and ParserState - provides an abstraction layer on top of underlying database repositories and separate state mechanisms for major classes.
- gguf_configs module: GGUFConfigs
- model_configs module: global_model_repo_catalog_list, global_model_finetuning_prompt_wrappers_lookup, global_default_prompt_catalog
- util module: Utilities
- setup module: Setup
- status module: Status
- exceptions module: LLMWare Exceptions
- web_services module: classes for Wikipedia, YFinance, and WebSite extraction
**End-to-End Use Cases** - we publish and maintain a number of end-to-end use cases in [examples/Use_Cases](https://www.github.com/llmware-ai/llmware/tree/main/examples/Use_Cases)
Need help or have questions?
============================
Check out the [llmware videos](https://www.youtube.com/@llmware) and [GitHub repository](https://github.com/llmware-ai/llmware).
Reach out to us on [GitHub Discussions](https://github.com/llmware-ai/llmware/discussions).
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discord channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/components/data_stores.md
+102
View File
@@ -0,0 +1,102 @@
---
layout: default
title: Data Stores
parent: Components
nav_order: 9
description: overview of the major modules and classes of LLMWare
permalink: /components/data_stores
---
# Data Stores
---
Simple-to-Scale Database Options - integrated data stores from laptop to parallelized cluster.
```python
from llmware.configs import LLMWareConfig
# to set the collection database - mongo, sqlite, postgres
LLMWareConfig().set_active_db("mongo")
# to set the vector database (or declare when installing)
# --options: milvus, pg_vector (postgres), redis, qdrant, faiss, pinecone, mongo atlas
LLMWareConfig().set_vector_db("milvus")
# for fast start - no installations required
LLMWareConfig().set_active_db("sqlite")
LLMWareConfig().set_vector_db("chromadb") # try also faiss and lancedb
# for single postgres deployment
LLMWareConfig().set_active_db("postgres")
LLMWareConfig().set_vector_db("postgres")
# to install mongo, milvus, postgres - see the docker-compose scripts as well as examples
```
Need help or have questions?
============================
Check out the [llmware videos](https://www.youtube.com/@llmware) and [GitHub repository](https://github.com/llmware-ai/llmware).
Reach out to us on [GitHub Discussions](https://github.com/llmware-ai/llmware/discussions).
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discord channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/components/embedding_models.md
+144
View File
@@ -0,0 +1,144 @@
---
layout: default
title: Embedding Models
parent: Components
nav_order: 6
description: overview of the major modules and classes of LLMWare
permalink: /components/embedding_models
---
# Embedding Models
---
llmware supports 30+ embedding models out of the box in the default ModelCatalog, with easy extensibility to add other
popular open source embedding models from HuggingFace or Sentence Transformers.
To get a list of the currently supported embedding models:
```python
from llmware.models import ModelCatalog
embedding_models = ModelCatalog().list_embedding_models()
for i, models in enumerate(embedding_models):
print(f"embedding models: {i} - {models}")
```
Supported popular models include:
- Sentence Transformers - `all-MiniLM-L6-v2`, `all-mpnet-base-v2`
- Jina AI - `jinaai/jina-embeddings-v2-base-en`, `jinaai/jina-embeddings-v2-small-en`
- Nomic - `nomic-ai/nomic-embed-text-v1`
- Industry BERT - `industry-bert-insurance`, `industry-bert-contracts`, `industry-bert-asset-management`, `industry-bert-sec`, `industry-bert-loans`
- OpenAI - `text-embedding-ada-002`, `text-embedding-3-small`, `text-embedding-3-large`
We also support top embedding models from BAAI, thenlper, llmrails/ember, Google, and Cohere. We are constantly looking to add new innovative open source models to this list
so please let us know if you are looking for support for a specific embedding model, and usually within 1-2 days, we can test and add to the ModelCatalog.
# Using an Embedding Model
Embedding models in llmware can be installed directly by `ModelCatalog().load_model("model_name")`, but in most cases,
the name of the embedding model will be passed to the `install_new_embedding` handler in the Library class when creating a new
embedding. Once that is completed, the embedding model is captured in the Library metadata on the LibraryCard as part of the
embedding record for that library, and as a result, often times, does not need to be used explicitly again, e.g.,
```python
from llmware.library import Library
library = Library().create_new_library("my_library")
# parses the content from the documents in the file path, text chunks and indexes in a text collection database
library.add_files(input_folder_path="/local/path/to/my_files", chunk_size=400, max_chunk_size=600, smart_chunking=1)
# creates embeddings - and keeps synchronized records of which text chunks have been embedded to enable incremental use
library.install_new_embedding(embedding_model_name="jinaai/jina-embeddings-v2-small-en",
vector_db="milvus",
batch_size=100)
```
Once the embeddings are installed on the library, you can look up the embedding status to see the updated embeddings, and confirm that
the model has been correctly captured:
```python
from llmware.library import Library
library = Library().load_library("my_library")
embedding_record = library.get_embedding_status()
print("\nupdate: embedding record - ", embedding_record)
```
And then you can run semantic retrievals on the Library, using the Query class in the retrievals module, e.g.:
```python
from llmware.library import Library
from llmware.retrieval import Query
library = Library().load_library("my_library")
# queries are constructed by creating a Query object, and passing a library as input
query_results = Query(library).semantic_query("my query", result_count=20)
for qr in query_results:
print("my query results: ", qr)
```
Need help or have questions?
============================
Check out the [llmware videos](https://www.youtube.com/@llmware) and [GitHub repository](https://github.com/llmware-ai/llmware).
Reach out to us on [GitHub Discussions](https://github.com/llmware-ai/llmware/discussions).
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discord channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/components/gguf.md
+186
View File
@@ -0,0 +1,186 @@
---
layout: default
title: GGUF
parent: Components
nav_order: 14
description: overview of the major modules and classes of LLMWare
permalink: /components/gguf
---
# GGUF
---
llmware packages its own build of the llama.cpp backend engine to enable running quantized models in GGUF format, which provides an
effective packaging to run small language models on both CPUs and GPUs, which fast loading and inference.
The GGUF capability is implemented in the models.py module in the class `GGUFGenerativeModel` with an extensive set of interfaces and
configurations provided in the gguf_configs.py module (which for most users and use cases do not need to adjusted).
To use a GGUF model is the same as using any other model in the ModelCatalog, e.g.,
```python
from llmware.models import ModelCatalog
gguf_model = ModelCatalog().load_model("phi-3-gguf")
response = gguf_model.inference("What are the benefits of small specialized language models?")
print("response: ", response)
```
# GGUF Platform Support
Within the llmware library, we currently package 6 separate builds of the gguf llama.cpp engine for the following platforms:
# Mac M1/M2/M3
- with Accelerate: "libllama_mac_metal.dylib"
- without Accelerate: "libllama_mac_metal_no_acc.dylib" (note: if you have an old Mac OS installed, it may not have full Accelerate support)
- By default on Mac M1/M2/M3, it will attempt to use the Accelerate (faster) back-end, and if that fails, then it will automatically revert to the no-acc version
# Windows
- CUDA version
- CPU version
- Will look for CUDA drivers, and if found, will try to use the CUDA build, but if that fails, then it will automatically revert to the CPU version.
# Linux
- CUDA version
- CPU version
- Will look for CUDA drivers, and if found, will try to use the CUDA build, but if that fails, then it will automatically revert to the CPU version.
# Troubleshooting CUDA on Windows and Linux
Requirement: Nvidia CUDA 12.1+
-- how to check: `nvcc --version` and `nvidia-smi` - if not found, then drivers are either not installed or not in $PATH and need to be configured
-- if you have older drivers (e.g., v11), then you will need to update them.
# Bring your own custom llama.cpp gguf backend
If you have a unique system requirement, or are looking to optimize for a particular BLAS library with your own build, you can bring your own as follows:
if you have a unique system requirement, you can build llama_cpp from source, and apply custom build settings - or find in the community a prebuilt llama_cpp library that matches your platform. Happy to help if you share the requirements.
```python
from llmware.gguf_configs import GGUFConfigs
GGUFConfigs().set_config("custom_lib_path", "/path/to/your/custom/llama_cpp_backend")
# ... and then load and run the model as usual - the GGUF model class will look at this config and load the llama.cpp found at the custom lib path.
```
# Streaming GGUF
```python
""" This example illustrates how to use the stream method for GGUF models for fast streaming of inference,
especially for real-time chat interactions.
Please note that the stream method has been implemented for GGUF models starting in llmware-0.2.13. This will be
any model with GGUFGenerativeModel class, and generally includes models with names that end in "gguf".
See also the chat UI example in the UI examples folder.
We would recommend using a chat optimized model, and have included a representative list below.
"""
from llmware.models import ModelCatalog
from llmware.gguf_configs import GGUFConfigs
# sets an absolute output maximum for the GGUF engine - normally set by default at 256
GGUFConfigs().set_config("max_output_tokens", 1000)
chat_models = ["phi-3-gguf",
"llama-2-7b-chat-gguf",
"llama-3-instruct-bartowski-gguf",
"openhermes-mistral-7b-gguf",
"zephyr-7b-gguf",
"tiny-llama-chat-gguf"]
model_name = chat_models[0]
# maximum output can be set optionally at any number up to the "max_output_tokens" set
model = ModelCatalog().load_model(model_name, max_output=500)
text_out = ""
token_count = 0
# prompt = "I am interested in gaining an understanding of the banking industry. What topics should I research?"
prompt = "What are the benefits of small specialized LLMs?"
# since model.stream provides a generator, then use as follows to consume the generator
for streamed_token in model.stream(prompt):
text_out += streamed_token
if text_out.strip():
print(streamed_token, end="")
token_count += 1
# final output text and token count
print("\n\n***total text out***: ", text_out)
print("\n***total tokens***: ", token_count)
```
Need help or have questions?
============================
Check out the [llmware videos](https://www.youtube.com/@llmware) and [GitHub repository](https://github.com/llmware-ai/llmware).
Reach out to us on [GitHub Discussions](https://github.com/llmware-ai/llmware/discussions).
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in Oktober 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/components/library.md
+150
View File
@@ -0,0 +1,150 @@
---
layout: default
title: Library
parent: Components
nav_order: 7
description: overview of the major modules and classes of LLMWare
permalink: /components/library
---
# Library: ingest, organize and index a collection of knowledge at scale - Parse, Text Chunk and Embed.
---
Library is the main organizing construct for unstructured information in LLMWare. Users can create one large library with all types of different content, or
can create multiple libraries with each library comprising a specific logical collection of information on a
particular subject matter, project/case/deal, or even different accounts/users/departments.
Each Library consists of the following components:
1. Collection on a Database - this is the core of the Library, and is created through parsing of documents, which
are then automatically chunked and indexed in a text collection database. This is the basis for retrieval,
and the collection that will be used as the basis for tracking any number of vector embeddings that can be
attached to a library collection.
2. File archives - found in the llmware_data path, within Accounts, there is a folder structure for each Library.
All file-based artifacts for the Library are organized in these folders, including copies of all files added in
the library (very useful for retrieval-based applications), images extracted and indexed from the source
documents, as well as derived artifacts such as nlp and knowledge graph and datasets.
3. Library Catalog - each Library is registered in the LibraryCatalog table, with a unique library_card that has
the key attributes and statistics of the Library.
When a Library object is passed to the Parser, the parser will automatically route all information into the
Library structure.
The Library also exposes convenience methods to easily install embeddings on a library, including tracking of
incremental progress.
To parse into a Library, there is the very useful convenience methods, "add_files" which will invoke the Parser,
collate and route the files within a selected folder path, check for duplicate files, execute the parsing,
text chunking and insertion into the database, and update all of the Library state automatically.
Libraries are the main index constructs that are used in executing a Query. Pass the library object when
constructing the Query object, and then all retrievals (text, semantic and hybrid) will be executed against
the content in that Library only.
```python
from llmware.library import Library
# to parse and text chunk a set of documents (pdf, pptx, docx, xlsx, txt, csv, md, json/jsonl, wav, png, jpg, html)
# step 1 - create a library, which is the 'knowledge-base container' construct
# - libraries have both text collection (DB) resources, and file resources (e.g., llmware_data/accounts/{library_name})
# - embeddings and queries are run against a library
lib = Library().create_new_library("my_library")
# step 2 - add_files is the universal ingestion function - point it at a local file folder with mixed file types
# - files will be routed by file extension to the correct parser, parsed, text chunked and indexed in text collection DB
lib.add_files("/folder/path/to/my/files")
# to install an embedding on a library - pick an embedding model and vector_db
lib.install_new_embedding(embedding_model_name="mini-lm-sbert", vector_db="milvus", batch_size=500)
# to add a second embedding to the same library (mix-and-match models + vector db)
lib.install_new_embedding(embedding_model_name="industry-bert-sec", vector_db="chromadb", batch_size=100)
# easy to create multiple libraries for different projects and groups
finance_lib = Library().create_new_library("finance_q4_2023")
finance_lib.add_files("/finance_folder/")
hr_lib = Library().create_new_library("hr_policies")
hr_lib.add_files("/hr_folder/")
# pull library card with key metadata - documents, text chunks, images, tables, embedding record
lib_card = Library().get_library_card("my_library")
# see all libraries
all_my_libs = Library().get_all_library_cards()
```
Need help or have questions?
============================
Check out the [llmware videos](https://www.youtube.com/@llmware) and [GitHub repository](https://github.com/llmware-ai/llmware).
Reach out to us on [GitHub Discussions](https://github.com/llmware-ai/llmware/discussions).
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in Oktober 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/components/model_catalog.md
+263
View File
@@ -0,0 +1,263 @@
---
layout: default
title: Model Catalog
parent: Components
nav_order: 2
description: overview of the major modules and classes of LLMWare
permalink: /components/model_catalog
---
# Model Catalog:
Access all models the same way with easy lookup, regardless of underlying implementation.
- 150+ Models in Catalog with 50+ RAG-optimized BLING, DRAGON and Industry BERT models
- 18 SLIM function-calling small language models for Agent use cases
- Full support for GGUF, HuggingFace, Sentence Transformers and major API-based models
- Easy to extend to add custom models - see examples
Generally, all models can be identified using either the `model_name` or `display_name`, which provides some flexibility
to expose a more "UI friendly" name or an informal short-name for a commonly-used model.
The default model list is implemented in the model_configs.py module, which is then generally accessed in the models.py module through
the `ModelCatalog` class, which also provides the ability to add models of various types, over-write by loading a custom model catalog from json file, and
other useful interfaces into the list of models.
```python
from llmware.models import ModelCatalog
from llmware.prompts import Prompt
# all models accessed through the ModelCatalog
models = ModelCatalog().list_all_models()
# to use any model in the ModelCatalog - "load_model" method and pass the model_name parameter
my_model = ModelCatalog().load_model("llmware/bling-phi-3-gguf")
output = my_model.inference("what is the future of AI?", add_context="Here is the article to read")
# to integrate model into a Prompt
prompter = Prompt().load_model("llmware/bling-tiny-llama-v0")
response = prompter.prompt_main("what is the future of AI?", context="Insert Sources of information")
```
# ADD a Custom GGUF to the ModelCatalog
```python
import time
import re
from llmware.models import ModelCatalog
from llmware.prompts import Prompt
# Step 1 - register new gguf model - we will pick the popular LLama-2-13B-chat-GGUF
ModelCatalog().register_gguf_model(model_name="TheBloke/Llama-2-13B-chat-GGUF-Q2",
gguf_model_repo="TheBloke/Llama-2-13B-chat-GGUF",
gguf_model_file_name="llama-2-13b-chat.Q2_K.gguf",
prompt_wrapper="my_version_inst")
# Step 2- if the prompt_wrapper is a standard, e.g., Meta's <INST>, then no need to do anything else
# -- however, if the model uses a custom prompt wrapper, then we need to define that too
# -- in this case, we are going to create our "own version" of the Meta <INST> wrapper
ModelCatalog().register_new_finetune_wrapper("my_version_inst", main_start="<INST>", llm_start="</INST>")
# Once we have completed these two steps, we are done - and can begin to use the model like any other
prompter = Prompt().load_model("TheBloke/Llama-2-13B-chat-GGUF-Q2")
question_list = ["I am interested in gaining an understanding of the banking industry. What topics should I research?",
"What are some tips for creating a successful business plan?",
"What are the best books to read for a class on American literature?"]
for i, entry in enumerate(question_list):
start_time = time.time()
print("\n")
print(f"query - {i + 1} - {entry}")
response = prompter.prompt_main(entry)
# Print results
time_taken = round(time.time() - start_time, 2)
llm_response = re.sub("[\n\n]", "\n", response['llm_response'])
print(f"llm_response - {i + 1} - {llm_response}")
print(f"time_taken - {i + 1} - {time_taken}")
```
# ADD an Ollama Model
```python
from llmware.models import ModelCatalog
# Step 1 - register your Ollama models in llmware ModelCatalog
# -- these two lines will register: llama2 and mistral models
# -- note: assumes that you have previously cached and installed both of these models with ollama locally
# register llama2
ModelCatalog().register_ollama_model(model_name="llama2",model_type="chat",host="localhost",port=11434)
# register mistral - note: if you are using ollama defaults, then OK to register with ollama model name only
ModelCatalog().register_ollama_model(model_name="mistral")
# optional - confirm that model was registered
my_new_model_card = ModelCatalog().lookup_model_card("llama2")
print("\nupdate: confirming - new ollama model card - ", my_new_model_card)
# Step 2 - start using the Ollama model like any other model in llmware
print("\nupdate: calling ollama llama 2 model ...")
model = ModelCatalog().load_model("llama2")
response = model.inference("why is the sky blue?")
print("update: example #1 - ollama llama 2 response - ", response)
# Tip: if you are loading 'llama2' chat model from Ollama, note that it is already included in
# the llmware model catalog under a different name, "TheBloke/Llama-2-7B-Chat-GGUF"
# the llmware model name maps to the original HuggingFace repository, and is a nod to "TheBloke" who has
# led the popularization of GGUF - and is responsible for creating most of the GGUF model versions.
# --llmware uses the "Q4_K_M" model by default, while Ollama generally prefers "Q4_0"
print("\nupdate: calling Llama-2-7B-Chat-GGUF in llmware catalog ...")
model = ModelCatalog().load_model("TheBloke/Llama-2-7B-Chat-GGUF")
response = model.inference("why is the sky blue?")
print("update: example #1 - [compare] - llmware / Llama-2-7B-Chat-GGUF response - ", response)
# Now, let's try the Ollama Mistral model with a context passage
model2 = ModelCatalog().load_model("mistral")
context_passage= ("NASA’s rover Perseverance has gathered data confirming the existence of ancient lake "
"sediments deposited by water that once filled a giant basin on Mars called Jerezo Crater, "
"according to a study published on Friday. The findings from ground-penetrating radar "
"observations conducted by the robotic rover substantiate previous orbital imagery and "
"other data leading scientists to theorize that portions of Mars were once covered in water "
"and may have harbored microbial life. The research, led by teams from the University of "
"California at Los Angeles (UCLA) and the University of Oslo, was published in the "
"journal Science Advances. It was based on subsurface scans taken by the car-sized, six-wheeled "
"rover over several months of 2022 as it made its way across the Martian surface from the "
"crater floor onto an adjacent expanse of braided, sedimentary-like features resembling, "
"from orbit, the river deltas found on Earth.")
response = model2.inference("What are the top 3 points?", add_context=context_passage)
print("\nupdate: calling ollama mistral model ...")
print("update: example #2 - ollama mistral response - ", response)
# Step 3 - using the ollama discovery API - optional
discovery = model2.discover_models()
print("\nupdate: example #3 - checking ollama model manifest list: ", discovery)
if len(discovery) > 0:
# note: assumes tht you have at least one model registered in ollama -otherwise, may throw error
for i, models in enumerate(discovery["models"]):
print("ollama models: ", i, models)
```
# Add a LM Studio Model
```python
from llmware.models import ModelCatalog
from llmware.prompts import Prompt
# one step process: add the open chat model to the Model Registry
# key params:
# model_name = "my_open_chat_model1"
# api_base = uri_path to the proposed endpoint
# prompt_wrapper = alpaca | <INST> | chat_ml | hf_chat | human_bot
# <INST> -> Llama2-Chat
# hf_chat -> Zephyr-Mistral
# chat_ml -> OpenHermes - Mistral
# human_bot -> Dragon models
# model_type = "chat" (alternative: "completion")
ModelCatalog().register_open_chat_model("my_open_chat_model1",
api_base="http://localhost:1234/v1",
prompt_wrapper="<INST>",
model_type="chat")
# once registered, you can invoke like any other model in llmware
prompter = Prompt().load_model("my_open_chat_model1")
response = prompter.prompt_main("What is the future of AI?")
# you can (optionally) register multiple open chat models with different api_base and model attributes
ModelCatalog().register_open_chat_model("my_open_chat_model2",
api_base="http://localhost:5678/v1",
prompt_wrapper="hf_chat",
model_type="chat")
```
Need help or have questions?
============================
Check out the [llmware videos](https://www.youtube.com/@llmware) and [GitHub repository](https://github.com/llmware-ai/llmware).
Reach out to us on [GitHub Discussions](https://github.com/llmware-ai/llmware/discussions).
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in Oktober 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/components/prompt_with_sources.md
+186
View File
@@ -0,0 +1,186 @@
---
layout: default
title: Prompt with Sources
parent: Components
nav_order: 10
description: overview of the major modules and classes of LLMWare
permalink: /components/prompt_with_sources
---
# Prompt with Sources
---
Prompt with Sources: the easiest way to combine knowledge retrieval with a LLM inference, and provides several high-level useful methods to
easily integrate a retrieval/query/parsing step into a prompt to be used as a source for running an inference on a model.
This is best illustrated with a simple example:
```python
from llmware.prompts import Prompt
# build a prompt and attach a model
prompter = Prompt().load_model("llmware/bling-tiny-llama-v0")
# add_source_document method: accepts any supported document type, parses the file, and creates text chunks
# if a query is passed, then it will run a quick in-memory filtering search against the text chunks
# the text chunks are packaged into sources with all of the accompanying metadata from the file, and made
# available automatically in batches to be used in prompting -
source = prompter.add_source_document("/folder/to/one/doc/", "filename", query="fast query")
# to run inference with 'prompt with sources' -> source will be automatically added to the prompt
responses = prompter.prompt_with_source("my query")
# depending upon the size of the source (and batching relative to the model context window, there may be more than
# a single inference run, so unpack potentially multiple responses
for i, response in enumerate(responses):
print("response: ", i, response)
```
# FACT CHECKING
Using prompt_with_source also provides integrated fact-checking methods that use the packaged source information to validate key
elements from the llm_response
```python
from llmware.prompts import Prompt
prompter = Prompt().load_model("bling-answer-tool", temperature=0.0, sample=False)
# contract is parsed, text-chunked, and then filtered by "base salary'
source = prompter.add_source_document("/local/folder/path", "my_document.pdf", query="exact filter query")
# calling the LLM with 'source' information from the contract automatically packaged into the prompt
responses = prompter.prompt_with_source("my question to the document", prompt_name="default_with_context")
# run several fact checks
# checks for numbers match
ev_numbers = prompter.evidence_check_numbers(responses)
# looks for statistical overlap to identify potential sources for the llm response
ev_sources = prompter.evidence_check_sources(responses)
# builds set of comparison stats between the llm_response and the sources
ev_stats = prompter.evidence_comparison_stats(responses)
# identifies if a response is a "not found" response
z = prompter.classify_not_found_response(responses, parse_response=True, evidence_match=True,ask_the_model=False)
for r, response in enumerate(responses):
print("LLM Response: ", response["llm_response"])
print("Numbers: ", ev_numbers[r]["fact_check"])
print("Sources: ", ev_sources[r]["source_review"])
print("Stats: ", ev_stats[r]["comparison_stats"])
print("Not Found Check: ", z[r])
```
In addition to `add_source_document`, the Prompt class implements the following other methods to easily integrate sources into prompts:
# Add Source - Query Results - Two Options
```python
from llmware.prompts import Prompt
from llmware.retrieval import Query
from llmware.library import Library
# build a prompt
prompter = Prompt().load_model("llmware/bling-tiny-llama-v0")
# Option A - run query and then add query_results to the prompt
my_lib = Library().load_library("my_library")
results = Query(my_lib).query("my query")
source2 = prompter.add_source_query_results(results)
# Option B - run a new query against a library and load directly into a prompt
source3 = prompter.add_source_new_query(my_lib, query="my new query", query_type="semantic", result_count=15)
```
# Add Other Sources
```python
from llmware.prompts import Prompt
# build a prompt
prompter = Prompt().load_model("llmware/bling-tiny-llama-v0")
# add wikipedia articles as a source
wiki_source = prompter.add_source_wikipedia("topic", article_count=5, query="filter among retrieved articles")
# add a website as a source
website_source = prompter.add_source_website("my_url", query="filter among website")
# add an entire library (should be small, e.g., just a couple of documents)
source = prompter.add_source_library("my_library")
```
Need help or have questions?
============================
Check out the [llmware videos](https://www.youtube.com/@llmware) and [GitHub repository](https://github.com/llmware-ai/llmware).
Reach out to us on [GitHub Discussions](https://github.com/llmware-ai/llmware/discussions).
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in Oktober 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/components/query.md
+115
View File
@@ -0,0 +1,115 @@
---
layout: default
title: Query
parent: Components
nav_order: 8
description: overview of the major modules and classes of LLMWare
permalink: /components/query
---
# Retrieval & Query
---
Query libraries with mix of text, semantic, hybrid, metadata, and custom filters. The retrieval.py module implements the
`Query` class, which is the primary way that search and retrieval is performed. Each `Query` object, when constructed,
requires that a Library is passed as a mandatory parameter in the constructor. The Query object will operate against that
Library, and have access to all of Library's specific attributes, metadata and methods.
Retrievals in llmware leverage the Library abstraction as the primary unit against which a particular query or retrieval is
executed. This provides the ability to have multiple distinct knowledge-bases, potentially aligned to different use cases, and/or
users, accounts and permissions.
# Executing Queries
```python
from llmware.retrieval import Query
from llmware.library import Library
# step 1 - load a previously created library
lib = Library().load_library("my_library")
# step 2 - create a query object
q = Query(lib)
# step 3 - run lots of different queries (many other options in the examples)
# basic text query
results1 = q.text_query("text query", result_count=20, exact_mode=False)
# semantic query
results2 = q.semantic_query("semantic query", result_count=10)
# combining a text query restricted to only certain documents in the library and "exact" match to the query
results3 = q.text_query_with_document_filter("new query", {"file_name": "selected file name"}, exact_mode=True)
# to apply a specific embedding (if multiple on library), pass the names when creating the query object
q2 = Query(lib, embedding_model_name="mini_lm_sbert", vector_db="milvus")
results4 = q2.semantic_query("new semantic query")
```
Need help or have questions?
============================
Check out the [llmware videos](https://www.youtube.com/@llmware) and [GitHub repository](https://github.com/llmware-ai/llmware).
Reach out to us on [GitHub Discussions](https://github.com/llmware-ai/llmware/discussions).
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in Oktober 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/components/rag_optimized_models.md
+326
View File
@@ -0,0 +1,326 @@
---
layout: default
title: RAG Optimized Models
parent: Components
nav_order: 3
description: overview of the major modules and classes of LLMWare
permalink: /components/rag_optimized_models
---
# RAG Optimized Models
---
RAG-Optimized Models - 1-7B parameter models designed for RAG workflow integration and running locally. </summary>
## Meet our Models
- **SLIM model series:** small, specialized models fine-tuned for function calling and multi-step, multi-model Agent workflows.
- **DRAGON model series:** Production-grade RAG-optimized 6-7B parameter models - "Delivering RAG on ..." the leading foundation base models.
- **BLING model series:** Small CPU-based RAG-optimized, instruct-following 1B-3B parameter models.
- **Industry BERT models:** out-of-the-box custom trained sentence transformer embedding models fine-tuned for the following industries: Insurance, Contracts, Asset Management, SEC.
- **GGUF Quantization:** we provide 'gguf' and 'tool' versions of many SLIM, DRAGON and BLING models, optimized for CPU deployment.
```python
""" This 'Hello World' example demonstrates how to get started using local BLING models with provided context, using both
Pytorch and GGUF versions. """
import time
from llmware.prompts import Prompt
def hello_world_questions():
test_list = [
{"query": "What is the total amount of the invoice?",
"answer": "$22,500.00",
"context": "Services Vendor Inc. \n100 Elm Street Pleasantville, NY \nTO Alpha Inc. 5900 1st Street "
"Los Angeles, CA \nDescription Front End Engineering Service $5000.00 \n Back End Engineering"
" Service $7500.00 \n Quality Assurance Manager $10,000.00 \n Total Amount $22,500.00 \n"
"Make all checks payable to Services Vendor Inc. Payment is due within 30 days."
"If you have any questions concerning this invoice, contact Bia Hermes. "
"THANK YOU FOR YOUR BUSINESS! INVOICE INVOICE # 0001 DATE 01/01/2022 FOR Alpha Project P.O. # 1000"},
{"query": "What was the amount of the trade surplus?",
"answer": "62.4 billion yen ($416.6 million)",
"context": "Japan’s September trade balance swings into surplus, surprising expectations"
"Japan recorded a trade surplus of 62.4 billion yen ($416.6 million) for September, "
"beating expectations from economists polled by Reuters for a trade deficit of 42.5 "
"billion yen. Data from Japan’s customs agency revealed that exports in September "
"increased 4.3% year on year, while imports slid 16.3% compared to the same period "
"last year. According to FactSet, exports to Asia fell for the ninth straight month, "
"which reflected ongoing China weakness. Exports were supported by shipments to "
"Western markets, FactSet added. — Lim Hui Jie"},
{"query": "When did the LISP machine market collapse?",
"answer": "1987.",
"context": "The attendees became the leaders of AI research in the 1960s."
" They and their students produced programs that the press described as 'astonishing': "
"computers were learning checkers strategies, solving word problems in algebra, "
"proving logical theorems and speaking English. By the middle of the 1960s, research in "
"the U.S. was heavily funded by the Department of Defense and laboratories had been "
"established around the world. Herbert Simon predicted, 'machines will be capable, "
"within twenty years, of doing any work a man can do'. Marvin Minsky agreed, writing, "
"'within a generation ... the problem of creating 'artificial intelligence' will "
"substantially be solved'. They had, however, underestimated the difficulty of the problem. "
"Both the U.S. and British governments cut off exploratory research in response "
"to the criticism of Sir James Lighthill and ongoing pressure from the US Congress "
"to fund more productive projects. Minsky's and Papert's book Perceptrons was understood "
"as proving that artificial neural networks approach would never be useful for solving "
"real-world tasks, thus discrediting the approach altogether. The 'AI winter', a period "
"when obtaining funding for AI projects was difficult, followed. In the early 1980s, "
"AI research was revived by the commercial success of expert systems, a form of AI "
"program that simulated the knowledge and analytical skills of human experts. By 1985, "
"the market for AI had reached over a billion dollars. At the same time, Japan's fifth "
"generation computer project inspired the U.S. and British governments to restore funding "
"for academic research. However, beginning with the collapse of the Lisp Machine market "
"in 1987, AI once again fell into disrepute, and a second, longer-lasting winter began."},
{"query": "What is the current rate on 10-year treasuries?",
"answer": "4.58%",
"context": "Stocks rallied Friday even after the release of stronger-than-expected U.S. jobs data "
"and a major increase in Treasury yields. The Dow Jones Industrial Average gained 195.12 points, "
"or 0.76%, to close at 31,419.58. The S&P 500 added 1.59% at 4,008.50. The tech-heavy "
"Nasdaq Composite rose 1.35%, closing at 12,299.68. The U.S. economy added 438,000 jobs in "
"August, the Labor Department said. Economists polled by Dow Jones expected 273,000 "
"jobs. However, wages rose less than expected last month. Stocks posted a stunning "
"turnaround on Friday, after initially falling on the stronger-than-expected jobs report. "
"At its session low, the Dow had fallen as much as 198 points; it surged by more than "
"500 points at the height of the rally. The Nasdaq and the S&P 500 slid by 0.8% during "
"their lowest points in the day. Traders were unclear of the reason for the intraday "
"reversal. Some noted it could be the softer wage number in the jobs report that made "
"investors rethink their earlier bearish stance. Others noted the pullback in yields from "
"the day’s highs. Part of the rally may just be to do a market that had gotten extremely "
"oversold with the S&P 500 at one point this week down more than 9% from its high earlier "
"this year. Yields initially surged after the report, with the 10-year Treasury rate trading "
"near its highest level in 14 years. The benchmark rate later eased from those levels, but "
"was still up around 6 basis points at 4.58%. 'We’re seeing a little bit of a give back "
"in yields from where we were around 4.8%. [With] them pulling back a bit, I think that’s "
"helping the stock market,' said Margaret Jones, chief investment officer at Vibrant Industries "
"Capital Advisors. 'We’ve had a lot of weakness in the market in recent weeks, and potentially "
"some oversold conditions.'"},
{"query": "Is the expected gross margin greater than 70%?",
"answer": "Yes, between 71.5% and 72.%",
"context": "Outlook NVIDIA’s outlook for the third quarter of fiscal 2024 is as follows:"
"Revenue is expected to be $16.00 billion, plus or minus 2%. GAAP and non-GAAP "
"gross margins are expected to be 71.5% and 72.5%, respectively, plus or minus "
"50 basis points. GAAP and non-GAAP operating expenses are expected to be "
"approximately $2.95 billion and $2.00 billion, respectively. GAAP and non-GAAP "
"other income and expense are expected to be an income of approximately $100 "
"million, excluding gains and losses from non-affiliated investments. GAAP and "
"non-GAAP tax rates are expected to be 14.5%, plus or minus 1%, excluding any discrete items."
"Highlights NVIDIA achieved progress since its previous earnings announcement "
"in these areas: Data Center Second-quarter revenue was a record $10.32 billion, "
"up 141% from the previous quarter and up 171% from a year ago. Announced that the "
"NVIDIA® GH200 Grace™ Hopper™ Superchip for complex AI and HPC workloads is shipping "
"this quarter, with a second-generation version with HBM3e memory expected to ship "
"in Q2 of calendar 2024. "},
{"query": "What is Bank of America's rating on Target?",
"answer": "Buy",
"context": "Here are some of the tickers on my radar for Thursday, Oct. 12, taken directly from "
"my reporter’s notebook: It’s the one-year anniversary of the S&P 500′s bear market bottom "
"of 3,577. Since then, as of Wednesday’s close of 4,376, the broad market index "
"soared more than 22%. Hotter than expected September consumer price index, consumer "
"inflation. The Social Security Administration issues announced a 3.2% cost-of-living "
"adjustment for 2024. Chipotle Mexican Grill (CMG) plans price increases. Pricing power. "
"Cites consumer price index showing sticky retail inflation for the fourth time "
"in two years. Bank of America upgrades Target (TGT) to buy from neutral. Cites "
"risk/reward from depressed levels. Traffic could improve. Gross margin upside. "
"Merchandising better. Freight and transportation better. Target to report quarter "
"next month. In retail, the CNBC Investing Club portfolio owns TJX Companies (TJX), "
"the off-price juggernaut behind T.J. Maxx, Marshalls and HomeGoods. Goldman Sachs "
"tactical buy trades on Club names Wells Fargo (WFC), which reports quarter Friday, "
"Humana (HUM) and Nvidia (NVDA). BofA initiates Snowflake (SNOW) with a buy rating."
"If you like this story, sign up for Jim Cramer’s Top 10 Morning Thoughts on the "
"Market email newsletter for free. Barclays cuts price targets on consumer products: "
"UTZ Brands (UTZ) to $16 per share from $17. Kraft Heinz (KHC) to $36 per share from "
"$38. Cyclical drag. J.M. Smucker (SJM) to $129 from $160. Secular headwinds. "
"Coca-Cola (KO) to $59 from $70. Barclays cut PTs on housing-related stocks: Toll Brothers"
"(TOL) to $74 per share from $82. Keeps underweight. Lowers Trex (TREX) and Azek"
"(AZEK), too. Goldman Sachs (GS) announces sale of fintech platform and warns on "
"third quarter of 19-cent per share drag on earnings. The buyer: investors led by "
"private equity firm Sixth Street. Exiting a mistake. Rise in consumer engagement for "
"Spotify (SPOT), says Morgan Stanley. The analysts hike price target to $190 per share "
"from $185. Keeps overweight (buy) rating. JPMorgan loves elf Beauty (ELF). Keeps "
"overweight (buy) rating but lowers price target to $139 per share from $150. "
"Sees “still challenging” environment into third-quarter print. The Club owns shares "
"in high-end beauty company Estee Lauder (EL). Barclays upgrades First Solar (FSLR) "
"to overweight from equal weight (buy from hold) but lowers price target to $224 per "
"share from $230. Risk reward upgrade. Best visibility of utility scale names."},
{"query": "What was the rate of decline in 3rd quarter sales?",
"answer": "20% year-on-year.",
"context": "Nokia said it would cut up to 14,000 jobs as part of a cost cutting plan following "
"third quarter earnings that plunged. The Finnish telecommunications giant said that "
"it will reduce its cost base and increase operation efficiency to “address the "
"challenging market environment. The substantial layoffs come after Nokia reported "
"third-quarter net sales declined 20% year-on-year to 4.98 billion euros. Profit over "
"the period plunged by 69% year-on-year to 133 million euros."},
{"query": "What is a list of the key points?",
"answer": "•Stocks rallied on Friday with stronger-than-expected U.S jobs data and increase in "
"Treasury yields;\n•Dow Jones gained 195.12 points;\n•S&P 500 added 1.59%;\n•Nasdaq Composite rose "
"1.35%;\n•U.S. economy added 438,000 jobs in August, better than the 273,000 expected;\n"
"•10-year Treasury rate trading near the highest level in 14 years at 4.58%.",
"context": "Stocks rallied Friday even after the release of stronger-than-expected U.S. jobs data "
"and a major increase in Treasury yields. The Dow Jones Industrial Average gained 195.12 points, "
"or 0.76%, to close at 31,419.58. The S&P 500 added 1.59% at 4,008.50. The tech-heavy "
"Nasdaq Composite rose 1.35%, closing at 12,299.68. The U.S. economy added 438,000 jobs in "
"August, the Labor Department said. Economists polled by Dow Jones expected 273,000 "
"jobs. However, wages rose less than expected last month. Stocks posted a stunning "
"turnaround on Friday, after initially falling on the stronger-than-expected jobs report. "
"At its session low, the Dow had fallen as much as 198 points; it surged by more than "
"500 points at the height of the rally. The Nasdaq and the S&P 500 slid by 0.8% during "
"their lowest points in the day. Traders were unclear of the reason for the intraday "
"reversal. Some noted it could be the softer wage number in the jobs report that made "
"investors rethink their earlier bearish stance. Others noted the pullback in yields from "
"the day’s highs. Part of the rally may just be to do a market that had gotten extremely "
"oversold with the S&P 500 at one point this week down more than 9% from its high earlier "
"this year. Yields initially surged after the report, with the 10-year Treasury rate trading "
"near its highest level in 14 years. The benchmark rate later eased from those levels, but "
"was still up around 6 basis points at 4.58%. 'We’re seeing a little bit of a give back "
"in yields from where we were around 4.8%. [With] them pulling back a bit, I think that’s "
"helping the stock market,' said Margaret Jones, chief investment officer at Vibrant Industries "
"Capital Advisors. 'We’ve had a lot of weakness in the market in recent weeks, and potentially "
"some oversold conditions.'"}
]
return test_list
# this is the main script to be run
def bling_meets_llmware_hello_world (model_name):
t0 = time.time()
# load the questions
test_list = hello_world_questions()
print(f"\n > Loading Model: {model_name}...")
# load the model
prompter = Prompt().load_model(model_name)
t1 = time.time()
print(f"\n > Model {model_name} load time: {t1-t0} seconds")
for i, entries in enumerate(test_list):
print(f"\n{i+1}. Query: {entries['query']}")
# run the prompt
output = prompter.prompt_main(entries["query"],context=entries["context"]
, prompt_name="default_with_context",temperature=0.30)
# print out the results
llm_response = output["llm_response"].strip("\n")
print(f"LLM Response: {llm_response}")
print(f"Gold Answer: {entries['answer']}")
print(f"LLM Usage: {output['usage']}")
t2 = time.time()
print(f"\nTotal processing time: {t2-t1} seconds")
return 0
if __name__ == "__main__":
# list of 'rag-instruct' laptop-ready small bling models on HuggingFace
pytorch_models = ["llmware/bling-1b-0.1", # most popular
"llmware/bling-tiny-llama-v0", # fastest
"llmware/bling-1.4b-0.1",
"llmware/bling-falcon-1b-0.1",
"llmware/bling-cerebras-1.3b-0.1",
"llmware/bling-sheared-llama-1.3b-0.1",
"llmware/bling-sheared-llama-2.7b-0.1",
"llmware/bling-red-pajamas-3b-0.1",
"llmware/bling-stable-lm-3b-4e1t-v0",
"llmware/bling-phi-3" # most accurate (and newest)
]
# Quantized GGUF versions generally load faster and run nicely on a laptop with at least 16 GB of RAM
gguf_models = ["bling-phi-3-gguf", "bling-stablelm-3b-tool", "dragon-llama-answer-tool", "dragon-yi-answer-tool", "dragon-mistral-answer-tool"]
# try model from either pytorch or gguf model list
# the newest (and most accurate) is 'bling-phi-3-gguf'
bling_meets_llmware_hello_world(gguf_models[0])
# check out the model card on Huggingface for RAG benchmark test performance results and other useful information
```
Need help or have questions?
============================
Check out the [llmware videos](https://www.youtube.com/@llmware) and [GitHub repository](https://github.com/llmware-ai/llmware).
Reach out to us on [GitHub Discussions](https://github.com/llmware-ai/llmware/discussions).
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in Oktober 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/components/release_history.md
+153
View File
@@ -0,0 +1,153 @@
---
layout: default
title: Release History
parent: Components
nav_order: 15
description: llmware is an integrated framework with over 50+ models for quickly developing LLM-based applications including Retrieval Augmented Generation (RAG) and Multi-Step Orchestration of Agent Workflows.
permalink: /components/release_history
---
Release History
---
- For Specific Wheels: [Wheel Archives](https://www.github.com/llmware-ai/llmware/tree/main/wheel_archives)
- For Features Details: [Main README-'Release notes and Change Log'](https://www.github.com/llmware-ai/llmware/tree/main/)
New wheels are built generally on PyPy on a weekly basis and updated on PyPy versioning. The development repo is updated
and current at all times, but may have updates that are not yet in the PyPy wheel.
All wheels are built and tested on:
1. Mac Metal
2. Windows x86 (+ with CUDA)
3. Linux x86 (+ with CUDA) - most testing on Ubuntu 22 and Ubuntu 20 - which are recommended.
4. Mac x86 (see 0.2.11 note below)
5. Linux aarch64* (see 0.2.7 note below)
**Release Notes**
--**0.3.0** released in the week of June 4, 2024 - continued pruning of required dependencies with split of python dependencies into a small minimal set of requirements (~10 in requirements.txt) that are included in the pip install, with an additional set of optional dependencies provided as 'extras', reflected in both the requirements_extras.txt file, and available over pip with the added instruction - `pip3 install 'llmware[full]'`. Notably, commonly used libraries such as transformers, torch and openai are now in the 'extras' as most llmware use cases do not require them, and this greatly simplifies the ability to install llmware. The `welcome_to_llmware.sh` and `welcome_to_llmware_windows.sh` have also been updated to install both the 'core' and 'extra' set of requirements. Other subtle, but significant, architectural changes include offering more extensibility for adding new model classes, configurable global base model methods for post_init and register, a new InferenceHistory state manager, and enhanced logging options.
--**0.2.15** released in the week of May 20, 2024 - removed pytorch dependency as a global import, and shifted to dynamically loading of torch in the event that it is called in a specific model class. This enables running most of llmware code and examples without pytorch or transformers loaded. The main areas of torch (and transformers) dependency is in using HFGenerativeModels and HFEmbeddingModels.
- note: we have seen some new errors caused with Pytorch 2.3 - which are resolved by down-leveling to `pip3 install torch==2.1`
- note: there are a couple of new warnings from within transformers and huggingface_hub libraries - these can be safely ignored. We have seen that keeping `local_dir_use_symlinks = False` when pulling model artifacts from Huggingface is still the safer option in some environments.
--**0.2.13** released in the week of May 12, 2024 - clean up of dependencies in both requirements.txt and Setup (PyPi) - install of vector db python sdk (e.g., pymilvus, chromadb, etc) is now required as a separate step outside of the pip3 install llmware - attempt to keep dependency matrix as simple as possible and avoid potential dependency conflicts on install, especially for packages which in turn have a large number of dependencies. If you run into any issues with install dependencies, please raise an issue.
--**0.2.12** released in the week of May 5, 2024 - added Python 3.12 support, and deprecated the use of faiss for v3.12+. We have changed the "Fast Start" no-install option to use chromadb or lancedb rather than faiss. Refactoring of code especially with Datasets, Graph and Web Services as separate modules.
--**0.2.11** released in the week of April 29, 2024 - updated GGUF libs for Phi-3 and Llama-3 support, and added new prebuilt shared libraries to support WhisperCPP. We are also deprecating support for Mac x86 going forward - will continue to support on most major components but not all new features going forward will be built specifically for Mac x86 (which Apple stopped shipping in 2022). Our intent is to keep narrowing our testing matrix to provide better support on key platforms. We have also added better safety checks for older versions of Mac OS running on M1/M2/M3 (no_acc option in GGUF and Whisper libs), as well as a custom check to find CUDA drivers on Windows (independent of Pytorch).
--**0.2.9** released in the week of April 15, 2024 - minor continued improvements to the parsers plus roll-out of new CustomTable class for rapidly integrating structured information into LLM-based workflows and data pipelines, including converting JSON/JSONL files and CSV files into structured DB tables.
--**0.2.8** released in the week of April 8, 2024 - significant improvements to the Office parser with new libs on all platforms. Conforming changes with the PDF parser in terms of exposing more options for text chunking strategies, encoding, and range of capture options (e.g., tables, images, header text, etc). Linux aarch64 libs deprecated and kept at 0.2.6 - some new features will not be available on Linux aarch64 - we recommend using Ubuntu20+ on x86_64 (with and without CUDA).
--**0.2.7** released in the week of April 1, 2024 - significant improvements to the PDF parser with new libs on all platforms. Important note that we are keeping linux aarch64 at 0.2.6 libs - and will be deprecating support going forward. For Linux, we recommend Ubuntu20+ and x86_64 (with and without CUDA).
--**0.2.5** released in the week of March 12, 2024 - continued enhancements of the GGUF implementation, especially for CUDA support, and re-compiling of all binaries to support Ubuntu 20 and Ubuntu 22. Ubuntu requirements are: CUDA 12.1 (to use GPU), and GLIBC 2.31+.
--**GGUF on Windows CUDA**: useful notes and debugging tips -
1. Requirement: Nvidia CUDA 12.1+
-- how to check: `nvcc --version` and `nvidia-smi` - if not found, then drivers are either not installed or not in $PATH and need to be configured
-- if you have older drivers (e.g., v11), then you will need to update them.
2. Requirement: CUDA-enabled Pytorch (pre-0.2.11)
-- starting with 0.2.11, we have implemented a custom check to evaluate if CUDA is present, independent of Pytorch.
-- for pre-0.2.11, we use Pytorch to check for CUDA drivers, e.g., `torch.cuda.is_available()` and `torch.version.cuda`
3. Installing a CUDA-enabled Pytorch - useful install script: (not required post-0.2.11 for GGUF on Windows)
-- `pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121`
4. Fall-back to CPU - if llmware can not load the CUDA-enabled drivers, it will automatically try to fall back to the CPU version of the drivers.
-- you can also adjust the GGUFConfigs().set_config - ("use_gpu", False) - and then it will automatically go to the CPU drivers.
5. Custom GGUF libraries - if you have a unique system requirement, you can build llama_cpp from source, and apply custom build settings - or find in the community a prebuilt llama_cpp library that matches your platform. Happy to help if you share the requirements.
-- to "bring your own GGUF": GGUFConfigs().set_config("custom_lib_path", "/path/to/your/custom/llama_cpp_backend" -> and llmware will try to load that library.
6. Issues? - please raise an Issue on Github, or on Discord - and we can work with you to get you up and running!
--**0.2.4** released in the week of February 26, 2024 - major upgrade of GGUF implementation to support more options, including CUDA support - which is the main source of growth in the size of the wheel package.
-- Note: We will look at making some of the CUDA builds as 'optional' or 'bring your own' over time.
-- Note: We will also start to 'prune' the list of wheels kept in the archive to keep the total repo size manageable for cloning.
--**0.2.2** introduced SLIM models and the new LLMfx class, and the capabilities for multi-model, multi-step Agent-based processes.
--**0.2.0** released in the week of January 22, 2024 - significant enhancements, including integration of Postgres and SQLite drivers into the c lib parsers.
--New examples involving Postgres or SQLite support (including 'Fast Start' examples) will require a fresh pip install of 0.2.0 or clone of the repo.
--If cloning the repo, please be especially careful to pick up the new updated /lib dependencies for your platform.
--New libs have new dependencies in Linux in particular - most extensive testing on Ubuntu 22. If any issues on a specific version of Linux, please raise a ticket.
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/components/slim_models.md
+304
View File
@@ -0,0 +1,304 @@
---
layout: default
title: SLIM Models
parent: Components
nav_order: 5
description: overview of the major modules and classes of LLMWare
permalink: /components/slim_models
---
# SLIM Models - Function Calling with Small Language Models
---
Generally, function-calling is a specialized capability of frontier language models, such as OpenAI GPT4.
We have adapted this concept to small language models through SLIMs (Structured Language Instruction Models),
which are 'single function' models fine-tuned to accept three main inputs to construct a prompt:
As of June 2024, there are 18 distinct SLIM function calling models with many more on the way, for most common
extraction, classification, and summarization tasks:
**Models List**
If you would like more information about any of the SLIM models, please check out their model card:
- extract - extract custom keys - [slim-extract](https://www.huggingface.co/llmware/slim-extract) & [slim-extract-tool](https://www.huggingface.co/llmware/slim-extract-tool)
- summary - summarize function call - [slim-summary](https://www.huggingface.co/llmware/slim-summary) & [slim-summary-tool](https://www.huggingface.co/llmware/slim-summary-tool)
- xsum - title/headline function call - [slim-xsum](https://www.huggingface.co/llmware/slim-xsum) & [slim-xsum-tool](https://www.huggingface.co/llmware/slim-xsum-tool)
- ner - extract named entities - [slim-ner](https://www.huggingface.co/llmware/slim-ner) & [slim-ner-tool](https://www.huggingface.co/llmware/slim-ner-tool)
- sentiment - evaluate sentiment - [slim-sentiment](https://www.huggingface.co/slim-sentiment) & [slim-sentiment-tool](https://www.huggingface.co/llmware/slim-sentiment-tool)
- topics - generate topic - [slim-topics](https://www.huggingface.co/slim-topics) & [slim-topics-tool](https://www.huggingface.co/llmware/slim-topics-tool)
- sa-ner - combo model (sentiment + named entities) - [slim-sa-ner](https://www.huggingface.co/slim-sa-ner) & [slim-sa-ner-tool](https://www.huggingface.co/llmware/slim-sa-ner-tool)
- boolean - provides a yes/no output with explanation - [slim-boolean](https://www.huggingface.co/slim-boolean) & [slim-boolean-tool](https://www.huggingface.com/llmware/slim-boolean-tool)
- ratings - apply 1 (low) - 5 (high) rating - [slim-ratings](https://www.huggingface.co/slim-ratings) & [slim-ratings-tool](https://www.huggingface.co/llmware/slim-ratings-tool)
- emotions - assess emotions - [slim-emotions](https://www.huggingface.co/slim-emotions) & [slim-emotions-tool](https://www.huggingface.co/llmware/slim-emotions-tool)
- tags - auto-generate list of tags - [slim-tags](https://www.huggingface.co/slim-tags) & [slim-tags-tool](https://www.huggingface.co/llmware/slim-tags-tool)
- tags-3b - enhanced auto-generation tagging model - [slim-tags-3b](https://www.huggingface.com/slim-tags-3b) & [slim-tags-3b-tool](https://www.huggingface.co/llmware/slim-tags-3b-tool)
- intent - identify intent - [slim-intent](https://www.huggingface.co/slim-intent) & [slim-intent-tool](https://www.huggingface.co/llmware/slim-intent-tool)
- category - high-level category - [slim-category](https://www.huggingface.co/slim-category) & [slim-category-tool](https://wwww.huggingface.co/llmware/slim-category-tool)
- nli - assess if evidence supports conclusion - [slim-nli](https://www.huggingface.co/slim-nli) & [slim-nli-tool](https://www.huggingface.co/llmware/slim-nli-tool)
- sql - convert text into sql - [slim-sql](https://www.huggingface.co/slim-sql) & [slim-sql-tool](https://www.huggingface.co/llmware/slim-sql-tool)
You may also want to check out these quantized 'answer' tools, which work well in conjunction with SLIMs for question-answer and summarization:
- bling-stablelm-3b-tool - 3b quantized RAG model - [bling-stablelm-3b-gguf](https://www.huggingface.co/llmware/bling-stablelm-3b-gguf)
- bling-answer-tool - 1b quantized RAG model - [bling-answer-tool](https://www.huggingface.co/llmware/bling-answer-tool)
- dragon-yi-answer-tool - 6b quantized RAG model - [dragon-yi-answer-tool](https://www.huggingface.co/llmware/dragon-yi-answer-tool)
- dragon-mistral-answer-tool - 7b quantized RAG model - [dragon-mistral-answer-tool](https://www.huggingface.co/llmware/dragon-mistral-answer-tool)
- dragon-llama-answer-tool - 7b quantized RAG model - [dragon-llama-answer-tool](https://www.huggingface.co/llmware/dragon-llama-answer-tool)
All SLIM models have a common prompting structure
Inputs:
-- text passage - this is the core passage or piece of text that you would like the model to assess
-- function - classify, extract, generate - this is handled by default by the model class, so usually does
not need to be explicitly declared - but is an option for SLIMs that support more than one function
-- params - depends upon the model, used to configure/guide the behavior of the function call - optional for
some SLIMs
Outputs:
-- structured python output, generally either a dictionary or list
Main objectives:
-- enable function calling with small, locally-running models,
-- simplify prompts by defining specific functions and fine-tuning the model to respond accordingly
without 'prompt magic'
-- standardized outputs that can be handled programmatically as part of a multi-step workflow.
```python
from llmware.models import ModelCatalog
def discover_slim_models():
""" Discover a list of SLIM tools in the Model Catalog.
-- SLIMs are available in both traditional Pytorch and quantized GGUF packages.
-- Generally, we train/fine-tune in Pytorch and then package in 4-bit quantized GGUF for inference.
-- By default, we designate the GGUF versions with 'tool' or 'gguf' in their names.
-- GGUF versions are generally faster to load, faster for inference and use less memory in most environments."""
tools = ModelCatalog().list_llm_tools()
tool_map = ModelCatalog().get_llm_fx_mapping()
print("\nList of SLIM model tools (GGUF) in the ModelCatalog\n")
for i, tool in enumerate(tools):
model_card = ModelCatalog().lookup_model_card(tool_map[tool])
print(f"{i} - tool: {tool} - "
f"model_name: {model_card['model_name']} - "
f"model_family: {model_card['model_family']}")
return 0
def hello_world_slim():
""" SLIM models can be identified in the ModelCatalog like any llmware model. Instead of using
inference method, SLIM models are used with the function_call method that prepares a special prompt
instruction, and takes optional parameters.
This example shows a series of function calls with different SLIM models.
Please note that the first time the models will be pulled from the llmware Huggingface repository, and will
take a couple of minutes. Future calls will be much faster once cached in memory locally. """
print("\nExecuting Function Call Inferences with SLIMs\n")
# Sentiment Analysis
passage1 = ("This is one of the best quarters we can remember for the industrial sector "
"with significant growth across the board in new order volume, as well as price "
"increases in excess of inflation. We continue to see very strong demand, especially "
"in Asia and Europe. Accordingly, we remain bullish on the tier 1 suppliers and would "
"be accumulating more stock on any dips.")
# here are the two key lines of code
model = ModelCatalog().load_model("slim-sentiment-tool")
response = model.function_call(passage1)
print("sentiment response: ", response['llm_response'])
# Named Entity Recognition
passage2 = "Michael Johnson was a famous Olympic sprinter from the U.S. in the early 2000s."
model = ModelCatalog().load_model("slim-ner-tool")
response = model.function_call(passage2)
print("ner response: ", response['llm_response'])
# Extract anything with Slim-extract
passage3 = ("Adobe shares tumbled as much as 11% in extended trading Thursday after the design software maker "
"issued strong fiscal first-quarter results but came up slightly short on quarterly revenue guidance. "
"Here’s how the company did, compared with estimates from analysts polled by LSEG, formerly known as Refinitiv: "
"Earnings per share: $4.48 adjusted vs. $4.38 expected Revenue: $5.18 billion vs. $5.14 billion expected "
"Adobe’s revenue grew 11% year over year in the quarter, which ended March 1, according to a statement. "
"Net income decreased to $620 million, or $1.36 per share, from $1.25 billion, or $2.71 per share, "
"in the same quarter a year ago. During the quarter, Adobe abandoned its $20 billion acquisition of "
"design software startup Figma after U.K. regulators found competitive concerns. The company paid "
"Figma a $1 billion termination fee.")
model = ModelCatalog().load_model("slim-extract-tool")
response = model.function_call(passage3, function="extract", params=["revenue growth"])
print("extract response: ", response['llm_response'])
# Generate questions with Slim-Q-Gen
model = ModelCatalog().load_model("slim-q-gen-tiny-tool", temperature=0.2, sample=True)
# supported params - "question", "multiple choice", "boolean"
response = model.function_call(passage3, params=['multiple choice'])
print("question generation response: ", response['llm_response'])
# Generate topic
model = ModelCatalog().load_model("slim-topics-tool")
response = model.function_call(passage3)
print("topics response: ", response['llm_response'])
# Generate headline summary with slim-xsum
model = ModelCatalog().load_model("slim-xsum-tool", temperature=0.0, sample=False)
response = model.function_call(passage3)
print("xsum response: ", response['llm_response'])
# Generate boolean with optional '(explain)` in parameter
model = ModelCatalog().load_model("slim-boolean-tool")
response = model.function_call(passage3, params=["Did Adobe revenue increase? (explain)"])
print("boolean response: ", response['llm_response'])
# Generate tags
model = ModelCatalog().load_model("slim-tags-tool", temperature=0.0, sample=False)
response = model.function_call(passage3)
print("tags response: ", response['llm_response'])
return 0
def using_logits_and_integrating_into_process():
""" This example shows two key elements of function calling SLIM models -
1. Using Logit Information to indicate confidence levels, especially for classifications.
2. Using the structured dictionary generated for programmatic handling in a larger process.
"""
print("\nExample: using logits and integrating into process\n")
text_passage = ("On balance, this was an average result, with earnings in line with expectations and "
"no big surprises to either the positive or the negative.")
# two key lines (load_model + execute function_call) + additional logit_analysis step
sentiment_model = ModelCatalog().load_model("slim-sentiment-tool", get_logits=True)
response = sentiment_model.function_call(text_passage)
analysis = ModelCatalog().logit_analysis(response,sentiment_model.model_card, sentiment_model.hf_tokenizer_name)
print("sentiment response: ", response['llm_response'])
print("\nAnalyzing response")
for keys, values in analysis.items():
print(f"{keys} - {values}")
# two key attributes of the sentiment output dictionary
sentiment_value = response["llm_response"]["sentiment"]
confidence_level = analysis["confidence_score"]
# use the sentiment classification as a 'if...then' decision point in a process
if "positive" in sentiment_value:
print("sentiment is positive .... will take 'positive' analysis path ...", sentiment_value)
else:
print("sentiment is negative .... will take 'negative' analysis path ...", sentiment_value)
if "positive" in sentiment_value and confidence_level > 0.8:
print("sentiment is positive with high confidence ... ", sentiment_value, confidence_level)
return 0
if __name__ == "__main__":
# discovering slim models in the llmware catalog
discover_slim_models()
# running function call inferences
hello_world_slim()
# doing interesting stuff with the output
using_logits_and_integrating_into_process()
```
Need help or have questions?
============================
Check out the [llmware videos](https://www.youtube.com/@llmware) and [GitHub repository](https://github.com/llmware-ai/llmware).
Reach out to us on [GitHub Discussions](https://github.com/llmware-ai/llmware/discussions).
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in Oktober 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/components/vector_databases.md
+173
View File
@@ -0,0 +1,173 @@
---
layout: default
title: Vector Databases
parent: Components
nav_order: 11
description: overview of the major modules and classes of LLMWare
permalink: /components/vector_databases
---
# Vector Databases
---
llmware supports the following vector databases:
- Milvus and Milvus-Lite - `milvus`
- Postgres (PG Vector) - `postgres`
- Qdrant - `qdrant`
- ChromaDB - `chromadb`
- Redis - `redis`
- Neo4j - `neo4j`
- LanceDB - `lancedb`
- FAISS - `faiss`
- Mongo-Atlas - `mongo-atlas`
- Pinecone - `pinecone`
In llmware, unstructured content is ingested and organized into a Library, and then embeddings are created against the
Library object, and usually, handled by implicitly through the Library method `.install_new_embedding`.
All embedding models are implemented through the embeddings.py module, and the `EmbeddingHandler` class, which routes
the embedding process to the vector db specific handler and provides a common set of utility functions.
In most cases, it is not necessarily to explicitly call the vector db class.
The design is intended to promote code re-use and to make it easy to experiment with different endpoint vector databases
without significant code changes, as well as to leverage the Library as the core organizing construct.
# Select Vector DB
To select a vector database in llmware is generally done is one of two ways:
1. Explicit Setting - `LLMWareConfig().set_vector_db("postgres")`
2. Pass the name of the vector database at the time of installing the embeddings:
`library.install_new_embedding(embedding_model_name=embedding_model, vector_db='milvus',batch_size=100)`
# Install Vector DB
No-install options: chromadb, lancedb, faiss, and milvus-lite
API-based options: mongo-atlas, pinecone
Install server options:
Generally, we have found that Docker (and Docker-Compose) are the easiest and most consistent ways to install vector
db across different platforms.
1. milvus - we provide a docker-compose script in the main repository root folder path, which installs mongodb as well.
```bash
curl -o docker-compose.yaml https://raw.githubusercontent.com/llmware-ai/llmware/main/docker-compose_mongo_milvus.yaml
docker compose up -d
```
2. qdrant
```bash
curl -o docker-compose.yaml https://raw.githubusercontent.com/llmware-ai/llmware/main/docker-compose-qdrant.yaml
docker compose up -d
```
3. postgres and pgvector
```bash
curl -o docker-compose.yaml https://raw.githubusercontent.com/llmware-ai/llmware/main/docker-compose-pgvector.yaml
docker compose up -d
```
4. redis
```bash
# scripts to deploy other options
curl -o docker-compose.yaml https://raw.githubusercontent.com/llmware-ai/llmware/main/docker-compose-redis-stack.yaml
```
5. neo4j
```bash
curl -o docker-compose.yaml https://raw.githubusercontent.com/llmware-ai/llmware/main/docker-compose-neo4j.yaml
docker compose up -d
```
# Configure Vector DB
To configure a vector database in llmware, we provide configuration objects in the `configs.py` module to adjust
authentication, port/host information, and other common configurations. To use the configuration, the pattern is
as follows through simple `get_config` and `set_config` methods:
```python
from llmware.configs import MilvusConfig
MilvusConfig().set_config("lite", True)
from llmware.configs import ChromaDBConfig
current_config = ChromaDBConfig().get_config("persistent_path")
ChromaDBConfig().set_config("persistent_path", "/new/local/path")
```
Configuration objects are provided for the following vector DB: `MilvusConfig`, `ChromaDBConfig`, `QdrantConfig`,
`Neo4jConfig`, `LanceDBConfig`, `PineConeConfig`, `MongoConfig`, `PostgresConfig`.
For 'out-of-the-box' testing and development, for most use cases, you will not need to change these configs.
Need help or have questions?
============================
Check out the [llmware videos](https://www.youtube.com/@llmware) and [GitHub repository](https://github.com/llmware-ai/llmware).
Reach out to us on [GitHub Discussions](https://github.com/llmware-ai/llmware/discussions).
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in Oktober 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/components/whisper_cpp.md
+203
View File
@@ -0,0 +1,203 @@
---
layout: default
title: Whisper CPP
parent: Components
nav_order: 14
description: overview of the major modules and classes of LLMWare
permalink: /components/whisper_cpp
---
# Whisper CPP
---
llmware has an integrated WhisperCPP backend which enables fast, easy local voice-to-text processing.
Whisper is a leading open voice voice-to-text model from OpenAI - https://github.com/openai/whisper
WhisperCPP is the implementation of Whisper packaged as a GGML deliverable - https://github.com/ggerganov/whisper.cpp
Starting with llmware 0.2.11, we have integrated WhisperCPPModel as a new model class,
providing options for direct inference, and coming soon, integration into the Parser for easy text chunking and
parsing into a Library with other document types.
llmware provides prebuilt shared libraries for WhisperCPP on the following platforms:
--Mac M series
--Linux x86 (no CUDA)
--Linux x86 (with CUDA) - really fast
--Windows x86 (only on CPU) currently.
We have added three Whisper models to the default model catalog:
1. ggml-base.en.bin - english-only base model
2. ggml-base.bin - multi-lingual base model
3. ggml-small.en-tdrz.bin - this is a 'tiny-diarize' implementation that has been finetuned to identify the
speakers and inserts special [_SOLM_] tags to indicate a conversation turn / change of speaker.
Main repo: https://github.com/akashmjn/tinydiarize/
Citation: @software{mahajan2023tinydiarize,
author = {Mahajan, Akash}, month = {08},
title = {tinydiarize: Minimal extension of Whisper for speaker segmentation with special tokens}
url = {https://github.com/akashmjn/tinydiarize},
year = {2023}
To use WAV files, there is one additional Python dependency required:
--pip install librosa
--Note: this has been added to the default requirements.txt and pypy build starting with 0.2.11
To use other popular audio/video file formats, such as MP3, MP4, M4A, etc., then the following dependencies are
required:
--pip install pydub
--ffmpeg library - which can be installed as follows:
-- Linux: `sudo apt install ffmpeg'
-- Mac: `brew install ffmpeg`
-- Windows: direct download and install from ffmpeg
```python
""" This example shows how to use llmware provided sample files for testing with WhisperCPP, integrated as of
llmware 0.2.11.
# examples - "famous_quotes" | "greatest_speeches" | "youtube_demos" | "earnings_calls"
-- famous_quotes - approximately 20 small .wav files with clips from old movies and speeches
-- greatest_speeches - approximately 60 famous historical speeches in english
-- youtube_videos - wav files of ~3 llmware youtube videos
-- earnings_calls - wav files of ~4 public company earnings calls (gathered from public investor relations)
These sample files are hosted in a non-restricted AWS S3 bucket, and downloaded via the Setup method
`load_sample_voice_files`. There are two options:
-- small_only = True: only pulls the 'famous_quotes' samples
-- small_only = False: pulls all of the samples (requires ~1.9 GB in total)
Please note that all of these samples have been pulled from open public domain sources, including the
Internet Archives, e.g., https://archive.org. These sample files are being provided solely for the purpose of
testing the code scripts below. Please do not use them for any other purpose.
To run these examples, please make sure to `pip install librosa`
"""
import os
from llmware.models import ModelCatalog
from llmware.gguf_configs import GGUFConfigs
from llmware.setup import Setup
# optional / to adjust various parameters of the model
GGUFConfigs().set_config("whisper_cpp_verbose", "OFF")
GGUFConfigs().set_config("whisper_cpp_realtime_display", True)
# note: english is default output - change to 'es' | 'fr' | 'de' | 'it' ...
GGUFConfigs().set_config("whisper_language", "en")
GGUFConfigs().set_config("whisper_remove_segment_markers", True)
def sample_files(example="famous_quotes", small_only=False):
""" Execute a basic inference on Voice-to-Text model passing a file_path string """
voice_samples = Setup().load_voice_sample_files(small_only=small_only)
examples = ["famous_quotes", "greatest_speeches", "youtube_demos", "earnings_calls"]
if example not in examples:
print("choose one of the following - ", examples)
return 0
fp = os.path.join(voice_samples,example)
files = os.listdir(fp)
# these are the two key lines
whisper_base_english = "whisper-cpp-base-english"
model = ModelCatalog().load_model(whisper_base_english)
for f in files:
if f.endswith(".wav"):
prompt = os.path.join(fp,f)
print(f"\n\nPROCESSING: prompt = {prompt}")
response = model.inference(prompt)
print("\nllm response: ", response["llm_response"])
print("usage: ", response["usage"])
return 0
if __name__ == "__main__":
# pick among the four examples: famous_quotes | greatest_speeches | youtube_demos | earnings_calls
sample_files(example="famous_quotes", small_only=False)
```
Need help or have questions?
============================
Check out the [llmware videos](https://www.youtube.com/@llmware) and [GitHub repository](https://github.com/llmware-ai/llmware).
Reach out to us on [GitHub Discussions](https://github.com/llmware-ai/llmware/discussions).
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in Oktober 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/contributing/code.md
+307
View File
@@ -0,0 +1,307 @@
---
layout: default
title: Code contributions
parent: Contributing
nav_order: 1
permalink: /contributing/code
---
# Contributing code
One way to contribute to ``llmware`` is by contributing to the code base.
We briefly describe some of the important modules of ``llmware`` next, so you can more easily navigate the code base.
You may also take a look at our [fast start series from YouTube](https://www.youtube.com/playlist?list=PL1-dn33KwsmD7SB9iSO6vx4ZLRAWea1DB).
## Core modules
### Library
<iframe width="560" height="315" src="https://www.youtube.com/embed/2xDefZ4oBOM?si=IAHkxpQkFwnWyYTL" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
The *library* module implements the classes **Library** and **LibraryCatalog**.
The **Library** class implements the *library* concept.
A *library* is a collection of documents, where a document can be PDF, an image, or an office document.
It is responsible for parsing, text chunking, and indexing.
In other words, it does the heavy lifting of adding content.
In the following, we shortly describe the functions for adding documents to the library.
```python
add_file(
self,
file_path):
```
This method adds one document of any supported type to the library.
```python
add_files(
self,
input_folder_path=None,
encoding="utf-8",
chunk_size=400,
get_images=True,get_tables=True,
smart_chunking=2,
max_chunk_size=600,
table_grid=True,
get_header_text=True,
table_strategy=1,
strip_header=False,
verbose_level=2,
copy_files_to_library=True):
```
This method adds the documents of one folder to the library.
```python
add_website(
self,
url,
get_links=True,
max_links=5):
```
This method adds a website, and links from the website, to the library.
```python
add_wiki(
self,
topic_list,
target_results=10):
```
This method adds a wikipedia article to the library.
```python
add_dialogs(
self,
input_folder=None):
```
This method adds an AWS dialog transcript to the library.
```python
add_image(
self,
input_folder=None):
```
This method adds images to the libary.
```python
add_pdf_by_ocr(
self,
input_folder=None):
```
This method adds scanned PDFs to the library.
```python
add_pdf(
self,
input_folder=None):
```
This method adds PDFs to the library.
```python
add_office(
self,
input_folder=None):
```
This method adds office documents to the library.
### Embeddings
<iframe width="560" height="315" src="https://www.youtube.com/embed/xQEk6ohvfV0?si=GAPle5gVdVPkYKWv" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
An *embedding* is a vector store and an embedding model.
It is responsible for applying an embedding model to a library, storing the embeddings in a vector store, and providing access to the embeddings with natural language queries.
We briefly describe the common methods offered by all vector stores below.
```python
def create_new_embedding(
self,
doc_ids=None,
batch_size=500):
```
This method creates the embeddings and adds them to the vector store.
```python
def search_index(
self,
query_embedding_vector,
sample_count=10):
```
This method searches the vector store given the query vector.
```python
def delete_index(self):
```
This method deletes the created vector store index.
### Prompts
<iframe width="560" height="315" src="https://www.youtube.com/embed/swiu4oBVfbA?si=rKbgO3USADCqICqr" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
A *prompt* is an input to model.
The prompt is used by the model to generate the response.
One important use case is that users want to augment a prompt, or a series of prompts, with additional information.
Next, we describe methods for augmenting a prompt with additional information.
```python
def add_source_new_query(
self,
library,
query=None,
query_type="semantic",
result_count=10):
```
This method adds the results of the ``query`` to the prompt.
```python
def add_source_query_results(
self,
query_results):
```
This method adds previous results from a query as a source to the prompt.
```python
def add_source_library(
self,
library_name):
```
This method adds an entire library to the prompt.
We recommend that you only use this when the library is sufficiently small.
```python
def add_source_wikipedia(
self,
topic,
article_count=3,
query=None):
```
This method adds wikipedia articles to the prompt based on the provided ``topic``.
```python
def add_source_yahoo_finance(
self,
ticker=None,
key_list=None):
```
This method adds a Yahoo finance ticker to the prompt.
```python
def add_source_knowledge_graph(
self,
library,
query):
```
This method adds the summary output elements from a knowledge graph based on the provided ``query``.
Please note that this method is experimental, i.e. unstable, and is subject to change dramatically in each new version.
```python
def add_source_website(
self,
url,
query=None):
```
This method adds the website pointed to by the ``url`` to the prompt.
```python
def add_source_document(
self,
input_fp,
input_fn,
query=None):
```
This method adds a document, or documents, of any supported type to the prompt.
If documents are added, then the ``query`` parameter can be used to filter the documents.
```python
def add_source_last_interaction_step(
self):
```
This method adds the last interaction to the prompt.
The use case for this is to enable interactive dialog, i.e. chatting.
### Model Catalog
A *model catalog* is a collection of models.
In the following, we briefly describe the methods for adding new models to the catalog.
```python
def register_new_hf_generative_model(
self,
hf_model_name=None,
context_window=2048,
prompt_wrapper="<INST>",
display_name=None,
temperature=0.3,
trailing_space="",
link=""):
```
This method adds a new generative model from hugging face.
Users can therefore add models from hugging face that are unsupported currently.
```python
def register_sentence_transformer_model(
self,
model_name,
embedding_dims,
context_window,
display_name=None,
link=""):
```
This method adds a new sentence transformer.
```python
def register_gguf_model(
self,
model_name,
gguf_model_repo,
gguf_model_file_name,
prompt_wrapper=None,
eos_token_id=0,
display_name=None,
trailing_space="",
temperature=0.3,
context_window=2048,
instruction_following=True):
```
This method adds a new GGUF model.
```python
def register_open_chat_model(
cls,
model_name,
api_base=None,
model_type="chat",
display_name=None,
context_window=4096,
instruction_following=True,
prompt_wrapper="",
temperature=0.5):
```
This method adds any chat model that is available through a web API, e.g. a chat model that is available locally
via localhost.
```python
def register_ollama_model(
cls,
model_name,
host="localhost",
port=11434,
model_type="chat",
raw=False,
stream=False,
display_name=None,
context_window=4096,
instruction_following=True,
prompt_wrapper="",
temperature=0.5):
```
This method adds an OLLama model that is available through a web API.
The method is similar to the ``register_open_chat_model`` method above.
### Categories of code contributions
#### New or Enhancement to existing Features
You want to submit a code contribution that adds a new feature or enhances an existing one?
Then the best way to start is by opening a discussion in our [GitHub discussions](https://github.com/llmware-ai/llmware/discussions).
Please do this before you work on it, so you do not put effort into it just to realise after submission that
it will not be merged.
#### Bugs
If you encounter a bug, you can
- File an issue about the bug.
- Provide a self-contained minimal example that reproduces the bug, which is extremely important.
- Provide possible solutions for the bug.
- Submit a pull a request to fix the bug.
We encourage you to read [How to create a Minimal, Reproducible Example](https://stackoverflow.com/help/minimal-reproducible-example) from the Stackoverflow helpcenter, and the tag description of [self-container](https://stackoverflow.com/tags/self-contained/info), also from Stackoverflow.
docs/contributing/contributing.md
+94
View File
@@ -0,0 +1,94 @@
---
layout: default
title: Contributing
nav_order: 7
has_children: true
description: llmware contributions.
permalink: /contributing
---
# Contributing to llmware
{: .note}
> The contributions to `llmware` are governed by our [Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md).
{: .warning}
> Have you found a security issue? Then please jump to [Security Vulnerabilities](#security-vulnerabilities).
On this page, we provide information ``llmware`` contributions.
There are **two ways** on how you can contribute.
The first is by making **code contributions**, and the second by making contributions to the **documentation**.
Please look at our [contribution suggestions](#how-can-you-contribute) if you need inspiration, or take a look at [open issues](#open-issues).
Contributions to `llmware` are welcome from everyone.
Our goal is to make the process simple, transparent, and straightforward.
We are happy to receive suggestions on how the process can be improved.
## How can you contribute?
{: .note}
> If you have never contributed before look for issues with the tag [``good first issue``](https://github.com/llmware-ai/llmware/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22).
The most usual ways to contribute is to add new features, fix bugs, add tests, or add documentation.
You can visit the [issues](https://github.com/llmware-ai/llmware/issues) site of the project and search for tags such as
``bug``, ``enhancement``, ``documentation``, or ``test``.
Here is a non exhaustive list of contributions you can make.
1. Code refactoring
2. Add new text data bases
3. Add new vector data bases
4. Fix bugs
5. Add usage examples (see for example the issues [jupyter notebook - more examples and better support](https://github.com/llmware-ai/llmware/issues/508) and [google colab examples and start up scripts](https://github.com/llmware-ai/llmware/issues/507))
6. Add experimental features
7. Improve code quality
8. Improve documentation in the docs (what you are reading right now)
9. Improve documentation by adding or updating docstrings in modules, classes, methods, or functions (see for example [Add docstrings](https://github.com/llmware-ai/llmware/issues/219))
10. Improve test coverage
11. Answer questions in our [Discord channel](https://discord.gg/MhZn5Nc39h), especially in the [technical support forum](https://discord.com/channels/1179245642770559067/1218498778915672194)
12. Post projects in which you use ``llmware`` in our Discord forum [made with llmware](https://discord.com/channels/1179245642770559067/1218567269471486012), ideially with a link to a public GitHub repository
## Open Issues
If you're interested in existing issues, you can
- Look for issues, if you are a new to the project, look for issues with the `good first issue` label.
- Provide answers for questions in our [GitHub discussions](https://github.com/llmware-ai/llmware/discussions)
- Provide help for bug or enhancement issues.
- Ask questions, reproduce the issues, or provide solutions.
- Pull a request to fix the issue.
## Security Vulnerabilities
**If you believe you've found a security vulnerability, then please _do not_ submit an issue ticket or pull request or otherwise publicly disclose the issue.**
Please follow the process at [Reporting a Vulnerability](https://github.com/llmware-ai/llmware/blob/main/Security.md)
## GitHub workflow
We follow the [``fork-and-pull``](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork) Git workflow.
1. [Fork](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) the repository on GitHub.
2. Clone your fork to your local machine with `git clone git@github.com:<yourname>/llmware.git`.
3. Create a branch with `git checkout -b my-topic-branch`.
4. Run the test suite by navigating to the tests/ folder and running ```./run-tests.py -s``` to ensure there are no failures
5. [Commit](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/committing-changes-to-a-pull-request-branch-created-from-a-fork) changes to your own branch, then push to GitHub with `git push origin my-topic-branch`.
6. Submit a [pull request](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests) so that we can review your changes.
Remember to [synchronize your forked repository](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo#keep-your-fork-synced) _before_ submitting proposed changes upstream. If you have an existing local repository, please update it before you start, to minimize the chance of merge conflicts.
```shell
git remote add upstream git@github.com:llmware-ai/llmware.git
git fetch upstream
git checkout upstream/main -b my-topic-branch
```
## Community
Questions and discussions are welcome in any shape or form.
Please fell free to join our community on our discord channel, on which we are active daily.
You are also welcome if you just want to post an idea!
- [Discord Channel](https://discord.gg/MhZn5Nc39h)
- [GitHub discussions](https://github.com/llmware-ai/llmware/discussions)
docs/contributing/documentation.md
+66
View File
@@ -0,0 +1,66 @@
---
layout: default
title: Documentation contributions
parent: Contributing
nav_order: 2
permalink: contributing/documentation
---
# Contributing documentation
One way to contribute to ``llmware`` is by contributing documentation.
There are **two ways** to contribute to the ``llmware`` documentation.
The first is via **docstrings in the code**, and the second is **the docs**, which is what you are *currently reading*.
In both areas, you can contribute in a lot of ways.
Here is a non exhaustive list of these ways for the docstrings which also apply to the docs.
1. Add documentation (e.g., adding a docstring to a function)
2. Update documentation (e.g., update a docstring that is not in sync with the code)
3. Simplify documentation (e.g., formulate a docstring more clearly)
4. Enhance documentation (e.g., add more examples to a docstring or fix typos)
## Docstrings
**Docstrings** document the code within the code, which allows programmers to easily have a look while they are programming.
For an exmaple, have a look at [this docstring](https://github.com/llmware-ai/llmware/blob/c9e12a7a150162986622738e127c37ac70f31cd6/llmware/agents.py#L27-L66) which documents the ``LLMfx`` class.
We follow the docstring style of **numpy**, for which you can find an example [here](https://github.com/numpy/numpydoc/blob/main/doc/example.py) and [here](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html).
Please be sure to follow the conventions and go over your pull request before you submit it.
## Docs
{: .note}
> All commands are executed from the `docs` sub-directory.
Contributing to this documentation is extremely important as many users will refer to it.
If you plan to contribute to the docs, we recommend that you locally install `jekyll` so you can test your changes locally.
We also recommend, that you install `jekyll` into a a ruby enviroment so it does not interfere with any other installations you might have.
We recommend that you install `rbenv` and `rvm` to manage your ruby installation.
`rbenv` is a tool that mangages different ruby versions, similar to what `conda` does for `python`.
Please [install rbenv](https://github.com/rbenv/rbenv?tab=readme-ov-file#installation) following their instructions, and the same for [install rvm](https://github.com/rvm/rvm?tab=readme-ov-file#installing-rvm).
We recommend that you install a ruby version `>=3.0`.
After having installed an isolated ruby version, you have to install the dependencies to build the docs locally.
The `docs` directory has a `Gemfile` which specifies the dependencies.
You can hence simply navigate to it and use the `bundle install` command.
```bash
bundle install
```
You should now be able to build and serve the documentation locally.
To do this, simply to the following.
```bash
bundle exec jekyll server --livereload --verbose
```
In the browser of your choice, you can then go to `http://127.0.0.1:4000/` and you will be served the documentation, which is re-build and re-loaded after any change to the `docs`.
``jekyll`` will create a ``_site`` directory where it saves the created files, please **never commit any files from the \_site directory**!
## Open Issues
If you're interested in existing issues, you can
- Look for issues with the `good first issue` and `documentation` label as a good place to get started.
- Provide answers for questions in our [GitHub discussions](https://github.com/llmware-ai/llmware/discussions)
- Provide help for bug or enhancement issues.
- Ask questions, reproduce the issues, or provide solutions.
- Pull a request to fix the issue.
docs/examples/agents.md
+98
View File
@@ -0,0 +1,98 @@
---
layout: default
title: Agents
parent: Examples
nav_order: 2
description: overview of the major modules and classes of LLMWare
permalink: /examples/agents
---
# Agents
🚀 Start Building Multi-Model Agents Locally on a Laptop 🚀
===============
**What is a SLIM?**
**SLIMs** are **S**tructured **L**anguage **I**nstruction **M**odels, which are small, specialized 1-3B parameter LLMs,
finetuned to generate structured outputs (Python dictionaries and lists, JSON and SQL) that can be handled programmatically, and
stacked together in multi-step, multi-model Agent workflows - all running on a local CPU.
**New SLIMS Just released** - check out slim-extract, slim-summarize, slim-xsum, slim-sa-ner, slim-boolean and slim-tags-3b
**Check out the new examples below marked with ⭐**
🔥🔥🔥 Web Services & Function Calls ([code](web_services_slim_fx.py)) 🔥🔥🔥
**Check out the Intro videos**
[SLIM Intro Video](https://www.youtube.com/watch?v=cQfdaTcmBpY)
There are 16 SLIM models, each delivered in two packages - a Pytorch/Huggingface FP16 model, and a
quantized "tool" designed for fast inference on a CPU, using LLMWare's embedded GGUF inference engine. In most cases,
we would recommend that you start with the "tools" version of the models.
**Getting Started**
We have several ready-to-run examples in this repository:
| Example | Detail |
|-----------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------|
| 1. Getting Started with SLIM Models ([code](slims-getting-started.py) / [video](https://www.youtube.com/watch?v=aWZFrTDmMPc&t=196s)) | Install the models and run hello world tests to see the models in action. |
| 2. Getting Started with Function-Calling Agent ([code](agent-llmfx-getting-started.py) / [video](https://www.youtube.com/watch?v=cQfdaTcmBpY)) | Generate a Structured Report with LLMfx |
| 3. Multi-step Complex Analysis with Agent ([code](agent-multistep-analysis.py) / [video](https://www.youtube.com/watch?v=y4WvwHqRR60)) | Delivering Complex Research Analysis with SLIM Agents | |
| 4. Document Clustering ([code](document-clustering.py)) | Multi-faceted automated document analysis with Topics, Tags and NER |
| 5. Two-Step NER Retrieval ([code](ner-retrieval.py)) | Using NER to extract name, and then using as basis for retrieval. | |
| 6. Using Sentiment Analysis ([code](sentiment-analysis.py)) | Using sentiment analysis on earnings transcripts and a 'if...then' condition |
| 7. Text2SQL - Intro ([code](text2sql-getting-started-1.py)) | Getting Started with SLIM-SQL-TOOL and Basic Text2SQL Inference | |
| 8. Text2SQL - E2E ([code](text2sql-end-to-end-2.py)) | End-to-End Natural Langugage Query to SQL DB Query | |
| 9. Text2SQL - MultiStep ([code](text2sql-multistep-example-3.py)) | Extract a customer name using NER and use in a Text2SQL query |
| 10. ⭐ Web Services & Function Calls ([code](web_services_slim_fx.py)) | Generate 30 key financial analysis with SLIM function calls and web services |
| 11. ⭐ Yes-No Questions with Explanations ([code](using_slim_boolean_model.py)) | Analyze earnings releases with SLIM Boolean |
| 12. ⭐ Extracting Revenue Growth ([code](using_slim_extract_model.py)) | Extract revenue growth from earnings releases with SLIM Extract |
| 13. ⭐ Summary as a Function Call ([code](using_slim_summary.py)) | Simple Summarization as a Function Call with List Length Parameters |
| 14. ⭐ Handling Not Found Extracts ([code](not_found_extract_with_lookup.py)) | Multi-step Lookup strategy and handling not-found answers |
| 15. ⭐ Extract + Lookup ([code](custom_extract_and_lookup.py)) | Extract Named Entity information and use for lookups with SLIM Extract |
| 16. ⭐ Headline/Title as XSUM Function Call ([code](using_slim_xsum.py)) | eXtreme Summarization (XSUM) with SLIM XSUM |
For information on all of the SLIM models, check out [LLMWare SLIM Model Collection](https://www.huggingface.co/llmware/).
**Models List**
If you would like more information about any of the SLIM models, please check out their model card:
- extract - extract custom keys - [slim-extract](https://www.huggingface.co/llmware/slim-extract) & [slim-extract-tool](https://www.huggingface.co/llmware/slim-extract-tool)
- summary - summarize function call - [slim-summary](https://www.huggingface.co/llmware/slim-summary) & [slim-summary-tool](https://www.huggingface.co/llmware/slim-summary-tool)
- xsum - title/headline function call - [slim-xsum](https://www.huggingface.co/llmware/slim-xsum) & [slim-xsum-tool](https://www.huggingface.co/llmware/slim-xsum-tool)
- ner - extract named entities - [slim-ner](https://www.huggingface.co/llmware/slim-ner) & [slim-ner-tool](https://www.huggingface.co/llmware/slim-ner-tool)
- sentiment - evaluate sentiment - [slim-sentiment](https://www.huggingface.co/slim-sentiment) & [slim-sentiment-tool](https://www.huggingface.co/llmware/slim-sentiment-tool)
- topics - generate topic - [slim-topics](https://www.huggingface.co/slim-topics) & [slim-topics-tool](https://www.huggingface.co/llmware/slim-topics-tool)
- sa-ner - combo model (sentiment + named entities) - [slim-sa-ner](https://www.huggingface.co/slim-sa-ner) & [slim-sa-ner-tool](https://www.huggingface.co/llmware/slim-sa-ner-tool)
- boolean - provides a yes/no output with explanation - [slim-boolean](https://www.huggingface.co/slim-boolean) & [slim-boolean-tool](https://www.huggingface.com/llmware/slim-boolean-tool)
- ratings - apply 1 (low) - 5 (high) rating - [slim-ratings](https://www.huggingface.co/slim-ratings) & [slim-ratings-tool](https://www.huggingface.co/llmware/slim-ratings-tool)
- emotions - assess emotions - [slim-emotions](https://www.huggingface.co/slim-emotions) & [slim-emotions-tool](https://www.huggingface.co/llmware/slim-emotions-tool)
- tags - auto-generate list of tags - [slim-tags](https://www.huggingface.co/slim-tags) & [slim-tags-tool](https://www.huggingface.co/llmware/slim-tags-tool)
- tags-3b - enhanced auto-generation tagging model - [slim-tags-3b](https://www.huggingface.com/slim-tags-3b) & [slim-tags-3b-tool](https://www.huggingface.co/llmware/slim-tags-3b-tool)
- intent - identify intent - [slim-intent](https://www.huggingface.co/slim-intent) & [slim-intent-tool](https://www.huggingface.co/llmware/slim-intent-tool)
- category - high-level category - [slim-category](https://www.huggingface.co/slim-category) & [slim-category-tool](https://wwww.huggingface.co/llmware/slim-category-tool)
- nli - assess if evidence supports conclusion - [slim-nli](https://www.huggingface.co/slim-nli) & [slim-nli-tool](https://www.huggingface.co/llmware/slim-nli-tool)
- sql - convert text into sql - [slim-sql](https://www.huggingface.co/slim-sql) & [slim-sql-tool](https://www.huggingface.co/llmware/slim-sql-tool)
You may also want to check out these quantized 'answer' tools, which work well in conjunction with SLIMs for question-answer and summarization:
- bling-stablelm-3b-tool - 3b quantized RAG model - [bling-stablelm-3b-gguf](https://www.huggingface.co/llmware/bling-stablelm-3b-gguf)
- bling-answer-tool - 1b quantized RAG model - [bling-answer-tool](https://www.huggingface.co/llmware/bling-answer-tool)
- dragon-yi-answer-tool - 6b quantized RAG model - [dragon-yi-answer-tool](https://www.huggingface.co/llmware/dragon-yi-answer-tool)
- dragon-mistral-answer-tool - 7b quantized RAG model - [dragon-mistral-answer-tool](https://www.huggingface.co/llmware/dragon-mistral-answer-tool)
- dragon-llama-answer-tool - 7b quantized RAG model - [dragon-llama-answer-tool](https://www.huggingface.co/llmware/dragon-llama-answer-tool)
**Set up**
No special setup for SLIMs is required other than to install llmware >=0.2.6, e.g., `pip3 install llmware`.
**Platforms:**
- Mac M1, Mac x86, Windows, Linux (Ubuntu 22 preferred, supported on Ubuntu 20 +)
- RAM: 16 GB minimum
- Python 3.9, 3.10, 3.11 (note: not supported on 3.12 yet)
- llmware >= 0.2.6 version
### **Let's get started! 🚀**
docs/examples/datasets.md
+134
View File
@@ -0,0 +1,134 @@
---
layout: default
title: Datasets
parent: Examples
nav_order: 10
description: overview of the major modules and classes of LLMWare
permalink: /examples/datasets
---
# Datasets - Introduction by Examples
llmware provides powerful capabilities to transform raw unstructured information into various model-ready datasets.
```python
import os
import json
from llmware.library import Library
from llmware.setup import Setup
from llmware.dataset_tools import Datasets
from llmware.retrieval import Query
def build_and_use_dataset(library_name):
# Setup a library and build a knowledge graph. Datasets will use the data in the knowledge graph
print (f"\n > Creating library {library_name}...")
library = Library().create_new_library(library_name)
sample_files_path = Setup().load_sample_files()
library.add_files(os.path.join(sample_files_path,"SmallLibrary"))
library.generate_knowledge_graph()
# Create a Datasets object from library
datasets = Datasets(library)
# Build a basic dataset useful for industry domain adaptation for fine-tuning embedding models
print (f"\n > Building basic text dataset...")
basic_embedding_dataset = datasets.build_text_ds(min_tokens=500, max_tokens=1000)
dataset_location = os.path.join(library.dataset_path, basic_embedding_dataset["ds_id"])
print (f"\n > Dataset:")
print (f"(Files referenced below are found in {dataset_location})")
print (f"\n{json.dumps(basic_embedding_dataset, indent=2)}")
sample = datasets.get_dataset_sample(datasets.current_ds_name)
print (f"\nRandom sample from the dataset:\n{json.dumps(sample, indent=2)}")
# Other Dataset Generation and Usage Examples:
# Build a simple self-supervised generative dataset- extracts text and splits into 'text' & 'completion'
# Several generative "prompt_wrappers" are available - chat_gpt | alpaca |
basic_generative_completion_dataset = datasets.build_gen_ds_targeted_text_completion(prompt_wrapper="alpaca")
# Build a generative self-supervised training sets created by pairing 'header_text' with 'text'
xsum_generative_completion_dataset = datasets.build_gen_ds_headline_text_xsum(prompt_wrapper="human_bot")
topic_prompter_dataset = datasets.build_gen_ds_headline_topic_prompter(prompt_wrapper="chat_gpt")
# Filter a library by a key term as part of building the dataset
filtered_dataset = datasets.build_text_ds(query="agreement", filter_dict={"master_index":1})
# Pass a set of query results to create a dataset from those results only
query_results = Query(library=library).query("africa")
query_filtered_dataset = datasets.build_text_ds(min_tokens=250,max_tokens=600, qr=query_results)
return 0
```
For more examples, see the [datasets example]((https://www.github.com/llmware-ai/llmware/tree/main/examples/Datasets/) in the main repo.
Check back often - we are updating these examples regularly - and many of these examples have companion videos as well.
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span><img src="assets/images/hf-logo.svg" alt="Hugging Face" class="hugging-face-logo"/></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/examples/embedding.md
+222
View File
@@ -0,0 +1,222 @@
---
layout: default
title: Embedding
parent: Examples
nav_order: 5
description: overview of the major modules and classes of LLMWare
permalink: /examples/embedding
---
# Embedding - Introduction by Examples
We introduce ``llmware`` through self-contained examples.
```python
""" This example is a fast start with Milvus Lite, which is a 'no-install' file-based version of Milvus, intended
for rapid prototyping. A couple of key points to note:
-- Platform - per Milvus docs, Milvus Lite is designed for Mac and Linux (not on Windows currently)
-- PyMilvus - need to `pip install pymilvus>=2.4.2`
-- within LLMWare: set MilvusConfig().set_config("lite", True)
"""
import os
from llmware.library import Library
from llmware.retrieval import Query
from llmware.setup import Setup
from llmware.status import Status
from llmware.models import ModelCatalog
from llmware.configs import LLMWareConfig, MilvusConfig
from importlib import util
if not util.find_spec("pymilvus"):
print("\nto run this example with pymilvus, you need to install pymilvus: pip3 install pymilvus>=2.4.2")
def setup_library(library_name):
""" Note: this setup_library method is provided to enable a self-contained example to create a test library """
# Step 1 - Create library which is the main 'organizing construct' in llmware
print ("\nupdate: Creating library: {}".format(library_name))
library = Library().create_new_library(library_name)
# check the embedding status 'before' installing the embedding
embedding_record = library.get_embedding_status()
print("embedding record - before embedding ", embedding_record)
# Step 2 - Pull down the sample files from S3 through the .load_sample_files() command
# --note: if you need to refresh the sample files, set 'over_write=True'
print ("update: Downloading Sample Files")
sample_files_path = Setup().load_sample_files(over_write=False)
# Step 3 - point ".add_files" method to the folder of documents that was just created
# this method parses the documents, text chunks, and captures in database
print("update: Parsing and Text Indexing Files")
library.add_files(input_folder_path=os.path.join(sample_files_path, "Agreements"),
chunk_size=400, max_chunk_size=600, smart_chunking=1)
return library
def install_vector_embeddings(library, embedding_model_name):
""" This method is the core example of installing an embedding on a library.
-- two inputs - (1) a pre-created library object and (2) the name of an embedding model """
library_name = library.library_name
vector_db = LLMWareConfig().get_vector_db()
print(f"\nupdate: Starting the Embedding: "
f"library - {library_name} - "
f"vector_db - {vector_db} - "
f"model - {embedding_model_name}")
# *** this is the one key line of code to create the embedding ***
library.install_new_embedding(embedding_model_name=embedding_model, vector_db=vector_db,batch_size=100)
# note: for using llmware as part of a larger application, you can check the real-time status by polling Status()
# --both the EmbeddingHandler and Parsers write to Status() at intervals while processing
update = Status().get_embedding_status(library_name, embedding_model)
print("update: Embeddings Complete - Status() check at end of embedding - ", update)
# Start using the new vector embeddings with Query
sample_query = "incentive compensation"
print("\n\nupdate: Run a sample semantic/vector query: {}".format(sample_query))
# queries are constructed by creating a Query object, and passing a library as input
query_results = Query(library).semantic_query(sample_query, result_count=20)
for i, entries in enumerate(query_results):
# each query result is a dictionary with many useful keys
text = entries["text"]
document_source = entries["file_source"]
page_num = entries["page_num"]
vector_distance = entries["distance"]
# to see all of the dictionary keys returned, uncomment the line below
# print("update: query_results - all - ", i, entries)
# for display purposes only, we will only show the first 125 characters of the text
if len(text) > 125: text = text[0:125] + " ... "
print("\nupdate: query results - {} - document - {} - page num - {} - distance - {} "
.format( i, document_source, page_num, vector_distance))
print("update: text sample - ", text)
# lets take a look at the library embedding status again at the end to confirm embeddings were created
embedding_record = library.get_embedding_status()
print("\nupdate: embedding record - ", embedding_record)
return 0
if __name__ == "__main__":
# Fast Start configuration - will use no-install embedded sqlite
# -- if you have installed Mongo or Postgres, then change the .set_active_db accordingly
LLMWareConfig().set_active_db("sqlite")
# set the "lite" flag in MilvusConfig to True -> to use server version, set to False (which is default)
MilvusConfig().set_config("lite", True)
LLMWareConfig().set_vector_db("milvus")
# Step 1 - create library
library = setup_library("ex2_milvus_lite")
# Step 2 - Select any embedding model in the LLMWare catalog
# to see a list of the embedding models supported, uncomment the line below and print the list
embedding_models = ModelCatalog().list_embedding_models()
# for i, models in enumerate(embedding_models):
# print("embedding models: ", i, models)
# for this first embedding, we will use a very popular and fast sentence transformer
embedding_model = "mini-lm-sbert"
# note: if you want to swap out "mini-lm-sbert" for Open AI 'text-embedding-ada-002', uncomment these lines:
# embedding_model = "text-embedding-ada-002"
# os.environ["USER_MANAGED_OPENAI_API_KEY"] = "<insert-your-openai-api-key>"
# run the core script
install_vector_embeddings(library, embedding_model)
```
For more examples, see the [embedding examples]((https://www.github.com/llmware-ai/llmware/tree/main/examples/Embedding/) in the main repo.
Check back often - we are updating these examples regularly - and many of these examples have companion videos as well.
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span><img src="assets/images/hf-logo.svg" alt="Hugging Face" class="hugging-face-logo"/></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/examples/examples.md
+25
View File
@@ -0,0 +1,25 @@
---
layout: default
title: Examples
nav_order: 5
has_children: true
description: examples, recipes and use cases
permalink: /examples
---
llmware offers a wide range of examples to cover the lifecycle of building RAG and Agent based applications using
small language models:
- [Parsing examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing) - ~14 stand-alone parsing examples for all common document types, including options for parsing in memory, outputting to JSON, parsing custom configured CSV and JSON files, running OCR on embedded images found in documents, table extraction, image extraction, text chunking, zip files, and web sources.
- [Embedding examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/Embedding) - ~15 stand-alone embedding examples to show how to use ~10 different vector databases and wide range of leading open source embedding models (including sentence transformers).
- [Retrieval examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/Retrieval) - ~10 stand-alone examples illustrating different query and retrieval techniques - semantic queries, text queries, document filters, page filters, 'hybrid' queries, author search, using query state, and generating bibliographies.
- [Dataset examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/Datasets) - ~5 stand-alone examples to show 'next steps' of how to leverage a Library to re-package content into various datasets and automated NLP analytics.
- [Fast start example #1-Parsing](https://github.com/llmware-ai/llmware/blob/main/fast_start/rag/example-1-create_first_library.py) - shows the basics of parsing.
- [Fast start example #2-Embedding](https://github.com/llmware-ai/llmware/blob/main/fast_start/rag/example-2-build_embeddings.py) - shows the basics of building embeddings.
- [CustomTable examples](https://github.com/llmware-ai/llmware/tree/main/examples/Structured_Tables) - ~5 examples to start building structured tables that can be used in conjunction with LLM-based workflows.
- [Models examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/Models) - ~20 examples showing a wide range of different model inferences and use cases, including the ability to integrate Ollama models, OpenChat (e.g., LMStudio) models, using LLama-3 and Phi-3, bringing your own models into the ModelCatalog, and configuring sampling settings.
- [Prompts examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/Prompts) - ~5 examples that illustrate how to use Prompt as an integrated workflow for integrating knowledge sources, managing prompt history, and applying fact-checking.
- [SLIM-Agents examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/SLIM-Agents) - ~20 examples showing how to build multi-model, multi-step Agent processes using locally-running SLIM function calling models.
- [Fast start example #3-Prompts and Models](https://github.com/llmware-ai/llmware/blob/main/fast_start/rag/example-3-prompts_and_models.py) - getting started with model inference.
docs/examples/getting_started.md
+124
View File
@@ -0,0 +1,124 @@
---
layout: default
title: Introduction by Examples
parent: Examples
nav_order: 9
permalink: /examples/getting_started
---
# Introduction by Examples
We introduce ``llmware`` through self-contained examples.
# Your first library and query
{: .note }
> The code here is a modified version from [example-1-create_first_library.py](https://github.com/llmware-ai/llmware/blob/main/fast_start/example-1-create_first_library.py).
> The adjustments are made to ease understanding for this post.
In this introduction, we will walk through the steps of creating a **library**.
To create a ``library`` in ``llmware`` we have to instantiate a ``library`` object and call
the ``add_files`` method, which will parse the files, chunk up the text and also index it.
We will also download the samples files we provide, which can be used for any experimentation you
might want to do.
**Configuring llmware**
Before we get started, we can influence the configuration of ``llmware``.
For example, we can decide on which **text collection** data base to use, and on the logging level.
By default, ``llmware`` uses MongoDB as the text collection data base and has a ``debug_mode`` level
of ``0``.
This means that by default, ``llmware`` will show the status manager and print errors.
The status manager is useful for large parsing jobs.
In this ``library`` introduction, we will change the text collection data base as well as the ``debug_mode``.
As the text collection data base, we will choose ``sqlite``.
And we will change the ``debug_mode`` to ``2``, which will show the file name that is being parsed, i.e. a file-by-file progress.
```python
from llmware.configs import LLMWareConfig
LLMWareConfig().set_active_db("sqlite")
LLMWareConfig().set_config("debug_mode", 2)
```
**Downloading sample files**
We start by downloading the sample files we need.
``llmware`` provides a set of sample files which we use throughout our examples.
The following code snippet downloads these sample files, and in doing so creates the directories
*Agreements*, *Invoices*, *UN-Resolutions-500*, *SmallLibrary*, *FinDocs*, and *AgreementsLarge*.
If you want to get the newest version of the sample files, you can set ``over_write=True``.
However, we encourage you to try it out with your own files once you are comfortable enough with ``llmware``.
```python
from llmware.setup import Setup
sample_files_path = Setup().load_sample_files(over_write=False)
```
``sample_files_path`` is the path where the files are stores.
Assume that your use name is ``foo``, then on Linux the path would be ``'/home/foo/llmware_data/sample_files'.``
**Creating a library**
Now that we have data, we can start to create our library.
In ``llmware``, a **library** is a collection of unstructured data.
Currently, ``llmware`` supports *text* and *images*.
The following code creates an empty ``library`` with the name ``my_llmware_library``.
```python
from llmware.library import Library
library = Library().create_new_library('my_llmware_library')
```
**Adding files to a library**
Now that we have created a ``library``, we are ready to *add files* to it.
Currently, the ``add_files`` method supports pdf, pptx, docx, xlsx, csv, md, txt, json, wav, and zip, jpg, and png.
The method will automatically choose the correct parser, based on the file extension.
```python
library.add_files('/home/foo/llmware_data/sample_files/Agreements')
```
**The library card**
A ``library`` keeps inventory of its files, similar to a good librarian.
We do this with a *library card*.
At the moment of this writing, a library card has the keys _id, library_name, embedding, knowledge_graph, unique_doc_id, documents, blocks, images, pages, tables, and account_name.
```python
updated_library_card = library.get_library_card()
doc_count = updated_library_card["documents"]
block_count = updated_library_card["blocks"]
library_card.keys()
```
You can also get where the library is stored via the ``library_main_path`` attribute.
Again, assuming your user name is *foo* and you are on a Linux system, then the ``library_path`` is ``'/home/foo/llmware_data/accounts/llmware/my_lib'``.
```python
library.library_main_path
```
**Querying a library**
Finally, we are ready to execute a query against our library.
Remember that the text is indexed automatically when we add it to the library.
The result of a ``Query`` is a list of dictionaries, where one dictionary is one result.
A result dictionary has a wide range of useful keys.
A few important keys in the dictionary are *text*, *file_source*, *page_num*, *doc_ID*, *block_ID*, and
*matches*.
In the following, we query the library for the base salary, return the first ten results, and
iterate over the results.
```python
query_results = Query(library).text_query('base salary', result_count=10)
for query_result in query_results:
text = query_result["text"]
file_source = query_result["file_source"]
page_number = query_result["page_num"]
doc_id = query_result["doc_ID"]
block_id = query_result["block_ID"]
matches = query_result["matches"]
```
You can take a look at all the keys that are returned by calling ``keys()``.
```python
query_results[0].keys()
```
docs/examples/models.md
+498
View File
@@ -0,0 +1,498 @@
---
layout: default
title: Models
parent: Examples
nav_order: 3
description: overview of the major modules and classes of LLMWare
permalink: /examples/models
---
# Models
We introduce ``llmware`` through self-contained examples.
```python
""" This example demonstrates prompting local BLING models with provided context - easy to select among different
BLING models between 1B - 4B, including both Pytorch versions and GGUF quantized versions, and to swap out the
hello_world questions with your own test set.
NOTE: if you are running on a CPU with limited memory (e.g., <16 GB of RAM), we would recommend sticking to
the 1B parameter models, or using the quantized GGUF versions. You may get out-of-memory errors and/or very
slow performance with ~3B parameter Pytorch models. Even with 16 GB+ of RAM, the 3B Pytorch models should run but
will be slow (without GPU acceleration). """
import time
from llmware.prompts import Prompt
def hello_world_questions():
test_list = [
{"query": "What is the total amount of the invoice?",
"answer": "$22,500.00",
"context": "Services Vendor Inc. \n100 Elm Street Pleasantville, NY \nTO Alpha Inc. 5900 1st Street "
"Los Angeles, CA \nDescription Front End Engineering Service $5000.00 \n Back End Engineering"
" Service $7500.00 \n Quality Assurance Manager $10,000.00 \n Total Amount $22,500.00 \n"
"Make all checks payable to Services Vendor Inc. Payment is due within 30 days."
"If you have any questions concerning this invoice, contact Bia Hermes. "
"THANK YOU FOR YOUR BUSINESS! INVOICE INVOICE # 0001 DATE 01/01/2022 FOR Alpha Project P.O. # 1000"},
{"query": "What was the amount of the trade surplus?",
"answer": "62.4 billion yen ($416.6 million)",
"context": "Japan’s September trade balance swings into surplus, surprising expectations"
"Japan recorded a trade surplus of 62.4 billion yen ($416.6 million) for September, "
"beating expectations from economists polled by Reuters for a trade deficit of 42.5 "
"billion yen. Data from Japan’s customs agency revealed that exports in September "
"increased 4.3% year on year, while imports slid 16.3% compared to the same period "
"last year. According to FactSet, exports to Asia fell for the ninth straight month, "
"which reflected ongoing China weakness. Exports were supported by shipments to "
"Western markets, FactSet added. — Lim Hui Jie"},
{"query": "What was Microsoft's revenue in the 3rd quarter?",
"answer": "$52.9 billion",
"context": "Microsoft Cloud Strength Drives Third Quarter Results \nREDMOND, Wash. — April 25, 2023 — "
"Microsoft Corp. today announced the following results for the quarter ended March 31, 2023,"
" as compared to the corresponding period of last fiscal year:\n· Revenue was $52.9 billion"
" and increased 7% (up 10% in constant currency)\n· Operating income was $22.4 billion "
"and increased 10% (up 15% in constant currency)\n· Net income was $18.3 billion and "
"increased 9% (up 14% in constant currency)\n· Diluted earnings per share was $2.45 "
"and increased 10% (up 14% in constant currency).\n"},
{"query": "When did the LISP machine market collapse?",
"answer": "1987.",
"context": "The attendees became the leaders of AI research in the 1960s."
" They and their students produced programs that the press described as 'astonishing': "
"computers were learning checkers strategies, solving word problems in algebra, "
"proving logical theorems and speaking English. By the middle of the 1960s, research in "
"the U.S. was heavily funded by the Department of Defense and laboratories had been "
"established around the world. Herbert Simon predicted, 'machines will be capable, "
"within twenty years, of doing any work a man can do'. Marvin Minsky agreed, writing, "
"'within a generation ... the problem of creating 'artificial intelligence' will "
"substantially be solved'. They had, however, underestimated the difficulty of the problem. "
"Both the U.S. and British governments cut off exploratory research in response "
"to the criticism of Sir James Lighthill and ongoing pressure from the US Congress "
"to fund more productive projects. Minsky's and Papert's book Perceptrons was understood "
"as proving that artificial neural networks approach would never be useful for solving "
"real-world tasks, thus discrediting the approach altogether. The 'AI winter', a period "
"when obtaining funding for AI projects was difficult, followed. In the early 1980s, "
"AI research was revived by the commercial success of expert systems, a form of AI "
"program that simulated the knowledge and analytical skills of human experts. By 1985, "
"the market for AI had reached over a billion dollars. At the same time, Japan's fifth "
"generation computer project inspired the U.S. and British governments to restore funding "
"for academic research. However, beginning with the collapse of the Lisp Machine market "
"in 1987, AI once again fell into disrepute, and a second, longer-lasting winter began."},
{"query": "When will employment start?",
"answer": "April 16, 2012.",
"context": "THIS EXECUTIVE EMPLOYMENT AGREEMENT (this “Agreement”) is entered "
"into this 2nd day of April, 2012, by and between Aphrodite Apollo "
"(“Executive”) and TestCo Software, Inc. (the “Company” or “Employer”), "
"and shall become effective upon Executive’s commencement of employment "
"(the “Effective Date”) which is expected to commence on April 16, 2012. "
"The Company and Executive agree that unless Executive has commenced "
"employment with the Company as of April 16, 2012 (or such later date as "
"agreed by each of the Company and Executive) this Agreement shall be "
"null and void and of no further effect."},
{"query": "What is the current rate on 10-year treasuries?",
"answer": "4.58%",
"context": "Stocks rallied Friday even after the release of stronger-than-expected U.S. jobs data "
"and a major increase in Treasury yields. The Dow Jones Industrial Average gained 195.12 points, "
"or 0.76%, to close at 31,419.58. The S&P 500 added 1.59% at 4,008.50. The tech-heavy "
"Nasdaq Composite rose 1.35%, closing at 12,299.68. The U.S. economy added 438,000 jobs in "
"August, the Labor Department said. Economists polled by Dow Jones expected 273,000 "
"jobs. However, wages rose less than expected last month. Stocks posted a stunning "
"turnaround on Friday, after initially falling on the stronger-than-expected jobs report. "
"At its session low, the Dow had fallen as much as 198 points; it surged by more than "
"500 points at the height of the rally. The Nasdaq and the S&P 500 slid by 0.8% during "
"their lowest points in the day. Traders were unclear of the reason for the intraday "
"reversal. Some noted it could be the softer wage number in the jobs report that made "
"investors rethink their earlier bearish stance. Others noted the pullback in yields from "
"the day’s highs. Part of the rally may just be to do a market that had gotten extremely "
"oversold with the S&P 500 at one point this week down more than 9% from its high earlier "
"this year. Yields initially surged after the report, with the 10-year Treasury rate trading "
"near its highest level in 14 years. The benchmark rate later eased from those levels, but "
"was still up around 6 basis points at 4.58%. 'We’re seeing a little bit of a give back "
"in yields from where we were around 4.8%. [With] them pulling back a bit, I think that’s "
"helping the stock market,' said Margaret Jones, chief investment officer at Vibrant Industries "
"Capital Advisors. 'We’ve had a lot of weakness in the market in recent weeks, and potentially "
"some oversold conditions.'"},
{"query": "What is the governing law?",
"answer": "State of Massachusetts",
"context": "19. Governing Law and Procedures. This Agreement shall be governed by and interpreted "
"under the laws of the State of Massachusetts, except with respect to Section 18(a) of this Agreement,"
" which shall be governed by the laws of the State of Delaware, without giving effect to any "
"conflict of laws provisions. Employer and Executive each irrevocably and unconditionally "
"(a) agrees that any action commenced by Employer for preliminary and permanent injunctive relief "
"or other equitable relief related to this Agreement or any action commenced by Executive pursuant "
"to any provision hereof, may be brought in the United States District Court for the federal "
"district in which Executive’s principal place of employment is located, or if such court does "
"not have jurisdiction or will not accept jurisdiction, in any court of general jurisdiction "
"in the state and county in which Executive’s principal place of employment is located, "
"(b) consents to the non-exclusive jurisdiction of any such court in any such suit, action o"
"r proceeding, and (c) waives any objection which Employer or Executive may have to the "
"laying of venue of any such suit, action or proceeding in any such court. Employer and "
"Executive each also irrevocably and unconditionally consents to the service of any process, "
"pleadings, notices or other papers in a manner permitted by the notice provisions of Section 8."},
{"query": "What is the amount of the base salary?",
"answer": "$200,000.",
"context": "2.2. Base Salary. For all the services rendered by Executive hereunder, during the "
"Employment Period, Employer shall pay Executive a base salary at the annual rate of "
"$200,000, payable semimonthly in accordance with Employer’s normal payroll practices. "
"Executive’s base salary shall be reviewed annually by the Board (or the compensation committee "
"of the Board), pursuant to Employer’s normal compensation and performance review policies "
"for senior level executives, and may be increased but not decreased. The amount of any "
"increase for each year shall be determined accordingly. For purposes of this Agreement, "
"the term “Base Salary” shall mean the amount of Executive’s base salary established "
"from time to time pursuant to this Section 2.2. "},
{"query": "Is the expected gross margin greater than 70%?",
"answer": "Yes, between 71.5% and 72.%",
"context": "Outlook NVIDIA’s outlook for the third quarter of fiscal 2024 is as follows:"
"Revenue is expected to be $16.00 billion, plus or minus 2%. GAAP and non-GAAP "
"gross margins are expected to be 71.5% and 72.5%, respectively, plus or minus "
"50 basis points. GAAP and non-GAAP operating expenses are expected to be "
"approximately $2.95 billion and $2.00 billion, respectively. GAAP and non-GAAP "
"other income and expense are expected to be an income of approximately $100 "
"million, excluding gains and losses from non-affiliated investments. GAAP and "
"non-GAAP tax rates are expected to be 14.5%, plus or minus 1%, excluding any discrete items."
"Highlights NVIDIA achieved progress since its previous earnings announcement "
"in these areas: Data Center Second-quarter revenue was a record $10.32 billion, "
"up 141% from the previous quarter and up 171% from a year ago. Announced that the "
"NVIDIA® GH200 Grace™ Hopper™ Superchip for complex AI and HPC workloads is shipping "
"this quarter, with a second-generation version with HBM3e memory expected to ship "
"in Q2 of calendar 2024. "},
{"query": "What is Bank of America's rating on Target?",
"answer": "Buy",
"context": "Here are some of the tickers on my radar for Thursday, Oct. 12, taken directly from "
"my reporter’s notebook: It’s the one-year anniversary of the S&P 500′s bear market bottom "
"of 3,577. Since then, as of Wednesday’s close of 4,376, the broad market index "
"soared more than 22%. Hotter than expected September consumer price index, consumer "
"inflation. The Social Security Administration issues announced a 3.2% cost-of-living "
"adjustment for 2024. Chipotle Mexican Grill (CMG) plans price increases. Pricing power. "
"Cites consumer price index showing sticky retail inflation for the fourth time "
"in two years. Bank of America upgrades Target (TGT) to buy from neutral. Cites "
"risk/reward from depressed levels. Traffic could improve. Gross margin upside. "
"Merchandising better. Freight and transportation better. Target to report quarter "
"next month. In retail, the CNBC Investing Club portfolio owns TJX Companies (TJX), "
"the off-price juggernaut behind T.J. Maxx, Marshalls and HomeGoods. Goldman Sachs "
"tactical buy trades on Club names Wells Fargo (WFC), which reports quarter Friday, "
"Humana (HUM) and Nvidia (NVDA). BofA initiates Snowflake (SNOW) with a buy rating."
"If you like this story, sign up for Jim Cramer’s Top 10 Morning Thoughts on the "
"Market email newsletter for free. Barclays cuts price targets on consumer products: "
"UTZ Brands (UTZ) to $16 per share from $17. Kraft Heinz (KHC) to $36 per share from "
"$38. Cyclical drag. J.M. Smucker (SJM) to $129 from $160. Secular headwinds. "
"Coca-Cola (KO) to $59 from $70. Barclays cut PTs on housing-related stocks: Toll Brothers"
"(TOL) to $74 per share from $82. Keeps underweight. Lowers Trex (TREX) and Azek"
"(AZEK), too. Goldman Sachs (GS) announces sale of fintech platform and warns on "
"third quarter of 19-cent per share drag on earnings. The buyer: investors led by "
"private equity firm Sixth Street. Exiting a mistake. Rise in consumer engagement for "
"Spotify (SPOT), says Morgan Stanley. The analysts hike price target to $190 per share "
"from $185. Keeps overweight (buy) rating. JPMorgan loves elf Beauty (ELF). Keeps "
"overweight (buy) rating but lowers price target to $139 per share from $150. "
"Sees “still challenging” environment into third-quarter print. The Club owns shares "
"in high-end beauty company Estee Lauder (EL). Barclays upgrades First Solar (FSLR) "
"to overweight from equal weight (buy from hold) but lowers price target to $224 per "
"share from $230. Risk reward upgrade. Best visibility of utility scale names."},
{"query": "Who is NVIDIA's partner for the driver assistance system?",
"answer": "MediaTek",
"context": "Automotive Second-quarter revenue was $253 million, down 15% from the previous "
"quarter and up 15% from a year ago. Announced that NVIDIA DRIVE Orin™ is powering "
"the new XPENG G6 Coupe SUV’s intelligent advanced driver assistance system. "
"Partnered with MediaTek, which will develop mainstream automotive systems on "
"chips for global OEMs, which integrate new NVIDIA GPU chiplet IP for AI and graphics."},
{"query": "What was the rate of decline in 3rd quarter sales?",
"answer": "20% year-on-year.",
"context": "Nokia said it would cut up to 14,000 jobs as part of a cost cutting plan following "
"third quarter earnings that plunged. The Finnish telecommunications giant said that "
"it will reduce its cost base and increase operation efficiency to “address the "
"challenging market environment. The substantial layoffs come after Nokia reported "
"third-quarter net sales declined 20% year-on-year to 4.98 billion euros. Profit over "
"the period plunged by 69% year-on-year to 133 million euros."},
{"query": "What was professional visualization revenue in the quarter?",
"answer": "$379 million",
"context": "Gaming Second-quarter revenue was $2.49 billion, up 11% from the previous quarter and up "
"22% from a year ago. Began shipping the GeForce RTX™ 4060 family of GPUs, "
"bringing to gamers NVIDIA Ada Lovelace architecture and DLSS, starting at $299."
"Announced NVIDIA Avatar Cloud Engine, or ACE, for Games, a custom AI model "
"foundry service using AI-powered natural language interactions to transform games "
"by bringing intelligence to non-playable characters. Added 35 DLSS games, including "
"Diablo IV, Ratchet & Clank: Rift Apart, Baldur’s Gate 3 and F1 23, as well as Portal: "
"Prelude RTX, a path-traced game made by the community using NVIDIA’s RTX Remix creator tool."
"Professional Visualization Second-quarter revenue was $379 million, up 28% from the "
"previous quarter and down 24% from a year ago. Announced three new desktop "
"workstation RTX GPUs based on the Ada Lovelace architecture — NVIDIA RTX 5000, RTX 4500 "
"and RTX 4000 — to deliver the latest AI, graphics and real-time rendering, which are "
"shipping this quarter. Announced a major release of the NVIDIA Omniverse platform, "
"with new foundation applications and services for developers and industrial "
"enterprises to optimize and enhance their 3D pipelines with OpenUSD and "
"generative AI. Joined with Pixar, Adobe, Apple and Autodesk to form the "
"Alliance for OpenUSD to promote the standardization, development, evolution and "
"growth of Universal Scene Description technology."},
{"query": "What is the executive's title?",
"answer": "Senior Vice President, Event Planning ('SVP') of the Workforce Optimization Division.",
"context": "2.1. Duties and Responsibilities and Extent of Service. During the Employment Period, "
"Executive shall serve as Senior Vice President, Event Planning (“SVP”) of the Employer’s "
"Workforce Optimization Division. In such role, Executive will report to the Board of "
"Directors of Employer (the “Board”) and shall devote substantially all of his business time "
"and attention and his best efforts and ability to the operations of Employer and its subsidiaries. "
"Executive shall be responsible for running Employer’s day-to-day operations and shall perform "
"faithfully, diligently and competently the duties and responsibilities of a SVP and such other "
"duties and responsibilities as directed by the Board and are consistent with such position. "
"The foregoing shall not be construed as preventing Executive from (a) making passive "
"investments in other businesses or enterprises consistent with Employer’s code of conduct, "
"or (b) engaging in any other business activity consistent with Employer’s code of conduct; "
"provided that Executive seeks and obtains the prior approval of the Board before engaging "
"in any other business activity. In addition, it shall not be a violation of this Agreement "
"for Executive to participate in civic or charitable activities, deliver lectures, fulfill "
"speaking engagements, teach at educational institutions, and/or manage personal investments "
"(subject to the immediately preceding sentence); provided that such activities do not "
"interfere in any substantial respect with the performance of Executive’s responsibilities "
"as an employee in accordance with this Agreement. Executive may also serve on one or more "
"corporate boards of another company (and committees thereof) upon giving advance notice "
"to the Board prior to commencing service on any other corporate board."},
{"query": "According to the CFO, what led to the increase in cloud revenue?",
"answer": "Focused execution by our sales teams and partners",
"context": "'The world's most advanced AI models "
"are coming together with the world's most universal user interface - natural language - "
"to create a new era of computing,' said Satya Nadella, chairman and chief "
"executive officer of Microsoft. 'Across the Microsoft Cloud, we are the platform "
"of choice to help customers get the most value out of their digital spend and innovate "
"for this next generation of AI.' 'Focused execution by our sales teams and partners "
"in this dynamic environment resulted in Microsoft Cloud revenue of $28.5 billion, "
"up 22% (up 25% in constant currency) year-over-year,' said Amy Hood, executive "
"vice president and chief financial officer of Microsoft.\n"},
{"query": "Which company is located in Nevada?",
"answer": "North Industries",
"context": "To send notices to Blue Moon Tech, mail to their headquarters at: "
"555 California Street, San Francisco, California 94123. To send notices to North Industries, mail to"
"their principal U.S. offices at: 19832 32nd Avenue, Las Vegas, Nevada 23593.\nTo send notices "
"to Red River Industries, send to: One Red River Road, Stamford, Connecticut 08234."},
{"query": "When can termination after a material breach occur?",
"answer": "If the breach is not cured within 15 days of notice of the breach.",
"context": "This Agreement shall remain in effect until terminated. Either party may terminate this "
"agreement, any Statement of Work or Services Description for convenience by giving the other "
"party 30 days written notice. Either party may terminate this Agreement or any work order or "
"services description if the other party is in material breach or default of any obligation "
"that is not cured within 15 days’ notice of such breach. The TestCo agrees to pay all fees "
"for services performed and expenses incurred prior to the termination of this Agreement. "
"Termination of this Agreement will terminate all outstanding Statement of Work or Services "
"Description entered into under this agreement."},
{"query": "What is a headline summary in 10 words or less?",
"answer": "Joe Biden is the 46th President of the United States.",
"context": "Joe Biden's tenure as the 46th president of the United States began with "
"his inauguration on January 20, 2021. Biden, a Democrat from Delaware who "
"previously served as vice president under Barack Obama, "
"took office following his victory in the 2020 presidential election over "
"Republican incumbent president Donald Trump. Upon his inauguration, he "
"became the oldest president in American history."},
{"query": "Who are the two people that won elections in Georgia?",
"answer": "Jon Ossoff and Raphael Warnock",
"context": "Though Biden was generally acknowledged as the winner, "
"General Services Administration head Emily W. Murphy "
"initially refused to begin the transition to the president-elect, "
"thereby denying funds and office space to his team. "
"On November 23, after Michigan certified its results, Murphy "
"issued the letter of ascertainment, granting the Biden transition "
"team access to federal funds and resources for an orderly transition. "
"Two days after becoming the projected winner of the 2020 election, "
"Biden announced the formation of a task force to advise him on the "
"COVID-19 pandemic during the transition, co-chaired by former "
"Surgeon General Vivek Murthy, former FDA commissioner David A. Kessler, "
"and Yale University's Marcella Nunez-Smith. On January 5, 2021, "
"the Democratic Party won control of the United States Senate, "
"effective January 20, as a result of electoral victories in "
"Georgia by Jon Ossoff in a runoff election for a six-year term "
"and Raphael Warnock in a special runoff election for a two-year term. "
"President-elect Biden had supported and campaigned for both "
"candidates prior to the runoff elections on January 5.On January 6, "
"a mob of thousands of Trump supporters violently stormed the Capitol "
"in the hope of overturning Biden's election, forcing Congress to "
"evacuate during the counting of the Electoral College votes. More "
"than 26,000 National Guard members were deployed to the capital "
"for the inauguration, with thousands remaining into the spring."},
{"query": "What is the list of the top financial highlights for the quarter?",
"answer": "•Revenue: $52.9 million, up 10% in constant currency;\n"
"•Operating income: $22.4 billion, up 15% in constant currency;\n"
"•Net income: $18.3 billion, up 14% in constant currency;\n"
"•Diluted earnings per share: $2.45 billion, up 14% in constant currency.",
"context": "Microsoft Cloud Strength Drives Third Quarter Results \nREDMOND, Wash. — April 25, 2023 — "
"Microsoft Corp. today announced the following results for the quarter ended March 31, 2023,"
" as compared to the corresponding period of last fiscal year:\n· Revenue was $52.9 billion"
" and increased 7% (up 10% in constant currency)\n· Operating income was $22.4 billion "
"and increased 10% (up 15% in constant currency)\n· Net income was $18.3 billion and "
"increased 9% (up 14% in constant currency)\n· Diluted earnings per share was $2.45 "
"and increased 10% (up 14% in constant currency).\n"},
{"query": "What is a list of the key points?",
"answer": "•Stocks rallied on Friday with stronger-than-expected U.S jobs data and increase in "
"Treasury yields;\n•Dow Jones gained 195.12 points;\n•S&P 500 added 1.59%;\n•Nasdaq Composite rose "
"1.35%;\n•U.S. economy added 438,000 jobs in August, better than the 273,000 expected;\n"
"•10-year Treasury rate trading near the highest level in 14 years at 4.58%.",
"context": "Stocks rallied Friday even after the release of stronger-than-expected U.S. jobs data "
"and a major increase in Treasury yields. The Dow Jones Industrial Average gained 195.12 points, "
"or 0.76%, to close at 31,419.58. The S&P 500 added 1.59% at 4,008.50. The tech-heavy "
"Nasdaq Composite rose 1.35%, closing at 12,299.68. The U.S. economy added 438,000 jobs in "
"August, the Labor Department said. Economists polled by Dow Jones expected 273,000 "
"jobs. However, wages rose less than expected last month. Stocks posted a stunning "
"turnaround on Friday, after initially falling on the stronger-than-expected jobs report. "
"At its session low, the Dow had fallen as much as 198 points; it surged by more than "
"500 points at the height of the rally. The Nasdaq and the S&P 500 slid by 0.8% during "
"their lowest points in the day. Traders were unclear of the reason for the intraday "
"reversal. Some noted it could be the softer wage number in the jobs report that made "
"investors rethink their earlier bearish stance. Others noted the pullback in yields from "
"the day’s highs. Part of the rally may just be to do a market that had gotten extremely "
"oversold with the S&P 500 at one point this week down more than 9% from its high earlier "
"this year. Yields initially surged after the report, with the 10-year Treasury rate trading "
"near its highest level in 14 years. The benchmark rate later eased from those levels, but "
"was still up around 6 basis points at 4.58%. 'We’re seeing a little bit of a give back "
"in yields from where we were around 4.8%. [With] them pulling back a bit, I think that’s "
"helping the stock market,' said Margaret Jones, chief investment officer at Vibrant Industries "
"Capital Advisors. 'We’ve had a lot of weakness in the market in recent weeks, and potentially "
"some oversold conditions.'"}
]
return test_list
def bling_meets_llmware_hello_world (model_name):
""" Simple inference loop that loads a model and runs through a series of test questions. """
t0 = time.time()
test_list = hello_world_questions()
print(f"\n > Loading Model: {model_name}...")
prompter = Prompt().load_model(model_name)
t1 = time.time()
print(f"\n > Model {model_name} load time: {t1-t0} seconds")
for i, entries in enumerate(test_list):
print(f"\n{i+1}. Query: {entries['query']}")
# run the prompt
output = prompter.prompt_main(entries["query"],context=entries["context"]
, prompt_name="default_with_context",temperature=0.30)
llm_response = output["llm_response"].strip("\n")
print(f"LLM Response: {llm_response}")
print(f"Gold Answer: {entries['answer']}")
print(f"LLM Usage: {output['usage']}")
t2 = time.time()
print(f"\nTotal processing time: {t2-t1} seconds")
return 0
if __name__ == "__main__":
# list of 'rag-instruct' laptop-ready bling models on HuggingFace
model_list = ["llmware/bling-1b-0.1",
"llmware/bling-tiny-llama-v0",
"llmware/bling-1.4b-0.1",
"llmware/bling-falcon-1b-0.1",
"llmware/bling-cerebras-1.3b-0.1",
"llmware/bling-sheared-llama-1.3b-0.1",
"llmware/bling-sheared-llama-2.7b-0.1",
"llmware/bling-red-pajamas-3b-0.1",
"llmware/bling-stable-lm-3b-4e1t-v0",
"llmware/bling-phi-3",
# use GGUF models too
"bling-phi-3-gguf", # quantized bling-phi-3
"bling-answer-tool", # quantized bling-tiny-llama
"bling-stablelm-3b-tool" # quantized bling-stablelm-3b
]
# try the newest bling model - 'tiny-llama'
bling_meets_llmware_hello_world(model_list[1])
```
For more examples, see the [models examples]((https://www.github.com/llmware-ai/llmware/tree/main/examples/Models/) in the main repo.
Check back often - we are updating these examples regularly - and many of these examples have companion videos as well.
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span><img src="assets/images/hf-logo.svg" alt="Hugging Face" class="hugging-face-logo"/></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/examples/notebooks.md
+52
View File
@@ -0,0 +1,52 @@
---
layout: default
title: Notebooks
parent: Examples
nav_order: 11
description: overview of the major modules and classes of LLMWare
permalink: /examples/notebooks
---
# Notebooks - Introduction by Examples
We introduce ``llmware`` through self-contained examples.
# Understanding Google Colab and Jupyter Notebooks
Welcome to our project documentation! A common point of confusion among developers new to data science and machine learning workflows is the relationship and differences between Google Colab and Jupyter Notebooks. This README aims to clarify these parts to ensure everyone is on the same page.
## What are Jupyter Notebooks?
Jupyter Notebooks is an open-source web application that lets you create and share documents that have live code, equations, visualizations, and narrative text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, data visualization, machine learning, and much more.
## What is Google Colab?
Google Colab (or Colaboratory) is a free Jupyter notebook environment that requires no setup and runs in the cloud. It offers a similar interface to Jupyter Notebooks and lets users write and execute Python in a web browser. Google Colab also provides free access to computing resources, including GPUs and TPUs, making it highly popular for machine learning and data analysis projects.
## Key Similarities
- **Interface:** Both platforms use the Jupyter Notebook interface, which supports mixing executable code, equations, visualizations, and narrative text in a single document.
- **Language Support:** Primarily, both are used for executing Python code. However, Jupyter Notebooks support other languages such as R and Julia.
- **Use Cases:** They are widely used for data analysis, machine learning, and education, allowing for easy sharing of results and methodologies.
## Increase Google Colab Computational Power with T4 GPU
Our models are designed to run on at least 16GB of RAM. By default Google Colab provides ~13GB of RAM, which significantly slows computational speed. To ensure the best performance when using our models, we highly recommend enabling the T4 GPU in Colab. This will provide the notebook with additional resources, including 16GB of RAM, allowing our models to run smoothly and efficiently.
Steps to enabling T4 GPU in Colab:
1. In your Colab notebook, click on the "Runtime" tab
2. Select "Change runtime type"
3. Under "Hardware Accelerator", select T4 GPU
NOTE: There is a weekly usage limit on using T4 for free.
## Key Differences
- **Execution Environment:** Jupyter Notebooks can be run locally on your machine or on a server, but Google Colab is hosted in the cloud.
- **Access to Resources:** Google Colab provides free access to hardware accelerators (GPUs and TPUs) which is not inherently available in Jupyter Notebooks unless specifically set up by the user on their servers.
- **Collaboration:** Google Colab offers easier collaboration features, similar to Google Docs, letting multiple users work on the same notebook simultaneously.
## Conclusion
While Google Colab and Jupyter Notebooks might seem different they are built on the same idea and offer similar functionalities with a few distinctions, mainly in execution environment and access to computing resources. Understanding these platforms' capabilities can significantly enhance your data science and machine learning projects.
We hope this guide has helped clarify the similarities and differences between Google Colab and Jupyter Notebooks. Happy coding!
docs/examples/parsing.md
+69
View File
@@ -0,0 +1,69 @@
---
layout: default
title: Parsing
parent: Examples
nav_order: 4
description: overview of the major modules and classes of LLMWare
permalink: /examples/parsing
---
# Parsing - Introduction by Examples
We introduce ``llmware`` through self-contained examples.
🚀 Parsing Examples 🚀
===============
**Parsing is the Humble Hero of Good RAG Pipelines**
LLMWare supports parsing of a wide range of unstructured content types, and views parsing, text chunking and indexing as the first step in the pipeline, and like any pipeline, care and attention to getting "great input" is usually the key to "great output."
In this repository, we show several key features of parsing with llmware:
**Parsing PDFs like a Pro**
- Configuring text chunking and extraction parameters - [**PDF Configuration**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing/pdf_parser_new_configs.py)
- PDF Table extraction - [**PDF Table**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing/pdf_table_extraction.py)
- Fallback to OCR - [**PDF-by-OCR**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing/parse_pdf_by_ocr.py)
**Parsing Office Documents (Powerpoints, Word, Excel)**
- Configuring text chunking and extraction parameters - [**Office Configuration**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing/office_parser_new_configs.py)
- Handling ZIPs and mixed file types - [**Microsoft IR Documents**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing/parsing_microsoft_ir_docs.py)
- Running OCR on Images Extracted - [**OCR Embedded Doc Images**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing/ocr_embedded_doc_images.py)
**Parsing without a Database**
- Parse in Memory - [**Parse in Memory**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing/parse_in_memory.py)
- Parse directly into a Prompt - [**Parse in Prompt**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing/parse_into_prompt.py)
- Parse to JSON file - [**Parse to JSON**](https://www.github.com/llmware-ai/llmware/tree/main/examples/main/examples/Parsing/parse_to_json.py)
**Other Content Types**
- Custom CSV - [**Custom CSV files**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing/parse_csv_custom.py)
- Custom JSON - [**Custom JSON files**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing/parse_jsonl_custom.py)
- Images - [**OCR on Images**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing/parse_images.py)
- Web/HTML - [**Website Extraction**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing/parse_web_sources_in_memory.py)
- Voice (WAV) - in Use_Cases - [**Parsing Great Speeches**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Use_Cases/parsing_great_speeches.py)
For more examples, see the [parsing examples]((https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing/) in the main repo.
Check back often - we are updating these examples regularly - and many of these examples have companion videos as well.
### **Let's get started! 🚀**
docs/examples/prompts.md
+263
View File
@@ -0,0 +1,263 @@
---
layout: default
title: Prompts
parent: Examples
nav_order: 6
description: overview of the major modules and classes of LLMWare
permalink: /examples/prompts
---
# Prompts - Introduction by Examples
We introduce ``llmware`` through self-contained examples.
# Basic RAG Scenario - Invoice Processing
```python
""" This example shows an end-to-end scenario for invoice processing that can be run locally and without a
database. The example shows how to combine the use of parsing combined with prompts_with_sources to rapidly
iterate through a batch of invoices and ask a set of questions, and then save the full output to both
(1) .jsonl for integration into an upstream application/database and (2) to a CSV for human review in excel.
note: the sample code pulls from a public repo to load the sample invoice documents the first time -
please feel free to substitute with your own invoice documents (PDF/DOCX/PPTX/XLSX/CSV/TXT) if you prefer.
this example does not require a database or embedding
this example can be run locally on a laptop by setting 'run_on_cpu=True'
if 'run_on_cpu==False", then please see the example 'launch_llmware_inference_server.py'
to configure and set up a 'pop-up' GPU inference server in just a few minutes
"""
import os
import re
from llmware.prompts import Prompt, HumanInTheLoop
from llmware.configs import LLMWareConfig
from llmware.setup import Setup
from llmware.models import ModelCatalog
def invoice_processing(run_on_cpu=True):
# Step 1 - Pull down the sample files from S3 through the .load_sample_files() command
# --note: if you need to refresh the sample files, set 'over_write=True'
print("update: Downloading Sample Files")
sample_files_path = Setup().load_sample_files(over_write=False)
invoices_path = os.path.join(sample_files_path, "Invoices")
# Step 2 - simple sample query list - each question will be asked to each invoice
query_list = ["What is the total amount of the invoice?",
"What is the invoice number?",
"What are the names of the two parties?"]
# Step 3 - Load Model
if run_on_cpu:
# load local bling model that can run on cpu/laptop
# note: bling-1b-0.1 is the *fastest* & *smallest*, but will make more errors than larger BLING models
# model_name = "llmware/bling-1b-0.1"
# try the new bling-phi-3 quantized with gguf - most accurate
model_name = 'bling-phi-3-gguf'
else:
# use GPU-based inference server to process
# *** see the launch_llmware_inference_server.py example script to setup ***
server_uri_string = "http://11.123.456.789:8088" # insert your server_uri_string
server_secret_key = "demo-test"
ModelCatalog().setup_custom_llmware_inference_server(server_uri_string, secret_key=server_secret_key)
model_name = "llmware-inference-server"
# attach inference server to prompt object
prompter = Prompt().load_model(model_name)
# Step 4 - main loop thru folder of invoices
for i, invoice in enumerate(os.listdir(invoices_path)):
# just in case (legacy on mac os file system - not needed on linux or windows)
if invoice != ".DS_Store":
print("\nAnalyzing invoice: ", str(i + 1), invoice)
for question in query_list:
# Step 4A - parses the invoices in memory and attaches as a source to the Prompt
source = prompter.add_source_document(invoices_path,invoice)
# Step 4B - executes the prompt on the LLM (with the loaded source)
output = prompter.prompt_with_source(question,prompt_name="default_with_context")
for i, response in enumerate(output):
print("LLM Response - ", question, " - ", re.sub("[\n]"," ", response["llm_response"]))
prompter.clear_source_materials()
# Save jsonl report with full transaction history to /prompt_history folder
print("\nupdate: prompt state saved at: ", os.path.join(LLMWareConfig.get_prompt_path(),prompter.prompt_id))
prompter.save_state()
# Generate CSV report for easy Human review in Excel
csv_output = HumanInTheLoop(prompter).export_current_interaction_to_csv()
print("\nupdate: csv output for human review - ", csv_output)
return 0
if __name__ == "__main__":
invoice_processing(run_on_cpu=True)
```
# Document Summarizer
```python
""" This Example shows a packaged 'document_summarizer' prompt using the slim-summary-tool. It shows a variety of
techniques to summarize documents generally larger than a LLM context window, and how to assemble multiple source
batches from the document, as well as using a 'query' and 'topic' to focus on specific segments of the document. """
import os
from llmware.prompts import Prompt
from llmware.setup import Setup
def test_summarize_document(example="jd salinger"):
# pull a sample document (or substitute a file_path and file_name of your own)
sample_files_path = Setup().load_sample_files(over_write=False)
topic = None
query = None
fp = None
fn = None
if example not in ["jd salinger", "employment terms", "just the comp", "un resolutions"]:
print ("not found example")
return []
if example == "jd salinger":
fp = os.path.join(sample_files_path, "SmallLibrary")
fn = "Jd-Salinger-Biography.docx"
topic = "jd salinger"
query = None
if example == "employment terms":
fp = os.path.join(sample_files_path, "Agreements")
fn = "Athena EXECUTIVE EMPLOYMENT AGREEMENT.pdf"
topic = "executive compensation terms"
query = None
if example == "just the comp":
fp = os.path.join(sample_files_path, "Agreements")
fn = "Athena EXECUTIVE EMPLOYMENT AGREEMENT.pdf"
topic = "executive compensation terms"
query = "base salary"
if example == "un resolutions":
fp = os.path.join(sample_files_path, "SmallLibrary")
fn = "N2126108.pdf"
# fn = "N2137825.pdf"
topic = "key points"
query = None
# optional parameters: 'query' - will select among blocks with the query term
# 'topic' - will pass a topic/issue as the parameter to the model to 'focus' the summary
# 'max_batch_cap' - caps the number of batches sent to the model
# 'text_only' - returns just the summary text aggregated
kp = Prompt().summarize_document_fc(fp, fn, topic=topic, query=query, text_only=True, max_batch_cap=15)
print(f"\nDocument summary completed - {len(kp)} Points")
for i, points in enumerate(kp):
print(i, points)
return 0
if __name__ == "__main__":
print(f"\nExample: Summarize Documents\n")
# 4 examples - ["jd salinger", "employment terms", "just the comp", "un resolutions"]
# -- "jd salinger" - summarizes key points about jd salinger from short biography document
# -- "employment terms" - summarizes the executive compensation terms across 15 page document
# -- "just the comp" - queries to find subset of document and then summarizes the key terms
# -- "un resolutions" - summarizes the un resolutions document
summary_direct = test_summarize_document(example="employment terms")
```
For more examples, see the [prompt examples]((https://www.github.com/llmware-ai/llmware/tree/main/examples/Prompts/) in the main repo.
Check back often - we are updating these examples regularly - and many of these examples have companion videos as well.
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span><img src="assets/images/hf-logo.svg" alt="Hugging Face" class="hugging-face-logo"/></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/examples/retrieval.md
+167
View File
@@ -0,0 +1,167 @@
---
layout: default
title: Retrieval
parent: Examples
nav_order: 7
description: overview of the major modules and classes of LLMWare
permalink: /examples/retrieval
---
# Retrieval - Introduction by Examples
We introduce ``llmware`` through self-contained examples.
# SEMANTIC Retrieval Example
```python
"""
This 'getting started' example demonstrates how to use basic semantic retrieval with the Query class
1. Create a sample library
2. Run a basic semantic query
3. View the results
"""
import os
from llmware.library import Library
from llmware.retrieval import Query
from llmware.setup import Setup
from llmware.configs import LLMWareConfig
def create_fin_docs_sample_library(library_name):
print(f"update: creating library - {library_name}")
library = Library().create_new_library(library_name)
sample_files_path = Setup().load_sample_files(over_write=False)
ingestion_folder_path = os.path.join(sample_files_path, "FinDocs")
parsing_output = library.add_files(ingestion_folder_path)
print(f"update: building embeddings - may take a few minutes the first time")
# note: if you have installed Milvus or another vector DB, please feel free to substitute
# note: if you have any memory constraints on laptop:
# (1) reduce embedding batch_size or ...
# (2) substitute "mini-lm-sbert" as embedding model
library.install_new_embedding(embedding_model_name="industry-bert-sec", vector_db="chromadb",batch_size=200)
return library
def basic_semantic_retrieval_example (library):
# Create a Query instance
q = Query(library)
# Set the keys that should be returned - optional - full set of keys will be returned by default
q.query_result_return_keys = ["distance","file_source", "page_num", "text"]
# perform a simple query
my_query = "ESG initiatives"
query_results1 = q.semantic_query(my_query, result_count=20)
# Iterate through query_results, which is a list of result dicts
print(f"\nQuery 1 - {my_query}")
for i, result in enumerate(query_results1):
print("results - ", i, result)
# perform another query
my_query2 = "stock performance"
query_results2 = q.semantic_query(my_query2, result_count=10)
print(f"\nQuery 2 - {my_query2}")
for i, result in enumerate(query_results2):
print("results - ", i, result)
# perform another query
my_query3 = "cloud computing"
# note: use of embedding_distance_threshold will cap results with distance < 1.0
query_results3 = q.semantic_query(my_query3, result_count=50, embedding_distance_threshold=1.0)
print(f"\nQuery 3 - {my_query3}")
for i, result in enumerate(query_results3):
print("result - ", i, result)
return [query_results1, query_results2, query_results3]
if __name__ == "__main__":
print(f"Example - Running a Basic Semantic Query")
LLMWareConfig().set_active_db("sqlite")
# step 1- will create library + embeddings with Financial Docs
lib = create_fin_docs_sample_library("lib_semantic_query_1")
# step 2- run query against the library and embeddings
my_results = basic_semantic_retrieval_example(lib)
```
For more examples, see the [retrieval examples]((https://www.github.com/llmware-ai/llmware/tree/main/examples/Retrieval/) in the main repo.
Check back often - we are updating these examples regularly - and many of these examples have companion videos as well.
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span><img src="assets/images/hf-logo.svg" alt="Hugging Face" class="hugging-face-logo"/></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/examples/structured_tables.md
+225
View File
@@ -0,0 +1,225 @@
---
layout: default
title: Structured Tables
parent: Examples
nav_order: 9
description: overview of the major modules and classes of LLMWare
permalink: /examples/structured_tables
---
# Structured Tables - Introduction by Examples
We introduce ``llmware`` through self-contained examples.
```python
""" This example shows the basic recipe for creating a CustomTable with LLMWare and a few of the basic methods
to quickly get started.
In this example, we will build a very simple 'hello world' Files table, which we will build upon in a future
example by aggregating a more interesting and useful set of attributes from a LLMWare Library collection.
CustomTable is designed to work with the text collection databases supported by LLMWare:
SQL DBs --- Postgres and SQLIte
NoSQL DB --- Mongo DB
Even though Mongo does not require a schema for inserting and retrieving information, the CustomTable method
will expect a defined schema to be provided (good best practice, in any case). """
from llmware.resources import CustomTable
def hello_world_custom_table():
# simple schema for a table to track Files/Documents
# note: the schema is a python dictionary, with named keys, and the value corresponding to the data type
# for sqlite and postgres, any standard sql data type should generally work
files_schema = {"custom_doc_num": "integer",
"file_name": "text",
"comments": "text"}
# create a CustomTable object
db_name = "sqlite"
table_name = "files_table_1000"
ct = CustomTable(db=db_name,table_name=table_name, schema=files_schema)
# insert a few sample rows - each row is a dictionary with keys from the schema, and the *actual* values
r1 = {"custom_doc_num": 1, "file_name": "technical_manual.pdf", "comments": "very useful overview"}
ct.write_new_record(r1)
r2 = {"custom_doc_num": 2, "file_name": "work_presentation.pptx", "comments": "need to save for future reference"}
ct.write_new_record(r2)
r3 = {"custom_doc_num": 3, "file_name": "dataset.json", "comments": "will use in next project"}
ct.write_new_record(r3)
# to see the entries - pull all items from the table
all_results = ct.get_all()
print("\nTEST #1 - Retrieving All Elements")
for i, res in enumerate(all_results):
print("results: ", i, res)
# look at the database schema
schema = ct.get_schema()
print("\nTEST #2 - Getting the Table Schema")
print("schema: ", schema)
schema_str = ct.sql_table_create_string()
print("table create sql: ", schema_str)
# perform a basic lookup with 'key' and 'value'
f = ct.lookup("custom_doc_num", 2)
print("\nTEST #3 - Basic Lookup - 'custom_doc_num' = 2")
print("lookup: ", f)
# if you prefer SQL, pass a SQL query directly (note: this will only work on Postgres and SQLite)
if db_name == "sqlite":
# note: our standard 'unpacking' of a row of sqlite includes the rowid attribute
custom_query = f"SELECT rowid, * FROM {table_name} WHERE custom_doc_num = 3;"
elif db_name == "postgres":
custom_query = f"SELECT * FROM {table_name} WHERE custom_doc_num = 3;"
elif db_name == "mongo":
custom_query = {"custom_doc_num": 3}
else:
print("must use either sqlite, postgres or mongo")
return -1
cf = ct.custom_lookup(custom_query)
print("\nTEST #4 - Custom SQL Lookup - 'custom_doc_num' = 3")
print("custom query lookup: ", cf)
print("\nTEST #5 - Making Updates and Deletes")
# to delete a record
ct.delete_record("custom_doc_num", 1)
print("deleted record")
# to update the values of a record
ct.update_record({"custom_doc_num": 2}, "file_name", "work_presentation_update_v2.pptx")
print("updated record")
updated_all_results = ct.get_all()
for i, res in enumerate(updated_all_results):
print("updated results: ", i, res)
print("\nTEST #6 - Delete Table - uncomment and set confirm=True")
# done? delete the table and start over
# -- note: confirm=True must be set
# ct.delete_table(confirm=False)
# look at all tables in the database
tables = ct.list_all_tables()
print("\nTEST #7 - View all of the tables on the DB")
for i, t in enumerate(tables):
print("tables:" ,i, t)
return 0
if __name__ == "__main__":
hello_world_custom_table()
```
These examples illustrate the use of the CustomTable class to quickly create SQL tables that can be used in conjunction with LLM-based workflows.
1. [**Intro to CustomTables**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Structured_Tables/create_custom_table-1.py)
- Getting started with using CustomTables
2. [**Loading CSV into CustomTables**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Structured_Tables/loading_csv_into_custom_table-2a.py)
- Loading CSV into CustomTables
3. [**Loading CSV into Library (Configured)**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Structured_Tables/loading_csv_w_config_options-2b.py)
- Loading CSV into Library
4. [**Loading JSON into CustomTables**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Stuctured_Tables/loading_json_custom_table-3a.py)
- Loading JSON into CustomTable database
5 [**Loading JSON into Library (Configured)**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Stuctured_Tables/loading_json_w_config_options-3b.py)
- Loading JSON into a library with configuration
For more examples, see the [structured tables example]((https://www.github.com/llmware-ai/llmware/tree/main/examples/Structured_Tables/) in the main repo.
Check back often - we are updating these examples regularly - and many of these examples have companion videos as well.
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span><img src="assets/images/hf-logo.svg" alt="Hugging Face" class="hugging-face-logo"/></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/examples/ui.md
+101
View File
@@ -0,0 +1,101 @@
---
layout: default
title: UI
parent: Examples
nav_order: 8
description: overview of the major modules and classes of LLMWare
permalink: /examples/ui
---
# UI - Introduction by Examples
We introduce ``llmware`` through self-contained examples.
**UI Scenarios**
We provide several 'UI' examples that show how to use LLMWare in a complex recipe combining different elements to accomplish a specific objective. While each example is still high-level, it is shared in the spirit of providing a high-level framework 'starting point' that can be developed in more detail for a variety of common use cases. All of these examples use small, specialized models, running locally - 'Small, but Mighty' !
1. [**GGUF Streaming Chatbot**](https://www.github.com/llmware-ai/llmware/tree/main/examples/UI/gguf_streaming_chatbot.py)
- Locally deployed chatbot using leading open source chat models, including Phi-3-GGUF
- Uses Streamlit
- Core simple framework of ~20 lines using llmware and Streamlit
2. [**Simple RAG UI with Streamlit**](https://www.github.com/llmware-ai/llmware/tree/main/examples/UI/simple_rag_ui_with_streamlit.py)
- Simple RAG UI
3. [**RAG UI with Query Topic with Streamlit**](https://www.github.com/llmware-ai/llmware/tree/main/examples/UI/rag_ui_with_query_topic_with_streamlit.py)
- UI demonstrating UI with query topic in RAG scenario
4. [**Using Streamlit Chat UI**](https://www.github.com/llmware-ai/llmware/tree/main/examples/UI/using_streamlit_chat_ui.py)
- Basic Streamlit Chat UI
For more examples, see the [UI examples]((https://www.github.com/llmware-ai/llmware/tree/main/examples/UI/) in the main repo.
Check back often - we are updating these examples regularly - and many of these examples have companion videos as well.
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span><img src="assets/images/hf-logo.svg" alt="Hugging Face" class="hugging-face-logo"/></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/examples/use_cases.md
+137
View File
@@ -0,0 +1,137 @@
---
layout: default
title: Use Cases
parent: Examples
nav_order: 1
description: overview of the major modules and classes of LLMWare
permalink: /examples/use_cases
---
🚀 Use Cases Examples 🚀
---
**End-to-End Scenarios**
We provide several 'end-to-end' examples that show how to use LLMWare in a complex recipe combining different elements to accomplish a specific objective. While each example is still high-level, it is shared in the spirit of providing a high-level framework 'starting point' that can be developed in more detail for a variety of common use cases. All of these examples use small, specialized models, running locally - 'Small, but Mighty' !
1. [**Research Automation with Agents and Web Services**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Use_Cases/web_services_slim_fx.py)
- Prepare a 30-key research analysis on a company
- Extract key lookup and other information from an earnings press release
- Automatically use the lookup data for real-time stock information from YFinance
- Automatically use the lookup date for background company history information in Wikipedia
- Run LLM prompts to ask key questions of the Wikipedia sources
- Aggregate into a consolidated research analysis
- All with local open source models
2. [**Invoice Processing**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Use_Cases/invoice_processing.py)
- Parse a batch of invoices (provided as sample files)
- Extract key information from the invoices
- Save the prompt state for follow-up review and analysis
3. [**Analyzing and Extracting Voice Transcripts**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Use_Cases/parsing_great_speeches.py)
- Voice transcription of 50+ wav files of great speeches of the 20th century
- Run text queries against the transcribed wav files
- Execute LLM agent inferences to extract and identify key elements of interest
- Prepare 'bibliography' with the key extracted points, including time-stamp
4. [**MSA Processing**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Use_Cases/msa_processing.py)
- Identify the termination provisions in Master Service Agreements among a larger batch of contracts
- Parse and query a large batch of contracts and identify the agreements with "Master Service Agreement" on the first page
- Find the termination provisions in each MSA
- Prompt LLM to read the termination provisions and answer a key question
- Run a fact-check and source-check on the LLM response
- Save all of the responses in CSV and JSON for follow-up review.
5. [**Querying a CSV**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Use_Cases/agent_with_custom_tables.py)
- Start running natural language queries on CSVs with Postgres and slim-sql-tool.
- Load a sample 'customer_table.csv' into Postgres
- Start running natural language queries that get converted into SQL and query the DB
6. [**Contract Analysis**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Use_Cases/contract_analysis_on_laptop_with_bling_models.py)
- Extract key information from set of employment agreement
- Use a simple retrieval strategy with keyword search to identify key provisions and topic areas
- Prompt LLM to read the key provisions and answer questions based on those source materials
7. [**Slicing and Dicing Office Docs**](https://www.github.com/llmware-ai/llmware/tree/main/examples/Use_Cases/slicing_and_dicing_office_docs.py)
- Shows a variety of advanced parsing techniques with Office document formats packaged in ZIP archives
- Extracts tables and images, runs OCR against the embedded images, exports the whole library, and creates dataset
For more examples, see the [use cases example]((https://www.github.com/llmware-ai/llmware/tree/main/examples/Use_Cases/) in the main repo.
Check back often - we are updating these examples regularly - and many of these examples have companion videos as well.
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <i class="fa-solid fa-face-smiling-hands"></i>
</span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/getting_started/clone_repo.md
+92
View File
@@ -0,0 +1,92 @@
---
layout: default
title: Clone Repo
parent: Getting Started
nav_order: 3
permalink: /getting_started/clone_repo
---
## ✍️ Working with the llmware Github repository
The llmware repo can be pulled locally to get access to all the examples, or to work directly with the latest version of the llmware code.
```bash
git clone git@github.com:llmware-ai/llmware.git
```
We have provided a **welcome_to_llmware** automation script in the root of the repository folder. After cloning:
- On Windows command line: `.\welcome_to_llmware_windows.sh`
- On Mac / Linux command line: `sh ./welcome_to_llmware.sh`
Alternatively, if you prefer to complete setup without the welcome automation script, then the next steps include:
1. **install requirements.txt** - inside the /llmware path - e.g., ```pip3 install -r llmware/requirements.txt```
2. **install requirements_extras.txt** - inside the /llmware path - e.g., ```pip3 install -r llmware/requirements_extras.txt``` (Depending upon your use case, you may not need all or any of these installs, but some of these will be used in the examples.)
3. **run examples** - copy one or more of the example .py files into the root project path. (We have seen several IDEs that will attempt to run interactively from the nested /example path, and then not have access to the /llmware module - the easy fix is to just copy the example you want to run into the root path).
4. **install vector db** - no-install vector db options include milvus lite, chromadb, faiss and lancedb - which do not require a server install, but do require that you install the python sdk library for that vector db, e.g., `pip3 install pymilvus`, or `pip3 install chromadb`. If you look in [examples/Embedding](https://github.com/llmware-ai/llmware/tree/main/examples/Embedding), you will see examples for getting started with various vector DB, and in the root of the repo, you will see easy-to-get-started docker compose scripts for installing milvus, postgres/pgvector, mongo, qdrant, neo4j, and redis.
5. Note: we have seen recently issues with Pytorch==2.3 on some platforms - if you run into any issues, we have seen that uninstalling Pytorch and downleveling to Pytorch==2.1 usually solves the problem.
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span><img src="assets/images/hf-logo.svg" alt="Hugging Face" class="hugging-face-logo"/></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/getting_started/fast_start.md
+143
View File
@@ -0,0 +1,143 @@
---
layout: default
title: Fast Start
parent: Getting Started
nav_order: 4
permalink: /getting_started/fast_start
---
Fast Start: Learning RAG with llmware through 6 examples
---
**Welcome to llmware!**
Fast Start is a structured series of 6 self-contained examples and accompanying videos that walk through the core foundational components of RAG with LLMWare.
Set up
`pip3 install llmware` or, if you prefer clone the github repo locally, e.g., `git clone git@github.com:llmware-ai/llmware.git
`.
Platforms:
- Mac M1/M2/M3, Windows, Linux (Ubuntu 20 or Ubuntu 22 preferred)
- RAM: 16 GB minimum
- Python 3.9, 3.10, 3.11, 3.12
- Pull the latest version of llmware == 0.2.14 (as of mid-May 2024)
- Please note that we have updated the examples from the original versions, to use new features in llmware, so there may be minor differences with the videos, which are annotated in the comments in each example.
There are 6 examples, designed to be used step-by-step, but each is self-contained,
so you can feel free to jump into any of the examples, in any order, that you prefer.
Each example has been designed to be "copy-paste" and RUN with lots of helpful comments and explanations embedded in the code samples.
Please check out our [Fast Start Youtube tutorials](https://www.youtube.com/playlist?list=PL1-dn33KwsmD7SB9iSO6vx4ZLRAWea1DB) that walk through each example below.
Examples:
**Section I - Learning the Main Components**
1. **Library** - parse, text chunk, and index to convert a "pile of files" into an AI-ready knowledge-base. [Video](https://youtu.be/2xDefZ4oBOM?si=8vRCvqj0-HG3zc4c)
2. **Embeddings** - apply an embedding model to the Library, store vectors, and start enabling natural language queries. [Video](https://youtu.be/xQEk6ohvfV0?si=B3X25ZsAZfW4AR_3)
3. **Prompts** & **Model Catalog** - start running inferences and building prompts. [Video](https://youtu.be/swiu4oBVfbA?si=0IVmLhiiYS3-pMIg)
**Section II - Connecting Knowledge with Prompts - 3 scenarios**
4. **RAG with Text Query** - start integrating documents into prompts. [Video](https://youtu.be/6oALi67HP7U?si=pAbvio4ULXTIXKdL)
5. **RAG with Semantic Query** - use natural language queries on documents and integrate with prompts. [Video](https://youtu.be/XT4kIXA9H3Q?si=EBCAxVXBt5vgYY8s)
6. **RAG with more complex retrieval** - start integrating more complex retrieval patterns. [Video](https://youtu.be/G1Q6Ar8THbo?si=vIVAv35uXAcnaUJy)
After completing these 6 examples, you should have a good foundation and set of recipes to start
exploring the other 100+ examples in the /examples folder, and build more sophisticated
LLM-based applications.
**Models**
- All of these examples are optimized for using local CPU-based models, primarily BLING and DRAGON.
- If you want to substitute for any other model in the catalog, it is generally as easy as
switching the model_name. If the model requires API keys, we show in the examples how to pass those keys as an
environment variable.
**Collection Databases**
- Our parsers are optimized to index text chunks directly into a persistent data store.
- For Fast Start, we will use "sqlite" which is an embedded database, requiring no install
- For more scalable deployment, we would recommend either "mongo" or "postgres"
- Install instructions for "mongo" and "postgres" are provided in docker-compose files in the repository
**Vector Databases**
- For Fast Start, we will use "chromadb" in persistent 'file' mode, requiring no install.
- Note: if you are using Python < 3.12, then please feel free to substitute for faiss (which was used in the videos).
- Note: depending upon how and when you installed llmware, you may need to `pip install chromadb`.
- For more scalable deployment, we would recommend installing one of 9 supported vector databases,
including Milvus, PGVector (Postgres), Redis, Qdrant, Neo4j, Mongo-Atlas, Chroma, LanceDB, or Pinecone.
- Install instructions provided in "examples/Embedding" for specific db, as well as docker-compose scripts.
**Local Private**
- All of the processing will take place locally on your laptop.
*This is an ongoing initiative to provide easy-to-get-started tutorials - we welcome and encourage feedback, as well
as contributions with examples and other tips for helping others on their LLM application journeys!*
**Let's get started!**
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span><img src="assets/images/hf-logo.svg" alt="Hugging Face" class="hugging-face-logo"/></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/getting_started/getting_started.md
+166
View File
@@ -0,0 +1,166 @@
---
layout: default
title: Getting Started
nav_order: 2
has_children: true
description: getting started with llmware
permalink: /getting_started
---
## Welcome to
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://llmware.ai/"><span><img src="assets/images/llmware_logo_color_cropped.png" alt="llmware" width="360" height="60"/></span></a>
</li>
</ul>
## 🧰🛠️🔩The Ultimate Toolkit for Enterprise RAG Pipelines with Small, Specialized Models
From quickly building POCs to scalable LLM Apps for the enterprise, LLMWare is packed with all the tools you need.
`llmware` is an integrated framework with over 50+ small, specialized, open source models for quickly developing LLM-based applications including Retrieval Augmented Generation (RAG) and Multi-Step Orchestration of Agent Workflows.
This project provides a comprehensive set of tools that anyone can use - from a beginner to the most sophisticated AI developer - to rapidly build industrial-grade, knowledge-based enterprise LLM applications.
Our specific focus is on making it easy to integrate open source small specialized models and connecting enterprise knowledge safely and securely.
## Getting Started
1. Install llmware - `pip3 install llmware`
2. Make sure that you are running on a [supported platform](https://github.com/llmware-ai/llmware/blob/main/docs/getting_started/platforms.md#platform-support).
3. Learn by example:
-- [Fast Start examples](https://www.github.com/llmware-ai/llmware/tree/main/fast_start) - structured set of 6 examples (with no DB installations required) to learn the main concepts of RAG with LLMWare - each example has extensive comments, and a supporting video on Youtube to walk you through it.
-- [Getting Started examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/Getting_Started) - heavily-annotated examples that review many getting started elements - selecting a database, loading sample files, working with libraries, and how to use the Model Catalog.
-- [Use Case examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/Use_Cases) - longer examples that integrate several components of LLMWare to provide a framework for a solution for common use case patterns.
-- Dive into specific area of interest - [Parsing](https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing) - [Models](https://www.github.com/llmware-ai/llmware/tree/main/examples/Models) - [Prompts](https://www.github.com/llmware-ai/llmware/tree/main/examples/Models) - [Agents](https://www.github.com/llmware-ai/llmware/tree/main/examples/SLIM-Agents) - and many more ...
4. We provide extensive [sample files](https://www.github.com/llmware-ai/tree/main/examples/Getting_Started/loading_sample_files.py) integrated into the examples, so you can copy-paste-run, and quickly validate that the installation is set up correctly, and to start seeing key classes and methods in action. We would encourage you to start with the 'out of the box' example first, and then use the example as the launching point for inserting your documents, models, queries, and workflows.
5. Learn by watching: check out the [LLMWare Youtube channel](https://www.youtube.com/@llmware).
6. Share with the community: join us on [Discord](https://discord.gg/MhZn5Nc39h).
[Install llmware](#install-llmware){: .btn .btn-primary .fs-5 .mb-4 .mb-md-0 .mr-2 }
[Common Setup & Configuration Items](#platform-support){: .btn .fs-5 .mb-4 .mb-md-0 }
[Architecture](architecture.md/#llmware-architecture){: .btn .fs-5 .mb-4 .mb-md-0 }
[View llmware on GitHub](https://www.github.com/llmware-ai/llmware/tree/main){: .btn .fs-5 .mb-4 .mb-md-0 }
[Open an Issue on GitHub](https://www.github.com/llmware-ai/llmware/issues){: .btn .fs-5 .mb-4 .mb-md-0 }
# Install llmware
___
**Using Pip Install**
- Installing llmware is easy: `pip3 install llmware`
- If you prefer, we also provide a set of recent wheels in the [wheel archives](https://www.github.com/llmware-ai/llmware/tree/main/wheel_archives) in this repository, which can be downloaded individually and used as follows:
```bash
pip3 install llmware-0.2.12-py3-none-any.wheel
````
- We generally keep the main branch of this repository current with all changes, but we only publish new wheels to PyPi approximately once per week
___
___
**Cloning the Repository**
- If you prefer to clone the repository:
```bash
git clone git@github.com:llmware-ai/llmware.git
```
- The llmware package is contained entirely in the /llmware folder path, so you should be able to drop this folder (with all of its contents) into a project tree, and use the llmware module essentially the same as a pip install.
- Please ensure that you are capturing and updating the /llmware/lib folder, which includes required compiled shared libraries. If you prefer, you can keep only those libs required for your OS platform.
- After cloning the repo, we provide a short 'welcome to llmware' automation script, which can be used to install the projects requirements (from llmware/requirements.txt), install several optional dependencies that are commonly used in examples, copy several good 'getting started' examples into the root folder, and then run a 'welcome_example.py' script to get started using our models. To use the "welcome to llmware" script:
Windows:
```bash
.\welcome_to_llmware_windows.sh
```
Mac/Linux:
```bash
sh ./welcome_to_llmware.sh
```
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span><img src="assets/images/hf-logo.svg" alt="Hugging Face" class="hugging-face-logo"/></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/getting_started/installation.md
+119
View File
@@ -0,0 +1,119 @@
---
layout: default
title: Installation
parent: Getting Started
nav_order: 2
permalink: /getting_started/installation
---
## Installation
Set up
`pip3 install llmware` or, if you prefer clone the github repo locally, e.g., `git clone git@github.com:llmware-ai/llmware.git
`.
Platforms:
- Mac M1/M2/M3, Windows, Linux (Ubuntu 20 or Ubuntu 22 preferred)
- RAM: 16 GB minimum
- Python 3.9, 3.10, 3.11 (note: not supported on 3.12 - coming soon!)
- Pull the latest version of llmware == 0.2.11 (as of end of April 2024)
- Please note that we have updated the examples from the original versions, to use new features in llmware, so there may be minor differences with the videos, which are annotated in the comments in each example.
## Wheel Archive
- If you prefer, we also provide a set of recent wheels in the [wheel archives](https://www.github.com/llmware-ai/llmware/tree/main/wheel_archives) in this repository, which can be downloaded individually and used as follows:
```bash
pip3 install llmware-0.2.12-py3-none-any.wheel
````
- We generally keep the main branch of this repository current with all changes, but we only publish new wheels to PyPi approximately once per week
___
___
**Cloning the Repository**
- If you prefer to clone the repository:
```bash
git clone git@github.com:llmware-ai/llmware.git
```
- The llmware package is contained entirely in the /llmware folder path, so you should be able to drop this folder (with all of its contents) into a project tree, and use the llmware module essentially the same as a pip install.
- Please ensure that you are capturing and updating the /llmware/lib folder, which includes required compiled shared libraries. If you prefer, you can keep only those libs required for your OS platform.
- After cloning the repo, we provide a short 'welcome to llmware' automation script, which can be used to install the projects requirements (from llmware/requirements.txt), install several optional dependencies that are commonly used in examples, copy several good 'getting started' examples into the root folder, and then run a 'welcome_example.py' script to get started using our models. To use the "welcome to llmware" script:
Windows:
```bash
.\welcome_to_llmware_windows.sh
```
Mac/Linux:
```bash
sh ./welcome_to_llmware.sh
```
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span><img src="assets/images/hf-logo.svg" alt="Hugging Face" class="hugging-face-logo"/></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/getting_started/overview.md
+653
View File
@@ -0,0 +1,653 @@
---
layout: default
title: Overview
parent: Getting Started
nav_order: 1
permalink: /getting_started/overview
---
## Welcome to
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://llmware.ai/"><span><img src="assets/images/llmware_logo_color_cropped.png" alt="llmware" width="360" height="60"/></span></a>
</li>
</ul>
## 🧰🛠️🔩Building Enterprise RAG Pipelines with Small, Specialized Models
`llmware` provides a unified framework for building LLM-based applications (e.g, RAG, Agents), using small, specialized models that can be deployed privately, integrated with enterprise knowledge sources safely and securely, and cost-effectively tuned and adapted for any business process.
`llmware` has two main components:
1. **RAG Pipeline** - integrated components for the full lifecycle of connecting knowledge sources to generative AI models; and
2. **50+ small, specialized models** fine-tuned for key tasks in enterprise process automation, including fact-based question-answering, classification, summarization, and extraction.
By bringing together both of these components, along with integrating leading open source models and underlying technologies, `llmware` offers a comprehensive set of tools to rapidly build knowledge-based enterprise LLM applications.
Most of our examples can be run without a GPU server - get started right away on your laptop.
## 🎯 Key features
Writing code with`llmware` is based on a few main concepts:
<details>
<summary><b>Model Catalog</b>: Access all models the same way with easy lookup, regardless of underlying implementation.
</summary>
```python
# 150+ Models in Catalog with 50+ RAG-optimized BLING, DRAGON and Industry BERT models
# Full support for GGUF, HuggingFace, Sentence Transformers and major API-based models
# Easy to extend to add custom models - see examples
from llmware.models import ModelCatalog
from llmware.prompts import Prompt
# all models accessed through the ModelCatalog
models = ModelCatalog().list_all_models()
# to use any model in the ModelCatalog - "load_model" method and pass the model_name parameter
my_model = ModelCatalog().load_model("llmware/bling-phi-3-gguf")
output = my_model.inference("what is the future of AI?", add_context="Here is the article to read")
# to integrate model into a Prompt
prompter = Prompt().load_model("llmware/bling-tiny-llama-v0")
response = prompter.prompt_main("what is the future of AI?", context="Insert Sources of information")
```
</details>
<details>
<summary><b>Library</b>: ingest, organize and index a collection of knowledge at scale - Parse, Text Chunk and Embed. </summary>
```python
from llmware.library import Library
# to parse and text chunk a set of documents (pdf, pptx, docx, xlsx, txt, csv, md, json/jsonl, wav, png, jpg, html)
# step 1 - create a library, which is the 'knowledge-base container' construct
# - libraries have both text collection (DB) resources, and file resources (e.g., llmware_data/accounts/{library_name})
# - embeddings and queries are run against a library
lib = Library().create_new_library("my_library")
# step 2 - add_files is the universal ingestion function - point it at a local file folder with mixed file types
# - files will be routed by file extension to the correct parser, parsed, text chunked and indexed in text collection DB
lib.add_files("/folder/path/to/my/files")
# to install an embedding on a library - pick an embedding model and vector_db
lib.install_new_embedding(embedding_model_name="mini-lm-sbert", vector_db="milvus", batch_size=500)
# to add a second embedding to the same library (mix-and-match models + vector db)
lib.install_new_embedding(embedding_model_name="industry-bert-sec", vector_db="chromadb", batch_size=100)
# easy to create multiple libraries for different projects and groups
finance_lib = Library().create_new_library("finance_q4_2023")
finance_lib.add_files("/finance_folder/")
hr_lib = Library().create_new_library("hr_policies")
hr_lib.add_files("/hr_folder/")
# pull library card with key metadata - documents, text chunks, images, tables, embedding record
lib_card = Library().get_library_card("my_library")
# see all libraries
all_my_libs = Library().get_all_library_cards()
```
</details>
<details>
<summary><b>Query</b>: query libraries with mix of text, semantic, hybrid, metadata, and custom filters. </summary>
```python
from llmware.retrieval import Query
from llmware.library import Library
# step 1 - load the previously created library
lib = Library().load_library("my_library")
# step 2 - create a query object and pass the library
q = Query(lib)
# step 3 - run lots of different queries (many other options in the examples)
# basic text query
results1 = q.text_query("text query", result_count=20, exact_mode=False)
# semantic query
results2 = q.semantic_query("semantic query", result_count=10)
# combining a text query restricted to only certain documents in the library and "exact" match to the query
results3 = q.text_query_with_document_filter("new query", {"file_name": "selected file name"}, exact_mode=True)
# to apply a specific embedding (if multiple on library), pass the names when creating the query object
q2 = Query(lib, embedding_model_name="mini_lm_sbert", vector_db="milvus")
results4 = q2.semantic_query("new semantic query")
```
</details>
<details>
<summary><b>Prompt with Sources</b>: the easiest way to combine knowledge retrieval with a LLM inference. </summary>
```python
from llmware.prompts import Prompt
from llmware.retrieval import Query
from llmware.library import Library
# build a prompt
prompter = Prompt().load_model("llmware/bling-tiny-llama-v0")
# add a file -> file is parsed, text chunked, filtered by query, and then packaged as model-ready context,
# including in batches, if needed, to fit the model context window
source = prompter.add_source_document("/folder/to/one/doc/", "filename", query="fast query")
# attach query results (from a Query) into a Prompt
my_lib = Library().load_library("my_library")
results = Query(my_lib).query("my query")
source2 = prompter.add_source_query_results(results)
# run a new query against a library and load directly into a prompt
source3 = prompter.add_source_new_query(my_lib, query="my new query", query_type="semantic", result_count=15)
# to run inference with 'prompt with sources'
responses = prompter.prompt_with_source("my query")
# to run fact-checks - post inference
fact_check = prompter.evidence_check_sources(responses)
# to view source materials (batched 'model-ready' and attached to prompt)
source_materials = prompter.review_sources_summary()
# to see the full prompt history
prompt_history = prompter.get_current_history()
```
</details>
<details>
<summary><b>RAG-Optimized Models</b> - 1-7B parameter models designed for RAG workflow integration and running locally. </summary>
```
""" This 'Hello World' example demonstrates how to get started using local BLING models with provided context, using both
Pytorch and GGUF versions. """
import time
from llmware.prompts import Prompt
def hello_world_questions():
test_list = [
{"query": "What is the total amount of the invoice?",
"answer": "$22,500.00",
"context": "Services Vendor Inc. \n100 Elm Street Pleasantville, NY \nTO Alpha Inc. 5900 1st Street "
"Los Angeles, CA \nDescription Front End Engineering Service $5000.00 \n Back End Engineering"
" Service $7500.00 \n Quality Assurance Manager $10,000.00 \n Total Amount $22,500.00 \n"
"Make all checks payable to Services Vendor Inc. Payment is due within 30 days."
"If you have any questions concerning this invoice, contact Bia Hermes. "
"THANK YOU FOR YOUR BUSINESS! INVOICE INVOICE # 0001 DATE 01/01/2022 FOR Alpha Project P.O. # 1000"},
{"query": "What was the amount of the trade surplus?",
"answer": "62.4 billion yen ($416.6 million)",
"context": "Japan’s September trade balance swings into surplus, surprising expectations"
"Japan recorded a trade surplus of 62.4 billion yen ($416.6 million) for September, "
"beating expectations from economists polled by Reuters for a trade deficit of 42.5 "
"billion yen. Data from Japan’s customs agency revealed that exports in September "
"increased 4.3% year on year, while imports slid 16.3% compared to the same period "
"last year. According to FactSet, exports to Asia fell for the ninth straight month, "
"which reflected ongoing China weakness. Exports were supported by shipments to "
"Western markets, FactSet added. — Lim Hui Jie"},
{"query": "When did the LISP machine market collapse?",
"answer": "1987.",
"context": "The attendees became the leaders of AI research in the 1960s."
" They and their students produced programs that the press described as 'astonishing': "
"computers were learning checkers strategies, solving word problems in algebra, "
"proving logical theorems and speaking English. By the middle of the 1960s, research in "
"the U.S. was heavily funded by the Department of Defense and laboratories had been "
"established around the world. Herbert Simon predicted, 'machines will be capable, "
"within twenty years, of doing any work a man can do'. Marvin Minsky agreed, writing, "
"'within a generation ... the problem of creating 'artificial intelligence' will "
"substantially be solved'. They had, however, underestimated the difficulty of the problem. "
"Both the U.S. and British governments cut off exploratory research in response "
"to the criticism of Sir James Lighthill and ongoing pressure from the US Congress "
"to fund more productive projects. Minsky's and Papert's book Perceptrons was understood "
"as proving that artificial neural networks approach would never be useful for solving "
"real-world tasks, thus discrediting the approach altogether. The 'AI winter', a period "
"when obtaining funding for AI projects was difficult, followed. In the early 1980s, "
"AI research was revived by the commercial success of expert systems, a form of AI "
"program that simulated the knowledge and analytical skills of human experts. By 1985, "
"the market for AI had reached over a billion dollars. At the same time, Japan's fifth "
"generation computer project inspired the U.S. and British governments to restore funding "
"for academic research. However, beginning with the collapse of the Lisp Machine market "
"in 1987, AI once again fell into disrepute, and a second, longer-lasting winter began."},
{"query": "What is the current rate on 10-year treasuries?",
"answer": "4.58%",
"context": "Stocks rallied Friday even after the release of stronger-than-expected U.S. jobs data "
"and a major increase in Treasury yields. The Dow Jones Industrial Average gained 195.12 points, "
"or 0.76%, to close at 31,419.58. The S&P 500 added 1.59% at 4,008.50. The tech-heavy "
"Nasdaq Composite rose 1.35%, closing at 12,299.68. The U.S. economy added 438,000 jobs in "
"August, the Labor Department said. Economists polled by Dow Jones expected 273,000 "
"jobs. However, wages rose less than expected last month. Stocks posted a stunning "
"turnaround on Friday, after initially falling on the stronger-than-expected jobs report. "
"At its session low, the Dow had fallen as much as 198 points; it surged by more than "
"500 points at the height of the rally. The Nasdaq and the S&P 500 slid by 0.8% during "
"their lowest points in the day. Traders were unclear of the reason for the intraday "
"reversal. Some noted it could be the softer wage number in the jobs report that made "
"investors rethink their earlier bearish stance. Others noted the pullback in yields from "
"the day’s highs. Part of the rally may just be to do a market that had gotten extremely "
"oversold with the S&P 500 at one point this week down more than 9% from its high earlier "
"this year. Yields initially surged after the report, with the 10-year Treasury rate trading "
"near its highest level in 14 years. The benchmark rate later eased from those levels, but "
"was still up around 6 basis points at 4.58%. 'We’re seeing a little bit of a give back "
"in yields from where we were around 4.8%. [With] them pulling back a bit, I think that’s "
"helping the stock market,' said Margaret Jones, chief investment officer at Vibrant Industries "
"Capital Advisors. 'We’ve had a lot of weakness in the market in recent weeks, and potentially "
"some oversold conditions.'"},
{"query": "Is the expected gross margin greater than 70%?",
"answer": "Yes, between 71.5% and 72.%",
"context": "Outlook NVIDIA’s outlook for the third quarter of fiscal 2024 is as follows:"
"Revenue is expected to be $16.00 billion, plus or minus 2%. GAAP and non-GAAP "
"gross margins are expected to be 71.5% and 72.5%, respectively, plus or minus "
"50 basis points. GAAP and non-GAAP operating expenses are expected to be "
"approximately $2.95 billion and $2.00 billion, respectively. GAAP and non-GAAP "
"other income and expense are expected to be an income of approximately $100 "
"million, excluding gains and losses from non-affiliated investments. GAAP and "
"non-GAAP tax rates are expected to be 14.5%, plus or minus 1%, excluding any discrete items."
"Highlights NVIDIA achieved progress since its previous earnings announcement "
"in these areas: Data Center Second-quarter revenue was a record $10.32 billion, "
"up 141% from the previous quarter and up 171% from a year ago. Announced that the "
"NVIDIA® GH200 Grace™ Hopper™ Superchip for complex AI and HPC workloads is shipping "
"this quarter, with a second-generation version with HBM3e memory expected to ship "
"in Q2 of calendar 2024. "},
{"query": "What is Bank of America's rating on Target?",
"answer": "Buy",
"context": "Here are some of the tickers on my radar for Thursday, Oct. 12, taken directly from "
"my reporter’s notebook: It’s the one-year anniversary of the S&P 500′s bear market bottom "
"of 3,577. Since then, as of Wednesday’s close of 4,376, the broad market index "
"soared more than 22%. Hotter than expected September consumer price index, consumer "
"inflation. The Social Security Administration issues announced a 3.2% cost-of-living "
"adjustment for 2024. Chipotle Mexican Grill (CMG) plans price increases. Pricing power. "
"Cites consumer price index showing sticky retail inflation for the fourth time "
"in two years. Bank of America upgrades Target (TGT) to buy from neutral. Cites "
"risk/reward from depressed levels. Traffic could improve. Gross margin upside. "
"Merchandising better. Freight and transportation better. Target to report quarter "
"next month. In retail, the CNBC Investing Club portfolio owns TJX Companies (TJX), "
"the off-price juggernaut behind T.J. Maxx, Marshalls and HomeGoods. Goldman Sachs "
"tactical buy trades on Club names Wells Fargo (WFC), which reports quarter Friday, "
"Humana (HUM) and Nvidia (NVDA). BofA initiates Snowflake (SNOW) with a buy rating."
"If you like this story, sign up for Jim Cramer’s Top 10 Morning Thoughts on the "
"Market email newsletter for free. Barclays cuts price targets on consumer products: "
"UTZ Brands (UTZ) to $16 per share from $17. Kraft Heinz (KHC) to $36 per share from "
"$38. Cyclical drag. J.M. Smucker (SJM) to $129 from $160. Secular headwinds. "
"Coca-Cola (KO) to $59 from $70. Barclays cut PTs on housing-related stocks: Toll Brothers"
"(TOL) to $74 per share from $82. Keeps underweight. Lowers Trex (TREX) and Azek"
"(AZEK), too. Goldman Sachs (GS) announces sale of fintech platform and warns on "
"third quarter of 19-cent per share drag on earnings. The buyer: investors led by "
"private equity firm Sixth Street. Exiting a mistake. Rise in consumer engagement for "
"Spotify (SPOT), says Morgan Stanley. The analysts hike price target to $190 per share "
"from $185. Keeps overweight (buy) rating. JPMorgan loves elf Beauty (ELF). Keeps "
"overweight (buy) rating but lowers price target to $139 per share from $150. "
"Sees “still challenging” environment into third-quarter print. The Club owns shares "
"in high-end beauty company Estee Lauder (EL). Barclays upgrades First Solar (FSLR) "
"to overweight from equal weight (buy from hold) but lowers price target to $224 per "
"share from $230. Risk reward upgrade. Best visibility of utility scale names."},
{"query": "What was the rate of decline in 3rd quarter sales?",
"answer": "20% year-on-year.",
"context": "Nokia said it would cut up to 14,000 jobs as part of a cost cutting plan following "
"third quarter earnings that plunged. The Finnish telecommunications giant said that "
"it will reduce its cost base and increase operation efficiency to “address the "
"challenging market environment. The substantial layoffs come after Nokia reported "
"third-quarter net sales declined 20% year-on-year to 4.98 billion euros. Profit over "
"the period plunged by 69% year-on-year to 133 million euros."},
{"query": "What is a list of the key points?",
"answer": "•Stocks rallied on Friday with stronger-than-expected U.S jobs data and increase in "
"Treasury yields;\n•Dow Jones gained 195.12 points;\n•S&P 500 added 1.59%;\n•Nasdaq Composite rose "
"1.35%;\n•U.S. economy added 438,000 jobs in August, better than the 273,000 expected;\n"
"•10-year Treasury rate trading near the highest level in 14 years at 4.58%.",
"context": "Stocks rallied Friday even after the release of stronger-than-expected U.S. jobs data "
"and a major increase in Treasury yields. The Dow Jones Industrial Average gained 195.12 points, "
"or 0.76%, to close at 31,419.58. The S&P 500 added 1.59% at 4,008.50. The tech-heavy "
"Nasdaq Composite rose 1.35%, closing at 12,299.68. The U.S. economy added 438,000 jobs in "
"August, the Labor Department said. Economists polled by Dow Jones expected 273,000 "
"jobs. However, wages rose less than expected last month. Stocks posted a stunning "
"turnaround on Friday, after initially falling on the stronger-than-expected jobs report. "
"At its session low, the Dow had fallen as much as 198 points; it surged by more than "
"500 points at the height of the rally. The Nasdaq and the S&P 500 slid by 0.8% during "
"their lowest points in the day. Traders were unclear of the reason for the intraday "
"reversal. Some noted it could be the softer wage number in the jobs report that made "
"investors rethink their earlier bearish stance. Others noted the pullback in yields from "
"the day’s highs. Part of the rally may just be to do a market that had gotten extremely "
"oversold with the S&P 500 at one point this week down more than 9% from its high earlier "
"this year. Yields initially surged after the report, with the 10-year Treasury rate trading "
"near its highest level in 14 years. The benchmark rate later eased from those levels, but "
"was still up around 6 basis points at 4.58%. 'We’re seeing a little bit of a give back "
"in yields from where we were around 4.8%. [With] them pulling back a bit, I think that’s "
"helping the stock market,' said Margaret Jones, chief investment officer at Vibrant Industries "
"Capital Advisors. 'We’ve had a lot of weakness in the market in recent weeks, and potentially "
"some oversold conditions.'"}
]
return test_list
# this is the main script to be run
def bling_meets_llmware_hello_world (model_name):
t0 = time.time()
# load the questions
test_list = hello_world_questions()
print(f"\n > Loading Model: {model_name}...")
# load the model
prompter = Prompt().load_model(model_name)
t1 = time.time()
print(f"\n > Model {model_name} load time: {t1-t0} seconds")
for i, entries in enumerate(test_list):
print(f"\n{i+1}. Query: {entries['query']}")
# run the prompt
output = prompter.prompt_main(entries["query"],context=entries["context"]
, prompt_name="default_with_context",temperature=0.30)
# print out the results
llm_response = output["llm_response"].strip("\n")
print(f"LLM Response: {llm_response}")
print(f"Gold Answer: {entries['answer']}")
print(f"LLM Usage: {output['usage']}")
t2 = time.time()
print(f"\nTotal processing time: {t2-t1} seconds")
return 0
if __name__ == "__main__":
# list of 'rag-instruct' laptop-ready small bling models on HuggingFace
pytorch_models = ["llmware/bling-1b-0.1", # most popular
"llmware/bling-tiny-llama-v0", # fastest
"llmware/bling-1.4b-0.1",
"llmware/bling-falcon-1b-0.1",
"llmware/bling-cerebras-1.3b-0.1",
"llmware/bling-sheared-llama-1.3b-0.1",
"llmware/bling-sheared-llama-2.7b-0.1",
"llmware/bling-red-pajamas-3b-0.1",
"llmware/bling-stable-lm-3b-4e1t-v0",
"llmware/bling-phi-3" # most accurate (and newest)
]
# Quantized GGUF versions generally load faster and run nicely on a laptop with at least 16 GB of RAM
gguf_models = ["bling-phi-3-gguf", "bling-stablelm-3b-tool", "dragon-llama-answer-tool", "dragon-yi-answer-tool", "dragon-mistral-answer-tool"]
# try model from either pytorch or gguf model list
# the newest (and most accurate) is 'bling-phi-3-gguf'
bling_meets_llmware_hello_world(gguf_models[0]
# check out the model card on Huggingface for RAG benchmark test performance results and other useful information
```
</details>
<details>
<summary><b>Simple-to-Scale Database Options </b> - integrated data stores from laptop to parallelized cluster. </summary>
```python
from llmware.configs import LLMWareConfig
# to set the collection database - mongo, sqlite, postgres
LLMWareConfig().set_active_db("mongo")
# to set the vector database (or declare when installing)
# --options: milvus, pg_vector (postgres), redis, qdrant, faiss, pinecone, mongo atlas
LLMWareConfig().set_vector_db("milvus")
# for fast start - no installations required
LLMWareConfig().set_active_db("sqlite")
LLMWareConfig().set_vector_db("chromadb") # try also faiss and lancedb
# for single postgres deployment
LLMWareConfig().set_active_db("postgres")
LLMWareConfig().set_vector_db("postgres")
# to install mongo, milvus, postgres - see the docker-compose scripts as well as examples
```
</details>
<details>
<summary> 🔥 <b> Agents with Function Calls and SLIM Models </b> 🔥 </summary>
```python
from llmware.agents import LLMfx
text = ("Tesla stock fell 8% in premarket trading after reporting fourth-quarter revenue and profit that "
"missed analysts’ estimates. The electric vehicle company also warned that vehicle volume growth in "
"2024 'may be notably lower' than last year’s growth rate. Automotive revenue, meanwhile, increased "
"just 1% from a year earlier, partly because the EVs were selling for less than they had in the past. "
"Tesla implemented steep price cuts in the second half of the year around the world. In a Wednesday "
"presentation, the company warned investors that it’s 'currently between two major growth waves.'")
# create an agent using LLMfx class
agent = LLMfx()
# load text to process
agent.load_work(text)
# load 'models' as 'tools' to be used in analysis process
agent.load_tool("sentiment")
agent.load_tool("extract")
agent.load_tool("topics")
agent.load_tool("boolean")
# run function calls using different tools
agent.sentiment()
agent.topics()
agent.extract(params=["company"])
agent.extract(params=["automotive revenue growth"])
agent.xsum()
agent.boolean(params=["is 2024 growth expected to be strong? (explain)"])
# at end of processing, show the report that was automatically aggregated by key
report = agent.show_report()
# displays a summary of the activity in the process
activity_summary = agent.activity_summary()
# list of the responses gathered
for i, entries in enumerate(agent.response_list):
print("update: response analysis: ", i, entries)
output = {"report": report, "activity_summary": activity_summary, "journal": agent.journal}
```
</details>
<details>
<summary> 🚀 <b>Start coding - Quick Start for RAG </b> 🚀 </summary>
```python
# This example illustrates a simple contract analysis
# using a RAG-optimized LLM running locally
import os
import re
from llmware.prompts import Prompt, HumanInTheLoop
from llmware.setup import Setup
from llmware.configs import LLMWareConfig
def contract_analysis_on_laptop (model_name):
# In this scenario, we will:
# -- download a set of sample contract files
# -- create a Prompt and load a BLING LLM model
# -- parse each contract, extract the relevant passages, and pass questions to a local LLM
# Main loop - Iterate thru each contract:
#
# 1. parse the document in memory (convert from PDF file into text chunks with metadata)
# 2. filter the parsed text chunks with a "topic" (e.g., "governing law") to extract relevant passages
# 3. package and assemble the text chunks into a model-ready context
# 4. ask three key questions for each contract to the LLM
# 5. print to the screen
# 6. save the results in both json and csv for furthe processing and review.
# Load the llmware sample files
print (f"\n > Loading the llmware sample files...")
sample_files_path = Setup().load_sample_files()
contracts_path = os.path.join(sample_files_path,"Agreements")
# Query list - these are the 3 main topics and questions that we would like the LLM to analyze for each contract
query_list = {"executive employment agreement": "What are the name of the two parties?",
"base salary": "What is the executive's base salary?",
"vacation": "How many vacation days will the executive receive?"}
# Load the selected model by name that was passed into the function
print (f"\n > Loading model {model_name}...")
prompter = Prompt().load_model(model_name, temperature=0.0, sample=False)
# Main loop
for i, contract in enumerate(os.listdir(contracts_path)):
# excluding Mac file artifact (annoying, but fact of life in demos)
if contract != ".DS_Store":
print("\nAnalyzing contract: ", str(i+1), contract)
print("LLM Responses:")
for key, value in query_list.items():
# step 1 + 2 + 3 above - contract is parsed, text-chunked, filtered by topic key,
# ... and then packaged into the prompt
source = prompter.add_source_document(contracts_path, contract, query=key)
# step 4 above - calling the LLM with 'source' information already packaged into the prompt
responses = prompter.prompt_with_source(value, prompt_name="default_with_context")
# step 5 above - print out to screen
for r, response in enumerate(responses):
print(key, ":", re.sub("[\n]"," ", response["llm_response"]).strip())
# We're done with this contract, clear the source from the prompt
prompter.clear_source_materials()
# step 6 above - saving the analysis to jsonl and csv
# Save jsonl report to jsonl to /prompt_history folder
print("\nPrompt state saved at: ", os.path.join(LLMWareConfig.get_prompt_path(),prompter.prompt_id))
prompter.save_state()
# Save csv report that includes the model, response, prompt, and evidence for human-in-the-loop review
csv_output = HumanInTheLoop(prompter).export_current_interaction_to_csv()
print("csv output saved at: ", csv_output)
if __name__ == "__main__":
# use local cpu model - try the newest - RAG finetune of Phi-3 quantized and packaged in GGUF
model = "bling-phi-3-gguf"
contract_analysis_on_laptop(model)
```
</details>
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span><img src="assets/images/hf-logo.svg" alt="Hugging Face" class="hugging-face-logo"/></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/getting_started/platforms.md
+155
View File
@@ -0,0 +1,155 @@
---
layout: default
title: Platforms Supported
parent: Getting Started
nav_order: 5
permalink: /getting_started/platforms
---
___
# Platform Support
___
**Platform Supported**
- **Python 3.9+** (note that we just added support for 3.12 starting in llmware version 0.2.12)
- **System RAM**: recommended 16 GB RAM minimum (to run most local models on CPU)
- **OS Supported**: Mac OS M1/M2/M3, Windows, Linux Ubuntu 20/22. We regularly build and test on Windows and Linux platforms with and without CUDA drivers.
- **Deprecated OS**: Linux Aarch64 (0.2.6) and Mac x86 (0.2.10) - most features of llmware should work on these platforms, but new features integrated since those versions will not be available. If you have a particular need to work on one of these platforms, please raise an Issue, and we can work with you to try to find a solution.
- **Linux**: we build to GLIBC 2.31+ - so Linux versions with older GLIBC drivers will generally not work (e.g., Ubuntu 18). To check the GLIBC version, you can use the command `ldd --version`. If it is 2.31 or any higher version, it should work.
___
___
**Database**
- LLMWare is an enterprise-grade data pipeline designed for persistent storage of key artifacts throughout the pipeline. We provide several options to parse 'in-memory' and write to jsonl files, but most of the functionality of LLMWare assumes that a persistent scalable data store will be used.
- There are three different types of data storage used in LLMWare:
1. **Text Collection database** - all of the LLMWare parsers, by default, parse and text chunk unstructured content (and associated metadata) into one of three databases used for text collections, organized in Libraries - **MongoDB**, **Postgres** and **SQLite**.
2. **Vector database** - for storing and retrieving semantic embedding vectors, LLMWare supports the following vector databases - Milvus, PG Vector / Postgres, Qdrant, ChromaDB, Redis, Neo4J, Lance DB, Mongo-Atlas, Pinecone and FAISS.
3. **SQL Tables database** - for easily integrating table-based data into LLM workflows through the CustomTable class and for using in conjunction with a Text-2-SQL workflow - supported on Postgres and SQLite.
- **Fast Start** option: you can start using SQLite locally without any separate installation by setting `LLMWareConfig.set_active_db("sqlite")` as shown in [configure_db_example](https://www.github.com/llmware-ai/llmware/blob/main/examples/Getting_Started/configure_db.py). For vector embedding examples, you can use ChromaDB, LanceDB or FAISS - all of which provide no-install options - just start using.
- **Install DB dependencies**: we provide a number of Docker-Compose scripts which can be used, or follow install instructions provided by the database - generally easiest to install locally with Docker.
**LLMWare File Storage**
- llmware stores a variety of artifacts during its operation locally in the /llmware_data path, which can be found as follows:
```python
from llmware.configs import LLMWareConfig
llmware_fp = LLMWareConfig().get_llmware_path()
print("llmware_data path: ", llmware_fp)
```
- to change the llmware path, we can change both the 'home' path, which is the main filepath, and the 'llmware_data' path name
as follows:
```python
from llmware.configs import LLMWareConfig
# changing the llmware home path - change home + llmware_path_name
LLMWareConfig().set_home("/my/new/local/home/path")
LLMWareConfig().set_llmware_path_name("llmware_data2")
# check the new llmware home path
llmware_fp = LLMWareConfig().get_llmware_path()
print("updated llmware path: ", llmware_fp)
```
___
___
**Local Models**
- LLMWare treats open source and locally deployed models as "first class citizens" with all classes, methods and examples designed to work first with smaller, specialized, locally-deployed models.
- By default, most models are pulled from public HuggingFace repositories, and cached locally. LLMWare will store all models locally at the /llmware_data/model_repo path, with all assets found in a folder tree with the models name.
- If a Pytorch model is pulled from HuggingFace, then it will appear in the default HuggingFace /.cache path.
- To view the local model path:
```python
from llmware.configs import LLMWareConfig
model_fp = LLMWareConfig().get_model_repo_path()
print("model repo path: ", model_fp)
```
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span><img src="assets/images/hf-logo.svg" alt="Hugging Face" class="hugging-face-logo"/></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/getting_started/working_with_docker.md
+157
View File
@@ -0,0 +1,157 @@
---
layout: default
title: Working with Docker
parent: Getting Started
nav_order: 6
permalink: /getting_started/working_with_docker
---
# Working with Docker Scripts
This section is a short guide on setting up a Linux environment with Docker and running LLMWare examples with different database systems.
## 1. Python and Pip
Python should come installed with your Linux environment.
To install Pip, run the following:
```
sudo apt-get update
sudo apt-get -y install python3-pip
pip3 install --upgrade pip
```
## 2. Docker and Docker Compose
The latest versions of Docker and Docker Comopse should be installed to be able to use the Docker Compose files in the LLMWare repository.
Instructions to install Docker: https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04 (Steps 1-2)
Note: Step 1 is necessary, Step 2 is optional but we highly recommend it.
Instructions to install Docker Compose: https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-compose-on-ubuntu-20-04 (Step 1)
Note: replace the URL in the `curl` command with the latest download from https://github.com/docker/compose/releases.
Check that Docker is running on your system:
```
sudo systemctl status docker
```
## 3. Running Docker Compose files
`cd` into the repository and ensure that you can see files of the format `docker-compose-database-name.yaml`.
To run a Compose file:
```
docker-compose -f docker-compose-database-name.yaml up -d
```
Check that the container is running:
```
docker ps
```
Note: this will list only the the containers that are currently running, add the `-a` flag (`docker ps -a`) to list all containers (even those that are stopped).
## 4. Test with Examples
The Compose files currently support 6 database systems:
- Mongo
- Postgres/PG Vector
- Neo4j
- Milvus
- Qdrant
- Redis
Note: PG Vector is an alias for Postgres and is used for vector embeddings.
1. Mongo and Postgres are used as the active database to store library text collections.
2. PG Vector, Neo4j, Milvus, Qdrant and Redis are used as the vector database to store vector embeddings.
To test that the containers are working as intended, you can modify an example provided in the LLMWare repository. The simplest example to do this is `fast_start/example-2-build_embeddings.py`.
Open the file in an editor.
1. Change the argument passed in as the active database on line 128 to an appropriate active database (Mongo or Postgres).
2. Change the argument passed in as the vector database on line 138 to an appropriate vector database (PG Vector, Neo4j, Milvus, Qdrant or Redis).
Run the example with these changes, and you should see updates in the terminal indicating that the embeddings are being generated correctly.
Note: It is possible that you will see an error:
```
llmware.exceptions.EmbeddingModelNotFoundException: Embedding model for 'example2_library' could not be located
```
In this case, use a unique name for the library name passed in on line 147.
## 5. Stopping/Deleting Containers
To stop a container, run:
```
docker stop container_ID_OR_container_name
```
To delete a container, run:
```
docker rm container_ID_OR_container_name
```
Note: passing in either the ID or the name will work.
To find the ID or name of a container, run:
```
docker ps -a
```
---
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span><img src="assets/images/hf-logo.svg" alt="Hugging Face" class="hugging-face-logo"/></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/index.md
+163
View File
@@ -0,0 +1,163 @@
---
layout: default
title: Home | llmware
nav_order: 1
description: llmware is an integrated framework with over 50+ models for quickly developing LLM-based applications including Retrieval Augmented Generation (RAG) and Multi-Step Orchestration of Agent Workflows.
permalink: /
---
## Welcome to
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://llmware.ai/"><span><img src="assets/images/llmware_logo_color_cropped.png" alt="llmware" width="360" height="60"/></span></a>
</li>
</ul>
## 🧰🛠️🔩The Ultimate Toolkit for Enterprise RAG Pipelines with Small, Specialized Models
From quickly building POCs to scalable LLM Apps for the enterprise, LLMWare is packed with all the tools you need.
`llmware` is an integrated framework with over 50+ small, specialized, open source models for quickly developing LLM-based applications including Retrieval Augmented Generation (RAG) and Multi-Step Orchestration of Agent Workflows.
This project provides a comprehensive set of tools that anyone can use - from a beginner to the most sophisticated AI developer - to rapidly build industrial-grade, knowledge-based enterprise LLM applications.
Our specific focus is on making it easy to integrate open source small specialized models and connecting enterprise knowledge safely and securely.
## Getting Started
1. Install llmware - `pip3 install llmware`
2. Make sure that you are running on a [supported platform](https://www.github.com/llmware-ai/llmware/tree/main/docs/getting_started/platforms.md#platform-support).
3. Learn by example:
-- [Fast Start examples](https://www.github.com/llmware-ai/llmware/tree/main/fast_start) - structured set of 6 examples (with no DB installations required) to learn the main concepts of RAG with LLMWare - each example has extensive comments, and a supporting video on Youtube to walk you through it.
-- [Getting Started examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/Getting_Started) - heavily-annotated examples that review many getting started elements - selecting a database, loading sample files, working with libraries, and how to use the Model Catalog.
-- [Use Case examples](https://www.github.com/llmware-ai/llmware/tree/main/examples/Use_Cases) - longer examples that integrate several components of LLMWare to provide a framework for a solution for common use case patterns.
-- Dive into specific area of interest - [Parsing](https://www.github.com/llmware-ai/llmware/tree/main/examples/Parsing) - [Models](https://www.github.com/llmware-ai/llmware/tree/main/examples/Models) - [Prompts](https://www.github.com/llmware-ai/llmware/tree/main/examples/Models) - [Agents](https://www.github.com/llmware-ai/llmware/tree/main/examples/SLIM-Agents) - and many more ...
4. We provide extensive [sample files](https://www.github.com/llmware-ai/tree/main/examples/Getting_Started/loading_sample_files.py) integrated into the examples, so you can copy-paste-run, and quickly validate that the installation is set up correctly, and to start seeing key classes and methods in action. We would encourage you to start with the 'out of the box' example first, and then use the example as the launching point for inserting your documents, models, queries, and workflows.
5. Learn by watching: check out the [LLMWare Youtube channel](https://www.youtube.com/@llmware).
6. Share with the community: join us on [Discord](https://discord.gg/MhZn5Nc39h).
[Install llmware](#install-llmware){: .btn .btn-primary .fs-5 .mb-4 .mb-md-0 .mr-2 }
[Common Setup & Configuration Items](#platform-support){: .btn .fs-5 .mb-4 .mb-md-0 }
[Architecture](architecture.md/#llmware-architecture){: .btn .fs-5 .mb-4 .mb-md-0 }
[View llmware on GitHub](https://www.github.com/llmware-ai/llmware/tree/main){: .btn .fs-5 .mb-4 .mb-md-0 }
[Open an Issue on GitHub](https://www.github.com/llmware-ai/llmware/issues){: .btn .fs-5 .mb-4 .mb-md-0 }
# Install llmware
___
**Using Pip Install**
- Installing llmware is easy: `pip3 install llmware`
- If you prefer, we also provide a set of recent wheels in the [wheel archives](https://www.github.com/llmware-ai/llmware/tree/main/wheel_archives) in this repository, which can be downloaded individually and used as follows:
```bash
pip3 install llmware-0.2.12-py3-none-any.wheel
````
- We generally keep the main branch of this repository current with all changes, but we only publish new wheels to PyPi approximately once per week
___
___
**Cloning the Repository**
- If you prefer to clone the repository:
```bash
git clone git@github.com:llmware-ai/llmware.git
```
- The llmware package is contained entirely in the /llmware folder path, so you should be able to drop this folder (with all of its contents) into a project tree, and use the llmware module essentially the same as a pip install.
- Please ensure that you are capturing and updating the /llmware/lib folder, which includes required compiled shared libraries. If you prefer, you can keep only those libs required for your OS platform.
- After cloning the repo, we provide a short 'welcome to llmware' automation script, which can be used to install the projects requirements (from llmware/requirements.txt), install several optional dependencies that are commonly used in examples, copy several good 'getting started' examples into the root folder, and then run a 'welcome_example.py' script to get started using our models. To use the "welcome to llmware" script:
Windows:
```bash
.\welcome_to_llmware_windows.sh
```
Mac/Linux:
```bash
sh ./welcome_to_llmware.sh
```
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span><img src="assets/images/hf-logo.svg" alt="Hugging Face" class="hugging-face-logo"/></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/learn/advanced_techniques_for_rag.md
+85
View File
@@ -0,0 +1,85 @@
---
layout: default
title: Advanced RAG
parent: Learn
nav_order: 4
description: overview of the major modules and classes of LLMWare
permalink: /learn/advanced_techniques_for_rag
---
llmware Youtube Video Channel
---
**Tutorial Videos** - check out our Youtube channel for high-impact 5-10 minute tutorials on the latest examples.
Check back often as this list is always growing ...
🎬 **Advanced RAG Techniques **
- [Best Small RAG Model - Bling-Phi-3](https://youtu.be/cViMonCAeSc?si=L6jX0sRdZAmKtRcz)
- [Agent Automation with Web Services for Financial Research](https://youtu.be/l0jzsg1_Ik0?si=oBGtALHLplouY9x2)
- [Voice Transcription and Automated Analysis of Greatest Speeches Dataset](https://youtu.be/5y0ez5ZBpPE?si=PAaCIqYou8nCGxYG)
- [Are you prompting wrong for RAG - Stochastic Sampling-Part I](https://youtu.be/7oMTGhSKuNY?si=_KSjuBnqArvWzYbx)
- [Are you prompting wrong for RAG - Stochastic Sampling-Part II- Code Experiments](https://youtu.be/iXp1tj-pPjM?si=3ZeMgipY0vJDHIMY)
- [Text2SQL Intro](https://youtu.be/BKZ6kO2XxNo?si=tXGt63pvrp_rOlIP)
- [Pop up LLMWare Inference Server](https://www.youtube.com/watch?v=qiEmLnSRDUA&t=20s)
- [Hardest Problem in RAG - handling 'Not Found'](https://youtu.be/slDeF7bYuv0?si=j1nkdwdGr5sgvUtK)
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/learn/core_rag_scenarios_running_locally.md
+86
View File
@@ -0,0 +1,86 @@
---
layout: default
title: Core RAG Scenarios Running Locally
parent: Learn
nav_order: 2
description: overview of the major modules and classes of LLMWare
permalink: /learn/core_rag_scenarios_running_locally
---
Core RAG Scenarios Run Locally
---
**Tutorial Videos** - check out our Youtube channel for high-impact 5-10 minute tutorials on the latest examples.
Check back often as this list is always growing ...
🎬 **Core RAG Scenarios**
- [Use small LLMs for RAG for Contract Analysis (feat. LLMWare)](https://www.youtube.com/watch?v=8aV5p3tErP0)
- [Invoice Processing with LLMware](https://www.youtube.com/watch?v=VHZSaBBG-Bo&t=10s)
- [Evaluate LLMs for RAG with LLMWare](https://www.youtube.com/watch?v=s0KWqYg5Buk&t=105s)
- [Fast Start to RAG with LLMWare Open Source Library](https://www.youtube.com/watch?v=0naqpH93eEU)
- [Use Retrieval Augmented Generation (RAG) without a Database](https://www.youtube.com/watch?v=tAGz6yR14lw)
- [Best Small RAG Model - Bling-Phi-3](https://youtu.be/cViMonCAeSc?si=L6jX0sRdZAmKtRcz)
- [RAG with BLING on your laptop](https://www.youtube.com/watch?v=JjgqOZ2v5oU)
- [DRAGON-7B-Models](https://www.youtube.com/watch?v=d_u7VaKu6Qk&t=37s)
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/learn/integrated_voice_transcription_with_whisper_cpp.md
+78
View File
@@ -0,0 +1,78 @@
---
layout: default
title: Voice Transcription with Whisper CPP
parent: Learn
nav_order: 6
description: overview of the major modules and classes of LLMWare
permalink: /learn/integrated_voice_transcription_with_whisper_cpp
---
Integrated Voice Transcription with Whisper CPP
---
**Tutorial Videos** - check out our Youtube channel for high-impact 5-10 minute tutorials on the latest examples.
Check back often as this list is always growing ...
🎬 **Using Whisper CPP Models**
- [Getting Started with Whisper.CPP](https://youtu.be/YG5u5AOU9MQ?si=5xQYZCILPSiR8n4s)
- [Voice Transcription and Automated Analysis of Greatest Speeches Dataset](https://youtu.be/5y0ez5ZBpPE?si=PAaCIqYou8nCGxYG)
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/learn/learn.md
+118
View File
@@ -0,0 +1,118 @@
---
layout: default
title: Learn
nav_order: 4
has_children: true
description: key learning resources
permalink: /learn
---
Learn: Youtube Video Series
---
**Tutorial Videos** - check out our Youtube channel for high-impact 5-10 minute tutorials on the latest examples.
Check back often as this list is always growing ...
🎬 **Some of our most recent videos**
- [Best Small RAG Model - Bling-Phi-3](https://youtu.be/cViMonCAeSc?si=L6jX0sRdZAmKtRcz)
- [Agent Automation with Web Services for Financial Research](https://youtu.be/l0jzsg1_Ik0?si=oBGtALHLplouY9x2)
- [Voice Transcription and Automated Analysis of Greatest Speeches Dataset](https://youtu.be/5y0ez5ZBpPE?si=PAaCIqYou8nCGxYG)
- [Are you prompting wrong for RAG - Stochastic Sampling-Part I](https://youtu.be/7oMTGhSKuNY?si=_KSjuBnqArvWzYbx)
- [Are you prompting wrong for RAG - Stochastic Sampling-Part II- Code Experiments](https://youtu.be/iXp1tj-pPjM?si=3ZeMgipY0vJDHIMY)
🎬 **Using Agents, Function Calls and SLIM models**
- [SLIMS Playlist](https://youtube.com/playlist?list=PL1-dn33KwsmAHWCWK6YjZrzicQ2yR6W8T&si=TSFGqQ3ObOO5vDde)
- [Agent-based Complex Research Analysis](https://youtu.be/y4WvwHqRR60?si=jX3KCrKcYkM95boe)
- [Getting Started with SLIMs (with code)](https://youtu.be/aWZFrTDmMPc?si=lmo98_quo_2Hrq0C)
- [SLIM Models Intro](https://www.youtube.com/watch?v=cQfdaTcmBpY)
- [Text2SQL Intro](https://youtu.be/BKZ6kO2XxNo?si=tXGt63pvrp_rOlIP)
- [Pop up LLMWare Inference Server](https://www.youtube.com/watch?v=qiEmLnSRDUA&t=20s)
- [Hardest Problem in RAG - handling 'Not Found'](https://youtu.be/slDeF7bYuv0?si=j1nkdwdGr5sgvUtK)
- [Extract Information from Earnings Releases](https://youtu.be/d6HFfyDk4YE?si=VmnIiWFmgBtR4DxS)
- [Summary Function Calls](https://youtu.be/yNg_KH5cPSk?si=Yl94tp_vKA8e7eT7)
- [Boolean Yes-No Function Calls](https://youtu.be/jZQZMMqAJXs?si=lU4YVI0H0tfc9k6e)
- [Autogenerate Topics, Tags and NER](https://youtu.be/N6oOxuyDsC4?si=vo2Fd8VG5xTbH4SD)
🎬 **Using GGUF Models**
- [Using LM Studio Models](https://www.youtube.com/watch?v=h2FDjUyvsKE)
- [Using Ollama Models](https://www.youtube.com/watch?v=qITahpVDuV0)
- [Use any GGUF Model](https://www.youtube.com/watch?v=9wXJgld7Yow)
- [Background on GGUF Quantization & DRAGON Model Example](https://www.youtube.com/watch?v=ZJyQIZNJ45E)
- [Getting Started with Whisper.CPP](https://youtu.be/YG5u5AOU9MQ?si=5xQYZCILPSiR8n4s)
🎬 **Core RAG Scenarios Running Locally**
- [RAG with BLING on your laptop](https://www.youtube.com/watch?v=JjgqOZ2v5oU)
- [DRAGON-7B-Models](https://www.youtube.com/watch?v=d_u7VaKu6Qk&t=37s)
- [Use small LLMs for RAG for Contract Analysis (feat. LLMWare)](https://www.youtube.com/watch?v=8aV5p3tErP0)
- [Invoice Processing with LLMware](https://www.youtube.com/watch?v=VHZSaBBG-Bo&t=10s)
- [Evaluate LLMs for RAG with LLMWare](https://www.youtube.com/watch?v=s0KWqYg5Buk&t=105s)
- [Fast Start to RAG with LLMWare Open Source Library](https://www.youtube.com/watch?v=0naqpH93eEU)
- [Use Retrieval Augmented Generation (RAG) without a Database](https://www.youtube.com/watch?v=tAGz6yR14lw)
🎬 **Parsing, Embedding, Data Pipelines and Extraction**
- [Ingest PDFs at Scale](https://www.youtube.com/watch?v=O0adUfrrxi8&t=10s)
- [Install and Compare Multiple Embeddings with Postgres and PGVector](https://www.youtube.com/watch?v=Bncvggy6m5Q)
- [Intro to Parsing and Text Chunking](https://youtu.be/2xDefZ4oBOM?si=YZzBUjDfQ0839EVF)
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/learn/other_topics.md
+85
View File
@@ -0,0 +1,85 @@
---
layout: default
title: Other Topics
parent: Learn
nav_order: 7
description: overview of the major modules and classes of LLMWare
permalink: /learn/other_topics
---
Other Notable Videos and Topics
---
**Tutorial Videos** - check out our Youtube channel for high-impact 5-10 minute tutorials on the latest examples.
Check back often as this list is always growing ...
🎬 **Some of our most recent videos**
- [Fast Local Chatbot with Phi-3-GGUF](https://youtu.be/gzzEVK8p3VM?si=HTMWQtN9XuaqjmpK)
- [Document Summarization](https://youtu.be/Ps3W-P9A1m8?si=mHvCcHvrKzndaNul)
- [Agent Server](https://youtu.be/nsA6-ZdnkXg?si=v7iGhC_rpj8TWbbl)
- [Best Small RAG Model - Bling-Phi-3](https://youtu.be/cViMonCAeSc?si=L6jX0sRdZAmKtRcz)
- [Agent Automation with Web Services for Financial Research](https://youtu.be/l0jzsg1_Ik0?si=oBGtALHLplouY9x2)
- [Voice Transcription and Automated Analysis of Greatest Speeches Dataset](https://youtu.be/5y0ez5ZBpPE?si=PAaCIqYou8nCGxYG)
- [Are you prompting wrong for RAG - Stochastic Sampling-Part I](https://youtu.be/7oMTGhSKuNY?si=_KSjuBnqArvWzYbx)
- [Are you prompting wrong for RAG - Stochastic Sampling-Part II- Code Experiments](https://youtu.be/iXp1tj-pPjM?si=3ZeMgipY0vJDHIMY)
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/learn/parsing_embedding_data_extraction.md
+82
View File
@@ -0,0 +1,82 @@
---
layout: default
title: Parsing Embedding and Data Extraction
parent: Learn
nav_order: 5
description: overview of the major modules and classes of LLMWare
permalink: /learn/parsing_embedding_data_extraction
---
Parsing, Embedding, and Data Extraction
---
**Tutorial Videos** - check out our Youtube channel for high-impact 5-10 minute tutorials on the latest examples.
Check back often as this list is always growing ...
🎬 **Parsing, Embedding, Data Pipelines and Extraction**
- [Advanced Parsing Techniques](https://youtu.be/dEsw8V_YBYY?si=B0GTVNhwfBYWkXyf)
- [Ingest PDFs at Scale](https://www.youtube.com/watch?v=O0adUfrrxi8&t=10s)
- [Install and Compare Multiple Embeddings with Postgres and PGVector](https://www.youtube.com/watch?v=Bncvggy6m5Q)
- [Intro to Parsing and Text Chunking](https://youtu.be/2xDefZ4oBOM?si=YZzBUjDfQ0839EVF)
- [Voice Transcription and Automated Analysis of Greatest Speeches Dataset](https://youtu.be/5y0ez5ZBpPE?si=PAaCIqYou8nCGxYG)
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/learn/using_agents_functions_slim_models.md
+90
View File
@@ -0,0 +1,90 @@
---
layout: default
title: Using Agents & Function Calls with SLIM Models
parent: Learn
nav_order: 1
description: overview of the major modules and classes of LLMWare
permalink: /learn/using_agents_functions_slim_models
---
Using Agents, Function Calls and SLIM Models
---
**Tutorial Videos** - check out our Youtube channel for high-impact 5-10 minute tutorials on the latest examples.
Check back often as this list is always growing ...
🎬 **Using Agents, Function Calls and SLIM models**
- [Agent Automation with Web Services for Financial Research](https://youtu.be/l0jzsg1_Ik0?si=oBGtALHLplouY9x2)
- [Sentiment Analysis](https://youtu.be/ERCHP21oAN8?si=fp6D4Tk9J2HdDRXa)
- [SLIMS Playlist](https://youtube.com/playlist?list=PL1-dn33KwsmAHWCWK6YjZrzicQ2yR6W8T&si=TSFGqQ3ObOO5vDde)
- [Agent-based Complex Research Analysis](https://youtu.be/y4WvwHqRR60?si=jX3KCrKcYkM95boe)
- [Getting Started with SLIMs (with code)](https://youtu.be/aWZFrTDmMPc?si=lmo98_quo_2Hrq0C)
- [SLIM Models Intro](https://www.youtube.com/watch?v=cQfdaTcmBpY)
- [Text2SQL Intro](https://youtu.be/BKZ6kO2XxNo?si=tXGt63pvrp_rOlIP)
- [Pop up LLMWare Inference Server](https://www.youtube.com/watch?v=qiEmLnSRDUA&t=20s)
- [Hardest Problem in RAG - handling 'Not Found'](https://youtu.be/slDeF7bYuv0?si=j1nkdwdGr5sgvUtK)
- [Extract Information from Earnings Releases](https://youtu.be/d6HFfyDk4YE?si=VmnIiWFmgBtR4DxS)
- [Summary Function Calls](https://youtu.be/yNg_KH5cPSk?si=Yl94tp_vKA8e7eT7)
- [Boolean Yes-No Function Calls](https://youtu.be/jZQZMMqAJXs?si=lU4YVI0H0tfc9k6e)
- [Autogenerate Topics, Tags and NER](https://youtu.be/N6oOxuyDsC4?si=vo2Fd8VG5xTbH4SD)
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
docs/learn/using_quantized_gguf_models.md
+87
View File
@@ -0,0 +1,87 @@
---
layout: default
title: Using Quantized GGUF Models
parent: Learn
nav_order: 3
description: overview of the major modules and classes of LLMWare
permalink: /learn/using_quantized_gguf_models
---
Using Quantized GGUF Models
---
**Tutorial Videos** - check out our Youtube channel for high-impact 5-10 minute tutorials on the latest examples.
Check back often as this list is always growing ...
🎬 **Using GGUF Models**
- [Using LM Studio Models](https://www.youtube.com/watch?v=h2FDjUyvsKE)
- [Using Ollama Models](https://www.youtube.com/watch?v=qITahpVDuV0)
- [Use any GGUF Model](https://www.youtube.com/watch?v=9wXJgld7Yow)
- [Background on GGUF Quantization & DRAGON Model Example](https://www.youtube.com/watch?v=ZJyQIZNJ45E)
- [Getting Started with Whisper.CPP](https://youtu.be/YG5u5AOU9MQ?si=5xQYZCILPSiR8n4s)
- [Best Small RAG Model - Bling-Phi-3](https://youtu.be/cViMonCAeSc?si=L6jX0sRdZAmKtRcz)
- [Agent Automation with Web Services for Financial Research](https://youtu.be/l0jzsg1_Ik0?si=oBGtALHLplouY9x2)
- [Voice Transcription and Automated Analysis of Greatest Speeches Dataset](https://youtu.be/5y0ez5ZBpPE?si=PAaCIqYou8nCGxYG)
- [Are you prompting wrong for RAG - Stochastic Sampling-Part I](https://youtu.be/7oMTGhSKuNY?si=_KSjuBnqArvWzYbx)
- [Are you prompting wrong for RAG - Stochastic Sampling-Part II- Code Experiments](https://youtu.be/iXp1tj-pPjM?si=3ZeMgipY0vJDHIMY)
# More information about the project - [see main repository](https://www.github.com/llmware-ai/llmware.git)
# About the project
`llmware` is &copy; 2023-{{ "now" | date: "%Y" }} by [AI Bloks](https://www.aibloks.com/home).
## Contributing
Please first discuss any change you want to make publicly, for example on GitHub via raising an [issue](https://github.com/llmware-ai/llmware/issues) or starting a [new discussion](https://github.com/llmware-ai/llmware/discussions).
You can also write an email or start a discussion on our Discrod channel.
Read more about becoming a contributor in the [GitHub repo](https://github.com/llmware-ai/llmware/blob/main/CONTRIBUTING.md).
## Code of conduct
We welcome everyone into the ``llmware`` community.
[View our Code of Conduct](https://github.com/llmware-ai/llmware/blob/main/CODE_OF_CONDUCT.md) in our GitHub repository.
## ``llmware`` and [AI Bloks](https://www.aibloks.com/home)
``llmware`` is an open source project from [AI Bloks](https://www.aibloks.com/home) - the company behind ``llmware``.
The company offers a Software as a Service (SaaS) Retrieval Augmented Generation (RAG) service.
[AI Bloks](https://www.aibloks.com/home) was founded by [Namee Oberst](https://www.linkedin.com/in/nameeoberst/) and [Darren Oberst](https://www.linkedin.com/in/darren-oberst-34a4b54/) in October 2022.
## License
`llmware` is distributed by an [Apache-2.0 license](https://www.github.com/llmware-ai/llmware/blob/main/LICENSE).
## Thank you to the contributors of ``llmware``!
<ul class="list-style-none">
{% for contributor in site.github.contributors %}
<li class="d-inline-block mr-1">
<a href="{{ contributor.html_url }}">
<img src="{{ contributor.avatar_url }}" width="32" height="32" alt="{{ contributor.login }}">
</a>
</li>
{% endfor %}
</ul>
---
<ul class="list-style-none">
<li class="d-inline-block mr-1">
<a href="https://discord.gg/MhZn5Nc39h"><span><i class="fa-brands fa-discord"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.youtube.com/@llmware"><span><i class="fa-brands fa-youtube"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://huggingface.co/llmware"><span> <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="Hugging Face" class="hugging-face-logo"/> </span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.linkedin.com/company/aibloks/"><span><i class="fa-brands fa-linkedin"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://twitter.com/AiBloks"><span><i class="fa-brands fa-square-x-twitter"></i></span></a>
</li>
<li class="d-inline-block mr-1">
<a href="https://www.instagram.com/aibloks/"><span><i class="fa-brands fa-instagram"></i></span></a>
</li>
</ul>
---
llmware/__init__.py
+26
View File
@@ -0,0 +1,26 @@
# Copyright 2023-2026 llmware
# Licensed under the Apache License, Version 2.0 (the "License"); you
# may not use this file except in compliance with the License. You
# may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied. See the License for the specific language governing
# permissions and limitations under the License.
"""The llmware package provides tools to build large language model (LLM) applications, this includes
a custom set of LLMs that are small and open source.
The llmware package aspires to be a middleware in LLM applications. In other words, it provides the
infrastructure between the components, such as the models, the prompts, the text databases, and
the vector databases.
"""
__version__ = '0.4.6'
__author__ = 'llmware'
__license__ = 'Apache 2.0 License'
llmware/agents.py
Executable
+1676
View File
File diff suppressed because it is too large Load Diff
llmware/configs.py
+1324
View File
File diff suppressed because it is too large Load Diff
llmware/embeddings.py
+2729
View File
File diff suppressed because it is too large Load Diff
llmware/gguf_configs.py
+1362
View File
File diff suppressed because it is too large Load Diff
llmware/lib/darwin/arm64/llmware/.dylibs/libbson-1.0.0.0.0.dylib
Executable
BIN
View File
Binary file not shown.
llmware/lib/darwin/arm64/llmware/.dylibs/liblzma.5.dylib
BIN
View File
Binary file not shown.
llmware/lib/darwin/arm64/llmware/.dylibs/libmongoc-1.0.0.0.0.dylib
Executable
BIN
View File
Binary file not shown.
llmware/lib/darwin/arm64/llmware/.dylibs/libpng16.16.dylib
Executable
BIN
View File
Binary file not shown.
llmware/lib/darwin/arm64/llmware/.dylibs/libpq.5.dylib
Executable
BIN
View File
Binary file not shown.
llmware/lib/darwin/arm64/llmware/.dylibs/libsnappy.1.1.10.dylib
BIN
View File
Binary file not shown.
llmware/lib/darwin/arm64/llmware/.dylibs/libsqlite3.0.dylib
Executable
BIN
View File
Binary file not shown.
llmware/lib/darwin/arm64/llmware/.dylibs/libxml2.2.dylib
Executable
BIN
View File
Binary file not shown.
llmware/lib/darwin/arm64/llmware/.dylibs/libzip.5.5.dylib
Executable
BIN
View File
Binary file not shown.
llmware/lib/darwin/arm64/llmware/.dylibs/libzstd.1.5.5.dylib
BIN
View File
Binary file not shown.
llmware/lib/darwin/arm64/llmware/libgraph_llmware.so
Executable
BIN
View File
Binary file not shown.
llmware/lib/darwin/arm64/llmware/liboffice_llmware.so
Executable
BIN
View File
Binary file not shown.
llmware/lib/darwin/arm64/llmware/libpdf_llmware.so
Executable
BIN
View File
Binary file not shown.
llmware/lib/gguf/gguf_dgx/libggml-base.so
BIN
View File
Binary file not shown.
llmware/lib/gguf/gguf_dgx/libggml-base.so.0
BIN
View File
Binary file not shown.
llmware/lib/gguf/gguf_dgx/libggml-base.so.0.9.5
BIN
View File
Binary file not shown.
llmware/lib/gguf/gguf_dgx/libggml-cpu.so
BIN
View File
Binary file not shown.
llmware/lib/gguf/gguf_dgx/libggml-cpu.so.0
BIN
View File
Binary file not shown.
llmware/lib/gguf/gguf_dgx/libggml-cpu.so.0.9.5
BIN
View File
Binary file not shown.
llmware/lib/gguf/gguf_dgx/libggml-cuda.so
BIN
View File
Binary file not shown.
llmware/lib/gguf/gguf_dgx/libggml-cuda.so.0
BIN
View File
Binary file not shown.
llmware/lib/gguf/gguf_dgx/libggml-cuda.so.0.9.5
BIN
View File
Binary file not shown.
llmware/lib/gguf/gguf_dgx/libggml.so
BIN
View File
Binary file not shown.
llmware/lib/gguf/gguf_dgx/libggml.so.0
BIN
View File
Binary file not shown.
llmware/lib/gguf/gguf_dgx/libggml.so.0.9.5
BIN
View File
Binary file not shown.
llmware/lib/gguf/gguf_dgx/libllama.so
BIN
View File
Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More