wehub-resource-sync
51 lines
1.3 KiB
Markdown
51 lines
1.3 KiB
Markdown
# ggml
|
|
|
|
[Manifesto](https://github.com/ggerganov/llama.cpp/discussions/205)
|
|
|
|
Tensor library for machine learning
|
|
|
|
***Note that this project is under active development. \
|
|
Some of the development is currently happening in the [llama.cpp](https://github.com/ggerganov/llama.cpp) and [whisper.cpp](https://github.com/ggerganov/whisper.cpp) repos***
|
|
|
|
## Features
|
|
|
|
- Low-level cross-platform implementation
|
|
- Integer quantization support
|
|
- Broad hardware support
|
|
- Automatic differentiation
|
|
- ADAM and L-BFGS optimizers
|
|
- No third-party dependencies
|
|
- Zero memory allocations during runtime
|
|
|
|
## Build
|
|
|
|
```bash
|
|
git clone https://github.com/ggml-org/ggml
|
|
cd ggml
|
|
|
|
# install python dependencies in a virtual environment
|
|
python3.10 -m venv .venv
|
|
source .venv/bin/activate
|
|
pip install -r requirements.txt
|
|
|
|
# build the examples
|
|
mkdir build && cd build
|
|
cmake ..
|
|
cmake --build . --config Release -j 8
|
|
```
|
|
|
|
## GPT inference (example)
|
|
|
|
```bash
|
|
# run the GPT-2 small 117M model
|
|
../examples/gpt-2/download-ggml-model.sh 117M
|
|
./bin/gpt-2-backend -m models/gpt-2-117M/ggml-model.bin -p "This is an example"
|
|
```
|
|
|
|
For more information, checkout the corresponding programs in the [examples](examples) folder.
|
|
|
|
## Resources
|
|
|
|
- [Introduction to ggml](https://huggingface.co/blog/introduction-to-ggml)
|
|
- [The GGUF file format](https://github.com/ggerganov/ggml/blob/master/docs/gguf.md)
|