chore: import upstream snapshot with attribution
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Update Platform Components Table / update (push) Has been cancelled
Docker image release / Build base image (push) Has been cancelled
Sync docs with Docusaurus / sync (push) Has been cancelled
Tests / Check if changed (push) Has been cancelled
Tests / format (push) Has been cancelled
Tests / check-imports (push) Has been cancelled
Tests / Unit / macos-latest (push) Has been cancelled
Tests / Unit / ubuntu-latest (push) Has been cancelled
Tests / Unit / windows-latest (push) Has been cancelled
Tests / mypy (push) Has been cancelled
Tests / Integration / ubuntu-latest (push) Has been cancelled
Tests / Integration / macos-latest (push) Has been cancelled
Tests / Integration / windows-latest (push) Has been cancelled
Tests / notify-slack-on-failure (push) Has been cancelled
Tests / Mark tests as completed (push) Has been cancelled
This commit is contained in:
@@ -0,0 +1,43 @@
|
||||
---
|
||||
title: "Enabling GPU Acceleration"
|
||||
id: enabling-gpu-acceleration
|
||||
slug: "/enabling-gpu-acceleration"
|
||||
description: "Speed up your Haystack application by engaging the GPU."
|
||||
---
|
||||
|
||||
import ClickableImage from "@site/src/components/ClickableImage";
|
||||
|
||||
# Enabling GPU Acceleration
|
||||
|
||||
Speed up your Haystack application by engaging the GPU.
|
||||
|
||||
The Transformer models used in Haystack are designed to be run on GPU-accelerated hardware. The steps for GPU acceleration setup depend on the environment that you're working in.
|
||||
|
||||
Once you have GPU enabled on your machine, you can set the `device` on which a given model for a component is loaded.
|
||||
|
||||
For example, to load a model for the `TransformersChatGenerator`, set `device=ComponentDevice.from_single(Device.gpu(id=0))` or `device = ComponentDevice.from_str("cuda:0")` when initializing.
|
||||
|
||||
You can find more information on the [Device management](../concepts/device-management.mdx) page.
|
||||
|
||||
### Enabling the GPU in Linux
|
||||
|
||||
1. Ensure that you have a fitting version of NVIDIA CUDA installed. To learn how to install CUDA, see the [NVIDIA CUDA Guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html).
|
||||
|
||||
2. Run the `nvidia-smi`in the command line to check if the GPU is enabled. If the GPU is enabled, the output shows a list of available GPUs and their memory usage:
|
||||
<ClickableImage src="/img/b44c7f4-gpu_enabled_cropped.png" alt="A screenshot of the command output with the name of the GPU device and its memory usage highlighted." />
|
||||
|
||||
### Enabling the GPU in Colab
|
||||
|
||||
1. In your Colab environment, select **Runtime>Change Runtime type**.
|
||||
<ClickableImage src="/img/85079c7-68747470733a2f2f7261772e67697468756275736572636f6e74656e742e636f6d2f646565707365742d61692f686179737461636b2f6d61696e2f646f63732f696d672f636f6c61625f6770755f72756e74696d652e6a7067.jpeg" alt="Google Colab Runtime menu with Change runtime type option highlighted for selecting GPU acceleration" size="large" />
|
||||
|
||||
2. Choose **Hardware accelerator>GPU**.
|
||||
3. To check if the GPU is enabled, run:
|
||||
|
||||
```python python
|
||||
%%bash
|
||||
|
||||
nvidia-smi
|
||||
```
|
||||
|
||||
The output should show the GPUs available and their usage.
|
||||
Reference in New Issue
Block a user