wehub/artidoro--qlora
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity

docs: make Chinese README the default

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 10:49:51 +00:00
parent 5ad7034dc7
commit 9c1712f7e8
1 changed files with 85 additions and 85 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+85 -85
View File
@@ -1,73 +1,77 @@
<!-- WEHUB_ZH_README -->
> [!NOTE]
> 本文档由 WeHub 基于上游 README 翻译整理，属于社区翻译，非官方中文文档。
> [English](./README.en.md) · [原始项目](https://github.com/artidoro/qlora) · [上游 README](https://github.com/artidoro/qlora/blob/HEAD/README.md)
> 原作者、版权与许可证归属以原始项目及本仓库 LICENSE 文件为准。
# QLoRA：量化 LLM 的高效微调
| [论文](https://arxiv.org/abs/2305.14314) | [适配器权重](https://huggingface.co/timdettmers) | [演示](https://huggingface.co/spaces/uwnlp/guanaco-playground-tgi) |
本仓库支持论文 "QLoRA: Efficient Finetuning of Quantized LLMs"，旨在让更多人能够参与 LLM 研究。
# QLoRA: Efficient Finetuning of Quantized LLMs
QLoRA 使用 [bitsandbytes](https://github.com/TimDettmers/bitsandbytes) 进行量化，并与 Hugging Face 的 [PEFT](https://github.com/huggingface/peft) 和 [transformers](https://github.com/huggingface/transformers/) 库集成。QLoRA 由 [华盛顿大学 UW NLP 小组](https://twitter.com/uwnlp?s=20). 的成员开发。
| [Paper](https://arxiv.org/abs/2305.14314) | [Adapter Weights](https://huggingface.co/timdettmers) | [Demo](https://huggingface.co/spaces/uwnlp/guanaco-playground-tgi) |
## 更新
- 2023/7/19 - 新增 LLaMA 2 示例脚本并更新版本要求
- 2023/7/18 - 修复添加新 token 时嵌入层未冻结的问题
This repo supports the paper "QLoRA: Efficient Finetuning of Quantized LLMs", an effort to democratize access to LLM research.
## 概述
我们提出了 QLoRA，一种高效的微调（finetuning）方法，可将显存占用降低到足以在单张 48GB GPU 上微调 65B 参数模型，同时保持与完整 16 位微调相当的下游任务性能。QLoRA 将梯度通过冻结的 4 位量化预训练语言模型反向传播到低秩适配器（Low Rank Adapters，LoRA）。我们表现最佳的模型系列命名为 Guanaco，在 Vicuna 基准上超越了此前所有公开发布的模型，达到 ChatGPT 性能水平的 99.3%，而仅需在单张 GPU 上微调 24 小时。QLoRA 引入了多项创新以节省显存且不牺牲性能：(a) 4 位 NormalFloat（NF4），一种对正态分布权重在信息论上最优的新数据类型；(b) 双重量化（Double Quantization），通过对量化常数再次量化来降低平均显存占用；(c) 分页优化器（Paged Optimizers）以管理显存峰值。我们使用 QLoRA 微调了超过 1,000 个模型，对 8 个指令数据集、多种模型类型（LLaMA、T5）以及常规微调难以运行的模型规模（例如 33B 和 65B 参数模型）上的指令遵循与聊天机器人性能进行了详细分析。结果表明，在小型高质量数据集上进行 QLoRA 微调即可达到最先进（state-of-the-art）结果，即使使用的模型规模小于此前的 SoTA。我们基于人工评估与 GPT-4 评估对聊天机器人性能进行了详细分析，表明 GPT-4 评估是人工评估的一种廉价且合理的替代方案。此外，我们发现当前的聊天机器人基准不足以可靠地准确评估聊天机器人的性能水平。我们发布了全部模型与代码，包括用于 4 位训练的 CUDA 内核。
## 许可证与预期用途
我们在 MIT 许可证下发布本仓库中与 QLoRA 微调相关的资源。
此外，我们发布 Guanaco 模型系列，涵盖基础 LLaMA 模型的 7B、13B、33B 和 65B 规模。这些模型的用途须符合 LLaMA 许可证，并需要获得 LLaMA 模型的访问权限。
## 演示
Guanaco 是纯研究用途的系统，可能产生有问题的输出。
1. 在此访问[在线演示](https://huggingface.co/spaces/uwnlp/guanaco-playground-tgi). 请注意这是 33B 模型，65B 模型演示将稍后推出。
2. 或者使用[此 notebook](https://colab.research.google.com/drive/17XEqL1JcmVWjHkT-WczdYkJlNINacwG7?usp=sharing). 在 Colab 中直接托管你自己的 Guanaco gradio 演示。免费 GPU 可运行 7B 和 13B 模型。
3. 另外，你能区分 ChatGPT 与 Guanaco 吗？来试试吧！
你可以在[此模型回复 Colab](https://colab.research.google.com/drive/1kK6xasHiav9nhiRUJjPMZb4fAED4qRHb?usp=sharing) 上对比 ChatGPT 与 Guanaco 65B 在 Vicuna 提示词上的回答。
QLoRA uses [bitsandbytes](https://github.com/TimDettmers/bitsandbytes) for quantization and is integrated with Hugging Face's [PEFT](https://github.com/huggingface/peft) and [transformers](https://github.com/huggingface/transformers/) libraries. QLoRA was developed by members of the [University of Washington's UW NLP group](https://twitter.com/uwnlp?s=20).
## Updates
- 7/19/2023 - Added LLaMA 2 example script and updated version requirements
- 7/18/2023 - Fixed non-frozen embeddings when adding new tokens
## Overview
We present QLoRA, an efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance. QLoRA backpropagates gradients through a frozen, 4-bit quantized pretrained language model into Low Rank Adapters (LoRA). Our best model family, which we name Guanaco, outperforms all previous openly released models on the Vicuna benchmark, reaching 99.3% of the performance level of ChatGPT while only requiring 24 hours of finetuning on a single GPU. QLoRA introduces a number of innovations to save memory without sacrificing performance: (a) 4-bit NormalFloat (NF4), a new data type that is information theoretically optimal for normally distributed weights (b) Double Quantization to reduce the average memory footprint by quantizing the quantization constants, and (c) Paged Optimizers to manage memory spikes. We use QLoRA to finetune more than 1,000 models, providing a detailed analysis of instruction following and chatbot performance across 8 instruction datasets, multiple model types (LLaMA, T5), and model scales that would be infeasible to run with regular finetuning (e.g. 33B and 65B parameter models). Our results show that QLoRA finetuning on a small high-quality dataset leads to state-of-the-art results, even when using smaller models than the previous SoTA. We provide a detailed analysis of chatbot performance based on both human and GPT-4 evaluations showing that GPT-4 evaluations are a cheap and reasonable alternative to human evaluation. Furthermore, we find that current chatbot benchmarks are not trustworthy to accurately evaluate the performance levels of chatbots. We release all of our models and code, including CUDA kernels for 4-bit training.
## License and Intended Use
We release the resources associated with QLoRA finetuning in this repository under MIT license.
In addition, we release the Guanaco model family for base LLaMA model sizes of 7B, 13B, 33B, and 65B. These models are intended for purposes in line with the LLaMA license and require access to the LLaMA models.
## Demo
Guanaco is a system purely intended for research purposes and could produce problematic outputs.
1. Access the [live demo here](https://huggingface.co/spaces/uwnlp/guanaco-playground-tgi). Note this is the 33B model, the 65B model demo will come later.
2. Or host your own Guanaco gradio demo directly in Colab with [this notebook](https://colab.research.google.com/drive/17XEqL1JcmVWjHkT-WczdYkJlNINacwG7?usp=sharing). Works with free GPUs for 7B and 13B models.
3. Alternatively, can you distinguish ChatGPT from Guanaco? Give it a try!
You can access [the model response Colab here](https://colab.research.google.com/drive/1kK6xasHiav9nhiRUJjPMZb4fAED4qRHb?usp=sharing) comparing ChatGPT and Guanaco 65B on Vicuna prompts.
## Installation
To load models in 4bits with transformers and bitsandbytes, you have to install accelerate and transformers from source and make sure you have the latest version of the bitsandbytes library. After installing PyTorch (follow instructions [here](https://pytorch.org/get-started/locally/)), you can achieve the above with the following command:
## 安装
要使用 transformers 和 bitsandbytes 以 4 位加载模型，你需要从源码安装 accelerate 和 transformers，并确保 bitsandbytes 库为最新版本。安装 PyTorch 后（请按[此处](https://pytorch.org/get-started/locally/)), 说明操作），可通过以下命令完成上述步骤：
```bash
pip install -U -r requirements.txt
```
## Getting Started
The `qlora.py` code is a starting point for finetuning and inference on various datasets.
Basic command for finetuning a baseline model on the Alpaca dataset:
## 快速开始
`qlora.py` 代码是在多种数据集上进行微调与推理的起点。
在 Alpaca 数据集上微调基线模型的基本命令：
```bash
python qlora.py --model_name_or_path <path_or_name>
```
For models larger than 13B, we recommend adjusting the learning rate:
对于大于 13B 的模型，我们建议调整学习率：
```bash
python qlora.py –learning_rate 0.0001 --model_name_or_path <path_or_name>
```
To replicate our Guanaco models see below.
要复现我们的 Guanaco 模型，请见下文。
### Tutorials and Demonstrations
Here is [a blog](https://huggingface.co/blog/4bit-transformers-bitsandbytes) discussing 4-bit quantization, QLoRA, and how they are integrated in transformers.
### 教程与演示
这里有[一篇博客](https://huggingface.co/blog/4bit-transformers-bitsandbytes) 讨论 4 位量化、QLoRA 以及它们如何集成到 transformers 中。
You can host your own gradio Guanaco demo directly in Colab following [this notebook](https://colab.research.google.com/drive/17XEqL1JcmVWjHkT-WczdYkJlNINacwG7?usp=sharing).
In addition, here are Colab notebooks with examples for inference and finetuning using QLoRA:
- [Inference notebook](https://colab.research.google.com/drive/1ge2F1QSK8Q7h0hn3YKuBCOAS0bK8E0wf?usp=sharing)
- [Finetuning notebook](https://colab.research.google.com/drive/1VoYNfYDKcKRQRor98Zbf2-9VQTtGJ24k?usp=sharing)
你可以按照[此 notebook](https://colab.research.google.com/drive/17XEqL1JcmVWjHkT-WczdYkJlNINacwG7?usp=sharing). 在 Colab 中直接托管你自己的 gradio Guanaco 演示。
此外，以下是使用 QLoRA 进行推理与微调的 Colab notebook 示例：
- [推理 notebook](https://colab.research.google.com/drive/1ge2F1QSK8Q7h0hn3YKuBCOAS0bK8E0wf?usp=sharing)
- [微调 notebook](https://colab.research.google.com/drive/1VoYNfYDKcKRQRor98Zbf2-9VQTtGJ24k?usp=sharing)
Other examples are found under the `examples/` folder. We include a generation getting started example with guanaco at `examples/guanaco_generate.py`.
其他示例见 `examples/` 文件夹。我们在 `examples/guanaco_generate.py` 提供了 guanaco 生成入门示例。
### Quantization
Quantization parameters are controlled from the `BitsandbytesConfig` ([see HF documenation](https://huggingface.co/docs/transformers/main_classes/quantization#transformers.BitsAndBytesConfig)) as follows:
- Loading in 4 bits is activated through `load_in_4bit`
- The datatype used for the linear layer computations with `bnb_4bit_compute_dtype`
- Nested quantization is activated through `bnb_4bit_use_double_quant`
- The datatype used for qunatization is specified with `bnb_4bit_quant_type`. Note that there are two supported quantization datatypes `fp4` (four bit float) and `nf4` (normal four bit float). The latter is theoretically optimal for normally distributed weights and we recommend using `nf4`.
### 量化
量化参数通过 `BitsandbytesConfig` 控制（[参见 HF 文档](https://huggingface.co/docs/transformers/main_classes/quantization#transformers.BitsAndBytesConfig))），如下：
- 通过 `load_in_4bit` 启用 4 位加载
- 通过 `bnb_4bit_compute_dtype` 指定线性层计算所用的数据类型
- 通过 `bnb_4bit_use_double_quant` 启用嵌套量化
- 通过 `bnb_4bit_quant_type` 指定量化所用的数据类型。注意支持两种量化数据类型：`fp4`（四位浮点）和 `nf4`（正态四位浮点）。后者对正态分布权重在理论上最优，我们建议使用 `nf4`。
```python
model = AutoModelForCausalLM.from_pretrained(
@@ -85,64 +89,60 @@ Quantization parameters are controlled from the `BitsandbytesConfig` ([see HF do
)
```
### Paged Optimizer
You can access the paged optimizer with the argument `--optim paged_adamw_32bit`
### 分页优化器
你可以通过参数 `--optim paged_adamw_32bit` 使用分页优化器。
### Guanaco Finetuning
You can select `--dataset oasst1` to load the OpenAssistant dataset that was used to train Guanaco. You can also find it on HF at [timdettmers/openassistant-guanaco](https://huggingface.co/datasets/timdettmers/openassistant-guanaco).
### Guanaco 微调
你可以选择 `--dataset oasst1` 来加载用于训练 Guanaco 的 OpenAssistant 数据集。你也可以在 HF 上找到：[timdettmers/openassistant-guanaco](https://huggingface.co/datasets/timdettmers/openassistant-guanaco).
We include scripts to reproduce the hyperparameters of Guanaco model training for various sizes at `./scripts/finetune_guanaco*.sh`. Make sure to adjust `per_device_train_batch_size` and `gradient_accumulation_steps` so that their product is 16 and training fits on your GPUs.
我们在 `./scripts/finetune_guanaco*.sh` 提供了复现各规模 Guanaco 模型训练超参数的脚本。请确保调整 `per_device_train_batch_size` 和 `gradient_accumulation_steps`，使二者乘积为 16，且训练能在你的 GPU 上运行。
### Using Local Datasets
### 使用本地数据集
You can specify the path to your dataset using the `--dataset` argument. If the `--dataset_format` argument is not set, it will default to the Alpaca format. Here are a few examples:
你可以使用 `--dataset` 参数指定数据集路径。如果未设置 `--dataset_format` 参数，将默认使用 Alpaca 格式。以下是几个示例：
- Training with an *alpaca* format dataset:
- 使用 *alpaca* 格式数据集训练：
```bash
python qlora.py --dataset="path/to/your/dataset"
```
- Training with a *self-instruct* format dataset:
- 使用 *self-instruct* 格式数据集训练：
```bash
python qlora.py --dataset="path/to/your/dataset" --dataset_format="self-instruct"
```
### Multi GPU
Multi GPU training and inference work out-of-the-box with Hugging Face's Accelerate. Note that the `per_device_train_batch_size` and `per_device_eval_batch_size` arguments are global batch sizes unlike what their name suggest.
### 多 GPU
多 GPU 训练与推理可通过 Hugging Face 的 Accelerate 开箱即用。请注意，`per_device_train_batch_size` 和 `per_device_eval_batch_size` 参数是全局批次大小，与其名称所暗示的含义不同。
When loading a model for training or inference on multiple GPUs you should pass something like the following to `AutoModelForCausalLM.from_pretrained()`:
在多个 GPU 上加载模型进行训练或推理时，应向 `AutoModelForCausalLM.from_pretrained()` 传入类似以下的配置：
```python
device_map = "auto"
max_memory = {i: '46000MB' for i in range(torch.cuda.device_count())}
```
## 示例输出
我们在 `eval/generations` 文件夹中提供了论文所述模型针对 OA 和 Vicuna 查询的生成结果。这些旨在促进对模型评估和分析的进一步研究。
## Sample Outputs
We provide generations for the models described in the paper for both OA and Vicuna queries in the `eval/generations` folder. These are intended to foster further research on model evaluation and analysis.
你能区分 ChatGPT 和 Guanaco 吗？来试试吧！
你可以在此处访问[模型响应 Colab](https://colab.research.google.com/drive/1kK6xasHiav9nhiRUJjPMZb4fAED4qRHb?usp=sharing) comparing ChatGPT and Guanaco 65B on Vicuna prompts.
Can you distinguish ChatGPT from Guanaco? Give it a try!
You can access [the model response Colab here](https://colab.research.google.com/drive/1kK6xasHiav9nhiRUJjPMZb4fAED4qRHb?usp=sharing) comparing ChatGPT and Guanaco 65B on Vicuna prompts.
## 评估
我们收录了改编自 FastChat 仓库的脚本，可使用 GPT-4 自动评估模型生成结果。其中包括相对于 ChatGPT 的对比脚本（十分制评分），以及采用三类标签（win、loose 或 tie）的「成对比较」（pairwise comparisons）。这些脚本位于 `eval` 文件夹。
## Evaluation
We include scripts adapted from the FastChat repo to automatically evaluate model generations using GPT-4. We include script for comparisons relative to ChatGPT with scores out of 10 as well as "pairwise comparisons" with three class labeling (win, loose, or tie). These are found in the `eval` folder.
为促进我们评估工作的复现以及该领域的后续研究，我们发布了各系统的 GPT-4 评分与人工评分。这些数据位于 `eval/ratings-human` 和 `eval/ratings-gpt4`。
To facilitate the replication of our evaluation and future work in this area, we release GPT-4 and human ratings of our systems. These are found under `eval/ratings-human` and `eval/ratings-gpt4`.
更多细节请参阅 `eval/EVAL_README.md`。
More details can be found at `eval/EVAL_README.md`.
## 已知问题与限制
以下是已知问题与缺陷列表。若你遇到的问题未在此列出，请新建 issue 并描述具体情况。
## Known Issues and Limitations
Here a list of known issues and bugs. If your issue is not reported here, please open a new issue and describe the problem.
1. 4-bit 推理速度较慢。目前，我们的 4-bit 推理实现尚未与 4-bit 矩阵乘法集成。
2. 使用 Trainer 恢复 LoRA 训练运行目前不受 HF（Hugging Face）支持。
3. 目前，使用 `bnb_4bit_compute_type='fp16'` 可能导致不稳定。对于 7B LLaMA，仅有 80% 的微调运行能无错误完成。我们已有解决方案，但尚未集成到 bitsandbytes 中。
4. 请确保使用 `tokenizer.bos_token_id = 1` 以避免生成问题。
5. 如果你遇到此 [issue](https://github.com/artidoro/qlora/issues/82) ("illegal memory access")，则应使用较新的 HF LLaMA 转换版本或降级 PyTorch 版本。
6. 添加新 token 的问题详见 #214。若添加新 token，需要更新 embedding 并存储/重新加载。
1. 4-bit inference is slow. Currently, our 4-bit inference implementation is not yet integrated with the 4-bit matrix multiplication
2. Resuming a LoRA training run with the Trainer currently not supported by HF.
3. Currently, using `bnb_4bit_compute_type='fp16'` can lead to instabilities. For 7B LLaMA, only 80% of finetuning runs complete without error. We have solutions, but they are not integrated yet into bitsandbytes.
4. Make sure that `tokenizer.bos_token_id = 1` to avoid generation issues.
5. If you get an this [issue](https://github.com/artidoro/qlora/issues/82) ("illegal memory access") then you should use a newer HF LLaMA conversion or downgrade your PyTorch version.
6. Problems with adding new tokens outlined in #214. Embeddings need to be updated and stored/reloaded if you are adding new tokens.
## Citation
## 引用
```bibtex
@article{dettmers2023qlora,
@@ -153,8 +153,8 @@ Here a list of known issues and bugs. If your issue is not reported here, please
}
```
## Acknowledgements
We thank the Hugging Face team, in particular Younes Belkada, for their support integrating QLoRA with PEFT and transformers libraries.
We also thank Meta for releasing the LLaMA models without which this work would not have been possible.
## 致谢
我们感谢 Hugging Face 团队，尤其是 Younes Belkada，在将 QLoRA 集成到 PEFT 和 transformers 库方面给予的支持。
我们也感谢 Meta 发布 LLaMA 模型，没有这些模型，本工作将无法实现。
This repo builds on the [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca) and [LMSYS FastChat](https://github.com/lm-sys/FastChat) repos.
本仓库基于 [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca) 和 [LMSYS FastChat](https://github.com/lm-sys/FastChat) 仓库构建。