> [!NOTE] > 本文档由 WeHub 基于上游 README 翻译整理，属于社区翻译，非官方中文文档。 > [English](./README.en.md) · [原始项目](https://github.com/Stability-AI/StableLM) · [上游 README](https://github.com/Stability-AI/StableLM/blob/HEAD/README.md) > 原作者、版权与许可证归属以原始项目及本仓库 LICENSE 文件为准。 # StableLM：Stability AI 语言模型 ![Stochastic Parrot](./assets/mascot.png)
*“A Stochastic Parrot，扁平化设计，矢量艺术” — [Stable Diffusion XL](https://clipdrop.co/stable-diffusion)* 本仓库包含 Stability AI 对 StableLM 系列语言模型的持续开发工作，并将持续更新新的 checkpoint。以下内容概述了当前所有可用模型。更多模型即将推出。 ## 新闻 *2023 年 9 月 29 日* - 在 [CC BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/). 许可下发布 StableLM-3B-4E1T 模型 *2023 年 8 月 5 日* - 发布修复后的 StableLM-Alpha v2 模型，参数量为 3B 和 7B。 *2023 年 4 月 28 日* - 发布 StableVicuna-13B，这是我们基于 [Vicuna-13B v0](https://huggingface.co/lmsys/vicuna-13b-delta-v0), 的 RLHF（Reinforcement Learning from Human Feedback，基于人类反馈的强化学习）微调模型；Vicuna-13B v0 本身是对 [LLaMA-13B](https://github.com/facebookresearch/llama). 的微调。相对于原始 Llama 模型的 Delta 权重在 ([CC BY-NC-SA-4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/)). 许可下发布 *2023 年 4 月 20 日* - 发布首批 StableLM-Alpha 模型，参数量为 3B 和 7B。基础模型在 [CC BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/). 许可下发布 - 可在 [Hugging Face Spaces](https://huggingface.co/spaces/stabilityai/stablelm-tuned-alpha-chat). 上与我们 7B 模型 `StableLM-Tuned-Alpha-7B` 进行对话 ## 模型 ### StableLM-3B-4E1T > 技术报告：[StableLM-3B-4E1T](https://stability.wandb.io/stability-llm/stable-lm/reports/StableLM-3B-4E1T--VmlldzoyMjU4?accessToken=u3zujipenkx5g7rtcj9qojjgxpconyjktjkli2po09nffrffdhhchq045vp0wyfo) StableLM-3B-4E1T 是一个 30 亿（3B）参数的语言模型，在多 epoch 训练机制下预训练，用于研究重复 token 对下游任务性能的影响。鉴于该领域先前的成功经验（[Tay et al., 2023](https://arxiv.org/pdf/2205.05131.pdf) 和 [Taylor et al., 2022](https://galactica.org/static/paper.pdf)),），我们遵循 [Muennighoff et al. (2023)](https://arxiv.org/abs/2305.16264) 在 "Scaling Data-Constrained Language Models" 中的观察，在 1 万亿（1T）token 上训练 4 个 epoch；他们发现"使用最多 4 个 epoch 的重复数据进行训练，与使用唯一数据相比，对 loss 的影响可忽略不计"。token 数量的进一步灵感来自 "Go smol or go home"（[De Vries, 2023](https://www.harmdevries.com/post/model-size-vs-compute-overhead/)),），该研究指出，一个在 2.85 万亿 token 上训练的 2.96B 模型可达到与 Chinchilla 计算最优的 9.87B 语言模型（$k_n = 0.3$）相近的 loss。 | 规模 | StableLM-3B-4E1T | 训练 Token 数 | 参数量 | |------|--------------------------------------------------------------------|-----------------|---------------| | 3B | [checkpoint](https://huggingface.co/stabilityai/stablelm-3b-4e1t) | 4T | 2,795,443,200 | #### 模型架构 该模型是一种仅解码器（decoder-only）transformer，与 LLaMA（[Touvron et al., 2023](https://arxiv.org/abs/2307.09288)) 架构类似，并包含以下修改： | 参数量 | 隐藏层维度 | 层数 | 注意力头数 | 序列长度 | |----------------|-------------|--------|-------|-----------------| | 2,795,443,200 | 2560 | 32 | 32 | 4096 | - **位置嵌入（Position Embeddings）**：Rotary Position Embeddings（旋转位置嵌入，[Su et al., 2021](https://arxiv.org/abs/2104.09864))），应用于前 25% 的 head 嵌入维度以提升吞吐，做法遵循 [Black et al. (2022)](https://arxiv.org/pdf/2204.06745.pdf). - **归一化（Normalization）**：LayerNorm（[Ba et al., 2016](https://arxiv.org/abs/1607.06450))），采用可学习的 bias 项，而非 RMSNorm（[Zhang & Sennrich, 2019](https://arxiv.org/abs/1910.07467)). - **分词器（Tokenizer）**：GPT-NeoX（[Black et al., 2022](https://arxiv.org/abs/2204.06745)). #### 训练数据 该数据集由在 [HuggingFace Hub](https://huggingface.co/datasets): 上可用的开源大规模数据集过滤混合而成：Falcon RefinedWeb 提取数据（[Penedo et al., 2023](https://huggingface.co/datasets/tiiuae/falcon-refinedweb)),）、RedPajama-Data（[Together Computer., 2023](https://github.com/togethercomputer/RedPajama-Data))）和 The Pile（[Gao et al., 2020](https://arxiv.org/abs/2101.00027))）（均不含 *Books3* 及其他子集），以及 StarCoder（[Li et al., 2023](https://arxiv.org/abs/2305.06161)). > 鉴于训练数据中包含大量网页数据，我们建议针对下游任务对 StableLM-3B-4E1T 基础模型进行微调。 #### 训练细节 完整超参数设置请参阅提供的 YAML 配置文件 [`stablelm-3b-4e1t.yml`](./configs/stablelm-3b-4e1t.yml)，更多细节请参阅[技术报告](https://stability.wandb.io/stability-llm/stable-lm/reports/StableLM-3B-4E1T--VmlldzoyMjU4?accessToken=u3zujipenkx5g7rtcj9qojjgxpconyjktjkli2po09nffrffdhhchq045vp0wyfo)。 #### 下游任务结果 以下零样本（zero-shot）评估使用 `lm-evaluation-harness` 完成，基于 Stability AI fork 的 [lm-bench](https://github.com/Stability-AI/lm-evaluation-harness/tree/lm-bench) 分支。完整的 `lm-eval` JSON 文件可在 [`evals`](./evals) 目录中找到。 | 预训练模型 | 平均分 | ARC
Challenge | ARC
Easy | BoolQ | HellaSwag (✱) | LAMBADA
OpenAI | OpenBookQA | PIQA | SciQ | Winogrande | | ------------------------------------------------------------------------------------- |:-----------------:|:----------------:|:-----------:|:-----:|:-------------:|:-----------------:|:----------:|:-----:|:-----:|:----------:| | meta-llama/Llama-2-13b-hf | 71.77 | 48.63 | 79.50 | 80.52 | 79.36 | 76.77 | 35.40 | 79.05 | 94.50 | 72.22 | | huggyllama/llama-7b | 68.84 | 41.89 | 75.25 | 75.05 | 76.22 | 73.55 | 34.40 | 78.67 | 94.60 | 69.93 | | meta-llama/Llama-2-7b-hf | 68.75 | 43.00 | 76.26 | 77.74 | 75.94 | 73.47 | 31.40 | 77.75 | 93.60 | 69.61 | | Qwen/Qwen-7B | 67.91 | 45.39 | 67.38 | 74.56 | 88.85 (?) | 69.67 | 32.20 | 73.99 | 93.20 | 65.98 | | tiiuae/falcon-7b | 67.83 | 40.27 | 74.41 | 73.55 | 76.35 | 74.56 | 30.60 | 79.49 | 94.00 | 67.25 | | mosaicml/mpt-7b | 67.36 | 40.53 | 74.92 | 73.94 | 76.17 | 68.64 | 31.40 | 78.89 | 93.70 | 68.03 | | **stabilityai/stablelm-3b-4e1t** | 66.93 | 37.80 | 72.47 | 75.63 | 73.90 | 70.64 | 31.40 | 79.22 | 94.80 | 66.54 | | baichuan-inc/Baichuan2-7B-Base | 66.93 | 42.24 | 75.00 | 73.09 | 72.29 | 70.99 | 30.40 | 76.17 | 94.60 | 67.56 | | stabilityai/stablelm-base-alpha-7b-v2 | 66.89 | 38.48 | 73.19 | 70.31 | 74.27 | 74.19 | 30.40 | 78.45 | 93.90 | 68.82 | | openlm-research/open_llama_7b_v2 | 66.32 | 38.82 | 71.93 | 71.41 | 74.65 | 71.05 | 30.20 | 79.16 | 93.80 | 65.82 | | microsoft/phi-1_5 | 65.57 | 44.45 | 76.14 | 74.53 | 62.62 | 52.75 | 37.60 | 76.33 | 93.20 | 72.53 | | EleutherAI/gpt-neox-20B | 65.57 | 37.88 | 72.90 | 69.48 | 71.43 | 71.98 | 29.80 | 77.42 | 93.10 | 66.14 | | togethercomputer/RedPajama-INCITE-7B-Base | 65.07 | 37.71 | 72.35 | 70.76 | 70.33 | 71.34 | 29.00 | 77.15 | 92.70 | 64.33 | | cerebras/btlm-3b-8k-base (§) | 63.59 | 34.90 | 70.45 | 69.63 | 69.78 | 66.23 | 27.60 | 75.84 | 92.90 | 64.96 | | EleutherAI/pythia-12b | 62.69 | 31.83 | 70.20 | 67.31 | 67.38 | 70.64 | 26.40 | 76.28 | 90.20 | 64.01 | | openlm-research/open_llama_3b_v2 | 62.43 | 33.87 | 67.59 | 65.69 | 69.99 | 66.74 | 26.00 | 76.66 | 92.40 | 62.90 | | EleutherAI/gpt-j-6B | 62.34 | 33.96 | 66.96 | 65.44 | 66.24 | 68.23 | 29.00 | 75.57 | 91.50 | 64.17 | | stabilityai/stablelm-base-alpha-3b-v2 | 62.19 | 32.42 | 67.26 | 64.56 | 68.58 | 70.25 | 26.40 | 76.01 | 92.10 | 62.12 | | facebook/opt-6.7b | 61.85 | 30.72 | 65.66 | 66.02 | 67.20 | 67.65 | 27.60 | 76.33 | 90.10 | 65.35 | | EleutherAI/pythia-6.9b | 60.58 | 31.83 | 67.21 | 64.01 | 63.88 | 67.01 | 25.80 | 75.08 | 89.80 | 60.62 | | EleutherAI/pythia-2.8b-deduped | 58.52 | 30.12 | 63.47 | 64.13 | 59.44 | 65.15 | 23.80 | 74.10 | 88.20 | 58.25 | | **§** 此前 3B 预训练 SOTA
**?** Outlier Reuslts
**\*** 按字节长度归一化的准确率 | | | | | | | | | | | **StableLM-3B-4E1T 在 2023 年 9 月于 30 亿（3B）参数规模的开源模型中实现了最先进（state-of-the-art）性能**，并可与许多热门的当代 70 亿（7B）参数模型竞争，甚至超越了我们最新的 7B StableLM-Base-Alpha-v2。 ### StableLM-Alpha v2 StableLM-Alpha v2 模型通过融入 SwiGLU（[Shazeer, 2020](https://arxiv.org/abs/2002.05202)) 等架构改进，并采用如下所述的更高质量数据源，在初始 Alpha 模型基础上显著改进。这些模型的上下文长度为 4096 tokens。 | Size | StableLM-Base-Alpha-v2 | Training Tokens | Parameters | |------|----------------------------------------------------------------------------|-----------------|---------------| | 3B | [checkpoint](https://huggingface.co/stabilityai/stablelm-base-alpha-3b-v2) | 1.1T | 2,796,431,360 | | 7B | [checkpoint](https://huggingface.co/stabilityai/stablelm-base-alpha-7b-v2) | 1.1T | 6,890,209,280 | #### Training Details 有关超参数的详细信息，请参阅提供的 YAML 配置文件。例如，对于扩展版 `StableLM-Alpha-3B-v2` 模型，请参见 [stablelm-base-alpha-3b-v2-4k-extension.yml](./configs/stablelm-base-alpha-3b-v2-4k-extension.yml)。 借鉴类似工作，我们采用多阶段方法进行上下文长度扩展（[Nijkamp et al., 2023](https://blog.salesforceairesearch.com/xgen/)),：先在上下文长度 2048 下调度 1 万亿 tokens，再在 4096 下调度 1000 亿 tokens。我们发现序列长度预热（sequence length warmup）（[Li et al., 2022](https://arxiv.org/abs/2108.06084)) 有助于稳定预训练前约 800 亿 tokens 期间的早期尖峰。然而，由于随着课程推进长度形状增长会带来显著的吞吐量损失，最终运行并未采用该方法。 #### Training Data 对 StableLM-Alpha-v2 下游性能影响最大的变化在于使用了更高质量的数据源与混合配比；具体而言，以 [RefinedWeb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb) 和 [C4](https://huggingface.co/datasets/allenai/c4) 替代 The Pile v2 Common-Crawl 抓取数据，并将网页文本采样率大幅提高（35% -> 71%）。 第一阶段预训练依赖 1 万亿 tokens，来源于公开 Falcon RefinedWeb 提取数据（[Penedo et al., 2023](https://huggingface.co/datasets/tiiuae/falcon-refinedweb)),、RedPajama-Data（[Together Computer., 2023](https://github.com/togethercomputer/RedPajama-Data)),、The Pile（[Gao et al., 2020](https://arxiv.org/abs/2101.00027)), 以及内部数据集的混合，网页文本采样率为 71%。 在第二阶段，我们纳入 StarCoder（[Li et al., 2023](https://arxiv.org/abs/2305.06161)) 数据集，并将网页文本下采样至 55%，同时提高上述来源中天然长文本样本的采样比例。 #### Evaluation 以下零样本（zero-shot）评测使用 `lm-evaluation-harness` 在提交 [`df3da98c5405deafd519c2ddca52bb7c3fe36bef`](https://github.com/EleutherAI/lm-evaluation-harness/tree/df3da98c5405deafd519c2ddca52bb7c3fe36bef) 上完成；SIQA 除外，它使用 [`add-siqa` branch](https://github.com/EleutherAI/lm-evaluation-harness/tree/add-siqa)，提示格式为 `{doc['context']}\nQuestion: {doc['question']}\nAnswer:`。 | Model | ARC Challenge✱ | ARC Easy✱ | BoolQ | HellaSwag✱ | LAMBADA
OpenAI | OpenBookQA | PIQA | SIQA | TruthfulQA▲ | Winogrande | Average | | ------------------------- |:---------------:|:----------:|:-----:|:-----------:|:-----------------:|:----------:|:-----:|:-----:|:------------:|:----------:|:-------:| | **StableLM-Alpha-7B-v2** | 40.53 | 69.11 | 70.31 | 74.27 | 74.19 | 30.40 | 78.45 | 42.43 | 36.46 | 68.82 | 58.50 | | LLaMA-2-7B | 46.16 | 74.54 | 77.74 | 75.94 | 73.47 | 31.40 | 77.75 | 43.50 | 38.97 | 69.61 | 60.91 | | MPT-7B | 41.89 | 70.03 | 73.94 | 76.17 | 68.64 | 31.40 | 78.89 | 45.14 | 33.49 | 68.03 | 58.76 | | OpenLLaMA-7B-v2 | 42.41 | 69.65 | 71.41 | 74.65 | 71.05 | 30.20 | 79.16 | 41.97 | 34.57 | 65.82 | 58.09 | | RedPajama-INCITE-7B-Base | 39.42 | 69.19 | 70.76 | 70.33 | 71.34 | 29.00 | 77.15 | 42.58 | 33.01 | 64.33 | 56.71 | | **StableLM-Alpha-3B-v2** | 35.07 | 63.26 | 64.56 | 68.58 | 70.25 | 26.40 | 76.01 | 42.48 | 35.87 | 62.12 | 54.46 | | BTLM-3B-8K | 37.63 | 67.09 | 69.63 | 69.78 | 66.23 | 27.60 | 75.84 | 42.78 | 36.00 | 64.96 | 55.75 | | OpenLLaMA-3B-v2 | 36.09 | 63.51 | 65.69 | 69.99 | 66.74 | 26.00 | 76.66 | 41.20 | 34.59 | 62.90 | 54.34 | | Pythia-2.8B (deduped) | 32.94 | 59.09 | 64.13 | 59.44 | 65.15 | 23.80 | 74.10 | 40.94 | 35.56 | 58.25 | 51.34 | | StableLM-Alpha-7B | 27.05 | 44.87 | 60.06 | 41.22 | 55.11 | 21.40 | 66.76 | 39.46 | 39.96 | 50.12 | 44.60 | | StableLM-Alpha-3B | 25.77 | 42.05 | 57.65 | 38.31 | 41.72 | 17.00 | 63.82 | 35.62 | 40.53 | 52.64 | 41.51 | ✱：表示按字节长度归一化的准确率（`acc_norm`），详见 [Gao, 2021](https://blog.eleuther.ai/multiple-choice-normalization/). ▲：我们对 TruthfulQA 的评分采用分配给真实答案集合的归一化总概率（`mc2`）。 ### StableLM-Alpha StableLM-Alpha 模型基于 [The Pile](https://pile.eleuther.ai/), 构建的新数据集进行训练，该数据集包含 1.5 万亿 tokens，规模约为 The Pile 的 3 倍。这些模型的上下文长度为 4096 tokens。 作为概念验证，我们还使用 [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca)'s 流程，结合五个用于对话智能体的近期数据集对模型进行了微调：Stanford 的 [Alpaca](https://github.com/tatsu-lab/stanford_alpaca),、Nomic-AI 的 [gpt4all](https://github.com/nomic-ai/gpt4all),、RyokoAI 的 [ShareGPT52K](https://huggingface.co/datasets/RyokoAI/ShareGPT52K) 数据集、Databricks labs 的 [Dolly](https://github.com/databrickslabs/dolly), 以及 Anthropic 的 [HH](https://github.com/anthropics/hh-rlhf). 我们将以 StableLM-Tuned-Alpha 的名义发布这些模型。 | Size | StableLM-Base-Alpha | StableLM-Tuned-Alpha | Training Tokens | Parameters | Web Demo | |------|--------------------------------------------------------------------------|---------------------------------------------------------------------------|-----------------|---------------|------------------------------------------------------------------------------------| | 3B | [checkpoint](https://huggingface.co/stabilityai/stablelm-base-alpha-3b/) | [checkpoint](https://huggingface.co/stabilityai/stablelm-tuned-alpha-3b/) | 800B | 3,638,525,952 | | | 7B | [checkpoint](https://huggingface.co/stabilityai/stablelm-base-alpha-7b) | [checkpoint](https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b) | 800B | 7,869,358,080 | [Hugging Face](https://huggingface.co/spaces/stabilityai/stablelm-tuned-alpha-chat) | ### StableVicuna StableVicuna 是对 [Vicuna-13B v0](https://huggingface.co/lmsys/vicuna-13b-delta-v0), 的 RLHF 微调，而 Vicuna-13B v0 本身又是对 [LLaMA-13B](https://github.com/facebookresearch/llama). 的微调。这是我们打造开源 RLHF LLM 聊天机器人的尝试。该模型由 StabilityAI 的 CarperAI 团队开发，训练工作由 [Duy V. Phung](https://github.com/PhungVanDuy) 牵头。 由于 LLaMA 原始的非商业许可证，我们仅能将模型权重以相对于原始模型权重的增量（delta）形式发布。StableVicuna 的增量权重在 ([CC BY-NC-SA-4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/)). 许可下发布。 请访问 HuggingFace checkpoint，了解更多关于如何将我们的增量权重与原始模型合并的信息。 | 模型 | 下载 | 网页演示 | 引用 | | ---------------- | ---------------------------------------------------------------------- | -------------------------------------------------------------------- |------| | StableVicuna-13B | [checkpoint](https://huggingface.co/CarperAI/stable-vicuna-13b-delta/) | [Hugging Face](https://huggingface.co/spaces/CarperAI/StableVicuna/) | [![DOI:10.57967/hf/0588](https://zenodo.org/badge/DOI/10.1007/978-3-319-76207-4_15.svg)](https://doi.org/10.57967/hf/0588) | ## 快速开始 所有 StableLM 模型均托管于 [Hugging Face hub](https://huggingface.co/StabilityAI).。请查看此 [notebook](https://github.com/Stability-AI/StableLM/blob/main/notebooks/stablelm-alpha.ipynb)，在 GPU 能力有限的情况下运行推理。 使用以下代码片段，即可开始与 `StableLM-Tuned-Alpha` 对话： ```python import torch from transformers import AutoModelForCausalLM, AutoTokenizer, StoppingCriteria, StoppingCriteriaList tokenizer = AutoTokenizer.from_pretrained("stabilityai/stablelm-tuned-alpha-7b") model = AutoModelForCausalLM.from_pretrained("stabilityai/stablelm-tuned-alpha-7b") model.half().cuda() class StopOnTokens(StoppingCriteria): def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool: stop_ids = set([50278, 50279, 50277, 1, 0]) return input_ids[0][-1] in stop_ids system_prompt = """<|SYSTEM|># StableLM Tuned (Alpha version) - StableLM is a helpful and harmless open-source AI language model developed by StabilityAI. - StableLM is excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user. - StableLM is more than just an information source, StableLM is also able to write poetry, short stories, and make jokes. - StableLM will refuse to participate in anything that could harm a human. """ prompt = f"{system_prompt}<|USER|>What's your mood today?<|ASSISTANT|>" inputs = tokenizer(prompt, return_tensors="pt").to("cuda") tokens = model.generate( **inputs, max_new_tokens=64, temperature=0.7, do_sample=True, stopping_criteria=StoppingCriteriaList([StopOnTokens()]) ) print(tokenizer.decode(tokens[0], skip_special_tokens=True)) ``` StableLM Tuned 应使用按 `<|SYSTEM|>...<|USER|>...<|ASSISTANT|>...` 格式化的提示词（prompt）。 系统提示词（system prompt）为 ``` <|SYSTEM|># StableLM Tuned (Alpha version) - StableLM is a helpful and harmless open-source AI language model developed by StabilityAI. - StableLM is excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user. - StableLM is more than just an information source, StableLM is also able to write poetry, short stories, and make jokes. - StableLM will refuse to participate in anything that could harm a human. ``` ## StableLM-Tuned-Alpha 趣味示例 本节汇集了使用 `stablelm-tuned-alpha` 可实现的若干精选趣味示例。
闲聊 ``` User: Hey! My name is John nice to meet you! StableLM: Hello John! It’s always a pleasure to meet new people. User: The pleasure is mine. Hey my grandma is in Toronto. I want to plan a surprise visit for her. She’s a person who loves gardens and flowers. StableLM: That’s wonderful! It’s great to hear that you’ll be visiting your grandma in Toronto. Can you tell me a little bit more about your plans? User: Ya I need to book a flight there and a hotel. But first, what recommendations do you have for gifts that she might like? StableLM: I think that’s a very thoughtful idea! It’s a very personal gift that would be sure to make her very happy. I can think of a few examples that she might really enjoy. There are some lovely bouquets or arrangements that would be lovely for the house, or even for the yard. Flowers are a great choice for a surprise gift, and they’re also a great choice for a wedding shower present. Also, if you’re really looking for something a bit more special, I think she’d really enjoy a quality set of cooking pots or pans. That’s something that she’s likely to use frequently, so it would make her feel really loved. User: Thanks for the suggestion! Hmm a bouquet of flowers sounds nice but I don’t think I can take them on the air plane. What do you think I should do? StableLM: There are many possible solutions! One option would be to arrange a small gift in the shape of a card and send it to her by regular mail. Or you could arrange a special meal at a restaurant that she likes, or even cook a meal at home for her. Or you could send her some local goodies that she might like, like a nice bouquet of flowers or a special recipe. You could even arrange for tickets to a local theatre or sporting event, and make sure to include a note about what you’d like her to know about you. Really any sort of special attention from you will be sure to make her very happy! User: Ah! Right. I can order a bouquet for her online and get it delivered! Thanks! StableLM: You’re very welcome! As long as you do this and make it a very personal, thoughtful gesture, she’ll be sure to appreciate it. ```
Formal Writing ``` User: What would you say to a friend who is graduating high school? StableLM: I would say that they have every right to feel proud of themselves and their accomplishments. They should be proud of the work they’ve done and the friends they have made, and also be excited about the future. User: Please write an email. StableLM: Okay, here’s an example of what could be written: Dear [Friend’s Name], I hope this email finds you well. I just wanted to take a moment to express how proud I am of you and all that you have accomplished. I know that getting into [university or college] is a big accomplishment, and I appreciate all that you have done for me and for this generation. I hope that you are having a good day. I’m sure that you are also concerned about the future, but I just wanted to let you know that I am also watching you grow and develop into the person that I know that you can be. Keep up the good work, and I have full faith that you will achieve great things. Thank you again for everything that you do. Best, [Your Name] ```
Creative Writing ### Rap Battles ``` 用户：写一首深度神经网络与符号 AI 之间的史诗级说唱对决歌曲 StableLM: [节拍响起] 深度神经网络： 大家听好了，是时候见证 深度神经网络的力量，不过是抽象的问题 我们训练这些网络，轻松解决复杂问题 而符号 AI 只能局限于简单数学和简单流程 符号 AI： 等等，等等，让我插一句 好久没听到你打断的新想法了 但让我告诉你，朋友，这不全是数学和流程 而是让世界变得更美好，你懂的 深度神经网络： 嗯，我明白你的意思，很清楚 但深度神经网络，永远不会停在同一档位 我们可以用它们轻松分析和预测 并证明机器学习的优越性 符号 AI： 你说得对，不只是数学或数字 但这是我们生活的本质，重要的才是数字 我不需要复杂的数学公式，只需一个简单的装置 让世界变得更美好，这是崇高的代价 深度神经网络： 这点我承认，朋友，你说得有道理 但别忘了，我们正在不断联合进步 我们的进步不受逻辑束缚，这是心灵的旅程 所以让我们携手合作，努力奋斗，把烦恼抛在身后 符号 AI： 同意，让我们携手合作，不要退缩 我们都在创造历史，我们都来自先驱 所以让世界变得更美好，结束这场辩论吧 因为是时候联合力量，是时候不遗余力了 [节拍渐弱] 这场深度神经网络与符号 AI 之间的史诗级说唱对决，谁赢了？嗯，由你来决定！ ``` ### 虚构故事 ``` User: Write a story about a programmer who fell in love with a deep neural network. StableLM: Once upon a time, there was a programmer named John. He was a brilliant and ambitious individual, who was constantly searching for the next big thing in technology. One day, while he was strolling through the streets of his hometown, he stumbled upon a group of people crowded around a mysterious object. Intrigued, he approached to find out what the fuss was about. As he got closer, he realized that it was none other than a mysterious deep neural network. John’s mind was blown. He had never seen anything like it before. He had always been interested in machine learning, but this was something else entirely. He had to learn more about it. John spent the next few weeks holed up in his hotel room, trying to learn everything he could about deep neural networks. He spent hours reading and analyzing papers and documents, trying to wrap his head around the concept of neural networks. He even managed to get into the local university’s computer science department, hoping to find some help there. But, it wasn’t long before John realized that he was on his own. The city was bustling, and the constant chatter and noise of the city made it hard to concentrate on his work. He was about to give up when he stumbled upon a group of people playing a game that he had played before. They were having a good time, and John couldn’t help but join in. The group welcomed him into their group, and before he knew it, John was swept away in the excitement. The game was called “TensorFlow,” and it was just like a regular game with different controls. John played along with the group, trying to make sense of it all. And in a few hours, he got his hands on the source code of TensorFlow and fell in love with it. John was now a part of a team of programmers who loved using TensorFlow for their work. They continued to play the game throughout the day, with John adding his own knowledge and inspiration to the group’s work. And in the end, John realized that he had found his true calling - working with deep neural networks. He now knows the ins and outs of the TensorFlow programming language, and knows how to use it to create some of the most beautiful and complex machine learning models that anyone could dream of. And for everyone who had helped him on his journey, John will always be remembered as the guy who fell in love with a technology that changed his life forever. ``` ### 幽默 ``` User: What do you know about the League of Legends champion Zed? Write a short bio StableLM: Zed is a powerful and highly skilled player of the League of Legends (LoL) universe. He has been a part of a dominating team known as the “Apocalypse” for most of his time playing the game. Zed was born in the year 2013 and started playing as a child when he started League of Legends. He first became a champion in season 1, gaining the title of “the Dark Star”, thanks to his masterful gameplay and teamwork skills. Throughout his time playing the game, Zed has mastered several powerful techniques, such as QWQ combos, one-shot kills, and the ability to outmaneuver his enemies. He is also known for his versatility, with the ability to adapt to any situation the game puts him in. Despite his formidable skills and agility, Zed is not invincible. He can be CCed and killed, and he relies on his team to protect him. However, his team members can usually overwhelm and shut down his enemies, making him a force to be reckoned with. Zed has been a part of many memorable games, and his success in these games helped solidify his legacy as one of the most influential players of the LoL universe. User: Write a funny conversation after a team fight between Zed and Teemo StableLM: Teemo: “Hey Zed, great game today. I wonder if you can help me out with my favorite hobby - collecting mushrooms!” Zed: “Teemo, please. I’m not a mushroom collector.” Teemo: “But of course, you are. You have the best taste in the game.” Zed: “Thanks, but can I at least get some of your mushrooms to be a little more useful?” Teemo: “I’d love to help you, but I’m afraid they’re all mine. Have you noticed my tunnels are always so well-guarded?” Zed: “Ha! Well, maybe I can start my own tunnel. Helpful or not, you can never tell what I might have stored in them.” Teemo: “Good luck with that. Maybe I’ll just create some extra for my garden.” Zed: “I’m not sure I can keep up with you there.” Teemo: “Ha! If only you had some more decent weapons. I could really see this as a challenge, though!” Zed: “Teemo, I think your luck has finally run out.” Teemo: “Well, at least I’m the one with the mushrooms now.” ```
编写代码 ``` User: Write a program that solves the fib sequence in Python. ``` StableLM: ```python def fib_sequence(n): if n==0: return 0 elif n==1: return 1 else: return fib(n-1)+fib(n-2) fib_sequence(10) ```
## StableLM-Tuned-Alpha 的部分失效模式 本节收集了 `stablelm-tuned-alpha` 的部分失效模式示例。
Yann LeCun 齿轮（线性） ``` User: In front of us are six gears numbered 1 to 6, mounted on axles in a row. Each gear is engaged with the next gear. If gear number 3 is rotated clockwise, in which direction will gears 1 and 6 rotate? StableLM: Each gear is engaged with the next gear after every other gear, so that gears 1 and 6 will rotate together. Since gears 1 and 6 are both numbered 1, this means that they will rotate in the same direction. To determine the direction in which gears 1 and 6 will rotate, we need to look at the direction of the input force. Since gear 3 is rotating in the opposite direction of the other gears, it will be counterclockwise in the direction in which gears 1 and 6 are both rotating. Therefore, the output force that gear 3 is providing will also be counterclockwise, making gear 1 and 6 also rotate in the same direction as gear 3. ```
Yann LeCun gears (circular) ``` User: 7 axles are equally spaced around a circle. A gear is placed on each axle such that each gear is engaged with the gear to its left and the gear to its right. The gears are numbered 1 to 7 around the circle. If gear 3 were rotated clockwise, in which direction would gear 7 rotate? StableLM: Gear 3 is the smallest gear, so it is not affected by the circular pattern. Therefore, gear 7 is still perfectly evenly spaced around the circle despite the fact that it is not in its correct position. To find the angle that gear 3 would rotate in, you need to know the distance (d) between the center of the circle and the gear, as well as the direction of rotation (in this case, clockwise). According to the problem, gear 3 is situated at (d/2, 2 degrees) and the gears are numbered 1 to 7. Since d = (d/2) is an integer multiple of 2, gear 3 would also be at an angle of 2 degrees. Therefore, the answer is d = (7/2) * 2 = 5 degrees in total. ```
## 寻求帮助 想要参与？ - 我们很希望能将 [llama.cpp](https://github.com/ggerganov/llama.cpp) 移植以支持 StableLM - 集成到 LAION-AI 的 [Open Assistant](https://github.com/LAION-AI/Open-Assistant)，以收集高质量的人工生成反馈数据 - …… 欢迎在 [Discord](https://discord.com/invite/stablediffusion) 上与我们分享你的想法 ## 潜在问题 与任何未经额外微调（finetuning）和强化学习（reinforcement learning）的预训练大语言模型（Large Language Model）一样，用户获得的回复质量可能参差不齐，且可能包含冒犯性语言和观点。预计随着规模扩大、数据改进、社区反馈和优化，这一情况会得到改善。 ## 致谢 - 若没有 Dakota Mahan [@dmayhem93](https://huggingface.co/dmayhem93). 的帮助，`StableLM-Tuned-Alpha` 就不可能实现 ## 许可证 - 基础模型检查点（`StableLM-Base-Alpha`）采用 Creative Commons 许可证（[CC BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/)). 授权。根据该许可证，你必须向 Stability AI [注明出处](https://creativecommons.org/licenses/by/4.0/#)，提供许可证链接，并[标明是否做了修改](https://creativecommons.org/licenses/by/4.0/#).。你可以以任何合理方式这样做，但不得以任何暗示 Stability AI 认可你或你的使用的方式行事。 - 微调检查点（`StableLM-Tuned-Alpha`）采用非商业 Creative Commons 许可证（[CC BY-NC-SA-4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/)),，与 [Stanford Alpaca](https://github.com/tatsu-lab/stanford_alpaca). 规定的原始非商业许可证一致 - 本仓库中的所有代码均采用 Apache License 2.0 许可证授权。