wehub/eugeneyan--open-llms
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity
Files
main
eugeneyan--open-llms/README.md
T
wehub-resource-sync 812578d9ae docs: make Chinese README the default
2026-07-13 10:46:49 +00:00

42 KiB
Raw Permalink Blame History

Note

本文档由 WeHub 基于上游 README 翻译整理，属于社区翻译，非官方中文文档。
English · 原始项目 · 上游 README
原作者、版权与许可证归属以原始项目及本仓库 LICENSE 文件为准。

开源大语言模型（Open LLMs）

这些 LLM（Large Language Models，大语言模型）均获准用于商业用途（例如 Apache 2.0、MIT、OpenRAIL-M）。欢迎贡献！

语言模型 发布日期 检查点 论文/博客 参数量 (B) 上下文长度 许可证 试用
T5 2019/10 T5 & Flan-T5, Flan-T5-xxl (HF) Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer 0.06 - 11 512 Apache 2.0 T5-Large
RWKV 4 2021/08 RWKV, ChatRWKV The RWKV Language Model (and my LM tricks) 0.1 - 14 infinity (RNN) Apache 2.0
GPT-NeoX-20B 2022/04 GPT-NEOX-20B GPT-NeoX-20B: An Open-Source Autoregressive Language Model 20 2048 Apache 2.0
YaLM-100B 2022/06 yalm-100b Yandex publishes YaLM 100B, the largest GPT-like neural network in open source 100 1024 Apache 2.0
UL2 2022/10 UL2 & Flan-UL2, Flan-UL2 (HF) UL2 20B: An Open Source Unified Language Learner 20 512, 2048 Apache 2.0
Bloom 2022/11 Bloom BLOOM: A 176B-Parameter Open-Access Multilingual Language Model 176 2048 OpenRAIL-M v1
ChatGLM 2023/03 chatglm-6b ChatGLM, Github 6 2048 Custom 免费，但有一定使用限制（可能需要注册）
Cerebras-GPT 2023/03 Cerebras-GPT Cerebras-GPT: A Family of Open, Compute-efficient, Large Language Models (Paper) 0.111 - 13 2048 Apache 2.0 Cerebras-GPT-1.3B
Open Assistant (Pythia family) 2023/03 OA-Pythia-12B-SFT-8, OA-Pythia-12B-SFT-4, OA-Pythia-12B-SFT-1 Democratizing Large Language Model Alignment 12 2048 Apache 2.0 Pythia-2.8B
Pythia 2023/04 pythia 70M - 12B Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling 0.07 - 12 2048 Apache 2.0
Dolly 2023/04 dolly-v2-12b Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM 3, 7, 12 2048 MIT
StableLM-Alpha 2023/04 StableLM-Alpha Stability AI Launches the First of its StableLM Suite of Language Models 3 - 65 4096 CC BY-SA-4.0
FastChat-T5 2023/04 fastchat-t5-3b-v1.0 We are excited to release FastChat-T5: our compact and commercial-friendly chatbot! 3 512 Apache 2.0
DLite 2023/05 dlite-v2-1_5b Announcing DLite V2: Lightweight, Open LLMs That Can Run Anywhere 0.124 - 1.5 1024 Apache 2.0 DLite-v2-1.5B
h2oGPT 2023/05 h2oGPT Building the World’s Best Open-Source Large Language Model: H2O.ai’s Journey 12 - 20 256 - 2048 Apache 2.0
MPT-7B 2023/05 MPT-7B, MPT-7B-Instruct Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs 7 84k (ALiBi) Apache 2.0, CC BY-SA-3.0
RedPajama-INCITE 2023/05 RedPajama-INCITE Releasing 3B and 7B RedPajama-INCITE family of models including base, instruction-tuned & chat models 3 - 7 2048 Apache 2.0 RedPajama-INCITE-Instruct-3B-v1
OpenLLaMA 2023/05 open_llama_3b, open_llama_7b, open_llama_13b OpenLLaMA: An Open Reproduction of LLaMA 3, 7 2048 Apache 2.0 OpenLLaMA-7B-Preview_200bt
Falcon 2023/05 Falcon-180B, Falcon-40B, Falcon-7B The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only 180, 40, 7 2048 Apache 2.0
GPT-J-6B 2023/06 GPT-J-6B, GPT4All-J GPT-J-6B: 6B JAX-Based Transformer 6 2048 Apache 2.0
MPT-30B 2023/06 MPT-30B, MPT-30B-instruct MPT-30B: Raising the bar for open-source foundation models 30 8192 Apache 2.0, CC BY-SA-3.0 MPT 30B inference code using CPU
LLaMA 2 2023/06 LLaMA 2 Weights  Llama 2: Open Foundation and Fine-Tuned Chat Models 7 - 70 4096 Custom 若用户数低于 7 亿则可免费使用，且不得使用 LLaMA 输出训练除 LLaMA 及其衍生模型以外的其他 LLM HuggingChat
ChatGLM2 2023/06 chatglm2-6b ChatGLM2-6B, Github 6 32k Custom 免费，但有一定使用限制（可能需要注册）
XGen-7B 2023/06 xgen-7b-4k-base, xgen-7b-8k-base Long Sequence Modeling with XGen 7 4096, 8192 Apache 2.0
Jais-13b 2023/08 jais-13b, jais-13b-chat Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models 13 2048 Apache 2.0
OpenHermes 2023/09 OpenHermes-7B, OpenHermes-13B Nous Research 7, 13 4096 MIT OpenHermes-V2 Finetuned on Mistral 7B
OpenLM 2023/09 OpenLM 1B, OpenLM 7B  Open LM: a minimal but performative language modeling (LM) repository 1, 7 2048 MIT
Mistral 7B 2023/09 Mistral-7B-v0.1, Mistral-7B-Instruct-v0.1 Mistral 7B 7 4096-16K with Sliding Windows Apache 2.0 Mistral Transformer
ChatGLM3 2023/10 chatglm3-6b, chatglm3-6b-base, chatglm3-6b-32k, chatglm3-6b-128k ChatGLM3 6 8192, 32k, 128k Custom 免费，但有一定使用限制（可能需要注册）
Skywork 2023/10 Skywork-13B-Base, Skywork-13B-Math Skywork 13 4096 Custom 免费，但有使用限制；基于 Skywork 输出训练的模型将成为 Skywork 衍生模型，并受本许可证约束。
Jais-30b 2023/11 jais-30b-v1, jais-30b-chat-v1 Jais-30B: Expanding the Horizon in Open-Source Arabic NLP 30 2048 Apache 2.0
Zephyr 2023/11 Zephyr 7B Website 7 8192 Apache 2.0
DeepSeek 2023/11 deepseek-llm-7b-base, deepseek-llm-7b-chat, deepseek-llm-67b-base, deepseek-llm-67b-chat Introducing DeepSeek LLM, 7, 67 4096 Custom 免费，但有使用限制；基于 DeepSeek 输出训练的模型将成为 DeepSeek 衍生模型，并受本许可证约束。
Mistral 7B v0.2 2023/12 Mistral-7B-v0.2, Mistral-7B-Instruct-v0.2 La Plateforme 7 32k Apache 2.0
Mixtral 8x7B v0.1 2023/12 Mixtral-8x7B-v0.1, Mixtral-8x7B-Instruct-v0.1 Mixtral of experts 46.7 32k Apache 2.0
LLM360 Amber 2023/12 Amber, AmberChat, AmberSafe Introducing LLM360: Fully Transparent Open-Source LLMs 6.7 2048 Apache 2.0
SOLAR 2023/12 Solar-10.7B Upstage 10.7 4096 apache-2.0
phi-2 2023/12 phi-2 2.7B Microsoft 2.7 2048 MIT
FLOR 2023/12 FLOR-760M, FLOR-1.3B, FLOR-1.3B-Instructed, FLOR-6.3B, FLOR-6.3B-Instructed FLOR-6.3B: a chinchilla-compliant model for Catalan, Spanish and English 0.76, 1.3, 6.3 2048 Apache 2.0，继承自 BLOOM 的使用限制
RWKV 5 v2 2024/01 rwkv-5-world-0.4b-2, rwkv-5-world-1.5b-2, rwkv-5-world-3b-2, rwkv-5-world-3b-2(16k), rwkv-5-world-7b-2 RWKV 5 0.4, 1.5, 3, 7 unlimited(RNN), trained on 4096 (and 16k for 3b) Apache 2.0
OLMo 2024/02 OLMo 1B, OLMo 7B, OLMo 7B Twin 2T AI2 1,7 2048 Apache 2.0
Qwen1.5 2024/02 Qwen1.5-7B, Qwen1.5-7B-Chat, Qwen1.5-14B, Qwen1.5-14B-Chat, Qwen1.5-72B, Qwen1.5-72B-Chat Introducing Qwen1.5 7, 14, 72 32k Custom 若用户数低于 1 亿则可免费使用，且不得使用 Qwen 输出训练除 Qwen 及其衍生模型以外的其他 LLM
LWM 2024/02 LWM-Text-Chat-128K, LWM-Text-Chat-256K, LWM-Text-Chat-512K, LWM-Text-Chat-1M, LWM-Text-128K, LWM-Text-256K, LWM-Text-512K, LWM-Text-1M Large World Model (LWM) 7 128k, 256k, 512k, 1M LLaMA 2 许可证
Jais-30b v3 2024/03 jais-30b-v3, jais-30b-chat-v3 Jais 30b v3 30 8192 Apache 2.0
Gemma 2024/02 Gemma 7B, Gemma 7B it, Gemma 2B, Gemma 2B it Technical report 2-7 8192 Gemma Terms of Use 免费，但有使用限制；基于 Gemma 输出训练的模型将成为 Gemma 衍生模型，并受本许可证约束。
Grok-1 2024/03 Grok-1 Open Release of Grok-1 314 8192 Apache 2.0
Qwen1.5 MoE 2024/03 Qwen1.5-MoE-A2.7B, Qwen1.5-MoE-A2.7B-Chat Qwen1.5-MoE: Matching 7B Model Performance with 1/3 Activated Parameters 14.3 8192 Custom 若用户数低于 1 亿则可免费使用，且不得使用 Qwen 输出训练除 Qwen 及其衍生模型以外的其他 LLM
Jamba 0.1 2024/03 Jamba-v0.1 Introducing Jamba: AI21's Groundbreaking SSM-Transformer Model 52 256k Apache 2.0
Qwen1.5 32B 2024/04 Qwen1.5-32B, Qwen1.5-32B-Chat Qwen1.5-32B: Fitting the Capstone of the Qwen1.5 Language Model Series 32 32k Custom 若用户数低于 1 亿则可免费使用，且不得使用 Qwen 输出训练除 Qwen 及其衍生模型以外的其他 LLM
Mamba-7B 2024/04 mamba-7b-rw Toyota Research Institute 7 unlimited(RNN), trained on 2048 Apache 2.0
Mixtral8x22B v0.1 2024/04 Mixtral-8x22B-v0.1, Mixtral-8x22B-Instruct-v0.1 Cheaper, Better, Faster, Stronger 141 64k Apache 2.0
Llama 3 2024/04 Llama-3-8B, Llama-3-8B-Instruct, Llama-3-70B, Llama-3-70B-Instruct, Llama-Guard-2-8B Introducing Meta Llama 3, Meta Llama 3 8, 70 8192 Meta Llama 3 Community License Agreement 若用户数低于 7 亿则可免费使用，且不得使用 LLaMA 3 输出训练除 LLaMA 3 及其衍生模型以外的其他 LLM
Phi-3 Mini 2024/04 Phi-3-mini-4k-instruct, Phi-3-mini-128k-instruct Introducing Phi-3, Technical Report 3.8 4096, 128k MIT
OpenELM 2024/04 OpenELM-270M, OpenELM-270M-Instruct, OpenELM-450M, OpenELM-450M-Instruct, OpenELM-1_1B, OpenELM-1_1B-Instruct, OpenELM-3B, OpenELM-3B-Instruct OpenELM: An Efficient Language Model Family with Open Training and Inference Framework 0.27, 0.45, 1.1, 3 2048 Custom open license 无使用或训练限制
Snowflake Arctic 2024/04 snowflake-arctic-base, snowflake-arctic-instruct Snowflake Arctic: The Best LLM for Enterprise AI — Efficiently Intelligent, Truly Open 480 4096 Apache 2.0
Qwen1.5 110B 2024/04 Qwen1.5-110B, Qwen1.5-110B-Chat Qwen1.5-110B: The First 100B+ Model of the Qwen1.5 Series 110 32k Custom 若用户数低于 1 亿则可免费使用，且不得使用 Qwen 输出训练除 Qwen 及其衍生模型以外的其他 LLM
RWKV 6 v2.1 2024/05 rwkv-6-world-1.6b-2.1, rwkv-6-world-3b-2.1, rwkv-6-world-7b-2.1 RWKV 6 1.6, 3, 7 unlimited(RNN), trained on 4096 Apache 2.0
DeepSeek-V2 2024/05 DeepSeek-V2, DeepSeek-V2-Chat DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model 236 128k Custom 免费，但有使用限制；基于 DeepSeek 输出训练的模型将成为 DeepSeek 衍生模型，并受本许可证约束。
Fugaku-LLM 2024/05 Fugaku-LLM-13B, Fugaku-LLM-13B-instruct Release of "Fugaku-LLM" – a large language model trained on the supercomputer "Fugaku" 13 2048 Custom 免费，但有使用限制
Falcon 2 2024/05 falcon2-11B Meet Falcon 2: TII Releases New AI Model Series, Outperforming Meta’s New Llama 3 11 8192 Custom Apache 2.0 附带轻度可接受使用政策
Yi-1.5 2024/05 Yi-1.5-6B, Yi-1.5-6B-Chat, Yi-1.5-9B, Yi-1.5-9B-Chat, Yi-1.5-34B, Yi-1.5-34B-Chat Yi-1.5 6, 9, 34 4096 Apache 2.0
DeepSeek-V2-Lite 2024/05 DeepSeek-V2-Lite, DeepSeek-V2-Lite-Chat DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model 16 32k Custom 免费，但有使用限制；基于 DeepSeek 输出训练的模型将成为 DeepSeek 衍生模型，并受本许可证约束。
Phi-3 small/medium 2024/05 Phi-3-mini-4k-instruct, Phi-3-mini-128k-instruct, Phi-3-medium-4k-instruct, Phi-3-medium-128k-instruct New models added to the Phi-3 family, available on Microsoft Azure, Technical Report 7, 14 4096, 128k MIT
Phi-4 2024/12 Phi-4 Introducing Phi-4: Microsoft’s Newest Small Language Model Specializing in Complex Reasoning, Technical Report 14 4096 MIT
YuLan-Mini 2024/12 YuLan-Mini YuLan-Mini: An Open Data-efficient Language Model, GitHub 14 28672 MIT YuLan-Mini
Selene Mini 2025/01 Selene Mini Atla Selene Mini: A General Purpose Evaluation Model, GitHub 8 128K Apache 2.0 Hugging Face Space

用于代码的开源大语言模型（Open LLMs）

Language Model Release Date Checkpoints Paper/Blog Params (B) Context Length Licence Try it
SantaCoder 2023/01 santacoder SantaCoder: don't reach for the stars! 1.1 2048 OpenRAIL-M v1 SantaCoder
CodeGen2 2023/04 codegen2 1B-16B CodeGen2: Lessons for Training LLMs on Programming and Natural Languages 1 - 16 2048 Apache 2.0
StarCoder 2023/05 starcoder StarCoder: A State-of-the-Art LLM for Code, StarCoder: May the source be with you! 1.1-15 8192 OpenRAIL-M v1
StarChat Alpha 2023/05 starchat-alpha Creating a Coding Assistant with StarCoder 16 8192 OpenRAIL-M v1
Replit Code 2023/05 replit-code-v1-3b Training a SOTA Code LLM in 1 week and Quantifying the Vibes — with Reza Shabani of Replit 2.7 infinity? (ALiBi) CC BY-SA-4.0 Replit-Code-v1-3B
CodeT5+ 2023/05 CodeT5+ CodeT5+: Open Code Large Language Models for Code Understanding and Generation 0.22 - 16 512 BSD-3-Clause Codet5+-6B
XGen-7B 2023/06 XGen-7B-8K-Base Long Sequence Modeling with XGen: A 7B LLM Trained on 8K Input Sequence Length 7 8192 Apache 2.0
CodeGen2.5 2023/07 CodeGen2.5-7B-multi CodeGen2.5: Small, but mighty 7 2048 Apache 2.0
DeciCoder-1B 2023/08 DeciCoder-1B Introducing DeciCoder: The New Gold Standard in Efficient and Accurate Code Generation 1.1 2048 Apache 2.0 DeciCoder Demo
Code Llama 2023/08 Inference Code for CodeLlama models  Code Llama: Open Foundation Models for Code 7 - 34 4096 Custom Free if you have under 700M users and you cannot use LLaMA outputs to train other LLMs besides LLaMA and its derivatives HuggingChat

用于预训练的开源大语言模型（LLM）数据集

Name Release Date Paper/Blog Dataset Tokens (T) License
RedPajama 2023/04 RedPajama, a project to create leading open-source models, starts by reproducing LLaMA training dataset of over 1.2 trillion tokens RedPajama-Data 1.2 Apache 2.0
starcoderdata 2023/05 StarCoder: A State-of-the-Art LLM for Code starcoderdata 0.25 Apache 2.0

用于指令微调（instruction-tuning）的开源大语言模型（LLM）数据集

Name Release Date Paper/Blog Dataset Samples (K) License
OIG (Open Instruction Generalist) 2023/03 THE OIG DATASET OIG 44,000 Apache 2.0
databricks-dolly-15k 2023/04 Free Dolly: Introducing the World's First Truly Open Instruction-Tuned LLM databricks-dolly-15k 15 CC BY-SA-3.0
MPT-7B-Instruct 2023/05 Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs dolly_hhrlhf 59 CC BY-SA-3.0

用于对齐微调（alignment-tuning）的开源大语言模型（LLM）数据集

Name Release Date Paper/Blog Dataset Samples (K) License
OpenAssistant Conversations Dataset 2023/04 OpenAssistant Conversations - Democratizing Large Language Model Alignment oasst1 161 Apache 2.0

开源大语言模型（LLM）评测

这些许可证分别是什么意思？

  • Apache 2.0: 允许用户出于任何目的使用软件、分发软件、修改软件，并在许可条款下分发修改后的版本，无需担心版税。
  • MIT: 与 Apache 2.0 类似，但更短、更简单。此外，与 Apache 2.0 不同的是，它不要求声明对原始代码所做的任何重大更改。
  • CC BY-SA-4.0: 允许 (i) 复制和再分发材料，以及 (ii) 出于任何目的（包括商业目的）对材料进行混编、转换和二次创作。但如果你进行后者，你必须在与原始材料相同的许可下分发你的贡献。（因此，可能不适用于内部团队。）
  • OpenRAIL-M v1: 允许免版税访问，并可灵活地对模型及其修改进行下游使用和共享，同时附带一组使用限制（见 Attachment A)
  • BSD-3-Clause: 该版本允许出于任何目的进行无限制再分发，只要保留其版权声明和许可中的免责声明即可。

免责声明： 本仓库提供的信息不构成、且无意构成法律建议。本仓库维护者不对使用相关模型的第三方行为负责。若出于商业目的使用模型，请先咨询律师。

改进

  • 补全上下文长度（context length）相关条目，并使用 ? 检查条目
  • 添加训练 token 数量？（参见 注意事项)
  • 添加（指向）训练代码的链接？
  • 添加（指向）评估基准（eval benchmarks）的链接？
Reference in New Issue View Git Blame Copy Permalink