wehub-resource-sync
151 lines
5.0 KiB
Plaintext
151 lines
5.0 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "2365faf7-39fb-4e53-a810-2e28c4f6b4c1",
|
|
"metadata": {},
|
|
"source": [
|
|
"# DNAGPT2-基因序列大模型最佳入门\n",
|
|
"\n",
|
|
"## 1 概要\n",
|
|
"自然语言大模型早已超出NLP研究领域，正在成为AI for science的基石。生物信息学中的基因序列，则是和自然语言最类似的，把大模型应用于生物序列研究，就成了最近一两年的热门研究方向，特别是2024年预测蛋白质结构的alphaFold获得诺贝尔化学奖，更是为生物学的研究指明了未来的方向。\n",
|
|
"\n",
|
|
"但对大多数从事生物学研究的工作者而言，大模型又非常陌生。事实上，在2023年之前，GPT等大模型还是NLP领域研究的小众课题，只是因为chatgpt的爆发，才进入公众视野。\n",
|
|
"\n",
|
|
"而大部生物学+大模型的研究，也都是2023年之后的工作，但领域跨度过大，这些论文一般都是大公司、大团队协作的产物，大部分研究者要学习或者重现这些工作，困难重重，我们在很多top论文的github issue中，都能感受到这一点。\n",
|
|
"\n",
|
|
"一方面，言必称大模型几乎是生物学研究确定的未来，另一方面，众多生物学研究者却在大模型的门槛前徘徊不前。如何在这个门槛前加一道梯子，就成了该领域一个迫切的需求。\n",
|
|
"\n",
|
|
"DNAGTP2就是这样的梯子，仅望能抛砖引玉，让更多的生物学工作者能够越过大模型的门槛，戴上大模型的翅膀，卷过同行。\n",
|
|
"\n",
|
|
"\n",
|
|
"<<img src='img/gpt2_bridge.png' width=\"600px\" />\n",
|
|
"\n",
|
|
"## 2 教程特色\n",
|
|
"本教程主要有以下特色：\n",
|
|
"\n",
|
|
"1 简单。代码简单，全部代码均为huggingface标准库构建，阅后即会。\n",
|
|
"\n",
|
|
"2 简单。理论简单，只讲最基础的网络构架，全部可视化讲解。\n",
|
|
"\n",
|
|
"3 简单。案例简单，均使用经典论文的代表性案例，一看就懂。\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"教程内容又不简单，从基础的分词器构建，到从头构建gpt、bert等典型模型，到llama模型微调，基本的deepspeed多卡分布式训练，到lucaone、ESM3等SOTA大模型的应用，结合序列分类、结构预测、回归分析等典型生物学任务，循序渐进，逐步展开。本教程会紧跟研究趋势，不断更新。\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"本教程面向人群：\n",
|
|
"\n",
|
|
"1 生物学领域科研人员、学生等，特别是生物信息学。\n",
|
|
"\n",
|
|
"2 大模型学习入门。不仅是生物学领域的，都可以看看，和一般大模型入门没啥差别，只是数据不同。\n",
|
|
"\n",
|
|
"\n",
|
|
"huggingface: https://huggingface.co/dnagpt/dnagpt2\n",
|
|
"\n",
|
|
"github: https://github.com/maris205/dnagpt2\n",
|
|
"\n",
|
|
"\n",
|
|
"## 3 教程大纲\n",
|
|
"\n",
|
|
"<img src='img/DNAGPT2.png' width=\"600px\" />\n",
|
|
"\n",
|
|
"\n",
|
|
"1 数据和环境\n",
|
|
"\n",
|
|
"1.1 大模型运行环境简介\n",
|
|
"\n",
|
|
"1.2 基因相关预训练和微调数据\n",
|
|
"\n",
|
|
"1.3 datasets库基本使用\n",
|
|
"\n",
|
|
"2 从头构建DNA的GPT2/Bert大模型\n",
|
|
"\n",
|
|
"\n",
|
|
"2.1 DNA分词器构建\n",
|
|
"\n",
|
|
"2.2 从头训练dna gpt2大模型\n",
|
|
"\n",
|
|
"2.3 从头训练dna bert大模型\n",
|
|
"\n",
|
|
"2.4 基因大模型的生物序列特征提取\n",
|
|
"\n",
|
|
"2.4 基于多模态数据构建大模型\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"3 基因大模型的生物序列任务\n",
|
|
"\n",
|
|
"3.1 序列分类任务\n",
|
|
"\n",
|
|
"3.2 序列结构预测\n",
|
|
"\n",
|
|
"3.3 多序列交互作用分析\n",
|
|
"\n",
|
|
"3.4 功能预测任务\n",
|
|
"\n",
|
|
"3.5 回归类任务\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"4 进入chatgpt时代: 基因指令构建和微调\n",
|
|
"\n",
|
|
"4.1 基于基因数据的llama词典扩充\n",
|
|
"\n",
|
|
"4.2 deepspeed分布式训练简介\n",
|
|
"\n",
|
|
"4.3 基于基因数据的llama模型持续预训练\n",
|
|
"\n",
|
|
"4.4 基于llama-gene大模型的分类任务\n",
|
|
"\n",
|
|
"4.5 基于llama-gene大模型的指令微调\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"5 生物领域SOTA大模型应用概要\n",
|
|
"\n",
|
|
"5.1 dnabert2应用\n",
|
|
"\n",
|
|
"5.2 lucaone使用\n",
|
|
"\n",
|
|
"5.3 ESM3使用\n",
|
|
"\n",
|
|
"5.4 Medgpt应用\n",
|
|
"\n",
|
|
"5.5 llama-gene应用"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "3252ef0f-3193-43f3-9dcf-5d2b625dbdf7",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.12.3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|