wehub-resource-sync
156 lines
8.9 KiB
Markdown
156 lines
8.9 KiB
Markdown
<!-- WEHUB_ZH_README -->
|
||
> [!NOTE]
|
||
> 本文档由 WeHub 基于上游 README 翻译整理，属于社区翻译，非官方中文文档。
|
||
> [English](./README.en.md) · [原始项目](https://github.com/labmlai/annotated_deep_learning_paper_implementations) · [上游 README](https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/HEAD/readme.md)
|
||
> 原作者、版权与许可证归属以原始项目及本仓库 LICENSE 文件为准。
|
||
|
||
[](https://twitter.com/labmlai)
|
||
|
||
# [labml.ai Deep Learning Paper Implementations](https://nn.labml.ai/index.html)
|
||
|
||
这是一组简洁的 PyTorch 神经网络及相关算法实现。
|
||
这些实现配有解释性文档，
|
||
|
||
[网站](https://nn.labml.ai/index.html)
|
||
将这些内容渲染为并排排版的笔记。
|
||
我们相信这些内容能帮助你更好地理解这些算法。
|
||
|
||
|
||
|
||
我们正在积极维护本仓库，几乎每周都会添加新的
|
||
实现。
|
||
[](https://twitter.com/labmlai) 获取更新。
|
||
|
||
## 论文实现
|
||
|
||
#### ✨ [Transformers](https://nn.labml.ai/transformers/index.html)
|
||
|
||
* [JAX 实现](https://nn.labml.ai/transformers/jax_transformer/index.html)
|
||
* [多头注意力（Multi-headed attention）](https://nn.labml.ai/transformers/mha.html)
|
||
* [Triton Flash Attention](https://nn.labml.ai/transformers/flash/index.html)
|
||
* [Transformer 构建模块](https://nn.labml.ai/transformers/models.html)
|
||
* [Transformer XL](https://nn.labml.ai/transformers/xl/index.html)
|
||
* [相对多头注意力（Relative multi-headed attention）](https://nn.labml.ai/transformers/xl/relative_mha.html)
|
||
* [Rotary Positional Embeddings](https://nn.labml.ai/transformers/rope/index.html)
|
||
* [Attention with Linear Biases (ALiBi)](https://nn.labml.ai/transformers/alibi/index.html)
|
||
* [RETRO](https://nn.labml.ai/transformers/retro/index.html)
|
||
* [Compressive Transformer](https://nn.labml.ai/transformers/compressive/index.html)
|
||
* [GPT Architecture](https://nn.labml.ai/transformers/gpt/index.html)
|
||
* [GLU Variants](https://nn.labml.ai/transformers/glu_variants/simple.html)
|
||
* [kNN-LM: Generalization through Memorization](https://nn.labml.ai/transformers/knn)
|
||
* [Feedback Transformer](https://nn.labml.ai/transformers/feedback/index.html)
|
||
* [Switch Transformer](https://nn.labml.ai/transformers/switch/index.html)
|
||
* [Fast Weights Transformer](https://nn.labml.ai/transformers/fast_weights/index.html)
|
||
* [FNet](https://nn.labml.ai/transformers/fnet/index.html)
|
||
* [Attention Free Transformer](https://nn.labml.ai/transformers/aft/index.html)
|
||
* [Masked Language Model](https://nn.labml.ai/transformers/mlm/index.html)
|
||
* [MLP-Mixer: An all-MLP Architecture for Vision](https://nn.labml.ai/transformers/mlp_mixer/index.html)
|
||
* [Pay Attention to MLPs (gMLP)](https://nn.labml.ai/transformers/gmlp/index.html)
|
||
* [Vision Transformer (ViT)](https://nn.labml.ai/transformers/vit/index.html)
|
||
* [Primer EZ](https://nn.labml.ai/transformers/primer_ez/index.html)
|
||
* [Hourglass](https://nn.labml.ai/transformers/hour_glass/index.html)
|
||
|
||
#### ✨ [Low-Rank Adaptation (LoRA)](https://nn.labml.ai/lora/index.html)
|
||
|
||
#### ✨ [Eleuther GPT-NeoX](https://nn.labml.ai/neox/index.html)
|
||
* [在 48GB GPU 上生成](https://nn.labml.ai/neox/samples/generate.html)
|
||
* [在两块 48GB GPU 上微调](https://nn.labml.ai/neox/samples/finetune.html)
|
||
* [LLM.int8()](https://nn.labml.ai/neox/utils/llm_int8.html)
|
||
|
||
#### ✨ [扩散模型（Diffusion models）](https://nn.labml.ai/diffusion/index.html)
|
||
|
||
* [Denoising Diffusion Probabilistic Models (DDPM)](https://nn.labml.ai/diffusion/ddpm/index.html)
|
||
* [Denoising Diffusion Implicit Models (DDIM)](https://nn.labml.ai/diffusion/stable_diffusion/sampler/ddim.html)
|
||
* [Latent Diffusion Models](https://nn.labml.ai/diffusion/stable_diffusion/latent_diffusion.html)
|
||
* [Stable Diffusion](https://nn.labml.ai/diffusion/stable_diffusion/index.html)
|
||
|
||
#### ✨ [生成对抗网络（Generative Adversarial Networks）](https://nn.labml.ai/gan/index.html)
|
||
* [原始 GAN](https://nn.labml.ai/gan/original/index.html)
|
||
* [基于深度卷积网络的 GAN](https://nn.labml.ai/gan/dcgan/index.html)
|
||
* [Cycle GAN](https://nn.labml.ai/gan/cycle_gan/index.html)
|
||
* [Wasserstein GAN](https://nn.labml.ai/gan/wasserstein/index.html)
|
||
* [Wasserstein GAN with Gradient Penalty](https://nn.labml.ai/gan/wasserstein/gradient_penalty/index.html)
|
||
* [StyleGAN 2](https://nn.labml.ai/gan/stylegan/index.html)
|
||
|
||
#### ✨ [循环高速公路网络（Recurrent Highway Networks）](https://nn.labml.ai/recurrent_highway_networks/index.html)
|
||
|
||
#### ✨ [LSTM](https://nn.labml.ai/lstm/index.html)
|
||
|
||
#### ✨ [HyperNetworks - HyperLSTM](https://nn.labml.ai/hypernetworks/hyper_lstm.html)
|
||
|
||
#### ✨ [ResNet](https://nn.labml.ai/resnet/index.html)
|
||
|
||
#### ✨ [ConvMixer](https://nn.labml.ai/conv_mixer/index.html)
|
||
|
||
#### ✨ [胶囊网络（Capsule Networks）](https://nn.labml.ai/capsule_networks/index.html)
|
||
|
||
#### ✨ [U-Net](https://nn.labml.ai/unet/index.html)
|
||
|
||
#### ✨ [Sketch RNN](https://nn.labml.ai/sketch_rnn/index.html)
|
||
|
||
#### ✨ 图神经网络（Graph Neural Networks）
|
||
|
||
* [图注意力网络（Graph Attention Networks，GAT）](https://nn.labml.ai/graphs/gat/index.html)
|
||
* [图注意力网络 v2（Graph Attention Networks v2，GATv2）](https://nn.labml.ai/graphs/gatv2/index.html)
|
||
|
||
#### ✨ [反事实遗憾最小化（Counterfactual Regret Minimization，CFR）](https://nn.labml.ai/cfr/index.html)
|
||
|
||
使用 CFR 求解具有不完全信息的博弈，例如扑克。
|
||
|
||
* [Kuhn 扑克（Kuhn Poker）](https://nn.labml.ai/cfr/kuhn/index.html)
|
||
|
||
#### ✨ [强化学习（Reinforcement Learning）](https://nn.labml.ai/rl/index.html)
|
||
* [近端策略优化（Proximal Policy Optimization）](https://nn.labml.ai/rl/ppo/index.html) 与
|
||
[广义优势估计（Generalized Advantage Estimation）](https://nn.labml.ai/rl/ppo/gae.html)
|
||
* [深度 Q 网络（Deep Q Networks）](https://nn.labml.ai/rl/dqn/index.html) 结合
|
||
[对决网络（Dueling Network）](https://nn.labml.ai/rl/dqn/model.html),
|
||
[优先经验回放（Prioritized Replay）](https://nn.labml.ai/rl/dqn/replay_buffer.html)
|
||
以及 Double Q Network。
|
||
|
||
#### ✨ [优化器（Optimizers）](https://nn.labml.ai/optimizers/index.html)
|
||
* [Adam](https://nn.labml.ai/optimizers/adam.html)
|
||
* [AMSGrad](https://nn.labml.ai/optimizers/amsgrad.html)
|
||
* [带 warmup 的 Adam 优化器（Adam Optimizer with warmup）](https://nn.labml.ai/optimizers/adam_warmup.html)
|
||
* [Noam 优化器（Noam Optimizer）](https://nn.labml.ai/optimizers/noam.html)
|
||
* [校正 Adam 优化器（Rectified Adam Optimizer）](https://nn.labml.ai/optimizers/radam.html)
|
||
* [AdaBelief 优化器（AdaBelief Optimizer）](https://nn.labml.ai/optimizers/ada_belief.html)
|
||
* [Sophia-G 优化器（Sophia-G Optimizer）](https://nn.labml.ai/optimizers/sophia.html)
|
||
|
||
#### ✨ [归一化层（Normalization Layers）](https://nn.labml.ai/normalization/index.html)
|
||
* [批归一化（Batch Normalization）](https://nn.labml.ai/normalization/batch_norm/index.html)
|
||
* [层归一化（Layer Normalization）](https://nn.labml.ai/normalization/layer_norm/index.html)
|
||
* [实例归一化（Instance Normalization）](https://nn.labml.ai/normalization/instance_norm/index.html)
|
||
* [组归一化（Group Normalization）](https://nn.labml.ai/normalization/group_norm/index.html)
|
||
* [权重标准化（Weight Standardization）](https://nn.labml.ai/normalization/weight_standardization/index.html)
|
||
* [批通道归一化（Batch-Channel Normalization）](https://nn.labml.ai/normalization/batch_channel_norm/index.html)
|
||
* [DeepNorm](https://nn.labml.ai/normalization/deep_norm/index.html)
|
||
|
||
#### ✨ [蒸馏（Distillation）](https://nn.labml.ai/distillation/index.html)
|
||
|
||
#### ✨ [自适应计算（Adaptive Computation）](https://nn.labml.ai/adaptive_computation/index.html)
|
||
|
||
* [PonderNet](https://nn.labml.ai/adaptive_computation/ponder_net/index.html)
|
||
|
||
#### ✨ [不确定性（Uncertainty）](https://nn.labml.ai/uncertainty/index.html)
|
||
|
||
* [用于量化分类不确定性的证据深度学习（Evidential Deep Learning to Quantify Classification Uncertainty）](https://nn.labml.ai/uncertainty/evidence/index.html)
|
||
|
||
#### ✨ [激活函数（Activations）](https://nn.labml.ai/activations/index.html)
|
||
|
||
* [模糊分块激活（Fuzzy Tiling Activations）](https://nn.labml.ai/activations/fta/index.html)
|
||
|
||
#### ✨ [语言模型采样技术（Langauge Model Sampling Techniques）](https://nn.labml.ai/sampling/index.html)
|
||
* [贪心采样（Greedy Sampling）](https://nn.labml.ai/sampling/greedy.html)
|
||
* [温度采样（Temperature Sampling）](https://nn.labml.ai/sampling/temperature.html)
|
||
* [Top-k 采样（Top-k Sampling）](https://nn.labml.ai/sampling/top_k.html)
|
||
* [核采样（Nucleus Sampling）](https://nn.labml.ai/sampling/nucleus.html)
|
||
|
||
#### ✨ [可扩展训练/推理（Scalable Training/Inference）](https://nn.labml.ai/scaling/index.html)
|
||
* [Zero3 内存优化（Zero3 memory optimizations）](https://nn.labml.ai/scaling/zero3/index.html)
|
||
|
||
### 安装
|
||
|
||
```bash
|
||
pip install labml-nn
|
||
```
|