chore: import upstream snapshot with attribution
This commit is contained in:
@@ -0,0 +1,14 @@
|
||||
Mixture of Experts (DeepSpeed MoE)
|
||||
==================================
|
||||
|
||||
DeepSpeed provides two forms of MoE support: DeepSpeed MoE and :doc:`AutoEP
|
||||
(Automatic Expert Parallelism) <autoep>`. DeepSpeed MoE is the explicit
|
||||
``deepspeed.moe.layer.MoE`` API for constructing MoE layers in model code. This
|
||||
page introduces the DeepSpeed MoE API.
|
||||
|
||||
See also the `Mixture of Experts (DeepSpeed MoE) tutorial
|
||||
<https://www.deepspeed.ai/tutorials/mixture-of-experts/>`__ for training
|
||||
examples and configuration details.
|
||||
|
||||
.. autoclass:: deepspeed.moe.layer.MoE
|
||||
:members:
|
||||
Reference in New Issue
Block a user