wehub-resource-sync
115 lines
1.2 KiB
ReStructuredText
115 lines
1.2 KiB
ReStructuredText
DeepSpeed
|
|
=========
|
|
|
|
Model Setup
|
|
-----------
|
|
|
|
.. toctree::
|
|
:maxdepth: 2
|
|
|
|
initialize
|
|
inference-init
|
|
|
|
Training API
|
|
------------
|
|
|
|
.. toctree::
|
|
:maxdepth: 2
|
|
|
|
training
|
|
|
|
Inference API
|
|
-------------
|
|
|
|
.. toctree::
|
|
:maxdepth: 2
|
|
|
|
inference-engine
|
|
|
|
Checkpointing API
|
|
-----------------
|
|
.. toctree::
|
|
:maxdepth: 2
|
|
|
|
model-checkpointing
|
|
activation-checkpointing
|
|
|
|
|
|
ZeRO API
|
|
--------
|
|
.. toctree::
|
|
:maxdepth: 2
|
|
|
|
zero3
|
|
|
|
Mixture of Experts
|
|
------------------
|
|
.. toctree::
|
|
:maxdepth: 2
|
|
|
|
autoep
|
|
moe
|
|
|
|
Transformer Kernel API
|
|
----------------------
|
|
.. toctree::
|
|
:maxdepth: 2
|
|
|
|
kernel
|
|
|
|
Pipeline Parallelism
|
|
--------------------
|
|
.. toctree::
|
|
:maxdepth: 2
|
|
|
|
pipeline
|
|
|
|
Optimizers
|
|
----------
|
|
.. toctree::
|
|
:maxdepth: 2
|
|
|
|
optimizers
|
|
|
|
Learning Rate Schedulers
|
|
------------------------
|
|
.. toctree::
|
|
:maxdepth: 2
|
|
|
|
schedulers
|
|
|
|
Flops Profiler
|
|
--------------------
|
|
.. toctree::
|
|
:maxdepth: 2
|
|
|
|
flops-profiler
|
|
|
|
Autotuning
|
|
--------------------
|
|
.. toctree::
|
|
:maxdepth: 2
|
|
|
|
autotuning
|
|
|
|
Memory Usage
|
|
------------------
|
|
.. toctree::
|
|
:maxdepth: 2
|
|
|
|
memory
|
|
|
|
Monitoring
|
|
----------
|
|
.. toctree::
|
|
:maxdepth: 2
|
|
|
|
monitor
|
|
|
|
Indices and tables
|
|
------------------
|
|
|
|
* :ref:`genindex`
|
|
* :ref:`modindex`
|
|
* :ref:`search`
|