chore: import upstream snapshot with attribution
This commit is contained in:
@@ -0,0 +1,17 @@
|
||||
Transformer Kernels
|
||||
===================
|
||||
|
||||
The transformer kernel API in DeepSpeed can be used to create BERT transformer layer for
|
||||
more efficient pre-training and fine-tuning, it includes the transformer layer configurations and
|
||||
transformer layer module initialization.
|
||||
|
||||
Here we present the transformer kernel API.
|
||||
Please see the `BERT pre-training tutorial <https://www.deepspeed.ai/tutorials/bert-pretraining/>`_ for usage details.
|
||||
|
||||
DeepSpeed Transformer Config
|
||||
----------------------------
|
||||
.. autoclass:: deepspeed.DeepSpeedTransformerConfig
|
||||
|
||||
DeepSpeed Transformer Layer
|
||||
----------------------------
|
||||
.. autoclass:: deepspeed.DeepSpeedTransformerLayer
|
||||
Reference in New Issue
Block a user