wehub-resource-sync
18 lines
653 B
ReStructuredText
18 lines
653 B
ReStructuredText
Transformer Kernels
|
|
===================
|
|
|
|
The transformer kernel API in DeepSpeed can be used to create BERT transformer layer for
|
|
more efficient pre-training and fine-tuning, it includes the transformer layer configurations and
|
|
transformer layer module initialization.
|
|
|
|
Here we present the transformer kernel API.
|
|
Please see the `BERT pre-training tutorial <https://www.deepspeed.ai/tutorials/bert-pretraining/>`_ for usage details.
|
|
|
|
DeepSpeed Transformer Config
|
|
----------------------------
|
|
.. autoclass:: deepspeed.DeepSpeedTransformerConfig
|
|
|
|
DeepSpeed Transformer Layer
|
|
----------------------------
|
|
.. autoclass:: deepspeed.DeepSpeedTransformerLayer
|