{ "
This defines configurations for a transformer. The configurations are calculate using option functions. These are lazy loaded and therefore only the necessary modules are calculated.
\n": "This defines configurations for a transformer. The configurations are calculate using option functions. These are lazy loaded and therefore only the necessary modules are calculated.
\n", "Dropout probability
\n": "Dropout probability
\n", "Number of attention heads
\n": "Number of attention heads
\n", "Number of layers
\n": "Number of layers
\n", "Number of tokens in the source vocabulary (for token embeddings)
\n": "Number of tokens in the source vocabulary (for token embeddings)
\n", "Number of tokens in the target vocabulary (to generate logits for prediction)
\n": "Number of tokens in the target vocabulary (to generate logits for prediction)
\n", "Transformer embedding size
\n": "Transformer embedding size
\n", "configs.py": "configs.py" }