{ "

Transformer Configurations

\n

This defines configurations for a transformer. The configurations are calculate using option functions. These are lazy loaded and therefore only the necessary modules are calculated.

\n": "

Transformer Configurations

\n

This defines configurations for a transformer. The configurations are calculate using option functions. These are lazy loaded and therefore only the necessary modules are calculated.

\n", "

Dropout probability

\n": "

Dropout probability

\n", "

Number of attention heads

\n": "

Number of attention heads

\n", "

Number of layers

\n": "

Number of layers

\n", "

Number of tokens in the source vocabulary (for token embeddings)

\n": "

Number of tokens in the source vocabulary (for token embeddings)

\n", "

Number of tokens in the target vocabulary (to generate logits for prediction)

\n": "

Number of tokens in the target vocabulary (to generate logits for prediction)

\n", "

Transformer embedding size

\n": "

Transformer embedding size

\n", "configs.py": "configs.py" }