# Attention Is All You Need

The transformer architecture uses multi-head attention.
Layer normalization is applied before each sub-layer.
The feed-forward network consists of two linear transformations.