{
"<h1><a href=\"https://nn.labml.ai/transformers/aft/index.html\">An Attention Free Transformer</a></h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of the paper <a href=\"https://arxiv.org/abs/2105.14103\">An Attention Free Transformer</a>.</p>\n<p>This paper replaces the <a href=\"https://nn.labml.ai/transformers/mha.html\">self-attention layer</a> with a new efficient operation, that has memory complexity of O(Td), where T is the sequence length and <span translate=no>_^_0_^_</span> is the dimensionality of embeddings.</p>\n<p>The paper introduces AFT along with AFT-local and AFT-conv. Here we have implemented AFT-local which pays attention to closeby tokens in an autoregressive model. </p>\n": "<h1><a href=\"https://nn.labml.ai/transformers/aft/index.html\">\u4e00\u79cd\u65e0\u6ce8\u610f\u529b\u7684 Transformer </a></h1>\n<p>\u8fd9\u662f\u8bba\u6587 <a href=\"https://arxiv.org/abs/2105.14103\">\u300a\u4e00\u79cd\u65e0\u6ce8\u610f\u529b\u7684 Transformer \u300b</a>\u7684<a href=\"https://pytorch.org\">PyTorch </a>\u5b9e\u73b0\u3002</p>\n<p>\u8fd9\u7bc7\u8bba\u6587\u7528\u4e00\u79cd\u65b0\u7684\u9ad8\u6548\u64cd\u4f5c\u66ff\u4ee3\u4e86<a href=\"https://nn.labml.ai/transformers/mha.html\">\u81ea\u6ce8\u610f\u529b\u5c42</a>\uff0c\u8be5\u8fd0\u7b97\u7684\u5b58\u50a8\u590d\u6742\u5ea6\u4e3aO\uff08Td\uff09\uff0c\u5176\u4e2d T \u662f\u5e8f\u5217\u957f\u5ea6\uff0c<span translate=no>_^_0_^_</span>\u662f\u5d4c\u5165\u7684\u7ef4\u5ea6\u3002</p>\n<p>\u8be5\u8bba\u6587\u4ecb\u7ecd\u4e86 AFT \u4ee5\u53ca AFT-local \u548c AFT-conv \u3002\u8fd9\u91cc\u6211\u4eec\u5b9e\u73b0\u4e86 AFT-local \uff0c\u5b83\u4f1a\u5728\u81ea\u56de\u5f52\u6a21\u578b\u4e2d\u5173\u6ce8\u90bb\u8fd1\u7684 token \u3002</p>\n",
"An Attention Free Transformer": "\u4e00\u79cd\u65e0\u6ce8\u610f\u529b\u7684 Transformer"
}
wehub-resource-sync