chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1,41 @@
|
||||
{
|
||||
"<h1>Vision Transformer (ViT)</h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of the paper <a href=\"https://arxiv.org/abs/2010.11929\">An Image Is Worth 16x16 Words: Transformers For Image Recognition At Scale</a>.</p>\n<p>Vision transformer applies a pure transformer to images without any convolution layers. They split the image into patches and apply a transformer on patch embeddings. <a href=\"#PathEmbeddings\">Patch embeddings</a> are generated by applying a simple linear transformation to the flattened pixel values of the patch. Then a standard transformer encoder is fed with the patch embeddings, along with a classification token <span translate=no>_^_0_^_</span>. The encoding on the <span translate=no>_^_1_^_</span> token is used to classify the image with an MLP.</p>\n<p>When feeding the transformer with the patches, learned positional embeddings are added to the patch embeddings, because the patch embeddings do not have any information about where that patch is from. The positional embeddings are a set of vectors for each patch location that get trained with gradient descent along with other parameters.</p>\n<p>ViTs perform well when they are pre-trained on large datasets. The paper suggests pre-training them with an MLP classification head and then using a single linear layer when fine-tuning. The paper beats SOTA with a ViT pre-trained on a 300 million image dataset. They also use higher resolution images during inference while keeping the patch size the same. The positional embeddings for new patch locations are calculated by interpolating learning positional embeddings.</p>\n<p>Here's <a href=\"experiment.html\">an experiment</a> that trains ViT on CIFAR-10. This doesn't do very well because it's trained on a small dataset. It's a simple experiment that anyone can run and play with ViTs.</p>\n": "<h1>\u89c6\u89c9\u53d8\u538b\u5668 (ViT)</h1>\n<p>\u8fd9\u662f <a href=\"https://pytorch.org\">PyTorch \u5bf9</a>\u300aAn <a href=\"https://arxiv.org/abs/2010.11929\">Image Is Worth 16x16 Words\uff1a\u7528\u4e8e\u5927\u89c4\u6a21\u56fe\u50cf\u8bc6\u522b\u7684\u53d8\u5f62\u91d1\u521a\u300b\u8bba\u6587</a>\u7684\u5b9e\u73b0\u3002</p>\n<p>\u89c6\u89c9\u8f6c\u6362\u5668\u5c06\u7eaf\u7cb9\u7684\u53d8\u6362\u5668\u5e94\u7528\u4e8e\u6ca1\u6709\u4efb\u4f55\u5377\u79ef\u5c42\u7684\u56fe\u50cf\u3002\u4ed6\u4eec\u5c06\u56fe\u50cf\u5206\u5272\u6210\u8865\u4e01\uff0c\u5e76\u5728\u8865\u4e01\u5d4c\u5165\u4e0a\u5e94\u7528\u8f6c\u6362\u5668\u3002<a href=\"#PathEmbeddings\">\u8865\u4e01\u5d4c\u5165</a>\u662f\u901a\u8fc7\u5bf9\u8865\u4e01\u7684\u6241\u5e73\u5316\u50cf\u7d20\u503c\u5e94\u7528\u7b80\u5355\u7684\u7ebf\u6027\u53d8\u6362\u6765\u751f\u6210\u7684\u3002\u7136\u540e\uff0c\u5411\u6807\u51c6\u53d8\u538b\u5668\u7f16\u7801\u5668\u63d0\u4f9b\u8865\u4e01\u5d4c\u5165\u4ee5\u53ca\u5206\u7c7b\u6807\u8bb0<span translate=no>_^_0_^_</span>\u3002<span translate=no>_^_1_^_</span>\u4ee4\u724c\u4e0a\u7684\u7f16\u7801\u7528\u4e8e\u4f7f\u7528 MLP \u5bf9\u56fe\u50cf\u8fdb\u884c\u5206\u7c7b\u3002</p>\n<p>\u5f53\u5411\u8f6c\u6362\u5668\u63d0\u4f9b\u8865\u4e01\u65f6\uff0c\u5b66\u5230\u7684\u4f4d\u7f6e\u5d4c\u5165\u4f1a\u6dfb\u52a0\u5230\u8865\u4e01\u5d4c\u5165\u4e2d\uff0c\u56e0\u4e3a\u8865\u4e01\u5d4c\u5165\u6ca1\u6709\u4efb\u4f55\u5173\u4e8e\u8be5\u8865\u4e01\u6765\u81ea\u4f55\u5904\u7684\u4fe1\u606f\u3002\u4f4d\u7f6e\u5d4c\u5165\u662f\u6bcf\u4e2a\u8865\u4e01\u4f4d\u7f6e\u7684\u4e00\u7ec4\u5411\u91cf\uff0c\u8fd9\u4e9b\u5411\u91cf\u4f7f\u7528\u68af\u5ea6\u4e0b\u964d\u548c\u5176\u4ed6\u53c2\u6570\u8fdb\u884c\u8bad\u7ec3\u3002</p>\n<p>VIT \u5728\u5927\u578b\u6570\u636e\u96c6\u4e0a\u8fdb\u884c\u9884\u8bad\u7ec3\u65f6\u8868\u73b0\u826f\u597d\u3002\u672c\u6587\u5efa\u8bae\u4f7f\u7528 MLP \u5206\u7c7b\u5934\u5bf9\u4ed6\u4eec\u8fdb\u884c\u9884\u8bad\u7ec3\uff0c\u7136\u540e\u5728\u5fae\u8c03\u65f6\u4f7f\u7528\u5355\u4e2a\u7ebf\u6027\u5c42\u3002\u8be5\u8bba\u6587\u57283\u4ebf\u5f20\u56fe\u50cf\u6570\u636e\u96c6\u4e0a\u9884\u5148\u8bad\u7ec3\u4e86ViT\uff0c\u51fb\u8d25\u4e86SOTA\u3002\u5b83\u4eec\u8fd8\u5728\u63a8\u7406\u8fc7\u7a0b\u4e2d\u4f7f\u7528\u66f4\u9ad8\u5206\u8fa8\u7387\u7684\u56fe\u50cf\uff0c\u540c\u65f6\u4fdd\u6301\u8865\u4e01\u5927\u5c0f\u4e0d\u53d8\u3002\u65b0\u8865\u4e01\u4f4d\u7f6e\u7684\u4f4d\u7f6e\u5d4c\u5165\u662f\u901a\u8fc7\u63d2\u503c\u5b66\u4e60\u4f4d\u7f6e\u5d4c\u5165\u6765\u8ba1\u7b97\u7684\u3002</p>\n<p>\u8fd9\u662f<a href=\"experiment.html\">\u4e00\u9879\u5728 CIFAR-10 \u4e0a\u8bad\u7ec3 ViT \u7684\u5b9e\u9a8c</a>\u3002\u8fd9\u6548\u679c\u4e0d\u592a\u597d\uff0c\u56e0\u4e3a\u5b83\u662f\u5728\u4e00\u4e2a\u5c0f\u6570\u636e\u96c6\u4e0a\u8bad\u7ec3\u7684\u3002\u8fd9\u662f\u4e00\u4e2a\u7b80\u5355\u7684\u5b9e\u9a8c\uff0c\u4efb\u4f55\u4eba\u90fd\u53ef\u4ee5\u4f7f\u7528Vits\u8fd0\u884c\u548c\u73a9\u6e38\u620f\u3002</p>\n",
|
||||
"<h2>Vision Transformer</h2>\n<p>This combines the <a href=\"#PatchEmbeddings\">patch embeddings</a>, <a href=\"#LearnedPositionalEmbeddings\">positional embeddings</a>, transformer and the <a href=\"#ClassificationHead\">classification head</a>.</p>\n": "<h2>\u89c6\u89c9\u53d8\u538b\u5668</h2>\n<p>\u8fd9\u7ed3\u5408\u4e86<a href=\"#PatchEmbeddings\">\u8865\u4e01\u5d4c\u5165</a>\u3001<a href=\"#LearnedPositionalEmbeddings\">\u4f4d\u7f6e\u5d4c\u5165</a>\u3001\u53d8\u538b\u5668\u548c<a href=\"#ClassificationHead\">\u5206\u7c7b\u5934</a>\u3002</p>\n",
|
||||
"<p> </p>\n": "<p></p>\n",
|
||||
"<p> <a id=\"ClassificationHead\"></a></p>\n<h2>MLP Classification Head</h2>\n<p>This is the two layer MLP head to classify the image based on <span translate=no>_^_0_^_</span> token embedding.</p>\n": "<p><a id=\"ClassificationHead\"></a></p>\n<h2>MLP \u5206\u7c7b\u4e3b\u7ba1</h2>\n<p>\u8fd9\u662f\u57fa\u4e8e<span translate=no>_^_0_^_</span>\u4ee4\u724c\u5d4c\u5165\u5bf9\u56fe\u50cf\u8fdb\u884c\u5206\u7c7b\u7684\u53cc\u5c42 MLP \u5934\u3002</p>\n",
|
||||
"<p> <a id=\"LearnedPositionalEmbeddings\"></a></p>\n<h2>Add parameterized positional encodings</h2>\n<p>This adds learned positional embeddings to patch embeddings.</p>\n": "<p><a id=\"LearnedPositionalEmbeddings\"></a></p>\n<h2>\u6dfb\u52a0\u53c2\u6570\u5316\u7684\u4f4d\u7f6e\u7f16\u7801</h2>\n<p>\u8fd9\u5c06\u5b66\u4e60\u7684\u4f4d\u7f6e\u5d4c\u5165\u6dfb\u52a0\u5230\u8865\u4e01\u5d4c\u5165\u4e2d\u3002</p>\n",
|
||||
"<p> <a id=\"PatchEmbeddings\"></a></p>\n<h2>Get patch embeddings</h2>\n<p>The paper splits the image into patches of equal size and do a linear transformation on the flattened pixels for each patch.</p>\n<p>We implement the same thing through a convolution layer, because it's simpler to implement.</p>\n": "<p><a id=\"PatchEmbeddings\"></a></p>\n<h2>\u83b7\u53d6\u8865\u4e01\u5d4c\u5165</h2>\n<p>\u7eb8\u5f20\u5c06\u56fe\u50cf\u5206\u5272\u6210\u5927\u5c0f\u76f8\u7b49\u7684\u6591\u5757\uff0c\u7136\u540e\u5bf9\u6bcf\u4e2a\u8865\u4e01\u7684\u6241\u5e73\u50cf\u7d20\u8fdb\u884c\u7ebf\u6027\u53d8\u6362\u3002</p>\n<p>\u6211\u4eec\u901a\u8fc7\u5377\u79ef\u5c42\u5b9e\u73b0\u540c\u6837\u7684\u4e1c\u897f\uff0c\u56e0\u4e3a\u5b83\u66f4\u5bb9\u6613\u5b9e\u73b0\u3002</p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span> token embedding </p>\n": "<p><span translate=no>_^_0_^_</span>\u4ee4\u724c\u5d4c\u5165</p>\n",
|
||||
"<p>Activation </p>\n": "<p>\u6fc0\u6d3b</p>\n",
|
||||
"<p>Add positional embeddings </p>\n": "<p>\u6dfb\u52a0\u4f4d\u7f6e\u5d4c\u5165</p>\n",
|
||||
"<p>Add to patch embeddings and return </p>\n": "<p>\u6dfb\u52a0\u5230\u8865\u4e01\u5d4c\u5165\u5e76\u8fd4\u56de</p>\n",
|
||||
"<p>Apply convolution layer </p>\n": "<p>\u5e94\u7528\u5377\u79ef\u5c42</p>\n",
|
||||
"<p>Classification head </p>\n": "<p>\u5206\u7c7b\u4e3b\u7ba1</p>\n",
|
||||
"<p>Classification head, to get logits </p>\n": "<p>\u5206\u7c7b\u5934\uff0c\u83b7\u53d6\u65e5\u5fd7</p>\n",
|
||||
"<p>Concatenate the <span translate=no>_^_0_^_</span> token embeddings before feeding the transformer </p>\n": "<p>\u5728\u7ed9\u53d8\u538b\u5668\u4f9b\u7535\u4e4b\u524d\u8fde\u63a5<span translate=no>_^_0_^_</span>\u4ee4\u724c\u5d4c\u5165</p>\n",
|
||||
"<p>Final normalization layer </p>\n": "<p>\u6700\u7ec8\u5f52\u4e00\u5316\u5c42</p>\n",
|
||||
"<p>First layer </p>\n": "<p>\u7b2c\u4e00\u5c42</p>\n",
|
||||
"<p>First layer and activation </p>\n": "<p>\u7b2c\u4e00\u5c42\u548c\u6fc0\u6d3b</p>\n",
|
||||
"<p>Get patch embeddings. This gives a tensor of shape <span translate=no>_^_0_^_</span> </p>\n": "<p>\u83b7\u53d6\u8865\u4e01\u5d4c\u5165\u3002\u8fd9\u7ed9\u51fa\u4e86\u5f62\u72b6\u7684\u5f20\u91cf<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Get the positional embeddings for the given patches </p>\n": "<p>\u83b7\u53d6\u7ed9\u5b9a\u8865\u4e01\u7684\u4f4d\u7f6e\u5d4c\u5165</p>\n",
|
||||
"<p>Get the shape. </p>\n": "<p>\u5f97\u5230\u5f62\u72b6\u3002</p>\n",
|
||||
"<p>Get the transformer output of the <span translate=no>_^_0_^_</span> token (which is the first in the sequence). </p>\n": "<p>\u83b7\u53d6<span translate=no>_^_0_^_</span>\u4ee4\u724c\u7684\u8f6c\u6362\u5668\u8f93\u51fa\uff08\u5e8f\u5217\u4e2d\u7684\u7b2c\u4e00\u4e2a\uff09\u3002</p>\n",
|
||||
"<p>Layer normalization </p>\n": "<p>\u5c42\u89c4\u8303\u5316</p>\n",
|
||||
"<p>Make copies of the transformer layer </p>\n": "<p>\u5236\u4f5c\u53d8\u538b\u5668\u5c42\u7684\u526f\u672c</p>\n",
|
||||
"<p>Pass through transformer layers with no attention masking </p>\n": "<p>\u7a7f\u8fc7\u53d8\u538b\u5668\u5c42\uff0c\u4e0d\u906e\u6321\u6ce8\u610f\u529b</p>\n",
|
||||
"<p>Patch embeddings </p>\n": "<p>\u8865\u4e01\u5d4c\u5165</p>\n",
|
||||
"<p>Positional embeddings for each location </p>\n": "<p>\u6bcf\u4e2a\u4f4d\u7f6e\u7684\u4f4d\u7f6e\u5d4c\u5165</p>\n",
|
||||
"<p>Rearrange to shape <span translate=no>_^_0_^_</span> </p>\n": "<p>\u91cd\u65b0\u6392\u5217\u6210\u5f62\u72b6<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Return the patch embeddings </p>\n": "<p>\u8fd4\u56de\u8865\u4e01\u5d4c\u5165</p>\n",
|
||||
"<p>Second layer </p>\n": "<p>\u7b2c\u4e8c\u5c42</p>\n",
|
||||
"<p>We create a convolution layer with a kernel size and and stride length equal to patch size. This is equivalent to splitting the image into patches and doing a linear transformation on each patch. </p>\n": "<p>\u6211\u4eec\u521b\u5efa\u4e00\u4e2a\u5377\u79ef\u5c42\uff0c\u5176\u5185\u6838\u5927\u5c0f\u548c\u6b65\u957f\u7b49\u4e8e\u8865\u4e01\u5927\u5c0f\u3002\u8fd9\u76f8\u5f53\u4e8e\u5c06\u56fe\u50cf\u5206\u5272\u6210\u8272\u5757\u5e76\u5728\u6bcf\u4e2a\u9762\u7247\u4e0a\u8fdb\u884c\u7ebf\u6027\u53d8\u6362\u3002</p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is a copy of a single <a href=\"../models.html#TransformerLayer\">transformer layer</a>. We make copies of it to make the transformer with <span translate=no>_^_1_^_</span>. </li>\n<li><span translate=no>_^_2_^_</span> is the number of <a href=\"../models.html#TransformerLayer\">transformer layers</a>. </li>\n<li><span translate=no>_^_3_^_</span> is the <a href=\"#PatchEmbeddings\">patch embeddings layer</a>. </li>\n<li><span translate=no>_^_4_^_</span> is the <a href=\"#LearnedPositionalEmbeddings\">positional embeddings layer</a>. </li>\n<li><span translate=no>_^_5_^_</span> is the <a href=\"#ClassificationHead\">classification head</a>.</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u662f\u5355\u4e2a<a href=\"../models.html#TransformerLayer\">\u53d8\u538b\u5668\u5c42</a>\u7684\u526f\u672c\u3002\u6211\u4eec\u5236\u4f5c\u4e86\u5b83\u7684\u526f\u672c\u6765\u5236\u4f5c\u53d8\u538b\u5668<span translate=no>_^_1_^_</span>\u3002</li>\n<li><span translate=no>_^_2_^_</span>\u662f\u53d8<a href=\"../models.html#TransformerLayer\">\u538b\u5668\u5c42</a>\u7684\u6570\u91cf\u3002</li>\n<li><span translate=no>_^_3_^_</span>\u662f<a href=\"#PatchEmbeddings\">\u8865\u4e01\u5d4c\u5165\u5c42</a>\u3002</li>\n<li><span translate=no>_^_4_^_</span>\u662f<a href=\"#LearnedPositionalEmbeddings\">\u4f4d\u7f6e\u5d4c\u5165\u5c42</a>\u3002</li>\n<li><span translate=no>_^_5_^_</span>\u662f<a href=\"#ClassificationHead\">\u5206\u7c7b\u5934</a>\u3002</li></ul>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the input image of shape <span translate=no>_^_1_^_</span></li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u662f\u5f62\u72b6\u7684\u8f93\u5165\u56fe\u50cf<span translate=no>_^_1_^_</span></li></ul>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the patch embeddings of shape <span translate=no>_^_1_^_</span></li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u662f\u5f62\u72b6\u7684\u8865\u4e01\u5d4c\u5165<span translate=no>_^_1_^_</span></li></ul>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the transformer embedding size </li>\n<li><span translate=no>_^_1_^_</span> is the size of the hidden layer </li>\n<li><span translate=no>_^_2_^_</span> is the number of classes in the classification task</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u53d8\u538b\u5668\u5d4c\u5165\u7684\u5927\u5c0f\u662f\u591a\u5c11</li>\n<li><span translate=no>_^_1_^_</span>\u662f\u9690\u85cf\u5c42\u7684\u5927\u5c0f</li>\n<li><span translate=no>_^_2_^_</span>\u662f\u5206\u7c7b\u4efb\u52a1\u4e2d\u7684\u7c7b\u6570</li></ul>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the transformer embeddings size </li>\n<li><span translate=no>_^_1_^_</span> is the maximum number of patches</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u53d8\u538b\u5668\u5d4c\u5165\u7684\u5927\u5c0f\u662f\u591a\u5c11</li>\n<li><span translate=no>_^_1_^_</span>\u662f\u8865\u4e01\u7684\u6700\u5927\u6570\u91cf</li></ul>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the transformer embeddings size </li>\n<li><span translate=no>_^_1_^_</span> is the size of the patch </li>\n<li><span translate=no>_^_2_^_</span> is the number of channels in the input image (3 for rgb)</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u53d8\u538b\u5668\u5d4c\u5165\u7684\u5927\u5c0f\u662f\u591a\u5c11</li>\n<li><span translate=no>_^_1_^_</span>\u662f\u8865\u4e01\u7684\u5927\u5c0f</li>\n<li><span translate=no>_^_2_^_</span>\u662f\u8f93\u5165\u56fe\u50cf\u4e2d\u7684\u901a\u9053\u6570\uff08rgb \u4e3a 3\uff09</li></ul>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the transformer encoding for <span translate=no>_^_1_^_</span> token</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u662f<span translate=no>_^_1_^_</span>\u4ee4\u724c\u7684\u8f6c\u6362\u5668\u7f16\u7801</li></ul>\n",
|
||||
"A PyTorch implementation/tutorial of the paper \"An Image Is Worth 16x16 Words: Transformers For Image Recognition At Scale\"": "\u8bba\u6587 \u201c\u4e00\u5f20\u56fe\u50cf\u503c\u5f97 16x16 \u5b57\uff1a\u7528\u4e8e\u5927\u89c4\u6a21\u56fe\u50cf\u8bc6\u522b\u7684\u53d8\u5f62\u91d1\u521a\u201d \u7684 PyTorch \u5b9e\u73b0/\u6559\u7a0b",
|
||||
"Vision Transformer (ViT)": "\u89c6\u89c9\u53d8\u538b\u5668 (ViT)"
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"<h1>Train a <a href=\"index.html\">Vision Transformer (ViT)</a> on CIFAR 10</h1>\n": "<h1>CIFAR 10 <a href=\"index.html\">\u3067\u30d3\u30b8\u30e7\u30f3\u30c8\u30e9\u30f3\u30b9\u30d5\u30a9\u30fc\u30de\u30fc (VIT)</a> \u3092\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3057\u307e\u3057\u3087\u3046</h1>\n",
|
||||
"<h2>Configurations</h2>\n<p>We use <a href=\"../../experiments/cifar10.html\"><span translate=no>_^_0_^_</span></a> which defines all the dataset related configurations, optimizer, and a training loop.</p>\n": "<h2>\u30b3\u30f3\u30d5\u30a3\u30ae\u30e5\u30ec\u30fc\u30b7\u30e7\u30f3</h2>\n<p>\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306b\u95a2\u9023\u3059\u308b\u3059\u3079\u3066\u306e\u69cb\u6210\u3001\u30aa\u30d7\u30c6\u30a3\u30de\u30a4\u30b6\u30fc\u3001\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30eb\u30fc\u30d7\u3092\u5b9a\u7fa9\u3059\u308b\u3082\u306e\u3092\u4f7f\u7528\u3057\u3066\u3044\u307e\u3059<a href=\"../../experiments/cifar10.html\"><span translate=no>_^_0_^_</span></a>\u3002</p>\n",
|
||||
"<h3>Create model</h3>\n": "<h3>\u30e2\u30c7\u30eb\u4f5c\u6210</h3>\n",
|
||||
"<p> </p>\n": "<p></p>\n",
|
||||
"<p> Create transformer configs</p>\n": "<p>\u30c8\u30e9\u30f3\u30b9\u30d5\u30a9\u30fc\u30de\u30fc\u69cb\u6210\u306e\u4f5c\u6210</p>\n",
|
||||
"<p><a href=\"../configs.html#TransformerConfigs\">Transformer configurations</a> to get <a href=\"../models.html#TransformerLayer\">transformer layer</a> </p>\n": "<p><a href=\"../configs.html#TransformerConfigs\"><a href=\"../models.html#TransformerLayer\">\u5909\u5727\u5668\u5c64\u3092\u53d6\u5f97\u3059\u308b\u305f\u3081\u306e\u5909\u5727\u5668\u69cb\u6210</a></a></p>\n",
|
||||
"<p>Augment CIFAR 10 images for training </p>\n": "<p>\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u7528\u306e CIFAR 10 \u30a4\u30e1\u30fc\u30b8\u306e\u62e1\u5f35</p>\n",
|
||||
"<p>Create a vision transformer </p>\n": "<p>\u30d3\u30b8\u30e7\u30f3\u30c8\u30e9\u30f3\u30b9\u30d5\u30a9\u30fc\u30de\u30fc\u306e\u4f5c\u6210</p>\n",
|
||||
"<p>Create configurations </p>\n": "<p>\u69cb\u6210\u306e\u4f5c\u6210</p>\n",
|
||||
"<p>Create experiment </p>\n": "<p>\u5b9f\u9a13\u3092\u4f5c\u6210</p>\n",
|
||||
"<p>Do not augment CIFAR 10 images for validation </p>\n": "<p>\u691c\u8a3c\u7528\u306b CIFAR 10 \u30a4\u30e1\u30fc\u30b8\u3092\u62e1\u5f35\u3057\u306a\u3044\u3067\u304f\u3060\u3055\u3044</p>\n",
|
||||
"<p>Load configurations </p>\n": "<p>\u69cb\u6210\u3092\u30ed\u30fc\u30c9</p>\n",
|
||||
"<p>Number of classes in the task </p>\n": "<p>\u30bf\u30b9\u30af\u5185\u306e\u30af\u30e9\u30b9\u6570</p>\n",
|
||||
"<p>Optimizer </p>\n": "<p>\u30aa\u30d7\u30c6\u30a3\u30de\u30a4\u30b6\u30fc</p>\n",
|
||||
"<p>Set model for saving/loading </p>\n": "<p>\u4fdd\u5b58/\u8aad\u307f\u8fbc\u307f\u7528\u306e\u30e2\u30c7\u30eb\u3092\u8a2d\u5b9a</p>\n",
|
||||
"<p>Size of a patch </p>\n": "<p>\u30d1\u30c3\u30c1\u306e\u30b5\u30a4\u30ba</p>\n",
|
||||
"<p>Size of the hidden layer in classification head </p>\n": "<p>\u5206\u985e\u30d8\u30c3\u30c9\u306e\u96a0\u308c\u5c64\u306e\u30b5\u30a4\u30ba</p>\n",
|
||||
"<p>Start the experiment and run the training loop </p>\n": "<p>\u5b9f\u9a13\u3092\u958b\u59cb\u3057\u3001\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30eb\u30fc\u30d7\u3092\u5b9f\u884c\u3057\u307e\u3059</p>\n",
|
||||
"<p>Training epochs and batch size </p>\n": "<p>\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30a8\u30dd\u30c3\u30af\u3068\u30d0\u30c3\u30c1\u30b5\u30a4\u30ba</p>\n",
|
||||
"<p>Transformer embedding size </p>\n": "<p>\u5909\u5727\u5668\u57cb\u3081\u8fbc\u307f\u30b5\u30a4\u30ba</p>\n",
|
||||
"<p>Transformer size from <a href=\"../configs.html#TransformerConfigs\">Transformer configurations</a> </p>\n": "<p><a href=\"../configs.html#TransformerConfigs\">\u30c8\u30e9\u30f3\u30b9\u30d5\u30a9\u30fc\u30de\u30fc\u69cb\u6210\u304b\u3089\u898b\u305f\u30c8\u30e9\u30f3\u30b9\u30b5\u30a4\u30ba</a></p>\n",
|
||||
"Train a Vision Transformer (ViT) on CIFAR 10": "CIFAR 10 \u3067\u30d3\u30b8\u30e7\u30f3\u30c8\u30e9\u30f3\u30b9\u30d5\u30a9\u30fc\u30de\u30fc (VIT) \u3092\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3057\u307e\u3057\u3087\u3046"
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"<h1>Train a <a href=\"index.html\">Vision Transformer (ViT)</a> on CIFAR 10</h1>\n<p><a href=\"https://app.labml.ai/run/afdd5332188b11edbdf543360515b595\"><span translate=no>_^_0_^_</span></a></p>\n": "<h1>CIFAR10 \u0db8\u0dad <a href=\"index.html\">\u0daf\u0dbb\u0dca\u0dc1\u0db1 \u0da7\u0dca\u0dbb\u0dcf\u0db1\u0dca\u0dc3\u0dca\u0dc6\u0ddd\u0db8\u0dbb\u0dca (VIT)</a> \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0d9a\u0dbb\u0db1\u0dca\u0db1</h1>\n<p><a href=\"https://app.labml.ai/run/afdd5332188b11edbdf543360515b595\"><span translate=no>_^_0_^_</span></a></p>\n",
|
||||
"<h2>Configurations</h2>\n<p>We use <a href=\"../../experiments/cifar10.html\"><span translate=no>_^_0_^_</span></a> which defines all the dataset related configurations, optimizer, and a training loop.</p>\n": "<h2>\u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3\u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dca</h2>\n<p>\u0dc3\u0dd2\u0dba\u0dbd\u0dd4\u0db8\u0daf\u0dad\u0dca\u0dad \u0d9a\u0da7\u0dca\u0da7\u0dbd \u0d86\u0dc1\u0dca\u0dbb\u0dd2\u0dad \u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3\u0dba\u0db1\u0dca, \u0db4\u0dca\u0dbb\u0dc1\u0dc3\u0dca\u0dad\u0d9a\u0dbb\u0dab\u0dba \u0dc3\u0dc4 \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0dbd\u0dd6\u0db4\u0dba\u0d9a\u0dca \u0db1\u0dd2\u0dbb\u0dca\u0dc0\u0da0\u0db1\u0dba \u0d9a\u0dbb\u0db1 \u0d85\u0db4\u0dd2 \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf <a href=\"../../experiments/cifar10.html\"><span translate=no>_^_0_^_</span></a> \u0d9a\u0dbb\u0db8\u0dd4. </p>\n",
|
||||
"<h3>Create model</h3>\n": "<h3>\u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba\u0dc3\u0dcf\u0daf\u0db1\u0dca\u0db1</h3>\n",
|
||||
"<p> </p>\n": "<p> </p>\n",
|
||||
"<p> Create transformer configs</p>\n": "<p> \u0da7\u0dca\u0dbb\u0dcf\u0db1\u0dca\u0dc3\u0dca\u0dc6\u0ddd\u0db8\u0dbb\u0dca\u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3 \u0dc3\u0dcf\u0daf\u0db1\u0dca\u0db1</p>\n",
|
||||
"<p><a href=\"../configs.html#TransformerConfigs\">Transformer configurations</a> to get <a href=\"../models.html#TransformerLayer\">transformer layer</a> </p>\n": "<p><a href=\"../configs.html#TransformerConfigs\">\u0da7\u0dca\u0dbb\u0dcf\u0db1\u0dca\u0dc3\u0dca\u0dc6\u0ddd\u0db8\u0dbb\u0dca <a href=\"../models.html#TransformerLayer\">\u0dc3\u0dca\u0dae\u0dbb\u0dba \u0dbd\u0db6\u0dcf \u0d9c\u0dd0\u0db1\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0da7\u0dca\u0dbb\u0dcf\u0db1\u0dca\u0dc3\u0dca\u0dc6\u0ddd\u0db8\u0dbb\u0dca</a> \u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3\u0dba\u0db1\u0dca</a> </p>\n",
|
||||
"<p>Augment CIFAR 10 images for training </p>\n": "<p>\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0d9a\u0dd2\u0dbb\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf CIFAR 10 \u0dbb\u0dd6\u0db4 </p>\n",
|
||||
"<p>Create a vision transformer </p>\n": "<p>\u0daf\u0dbb\u0dca\u0dc1\u0db1\u0da7\u0dca\u0dbb\u0dcf\u0db1\u0dca\u0dc3\u0dca\u0dc6\u0ddd\u0db8\u0dbb\u0dba\u0d9a\u0dca \u0dc3\u0dcf\u0daf\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Create configurations </p>\n": "<p>\u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3\u0dba\u0db1\u0dca\u0dc3\u0dcf\u0daf\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Create experiment </p>\n": "<p>\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf\u0db6\u0dd0\u0dbd\u0dd3\u0db8 \u0dc3\u0dcf\u0daf\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Do not augment CIFAR 10 images for validation </p>\n": "<p>CIFAR\u0dc0\u0dbb\u0dca\u0db0\u0db1\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1 \u0d91\u0db4\u0dcf 10 \u0dc0\u0dbd\u0d82\u0d9c\u0dd4 \u0d9a\u0dd2\u0dbb\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0dbb\u0dd6\u0db4 </p>\n",
|
||||
"<p>Load configurations </p>\n": "<p>\u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3\u0dba\u0db1\u0dca\u0db4\u0dd6\u0dbb\u0dab\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Number of classes in the task </p>\n": "<p>\u0d9a\u0dbb\u0dca\u0dad\u0dc0\u0dca\u0dba\u0dba\u0dda\u0db4\u0db1\u0dca\u0dad\u0dd2 \u0d9c\u0dab\u0db1 </p>\n",
|
||||
"<p>Optimizer </p>\n": "<p>\u0db4\u0dca\u0dbb\u0dc1\u0dc3\u0dca\u0dad\u0d9a\u0dbb\u0dab\u0dba </p>\n",
|
||||
"<p>Set model for saving/loading </p>\n": "<p>\u0d89\u0dad\u0dd2\u0dbb\u0dd2\u0d9a\u0dd2\u0dbb\u0dd3\u0db8/\u0db4\u0dd0\u0da7\u0dc0\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba \u0dc3\u0d9a\u0dc3\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Size of a patch </p>\n": "<p>\u0db4\u0dd0\u0da0\u0dca\u0d91\u0d9a\u0d9a \u0db4\u0dca\u0dbb\u0db8\u0dcf\u0dab\u0dba </p>\n",
|
||||
"<p>Size of the hidden layer in classification head </p>\n": "<p>\u0dc0\u0dbb\u0dca\u0d9c\u0dd3\u0d9a\u0dbb\u0dab\u0dc4\u0dd2\u0dc3\u0dd9\u0dc4\u0dd2 \u0dc3\u0dd0\u0d9f\u0dc0\u0dd4\u0dab\u0dd4 \u0dc3\u0dca\u0dae\u0dbb\u0dba\u0dda \u0db4\u0dca\u0dbb\u0db8\u0dcf\u0dab\u0dba </p>\n",
|
||||
"<p>Start the experiment and run the training loop </p>\n": "<p>\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf\u0db6\u0dd0\u0dbd\u0dd3\u0db8 \u0d86\u0dbb\u0db8\u0dca\u0db7 \u0d9a\u0dbb \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0dbd\u0dd6\u0db4\u0dba \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Training epochs and batch size </p>\n": "<p>\u0d8a\u0db4\u0ddc\u0da0\u0dca\u0dc3\u0dc4 \u0d9a\u0dab\u0dca\u0da9\u0dcf\u0dba\u0db8\u0dca \u0db4\u0dca\u0dbb\u0db8\u0dcf\u0dab\u0dba \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0d9a\u0dd2\u0dbb\u0dd3\u0db8 </p>\n",
|
||||
"<p>Transformer embedding size </p>\n": "<p>\u0da7\u0dca\u0dbb\u0dcf\u0db1\u0dca\u0dc3\u0dca\u0dc6\u0ddd\u0db8\u0dbb\u0dca\u0d9a\u0dcf\u0dc0\u0dd0\u0daf\u0dca\u0daf\u0dd3\u0db8\u0dda \u0db4\u0dca\u0dbb\u0db8\u0dcf\u0dab\u0dba </p>\n",
|
||||
"<p>Transformer size from <a href=\"../configs.html#TransformerConfigs\">Transformer configurations</a> </p>\n": "<p>\u0da7\u0dca\u0dbb\u0dcf\u0db1\u0dca\u0dc3\u0dca\u0dc6\u0ddd\u0db8\u0dbb\u0dca <a href=\"../configs.html#TransformerConfigs\">\u0db8\u0dcf\u0db1\u0d9a\u0dbb\u0dab \u0dc3\u0dd2\u0da7 \u0da7\u0dca\u0dbb\u0dcf\u0db1\u0dca\u0dc3\u0dca\u0dc6\u0ddd\u0db8\u0dbb\u0dca</a> \u0db4\u0dca\u0dbb\u0db8\u0dcf\u0dab\u0dba </p>\n",
|
||||
"Train a Vision Transformer (ViT) on CIFAR 10": "CIFAR 10 \u0db8\u0dad \u0daf\u0dbb\u0dca\u0dc1\u0db1 \u0da7\u0dca\u0dbb\u0dcf\u0db1\u0dca\u0dc3\u0dca\u0dc6\u0ddd\u0db8\u0dbb\u0dca (VIT) \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0d9a\u0dbb\u0db1\u0dca\u0db1"
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
{
|
||||
"<h1>Train a <a href=\"index.html\">Vision Transformer (ViT)</a> on CIFAR 10</h1>\n": "<h1>\u5728 CIFA <a href=\"index.html\">R 10 \u4e0a\u8bad\u7ec3\u89c6\u89c9\u53d8\u538b\u5668 (ViT)</a></h1>\n",
|
||||
"<h2>Configurations</h2>\n<p>We use <a href=\"../../experiments/cifar10.html\"><span translate=no>_^_0_^_</span></a> which defines all the dataset related configurations, optimizer, and a training loop.</p>\n": "<h2>\u914d\u7f6e</h2>\n<p>\u6211\u4eec\u4f7f\u7528<a href=\"../../experiments/cifar10.html\"><span translate=no>_^_0_^_</span></a>\u5b83\u6765\u5b9a\u4e49\u6240\u6709\u4e0e\u6570\u636e\u96c6\u76f8\u5173\u7684\u914d\u7f6e\u3001\u4f18\u5316\u5668\u548c\u8bad\u7ec3\u5faa\u73af\u3002</p>\n",
|
||||
"<h3>Create model</h3>\n": "<h3>\u521b\u5efa\u6a21\u578b</h3>\n",
|
||||
"<p> </p>\n": "<p></p>\n",
|
||||
"<p> Create transformer configs</p>\n": "<p>\u521b\u5efa\u53d8\u538b\u5668\u914d\u7f6e</p>\n",
|
||||
"<p><a href=\"../configs.html#TransformerConfigs\">Transformer configurations</a> to get <a href=\"../models.html#TransformerLayer\">transformer layer</a> </p>\n": "<p>\u83b7\u5f97<a href=\"../configs.html#TransformerConfigs\">\u53d8\u538b\u5668<a href=\"../models.html#TransformerLayer\">\u5c42\u7684\u53d8\u538b\u5668</a>\u914d\u7f6e</a></p>\n",
|
||||
"<p>Augment CIFAR 10 images for training </p>\n": "<p>\u589e\u5f3a CIFAR 10 \u56fe\u50cf\u7528\u4e8e\u8bad\u7ec3</p>\n",
|
||||
"<p>Create a vision transformer </p>\n": "<p>\u521b\u5efa\u89c6\u89c9\u53d8\u538b\u5668</p>\n",
|
||||
"<p>Create configurations </p>\n": "<p>\u521b\u5efa\u914d\u7f6e</p>\n",
|
||||
"<p>Create experiment </p>\n": "<p>\u521b\u5efa\u5b9e\u9a8c</p>\n",
|
||||
"<p>Do not augment CIFAR 10 images for validation </p>\n": "<p>\u4e0d\u8981\u6269\u5927 CIFAR 10 \u56fe\u50cf\u8fdb\u884c\u9a8c\u8bc1</p>\n",
|
||||
"<p>Load configurations </p>\n": "<p>\u88c5\u8f7d\u914d\u7f6e</p>\n",
|
||||
"<p>Number of classes in the task </p>\n": "<p>\u4efb\u52a1\u4e2d\u7684\u7c7b\u6570</p>\n",
|
||||
"<p>Optimizer </p>\n": "<p>\u4f18\u5316\u5668</p>\n",
|
||||
"<p>Set model for saving/loading </p>\n": "<p>\u8bbe\u7f6e\u4fdd\u5b58/\u52a0\u8f7d\u7684\u6a21\u578b</p>\n",
|
||||
"<p>Size of a patch </p>\n": "<p>\u8865\u4e01\u7684\u5927\u5c0f</p>\n",
|
||||
"<p>Size of the hidden layer in classification head </p>\n": "<p>\u5206\u7c7b\u5934\u4e2d\u9690\u85cf\u5c42\u7684\u5927\u5c0f</p>\n",
|
||||
"<p>Start the experiment and run the training loop </p>\n": "<p>\u5f00\u59cb\u5b9e\u9a8c\u5e76\u8fd0\u884c\u8bad\u7ec3\u5faa\u73af</p>\n",
|
||||
"<p>Training epochs and batch size </p>\n": "<p>\u8bad\u7ec3\u5468\u671f\u548c\u6279\u6b21\u5927\u5c0f</p>\n",
|
||||
"<p>Transformer embedding size </p>\n": "<p>\u53d8\u538b\u5668\u5d4c\u5165\u5c3a\u5bf8</p>\n",
|
||||
"<p>Transformer size from <a href=\"../configs.html#TransformerConfigs\">Transformer configurations</a> </p>\n": "<p>\u53d8\u538b\u5668<a href=\"../configs.html#TransformerConfigs\">\u914d\u7f6e\u4e2d\u7684\u53d8\u538b\u5668</a>\u5c3a\u5bf8</p>\n",
|
||||
"Train a Vision Transformer (ViT) on CIFAR 10": "\u5728 CIFAR 10 \u4e0a\u8bad\u7ec3\u89c6\u89c9\u53d8\u538b\u5668 (ViT)"
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user