wehub/labmlai--annotated_deep_learning_paper_implementations
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity

chore: import upstream snapshot with attribution

Browse Source
This commit is contained in:
wehub-resource-sync
2026-07-13 12:19:01 +08:00
commit 3b90d1192f
2172 changed files with 594509 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
translate_cache/conv_mixer/__init__.ja.json
+40
View File
@@ -0,0 +1,40 @@
{
"<h1>Patches Are All You Need? (ConvMixer)</h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of the paper <a href=\"https://arxiv.org/abs/2201.09792\">Patches Are All You Need?</a>.</p>\n<p><span translate=no>_^_0_^_</span></p>\n<p>ConvMixer is Similar to <a href=\"../transformers/mlp_mixer/index.html\">MLP-Mixer</a>. MLP-Mixer separates mixing of spatial and channel dimensions, by applying an MLP across spatial dimension and then an MLP across the channel dimension (spatial MLP replaces the <a href=\"../transformers/vit/index.html\">ViT</a> attention and channel MLP is the <a href=\"../transformers/feed_forward.html\">FFN</a> of ViT).</p>\n<p>ConvMixer uses a <span translate=no>_^_1_^_</span> convolution for channel mixing and a depth-wise convolution for spatial mixing. Since it&#x27;s a convolution instead of a full MLP across the space, it mixes only the nearby batches in contrast to ViT or MLP-Mixer. Also, the MLP-mixer uses MLPs of two layers for each mixing and ConvMixer uses a single layer for each mixing.</p>\n<p>The paper recommends removing the residual connection across the channel mixing (point-wise convolution) and having only a residual connection over the spatial mixing (depth-wise convolution). They also use <a href=\"../normalization/batch_norm/index.html\">Batch normalization</a> instead of <a href=\"../normalization/layer_norm/index.html\">Layer normalization</a>.</p>\n<p>Here&#x27;s <a href=\"experiment.html\">an experiment</a> that trains ConvMixer on CIFAR-10.</p>\n": "<h1>\u5fc5\u8981\u306a\u306e\u306f\u30d1\u30c3\u30c1\u3060\u3051\uff1f(\u30b3\u30f3\u30d0\u30fc\u30b8\u30e7\u30f3\u30df\u30ad\u30b5\u30fc</h1>)\n<p><a href=\"https://pytorch.org\">\u3053\u308c\u306f\u7d19\u306e\u30d1\u30c3\u30c1\u3092PyTorch\u3067\u5b9f\u88c5\u3057\u305f\u3082\u306e\u3067\u3059</a><a href=\"https://arxiv.org/abs/2201.09792\">\u3002\u5fc5\u8981\u306a\u306e\u306f\u30d1\u30c3\u30c1\u3060\u3051\u3067\u3059\u304b</a>\uff1f</p>\u3002\n<p><span translate=no>_^_0_^_</span></p>\n<p><a href=\"../transformers/mlp_mixer/index.html\">ConvMixer\u306fMLP\u30df\u30ad\u30b5\u30fc\u306b\u4f3c\u3066\u3044\u307e\u3059\u3002</a></p><a href=\"../transformers/feed_forward.html\">MLP-Mixer\u306f\u3001\u7a7a\u9593\u6b21\u5143\u5168\u4f53\u306bMLP\u3092\u9069\u7528\u3057\u3001\u6b21\u306b\u30c1\u30e3\u30cd\u30eb\u6b21\u5143\u5168\u4f53\u306bMLP\u3092\u9069\u7528\u3059\u308b\u3053\u3068\u3067\u3001\u7a7a\u9593\u6b21\u5143\u3068\u30c1\u30e3\u30cd\u30eb\u6b21\u5143\u306e\u6df7\u5408\u3092\u5206\u96e2\u3057\u307e\u3059\uff08\u7a7a\u9593MLP\u306fvIT\u306e\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u306b\u4ee3\u308f\u308a\u3001<a href=\"../transformers/vit/index.html\">\u30c1\u30e3\u30cd\u30ebMLP\u306fVIT\u306eFFN\u3067\u3059</a>\uff09\u3002</a>\n<p>ConvMixer\u306f\u3001<span translate=no>_^_1_^_</span>\u30c1\u30e3\u30f3\u30cd\u30eb\u30df\u30ad\u30b7\u30f3\u30b0\u306b\u306f\u30b3\u30f3\u30dc\u30ea\u30e5\u30fc\u30b7\u30e7\u30f3\u3092\u4f7f\u7528\u3057\u3001\u7a7a\u9593\u30df\u30ad\u30b7\u30f3\u30b0\u306b\u306f\u5965\u884c\u304d\u30b3\u30f3\u30dc\u30ea\u30e5\u30fc\u30b7\u30e7\u30f3\u3092\u4f7f\u7528\u3057\u307e\u3059\u3002\u30b9\u30da\u30fc\u30b9\u5168\u4f53\u3067\u30d5\u30ebMLP\u3067\u306f\u306a\u304f\u7573\u307f\u8fbc\u307f\u306a\u306e\u3067\u3001VIT\u3084MLP\u30df\u30ad\u30b5\u30fc\u3068\u306f\u5bfe\u7167\u7684\u306b\u3001\u8fd1\u304f\u306e\u30d0\u30c3\u30c1\u306e\u307f\u3092\u30df\u30ad\u30b7\u30f3\u30b0\u3057\u307e\u3059\u3002\u307e\u305f\u3001MLP\u30df\u30ad\u30b5\u30fc\u306f\u30df\u30ad\u30b7\u30f3\u30b0\u3054\u3068\u306b2\u5c64\u306eMLP\u3092\u4f7f\u7528\u3057\u3001ConvMixer\u306f\u30df\u30ad\u30b7\u30f3\u30b0\u3054\u3068\u306b1\u5c64\u306eMLP\u3092\u4f7f\u7528\u3057\u307e\u3059</p>\u3002\n<p>\u3053\u306e\u8ad6\u6587\u3067\u306f\u3001\u30c1\u30e3\u30cd\u30eb\u30df\u30ad\u30b7\u30f3\u30b0\u5168\u4f53\u306e\u6b8b\u7559\u63a5\u7d9a\u3092\u524a\u9664\u3057\uff08\u70b9\u5358\u4f4d\u306e\u7573\u307f\u8fbc\u307f\uff09\u3001\u7a7a\u9593\u30df\u30ad\u30b7\u30f3\u30b0\u3067\u306f\u6b8b\u7559\u63a5\u7d9a\u306e\u307f\u306b\u3059\u308b\uff08\u6df1\u3055\u65b9\u5411\u306e\u7573\u307f\u8fbc\u307f\uff09\u3053\u3068\u3092\u63a8\u5968\u3057\u3066\u3044\u307e\u3059\u3002\u307e\u305f\u3001</p><a href=\"../normalization/batch_norm/index.html\"><a href=\"../normalization/layer_norm/index.html\">\u30ec\u30a4\u30e4\u30fc\u6b63\u898f\u5316\u306e\u4ee3\u308f\u308a\u306b\u30d0\u30c3\u30c1\u6b63\u898f\u5316\u3092\u4f7f\u7528\u3057\u307e\u3059</a></a>\u3002\n<p>\u3053\u308c\u306f<a href=\"experiment.html\">\u3001CIFAR-10 \u3067 ConvMixer \u3092\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3059\u308b\u5b9f\u9a13\u3067\u3059</a>\u3002</p>\n",
"<h2>ConvMixer</h2>\n<p>This combines the patch embeddings block, a number of ConvMixer layers and a classification head.</p>\n": "<h2>\u30b3\u30f3\u30d0\u30fc\u30b8\u30e7\u30f3\u30df\u30ad\u30b5\u30fc</h2>\n<p>\u3053\u308c\u306b\u3088\u308a\u3001\u30d1\u30c3\u30c1\u57cb\u3081\u8fbc\u307f\u30d6\u30ed\u30c3\u30af\u3001\u591a\u6570\u306e ConvMixer \u30ec\u30a4\u30e4\u30fc\u3001\u304a\u3088\u3073\u5206\u985e\u30d8\u30c3\u30c9\u304c\u7d44\u307f\u5408\u308f\u3055\u308c\u307e\u3059\u3002</p>\n",
"<p> </p>\n": "<p></p>\n",
"<p> <a id=\"ClassificationHead\"></a></p>\n<h2>Classification Head</h2>\n<p>They do average pooling (taking the mean of all patch embeddings) and a final linear transformation to predict the log-probabilities of the image classes.</p>\n": "<p><a id=\"ClassificationHead\"></a></p>\n<h2>\u5206\u985e\u8cac\u4efb\u8005</h2>\n<p>\u5e73\u5747\u30d7\u30fc\u30ea\u30f3\u30b0\uff08\u3059\u3079\u3066\u306e\u30d1\u30c3\u30c1\u57cb\u3081\u8fbc\u307f\u306e\u5e73\u5747\u3092\u53d6\u308b\uff09\u3068\u6700\u7d42\u7684\u306a\u7dda\u5f62\u5909\u63db\u3092\u884c\u3063\u3066\u3001\u753b\u50cf\u30af\u30e9\u30b9\u306e\u5bfe\u6570\u78ba\u7387\u3092\u4e88\u6e2c\u3057\u307e\u3059\u3002</p>\n",
"<p> <a id=\"ConvMixerLayer\"></a></p>\n<h2>ConvMixer layer</h2>\n<p>This is a single ConvMixer layer. The model will have a series of these.</p>\n": "<p><a id=\"ConvMixerLayer\"></a></p>\n<h2>ConvMixer \u30ec\u30a4\u30e4\u30fc</h2>\n<p>\u3053\u308c\u306f\u5358\u4e00\u306e ConvMixer \u30ec\u30a4\u30e4\u30fc\u3067\u3059\u3002\u30e2\u30c7\u30eb\u306b\u306f\u3053\u308c\u3089\u306e\u30b7\u30ea\u30fc\u30ba\u304c\u3042\u308a\u307e\u3059\u3002</p>\n",
"<p> <a id=\"PatchEmbeddings\"></a></p>\n<h2>Get patch embeddings</h2>\n<p>This splits the image into patches of size <span translate=no>_^_0_^_</span> and gives an embedding for each patch.</p>\n": "<p><a id=\"PatchEmbeddings\"></a></p>\n<h2>\u30d1\u30c3\u30c1\u57cb\u3081\u8fbc\u307f\u3092\u5165\u624b</h2>\n<p>\u3053\u308c\u306b\u3088\u308a\u3001<span translate=no>_^_0_^_</span>\u753b\u50cf\u304c\u8907\u6570\u306e\u30b5\u30a4\u30ba\u306e\u30d1\u30c3\u30c1\u306b\u5206\u5272\u3055\u308c\u3001\u5404\u30d1\u30c3\u30c1\u304c\u57cb\u3081\u8fbc\u307e\u308c\u307e\u3059\u3002</p>\n",
"<p>Activation after depth-wise convolution </p>\n": "<p>\u6df1\u3055\u65b9\u5411\u306e\u7573\u307f\u8fbc\u307f\u5f8c\u306e\u30a2\u30af\u30c6\u30a3\u30d9\u30fc\u30b7\u30e7\u30f3</p>\n",
"<p>Activation after point-wise convolution </p>\n": "<p>\u70b9\u5358\u4f4d\u306e\u7573\u307f\u8fbc\u307f\u5f8c\u306e\u30a2\u30af\u30c6\u30a3\u30d9\u30fc\u30b7\u30e7\u30f3</p>\n",
"<p>Activation and normalization </p>\n": "<p>\u30a2\u30af\u30c6\u30a3\u30d9\u30fc\u30b7\u30e7\u30f3\u3068\u6b63\u898f\u5316</p>\n",
"<p>Activation function </p>\n": "<p>\u30a2\u30af\u30c6\u30a3\u30d9\u30fc\u30b7\u30e7\u30f3\u6a5f\u80fd</p>\n",
"<p>Add residual connection </p>\n": "<p>\u6b8b\u4f59\u63a5\u7d9a\u3092\u8ffd\u52a0</p>\n",
"<p>Apply convolution layer </p>\n": "<p>\u7573\u307f\u8fbc\u307f\u5c64\u3092\u9069\u7528</p>\n",
"<p>Average Pool </p>\n": "<p>\u30a2\u30d9\u30ec\u30fc\u30b8\u30d7\u30fc\u30eb</p>\n",
"<p>Average pooling </p>\n": "<p>\u5e73\u5747\u30d7\u30fc\u30ea\u30f3\u30b0</p>\n",
"<p>Batch normalization </p>\n": "<p>\u30d0\u30c3\u30c1\u6b63\u898f\u5316</p>\n",
"<p>Classification head </p>\n": "<p>\u5206\u985e\u30d8\u30c3\u30c9</p>\n",
"<p>Classification head, to get logits </p>\n": "<p>\u30ed\u30b8\u30c3\u30c8\u3092\u53d6\u5f97\u3059\u308b\u305f\u3081\u306e\u5206\u985e\u30d8\u30c3\u30c9</p>\n",
"<p>Depth-wise convolution is separate convolution for each channel. We do this with a convolution layer with the number of groups equal to the number of channels. So that each channel is it&#x27;s own group. </p>\n": "<p>\u6df1\u5ea6\u65b9\u5411\u306e\u7573\u307f\u8fbc\u307f\u306f\u3001\u30c1\u30e3\u30f3\u30cd\u30eb\u3054\u3068\u306b\u5225\u3005\u306e\u7573\u307f\u8fbc\u307f\u306b\u306a\u308a\u307e\u3059\u3002\u3053\u308c\u306f\u3001\u30b0\u30eb\u30fc\u30d7\u306e\u6570\u304c\u30c1\u30e3\u30cd\u30eb\u6570\u3068\u7b49\u3057\u3044\u7573\u307f\u8fbc\u307f\u5c64\u3067\u884c\u3044\u307e\u3059\u3002\u305d\u306e\u305f\u3081\u3001\u5404\u30c1\u30e3\u30f3\u30cd\u30eb\u306f\u305d\u308c\u305e\u308c\u72ec\u81ea\u306e\u30b0\u30eb\u30fc\u30d7\u306b\u306a\u308a\u307e\u3059\u3002</p>\n",
"<p>Depth-wise convolution, activation and normalization </p>\n": "<p>\u6df1\u5ea6\u65b9\u5411\u306e\u7573\u307f\u8fbc\u307f\u3001\u6d3b\u6027\u5316\u3001\u6b63\u898f\u5316</p>\n",
"<p>For the residual connection around the depth-wise convolution </p>\n": "<p>\u6df1\u3055\u65b9\u5411\u306e\u7573\u307f\u8fbc\u307f\u306e\u5468\u308a\u306e\u6b8b\u5dee\u7d50\u5408\u306b\u3064\u3044\u3066</p>\n",
"<p>Get patch embeddings. This gives a tensor of shape <span translate=no>_^_0_^_</span>. </p>\n": "<p>\u30d1\u30c3\u30c1\u306e\u57cb\u3081\u8fbc\u307f\u3092\u5165\u624b\u3057\u3066\u304f\u3060\u3055\u3044\u3002<span translate=no>_^_0_^_</span>\u3053\u308c\u306b\u3088\u308a\u5f62\u72b6\u306e\u30c6\u30f3\u30bd\u30eb\u304c\u5f97\u3089\u308c\u307e\u3059</p>\u3002\n",
"<p>Get the embedding, <span translate=no>_^_0_^_</span> will have shape <span translate=no>_^_1_^_</span> </p>\n": "<p><span translate=no>_^_0_^_</span>\u57cb\u3081\u8fbc\u307f\u3092\u5165\u308c\u308b\u3068\u3001\u5f62\u304c\u6574\u3044\u307e\u3059 <span translate=no>_^_1_^_</span></p>\n",
"<p>Linear layer </p>\n": "<p>\u30ea\u30cb\u30a2\u30ec\u30a4\u30e4\u30fc</p>\n",
"<p>Make copies of the <a href=\"#ConvMixerLayer\">ConvMixer layer</a> </p>\n": "<p><a href=\"#ConvMixerLayer\">ConvMixer</a> \u30ec\u30a4\u30e4\u30fc\u306e\u30b3\u30d4\u30fc\u3092\u4f5c\u6210</p>\n",
"<p>Normalization after depth-wise convolution </p>\n": "<p>\u6df1\u3055\u65b9\u5411\u306e\u7573\u307f\u8fbc\u307f\u5f8c\u306e\u6b63\u898f\u5316</p>\n",
"<p>Normalization after point-wise convolution </p>\n": "<p>\u70b9\u5358\u4f4d\u306e\u7573\u307f\u8fbc\u307f\u5f8c\u306e\u6b63\u898f\u5316</p>\n",
"<p>Pass through <a href=\"#ConvMixerLayer\">ConvMixer layers</a> </p>\n": "<p><a href=\"#ConvMixerLayer\">ConvMixer \u30ec\u30a4\u30e4\u30fc\u3092\u30d1\u30b9\u30b9\u30eb\u30fc\u3059\u308b</a></p>\n",
"<p>Patch embeddings </p>\n": "<p>\u30d1\u30c3\u30c1\u57cb\u3081\u8fbc\u307f</p>\n",
"<p>Point-wise convolution is a <span translate=no>_^_0_^_</span> convolution. i.e. a linear transformation of patch embeddings </p>\n": "<p>\u70b9\u5358\u4f4d\u306e\u7573\u307f\u8fbc\u307f\u306f\u7573\u307f\u8fbc\u307f\u3067\u3059\u3002\u3064\u307e\u308a\u3001<span translate=no>_^_0_^_</span>\u30d1\u30c3\u30c1\u57cb\u3081\u8fbc\u307f\u306e\u7dda\u5f62\u5909\u63db\u3067\u3059</p>\n",
"<p>Point-wise convolution, activation and normalization </p>\n": "<p>\u70b9\u5358\u4f4d\u306e\u7573\u307f\u8fbc\u307f\u3001\u6d3b\u6027\u5316\u3001\u6b63\u898f\u5316</p>\n",
"<p>We create a convolution layer with a kernel size and and stride length equal to patch size. This is equivalent to splitting the image into patches and doing a linear transformation on each patch. </p>\n": "<p>\u30ab\u30fc\u30cd\u30eb\u30b5\u30a4\u30ba\u3067\u30b9\u30c8\u30e9\u30a4\u30c9\u306e\u9577\u3055\u304c\u30d1\u30c3\u30c1\u30b5\u30a4\u30ba\u3068\u540c\u3058\u30b3\u30f3\u30dc\u30ea\u30e5\u30fc\u30b7\u30e7\u30f3\u30ec\u30a4\u30e4\u30fc\u3092\u4f5c\u6210\u3057\u307e\u3059\u3002\u3053\u308c\u306f\u3001\u753b\u50cf\u3092\u30d1\u30c3\u30c1\u306b\u5206\u5272\u3057\u3001\u5404\u30d1\u30c3\u30c1\u3067\u7dda\u5f62\u5909\u63db\u3092\u884c\u3046\u306e\u3068\u540c\u3058\u3067\u3059</p>\u3002\n",
"<ul><li><span translate=no>_^_0_^_</span> is a copy of a single <a href=\"#ConvMixerLayer\">ConvMixer layer</a>. We make copies of it to make ConvMixer with <span translate=no>_^_1_^_</span>. </li>\n<li><span translate=no>_^_2_^_</span> is the number of ConvMixer layers (or depth), <span translate=no>_^_3_^_</span>. </li>\n<li><span translate=no>_^_4_^_</span> is the <a href=\"#PatchEmbeddings\">patch embeddings layer</a>. </li>\n<li><span translate=no>_^_5_^_</span> is the <a href=\"#ClassificationHead\">classification head</a>.</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u5358\u4e00\u306e <a href=\"#ConvMixerLayer\">ConvMixer</a> \u30ec\u30a4\u30e4\u30fc\u306e\u30b3\u30d4\u30fc\u3067\u3059\u3002\u305d\u306e\u30b3\u30d4\u30fc\u3092\u4f5c\u6210\u3057\u3066 ConvMixer</li> \u3092\u4f5c\u6210\u3057\u307e\u3059\u3002<span translate=no>_^_1_^_</span>\n<li><span translate=no>_^_2_^_</span>\u306f ConvMixer \u30ec\u30a4\u30e4\u30fc\u306e\u6570 (\u307e\u305f\u306f\u6df1\u3055) \u3067\u3059\u3002<span translate=no>_^_3_^_</span></li>\n<li><span translate=no>_^_4_^_</span><a href=\"#PatchEmbeddings\">\u30d1\u30c3\u30c1\u57cb\u3081\u8fbc\u307f\u30ec\u30a4\u30e4\u30fc\u3067\u3059</a>\u3002</li>\n<li><span translate=no>_^_5_^_</span><a href=\"#ClassificationHead\">\u5206\u985e\u8cac\u4efb\u8005\u3067\u3059</a>\u3002</li></ul>\n",
"<ul><li><span translate=no>_^_0_^_</span> is the input image of shape <span translate=no>_^_1_^_</span></li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u5f62\u72b6\u306e\u5165\u529b\u753b\u50cf\u3067\u3059 <span translate=no>_^_1_^_</span></li></ul>\n",
"<ul><li><span translate=no>_^_0_^_</span> is the number of channels in patch embeddings <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> is the size of the patch, <span translate=no>_^_3_^_</span> </li>\n<li><span translate=no>_^_4_^_</span> is the number of channels in the input image (3 for rgb)</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u30d1\u30c3\u30c1\u57cb\u3081\u8fbc\u307f\u306e\u30c1\u30e3\u30f3\u30cd\u30eb\u6570\u3067\u3059 <span translate=no>_^_1_^_</span></li>\n<li><span translate=no>_^_2_^_</span>\u306f\u30d1\u30c3\u30c1\u306e\u30b5\u30a4\u30ba\u3001<span translate=no>_^_3_^_</span></li>\n<li><span translate=no>_^_4_^_</span>\u306f\u5165\u529b\u753b\u50cf\u306e\u30c1\u30e3\u30f3\u30cd\u30eb\u6570 (RGB \u306e\u5834\u5408\u306f 3)</li></ul>\n",
"<ul><li><span translate=no>_^_0_^_</span> is the number of channels in patch embeddings, <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> is the number of classes in the classification task</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u306f\u30d1\u30c3\u30c1\u57cb\u3081\u8fbc\u307f\u306e\u30c1\u30e3\u30f3\u30cd\u30eb\u6570\u3001<span translate=no>_^_1_^_</span></li>\n<li><span translate=no>_^_2_^_</span>\u5206\u985e\u30bf\u30b9\u30af\u5185\u306e\u30af\u30e9\u30b9\u6570\u3067\u3059</li></ul>\n",
"<ul><li><span translate=no>_^_0_^_</span> is the number of channels in patch embeddings, <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> is the size of the kernel of spatial convolution, <span translate=no>_^_3_^_</span></li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u306f\u30d1\u30c3\u30c1\u57cb\u3081\u8fbc\u307f\u306e\u30c1\u30e3\u30f3\u30cd\u30eb\u6570\u3001<span translate=no>_^_1_^_</span></li>\n<li><span translate=no>_^_2_^_</span>\u306f\u7a7a\u9593\u7573\u307f\u8fbc\u307f\u306e\u30ab\u30fc\u30cd\u30eb\u306e\u5927\u304d\u3055\u3067\u3059 <span translate=no>_^_3_^_</span></li></ul>\n",
"A PyTorch implementation/tutorial of the paper \"Patches Are All You Need?\"": "\u300c\u5fc5\u8981\u306a\u306e\u306f\u30d1\u30c3\u30c1\u3060\u3051\uff1f\u300d\u3068\u3044\u3046\u8ad6\u6587\u306ePyTorch\u5b9f\u88c5/\u30c1\u30e5\u30fc\u30c8\u30ea\u30a2\u30eb",
"Patches Are All You Need? (ConvMixer)": "\u5fc5\u8981\u306a\u306e\u306f\u30d1\u30c3\u30c1\u3060\u3051\uff1f(\u30b3\u30f3\u30d0\u30fc\u30b8\u30e7\u30f3\u30df\u30ad\u30b5\u30fc)"
}
translate_cache/conv_mixer/__init__.si.json
+40
View File
File diff suppressed because one or more lines are too long
translate_cache/conv_mixer/__init__.zh.json
+40
View File
@@ -0,0 +1,40 @@
{
"<h1>Patches Are All You Need? (ConvMixer)</h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of the paper <a href=\"https://arxiv.org/abs/2201.09792\">Patches Are All You Need?</a>.</p>\n<p><span translate=no>_^_0_^_</span></p>\n<p>ConvMixer is Similar to <a href=\"../transformers/mlp_mixer/index.html\">MLP-Mixer</a>. MLP-Mixer separates mixing of spatial and channel dimensions, by applying an MLP across spatial dimension and then an MLP across the channel dimension (spatial MLP replaces the <a href=\"../transformers/vit/index.html\">ViT</a> attention and channel MLP is the <a href=\"../transformers/feed_forward.html\">FFN</a> of ViT).</p>\n<p>ConvMixer uses a <span translate=no>_^_1_^_</span> convolution for channel mixing and a depth-wise convolution for spatial mixing. Since it&#x27;s a convolution instead of a full MLP across the space, it mixes only the nearby batches in contrast to ViT or MLP-Mixer. Also, the MLP-mixer uses MLPs of two layers for each mixing and ConvMixer uses a single layer for each mixing.</p>\n<p>The paper recommends removing the residual connection across the channel mixing (point-wise convolution) and having only a residual connection over the spatial mixing (depth-wise convolution). They also use <a href=\"../normalization/batch_norm/index.html\">Batch normalization</a> instead of <a href=\"../normalization/layer_norm/index.html\">Layer normalization</a>.</p>\n<p>Here&#x27;s <a href=\"experiment.html\">an experiment</a> that trains ConvMixer on CIFAR-10.</p>\n": "<h1>\u4f60\u53ea\u9700\u8981\u8865\u4e01\u5417\uff1f\uff08convMixer\uff09</h1>\n<p>\u8fd9\u662f <a href=\"https://pytorch.org\">PyTorch</a> \u5bf9\u8bba\u6587\u300a<a href=\"https://arxiv.org/abs/2201.09792\">\u8865\u4e01\u5c31\u662f\u4f60\u6240\u9700\u8981\u7684\uff1f</a>\u300b\u7684\u5b9e\u73b0</p>\u3002\n<p><span translate=no>_^_0_^_</span></p>\n<p>convMixer \u7c7b\u4f3c\u4e8e <a href=\"../transformers/mlp_mixer/index.html\">MLP \u6df7\u97f3\u5668</a>\u3002MLP-Mixer \u901a\u8fc7\u5728\u7a7a\u95f4\u7ef4\u5ea6\u4e0a\u5e94\u7528 MLP\uff0c\u7136\u540e\u5728\u4fe1\u9053\u7ef4\u5ea6\u4e0a\u5e94\u7528 MLP \u6765\u5206\u79bb\u7a7a\u95f4\u7ef4\u5ea6\u548c\u4fe1\u9053\u7ef4\u5ea6\u7684\u6df7\u97f3\uff08\u7a7a\u95f4 MLP \u53d6\u4ee3 <a href=\"../transformers/vit/index.html\">vIT</a> \u6ce8\u610f\u529b\uff0c\u4fe1\u9053 MLP \u662f ViT \u7684 <a href=\"../transformers/feed_forward.html\">FFN</a>\uff09\u3002</p>\n<p>ConvMixer \u4f7f\u7528<span translate=no>_^_1_^_</span>\u5377\u79ef\u8fdb\u884c\u901a\u9053\u6df7\u5408\uff0c\u4f7f\u7528\u6df1\u5ea6\u5377\u79ef\u8fdb\u884c\u7a7a\u95f4\u6df7\u5408\u3002\u7531\u4e8e\u5b83\u662f\u5377\u79ef\u800c\u4e0d\u662f\u6574\u4e2a\u7a7a\u95f4\u7684\u5b8c\u6574\u7684 MLP\uff0c\u56e0\u6b64\u4e0e vIT \u6216 MLP-Mixer \u76f8\u6bd4\uff0c\u5b83\u53ea\u6df7\u5408\u9644\u8fd1\u7684\u6279\u6b21\u3002\u6b64\u5916\uff0cMLP-Mixer \u6bcf\u6b21\u6df7\u5408\u4f7f\u7528\u4e24\u5c42 MLP\uff0cConvMixer \u6bcf\u6b21\u6df7\u5408\u4f7f\u7528\u5355\u5c42\u3002</p>\n<p>\u8be5\u8bba\u6587\u5efa\u8bae\u5220\u9664\u4fe1\u9053\u6df7\u5408\uff08\u9010\u70b9\u5377\u79ef\uff09\u4e0a\u7684\u5269\u4f59\u8fde\u63a5\uff0c\u5728\u7a7a\u95f4\u6df7\u5408\uff08\u6df1\u5ea6\u5377\u79ef\uff09\u4e0a\u4ec5\u4f7f\u7528\u6b8b\u5dee\u8fde\u63a5\u3002\u4ed6\u4eec\u8fd8\u4f7f\u7528<a href=\"../normalization/batch_norm/index.html\">\u6279\u91cf\u6807\u51c6\u5316</a>\u800c\u4e0d\u662f<a href=\"../normalization/layer_norm/index.html\">\u56fe\u5c42\u6807\u51c6\u5316</a>\u3002</p>\n<p>\u8fd9\u662f<a href=\"experiment.html\">\u4e00\u9879\u5728 CIFAR-10 \u4e0a\u8bad\u7ec3 ConvMixer \u7684\u5b9e\u9a8c</a>\u3002</p>\n",
"<h2>ConvMixer</h2>\n<p>This combines the patch embeddings block, a number of ConvMixer layers and a classification head.</p>\n": "<h2>\u6df7\u97f3\u5668</h2>\n<p>\u5b83\u7ed3\u5408\u4e86\u8865\u4e01\u5d4c\u5165\u5757\u3001\u8bb8\u591a ConvMixer \u5c42\u548c\u4e00\u4e2a\u5206\u7c7b\u5934\u3002</p>\n",
"<p> </p>\n": "<p></p>\n",
"<p> <a id=\"ClassificationHead\"></a></p>\n<h2>Classification Head</h2>\n<p>They do average pooling (taking the mean of all patch embeddings) and a final linear transformation to predict the log-probabilities of the image classes.</p>\n": "<p><a id=\"ClassificationHead\"></a></p>\n<h2>\u5206\u7c7b\u4e3b\u7ba1</h2>\n<p>\u5b83\u4eec\u8fdb\u884c\u5e73\u5747\u6c60\uff08\u53d6\u6240\u6709\u8865\u4e01\u5d4c\u5165\u7684\u5747\u503c\uff09\u548c\u6700\u7ec8\u7684\u7ebf\u6027\u53d8\u6362\u6765\u9884\u6d4b\u5f71\u50cf\u7c7b\u7684\u5bf9\u6570\u6982\u7387\u3002</p>\n",
"<p> <a id=\"ConvMixerLayer\"></a></p>\n<h2>ConvMixer layer</h2>\n<p>This is a single ConvMixer layer. The model will have a series of these.</p>\n": "<p><a id=\"ConvMixerLayer\"></a></p>\n<h2>\u6df7\u97f3\u5668\u5c42</h2>\n<p>\u8fd9\u662f\u5355\u4e2a ConvMixer \u5c42\u3002\u8be5\u6a21\u578b\u5c06\u6709\u4e00\u7cfb\u5217\u8fd9\u6837\u7684\u3002</p>\n",
"<p> <a id=\"PatchEmbeddings\"></a></p>\n<h2>Get patch embeddings</h2>\n<p>This splits the image into patches of size <span translate=no>_^_0_^_</span> and gives an embedding for each patch.</p>\n": "<p><a id=\"PatchEmbeddings\"></a></p>\n<h2>\u83b7\u53d6\u8865\u4e01\u5d4c\u5165</h2>\n<p>\u8fd9\u4f1a\u5c06\u56fe\u50cf\u62c6\u5206\u4e3a\u5927\u5c0f\u7684\u8865\u4e01\uff0c<span translate=no>_^_0_^_</span>\u5e76\u4e3a\u6bcf\u4e2a\u8865\u4e01\u63d0\u4f9b\u5d4c\u5165\u3002</p>\n",
"<p>Activation after depth-wise convolution </p>\n": "<p>\u6df1\u5ea6\u5377\u79ef\u540e\u6fc0\u6d3b</p>\n",
"<p>Activation after point-wise convolution </p>\n": "<p>\u9010\u70b9\u5377\u79ef\u540e\u6fc0\u6d3b</p>\n",
"<p>Activation and normalization </p>\n": "<p>\u6fc0\u6d3b\u548c\u89c4\u8303\u5316</p>\n",
"<p>Activation function </p>\n": "<p>\u6fc0\u6d3b\u529f\u80fd</p>\n",
"<p>Add residual connection </p>\n": "<p>\u6dfb\u52a0\u5269\u4f59\u8fde\u63a5</p>\n",
"<p>Apply convolution layer </p>\n": "<p>\u5e94\u7528\u5377\u79ef\u5c42</p>\n",
"<p>Average Pool </p>\n": "<p>\u5e73\u5747\u6c60</p>\n",
"<p>Average pooling </p>\n": "<p>\u5e73\u5747\u6c47\u96c6</p>\n",
"<p>Batch normalization </p>\n": "<p>\u6279\u91cf\u6807\u51c6\u5316</p>\n",
"<p>Classification head </p>\n": "<p>\u5206\u7c7b\u4e3b\u7ba1</p>\n",
"<p>Classification head, to get logits </p>\n": "<p>\u5206\u7c7b\u5934\uff0c\u83b7\u53d6\u65e5\u5fd7</p>\n",
"<p>Depth-wise convolution is separate convolution for each channel. We do this with a convolution layer with the number of groups equal to the number of channels. So that each channel is it&#x27;s own group. </p>\n": "<p>\u6df1\u5ea6\u5377\u79ef\u662f\u6bcf\u4e2a\u901a\u9053\u7684\u5355\u72ec\u5377\u79ef\u3002\u6211\u4eec\u4f7f\u7528\u5377\u79ef\u5c42\u6765\u5b8c\u6210\u6b64\u64cd\u4f5c\uff0c\u8be5\u5377\u79ef\u5c42\u7684\u7ec4\u6570\u7b49\u4e8e\u901a\u9053\u6570\u3002\u56e0\u6b64\uff0c\u6bcf\u4e2a\u9891\u9053\u90fd\u662f\u5b83\u81ea\u5df1\u7684\u7ec4\u3002</p>\n",
"<p>Depth-wise convolution, activation and normalization </p>\n": "<p>\u6df1\u5ea6\u5377\u79ef\u3001\u6fc0\u6d3b\u548c\u5f52\u4e00\u5316</p>\n",
"<p>For the residual connection around the depth-wise convolution </p>\n": "<p>\u5bf9\u4e8e\u56f4\u7ed5\u6df1\u5ea6\u5377\u79ef\u7684\u5269\u4f59\u8fde\u63a5</p>\n",
"<p>Get patch embeddings. This gives a tensor of shape <span translate=no>_^_0_^_</span>. </p>\n": "<p>\u83b7\u53d6\u8865\u4e01\u5d4c\u5165\u3002\u8fd9\u7ed9\u51fa\u4e86\u5f62\u72b6\u7684\u5f20\u91cf<span translate=no>_^_0_^_</span>\u3002</p>\n",
"<p>Get the embedding, <span translate=no>_^_0_^_</span> will have shape <span translate=no>_^_1_^_</span> </p>\n": "<p>\u5f97\u5230\u5d4c\u5165\uff0c<span translate=no>_^_0_^_</span>\u4f1a\u6709\u5f62\u72b6<span translate=no>_^_1_^_</span></p>\n",
"<p>Linear layer </p>\n": "<p>\u7ebf\u6027\u5c42</p>\n",
"<p>Make copies of the <a href=\"#ConvMixerLayer\">ConvMixer layer</a> </p>\n": "<p>\u5236\u4f5c C <a href=\"#ConvMixerLayer\">onvMixer \u56fe\u5c42</a>\u7684\u526f\u672c</p>\n",
"<p>Normalization after depth-wise convolution </p>\n": "<p>\u6df1\u5ea6\u5377\u79ef\u540e\u7684\u5f52\u4e00\u5316</p>\n",
"<p>Normalization after point-wise convolution </p>\n": "<p>\u9010\u70b9\u5377\u79ef\u540e\u7684\u5f52\u4e00\u5316</p>\n",
"<p>Pass through <a href=\"#ConvMixerLayer\">ConvMixer layers</a> </p>\n": "<p>\u7a7f\u8fc7 <a href=\"#ConvMixerLayer\">ConvMixer \u56fe\u5c42</a></p>\n",
"<p>Patch embeddings </p>\n": "<p>\u8865\u4e01\u5d4c\u5165</p>\n",
"<p>Point-wise convolution is a <span translate=no>_^_0_^_</span> convolution. i.e. a linear transformation of patch embeddings </p>\n": "<p>\u9010\u70b9\u5377\u79ef\u662f\u4e00\u79cd<span translate=no>_^_0_^_</span>\u5377\u79ef\u3002\u5373\u8865\u4e01\u5d4c\u5165\u7684\u7ebf\u6027\u53d8\u6362</p>\n",
"<p>Point-wise convolution, activation and normalization </p>\n": "<p>\u9010\u70b9\u5377\u79ef\u3001\u6fc0\u6d3b\u548c\u5f52\u4e00\u5316</p>\n",
"<p>We create a convolution layer with a kernel size and and stride length equal to patch size. This is equivalent to splitting the image into patches and doing a linear transformation on each patch. </p>\n": "<p>\u6211\u4eec\u521b\u5efa\u4e00\u4e2a\u5377\u79ef\u5c42\uff0c\u5176\u5185\u6838\u5927\u5c0f\u548c\u6b65\u957f\u7b49\u4e8e\u8865\u4e01\u5927\u5c0f\u3002\u8fd9\u76f8\u5f53\u4e8e\u5c06\u56fe\u50cf\u5206\u5272\u6210\u8272\u5757\u5e76\u5728\u6bcf\u4e2a\u9762\u7247\u4e0a\u8fdb\u884c\u7ebf\u6027\u53d8\u6362\u3002</p>\n",
"<ul><li><span translate=no>_^_0_^_</span> is a copy of a single <a href=\"#ConvMixerLayer\">ConvMixer layer</a>. We make copies of it to make ConvMixer with <span translate=no>_^_1_^_</span>. </li>\n<li><span translate=no>_^_2_^_</span> is the number of ConvMixer layers (or depth), <span translate=no>_^_3_^_</span>. </li>\n<li><span translate=no>_^_4_^_</span> is the <a href=\"#PatchEmbeddings\">patch embeddings layer</a>. </li>\n<li><span translate=no>_^_5_^_</span> is the <a href=\"#ClassificationHead\">classification head</a>.</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u662f\u5355\u4e2a C <a href=\"#ConvMixerLayer\">onvMixer \u5c42</a>\u7684\u526f\u672c\u3002\u6211\u4eec\u5236\u4f5c\u5b83\u7684\u526f\u672c\u6765\u5236\u4f5c ConvMixer<span translate=no>_^_1_^_</span>\u3002</li>\n<li><span translate=no>_^_2_^_</span>\u662f ConvMixer \u5c42\uff08\u6216\u6df1\u5ea6\uff09\u7684\u6570\u91cf<span translate=no>_^_3_^_</span>\u3002</li>\n<li><span translate=no>_^_4_^_</span>\u662f<a href=\"#PatchEmbeddings\">\u8865\u4e01\u5d4c\u5165\u5c42</a>\u3002</li>\n<li><span translate=no>_^_5_^_</span>\u662f<a href=\"#ClassificationHead\">\u5206\u7c7b\u5934</a>\u3002</li></ul>\n",
"<ul><li><span translate=no>_^_0_^_</span> is the input image of shape <span translate=no>_^_1_^_</span></li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u662f\u5f62\u72b6\u7684\u8f93\u5165\u56fe\u50cf<span translate=no>_^_1_^_</span></li></ul>\n",
"<ul><li><span translate=no>_^_0_^_</span> is the number of channels in patch embeddings <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> is the size of the patch, <span translate=no>_^_3_^_</span> </li>\n<li><span translate=no>_^_4_^_</span> is the number of channels in the input image (3 for rgb)</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u662f\u8865\u4e01\u5d4c\u5165\u4e2d\u7684\u901a\u9053\u6570<span translate=no>_^_1_^_</span></li>\n<li><span translate=no>_^_2_^_</span>\u662f\u8865\u4e01\u7684\u5927\u5c0f\uff0c<span translate=no>_^_3_^_</span></li>\n<li><span translate=no>_^_4_^_</span>\u662f\u8f93\u5165\u56fe\u50cf\u4e2d\u7684\u901a\u9053\u6570\uff08rgb \u4e3a 3\uff09</li></ul>\n",
"<ul><li><span translate=no>_^_0_^_</span> is the number of channels in patch embeddings, <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> is the number of classes in the classification task</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u662f\u8865\u4e01\u5d4c\u5165\u4e2d\u7684\u901a\u9053\u6570\uff0c<span translate=no>_^_1_^_</span></li>\n<li><span translate=no>_^_2_^_</span>\u662f\u5206\u7c7b\u4efb\u52a1\u4e2d\u7684\u7c7b\u6570</li></ul>\n",
"<ul><li><span translate=no>_^_0_^_</span> is the number of channels in patch embeddings, <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> is the size of the kernel of spatial convolution, <span translate=no>_^_3_^_</span></li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u662f\u8865\u4e01\u5d4c\u5165\u4e2d\u7684\u901a\u9053\u6570\uff0c<span translate=no>_^_1_^_</span></li>\n<li><span translate=no>_^_2_^_</span>\u662f\u7a7a\u95f4\u5377\u79ef\u5185\u6838\u7684\u5927\u5c0f\uff0c<span translate=no>_^_3_^_</span></li></ul>\n",
"A PyTorch implementation/tutorial of the paper \"Patches Are All You Need?\"": "\u8bba\u6587 \u201c\u8865\u4e01\u5c31\u662f\u4f60\u6240\u9700\u8981\u7684\u5417\uff1f\u201d \u7684 PyTorch \u5b9e\u73b0/\u6559\u7a0b",
"Patches Are All You Need? (ConvMixer)": "\u8865\u4e01\u662f\u4f60\u6240\u9700\u8981\u7684\u5417\uff1f\uff08convMixer\uff09"
}
translate_cache/conv_mixer/experiment.ja.json
+22
View File
@@ -0,0 +1,22 @@
{
"<h1>Train a <a href=\"index.html\">ConvMixer</a> on CIFAR 10</h1>\n<p>This script trains a ConvMixer on CIFAR 10 dataset.</p>\n<p>This is not an attempt to reproduce the results of the paper. The paper uses image augmentations present in <a href=\"https://github.com/rwightman/pytorch-image-models\">PyTorch Image Models (timm)</a> for training. We haven&#x27;t done this for simplicity - which causes our validation accuracy to drop.</p>\n": "<h1>CIFAR 10 <a href=\"index.html\">\u3067\u306e\u30b3\u30f3\u30d0\u30fc\u30b8\u30e7\u30f3\u30df\u30ad\u30b5\u30fc\u306e\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0</a></h1>\n<p>\u3053\u306e\u30b9\u30af\u30ea\u30d7\u30c8\u306f\u3001CIFAR 10 \u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3067 ConvMixer \u3092\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3057\u307e\u3059\u3002</p>\n<p>\u3053\u308c\u306f\u8ad6\u6587\u306e\u7d50\u679c\u3092\u518d\u73fe\u3059\u308b\u8a66\u307f\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002\u3053\u306e\u8ad6\u6587\u3067\u306f\u3001<a href=\"https://github.com/rwightman/pytorch-image-models\">PyTorch\u753b\u50cf\u30e2\u30c7\u30eb\uff08timm\uff09\u306b\u3042\u308b\u753b\u50cf\u62e1\u5f35\u3092\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u306b\u4f7f\u7528\u3057\u3066\u3044\u307e\u3059</a>\u3002\u7c21\u7565\u5316\u306e\u305f\u3081\u306b\u3053\u308c\u3092\u884c\u3063\u305f\u308f\u3051\u3067\u306f\u3042\u308a\u307e\u305b\u3093\u3002\u305d\u306e\u305f\u3081\u3001\u691c\u8a3c\u306e\u7cbe\u5ea6\u304c\u4f4e\u4e0b\u3057\u307e\u3059</p>\u3002\n",
"<h2>Configurations</h2>\n<p>We use <a href=\"../experiments/cifar10.html\"><span translate=no>_^_0_^_</span></a> which defines all the dataset related configurations, optimizer, and a training loop.</p>\n": "<h2>\u30b3\u30f3\u30d5\u30a3\u30ae\u30e5\u30ec\u30fc\u30b7\u30e7\u30f3</h2>\n<p>\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306b\u95a2\u9023\u3059\u308b\u3059\u3079\u3066\u306e\u69cb\u6210\u3001\u30aa\u30d7\u30c6\u30a3\u30de\u30a4\u30b6\u30fc\u3001\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30eb\u30fc\u30d7\u3092\u5b9a\u7fa9\u3059\u308b\u3082\u306e\u3092\u4f7f\u7528\u3057\u3066\u3044\u307e\u3059<a href=\"../experiments/cifar10.html\"><span translate=no>_^_0_^_</span></a>\u3002</p>\n",
"<h3>Create model</h3>\n": "<h3>\u30e2\u30c7\u30eb\u4f5c\u6210</h3>\n",
"<p> </p>\n": "<p></p>\n",
"<p>Create ConvMixer </p>\n": "<p>ConvMixer \u306e\u4f5c\u6210</p>\n",
"<p>Create configurations </p>\n": "<p>\u69cb\u6210\u306e\u4f5c\u6210</p>\n",
"<p>Create experiment </p>\n": "<p>\u5b9f\u9a13\u3092\u4f5c\u6210</p>\n",
"<p>Do not augment images for validation </p>\n": "<p>\u691c\u8a3c\u306e\u305f\u3081\u306b\u753b\u50cf\u3092\u88dc\u8db3\u3057\u306a\u3044\u3067\u304f\u3060\u3055\u3044</p>\n",
"<p>Kernel size of the depth-wise convolution, <span translate=no>_^_0_^_</span> </p>\n": "<p>\u6df1\u3055\u65b9\u5411\u306e\u7573\u307f\u8fbc\u307f\u306e\u30ab\u30fc\u30cd\u30eb\u30b5\u30a4\u30ba\u3001<span translate=no>_^_0_^_</span></p>\n",
"<p>Load configurations </p>\n": "<p>\u69cb\u6210\u3092\u30ed\u30fc\u30c9</p>\n",
"<p>Number of <a href=\"#ConvMixerLayer\">ConvMixer layers</a> or depth, <span translate=no>_^_0_^_</span> </p>\n": "<p><a href=\"#ConvMixerLayer\">ConvMixer \u30ec\u30a4\u30e4\u30fc\u306e\u6570\u307e\u305f\u306f\u6df1\u3055</a>\u3001<span translate=no>_^_0_^_</span></p>\n",
"<p>Number of channels in patch embeddings, <span translate=no>_^_0_^_</span> </p>\n": "<p>\u30d1\u30c3\u30c1\u57cb\u3081\u8fbc\u307f\u306e\u30c1\u30e3\u30f3\u30cd\u30eb\u6570\u3001<span translate=no>_^_0_^_</span></p>\n",
"<p>Number of classes in the task </p>\n": "<p>\u30bf\u30b9\u30af\u5185\u306e\u30af\u30e9\u30b9\u6570</p>\n",
"<p>Optimizer </p>\n": "<p>\u30aa\u30d7\u30c6\u30a3\u30de\u30a4\u30b6\u30fc</p>\n",
"<p>Set model for saving/loading </p>\n": "<p>\u4fdd\u5b58/\u8aad\u307f\u8fbc\u307f\u7528\u306e\u30e2\u30c7\u30eb\u3092\u8a2d\u5b9a</p>\n",
"<p>Simple image augmentations </p>\n": "<p>\u30b7\u30f3\u30d7\u30eb\u306a\u753b\u50cf\u88dc\u6b63</p>\n",
"<p>Size of a patch, <span translate=no>_^_0_^_</span> </p>\n": "<p>\u30d1\u30c3\u30c1\u306e\u30b5\u30a4\u30ba\u3001<span translate=no>_^_0_^_</span></p>\n",
"<p>Start the experiment and run the training loop </p>\n": "<p>\u5b9f\u9a13\u3092\u958b\u59cb\u3057\u3001\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30eb\u30fc\u30d7\u3092\u5b9f\u884c\u3057\u307e\u3059</p>\n",
"<p>Training epochs and batch size </p>\n": "<p>\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30a8\u30dd\u30c3\u30af\u3068\u30d0\u30c3\u30c1\u30b5\u30a4\u30ba</p>\n",
"Train ConvMixer on CIFAR 10": "CIFAR 10 \u306e\u30c8\u30ec\u30a4\u30f3\u30b3\u30f3\u30d0\u30fc\u30df\u30ad\u30b5\u30fc"
}
translate_cache/conv_mixer/experiment.si.json
+22
View File
@@ -0,0 +1,22 @@
{
"<h1>Train a <a href=\"index.html\">ConvMixer</a> on CIFAR 10</h1>\n<p>This script trains a ConvMixer on CIFAR 10 dataset.</p>\n<p>This is not an attempt to reproduce the results of the paper. The paper uses image augmentations present in <a href=\"https://github.com/rwightman/pytorch-image-models\">PyTorch Image Models (timm)</a> for training. We haven&#x27;t done this for simplicity - which causes our validation accuracy to drop.</p>\n<p><a href=\"https://app.labml.ai/run/0fc344da2cd011ecb0bc3fdb2e774a3d\"><span translate=no>_^_0_^_</span></a></p>\n": "<h1><a href=\"index.html\">CIFAR 10 \u0db8\u0dad \u0d9a\u0ddc\u0db1\u0dca\u0dc0\u0dd3 \u0db8\u0dd2\u0d9a\u0dca\u0dc3\u0dbb\u0dca</a> \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0d9a\u0dbb\u0db1\u0dca\u0db1</h1>\n<p>\u0db8\u0dd9\u0db8\u0dad\u0dd2\u0dbb \u0dbb\u0da0\u0db1\u0dba CIFAR \u0db8\u0dad ConvMixer \u0daf\u0dd4\u0db8\u0dca\u0dbb\u0dd2\u0dba 10 \u0daf\u0dad\u0dca\u0dad \u0dc3\u0db8\u0dd4\u0daf\u0dcf\u0dba. </p>\n<p>\u0db8\u0dd9\u0dba\u0d9a\u0da9\u0daf\u0dcf\u0dc3\u0dd2 \u0db4\u0dca\u0dbb\u0dad\u0dd2 results \u0dbd \u0db4\u0dca\u0dbb\u0dad\u0dd2\u0db1\u0dd2\u0dc2\u0dca\u0db4\u0dcf\u0daf\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0d8b\u0dad\u0dca\u0dc3\u0dcf\u0dc4\u0dba\u0d9a\u0dca \u0db1\u0ddc\u0dc0\u0dda. \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0dc0 \u0dc3\u0db3\u0dc4\u0dcf <a href=\"https://github.com/rwightman/pytorch-image-models\">PyTorch Image Models (timm)</a> \u0dc4\u0dd2 \u0d87\u0dad\u0dd2 \u0dbb\u0dd6\u0db4 \u0dc0\u0dd0\u0da9\u0dd2\u0daf\u0dd2\u0dba\u0dd4\u0dab\u0dd4 \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dca \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dbb\u0dba\u0dd2. \u0dc3\u0dbb\u0dbd \u0db6\u0dc0 \u0dc3\u0db3\u0dc4\u0dcf \u0d85\u0db4\u0dd2 \u0db8\u0dd9\u0dba \u0d9a\u0dbb \u0db1\u0dd0\u0dad - \u0d91\u0db8\u0d9f\u0dd2\u0db1\u0dca \u0d85\u0db4\u0d9c\u0dda \u0dc0\u0dbd\u0d82\u0d9c\u0dd4 \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0db1\u0dd2\u0dbb\u0dc0\u0daf\u0dca\u0dba\u0dad\u0dcf\u0dc0\u0dba \u0db4\u0dc4\u0dad \u0dc0\u0dd0\u0da7\u0dd3\u0db8\u0da7 \u0dc4\u0dda\u0dad\u0dd4 \u0dc0\u0dda. </p>\n<p><a href=\"https://app.labml.ai/run/0fc344da2cd011ecb0bc3fdb2e774a3d\"><span translate=no>_^_0_^_</span></a></p>\n",
"<h2>Configurations</h2>\n<p>We use <a href=\"../experiments/cifar10.html\"><span translate=no>_^_0_^_</span></a> which defines all the dataset related configurations, optimizer, and a training loop.</p>\n": "<h2>\u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3\u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dca</h2>\n<p>\u0dc3\u0dd2\u0dba\u0dbd\u0dd4\u0db8\u0daf\u0dad\u0dca\u0dad \u0d9a\u0da7\u0dca\u0da7\u0dbd \u0d86\u0dc1\u0dca\u0dbb\u0dd2\u0dad \u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3\u0dba\u0db1\u0dca, \u0db4\u0dca\u0dbb\u0dc1\u0dc3\u0dca\u0dad\u0d9a\u0dbb\u0dab\u0dba \u0dc3\u0dc4 \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0dbd\u0dd6\u0db4\u0dba\u0d9a\u0dca \u0db1\u0dd2\u0dbb\u0dca\u0dc0\u0da0\u0db1\u0dba \u0d9a\u0dbb\u0db1 \u0d85\u0db4\u0dd2 \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf <a href=\"../experiments/cifar10.html\"><span translate=no>_^_0_^_</span></a> \u0d9a\u0dbb\u0db8\u0dd4. </p>\n",
"<h3>Create model</h3>\n": "<h3>\u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba\u0dc3\u0dcf\u0daf\u0db1\u0dca\u0db1</h3>\n",
"<p> </p>\n": "<p> </p>\n",
"<p>Create ConvMixer </p>\n": "<p>ConvMixer\u0db1\u0dd2\u0dbb\u0dca\u0db8\u0dcf\u0dab\u0dba </p>\n",
"<p>Create configurations </p>\n": "<p>\u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3\u0dba\u0db1\u0dca\u0dc3\u0dcf\u0daf\u0db1\u0dca\u0db1 </p>\n",
"<p>Create experiment </p>\n": "<p>\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf\u0db6\u0dd0\u0dbd\u0dd3\u0db8 \u0dc3\u0dcf\u0daf\u0db1\u0dca\u0db1 </p>\n",
"<p>Do not augment images for validation </p>\n": "<p>\u0dc0\u0dbd\u0d82\u0d9c\u0dd4\u0d9a\u0dd2\u0dbb\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0dbb\u0dd6\u0db4 \u0dc0\u0dd0\u0da9\u0dd2 \u0db1\u0ddc\u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
"<p>Kernel size of the depth-wise convolution, <span translate=no>_^_0_^_</span> </p>\n": "<p>\u0d9c\u0dd0\u0db9\u0dd4\u0dbb-\u0db1\u0dd0\u0dab\u0dc0\u0dad\u0dca\u0dc3\u0d82\u0dc0\u0dc4\u0db1\u0dba\u0dda \u0d9a\u0dbb\u0dca\u0db1\u0dbd\u0dca \u0db4\u0dca\u0dbb\u0db8\u0dcf\u0dab\u0dba, <span translate=no>_^_0_^_</span> </p>\n",
"<p>Load configurations </p>\n": "<p>\u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3\u0dba\u0db1\u0dca\u0db4\u0dd6\u0dbb\u0dab\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
"<p>Number of <a href=\"#ConvMixerLayer\">ConvMixer layers</a> or depth, <span translate=no>_^_0_^_</span> </p>\n": "<p><a href=\"#ConvMixerLayer\">ConvMixer \u0dc3\u0dca\u0dae\u0dbb</a> \u0dc4\u0ddd \u0d9c\u0dd0\u0db9\u0dd4\u0dbb \u0d9c\u0dab\u0db1, <span translate=no>_^_0_^_</span> </p>\n",
"<p>Number of channels in patch embeddings, <span translate=no>_^_0_^_</span> </p>\n": "<p>\u0db4\u0dd0\u0da0\u0dca\u0d9a\u0dcf\u0dc0\u0dd0\u0daf\u0dca\u0daf\u0dd3\u0db8\u0dca \u0dc0\u0dbd \u0db1\u0dcf\u0dbd\u0dd2\u0d9a\u0dcf \u0d9c\u0dab\u0db1, <span translate=no>_^_0_^_</span> </p>\n",
"<p>Number of classes in the task </p>\n": "<p>\u0d9a\u0dbb\u0dca\u0dad\u0dc0\u0dca\u0dba\u0dba\u0dda\u0db4\u0db1\u0dca\u0dad\u0dd2 \u0d9c\u0dab\u0db1 </p>\n",
"<p>Optimizer </p>\n": "<p>\u0db4\u0dca\u0dbb\u0dc1\u0dc3\u0dca\u0dad\u0d9a\u0dbb\u0dab\u0dba </p>\n",
"<p>Set model for saving/loading </p>\n": "<p>\u0d89\u0dad\u0dd2\u0dbb\u0dd2\u0d9a\u0dd2\u0dbb\u0dd3\u0db8/\u0db4\u0dd0\u0da7\u0dc0\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba \u0dc3\u0d9a\u0dc3\u0db1\u0dca\u0db1 </p>\n",
"<p>Simple image augmentations </p>\n": "<p>\u0dc3\u0dbb\u0dbd\u0dbb\u0dd6\u0db4 \u0dc0\u0dd0\u0da9\u0dd2 \u0d9a\u0dd2\u0dbb\u0dd3\u0db8 </p>\n",
"<p>Size of a patch, <span translate=no>_^_0_^_</span> </p>\n": "<p>\u0db4\u0dd0\u0da0\u0dca\u0d91\u0d9a\u0d9a \u0db4\u0dca\u0dbb\u0db8\u0dcf\u0dab\u0dba, <span translate=no>_^_0_^_</span> </p>\n",
"<p>Start the experiment and run the training loop </p>\n": "<p>\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf\u0db6\u0dd0\u0dbd\u0dd3\u0db8 \u0d86\u0dbb\u0db8\u0dca\u0db7 \u0d9a\u0dbb \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0dbd\u0dd6\u0db4\u0dba \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
"<p>Training epochs and batch size </p>\n": "<p>\u0d8a\u0db4\u0ddc\u0da0\u0dca\u0dc3\u0dc4 \u0d9a\u0dab\u0dca\u0da9\u0dcf\u0dba\u0db8\u0dca \u0db4\u0dca\u0dbb\u0db8\u0dcf\u0dab\u0dba \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0d9a\u0dd2\u0dbb\u0dd3\u0db8 </p>\n",
"Train ConvMixer on CIFAR 10": "CIFAR 10 \u0db8\u0dad \u0daf\u0dd4\u0db8\u0dca\u0dbb\u0dd2\u0dba \u0d9a\u0ddc\u0db1\u0dca\u0dc0\u0dd3 \u0db8\u0dd2\u0d9a\u0dca\u0dc3\u0dbb\u0dca"
}
translate_cache/conv_mixer/experiment.zh.json
+22
View File
@@ -0,0 +1,22 @@
{
"<h1>Train a <a href=\"index.html\">ConvMixer</a> on CIFAR 10</h1>\n<p>This script trains a ConvMixer on CIFAR 10 dataset.</p>\n<p>This is not an attempt to reproduce the results of the paper. The paper uses image augmentations present in <a href=\"https://github.com/rwightman/pytorch-image-models\">PyTorch Image Models (timm)</a> for training. We haven&#x27;t done this for simplicity - which causes our validation accuracy to drop.</p>\n": "<h1>\u5728 CIFA <a href=\"index.html\">R 10 \u4e0a\u8bad\u7ec3 convMixer</a></h1>\n<p>\u6b64\u811a\u672c\u5728 CIFAR 10 \u6570\u636e\u96c6\u4e0a\u8bad\u7ec3 ConvMixer\u3002</p>\n<p>\u8fd9\u5e76\u4e0d\u662f\u8bd5\u56fe\u91cd\u73b0\u8bba\u6587\u7684\u7ed3\u679c\u3002\u672c\u6587\u4f7f\u7528 <a href=\"https://github.com/rwightman/pytorch-image-models\">PyTorch \u56fe\u50cf\u6a21\u578b (timm) \u4e2d\u5b58\u5728\u7684\u56fe\u50cf</a>\u589e\u5f3a\u8fdb\u884c\u8bad\u7ec3\u3002\u4e3a\u4e86\u7b80\u5355\u8d77\u89c1\uff0c\u6211\u4eec\u6ca1\u6709\u8fd9\u6837\u505a\u2014\u2014\u8fd9\u4f1a\u5bfc\u81f4\u6211\u4eec\u7684\u9a8c\u8bc1\u7cbe\u5ea6\u4e0b\u964d\u3002</p>\n",
"<h2>Configurations</h2>\n<p>We use <a href=\"../experiments/cifar10.html\"><span translate=no>_^_0_^_</span></a> which defines all the dataset related configurations, optimizer, and a training loop.</p>\n": "<h2>\u914d\u7f6e</h2>\n<p>\u6211\u4eec\u4f7f\u7528<a href=\"../experiments/cifar10.html\"><span translate=no>_^_0_^_</span></a>\u5b83\u6765\u5b9a\u4e49\u6240\u6709\u4e0e\u6570\u636e\u96c6\u76f8\u5173\u7684\u914d\u7f6e\u3001\u4f18\u5316\u5668\u548c\u8bad\u7ec3\u5faa\u73af\u3002</p>\n",
"<h3>Create model</h3>\n": "<h3>\u521b\u5efa\u6a21\u578b</h3>\n",
"<p> </p>\n": "<p></p>\n",
"<p>Create ConvMixer </p>\n": "<p>\u521b\u5efa\u6df7\u97f3\u5668</p>\n",
"<p>Create configurations </p>\n": "<p>\u521b\u5efa\u914d\u7f6e</p>\n",
"<p>Create experiment </p>\n": "<p>\u521b\u5efa\u5b9e\u9a8c</p>\n",
"<p>Do not augment images for validation </p>\n": "<p>\u4e0d\u8981\u6269\u5145\u56fe\u50cf\u4ee5\u8fdb\u884c\u9a8c\u8bc1</p>\n",
"<p>Kernel size of the depth-wise convolution, <span translate=no>_^_0_^_</span> </p>\n": "<p>\u6df1\u5ea6\u5377\u79ef\u7684\u5185\u6838\u5927\u5c0f\uff0c<span translate=no>_^_0_^_</span></p>\n",
"<p>Load configurations </p>\n": "<p>\u88c5\u8f7d\u914d\u7f6e</p>\n",
"<p>Number of <a href=\"#ConvMixerLayer\">ConvMixer layers</a> or depth, <span translate=no>_^_0_^_</span> </p>\n": "<p><a href=\"#ConvMixerLayer\">ConvMixer \u5c42</a>\u6570\u6216\u6df1\u5ea6\uff0c<span translate=no>_^_0_^_</span></p>\n",
"<p>Number of channels in patch embeddings, <span translate=no>_^_0_^_</span> </p>\n": "<p>\u8865\u4e01\u5d4c\u5165\u4e2d\u7684\u901a\u9053\u6570\uff0c<span translate=no>_^_0_^_</span></p>\n",
"<p>Number of classes in the task </p>\n": "<p>\u4efb\u52a1\u4e2d\u7684\u7c7b\u6570</p>\n",
"<p>Optimizer </p>\n": "<p>\u4f18\u5316\u5668</p>\n",
"<p>Set model for saving/loading </p>\n": "<p>\u8bbe\u7f6e\u4fdd\u5b58/\u52a0\u8f7d\u7684\u6a21\u578b</p>\n",
"<p>Simple image augmentations </p>\n": "<p>\u7b80\u5355\u7684\u56fe\u50cf\u589e\u5f3a</p>\n",
"<p>Size of a patch, <span translate=no>_^_0_^_</span> </p>\n": "<p>\u8865\u4e01\u7684\u5927\u5c0f\uff0c<span translate=no>_^_0_^_</span></p>\n",
"<p>Start the experiment and run the training loop </p>\n": "<p>\u5f00\u59cb\u5b9e\u9a8c\u5e76\u8fd0\u884c\u8bad\u7ec3\u5faa\u73af</p>\n",
"<p>Training epochs and batch size </p>\n": "<p>\u8bad\u7ec3\u5468\u671f\u548c\u6279\u6b21\u5927\u5c0f</p>\n",
"Train ConvMixer on CIFAR 10": "\u5728 CIFAR 10 \u4e0a\u8bad\u7ec3 ConvMixer"
}
translate_cache/conv_mixer/readme.ja.json
+4
View File
@@ -0,0 +1,4 @@
{
" Patches Are All You Need?": " \u5fc5\u8981\u306a\u306e\u306f\u30d1\u30c3\u30c1\u3060\u3051\uff1f",
"<h1><a href=\"https://nn.labml.ai/conv_mixer/index.html\">Patches Are All You Need?</a></h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of the paper <a href=\"https://arxiv.org/abs/2201.09792\">Patches Are All You Need?</a>.</p>\n<p>ConvMixer is Similar to <a href=\"https://nn.labml.ai/transformers/mlp_mixer/index.html\">MLP-Mixer</a>. MLP-Mixer separates mixing of spatial and channel dimensions, by applying an MLP across spatial dimension and then an MLP across the channel dimension (spatial MLP replaces the <a href=\"https://nn.labml.ai/transformers/vit/index.html\">ViT</a> attention and channel MLP is the <a href=\"https://nn.labml.ai/transformers/feed_forward.html\">FFN</a> of ViT).</p>\n<p>ConvMixer uses a 1x1 convolution for channel mixing and a depth-wise convolution for spatial mixing. Since it&#x27;s a convolution instead of a full MLP across the space, it mixes only the nearby batches in contrast to ViT or MLP-Mixer. Also, the MLP-mixer uses MLPs of two layers for each mixing and ConvMixer uses a single layer for each mixing.</p>\n<p>The paper recommends removing the residual connection across the channel mixing (point-wise convolution) and having only a residual connection over the spatial mixing (depth-wise convolution). They also use <a href=\"https://nn.labml.ai/normalization/batch_norm/index.html\">Batch normalization</a> instead of <a href=\"../normalization/layer_norm/index.html\">Layer normalization</a>.</p>\n<p>Here&#x27;s <a href=\"https://nn.labml.ai/conv_mixer/experiment.html\">an experiment</a> that trains ConvMixer on CIFAR-10. </p>\n": "<h1><a href=\"https://nn.labml.ai/conv_mixer/index.html\">\u5fc5\u8981\u306a\u306e\u306f\u30d1\u30c3\u30c1\u3060\u3051\uff1f</a></h1>\n<p><a href=\"https://pytorch.org\">\u3053\u308c\u306f\u7d19\u306e\u30d1\u30c3\u30c1\u3092PyTorch\u3067\u5b9f\u88c5\u3057\u305f\u3082\u306e\u3067\u3059</a><a href=\"https://arxiv.org/abs/2201.09792\">\u3002\u5fc5\u8981\u306a\u306e\u306f\u30d1\u30c3\u30c1\u3060\u3051\u3067\u3059\u304b</a>\uff1f</p>\u3002\n<p><a href=\"https://nn.labml.ai/transformers/mlp_mixer/index.html\">ConvMixer\u306fMLP\u30df\u30ad\u30b5\u30fc\u306b\u4f3c\u3066\u3044\u307e\u3059\u3002</a></p><a href=\"https://nn.labml.ai/transformers/feed_forward.html\">MLP-Mixer\u306f\u3001\u7a7a\u9593\u6b21\u5143\u5168\u4f53\u306bMLP\u3092\u9069\u7528\u3057\u3001\u6b21\u306b\u30c1\u30e3\u30cd\u30eb\u6b21\u5143\u5168\u4f53\u306bMLP\u3092\u9069\u7528\u3059\u308b\u3053\u3068\u3067\u3001\u7a7a\u9593\u6b21\u5143\u3068\u30c1\u30e3\u30cd\u30eb\u6b21\u5143\u306e\u6df7\u5408\u3092\u5206\u96e2\u3057\u307e\u3059\uff08\u7a7a\u9593MLP\u306fvIT\u306e\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u306b\u4ee3\u308f\u308a\u3001<a href=\"https://nn.labml.ai/transformers/vit/index.html\">\u30c1\u30e3\u30cd\u30ebMLP\u306fVIT\u306eFFN\u3067\u3059</a>\uff09\u3002</a>\n<p>ConvMixer\u306f\u3001\u30c1\u30e3\u30f3\u30cd\u30eb\u30df\u30ad\u30b7\u30f3\u30b0\u306b1x1\u306e\u30b3\u30f3\u30dc\u30ea\u30e5\u30fc\u30b7\u30e7\u30f3\u3092\u4f7f\u7528\u3057\u3001\u7a7a\u9593\u30df\u30ad\u30b7\u30f3\u30b0\u306b\u5965\u884c\u304d\u30b3\u30f3\u30dc\u30ea\u30e5\u30fc\u30b7\u30e7\u30f3\u3092\u4f7f\u7528\u3057\u307e\u3059\u3002\u30b9\u30da\u30fc\u30b9\u5168\u4f53\u3067\u30d5\u30ebMLP\u3067\u306f\u306a\u304f\u7573\u307f\u8fbc\u307f\u306a\u306e\u3067\u3001VIT\u3084MLP\u30df\u30ad\u30b5\u30fc\u3068\u306f\u5bfe\u7167\u7684\u306b\u3001\u8fd1\u304f\u306e\u30d0\u30c3\u30c1\u306e\u307f\u3092\u30df\u30ad\u30b7\u30f3\u30b0\u3057\u307e\u3059\u3002\u307e\u305f\u3001MLP\u30df\u30ad\u30b5\u30fc\u306f\u30df\u30ad\u30b7\u30f3\u30b0\u3054\u3068\u306b2\u5c64\u306eMLP\u3092\u4f7f\u7528\u3057\u3001ConvMixer\u306f\u30df\u30ad\u30b7\u30f3\u30b0\u3054\u3068\u306b1\u5c64\u306eMLP\u3092\u4f7f\u7528\u3057\u307e\u3059</p>\u3002\n<p>\u3053\u306e\u8ad6\u6587\u3067\u306f\u3001\u30c1\u30e3\u30cd\u30eb\u30df\u30ad\u30b7\u30f3\u30b0\u5168\u4f53\u306e\u6b8b\u7559\u63a5\u7d9a\u3092\u524a\u9664\u3057\uff08\u70b9\u5358\u4f4d\u306e\u7573\u307f\u8fbc\u307f\uff09\u3001\u7a7a\u9593\u30df\u30ad\u30b7\u30f3\u30b0\u3067\u306f\u6b8b\u7559\u63a5\u7d9a\u306e\u307f\u306b\u3059\u308b\uff08\u6df1\u3055\u65b9\u5411\u306e\u7573\u307f\u8fbc\u307f\uff09\u3053\u3068\u3092\u63a8\u5968\u3057\u3066\u3044\u307e\u3059\u3002\u307e\u305f\u3001</p><a href=\"https://nn.labml.ai/normalization/batch_norm/index.html\"><a href=\"../normalization/layer_norm/index.html\">\u30ec\u30a4\u30e4\u30fc\u6b63\u898f\u5316\u306e\u4ee3\u308f\u308a\u306b\u30d0\u30c3\u30c1\u6b63\u898f\u5316\u3092\u4f7f\u7528\u3057\u307e\u3059</a></a>\u3002\n<p>\u3053\u308c\u306f<a href=\"https://nn.labml.ai/conv_mixer/experiment.html\">\u3001CIFAR-10 \u3067 ConvMixer \u3092\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3059\u308b\u5b9f\u9a13\u3067\u3059</a>\u3002</p>\n"
}
translate_cache/conv_mixer/readme.si.json
+4
View File
File diff suppressed because one or more lines are too long
translate_cache/conv_mixer/readme.zh.json
+4
View File
@@ -0,0 +1,4 @@
{
" Patches Are All You Need?": " \u8865\u4e01\u662f\u4f60\u6240\u9700\u8981\u7684\u5417\uff1f",
"<h1><a href=\"https://nn.labml.ai/conv_mixer/index.html\">Patches Are All You Need?</a></h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of the paper <a href=\"https://arxiv.org/abs/2201.09792\">Patches Are All You Need?</a>.</p>\n<p>ConvMixer is Similar to <a href=\"https://nn.labml.ai/transformers/mlp_mixer/index.html\">MLP-Mixer</a>. MLP-Mixer separates mixing of spatial and channel dimensions, by applying an MLP across spatial dimension and then an MLP across the channel dimension (spatial MLP replaces the <a href=\"https://nn.labml.ai/transformers/vit/index.html\">ViT</a> attention and channel MLP is the <a href=\"https://nn.labml.ai/transformers/feed_forward.html\">FFN</a> of ViT).</p>\n<p>ConvMixer uses a 1x1 convolution for channel mixing and a depth-wise convolution for spatial mixing. Since it&#x27;s a convolution instead of a full MLP across the space, it mixes only the nearby batches in contrast to ViT or MLP-Mixer. Also, the MLP-mixer uses MLPs of two layers for each mixing and ConvMixer uses a single layer for each mixing.</p>\n<p>The paper recommends removing the residual connection across the channel mixing (point-wise convolution) and having only a residual connection over the spatial mixing (depth-wise convolution). They also use <a href=\"https://nn.labml.ai/normalization/batch_norm/index.html\">Batch normalization</a> instead of <a href=\"../normalization/layer_norm/index.html\">Layer normalization</a>.</p>\n<p>Here&#x27;s <a href=\"https://nn.labml.ai/conv_mixer/experiment.html\">an experiment</a> that trains ConvMixer on CIFAR-10. </p>\n": "<h1><a href=\"https://nn.labml.ai/conv_mixer/index.html\">\u4f60\u53ea\u9700\u8981\u8865\u4e01\u5417\uff1f</a></h1>\n<p>\u8fd9\u662f <a href=\"https://pytorch.org\">PyTorch</a> \u5bf9\u8bba\u6587\u300a<a href=\"https://arxiv.org/abs/2201.09792\">\u8865\u4e01\u5c31\u662f\u4f60\u6240\u9700\u8981\u7684\uff1f</a>\u300b\u7684\u5b9e\u73b0</p>\u3002\n<p>convMixer \u7c7b\u4f3c\u4e8e <a href=\"https://nn.labml.ai/transformers/mlp_mixer/index.html\">MLP \u6df7\u97f3\u5668</a>\u3002MLP-Mixer \u901a\u8fc7\u5728\u7a7a\u95f4\u7ef4\u5ea6\u4e0a\u5e94\u7528 MLP\uff0c\u7136\u540e\u5728\u4fe1\u9053\u7ef4\u5ea6\u4e0a\u5e94\u7528 MLP \u6765\u5206\u79bb\u7a7a\u95f4\u7ef4\u5ea6\u548c\u4fe1\u9053\u7ef4\u5ea6\u7684\u6df7\u97f3\uff08\u7a7a\u95f4 MLP \u53d6\u4ee3 <a href=\"https://nn.labml.ai/transformers/vit/index.html\">vIT</a> \u6ce8\u610f\u529b\uff0c\u4fe1\u9053 MLP \u662f ViT \u7684 <a href=\"https://nn.labml.ai/transformers/feed_forward.html\">FFN</a>\uff09\u3002</p>\n<p>ConvMixer \u4f7f\u7528 1x1 \u5377\u79ef\u8fdb\u884c\u901a\u9053\u6df7\u5408\uff0c\u4f7f\u7528\u6df1\u5ea6\u5377\u79ef\u8fdb\u884c\u7a7a\u95f4\u6df7\u5408\u3002\u7531\u4e8e\u5b83\u662f\u5377\u79ef\u800c\u4e0d\u662f\u6574\u4e2a\u7a7a\u95f4\u7684\u5b8c\u6574\u7684 MLP\uff0c\u56e0\u6b64\u4e0e vIT \u6216 MLP-Mixer \u76f8\u6bd4\uff0c\u5b83\u53ea\u6df7\u5408\u9644\u8fd1\u7684\u6279\u6b21\u3002\u6b64\u5916\uff0cMLP-Mixer \u6bcf\u6b21\u6df7\u5408\u4f7f\u7528\u4e24\u5c42 MLP\uff0cConvMixer \u6bcf\u6b21\u6df7\u5408\u4f7f\u7528\u5355\u5c42\u3002</p>\n<p>\u8be5\u8bba\u6587\u5efa\u8bae\u5220\u9664\u4fe1\u9053\u6df7\u5408\uff08\u9010\u70b9\u5377\u79ef\uff09\u4e0a\u7684\u5269\u4f59\u8fde\u63a5\uff0c\u5728\u7a7a\u95f4\u6df7\u5408\uff08\u6df1\u5ea6\u5377\u79ef\uff09\u4e0a\u4ec5\u4f7f\u7528\u6b8b\u5dee\u8fde\u63a5\u3002\u4ed6\u4eec\u8fd8\u4f7f\u7528<a href=\"https://nn.labml.ai/normalization/batch_norm/index.html\">\u6279\u91cf\u6807\u51c6\u5316</a>\u800c\u4e0d\u662f<a href=\"../normalization/layer_norm/index.html\">\u56fe\u5c42\u6807\u51c6\u5316</a>\u3002</p>\n<p>\u8fd9\u662f<a href=\"https://nn.labml.ai/conv_mixer/experiment.html\">\u4e00\u9879\u5728 CIFAR-10 \u4e0a\u8bad\u7ec3 ConvMixer \u7684\u5b9e\u9a8c</a>\u3002</p>\n"
}