chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"<h1>Graph Attention Networks (GAT)</h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of the paper <a href=\"https://arxiv.org/abs/1710.10903\">Graph Attention Networks</a>.</p>\n<p>GATs work on graph data. A graph consists of nodes and edges connecting nodes. For example, in Cora dataset the nodes are research papers and the edges are citations that connect the papers.</p>\n<p>GAT uses masked self-attention, kind of similar to <a href=\"../../transformers/mha.html\">transformers</a>. GAT consists of graph attention layers stacked on top of each other. Each graph attention layer gets node embeddings as inputs and outputs transformed embeddings. The node embeddings pay attention to the embeddings of other nodes it's connected to. The details of graph attention layers are included alongside the implementation.</p>\n<p>Here is <a href=\"experiment.html\">the training code</a> for training a two-layer GAT on Cora dataset.</p>\n": "<h1>\u30b0\u30e9\u30d5\u30fb\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30fb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af (GAT)</h1>\n<p>\u3053\u308c\u306f\u8ad6\u6587\u306e\u300c<a href=\"https://arxiv.org/abs/1710.10903\">\u30b0\u30e9\u30d5\u30fb\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30fb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af</a>\u300d\u306e <a href=\"https://pytorch.org\">PyTorch</a> \u5b9f\u88c5\u3067\u3059\u3002</p>\n<p>GAT \u306f\u30b0\u30e9\u30d5\u30c7\u30fc\u30bf\u3092\u51e6\u7406\u3057\u307e\u3059\u3002\u30b0\u30e9\u30d5\u306f\u3001\u30ce\u30fc\u30c9\u3068\u30ce\u30fc\u30c9\u3092\u63a5\u7d9a\u3059\u308b\u30a8\u30c3\u30b8\u3067\u69cb\u6210\u3055\u308c\u307e\u3059\u3002\u305f\u3068\u3048\u3070\u3001Cora\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3067\u306f\u3001\u30ce\u30fc\u30c9\u306f\u7814\u7a76\u8ad6\u6587\u3067\u3001\u7aef\u306f\u8ad6\u6587\u3092\u3064\u306a\u3050\u5f15\u7528\u3067\u3059</p>\u3002\n<p><a href=\"../../transformers/mha.html\">GAT\u306f\u3001\u30c8\u30e9\u30f3\u30b9\u30d5\u30a9\u30fc\u30de\u30fc\u306b\u4f3c\u305f\u3001\u30de\u30b9\u30af\u3055\u308c\u305f\u30bb\u30eb\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u3092\u4f7f\u3044\u307e\u3059\u3002</a>GAT\u306f\u3001\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30ec\u30a4\u30e4\u30fc\u304c\u4e92\u3044\u306b\u91cd\u306a\u308a\u5408\u3063\u3066\u69cb\u6210\u3055\u308c\u3066\u3044\u307e\u3059\u3002\u5404\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30ec\u30a4\u30e4\u30fc\u306f\u3001\u5165\u529b\u3068\u3057\u3066\u30ce\u30fc\u30c9\u57cb\u3081\u8fbc\u307f\u3092\u53d6\u5f97\u3057\u3001\u5909\u63db\u3055\u308c\u305f\u57cb\u3081\u8fbc\u307f\u3092\u51fa\u529b\u3057\u307e\u3059\u3002\u30ce\u30fc\u30c9\u57cb\u3081\u8fbc\u307f\u306f\u3001\u63a5\u7d9a\u3055\u308c\u3066\u3044\u308b\u4ed6\u306e\u30ce\u30fc\u30c9\u306e\u57cb\u3081\u8fbc\u307f\u306b\u6ce8\u76ee\u3057\u307e\u3059\u3002\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30ec\u30a4\u30e4\u30fc\u306e\u8a73\u7d30\u306f\u3001\u5b9f\u88c5\u3068\u3068\u3082\u306b\u542b\u307e\u308c\u3066\u3044\u307e\u3059\u3002</p>\n<p><a href=\"experiment.html\">Cora \u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3067 2 \u5c64 GAT \u3092\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3059\u308b\u305f\u3081\u306e\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30b3\u30fc\u30c9\u3092\u6b21\u306b\u793a\u3057\u307e\u3059</a>\u3002</p>\n",
|
||||
"<h2>Graph attention layer</h2>\n<p>This is a single graph attention layer. A GAT is made up of multiple such layers.</p>\n<p>It takes <span translate=no>_^_0_^_</span>, where <span translate=no>_^_1_^_</span> as input and outputs <span translate=no>_^_2_^_</span>, where <span translate=no>_^_3_^_</span>.</p>\n": "<h2>\u30b0\u30e9\u30d5\u30fb\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30fb\u30ec\u30a4\u30e4\u30fc</h2>\n<p>\u3053\u308c\u306f\u5358\u4e00\u306e\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30ec\u30a4\u30e4\u30fc\u3067\u3059\u3002GAT \u306f\u3053\u306e\u3088\u3046\u306a\u8907\u6570\u306e\u30ec\u30a4\u30e4\u30fc\u3067\u69cb\u6210\u3055\u308c\u3066\u3044\u307e\u3059</p>\u3002\n<p><span translate=no>_^_1_^_</span>\u5165\u529b\u3068\u3057\u3066<span translate=no>_^_0_^_</span>\u3001where \u3092\u3001\u51fa\u529b\u3068\u3057\u3066<span translate=no>_^_2_^_</span>\u3001where <span translate=no>_^_3_^_</span> \u3092\u53d6\u308a\u307e\u3059\u3002</p>\n",
|
||||
"<h4>Calculate attention score</h4>\n<p>We calculate these for each head <span translate=no>_^_0_^_</span>. <em>We have omitted <span translate=no>_^_1_^_</span> for simplicity</em>.</p>\n<p><span translate=no>_^_2_^_</span></p>\n<p><span translate=no>_^_3_^_</span> is the attention score (importance) from node <span translate=no>_^_4_^_</span> to node <span translate=no>_^_5_^_</span>. We calculate this for each head.</p>\n<p><span translate=no>_^_6_^_</span> is the attention mechanism, that calculates the attention score. The paper concatenates <span translate=no>_^_7_^_</span>, <span translate=no>_^_8_^_</span> and does a linear transformation with a weight vector <span translate=no>_^_9_^_</span> followed by a <span translate=no>_^_10_^_</span>.</p>\n<p><span translate=no>_^_11_^_</span> </p>\n": "<h4>\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30b9\u30b3\u30a2\u306e\u8a08\u7b97</h4>\n<p><span translate=no>_^_0_^_</span>\u3053\u308c\u3089\u306f\u982d\u3054\u3068\u306b\u8a08\u7b97\u3057\u307e\u3059\u3002<em><span translate=no>_^_1_^_</span>\u308f\u304b\u308a\u3084\u3059\u304f\u3059\u308b\u305f\u3081\u306b\u7701\u7565\u3057\u307e\u3057\u305f\u3002</em></p>\n<p><span translate=no>_^_2_^_</span></p>\n<p><span translate=no>_^_3_^_</span><span translate=no>_^_4_^_</span>\u30ce\u30fc\u30c9\u3054\u3068\u306e\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30b9\u30b3\u30a2\uff08\u91cd\u8981\u5ea6\uff09<span translate=no>_^_5_^_</span>\u3067\u3059\u3002\u3053\u308c\u3092\u982d\u3054\u3068\u306b\u8a08\u7b97\u3057\u307e\u3059\u3002</p>\n<p><span translate=no>_^_6_^_</span>\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30b9\u30b3\u30a2\u3092\u8a08\u7b97\u3059\u308b\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30e1\u30ab\u30cb\u30ba\u30e0\u3067\u3059\u3002\u3053\u306e\u8ad6\u6587\u3067\u306f<span translate=no>_^_7_^_</span>\u3001<span translate=no>_^_8_^_</span><span translate=no>_^_9_^_</span>\u91cd\u307f\u30d9\u30af\u30c8\u30eb\u306e\u5f8c\u306ba\u3092\u9023\u7d50\u3057\u3001\u7dda\u5f62\u5909\u63db\u3092\u884c\u3044\u307e\u3059</p>\u3002<span translate=no>_^_10_^_</span>\n<p><span translate=no>_^_11_^_</span></p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span> </p>\n": "<p><span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span> gets <span translate=no>_^_1_^_</span> where each node embedding is repeated <span translate=no>_^_2_^_</span> times. </p>\n": "<p><span translate=no>_^_0_^_</span><span translate=no>_^_1_^_</span><span translate=no>_^_2_^_</span>\u5404\u30ce\u30fc\u30c9\u306e\u57cb\u3081\u8fbc\u307f\u304c\u4f55\u5ea6\u3082\u7e70\u308a\u8fd4\u3055\u308c\u308b\u5834\u6240\u3092\u53d6\u5f97\u3057\u307e\u3059\u3002</p>\n",
|
||||
"<p>Apply dropout regularization </p>\n": "<p>\u30c9\u30ed\u30c3\u30d7\u30a2\u30a6\u30c8\u6b63\u5247\u5316\u3092\u9069\u7528</p>\n",
|
||||
"<p>Calculate <span translate=no>_^_0_^_</span> <span translate=no>_^_1_^_</span> is of shape <span translate=no>_^_2_^_</span> </p>\n": "<p><span translate=no>_^_0_^_</span><span translate=no>_^_1_^_</span>\u5f62\u72b6\u306e\u8a08\u7b97 <span translate=no>_^_2_^_</span></p>\n",
|
||||
"<p>Calculate final output for each head <span translate=no>_^_0_^_</span></p>\n<p><em>Note:</em> The paper includes the final activation <span translate=no>_^_1_^_</span> in <span translate=no>_^_2_^_</span> We have omitted this from the Graph Attention Layer implementation and use it on the GAT model to match with how other PyTorch modules are defined - activation as a separate layer. </p>\n": "<p>\u5404\u30d8\u30c3\u30c9\u306e\u6700\u7d42\u51fa\u529b\u3092\u8a08\u7b97 <span translate=no>_^_0_^_</span></p>\n<p><em>\u6ce8:</em><span translate=no>_^_1_^_</span><span translate=no>_^_2_^_</span>\u3053\u306e\u30db\u30ef\u30a4\u30c8\u30da\u30fc\u30d1\u30fc\u3067\u306f\u3001Graph Attention Layer \u306e\u5b9f\u88c5\u304b\u3089\u306f\u7701\u7565\u3057\u3001\u4ed6\u306e PyTorch \u30e2\u30b8\u30e5\u30fc\u30eb\u306e\u5b9a\u7fa9\u306b\u5408\u308f\u305b\u3066 GAT \u30e2\u30c7\u30eb\u3067\u4f7f\u7528\u3057\u3066\u3044\u307e\u3059\u3002\u3064\u307e\u308a\u3001\u30a2\u30af\u30c6\u30a3\u30d9\u30fc\u30b7\u30e7\u30f3\u306f\u5225\u306e\u30ec\u30a4\u30e4\u30fc\u3068\u3057\u3066\u884c\u308f\u308c\u307e\u3059\u3002</p>\n",
|
||||
"<p>Calculate the number of dimensions per head </p>\n": "<p>\u982d\u3042\u305f\u308a\u306e\u5bf8\u6cd5\u6570\u306e\u8a08\u7b97</p>\n",
|
||||
"<p>Concatenate the heads </p>\n": "<p>\u30d8\u30c3\u30c9\u3092\u9023\u7d50\u3057\u3066\u304f\u3060\u3055\u3044</p>\n",
|
||||
"<p>Dropout layer to be applied for attention </p>\n": "<p>\u6ce8\u76ee\u3059\u3079\u304d\u30c9\u30ed\u30c3\u30d7\u30a2\u30a6\u30c8\u5c64</p>\n",
|
||||
"<p>First we calculate <span translate=no>_^_0_^_</span> for all pairs of <span translate=no>_^_1_^_</span>.</p>\n<p><span translate=no>_^_2_^_</span> gets <span translate=no>_^_3_^_</span> where each node embedding is repeated <span translate=no>_^_4_^_</span> times. </p>\n": "<p>\u307e\u305a<span translate=no>_^_0_^_</span>\u3001\u3059\u3079\u3066\u306e\u30da\u30a2\u3092\u8a08\u7b97\u3057\u307e\u3059<span translate=no>_^_1_^_</span>.</p>\n<p><span translate=no>_^_2_^_</span><span translate=no>_^_3_^_</span><span translate=no>_^_4_^_</span>\u5404\u30ce\u30fc\u30c9\u306e\u57cb\u3081\u8fbc\u307f\u304c\u4f55\u5ea6\u3082\u7e70\u308a\u8fd4\u3055\u308c\u308b\u5834\u6240\u3092\u53d6\u5f97\u3057\u307e\u3059\u3002</p>\n",
|
||||
"<p>If we are averaging the multiple heads </p>\n": "<p>\u8907\u6570\u306e\u30d8\u30c3\u30c9\u3092\u5e73\u5747\u5316\u3059\u308b\u5834\u5408</p>\n",
|
||||
"<p>If we are concatenating the multiple heads </p>\n": "<p>\u8907\u6570\u306e\u30d8\u30c3\u30c9\u3092\u9023\u7d50\u3059\u308b\u5834\u5408</p>\n",
|
||||
"<p>Linear layer for initial transformation; i.e. to transform the node embeddings before self-attention </p>\n": "<p>\u521d\u671f\u5909\u63db\u7528\u306e\u7dda\u5f62\u30ec\u30a4\u30e4\u30fc\u3002\u3064\u307e\u308a\u3001\u81ea\u5df1\u51e6\u7406\u306e\u524d\u306b\u30ce\u30fc\u30c9\u57cb\u3081\u8fbc\u307f\u3092\u5909\u63db\u3059\u308b</p>\n",
|
||||
"<p>Linear layer to compute attention score <span translate=no>_^_0_^_</span> </p>\n": "<p>\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30b9\u30b3\u30a2\u3092\u8a08\u7b97\u3059\u308b\u7dda\u5f62\u30ec\u30a4\u30e4\u30fc <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Mask <span translate=no>_^_0_^_</span> based on adjacency matrix. <span translate=no>_^_1_^_</span> is set to <span translate=no>_^_2_^_</span> if there is no edge from <span translate=no>_^_3_^_</span> to <span translate=no>_^_4_^_</span>. </p>\n": "<p><span translate=no>_^_0_^_</span>\u96a3\u63a5\u30de\u30c8\u30ea\u30c3\u30af\u30b9\u306b\u57fa\u3065\u304f\u30de\u30b9\u30af\u3002<span translate=no>_^_1_^_</span><span translate=no>_^_2_^_</span><span translate=no>_^_3_^_</span>\u304b\u3089\u307e\u3067\u306e\u30a8\u30c3\u30b8\u304c\u306a\u3044\u5834\u5408\u306f\u3001\u306b\u8a2d\u5b9a\u3055\u308c\u307e\u3059<span translate=no>_^_4_^_</span>\u3002</p>\n",
|
||||
"<p>Now we concatenate to get <span translate=no>_^_0_^_</span> </p>\n": "<p>\u6b21\u306b\u3001\u9023\u7d50\u3057\u3066\u6b21\u306e\u3088\u3046\u306b\u306a\u308a\u307e\u3059\u3002<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Number of nodes </p>\n": "<p>\u30ce\u30fc\u30c9\u6570</p>\n",
|
||||
"<p>Remove the last dimension of size <span translate=no>_^_0_^_</span> </p>\n": "<p>\u30b5\u30a4\u30ba\u306e\u6700\u5f8c\u306e\u30c7\u30a3\u30e1\u30f3\u30b7\u30e7\u30f3\u3092\u524a\u9664 <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Reshape so that <span translate=no>_^_0_^_</span> is <span translate=no>_^_1_^_</span> </p>\n": "<p><span translate=no>_^_0_^_</span>\u305d\u306e\u3088\u3046\u306b\u5f62\u3092\u5909\u3048\u3066\u304f\u3060\u3055\u3044 <span translate=no>_^_1_^_</span></p>\n",
|
||||
"<p>Softmax to compute attention <span translate=no>_^_0_^_</span> </p>\n": "<p>\u6ce8\u610f\u529b\u3092\u8a08\u7b97\u3059\u308b\u30bd\u30d5\u30c8\u30de\u30c3\u30af\u30b9 <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Take the mean of the heads </p>\n": "<p>\u982d\u306e\u4e2d\u3092\u5e73\u5747\u3057\u3066</p>\n",
|
||||
"<p>The activation for attention score <span translate=no>_^_0_^_</span> </p>\n": "<p>\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30b9\u30b3\u30a2\u306e\u30a2\u30af\u30c6\u30a3\u30d9\u30fc\u30b7\u30e7\u30f3 <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>The adjacency matrix should have shape <span translate=no>_^_0_^_</span> or<span translate=no>_^_1_^_</span> </p>\n": "<p>\u96a3\u63a5\u30de\u30c8\u30ea\u30c3\u30af\u30b9\u306f\u3001<span translate=no>_^_0_^_</span>\u307e\u305f\u306f\u306e\u5f62\u72b6\u3067\u306a\u3051\u308c\u3070\u306a\u308a\u307e\u305b\u3093 <span translate=no>_^_1_^_</span></p>\n",
|
||||
"<p>The initial transformation, <span translate=no>_^_0_^_</span> for each head. We do single linear transformation and then split it up for each head. </p>\n": "<p><span translate=no>_^_0_^_</span>\u5404\u30d8\u30c3\u30c9\u306e\u521d\u671f\u5909\u5f62\u3002\u7dda\u5f62\u5909\u63db\u30921\u3064\u884c\u3044\u3001\u305d\u308c\u3092\u982d\u3054\u3068\u306b\u5206\u5272\u3057\u307e\u3059\u3002</p>\n",
|
||||
"<p>We then normalize attention scores (or coefficients) <span translate=no>_^_0_^_</span></p>\n<p>where <span translate=no>_^_1_^_</span> is the set of nodes connected to <span translate=no>_^_2_^_</span>.</p>\n<p>We do this by setting unconnected <span translate=no>_^_3_^_</span> to <span translate=no>_^_4_^_</span> which makes <span translate=no>_^_5_^_</span> for unconnected pairs. </p>\n": "<p>\u6b21\u306b\u3001\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30b9\u30b3\u30a2 (\u307e\u305f\u306f\u4fc2\u6570) \u3092\u6b63\u898f\u5316\u3057\u307e\u3059 <span translate=no>_^_0_^_</span></p>\n<p><span translate=no>_^_1_^_</span>\u306f\u63a5\u7d9a\u5148\u306e\u30ce\u30fc\u30c9\u30bb\u30c3\u30c8\u304c\u3069\u3053\u306b\u3042\u308b\u304b<span translate=no>_^_2_^_</span>\u3002</p>\n<p>\u305d\u306e\u305f\u3081\u306b\u306f\u3001\u300c\u672a\u63a5\u7d9a\u300d\u3092\u300c\u672a\u63a5\u7d9a\u300d\u306b\u8a2d\u5b9a\u3059\u308b\u3053\u3068\u3067<span translate=no>_^_3_^_</span>\u3001<span translate=no>_^_4_^_</span>\u30da\u30a2\u304c\u63a5\u7d9a\u3055\u308c\u3066\u3044\u306a\u3044\u72b6\u614b\u306b\u306a\u308a\u307e\u3059<span translate=no>_^_5_^_</span>\u3002</p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span>, <span translate=no>_^_1_^_</span> is the input node embeddings of shape <span translate=no>_^_2_^_</span>. </li>\n<li><span translate=no>_^_3_^_</span> is the adjacency matrix of shape <span translate=no>_^_4_^_</span>. We use shape <span translate=no>_^_5_^_</span> since the adjacency is the same for each head.</li></ul>\n<p>Adjacency matrix represent the edges (or connections) among nodes. <span translate=no>_^_6_^_</span> is <span translate=no>_^_7_^_</span> if there is an edge from node <span translate=no>_^_8_^_</span> to node <span translate=no>_^_9_^_</span>.</p>\n": "<ul><li><span translate=no>_^_0_^_</span>\u3001<span translate=no>_^_1_^_</span>\u306f\u30b7\u30a7\u30a4\u30d7\u306e\u5165\u529b\u30ce\u30fc\u30c9\u57cb\u3081\u8fbc\u307f\u3067\u3059\u3002<span translate=no>_^_2_^_</span></li>\n<li><span translate=no>_^_3_^_</span><span translate=no>_^_4_^_</span>\u306f\u5f62\u72b6\u306e\u96a3\u63a5\u884c\u5217\u3067\u3059\u3002<span translate=no>_^_5_^_</span>\u5404\u30d8\u30c3\u30c9\u306e\u96a3\u63a5\u95a2\u4fc2\u304c\u540c\u3058\u306a\u306e\u3067\u3001\u5f62\u72b6\u3092\u4f7f\u7528\u3057\u307e\u3059</li></ul>\u3002\n<p>\u96a3\u63a5\u30de\u30c8\u30ea\u30c3\u30af\u30b9\u306f\u3001\u30ce\u30fc\u30c9\u9593\u306e\u30a8\u30c3\u30b8 (\u307e\u305f\u306f\u63a5\u7d9a) \u3092\u8868\u3057\u307e\u3059\u3002<span translate=no>_^_6_^_</span><span translate=no>_^_7_^_</span><span translate=no>_^_8_^_</span>\u30ce\u30fc\u30c9\u9593\u3067\u30a8\u30c3\u30b8\u304c\u3042\u308b\u5834\u5408\u3067\u3059<span translate=no>_^_9_^_</span>\u3002</p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span>, <span translate=no>_^_1_^_</span>, is the number of input features per node </li>\n<li><span translate=no>_^_2_^_</span>, <span translate=no>_^_3_^_</span>, is the number of output features per node </li>\n<li><span translate=no>_^_4_^_</span>, <span translate=no>_^_5_^_</span>, is the number of attention heads </li>\n<li><span translate=no>_^_6_^_</span> whether the multi-head results should be concatenated or averaged </li>\n<li><span translate=no>_^_7_^_</span> is the dropout probability </li>\n<li><span translate=no>_^_8_^_</span> is the negative slope for leaky relu activation</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span><span translate=no>_^_1_^_</span>\u3001\u306f\u30ce\u30fc\u30c9\u3042\u305f\u308a\u306e\u5165\u529b\u30d5\u30a3\u30fc\u30c1\u30e3\u306e\u6570\u3067\u3059</li>\n<li><span translate=no>_^_2_^_</span><span translate=no>_^_3_^_</span>\u3001\u306f\u30ce\u30fc\u30c9\u3054\u3068\u306e\u51fa\u529b\u30d5\u30a3\u30fc\u30c1\u30e3\u306e\u6570\u3067\u3059</li>\n<li><span translate=no>_^_4_^_</span><span translate=no>_^_5_^_</span>\u3001\u306f\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30fb\u30d8\u30c3\u30c9\u306e\u6570</li>\n<li><span translate=no>_^_6_^_</span>\u30de\u30eb\u30c1\u30d8\u30c3\u30c9\u306e\u7d50\u679c\u3092\u9023\u7d50\u3059\u3079\u304d\u304b\u5e73\u5747\u5316\u3059\u3079\u304d\u304b</li>\n<li><span translate=no>_^_7_^_</span>\u306f\u8131\u843d\u78ba\u7387\u3067\u3059</li>\n<li><span translate=no>_^_8_^_</span>\u30ea\u30fc\u30af\u306e\u3042\u308b\u30ea\u30ec\u30fc\u30a2\u30af\u30c6\u30a3\u30d9\u30fc\u30b7\u30e7\u30f3\u306e\u8ca0\u306e\u50be\u304d\u3067\u3059</li></ul>\n",
|
||||
"A PyTorch implementation/tutorial of Graph Attention Networks.": "\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u306ePyTorch\u5b9f\u88c5/\u30c1\u30e5\u30fc\u30c8\u30ea\u30a2\u30eb\u3002",
|
||||
"Graph Attention Networks (GAT)": "\u30b0\u30e9\u30d5\u30fb\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30fb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af (GAT)"
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"<h1>Graph Attention Networks (GAT)</h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of the paper <a href=\"https://arxiv.org/abs/1710.10903\">Graph Attention Networks</a>.</p>\n<p>GATs work on graph data. A graph consists of nodes and edges connecting nodes. For example, in Cora dataset the nodes are research papers and the edges are citations that connect the papers.</p>\n<p>GAT uses masked self-attention, kind of similar to <a href=\"../../transformers/mha.html\">transformers</a>. GAT consists of graph attention layers stacked on top of each other. Each graph attention layer gets node embeddings as inputs and outputs transformed embeddings. The node embeddings pay attention to the embeddings of other nodes it's connected to. The details of graph attention layers are included alongside the implementation.</p>\n<p>Here is <a href=\"experiment.html\">the training code</a> for training a two-layer GAT on Cora dataset.</p>\n": "<h1>\u56fe\u8868\u6ce8\u610f\u529b\u7f51\u7edc (GAT)</h1>\n<p>\u8fd9\u662f <a href=\"https://pytorch.org\">PyTorch</a> \u5bf9\u300a<a href=\"https://arxiv.org/abs/1710.10903\">\u56fe\u5f62\u6ce8\u610f\u529b\u7f51\u7edc</a>\u300b\u8bba\u6587\u7684\u5b9e\u73b0\u3002</p>\n<p>GAT \u5904\u7406\u56fe\u8868\u6570\u636e\u3002\u56fe\u7531\u8282\u70b9\u548c\u8fde\u63a5\u8282\u70b9\u7684\u8fb9\u7ec4\u6210\u3002\u4f8b\u5982\uff0c\u5728 Cora \u6570\u636e\u96c6\u4e2d\uff0c\u8282\u70b9\u662f\u7814\u7a76\u8bba\u6587\uff0c\u8fb9\u7f18\u662f\u8fde\u63a5\u8bba\u6587\u7684\u5f15\u6587\u3002</p>\n<p>GAT \u4f7f\u7528\u8499\u9762\u81ea\u6ce8\u610f\u529b\uff0c\u6709\u70b9\u7c7b\u4f3c\u4e8e<a href=\"../../transformers/mha.html\">\u53d8\u5f62\u91d1\u521a</a>\u3002GAT \u7531\u76f8\u4e92\u5806\u53e0\u7684\u56fe\u8868\u6ce8\u610f\u529b\u5c42\u7ec4\u6210\u3002\u6bcf\u4e2a\u56fe\u6ce8\u610f\u529b\u5c42\u90fd\u5c06\u8282\u70b9\u5d4c\u5165\u4f5c\u4e3a\u8f6c\u6362\u540e\u7684\u5d4c\u5165\u7684\u8f93\u5165\u548c\u8f93\u51fa\u83b7\u5f97\u8282\u70b9\u3002\u8282\u70b9\u5d4c\u5165\u4f1a\u6ce8\u610f\u5b83\u6240\u8fde\u63a5\u7684\u5176\u4ed6\u8282\u70b9\u7684\u5d4c\u5165\u3002\u56fe\u5f62\u6ce8\u610f\u529b\u5c42\u7684\u8be6\u7ec6\u4fe1\u606f\u4e0e\u5b9e\u73b0\u4e00\u8d77\u5305\u62ec\u5728\u5185\u3002</p>\n<p>\u4ee5\u4e0b\u662f<a href=\"experiment.html\">\u5728 Cora \u6570\u636e\u96c6\u4e0a\u8bad\u7ec3\u4e24\u5c42 GAT \u7684\u8bad\u7ec3\u4ee3\u7801</a>\u3002</p>\n",
|
||||
"<h2>Graph attention layer</h2>\n<p>This is a single graph attention layer. A GAT is made up of multiple such layers.</p>\n<p>It takes <span translate=no>_^_0_^_</span>, where <span translate=no>_^_1_^_</span> as input and outputs <span translate=no>_^_2_^_</span>, where <span translate=no>_^_3_^_</span>.</p>\n": "<h2>\u56fe\u5f62\u5173\u6ce8\u5c42</h2>\n<p>\u8fd9\u662f\u4e00\u4e2a\u5355\u4e00\u7684\u56fe\u5f62\u5173\u6ce8\u5c42\u3002\u4e00\u4e2a GAT \u7531\u591a\u4e2a\u8fd9\u6837\u7684\u5c42\u7ec4\u6210\u3002</p>\n<p>\u5b83\u9700\u8981<span translate=no>_^_0_^_</span>\uff0c\u5176\u4e2d<span translate=no>_^_1_^_</span>\u4f5c\u4e3a\u8f93\u5165\u548c\u8f93\u51fa<span translate=no>_^_2_^_</span>\uff0c\u5728\u54ea\u91cc<span translate=no>_^_3_^_</span>\u3002</p>\n",
|
||||
"<h4>Calculate attention score</h4>\n<p>We calculate these for each head <span translate=no>_^_0_^_</span>. <em>We have omitted <span translate=no>_^_1_^_</span> for simplicity</em>.</p>\n<p><span translate=no>_^_2_^_</span></p>\n<p><span translate=no>_^_3_^_</span> is the attention score (importance) from node <span translate=no>_^_4_^_</span> to node <span translate=no>_^_5_^_</span>. We calculate this for each head.</p>\n<p><span translate=no>_^_6_^_</span> is the attention mechanism, that calculates the attention score. The paper concatenates <span translate=no>_^_7_^_</span>, <span translate=no>_^_8_^_</span> and does a linear transformation with a weight vector <span translate=no>_^_9_^_</span> followed by a <span translate=no>_^_10_^_</span>.</p>\n<p><span translate=no>_^_11_^_</span> </p>\n": "<h4>\u8ba1\u7b97\u6ce8\u610f\u529b\u5206\u6570</h4>\n<p>\u6211\u4eec\u4e3a\u6bcf\u4e2a\u5934\u90e8\u8ba1\u7b97\u8fd9\u4e9b<span translate=no>_^_0_^_</span>\u3002<em><span translate=no>_^_1_^_</span>\u4e3a\u7b80\u5355\u8d77\u89c1\uff0c\u6211\u4eec\u7701\u7565\u4e86</em>\u3002</p>\n<p><span translate=no>_^_2_^_</span></p>\n<p><span translate=no>_^_3_^_</span>\u662f\u4ece\u4e00\u4e2a\u8282\u70b9\u5230\u53e6\u4e00\u4e2a\u8282\u70b9\u7684<span translate=no>_^_4_^_</span>\u6ce8\u610f\u529b\u5206\u6570\uff08\u91cd\u8981\u6027\uff09<span translate=no>_^_5_^_</span>\u3002\u6211\u4eec\u4e3a\u6bcf\u4e2a\u5934\u90e8\u8ba1\u7b97\u8fd9\u4e2a\u503c\u3002</p>\n<p><span translate=no>_^_6_^_</span>\u662f\u8ba1\u7b97\u6ce8\u610f\u529b\u5206\u6570\u7684\u6ce8\u610f\u529b\u673a\u5236\u3002\u672c\u6587\u8fde\u63a5\u8d77\u6765<span translate=no>_^_7_^_</span>\uff0c<span translate=no>_^_8_^_</span>\u7136\u540e\u4f7f\u7528\u6743\u91cd\u5411\u91cf<span translate=no>_^_9_^_</span>\u540e\u8ddf a \u8fdb\u884c\u7ebf\u6027\u53d8\u6362<span translate=no>_^_10_^_</span>\u3002</p>\n<p><span translate=no>_^_11_^_</span></p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span> </p>\n": "<p><span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span> gets <span translate=no>_^_1_^_</span> where each node embedding is repeated <span translate=no>_^_2_^_</span> times. </p>\n": "<p><span translate=no>_^_0_^_</span>\u83b7\u53d6\u6bcf\u4e2a\u8282\u70b9\u5d4c\u5165\u91cd\u590d<span translate=no>_^_2_^_</span>\u6b21\u6570<span translate=no>_^_1_^_</span>\u7684\u4f4d\u7f6e\u3002</p>\n",
|
||||
"<p>Apply dropout regularization </p>\n": "<p>\u5e94\u7528\u8f8d\u5b66\u6b63\u5219\u5316</p>\n",
|
||||
"<p>Calculate <span translate=no>_^_0_^_</span> <span translate=no>_^_1_^_</span> is of shape <span translate=no>_^_2_^_</span> </p>\n": "<p>\u8ba1\u7b97<span translate=no>_^_0_^_</span><span translate=no>_^_1_^_</span>\u662f\u5f62\u72b6\u7684<span translate=no>_^_2_^_</span></p>\n",
|
||||
"<p>Calculate final output for each head <span translate=no>_^_0_^_</span></p>\n<p><em>Note:</em> The paper includes the final activation <span translate=no>_^_1_^_</span> in <span translate=no>_^_2_^_</span> We have omitted this from the Graph Attention Layer implementation and use it on the GAT model to match with how other PyTorch modules are defined - activation as a separate layer. </p>\n": "<p>\u8ba1\u7b97\u6bcf\u4e2a\u5934\u7684\u6700\u7ec8\u8f93\u51fa<span translate=no>_^_0_^_</span></p>\n<p><em>\u6ce8\u610f\uff1a</em>\u672c\u6587\u5305\u542b\u4e86\u6700\u540e\u7684\u6fc0\u6d3b\u3002<span translate=no>_^_2_^_</span>\u6211\u4eec\u5728Graph Attention Layer\u5b9e\u73b0<span translate=no>_^_1_^_</span>\u4e2d\u7701\u7565\u4e86\u8fd9\u4e00\u70b9\uff0c\u5e76\u5c06\u5176\u7528\u4e8eGAT\u6a21\u578b\u4ee5\u5339\u914d\u5176\u4ed6 PyTorch \u6a21\u5757\u7684\u5b9a\u4e49\u65b9\u5f0f\u2014\u2014\u6fc0\u6d3b\u4f5c\u4e3a\u5355\u72ec\u7684\u56fe\u5c42\u3002</p>\n",
|
||||
"<p>Calculate the number of dimensions per head </p>\n": "<p>\u8ba1\u7b97\u6bcf\u5934\u7684\u5c3a\u5bf8\u6570</p>\n",
|
||||
"<p>Concatenate the heads </p>\n": "<p>\u8fde\u63a5\u5934\u90e8</p>\n",
|
||||
"<p>Dropout layer to be applied for attention </p>\n": "<p>\u8981\u5e94\u7528\u7684\u6389\u843d\u5c42\u4ee5\u5f15\u8d77\u6ce8\u610f</p>\n",
|
||||
"<p>First we calculate <span translate=no>_^_0_^_</span> for all pairs of <span translate=no>_^_1_^_</span>.</p>\n<p><span translate=no>_^_2_^_</span> gets <span translate=no>_^_3_^_</span> where each node embedding is repeated <span translate=no>_^_4_^_</span> times. </p>\n": "<p>\u9996\u5148\uff0c\u6211\u4eec\u8ba1\u7b97<span translate=no>_^_0_^_</span>\u6240\u6709\u5bf9<span translate=no>_^_1_^_</span>.</p>\n<p><span translate=no>_^_2_^_</span>\u83b7\u53d6\u6bcf\u4e2a\u8282\u70b9\u5d4c\u5165\u91cd\u590d<span translate=no>_^_4_^_</span>\u6b21\u6570<span translate=no>_^_3_^_</span>\u7684\u4f4d\u7f6e\u3002</p>\n",
|
||||
"<p>If we are averaging the multiple heads </p>\n": "<p>\u5982\u679c\u6211\u4eec\u5e73\u5747\u591a\u5934</p>\n",
|
||||
"<p>If we are concatenating the multiple heads </p>\n": "<p>\u5982\u679c\u6211\u4eec\u8981\u8fde\u63a5\u591a\u4e2a\u5934</p>\n",
|
||||
"<p>Linear layer for initial transformation; i.e. to transform the node embeddings before self-attention </p>\n": "<p>\u7528\u4e8e\u521d\u59cb\u53d8\u6362\u7684\u7ebf\u6027\u5c42\uff1b\u5373\u5728\u81ea\u6211\u5173\u6ce8\u4e4b\u524d\u8f6c\u6362\u8282\u70b9\u5d4c\u5165</p>\n",
|
||||
"<p>Linear layer to compute attention score <span translate=no>_^_0_^_</span> </p>\n": "<p>\u7528\u4e8e\u8ba1\u7b97\u6ce8\u610f\u529b\u5206\u6570\u7684\u7ebf\u6027\u56fe\u5c42<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Mask <span translate=no>_^_0_^_</span> based on adjacency matrix. <span translate=no>_^_1_^_</span> is set to <span translate=no>_^_2_^_</span> if there is no edge from <span translate=no>_^_3_^_</span> to <span translate=no>_^_4_^_</span>. </p>\n": "<p><span translate=no>_^_0_^_</span>\u57fa\u4e8e\u90bb\u63a5\u77e9\u9635\u7684\u63a9\u7801\u3002<span translate=no>_^_1_^_</span><span translate=no>_^_2_^_</span>\u5982\u679c\u6ca1\u6709\u4ece\u5230\u7684\u8fb9\u7f18\uff0c\u5219\u8bbe\u7f6e<span translate=no>_^_3_^_</span>\u4e3a<span translate=no>_^_4_^_</span>\u3002</p>\n",
|
||||
"<p>Now we concatenate to get <span translate=no>_^_0_^_</span> </p>\n": "<p>\u73b0\u5728\u6211\u4eec\u8fde\u63a5\u6765\u83b7\u5f97<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Number of nodes </p>\n": "<p>\u8282\u70b9\u6570\u91cf</p>\n",
|
||||
"<p>Remove the last dimension of size <span translate=no>_^_0_^_</span> </p>\n": "<p>\u79fb\u9664\u5927\u5c0f\u7684\u6700\u540e\u4e00\u4e2a\u7ef4\u5ea6<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Reshape so that <span translate=no>_^_0_^_</span> is <span translate=no>_^_1_^_</span> </p>\n": "<p>\u91cd\u5851<span translate=no>_^_0_^_</span>\u5c31\u662f\u8fd9\u6837<span translate=no>_^_1_^_</span></p>\n",
|
||||
"<p>Softmax to compute attention <span translate=no>_^_0_^_</span> </p>\n": "<p>Softmax \u9700\u8981\u8ba1\u7b97\u6ce8\u610f\u529b<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Take the mean of the heads </p>\n": "<p>\u4ee5\u5934\u8111\u7684\u610f\u601d\u4e3a\u4f8b</p>\n",
|
||||
"<p>The activation for attention score <span translate=no>_^_0_^_</span> </p>\n": "<p>\u6fc0\u6d3b\u6ce8\u610f\u529b\u5206\u6570<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>The adjacency matrix should have shape <span translate=no>_^_0_^_</span> or<span translate=no>_^_1_^_</span> </p>\n": "<p>\u90bb\u63a5\u77e9\u9635\u7684\u5f62\u72b6\u5e94<span translate=no>_^_0_^_</span>\u4e3a<span translate=no>_^_1_^_</span></p>\n",
|
||||
"<p>The initial transformation, <span translate=no>_^_0_^_</span> for each head. We do single linear transformation and then split it up for each head. </p>\n": "<p>\u6bcf\u4e2a\u5934\u90e8\u7684\u521d\u59cb\u53d8\u6362\u3002<span translate=no>_^_0_^_</span>\u6211\u4eec\u505a\u5355\u4e2a\u7ebf\u6027\u53d8\u6362\uff0c\u7136\u540e\u5c06\u5176\u62c6\u5206\u4e3a\u6bcf\u4e2a\u5934\u90e8\u3002</p>\n",
|
||||
"<p>We then normalize attention scores (or coefficients) <span translate=no>_^_0_^_</span></p>\n<p>where <span translate=no>_^_1_^_</span> is the set of nodes connected to <span translate=no>_^_2_^_</span>.</p>\n<p>We do this by setting unconnected <span translate=no>_^_3_^_</span> to <span translate=no>_^_4_^_</span> which makes <span translate=no>_^_5_^_</span> for unconnected pairs. </p>\n": "<p>\u7136\u540e\uff0c\u6211\u4eec\u5c06\u6ce8\u610f\u529b\u5206\u6570\uff08\u6216\u7cfb\u6570\uff09\u5f52\u4e00\u5316<span translate=no>_^_0_^_</span></p>\n<p>\u5176\u4e2d<span translate=no>_^_1_^_</span>\u662f\u8fde\u63a5\u5230\u7684\u8282\u70b9\u96c6<span translate=no>_^_2_^_</span>\u3002</p>\n<p>\u6211\u4eec\u901a\u8fc7<span translate=no>_^_3_^_</span>\u5c06\u672a\u8fde\u63a5\u7684\u914d\u5bf9\u8bbe\u7f6e<span translate=no>_^_5_^_</span>\u4e3a\u672a\u8fde\u63a5<span translate=no>_^_4_^_</span>\u7684\u914d\u5bf9\u6765\u5b9e\u73b0\u6b64\u76ee\u7684\u3002</p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span>, <span translate=no>_^_1_^_</span> is the input node embeddings of shape <span translate=no>_^_2_^_</span>. </li>\n<li><span translate=no>_^_3_^_</span> is the adjacency matrix of shape <span translate=no>_^_4_^_</span>. We use shape <span translate=no>_^_5_^_</span> since the adjacency is the same for each head.</li></ul>\n<p>Adjacency matrix represent the edges (or connections) among nodes. <span translate=no>_^_6_^_</span> is <span translate=no>_^_7_^_</span> if there is an edge from node <span translate=no>_^_8_^_</span> to node <span translate=no>_^_9_^_</span>.</p>\n": "<ul><li><span translate=no>_^_0_^_</span>\uff0c<span translate=no>_^_1_^_</span>\u662f shape \u7684\u8f93\u5165\u8282\u70b9\u5d4c\u5165<span translate=no>_^_2_^_</span>\u3002</li>\n<li><span translate=no>_^_3_^_</span>\u662f\u5f62\u72b6\u7684\u90bb\u63a5\u77e9\u9635<span translate=no>_^_4_^_</span>\u3002\u6211\u4eec\u4f7f\u7528\u5f62\u72b6\uff0c<span translate=no>_^_5_^_</span>\u56e0\u4e3a\u6bcf\u4e2a\u5934\u90e8\u7684\u90bb\u63a5\u662f\u76f8\u540c\u7684\u3002</li></ul>\n<p>\u90bb\u63a5\u77e9\u9635\u8868\u793a\u8282\u70b9\u4e4b\u95f4\u7684\u8fb9\uff08\u6216\u8fde\u63a5\uff09\u3002<span translate=no>_^_6_^_</span><span translate=no>_^_7_^_</span>\u5982\u679c\u8282\u70b9\u4e0e\u8282<span translate=no>_^_8_^_</span>\u70b9\u4e4b\u95f4\u5b58\u5728\u8fb9\u7f18<span translate=no>_^_9_^_</span>\u3002</p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span>, <span translate=no>_^_1_^_</span>, is the number of input features per node </li>\n<li><span translate=no>_^_2_^_</span>, <span translate=no>_^_3_^_</span>, is the number of output features per node </li>\n<li><span translate=no>_^_4_^_</span>, <span translate=no>_^_5_^_</span>, is the number of attention heads </li>\n<li><span translate=no>_^_6_^_</span> whether the multi-head results should be concatenated or averaged </li>\n<li><span translate=no>_^_7_^_</span> is the dropout probability </li>\n<li><span translate=no>_^_8_^_</span> is the negative slope for leaky relu activation</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span><span translate=no>_^_1_^_</span>\uff0c\u662f\u6bcf\u4e2a\u8282\u70b9\u7684\u8f93\u5165\u8981\u7d20\u6570</li>\n<li><span translate=no>_^_2_^_</span><span translate=no>_^_3_^_</span>\uff0c\u662f\u6bcf\u4e2a\u8282\u70b9\u7684\u8f93\u51fa\u8981\u7d20\u6570</li>\n<li><span translate=no>_^_4_^_</span><span translate=no>_^_5_^_</span>\uff0c\u662f\u6ce8\u610f\u5934\u7684\u6570\u91cf</li>\n<li><span translate=no>_^_6_^_</span>\u591a\u5934\u7ed3\u679c\u5e94\u8be5\u662f\u4e32\u8054\u8fd8\u662f\u6c42\u5e73\u5747\u503c</li>\n<li><span translate=no>_^_7_^_</span>\u662f\u8f8d\u5b66\u6982\u7387</li>\n<li><span translate=no>_^_8_^_</span>\u662f\u6cc4\u6f0f\u7684 relu \u6fc0\u6d3b\u7684\u8d1f\u659c\u7387</li></ul>\n",
|
||||
"A PyTorch implementation/tutorial of Graph Attention Networks.": "Graph \u6ce8\u610f\u529b\u7f51\u7edc\u7684 PyTorch \u5b9e\u73b0/\u6559\u7a0b\u3002",
|
||||
"Graph Attention Networks (GAT)": "\u56fe\u5173\u6ce8\u7f51\u7edc (GAT)"
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
{
|
||||
"<h1>Train a Graph Attention Network (GAT) on Cora dataset</h1>\n": "<h1>Cora \u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306e\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30cd\u30c3\u30c8\u30ef\u30fc\u30af (GAT) \u306e\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0</h1>\n",
|
||||
"<h2><a href=\"https://linqs.soe.ucsc.edu/data\">Cora Dataset</a></h2>\n<p>Cora dataset is a dataset of research papers. For each paper we are given a binary feature vector that indicates the presence of words. Each paper is classified into one of 7 classes. The dataset also has the citation network.</p>\n<p>The papers are the nodes of the graph and the edges are the citations.</p>\n<p>The task is to classify the nodes to the 7 classes with feature vectors and citation network as input.</p>\n": "<h2><a href=\"https://linqs.soe.ucsc.edu/data\">\u30b3\u30fc\u30e9\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8</a></h2>\n<p>Cora\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306f\u7814\u7a76\u8ad6\u6587\u306e\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3067\u3059\u3002\u5404\u8ad6\u6587\u306b\u306f\u3001\u5358\u8a9e\u306e\u5b58\u5728\u3092\u793a\u3059\u30d0\u30a4\u30ca\u30ea\u7279\u5fb4\u30d9\u30af\u30c8\u30eb\u304c\u4e0e\u3048\u3089\u308c\u307e\u3059\u3002\u5404\u8ad6\u6587\u306f7\u3064\u306e\u30af\u30e9\u30b9\u306e\u3044\u305a\u308c\u304b\u306b\u5206\u985e\u3055\u308c\u307e\u3059\u3002\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306b\u306f\u5f15\u7528\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3082\u3042\u308a\u307e\u3059</p>\u3002\n<p>\u8ad6\u6587\u306f\u30b0\u30e9\u30d5\u306e\u7bc0\u70b9\u3067\u3001\u7aef\u306f\u5f15\u7528\u3067\u3059\u3002</p>\n<p>\u30bf\u30b9\u30af\u306f\u3001\u7279\u5fb4\u30d9\u30af\u30c8\u30eb\u3068\u5f15\u7528\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3092\u5165\u529b\u3068\u3057\u3066\u3001\u30ce\u30fc\u30c9\u30927\u3064\u306e\u30af\u30e9\u30b9\u306b\u5206\u985e\u3059\u308b\u3053\u3068\u3067\u3059\u3002</p>\n",
|
||||
"<h2>Configurations</h2>\n": "<h2>\u30b3\u30f3\u30d5\u30a3\u30ae\u30e5\u30ec\u30fc\u30b7\u30e7\u30f3</h2>\n",
|
||||
"<h2>Graph Attention Network (GAT)</h2>\n<p>This graph attention network has two <a href=\"index.html\">graph attention layers</a>.</p>\n": "<h2>\u30b0\u30e9\u30d5\u30fb\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30fb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af (GAT)</h2>\n<p>\u3053\u306e\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u306b\u306f 2 <a href=\"index.html\">\u3064\u306e\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30ec\u30a4\u30e4\u30fc\u304c\u3042\u308a\u307e\u3059</a>\u3002</p>\n",
|
||||
"<h3>Training loop</h3>\n<p>We do full batch training since the dataset is small. If we were to sample and train we will have to sample a set of nodes for each training step along with the edges that span across those selected nodes.</p>\n": "<h3>\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30eb\u30fc\u30d7</h3>\n<p>\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u304c\u5c0f\u3055\u3044\u306e\u3067\u3001\u30d5\u30eb\u30d0\u30c3\u30c1\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3092\u884c\u3044\u307e\u3059\u3002\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3057\u3066\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3059\u308b\u5834\u5408\u3001\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30b9\u30c6\u30c3\u30d7\u3054\u3068\u306b\u4e00\u9023\u306e\u30ce\u30fc\u30c9\u3068\u3001\u9078\u629e\u3057\u305f\u30ce\u30fc\u30c9\u306b\u307e\u305f\u304c\u308b\u30a8\u30c3\u30b8\u3092\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3059\u308b\u5fc5\u8981\u304c\u3042\u308a\u307e\u3059\u3002</p>\n",
|
||||
"<p> </p>\n": "<p></p>\n",
|
||||
"<p> A simple function to calculate the accuracy</p>\n": "<p>\u7cbe\u5ea6\u3092\u8a08\u7b97\u3059\u308b\u7c21\u5358\u306a\u95a2\u6570</p>\n",
|
||||
"<p> Create Cora dataset</p>\n": "<p>Cora \u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306e\u4f5c\u6210</p>\n",
|
||||
"<p> Create GAT model</p>\n": "<p>GAT \u30e2\u30c7\u30eb\u306e\u4f5c\u6210</p>\n",
|
||||
"<p> Create configurable optimizer</p>\n": "<p>\u8a2d\u5b9a\u53ef\u80fd\u306a\u30aa\u30d7\u30c6\u30a3\u30de\u30a4\u30b6\u30fc\u306e\u4f5c\u6210</p>\n",
|
||||
"<p> Download the dataset</p>\n": "<p>\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306e\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9</p>\n",
|
||||
"<p> Load the dataset</p>\n": "<p>\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306e\u8aad\u307f\u8fbc\u307f</p>\n",
|
||||
"<p>Activation function </p>\n": "<p>\u30a2\u30af\u30c6\u30a3\u30d9\u30fc\u30b7\u30e7\u30f3\u6a5f\u80fd</p>\n",
|
||||
"<p>Activation function after first graph attention layer </p>\n": "<p>\u6700\u521d\u306e\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30ec\u30a4\u30e4\u30fc\u5f8c\u306e\u30a2\u30af\u30c6\u30a3\u30d9\u30fc\u30b7\u30e7\u30f3\u6a5f\u80fd</p>\n",
|
||||
"<p>Adam optimizer </p>\n": "<p>\u30a2\u30c0\u30e0\u30fb\u30aa\u30d7\u30c6\u30a3\u30de\u30a4\u30b6\u30fc</p>\n",
|
||||
"<p>Add an empty third dimension for the heads </p>\n": "<p>\u982d\u90e8\u306b\u7a7a\u306e 3 \u756a\u76ee\u306e\u30c7\u30a3\u30e1\u30f3\u30b7\u30e7\u30f3\u3092\u8ffd\u52a0</p>\n",
|
||||
"<p>Adjacency matrix with the edge information. <span translate=no>_^_0_^_</span> is <span translate=no>_^_1_^_</span> if there is an edge from <span translate=no>_^_2_^_</span> to <span translate=no>_^_3_^_</span>. </p>\n": "<p>\u30a8\u30c3\u30b8\u60c5\u5831\u3092\u542b\u3080\u96a3\u63a5\u30de\u30c8\u30ea\u30c3\u30af\u30b9\u3002<span translate=no>_^_0_^_</span><span translate=no>_^_1_^_</span><span translate=no>_^_2_^_</span><span translate=no>_^_3_^_</span>\u3082\u3057\u3082\u304b\u3089\u7aef\u304c\u3042\u3063\u305f\u3089\u306d</p>\n",
|
||||
"<p>Apply dropout to the input </p>\n": "<p>\u5165\u529b\u306b\u30c9\u30ed\u30c3\u30d7\u30a2\u30a6\u30c8\u3092\u9069\u7528</p>\n",
|
||||
"<p>Calculate configurations. </p>\n": "<p>\u69cb\u6210\u3092\u8a08\u7b97\u3057\u307e\u3059\u3002</p>\n",
|
||||
"<p>Calculate gradients </p>\n": "<p>\u52fe\u914d\u306e\u8a08\u7b97</p>\n",
|
||||
"<p>Calculate the loss for validation nodes </p>\n": "<p>\u691c\u8a3c\u30ce\u30fc\u30c9\u306e\u640d\u5931\u306e\u8a08\u7b97</p>\n",
|
||||
"<p>Create an experiment </p>\n": "<p>\u30c6\u30b9\u30c8\u3092\u4f5c\u6210</p>\n",
|
||||
"<p>Create configurations </p>\n": "<p>\u69cb\u6210\u306e\u4f5c\u6210</p>\n",
|
||||
"<p>Dataset </p>\n": "<p>\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8</p>\n",
|
||||
"<p>Device to train on</p>\n<p>This creates configs for device, so that we can change the device by passing a config value </p>\n": "<p>\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3059\u308b\u30c7\u30d0\u30a4\u30b9</p>\n<p>\u3053\u308c\u306b\u3088\u308a\u30c7\u30d0\u30a4\u30b9\u306e\u8a2d\u5b9a\u304c\u4f5c\u6210\u3055\u308c\u308b\u306e\u3067\u3001\u8a2d\u5b9a\u5024\u3092\u6e21\u3059\u3053\u3068\u3067\u30c7\u30d0\u30a4\u30b9\u3092\u5909\u66f4\u3067\u304d\u307e\u3059</p>\n",
|
||||
"<p>Download dataset </p>\n": "<p>\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306e\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9</p>\n",
|
||||
"<p>Dropout </p>\n": "<p>\u30c9\u30ed\u30c3\u30d7\u30a2\u30a6\u30c8</p>\n",
|
||||
"<p>Dropout probability </p>\n": "<p>\u8131\u843d\u78ba\u7387</p>\n",
|
||||
"<p>Empty adjacency matrix - an identity matrix </p>\n": "<p>\u7a7a\u306e\u96a3\u63a5\u884c\u5217-\u5358\u4f4d\u884c\u5217</p>\n",
|
||||
"<p>Evaluate the model </p>\n": "<p>\u30e2\u30c7\u30eb\u306e\u8a55\u4fa1</p>\n",
|
||||
"<p>Evaluate the model again </p>\n": "<p>\u30e2\u30c7\u30eb\u3092\u518d\u5ea6\u8a55\u4fa1\u3057\u3066\u304f\u3060\u3055\u3044</p>\n",
|
||||
"<p>Feature vectors for all nodes </p>\n": "<p>\u5168\u30ce\u30fc\u30c9\u306e\u7279\u5fb4\u30d9\u30af\u30c8\u30eb</p>\n",
|
||||
"<p>Final graph attention layer where we average the heads </p>\n": "<p>\u30d8\u30c3\u30c9\u3092\u5e73\u5747\u5316\u3059\u308b\u6700\u5f8c\u306e\u30b0\u30e9\u30d5\u30fb\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30fb\u30ec\u30a4\u30e4\u30fc</p>\n",
|
||||
"<p>First graph attention layer </p>\n": "<p>\u6700\u521d\u306e\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30ec\u30a4\u30e4\u30fc</p>\n",
|
||||
"<p>First graph attention layer where we concatenate the heads </p>\n": "<p>\u30d8\u30c3\u30c9\u3092\u9023\u7d50\u3059\u308b\u6700\u521d\u306e\u30b0\u30e9\u30d5\u30fb\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30fb\u30ec\u30a4\u30e4\u30fc</p>\n",
|
||||
"<p>Get the class names and assign an unique integer to each of them </p>\n": "<p>\u30af\u30e9\u30b9\u540d\u3092\u53d6\u5f97\u3057\u3001\u305d\u308c\u305e\u308c\u306b\u4e00\u610f\u306e\u6574\u6570\u3092\u5272\u308a\u5f53\u3066\u307e\u3059\u3002</p>\n",
|
||||
"<p>Get the feature vectors </p>\n": "<p>\u7279\u5fb4\u30d9\u30af\u30c8\u30eb\u3092\u53d6\u5f97</p>\n",
|
||||
"<p>Get the labels as those integers </p>\n": "<p>\u30e9\u30d9\u30eb\u3092\u305d\u308c\u3089\u306e\u6574\u6570\u3068\u3057\u3066\u53d6\u5f97</p>\n",
|
||||
"<p>Get the loss for training nodes </p>\n": "<p>\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30ce\u30fc\u30c9\u3067\u640d\u5931\u3092\u88ab\u308b</p>\n",
|
||||
"<p>Get the number of classes </p>\n": "<p>\u30af\u30e9\u30b9\u6570\u3092\u53d6\u5f97</p>\n",
|
||||
"<p>Get the paper ids </p>\n": "<p>\u7d19\u306e ID \u3092\u5165\u624b</p>\n",
|
||||
"<p>Labels for each node </p>\n": "<p>\u5404\u30ce\u30fc\u30c9\u306e\u30e9\u30d9\u30eb</p>\n",
|
||||
"<p>Load the citations, it's a list of pairs of integers. </p>\n": "<p>\u5f15\u7528\u3092\u30ed\u30fc\u30c9\u3057\u307e\u3059\u3002\u6574\u6570\u306e\u30da\u30a2\u306e\u30ea\u30b9\u30c8\u3067\u3059\u3002</p>\n",
|
||||
"<p>Log the accuracy </p>\n": "<p>\u7cbe\u5ea6\u3092\u8a18\u9332\u3059\u308b</p>\n",
|
||||
"<p>Log the loss </p>\n": "<p>\u640d\u5931\u3092\u8a18\u9332\u3059\u308b</p>\n",
|
||||
"<p>Loss function </p>\n": "<p>\u640d\u5931\u95a2\u6570</p>\n",
|
||||
"<p>Make all the gradients zero </p>\n": "<p>\u3059\u3079\u3066\u306e\u30b0\u30e9\u30c7\u30fc\u30b7\u30e7\u30f3\u3092\u30bc\u30ed\u306b\u3059\u308b</p>\n",
|
||||
"<p>Map of paper id to index </p>\n": "<p>\u7d19ID\u3068\u7d22\u5f15\u306e\u30de\u30c3\u30d7</p>\n",
|
||||
"<p>Mark the citations in the adjacency matrix </p>\n": "<p>\u5f15\u7528\u6587\u732e\u3092\u96a3\u63a5\u30de\u30c8\u30ea\u30c3\u30af\u30b9\u306b\u8a18\u5165</p>\n",
|
||||
"<p>Model </p>\n": "<p>\u30e2\u30c7\u30eb</p>\n",
|
||||
"<p>Move the adjacency matrix to the device </p>\n": "<p>\u96a3\u63a5\u30de\u30c8\u30ea\u30c3\u30af\u30b9\u3092\u30c7\u30d0\u30a4\u30b9\u306b\u79fb\u52d5</p>\n",
|
||||
"<p>Move the feature vectors to the device </p>\n": "<p>\u7279\u5fb4\u30d9\u30af\u30c8\u30eb\u3092\u30c7\u30d0\u30a4\u30b9\u306b\u79fb\u52d5\u3057\u307e\u3059</p>\n",
|
||||
"<p>Move the labels to the device </p>\n": "<p>\u30e9\u30d9\u30eb\u3092\u30c7\u30d0\u30a4\u30b9\u306b\u79fb\u52d5</p>\n",
|
||||
"<p>No need to compute gradients </p>\n": "<p>\u52fe\u914d\u3092\u8a08\u7b97\u3059\u308b\u5fc5\u8981\u306f\u3042\u308a\u307e\u305b\u3093</p>\n",
|
||||
"<p>Nodes for training </p>\n": "<p>\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u7528\u30ce\u30fc\u30c9</p>\n",
|
||||
"<p>Nodes for validation </p>\n": "<p>\u691c\u8a3c\u7528\u30ce\u30fc\u30c9</p>\n",
|
||||
"<p>Normalize the feature vectors </p>\n": "<p>\u7279\u5fb4\u30d9\u30af\u30c8\u30eb\u3092\u6b63\u898f\u5316</p>\n",
|
||||
"<p>Number of classes for classification </p>\n": "<p>\u5206\u985e\u3059\u308b\u30af\u30e9\u30b9\u6570</p>\n",
|
||||
"<p>Number of features in the first graph attention layer </p>\n": "<p>\u6700\u521d\u306e\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30ec\u30a4\u30e4\u30fc\u306b\u542b\u307e\u308c\u308b\u30d5\u30a3\u30fc\u30c1\u30e3\u306e\u6570</p>\n",
|
||||
"<p>Number of features in the input </p>\n": "<p>\u5165\u529b\u5185\u306e\u30d5\u30a3\u30fc\u30c1\u30e3\u306e\u6570</p>\n",
|
||||
"<p>Number of features per node in the input </p>\n": "<p>\u5165\u529b\u5185\u306e\u30ce\u30fc\u30c9\u3042\u305f\u308a\u306e\u30d5\u30a3\u30fc\u30c1\u30e3\u6570</p>\n",
|
||||
"<p>Number of heads </p>\n": "<p>\u30d8\u30c3\u30c9\u6570</p>\n",
|
||||
"<p>Number of nodes to train on </p>\n": "<p>\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3059\u308b\u30ce\u30fc\u30c9\u6570</p>\n",
|
||||
"<p>Number of training iterations </p>\n": "<p>\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u306e\u53cd\u5fa9\u56de\u6570</p>\n",
|
||||
"<p>Optimizer </p>\n": "<p>\u30aa\u30d7\u30c6\u30a3\u30de\u30a4\u30b6\u30fc</p>\n",
|
||||
"<p>Output layer (without activation) for logits </p>\n": "<p>\u30ed\u30b8\u30c3\u30c8\u306e\u51fa\u529b\u30ec\u30a4\u30e4\u30fc (\u30a2\u30af\u30c6\u30a3\u30d9\u30fc\u30b7\u30e7\u30f3\u306a\u3057)</p>\n",
|
||||
"<p>Random indexes </p>\n": "<p>\u30e9\u30f3\u30c0\u30e0\u30a4\u30f3\u30c7\u30c3\u30af\u30b9</p>\n",
|
||||
"<p>Read the paper ids, feature vectors, and labels </p>\n": "<p>\u8ad6\u6587ID\u3001\u7279\u5fb4\u30d9\u30af\u30c8\u30eb\u3001\u30e9\u30d9\u30eb\u3092\u8aad\u3080</p>\n",
|
||||
"<p>Run the training </p>\n": "<p>\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3092\u5b9f\u884c</p>\n",
|
||||
"<p>Save logs </p>\n": "<p>\u30ed\u30b0\u3092\u4fdd\u5b58</p>\n",
|
||||
"<p>Set mode to evaluation mode for validation </p>\n": "<p>\u691c\u8a3c\u7528\u306b\u30e2\u30fc\u30c9\u3092\u8a55\u4fa1\u30e2\u30fc\u30c9\u306b\u8a2d\u5b9a</p>\n",
|
||||
"<p>Set of class names and an unique integer index </p>\n": "<p>\u30af\u30e9\u30b9\u540d\u3068\u4e00\u610f\u306e\u6574\u6570\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u306e\u30bb\u30c3\u30c8</p>\n",
|
||||
"<p>Set the model to training mode </p>\n": "<p>\u30e2\u30c7\u30eb\u3092\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30e2\u30fc\u30c9\u306b\u8a2d\u5b9a</p>\n",
|
||||
"<p>Start and watch the experiment </p>\n": "<p>\u5b9f\u9a13\u3092\u958b\u59cb\u3057\u3066\u898b\u308b</p>\n",
|
||||
"<p>Take optimization step </p>\n": "<p>\u6700\u9069\u5316\u306e\u4e00\u6b69\u3092\u8e0f\u307f\u51fa\u3059</p>\n",
|
||||
"<p>The pair of paper indexes </p>\n": "<p>\u4e00\u5bfe\u306e\u30da\u30fc\u30d1\u30fc\u30a4\u30f3\u30c7\u30c3\u30af\u30b9</p>\n",
|
||||
"<p>Training loop </p>\n": "<p>\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30eb\u30fc\u30d7</p>\n",
|
||||
"<p>We build a symmetrical graph, where if paper <span translate=no>_^_0_^_</span> referenced paper <span translate=no>_^_1_^_</span> we place an adge from <span translate=no>_^_2_^_</span> to <span translate=no>_^_3_^_</span> as well as an edge from <span translate=no>_^_4_^_</span> to <span translate=no>_^_5_^_</span>. </p>\n": "<p>\u5bfe\u79f0\u7684\u306a\u30b0\u30e9\u30d5\u3092\u4f5c\u6210\u3057\u307e\u3059\u3002<span translate=no>_^_0_^_</span>\u7d19\u304c\u53c2\u7167\u3057\u3066\u3044\u308b\u7d19\u306e\u5834\u5408\u306f\u3001<span translate=no>_^_1_^_</span>\u7aef\u3092\u7aef\u304b\u3089\u7aef\u306b\u3001<span translate=no>_^_2_^_</span><span translate=no>_^_3_^_</span>\u7aef\u3092\u7aef\u3068\u3057\u3066\u914d\u7f6e\u3057\u307e\u3059\u3002<span translate=no>_^_4_^_</span> <span translate=no>_^_5_^_</span></p>\n",
|
||||
"<p>Whether to include edges. This is test how much accuracy is lost if we ignore the citation network. </p>\n": "<p>\u30a8\u30c3\u30b8\u3092\u542b\u3081\u308b\u304b\u3069\u3046\u304b\u3002\u3053\u308c\u306f\u3001\u5f15\u7528\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3092\u7121\u8996\u3059\u308b\u3068\u7cbe\u5ea6\u304c\u3069\u308c\u3060\u3051\u5931\u308f\u308c\u308b\u304b\u3092\u30c6\u30b9\u30c8\u3059\u308b\u3082\u306e\u3067\u3059</p>\u3002\n",
|
||||
"<p>Whether to include the citation network </p>\n": "<p>\u5f15\u7528\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3092\u542b\u3081\u308b\u304b\u3069\u3046\u304b</p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the features vectors of shape <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> is the adjacency matrix of the form <span translate=no>_^_3_^_</span> or <span translate=no>_^_4_^_</span></li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u306f\u5f62\u72b6\u306e\u7279\u5fb4\u30d9\u30af\u30c8\u30eb\u3067\u3059 <span translate=no>_^_1_^_</span></li>\n<li><span translate=no>_^_2_^_</span><span translate=no>_^_3_^_</span>\u306f\u6b21\u306e\u5f62\u5f0f\u306e\u96a3\u63a5\u884c\u5217\u3067\u3059 <span translate=no>_^_4_^_</span></li></ul>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the number of features per node </li>\n<li><span translate=no>_^_1_^_</span> is the number of features in the first graph attention layer </li>\n<li><span translate=no>_^_2_^_</span> is the number of classes </li>\n<li><span translate=no>_^_3_^_</span> is the number of heads in the graph attention layers </li>\n<li><span translate=no>_^_4_^_</span> is the dropout probability</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u306f\u30ce\u30fc\u30c9\u3042\u305f\u308a\u306e\u30d5\u30a3\u30fc\u30c1\u30e3\u6570</li>\n<li><span translate=no>_^_1_^_</span>\u306f\u6700\u521d\u306e\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30ec\u30a4\u30e4\u30fc\u306b\u542b\u307e\u308c\u308b\u30d5\u30a3\u30fc\u30c1\u30e3\u306e\u6570\u3067\u3059</li>\n<li><span translate=no>_^_2_^_</span>\u306f\u30af\u30e9\u30b9\u306e\u6570</li>\n<li><span translate=no>_^_3_^_</span>\u30b0\u30e9\u30d5\u30fb\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30fb\u30ec\u30a4\u30e4\u30fc\u306e\u30d8\u30c3\u30c9\u6570\u3067\u3059</li>\n<li><span translate=no>_^_4_^_</span>\u306f\u8131\u843d\u78ba\u7387\u3067\u3059</li></ul>\n",
|
||||
"This trains is a Graph Attention Network (GAT) on Cora dataset": "\u3053\u306e\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u306f\u3001Cora\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306e\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\uff08GAT\uff09\u3067\u3059",
|
||||
"Train a Graph Attention Network (GAT) on Cora dataset": "Cora \u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306e\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30cd\u30c3\u30c8\u30ef\u30fc\u30af (GAT) \u306e\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0"
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
{
|
||||
"<h1>Train a Graph Attention Network (GAT) on Cora dataset</h1>\n<p><a href=\"https://app.labml.ai/run/d6c636cadf3511eba2f1e707f612f95d\"><span translate=no>_^_0_^_</span></a></p>\n": "<h1>\u0d9a\u0ddd\u0dbb\u0dcf\u0daf\u0dad\u0dca\u0dad \u0d9a\u0da7\u0dca\u0da7\u0dbd\u0dba \u0db8\u0dad \u0db4\u0dca\u0dbb\u0dc3\u0dca\u0dad\u0dcf\u0dbb\u0dba \u0d85\u0dc0\u0db0\u0dcf\u0db1\u0dba \u0da2\u0dcf\u0dbd\u0dba (GAT) \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4</h1>\n<p><a href=\"https://app.labml.ai/run/d6c636cadf3511eba2f1e707f612f95d\"><span translate=no>_^_0_^_</span></a></p>\n",
|
||||
"<h2><a href=\"https://linqs.soe.ucsc.edu/data\">Cora Dataset</a></h2>\n<p>Cora dataset is a dataset of research papers. For each paper we are given a binary feature vector that indicates the presence of words. Each paper is classified into one of 7 classes. The dataset also has the citation network.</p>\n<p>The papers are the nodes of the graph and the edges are the citations.</p>\n<p>The task is to classify the nodes to the 7 classes with feature vectors and citation network as input.</p>\n": "<h2><a href=\"https://linqs.soe.ucsc.edu/data\">\u0d9a\u0ddd\u0dbb\u0dcf \u0daf\u0dad\u0dca\u0dad \u0d9a\u0da7\u0dca\u0da7\u0dbd\u0dba</a></h2>\n<p>\u0d9a\u0ddd\u0dbb\u0dcf \u0daf\u0dad\u0dca\u0dad \u0d9a\u0da7\u0dca\u0da7\u0dbd\u0dba \u0db4\u0dbb\u0dca\u0dba\u0dda\u0dc2\u0dab \u0db4\u0dad\u0dca\u0dbb\u0dd2\u0d9a\u0dcf \u0daf\u0dad\u0dca\u0dad \u0d9a\u0da7\u0dca\u0da7\u0dbd\u0dba\u0d9a\u0dd2. \u0dc3\u0dd1\u0db8 \u0d9a\u0da9\u0daf\u0dcf\u0dc3\u0dd2 \u0dc3\u0db3\u0dc4\u0dcf\u0db8 \u0d85\u0db4\u0da7 \u0daf\u0dca\u0dc0\u0dd2\u0db8\u0dba \u0d85\u0d82\u0d9c \u0daf\u0ddb\u0dc1\u0dd2\u0d9a\u0dba\u0d9a\u0dca \u0dbd\u0db6\u0dcf \u0daf\u0dd3 \u0d87\u0dad\u0dd2 \u0d85\u0dad\u0dbb \u0d91\u0dba \u0dc0\u0da0\u0db1 \u0d87\u0dad\u0dd2 \u0db6\u0dc0 \u0db4\u0dd9\u0db1\u0dca\u0db1\u0dd4\u0db8\u0dca \u0d9a\u0dbb\u0dba\u0dd2. \u0dc3\u0dd1\u0db8 \u0d9a\u0da9\u0daf\u0dcf\u0dc3\u0dd2\u0dba\u0d9a\u0dca\u0db8 \u0db4\u0db1\u0dca\u0dad\u0dd2 7 \u0db1\u0dca \u0d91\u0d9a\u0d9a\u0da7 \u0dc0\u0dbb\u0dca\u0d9c\u0dd3\u0d9a\u0dbb\u0dab\u0dba \u0d9a\u0dbb \u0d87\u0dad. \u0daf\u0dad\u0dca\u0dad \u0d9a\u0da7\u0dca\u0da7\u0dbd\u0dba\u0da7 \u0d8b\u0db4\u0dd4\u0da7\u0dcf \u0daf\u0dd0\u0d9a\u0dca\u0dc0\u0dd3\u0db8\u0dda \u0da2\u0dcf\u0dbd\u0dba \u0daf \u0d87\u0dad.</p>\n<p>\u0db4\u0dad\u0dca\u0dbb\u0dd2\u0d9a\u0dcf \u0dba\u0db1\u0dd4 \u0db4\u0dca\u0dbb\u0dc3\u0dca\u0dae\u0dcf\u0dbb\u0dba\u0dda \u0db1\u0ddd\u0da9\u0dca \u0dc0\u0db1 \u0d85\u0dad\u0dbb \u0daf\u0dcf\u0dbb \u0d8b\u0db4\u0dd4\u0da7\u0dcf \u0daf\u0dd0\u0d9a\u0dca\u0dc0\u0dd3\u0db8\u0dca \u0dc0\u0dda.</p>\n<p>\u0d9a\u0dbb\u0dca\u0dad\u0dc0\u0dca\u0dba\u0dba \u0dc0\u0db1\u0dca\u0db1\u0dda \u0db1\u0ddd\u0da9\u0dca 7 \u0db4\u0db1\u0dca\u0dad\u0dd2 \u0dc0\u0dbd\u0da7 \u0dc0\u0dd2\u0dc1\u0dda\u0dc2\u0dcf\u0d82\u0d9c \u0daf\u0ddb\u0dc1\u0dd2\u0d9a \u0dc3\u0dc4 \u0d8b\u0db4\u0dd4\u0da7\u0dcf \u0daf\u0dd0\u0d9a\u0dca\u0dc0\u0dd3\u0db8\u0dda \u0da2\u0dcf\u0dbd\u0dba \u0d86\u0daf\u0dcf\u0db1\u0dba \u0dbd\u0dd9\u0dc3 \u0dc0\u0dbb\u0dca\u0d9c\u0dd3\u0d9a\u0dbb\u0dab\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dba\u0dd2.</p>\n",
|
||||
"<h2>Configurations</h2>\n": "<h2>\u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3\u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dca</h2>\n",
|
||||
"<h2>Graph Attention Network (GAT)</h2>\n<p>This graph attention network has two <a href=\"index.html\">graph attention layers</a>.</p>\n": "<h2>\u0db4\u0dca\u0dbb\u0dc3\u0dca\u0dad\u0dcf\u0dbb\u0dba\u0d85\u0dc0\u0db0\u0dcf\u0db1\u0dba \u0da2\u0dcf\u0dbd\u0dba (GAT)</h2>\n<p>\u0db8\u0dd9\u0db8\u0db4\u0dca\u0dbb\u0dc3\u0dca\u0dad\u0dcf\u0dbb \u0d85\u0dc0\u0db0\u0dcf\u0db1\u0dba \u0da2\u0dcf\u0dbd\u0dba <a href=\"index.html\">\u0db4\u0dca\u0dbb\u0dc3\u0dca\u0dad\u0dcf\u0dbb\u0dba \u0d85\u0dc0\u0db0\u0dcf\u0db1\u0dba \u0dc3\u0dca\u0dae\u0dbb</a>\u0daf\u0dd9\u0d9a\u0d9a\u0dca \u0d87\u0dad. </p>\n",
|
||||
"<h3>Training loop</h3>\n<p>We do full batch training since the dataset is small. If we were to sample and train we will have to sample a set of nodes for each training step along with the edges that span across those selected nodes.</p>\n": "<h3>\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0dbd\u0dd6\u0db4\u0dba</h3>\n<p>\u0daf\u0dad\u0dca\u0dad\u0dc3\u0db8\u0dd4\u0daf\u0dcf\u0dba \u0d9a\u0dd4\u0da9\u0dcf \u0db6\u0dd0\u0dc0\u0dd2\u0db1\u0dca \u0d85\u0db4\u0dd2 \u0db4\u0dd6\u0dbb\u0dca\u0dab \u0d9a\u0dab\u0dca\u0da9\u0dcf\u0dba\u0db8\u0dca \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0dc0 \u0d9a\u0dbb\u0db1\u0dca\u0db1\u0dd9\u0db8\u0dd4. \u0d85\u0db4 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba \u0dc4\u0dcf \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0da7 \u0db1\u0db8\u0dca, \u0d91\u0d9a\u0dca \u0d91\u0d9a\u0dca \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0db4\u0dd2\u0dba\u0dc0\u0dbb \u0dc3\u0db3\u0dc4\u0dcf \u0db1\u0ddd\u0da9\u0dca \u0d9a\u0da7\u0dca\u0da7\u0dbd\u0dba\u0d9a\u0dca \u0dc3\u0dcf\u0db8\u0dca\u0db4\u0dbd \u0dbd\u0db6\u0dcf \u0d9c\u0dad \u0dba\u0dd4\u0dad\u0dd4 \u0d85\u0dad\u0dbb \u0d91\u0db8 \u0dad\u0ddd\u0dbb\u0dcf\u0d9c\u0dad\u0dca \u0db1\u0ddd\u0da9\u0dca \u0dc4\u0dbb\u0dc4\u0dcf \u0dc0\u0dd2\u0dc4\u0dd2\u0daf\u0dd9\u0db1 \u0daf\u0dcf\u0dbb \u0dc3\u0db8\u0d9f. </p>\n",
|
||||
"<p> </p>\n": "<p> </p>\n",
|
||||
"<p> A simple function to calculate the accuracy</p>\n": "<p> \u0db1\u0dd2\u0dbb\u0dc0\u0daf\u0dca\u0dba\u0dad\u0dcf\u0dc0\u0d9c\u0dab\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0dc3\u0dbb\u0dbd \u0dc1\u0dca\u0dbb\u0dd2\u0dad\u0dba\u0d9a\u0dca</p>\n",
|
||||
"<p> Create Cora dataset</p>\n": "<p> \u0d9a\u0ddd\u0dbb\u0dcf\u0daf\u0dad\u0dca\u0dad \u0d9a\u0da7\u0dca\u0da7\u0dbd\u0dba \u0dc3\u0dcf\u0daf\u0db1\u0dca\u0db1</p>\n",
|
||||
"<p> Create GAT model</p>\n": "<p> GAT\u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba \u0dc3\u0dcf\u0daf\u0db1\u0dca\u0db1</p>\n",
|
||||
"<p> Create configurable optimizer</p>\n": "<p> \u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3\u0d9c\u0dad\u0d9a\u0dc5 \u0dc4\u0dd0\u0d9a\u0dd2 \u0db4\u0dca\u0dbb\u0dc1\u0dc3\u0dca\u0dad\u0d9a\u0dbb\u0dab\u0dba</p>\n",
|
||||
"<p> Download the dataset</p>\n": "<p> \u0daf\u0dad\u0dca\u0dad\u0d9a\u0da7\u0dca\u0da7\u0dbd\u0dba \u0db6\u0dcf\u0d9c\u0db1\u0dca\u0db1</p>\n",
|
||||
"<p> Load the dataset</p>\n": "<p> \u0daf\u0dad\u0dca\u0dad\u0d9a\u0da7\u0dca\u0da7\u0dbd\u0dba \u0db4\u0dd6\u0dbb\u0dab\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1</p>\n",
|
||||
"<p>Activation function </p>\n": "<p>\u0dc3\u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0d9a\u0dcf\u0dbb\u0dca\u0dba\u0dba </p>\n",
|
||||
"<p>Activation function after first graph attention layer </p>\n": "<p>\u0db4\u0dc5\u0db8\u0dd4\u0db4\u0dca\u0dbb\u0dc3\u0dca\u0dae\u0dcf\u0dbb \u0d85\u0dc0\u0db0\u0dcf\u0db1\u0dba \u0dc3\u0dca\u0dae\u0dbb\u0dba\u0dd9\u0db1\u0dca \u0db4\u0dc3\u0dd4 \u0dc3\u0d9a\u0dca\u0dbb\u0dd2\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0d9a\u0dcf\u0dbb\u0dca\u0dba\u0dba </p>\n",
|
||||
"<p>Adam optimizer </p>\n": "<p>\u0d86\u0daf\u0db8\u0dca\u0db4\u0dca\u0dbb\u0dc1\u0dc3\u0dca\u0dad\u0d9a\u0dbb\u0dab\u0dba </p>\n",
|
||||
"<p>Add an empty third dimension for the heads </p>\n": "<p>\u0dc4\u0dd2\u0dc3\u0dca\u0dc3\u0db3\u0dc4\u0dcf \u0dc4\u0dd2\u0dc3\u0dca \u0dad\u0dd9\u0dc0\u0db1 \u0db8\u0dcf\u0db1\u0dba\u0d9a\u0dca \u0d91\u0d9a\u0dca \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Adjacency matrix with the edge information. <span translate=no>_^_0_^_</span> is <span translate=no>_^_1_^_</span> if there is an edge from <span translate=no>_^_2_^_</span> to <span translate=no>_^_3_^_</span>. </p>\n": "<p>\u0daf\u0dcf\u0dbb\u0dba\u0dda\u0dad\u0ddc\u0dbb\u0dad\u0dd4\u0dbb\u0dd4 \u0dc3\u0db8\u0d9f \u0dc3\u0db8\u0dca\u0db6\u0db1\u0dca\u0db0 \u0dc0\u0dd3\u0db8\u0dda \u0d85\u0db1\u0dd4\u0d9a\u0dd8\u0dad\u0dd2\u0dba. <span translate=no>_^_0_^_</span> \u0dc3\u0dd2\u0da7 \u0daf\u0dcf\u0dbb\u0dba\u0d9a\u0dca \u0dad\u0dd2\u0db6\u0dda <span translate=no>_^_1_^_</span> <span translate=no>_^_2_^_</span> \u0db1\u0db8\u0dca <span translate=no>_^_3_^_</span>. </p>\n",
|
||||
"<p>Apply dropout to the input </p>\n": "<p>\u0d86\u0daf\u0dcf\u0db1\u0dba\u0da7\u0d85\u0dad\u0dc4\u0dd0\u0dbb \u0daf\u0dd0\u0db8\u0dd3\u0db8 \u0dba\u0ddc\u0daf\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Calculate configurations. </p>\n": "<p>\u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3\u0dba\u0db1\u0dca\u0d9c\u0dab\u0db1\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1. </p>\n",
|
||||
"<p>Calculate gradients </p>\n": "<p>\u0d85\u0db1\u0dd4\u0d9a\u0dca\u0dbb\u0db8\u0dd2\u0d9a\u0d9c\u0dab\u0db1\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Calculate the loss for validation nodes </p>\n": "<p>\u0dc0\u0dbd\u0d82\u0d9c\u0dd4\u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0db1\u0ddd\u0da9\u0dca \u0dc3\u0db3\u0dc4\u0dcf \u0d85\u0dbd\u0dcf\u0db7\u0dba \u0d9c\u0dab\u0db1\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Create an experiment </p>\n": "<p>\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf\u0db6\u0dd0\u0dbd\u0dd3\u0db8\u0d9a\u0dca \u0dc3\u0dcf\u0daf\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Create configurations </p>\n": "<p>\u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3\u0dba\u0db1\u0dca\u0dc3\u0dcf\u0daf\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Dataset </p>\n": "<p>\u0daf\u0dad\u0dca\u0dad\u0d9a\u0da7\u0dca\u0da7\u0dbd\u0dba </p>\n",
|
||||
"<p>Device to train on</p>\n<p>This creates configs for device, so that we can change the device by passing a config value </p>\n": "<p>\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0da7 \u0d8b\u0db4\u0dcf\u0d82\u0d9c\u0dba</p>\n<p>\u0db8\u0dd9\u0dba\u0d8b\u0db4\u0dcf\u0d82\u0d9c\u0dba \u0dc3\u0db3\u0dc4\u0dcf \u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3 \u0db1\u0dd2\u0dbb\u0dca\u0db8\u0dcf\u0dab\u0dba \u0d9a\u0dbb\u0dba\u0dd2, \u0d91\u0dc0\u0dd2\u0da7 \u0d85\u0db4\u0da7 \u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3 \u0d85\u0d9c\u0dba\u0d9a\u0dca \u0db4\u0dc3\u0dd4 \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dd9\u0db1\u0dca \u0d8b\u0db4\u0dcf\u0d82\u0d9c\u0dba \u0dc0\u0dd9\u0db1\u0dc3\u0dca \u0d9a\u0dc5 \u0dc4\u0dd0\u0d9a\u0dd2\u0dba </p>\n",
|
||||
"<p>Download dataset </p>\n": "<p>\u0db6\u0dcf\u0d9c\u0dad\u0daf\u0dad\u0dca\u0dad \u0d9a\u0da7\u0dca\u0da7\u0dbd\u0dba </p>\n",
|
||||
"<p>Dropout </p>\n": "<p>\u0dc4\u0dd0\u0dbd\u0dd3\u0db8 </p>\n",
|
||||
"<p>Dropout probability </p>\n": "<p>\u0d85\u0dad\u0dc4\u0dd0\u0dbb\u0daf\u0dd0\u0db8\u0dd3\u0db8\u0dda \u0dc3\u0db8\u0dca\u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf\u0dc0 </p>\n",
|
||||
"<p>Empty adjacency matrix - an identity matrix </p>\n": "<p>\u0dc4\u0dd2\u0dc3\u0dcaadjacency \u0db1\u0dca\u0dba\u0dcf\u0dc3\u0dba - \u0d85\u0db1\u0db1\u0dca\u0dba\u0dad\u0dcf \u0d85\u0db1\u0dd4\u0d9a\u0dd8\u0dad\u0dd2\u0dba\u0d9a\u0dca </p>\n",
|
||||
"<p>Evaluate the model </p>\n": "<p>\u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba\u0dad\u0d9a\u0dca\u0dc3\u0dda\u0dbb\u0dd4 \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Evaluate the model again </p>\n": "<p>\u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba\u0db1\u0dd0\u0dc0\u0dad \u0d87\u0d9c\u0dba\u0dd3\u0db8\u0da7 \u0dbd\u0d9a\u0dca \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Feature vectors for all nodes </p>\n": "<p>\u0dc3\u0dd2\u0dba\u0dbd\u0dd4\u0db8\u0db1\u0ddd\u0da9\u0dca \u0dc3\u0db3\u0dc4\u0dcf \u0dc0\u0dd2\u0dc1\u0dda\u0dc2\u0dcf\u0d82\u0d9c \u0dc0\u0dcf\u0dc4\u0d9a\u0dba\u0db1\u0dca </p>\n",
|
||||
"<p>Final graph attention layer where we average the heads </p>\n": "<p>\u0d85\u0db4\u0dd2\u0dc4\u0dd2\u0dc3\u0dca \u0dc3\u0dcf\u0db8\u0dcf\u0db1\u0dca\u0dba\u0dba \u0d91\u0dc4\u0dd2\u0daf\u0dd3 \u0d85\u0dc0\u0dc3\u0db1\u0dca \u0db4\u0dca\u0dbb\u0dc3\u0dca\u0dad\u0dcf\u0dbb\u0dba \u0d85\u0dc0\u0db0\u0dcf\u0db1\u0dba \u0dc3\u0dca\u0dae\u0dbb\u0dba </p>\n",
|
||||
"<p>First graph attention layer </p>\n": "<p>\u0db4\u0dc5\u0db8\u0dd4\u0db4\u0dca\u0dbb\u0dc3\u0dca\u0dae\u0dcf\u0dbb \u0d85\u0dc0\u0db0\u0dcf\u0db1\u0dba \u0dc3\u0dca\u0dae\u0dbb\u0dba </p>\n",
|
||||
"<p>First graph attention layer where we concatenate the heads </p>\n": "<p>\u0d85\u0db4\u0dd2\u0dc4\u0dd2\u0dc3\u0dca concatenate \u0d91\u0dc4\u0dd2\u0daf\u0dd3 \u0db4\u0dc5\u0db8\u0dd4 \u0db4\u0dca\u0dbb\u0dc3\u0dca\u0dad\u0dcf\u0dbb\u0dba \u0d85\u0dc0\u0db0\u0dcf\u0db1\u0dba \u0dc3\u0dca\u0dae\u0dbb\u0dba </p>\n",
|
||||
"<p>Get the class names and assign an unique integer to each of them </p>\n": "<p>\u0db4\u0db1\u0dca\u0dad\u0dd2\u0db1\u0dcf\u0db8 \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 \u0dc3\u0dc4 \u0d92 \u0d91\u0d9a\u0dca \u0d91\u0d9a\u0dca \u0dc3\u0db3\u0dc4\u0dcf \u0d85\u0daf\u0dca\u0dc0\u0dd2\u0dad\u0dd3\u0dba \u0db4\u0dd6\u0dbb\u0dca\u0dab \u0dc3\u0d82\u0d9b\u0dca\u0dba\u0dcf\u0dc0\u0d9a\u0dca \u0db4\u0dc0\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Get the feature vectors </p>\n": "<p>\u0dc0\u0dd2\u0dc1\u0dda\u0dc2\u0dcf\u0d82\u0d9c\u0daf\u0ddb\u0dc1\u0dd2\u0d9a \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Get the labels as those integers </p>\n": "<p>\u0d91\u0db8\u0dbd\u0dda\u0db6\u0dbd\u0dca \u0d91\u0db8 \u0db1\u0dd2\u0d9b\u0dd2\u0dbd \u0dbd\u0dd9\u0dc3 \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Get the loss for training nodes </p>\n": "<p>\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0db1\u0ddd\u0da9\u0dca \u0dc3\u0db3\u0dc4\u0dcf \u0d85\u0dbd\u0dcf\u0db7\u0dba \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Get the number of classes </p>\n": "<p>\u0db4\u0db1\u0dca\u0dad\u0dd2\u0d9c\u0dab\u0db1 \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Get the paper ids </p>\n": "<p>\u0d9a\u0da9\u0daf\u0dcf\u0dc3\u0dd2\u0dc4\u0dd0\u0db3\u0dd4\u0db1\u0dd4\u0db8\u0dca\u0db4\u0dad\u0dca \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Labels for each node </p>\n": "<p>\u0d91\u0d9a\u0dca\u0d91\u0d9a\u0dca \u0db1\u0ddd\u0da9\u0dba \u0dc3\u0db3\u0dc4\u0dcf \u0dbd\u0dda\u0db6\u0dbd </p>\n",
|
||||
"<p>Load the citations, it's a list of pairs of integers. </p>\n": "<p>\u0d8b\u0db4\u0dd4\u0da7\u0dcf\u0daf\u0dd0\u0d9a\u0dca\u0dc0\u0dd3\u0db8\u0dca \u0db4\u0dd6\u0dbb\u0dab\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1, \u0d91\u0dba \u0db1\u0dd2\u0d9b\u0dd2\u0dbd \u0dba\u0dd4\u0d9c\u0dbd \u0dbd\u0dd0\u0dba\u0dd2\u0dc3\u0dca\u0dad\u0dd4\u0dc0\u0d9a\u0dd2. </p>\n",
|
||||
"<p>Log the accuracy </p>\n": "<p>\u0db1\u0dd2\u0dbb\u0dc0\u0daf\u0dca\u0dba\u0dad\u0dcf\u0dc0\u0dba\u0dbd\u0ddc\u0d9c\u0dca \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Log the loss </p>\n": "<p>\u0d85\u0dbd\u0dcf\u0db7\u0dba\u0dbd\u0ddc\u0d9c\u0dca \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Loss function </p>\n": "<p>\u0db4\u0dcf\u0da9\u0dd4\u0dc1\u0dca\u0dbb\u0dd2\u0dad\u0dba </p>\n",
|
||||
"<p>Make all the gradients zero </p>\n": "<p>\u0dc3\u0dd2\u0dba\u0dbd\u0dd4\u0db8\u0d85\u0db1\u0dd4\u0d9a\u0dca\u0dbb\u0db8\u0dd2\u0d9a \u0dc1\u0dd4\u0db1\u0dca\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Map of paper id to index </p>\n": "<p>\u0d9a\u0da9\u0daf\u0dcf\u0dc3\u0dd2\u0dc4\u0dd0\u0db3\u0dd4\u0db1\u0dd4\u0db8\u0dca\u0db4\u0dad\u0dda \u0dc3\u0dd2\u0da7 \u0daf\u0dbb\u0dca\u0dc1\u0d9a\u0dba \u0daf\u0d9a\u0dca\u0dc0\u0dcf \u0dc3\u0dd2\u0dad\u0dd2\u0dba\u0db8 </p>\n",
|
||||
"<p>Mark the citations in the adjacency matrix </p>\n": "<p>\u0db8\u0dd9\u0db8adjacency \u0db1\u0dca\u0dba\u0dcf\u0dc3\u0dba \u0dad\u0dd4\u0dc5 \u0d8b\u0db4\u0dd4\u0da7\u0dcf \u0daf\u0dd0\u0d9a\u0dca\u0dc0\u0dd3\u0db8\u0dca \u0dc3\u0dbd\u0d9a\u0dd4\u0dab\u0dd4 </p>\n",
|
||||
"<p>Model </p>\n": "<p>\u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba </p>\n",
|
||||
"<p>Move the adjacency matrix to the device </p>\n": "<p>\u0d8b\u0db4\u0d9a\u0dbb\u0dab\u0dba\u0da7\u0d85\u0db1\u0dd4\u0d9a\u0dd8\u0dad\u0dd2\u0dba \u0d9c\u0dd9\u0db1 \u0dba\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Move the feature vectors to the device </p>\n": "<p>\u0dc0\u0dd2\u0dc1\u0dda\u0dc2\u0dcf\u0d82\u0d9c\u0daf\u0ddb\u0dc1\u0dd2\u0d9a \u0d8b\u0db4\u0dcf\u0d82\u0d9c\u0dba\u0da7 \u0d9c\u0dd9\u0db1\u0dba\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Move the labels to the device </p>\n": "<p>\u0dbd\u0dda\u0db6\u0dbd\u0d8b\u0db4\u0dcf\u0d82\u0d9c\u0dba\u0da7 \u0d9c\u0dd9\u0db1\u0dba\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>No need to compute gradients </p>\n": "<p>\u0d85\u0db1\u0dd4\u0d9a\u0dca\u0dbb\u0db8\u0dd2\u0d9a\u0d9c\u0dab\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0da7 \u0d85\u0dc0\u0dc1\u0dca\u0dba \u0db1\u0dd0\u0dad </p>\n",
|
||||
"<p>Nodes for training </p>\n": "<p>\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0dc0\u0dc3\u0db3\u0dc4\u0dcf \u0db1\u0ddd\u0da9\u0dca </p>\n",
|
||||
"<p>Nodes for validation </p>\n": "<p>\u0dc0\u0dbd\u0d82\u0d9c\u0dd4\u0d9a\u0dd2\u0dbb\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0db1\u0ddd\u0da9\u0dca </p>\n",
|
||||
"<p>Normalize the feature vectors </p>\n": "<p>\u0dc0\u0dd2\u0dc1\u0dda\u0dc2\u0dcf\u0d82\u0d9c\u0daf\u0ddb\u0dc1\u0dd2\u0d9a \u0dc3\u0dcf\u0db8\u0dcf\u0db1\u0dca\u0dba\u0d9a\u0dbb\u0dab\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Number of classes for classification </p>\n": "<p>\u0dc0\u0dbb\u0dca\u0d9c\u0dd3\u0d9a\u0dbb\u0dab\u0dba\u0dc3\u0db3\u0dc4\u0dcf \u0db4\u0db1\u0dca\u0dad\u0dd2 \u0d9c\u0dab\u0db1 </p>\n",
|
||||
"<p>Number of features in the first graph attention layer </p>\n": "<p>\u0db4\u0dc5\u0db8\u0dd4\u0db4\u0dca\u0dbb\u0dc3\u0dca\u0dad\u0dcf\u0dbb\u0dba\u0dda \u0d85\u0dc0\u0db0\u0dcf\u0db1\u0dba \u0dc3\u0dca\u0dae\u0dbb\u0dba\u0dda \u0dc0\u0dd2\u0dc1\u0dda\u0dc2\u0dcf\u0d82\u0d9c \u0d9c\u0dab\u0db1 </p>\n",
|
||||
"<p>Number of features in the input </p>\n": "<p>\u0d86\u0daf\u0dcf\u0db1\u0dba\u0dda\u0dc0\u0dd2\u0dc1\u0dda\u0dc2\u0dcf\u0d82\u0d9c \u0d9c\u0dab\u0db1 </p>\n",
|
||||
"<p>Number of features per node in the input </p>\n": "<p>\u0d86\u0daf\u0dcf\u0db1\u0dba\u0dda\u0db1\u0ddd\u0da9\u0dba\u0d9a\u0da7 \u0dc0\u0dd2\u0dc1\u0dda\u0dc2\u0dcf\u0d82\u0d9c \u0d9c\u0dab\u0db1 </p>\n",
|
||||
"<p>Number of heads </p>\n": "<p>\u0dc4\u0dd2\u0dc3\u0dca\u0d9c\u0dab\u0db1 </p>\n",
|
||||
"<p>Number of nodes to train on </p>\n": "<p>\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0da7 \u0db1\u0ddd\u0da9\u0dca \u0d9c\u0dab\u0db1 </p>\n",
|
||||
"<p>Number of training iterations </p>\n": "<p>\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0db4\u0dd4\u0db1\u0dbb\u0dcf\u0dc0\u0dbb\u0dca\u0dad\u0db1 \u0d9c\u0dab\u0db1 </p>\n",
|
||||
"<p>Optimizer </p>\n": "<p>\u0db4\u0dca\u0dbb\u0dc1\u0dc3\u0dca\u0dad\u0d9a\u0dbb\u0dab\u0dba </p>\n",
|
||||
"<p>Output layer (without activation) for logits </p>\n": "<p>\u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca\u0dc3\u0db3\u0dc4\u0dcf \u0db4\u0dca\u0dbb\u0dad\u0dd2\u0daf\u0dcf\u0db1 \u0dc3\u0dca\u0dae\u0dbb\u0dba (\u0dc3\u0d9a\u0dca\u0dbb\u0dd2\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0d9a\u0dd2\u0db1\u0dca \u0dad\u0ddc\u0dbb\u0dc0) </p>\n",
|
||||
"<p>Random indexes </p>\n": "<p>\u0d85\u0dc4\u0db9\u0dd4\u0daf\u0dbb\u0dca\u0dc1\u0d9a </p>\n",
|
||||
"<p>Read the paper ids, feature vectors, and labels </p>\n": "<p>\u0d9a\u0da9\u0daf\u0dcf\u0dc3\u0dd2\u0dc4\u0dd0\u0db3\u0dd4\u0db1\u0dd4\u0db8\u0dca\u0db4\u0dad\u0dca, \u0dc0\u0dd2\u0dc1\u0dda\u0dc2\u0dcf\u0d82\u0d9c \u0daf\u0ddb\u0dc1\u0dd2\u0d9a \u0dc3\u0dc4 \u0dbd\u0dda\u0db6\u0dbd \u0d9a\u0dd2\u0dba\u0dc0\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Run the training </p>\n": "<p>\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0dc0\u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Save logs </p>\n": "<p>\u0dbd\u0ddc\u0d9c\u0dca\u0dc3\u0dd4\u0dbb\u0d9a\u0dd2\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Set mode to evaluation mode for validation </p>\n": "<p>\u0dc0\u0dbd\u0d82\u0d9c\u0dd4\u0d9a\u0dd2\u0dbb\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0d87\u0d9c\u0dba\u0dd3\u0db8\u0dda \u0db4\u0dca\u0dbb\u0d9a\u0dcf\u0dbb\u0dba\u0da7 \u0dc3\u0d9a\u0dc3\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Set of class names and an unique integer index </p>\n": "<p>\u0db4\u0db1\u0dca\u0dad\u0dd2\u0db1\u0dcf\u0db8 \u0d9a\u0da7\u0dca\u0da7\u0dbd\u0dba\u0d9a\u0dca \u0dc3\u0dc4 \u0d85\u0daf\u0dca\u0dc0\u0dd2\u0dad\u0dd3\u0dba \u0db4\u0dd6\u0dbb\u0dca\u0dab \u0dc3\u0d82\u0d9b\u0dca\u0dba\u0dcf \u0daf\u0dbb\u0dca\u0dc1\u0d9a\u0dba </p>\n",
|
||||
"<p>Set the model to training mode </p>\n": "<p>\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0db8\u0dcf\u0daf\u0dd2\u0dbd\u0dd2\u0dba\u0da7 \u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba \u0dc3\u0d9a\u0dc3\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Start and watch the experiment </p>\n": "<p>\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf\u0db6\u0dd0\u0dbd\u0dd3\u0db8 \u0d86\u0dbb\u0db8\u0dca\u0db7 \u0d9a\u0dbb \u0db1\u0dbb\u0db9\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Take optimization step </p>\n": "<p>\u0db4\u0dca\u0dbb\u0dc1\u0dc3\u0dca\u0dad\u0dd2\u0d9a\u0dbb\u0dab\u0db4\u0dd2\u0dba\u0dc0\u0dbb \u0d9c\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>The pair of paper indexes </p>\n": "<p>\u0d9a\u0da9\u0daf\u0dcf\u0dc3\u0dd2\u0daf\u0dbb\u0dca\u0dc1\u0d9a \u0dba\u0dd4\u0d9c\u0dbd\u0dba </p>\n",
|
||||
"<p>Training loop </p>\n": "<p>\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0dbd\u0dd6\u0db4\u0dba </p>\n",
|
||||
"<p>We build a symmetrical graph, where if paper <span translate=no>_^_0_^_</span> referenced paper <span translate=no>_^_1_^_</span> we place an adge from <span translate=no>_^_2_^_</span> to <span translate=no>_^_3_^_</span> as well as an edge from <span translate=no>_^_4_^_</span> to <span translate=no>_^_5_^_</span>. </p>\n": "<p>\u0d85\u0db4\u0dd2\u0dc3\u0db8\u0db8\u0dd2\u0dad\u0dd2\u0d9a \u0db4\u0dca\u0dbb\u0dc3\u0dca\u0dad\u0dcf\u0dbb\u0dba \u0d89\u0daf\u0dd2, \u0d91\u0dc4\u0dd2\u0daf\u0dd3 \u0d9a\u0da9\u0daf\u0dcf\u0dc3\u0dd2 <span translate=no>_^_0_^_</span> \u0dba\u0ddc\u0db8\u0dd4 \u0d9a\u0da9\u0daf\u0dcf\u0dc3\u0dd2 \u0db1\u0db8\u0dca <span translate=no>_^_1_^_</span> \u0d85\u0db4\u0dd2 \u0dc3\u0dd2\u0da7 <span translate=no>_^_2_^_</span> adge \u0dad\u0dd0\u0db1\u0dd2\u0db1\u0dca <span translate=no>_^_3_^_</span> \u0db8\u0dd9\u0db1\u0dca\u0db8 \u0dc3\u0dd2\u0da7 \u0d85\u0daf\u0dca\u0daf\u0dbb <span translate=no>_^_4_^_</span> \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0da7 <span translate=no>_^_5_^_</span>. </p>\n",
|
||||
"<p>Whether to include edges. This is test how much accuracy is lost if we ignore the citation network. </p>\n": "<p>\u0daf\u0dcf\u0dbb\u0d87\u0dad\u0dd4\u0dc5\u0dad\u0dca \u0d9a\u0dc5 \u0dba\u0dd4\u0dad\u0dd4\u0daf \u0dba\u0db1\u0dca\u0db1. \u0db8\u0dd9\u0db8 \u0d85\u0db4\u0dd2 \u0d8b\u0db4\u0dd4\u0da7\u0dcf \u0daf\u0dd0\u0d9a\u0dca\u0dc0\u0dd3\u0db8\u0dda \u0da2\u0dcf\u0dbd\u0dba \u0db1\u0ddc\u0dc3\u0dbd\u0d9a\u0dcf \u0db1\u0db8\u0dca \u0d9a\u0ddc\u0db4\u0db8\u0dab \u0db1\u0dd2\u0dbb\u0dc0\u0daf\u0dca\u0dba\u0dad\u0dcf\u0dc0 \u0d85\u0dc4\u0dd2\u0db8\u0dd2 \u0dc0\u0dda \u0db4\u0dbb\u0dd3\u0d9a\u0dca\u0dc2\u0dcf \u0dc0\u0dda. </p>\n",
|
||||
"<p>Whether to include the citation network </p>\n": "<p>\u0d8b\u0db4\u0dd4\u0da7\u0dcf\u0daf\u0dd0\u0d9a\u0dca\u0dc0\u0dd3\u0db8\u0dda \u0da2\u0dcf\u0dbd\u0dba \u0d87\u0dad\u0dd4\u0dc5\u0dad\u0dca \u0d9a\u0dc5 \u0dba\u0dd4\u0dad\u0dd4\u0daf \u0dba\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the features vectors of shape <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> is the adjacency matrix of the form <span translate=no>_^_3_^_</span> or <span translate=no>_^_4_^_</span></li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span> \u0dc4\u0dd0\u0da9\u0dba\u0dda \u0dbd\u0d9a\u0dca\u0dc2\u0dab \u0daf\u0ddb\u0dc1\u0dd2\u0d9a \u0dc0\u0dda <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> \u0dba\u0db1\u0dd4 \u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba\u0dda \u0d85\u0db1\u0dd4\u0d9a\u0dd8\u0dad\u0dd2\u0dba\u0dda \u0d85\u0db1\u0dd4\u0d9a\u0dd8\u0dad\u0dd2\u0dba <span translate=no>_^_3_^_</span> \u0dc4\u0ddd <span translate=no>_^_4_^_</span></li></ul>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the number of features per node </li>\n<li><span translate=no>_^_1_^_</span> is the number of features in the first graph attention layer </li>\n<li><span translate=no>_^_2_^_</span> is the number of classes </li>\n<li><span translate=no>_^_3_^_</span> is the number of heads in the graph attention layers </li>\n<li><span translate=no>_^_4_^_</span> is the dropout probability</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span> node \u0d91\u0d9a\u0d9a\u0dca \u0db8\u0dad\u0db8 \u0d8a\u0da7 \u0d85\u0daf\u0dcf\u0dbd \u0dc0\u0dd2\u0dc1\u0dda\u0dc2\u0dcf\u0d82\u0d9c \u0dc3\u0d82\u0d9b\u0dca\u0dba\u0dcf\u0dc0 \u0dc0\u0dda </li>\n<li><span translate=no>_^_1_^_</span> \u0db4\u0dc5\u0db8\u0dd4 \u0db4\u0dca\u0dbb\u0dc3\u0dca\u0dad\u0dcf\u0dbb\u0dba \u0d85\u0dc0\u0db0\u0dcf\u0db1\u0dba \u0dc3\u0dca\u0dae\u0dbb\u0dba \u0dbd\u0d9a\u0dca\u0dc2\u0dab \u0dc3\u0d82\u0d9b\u0dca\u0dba\u0dcf\u0dc0 \u0dc0\u0dda </li>\n<li><span translate=no>_^_2_^_</span> \u0dba\u0db1\u0dd4 \u0db4\u0db1\u0dca\u0dad\u0dd2 \u0d9c\u0dab\u0db1 </li>\n<li><span translate=no>_^_3_^_</span> \u0db4\u0dca\u0dbb\u0dc3\u0dca\u0dae\u0dcf\u0dbb \u0d85\u0dc0\u0db0\u0dcf\u0db1\u0dba \u0dc3\u0dca\u0dae\u0dbb \u0dc0\u0dbd \u0dc4\u0dd2\u0dc3\u0dca \u0d9c\u0dab\u0db1 </li>\n<li><span translate=no>_^_4_^_</span> \u0d85\u0dad\u0dc4\u0dd0\u0dbb \u0daf\u0dd0\u0db8\u0dd3\u0db8\u0dda \u0dc3\u0db8\u0dca\u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf\u0dc0</li></ul>\n",
|
||||
"This trains is a Graph Attention Network (GAT) on Cora dataset": "\u0db8\u0dd9\u0db8 \u0daf\u0dd4\u0db8\u0dca\u0dbb\u0dd2\u0dba \u0d9a\u0ddd\u0dbb\u0dcf \u0daf\u0dad\u0dca\u0dad \u0dc3\u0db8\u0dd4\u0daf\u0dcf\u0dba \u0db8\u0dad \u0db4\u0dca\u0dbb\u0dc3\u0dca\u0dad\u0dcf\u0dbb\u0dba \u0d85\u0dc0\u0db0\u0dcf\u0db1\u0dba \u0da2\u0dcf\u0dbd\u0dba (GAT) \u0dc0\u0dda",
|
||||
"Train a Graph Attention Network (GAT) on Cora dataset": "\u0d9a\u0ddd\u0dbb\u0dcf \u0daf\u0dad\u0dca\u0dad \u0d9a\u0da7\u0dca\u0da7\u0dbd\u0dba \u0db8\u0dad \u0db4\u0dca\u0dbb\u0dc3\u0dca\u0dad\u0dcf\u0dbb\u0dba \u0d85\u0dc0\u0db0\u0dcf\u0db1\u0dba \u0da2\u0dcf\u0dbd\u0dba (GAT) \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4"
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
{
|
||||
"<h1>Train a Graph Attention Network (GAT) on Cora dataset</h1>\n": "<h1>\u5728 Cora \u6570\u636e\u96c6\u4e0a\u8bad\u7ec3\u56fe\u6ce8\u610f\u529b\u7f51\u7edc (GAT)</h1>\n",
|
||||
"<h2><a href=\"https://linqs.soe.ucsc.edu/data\">Cora Dataset</a></h2>\n<p>Cora dataset is a dataset of research papers. For each paper we are given a binary feature vector that indicates the presence of words. Each paper is classified into one of 7 classes. The dataset also has the citation network.</p>\n<p>The papers are the nodes of the graph and the edges are the citations.</p>\n<p>The task is to classify the nodes to the 7 classes with feature vectors and citation network as input.</p>\n": "<h2><a href=\"https://linqs.soe.ucsc.edu/data\">Cora \u6570\u636e\u96c6</a></h2>\n<p>Cora \u6570\u636e\u96c6\u662f\u7814\u7a76\u8bba\u6587\u7684\u6570\u636e\u96c6\u3002\u5bf9\u4e8e\u6bcf\u7bc7\u8bba\u6587\uff0c\u6211\u4eec\u90fd\u5f97\u5230\u4e00\u4e2a\u4e8c\u8fdb\u5236\u7279\u5f81\u5411\u91cf\uff0c\u8be5\u5411\u91cf\u8868\u793a\u5355\u8bcd\u7684\u5b58\u5728\u3002\u6bcf\u7bc7\u8bba\u6587\u5206\u4e3a 7 \u4e2a\u7c7b\u522b\u4e4b\u4e00\u3002\u8be5\u6570\u636e\u96c6\u8fd8\u5177\u6709\u5f15\u6587\u7f51\u7edc\u3002</p>\n<p>\u8bba\u6587\u662f\u56fe\u7684\u8282\u70b9\uff0c\u8fb9\u7f18\u662f\u5f15\u6587\u3002</p>\n<p>\u4efb\u52a1\u662f\u4f7f\u7528\u7279\u5f81\u5411\u91cf\u548c\u5f15\u6587\u7f51\u7edc\u4f5c\u4e3a\u8f93\u5165\uff0c\u5c06\u8282\u70b9\u5206\u7c7b\u4e3a 7 \u7c7b\u3002</p>\n",
|
||||
"<h2>Configurations</h2>\n": "<h2>\u914d\u7f6e</h2>\n",
|
||||
"<h2>Graph Attention Network (GAT)</h2>\n<p>This graph attention network has two <a href=\"index.html\">graph attention layers</a>.</p>\n": "<h2>Graph \u6ce8\u610f\u529b\u7f51\u7edc (GAT)</h2>\n<p>\u8fd9\u4e2a\u56fe\u5f62\u5173\u6ce8\u7f51\u7edc\u6709\u4e24\u4e2a<a href=\"index.html\">\u56fe\u5f62\u5173\u6ce8\u5c42</a>\u3002</p>\n",
|
||||
"<h3>Training loop</h3>\n<p>We do full batch training since the dataset is small. If we were to sample and train we will have to sample a set of nodes for each training step along with the edges that span across those selected nodes.</p>\n": "<h3>\u8bad\u7ec3\u5faa\u73af</h3>\n<p>\u7531\u4e8e\u6570\u636e\u96c6\u5f88\u5c0f\uff0c\u6211\u4eec\u8fdb\u884c\u5168\u6279\u91cf\u8bad\u7ec3\u3002\u5982\u679c\u8981\u8fdb\u884c\u91c7\u6837\u548c\u8bad\u7ec3\uff0c\u6211\u4eec\u5c06\u4e0d\u5f97\u4e0d\u4e3a\u6bcf\u4e2a\u8bad\u7ec3\u6b65\u9aa4\u5bf9\u4e00\u7ec4\u8282\u70b9\u4ee5\u53ca\u8de8\u8d8a\u8fd9\u4e9b\u9009\u5b9a\u8282\u70b9\u7684\u8fb9\u8fdb\u884c\u91c7\u6837\u3002</p>\n",
|
||||
"<p> </p>\n": "<p></p>\n",
|
||||
"<p> A simple function to calculate the accuracy</p>\n": "<p>\u8ba1\u7b97\u7cbe\u5ea6\u7684\u7b80\u5355\u51fd\u6570</p>\n",
|
||||
"<p> Create Cora dataset</p>\n": "<p>\u521b\u5efa Cora \u6570\u636e\u96c6</p>\n",
|
||||
"<p> Create GAT model</p>\n": "<p>\u521b\u5efa GAT \u6a21\u578b</p>\n",
|
||||
"<p> Create configurable optimizer</p>\n": "<p>\u521b\u5efa\u53ef\u914d\u7f6e\u7684\u4f18\u5316\u5668</p>\n",
|
||||
"<p> Download the dataset</p>\n": "<p>\u4e0b\u8f7d\u6570\u636e\u96c6</p>\n",
|
||||
"<p> Load the dataset</p>\n": "<p>\u52a0\u8f7d\u6570\u636e\u96c6</p>\n",
|
||||
"<p>Activation function </p>\n": "<p>\u6fc0\u6d3b\u529f\u80fd</p>\n",
|
||||
"<p>Activation function after first graph attention layer </p>\n": "<p>\u7b2c\u4e00\u4e2a\u56fe\u5f62\u5173\u6ce8\u5c42\u4e4b\u540e\u7684\u6fc0\u6d3b\u529f\u80fd</p>\n",
|
||||
"<p>Adam optimizer </p>\n": "<p>Adam \u4f18\u5316\u5668</p>\n",
|
||||
"<p>Add an empty third dimension for the heads </p>\n": "<p>\u4e3a\u5934\u90e8\u6dfb\u52a0\u4e00\u4e2a\u7a7a\u7684\u7b2c\u4e09\u4e2a\u7ef4\u5ea6</p>\n",
|
||||
"<p>Adjacency matrix with the edge information. <span translate=no>_^_0_^_</span> is <span translate=no>_^_1_^_</span> if there is an edge from <span translate=no>_^_2_^_</span> to <span translate=no>_^_3_^_</span>. </p>\n": "<p>\u5305\u542b\u8fb9\u4fe1\u606f\u7684\u90bb\u63a5\u77e9\u9635\u3002<span translate=no>_^_0_^_</span><span translate=no>_^_1_^_</span>\u5982\u679c\u5b58\u5728\u4ece<span translate=no>_^_2_^_</span>\u5230\u7684\u8fb9\u7f18<span translate=no>_^_3_^_</span>\u3002</p>\n",
|
||||
"<p>Apply dropout to the input </p>\n": "<p>\u5c06\u4e22\u5931\u5e94\u7528\u4e8e\u8f93\u5165</p>\n",
|
||||
"<p>Calculate configurations. </p>\n": "<p>\u8ba1\u7b97\u914d\u7f6e\u3002</p>\n",
|
||||
"<p>Calculate gradients </p>\n": "<p>\u8ba1\u7b97\u68af\u5ea6</p>\n",
|
||||
"<p>Calculate the loss for validation nodes </p>\n": "<p>\u8ba1\u7b97\u9a8c\u8bc1\u8282\u70b9\u7684\u635f\u5931</p>\n",
|
||||
"<p>Create an experiment </p>\n": "<p>\u521b\u5efa\u5b9e\u9a8c</p>\n",
|
||||
"<p>Create configurations </p>\n": "<p>\u521b\u5efa\u914d\u7f6e</p>\n",
|
||||
"<p>Dataset </p>\n": "<p>\u6570\u636e\u96c6</p>\n",
|
||||
"<p>Device to train on</p>\n<p>This creates configs for device, so that we can change the device by passing a config value </p>\n": "<p>\u7528\u4e8e\u8bad\u7ec3\u7684\u8bbe\u5907</p>\n<p>\u8fd9\u5c06\u4e3a\u8bbe\u5907\u521b\u5efa\u914d\u7f6e\uff0c\u4ee5\u4fbf\u6211\u4eec\u53ef\u4ee5\u901a\u8fc7\u4f20\u9012\u914d\u7f6e\u503c\u6765\u66f4\u6539\u8bbe\u5907</p>\n",
|
||||
"<p>Download dataset </p>\n": "<p>\u4e0b\u8f7d\u6570\u636e\u96c6</p>\n",
|
||||
"<p>Dropout </p>\n": "<p>\u8f8d\u5b66</p>\n",
|
||||
"<p>Dropout probability </p>\n": "<p>\u8f8d\u5b66\u6982\u7387</p>\n",
|
||||
"<p>Empty adjacency matrix - an identity matrix </p>\n": "<p>\u7a7a\u90bb\u63a5\u77e9\u9635-\u6052\u7b49\u77e9\u9635</p>\n",
|
||||
"<p>Evaluate the model </p>\n": "<p>\u8bc4\u4f30\u6a21\u578b</p>\n",
|
||||
"<p>Evaluate the model again </p>\n": "<p>\u518d\u6b21\u8bc4\u4f30\u6a21\u578b</p>\n",
|
||||
"<p>Feature vectors for all nodes </p>\n": "<p>\u6240\u6709\u8282\u70b9\u7684\u7279\u5f81\u5411\u91cf</p>\n",
|
||||
"<p>Final graph attention layer where we average the heads </p>\n": "<p>\u6700\u540e\u4e00\u5f20\u56fe\u5173\u6ce8\u5c42\uff0c\u6211\u4eec\u5e73\u5747\u5934\u90e8</p>\n",
|
||||
"<p>First graph attention layer </p>\n": "<p>\u7b2c\u4e00\u4e2a\u56fe\u5f62\u5173\u6ce8\u5c42</p>\n",
|
||||
"<p>First graph attention layer where we concatenate the heads </p>\n": "<p>\u6211\u4eec\u8fde\u63a5\u5934\u90e8\u7684\u7b2c\u4e00\u4e2a\u56fe\u5f62\u6ce8\u610f\u5c42</p>\n",
|
||||
"<p>Get the class names and assign an unique integer to each of them </p>\n": "<p>\u83b7\u53d6\u7c7b\u540d\u5e76\u4e3a\u6bcf\u4e2a\u7c7b\u5206\u914d\u4e00\u4e2a\u552f\u4e00\u7684\u6574\u6570</p>\n",
|
||||
"<p>Get the feature vectors </p>\n": "<p>\u83b7\u53d6\u7279\u5f81\u5411\u91cf</p>\n",
|
||||
"<p>Get the labels as those integers </p>\n": "<p>\u83b7\u53d6\u8fd9\u4e9b\u6574\u6570\u7684\u6807\u7b7e</p>\n",
|
||||
"<p>Get the loss for training nodes </p>\n": "<p>\u83b7\u5f97\u8bad\u7ec3\u8282\u70b9\u7684\u635f\u5931</p>\n",
|
||||
"<p>Get the number of classes </p>\n": "<p>\u83b7\u53d6\u73ed\u7ea7\u6570</p>\n",
|
||||
"<p>Get the paper ids </p>\n": "<p>\u83b7\u53d6\u7eb8\u8d28\u8bc1\u4ef6</p>\n",
|
||||
"<p>Labels for each node </p>\n": "<p>\u6bcf\u4e2a\u8282\u70b9\u7684\u6807\u7b7e</p>\n",
|
||||
"<p>Load the citations, it's a list of pairs of integers. </p>\n": "<p>\u52a0\u8f7d\u5f15\u6587\uff0c\u8fd9\u662f\u4e00\u4e2a\u6574\u6570\u5bf9\u7684\u5217\u8868\u3002</p>\n",
|
||||
"<p>Log the accuracy </p>\n": "<p>\u8bb0\u5f55\u51c6\u786e\u6027</p>\n",
|
||||
"<p>Log the loss </p>\n": "<p>\u8bb0\u5f55\u635f\u5931</p>\n",
|
||||
"<p>Loss function </p>\n": "<p>\u4e8f\u635f\u51fd\u6570</p>\n",
|
||||
"<p>Make all the gradients zero </p>\n": "<p>\u5c06\u6240\u6709\u6e10\u53d8\u8bbe\u4e3a\u96f6</p>\n",
|
||||
"<p>Map of paper id to index </p>\n": "<p>\u7eb8\u5f20 ID \u5230\u7d22\u5f15\u7684\u6620\u5c04</p>\n",
|
||||
"<p>Mark the citations in the adjacency matrix </p>\n": "<p>\u5728\u90bb\u63a5\u77e9\u9635\u4e2d\u6807\u8bb0\u5f15\u7528</p>\n",
|
||||
"<p>Model </p>\n": "<p>\u578b\u53f7</p>\n",
|
||||
"<p>Move the adjacency matrix to the device </p>\n": "<p>\u5c06\u90bb\u63a5\u77e9\u9635\u79fb\u81f3\u8bbe\u5907</p>\n",
|
||||
"<p>Move the feature vectors to the device </p>\n": "<p>\u5c06\u7279\u5f81\u5411\u91cf\u79fb\u52a8\u5230\u8bbe\u5907</p>\n",
|
||||
"<p>Move the labels to the device </p>\n": "<p>\u5c06\u6807\u7b7e\u79fb\u5230\u8bbe\u5907\u4e0a</p>\n",
|
||||
"<p>No need to compute gradients </p>\n": "<p>\u65e0\u9700\u8ba1\u7b97\u68af\u5ea6</p>\n",
|
||||
"<p>Nodes for training </p>\n": "<p>\u8bad\u7ec3\u8282\u70b9</p>\n",
|
||||
"<p>Nodes for validation </p>\n": "<p>\u7528\u4e8e\u9a8c\u8bc1\u7684\u8282\u70b9</p>\n",
|
||||
"<p>Normalize the feature vectors </p>\n": "<p>\u5f52\u4e00\u5316\u7279\u5f81\u5411\u91cf</p>\n",
|
||||
"<p>Number of classes for classification </p>\n": "<p>\u7528\u4e8e\u5206\u7c7b\u7684\u7c7b\u6570</p>\n",
|
||||
"<p>Number of features in the first graph attention layer </p>\n": "<p>\u7b2c\u4e00\u4e2a\u56fe\u5f62\u5173\u6ce8\u56fe\u5c42\u4e2d\u7684\u8981\u7d20\u6570</p>\n",
|
||||
"<p>Number of features in the input </p>\n": "<p>\u8f93\u5165\u4e2d\u7684\u8981\u7d20\u6570\u91cf</p>\n",
|
||||
"<p>Number of features per node in the input </p>\n": "<p>\u8f93\u5165\u4e2d\u6bcf\u4e2a\u8282\u70b9\u7684\u8981\u7d20\u6570</p>\n",
|
||||
"<p>Number of heads </p>\n": "<p>\u5934\u6570</p>\n",
|
||||
"<p>Number of nodes to train on </p>\n": "<p>\u8981\u8bad\u7ec3\u7684\u8282\u70b9\u6570</p>\n",
|
||||
"<p>Number of training iterations </p>\n": "<p>\u8bad\u7ec3\u8fed\u4ee3\u6b21\u6570</p>\n",
|
||||
"<p>Optimizer </p>\n": "<p>\u4f18\u5316\u5668</p>\n",
|
||||
"<p>Output layer (without activation) for logits </p>\n": "<p>logits \u7684\u8f93\u51fa\u5c42\uff08\u672a\u6fc0\u6d3b\uff09</p>\n",
|
||||
"<p>Random indexes </p>\n": "<p>\u968f\u673a\u7d22\u5f15</p>\n",
|
||||
"<p>Read the paper ids, feature vectors, and labels </p>\n": "<p>\u9605\u8bfb\u7eb8\u5f20 ID\u3001\u7279\u5f81\u77e2\u91cf\u548c\u6807\u7b7e</p>\n",
|
||||
"<p>Run the training </p>\n": "<p>\u8fd0\u884c\u8bad\u7ec3</p>\n",
|
||||
"<p>Save logs </p>\n": "<p>\u4fdd\u5b58\u65e5\u5fd7</p>\n",
|
||||
"<p>Set mode to evaluation mode for validation </p>\n": "<p>\u5c06\u6a21\u5f0f\u8bbe\u7f6e\u4e3a\u8bc4\u4f30\u6a21\u5f0f\u4ee5\u8fdb\u884c\u9a8c\u8bc1</p>\n",
|
||||
"<p>Set of class names and an unique integer index </p>\n": "<p>\u4e00\u7ec4\u7c7b\u540d\u548c\u4e00\u4e2a\u552f\u4e00\u7684\u6574\u6570\u7d22\u5f15</p>\n",
|
||||
"<p>Set the model to training mode </p>\n": "<p>\u5c06\u6a21\u578b\u8bbe\u7f6e\u4e3a\u8bad\u7ec3\u6a21\u5f0f</p>\n",
|
||||
"<p>Start and watch the experiment </p>\n": "<p>\u5f00\u59cb\u89c2\u770b\u5b9e\u9a8c</p>\n",
|
||||
"<p>Take optimization step </p>\n": "<p>\u91c7\u53d6\u4f18\u5316\u6b65\u9aa4</p>\n",
|
||||
"<p>The pair of paper indexes </p>\n": "<p>\u4e00\u5bf9\u7eb8\u8d28\u7d22\u5f15</p>\n",
|
||||
"<p>Training loop </p>\n": "<p>\u8bad\u7ec3\u5faa\u73af</p>\n",
|
||||
"<p>We build a symmetrical graph, where if paper <span translate=no>_^_0_^_</span> referenced paper <span translate=no>_^_1_^_</span> we place an adge from <span translate=no>_^_2_^_</span> to <span translate=no>_^_3_^_</span> as well as an edge from <span translate=no>_^_4_^_</span> to <span translate=no>_^_5_^_</span>. </p>\n": "<p>\u6211\u4eec\u6784\u5efa\u4e00\u4e2a\u5bf9\u79f0\u7684\u56fe\u5f62\uff0c\u5982\u679c\u7eb8\u5f20<span translate=no>_^_0_^_</span>\u5f15\u7528\u4e86\u7eb8\u5f20\uff0c<span translate=no>_^_1_^_</span>\u6211\u4eec\u4f1a\u5728\u5176\u4e2d\u653e\u7f6e\u4e00\u4e2a\u4ece<span translate=no>_^_2_^_</span>\u5230\u7684\u5fbd\u7ae0<span translate=no>_^_3_^_</span>\u4ee5\u53ca\u4ece<span translate=no>_^_4_^_</span>\u5230<span translate=no>_^_5_^_</span>\u3002</p>\n",
|
||||
"<p>Whether to include edges. This is test how much accuracy is lost if we ignore the citation network. </p>\n": "<p>\u662f\u5426\u5305\u62ec\u8fb9\u7f18\u3002\u8fd9\u662f\u6d4b\u8bd5\u5982\u679c\u6211\u4eec\u5ffd\u7565\u5f15\u6587\u7f51\u7edc\u4f1a\u635f\u5931\u591a\u5c11\u51c6\u786e\u6027\u3002</p>\n",
|
||||
"<p>Whether to include the citation network </p>\n": "<p>\u662f\u5426\u5305\u62ec\u5f15\u6587\u7f51\u7edc</p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the features vectors of shape <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> is the adjacency matrix of the form <span translate=no>_^_3_^_</span> or <span translate=no>_^_4_^_</span></li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u662f\u5f62\u72b6\u7684\u7279\u5f81\u5411\u91cf<span translate=no>_^_1_^_</span></li>\n<li><span translate=no>_^_2_^_</span>\u662f\u5f62\u5f0f\u7684\u90bb\u63a5\u77e9\u9635<span translate=no>_^_3_^_</span>\u6216<span translate=no>_^_4_^_</span></li></ul>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the number of features per node </li>\n<li><span translate=no>_^_1_^_</span> is the number of features in the first graph attention layer </li>\n<li><span translate=no>_^_2_^_</span> is the number of classes </li>\n<li><span translate=no>_^_3_^_</span> is the number of heads in the graph attention layers </li>\n<li><span translate=no>_^_4_^_</span> is the dropout probability</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u662f\u6bcf\u4e2a\u8282\u70b9\u7684\u8981\u7d20\u6570</li>\n<li><span translate=no>_^_1_^_</span>\u662f\u7b2c\u4e00\u4e2a\u56fe\u5f62\u5173\u6ce8\u5c42\u4e2d\u7684\u8981\u7d20\u6570</li>\n<li><span translate=no>_^_2_^_</span>\u662f\u7c7b\u7684\u6570\u91cf</li>\n<li><span translate=no>_^_3_^_</span>\u662f\u56fe\u8868\u5173\u6ce8\u5c42\u4e2d\u7684\u5934\u90e8\u6570\u91cf</li>\n<li><span translate=no>_^_4_^_</span>\u662f\u8f8d\u5b66\u6982\u7387</li></ul>\n",
|
||||
"This trains is a Graph Attention Network (GAT) on Cora dataset": "\u8fd9\u5217\u706b\u8f66\u662f Cora \u6570\u636e\u96c6\u4e0a\u7684\u56fe\u5f62\u6ce8\u610f\u529b\u7f51\u7edc (GAT)",
|
||||
"Train a Graph Attention Network (GAT) on Cora dataset": "\u5728 Cora \u6570\u636e\u96c6\u4e0a\u8bad\u7ec3\u56fe\u5f62\u6ce8\u610f\u529b\u7f51\u7edc (GAT)"
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"<h1><a href=\"https://nn.labml.ai/graphs/gat/index.html\">Graph Attention Networks (GAT)</a></h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of the paper <a href=\"https://arxiv.org/abs/1710.10903\">Graph Attention Networks</a>.</p>\n<p>GATs work on graph data. A graph consists of nodes and edges connecting nodes. For example, in Cora dataset the nodes are research papers and the edges are citations that connect the papers.</p>\n<p>GAT uses masked self-attention, kind of similar to <a href=\"https://nn.labml.ai/transformers/mha.html\">transformers</a>. GAT consists of graph attention layers stacked on top of each other. Each graph attention layer gets node embeddings as inputs and outputs transformed embeddings. The node embeddings pay attention to the embeddings of other nodes it's connected to. The details of graph attention layers are included alongside the implementation.</p>\n<p>Here is <a href=\"https://nn.labml.ai/graphs/gat/experiment.html\">the training code</a> for training a two-layer GAT on Cora dataset. </p>\n": "<h1><a href=\"https://nn.labml.ai/graphs/gat/index.html\">\u30b0\u30e9\u30d5\u30fb\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30fb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af (GAT)</a></h1>\n<p>\u3053\u308c\u306f\u8ad6\u6587\u306e\u300c<a href=\"https://arxiv.org/abs/1710.10903\">\u30b0\u30e9\u30d5\u30fb\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30fb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af</a>\u300d\u306e <a href=\"https://pytorch.org\">PyTorch</a> \u5b9f\u88c5\u3067\u3059\u3002</p>\n<p>GAT \u306f\u30b0\u30e9\u30d5\u30c7\u30fc\u30bf\u3092\u51e6\u7406\u3057\u307e\u3059\u3002\u30b0\u30e9\u30d5\u306f\u3001\u30ce\u30fc\u30c9\u3068\u30ce\u30fc\u30c9\u3092\u63a5\u7d9a\u3059\u308b\u30a8\u30c3\u30b8\u3067\u69cb\u6210\u3055\u308c\u307e\u3059\u3002\u305f\u3068\u3048\u3070\u3001Cora\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3067\u306f\u3001\u30ce\u30fc\u30c9\u306f\u7814\u7a76\u8ad6\u6587\u3067\u3001\u7aef\u306f\u8ad6\u6587\u3092\u3064\u306a\u3050\u5f15\u7528\u3067\u3059</p>\u3002\n<p><a href=\"https://nn.labml.ai/transformers/mha.html\">GAT\u306f\u3001\u30c8\u30e9\u30f3\u30b9\u30d5\u30a9\u30fc\u30de\u30fc\u306b\u4f3c\u305f\u3001\u30de\u30b9\u30af\u3055\u308c\u305f\u30bb\u30eb\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u3092\u4f7f\u3044\u307e\u3059\u3002</a>GAT\u306f\u3001\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30ec\u30a4\u30e4\u30fc\u304c\u4e92\u3044\u306b\u91cd\u306a\u308a\u5408\u3063\u3066\u69cb\u6210\u3055\u308c\u3066\u3044\u307e\u3059\u3002\u5404\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30ec\u30a4\u30e4\u30fc\u306f\u3001\u5165\u529b\u3068\u3057\u3066\u30ce\u30fc\u30c9\u57cb\u3081\u8fbc\u307f\u3092\u53d6\u5f97\u3057\u3001\u5909\u63db\u3055\u308c\u305f\u57cb\u3081\u8fbc\u307f\u3092\u51fa\u529b\u3057\u307e\u3059\u3002\u30ce\u30fc\u30c9\u57cb\u3081\u8fbc\u307f\u306f\u3001\u63a5\u7d9a\u3055\u308c\u3066\u3044\u308b\u4ed6\u306e\u30ce\u30fc\u30c9\u306e\u57cb\u3081\u8fbc\u307f\u306b\u6ce8\u76ee\u3057\u307e\u3059\u3002\u30b0\u30e9\u30d5\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30ec\u30a4\u30e4\u30fc\u306e\u8a73\u7d30\u306f\u3001\u5b9f\u88c5\u3068\u3068\u3082\u306b\u542b\u307e\u308c\u3066\u3044\u307e\u3059\u3002</p>\n<p><a href=\"https://nn.labml.ai/graphs/gat/experiment.html\">Cora \u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u3067 2 \u5c64 GAT \u3092\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3059\u308b\u305f\u3081\u306e\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30b3\u30fc\u30c9\u3092\u6b21\u306b\u793a\u3057\u307e\u3059</a>\u3002</p>\n",
|
||||
"Graph Attention Networks (GAT)": "\u30b0\u30e9\u30d5\u30fb\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30fb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af (GAT)"
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"<h1><a href=\"https://nn.labml.ai/graphs/gat/index.html\">Graph Attention Networks (GAT)</a></h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of the paper <a href=\"https://arxiv.org/abs/1710.10903\">Graph Attention Networks</a>.</p>\n<p>GATs work on graph data. A graph consists of nodes and edges connecting nodes. For example, in Cora dataset the nodes are research papers and the edges are citations that connect the papers.</p>\n<p>GAT uses masked self-attention, kind of similar to <a href=\"https://nn.labml.ai/transformers/mha.html\">transformers</a>. GAT consists of graph attention layers stacked on top of each other. Each graph attention layer gets node embeddings as inputs and outputs transformed embeddings. The node embeddings pay attention to the embeddings of other nodes it's connected to. The details of graph attention layers are included alongside the implementation.</p>\n<p>Here is <a href=\"https://nn.labml.ai/graphs/gat/experiment.html\">the training code</a> for training a two-layer GAT on Cora dataset. </p>\n": "<h1><a href=\"https://nn.labml.ai/graphs/gat/index.html\">\u56fe\u8868\u6ce8\u610f\u529b\u7f51\u7edc (GAT)</a></h1>\n<p>\u8fd9\u662f <a href=\"https://pytorch.org\">PyTorch</a> \u5bf9\u300a<a href=\"https://arxiv.org/abs/1710.10903\">\u56fe\u5f62\u6ce8\u610f\u529b\u7f51\u7edc</a>\u300b\u8bba\u6587\u7684\u5b9e\u73b0\u3002</p>\n<p>GAT \u5904\u7406\u56fe\u8868\u6570\u636e\u3002\u56fe\u7531\u8282\u70b9\u548c\u8fde\u63a5\u8282\u70b9\u7684\u8fb9\u7ec4\u6210\u3002\u4f8b\u5982\uff0c\u5728 Cora \u6570\u636e\u96c6\u4e2d\uff0c\u8282\u70b9\u662f\u7814\u7a76\u8bba\u6587\uff0c\u8fb9\u7f18\u662f\u8fde\u63a5\u8bba\u6587\u7684\u5f15\u6587\u3002</p>\n<p>GAT \u4f7f\u7528\u8499\u9762\u81ea\u6ce8\u610f\u529b\uff0c\u6709\u70b9\u7c7b\u4f3c\u4e8e<a href=\"https://nn.labml.ai/transformers/mha.html\">\u53d8\u5f62\u91d1\u521a</a>\u3002GAT \u7531\u76f8\u4e92\u5806\u53e0\u7684\u56fe\u8868\u6ce8\u610f\u529b\u5c42\u7ec4\u6210\u3002\u6bcf\u4e2a\u56fe\u6ce8\u610f\u529b\u5c42\u90fd\u5c06\u8282\u70b9\u5d4c\u5165\u4f5c\u4e3a\u8f6c\u6362\u540e\u7684\u5d4c\u5165\u7684\u8f93\u5165\u548c\u8f93\u51fa\u83b7\u5f97\u8282\u70b9\u3002\u8282\u70b9\u5d4c\u5165\u4f1a\u6ce8\u610f\u5b83\u6240\u8fde\u63a5\u7684\u5176\u4ed6\u8282\u70b9\u7684\u5d4c\u5165\u3002\u56fe\u5f62\u6ce8\u610f\u529b\u5c42\u7684\u8be6\u7ec6\u4fe1\u606f\u4e0e\u5b9e\u73b0\u4e00\u8d77\u5305\u62ec\u5728\u5185\u3002</p>\n<p>\u4ee5\u4e0b\u662f<a href=\"https://nn.labml.ai/graphs/gat/experiment.html\">\u5728 Cora \u6570\u636e\u96c6\u4e0a\u8bad\u7ec3\u4e24\u5c42 GAT \u7684\u8bad\u7ec3\u4ee3\u7801</a>\u3002</p>\n",
|
||||
"Graph Attention Networks (GAT)": "\u56fe\u5173\u6ce8\u7f51\u7edc (GAT)"
|
||||
}
|
||||
Reference in New Issue
Block a user