chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"<h1>Deep Q Networks (DQN)</h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of paper <a href=\"https://arxiv.org/abs/1312.5602\">Playing Atari with Deep Reinforcement Learning</a> along with <a href=\"model.html\">Dueling Network</a>, <a href=\"replay_buffer.html\">Prioritized Replay</a> and Double Q Network.</p>\n<p>Here is the <a href=\"experiment.html\">experiment</a> and <a href=\"model.html\">model</a> implementation.</p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a></p>\n": "<h1>\u30c7\u30a3\u30fc\u30d7Q\u30cd\u30c3\u30c8\u30ef\u30fc\u30af (DQN)</h1>\n<p>\u3053\u308c\u306f\u3001<a href=\"https://arxiv.org/abs/1312.5602\">\u30c7\u30a3\u30fc\u30d7\u5f37\u5316\u5b66\u7fd2\u3092\u4f7f\u3063\u305f\u30a2\u30bf\u30ea\u30d7\u30ec\u30a4\u3068\u30c7\u30e5\u30a8\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af</a><a href=\"model.html\">\u3001<a href=\"replay_buffer.html\">\u512a\u5148\u30ea\u30d7\u30ec\u30a4</a>\u3001<a href=\"https://pytorch.org\">\u30c0\u30d6\u30ebQ\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3092PyTorch\u3067\u5b9f\u88c5\u3057\u305f\u3082\u306e\u3067\u3059</a></a>\u3002</p>\n<p><a href=\"experiment.html\"><a href=\"model.html\">\u3053\u308c\u304c\u5b9f\u9a13\u3068\u30e2\u30c7\u30eb\u306e\u5b9f\u88c5\u3067\u3059</a></a>\u3002</p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a></p>\n",
|
||||
"<h2>Train the model</h2>\n<p>We want to find optimal action-value function.</p>\n<span translate=no>_^_0_^_</span><h3>Target network \ud83c\udfaf</h3>\n<p>In order to improve stability we use experience replay that randomly sample from previous experience <span translate=no>_^_1_^_</span>. We also use a Q network with a separate set of parameters <span translate=no>_^_2_^_</span> to calculate the target. <span translate=no>_^_3_^_</span> is updated periodically. This is according to paper <a href=\"https://deepmind.com/research/dqn/\">Human Level Control Through Deep Reinforcement Learning</a>.</p>\n<p>So the loss function is, <span translate=no>_^_4_^_</span></p>\n<h3>Double <span translate=no>_^_5_^_</span>-Learning</h3>\n<p>The max operator in the above calculation uses same network for both selecting the best action and for evaluating the value. That is, <span translate=no>_^_6_^_</span> We use <a href=\"https://arxiv.org/abs/1509.06461\">double Q-learning</a>, where the <span translate=no>_^_7_^_</span> is taken from <span translate=no>_^_8_^_</span> and the value is taken from <span translate=no>_^_9_^_</span>.</p>\n<p>And the loss function becomes,</p>\n<span translate=no>_^_10_^_</span>": "<h2>\u30e2\u30c7\u30eb\u306e\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0</h2>\n<p>\u6700\u9069\u306a\u30a2\u30af\u30b7\u30e7\u30f3\u30d0\u30ea\u30e5\u30fc\u95a2\u6570\u3092\u898b\u3064\u3051\u305f\u3044\u3002</p>\n<span translate=no>_^_0_^_</span><h3>\u30bf\u30fc\u30b2\u30c3\u30c8\u30cd\u30c3\u30c8\u30ef\u30fc\u30af \ud83c\udfaf</h3>\n<p>\u5b89\u5b9a\u6027\u3092\u5411\u4e0a\u3055\u305b\u308b\u305f\u3081\u306b\u3001\u4ee5\u524d\u306e\u30a8\u30af\u30b9\u30da\u30ea\u30a8\u30f3\u30b9\u304b\u3089\u30e9\u30f3\u30c0\u30e0\u306b\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3055\u308c\u308b\u30a8\u30af\u30b9\u30da\u30ea\u30a8\u30f3\u30b9\u306e\u30ea\u30d7\u30ec\u30a4\u3092\u4f7f\u7528\u3057\u3066\u3044\u307e\u3059\u3002<span translate=no>_^_1_^_</span>\u307e\u305f\u3001<span translate=no>_^_2_^_</span>\u5225\u306e\u30d1\u30e9\u30e1\u30fc\u30bf\u30bb\u30c3\u30c8\u3092\u6301\u3064Q\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3092\u4f7f\u7528\u3057\u3066\u30bf\u30fc\u30b2\u30c3\u30c8\u3092\u8a08\u7b97\u3057\u307e\u3059\u3002<span translate=no>_^_3_^_</span>\u5b9a\u671f\u7684\u306b\u66f4\u65b0\u3055\u308c\u307e\u3059\u3002\u3053\u308c\u306f\u3001<a href=\"https://deepmind.com/research/dqn/\">\u6df1\u5c64\u5f37\u5316\u5b66\u7fd2\u306b\u3088\u308b\u30d2\u30e5\u30fc\u30de\u30f3\u30ec\u30d9\u30eb\u5236\u5fa1\u306e\u8ad6\u6587\u306b\u3088\u308b\u3082\u306e\u3067\u3059</a></p>\u3002\n<p>\u3057\u305f\u304c\u3063\u3066\u3001\u640d\u5931\u95a2\u6570\u306f\u3001<span translate=no>_^_4_^_</span></p>\n<h3><span translate=no>_^_5_^_</span>\u30c0\u30d6\u30eb\u30e9\u30fc\u30cb\u30f3\u30b0</h3>\n<p>\u4e0a\u306e\u8a08\u7b97\u306e max \u6f14\u7b97\u5b50\u306f\u3001\u6700\u9069\u306a\u30a2\u30af\u30b7\u30e7\u30f3\u306e\u9078\u629e\u3068\u5024\u306e\u8a55\u4fa1\u306e\u4e21\u65b9\u306b\u540c\u3058\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3092\u4f7f\u7528\u3057\u307e\u3059\u3002\u3064\u307e\u308a<span translate=no>_^_6_^_</span>\u3001<a href=\"https://arxiv.org/abs/1509.06461\"><span translate=no>_^_7_^_</span><span translate=no>_^_8_^_</span>\u306e\u53d6\u5f97\u5143\u3068\u5024\u306e\u53d6\u5f97\u5143\u3068\u3044\u3046\u4e8c\u91cdQ\u30e9\u30fc\u30cb\u30f3\u30b0\u3092\u4f7f\u7528\u3057\u3066\u3044\u307e\u3059</a></p>\u3002<span translate=no>_^_9_^_</span>\n<p>\u305d\u3057\u3066\u3001\u640d\u5931\u95a2\u6570\u306f\u6b21\u306e\u3088\u3046\u306b\u306a\u308a\u307e\u3059\u3002</p>\n<span translate=no>_^_10_^_</span>",
|
||||
"<p><span translate=no>_^_0_^_</span> </p>\n": "<p><span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Calculate the desired Q value. We multiply by <span translate=no>_^_0_^_</span> to zero out the next state Q values if the game ended.</p>\n<p><span translate=no>_^_1_^_</span> </p>\n": "<p>\u76ee\u7684\u306e Q \u5024\u3092\u8a08\u7b97\u3057\u307e\u3059\u3002\u30b2\u30fc\u30e0\u304c\u7d42\u4e86\u3057\u305f\u3089<span translate=no>_^_0_^_</span>\u3001\u3092\u639b\u3051\u3066\u6b21\u306e\u30b9\u30c6\u30fc\u30c8\u306eQ\u5024\u3092\u30bc\u30ed\u306b\u3057\u307e\u3059</p>\u3002\n<p><span translate=no>_^_1_^_</span></p>\n",
|
||||
"<p>Get the best action at state <span translate=no>_^_0_^_</span> <span translate=no>_^_1_^_</span> </p>\n": "<p>\u5dde\u3067\u6700\u9ad8\u306e\u30a2\u30af\u30b7\u30e7\u30f3\u3092 <span translate=no>_^_0_^_</span> <span translate=no>_^_1_^_</span></p>\n",
|
||||
"<p>Get the q value from the target network for the best action at state <span translate=no>_^_0_^_</span> <span translate=no>_^_1_^_</span> </p>\n": "<p>\u30bf\u30fc\u30b2\u30c3\u30c8\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u304b\u3089 q \u5024\u3092\u53d6\u5f97\u3057\u3066\u3001\u72b6\u614b\u3067\u306e\u6700\u9069\u306a\u30a2\u30af\u30b7\u30e7\u30f3\u3092\u5b9f\u73fe\u3059\u308b <span translate=no>_^_0_^_</span> <span translate=no>_^_1_^_</span></p>\n",
|
||||
"<p>Get weighted means </p>\n": "<p>\u52a0\u91cd\u5e73\u5747\u3092\u53d6\u5f97</p>\n",
|
||||
"<p>Gradients shouldn't propagate gradients <span translate=no>_^_0_^_</span> </p>\n": "<p>\u30b0\u30e9\u30c7\u30fc\u30b7\u30e7\u30f3\u306f\u30b0\u30e9\u30c7\u30fc\u30b7\u30e7\u30f3\u3092\u4f1d\u64ad\u3057\u3066\u306f\u3044\u3051\u307e\u305b\u3093 <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Temporal difference error <span translate=no>_^_0_^_</span> is used to weigh samples in replay buffer </p>\n": "<p><span translate=no>_^_0_^_</span>\u6642\u9593\u5dee\u30a8\u30e9\u30fc\u306f\u30ea\u30d7\u30ec\u30a4\u30d0\u30c3\u30d5\u30a1\u5185\u306e\u30b5\u30f3\u30d7\u30eb\u306e\u91cd\u307f\u4ed8\u3051\u306b\u4f7f\u7528\u3055\u308c\u307e\u3059</p>\n",
|
||||
"<p>We take <a href=\"https://en.wikipedia.org/wiki/Huber_loss\">Huber loss</a> instead of mean squared error loss because it is less sensitive to outliers </p>\n": "<p>\u5916\u308c\u5024\u306e\u5f71\u97ff\u3092\u53d7\u3051\u306b\u304f\u3044\u306e\u3067\u3001<a href=\"https://en.wikipedia.org/wiki/Huber_loss\">\u5e73\u5747\u4e8c\u4e57\u8aa4\u5dee\u640d\u5931\u306e\u4ee3\u308f\u308a\u306b\u30d5\u30fc\u30d0\u30fc\u640d\u5931\u3092\u4f7f\u7528\u3057\u307e\u3059</a>\u3002</p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> - <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> - <span translate=no>_^_3_^_</span> </li>\n<li><span translate=no>_^_4_^_</span> - <span translate=no>_^_5_^_</span> </li>\n<li><span translate=no>_^_6_^_</span> - <span translate=no>_^_7_^_</span> </li>\n<li><span translate=no>_^_8_^_</span> - whether the game ended after taking the action </li>\n<li><span translate=no>_^_9_^_</span> - <span translate=no>_^_10_^_</span> </li>\n<li><span translate=no>_^_11_^_</span> - weights of the samples from prioritized experienced replay</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>-<span translate=no>_^_1_^_</span></li>\n<li><span translate=no>_^_2_^_</span>-<span translate=no>_^_3_^_</span></li>\n<li><span translate=no>_^_4_^_</span>-<span translate=no>_^_5_^_</span></li>\n<li><span translate=no>_^_6_^_</span>-<span translate=no>_^_7_^_</span></li>\n<li><span translate=no>_^_8_^_</span>-\u30a2\u30af\u30b7\u30e7\u30f3\u3092\u5b9f\u884c\u3057\u305f\u5f8c\u306b\u30b2\u30fc\u30e0\u304c\u7d42\u4e86\u3057\u305f\u304b\u3069\u3046\u304b</li>\n<li><span translate=no>_^_9_^_</span>-<span translate=no>_^_10_^_</span></li>\n<li><span translate=no>_^_11_^_</span>-\u7d4c\u9a13\u8c4a\u304b\u306a\u30ea\u30d7\u30ec\u30a4\u3092\u512a\u5148\u3057\u3066\u62bd\u51fa\u3057\u305f\u30b5\u30f3\u30d7\u30eb\u306e\u91cd\u307f</li></ul>\n",
|
||||
"Deep Q Networks (DQN)": "\u30c7\u30a3\u30fc\u30d7Q\u30cd\u30c3\u30c8\u30ef\u30fc\u30af (DQN)",
|
||||
"This is a PyTorch implementation/tutorial of Deep Q Networks (DQN) from paper Playing Atari with Deep Reinforcement Learning. This includes dueling network architecture, a prioritized replay buffer and double-Q-network training.": "\u3053\u308c\u306f\u30c7\u30a3\u30fc\u30d7Q\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\uff08DQN\uff09\u306ePyTorch\u5b9f\u88c5/\u30c1\u30e5\u30fc\u30c8\u30ea\u30a2\u30eb\u3067\u3001\u300c\u30c7\u30a3\u30fc\u30d7\u5f37\u5316\u5b66\u7fd2\u3067\u30a2\u30bf\u30ea\u3092\u30d7\u30ec\u30a4\u300d\u3068\u3044\u3046\u8ad6\u6587\u304b\u3089\u5f15\u7528\u3057\u3066\u3044\u307e\u3059\u3002\u3053\u308c\u306b\u306f\u3001\u30c7\u30e5\u30a8\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u30a2\u30fc\u30ad\u30c6\u30af\u30c1\u30e3\u3001\u512a\u5148\u9806\u4f4d\u4ed8\u3051\u3055\u308c\u305f\u30ea\u30d7\u30ec\u30a4\u30d0\u30c3\u30d5\u30a1\u3001\u30c0\u30d6\u30ebQ\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u304c\u542b\u307e\u308c\u307e\u3059\u3002"
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"<h1>Deep Q Networks (DQN)</h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of paper <a href=\"https://arxiv.org/abs/1312.5602\">Playing Atari with Deep Reinforcement Learning</a> along with <a href=\"model.html\">Dueling Network</a>, <a href=\"replay_buffer.html\">Prioritized Replay</a> and Double Q Network.</p>\n<p>Here is the <a href=\"experiment.html\">experiment</a> and <a href=\"model.html\">model</a> implementation.</p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a> <a href=\"https://app.labml.ai/run/fe1ad986237511ec86e8b763a2d3f710\"><span translate=no>_^_1_^_</span></a></p>\n": "<h1>\u0d9c\u0dd0\u0db9\u0dd4\u0dbb\u0dd4Q \u0da2\u0dcf\u0dbd (DQN)</h1>\n<p>\u0db8\u0dd9\u0dba <a href=\"https://pytorch.org\">PyTorch</a> \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0d9a\u0dd2 <a href=\"https://arxiv.org/abs/1312.5602\">Atari \u0d9a\u0da9\u0daf\u0dcf\u0dc3\u0dd2 \u0dc3\u0dd9\u0dbd\u0dca\u0dbd\u0db8\u0dca \u0d9a\u0dd2\u0dbb\u0dd3\u0db8 \u0d9c\u0dd0\u0db9\u0dd4\u0dbb\u0dd4 \u0dc1\u0d9a\u0dca\u0dad\u0dd2\u0db8\u0dad\u0dca \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0d89\u0d9c\u0dd9\u0db1\u0dd3\u0db8</a> \u0dc3\u0dc4 <a href=\"model.html\">\u0da9\u0dd4\u0dc0\u0dbd\u0dd2\u0d82 \u0da2\u0dcf\u0dbd\u0dba</a> , <a href=\"replay_buffer.html\">\u0db4\u0dca\u0dbb\u0db8\u0dd4\u0d9b\u0dad\u0dcf \u0db1\u0dd0\u0dc0\u0dad \u0db0\u0dcf\u0dc0\u0db1\u0dba</a> \u0dc3\u0dc4 \u0daf\u0dca\u0dc0\u0dd2\u0dad\u0dca\u0dc0 Q \u0da2\u0dcf\u0dbd\u0dba \u0dc3\u0db8\u0d9f. </p>\n<p>\u0db8\u0dd9\u0db1\u0dca\u0db1 <a href=\"experiment.html\">\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf</a> \u0db6\u0dd0\u0dbd\u0dd3\u0db8 \u0dc3\u0dc4 <a href=\"model.html\">\u0d86\u0daf\u0dbb\u0dca\u0dc1</a> \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dd2\u0dbb\u0dd3\u0db8. </p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a> <a href=\"https://app.labml.ai/run/fe1ad986237511ec86e8b763a2d3f710\"> <span translate=no>_^_1_^_</span></a></p>\n",
|
||||
"<h2>Train the model</h2>\n<p>We want to find optimal action-value function.</p>\n<span translate=no>_^_0_^_</span><h3>Target network \ud83c\udfaf</h3>\n<p>In order to improve stability we use experience replay that randomly sample from previous experience <span translate=no>_^_1_^_</span>. We also use a Q network with a separate set of parameters <span translate=no>_^_2_^_</span> to calculate the target. <span translate=no>_^_3_^_</span> is updated periodically. This is according to paper <a href=\"https://deepmind.com/research/dqn/\">Human Level Control Through Deep Reinforcement Learning</a>.</p>\n<p>So the loss function is, <span translate=no>_^_4_^_</span></p>\n<h3>Double <span translate=no>_^_5_^_</span>-Learning</h3>\n<p>The max operator in the above calculation uses same network for both selecting the best action and for evaluating the value. That is, <span translate=no>_^_6_^_</span> We use <a href=\"https://arxiv.org/abs/1509.06461\">double Q-learning</a>, where the <span translate=no>_^_7_^_</span> is taken from <span translate=no>_^_8_^_</span> and the value is taken from <span translate=no>_^_9_^_</span>.</p>\n<p>And the loss function becomes,</p>\n<span translate=no>_^_10_^_</span>": "<h2>\u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0d9a\u0dbb\u0db1\u0dca\u0db1</h2>\n<p>\u0db4\u0dca\u0dbb\u0dc1\u0dc3\u0dca\u0dad \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0d9a\u0dcf\u0dbb\u0dd3 \u0d85\u0d9c\u0dba \u0dc1\u0dca\u0dbb\u0dd2\u0dad\u0dba \u0dc3\u0ddc\u0dba\u0dcf \u0d9c\u0dd0\u0db1\u0dd3\u0db8\u0da7 \u0d85\u0db4\u0da7 \u0d85\u0dc0\u0dc1\u0dca\u0dba\u0dba.</p>\n<span translate=no>_^_0_^_</span><h3>\u0d89\u0dbd\u0d9a\u0dca\u0d9a \u0da2\u0dcf\u0dbd\u0dba \ud83c\udfaf</h3>\n<p>\u0dc3\u0dca\u0dae\u0dcf\u0dc0\u0dbb\u0dad\u0dca\u0dc0\u0dba \u0dc0\u0dd0\u0da9\u0dd2 \u0daf\u0dd2\u0dba\u0dd4\u0dab\u0dd4 \u0d9a\u0dd2\u0dbb\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0d85\u0db4\u0dd2 \u0db4\u0dd9\u0dbb \u0d85\u0dad\u0dca\u0daf\u0dd0\u0d9a\u0dd3\u0db8\u0dca \u0dc0\u0dbd\u0dd2\u0db1\u0dca \u0d85\u0dc4\u0db9\u0dd4 \u0dbd\u0dd9\u0dc3 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba \u0d85\u0dad\u0dca\u0daf\u0dd0\u0d9a\u0dd3\u0db8\u0dca \u0db1\u0dd0\u0dc0\u0dad \u0db0\u0dcf\u0dc0\u0db1\u0dba \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dbb\u0db8\u0dd4<span translate=no>_^_1_^_</span>. \u0d89\u0dbd\u0d9a\u0dca\u0d9a\u0dba \u0d9c\u0dab\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0dc0\u0dd9\u0db1\u0db8 \u0db4\u0dbb\u0dcf\u0db8\u0dd2\u0dad\u0dd3\u0db1\u0dca<span translate=no>_^_2_^_</span> \u0dc3\u0db8\u0dd6\u0dc4\u0dba\u0d9a\u0dca \u0dc3\u0dc4\u0dd2\u0dad Q \u0da2\u0dcf\u0dbd\u0dba\u0d9a\u0dca \u0daf \u0d85\u0db4\u0dd2 \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dbb\u0db8\u0dd4. <span translate=no>_^_3_^_</span>\u0dc0\u0dbb\u0dd2\u0db1\u0dca \u0dc0\u0dbb \u0dba\u0dcf\u0dc0\u0dad\u0dca\u0d9a\u0dcf\u0dbd\u0dd3\u0db1 \u0dc0\u0dda. \u0db8\u0dd9\u0dba \u0d9c\u0dd0\u0db9\u0dd4\u0dbb\u0dd4 \u0dc1\u0d9a\u0dca\u0dad\u0dd2\u0db8\u0dad\u0dca \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0d89\u0d9c\u0dd9\u0db1\u0dd3\u0db8 \u0dad\u0dd4\u0dc5\u0dd2\u0db1\u0dca \u0d9a\u0da9\u0daf\u0dcf\u0dc3\u0dd2 <a href=\"https://deepmind.com/research/dqn/\">\u0db8\u0dcf\u0db1\u0dc0 \u0db8\u0da7\u0dca\u0da7\u0db8\u0dca \u0db4\u0dcf\u0dbd\u0db1\u0dba\u0da7 \u0d85\u0db1\u0dd4\u0dc0</a> \u0dba.</p>\n<p>\u0d91\u0db6\u0dd0\u0dc0\u0dd2\u0db1\u0dca \u0db4\u0dcf\u0da9\u0dd4 \u0dc1\u0dca\u0dbb\u0dd2\u0dad\u0dba \u0dc0\u0db1\u0dca\u0db1\u0dda,<span translate=no>_^_4_^_</span></p>\n<h3><span translate=no>_^_5_^_</span>\u0daf\u0dca\u0dc0\u0dd2\u0dad\u0dca\u0dc0-\u0d89\u0d9c\u0dd9\u0db1\u0dd4\u0db8\u0dca</h3>\n<p>\u0d89\u0dc4\u0dad \u0d9c\u0dab\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0d8b\u0db4\u0dbb\u0dd2\u0db8 \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0d9a\u0dbb\u0dd4 \u0dc4\u0ddc\u0db3\u0db8 \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dc0 \u0dad\u0ddd\u0dbb\u0dcf \u0d9c\u0dd0\u0db1\u0dd3\u0db8 \u0dc3\u0dc4 \u0dc0\u0da7\u0dd2\u0db1\u0dcf\u0d9a\u0db8 \u0d87\u0d9c\u0dba\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0d91\u0d9a\u0db8 \u0da2\u0dcf\u0dbd\u0dba\u0d9a\u0dca \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dbb\u0dba\u0dd2. \u0d91\u0db1\u0db8\u0dca,<span translate=no>_^_6_^_</span> \u0d85\u0db4\u0dd2 <a href=\"https://arxiv.org/abs/1509.06461\">\u0daf\u0dca\u0dc0\u0dd2\u0dad\u0dca\u0dc0 Q- \u0d89\u0d9c\u0dd9\u0db1\u0dd3\u0db8</a> \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dbb\u0db8\u0dd4,<span translate=no>_^_7_^_</span> \u0d91\u0dba \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1\u0dda \u0d9a\u0ddc\u0dad\u0dd0\u0db1\u0dd2\u0db1\u0dca\u0daf<span translate=no>_^_8_^_</span> \u0dc3\u0dc4 \u0dc0\u0da7\u0dd2\u0db1\u0dcf\u0d9a\u0db8 \u0dbd\u0db6\u0dcf<span translate=no>_^_9_^_</span> \u0d9c\u0db1\u0dd3.</p>\n<p>\u0db4\u0dcf\u0da9\u0dd4 \u0dc1\u0dca\u0dbb\u0dd2\u0dad\u0dba \u0db6\u0dc0\u0da7 \u0db4\u0dad\u0dca\u0dc0\u0dda,</p>\n<span translate=no>_^_10_^_</span>",
|
||||
"<p><span translate=no>_^_0_^_</span> </p>\n": "<p><span translate=no>_^_0_^_</span> </p>\n",
|
||||
"<p>Calculate the desired Q value. We multiply by <span translate=no>_^_0_^_</span> to zero out the next state Q values if the game ended.</p>\n<p><span translate=no>_^_1_^_</span> </p>\n": "<p>\u0d85\u0db4\u0dda\u0d9a\u0dca\u0dc2\u0dd2\u0dadQ \u0d85\u0d9c\u0dba \u0d9c\u0dab\u0db1\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1. \u0d9a\u0dca\u0dbb\u0dd3\u0da9\u0dcf\u0dc0 \u0d85\u0dc0\u0dc3\u0db1\u0dca \u0dc0\u0dd6\u0dba\u0dda \u0db1\u0db8\u0dca \u0d8a\u0dc5\u0d9f \u0dbb\u0dcf\u0da2\u0dca\u0dba Q \u0d85\u0d9c\u0dba\u0db1\u0dca \u0dc1\u0dd4\u0db1\u0dca\u0dba <span translate=no>_^_0_^_</span> \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0da7 \u0d85\u0db4\u0dd2 \u0d9c\u0dd4\u0dab \u0d9a\u0dbb\u0db8\u0dd4. </p>\n<p><span translate=no>_^_1_^_</span> </p>\n",
|
||||
"<p>Get the best action at state <span translate=no>_^_0_^_</span> <span translate=no>_^_1_^_</span> </p>\n": "<p>\u0dbb\u0dcf\u0da2\u0dca\u0dba\u0dba\u0dd9\u0db1\u0dca\u0dc4\u0ddc\u0db3\u0db8 \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dc0 \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 <span translate=no>_^_0_^_</span> <span translate=no>_^_1_^_</span> </p>\n",
|
||||
"<p>Get the q value from the target network for the best action at state <span translate=no>_^_0_^_</span> <span translate=no>_^_1_^_</span> </p>\n": "<p>\u0dbb\u0dcf\u0da2\u0dca\u0dba\u0dc4\u0ddc\u0db3\u0db8 \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dc0 \u0dc3\u0db3\u0dc4\u0dcf \u0d89\u0dbd\u0d9a\u0dca\u0d9a \u0da2\u0dcf\u0dbd\u0dba\u0dd9\u0db1\u0dca q \u0d85\u0d9c\u0dba \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 <span translate=no>_^_0_^_</span> <span translate=no>_^_1_^_</span> </p>\n",
|
||||
"<p>Get weighted means </p>\n": "<p>\u0db6\u0dbb\u0dad\u0dd0\u0db6\u0dd6 \u0d9a\u0dca\u0dbb\u0db8 \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Gradients shouldn't propagate gradients <span translate=no>_^_0_^_</span> </p>\n": "<p>\u0d85\u0db1\u0dd4\u0d9a\u0dca\u0dbb\u0db8\u0dd2\u0d9a\u0d85\u0db1\u0dd4\u0d9a\u0dca\u0dbb\u0db8\u0dd2\u0d9a \u0db4\u0dca\u0dbb\u0da0\u0dcf\u0dbb\u0dba \u0db1\u0ddc\u0d9a\u0dc5 \u0dba\u0dd4\u0dad\u0dd4\u0dba <span translate=no>_^_0_^_</span> </p>\n",
|
||||
"<p>Temporal difference error <span translate=no>_^_0_^_</span> is used to weigh samples in replay buffer </p>\n": "<p>\u0db1\u0dd0\u0dc0\u0dad\u0db0\u0dcf\u0dc0\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0db6\u0dc6\u0dbb\u0dba\u0dda \u0dc3\u0dcf\u0db8\u0dca\u0db4\u0dbd \u0d9a\u0dd2\u0dbb\u0dcf \u0db8\u0dd0\u0db1 \u0db6\u0dd0\u0dbd\u0dd3\u0db8\u0da7 \u0dad\u0dcf\u0dc0\u0d9a\u0dcf\u0dbd\u0dd2\u0d9a \u0dc0\u0dd9\u0db1\u0dc3 \u0daf\u0ddd\u0dc2\u0dba <span translate=no>_^_0_^_</span> \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dbb\u0dba\u0dd2 </p>\n",
|
||||
"<p>We take <a href=\"https://en.wikipedia.org/wiki/Huber_loss\">Huber loss</a> instead of mean squared error loss because it is less sensitive to outliers </p>\n": "<p>\u0d91\u0dbaoutliers \u0d85\u0da9\u0dd4 \u0dc3\u0d82\u0dc0\u0dda\u0daf\u0dd3 \u0db1\u0dd2\u0dc3\u0dcf \u0d85\u0db4\u0dd2 \u0d92 \u0dc0\u0dd9\u0db1\u0dd4\u0dc0\u0da7 \u0db8\u0db0\u0dca\u0dba\u0db1\u0dca\u0dba \u0dc0\u0dbb\u0dca\u0d9c \u0daf\u0ddd\u0dc2\u0dba\u0d9a\u0dca \u0d85\u0dc4\u0dd2\u0db8\u0dd2 <a href=\"https://en.wikipedia.org/wiki/Huber_loss\">Huber</a> \u0d85\u0dc4\u0dd2\u0db8\u0dd2 \u0d9c\u0dad </p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> - <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> - <span translate=no>_^_3_^_</span> </li>\n<li><span translate=no>_^_4_^_</span> - <span translate=no>_^_5_^_</span> </li>\n<li><span translate=no>_^_6_^_</span> - <span translate=no>_^_7_^_</span> </li>\n<li><span translate=no>_^_8_^_</span> - whether the game ended after taking the action </li>\n<li><span translate=no>_^_9_^_</span> - <span translate=no>_^_10_^_</span> </li>\n<li><span translate=no>_^_11_^_</span> - weights of the samples from prioritized experienced replay</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span> - <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> - <span translate=no>_^_3_^_</span> </li>\n<li><span translate=no>_^_4_^_</span> - <span translate=no>_^_5_^_</span> </li>\n<li><span translate=no>_^_6_^_</span> - <span translate=no>_^_7_^_</span> </li>\n<li><span translate=no>_^_8_^_</span> - \u0db4\u0dd2\u0dba\u0dc0\u0dbb \u0d9c\u0dd0\u0db1\u0dd3\u0db8\u0dd9\u0db1\u0dca \u0db4\u0dc3\u0dd4 \u0d9a\u0dca\u0dbb\u0dd3\u0da9\u0dcf\u0dc0 \u0d85\u0dc0\u0dc3\u0db1\u0dca \u0dc0\u0dd6\u0dc0\u0dcf\u0daf \u0dba\u0db1\u0dca\u0db1 </li>\n<li><span translate=no>_^_9_^_</span> - <span translate=no>_^_10_^_</span> </li>\n<li><span translate=no>_^_11_^_</span> - \u0db4\u0dca\u0dbb\u0db8\u0dd4\u0d9b\u0dad\u0dcf\u0dc0\u0dba \u0db4\u0dc5\u0db4\u0dd4\u0dbb\u0dd4\u0daf\u0dd4 \u0db1\u0dd0\u0dc0\u0dad \u0db0\u0dcf\u0dc0\u0db1\u0dba \u0dc3\u0dd2\u0da7 \u0dc3\u0dcf\u0db8\u0dca\u0db4\u0dbd \u0db6\u0dbb</li></ul>\n",
|
||||
"Deep Q Networks (DQN)": "\u0d9c\u0dd0\u0db9\u0dd4\u0dbb\u0dd4 Q \u0da2\u0dcf\u0dbd (DQN)",
|
||||
"This is a PyTorch implementation/tutorial of Deep Q Networks (DQN) from paper Playing Atari with Deep Reinforcement Learning. This includes dueling network architecture, a prioritized replay buffer and double-Q-network training.": "\u0db8\u0dd9\u0dba PyTorch \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dd2\u0dbb\u0dd3\u0db8/\u0db1\u0dd2\u0db6\u0db1\u0dca\u0db0\u0db1\u0dba\u0d9a\u0dd2 \u0d9c\u0dd0\u0db9\u0dd4\u0dbb\u0dd4 Q \u0db1\u0dd9\u0da7\u0dca\u0dc0\u0dbb\u0dca\u0d9a\u0dca\u0dc3\u0dca (DQN) \u0d9a\u0da9\u0daf\u0dcf\u0dc3\u0dd2 \u0dc3\u0dd9\u0dbd\u0dca\u0dbd\u0db8\u0dca \u0d85\u0da7\u0dcf\u0dbb\u0dd2 \u0d9c\u0dd0\u0db9\u0dd4\u0dbb\u0dd4 \u0dc1\u0d9a\u0dca\u0dad\u0dd2\u0db8\u0dad\u0dca \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0d89\u0d9c\u0dd9\u0db1\u0dd3\u0db8 \u0dc3\u0db8\u0d9f. \u0da2\u0dcf\u0dbd \u0d9c\u0dd8\u0dc4 \u0db1\u0dd2\u0dbb\u0dca\u0db8\u0dcf\u0dab \u0dc1\u0dd2\u0dbd\u0dca\u0db4\u0dba \u0da9\u0db6\u0dbd\u0dca \u0d9a\u0dd2\u0dbb\u0dd3\u0db8, \u0db4\u0dca\u0dbb\u0db8\u0dd4\u0d9b\u0dad\u0dcf \u0db1\u0dd0\u0dc0\u0dad \u0db0\u0dcf\u0dc0\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0db6\u0dc6\u0dbb\u0dba\u0d9a\u0dca \u0dc3\u0dc4 \u0daf\u0dca\u0dc0\u0dd2\u0dad\u0dca\u0dc0 Q-\u0da2\u0dcf\u0dbd \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0dc0 \u0db8\u0dd9\u0dba\u0da7 \u0d87\u0dad\u0dd4\u0dc5\u0dad\u0dca \u0dba."
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
{
|
||||
"<h1>Deep Q Networks (DQN)</h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of paper <a href=\"https://arxiv.org/abs/1312.5602\">Playing Atari with Deep Reinforcement Learning</a> along with <a href=\"model.html\">Dueling Network</a>, <a href=\"replay_buffer.html\">Prioritized Replay</a> and Double Q Network.</p>\n<p>Here is the <a href=\"experiment.html\">experiment</a> and <a href=\"model.html\">model</a> implementation.</p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a></p>\n": "<h1>\u6df1\u5ea6 Q \u7f51\u7edc (DQN)</h1>\n<p>\u8fd9\u662f <a href=\"https://pytorch.org\">PyTorch</a> \u5b9e\u73b0\u7684 PyTorch <a href=\"https://arxiv.org/abs/1312.5602\">\u4f7f\u7528\u6df1\u5ea6\u5f3a\u5316\u5b66\u4e60\u73a9\u96c5</a>\u8fbe\u5229\u4ee5\u53ca<a href=\"model.html\">\u51b3\u6597\u7f51\u7edc</a>\u3001<a href=\"replay_buffer.html\">\u4f18\u5148\u56de\u653e</a>\u548c Double Q Network\u3002</p>\n<p>\u8fd9\u662f<a href=\"experiment.html\">\u5b9e\u9a8c</a>\u548c<a href=\"model.html\">\u6a21\u578b</a>\u5b9e\u73b0\u3002</p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a></p>\n",
|
||||
"<h2>Train the model</h2>\n<p>We want to find optimal action-value function.</p>\n<span translate=no>_^_0_^_</span><h3>Target network \ud83c\udfaf</h3>\n<p>In order to improve stability we use experience replay that randomly sample from previous experience <span translate=no>_^_1_^_</span>. We also use a Q network with a separate set of parameters <span translate=no>_^_2_^_</span> to calculate the target. <span translate=no>_^_3_^_</span> is updated periodically. This is according to paper <a href=\"https://deepmind.com/research/dqn/\">Human Level Control Through Deep Reinforcement Learning</a>.</p>\n<p>So the loss function is, <span translate=no>_^_4_^_</span></p>\n<h3>Double <span translate=no>_^_5_^_</span>-Learning</h3>\n<p>The max operator in the above calculation uses same network for both selecting the best action and for evaluating the value. That is, <span translate=no>_^_6_^_</span> We use <a href=\"https://arxiv.org/abs/1509.06461\">double Q-learning</a>, where the <span translate=no>_^_7_^_</span> is taken from <span translate=no>_^_8_^_</span> and the value is taken from <span translate=no>_^_9_^_</span>.</p>\n<p>And the loss function becomes,</p>\n<span translate=no>_^_10_^_</span>": "<h2>\u8bad\u7ec3\u6a21\u578b</h2>\n<p>\u6211\u4eec\u60f3\u627e\u5230\u6700\u4f73\u7684\u52a8\u4f5c\u503c\u51fd\u6570\u3002</p>\n<span translate=no>_^_0_^_</span><h3>\u76ee\u6807\u7f51\u7edc \ud83c\udfaf</h3>\n<p>\u4e3a\u4e86\u63d0\u9ad8\u7a33\u5b9a\u6027\uff0c\u6211\u4eec\u4f7f\u7528\u7ecf\u9a8c\u56de\u653e\uff0c\u4ece\u4ee5\u524d\u7684\u7ecf\u9a8c\u4e2d\u968f\u673a\u62bd\u6837<span translate=no>_^_1_^_</span>\u3002\u6211\u4eec\u8fd8\u4f7f\u7528\u5177\u6709\u4e00\u7ec4\u5355\u72ec\u53c2\u6570\u7684 Q \u7f51\u7edc<span translate=no>_^_2_^_</span>\u6765\u8ba1\u7b97\u76ee\u6807\u3002<span translate=no>_^_3_^_</span>\u5b9a\u671f\u66f4\u65b0\u3002\u8fd9\u662f\u6839\u636e\u8bba\u6587\u300a\u901a\u8fc7\u6df1\u5ea6\u5f3a\u5316\u5b66\u4e60\u8fdb\u884c<a href=\"https://deepmind.com/research/dqn/\">\u4eba\u4f53\u6c34\u5e73\u63a7\u5236</a>\u300b\u5f97\u51fa\u7684\u3002</p>\n<p>\u6240\u4ee5\u635f\u5931\u51fd\u6570\u662f\uff0c<span translate=no>_^_4_^_</span></p>\n<h3>\u53cc<span translate=no>_^_5_^_</span>\u91cd\u5b66\u4e60</h3>\n<p>\u4e0a\u8ff0\u8ba1\u7b97\u4e2d\u7684\u6700\u5927\u503c\u8fd0\u7b97\u7b26\u4f7f\u7528\u76f8\u540c\u7684\u7f51\u7edc\u6765\u9009\u62e9\u6700\u4f73\u52a8\u4f5c\u548c\u8bc4\u4f30\u503c\u3002\u4e5f\u5c31\u662f\u8bf4\uff0c<span translate=no>_^_6_^_</span>\u6211\u4eec\u4f7f\u7528<a href=\"https://arxiv.org/abs/1509.06461\">\u53cc\u91cdQ-L</a><span translate=no>_^_7_^_</span> earning<span translate=no>_^_8_^_</span>\uff0c\u5176\u4e2d\u53d6\u81ea\u503c\uff0c\u53d6\u81ea\u503c<span translate=no>_^_9_^_</span>\u3002</p>\n<p>\u635f\u5931\u51fd\u6570\u53d8\u6210\uff0c</p>\n<span translate=no>_^_10_^_</span>",
|
||||
"<p><span translate=no>_^_0_^_</span> </p>\n": "<p><span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Calculate the desired Q value. We multiply by <span translate=no>_^_0_^_</span> to zero out the next state Q values if the game ended.</p>\n<p><span translate=no>_^_1_^_</span> </p>\n": "<p>\u8ba1\u7b97\u6240\u9700\u7684 Q \u503c\u3002\u5982\u679c\u6e38\u620f\u7ed3\u675f\uff0c\u6211\u4eec\u5c06\u4e58<span translate=no>_^_0_^_</span>\u4ee5\u5c06\u4e0b\u4e00\u4e2a\u72b6\u6001 Q \u503c\u5f52\u96f6\u3002</p>\n<p><span translate=no>_^_1_^_</span></p>\n",
|
||||
"<p>Get the best action at state <span translate=no>_^_0_^_</span> <span translate=no>_^_1_^_</span> </p>\n": "<p>\u5728\u5dde\u5185\u91c7\u53d6\u6700\u4f73\u884c\u52a8<span translate=no>_^_0_^_</span><span translate=no>_^_1_^_</span></p>\n",
|
||||
"<p>Get the q value from the target network for the best action at state <span translate=no>_^_0_^_</span> <span translate=no>_^_1_^_</span> </p>\n": "<p>\u4ece\u76ee\u6807\u7f51\u7edc\u83b7\u53d6 q \u503c\uff0c\u4ee5\u4fbf\u5728\u5dde\u5185\u91c7\u53d6\u6700\u4f73\u884c\u52a8<span translate=no>_^_0_^_</span><span translate=no>_^_1_^_</span></p>\n",
|
||||
"<p>Get weighted means </p>\n": "<p>\u83b7\u53d6\u52a0\u6743\u5747\u503c</p>\n",
|
||||
"<p>Gradients shouldn't propagate gradients <span translate=no>_^_0_^_</span> </p>\n": "<p>\u6e10\u53d8\u4e0d\u5e94\u4f20\u64ad\u6e10\u53d8<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Temporal difference error <span translate=no>_^_0_^_</span> is used to weigh samples in replay buffer </p>\n": "<p>\u65f6\u5dee\u8bef\u5dee<span translate=no>_^_0_^_</span>\u7528\u4e8e\u79f0\u91cf\u91cd\u653e\u7f13\u51b2\u533a\u4e2d\u7684\u6837\u672c</p>\n",
|
||||
"<p>We take <a href=\"https://en.wikipedia.org/wiki/Huber_loss\">Huber loss</a> instead of mean squared error loss because it is less sensitive to outliers </p>\n": "<p>\u6211\u4eec\u91c7\u7528 <a href=\"https://en.wikipedia.org/wiki/Huber_loss\">Huber \u635f\u5931</a>\u800c\u4e0d\u662f\u5747\u65b9\u8bef\u5dee\u635f\u5931\uff0c\u56e0\u4e3a\u5b83\u5bf9\u5f02\u5e38\u503c\u4e0d\u592a\u654f\u611f</p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> - <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> - <span translate=no>_^_3_^_</span> </li>\n<li><span translate=no>_^_4_^_</span> - <span translate=no>_^_5_^_</span> </li>\n<li><span translate=no>_^_6_^_</span> - <span translate=no>_^_7_^_</span> </li>\n<li><span translate=no>_^_8_^_</span> - whether the game ended after taking the action </li>\n<li><span translate=no>_^_9_^_</span> - <span translate=no>_^_10_^_</span> </li>\n<li><span translate=no>_^_11_^_</span> - weights of the samples from prioritized experienced replay</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>-<span translate=no>_^_1_^_</span></li>\n<li><span translate=no>_^_2_^_</span>-<span translate=no>_^_3_^_</span></li>\n<li><span translate=no>_^_4_^_</span>-<span translate=no>_^_5_^_</span></li>\n<li><span translate=no>_^_6_^_</span>-<span translate=no>_^_7_^_</span></li>\n<li><span translate=no>_^_8_^_</span>-\u6e38\u620f\u5728\u91c7\u53d6\u884c\u52a8\u540e\u662f\u5426\u7ed3\u675f</li>\n<li><span translate=no>_^_9_^_</span>-<span translate=no>_^_10_^_</span></li>\n</ul><li><span translate=no>_^_11_^_</span>-\u6765\u81ea\u6709\u7ecf\u9a8c\u7684\u4f18\u5148\u91cd\u64ad\u7684\u6837\u672c\u7684\u6743\u91cd</li>\n",
|
||||
"Deep Q Networks (DQN)": "\u6df1\u5ea6\u95ee\u7b54\u7f51\u7edc (DQN)",
|
||||
"This is a PyTorch implementation/tutorial of Deep Q Networks (DQN) from paper Playing Atari with Deep Reinforcement Learning. This includes dueling network architecture, a prioritized replay buffer and double-Q-network training.": "\u8fd9\u662f Deep Q Networks\uff08DQN\uff09\u7684 PyTorch \u5b9e\u73b0/\u6559\u7a0b\uff0c\u6765\u81ea\u8bba\u6587\u300a\u7528\u6df1\u5ea6\u5f3a\u5316\u5b66\u4e60\u73a9\u96c5\u8fbe\u5229\u300b\u3002\u8fd9\u5305\u62ec\u51b3\u6597\u7f51\u7edc\u67b6\u6784\u3001\u4f18\u5148\u91cd\u64ad\u7f13\u51b2\u533a\u548c Double-Q-Network \u8bad\u7ec3\u3002"
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
{
|
||||
"<h1>DQN Experiment with Atari Breakout</h1>\n<p>This experiment trains a Deep Q Network (DQN) to play Atari Breakout game on OpenAI Gym. It runs the <a href=\"../game.html\">game environments on multiple processes</a> to sample efficiently.</p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a></p>\n": "<h1>\u30a2\u30bf\u30ea\u30fb\u30d6\u30ec\u30a4\u30af\u30a2\u30a6\u30c8\u306b\u3088\u308bDQN\u5b9f\u9a13</h1>\n<p>\u3053\u306e\u5b9f\u9a13\u3067\u306f\u3001\u30c7\u30a3\u30fc\u30d7Q\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\uff08DQN\uff09\u306bOpenAI Gym\u3067\u30a2\u30bf\u30ea\u30d6\u30ec\u30a4\u30af\u30a2\u30a6\u30c8\u30b2\u30fc\u30e0\u3092\u30d7\u30ec\u30a4\u3059\u308b\u3088\u3046\u306b\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3057\u307e\u3059\u3002<a href=\"../game.html\">\u30b2\u30fc\u30e0\u74b0\u5883\u3092\u8907\u6570\u306e\u30d7\u30ed\u30bb\u30b9\u3067\u5b9f\u884c\u3057\u3066\u52b9\u7387\u7684\u306b\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3057\u307e\u3059</a>\u3002</p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a></p>\n",
|
||||
"<h2>Run it</h2>\n": "<h2>\u5b9f\u884c\u3057\u3066\u304f\u3060\u3055\u3044</h2>\n",
|
||||
"<h2>Trainer</h2>\n": "<h2>\u30c8\u30ec\u30fc\u30ca\u30fc</h2>\n",
|
||||
"<h3>Destroy</h3>\n<p>Stop the workers</p>\n": "<h3>\u7834\u58ca</h3>\n<p>\u52b4\u50cd\u8005\u3092\u6b62\u3081\u308d</p>\n",
|
||||
"<h3>Run training loop</h3>\n": "<h3>\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u30eb\u30fc\u30d7\u3092\u5b9f\u884c</h3>\n",
|
||||
"<h3>Sample data</h3>\n": "<h3>\u30b5\u30f3\u30d7\u30eb\u30c7\u30fc\u30bf</h3>\n",
|
||||
"<h3>Train the model</h3>\n": "<h3>\u30e2\u30c7\u30eb\u306e\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0</h3>\n",
|
||||
"<h4><span translate=no>_^_0_^_</span>-greedy Sampling</h4>\n<p>When sampling actions we use a <span translate=no>_^_1_^_</span>-greedy strategy, where we take a greedy action with probabiliy <span translate=no>_^_2_^_</span> and take a random action with probability <span translate=no>_^_3_^_</span>. We refer to <span translate=no>_^_4_^_</span> as <span translate=no>_^_5_^_</span>.</p>\n": "<h4><span translate=no>_^_0_^_</span>-\u8caa\u6b32\u306a\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</h4>\n<p>\u30a2\u30af\u30b7\u30e7\u30f3\u3092\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3059\u308b\u3068\u304d\u306f\u3001<span translate=no>_^_1_^_</span>-greedy \u30b9\u30c8\u30e9\u30c6\u30b8\u30fc\u3092\u4f7f\u7528\u3057\u307e\u3059\u3002\u3064\u307e\u308a\u3001<span translate=no>_^_2_^_</span>\u78ba\u7387\u306e\u3042\u308b\u8caa\u6b32\u306a\u30a2\u30af\u30b7\u30e7\u30f3\u3092\u5b9f\u884c\u3057\u3001\u78ba\u7387\u306e\u3042\u308b\u30e9\u30f3\u30c0\u30e0\u306a\u30a2\u30af\u30b7\u30e7\u30f3\u3092\u5b9f\u884c\u3057\u307e\u3059\u3002<span translate=no>_^_3_^_</span><span translate=no>_^_4_^_</span>\u3068\u547c\u3073\u307e\u3059<span translate=no>_^_5_^_</span>\u3002</p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span> for prioritized replay </p>\n": "<p><span translate=no>_^_0_^_</span>\u512a\u5148\u518d\u751f\u7528</p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span> for replay buffer as a function of updates </p>\n": "<p><span translate=no>_^_0_^_</span>\u66f4\u65b0\u6a5f\u80fd\u3068\u3057\u3066\u306e\u518d\u751f\u30d0\u30c3\u30d5\u30a1\u7528</p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span>, exploration fraction </p>\n": "<p><span translate=no>_^_0_^_</span>\u3001\u63a2\u67fb\u30d5\u30e9\u30af\u30b7\u30e7\u30f3</p>\n",
|
||||
"<p>Add a new line to the screen periodically </p>\n": "<p>\u753b\u9762\u306b\u5b9a\u671f\u7684\u306b\u65b0\u3057\u3044\u884c\u3092\u8ffd\u52a0\u3057\u3066\u304f\u3060\u3055\u3044</p>\n",
|
||||
"<p>Add transition to replay buffer </p>\n": "<p>\u518d\u751f\u30d0\u30c3\u30d5\u30a1\u306b\u30c8\u30e9\u30f3\u30b8\u30b7\u30e7\u30f3\u3092\u8ffd\u52a0</p>\n",
|
||||
"<p>Calculate gradients </p>\n": "<p>\u52fe\u914d\u306e\u8a08\u7b97</p>\n",
|
||||
"<p>Calculate priorities for replay buffer <span translate=no>_^_0_^_</span> </p>\n": "<p>\u518d\u751f\u30d0\u30c3\u30d5\u30a1\u306e\u512a\u5148\u5ea6\u3092\u8a08\u7b97 <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Clip gradients </p>\n": "<p>\u30af\u30ea\u30c3\u30d7\u30b0\u30e9\u30c7\u30fc\u30b7\u30e7\u30f3</p>\n",
|
||||
"<p>Collect information from each worker </p>\n": "<p>\u5404\u4f5c\u696d\u8005\u304b\u3089\u60c5\u5831\u3092\u53ce\u96c6\u3059\u308b</p>\n",
|
||||
"<p>Compute Temporal Difference (TD) errors, <span translate=no>_^_0_^_</span>, and the loss, <span translate=no>_^_1_^_</span>. </p>\n": "<p>\u6642\u5dee (TD) \u8aa4\u5dee<span translate=no>_^_0_^_</span>\u3001\u304a\u3088\u3073\u640d\u5931\u3092\u8a08\u7b97\u3057\u307e\u3059\u3002<span translate=no>_^_1_^_</span></p>\n",
|
||||
"<p>Configurations </p>\n": "<p>\u30b3\u30f3\u30d5\u30a3\u30ae\u30e5\u30ec\u30fc\u30b7\u30e7\u30f3</p>\n",
|
||||
"<p>Copy to target network initially </p>\n": "<p>\u6700\u521d\u306b\u30bf\u30fc\u30b2\u30c3\u30c8\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u306b\u30b3\u30d4\u30fc</p>\n",
|
||||
"<p>Create the experiment </p>\n": "<p>\u5b9f\u9a13\u3092\u4f5c\u6210</p>\n",
|
||||
"<p>Get <span translate=no>_^_0_^_</span> </p>\n": "<p>\u53d6\u5f97 <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Get Q_values for the current observation </p>\n": "<p>\u73fe\u5728\u306e\u89b3\u6e2c\u5024\u306e Q_value \u3092\u53d6\u5f97</p>\n",
|
||||
"<p>Get results after executing the actions </p>\n": "<p>\u30a2\u30af\u30b7\u30e7\u30f3\u3092\u5b9f\u884c\u3057\u305f\u5f8c\u306b\u7d50\u679c\u3092\u53d6\u5f97</p>\n",
|
||||
"<p>Get the Q-values of the next state for <a href=\"index.html\">Double Q-learning</a>. Gradients shouldn't propagate for these </p>\n": "<p><a href=\"index.html\">\u4e8c\u91cdQ\u5b66\u7fd2\u306e\u6b21\u306e\u72b6\u614b\u306eQ\u5024\u3092\u53d6\u5f97\u3057\u307e\u3059</a>\u3002\u3053\u308c\u3089\u306e\u5834\u5408\u3001\u30b0\u30e9\u30c7\u30fc\u30b7\u30e7\u30f3\u306f\u4f1d\u64ad\u3057\u306a\u3044\u306f\u305a\u3067\u3059</p>\n",
|
||||
"<p>Get the predicted Q-value </p>\n": "<p>\u4e88\u6e2c\u3055\u308c\u305f Q \u5024\u306e\u53d6\u5f97</p>\n",
|
||||
"<p>Initialize the trainer </p>\n": "<p>\u30c8\u30ec\u30fc\u30ca\u30fc\u3092\u521d\u671f\u5316</p>\n",
|
||||
"<p>Last 100 episode information </p>\n": "<p>\u6700\u65b0100\u8a71\u306e\u60c5\u5831</p>\n",
|
||||
"<p>Learning rate. </p>\n": "<p>\u5b66\u7fd2\u7387\u3002</p>\n",
|
||||
"<p>Mini batch size </p>\n": "<p>\u30df\u30cb\u30d0\u30c3\u30c1\u30b5\u30a4\u30ba</p>\n",
|
||||
"<p>Model for sampling and training </p>\n": "<p>\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3068\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u7528\u306e\u30e2\u30c7\u30eb</p>\n",
|
||||
"<p>Number of epochs to train the model with sampled data. </p>\n": "<p>\u30b5\u30f3\u30d7\u30eb\u30c7\u30fc\u30bf\u3092\u4f7f\u7528\u3057\u3066\u30e2\u30c7\u30eb\u3092\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3059\u308b\u30a8\u30dd\u30c3\u30af\u306e\u6570\u3002</p>\n",
|
||||
"<p>Number of steps to run on each process for a single update </p>\n": "<p>1 \u56de\u306e\u66f4\u65b0\u3067\u5404\u30d7\u30ed\u30bb\u30b9\u3067\u5b9f\u884c\u3059\u308b\u30b9\u30c6\u30c3\u30d7\u306e\u6570</p>\n",
|
||||
"<p>Number of updates </p>\n": "<p>\u66f4\u65b0\u56de\u6570</p>\n",
|
||||
"<p>Number of worker processes </p>\n": "<p>\u30ef\u30fc\u30ab\u30fc\u30d7\u30ed\u30bb\u30b9\u306e\u6570</p>\n",
|
||||
"<p>Periodically update target network </p>\n": "<p>\u30bf\u30fc\u30b2\u30c3\u30c8\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3092\u5b9a\u671f\u7684\u306b\u66f4\u65b0</p>\n",
|
||||
"<p>Pick the action based on <span translate=no>_^_0_^_</span> </p>\n": "<p>\u4ee5\u4e0b\u306b\u57fa\u3065\u3044\u3066\u30a2\u30af\u30b7\u30e7\u30f3\u3092\u9078\u629e\u3057\u3066\u304f\u3060\u3055\u3044 <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Replay buffer with <span translate=no>_^_0_^_</span>. Capacity of the replay buffer must be a power of 2. </p>\n": "<p>\u30ea\u30d7\u30ec\u30a4\u30d0\u30c3\u30d5\u30a1\u306f<span translate=no>_^_0_^_</span>.\u518d\u751f\u30d0\u30c3\u30d5\u30a1\u306e\u5bb9\u91cf\u306f 2 \u306e\u7d2f\u4e57\u3067\u306a\u3051\u308c\u3070\u306a\u308a\u307e\u305b\u3093</p>\u3002\n",
|
||||
"<p>Run and monitor the experiment </p>\n": "<p>\u5b9f\u9a13\u306e\u5b9f\u884c\u3068\u76e3\u8996</p>\n",
|
||||
"<p>Run sampled actions on each worker </p>\n": "<p>\u5404\u30ef\u30fc\u30ab\u30fc\u3067\u30b5\u30f3\u30d7\u30eb\u30a2\u30af\u30b7\u30e7\u30f3\u3092\u5b9f\u884c</p>\n",
|
||||
"<p>Sample <span translate=no>_^_0_^_</span> </p>\n": "<p>[\u30b5\u30f3\u30d7\u30eb] <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Sample actions </p>\n": "<p>\u30b5\u30f3\u30d7\u30eb\u30a2\u30af\u30b7\u30e7\u30f3</p>\n",
|
||||
"<p>Sample from priority replay buffer </p>\n": "<p>\u30d7\u30e9\u30a4\u30aa\u30ea\u30c6\u30a3\u30fb\u30ea\u30d7\u30ec\u30a4\u30fb\u30d0\u30c3\u30d5\u30a1\u304b\u3089\u306e\u30b5\u30f3\u30d7\u30eb</p>\n",
|
||||
"<p>Sample the action with highest Q-value. This is the greedy action. </p>\n": "<p>Q\u5024\u304c\u6700\u3082\u9ad8\u3044\u30a2\u30af\u30b7\u30e7\u30f3\u3092\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3057\u307e\u3059\u3002\u3053\u308c\u306f\u8caa\u6b32\u306a\u884c\u52d5\u3067\u3059</p>\u3002\n",
|
||||
"<p>Sample with current policy </p>\n": "<p>\u73fe\u5728\u306e\u30dd\u30ea\u30b7\u30fc\u3092\u542b\u3080\u30b5\u30f3\u30d7\u30eb</p>\n",
|
||||
"<p>Sampling doesn't need gradients </p>\n": "<p>\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u306b\u306f\u30b0\u30e9\u30c7\u30fc\u30b7\u30e7\u30f3\u306f\u5fc5\u8981\u3042\u308a\u307e\u305b\u3093</p>\n",
|
||||
"<p>Save tracked indicators. </p>\n": "<p>\u8ffd\u8de1\u6307\u6a19\u3092\u4fdd\u5b58\u3057\u307e\u3059\u3002</p>\n",
|
||||
"<p>Scale observations from <span translate=no>_^_0_^_</span> to <span translate=no>_^_1_^_</span> </p>\n": "<p><span translate=no>_^_0_^_</span>\u89b3\u6e2c\u5024\u3092\u304b\u3089\u306b\u30b9\u30b1\u30fc\u30ea\u30f3\u30b0 <span translate=no>_^_1_^_</span></p>\n",
|
||||
"<p>Select device </p>\n": "<p>\u30c7\u30d0\u30a4\u30b9\u3092\u9078\u629e</p>\n",
|
||||
"<p>Set learning rate </p>\n": "<p>\u5b66\u7fd2\u7387\u3092\u8a2d\u5b9a</p>\n",
|
||||
"<p>Start training after the buffer is full </p>\n": "<p>\u30d0\u30c3\u30d5\u30a1\u30fc\u304c\u3044\u3063\u3071\u3044\u306b\u306a\u3063\u305f\u3089\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u3092\u958b\u59cb\u3059\u308b</p>\n",
|
||||
"<p>Stop the workers </p>\n": "<p>\u52b4\u50cd\u8005\u3092\u6b62\u3081\u308d</p>\n",
|
||||
"<p>Target model updating interval </p>\n": "<p>\u5bfe\u8c61\u30e2\u30c7\u30eb\u306e\u66f4\u65b0\u9593\u9694</p>\n",
|
||||
"<p>This doesn't need gradients </p>\n": "<p>\u3053\u308c\u306b\u306f\u30b0\u30e9\u30c7\u30fc\u30b7\u30e7\u30f3\u306f\u5fc5\u8981\u3042\u308a\u307e\u305b\u3093</p>\n",
|
||||
"<p>Train the model </p>\n": "<p>\u30e2\u30c7\u30eb\u306e\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0</p>\n",
|
||||
"<p>Uniformly sample and action </p>\n": "<p>\u30b5\u30f3\u30d7\u30eb\u3068\u30a2\u30af\u30b7\u30e7\u30f3\u3092\u5747\u4e00\u306b</p>\n",
|
||||
"<p>Update parameters based on gradients </p>\n": "<p>\u30b0\u30e9\u30c7\u30fc\u30b7\u30e7\u30f3\u306b\u57fa\u3065\u3044\u3066\u30d1\u30e9\u30e1\u30fc\u30bf\u3092\u66f4\u65b0</p>\n",
|
||||
"<p>Update replay buffer priorities </p>\n": "<p>\u30ea\u30d7\u30ec\u30a4\u30d0\u30c3\u30d5\u30a1\u306e\u512a\u5148\u9806\u4f4d\u3092\u66f4\u65b0</p>\n",
|
||||
"<p>Whether to chose greedy action or the random action </p>\n": "<p>\u6b32\u5f35\u308a\u30a2\u30af\u30b7\u30e7\u30f3\u3068\u30e9\u30f3\u30c0\u30e0\u30a2\u30af\u30b7\u30e7\u30f3\u306e\u3069\u3061\u3089\u3092\u9078\u3076\u304b</p>\n",
|
||||
"<p>Zero out the previously calculated gradients </p>\n": "<p>\u4ee5\u524d\u306b\u8a08\u7b97\u3057\u305f\u30b0\u30e9\u30c7\u30fc\u30b7\u30e7\u30f3\u3092\u30bc\u30ed\u306b\u3057\u307e\u3059</p>\n",
|
||||
"<p>create workers </p>\n": "<p>\u30ef\u30fc\u30ab\u30fc\u3092\u4f5c\u6210</p>\n",
|
||||
"<p>exploration as a function of updates </p>\n": "<p>\u66f4\u65b0\u6a5f\u80fd\u3068\u3057\u3066\u306e\u63a2\u7d22</p>\n",
|
||||
"<p>get the initial observations </p>\n": "<p>\u521d\u671f\u89b3\u6e2c\u5024\u3092\u53d6\u5f97</p>\n",
|
||||
"<p>initialize tensors for observations </p>\n": "<p>\u89b3\u6e2c\u7528\u306e\u30c6\u30f3\u30bd\u30eb\u3092\u521d\u671f\u5316</p>\n",
|
||||
"<p>learning rate </p>\n": "<p>\u5b66\u7fd2\u7387</p>\n",
|
||||
"<p>loss function </p>\n": "<p>\u640d\u5931\u95a2\u6570</p>\n",
|
||||
"<p>number of training iterations </p>\n": "<p>\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u306e\u53cd\u5fa9\u56de\u6570</p>\n",
|
||||
"<p>number of updates </p>\n": "<p>\u66f4\u65b0\u56de\u6570</p>\n",
|
||||
"<p>number of workers </p>\n": "<p>\u52b4\u50cd\u8005\u306e\u6570</p>\n",
|
||||
"<p>optimizer </p>\n": "<p>\u30aa\u30d7\u30c6\u30a3\u30de\u30a4\u30b6\u30fc</p>\n",
|
||||
"<p>reset the workers </p>\n": "<p>\u30ef\u30fc\u30ab\u30fc\u3092\u30ea\u30bb\u30c3\u30c8</p>\n",
|
||||
"<p>size of mini batch for training </p>\n": "<p>\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u7528\u30df\u30cb\u30d0\u30c3\u30c1\u306e\u30b5\u30a4\u30ba</p>\n",
|
||||
"<p>steps sampled on each update </p>\n": "<p>\u66f4\u65b0\u306e\u305f\u3073\u306b\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3055\u308c\u308b\u30b9\u30c6\u30c3\u30d7</p>\n",
|
||||
"<p>target model to get <span translate=no>_^_0_^_</span> </p>\n": "<p>\u53d6\u5f97\u3059\u308b\u5bfe\u8c61\u30e2\u30c7\u30eb <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>update current observation </p>\n": "<p>\u73fe\u5728\u306e\u89b3\u6e2c\u5024\u3092\u66f4\u65b0</p>\n",
|
||||
"<p>update episode information. collect episode info, which is available if an episode finished; this includes total reward and length of the episode - look at <span translate=no>_^_0_^_</span> to see how it works. </p>\n": "<p>\u30a8\u30d4\u30bd\u30fc\u30c9\u60c5\u5831\u3092\u66f4\u65b0\u3057\u307e\u3059\u3002\u30a8\u30d4\u30bd\u30fc\u30c9\u304c\u7d42\u4e86\u3057\u305f\u5834\u5408\u306b\u5229\u7528\u3067\u304d\u308b\u30a8\u30d4\u30bd\u30fc\u30c9\u60c5\u5831\u3092\u53ce\u96c6\u3057\u307e\u3059\u3002\u3053\u308c\u306b\u306f\u3001\u5408\u8a08\u5831\u916c\u3068\u30a8\u30d4\u30bd\u30fc\u30c9\u306e\u9577\u3055\u304c\u542b\u307e\u308c\u307e\u3059\u3002\u4ed5\u7d44\u307f\u3092\u78ba\u8a8d\u3057\u3066\u307f\u3066\u304f\u3060\u3055\u3044\u3002<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>update target network every 250 update </p>\n": "<p>250 \u56de\u306e\u66f4\u65b0\u3054\u3068\u306b\u30bf\u30fc\u30b2\u30c3\u30c8\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3092\u66f4\u65b0</p>\n",
|
||||
"DQN Experiment with Atari Breakout": "\u30a2\u30bf\u30ea\u30fb\u30d6\u30ec\u30a4\u30af\u30a2\u30a6\u30c8\u306b\u3088\u308bDQN\u5b9f\u9a13",
|
||||
"Implementation of DQN experiment with Atari Breakout": "\u30a2\u30bf\u30ea\u30fb\u30d6\u30ec\u30a4\u30af\u30a2\u30a6\u30c8\u306b\u3088\u308bDQN\u5b9f\u9a13\u306e\u5b9f\u65bd"
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
{
|
||||
"<h1>DQN Experiment with Atari Breakout</h1>\n<p>This experiment trains a Deep Q Network (DQN) to play Atari Breakout game on OpenAI Gym. It runs the <a href=\"../game.html\">game environments on multiple processes</a> to sample efficiently.</p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a> <a href=\"https://app.labml.ai/run/fe1ad986237511ec86e8b763a2d3f710\"><span translate=no>_^_1_^_</span></a></p>\n": "<h1>\u0d85\u0da7\u0dcf\u0dbb\u0dd2\u0db6\u0dca\u0dbb\u0dda\u0d9a\u0dca\u0d85\u0dc0\u0dd4\u0da7\u0dca \u0dc3\u0db8\u0d9f \u0da9\u0dd3QN \u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf \u0db6\u0dd0\u0dbd\u0dd3\u0db8</h1>\n<p>\u0db8\u0dd9\u0db8\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf \u0db6\u0dd0\u0dbd\u0dd3\u0db8 OpenAI \u0da2\u0dd2\u0db8\u0dca \u0dc4\u0dd2 \u0d85\u0da7\u0dcf\u0dbb\u0dd2 \u0db6\u0dca\u0dbb\u0dda\u0d9a\u0dca \u0d85\u0dc0\u0dd4\u0da7\u0dca \u0d9a\u0dca\u0dbb\u0dd3\u0da9\u0dcf\u0dc0 \u0dc3\u0db3\u0dc4\u0dcf \u0d9c\u0dd0\u0db9\u0dd4\u0dbb\u0dd4 Q \u0da2\u0dcf\u0dbd\u0dba\u0d9a\u0dca (DQN) \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0d9a\u0dbb\u0dba\u0dd2. \u0d9a\u0dcf\u0dbb\u0dca\u0dba\u0d9a\u0dca\u0dc2\u0db8\u0dc0 \u0dc3\u0dcf\u0db8\u0dca\u0db4\u0dbd \u0dbd\u0db6\u0dcf \u0d9c\u0dd0\u0db1\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0d91\u0dba <a href=\"../game.html\">\u0db6\u0dc4\u0dd4 \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dc0\u0dbd\u0dd3\u0db1\u0dca\u0dc4\u0dd2 \u0d9a\u0dca\u0dbb\u0dd3\u0da9\u0dcf \u0db4\u0dbb\u0dd2\u0dc3\u0dbb\u0dba\u0db1\u0dca</a> \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dbb\u0dba\u0dd2. </p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a> <a href=\"https://app.labml.ai/run/fe1ad986237511ec86e8b763a2d3f710\"> <span translate=no>_^_1_^_</span></a></p>\n",
|
||||
"<h2>Run it</h2>\n": "<h2>\u0d91\u0dba\u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dbb\u0db1\u0dca\u0db1</h2>\n",
|
||||
"<h2>Trainer</h2>\n": "<h2>\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0d9a\u0dbb\u0dd4</h2>\n",
|
||||
"<h3>Destroy</h3>\n<p>Stop the workers</p>\n": "<h3>\u0dc0\u0dd2\u0db1\u0dcf\u0dc1\u0d9a\u0dbb\u0db1\u0dca\u0db1</h3>\n<p>\u0d9a\u0db8\u0dca\u0d9a\u0dbb\u0dd4\u0dc0\u0db1\u0dca\u0db1\u0dc0\u0dad\u0dca\u0dc0\u0db1\u0dca\u0db1</p>\n",
|
||||
"<h3>Run training loop</h3>\n": "<h3>\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0dbd\u0dd6\u0db4\u0dba \u0db0\u0dcf\u0dc0\u0db1\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1</h3>\n",
|
||||
"<h3>Sample data</h3>\n": "<h3>\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0daf\u0dad\u0dca\u0dad</h3>\n",
|
||||
"<h3>Train the model</h3>\n": "<h3>\u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0d9a\u0dbb\u0db1\u0dca\u0db1</h3>\n",
|
||||
"<h4><span translate=no>_^_0_^_</span>-greedy Sampling</h4>\n<p>When sampling actions we use a <span translate=no>_^_1_^_</span>-greedy strategy, where we take a greedy action with probabiliy <span translate=no>_^_2_^_</span> and take a random action with probability <span translate=no>_^_3_^_</span>. We refer to <span translate=no>_^_4_^_</span> as <span translate=no>_^_5_^_</span>.</p>\n": "<h4><span translate=no>_^_0_^_</span>\u0d9a\u0dd1\u0daf\u0dbb \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8</h4>\n<p>\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8\u0dca\u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda\u0daf\u0dd3 \u0d85\u0db4\u0dd2 <span translate=no>_^_1_^_</span>\u0d9a\u0dd1\u0daf\u0dbb \u0d8b\u0db4\u0dcf\u0dba \u0db8\u0dcf\u0dbb\u0dca\u0d9c\u0dba\u0d9a\u0dca \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dbb\u0db8\u0dd4, \u0d91\u0dc4\u0dd2\u0daf\u0dd3 \u0d85\u0db4\u0dd2 \u0dc3\u0db8\u0dca\u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf\u0dc0 \u0dc3\u0db8\u0d9f \u0d9a\u0dd1\u0daf\u0dbb \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0db8\u0dcf\u0dbb\u0dca\u0d9c\u0dba\u0d9a\u0dca \u0d9c\u0db1\u0dca\u0db1\u0dcf <span translate=no>_^_2_^_</span> \u0d85\u0dad\u0dbb \u0d85\u0dc4\u0db9\u0dd4 \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0db8\u0dcf\u0dbb\u0dca\u0d9c\u0dba\u0d9a\u0dca \u0d9c\u0db1\u0dd2\u0db8\u0dd4 <span translate=no>_^_3_^_</span>. \u0d85\u0db4\u0dd2 <span translate=no>_^_4_^_</span> \u0dc4\u0db3\u0dd4\u0db1\u0dca\u0dc0\u0db1\u0dca\u0db1\u0dda <span translate=no>_^_5_^_</span>. </p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span> for prioritized replay </p>\n": "<p><span translate=no>_^_0_^_</span> \u0db4\u0dca\u0dbb\u0db8\u0dd4\u0d9b\u0dad\u0dcf \u0db1\u0dd0\u0dc0\u0dad \u0db0\u0dcf\u0dc0\u0db1\u0dba \u0dc3\u0db3\u0dc4\u0dcf </p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span> for replay buffer as a function of updates </p>\n": "<p><span translate=no>_^_0_^_</span> \u0dba\u0dcf\u0dc0\u0dad\u0dca\u0d9a\u0dcf\u0dbd\u0dd3\u0db1 \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0d9a\u0dcf\u0dbb\u0dca\u0dba\u0dba\u0d9a\u0dca \u0dbd\u0dd9\u0dc3 \u0db6\u0dc6\u0dbb\u0dba \u0db1\u0dd0\u0dc0\u0dad \u0db0\u0dcf\u0dc0\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf </p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span>, exploration fraction </p>\n": "<p><span translate=no>_^_0_^_</span>, \u0d9c\u0dc0\u0dda\u0dc2\u0dab \u0db7\u0dcf\u0d9c\u0dba </p>\n",
|
||||
"<p>Add a new line to the screen periodically </p>\n": "<p>\u0dc0\u0dbb\u0dd2\u0db1\u0dca\u0dc0\u0dbb \u0dad\u0dd2\u0dbb\u0dba\u0da7 \u0db1\u0dc0 \u0dbb\u0dda\u0d9b\u0dcf\u0dc0\u0d9a\u0dca \u0d91\u0d9a\u0dca \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Add transition to replay buffer </p>\n": "<p>\u0db6\u0dc6\u0dbb\u0dba\u0db1\u0dd0\u0dc0\u0dad \u0db0\u0dcf\u0dc0\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0dc3\u0d82\u0d9a\u0dca\u0dbb\u0dcf\u0db1\u0dca\u0dad\u0dd2\u0dba \u0d91\u0d9a\u0dca \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Calculate gradients </p>\n": "<p>\u0d85\u0db1\u0dd4\u0d9a\u0dca\u0dbb\u0db8\u0dd2\u0d9a\u0d9c\u0dab\u0db1\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Calculate priorities for replay buffer <span translate=no>_^_0_^_</span> </p>\n": "<p>\u0db1\u0dd0\u0dc0\u0dad\u0db0\u0dcf\u0dc0\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0db6\u0dc6\u0dbb\u0dba \u0dc3\u0db3\u0dc4\u0dcf \u0db4\u0dca\u0dbb\u0db8\u0dd4\u0d9b\u0dad\u0dcf \u0d9c\u0dab\u0db1\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1 <span translate=no>_^_0_^_</span> </p>\n",
|
||||
"<p>Clip gradients </p>\n": "<p>\u0d9a\u0dca\u0dbd\u0dd2\u0db4\u0dca\u0d85\u0db1\u0dd4\u0d9a\u0dca\u0dbb\u0db8\u0dd2\u0d9a </p>\n",
|
||||
"<p>Collect information from each worker </p>\n": "<p>\u0d91\u0d9a\u0dca\u0d91\u0d9a\u0dca \u0dc3\u0dda\u0dc0\u0d9a\u0dba\u0dcf\u0d9c\u0dd9\u0db1\u0dca \u0dad\u0ddc\u0dbb\u0dad\u0dd4\u0dbb\u0dd4 \u0dbb\u0dd0\u0dc3\u0dca \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Compute Temporal Difference (TD) errors, <span translate=no>_^_0_^_</span>, and the loss, <span translate=no>_^_1_^_</span>. </p>\n": "<p>\u0dad\u0dcf\u0dc0\u0d9a\u0dcf\u0dbd\u0dd2\u0d9a\u0dc0\u0dd9\u0db1\u0dc3 \u0d9c\u0dab\u0db1\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1 (TD) \u0daf\u0ddd\u0dc2 <span translate=no>_^_0_^_</span>, \u0dc3\u0dc4 \u0d85\u0dbd\u0dcf\u0db7\u0dba, <span translate=no>_^_1_^_</span>. </p>\n",
|
||||
"<p>Configurations </p>\n": "<p>\u0dc0\u0dd2\u0db1\u0dca\u0dba\u0dcf\u0dc3\u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dca </p>\n",
|
||||
"<p>Copy to target network initially </p>\n": "<p>\u0db8\u0dd4\u0dbd\u0dd2\u0db1\u0dca\u0d89\u0dbd\u0d9a\u0dca\u0d9a\u0d9c\u0dad \u0da2\u0dcf\u0dbd\u0dba\u0da7 \u0db4\u0dd2\u0da7\u0db4\u0dad\u0dca \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Create the experiment </p>\n": "<p>\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf\u0db6\u0dd0\u0dbd\u0dd3\u0db8 \u0dc3\u0dcf\u0daf\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Get <span translate=no>_^_0_^_</span> </p>\n": "<p>\u0dbd\u0db6\u0dcf\u0d9c\u0db1\u0dca\u0db1 <span translate=no>_^_0_^_</span> </p>\n",
|
||||
"<p>Get Q_values for the current observation </p>\n": "<p>\u0dc0\u0dad\u0dca\u0db8\u0db1\u0dca\u0db1\u0dd2\u0dbb\u0dd3\u0d9a\u0dca\u0dc2\u0dab \u0dc3\u0db3\u0dc4\u0dcf Q_\u0d85\u0d9c\u0dba\u0db1\u0dca \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Get results after executing the actions </p>\n": "<p>\u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dc0\u0db1\u0dca\u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dd9\u0db1\u0dca \u0db4\u0dc3\u0dd4 \u0db4\u0dca\u0dbb\u0dad\u0dd2 results \u0dbd \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Get the Q-values of the next state for <a href=\"index.html\">Double Q-learning</a>. Gradients shouldn't propagate for these </p>\n": "<p><a href=\"index.html\">\u0daf\u0dca\u0dc0\u0dd2\u0dad\u0dca\u0dc0 Q- \u0d89\u0d9c\u0dd9\u0db1\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0d8a\u0dc5\u0d9f \u0dad\u0dad\u0dca\u0dc0\u0dba\u0dda Q-\u0d85\u0d9c\u0dba\u0db1\u0dca</a>\u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1. \u0db8\u0dda\u0dc0\u0dcf \u0dc3\u0db3\u0dc4\u0dcf \u0d85\u0db1\u0dd4\u0d9a\u0dca\u0dbb\u0db8\u0dd2\u0d9a \u0db4\u0dca\u0dbb\u0da0\u0dcf\u0dbb\u0dab\u0dba \u0db1\u0ddc\u0d9a\u0dc5 \u0dba\u0dd4\u0dad\u0dd4\u0dba </p>\n",
|
||||
"<p>Get the predicted Q-value </p>\n": "<p>\u0db4\u0dd4\u0dbb\u0ddd\u0d9a\u0dae\u0db1\u0dba\u0d9a\u0dbb\u0db1 \u0dbd\u0daf Q-\u0d85\u0d9c\u0dba \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Initialize the trainer </p>\n": "<p>\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0d9a\u0dbb\u0dd4\u0d86\u0dbb\u0db8\u0dca\u0db7 \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Last 100 episode information </p>\n": "<p>\u0d85\u0dc0\u0dc3\u0db1\u0dca100 \u0d9a\u0dae\u0dcf\u0d82\u0d9c \u0dad\u0ddc\u0dbb\u0dad\u0dd4\u0dbb\u0dd4 </p>\n",
|
||||
"<p>Learning rate. </p>\n": "<p>\u0d89\u0d9c\u0dd9\u0db1\u0dd4\u0db8\u0dca\u0d85\u0db1\u0dd4\u0db4\u0dcf\u0dad\u0dba. </p>\n",
|
||||
"<p>Mini batch size </p>\n": "<p>\u0d9a\u0dd4\u0da9\u0dcf\u0d9a\u0dab\u0dca\u0da9\u0dcf\u0dba\u0db8\u0dca \u0db4\u0dca\u0dbb\u0db8\u0dcf\u0dab\u0dba </p>\n",
|
||||
"<p>Model for sampling and training </p>\n": "<p>\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8\u0dca\u0dc4\u0dcf \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0d9a\u0dd2\u0dbb\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba </p>\n",
|
||||
"<p>Number of epochs to train the model with sampled data. </p>\n": "<p>\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0daf\u0dad\u0dca\u0dad \u0dc3\u0db8\u0d9f \u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0d9a\u0dd2\u0dbb\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0d91\u0db4\u0ddc\u0da0\u0dca \u0d9c\u0dab\u0db1. </p>\n",
|
||||
"<p>Number of steps to run on each process for a single update </p>\n": "<p>\u0dad\u0db1\u0dd2\u0dba\u0dcf\u0dc0\u0dad\u0dca\u0d9a\u0dcf\u0dbd\u0dd3\u0db1 \u0d9a\u0dd2\u0dbb\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0d91\u0d9a\u0dca \u0d91\u0d9a\u0dca \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dc0\u0dbd\u0dd2\u0dba \u0db8\u0dad \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0da7 \u0db4\u0dd2\u0dba\u0dc0\u0dbb \u0d9c\u0dab\u0db1 </p>\n",
|
||||
"<p>Number of updates </p>\n": "<p>\u0dba\u0dcf\u0dc0\u0dad\u0dca\u0d9a\u0dcf\u0dbd\u0dd3\u0db1\u0d9c\u0dab\u0db1 </p>\n",
|
||||
"<p>Number of worker processes </p>\n": "<p>\u0dc3\u0dda\u0dc0\u0d9a\u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dc0\u0dbd\u0dd2 \u0d9c\u0dab\u0db1 </p>\n",
|
||||
"<p>Periodically update target network </p>\n": "<p>\u0d89\u0dbd\u0d9a\u0dca\u0d9a\u0da2\u0dcf\u0dbd\u0dba \u0dc0\u0dbb\u0dd2\u0db1\u0dca \u0dc0\u0dbb \u0dba\u0dcf\u0dc0\u0dad\u0dca\u0d9a\u0dcf\u0dbd\u0dd3\u0db1 \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Pick the action based on <span translate=no>_^_0_^_</span> </p>\n": "<p>\u0db8\u0dad\u0db4\u0daf\u0db1\u0db8\u0dca\u0dc0 \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dc0 \u0dad\u0ddd\u0dbb\u0db1\u0dca\u0db1 <span translate=no>_^_0_^_</span> </p>\n",
|
||||
"<p>Replay buffer with <span translate=no>_^_0_^_</span>. Capacity of the replay buffer must be a power of 2. </p>\n": "<p>\u0dc3\u0db8\u0d9f\u0db6\u0dc6\u0dbb\u0dba \u0db1\u0dd0\u0dc0\u0dad \u0db0\u0dcf\u0dc0\u0db1\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1 <span translate=no>_^_0_^_</span>. \u0db1\u0dd0\u0dc0\u0dad \u0db0\u0dcf\u0dc0\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0db6\u0dc6\u0dbb\u0dba\u0dda \u0db0\u0dcf\u0dbb\u0dd2\u0dad\u0dcf\u0dc0 2 \u0db6\u0dbd\u0dba\u0d9a\u0dca \u0dc0\u0dd2\u0dba \u0dba\u0dd4\u0dad\u0dd4\u0dba. </p>\n",
|
||||
"<p>Run and monitor the experiment </p>\n": "<p>\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf\u0db6\u0dd0\u0dbd\u0dd3\u0db8 \u0db0\u0dcf\u0dc0\u0db1\u0dba \u0d9a\u0dbb \u0d85\u0db0\u0dd3\u0d9a\u0dca\u0dc2\u0dab\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Run sampled actions on each worker </p>\n": "<p>\u0d91\u0d9a\u0dca\u0d91\u0d9a\u0dca \u0dc3\u0dda\u0dc0\u0d9a\u0dba\u0dcf \u0db8\u0dad \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a </p>\n",
|
||||
"<p>Sample <span translate=no>_^_0_^_</span> </p>\n": "<p>\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba <span translate=no>_^_0_^_</span> </p>\n",
|
||||
"<p>Sample actions </p>\n": "<p>\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf </p>\n",
|
||||
"<p>Sample from priority replay buffer </p>\n": "<p>\u0db4\u0dca\u0dbb\u0db8\u0dd4\u0d9b\u0dad\u0dcf\u0db1\u0dd0\u0dc0\u0dad \u0db0\u0dcf\u0dc0\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0db6\u0dc6\u0dbb\u0dba\u0dd9\u0db1\u0dca \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba </p>\n",
|
||||
"<p>Sample the action with highest Q-value. This is the greedy action. </p>\n": "<p>\u0d89\u0dc4\u0dc5\u0db8Q- \u0d85\u0d9c\u0dba \u0dc3\u0db8\u0d9f \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dc0 \u0dc3\u0dcf\u0db8\u0dca\u0db4\u0dbd \u0d9a\u0dbb\u0db1\u0dca\u0db1. \u0db8\u0dd9\u0dba \u0d9a\u0dd1\u0daf\u0dbb \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dc0\u0d9a\u0dd2. </p>\n",
|
||||
"<p>Sample with current policy </p>\n": "<p>\u0dc0\u0dad\u0dca\u0db8\u0db1\u0dca\u0db4\u0dca\u0dbb\u0dad\u0dd2\u0db4\u0dad\u0dca\u0dad\u0dd2\u0dba \u0dc3\u0db8\u0d9f \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba </p>\n",
|
||||
"<p>Sampling doesn't need gradients </p>\n": "<p>\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8\u0dca\u0dc0\u0dbd\u0da7 \u0d85\u0db1\u0dd4\u0d9a\u0dca\u0dbb\u0db8\u0dd2\u0d9a \u0d85\u0dc0\u0dc1\u0dca\u0dba \u0db1\u0ddc\u0dc0\u0dda </p>\n",
|
||||
"<p>Save tracked indicators. </p>\n": "<p>\u0dbd\u0dd4\u0dc4\u0dd4\u0db6\u0dd0\u0db3\u0d87\u0dad\u0dd2 \u0daf\u0dbb\u0dca\u0dc1\u0d9a \u0dc3\u0dd4\u0dbb\u0d9a\u0dd2\u0db1\u0dca\u0db1. </p>\n",
|
||||
"<p>Scale observations from <span translate=no>_^_0_^_</span> to <span translate=no>_^_1_^_</span> </p>\n": "<p>\u0dc3\u0dd2\u0da7 <span translate=no>_^_0_^_</span> \u0db4\u0dbb\u0dd2\u0db8\u0dcf\u0dab \u0db1\u0dd2\u0dbb\u0dd3\u0d9a\u0dca\u0dc2\u0dab <span translate=no>_^_1_^_</span> </p>\n",
|
||||
"<p>Select device </p>\n": "<p>\u0d8b\u0db4\u0dcf\u0d82\u0d9c\u0dba\u0dad\u0ddd\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Set learning rate </p>\n": "<p>\u0d89\u0d9c\u0dd9\u0db1\u0dd4\u0db8\u0dca\u0d85\u0db1\u0dd4\u0db4\u0dcf\u0dad\u0dba \u0dc3\u0d9a\u0dc3\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Start training after the buffer is full </p>\n": "<p>\u0db6\u0dc6\u0dbb\u0dba\u0db4\u0dd2\u0dbb\u0dd3 \u0d9c\u0dd2\u0dba \u0db4\u0dc3\u0dd4 \u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0dc0 \u0d86\u0dbb\u0db8\u0dca\u0db7 \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Stop the workers </p>\n": "<p>\u0d9a\u0db8\u0dca\u0d9a\u0dbb\u0dd4\u0dc0\u0db1\u0dca\u0db1\u0dc0\u0dad\u0dca\u0dc0\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Target model updating interval </p>\n": "<p>\u0d89\u0dbd\u0d9a\u0dca\u0d9a\u0d9c\u0dad\u0d86\u0d9a\u0dd8\u0dad\u0dd2 \u0dba\u0dcf\u0dc0\u0dad\u0dca\u0d9a\u0dcf\u0dbd\u0dd3\u0db1 \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0db4\u0dbb\u0dad\u0dbb\u0dba </p>\n",
|
||||
"<p>This doesn't need gradients </p>\n": "<p>\u0db8\u0dda\u0dc3\u0db3\u0dc4\u0dcf \u0d85\u0db1\u0dd4\u0d9a\u0dca\u0dbb\u0db8\u0dd2\u0d9a \u0d85\u0dc0\u0dc1\u0dca\u0dba \u0db1\u0ddc\u0dc0\u0dda </p>\n",
|
||||
"<p>Train the model </p>\n": "<p>\u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4 \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Uniformly sample and action </p>\n": "<p>\u0d92\u0d9a\u0dcf\u0d9a\u0dcf\u0dbb\u0dc0\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba \u0dc3\u0dc4 \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dc0 </p>\n",
|
||||
"<p>Update parameters based on gradients </p>\n": "<p>\u0d85\u0db1\u0dd4\u0d9a\u0dca\u0dbb\u0db8\u0dd2\u0d9a\u0db8\u0dad \u0db4\u0daf\u0db1\u0db8\u0dca\u0dc0 \u0db4\u0dbb\u0dcf\u0db8\u0dd2\u0dad\u0dd3\u0db1\u0dca \u0dba\u0dcf\u0dc0\u0dad\u0dca\u0d9a\u0dcf\u0dbd\u0dd3\u0db1 \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Update replay buffer priorities </p>\n": "<p>\u0db1\u0dd0\u0dc0\u0dad\u0db0\u0dcf\u0dc0\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0dc3\u0dca\u0dc0\u0dcf\u0dbb\u0d9a\u0dca\u0dc2\u0d9a \u0db4\u0dca\u0dbb\u0db8\u0dd4\u0d9b\u0dad\u0dcf \u0dba\u0dcf\u0dc0\u0dad\u0dca\u0d9a\u0dcf\u0dbd\u0dd3\u0db1 \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Whether to chose greedy action or the random action </p>\n": "<p>\u0d9a\u0dd1\u0daf\u0dbb\u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dc0 \u0dc4\u0ddd \u0d85\u0dc4\u0db9\u0dd4 \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dc0 \u0dad\u0ddd\u0dbb\u0dcf \u0d9c\u0dad \u0dba\u0dd4\u0dad\u0dd4\u0daf \u0dba\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Zero out the previously calculated gradients </p>\n": "<p>\u0d9a\u0dbd\u0dd2\u0db1\u0dca\u0d9c\u0dab\u0db1\u0dba \u0d9a\u0dbb\u0db1 \u0dbd\u0daf \u0d85\u0db1\u0dd4\u0d9a\u0dca\u0dbb\u0db8\u0dd2\u0d9a \u0dc1\u0dd4\u0db1\u0dca\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8 </p>\n",
|
||||
"<p>create workers </p>\n": "<p>\u0d9a\u0db8\u0dca\u0d9a\u0dbb\u0dd4\u0dc0\u0db1\u0dca\u0db1\u0dd2\u0dbb\u0dca\u0db8\u0dcf\u0dab\u0dba </p>\n",
|
||||
"<p>exploration as a function of updates </p>\n": "<p>\u0dba\u0dcf\u0dc0\u0dad\u0dca\u0d9a\u0dcf\u0dbd\u0dd3\u0db1\u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0d9a\u0dcf\u0dbb\u0dca\u0dba\u0dba\u0d9a\u0dca \u0dbd\u0dd9\u0dc3 \u0d9c\u0dc0\u0dda\u0dc2\u0dab\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8 </p>\n",
|
||||
"<p>get the initial observations </p>\n": "<p>\u0db8\u0dd6\u0dbd\u0dd2\u0d9a\u0db1\u0dd2\u0dbb\u0dd3\u0d9a\u0dca\u0dc2\u0dab \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>initialize tensors for observations </p>\n": "<p>\u0db1\u0dd2\u0dbb\u0dd3\u0d9a\u0dca\u0dc2\u0dab\u0dc3\u0db3\u0dc4\u0dcf \u0d86\u0dad\u0dad\u0dd3\u0db1\u0dca \u0d86\u0dbb\u0db8\u0dca\u0db7 \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>learning rate </p>\n": "<p>\u0d89\u0d9c\u0dd9\u0db1\u0dd4\u0db8\u0dca\u0d85\u0db1\u0dd4\u0db4\u0dcf\u0dad\u0dba </p>\n",
|
||||
"<p>loss function </p>\n": "<p>\u0db4\u0dcf\u0da9\u0dd4\u0dc1\u0dca\u0dbb\u0dd2\u0dad\u0dba </p>\n",
|
||||
"<p>number of training iterations </p>\n": "<p>\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0db4\u0dd4\u0db1\u0dbb\u0dcf\u0dc0\u0dbb\u0dca\u0dad\u0db1 \u0d9c\u0dab\u0db1 </p>\n",
|
||||
"<p>number of updates </p>\n": "<p>\u0dba\u0dcf\u0dc0\u0dad\u0dca\u0d9a\u0dcf\u0dbd\u0dd3\u0db1\u0d9c\u0dab\u0db1 </p>\n",
|
||||
"<p>number of workers </p>\n": "<p>\u0d9a\u0db8\u0dca\u0d9a\u0dbb\u0dd4\u0dc0\u0db1\u0dca\u0dc3\u0d82\u0d9b\u0dca\u0dba\u0dcf\u0dc0 </p>\n",
|
||||
"<p>optimizer </p>\n": "<p>\u0db4\u0dca\u200d\u0dbb\u0dc1\u0dc3\u0dca\u0dad\u0d9a\u0dbb\u0dab\u0dba </p>\n",
|
||||
"<p>reset the workers </p>\n": "<p>\u0d9a\u0db8\u0dca\u0d9a\u0dbb\u0dd4\u0dc0\u0db1\u0dca\u0dba\u0dc5\u0dd2 \u0db4\u0dd2\u0dc4\u0dd2\u0da7\u0dd4\u0dc0\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>size of mini batch for training </p>\n": "<p>\u0db4\u0dd4\u0dc4\u0dd4\u0dab\u0dd4\u0dc0\u0dc3\u0db3\u0dc4\u0dcf \u0d9a\u0dd4\u0da9\u0dcf \u0d9a\u0dab\u0dca\u0da9\u0dcf\u0dba\u0db8\u0dda \u0db4\u0dca\u0dbb\u0db8\u0dcf\u0dab\u0dba </p>\n",
|
||||
"<p>steps sampled on each update </p>\n": "<p>\u0d91\u0d9a\u0dca\u0d91\u0d9a\u0dca \u0dba\u0dcf\u0dc0\u0dad\u0dca\u0d9a\u0dcf\u0dbd\u0dd3\u0db1\u0dba\u0dda \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0db4\u0dd2\u0dba\u0dc0\u0dbb </p>\n",
|
||||
"<p>target model to get <span translate=no>_^_0_^_</span> </p>\n": "<p>\u0dbd\u0db6\u0dcf\u0d9c\u0dd0\u0db1\u0dd3\u0db8\u0da7 \u0d89\u0dbd\u0d9a\u0dca\u0d9a \u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba <span translate=no>_^_0_^_</span> </p>\n",
|
||||
"<p>update current observation </p>\n": "<p>\u0dc0\u0dad\u0dca\u0db8\u0db1\u0dca\u0db1\u0dd2\u0dbb\u0dd3\u0d9a\u0dca\u0dc2\u0dab \u0dba\u0dcf\u0dc0\u0dad\u0dca\u0d9a\u0dcf\u0dbd\u0dd3\u0db1 \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>update episode information. collect episode info, which is available if an episode finished; this includes total reward and length of the episode - look at <span translate=no>_^_0_^_</span> to see how it works. </p>\n": "<p>\u0d9a\u0dae\u0dcf\u0d82\u0d9c\u0dad\u0ddc\u0dbb\u0dad\u0dd4\u0dbb\u0dd4 \u0dba\u0dcf\u0dc0\u0dad\u0dca\u0d9a\u0dcf\u0dbd\u0dd3\u0db1 \u0d9a\u0dbb\u0db1\u0dca\u0db1. \u0d9a\u0dae\u0dcf\u0d82\u0d9c \u0dad\u0ddc\u0dbb\u0dad\u0dd4\u0dbb\u0dd4 \u0d91\u0d9a\u0dad\u0dd4 \u0d9a\u0dbb\u0db1\u0dca\u0db1, \u0d9a\u0dae\u0dcf\u0d82\u0d9c\u0dba\u0d9a\u0dca \u0d85\u0dc0\u0dc3\u0db1\u0dca \u0dc0\u0dd4\u0dc0\u0dc4\u0ddc\u0dad\u0dca \u0dbd\u0db6\u0dcf \u0d9c\u0dad \u0dc4\u0dd0\u0d9a\u0dd2\u0dba; \u0db8\u0dd9\u0dba\u0da7 \u0d9a\u0dae\u0dcf\u0d82\u0d9c\u0dba\u0dda \u0dc3\u0db8\u0dca\u0db4\u0dd6\u0dbb\u0dca\u0dab \u0dc0\u0dd2\u0db4\u0dcf\u0d9a\u0dba \u0dc3\u0dc4 \u0daf\u0dd2\u0d9c \u0d87\u0dad\u0dd4\u0dc5\u0dad\u0dca \u0dc0\u0dda - \u0d91\u0dba \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0dc0\u0db1 \u0d86\u0d9a\u0dcf\u0dbb\u0dba <span translate=no>_^_0_^_</span> \u0db6\u0dd0\u0dbd\u0dd3\u0db8\u0da7 \u0db6\u0dbd\u0db1\u0dca\u0db1. </p>\n",
|
||||
"<p>update target network every 250 update </p>\n": "<p>\u0dc3\u0dd1\u0db8250 \u0dba\u0dcf\u0dc0\u0dad\u0dca\u0d9a\u0dcf\u0dbd\u0dd3\u0db1 \u0d89\u0dbd\u0d9a\u0dca\u0d9a \u0da2\u0dcf\u0dbd\u0dba \u0dba\u0dcf\u0dc0\u0dad\u0dca\u0d9a\u0dcf\u0dbd\u0dd3\u0db1 </p>\n",
|
||||
"DQN Experiment with Atari Breakout": "\u0d85\u0da7\u0dcf\u0dbb\u0dd2 \u0db6\u0dca\u0dbb\u0dda\u0d9a\u0dca\u0d85\u0dc0\u0dd4\u0da7\u0dca \u0dc3\u0db8\u0d9f \u0da9\u0dd3QN \u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf \u0db6\u0dd0\u0dbd\u0dd3\u0db8",
|
||||
"Implementation of DQN experiment with Atari Breakout": "Atari Breakout \u0dc3\u0db8\u0d9f DQN \u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf \u0db6\u0dd0\u0dbd\u0dd3\u0db8 \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dd2\u0dbb\u0dd3\u0db8"
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
{
|
||||
"<h1>DQN Experiment with Atari Breakout</h1>\n<p>This experiment trains a Deep Q Network (DQN) to play Atari Breakout game on OpenAI Gym. It runs the <a href=\"../game.html\">game environments on multiple processes</a> to sample efficiently.</p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a></p>\n": "<h1>\u4f7f\u7528 Atari Breakout \u8fdb\u884c DQN \u5b9e\u9a8c</h1>\n<p>\u8be5\u5b9e\u9a8c\u8bad\u7ec3 Deep Q Network (DQN) \u5728 OpenAI Gym \u4e0a\u73a9 Atari Breakout \u6e38\u620f\u3002\u5b83\u5728<a href=\"../game.html\">\u591a\u4e2a\u8fdb\u7a0b\u4e0a\u8fd0\u884c\u6e38\u620f\u73af\u5883</a>\u4ee5\u9ad8\u6548\u91c7\u6837\u3002</p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a></p>\n",
|
||||
"<h2>Run it</h2>\n": "<h2>\u8fd0\u884c\u5b83</h2>\n",
|
||||
"<h2>Trainer</h2>\n": "<h2>\u8bad\u7ec3\u5e08</h2>\n",
|
||||
"<h3>Destroy</h3>\n<p>Stop the workers</p>\n": "<h3>\u6467\u6bc1</h3>\n<p>\u963b\u6b62\u5de5\u4eba</p>\n",
|
||||
"<h3>Run training loop</h3>\n": "<h3>\u8dd1\u6b65\u8bad\u7ec3\u5faa\u73af</h3>\n",
|
||||
"<h3>Sample data</h3>\n": "<h3>\u6837\u672c\u6570\u636e</h3>\n",
|
||||
"<h3>Train the model</h3>\n": "<h3>\u8bad\u7ec3\u6a21\u578b</h3>\n",
|
||||
"<h4><span translate=no>_^_0_^_</span>-greedy Sampling</h4>\n<p>When sampling actions we use a <span translate=no>_^_1_^_</span>-greedy strategy, where we take a greedy action with probabiliy <span translate=no>_^_2_^_</span> and take a random action with probability <span translate=no>_^_3_^_</span>. We refer to <span translate=no>_^_4_^_</span> as <span translate=no>_^_5_^_</span>.</p>\n": "<h4><span translate=no>_^_0_^_</span>-\u8d2a\u5a6a\u91c7\u6837</h4>\n\u5728\u5bf9@@ <p>\u52a8\u4f5c\u8fdb\u884c\u62bd\u6837\u65f6\uff0c\u6211\u4eec\u4f7f\u7528<span translate=no>_^_1_^_</span>-greedy\u7b56\u7565\uff0c\u5176\u4e2d\u6211\u4eec\u91c7\u53d6\u6982\u7387\u7684\u8d2a\u5a6a\u52a8\u4f5c\uff0c<span translate=no>_^_2_^_</span>\u5e76\u968f\u673a\u91c7\u53d6\u6982\u7387\u52a8\u4f5c<span translate=no>_^_3_^_</span>\u3002\u6211\u4eec\u79f0\u4e4b<span translate=no>_^_4_^_</span>\u4e3a<span translate=no>_^_5_^_</span>\u3002</p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span> for prioritized replay </p>\n": "<p><span translate=no>_^_0_^_</span>\u7528\u4e8e\u4f18\u5148\u91cd\u64ad</p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span> for replay buffer as a function of updates </p>\n": "<p><span translate=no>_^_0_^_</span>\u4f5c\u4e3a\u66f4\u65b0\u51fd\u6570\u7684\u91cd\u64ad\u7f13\u51b2\u533a</p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span>, exploration fraction </p>\n": "<p><span translate=no>_^_0_^_</span>\uff0c\u52d8\u63a2\u5206\u6570</p>\n",
|
||||
"<p>Add a new line to the screen periodically </p>\n": "<p>\u5b9a\u671f\u5728\u5c4f\u5e55\u4e0a\u6dfb\u52a0\u65b0\u884c</p>\n",
|
||||
"<p>Add transition to replay buffer </p>\n": "<p>\u5c06\u8fc7\u6e21\u6dfb\u52a0\u5230\u91cd\u64ad\u7f13\u51b2\u533a</p>\n",
|
||||
"<p>Calculate gradients </p>\n": "<p>\u8ba1\u7b97\u68af\u5ea6</p>\n",
|
||||
"<p>Calculate priorities for replay buffer <span translate=no>_^_0_^_</span> </p>\n": "<p>\u8ba1\u7b97\u91cd\u64ad\u7f13\u51b2\u533a\u7684\u4f18\u5148\u7ea7<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Clip gradients </p>\n": "<p>\u526a\u8f91\u6e10\u53d8</p>\n",
|
||||
"<p>Collect information from each worker </p>\n": "<p>\u6536\u96c6\u6bcf\u4f4d\u5458\u5de5\u7684\u4fe1\u606f</p>\n",
|
||||
"<p>Compute Temporal Difference (TD) errors, <span translate=no>_^_0_^_</span>, and the loss, <span translate=no>_^_1_^_</span>. </p>\n": "<p>\u8ba1\u7b97\u65f6\u5dee (TD) \u8bef\u5dee\u548c\u635f\u5931<span translate=no>_^_1_^_</span>\u3002<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Configurations </p>\n": "<p>\u914d\u7f6e</p>\n",
|
||||
"<p>Copy to target network initially </p>\n": "<p>\u6700\u521d\u590d\u5236\u5230\u76ee\u6807\u7f51\u7edc</p>\n",
|
||||
"<p>Create the experiment </p>\n": "<p>\u521b\u5efa\u5b9e\u9a8c</p>\n",
|
||||
"<p>Get <span translate=no>_^_0_^_</span> </p>\n": "<p>\u5f97\u5230<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Get Q_values for the current observation </p>\n": "<p>\u83b7\u53d6\u5f53\u524d\u89c2\u6d4b\u503c\u7684 Q_Values</p>\n",
|
||||
"<p>Get results after executing the actions </p>\n": "<p>\u6267\u884c\u64cd\u4f5c\u540e\u83b7\u53d6\u7ed3\u679c</p>\n",
|
||||
"<p>Get the Q-values of the next state for <a href=\"index.html\">Double Q-learning</a>. Gradients shouldn't propagate for these </p>\n": "<p>\u83b7\u53d6 \u201c<a href=\"index.html\">\u53cc Q \u5b66\u4e60\u201d \u7684\u4e0b\u4e00\u4e2a\u72b6\u6001\u7684 Q</a> \u503c\u3002\u68af\u5ea6\u4e0d\u5e94\u8be5\u4e3a\u8fd9\u4e9b\u4f20\u64ad</p>\n",
|
||||
"<p>Get the predicted Q-value </p>\n": "<p>\u83b7\u53d6\u9884\u6d4b\u7684 Q \u503c</p>\n",
|
||||
"<p>Initialize the trainer </p>\n": "<p>\u521d\u59cb\u5316\u8bad\u7ec3\u5668</p>\n",
|
||||
"<p>Last 100 episode information </p>\n": "<p>\u6700\u8fd1 100 \u96c6\u4fe1\u606f</p>\n",
|
||||
"<p>Learning rate. </p>\n": "<p>\u5b66\u4e60\u7387\u3002</p>\n",
|
||||
"<p>Mini batch size </p>\n": "<p>\u5c0f\u6279\u91cf</p>\n",
|
||||
"<p>Model for sampling and training </p>\n": "<p>\u91c7\u6837\u548c\u8bad\u7ec3\u6a21\u578b</p>\n",
|
||||
"<p>Number of epochs to train the model with sampled data. </p>\n": "\u4f7f\u7528@@ <p>\u91c7\u6837\u6570\u636e\u8bad\u7ec3\u6a21\u578b\u7684\u5468\u671f\u6570\u3002</p>\n",
|
||||
"<p>Number of steps to run on each process for a single update </p>\n": "<p>\u5355\u6b21\u66f4\u65b0\u7684\u6bcf\u4e2a\u8fdb\u7a0b\u8981\u8fd0\u884c\u7684\u6b65\u9aa4\u6570</p>\n",
|
||||
"<p>Number of updates </p>\n": "<p>\u66f4\u65b0\u6b21\u6570</p>\n",
|
||||
"<p>Number of worker processes </p>\n": "<p>\u5de5\u4f5c\u8fdb\u7a0b\u6570</p>\n",
|
||||
"<p>Periodically update target network </p>\n": "<p>\u5b9a\u671f\u66f4\u65b0\u76ee\u6807\u7f51\u7edc</p>\n",
|
||||
"<p>Pick the action based on <span translate=no>_^_0_^_</span> </p>\n": "<p>\u6839\u636e\u4ee5\u4e0b\u5185\u5bb9\u9009\u62e9\u64cd\u4f5c<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Replay buffer with <span translate=no>_^_0_^_</span>. Capacity of the replay buffer must be a power of 2. </p>\n": "\u4f7f\u7528@@ <p>\u91cd\u64ad\u7f13\u51b2\u533a<span translate=no>_^_0_^_</span>\u3002\u91cd\u64ad\u7f13\u51b2\u533a\u7684\u5bb9\u91cf\u5fc5\u987b\u662f 2 \u7684\u5e42\u3002</p>\n",
|
||||
"<p>Run and monitor the experiment </p>\n": "<p>\u8fd0\u884c\u5e76\u76d1\u63a7\u5b9e\u9a8c</p>\n",
|
||||
"<p>Run sampled actions on each worker </p>\n": "<p>\u5bf9\u6bcf\u4e2a\u5de5\u4f5c\u5668\u8fd0\u884c\u91c7\u6837\u64cd\u4f5c</p>\n",
|
||||
"<p>Sample <span translate=no>_^_0_^_</span> </p>\n": "<p>\u6837\u672c<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Sample actions </p>\n": "<p>\u64cd\u4f5c\u793a\u4f8b</p>\n",
|
||||
"<p>Sample from priority replay buffer </p>\n": "<p>\u6765\u81ea\u4f18\u5148\u7ea7\u91cd\u64ad\u7f13\u51b2\u533a\u7684\u6837\u672c</p>\n",
|
||||
"<p>Sample the action with highest Q-value. This is the greedy action. </p>\n": "<p>\u91c7\u6837\u5177\u6709\u6700\u9ad8 Q \u503c\u7684\u52a8\u4f5c\u3002\u8fd9\u662f\u8d2a\u5a6a\u7684\u884c\u52a8\u3002</p>\n",
|
||||
"<p>Sample with current policy </p>\n": "<p>\u5f53\u524d\u653f\u7b56\u7684\u793a\u4f8b</p>\n",
|
||||
"<p>Sampling doesn't need gradients </p>\n": "<p>\u91c7\u6837\u4e0d\u9700\u8981\u6e10\u53d8</p>\n",
|
||||
"<p>Save tracked indicators. </p>\n": "<p>\u4fdd\u5b58\u8ddf\u8e2a\u7684\u6307\u6807\u3002</p>\n",
|
||||
"<p>Scale observations from <span translate=no>_^_0_^_</span> to <span translate=no>_^_1_^_</span> </p>\n": "<p>\u5c06\u89c2\u6d4b\u503c\u4ece\u7f29\u653e<span translate=no>_^_0_^_</span>\u5230<span translate=no>_^_1_^_</span></p>\n",
|
||||
"<p>Select device </p>\n": "<p>\u9009\u62e9\u8bbe\u5907</p>\n",
|
||||
"<p>Set learning rate </p>\n": "<p>\u8bbe\u7f6e\u5b66\u4e60\u901f\u7387</p>\n",
|
||||
"<p>Start training after the buffer is full </p>\n": "<p>\u7f13\u51b2\u533a\u6ee1\u540e\u5f00\u59cb\u8bad\u7ec3</p>\n",
|
||||
"<p>Stop the workers </p>\n": "<p>\u963b\u6b62\u5de5\u4eba</p>\n",
|
||||
"<p>Target model updating interval </p>\n": "<p>\u76ee\u6807\u6a21\u578b\u66f4\u65b0\u95f4\u9694</p>\n",
|
||||
"<p>This doesn't need gradients </p>\n": "<p>\u8fd9\u4e0d\u9700\u8981\u6e10\u53d8</p>\n",
|
||||
"<p>Train the model </p>\n": "<p>\u8bad\u7ec3\u6a21\u578b</p>\n",
|
||||
"<p>Uniformly sample and action </p>\n": "<p>\u7edf\u4e00\u91c7\u6837\u548c\u884c\u52a8</p>\n",
|
||||
"<p>Update parameters based on gradients </p>\n": "<p>\u6839\u636e\u6e10\u53d8\u66f4\u65b0\u53c2\u6570</p>\n",
|
||||
"<p>Update replay buffer priorities </p>\n": "<p>\u66f4\u65b0\u91cd\u64ad\u7f13\u51b2\u533a\u4f18\u5148\u7ea7</p>\n",
|
||||
"<p>Whether to chose greedy action or the random action </p>\n": "<p>\u9009\u62e9\u8d2a\u5a6a\u52a8\u4f5c\u8fd8\u662f\u968f\u673a\u52a8\u4f5c</p>\n",
|
||||
"<p>Zero out the previously calculated gradients </p>\n": "<p>\u5c06\u5148\u524d\u8ba1\u7b97\u7684\u68af\u5ea6\u5f52\u96f6</p>\n",
|
||||
"<p>create workers </p>\n": "<p>\u521b\u5efa\u5de5\u4f5c\u4eba\u5458</p>\n",
|
||||
"<p>exploration as a function of updates </p>\n": "<p>\u4f5c\u4e3a\u66f4\u65b0\u51fd\u6570\u7684\u63a2\u7d22</p>\n",
|
||||
"<p>get the initial observations </p>\n": "<p>\u83b7\u5f97\u521d\u6b65\u89c2\u6d4b\u503c</p>\n",
|
||||
"<p>initialize tensors for observations </p>\n": "<p>\u521d\u59cb\u5316\u89c2\u6d4b\u503c\u7684\u5f20\u91cf</p>\n",
|
||||
"<p>learning rate </p>\n": "<p>\u5b66\u4e60\u7387</p>\n",
|
||||
"<p>loss function </p>\n": "<p>\u635f\u5931\u51fd\u6570</p>\n",
|
||||
"<p>number of training iterations </p>\n": "<p>\u8bad\u7ec3\u8fed\u4ee3\u6b21\u6570</p>\n",
|
||||
"<p>number of updates </p>\n": "<p>\u66f4\u65b0\u6b21\u6570</p>\n",
|
||||
"<p>number of workers </p>\n": "<p>\u5de5\u4f5c\u4eba\u5458\u4eba\u6570</p>\n",
|
||||
"<p>optimizer </p>\n": "<p>\u4f18\u5316\u8005</p>\n",
|
||||
"<p>reset the workers </p>\n": "<p>\u91cd\u7f6e\u5de5\u4f5c\u4eba\u5458</p>\n",
|
||||
"<p>size of mini batch for training </p>\n": "<p>\u7528\u4e8e\u8bad\u7ec3\u7684\u5fae\u578b\u6279\u6b21\u7684\u5927\u5c0f</p>\n",
|
||||
"<p>steps sampled on each update </p>\n": "<p>\u6bcf\u6b21\u66f4\u65b0\u65f6\u91c7\u6837\u7684\u6b65\u9aa4</p>\n",
|
||||
"<p>target model to get <span translate=no>_^_0_^_</span> </p>\n": "<p>\u8981\u83b7\u53d6\u7684\u76ee\u6807\u6a21\u578b<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>update current observation </p>\n": "<p>\u66f4\u65b0\u5f53\u524d\u89c2\u6d4b\u503c</p>\n",
|
||||
"<p>update episode information. collect episode info, which is available if an episode finished; this includes total reward and length of the episode - look at <span translate=no>_^_0_^_</span> to see how it works. </p>\n": "<p>\u66f4\u65b0\u5267\u96c6\u4fe1\u606f\u3002\u6536\u96c6\u5267\u96c6\u4fe1\u606f\uff0c\u5982\u679c\u5267\u96c6\u7ed3\u675f\u5219\u53ef\u7528\uff1b\u8fd9\u5305\u62ec\u603b\u5956\u52b1\u548c\u5267\u96c6\u65f6\u957f\u2014\u2014\u770b\u770b<span translate=no>_^_0_^_</span>\u5b83\u662f\u5982\u4f55\u8fd0\u4f5c\u7684\u3002</p>\n",
|
||||
"<p>update target network every 250 update </p>\n": "<p>\u6bcf 250 \u6b21\u66f4\u65b0\u4e00\u6b21\u76ee\u6807\u7f51\u7edc</p>\n",
|
||||
"DQN Experiment with Atari Breakout": "\u4f7f\u7528 Atari Breakout \u8fdb\u884c DQN \u5b9e",
|
||||
"Implementation of DQN experiment with Atari Breakout": "\u4f7f\u7528 Atari Breakout \u5b9e\u65bd DQN \u5b9e\u9a8c"
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"<h1>Deep Q Network (DQN) Model</h1>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a></p>\n": "<h1>\u30c7\u30a3\u30fc\u30d7Q\u30cd\u30c3\u30c8\u30ef\u30fc\u30af (DQN) \u30e2\u30c7\u30eb</h1>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a></p>\n",
|
||||
"<h2>Dueling Network \u2694\ufe0f Model for <span translate=no>_^_0_^_</span> Values</h2>\n<p>We are using a <a href=\"https://arxiv.org/abs/1511.06581\">dueling network</a> to calculate Q-values. Intuition behind dueling network architecture is that in most states the action doesn't matter, and in some states the action is significant. Dueling network allows this to be represented very well.</p>\n<span translate=no>_^_1_^_</span><p>So we create two networks for <span translate=no>_^_2_^_</span> and <span translate=no>_^_3_^_</span> and get <span translate=no>_^_4_^_</span> from them. <span translate=no>_^_5_^_</span> We share the initial layers of the <span translate=no>_^_6_^_</span> and <span translate=no>_^_7_^_</span> networks.</p>\n": "<h2>\u30c7\u30e5\u30a8\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af \u2694\ufe0f \u4fa1\u5024\u30e2\u30c7\u30eb <span translate=no>_^_0_^_</span></h2>\n<p><a href=\"https://arxiv.org/abs/1511.06581\">Q\u5024\u306e\u8a08\u7b97\u306b\u306f\u30c7\u30e5\u30a8\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3092\u4f7f\u7528\u3057\u3066\u3044\u307e\u3059</a>\u3002\u30c7\u30e5\u30a8\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u30a2\u30fc\u30ad\u30c6\u30af\u30c1\u30e3\u306e\u80cc\u5f8c\u306b\u3042\u308b\u76f4\u611f\u306f\u3001\u307b\u3068\u3093\u3069\u306e\u5dde\u3067\u306f\u30a2\u30af\u30b7\u30e7\u30f3\u306f\u91cd\u8981\u3067\u306f\u306a\u304f\u3001\u4e00\u90e8\u306e\u5dde\u3067\u306f\u30a2\u30af\u30b7\u30e7\u30f3\u304c\u91cd\u8981\u3067\u3042\u308b\u3068\u3044\u3046\u3053\u3068\u3067\u3059\u3002\u30c7\u30e5\u30a8\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3067\u306f\u3001\u3053\u308c\u3092\u975e\u5e38\u306b\u3088\u304f\u8868\u73fe\u3067\u304d\u307e\u3059</p>\u3002\n<span translate=no>_^_1_^_</span><p>\u305d\u3053\u3067\u3001<span translate=no>_^_2_^_</span><span translate=no>_^_3_^_</span>\u3068\u304b\u3089\u306e 2 \u3064\u306e\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3092\u4f5c\u6210\u3057\u3066\u3001\u305d\u306e 2 <span translate=no>_^_4_^_</span> \u3064\u306e\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u304b\u3089\u53d6\u5f97\u3057\u307e\u3059\u3002<span translate=no>_^_5_^_</span><span translate=no>_^_6_^_</span><span translate=no>_^_7_^_</span>\u3068\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u306e\u521d\u671f\u30ec\u30a4\u30e4\u30fc\u3092\u5171\u6709\u3057\u307e\u3059\u3002</p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span> </p>\n": "<p><span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>A fully connected layer takes the flattened frame from third convolution layer, and outputs <span translate=no>_^_0_^_</span> features </p>\n": "<p>\u5b8c\u5168\u306b\u63a5\u7d9a\u3055\u308c\u305f\u30ec\u30a4\u30e4\u30fc\u306f\u30013 \u756a\u76ee\u306e\u30b3\u30f3\u30dc\u30ea\u30e5\u30fc\u30b7\u30e7\u30f3\u30ec\u30a4\u30e4\u30fc\u304b\u3089\u30d5\u30e9\u30c3\u30c8\u5316\u3055\u308c\u305f\u30d5\u30ec\u30fc\u30e0\u3092\u53d6\u308a\u51fa\u3057\u3001\u30d5\u30a3\u30fc\u30c1\u30e3\u3092\u51fa\u529b\u3057\u307e\u3059\u3002<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Convolution </p>\n": "<p>\u30b3\u30f3\u30dc\u30ea\u30e5\u30fc\u30b7\u30e7\u30f3</p>\n",
|
||||
"<p>Linear layer </p>\n": "<p>\u30ea\u30cb\u30a2\u30ec\u30a4\u30e4\u30fc</p>\n",
|
||||
"<p>Reshape for linear layers </p>\n": "<p>\u7dda\u5f62\u30ec\u30a4\u30e4\u30fc\u306e\u5f62\u72b6\u3092\u5909\u66f4</p>\n",
|
||||
"<p>The first convolution layer takes a <span translate=no>_^_0_^_</span> frame and produces a <span translate=no>_^_1_^_</span> frame </p>\n": "<p><span translate=no>_^_0_^_</span>\u6700\u521d\u306e\u7573\u307f\u8fbc\u307f\u5c64\u306f\u30d5\u30ec\u30fc\u30e0\u3092\u53d6\u308a\u3001\u30d5\u30ec\u30fc\u30e0\u3092\u751f\u6210\u3057\u307e\u3059\u3002<span translate=no>_^_1_^_</span></p>\n",
|
||||
"<p>The second convolution layer takes a <span translate=no>_^_0_^_</span> frame and produces a <span translate=no>_^_1_^_</span> frame </p>\n": "<p>2 \u756a\u76ee\u306e\u7573\u307f\u8fbc\u307f\u5c64\u306f\u3001<span translate=no>_^_0_^_</span>\u30d5\u30ec\u30fc\u30e0\u3092\u53d6\u5f97\u3057\u3066\u30d5\u30ec\u30fc\u30e0\u3092\u751f\u6210\u3057\u307e\u3059\u3002<span translate=no>_^_1_^_</span></p>\n",
|
||||
"<p>The third convolution layer takes a <span translate=no>_^_0_^_</span> frame and produces a <span translate=no>_^_1_^_</span> frame </p>\n": "<p>3 \u756a\u76ee\u306e\u7573\u307f\u8fbc\u307f\u5c64\u306f\u3001<span translate=no>_^_0_^_</span>\u30d5\u30ec\u30fc\u30e0\u3092\u53d6\u5f97\u3057\u3066\u30d5\u30ec\u30fc\u30e0\u3092\u751f\u6210\u3057\u307e\u3059\u3002<span translate=no>_^_1_^_</span></p>\n",
|
||||
"<p>This head gives the action value <span translate=no>_^_0_^_</span> </p>\n": "<p>\u3053\u306e\u30d8\u30c3\u30c9\u306f\u30a2\u30af\u30b7\u30e7\u30f3\u5024\u3092\u4e0e\u3048\u307e\u3059 <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>This head gives the state value <span translate=no>_^_0_^_</span> </p>\n": "<p>\u3053\u306e\u30d8\u30c3\u30c9\u306f\u72b6\u614b\u5024\u3092\u4e0e\u3048\u307e\u3059 <span translate=no>_^_0_^_</span></p>\n",
|
||||
"Deep Q Network (DQN) Model": "\u30c7\u30a3\u30fc\u30d7Q\u30cd\u30c3\u30c8\u30ef\u30fc\u30af (DQN) \u30e2\u30c7\u30eb",
|
||||
"Implementation of neural network model for Deep Q Network (DQN).": "\u30c7\u30a3\u30fc\u30d7Q\u30cd\u30c3\u30c8\u30ef\u30fc\u30af (DQN) \u7528\u306e\u30cb\u30e5\u30fc\u30e9\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u30e2\u30c7\u30eb\u306e\u5b9f\u88c5\u3002"
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"<h1>Deep Q Network (DQN) Model</h1>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a> <a href=\"https://app.labml.ai/run/fe1ad986237511ec86e8b763a2d3f710\"><span translate=no>_^_1_^_</span></a></p>\n": "<h1>\u0d9c\u0dd0\u0db9\u0dd4\u0dbb\u0dd4Q \u0da2\u0dcf\u0dbd (DQN) \u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba</h1>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a> <a href=\"https://app.labml.ai/run/fe1ad986237511ec86e8b763a2d3f710\"> <span translate=no>_^_1_^_</span></a></p>\n",
|
||||
"<h2>Dueling Network \u2694\ufe0f Model for <span translate=no>_^_0_^_</span> Values</h2>\n<p>We are using a <a href=\"https://arxiv.org/abs/1511.06581\">dueling network</a> to calculate Q-values. Intuition behind dueling network architecture is that in most states the action doesn't matter, and in some states the action is significant. Dueling network allows this to be represented very well.</p>\n<span translate=no>_^_1_^_</span><p>So we create two networks for <span translate=no>_^_2_^_</span> and <span translate=no>_^_3_^_</span> and get <span translate=no>_^_4_^_</span> from them. <span translate=no>_^_5_^_</span> We share the initial layers of the <span translate=no>_^_6_^_</span> and <span translate=no>_^_7_^_</span> networks.</p>\n": "<h2>\u0da2\u0dcf\u0dbd\u0dbaDueling \u2694\ufe0f <span translate=no>_^_0_^_</span> \u0dc0\u0da7\u0dd2\u0db1\u0dcf\u0d9a\u0db8\u0dca \u0dc3\u0db3\u0dc4\u0dcf \u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba</h2>\n<p>Q-\u0d85\u0d9c\u0dba\u0db1\u0dca\u0d9c\u0dab\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0d85\u0db4\u0dd2 <a href=\"https://arxiv.org/abs/1511.06581\">\u0da9\u0dd6\u0dbd\u0dd2\u0d82 \u0da2\u0dcf\u0dbd\u0dba\u0d9a\u0dca</a> \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dbb\u0db8\u0dd4. \u0da2\u0dcf\u0dbd \u0d9c\u0dd8\u0dc4 \u0db1\u0dd2\u0dbb\u0dca\u0db8\u0dcf\u0dab \u0dc1\u0dd2\u0dbd\u0dca\u0db4\u0dba dueling \u0db4\u0dd2\u0da7\u0dd4\u0db4\u0dc3 \u0d87\u0dad\u0dd2 \u0db4\u0dca\u0dbb\u0dad\u0dd2\u0db7\u0dcf\u0db1\u0dba \u0db1\u0db8\u0dca, \u0db6\u0ddc\u0dc4\u0ddd \u0db4\u0dca\u0dbb\u0dcf\u0db1\u0dca\u0dad\u0dc0\u0dbd \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dc0 \u0dc0\u0dd0\u0daf\u0d9c\u0dad\u0dca \u0db1\u0ddc\u0dc0\u0db1 \u0d85\u0dad\u0dbb \u0dc3\u0db8\u0dc4\u0dbb \u0db4\u0dca\u0dbb\u0dcf\u0db1\u0dca\u0dad\u0dc0\u0dbd \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dc0 \u0dc3\u0dd0\u0dbd\u0d9a\u0dd2\u0dba \u0dba\u0dd4\u0dad\u0dd4 \u0dba. Dueling \u0da2\u0dcf\u0dbd\u0dba \u0db8\u0dd9\u0dba \u0d89\u0dad\u0dcf \u0dc4\u0ddc\u0db3\u0dd2\u0db1\u0dca \u0db1\u0dd2\u0dbb\u0dd6\u0db4\u0dab\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0da7 \u0d89\u0da9 \u0daf\u0dd9\u0dba\u0dd2. </p>\n<span translate=no>_^_1_^_</span><p>\u0d91\u0db6\u0dd0\u0dc0\u0dd2\u0db1\u0dca\u0d85\u0db4\u0dd2 \u0da2\u0dcf\u0dbd \u0daf\u0dd9\u0d9a\u0d9a\u0dca \u0db1\u0dd2\u0dbb\u0dca\u0db8\u0dcf\u0dab\u0dba <span translate=no>_^_2_^_</span> <span translate=no>_^_3_^_</span> \u0d9a\u0dbb <span translate=no>_^_4_^_</span> \u0d94\u0dc0\u0dd4\u0db1\u0dca\u0d9c\u0dd9\u0db1\u0dca \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dd2\u0db8\u0dd4. <span translate=no>_^_5_^_</span> \u0d85\u0db4\u0dd2 <span translate=no>_^_6_^_</span> \u0dc3\u0dc4 <span translate=no>_^_7_^_</span> \u0da2\u0dcf\u0dbd \u0dc0\u0dbd \u0d86\u0dbb\u0db8\u0dca\u0db7\u0d9a \u0dc3\u0dca\u0dae\u0dbb \u0db6\u0dd9\u0daf\u0dcf \u0d9c\u0db1\u0dd2\u0db8\u0dd4. </p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span> </p>\n": "<p><span translate=no>_^_0_^_</span> </p>\n",
|
||||
"<p>A fully connected layer takes the flattened frame from third convolution layer, and outputs <span translate=no>_^_0_^_</span> features </p>\n": "<p>\u0dc3\u0db8\u0dca\u0db4\u0dd4\u0dbb\u0dca\u0dab\u0dba\u0dd9\u0db1\u0dca\u0db8\u0dc3\u0db8\u0dca\u0db6\u0db1\u0dca\u0db0\u0dd2\u0dad \u0dad\u0da7\u0dca\u0da7\u0dd4\u0dc0\u0d9a\u0dca \u0db4\u0dd0\u0dad\u0dbd\u0dd2 \u0dbb\u0dcf\u0db8\u0dd4\u0dc0 \u0dad\u0dd9\u0dc0\u0db1 \u0d9a\u0dd0\u0da7\u0dd2 \u0d9c\u0dd0\u0dc3\u0dd4\u0dab\u0dd4 \u0dc3\u0dca\u0dae\u0dbb\u0dba\u0dd9\u0db1\u0dca \u0d9c\u0db1\u0dca\u0db1\u0dcf \u0d85\u0dad\u0dbb <span translate=no>_^_0_^_</span> \u0dc0\u0dd2\u0dc1\u0dda\u0dc2\u0dcf\u0d82\u0d9c \u0db4\u0dca\u0dbb\u0dad\u0dd2\u0daf\u0dcf\u0db1\u0dba \u0d9a\u0dbb\u0dba\u0dd2 </p>\n",
|
||||
"<p>Convolution </p>\n": "<p>\u0dc3\u0d82\u0dc0\u0dbd\u0dd2\u0dad </p>\n",
|
||||
"<p>Linear layer </p>\n": "<p>\u0dbb\u0dda\u0d9b\u0dd3\u0dba\u0dc3\u0dca\u0dae\u0dbb\u0dba </p>\n",
|
||||
"<p>Reshape for linear layers </p>\n": "<p>\u0dbb\u0dda\u0d9b\u0dd3\u0dba\u0dc3\u0dca\u0dae\u0dbb \u0dc3\u0db3\u0dc4\u0dcf \u0db1\u0dd0\u0dc0\u0dad \u0dc3\u0d9a\u0dc3\u0dca \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>The first convolution layer takes a <span translate=no>_^_0_^_</span> frame and produces a <span translate=no>_^_1_^_</span> frame </p>\n": "<p>\u0db4\u0dc5\u0db8\u0dd4\u0d9a\u0dd0\u0da7\u0dd2 \u0d9c\u0dd0\u0dc3\u0dd4\u0dab\u0dd4 \u0dc3\u0dca\u0dad\u0dbb\u0dba <span translate=no>_^_0_^_</span> \u0dbb\u0dcf\u0db8\u0dd4\u0dc0\u0d9a\u0dca \u0d9c\u0dd9\u0db1 <span translate=no>_^_1_^_</span> \u0dbb\u0dcf\u0db8\u0dd4\u0dc0\u0d9a\u0dca \u0db1\u0dd2\u0db4\u0daf\u0dc0\u0dba\u0dd2 </p>\n",
|
||||
"<p>The second convolution layer takes a <span translate=no>_^_0_^_</span> frame and produces a <span translate=no>_^_1_^_</span> frame </p>\n": "<p>\u0daf\u0dd9\u0dc0\u0db1\u0d9a\u0dd0\u0da7\u0dd2 \u0d9c\u0dd0\u0dc3\u0dd4\u0dab\u0dd4 \u0dc3\u0dca\u0dad\u0dbb\u0dba <span translate=no>_^_0_^_</span> \u0dbb\u0dcf\u0db8\u0dd4\u0dc0\u0d9a\u0dca \u0d9c\u0dd9\u0db1 <span translate=no>_^_1_^_</span> \u0dbb\u0dcf\u0db8\u0dd4\u0dc0\u0d9a\u0dca \u0db1\u0dd2\u0db4\u0daf\u0dc0\u0dba\u0dd2 </p>\n",
|
||||
"<p>The third convolution layer takes a <span translate=no>_^_0_^_</span> frame and produces a <span translate=no>_^_1_^_</span> frame </p>\n": "<p>\u0dad\u0dd9\u0dc0\u0db1\u0d9a\u0dd0\u0da7\u0dd2 \u0d9c\u0dd0\u0dc3\u0dd4\u0dab\u0dd4 \u0dc3\u0dca\u0dad\u0dbb\u0dba <span translate=no>_^_0_^_</span> \u0dbb\u0dcf\u0db8\u0dd4\u0dc0\u0d9a\u0dca \u0d9c\u0dd9\u0db1 <span translate=no>_^_1_^_</span> \u0dbb\u0dcf\u0db8\u0dd4\u0dc0\u0d9a\u0dca \u0db1\u0dd2\u0db4\u0daf\u0dc0\u0dba\u0dd2 </p>\n",
|
||||
"<p>This head gives the action value <span translate=no>_^_0_^_</span> </p>\n": "<p>\u0db8\u0dd9\u0db8\u0dc4\u0dd2\u0dc3 \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0d9a\u0dcf\u0dbb\u0dd3 \u0d85\u0d9c\u0dba \u0dbd\u0db6\u0dcf \u0daf\u0dd9\u0dba\u0dd2 <span translate=no>_^_0_^_</span> </p>\n",
|
||||
"<p>This head gives the state value <span translate=no>_^_0_^_</span> </p>\n": "<p>\u0db8\u0dd9\u0db8\u0dc4\u0dd2\u0dc3 \u0dbb\u0dcf\u0da2\u0dca\u0dba \u0dc0\u0da7\u0dd2\u0db1\u0dcf\u0d9a\u0db8 \u0dbd\u0db6\u0dcf \u0daf\u0dd9\u0dba\u0dd2 <span translate=no>_^_0_^_</span> </p>\n",
|
||||
"Deep Q Network (DQN) Model": "\u0d9c\u0dd0\u0db9\u0dd4\u0dbb\u0dd4 Q \u0da2\u0dcf\u0dbd (DQN) \u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba",
|
||||
"Implementation of neural network model for Deep Q Network (DQN).": "Deep Q Network (DQN) \u0dc3\u0db3\u0dc4\u0dcf \u0dc3\u0dca\u0db1\u0dcf\u0dba\u0dd4\u0d9a \u0da2\u0dcf\u0dbd \u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dd2\u0dbb\u0dd3\u0db8."
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
{
|
||||
"<h1>Deep Q Network (DQN) Model</h1>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a></p>\n": "<h1>\u6df1\u5ea6 Q \u7f51\u7edc (DQN) \u6a21\u578b</h1>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a></p>\n",
|
||||
"<h2>Dueling Network \u2694\ufe0f Model for <span translate=no>_^_0_^_</span> Values</h2>\n<p>We are using a <a href=\"https://arxiv.org/abs/1511.06581\">dueling network</a> to calculate Q-values. Intuition behind dueling network architecture is that in most states the action doesn't matter, and in some states the action is significant. Dueling network allows this to be represented very well.</p>\n<span translate=no>_^_1_^_</span><p>So we create two networks for <span translate=no>_^_2_^_</span> and <span translate=no>_^_3_^_</span> and get <span translate=no>_^_4_^_</span> from them. <span translate=no>_^_5_^_</span> We share the initial layers of the <span translate=no>_^_6_^_</span> and <span translate=no>_^_7_^_</span> networks.</p>\n": "<h2>\u51b3\u6597\u7f51\u7edc \u2694\ufe0f<span translate=no>_^_0_^_</span> \u4ef7\u503c\u89c2\u6a21\u578b</h2>\n<p>\u6211\u4eec\u6b63\u5728\u4f7f\u7528\u51b3<a href=\"https://arxiv.org/abs/1511.06581\">\u6597\u7f51\u7edc</a>\u6765\u8ba1\u7b97 Q \u503c\u3002\u51b3\u6597\u7f51\u7edc\u67b6\u6784\u80cc\u540e\u7684\u76f4\u89c9\u662f\uff0c\u5728\u5927\u591a\u6570\u5dde\uff0c\u884c\u52a8\u65e0\u5173\u7d27\u8981\uff0c\u800c\u5728\u67d0\u4e9b\u5dde\uff0c\u884c\u52a8\u610f\u4e49\u91cd\u5927\u3002\u51b3\u6597\u7f51\u7edc\u53ef\u4ee5\u5f88\u597d\u5730\u4f53\u73b0\u8fd9\u4e00\u70b9\u3002</p>\n<span translate=no>_^_1_^_</span><p>\u56e0\u6b64\uff0c\u6211\u4eec\u4e3a<span translate=no>_^_2_^_</span>\u548c\u521b\u5efa\u4e86\u4e24\u4e2a\u7f51\u7edc\uff0c<span translate=no>_^_3_^_</span>\u7136\u540e<span translate=no>_^_4_^_</span>\u4ece\u4e2d\u83b7\u53d6\u3002<span translate=no>_^_5_^_</span>\u6211\u4eec\u5171\u4eab<span translate=no>_^_6_^_</span>\u548c<span translate=no>_^_7_^_</span>\u7f51\u7edc\u7684\u521d\u59cb\u5c42\u3002</p>\n",
|
||||
"<p><span translate=no>_^_0_^_</span> </p>\n": "<p><span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>A fully connected layer takes the flattened frame from third convolution layer, and outputs <span translate=no>_^_0_^_</span> features </p>\n": "<p>\u5b8c\u5168\u8fde\u63a5\u7684\u56fe\u5c42\u4ece\u7b2c\u4e09\u4e2a\u5377\u79ef\u56fe\u5c42\u83b7\u53d6\u5c55\u5e73\u7684\u5e27\uff0c\u5e76\u8f93\u51fa<span translate=no>_^_0_^_</span>\u8981\u7d20</p>\n",
|
||||
"<p>Convolution </p>\n": "<p>\u5377\u79ef</p>\n",
|
||||
"<p>Linear layer </p>\n": "<p>\u7ebf\u6027\u5c42</p>\n",
|
||||
"<p>Reshape for linear layers </p>\n": "<p>\u7ebf\u6027\u56fe\u5c42\u7684\u6574\u5f62</p>\n",
|
||||
"<p>The first convolution layer takes a <span translate=no>_^_0_^_</span> frame and produces a <span translate=no>_^_1_^_</span> frame </p>\n": "<p>\u7b2c\u4e00\u4e2a\u5377\u79ef\u5c42\u9700\u8981\u4e00\u4e2a<span translate=no>_^_0_^_</span>\u5e27\u5e76\u751f\u6210\u4e00\u4e2a<span translate=no>_^_1_^_</span>\u5e27</p>\n",
|
||||
"<p>The second convolution layer takes a <span translate=no>_^_0_^_</span> frame and produces a <span translate=no>_^_1_^_</span> frame </p>\n": "<p>\u7b2c\u4e8c\u4e2a\u5377\u79ef\u5c42\u83b7\u53d6\u4e00\u4e2a<span translate=no>_^_0_^_</span>\u5e27\u5e76\u751f\u6210\u4e00\u4e2a<span translate=no>_^_1_^_</span>\u5e27</p>\n",
|
||||
"<p>The third convolution layer takes a <span translate=no>_^_0_^_</span> frame and produces a <span translate=no>_^_1_^_</span> frame </p>\n": "<p>\u7b2c\u4e09\u4e2a\u5377\u79ef\u5c42\u83b7\u53d6\u4e00\u4e2a<span translate=no>_^_0_^_</span>\u5e27\u5e76\u751f\u6210\u4e00\u4e2a<span translate=no>_^_1_^_</span>\u5e27</p>\n",
|
||||
"<p>This head gives the action value <span translate=no>_^_0_^_</span> </p>\n": "<p>\u8fd9\u4e2a\u5934\u7ed9\u51fa\u4e86\u52a8\u4f5c\u503c<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>This head gives the state value <span translate=no>_^_0_^_</span> </p>\n": "<p>\u8fd9\u4e2a\u5934\u7ed9\u51fa\u4e86\u72b6\u6001\u503c<span translate=no>_^_0_^_</span></p>\n",
|
||||
"Deep Q Network (DQN) Model": "\u6df1\u5ea6\u95ee\u7f51\u7edc (DQN) \u6a21\u578b",
|
||||
"Implementation of neural network model for Deep Q Network (DQN).": "\u6df1\u5ea6\u95ee\u7f51\u7edc (DQN) \u795e\u7ecf\u7f51\u7edc\u6a21\u578b\u7684\u5b9e\u73b0\u3002"
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"<h1><a href=\"https://nn.labml.ai/rl/dqn/index.html\">Deep Q Networks (DQN)</a></h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of paper <a href=\"https://arxiv.org/abs/1312.5602\">Playing Atari with Deep Reinforcement Learning</a> along with <a href=\"https://nn.labml.ai/rl/dqn/model.html\">Dueling Network</a>, <a href=\"https://nn.labml.ai/rl/dqn/replay_buffer.html\">Prioritized Replay</a> and Double Q Network.</p>\n<p>Here is the <a href=\"https://nn.labml.ai/rl/dqn/experiment.html\">experiment</a> and <a href=\"https://nn.labml.ai/rl/dqn/model.html\">model</a> implementation.</p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a> </p>\n": "<h1><a href=\"https://nn.labml.ai/rl/dqn/index.html\">\u30c7\u30a3\u30fc\u30d7Q\u30cd\u30c3\u30c8\u30ef\u30fc\u30af (DQN)</a></h1>\n<p>\u3053\u308c\u306f\u3001<a href=\"https://arxiv.org/abs/1312.5602\">\u30c7\u30a3\u30fc\u30d7\u5f37\u5316\u5b66\u7fd2\u3092\u4f7f\u3063\u305f\u30a2\u30bf\u30ea\u30d7\u30ec\u30a4\u3068\u30c7\u30e5\u30a8\u30eb\u30cd\u30c3\u30c8\u30ef\u30fc\u30af</a><a href=\"https://nn.labml.ai/rl/dqn/model.html\">\u3001<a href=\"https://nn.labml.ai/rl/dqn/replay_buffer.html\">\u512a\u5148\u30ea\u30d7\u30ec\u30a4</a>\u3001<a href=\"https://pytorch.org\">\u30c0\u30d6\u30ebQ\u30cd\u30c3\u30c8\u30ef\u30fc\u30af\u3092PyTorch\u3067\u5b9f\u88c5\u3057\u305f\u3082\u306e\u3067\u3059</a></a>\u3002</p>\n<p><a href=\"https://nn.labml.ai/rl/dqn/experiment.html\"><a href=\"https://nn.labml.ai/rl/dqn/model.html\">\u3053\u308c\u304c\u5b9f\u9a13\u3068\u30e2\u30c7\u30eb\u306e\u5b9f\u88c5\u3067\u3059</a></a>\u3002</p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a></p>\n",
|
||||
"Deep Q Networks (DQN)": "\u30c7\u30a3\u30fc\u30d7Q\u30cd\u30c3\u30c8\u30ef\u30fc\u30af (DQN)"
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"<h1><a href=\"https://nn.labml.ai/rl/dqn/index.html\">Deep Q Networks (DQN)</a></h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of paper <a href=\"https://arxiv.org/abs/1312.5602\">Playing Atari with Deep Reinforcement Learning</a> along with <a href=\"https://nn.labml.ai/rl/dqn/model.html\">Dueling Network</a>, <a href=\"https://nn.labml.ai/rl/dqn/replay_buffer.html\">Prioritized Replay</a> and Double Q Network.</p>\n<p>Here is the <a href=\"https://nn.labml.ai/rl/dqn/experiment.html\">experiment</a> and <a href=\"https://nn.labml.ai/rl/dqn/model.html\">model</a> implementation.</p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a> <a href=\"https://app.labml.ai/run/fe1ad986237511ec86e8b763a2d3f710\"><span translate=no>_^_1_^_</span></a> </p>\n": "<h1><a href=\"https://nn.labml.ai/rl/dqn/index.html\">\u0d9c\u0dd0\u0db9\u0dd4\u0dbb\u0dd4 Q \u0da2\u0dcf\u0dbd (DQN)</a></h1>\n<p>\u0db8\u0dd9\u0dba <a href=\"https://pytorch.org\">PyTorch</a> \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0d9a\u0dd2 \u0d9a\u0da9\u0daf\u0dcf\u0dc3\u0dd2 <a href=\"https://arxiv.org/abs/1312.5602\">\u0dc3\u0dd9\u0dbd\u0dca\u0dbd\u0db8\u0dca \u0d85\u0da7\u0dcf\u0dbb\u0dd2 \u0d9c\u0dd0\u0db9\u0dd4\u0dbb\u0dd4 \u0dc1\u0d9a\u0dca\u0dad\u0dd2\u0db8\u0dad\u0dca \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0dda \u0d89\u0d9c\u0dd9\u0db1\u0dd3\u0db8</a> \u0dc3\u0dc4 <a href=\"https://nn.labml.ai/rl/dqn/model.html\">\u0da9\u0dd4\u0dbd\u0dd2\u0d82 \u0da2\u0dcf\u0dbd\u0dba</a> \u0dc3\u0db8\u0d9f, <a href=\"https://nn.labml.ai/rl/dqn/replay_buffer.html\">\u0db4\u0dca\u0dbb\u0db8\u0dd4\u0d9b\u0dad\u0dcf \u0db1\u0dd0\u0dc0\u0dad \u0db0\u0dcf\u0dc0\u0db1\u0dba</a> \u0dc3\u0dc4 \u0daf\u0dca\u0dc0\u0dd2\u0dad\u0dca\u0dc0 Q \u0da2\u0dcf\u0dbd\u0dba. </p>\n<p>\u0db8\u0dd9\u0db1\u0dca\u0db1 <a href=\"https://nn.labml.ai/rl/dqn/experiment.html\">\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf</a> \u0db6\u0dd0\u0dbd\u0dd3\u0db8 \u0dc3\u0dc4 <a href=\"https://nn.labml.ai/rl/dqn/model.html\">\u0d86\u0daf\u0dbb\u0dca\u0dc1</a> \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dd2\u0dbb\u0dd3\u0db8. </p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a> <a href=\"https://app.labml.ai/run/fe1ad986237511ec86e8b763a2d3f710\"> <span translate=no>_^_1_^_</span></a> </p>\n",
|
||||
"Deep Q Networks (DQN)": "\u0d9c\u0dd0\u0db9\u0dd4\u0dbb\u0dd4 Q \u0da2\u0dcf\u0dbd (DQN)"
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"<h1><a href=\"https://nn.labml.ai/rl/dqn/index.html\">Deep Q Networks (DQN)</a></h1>\n<p>This is a <a href=\"https://pytorch.org\">PyTorch</a> implementation of paper <a href=\"https://arxiv.org/abs/1312.5602\">Playing Atari with Deep Reinforcement Learning</a> along with <a href=\"https://nn.labml.ai/rl/dqn/model.html\">Dueling Network</a>, <a href=\"https://nn.labml.ai/rl/dqn/replay_buffer.html\">Prioritized Replay</a> and Double Q Network.</p>\n<p>Here is the <a href=\"https://nn.labml.ai/rl/dqn/experiment.html\">experiment</a> and <a href=\"https://nn.labml.ai/rl/dqn/model.html\">model</a> implementation.</p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a> </p>\n": "<h1><a href=\"https://nn.labml.ai/rl/dqn/index.html\">\u6df1\u5ea6 Q \u7f51\u7edc (DQN)</a></h1>\n<p>\u8fd9\u662f <a href=\"https://pytorch.org\">PyTorch</a> \u5b9e\u73b0\u7684 PyTorch <a href=\"https://arxiv.org/abs/1312.5602\">\u4f7f\u7528\u6df1\u5ea6\u5f3a\u5316\u5b66\u4e60\u73a9\u96c5</a>\u8fbe\u5229\u4ee5\u53ca<a href=\"https://nn.labml.ai/rl/dqn/model.html\">\u51b3\u6597\u7f51\u7edc</a>\u3001<a href=\"https://nn.labml.ai/rl/dqn/replay_buffer.html\">\u4f18\u5148\u56de\u653e</a>\u548c Double Q Network\u3002</p>\n<p>\u8fd9\u662f<a href=\"https://nn.labml.ai/rl/dqn/experiment.html\">\u5b9e\u9a8c</a>\u548c<a href=\"https://nn.labml.ai/rl/dqn/model.html\">\u6a21\u578b</a>\u5b9e\u73b0\u3002</p>\n<p><a href=\"https://colab.research.google.com/github/labmlai/annotated_deep_learning_paper_implementations/blob/master/labml_nn/rl/dqn/experiment.ipynb\"><span translate=no>_^_0_^_</span></a></p>\n",
|
||||
"Deep Q Networks (DQN)": "\u6df1\u5ea6\u95ee\u7b54\u7f51\u7edc (DQN)"
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user