This extends AMSGrad optimizer and adds a warmup stage.
\n": "\u3053\u308c\u306b\u3088\u308a AMSgrad \u30aa\u30d7\u30c6\u30a3\u30de\u30a4\u30b6\u304c\u62e1\u5f35\u3055\u308c\u3001\u30a6\u30a9\u30fc\u30e0\u30a2\u30c3\u30d7\u30b9\u30c6\u30fc\u30b8\u304c\u8ffd\u52a0\u3055\u308c\u307e\u3059\u3002
\n", "This class extends from AMSGrad optimizer defined in _^_0_^_.
\n": "\u3053\u306e\u30af\u30e9\u30b9\u306f\u3001\u3067\u5b9a\u7fa9\u3055\u308c\u3066\u3044\u308b AMSGrad \u30aa\u30d7\u30c6\u30a3\u30de\u30a4\u30b6\u3092\u62e1\u5f35\u3057\u305f\u3082\u306e\u3067\u3059\u3002_^_0_^_
\n", "_^_0_^_ where _^_1_^_ is the number of warmup steps.
\n": "_^_0_^__^_1_^_\u306f\u30a6\u30a9\u30fc\u30e0\u30a2\u30c3\u30d7\u30b9\u30c6\u30c3\u30d7\u306e\u6570\u3067\u3059\u3002
\n", "A linearly increasing learning rate from _^_0_^_ to _^_1_^_
\n": "\u5b66\u7fd2\u7387\u304c 1 \u304b\u3089 _^_0_^_ 1 \u306b\u76f4\u7dda\u7684\u306b\u5897\u52a0\u3057\u3066\u3044\u308b _^_1_^_
\n", "Constant learning rate _^_0_^_
\n": "\u4e00\u5b9a\u306e\u5b66\u7fd2\u7387 _^_0_^_
\n", "If we are in warmup stage
\n": "\u30a6\u30a9\u30fc\u30e0\u30a2\u30c3\u30d7\u6bb5\u968e\u306e\u5834\u5408
\n", "A simple PyTorch implementation/tutorial of Adam optimizer with warm-up.": "\u30a6\u30a9\u30fc\u30e0\u30a2\u30c3\u30d7\u6a5f\u80fd\u4ed8\u304d\u306e Adam \u30aa\u30d7\u30c6\u30a3\u30de\u30a4\u30b6\u30fc\u306e\u7c21\u5358\u306a PyTorch \u5b9f\u88c5/\u30c1\u30e5\u30fc\u30c8\u30ea\u30a2\u30eb\u3002", "Adam optimizer with warm-up": "\u30a6\u30a9\u30fc\u30e0\u30a2\u30c3\u30d7\u6a5f\u80fd\u4ed8\u304d Adam \u30aa\u30d7\u30c6\u30a3\u30de\u30a4\u30b6\u30fc" }