This extends AMSGrad optimizer and adds a warmup stage.
\n": "\u8fd9\u6269\u5c55\u4e86 AmsGrad \u4f18\u5316\u5668\u5e76\u589e\u52a0\u4e86\u9884\u70ed\u9636\u6bb5\u3002
\n", "This class extends from AMSGrad optimizer defined in _^_0_^_.
\n": "\u8fd9\u4e2a\u7c7b\u662f\u4ece\u4e2d\u5b9a\u4e49\u7684 AmsGrad \u4f18\u5316\u5668\u6269\u5c55\u800c\u6765\u7684_^_0_^_\u3002
\n", "_^_0_^_ where _^_1_^_ is the number of warmup steps.
\n": "_^_0_^_\u5176\u4e2d_^_1_^_\u662f\u9884\u70ed\u6b65\u9aa4\u7684\u6570\u91cf\u3002
\n", "A linearly increasing learning rate from _^_0_^_ to _^_1_^_
\n": "\u5b66\u4e60\u7387\u4ece\u7ebf\u6027\u589e\u52a0_^_0_^_\u5230_^_1_^_
\n", "Constant learning rate _^_0_^_
\n": "\u6301\u7eed\u7684\u5b66\u4e60\u901f\u7387_^_0_^_
\n", "If we are in warmup stage
\n": "\u5982\u679c\u6211\u4eec\u5904\u4e8e\u70ed\u8eab\u9636\u6bb5
\n", "A simple PyTorch implementation/tutorial of Adam optimizer with warm-up.": "\u4e00\u4e2a\u7b80\u5355\u7684 PyTorch \u5b9e\u73b0/\u6559\u7a0b\u7684 Adam \u4f18\u5316\u5668\uff0c\u5e26\u6709\u9884\u70ed\u529f\u80fd\u3002", "Adam optimizer with warm-up": "\u5e26\u70ed\u8eab\u529f\u80fd\u7684 Adam \u4f18\u5316\u5668" }