This is the PyTorch implementation of optimizer introduced in the paper Attention Is All You Need.
\n": "\u3053\u308c\u306f\u3001\u8ad6\u6587\u300c\u5fc5\u8981\u306a\u306e\u306f\u6ce8\u610f\u3060\u3051\u300d\u3067\u7d39\u4ecb\u3055\u308c\u3066\u3044\u308b\u30aa\u30d7\u30c6\u30a3\u30de\u30a4\u30b6\u30fc\u306ePyTorch\u5b9f\u88c5\u3067\u3059\u3002
\n", "This class extends from Adam optimizer defined in _^_0_^_.
\n": "\u3053\u306e\u30af\u30e9\u30b9\u306f\u3001\u3067\u5b9a\u7fa9\u3055\u308c\u3066\u3044\u308b Adam \u30aa\u30d7\u30c6\u30a3\u30de\u30a4\u30b6\u3092\u62e1\u5f35\u3057\u305f\u3082\u306e\u3067\u3059\u3002_^_0_^_
\n", "_^_0_^_ where _^_1_^_ is the number of warmup steps.
\n": "_^_0_^__^_1_^_\u306f\u30a6\u30a9\u30fc\u30e0\u30a2\u30c3\u30d7\u30b9\u30c6\u30c3\u30d7\u306e\u6570\u3067\u3059\u3002
\n", "_^_0_^_
\n": "_^_0_^_
\n", "_^_0_^_
\n": "_^_0_^_
\n", "Noam optimizer from Attention is All You Need paper": "\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30fb\u30a4\u30ba\u30fb\u30aa\u30fc\u30eb\u30fb\u30e6\u30fc\u30fb\u30cb\u30fc\u30c9\u8ad6\u6587\u306e Noam \u30aa\u30d7\u30c6\u30a3\u30de\u30a4\u30b6\u30fc", "This is a tutorial/implementation of Noam optimizer. Noam optimizer has a warm-up period and then an exponentially decaying learning rate.": "\u3053\u308c\u306fNoam\u30aa\u30d7\u30c6\u30a3\u30de\u30a4\u30b6\u30fc\u306e\u30c1\u30e5\u30fc\u30c8\u30ea\u30a2\u30eb/\u5b9f\u88c5\u3067\u3059\u3002Noam Optimizer\u306b\u306f\u30a6\u30a9\u30fc\u30e0\u30a2\u30c3\u30d7\u671f\u9593\u304c\u3042\u308a\u3001\u305d\u306e\u5f8c\u306f\u5b66\u7fd2\u7387\u304c\u6307\u6570\u95a2\u6570\u7684\u306b\u4f4e\u4e0b\u3057\u307e\u3059\u3002" }