{
 "<h1>Fine Tune GPT-NeoX</h1>\n<p>This shows how to fine tune GPT-NeoX with pipeline parallelism.</p>\n": "<h1>\u30d5\u30a1\u30a4\u30f3\u30c1\u30e5\u30fc\u30f3 GPT-\u30cd\u30aa\u30c3\u30af\u30b9</h1>\n<p>\u3053\u308c\u306f\u3001\u30d1\u30a4\u30d7\u30e9\u30a4\u30f3\u4e26\u5217\u51e6\u7406\u3067GPT-Neox\u3092\u5fae\u8abf\u6574\u3059\u308b\u65b9\u6cd5\u3092\u793a\u3057\u3066\u3044\u307e\u3059\u3002</p>\n",
 "<h3>Create fine tuner for biases</h3>\n": "<h3>\u30d0\u30a4\u30a2\u30b9\u306e\u5fae\u8abf\u6574\u5668\u306e\u4f5c\u6210</h3>\n",
 "<h3>Create pipeline parallel model</h3>\n": "<h3>\u30d1\u30a4\u30d7\u30e9\u30a4\u30f3\u4e26\u5217\u30e2\u30c7\u30eb\u306e\u4f5c\u6210</h3>\n",
 "<h3>Load GPT-NeoX layers</h3>\n": "<h3>GPT-Neox \u30ec\u30a4\u30e4\u30fc\u3092\u30ed\u30fc\u30c9</h3>\n",
 "<h4>Tiny Shakespeare dataset</h4>\n": "<h4>\u5c0f\u3055\u306a\u30b7\u30a7\u30a4\u30af\u30b9\u30d4\u30a2\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8</h4>\n",
 "<p> </p>\n": "<p></p>\n",
 "<p>Create Fairscale Pipe module </p>\n": "<p>\u30d5\u30a7\u30a2\u30b9\u30b1\u30fc\u30eb\u30d1\u30a4\u30d7\u30e2\u30b8\u30e5\u30fc\u30eb\u3092\u4f5c\u6210</p>\n",
 "<p>Create experiment </p>\n": "<p>\u5b9f\u9a13\u3092\u4f5c\u6210</p>\n",
 "<p>Create the Pipe module </p>\n": "<p>\u30d1\u30a4\u30d7\u30e2\u30b8\u30e5\u30fc\u30eb\u3092\u4f5c\u6210\u3059\u308b</p>\n",
 "<p>Devices for each GPU </p>\n": "<p>\u5404 GPU \u306e\u30c7\u30d0\u30a4\u30b9</p>\n",
 "<p>Get the layer distribution across GPUs </p>\n": "<p>GPU \u5168\u4f53\u306e\u30ec\u30a4\u30e4\u30fc\u5206\u5e03\u3092\u53d6\u5f97</p>\n",
 "<p>Initialize configs </p>\n": "<p>\u30b3\u30f3\u30d5\u30a3\u30b0\u3092\u521d\u671f\u5316</p>\n",
 "<p>Initialize the model. Do this before the loop for cleaner logs. </p>\n": "<p>\u30e2\u30c7\u30eb\u3092\u521d\u671f\u5316\u3057\u307e\u3059\u3002\u3053\u308c\u3092\u30eb\u30fc\u30d7\u306e\u524d\u306b\u884c\u3063\u3066\u3001\u30ed\u30b0\u3092\u30af\u30ea\u30fc\u30f3\u30a2\u30c3\u30d7\u3057\u3066\u304f\u3060\u3055\u3044\u3002</p>\n",
 "<p>Make sure the finetuner is initialized </p>\n": "<p>\u30d5\u30a1\u30a4\u30f3\u30c1\u30e5\u30fc\u30ca\u30fc\u304c\u521d\u671f\u5316\u3055\u308c\u3066\u3044\u308b\u3053\u3068\u3092\u78ba\u8a8d\u3057\u3066\u304f\u3060\u3055\u3044</p>\n",
 "<p>Mark biases as trainable </p>\n": "<p>\u30d0\u30a4\u30a2\u30b9\u3092\u30c8\u30ec\u30fc\u30cb\u30f3\u30b0\u53ef\u80fd\u3068\u30de\u30fc\u30af\u3059\u308b</p>\n",
 "<p>Start the experiment </p>\n": "<p>\u5b9f\u9a13\u3092\u59cb\u3081\u308b</p>\n",
 "<p>Train </p>\n": "<p>\u5217\u8eca</p>\n",
 "Fine Tune GPT-NeoX": "\u30d5\u30a1\u30a4\u30f3\u30c1\u30e5\u30fc\u30f3 GPT-\u30cd\u30aa\u30c3\u30af\u30b9",
 "Fine tune GPT-NeoX biases with Fairscale pipeline parallel module": "\u30d5\u30a7\u30a2\u30b9\u30b1\u30fc\u30eb\u30fb\u30d1\u30a4\u30d7\u30e9\u30a4\u30f3\u30fb\u30d1\u30e9\u30ec\u30eb\u30fb\u30e2\u30b8\u30e5\u30fc\u30eb\u306b\u3088\u308bGPT-Neox\u30d0\u30a4\u30a2\u30b9\u306e\u5fae\u8abf\u6574"
}