{ "

Generate Text with GPT-NeoX

\n

This shows how to generate text from GPT-NeoX with a single GPU.

\n

This needs a GPU with more than 45GB memory.

\n": "

GPT-\u30cd\u30aa\u30c3\u30af\u30b9\u3067\u30c6\u30ad\u30b9\u30c8\u3092\u751f\u6210

\n

\u3053\u308c\u306f\u3001\u5358\u4e00\u306eGPU\u3067GPT-Neox\u304b\u3089\u30c6\u30ad\u30b9\u30c8\u3092\u751f\u6210\u3059\u308b\u65b9\u6cd5\u3092\u793a\u3057\u3066\u3044\u307e\u3059\u3002

\n

\u3053\u308c\u306b\u306f\u300145 GB \u4ee5\u4e0a\u306e\u30e1\u30e2\u30ea\u3092\u642d\u8f09\u3057\u305f GPU \u304c\u5fc5\u8981\u3067\u3059\u3002

\n", "

Generate text

\n": "

\u30c6\u30ad\u30b9\u30c8\u3092\u751f\u6210

\n", "

Predict the next token

\n\n": "

\u6b21\u306e\u30c8\u30fc\u30af\u30f3\u3092\u4e88\u6e2c

\n\n", "

\n": "

\n", "

Append the predicted token

\n": "

\u4e88\u6e2c\u30c8\u30fc\u30af\u30f3\u3092\u8ffd\u52a0

\n", "

Device

\n": "

\u7aef\u672b

\n", "

Eval model

\n": "

\u8a55\u4fa1\u30e2\u30c7\u30eb

\n", "

Get next token. Note that we only feed the last token to the model because we cache the key/value pairs of previous tokens.

\n": "

\u6b21\u306e\u30c8\u30fc\u30af\u30f3\u3092\u5165\u624b\u3057\u3066\u304f\u3060\u3055\u3044\u3002\u4ee5\u524d\u306e\u30c8\u30fc\u30af\u30f3\u306e\u30ad\u30fc\u3068\u5024\u306e\u30da\u30a2\u3092\u30ad\u30e3\u30c3\u30b7\u30e5\u3059\u308b\u306e\u3067\u3001\u6700\u5f8c\u306e\u30c8\u30fc\u30af\u30f3\u306e\u307f\u3092\u30e2\u30c7\u30eb\u306b\u30d5\u30a3\u30fc\u30c9\u3059\u308b\u3053\u3068\u306b\u6ce8\u610f\u3057\u3066\u304f\u3060\u3055\u3044

\u3002\n", "

Get the tokens

\n": "

\u30c8\u30fc\u30af\u30f3\u3092\u5165\u624b

\n", "

Get token ids

\n": "

\u30c8\u30fc\u30af\u30f3 ID \u3092\u53d6\u5f97

\n", "

Imports

\n": "

\u8f38\u5165

\n", "

List of layers to load. This is used for testing. You can assign a subset of layers like _^_0_^_ so that it only loads the first to transformer layers.

\n": "

\u30ed\u30fc\u30c9\u3059\u308b\u30ec\u30a4\u30e4\u30fc\u306e\u30ea\u30b9\u30c8\u3002\u3053\u308c\u306f\u30c6\u30b9\u30c8\u306b\u4f7f\u7528\u3055\u308c\u307e\u3059\u3002_^_0_^_\u306e\u3088\u3046\u306b\u30ec\u30a4\u30e4\u30fc\u306e\u30b5\u30d6\u30bb\u30c3\u30c8\u3092\u5272\u308a\u5f53\u3066\u3066\u3001\u6700\u521d\u306e\u30ec\u30a4\u30e4\u30fc\u306e\u307f\u3092\u30c8\u30e9\u30f3\u30b9\u30d5\u30a9\u30fc\u30de\u30fc\u30ec\u30a4\u30e4\u30fc\u306b\u8aad\u307f\u8fbc\u3080\u3053\u3068\u304c\u3067\u304d\u307e\u3059

\u3002\n", "

Load layers

\n": "

\u30ec\u30a4\u30e4\u30fc\u3092\u30ed\u30fc\u30c9

\n", "

Predict 100 tokens

\n": "

\u30c8\u30fc\u30af\u30f3\u3092100\u500b\u4e88\u6e2c\u3059\u308b

\n", "

Print

\n": "

\u30d7\u30ea\u30f3\u30c8

\n", "

Prompt to complete

\n": "

\u5b8c\u4e86\u3092\u4fc3\u3059\u30d7\u30ed\u30f3\u30d7\u30c8

\n", "

Return predicted token

\n": "

\u4e88\u6e2c\u30c8\u30fc\u30af\u30f3\u3092\u8fd4\u3059

\n", "

Run the model

\n": "

\u30e2\u30c7\u30eb\u3092\u5b9f\u884c

\n", "

Set the state to use cached activations

\n": "

\u30ad\u30e3\u30c3\u30b7\u30e5\u3055\u308c\u305f\u30a2\u30af\u30c6\u30a3\u30d9\u30fc\u30b7\u30e7\u30f3\u3092\u4f7f\u7528\u3059\u308b\u3088\u3046\u306b\u72b6\u614b\u3092\u8a2d\u5b9a\u3057\u307e\u3059

\n", "

Setup cache to cache intermediate key/value pairs for faster generation

\n": "

\u751f\u6210\u3092\u9ad8\u901f\u5316\u3059\u308b\u305f\u3081\u306b\u4e2d\u9593\u30ad\u30fc\u3068\u5024\u306e\u30da\u30a2\u3092\u30ad\u30e3\u30c3\u30b7\u30e5\u3059\u308b\u3088\u3046\u306b\u30ad\u30e3\u30c3\u30b7\u30e5\u3092\u8a2d\u5b9a

\n", "Generate Text with GPT-NeoX": "GPT-\u30cd\u30aa\u30c3\u30af\u30b9\u3067\u30c6\u30ad\u30b9\u30c8\u3092\u751f\u6210" }