{ "

Generate Text with GPT-NeoX

\n

This shows how to generate text from GPT-NeoX with a single GPU.

\n

This needs a GPU with more than 45GB memory.

\n": "

\u4f7f\u7528 GPT-NEOX \u751f\u6210\u6587\u672c

\n

\u8fd9\u8bf4\u660e\u4e86\u5982\u4f55\u4f7f\u7528\u5355\u4e2a GPU \u4ece GPT-NEOX \u751f\u6210\u6587\u672c\u3002

\n

\u8fd9\u9700\u8981\u4e00\u4e2a\u5185\u5b58\u8d85\u8fc745GB\u7684GPU\u3002

\n", "

Generate text

\n": "

\u751f\u6210\u6587\u672c

\n", "

Predict the next token

\n\n": "

\u9884\u6d4b\u4e0b\u4e00\u4e2a\u4ee3\u5e01

\n\n", "

\n": "

\n", "

Append the predicted token

\n": "

\u8ffd\u52a0\u9884\u6d4b\u7684\u4ee4\u724c

\n", "

Device

\n": "

\u8bbe\u5907

\n", "

Eval model

\n": "

\u8bc4\u4f30\u6a21\u578b

\n", "

Get next token. Note that we only feed the last token to the model because we cache the key/value pairs of previous tokens.

\n": "

\u83b7\u53d6\u4e0b\u4e00\u4e2a\u4ee4\u724c\u3002\u8bf7\u6ce8\u610f\uff0c\u6211\u4eec\u53ea\u5c06\u6700\u540e\u4e00\u4e2a\u4ee4\u724c\u63d0\u4f9b\u7ed9\u6a21\u578b\uff0c\u56e0\u4e3a\u6211\u4eec\u7f13\u5b58\u4e86\u5148\u524d\u4ee4\u724c\u7684\u952e/\u503c\u5bf9\u3002

\n", "

Get the tokens

\n": "

\u83b7\u53d6\u4ee3\u5e01

\n", "

Get token ids

\n": "

\u83b7\u53d6\u4ee3\u5e01 ID

\n", "

Imports

\n": "

\u8fdb\u53e3

\n", "

List of layers to load. This is used for testing. You can assign a subset of layers like _^_0_^_ so that it only loads the first to transformer layers.

\n": "

\u8981\u52a0\u8f7d\u7684\u56fe\u5c42\u5217\u8868\u3002\u8fd9\u7528\u4e8e\u6d4b\u8bd5\u3002\u60a8\u53ef\u4ee5\u5c06\u5c42\u7684\u5b50\u96c6\u5206\u914d\u7ed9\u53d8\u538b\u5668\u5c42\uff0c_^_0_^_\u4f7f\u5176\u4ec5\u5c06\u7b2c\u4e00\u4e2a\u5c42\u52a0\u8f7d\u5230\u53d8\u538b\u5668\u5c42\u3002

\n", "

Load layers

\n": "

\u52a0\u8f7d\u56fe\u5c42

\n", "

Predict 100 tokens

\n": "

\u9884\u6d4b 100 \u4e2a\u4ee3\u5e01

\n", "

Print

\n": "

\u6253\u5370

\n", "

Prompt to complete

\n": "

\u63d0\u793a\u5b8c\u6210

\n", "

Return predicted token

\n": "

\u8fd4\u56de\u9884\u6d4b\u7684\u4ee3\u5e01

\n", "

Run the model

\n": "

\u8fd0\u884c\u6a21\u578b

\n", "

Set the state to use cached activations

\n": "

\u8bbe\u7f6e\u72b6\u6001\u4ee5\u4f7f\u7528\u7f13\u5b58\u7684\u6fc0\u6d3b

\n", "

Setup cache to cache intermediate key/value pairs for faster generation

\n": "

\u8bbe\u7f6e\u7f13\u5b58\u4ee5\u7f13\u5b58\u4e2d\u95f4\u952e/\u503c\u5bf9\u4ee5\u52a0\u5feb\u751f\u6210\u901f\u5ea6

\n", "Generate Text with GPT-NeoX": "\u4f7f\u7528 GPT-NEOX \u751f\u6210\u6587\u672c" }