{ "
This shows how to generate text from GPT-NeoX with a single GPU.
\nThis needs a GPU with more than 45GB memory.
\n": "\u8fd9\u8bf4\u660e\u4e86\u5982\u4f55\u4f7f\u7528\u5355\u4e2a GPU \u4ece GPT-NEOX \u751f\u6210\u6587\u672c\u3002
\n\u8fd9\u9700\u8981\u4e00\u4e2a\u5185\u5b58\u8d85\u8fc745GB\u7684GPU\u3002
\n", "\n": "\n", "
Append the predicted token
\n": "\u8ffd\u52a0\u9884\u6d4b\u7684\u4ee4\u724c
\n", "Device
\n": "\u8bbe\u5907
\n", "Eval model
\n": "\u8bc4\u4f30\u6a21\u578b
\n", "Get next token. Note that we only feed the last token to the model because we cache the key/value pairs of previous tokens.
\n": "\u83b7\u53d6\u4e0b\u4e00\u4e2a\u4ee4\u724c\u3002\u8bf7\u6ce8\u610f\uff0c\u6211\u4eec\u53ea\u5c06\u6700\u540e\u4e00\u4e2a\u4ee4\u724c\u63d0\u4f9b\u7ed9\u6a21\u578b\uff0c\u56e0\u4e3a\u6211\u4eec\u7f13\u5b58\u4e86\u5148\u524d\u4ee4\u724c\u7684\u952e/\u503c\u5bf9\u3002
\n", "Get the tokens
\n": "\u83b7\u53d6\u4ee3\u5e01
\n", "Get token ids
\n": "\u83b7\u53d6\u4ee3\u5e01 ID
\n", "Imports
\n": "\u8fdb\u53e3
\n", "List of layers to load. This is used for testing. You can assign a subset of layers like _^_0_^_ so that it only loads the first to transformer layers.
\n": "\u8981\u52a0\u8f7d\u7684\u56fe\u5c42\u5217\u8868\u3002\u8fd9\u7528\u4e8e\u6d4b\u8bd5\u3002\u60a8\u53ef\u4ee5\u5c06\u5c42\u7684\u5b50\u96c6\u5206\u914d\u7ed9\u53d8\u538b\u5668\u5c42\uff0c_^_0_^_\u4f7f\u5176\u4ec5\u5c06\u7b2c\u4e00\u4e2a\u5c42\u52a0\u8f7d\u5230\u53d8\u538b\u5668\u5c42\u3002
\n", "Load layers
\n": "\u52a0\u8f7d\u56fe\u5c42
\n", "Predict 100 tokens
\n": "\u9884\u6d4b 100 \u4e2a\u4ee3\u5e01
\n", "\u6253\u5370
\n", "Prompt to complete
\n": "\u63d0\u793a\u5b8c\u6210
\n", "Return predicted token
\n": "\u8fd4\u56de\u9884\u6d4b\u7684\u4ee3\u5e01
\n", "Run the model
\n": "\u8fd0\u884c\u6a21\u578b
\n", "Set the state to use cached activations
\n": "\u8bbe\u7f6e\u72b6\u6001\u4ee5\u4f7f\u7528\u7f13\u5b58\u7684\u6fc0\u6d3b
\n", "Setup cache to cache intermediate key/value pairs for faster generation
\n": "\u8bbe\u7f6e\u7f13\u5b58\u4ee5\u7f13\u5b58\u4e2d\u95f4\u952e/\u503c\u5bf9\u4ee5\u52a0\u5feb\u751f\u6210\u901f\u5ea6
\n", "Generate Text with GPT-NeoX": "\u4f7f\u7528 GPT-NEOX \u751f\u6210\u6587\u672c" }