chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"<h1>Sampling Techniques for Language Models</h1>\n<ul><li><a href=\"greedy.html\">Greedy Sampling</a> </li>\n<li><a href=\"temperature.html\">Temperature Sampling</a> </li>\n<li><a href=\"top_k.html\">Top-k Sampling</a> </li>\n<li><a href=\"nucleus.html\">Nucleus Sampling</a></li></ul>\n<p>Here's an <a href=\"experiment.html\">experiment</a> that uses these sampling techniques.</p>\n": "<h1>\u8a00\u8a9e\u30e2\u30c7\u30eb\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u624b\u6cd5</h1>\n<ul><li><a href=\"greedy.html\">\u6b32\u5f35\u308a\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</a></li>\n<li><a href=\"temperature.html\">\u6e29\u5ea6\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</a></li>\n<li><a href=\"top_k.html\">\u30c8\u30c3\u30d7k\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</a></li>\n<li><a href=\"nucleus.html\">\u6838\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</a></li></ul>\n<p>\u3053\u308c\u306f\u3001<a href=\"experiment.html\">\u3053\u308c\u3089\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u624b\u6cd5\u3092\u4f7f\u7528\u3057\u305f\u5b9f\u9a13\u3067\u3059</a>\u3002</p>\n",
|
||||
"<h3>Sample from logits</h3>\n<ul><li><span translate=no>_^_0_^_</span> are the logits of the distribution of shape <span translate=no>_^_1_^_</span></li></ul>\n": "<h3>\u30ed\u30b8\u30c3\u30c8\u304b\u3089\u306e\u30b5\u30f3\u30d7\u30eb</h3>\n<ul><li><span translate=no>_^_0_^_</span>\u5f62\u72b6\u5206\u5e03\u306e\u30ed\u30b8\u30c3\u30c8\u3067\u3059 <span translate=no>_^_1_^_</span></li></ul>\n",
|
||||
"<h3>Sampler base class</h3>\n": "<h3>\u30b5\u30f3\u30d7\u30e9\u30fc\u57fa\u672c\u30af\u30e9\u30b9</h3>\n",
|
||||
"A set of PyTorch implementations/tutorials of sampling techniques for language models.": "\u8a00\u8a9e\u30e2\u30c7\u30eb\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u624b\u6cd5\u306ePyTorch\u5b9f\u88c5/\u30c1\u30e5\u30fc\u30c8\u30ea\u30a2\u30eb\u306e\u30bb\u30c3\u30c8\u3002",
|
||||
"Sampling Techniques for Language Models": "\u8a00\u8a9e\u30e2\u30c7\u30eb\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u624b\u6cd5"
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"<h1>Sampling Techniques for Language Models</h1>\n<ul><li><a href=\"greedy.html\">Greedy Sampling</a> </li>\n<li><a href=\"temperature.html\">Temperature Sampling</a> </li>\n<li><a href=\"top_k.html\">Top-k Sampling</a> </li>\n<li><a href=\"nucleus.html\">Nucleus Sampling</a></li></ul>\n<p>Here's an <a href=\"experiment.html\">experiment</a> that uses these sampling techniques.</p>\n": "<h1>\u0db7\u0dcf\u0dc2\u0dcf\u0d86\u0d9a\u0dd8\u0dad\u0dd2 \u0dc3\u0db3\u0dc4\u0dcf \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0d9a\u0dca\u0dbb\u0db8</h1>\n<ul><li><a href=\"greedy.html\">\u0d9a\u0dd1\u0daf\u0dbb \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8</a> </li>\n<li><a href=\"temperature.html\">\u0d8b\u0dc2\u0dca\u0dab\u0dad\u0dca\u0dc0 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8</a> </li>\n<li><a href=\"top_k.html\">\u0d89\u0dc4\u0dc5-K \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8\u0dca</a> </li>\n<li><a href=\"nucleus.html\">\u0db1\u0dca\u0dba\u0dc2\u0dca\u0da7\u0dd2\u0d9a \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8</a></li></ul>\n<p>\u0db8\u0dd9\u0db1\u0dca\u0db1\u0db8\u0dd9\u0db8 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0dc1\u0dd2\u0dbd\u0dca\u0db4\u0dd3\u0dba \u0d9a\u0dca\u0dbb\u0db8 \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dbb\u0db1 <a href=\"experiment.html\">\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf \u0db6\u0dd0\u0dbd\u0dd3\u0db8\u0d9a\u0dca</a> . </p>\n",
|
||||
"<h3>Sample from logits</h3>\n<ul><li><span translate=no>_^_0_^_</span> are the logits of the distribution of shape <span translate=no>_^_1_^_</span></li></ul>\n": "<h3>\u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca\u0dc0\u0dbd\u0dd2\u0db1\u0dca \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba</h3>\n<ul><li><span translate=no>_^_0_^_</span> \u0dc4\u0dd0\u0da9\u0dba \u0db6\u0dd9\u0daf\u0dcf \u0dc4\u0dd0\u0dbb\u0dd3\u0db8\u0dda \u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca \u0dc0\u0dda <span translate=no>_^_1_^_</span></li></ul>\n",
|
||||
"<h3>Sampler base class</h3>\n": "<h3>\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0db8\u0dd6\u0dbd\u0dd2\u0d9a \u0db4\u0db1\u0dca\u0dad\u0dd2\u0dba</h3>\n",
|
||||
"A set of PyTorch implementations/tutorials of sampling techniques for language models.": "\u0db7\u0dcf\u0dc2\u0dcf \u0d86\u0d9a\u0dd8\u0dad\u0dd2 \u0dc3\u0db3\u0dc4\u0dcf \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0dc1\u0dd2\u0dbd\u0dca\u0db4\u0dd3\u0dba \u0d9a\u0dca\u0dbb\u0db8 PyTorch \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dd2\u0dbb\u0dd3\u0db8/\u0db1\u0dd2\u0db6\u0db1\u0dca\u0db0\u0db1 \u0db8\u0dcf\u0dbd\u0dcf\u0dc0\u0d9a\u0dca.",
|
||||
"Sampling Techniques for Language Models": "\u0db7\u0dcf\u0dc2\u0dcf \u0d86\u0d9a\u0dd8\u0dad\u0dd2 \u0dc3\u0db3\u0dc4\u0dcf \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0d9a\u0dca\u0dbb\u0db8"
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"<h1>Sampling Techniques for Language Models</h1>\n<ul><li><a href=\"greedy.html\">Greedy Sampling</a> </li>\n<li><a href=\"temperature.html\">Temperature Sampling</a> </li>\n<li><a href=\"top_k.html\">Top-k Sampling</a> </li>\n<li><a href=\"nucleus.html\">Nucleus Sampling</a></li></ul>\n<p>Here's an <a href=\"experiment.html\">experiment</a> that uses these sampling techniques.</p>\n": "<h1>\u8bed\u8a00\u6a21\u578b\u7684\u91c7\u6837\u6280\u672f</h1>\n<ul><li><a href=\"greedy.html\">\u8d2a\u5a6a\u91c7\u6837</a></li>\n<li><a href=\"temperature.html\">\u6e29\u5ea6\u91c7\u6837</a></li>\n<li><a href=\"top_k.html\">\u524d k \u4e2a\u91c7\u6837</a></li>\n<li><a href=\"nucleus.html\">\u539f\u5b50\u6838\u91c7\u6837</a></li></ul>\n<p>\u8fd9\u662f\u4e00\u4e2a\u4f7f\u7528\u8fd9\u4e9b\u91c7\u6837\u6280\u672f\u7684<a href=\"experiment.html\">\u5b9e\u9a8c</a>\u3002</p>\n",
|
||||
"<h3>Sample from logits</h3>\n<ul><li><span translate=no>_^_0_^_</span> are the logits of the distribution of shape <span translate=no>_^_1_^_</span></li></ul>\n": "<h3>\u6765\u81ea logits \u7684\u6837\u672c</h3>\n<ul><li><span translate=no>_^_0_^_</span>\u662f\u5f62\u72b6\u5206\u5e03\u7684\u5bf9\u6570<span translate=no>_^_1_^_</span></li></ul>\n",
|
||||
"<h3>Sampler base class</h3>\n": "<h3>\u91c7\u6837\u5668\u57fa\u7c7b</h3>\n",
|
||||
"A set of PyTorch implementations/tutorials of sampling techniques for language models.": "\u4e00\u5957\u8bed\u8a00\u6a21\u578b\u91c7\u6837\u6280\u672f\u7684 PyTorch \u5b9e\u73b0/\u6559\u7a0b\u3002",
|
||||
"Sampling Techniques for Language Models": "\u8bed\u8a00\u6a21\u578b\u7684\u91c7\u6837\u6280\u672f"
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"<h1>Trying out Sampling Techniques for Language Models</h1>\n<ul><li><a href=\"greedy.html\">Greedy Sampling</a> </li>\n<li><a href=\"temperature.html\">Temperature Sampling</a> </li>\n<li><a href=\"top_k.html\">Top-k Sampling</a> </li>\n<li><a href=\"nucleus.html\">Nucleus Sampling</a></li></ul>\n<p>This experiment uses the above sampling techniques, on HuggingFace's GPT2 model.</p>\n": "<h1>\u8a00\u8a9e\u30e2\u30c7\u30eb\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u624b\u6cd5\u306e\u8a66\u307f</h1>\n<ul><li><a href=\"greedy.html\">\u6b32\u5f35\u308a\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</a></li>\n<li><a href=\"temperature.html\">\u6e29\u5ea6\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</a></li>\n<li><a href=\"top_k.html\">\u30c8\u30c3\u30d7k\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</a></li>\n<li><a href=\"nucleus.html\">\u6838\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</a></li></ul>\n<p>\u3053\u306e\u5b9f\u9a13\u3067\u306f\u3001HuggingFace\u306eGPT2\u30e2\u30c7\u30eb\u3067\u4e0a\u8a18\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u624b\u6cd5\u3092\u4f7f\u7528\u3057\u3066\u3044\u307e\u3059\u3002</p>\n",
|
||||
"<h2>Sample from model</h2>\n<ul><li><span translate=no>_^_0_^_</span> is the model to sample from </li>\n<li><span translate=no>_^_1_^_</span> is the tokenizer to use </li>\n<li><span translate=no>_^_2_^_</span> is the sampler to use </li>\n<li><span translate=no>_^_3_^_</span> is the number of samples to generate </li>\n<li><span translate=no>_^_4_^_</span> is the number of tokens to generate </li>\n<li><span translate=no>_^_5_^_</span> is the maximum sequence length for the model </li>\n<li><span translate=no>_^_6_^_</span> is the starting prompt</li></ul>\n": "<h2>\u30e2\u30c7\u30eb\u304b\u3089\u306e\u30b5\u30f3\u30d7\u30eb</h2>\n<ul><li><span translate=no>_^_0_^_</span>\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u5143\u306e\u30e2\u30c7\u30eb\u3067\u3059</li>\n<li><span translate=no>_^_1_^_</span>\u4f7f\u7528\u3059\u308b\u30c8\u30fc\u30af\u30ca\u30a4\u30b6\u30fc\u3067\u3059</li>\n<li><span translate=no>_^_2_^_</span>\u4f7f\u7528\u3059\u308b\u30b5\u30f3\u30d7\u30e9\u30fc\u306f</li>\n<li><span translate=no>_^_3_^_</span>\u306f\u751f\u6210\u3059\u308b\u30b5\u30f3\u30d7\u30eb\u306e\u6570\u3067\u3059</li>\n<li><span translate=no>_^_4_^_</span>\u306f\u751f\u6210\u3059\u308b\u30c8\u30fc\u30af\u30f3\u306e\u6570\u3067\u3059</li>\n<li><span translate=no>_^_5_^_</span>\u30e2\u30c7\u30eb\u306e\u6700\u5927\u30b7\u30fc\u30b1\u30f3\u30b9\u9577\u3067\u3059</li>\n<li><span translate=no>_^_6_^_</span>\u306f\u958b\u59cb\u30d7\u30ed\u30f3\u30d7\u30c8\u3067\u3059</li></ul>\n",
|
||||
"<h3>Try different sampling techniques</h3>\n": "<h3>\u3055\u307e\u3056\u307e\u306a\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u624b\u6cd5\u3092\u8a66\u3057\u3066\u304f\u3060\u3055\u3044</h3>\n",
|
||||
"<p> </p>\n": "<p></p>\n",
|
||||
"<p><a href=\"greedy.html\">Greedy Sampling</a> </p>\n": "<p><a href=\"greedy.html\">\u6b32\u5f35\u308a\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</a></p>\n",
|
||||
"<p><a href=\"nucleus.html\">Nucleus Sampling</a> </p>\n": "<p><a href=\"nucleus.html\">\u6838\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</a></p>\n",
|
||||
"<p><a href=\"temperature.html\">Temperature Sampling</a> </p>\n": "<p><a href=\"temperature.html\">\u6e29\u5ea6\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</a></p>\n",
|
||||
"<p><a href=\"top_k.html\">Top-k Sampling</a> </p>\n": "<p><a href=\"top_k.html\">\u30c8\u30c3\u30d7k\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</a></p>\n",
|
||||
"<p>Add the sampled token to the data </p>\n": "<p>\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3057\u305f\u30c8\u30fc\u30af\u30f3\u3092\u30c7\u30fc\u30bf\u306b\u8ffd\u52a0\u3057\u307e\u3059</p>\n",
|
||||
"<p>Collect output for printing </p>\n": "<p>\u5370\u5237\u7528\u306e\u51fa\u529b\u3092\u53ce\u96c6</p>\n",
|
||||
"<p>Decode and add the sampled token for logging </p>\n": "<p>\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3057\u305f\u30c8\u30fc\u30af\u30f3\u3092\u30c7\u30b3\u30fc\u30c9\u3057\u3066\u30ed\u30ae\u30f3\u30b0\u7528\u306b\u8ffd\u52a0</p>\n",
|
||||
"<p>Get the <span translate=no>_^_0_^_</span> of the last token </p>\n": "<p>\u6700\u5f8c\u306e\u30c8\u30fc\u30af\u30f3\u306e\u53d6\u5f97 <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Get the model output. The 'logits' has shape <span translate=no>_^_0_^_</span> </p>\n": "<p>\u30e2\u30c7\u30eb\u51fa\u529b\u3092\u53d6\u5f97\u3057\u307e\u3059\u3002\u300c\u30ed\u30b8\u30c3\u30c8\u300d\u306b\u306f\u5f62\u304c\u3042\u308a\u307e\u3059 <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Load the model and tokenizer </p>\n": "<p>\u30e2\u30c7\u30eb\u3068\u30c8\u30fc\u30af\u30ca\u30a4\u30b6\u30fc\u3092\u30ed\u30fc\u30c9</p>\n",
|
||||
"<p>Print the sampled outputs </p>\n": "<p>\u30b5\u30f3\u30d7\u30eb\u51fa\u529b\u3092\u5370\u5237</p>\n",
|
||||
"<p>Prompts to use for sampling </p>\n": "<p>\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u306b\u4f7f\u7528\u3059\u308b\u30d7\u30ed\u30f3\u30d7\u30c8</p>\n",
|
||||
"<p>Sample <span translate=no>_^_0_^_</span> </p>\n": "<p>[\u30b5\u30f3\u30d7\u30eb] <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Sample from the <span translate=no>_^_0_^_</span> </p>\n": "<p>\u304b\u3089\u306e\u30b5\u30f3\u30d7\u30eb <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Set the model to eval mode </p>\n": "<p>\u30e2\u30c7\u30eb\u3092 eval \u30e2\u30fc\u30c9\u306b\u8a2d\u5b9a</p>\n",
|
||||
"<p>Tokenize the <span translate=no>_^_0_^_</span> and make <span translate=no>_^_1_^_</span> copies of it </p>\n": "<p><span translate=no>_^_0_^_</span><span translate=no>_^_1_^_</span>\u3092\u30c8\u30fc\u30af\u30f3\u5316\u3057\u3066\u30b3\u30d4\u30fc\u3092\u4f5c\u6210</p>\n",
|
||||
"<p>Truncate the data to the maximum sequence length </p>\n": "<p>\u30c7\u30fc\u30bf\u3092\u6700\u5927\u30b7\u30fc\u30b1\u30f3\u30b9\u9577\u307e\u3067\u5207\u308a\u6368\u3066\u308b</p>\n",
|
||||
"Trying out Sampling Techniques for Language Models": "\u8a00\u8a9e\u30e2\u30c7\u30eb\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u624b\u6cd5\u306e\u8a66\u307f",
|
||||
"We try out different sampling techniques for language models on HuggingFace's GPT2 model.": "HuggingFace\u306eGPT2\u30e2\u30c7\u30eb\u3067\u3001\u8a00\u8a9e\u30e2\u30c7\u30eb\u7528\u306b\u3055\u307e\u3056\u307e\u306a\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u624b\u6cd5\u3092\u8a66\u3057\u3066\u3044\u307e\u3059\u3002"
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"<h1>Trying out Sampling Techniques for Language Models</h1>\n<ul><li><a href=\"greedy.html\">Greedy Sampling</a> </li>\n<li><a href=\"temperature.html\">Temperature Sampling</a> </li>\n<li><a href=\"top_k.html\">Top-k Sampling</a> </li>\n<li><a href=\"nucleus.html\">Nucleus Sampling</a></li></ul>\n<p>This experiment uses the above sampling techniques, on HuggingFace's GPT2 model.</p>\n": "<h1>\u0db7\u0dcf\u0dc2\u0dcf\u0d86\u0d9a\u0dd8\u0dad\u0dd2 \u0dc3\u0db3\u0dc4\u0dcf \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0d9a\u0dca\u0dbb\u0db8 \u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf \u0db6\u0dd0\u0dbd\u0dd3\u0db8</h1>\n<ul><li><a href=\"greedy.html\">\u0d9a\u0dd1\u0daf\u0dbb \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8</a> </li>\n<li><a href=\"temperature.html\">\u0d8b\u0dc2\u0dca\u0dab\u0dad\u0dca\u0dc0 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8</a> </li>\n<li><a href=\"top_k.html\">\u0d89\u0dc4\u0dc5-K \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8\u0dca</a> </li>\n<li><a href=\"nucleus.html\">\u0db1\u0dca\u0dba\u0dc2\u0dca\u0da7\u0dd2\u0d9a \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8</a></li></ul>\n<p>\u0db8\u0dd9\u0db8\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf \u0db6\u0dd0\u0dbd\u0dd3\u0db8 HugingFace \u0dc4\u0dd2 GPT2 \u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba \u0db8\u0dad \u0d89\u0dc4\u0dad \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0dc1\u0dd2\u0dbd\u0dca\u0db4\u0dd3\u0dba \u0d9a\u0dca\u0dbb\u0db8 \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dbb\u0dba\u0dd2. </p>\n",
|
||||
"<h2>Sample from model</h2>\n<ul><li><span translate=no>_^_0_^_</span> is the model to sample from </li>\n<li><span translate=no>_^_1_^_</span> is the tokenizer to use </li>\n<li><span translate=no>_^_2_^_</span> is the sampler to use </li>\n<li><span translate=no>_^_3_^_</span> is the number of samples to generate </li>\n<li><span translate=no>_^_4_^_</span> is the number of tokens to generate </li>\n<li><span translate=no>_^_5_^_</span> is the maximum sequence length for the model </li>\n<li><span translate=no>_^_6_^_</span> is the starting prompt</li></ul>\n": "<h2>\u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba\u0dd9\u0db1\u0dca\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba</h2>\n<ul><li><span translate=no>_^_0_^_</span> \u0dc3\u0dd2\u0da7 \u0d86\u0daf\u0dbb\u0dca\u0dc1 \u0d86\u0daf\u0dbb\u0dca\u0dc1 \u0dc0\u0dda </li>\n<li><span translate=no>_^_1_^_</span> \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0da7 \u0da7\u0ddd\u0d9a\u0db1\u0dba\u0dd2\u0dc3\u0dbb\u0dca \u0dc0\u0dda </li>\n<li><span translate=no>_^_2_^_</span> \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0da7 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0d9a\u0dbb\u0dd4 \u0dc0\u0dda </li>\n<li><span translate=no>_^_3_^_</span> \u0da2\u0db1\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0da7 \u0dc3\u0dcf\u0db8\u0dca\u0db4\u0dbd \u0dc3\u0d82\u0d9b\u0dca\u0dba\u0dcf\u0dc0 \u0dc0\u0dda </li>\n<li><span translate=no>_^_4_^_</span> \u0d8b\u0dad\u0dca\u0db4\u0dcf\u0daf\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0da7 \u0da7\u0ddd\u0d9a\u0db1 \u0d9c\u0dab\u0db1 \u0dc0\u0dda </li>\n<li><span translate=no>_^_5_^_</span> \u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba \u0dc3\u0db3\u0dc4\u0dcf \u0d8b\u0db4\u0dbb\u0dd2\u0db8 \u0d85\u0db1\u0dd4\u0d9a\u0dca\u0dbb\u0db8\u0dba \u0daf\u0dd2\u0d9c \u0dc0\u0dda </li>\n<li><span translate=no>_^_6_^_</span> \u0d86\u0dbb\u0db8\u0dca\u0db7\u0d9a \u0dc0\u0dd2\u0db8\u0dc3\u0dd4\u0db8 \u0dc0\u0dda</li></ul>\n",
|
||||
"<h3>Try different sampling techniques</h3>\n": "<h3>\u0dc0\u0dd2\u0dc0\u0dd2\u0db0\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0d9a\u0dca\u0dbb\u0db8 \u0d8b\u0dad\u0dca\u0dc3\u0dcf\u0dc4 \u0d9a\u0dbb\u0db1\u0dca\u0db1</h3>\n",
|
||||
"<p> </p>\n": "<p> </p>\n",
|
||||
"<p><a href=\"greedy.html\">Greedy Sampling</a> </p>\n": "<p><a href=\"greedy.html\">\u0d9a\u0dd1\u0daf\u0dbb \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8</a> </p>\n",
|
||||
"<p><a href=\"nucleus.html\">Nucleus Sampling</a> </p>\n": "<p><a href=\"nucleus.html\">\u0db1\u0dca\u0dba\u0dc2\u0dca\u0da7\u0dd2\u0d9a \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8</a> </p>\n",
|
||||
"<p><a href=\"temperature.html\">Temperature Sampling</a> </p>\n": "<p><a href=\"temperature.html\">\u0d8b\u0dc2\u0dca\u0dab\u0dad\u0dca\u0dc0 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8</a> </p>\n",
|
||||
"<p><a href=\"top_k.html\">Top-k Sampling</a> </p>\n": "<p><a href=\"top_k.html\">\u0d89\u0dc4\u0dc5-K \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8\u0dca</a> </p>\n",
|
||||
"<p>Add the sampled token to the data </p>\n": "<p>\u0daf\u0dad\u0dca\u0dad\u0dc0\u0dbd\u0da7 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0da7\u0ddd\u0d9a\u0db1\u0dba \u0d91\u0d9a\u0dca \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Collect output for printing </p>\n": "<p>\u0db8\u0dd4\u0daf\u0dca\u0dbb\u0dab\u0dba\u0dc3\u0db3\u0dc4\u0dcf \u0db4\u0dca\u0dbb\u0dad\u0dd2\u0daf\u0dcf\u0db1\u0dba \u0d91\u0d9a\u0dad\u0dd4 \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Decode and add the sampled token for logging </p>\n": "<p>\u0dbd\u0ddc\u0d9c\u0dca\u0dc0\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0da7\u0ddd\u0d9a\u0db1\u0dba \u0dc0\u0dd2\u0d9a\u0dda\u0dad\u0db1\u0dba \u0d9a\u0dbb \u0d91\u0d9a\u0dca \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Get the <span translate=no>_^_0_^_</span> of the last token </p>\n": "<p>\u0d85\u0dc0\u0dc3\u0dcf\u0db1\u0da7\u0ddd\u0d9a\u0db1\u0dba \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 <span translate=no>_^_0_^_</span> </p>\n",
|
||||
"<p>Get the model output. The 'logits' has shape <span translate=no>_^_0_^_</span> </p>\n": "<p>\u0d86\u0daf\u0dbb\u0dca\u0dc1\u0db4\u0dca\u0dbb\u0dad\u0dd2\u0daf\u0dcf\u0db1\u0dba \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1. '\u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd3\u0db8\u0da7' \u0dc4\u0dd0\u0da9\u0dba \u0d87\u0dad <span translate=no>_^_0_^_</span> </p>\n",
|
||||
"<p>Load the model and tokenizer </p>\n": "<p>\u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba\u0dc3\u0dc4 \u0da7\u0ddd\u0d9a\u0db1\u0dba\u0dd2\u0dc3\u0dbb\u0dca \u0db4\u0da7\u0dc0\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Print the sampled outputs </p>\n": "<p>\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0db4\u0dca\u0dbb\u0dad\u0dd2\u0daf\u0dcf\u0db1\u0dba\u0db1\u0dca \u0db8\u0dd4\u0daf\u0dca\u0dbb\u0dab\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Prompts to use for sampling </p>\n": "<p>\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8\u0dc3\u0db3\u0dc4\u0dcf \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0da7 \u0dc0\u0dd2\u0db8\u0dc3\u0db1\u0dd4 \u0dbd\u0dd0\u0db6\u0dda </p>\n",
|
||||
"<p>Sample <span translate=no>_^_0_^_</span> </p>\n": "<p>\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba <span translate=no>_^_0_^_</span> </p>\n",
|
||||
"<p>Sample from the <span translate=no>_^_0_^_</span> </p>\n": "<p>\u0dc0\u0dd9\u0dad\u0dd2\u0db1\u0dca\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba <span translate=no>_^_0_^_</span> </p>\n",
|
||||
"<p>Set the model to eval mode </p>\n": "<p>\u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dbaeval \u0db4\u0dca\u0dbb\u0d9a\u0dcf\u0dbb\u0dba\u0da7 \u0dc3\u0d9a\u0dc3\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Tokenize the <span translate=no>_^_0_^_</span> and make <span translate=no>_^_1_^_</span> copies of it </p>\n": "<p>\u0da7\u0ddd\u0d9a\u0dd3\u0dc3\u0dca <span translate=no>_^_0_^_</span> \u0d9a\u0dbb <span translate=no>_^_1_^_</span> \u0d91\u0dc4\u0dd2 \u0db4\u0dd2\u0da7\u0db4\u0dad\u0dca \u0dc3\u0dcf\u0daf\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Truncate the data to the maximum sequence length </p>\n": "<p>\u0d8b\u0db4\u0dbb\u0dd2\u0db8\u0d85\u0db1\u0dd4\u0d9a\u0dca\u0dbb\u0db8\u0dd2\u0d9a \u0daf\u0dd2\u0d9c\u0da7 \u0daf\u0dad\u0dca\u0dad \u0d9a\u0db4\u0dcf </p>\n",
|
||||
"Trying out Sampling Techniques for Language Models": "\u0db7\u0dcf\u0dc2\u0dcf \u0d86\u0d9a\u0dd8\u0dad\u0dd2 \u0dc3\u0db3\u0dc4\u0dcf \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0d9a\u0dca\u0dbb\u0db8 \u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf \u0db6\u0dd0\u0dbd\u0dd3\u0db8",
|
||||
"We try out different sampling techniques for language models on HuggingFace's GPT2 model.": "HugingFace \u0dc4\u0dd2 GPT2 \u0d86\u0d9a\u0dd8\u0dad\u0dd2\u0dba\u0dda \u0db7\u0dcf\u0dc2\u0dcf \u0d86\u0d9a\u0dd8\u0dad\u0dd2 \u0dc3\u0db3\u0dc4\u0dcf \u0dc0\u0dd2\u0dc0\u0dd2\u0db0 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0d9a\u0dca\u0dbb\u0db8 \u0d85\u0db4\u0dd2 \u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf \u0db6\u0dbd\u0db8\u0dd4."
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"<h1>Trying out Sampling Techniques for Language Models</h1>\n<ul><li><a href=\"greedy.html\">Greedy Sampling</a> </li>\n<li><a href=\"temperature.html\">Temperature Sampling</a> </li>\n<li><a href=\"top_k.html\">Top-k Sampling</a> </li>\n<li><a href=\"nucleus.html\">Nucleus Sampling</a></li></ul>\n<p>This experiment uses the above sampling techniques, on HuggingFace's GPT2 model.</p>\n": "<h1>\u5c1d\u8bd5\u8bed\u8a00\u6a21\u578b\u7684\u91c7\u6837\u6280\u672f</h1>\n<ul><li><a href=\"greedy.html\">\u8d2a\u5a6a\u91c7\u6837</a></li>\n<li><a href=\"temperature.html\">\u6e29\u5ea6\u91c7\u6837</a></li>\n<li><a href=\"top_k.html\">\u524d k \u4e2a\u91c7\u6837</a></li>\n<li><a href=\"nucleus.html\">\u539f\u5b50\u6838\u91c7\u6837</a></li></ul>\n<p>\u672c\u5b9e\u9a8c\u5728HuggingFace\u7684GPT2\u6a21\u578b\u4e0a\u4f7f\u7528\u4e86\u4e0a\u8ff0\u91c7\u6837\u6280\u672f\u3002</p>\n",
|
||||
"<h2>Sample from model</h2>\n<ul><li><span translate=no>_^_0_^_</span> is the model to sample from </li>\n<li><span translate=no>_^_1_^_</span> is the tokenizer to use </li>\n<li><span translate=no>_^_2_^_</span> is the sampler to use </li>\n<li><span translate=no>_^_3_^_</span> is the number of samples to generate </li>\n<li><span translate=no>_^_4_^_</span> is the number of tokens to generate </li>\n<li><span translate=no>_^_5_^_</span> is the maximum sequence length for the model </li>\n<li><span translate=no>_^_6_^_</span> is the starting prompt</li></ul>\n": "<h2>\u6765\u81ea\u6a21\u578b\u7684\u6837\u672c</h2>\n<ul><li><span translate=no>_^_0_^_</span>\u662f\u8981\u91c7\u6837\u7684\u6a21\u578b</li>\n<li><span translate=no>_^_1_^_</span>\u662f\u8981\u4f7f\u7528\u7684\u5206\u8bcd\u5668</li>\n<li><span translate=no>_^_2_^_</span>\u662f\u8981\u4f7f\u7528\u7684\u91c7\u6837\u5668</li>\n<li><span translate=no>_^_3_^_</span>\u662f\u8981\u751f\u6210\u7684\u6837\u672c\u6570</li>\n<li><span translate=no>_^_4_^_</span>\u662f\u8981\u751f\u6210\u7684\u4ee4\u724c\u6570\u91cf</li>\n<li><span translate=no>_^_5_^_</span>\u662f\u6a21\u578b\u7684\u6700\u5927\u5e8f\u5217\u957f\u5ea6</li>\n<li><span translate=no>_^_6_^_</span>\u662f\u8d77\u59cb\u63d0\u793a</li></ul>\n",
|
||||
"<h3>Try different sampling techniques</h3>\n": "<h3>\u5c1d\u8bd5\u4e0d\u540c\u7684\u91c7\u6837\u6280\u5de7</h3>\n",
|
||||
"<p> </p>\n": "<p></p>\n",
|
||||
"<p><a href=\"greedy.html\">Greedy Sampling</a> </p>\n": "<p><a href=\"greedy.html\">\u8d2a\u5a6a\u91c7\u6837</a></p>\n",
|
||||
"<p><a href=\"nucleus.html\">Nucleus Sampling</a> </p>\n": "<p><a href=\"nucleus.html\">\u539f\u5b50\u6838\u91c7\u6837</a></p>\n",
|
||||
"<p><a href=\"temperature.html\">Temperature Sampling</a> </p>\n": "<p><a href=\"temperature.html\">\u6e29\u5ea6\u91c7\u6837</a></p>\n",
|
||||
"<p><a href=\"top_k.html\">Top-k Sampling</a> </p>\n": "<p><a href=\"top_k.html\">\u524d k \u4e2a\u91c7\u6837</a></p>\n",
|
||||
"<p>Add the sampled token to the data </p>\n": "<p>\u5c06\u91c7\u6837\u4ee4\u724c\u6dfb\u52a0\u5230\u6570\u636e\u4e2d</p>\n",
|
||||
"<p>Collect output for printing </p>\n": "<p>\u6536\u96c6\u8f93\u51fa\u4ee5\u8fdb\u884c\u6253\u5370</p>\n",
|
||||
"<p>Decode and add the sampled token for logging </p>\n": "<p>\u89e3\u7801\u5e76\u6dfb\u52a0\u7528\u4e8e\u65e5\u5fd7\u8bb0\u5f55\u7684\u91c7\u6837\u4ee4\u724c</p>\n",
|
||||
"<p>Get the <span translate=no>_^_0_^_</span> of the last token </p>\n": "<p>\u83b7\u53d6\u6700\u540e<span translate=no>_^_0_^_</span>\u4e00\u4e2a\u4ee4\u724c\u7684</p>\n",
|
||||
"<p>Get the model output. The 'logits' has shape <span translate=no>_^_0_^_</span> </p>\n": "<p>\u83b7\u53d6\u6a21\u578b\u8f93\u51fa\u3002\u201clogits\u201d \u6709\u5f62\u72b6<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Load the model and tokenizer </p>\n": "<p>\u52a0\u8f7d\u6a21\u578b\u548c\u5206\u8bcd\u5668</p>\n",
|
||||
"<p>Print the sampled outputs </p>\n": "<p>\u6253\u5370\u91c7\u6837\u8f93\u51fa</p>\n",
|
||||
"<p>Prompts to use for sampling </p>\n": "<p>\u91c7\u6837\u65f6\u4f7f\u7528\u7684\u63d0\u793a</p>\n",
|
||||
"<p>Sample <span translate=no>_^_0_^_</span> </p>\n": "<p>\u6837\u672c<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Sample from the <span translate=no>_^_0_^_</span> </p>\n": "<p>\u6837\u672c\u6765\u81ea<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Set the model to eval mode </p>\n": "<p>\u5c06\u6a21\u578b\u8bbe\u7f6e\u4e3a\u8bc4\u4f30\u6a21\u5f0f</p>\n",
|
||||
"<p>Tokenize the <span translate=no>_^_0_^_</span> and make <span translate=no>_^_1_^_</span> copies of it </p>\n": "<p>\u6807\u8bb0\u5316<span translate=no>_^_0_^_</span>\u5e76\u5236\u4f5c\u5176<span translate=no>_^_1_^_</span>\u526f\u672c</p>\n",
|
||||
"<p>Truncate the data to the maximum sequence length </p>\n": "<p>\u5c06\u6570\u636e\u622a\u65ad\u4e3a\u6700\u5927\u5e8f\u5217\u957f\u5ea6</p>\n",
|
||||
"Trying out Sampling Techniques for Language Models": "\u5c1d\u8bd5\u8bed\u8a00\u6a21\u578b\u7684\u91c7\u6837\u6280\u672f",
|
||||
"We try out different sampling techniques for language models on HuggingFace's GPT2 model.": "\u6211\u4eec\u5728HuggingFace\u7684GPT2\u6a21\u578b\u4e0a\u4e3a\u8bed\u8a00\u6a21\u578b\u5c1d\u8bd5\u4e86\u4e0d\u540c\u7684\u91c7\u6837\u6280\u672f\u3002"
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"<p>Add the prediction for logging </p>\n": "<p>\u30ed\u30ae\u30f3\u30b0\u7528\u306e\u4e88\u6e2c\u3092\u8ffd\u52a0</p>\n",
|
||||
"<p>Collect output for printing </p>\n": "<p>\u5370\u5237\u7528\u306e\u51fa\u529b\u3092\u53ce\u96c6</p>\n",
|
||||
"<p>Get the model output </p>\n": "<p>\u30e2\u30c7\u30eb\u51fa\u529b\u3092\u53d6\u5f97</p>\n",
|
||||
"<p>Get the model prediction (greedy) </p>\n": "<p>\u30e2\u30c7\u30eb\u4e88\u6e2c\u3092\u53d6\u5f97 (\u6b32\u5f35\u308a)</p>\n",
|
||||
"<p>Print the sampled output </p>\n": "<p>\u30b5\u30f3\u30d7\u30eb\u51fa\u529b\u3092\u5370\u5237\u3059\u308b</p>\n",
|
||||
"<p>Sample 25 tokens </p>\n": "<p>25\u30c8\u30fc\u30af\u30f3\u306e\u30b5\u30f3\u30d7\u30eb</p>\n",
|
||||
"<p>Tokenize the prompt </p>\n": "<p>\u30d7\u30ed\u30f3\u30d7\u30c8\u3092\u30c8\u30fc\u30af\u30f3\u5316</p>\n",
|
||||
"experiment_tiny.py": "experiment_tiny.py"
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"<p>Add the prediction for logging </p>\n": "<p>\u0dbd\u0ddc\u0d9c\u0dca\u0dc0\u0dd3\u0db8 \u0dc3\u0db3\u0dc4\u0dcf \u0d85\u0db1\u0dcf\u0dc0\u0dd0\u0d9a\u0dd2\u0dba \u0d91\u0d9a\u0dca \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Collect output for printing </p>\n": "<p>\u0db8\u0dd4\u0daf\u0dca\u0dbb\u0dab\u0dba\u0dc3\u0db3\u0dc4\u0dcf \u0db4\u0dca\u0dbb\u0dad\u0dd2\u0daf\u0dcf\u0db1\u0dba \u0d91\u0d9a\u0dad\u0dd4 \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Get the model output </p>\n": "<p>\u0d86\u0daf\u0dbb\u0dca\u0dc1\u0db4\u0dca\u0dbb\u0dad\u0dd2\u0daf\u0dcf\u0db1\u0dba \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Get the model prediction (greedy) </p>\n": "<p>\u0d86\u0daf\u0dbb\u0dca\u0dc1\u0d85\u0db1\u0dcf\u0dc0\u0dd0\u0d9a\u0dd2\u0dba \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 (\u0d9a\u0dd1\u0daf\u0dbb) </p>\n",
|
||||
"<p>Print the sampled output </p>\n": "<p>\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0db4\u0dca\u0dbb\u0dad\u0dd2\u0daf\u0dcf\u0db1\u0dba \u0db8\u0dd4\u0daf\u0dca\u0dbb\u0dab\u0dba \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Sample 25 tokens </p>\n": "<p>\u0dc3\u0dcf\u0db8\u0dca\u0db4\u0dbd25 \u0da7\u0ddd\u0d9a\u0db1 </p>\n",
|
||||
"<p>Tokenize the prompt </p>\n": "<p>\u0dc0\u0dd2\u0db8\u0dc3\u0dd4\u0db8\u0da7\u0ddd\u0d9a\u0dd9\u0db1\u0dca\u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"experiment_tiny.py": "experiment_tiny.py"
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"<p>Add the prediction for logging </p>\n": "<p>\u6dfb\u52a0\u65e5\u5fd7\u8bb0\u5f55\u7684\u9884\u6d4b</p>\n",
|
||||
"<p>Collect output for printing </p>\n": "<p>\u6536\u96c6\u8f93\u51fa\u4ee5\u8fdb\u884c\u6253\u5370</p>\n",
|
||||
"<p>Get the model output </p>\n": "<p>\u83b7\u53d6\u6a21\u578b\u8f93\u51fa</p>\n",
|
||||
"<p>Get the model prediction (greedy) </p>\n": "<p>\u83b7\u53d6\u6a21\u578b\u9884\u6d4b\uff08\u8d2a\u5a6a\uff09</p>\n",
|
||||
"<p>Print the sampled output </p>\n": "<p>\u6253\u5370\u91c7\u6837\u8f93\u51fa</p>\n",
|
||||
"<p>Sample 25 tokens </p>\n": "<p>\u6837\u672c 25 \u4e2a\u4ee3\u5e01</p>\n",
|
||||
"<p>Tokenize the prompt </p>\n": "<p>\u5c06\u63d0\u793a\u7b26\u53f7\u5316</p>\n",
|
||||
"experiment_tiny.py": "experiment_tiny.py"
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"<h1>Greedy Sampling</h1>\n<p>Here we sample the most likely token from the distribution of logits.</p>\n<p>Here's an <a href=\"experiment.html\">experiment</a> that uses these sampling techniques.</p>\n": "<h1>\u6b32\u5f35\u308a\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</h1>\n<p>\u3053\u3053\u3067\u306f\u3001\u30ed\u30b8\u30c3\u30c8\u306e\u5206\u5e03\u304b\u3089\u6700\u3082\u53ef\u80fd\u6027\u306e\u9ad8\u3044\u30c8\u30fc\u30af\u30f3\u3092\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3057\u307e\u3059\u3002</p>\n<p>\u3053\u308c\u306f\u3001<a href=\"experiment.html\">\u3053\u308c\u3089\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u624b\u6cd5\u3092\u4f7f\u7528\u3057\u305f\u5b9f\u9a13\u3067\u3059</a>\u3002</p>\n",
|
||||
"<p> Sample the most likely token from the distribution of logits</p>\n": "<p>\u30ed\u30b8\u30c3\u30c8\u306e\u5206\u5e03\u304b\u3089\u6700\u3082\u53ef\u80fd\u6027\u306e\u9ad8\u3044\u30c8\u30fc\u30af\u30f3\u3092\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3059\u308b</p>\n",
|
||||
"A PyTorch implementation of greedy sampling from language models.": "\u8a00\u8a9e\u30e2\u30c7\u30eb\u304b\u3089\u306e\u8caa\u6b32\u306a\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u306e PyTorch \u5b9f\u88c5\u3002",
|
||||
"Greedy Sampling": "\u6b32\u5f35\u308a\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0"
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"<h1>Greedy Sampling</h1>\n<p>Here we sample the most likely token from the distribution of logits.</p>\n<p>Here's an <a href=\"experiment.html\">experiment</a> that uses these sampling techniques.</p>\n": "<h1>\u0d9a\u0dd1\u0daf\u0dbb\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8</h1>\n<p>\u0db8\u0dd9\u0db1\u0dca\u0db1\u0d85\u0db4\u0dd2 \u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca \u0db6\u0dd9\u0daf\u0dcf \u0dc4\u0dd0\u0dbb\u0dd3\u0db8\u0dd9\u0db1\u0dca \u0db6\u0ddc\u0dc4\u0ddd \u0daf\u0dd4\u0dbb\u0da7 \u0da7\u0ddd\u0d9a\u0db1\u0dba \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0d9a\u0dbb\u0db8\u0dd4. </p>\n<p>\u0db8\u0dd9\u0db1\u0dca\u0db1\u0db8\u0dd9\u0db8 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0dc1\u0dd2\u0dbd\u0dca\u0db4\u0dd3\u0dba \u0d9a\u0dca\u0dbb\u0db8 \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dbb\u0db1 <a href=\"experiment.html\">\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf \u0db6\u0dd0\u0dbd\u0dd3\u0db8\u0d9a\u0dca</a> . </p>\n",
|
||||
"<p> Sample the most likely token from the distribution of logits</p>\n": "<p> \u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca\u0db6\u0dd9\u0daf\u0dcf \u0dc4\u0dd0\u0dbb\u0dd3\u0db8\u0dd9\u0db1\u0dca \u0db6\u0ddc\u0dc4\u0ddd \u0daf\u0dd4\u0dbb\u0da7 \u0da7\u0ddd\u0d9a\u0db1\u0dba \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0d9a\u0dbb\u0db1\u0dca\u0db1</p>\n",
|
||||
"A PyTorch implementation of greedy sampling from language models.": "\u0db7\u0dcf\u0dc2\u0dcf \u0d86\u0d9a\u0dd8\u0dad\u0dd2 \u0dc0\u0dbd\u0dd2\u0db1\u0dca \u0d9a\u0dd1\u0daf\u0dbb \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8\u0dca PyTorch \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dd2\u0dbb\u0dd3\u0db8.",
|
||||
"Greedy Sampling": "\u0d9a\u0dd1\u0daf\u0dbb \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8"
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"<h1>Greedy Sampling</h1>\n<p>Here we sample the most likely token from the distribution of logits.</p>\n<p>Here's an <a href=\"experiment.html\">experiment</a> that uses these sampling techniques.</p>\n": "<h1>\u8d2a\u5a6a\u91c7\u6837</h1>\n<p>\u5728\u8fd9\u91cc\uff0c\u6211\u4eec\u4ece\u65e5\u5fd7\u5206\u5e03\u4e2d\u62bd\u53d6\u6700\u6709\u53ef\u80fd\u7684\u4ee4\u724c\u3002</p>\n<p>\u8fd9\u662f\u4e00\u4e2a\u4f7f\u7528\u8fd9\u4e9b\u91c7\u6837\u6280\u672f\u7684<a href=\"experiment.html\">\u5b9e\u9a8c</a>\u3002</p>\n",
|
||||
"<p> Sample the most likely token from the distribution of logits</p>\n": "<p>\u4ece\u65e5\u5fd7\u5206\u5e03\u4e2d\u62bd\u53d6\u6700\u6709\u53ef\u80fd\u7684\u4ee4\u724c</p>\n",
|
||||
"A PyTorch implementation of greedy sampling from language models.": "\u4ece\u8bed\u8a00\u6a21\u578b\u4e2d\u8fdb\u884c\u8d2a\u5a6a\u91c7\u6837\u7684 PyTorch \u5b9e\u73b0\u3002",
|
||||
"Greedy Sampling": "\u8d2a\u5a6a\u91c7\u6837"
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
{
|
||||
"<h1>Nucleus Sampling</h1>\n<p>This is an implementation of nucleus sampling, introduced in the paper <a href=\"https://arxiv.org/abs/1904.09751\">The Curious Case of Neural Text Degeneration</a>.</p>\n<p>The paper discusses the problems with other sampling methods such as Beam Search, <a href=\"temperature.html\">Pure sampling</a>, <a href=\"temperature.html\">Temperature sampling</a>, and <a href=\"top_k.html\">Top-k sampling</a>. The paper introduces the idea of nucleus sampling, which practically performs better than other sampling methods for text generation.</p>\n<p>Nucleus sampling first picks a subset of the vocabulary <span translate=no>_^_0_^_</span>, where <span translate=no>_^_1_^_</span> is smallest set of tokens such that</p>\n<p><span translate=no>_^_2_^_</span></p>\n<p>That is, we pick the highest probable tokens until the sum of their probabilities is less that <span translate=no>_^_3_^_</span>.</p>\n<p>Then we sample from the selected tokens.</p>\n<p>Here's an <a href=\"experiment.html\">experiment</a> that uses these sampling techniques.</p>\n": "<h1>\u6838\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</h1>\n<p>\u3053\u308c\u306f\u6838\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u306e\u5b9f\u88c5\u3067\u3001\u8ad6\u6587\u300c<a href=\"https://arxiv.org/abs/1904.09751\">\u795e\u7d4c\u30c6\u30ad\u30b9\u30c8\u5909\u6027\u306e\u5947\u5999\u306a\u4e8b\u4f8b</a>\u300d\u3067\u7d39\u4ecb\u3055\u308c\u3066\u3044\u307e\u3059\u3002</p>\n<p>\u3053\u306e\u8ad6\u6587\u3067\u306f\u3001\u30d3\u30fc\u30e0\u30b5\u30fc\u30c1\u3001<a href=\"temperature.html\">\u30d4\u30e5\u30a2\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3001\u6e29\u5ea6\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</a><a href=\"temperature.html\">\u3001<a href=\"top_k.html\">TOP-K\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u306a\u3069\u306e\u4ed6\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u65b9\u6cd5\u306e\u554f\u984c\u306b\u3064\u3044\u3066\u8aac\u660e\u3057\u307e\u3059</a></a>\u3002\u3053\u306e\u8ad6\u6587\u3067\u306f\u3001\u6838\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u306e\u30a2\u30a4\u30c7\u30a2\u3092\u7d39\u4ecb\u3057\u3066\u3044\u307e\u3059\u3002\u6838\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u306f\u3001\u30c6\u30ad\u30b9\u30c8\u751f\u6210\u306b\u304a\u3044\u3066\u4ed6\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u65b9\u6cd5\u3088\u308a\u3082\u5b9f\u8cea\u7684\u306b\u512a\u308c\u3066\u3044\u307e\u3059</p>\u3002\n<p>Nucleus \u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3067\u306f\u3001\u6700\u521d\u306b\u30dc\u30ad\u30e3\u30d6\u30e9\u30ea\u306e\u30b5\u30d6\u30bb\u30c3\u30c8\u3092\u9078\u629e\u3057\u307e\u3059\u3002\u3053\u3053\u3067<span translate=no>_^_0_^_</span>\u3001<span translate=no>_^_1_^_</span>\u306f\u6b21\u306e\u3088\u3046\u306a\u30c8\u30fc\u30af\u30f3\u306e\u6700\u5c0f\u30bb\u30c3\u30c8\u3092\u9078\u629e\u3057\u307e\u3059\u3002</p>\n<p><span translate=no>_^_2_^_</span></p>\n<p>\u3064\u307e\u308a\u3001\u78ba\u7387\u306e\u5408\u8a08\u304c\u305d\u308c\u3088\u308a\u5c0f\u3055\u304f\u306a\u308b\u307e\u3067\u3001\u6700\u3082\u53ef\u80fd\u6027\u306e\u9ad8\u3044\u30c8\u30fc\u30af\u30f3\u3092\u9078\u629e\u3057\u307e\u3059\u3002<span translate=no>_^_3_^_</span></p>\n<p>\u6b21\u306b\u3001\u9078\u629e\u3057\u305f\u30c8\u30fc\u30af\u30f3\u304b\u3089\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3057\u307e\u3059\u3002</p>\n<p>\u3053\u308c\u306f\u3001<a href=\"experiment.html\">\u3053\u308c\u3089\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u624b\u6cd5\u3092\u4f7f\u7528\u3057\u305f\u5b9f\u9a13\u3067\u3059</a>\u3002</p>\n",
|
||||
"<h2>Nucleus Sampler</h2>\n": "<h2>\u6838\u30b5\u30f3\u30d7\u30e9\u30fc</h2>\n",
|
||||
"<p> </p>\n": "<p></p>\n",
|
||||
"<p> Sample from logits with Nucleus Sampling</p>\n": "<p>Nucleus \u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u306b\u3088\u308b\u30ed\u30b8\u30c3\u30c8\u304b\u3089\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</p>\n",
|
||||
"<p>Find the cumulative sums less than <span translate=no>_^_0_^_</span>. </p>\n": "<p>\u3088\u308a\u5c0f\u3055\u3044\u7d2f\u7a4d\u548c\u3092\u6c42\u3081\u307e\u3059\u3002<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Get log probabilities and mask out the non-nucleus </p>\n": "<p>\u5bfe\u6570\u78ba\u7387\u3092\u53d6\u5f97\u3057\u3066\u975e\u6838\u3092\u30de\u30b9\u30af\u3059\u308b</p>\n",
|
||||
"<p>Get probabilities <span translate=no>_^_0_^_</span> </p>\n": "<p>\u78ba\u7387\u3092\u53d6\u5f97 <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Get the actual indexes </p>\n": "<p>\u5b9f\u969b\u306e\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u3092\u53d6\u5f97</p>\n",
|
||||
"<p>Get the cumulative sum of probabilities in the sorted order </p>\n": "<p>\u78ba\u7387\u306e\u7d2f\u7a4d\u5408\u8a08\u3092\u30bd\u30fc\u30c8\u3055\u308c\u305f\u9806\u5e8f\u3067\u6c42\u3081\u308b</p>\n",
|
||||
"<p>Prepend ones so that we add one token after the minimum number of tokens with cumulative probability less that <span translate=no>_^_0_^_</span>. </p>\n": "<p>\u7d2f\u7a4d\u78ba\u7387\u304c\u305d\u308c\u3088\u308a\u5c0f\u3055\u3044\u30c8\u30fc\u30af\u30f3\u306e\u6700\u5c0f\u6570\u306e\u5f8c\u306b\u30c8\u30fc\u30af\u30f3\u30921\u3064\u8ffd\u52a0\u3059\u308b\u3088\u3046\u306b\u30011\u3092\u5148\u982d\u306b\u8ffd\u52a0\u3057\u307e\u3059\u3002<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Sample from the sampler </p>\n": "<p>\u30b5\u30f3\u30d7\u30e9\u30fc\u304b\u3089\u306e\u30b5\u30f3\u30d7\u30eb</p>\n",
|
||||
"<p>Softmax to compute <span translate=no>_^_0_^_</span> from the logits </p>\n": "<p><span translate=no>_^_0_^_</span>\u30ed\u30b8\u30c3\u30c8\u304b\u3089\u8a08\u7b97\u3059\u308b\u30bd\u30d5\u30c8\u30de\u30c3\u30af\u30b9</p>\n",
|
||||
"<p>Sort probabilities in descending order </p>\n": "<p>\u78ba\u7387\u3092\u964d\u9806\u306b\u4e26\u3079\u66ff\u3048\u308b</p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the sum of probabilities of tokens to pick <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> is the sampler to use for the selected tokens</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u30d4\u30c3\u30af\u3059\u308b\u30c8\u30fc\u30af\u30f3\u306e\u78ba\u7387\u306e\u5408\u8a08\u3067\u3059 <span translate=no>_^_1_^_</span></li>\n<li><span translate=no>_^_2_^_</span>\u9078\u629e\u3057\u305f\u30c8\u30fc\u30af\u30f3\u306b\u4f7f\u7528\u3059\u308b\u30b5\u30f3\u30d7\u30e9\u30fc\u3067\u3059</li></ul>\n",
|
||||
"A PyTorch implementation of nucleus sampling from language models.": "\u8a00\u8a9e\u30e2\u30c7\u30eb\u304b\u3089\u306e\u6838\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u306ePyTorch\u5b9f\u88c5\u3002",
|
||||
"Nucleus Sampling": "\u6838\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0"
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
{
|
||||
"<h1>Nucleus Sampling</h1>\n<p>This is an implementation of nucleus sampling, introduced in the paper <a href=\"https://arxiv.org/abs/1904.09751\">The Curious Case of Neural Text Degeneration</a>.</p>\n<p>The paper discusses the problems with other sampling methods such as Beam Search, <a href=\"temperature.html\">Pure sampling</a>, <a href=\"temperature.html\">Temperature sampling</a>, and <a href=\"top_k.html\">Top-k sampling</a>. The paper introduces the idea of nucleus sampling, which practically performs better than other sampling methods for text generation.</p>\n<p>Nucleus sampling first picks a subset of the vocabulary <span translate=no>_^_0_^_</span>, where <span translate=no>_^_1_^_</span> is smallest set of tokens such that</p>\n<p><span translate=no>_^_2_^_</span></p>\n<p>That is, we pick the highest probable tokens until the sum of their probabilities is less that <span translate=no>_^_3_^_</span>.</p>\n<p>Then we sample from the selected tokens.</p>\n<p>Here's an <a href=\"experiment.html\">experiment</a> that uses these sampling techniques.</p>\n": "<h1>\u0db1\u0dca\u0dba\u0dc2\u0dca\u0da7\u0dd2\u0d9a\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8</h1>\n<p>\u0db8\u0dd9\u0dba\u0db1\u0dca\u0dba\u0dc2\u0dca\u0da7\u0dd2\u0d9a \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8\u0dca \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0d9a\u0dca \u0dc0\u0db1 \u0d85\u0dad\u0dbb \u0d91\u0dba \u0d9a\u0da9\u0daf\u0dcf\u0dc3\u0dd2 \u0dc0\u0dbd\u0dd2\u0db1\u0dca \u0dc4\u0db3\u0dd4\u0db1\u0dca\u0dc0\u0dcf <a href=\"https://arxiv.org/abs/1904.09751\">\u0daf\u0dd3 \u0d87\u0dad \u0dc3\u0dca\u0db1\u0dcf\u0dba\u0dd4 \u0db4\u0dd9\u0dc5 \u0db4\u0dbb\u0dd2\u0dc4\u0dcf\u0db1\u0dd2\u0dba \u0db4\u0dd2\u0dc5\u0dd2\u0db6\u0db3 \u0d9a\u0dd4\u0dad\u0dd4\u0dc4\u0dbd\u0dba</a>. </p>\n<p>\u0d9a\u0daf\u0db8\u0dca\u0db6\u0dc3\u0dd9\u0dc0\u0dd3\u0db8, <a href=\"temperature.html\">\u0db4\u0dd2\u0dbb\u0dd2\u0dc3\u0dd2\u0daf\u0dd4 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8, \u0d8b\u0dc2\u0dca\u0dab\u0dad\u0dca\u0dc0 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8 \u0dc3\u0dc4 <a href=\"top_k.html\">\u0d89\u0dc4\u0dc5 \u0d9a\u0dda</a>\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8</a><a href=\"temperature.html\">\u0dc0\u0dd0\u0db1\u0dd2 \u0dc0\u0dd9\u0db1\u0dad\u0dca \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2</a>\u0d9a\u0dca\u0dbb\u0db8\u0dc0\u0dbd \u0d87\u0dad\u0dd2 \u0d9c\u0dd0\u0da7\u0dc5\u0dd4 \u0db4\u0dd2\u0dc5\u0dd2\u0db6\u0db3\u0dc0 \u0d9a\u0da9\u0daf\u0dcf\u0dc3\u0dd2 \u0dc3\u0dcf\u0d9a\u0da0\u0dca\u0da1\u0dcf \u0d9a\u0dbb\u0dba\u0dd2. \u0db8\u0dd9\u0db8 \u0db4\u0dad\u0dca\u0dbb\u0dd2\u0d9a\u0dcf\u0dc0 \u0db1\u0dca\u0dba\u0dc2\u0dca\u0da7\u0dd2\u0d9a \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8\u0dca \u0db4\u0dd2\u0dc5\u0dd2\u0db6\u0db3 \u0d85\u0daf\u0dc4\u0dc3 \u0dc4\u0db3\u0dd4\u0db1\u0dca\u0dc0\u0dcf \u0daf\u0dd9\u0dba\u0dd2, \u0d91\u0dba \u0db4\u0dd9\u0dc5 \u0d8b\u0dad\u0dca\u0db4\u0dcf\u0daf\u0db1\u0dba \u0dc3\u0db3\u0dc4\u0dcf \u0dc0\u0dd9\u0db1\u0dad\u0dca \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0d9a\u0dca\u0dbb\u0db8\u0dc0\u0dbd\u0da7 \u0dc0\u0da9\u0dcf \u0db4\u0dca\u0dbb\u0dcf\u0dba\u0ddd\u0d9c\u0dd2\u0d9a\u0dc0 \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf \u0d9a\u0dbb\u0dba\u0dd2. </p>\n<p>\u0db1\u0dca\u0dba\u0dc2\u0dca\u0da7\u0dd2\u0d9a\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8 \u0db4\u0dc5\u0db8\u0dd4\u0dc0 \u0dc0\u0da0\u0db1 \u0db8\u0dcf\u0dbd\u0dcf\u0dc0\u0dda \u0d8b\u0db4 \u0d9a\u0dd4\u0dbd\u0d9a\u0dba\u0d9a\u0dca \u0dad\u0ddd\u0dbb\u0dcf \u0d9c\u0db1\u0dd3 <span translate=no>_^_0_^_</span>, \u0d91\u0dc4\u0dd2\u0daf\u0dd3 <span translate=no>_^_1_^_</span> \u0d9a\u0dd4\u0da9\u0dcf\u0db8 \u0da7\u0ddd\u0d9a\u0db1 \u0d9a\u0da7\u0dca\u0da7\u0dbd\u0dba\u0d9a\u0dca</p>\n<p><span translate=no>_^_2_^_</span></p>\n<p>\u0d91\u0db1\u0db8\u0dca, \u0d92\u0dc0\u0dcf\u0dba\u0dda \u0dc3\u0db8\u0dca\u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf\u0dc0\u0db1\u0dca\u0d9c\u0dda \u0d91\u0d9a\u0dad\u0dd4\u0dc0 \u0d85\u0da9\u0dd4 \u0dc0\u0db1 \u0dad\u0dd9\u0d9a\u0dca \u0d85\u0db4\u0dd2 \u0d89\u0dc4\u0dc5\u0db8 \u0dc3\u0db8\u0dca\u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf\u0dc0 \u0dc3\u0dc4\u0dd2\u0dad \u0da7\u0ddd\u0d9a\u0db1 \u0dad\u0ddd\u0dbb\u0dcf \u0d9c\u0db1\u0dd2\u0db8\u0dd4 <span translate=no>_^_3_^_</span>. </p>\n<p>\u0d89\u0db1\u0dca\u0db4\u0dc3\u0dd4\u0d85\u0db4\u0dd2 \u0dad\u0ddd\u0dbb\u0dcf\u0d9c\u0dad\u0dca \u0da7\u0ddd\u0d9a\u0db1 \u0dc0\u0dbd\u0dd2\u0db1\u0dca \u0dc3\u0dcf\u0db8\u0dca\u0db4\u0dbd \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dd2\u0db8\u0dd4. </p>\n<p>\u0db8\u0dd9\u0db1\u0dca\u0db1\u0db8\u0dd9\u0db8 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0dc1\u0dd2\u0dbd\u0dca\u0db4\u0dd3\u0dba \u0d9a\u0dca\u0dbb\u0db8 \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dbb\u0db1 <a href=\"experiment.html\">\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf \u0db6\u0dd0\u0dbd\u0dd3\u0db8\u0d9a\u0dca</a> . </p>\n",
|
||||
"<h2>Nucleus Sampler</h2>\n": "<h2>\u0db1\u0dca\u0dba\u0dc2\u0dca\u0da7\u0dd2\u0d9a\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2</h2>\n",
|
||||
"<p> </p>\n": "<p> </p>\n",
|
||||
"<p> Sample from logits with Nucleus Sampling</p>\n": "<p> \u0db1\u0dca\u0dba\u0dc2\u0dca\u0da7\u0dd2\u0d9a\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8 \u0dc3\u0db8\u0d9f \u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca \u0dc0\u0dbd\u0dd2\u0db1\u0dca \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba</p>\n",
|
||||
"<p>Find the cumulative sums less than <span translate=no>_^_0_^_</span>. </p>\n": "<p>\u0dc0\u0da9\u0dcf\u0d85\u0da9\u0dd4 \u0dc3\u0db8\u0dd4\u0da0\u0dca\u0da0\u0dd2\u0dad \u0db8\u0dd4\u0daf\u0dbd\u0d9a\u0dca \u0dc3\u0ddc\u0dba\u0dcf \u0d9c\u0db1\u0dca\u0db1 <span translate=no>_^_0_^_</span>. </p>\n",
|
||||
"<p>Get log probabilities and mask out the non-nucleus </p>\n": "<p>\u0dbd\u0ddc\u0d9c\u0dca\u0dc3\u0db8\u0dca\u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf\u0dc0 \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 \u0dc3\u0dc4 \u0db1\u0dca\u0dba\u0dc2\u0dca\u0da7\u0dd2\u0dba \u0db1\u0ddc\u0dc0\u0db1 \u0dc0\u0dc3\u0d82 \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Get probabilities <span translate=no>_^_0_^_</span> </p>\n": "<p>\u0dc3\u0db8\u0dca\u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf\u0dc0\u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 <span translate=no>_^_0_^_</span> </p>\n",
|
||||
"<p>Get the actual indexes </p>\n": "<p>\u0dc3\u0dad\u0dca\u0dba\u0daf\u0dbb\u0dca\u0dc1\u0d9a \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Get the cumulative sum of probabilities in the sorted order </p>\n": "<p>\u0dc0\u0dbb\u0dca\u0d9c\u0d9a\u0dc5 \u0d85\u0db1\u0dd4\u0db4\u0dd2\u0dc5\u0dd2\u0dc0\u0dd9\u0dbd\u0dd9\u0dc4\u0dd2 \u0dc3\u0db8\u0dd4\u0da0\u0dca\u0da0\u0dd2\u0dad \u0dc3\u0db8\u0dca\u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf\u0dc0\u0db1\u0dca\u0d9c\u0dda \u0d91\u0d9a\u0dad\u0dd4\u0dc0 \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Prepend ones so that we add one token after the minimum number of tokens with cumulative probability less that <span translate=no>_^_0_^_</span>. </p>\n": "<p>\u0dc3\u0db8\u0dd4\u0da0\u0dca\u0da0\u0dd2\u0dad\u0dc3\u0db8\u0dca\u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf\u0dc0 \u0d85\u0da9\u0dd4 \u0d85\u0dc0\u0db8 \u0da7\u0ddd\u0d9a\u0db1 \u0dc3\u0d82\u0d9b\u0dca\u0dba\u0dcf\u0dc0\u0dd9\u0db1\u0dca \u0db4\u0dc3\u0dd4\u0dc0 \u0d85\u0db4\u0dd2 \u0d91\u0d9a\u0dca \u0da7\u0ddd\u0d9a\u0db1\u0dba\u0d9a\u0dca \u0d91\u0d9a\u0dad\u0dd4 \u0d9a\u0dbb\u0db1 \u0db4\u0dbb\u0dd2\u0daf\u0dd2 \u0d92\u0dc0\u0dcf \u0dc3\u0d9a\u0dc3\u0dca \u0d9a\u0dbb\u0db1\u0dca\u0db1 <span translate=no>_^_0_^_</span>. </p>\n",
|
||||
"<p>Sample from the sampler </p>\n": "<p>\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba\u0dd9\u0db1\u0dca\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba </p>\n",
|
||||
"<p>Softmax to compute <span translate=no>_^_0_^_</span> from the logits </p>\n": "<p>\u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca <span translate=no>_^_0_^_</span> \u0dc0\u0dbd\u0dd2\u0db1\u0dca \u0d9c\u0dab\u0db1\u0dba \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0da7 \u0dc3\u0ddc\u0dc6\u0dca\u0da7\u0dca\u0db8\u0dd0\u0d9a\u0dca\u0dc3\u0dca </p>\n",
|
||||
"<p>Sort probabilities in descending order </p>\n": "<p>\u0db6\u0dd0\u0dc3\u0dd3\u0db8\u0dda\u0d85\u0db1\u0dd4\u0db4\u0dd2\u0dc5\u0dd2\u0dc0\u0dd9\u0dbd\u0dd9\u0dc4\u0dd2 \u0dc3\u0db8\u0dca\u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf\u0dc0\u0db1\u0dca \u0dc0\u0dbb\u0dca\u0d9c \u0d9a\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the sum of probabilities of tokens to pick <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> is the sampler to use for the selected tokens</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span> \u0dba\u0db1\u0dd4 \u0dad\u0ddd\u0dbb\u0dcf \u0d9c\u0dd0\u0db1\u0dd3\u0db8\u0da7 \u0da7\u0ddd\u0d9a\u0db1 \u0dc0\u0dbd \u0dc3\u0db8\u0dca\u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf\u0dc0\u0db1\u0dca\u0d9c\u0dda \u0d91\u0d9a\u0dad\u0dd4\u0dc0\u0dba\u0dd2 <span translate=no>_^_1_^_</span> </li>\n</ul><li><span translate=no>_^_2_^_</span> \u0dad\u0ddd\u0dbb\u0dcf\u0d9c\u0dad\u0dca \u0da7\u0ddd\u0d9a\u0db1 \u0dc3\u0db3\u0dc4\u0dcf \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dc5 \u0dba\u0dd4\u0dad\u0dd4 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0d9a\u0dbb\u0dd4 \u0dc0\u0dda</li>\n",
|
||||
"A PyTorch implementation of nucleus sampling from language models.": "\u0db7\u0dcf\u0dc2\u0dcf \u0d86\u0d9a\u0dd8\u0dad\u0dd2 \u0dc0\u0dbd\u0dd2\u0db1\u0dca \u0db1\u0dca\u0dba\u0dc2\u0dca\u0da7\u0dd2\u0d9a \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8\u0dca PyTorch \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dd2\u0dbb\u0dd3\u0db8.",
|
||||
"Nucleus Sampling": "\u0db1\u0dca\u0dba\u0dc2\u0dca\u0da7\u0dd2\u0d9a \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8"
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
{
|
||||
"<h1>Nucleus Sampling</h1>\n<p>This is an implementation of nucleus sampling, introduced in the paper <a href=\"https://arxiv.org/abs/1904.09751\">The Curious Case of Neural Text Degeneration</a>.</p>\n<p>The paper discusses the problems with other sampling methods such as Beam Search, <a href=\"temperature.html\">Pure sampling</a>, <a href=\"temperature.html\">Temperature sampling</a>, and <a href=\"top_k.html\">Top-k sampling</a>. The paper introduces the idea of nucleus sampling, which practically performs better than other sampling methods for text generation.</p>\n<p>Nucleus sampling first picks a subset of the vocabulary <span translate=no>_^_0_^_</span>, where <span translate=no>_^_1_^_</span> is smallest set of tokens such that</p>\n<p><span translate=no>_^_2_^_</span></p>\n<p>That is, we pick the highest probable tokens until the sum of their probabilities is less that <span translate=no>_^_3_^_</span>.</p>\n<p>Then we sample from the selected tokens.</p>\n<p>Here's an <a href=\"experiment.html\">experiment</a> that uses these sampling techniques.</p>\n": "<h1>\u539f\u5b50\u6838\u91c7\u6837</h1>\n<p>\u8fd9\u662f\u539f\u5b50\u6838\u91c7\u6837\u7684\u4e00\u79cd\u5b9e\u73b0\uff0c\u5728\u8bba\u6587<a href=\"https://arxiv.org/abs/1904.09751\">\u300a\u795e\u7ecf\u6587\u672c\u53d8\u6027\u7684\u597d\u5947\u6848\u4f8b\u300b</a>\u4e2d\u8fdb\u884c\u4e86\u4ecb\u7ecd\u3002</p>\n<p>\u672c\u6587\u8ba8\u8bba\u4e86\u5176\u4ed6\u91c7\u6837\u65b9\u6cd5\uff08\u4f8b\u5982\u5149\u675f\u641c\u7d22\u3001<a href=\"temperature.html\">\u7eaf\u91c7\u6837\u3001<a href=\"temperature.html\">\u6e29\u5ea6</a>\u91c7\u6837</a>\u548cT <a href=\"top_k.html\">op-K\u91c7\u6837</a>\uff09\u5b58\u5728\u7684\u95ee\u9898\u3002\u672c\u6587\u4ecb\u7ecd\u4e86\u539f\u5b50\u6838\u91c7\u6837\u7684\u6982\u5ff5\uff0c\u5728\u6587\u672c\u751f\u6210\u65b9\u9762\uff0c\u6838\u91c7\u6837\u7684\u6548\u679c\u5b9e\u9645\u4e0a\u6bd4\u5176\u4ed6\u91c7\u6837\u65b9\u6cd5\u8981\u597d\u3002</p>\n<p>Nucleus \u91c7\u6837\u9996\u5148\u9009\u62e9\u8bcd\u6c47\u7684\u4e00\u4e2a\u5b50\u96c6<span translate=no>_^_0_^_</span>\uff0c\u5176\u4e2d<span translate=no>_^_1_^_</span>\u662f\u6700\u5c0f\u7684\u4ee4\u724c\u96c6\u5408</p>\n<p><span translate=no>_^_2_^_</span></p>\n<p>\u4e5f\u5c31\u662f\u8bf4\uff0c\u6211\u4eec\u9009\u62e9\u53ef\u80fd\u6027\u6700\u9ad8\u7684\u4ee3\u5e01\uff0c\u76f4\u5230\u5b83\u4eec\u7684\u6982\u7387\u603b\u548c\u5c0f\u4e8e\u8be5\u503c\u4e3a\u6b62<span translate=no>_^_3_^_</span>\u3002</p>\n<p>\u7136\u540e\u6211\u4eec\u4ece\u9009\u5b9a\u7684\u4ee4\u724c\u4e2d\u62bd\u6837\u3002</p>\n<p>\u8fd9\u662f\u4e00\u4e2a\u4f7f\u7528\u8fd9\u4e9b\u91c7\u6837\u6280\u672f\u7684<a href=\"experiment.html\">\u5b9e\u9a8c</a>\u3002</p>\n",
|
||||
"<h2>Nucleus Sampler</h2>\n": "<h2>Nucleus \u91c7\u6837\u5668</h2>\n",
|
||||
"<p> </p>\n": "<p></p>\n",
|
||||
"<p> Sample from logits with Nucleus Sampling</p>\n": "<p>\u4f7f\u7528 Nucleus \u91c7\u6837\u4ece logits \u4e2d\u63d0\u53d6\u6837\u672c</p>\n",
|
||||
"<p>Find the cumulative sums less than <span translate=no>_^_0_^_</span>. </p>\n": "<p>\u627e\u51fa\u5c0f\u4e8e\u7684\u7d2f\u8ba1\u603b\u548c<span translate=no>_^_0_^_</span>\u3002</p>\n",
|
||||
"<p>Get log probabilities and mask out the non-nucleus </p>\n": "<p>\u83b7\u53d6\u5bf9\u6570\u6982\u7387\u5e76\u63a9\u76d6\u975e\u6838</p>\n",
|
||||
"<p>Get probabilities <span translate=no>_^_0_^_</span> </p>\n": "<p>\u83b7\u53d6\u6982\u7387<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<p>Get the actual indexes </p>\n": "<p>\u83b7\u53d6\u5b9e\u9645\u7d22\u5f15</p>\n",
|
||||
"<p>Get the cumulative sum of probabilities in the sorted order </p>\n": "<p>\u6309\u6392\u5e8f\u987a\u5e8f\u83b7\u53d6\u6982\u7387\u7684\u7d2f\u79ef\u603b\u548c</p>\n",
|
||||
"<p>Prepend ones so that we add one token after the minimum number of tokens with cumulative probability less that <span translate=no>_^_0_^_</span>. </p>\n": "\u5728@@ <p>\u524d\u9762\u52a0\u4e00\u4e2a\uff0c\u8fd9\u6837\u6211\u4eec\u5c31\u53ef\u4ee5\u5728\u7d2f\u79ef\u6982\u7387\u5c0f\u4e8e\u8be5\u503c\u7684\u6700\u5c0f\u4ee3\u5e01\u6570\u91cf\u4e4b\u540e\u6dfb\u52a0\u4e00\u4e2a\u4ee4\u724c<span translate=no>_^_0_^_</span>\u3002</p>\n",
|
||||
"<p>Sample from the sampler </p>\n": "<p>\u6765\u81ea\u91c7\u6837\u5668\u7684\u6837\u672c</p>\n",
|
||||
"<p>Softmax to compute <span translate=no>_^_0_^_</span> from the logits </p>\n": "<p>\u8981\u6839\u636e\u5bf9\u6570\u8ba1\u7b97<span translate=no>_^_0_^_</span>\u7684 softmax</p>\n",
|
||||
"<p>Sort probabilities in descending order </p>\n": "<p>\u6309\u964d\u5e8f\u5bf9\u6982\u7387\u8fdb\u884c\u6392\u5e8f</p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the sum of probabilities of tokens to pick <span translate=no>_^_1_^_</span> </li>\n<li><span translate=no>_^_2_^_</span> is the sampler to use for the selected tokens</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u662f\u8981\u9009\u62e9\u7684\u4ee3\u5e01\u6982\u7387\u4e4b\u548c<span translate=no>_^_1_^_</span></li>\n<li><span translate=no>_^_2_^_</span>\u662f\u7528\u4e8e\u9009\u5b9a\u4ee4\u724c\u7684\u91c7\u6837\u5668</li></ul>\n",
|
||||
"A PyTorch implementation of nucleus sampling from language models.": "\u4ece\u8bed\u8a00\u6a21\u578b\u8fdb\u884c\u6838\u91c7\u6837\u7684 PyTorch \u5b9e\u73b0\u3002",
|
||||
"Nucleus Sampling": "\u539f\u5b50\u6838\u91c7\u6837"
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"<h1>Sampling from Language Models with Temperature</h1>\n<p>Here we sample from the following probability distribution where <span translate=no>_^_0_^_</span> is the vocabulary, <span translate=no>_^_1_^_</span> are the logits of the distribution and T is the temperature:</p>\n<p><span translate=no>_^_2_^_</span></p>\n<p><span translate=no>_^_3_^_</span> is normal random sampling.</p>\n<p>Here's an <a href=\"experiment.html\">experiment</a> that uses these sampling techniques.</p>\n": "<h1>\u6e29\u5ea6\u3092\u7528\u3044\u305f\u8a00\u8a9e\u30e2\u30c7\u30eb\u304b\u3089\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</h1>\n<p>\u3053\u3053\u3067\u306f\u3001\u6b21\u306e\u78ba\u7387\u5206\u5e03\u304b\u3089\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3057\u307e\u3059\u3002\u3053\u3053\u3067\u3001<span translate=no>_^_0_^_</span>\u306f\u30dc\u30ad\u30e3\u30d6\u30e9\u30ea\u30fc\u3001<span translate=no>_^_1_^_</span>\u306f\u5206\u5e03\u306e\u30ed\u30b8\u30c3\u30c8\u3001T\u306f\u6e29\u5ea6\u3067\u3059\u3002</p>\n<p><span translate=no>_^_2_^_</span></p>\n<p><span translate=no>_^_3_^_</span>\u306f\u901a\u5e38\u306e\u30e9\u30f3\u30c0\u30e0\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3067\u3059\u3002</p>\n<p>\u3053\u308c\u306f\u3001<a href=\"experiment.html\">\u3053\u308c\u3089\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u624b\u6cd5\u3092\u4f7f\u7528\u3057\u305f\u5b9f\u9a13\u3067\u3059</a>\u3002</p>\n",
|
||||
"<h2>Sampler with Temperature</h2>\n": "<h2>\u6e29\u5ea6\u6a5f\u80fd\u4ed8\u304d\u30b5\u30f3\u30d7\u30e9\u30fc</h2>\n",
|
||||
"<p> Sample from logits</p>\n": "<p>\u30ed\u30b8\u30c3\u30c8\u304b\u3089\u306e\u30b5\u30f3\u30d7\u30eb</p>\n",
|
||||
"<p>Create a categorical distribution with temperature adjusted logits </p>\n": "<p>\u6e29\u5ea6\u8abf\u6574\u6e08\u307f\u30ed\u30b8\u30c3\u30c8\u306b\u3088\u308b\u30ab\u30c6\u30b4\u30ea\u5206\u5e03\u306e\u4f5c\u6210</p>\n",
|
||||
"<p>Sample </p>\n": "<p>[\u30b5\u30f3\u30d7\u30eb]</p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the temperature to sample with</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u306f\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3059\u308b\u6e29\u5ea6</li></ul>\n",
|
||||
"A PyTorch implementation of sampling from language models with temperature.": "\u6e29\u5ea6\u3092\u542b\u3080\u8a00\u8a9e\u30e2\u30c7\u30eb\u304b\u3089\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u306e PyTorch \u5b9f\u88c5\u3002",
|
||||
"Sampling from Language Models with Temperature": "\u6e29\u5ea6\u3092\u7528\u3044\u305f\u8a00\u8a9e\u30e2\u30c7\u30eb\u304b\u3089\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0"
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"<h1>Sampling from Language Models with Temperature</h1>\n<p>Here we sample from the following probability distribution where <span translate=no>_^_0_^_</span> is the vocabulary, <span translate=no>_^_1_^_</span> are the logits of the distribution and T is the temperature:</p>\n<p><span translate=no>_^_2_^_</span></p>\n<p><span translate=no>_^_3_^_</span> is normal random sampling.</p>\n<p>Here's an <a href=\"experiment.html\">experiment</a> that uses these sampling techniques.</p>\n": "<h1>\u0d8b\u0dc2\u0dca\u0dab\u0dad\u0dca\u0dc0\u0dba\u0dc3\u0db8\u0d9f \u0db7\u0dcf\u0dc2\u0dcf \u0d86\u0d9a\u0dd8\u0dad\u0dd2 \u0dc0\u0dbd\u0dd2\u0db1\u0dca \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8</h1>\n<p>\u0db8\u0dd9\u0db1\u0dca\u0db1\u0d85\u0db4\u0dd2 \u0db4\u0dc4\u0dad \u0dc3\u0db3\u0dc4\u0db1\u0dca \u0dc3\u0db8\u0dca\u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0dc0\u0dca\u0dba\u0dcf\u0db4\u0dca\u0dad\u0dd2\u0dba\u0dd9\u0db1\u0dca \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba, <span translate=no>_^_1_^_</span> \u0dc0\u0dcf\u0d9c\u0dca \u0db8\u0dcf\u0dbd\u0dcf\u0dc0 \u0d9a\u0ddc\u0dc4\u0dd9\u0daf <span translate=no>_^_0_^_</span> , \u0db6\u0dd9\u0daf\u0dcf \u0dc4\u0dd0\u0dbb\u0dd3\u0db8\u0dda \u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca \u0dc3\u0dc4 T \u0dba\u0db1\u0dd4 \u0d8b\u0dc2\u0dca\u0dab\u0dad\u0dca\u0dc0\u0dba:</p>\n<p><span translate=no>_^_2_^_</span></p>\n<p><span translate=no>_^_3_^_</span> \u0dc3\u0dcf\u0db8\u0dcf\u0db1\u0dca\u0dba \u0d85\u0dc4\u0db9\u0dd4 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8\u0dca \u0dc0\u0dda. </p>\n<p>\u0db8\u0dd9\u0db1\u0dca\u0db1\u0db8\u0dd9\u0db8 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0dc1\u0dd2\u0dbd\u0dca\u0db4\u0dd3\u0dba \u0d9a\u0dca\u0dbb\u0db8 \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dbb\u0db1 <a href=\"experiment.html\">\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf \u0db6\u0dd0\u0dbd\u0dd3\u0db8\u0d9a\u0dca</a> . </p>\n",
|
||||
"<h2>Sampler with Temperature</h2>\n": "<h2>\u0d8b\u0dc2\u0dca\u0dab\u0dad\u0dca\u0dc0\u0dba\u0dc3\u0db8\u0d9f \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0d9a\u0dbb\u0dd4</h2>\n",
|
||||
"<p> Sample from logits</p>\n": "<p> \u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca\u0dc0\u0dbd\u0dd2\u0db1\u0dca \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba</p>\n",
|
||||
"<p>Create a categorical distribution with temperature adjusted logits </p>\n": "<p>\u0d8b\u0dc2\u0dca\u0dab\u0dad\u0dca\u0dc0\u0d9c\u0dd0\u0dbd\u0db4\u0dd4\u0db8\u0dca \u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca \u0dc3\u0db8\u0d9f \u0dc0\u0dbb\u0dca\u0d9c\u0dd3\u0d9a\u0dbb\u0dab \u0db6\u0dd9\u0daf\u0dcf\u0dc4\u0dd0\u0dbb\u0dd3\u0db8\u0d9a\u0dca \u0dc3\u0dcf\u0daf\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Sample </p>\n": "<p>\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba </p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the temperature to sample with</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span> \u0dc3\u0db8\u0d9f \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba \u0dba\u0dd4\u0dad\u0dd4 \u0d8b\u0dc2\u0dca\u0dab\u0dad\u0dca\u0dc0\u0dba \u0dc0\u0dda</li></ul>\n",
|
||||
"A PyTorch implementation of sampling from language models with temperature.": "\u0d8b\u0dc2\u0dca\u0dab\u0dad\u0dca\u0dc0\u0dba \u0dc3\u0dc4\u0dd2\u0dad \u0db7\u0dcf\u0dc2\u0dcf \u0d86\u0d9a\u0dd8\u0dad\u0dd2 \u0dc0\u0dbd\u0dd2\u0db1\u0dca \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8\u0dca PyTorch \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dd2\u0dbb\u0dd3\u0db8.",
|
||||
"Sampling from Language Models with Temperature": "\u0d8b\u0dc2\u0dca\u0dab\u0dad\u0dca\u0dc0\u0dba \u0dc3\u0db8\u0d9f \u0db7\u0dcf\u0dc2\u0dcf \u0d86\u0d9a\u0dd8\u0dad\u0dd2 \u0dc0\u0dbd\u0dd2\u0db1\u0dca \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8"
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"<h1>Sampling from Language Models with Temperature</h1>\n<p>Here we sample from the following probability distribution where <span translate=no>_^_0_^_</span> is the vocabulary, <span translate=no>_^_1_^_</span> are the logits of the distribution and T is the temperature:</p>\n<p><span translate=no>_^_2_^_</span></p>\n<p><span translate=no>_^_3_^_</span> is normal random sampling.</p>\n<p>Here's an <a href=\"experiment.html\">experiment</a> that uses these sampling techniques.</p>\n": "<h1>\u4ece\u5e26\u6e29\u5ea6\u7684\u8bed\u8a00\u6a21\u578b\u4e2d\u91c7\u6837</h1>\n<p>\u5728\u8fd9\u91cc\uff0c\u6211\u4eec\u4ece\u4ee5\u4e0b\u6982\u7387\u5206\u5e03\u4e2d\u62bd\u6837\uff0c\u5176\u4e2d<span translate=no>_^_0_^_</span>\u662f\u8bcd\u6c47\uff0c<span translate=no>_^_1_^_</span>\u662f\u5206\u5e03\u7684\u5bf9\u6570\uff0cT \u662f\u6e29\u5ea6\uff1a</p>\n<p><span translate=no>_^_2_^_</span></p>\n<p><span translate=no>_^_3_^_</span>\u662f\u6b63\u5e38\u7684\u968f\u673a\u62bd\u6837\u3002</p>\n<p>\u8fd9\u662f\u4e00\u4e2a\u4f7f\u7528\u8fd9\u4e9b\u91c7\u6837\u6280\u672f\u7684<a href=\"experiment.html\">\u5b9e\u9a8c</a>\u3002</p>\n",
|
||||
"<h2>Sampler with Temperature</h2>\n": "<h2>\u5e26\u6e29\u5ea6\u7684\u91c7\u6837\u5668</h2>\n",
|
||||
"<p> Sample from logits</p>\n": "<p>\u6765\u81ea logits \u7684\u6837\u672c</p>\n",
|
||||
"<p>Create a categorical distribution with temperature adjusted logits </p>\n": "<p>\u4f7f\u7528\u6e29\u5ea6\u8c03\u6574\u540e\u7684\u5bf9\u6570\u521b\u5efa\u5206\u7c7b\u5206\u5e03</p>\n",
|
||||
"<p>Sample </p>\n": "<p>\u6837\u672c</p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the temperature to sample with</li></ul>\n": "<ul><li><span translate=no>_^_0_^_</span>\u662f\u8981\u91c7\u6837\u7684\u6e29\u5ea6</li></ul>\n",
|
||||
"A PyTorch implementation of sampling from language models with temperature.": "\u4f7f\u7528\u6e29\u5ea6\u4ece\u8bed\u8a00\u6a21\u578b\u4e2d\u91c7\u6837\u7684 PyTorch \u5b9e\u73b0\u3002",
|
||||
"Sampling from Language Models with Temperature": "\u4ece\u5e26\u6e29\u5ea6\u7684\u8bed\u8a00\u6a21\u578b\u4e2d\u91c7\u6837"
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"<h1>Top-k Sampling</h1>\n<p>Here we first pick the top-k tokens from the distribution of logits, and then sample from them.</p>\n<p>Here's an <a href=\"experiment.html\">experiment</a> that uses these sampling techniques.</p>\n": "<h1>\u30c8\u30c3\u30d7k\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0</h1>\n<p>\u3053\u3053\u3067\u306f\u3001\u6700\u521d\u306b\u30ed\u30b8\u30c3\u30c8\u306e\u5206\u5e03\u304b\u3089\u4e0a\u4f4dk\u500b\u306e\u30c8\u30fc\u30af\u30f3\u3092\u9078\u629e\u3057\u3001\u6b21\u306b\u305d\u308c\u3089\u304b\u3089\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3057\u307e\u3059\u3002</p>\n<p>\u3053\u308c\u306f\u3001<a href=\"experiment.html\">\u3053\u308c\u3089\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u624b\u6cd5\u3092\u4f7f\u7528\u3057\u305f\u5b9f\u9a13\u3067\u3059</a>\u3002</p>\n",
|
||||
"<h2>Top-k Sampler</h2>\n": "<h2>\u30c8\u30c3\u30d7k\u30b5\u30f3\u30d7\u30e9\u30fc</h2>\n",
|
||||
"<p> Sample from logits</p>\n": "<p>\u30ed\u30b8\u30c3\u30c8\u304b\u3089\u306e\u30b5\u30f3\u30d7\u30eb</p>\n",
|
||||
"<p>New logits filled with <span translate=no>_^_0_^_</span>; i.e. zero probability </p>\n": "<p><span translate=no>_^_0_^_</span>\u65b0\u3057\u3044\u30ed\u30b8\u30c3\u30c8\u3092\u57cb\u3081\u308b\u3001\u3064\u307e\u308a\u78ba\u7387\u304c\u30bc\u30ed</p>\n",
|
||||
"<p>Pick the largest <span translate=no>_^_0_^_</span> logits and their indices </p>\n": "<p><span translate=no>_^_0_^_</span>\u6700\u5927\u306e\u30ed\u30b8\u30c3\u30c8\u3068\u305d\u306e\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u3092\u9078\u629e\u3057\u3066\u304f\u3060\u3055\u3044</p>\n",
|
||||
"<p>Sample from the top-k logits with the specified sampler. </p>\n": "<p>\u6307\u5b9a\u3055\u308c\u305f\u30b5\u30f3\u30d7\u30e9\u30fc\u3092\u4f7f\u7528\u3057\u3066\u3001\u4e0a\u304b\u3089k\u500b\u306e\u30ed\u30b8\u30c3\u30c8\u3092\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3057\u307e\u3059\u3002</p>\n",
|
||||
"<p>Set the values of the top-k selected indices to actual logits. Logits of other tokens remain <span translate=no>_^_0_^_</span> </p>\n": "<p>\u9078\u629e\u3057\u305f\u4e0a\u4f4dk\u306e\u30a4\u30f3\u30c7\u30c3\u30af\u30b9\u306e\u5024\u3092\u5b9f\u969b\u306e\u30ed\u30b8\u30c3\u30c8\u306b\u8a2d\u5b9a\u3057\u307e\u3059\u3002\u4ed6\u306e\u30c8\u30fc\u30af\u30f3\u306e\u30ed\u30b8\u30c3\u30c8\u306f\u6b8b\u308a\u307e\u3059 <span translate=no>_^_0_^_</span></p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the number of tokens to pick </li>\n<li><span translate=no>_^_1_^_</span> is the sampler to use for the top-k tokens</li></ul>\n<p><span translate=no>_^_2_^_</span> can be any sampler that takes a logits tensor as input and returns a token tensor; e.g. <a href=\"temperature.html\">`TemperatureSampler'</a>.</p>\n": "<ul><li><span translate=no>_^_0_^_</span>\u9078\u629e\u3059\u308b\u30c8\u30fc\u30af\u30f3\u306e\u6570\u3067\u3059</li>\n<li><span translate=no>_^_1_^_</span>\u30c8\u30c3\u30d7k\u306e\u30c8\u30fc\u30af\u30f3\u306b\u4f7f\u7528\u3059\u308b\u30b5\u30f3\u30d7\u30e9\u30fc\u3067\u3059</li></ul>\n<p><span translate=no>_^_2_^_</span><a href=\"temperature.html\">\u30ed\u30b8\u30c3\u30c4\u30c6\u30f3\u30bd\u30eb\u3092\u5165\u529b\u3068\u3057\u3066\u53d7\u3051\u53d6\u308a\u3001\u30c8\u30fc\u30af\u30f3\u30c6\u30f3\u30bd\u30eb\u3092\u8fd4\u3059\u30b5\u30f3\u30d7\u30e9\u30fc\u306a\u3089\u3069\u308c\u3067\u3082\u304b\u307e\u3044\u307e\u305b\u3093\uff08\u4f8b\uff1a`TemperatureSampler'\uff09\u3002</a></p>\n",
|
||||
"A PyTorch implementation of top-k sampling from language models.": "\u8a00\u8a9e\u30e2\u30c7\u30eb\u304b\u3089\u306e top-k \u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u306e PyTorch \u5b9f\u88c5\u3002",
|
||||
"Top-k Sampling": "\u30c8\u30c3\u30d7k\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0"
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"<h1>Top-k Sampling</h1>\n<p>Here we first pick the top-k tokens from the distribution of logits, and then sample from them.</p>\n<p>Here's an <a href=\"experiment.html\">experiment</a> that uses these sampling techniques.</p>\n": "<h1>\u0d89\u0dc4\u0dc5-K\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8\u0dca</h1>\n<p>\u0db8\u0dd9\u0db1\u0dca\u0db1\u0d85\u0db4\u0dd2 \u0db4\u0dc5\u0db8\u0dd4\u0dc0 \u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca \u0db6\u0dd9\u0daf\u0dcf \u0dc4\u0dd0\u0dbb\u0dd3\u0db8\u0dd9\u0db1\u0dca \u0d89\u0dc4\u0dc5\u0db8 \u0d9a\u0dda \u0da7\u0ddd\u0d9a\u0db1 \u0dad\u0ddd\u0dbb\u0dcf\u0d9c\u0dd9\u0db1 \u0d92\u0dc0\u0dcf\u0dba\u0dd2\u0db1\u0dca \u0dc3\u0dcf\u0db8\u0dca\u0db4\u0dbd \u0dbd\u0db6\u0dcf \u0d9c\u0db1\u0dd2\u0db8\u0dd4. </p>\n<p>\u0db8\u0dd9\u0db1\u0dca\u0db1\u0db8\u0dd9\u0db8 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2 \u0dc1\u0dd2\u0dbd\u0dca\u0db4\u0dd3\u0dba \u0d9a\u0dca\u0dbb\u0db8 \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dbb\u0db1 <a href=\"experiment.html\">\u0d85\u0dad\u0dca\u0dc4\u0daf\u0dcf \u0db6\u0dd0\u0dbd\u0dd3\u0db8\u0d9a\u0dca</a> . </p>\n",
|
||||
"<h2>Top-k Sampler</h2>\n": "<h2>\u0d89\u0dc4\u0dc5-K\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2</h2>\n",
|
||||
"<p> Sample from logits</p>\n": "<p> \u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca\u0dc0\u0dbd\u0dd2\u0db1\u0dca \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba</p>\n",
|
||||
"<p>New logits filled with <span translate=no>_^_0_^_</span>; i.e. zero probability </p>\n": "<p>\u0db1\u0dc0\u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca \u0db4\u0dd2\u0dbb\u0dd3 \u0d87\u0dad <span translate=no>_^_0_^_</span>; i.e. \u0dc1\u0dd4\u0db1\u0dca\u0dba \u0dc3\u0db8\u0dca\u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf\u0dc0 </p>\n",
|
||||
"<p>Pick the largest <span translate=no>_^_0_^_</span> logits and their indices </p>\n": "<p>\u0dc0\u0dd2\u0dc1\u0dcf\u0dbd\u0dad\u0db8 <span translate=no>_^_0_^_</span> \u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca \u0dc3\u0dc4 \u0d92\u0dc0\u0dcf\u0dba\u0dda \u0daf\u0dbb\u0dca\u0dc1\u0d9a \u0dad\u0ddd\u0dbb\u0db1\u0dca\u0db1 </p>\n",
|
||||
"<p>Sample from the top-k logits with the specified sampler. </p>\n": "<p>\u0db1\u0dd2\u0dc1\u0dca\u0da0\u0dd2\u0dad\u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba \u0dc3\u0db8\u0d9f \u0d89\u0dc4\u0dc5-k \u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca \u0dc0\u0dbd\u0dd2\u0db1\u0dca \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba. </p>\n",
|
||||
"<p>Set the values of the top-k selected indices to actual logits. Logits of other tokens remain <span translate=no>_^_0_^_</span> </p>\n": "<p>Top-k\u0dad\u0ddd\u0dbb\u0dcf\u0d9c\u0dad\u0dca \u0daf\u0dbb\u0dca\u0dc1\u0d9a\u0dc0\u0dbd \u0d85\u0d9c\u0dba\u0db1\u0dca \u0dc3\u0dd0\u0db6\u0dd1 \u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca \u0dc0\u0dbd\u0da7 \u0dc3\u0d9a\u0dc3\u0db1\u0dca\u0db1. \u0dc0\u0dd9\u0db1\u0dad\u0dca \u0da7\u0ddd\u0d9a\u0db1 \u0dc0\u0dbd \u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca \u0d89\u0dad\u0dd2\u0dbb\u0dd2\u0dc0 \u0db4\u0dc0\u0dad\u0dd3 <span translate=no>_^_0_^_</span> </p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the number of tokens to pick </li>\n<li><span translate=no>_^_1_^_</span> is the sampler to use for the top-k tokens</li></ul>\n<p><span translate=no>_^_2_^_</span> can be any sampler that takes a logits tensor as input and returns a token tensor; e.g. <a href=\"temperature.html\">`TemperatureSampler'</a>.</p>\n": "<ul><li><span translate=no>_^_0_^_</span> \u0dad\u0ddd\u0dbb\u0dcf \u0d9c\u0dd0\u0db1\u0dd3\u0db8\u0da7 \u0da7\u0ddd\u0d9a\u0db1 \u0d9c\u0dab\u0db1 \u0dc0\u0dda </li>\n</ul><li><span translate=no>_^_1_^_</span> \u0d89\u0dc4\u0dc5-k \u0da7\u0ddd\u0d9a\u0db1 \u0dc3\u0db3\u0dc4\u0dcf \u0db7\u0dcf\u0dc0\u0dd2\u0dad\u0dcf \u0d9a\u0dd2\u0dbb\u0dd3\u0db8\u0da7 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0d9a\u0dbb\u0dd4 \u0dc0\u0dda</li>\n<p><span translate=no>_^_2_^_</span> \u0db4\u0dd2\u0dc0\u0dd2\u0dc3\u0dd4\u0db8\u0dca \u0da7\u0dd9\u0db1\u0dca\u0dc3\u0dbb\u0dba\u0d9a\u0dca \u0d86\u0daf\u0dcf\u0db1 \u0dbd\u0dd9\u0dc3 \u0d9c\u0dd9\u0db1 \u0da7\u0ddd\u0d9a\u0db1\u0dca \u0da7\u0dd9\u0db1\u0dca\u0dc3\u0dbb\u0dba\u0d9a\u0dca \u0db1\u0dd0\u0dc0\u0dad \u0dbd\u0db6\u0dcf \u0daf\u0dd9\u0db1 \u0d95\u0db1\u0dd1\u0db8 \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd2\u0dba\u0d9a \u0dc0\u0dd2\u0dba \u0dc4\u0dd0\u0d9a\u0dd2\u0dba; \u0d8b\u0daf\u0dcf: <a href=\"temperature.html\">`\u0d8b\u0dc2\u0dca\u0dab\u0dad\u0dca\u0dc0 \u0dc3\u0dcf\u0db8\u0dca\u0db4\u0dbd\u0dba\u0d9a\u0dca'</a>. </p>\n",
|
||||
"A PyTorch implementation of top-k sampling from language models.": "\u0db7\u0dcf\u0dc2\u0dcf \u0d86\u0d9a\u0dd8\u0dad\u0dd2 \u0dc0\u0dbd\u0dd2\u0db1\u0dca \u0d89\u0dc4\u0dc5 \u0d9a\u0dda \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8\u0dca PyTorch \u0d9a\u0dca\u0dbb\u0dd2\u0dba\u0dcf\u0dad\u0dca\u0db8\u0d9a \u0d9a\u0dd2\u0dbb\u0dd3\u0db8.",
|
||||
"Top-k Sampling": "\u0d89\u0dc4\u0dc5-K \u0db1\u0dd2\u0dba\u0dd0\u0daf\u0dd3\u0db8\u0dca"
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
{
|
||||
"<h1>Top-k Sampling</h1>\n<p>Here we first pick the top-k tokens from the distribution of logits, and then sample from them.</p>\n<p>Here's an <a href=\"experiment.html\">experiment</a> that uses these sampling techniques.</p>\n": "<h1>\u524d k \u4e2a\u91c7\u6837</h1>\n<p>\u5728\u8fd9\u91cc\uff0c\u6211\u4eec\u9996\u5148\u4ecelogits\u5206\u5e03\u4e2d\u6311\u9009top-k\u4ee3\u5e01\uff0c\u7136\u540e\u4ece\u4e2d\u91c7\u6837\u3002</p>\n<p>\u8fd9\u662f\u4e00\u4e2a\u4f7f\u7528\u8fd9\u4e9b\u91c7\u6837\u6280\u672f\u7684<a href=\"experiment.html\">\u5b9e\u9a8c</a>\u3002</p>\n",
|
||||
"<h2>Top-k Sampler</h2>\n": "<h2>Top-k \u91c7\u6837\u5668</h2>\n",
|
||||
"<p> Sample from logits</p>\n": "<p>\u6765\u81ea logits \u7684\u6837\u672c</p>\n",
|
||||
"<p>New logits filled with <span translate=no>_^_0_^_</span>; i.e. zero probability </p>\n": "<p>\u65b0\u7684 logit \u586b\u5145\u4e86<span translate=no>_^_0_^_</span>\uff1b\u5373\u96f6\u6982\u7387</p>\n",
|
||||
"<p>Pick the largest <span translate=no>_^_0_^_</span> logits and their indices </p>\n": "<p>\u9009\u62e9\u6700\u5927\u7684\u5bf9<span translate=no>_^_0_^_</span>\u6570\u53ca\u5176\u6307\u6570</p>\n",
|
||||
"<p>Sample from the top-k logits with the specified sampler. </p>\n": "<p>\u4f7f\u7528\u6307\u5b9a\u91c7\u6837\u5668\u4ece top-k logits \u4e2d\u62bd\u6837\u3002</p>\n",
|
||||
"<p>Set the values of the top-k selected indices to actual logits. Logits of other tokens remain <span translate=no>_^_0_^_</span> </p>\n": "<p>\u5c06\u9009\u5b9a\u524d k \u4e2a\u7d22\u5f15\u7684\u503c\u8bbe\u7f6e\u4e3a\u5b9e\u9645\u5bf9\u6570\u3002\u5176\u4ed6\u4ee3\u5e01\u7684\u8bb0\u5f55\u4ecd\u7136\u5b58\u5728<span translate=no>_^_0_^_</span></p>\n",
|
||||
"<ul><li><span translate=no>_^_0_^_</span> is the number of tokens to pick </li>\n<li><span translate=no>_^_1_^_</span> is the sampler to use for the top-k tokens</li></ul>\n<p><span translate=no>_^_2_^_</span> can be any sampler that takes a logits tensor as input and returns a token tensor; e.g. <a href=\"temperature.html\">`TemperatureSampler'</a>.</p>\n": "<ul><li><span translate=no>_^_0_^_</span>\u662f\u8981\u6311\u9009\u7684\u4ee3\u5e01\u6570\u91cf</li>\n<li><span translate=no>_^_1_^_</span>\u662f\u7528\u4e8e\u524d k \u4e2a\u4ee3\u5e01\u7684\u91c7\u6837\u5668</li></ul>\n<p><span translate=no>_^_2_^_</span>\u53ef\u4ee5\u662f\u4efb\u4f55\u4ee5 logits \u5f20\u91cf\u4f5c\u4e3a\u8f93\u5165\u5e76\u8fd4\u56de\u4ee4\u724c\u5f20\u91cf\u7684\u91c7\u6837\u5668\uff1b\u4f8b\u5982 <a href=\"temperature.html\">\u201cTemperatureSample\u201d</a>\u3002</p>\n",
|
||||
"A PyTorch implementation of top-k sampling from language models.": "\u6765\u81ea\u8bed\u8a00\u6a21\u578b\u7684 top-k \u91c7\u6837\u7684 PyTorch \u5b9e\u73b0\u3002",
|
||||
"Top-k Sampling": "\u524d k \u4e2a\u91c7\u6837"
|
||||
}
|
||||
Reference in New Issue
Block a user