{ "

Sampling from Language Models with Temperature

\n

Here we sample from the following probability distribution where _^_0_^_ is the vocabulary, _^_1_^_ are the logits of the distribution and T is the temperature:

\n

_^_2_^_

\n

_^_3_^_ is normal random sampling.

\n

Here's an experiment that uses these sampling techniques.

\n": "

\u6e29\u5ea6\u3092\u7528\u3044\u305f\u8a00\u8a9e\u30e2\u30c7\u30eb\u304b\u3089\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0

\n

\u3053\u3053\u3067\u306f\u3001\u6b21\u306e\u78ba\u7387\u5206\u5e03\u304b\u3089\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3057\u307e\u3059\u3002\u3053\u3053\u3067\u3001_^_0_^_\u306f\u30dc\u30ad\u30e3\u30d6\u30e9\u30ea\u30fc\u3001_^_1_^_\u306f\u5206\u5e03\u306e\u30ed\u30b8\u30c3\u30c8\u3001T\u306f\u6e29\u5ea6\u3067\u3059\u3002

\n

_^_2_^_

\n

_^_3_^_\u306f\u901a\u5e38\u306e\u30e9\u30f3\u30c0\u30e0\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u3067\u3059\u3002

\n

\u3053\u308c\u306f\u3001\u3053\u308c\u3089\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u624b\u6cd5\u3092\u4f7f\u7528\u3057\u305f\u5b9f\u9a13\u3067\u3059\u3002

\n", "

Sampler with Temperature

\n": "

\u6e29\u5ea6\u6a5f\u80fd\u4ed8\u304d\u30b5\u30f3\u30d7\u30e9\u30fc

\n", "

Sample from logits

\n": "

\u30ed\u30b8\u30c3\u30c8\u304b\u3089\u306e\u30b5\u30f3\u30d7\u30eb

\n", "

Create a categorical distribution with temperature adjusted logits

\n": "

\u6e29\u5ea6\u8abf\u6574\u6e08\u307f\u30ed\u30b8\u30c3\u30c8\u306b\u3088\u308b\u30ab\u30c6\u30b4\u30ea\u5206\u5e03\u306e\u4f5c\u6210

\n", "

Sample

\n": "

[\u30b5\u30f3\u30d7\u30eb]

\n", "\n": "\n", "A PyTorch implementation of sampling from language models with temperature.": "\u6e29\u5ea6\u3092\u542b\u3080\u8a00\u8a9e\u30e2\u30c7\u30eb\u304b\u3089\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0\u306e PyTorch \u5b9f\u88c5\u3002", "Sampling from Language Models with Temperature": "\u6e29\u5ea6\u3092\u7528\u3044\u305f\u8a00\u8a9e\u30e2\u30c7\u30eb\u304b\u3089\u306e\u30b5\u30f3\u30d7\u30ea\u30f3\u30b0" }