This is a U-Net based model to predict noise _^_0_^_.
\nU-Net is a gets it's name from the U shape in the model diagram. It processes a given image by progressively lowering (halving) the feature map resolution and then increasing the resolution. There are pass-through connection at each resolution.
\n_^_1_^_
\nThis implementation contains a bunch of modifications to original U-Net (residual blocks, multi-head attention) and also adds time-step embeddings _^_2_^_.
\n": "\u3053\u308c\u306f U-Net _^_0_^_ \u30d9\u30fc\u30b9\u306e\u30ce\u30a4\u30ba\u4e88\u6e2c\u30e2\u30c7\u30eb\u3067\u3059\u3002
\nU-Net\u306f\u3001\u30e2\u30c7\u30eb\u56f3\u306eU\u5b57\u5f62\u306b\u3061\u306a\u3093\u3067\u540d\u4ed8\u3051\u3089\u308c\u307e\u3057\u305f\u3002\u7279\u5fb4\u30de\u30c3\u30d7\u306e\u89e3\u50cf\u5ea6\u3092\u6bb5\u968e\u7684\u306b\u4f4e\u304f (\u534a\u5206\u306b)\u3001\u6b21\u306b\u89e3\u50cf\u5ea6\u3092\u4e0a\u3052\u308b\u3053\u3068\u306b\u3088\u3063\u3066\u3001\u7279\u5b9a\u306e\u753b\u50cf\u3092\u51e6\u7406\u3057\u307e\u3059\u3002\u5404\u89e3\u50cf\u5ea6\u306b\u306f\u30d1\u30b9\u30b9\u30eb\u30fc\u63a5\u7d9a\u304c\u3042\u308a\u307e\u3059
\u3002\n_^_1_^_
\n\u3053\u306e\u5b9f\u88c5\u306b\u306f\u3001\u30aa\u30ea\u30b8\u30ca\u30eb\u306e U-Net \u306b\u591a\u6570\u306e\u5909\u66f4\uff08\u6b8b\u7559\u30d6\u30ed\u30c3\u30af\u3001\u30de\u30eb\u30c1\u30d8\u30c3\u30c9\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\uff09\u304c\u542b\u307e\u308c\u3066\u304a\u308a\u3001\u30bf\u30a4\u30e0\u30b9\u30c6\u30c3\u30d7\u306e\u57cb\u3081\u8fbc\u307f\u3082\u8ffd\u52a0\u3055\u308c\u3066\u3044\u307e\u3059\u3002_^_2_^_
\n", "This is similar to transformer multi-head attention.
\n": "This combines _^_0_^_ and _^_1_^_. These are used in the first half of U-Net at each resolution.
\n": "_^_0_^_\u3053\u308c\u306f\u3068\u3092\u7d44\u307f\u5408\u308f\u305b\u305f\u3082\u306e\u3067\u3059_^_1_^_\u3002\u3053\u308c\u3089\u306fU-Net\u306e\u524d\u534a\u3067\u305d\u308c\u305e\u308c\u306e\u89e3\u50cf\u5ea6\u3067\u4f7f\u308f\u308c\u3066\u3044\u307e\u3059
\u3002\n", "It combines a _^_0_^_, _^_1_^_, followed by another _^_2_^_. This block is applied at the lowest resolution of the U-Net.
\n": "a \u3068_^_0_^__^_1_^_\u3001_^_2_^_\u306e\u5f8c\u306b\u7d9a\u304f\u5225\u306e\u3082\u306e\u3092\u7d44\u307f\u5408\u308f\u305b\u307e\u3059\u3002\u3053\u306e\u30d6\u30ed\u30c3\u30af\u306f U-Net \u306e\u6700\u4f4e\u89e3\u50cf\u5ea6\u3067\u9069\u7528\u3055\u308c\u307e\u3059
\u3002\n", "A residual block has two convolution layers with group normalization. Each resolution is processed with two residual blocks.
\n": "\u6b8b\u5dee\u30d6\u30ed\u30c3\u30af\u306b\u306f\u3001\u30b0\u30eb\u30fc\u30d7\u6b63\u898f\u5316\u3055\u308c\u305f 2 \u3064\u306e\u7573\u307f\u8fbc\u307f\u5c64\u304c\u3042\u308a\u307e\u3059\u3002\u5404\u89e3\u50cf\u5ea6\u306f 2 \u3064\u306e\u6b8b\u5dee\u30d6\u30ed\u30c3\u30af\u3067\u51e6\u7406\u3055\u308c\u307e\u3059
\u3002\n", "_^_0_^_
\n": "_^_0_^_
\n", "This combines _^_0_^_ and _^_1_^_. These are used in the second half of U-Net at each resolution.
\n": "_^_0_^_\u3053\u308c\u306f\u3068\u3092\u7d44\u307f\u5408\u308f\u305b\u305f\u3082\u306e\u3067\u3059_^_1_^_\u3002\u3053\u308c\u3089\u306fU-Net\u306e\u5f8c\u534a\u3067\u305d\u308c\u305e\u308c\u306e\u89e3\u50cf\u5ea6\u3067\u4f7f\u308f\u308c\u3066\u3044\u307e\u3059
\u3002\n", "\n": "\n", "
_^_0_^_ at the same resolution
\n": "_^_0_^_\u540c\u3058\u89e3\u50cf\u5ea6\u3067
\n", "_^_0_^_ is not used, but it's kept in the arguments because for the attention layer function signature to match with _^_1_^_.
\n": "_^_0_^_\u306f\u4f7f\u308f\u308c\u3066\u3044\u307e\u305b\u3093\u304c\u3001_^_1_^_\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30ec\u30a4\u30e4\u30fc\u306e\u95a2\u6570\u30b7\u30b0\u30cd\u30c1\u30e3\u3068\u306e\u30de\u30c3\u30c1\u30f3\u30b0\u306e\u305f\u3081\u5f15\u6570\u306b\u306f\u6b8b\u3055\u308c\u3066\u3044\u307e\u3059\u3002
\n", "_^_0_^_ will store outputs at each resolution for skip connection
\n": "_^_0_^_\u63a5\u7d9a\u3092\u30b9\u30ad\u30c3\u30d7\u3067\u304d\u308b\u3088\u3046\u306b\u3001\u51fa\u529b\u3092\u5404\u89e3\u50cf\u5ea6\u3067\u4fdd\u5b58\u3057\u307e\u3059
\n", "Activation
\n": "\u30a2\u30af\u30c6\u30a3\u30d9\u30fc\u30b7\u30e7\u30f3
\n", "Add _^_0_^_
\n": "[\u8ffd\u52a0] _^_0_^_
\n", "Add skip connection
\n": "\u30b9\u30ad\u30c3\u30d7\u63a5\u7d9a\u3092\u8ffd\u52a0
\n", "Add the shortcut connection and return
\n": "\u30b7\u30e7\u30fc\u30c8\u30ab\u30c3\u30c8\u63a5\u7d9a\u3092\u8ffd\u52a0\u3057\u3066\u623b\u308b
\n", "Add time embeddings
\n": "\u6642\u9593\u57cb\u3081\u8fbc\u307f\u3092\u8ffd\u52a0
\n", "Calculate scaled dot-product _^_0_^_
\n": "\u30b9\u30b1\u30fc\u30ea\u30f3\u30b0\u3055\u308c\u305f\u30c9\u30c3\u30c8\u30d7\u30ed\u30c0\u30af\u30c8\u306e\u8a08\u7b97 _^_0_^_
\n", "Change _^_0_^_ to shape _^_1_^_
\n": "_^_0_^_\u5f62\u72b6\u306b\u5909\u66f4 _^_1_^_
\n", "Change to shape _^_0_^_
\n": "\u5f62\u72b6\u306b\u5909\u66f4 _^_0_^_
\n", "Combine the set of modules
\n": "\u30e2\u30b8\u30e5\u30fc\u30eb\u30bb\u30c3\u30c8\u3092\u7d44\u307f\u5408\u308f\u305b\u308b
\n", "Create sinusoidal position embeddings same as those from the transformer
\n_^_0_^_where _^_1_^_ is _^_2_^_
\n": "\n_^_0_^__^_1_^_\u3069\u3053 _^_2_^_
\n", "Default _^_0_^_
\n": "\u30c7\u30d5\u30a9\u30eb\u30c8 _^_0_^_
\n", "Down sample at all resolutions except the last
\n": "\u6700\u5f8c\u306e\u89e3\u50cf\u5ea6\u3092\u9664\u304f\u3059\u3079\u3066\u306e\u89e3\u50cf\u5ea6\u306e\u30c0\u30a6\u30f3\u30b5\u30f3\u30d7\u30eb
\n", "Final block to reduce the number of channels
\n": "\u30c1\u30e3\u30f3\u30cd\u30eb\u6570\u3092\u6e1b\u3089\u3059\u305f\u3081\u306e\u6700\u5f8c\u306e\u30d6\u30ed\u30c3\u30af
\n", "Final normalization and convolution
\n": "\u6700\u7d42\u7684\u306a\u6b63\u898f\u5316\u3068\u7573\u307f\u8fbc\u307f
\n", "Final normalization and convolution layer
\n": "\u6700\u7d42\u6b63\u898f\u5316\u3068\u7573\u307f\u8fbc\u307f\u5c64
\n", "First convolution layer
\n": "\u6700\u521d\u306e\u7573\u307f\u8fbc\u307f\u5c64
\n", "First half of U-Net
\n": "\u30e6\u30fc\u30cd\u30c3\u30c8\u524d\u534a
\n", "First linear layer
\n": "\u7b2c 1 \u7dda\u5f62\u30ec\u30a4\u30e4\u30fc
\n", "For each resolution
\n": "\u5404\u89e3\u50cf\u5ea6\u306b\u3064\u3044\u3066
\n", "Get image projection
\n": "\u30a4\u30e1\u30fc\u30b8\u30d7\u30ed\u30b8\u30a7\u30af\u30b7\u30e7\u30f3\u3092\u53d6\u5f97
\n", "Get query, key, and values (concatenated) and shape it to _^_0_^_
\n": "\u30af\u30a8\u30ea\u3001\u30ad\u30fc\u3001\u5024 (\u9023\u7d50) \u3092\u53d6\u5f97\u3057\u3001\u4ee5\u4e0b\u306e\u3088\u3046\u306b\u5f62\u3092\u6574\u3048\u307e\u3059 _^_0_^_
\n", "Get shape
\n": "\u30b7\u30a7\u30a4\u30d7\u3092\u53d6\u5f97
\n", "Get the skip connection from first half of U-Net and concatenate
\n": "U-Net\u306e\u524d\u534a\u304b\u3089\u30b9\u30ad\u30c3\u30d7\u63a5\u7d9a\u3092\u53d6\u5f97\u3057\u3066\u9023\u7d50\u3059\u308b
\n", "Get time-step embeddings
\n": "\u30bf\u30a4\u30e0\u30b9\u30c6\u30c3\u30d7\u306e\u57cb\u3081\u8fbc\u307f\u3092\u5165\u624b
\n", "Group normalization and the first convolution layer
\n": "\u30b0\u30eb\u30fc\u30d7\u6b63\u898f\u5316\u3068\u6700\u521d\u306e\u7573\u307f\u8fbc\u307f\u5c64
\n", "Group normalization and the second convolution layer
\n": "\u30b0\u30eb\u30fc\u30d7\u6b63\u898f\u5316\u3068 2 \u756a\u76ee\u306e\u7573\u307f\u8fbc\u307f\u5c64
\n", "If the number of input channels is not equal to the number of output channels we have to project the shortcut connection
\n": "\u5165\u529b\u30c1\u30e3\u30f3\u30cd\u30eb\u306e\u6570\u304c\u51fa\u529b\u30c1\u30e3\u30f3\u30cd\u30eb\u306e\u6570\u3068\u7b49\u3057\u304f\u306a\u3044\u5834\u5408\u306f\u3001\u30b7\u30e7\u30fc\u30c8\u30ab\u30c3\u30c8\u63a5\u7d9a\u3092\u6295\u5f71\u3059\u308b\u5fc5\u8981\u304c\u3042\u308a\u307e\u3059\u3002
\n", "Linear layer for final transformation
\n": "\u6700\u7d42\u5909\u63db\u7528\u306e\u7dda\u5f62\u30ec\u30a4\u30e4\u30fc
\n", "Linear layer for time embeddings
\n": "\u6642\u9593\u57cb\u3081\u8fbc\u307f\u7528\u306e\u7dda\u5f62\u30ec\u30a4\u30e4\u30fc
\n", "Middle (bottom)
\n": "\u4e2d\u592e (\u4e0b\u90e8)
\n", "Middle block
\n": "\u30df\u30c9\u30eb\u30d6\u30ed\u30c3\u30af
\n", "Multiply by values
\n": "\u5024\u306b\u3088\u308b\u4e57\u7b97
\n", "Normalization layer
\n": "\u6b63\u898f\u5316\u30ec\u30a4\u30e4\u30fc
\n", "Number of channels
\n": "\u30c1\u30e3\u30f3\u30cd\u30eb\u6570
\n", "Number of output channels at this resolution
\n": "\u3053\u306e\u89e3\u50cf\u5ea6\u3067\u306e\u51fa\u529b\u30c1\u30e3\u30f3\u30cd\u30eb\u6570
\n", "Number of resolutions
\n": "\u89e3\u50cf\u5ea6\u306e\u6570
\n", "Project image into feature map
\n": "\u753b\u50cf\u3092\u30d5\u30a3\u30fc\u30c1\u30e3\u30de\u30c3\u30d7\u306b\u6295\u5f71
\n", "Projections for query, key and values
\n": "\u30af\u30a8\u30ea\u3001\u30ad\u30fc\u3001\u5024\u306e\u6295\u5f71
\n", "Reshape to _^_0_^_
\n": "\u5f62\u72b6\u3092\u6b21\u306e\u5f62\u5f0f\u306b\u5909\u66f4 _^_0_^_
\n", "Scale for dot-product attention
\n": "\u30c9\u30c3\u30c8\u30d7\u30ed\u30c0\u30af\u30c8\u30fb\u30a2\u30c6\u30f3\u30b7\u30e7\u30f3\u30fb\u30b9\u30b1\u30fc\u30eb
\n", "Second convolution layer
\n": "2 \u756a\u76ee\u306e\u7573\u307f\u8fbc\u307f\u5c64
\n", "Second half of U-Net
\n": "\u30e6\u30fc\u30cd\u30c3\u30c8\u5f8c\u534a
\n", "Second linear layer
\n": "2 \u756a\u76ee\u306e\u7dda\u5f62\u30ec\u30a4\u30e4\u30fc
\n", "Softmax along the sequence dimension _^_0_^_
\n": "\u30b7\u30fc\u30b1\u30f3\u30b9\u6b21\u5143\u306b\u6cbf\u3063\u305f\u30bd\u30d5\u30c8\u30de\u30c3\u30af\u30b9 _^_0_^_
\n", "Split query, key, and values. Each of them will have shape _^_0_^_
\n": "\u30af\u30a8\u30ea\u3001\u30ad\u30fc\u3001\u5024\u3092\u5206\u5272\u3057\u307e\u3059\u3002\u305d\u308c\u305e\u308c\u306b\u5f62\u304c\u3042\u308a\u307e\u3059 _^_0_^_
\n", "The input has _^_0_^_ because we concatenate the output of the same resolution from the first half of the U-Net
\n": "\u5165\u529b\u306f\u3001_^_0_^_ U-Net\u306e\u524d\u534a\u304b\u3089\u540c\u3058\u89e3\u50cf\u5ea6\u306e\u51fa\u529b\u3092\u9023\u7d50\u3057\u3066\u3044\u308b\u305f\u3081\u3067\u3059\u3002
\n", "Time embedding layer. Time embedding has _^_0_^_ channels
\n": "\u6642\u9593\u57cb\u3081\u8fbc\u307f\u30ec\u30a4\u30e4\u30fc\u3002\u6642\u9593\u57cb\u3081\u8fbc\u307f\u306b\u306f\u30c1\u30e3\u30f3\u30cd\u30eb\u304c\u3042\u308a\u307e\u3059 _^_0_^_
\n", "Transform to _^_0_^_
\n": "\u306b\u5909\u63db _^_0_^_
\n", "Transform with the MLP
\n": "MLP \u306b\u3088\u308b\u30c8\u30e9\u30f3\u30b9\u30d5\u30a9\u30fc\u30e1\u30fc\u30b7\u30e7\u30f3
\n", "Up sample at all resolutions except last
\n": "\u524d\u56de\u3092\u9664\u304f\u3059\u3079\u3066\u306e\u89e3\u50cf\u5ea6\u3067\u30b5\u30f3\u30d7\u30eb\u3092\u30a2\u30c3\u30d7
\n", "