chore: import upstream snapshot with attribution

tools/Polygraphy/examples/cli/surgeon/01_isolating_subgraphs/README.md

@@ -0,0 +1,65 @@
+# Using Extract To Isolate A Subgraph
+
+
+## Introduction
+
+The `surgeon extract` subtool can be used to extract a subgraph from a model with a single command.
+
+In this example, we'll extract a subgraph from a model that computes `Y = x0 + (a * x1 + b)`:
+
+![./model.png](./model.png)
+
+Let's assume that we want to isolate the subgraph that computes `(a * x1 + b)`, and that we've
+used `polygraphy inspect model model.onnx --show layers` to determine the names of the input/output tensors
+of this subgraph, but that we don't know the shapes or data types of any of the tensors involved.
+
+When shapes and data types are unknown, you can use `auto` to indicate that Polygraphy should
+attempt to automatically determine these.
+For inputs, we must specify both shape and data type, whereas outputs only require the data
+type - hence `--inputs` requires 2 `auto`s and `--outputs` requires only 1.
+
+
+## Running The Example
+
+1. Extract the subgraph:
+
+    ```bash
+    polygraphy surgeon extract model.onnx \
+        --inputs x1:auto:auto \
+        --outputs add_out:auto \
+        -o subgraph.onnx
+    ```
+
+    If we knew the shapes and/or data types, we could instead write, for example:
+
+    ```bash
+    polygraphy surgeon extract model.onnx \
+        --inputs x1:[1,3,224,224]:float32 \
+        --outputs add_out:float32 \
+        -o subgraph.onnx
+    ```
+
+    The resulting subgraph will look like this:
+
+    ![./subgraph.png](./subgraph.png)
+
+2. **[Optional]** At this point, the model is ready for use. You can use `inspect model`
+    to confirm whether it looks correct:
+
+    ```bash
+    polygraphy inspect model subgraph.onnx --show layers
+    ```
+
+## A Note On `auto`
+
+When `auto` is specified as a shape or data type, Polygraphy relies on ONNX shape
+inference to determine the shapes and data types of intermediate tensors.
+
+In cases where ONNX shape inference cannot determine shapes, Polygraphy
+will run inference on the model using ONNX-Runtime with synthetic input data
+You can control the shape of this input data using the `--model-inputs` argument
+and the contents using the `Data Loader` options.
+
+This will cause the inputs of the resulting subgraph to have fixed shapes. You can change
+these back to dynamic by using the extract command again on the subgraph, and specifying
+the same inputs, but using shapes with dynamic dimensions, e.g. `--inputs identity_out_0:[-1,-1]:auto`

tools/Polygraphy/examples/cli/surgeon/02_folding_constants/README.md

@@ -0,0 +1,41 @@
+# Using Sanitize To Fold Constants
+
+
+## Introduction
+
+The `surgeon sanitize` subtool can be used to fold constants in graphs,
+remove unused nodes, and topologically sort nodes. In cases where shapes
+are statically known, it can also simplify subgraphs involving shape operations.
+
+In this example, we'll fold constants in a graph that computes `output = input + ((a + b) + d)`,
+where `a`, `b`, and `d` are constants:
+
+![./model.png](./model.png)
+
+
+## Running The Example
+
+1. Fold constants with:
+
+    ```bash
+    polygraphy surgeon sanitize model.onnx \
+        --fold-constants \
+        -o folded.onnx
+    ```
+
+    This collapses `a`, `b`, and `d` into a constant tensor, and the resulting graph
+    computes `output = input + e`:
+
+    ![./folded.png](./folded.png)
+
+    *TIP: Sometimes, models include operations like `Tile` or `ConstantOfShape`, that may*
+        *generate large constant tensors. Folding these can bloat the model size*
+        *to an undesirable degree. You can use the `--fold-size-threshold` to control*
+        *the maximum size, in bytes, for which to fold tensors. Any nodes that generate*
+        *tensors over this limit will not be folded, but instead computed at runtime.*
+
+2. **[Optional]** You can use `inspect model` to confirm whether it looks correct:
+
+    ```bash
+    polygraphy inspect model folded.onnx --show layers
+    ```

tools/Polygraphy/examples/cli/surgeon/03_modifying_input_shapes/README.md

@@ -0,0 +1,31 @@
+# Modifying Input Shapes
+
+
+## Introduction
+
+The `surgeon sanitize` subtool can be used to modify the input shapes of an ONNX model.
+This does not change the intermediate layers of the model, and as such, may cause issues if
+the model makes assumptions about the input shapes (for example, a `Reshape` node with a hard-coded
+new shape).
+
+Output shapes can be inferred and so these are not modified (nor do they need to be).
+
+*NOTE: Re-exporting the ONNX model with the desired shapes is strongly recommended.*
+    *The method shown here should only be used when doing so is not possible.*
+
+## Running The Example
+
+1. Change the input shape of the model to a shape with a dynamic batch dimension,
+    keeping other dimensions the same:
+
+    ```bash
+    polygraphy surgeon sanitize identity.onnx \
+        --override-input-shapes x:['batch',1,2,2] \
+        -o dynamic_identity.onnx
+    ```
+
+2. **[Optional]** You can use `inspect model` to confirm whether it looks correct:
+
+    ```bash
+    polygraphy inspect model dynamic_identity.onnx --show layers
+    ```

tools/Polygraphy/examples/cli/surgeon/03_modifying_input_shapes/identity.onnx

@@ -0,0 +1,15 @@
+backend-test:[
+
+
x
y"Identity
+test_identityZ
+
x
+

+

+

+
+b
+
y
+

+

+

+

tools/Polygraphy/examples/cli/surgeon/04_setting_upper_bounds/README.md

@@ -0,0 +1,56 @@
+# Using Sanitize To Set Upper Bounds For Unbounded Data-Dependent Shapes (DDS)
+
+
+## Introduction
+
+The `surgeon sanitize` subtool can be used to set upper bounds for unbounded Data-Dependent Shapes (DDS).
+When the shape of a tensor depends on the runtime value of another tensor, such shape is called DDS.
+Some DDS has a limited upper bound. For example, the output shape of a `NonZero` operator is a DDS, but its output shape will not exceed the shape of its input.
+While, some other DDS has no upper bound. For example, the output of a `Range` operator has an unbounded DDS when the `limit` input is a runtime tensor.
+Tensors with unbounded DDS are difficult for TensorRT to optimize inference performance and memory usage at builder stage. 
+In the worst case, they can cause TensorRT engine building failures. 
+
+In this example, we'll use polygraphy to set upper bounds for an unbounded DDS in a graph:
+
+![./model.png](./model.png)
+
+
+## Running The Example
+
+1. Run constant folding for the model first:
+
+    ```bash
+    polygraphy surgeon sanitize model.onnx -o folded.onnx --fold-constants
+    ```
+
+    Note that const folding and symbolic shape inference are required for listing unbounded DDS and setting upper bounds.
+
+2. Find tensors with unbounded DDS with:
+
+    ```bash
+    polygraphy inspect model folded.onnx --list-unbounded-dds
+    ```
+
+    Polygraphy will show all tensors with unbounded DDS.
+
+3. Set upper bounds for unbounded DDS with:
+
+    ```bash
+    polygraphy surgeon sanitize folded.onnx --set-unbounded-dds-upper-bound 1000 -o modified.onnx 
+    ```
+
+    Polygraphy will first search all tensors with unbounded DDS. 
+    Then it will insert min operators with the provided upper bound values to limit the DDS tensor size.
+    In this example, a min operator is inserted before the `Range` operator.
+    With the modified model, TensorRT will know that the output shape of the `Range` operator will not exceed 1000.
+    Thus more kernels can be selected for the following layers.
+
+    ![./modified.png](./modified.png)
+
+4. Check that there is no tensors with unbounded DDS now:
+
+    ```bash
+    polygraphy inspect model modified.onnx --list-unbounded-dds
+    ```
+
+    The modified.onnx should contain no unbounded DDS now.