chore: import upstream snapshot with attribution

tools/Polygraphy/examples/cli/convert/01_int8_calibration_in_tensorrt/README.md

@@ -0,0 +1,52 @@
+# Int8 Calibration In TensorRT
+
+
+## Introduction
+
+In [API example 04](../../../api/04_int8_calibration_in_tensorrt/), we saw how we can leverage
+Polygraphy's included calibrator to easily run int8 calibration with TensorRT.
+
+But what if we wanted to do the same thing on the command-line?
+
+To do this, we need a way to supply custom input data to our command-line tools.
+Polygraphy provides multiple ways to do so, which are detailed [here](../../../../how-to/use_custom_input_data.md).
+
+In this example, we'll use a data loader script by defining a `load_data` function in a Python
+script called `data_loader.py` and then use `polygraphy convert` to build the TensorRT engine.
+
+*TIP: We can use a similar approach with `polygraphy run` to build and run the engine.*
+
+## Running The Example
+
+1. Convert the model, using the custom data loader script to supply calibration data,
+    saving a calibration cache for future use:
+
+    ```bash
+    polygraphy convert identity.onnx --int8 \
+        --data-loader-script ./data_loader.py \
+        --calibration-cache identity_calib.cache \
+        -o identity.engine
+    ```
+
+2. **[Optional]** Rebuild the engine using the cache to skip calibration:
+
+    ```bash
+    polygraphy convert identity.onnx --int8 \
+        --calibration-cache identity_calib.cache \
+        -o identity.engine
+    ```
+
+    Since the calibration cache is already populated, calibration will be skipped.
+    Hence, we do *not* need to supply input data.
+
+
+3. **[Optional]** Use the data loader directly from the API example.
+
+    The method outlined here is so flexible that we can even use the data loader we defined in the API example!
+    We just need to specify the function name since the example does not call it `load_data`:
+
+    ```bash
+    polygraphy convert identity.onnx --int8 \
+        --data-loader-script ../../../api/04_int8_calibration_in_tensorrt/example.py:calib_data \
+        -o identity.engine
+    ```

tools/Polygraphy/examples/cli/convert/01_int8_calibration_in_tensorrt/data_loader.py

@@ -0,0 +1,33 @@
+#!/usr/bin/env python3
+#
+# SPDX-FileCopyrightText: Copyright (c) 1993-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""
+Defines a `load_data` function that returns a generator yielding
+feed_dicts so that this script can be used as the argument for
+the --data-loader-script command-line parameter.
+"""
+import numpy as np
+
+INPUT_SHAPE = (1, 1, 2, 2)
+
+
+def load_data():
+    for _ in range(5):
+        yield {
+            "x": np.ones(shape=INPUT_SHAPE, dtype=np.float32)
+        }  # Still totally real data

tools/Polygraphy/examples/cli/convert/01_int8_calibration_in_tensorrt/identity.onnx

@@ -0,0 +1,15 @@
+backend-test:[
+
+
x
y"Identity
+test_identityZ
+
x
+

+

+

+
+b
+
y
+

+

+

+

tools/Polygraphy/examples/cli/convert/02_deterministic_engine_builds_in_tensorrt/README.md

@@ -0,0 +1,41 @@
+# Deterministic Engine Building In TensorRT
+
+**NOTE: This example requires TensorRT 8.7 or newer.**
+
+## Introduction
+
+During engine building, TensorRT runs and times several kernels in order to select
+the most optimal ones. Since kernel timings may vary slightly from run to run, this
+process is inherently non-deterministic.
+
+In many cases, deterministic engine builds may be desirable. One way of achieving this
+is to use a timing cache to ensure the same kernels are picked each time.
+
+## Running The Example
+
+1. Build an engine and save a timing cache:
+
+    ```bash
+    polygraphy convert identity.onnx \
+        --save-timing-cache timing.cache \
+        -o 0.engine
+    ```
+
+2. Use the timing cache for another engine build:
+
+    ```bash
+    polygraphy convert identity.onnx \
+        --load-timing-cache timing.cache --error-on-timing-cache-miss \
+        -o 1.engine
+    ```
+
+    We specify `--error-on-timing-cache-miss` so that we can be sure that the new engine
+    used the entries from the timing cache for each layer.
+
+3. Verify that the engines are exactly the same:
+
+    <!-- Polygraphy Test: Ignore Start -->
+    ```bash
+    diff <(polygraphy inspect model 0.engine --show layers attrs) <(polygraphy inspect model 1.engine --show layers attrs)
+    ```
+    <!-- Polygraphy Test: Ignore End -->

tools/Polygraphy/examples/cli/convert/02_deterministic_engine_builds_in_tensorrt/identity.onnx

@@ -0,0 +1,15 @@
+backend-test:[
+
+
x
y"Identity
+test_identityZ
+
x
+

+

+

+
+b
+
y
+

+

+

+

tools/Polygraphy/examples/cli/convert/03_dynamic_shapes_in_tensorrt/README.md

@@ -0,0 +1,42 @@
+# Working With Models With Dynamic Shapes In TensorRT
+
+## Introduction
+
+In order to use dynamic input shapes with TensorRT, we have to specify a range
+(or multiple ranges) of possible shapes when we build the engine.
+For details on how this works, refer to
+[API example 07](../../../api/07_tensorrt_and_dynamic_shapes/).
+
+When using the CLI, we can specify the per-input minimum, optimum, and maximum
+shapes one or more times. If shapes are specified more than
+once per input, multiple optimization profiles are created.
+
+## Running The Example
+
+1. Build an engine with 3 separate profiles:
+
+    ```bash
+    polygraphy convert dynamic_identity.onnx -o dynamic_identity.engine \
+        --trt-min-shapes X:[1,3,28,28] --trt-opt-shapes X:[1,3,28,28] --trt-max-shapes X:[1,3,28,28] \
+        --trt-min-shapes X:[1,3,28,28] --trt-opt-shapes X:[4,3,28,28] --trt-max-shapes X:[32,3,28,28] \
+        --trt-min-shapes X:[128,3,28,28] --trt-opt-shapes X:[128,3,28,28] --trt-max-shapes X:[128,3,28,28]
+    ```
+
+    For models with multiple inputs, simply provide multiple arguments to each `--trt-*-shapes` parameter.
+    For example: `--trt-min-shapes input0:[10,10] input1:[10,10] input2:[10,10] ...`
+
+    *TIP: If we want to use only a single profile where min == opt == max, we can leverage the runtime input*
+        *shapes option: `--input-shapes` as a conveneint shorthand instead of setting min/opt/max separately.*
+
+
+2. **[Optional]** Inspect the resulting engine:
+
+    ```bash
+    polygraphy inspect model dynamic_identity.engine
+    ```
+
+
+## Further Reading
+
+For more information on using dynamic shapes with TensorRT, see the
+[developer guide](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#work_dynamic_shapes)

tools/Polygraphy/examples/cli/convert/03_dynamic_shapes_in_tensorrt/dynamic_identity.onnx

@@ -0,0 +1,12 @@
+:[
+
+
X
Y"Identityonnx_dynamic_identityZ%
+
X 
+

+
+batch_size
+
+
+b	
+
Y
+
B

tools/Polygraphy/examples/cli/convert/04_converting_models_to_fp16/README.md

@@ -0,0 +1,49 @@
+# Converting ONNX Models To FP16
+
+## Introduction
+
+When debugging accuracy issues with using TensorRT reduced precision
+optimizations (`--fp16` and `--tf32` flags) on an ONNX model trained in FP32,
+it can be helpful to convert the model to FP16 and run it under ONNX-Runtime
+to check if there are might be problems inherent to running the model
+with reduced precision.
+
+## Running The Example
+
+1. Convert the model to FP16:
+
+   ```bash
+   polygraphy convert --fp-to-fp16 -o identity_fp16.onnx identity.onnx
+   ```
+
+2. **[Optional]** Inspect the resulting model:
+
+   ```bash
+   polygraphy inspect model identity_fp16.onnx
+   ```
+
+3. **[Optional]** Run the FP32 and FP16 models under ONNX-Runtime and then compare the results:
+
+   ```bash
+   polygraphy run --onnxrt identity.onnx \
+      --save-inputs inputs.json --save-outputs outputs_fp32.json
+   ```
+
+   ```bash
+   polygraphy run --onnxrt identity_fp16.onnx \
+      --load-inputs inputs.json --load-outputs outputs_fp32.json \
+      --atol 0.001 --rtol 0.001
+   ```
+
+4. **[Optional]** Check if any intermediate outputs of the FP16 model
+   contain NaN or infinity (see [Checking for Intermediate NaN or Infinities](../../../../examples/cli/run/07_checking_nan_inf)):
+
+   ```bash
+   polygraphy run --onnxrt identity_fp16.onnx --onnx-outputs mark all --validate
+   ```
+
+## See Also
+
+* [Comparing Across Runs](../../../../examples/cli/run/02_comparing_across_runs)
+* [Checking for Intermediate NaN or Infinities](../../../../examples/cli/run/07_checking_nan_inf)
+* [Debugging TensorRT Accuracy Issues](../../../../how-to/debug_accuracy.md)

tools/Polygraphy/examples/cli/convert/04_converting_models_to_fp16/identity.onnx

@@ -0,0 +1,15 @@
+backend-test:[
+
+
x
y"Identity
+test_identityZ
+
x
+

+

+

+
+b
+
y
+

+

+

+