chore: import upstream snapshot with attribution

samples/python/onnx_packnet/README.md

@@ -0,0 +1,112 @@
+# TensorRT Inference of ONNX models with custom layers.
+
+**Table Of Contents**
+- [Description](#description)
+- [How does this sample work?](#how-does-this-sample-work)
+- [Prerequisites](#prerequisites)
+- [Running the sample](#running-the-sample)
+    * [Cloning the packnet repository](#cloning-the-packnet-repository)
+    * [Conversion to ONNX](#conversion-to-onnx)
+    * [Inference with TensorRT](#inference-with-tensorrt)
+	* [Sample `--help` options](#sample-help-options)
+- [Additional resources](#additional-resources)
+- [License](#license)
+- [Changelog](#changelog)
+- [Known issues](#known-issues)
+
+## Description
+
+This sample, samplePackNet, is a Python sample which uses TensorRT to perform inference with PackNet network. PackNet is a self-supervised monocular depth estimation network used in autonomous driving.
+
+
+## How does this sample work?
+
+This sample converts the Pytorch graph into ONNX and uses ONNX-parser included in TensorRT to parse the ONNX graph. The sample also demonstrates
+
+* Use of custom layers (plugins) in ONNX graph. These plugins would be automatically registered in TensorRT by using `REGISTER_TENSORRT_PLUGIN` API.
+* Use of ONNX-graphsurgeon (ONNX-GS) API to modify layers or subgraphs in the ONNX graph. For this network, we transform Group Normalization, upsample and pad layers to remove unnecessary
+  nodes for inference with TensorRT.
+
+
+## Prerequisites
+
+1. Upgrade pip version and install the sample dependencies.
+    ```bash
+    pip3 install --upgrade pip
+    pip3 install -r requirements.txt
+    ```
+
+On PowerPC systems, you will need to manually install PyTorch using IBM's [PowerAI](https://www.ibm.com/support/knowledgecenter/SS5SF7_1.6.0/navigation/pai_install.htm).
+
+
+## Running the sample
+
+### Preparing packnet
+
+Clone the [packnet](https://github.com/TRI-ML/packnet-sfm) repository and update `PYTHONPATH`.
+
+```
+git clone https://github.com/TRI-ML/packnet-sfm.git packnet-sfm
+pushd packnet-sfm && git checkout tags/v0.1.2 && popd
+export PYTHONPATH=$PWD/packnet-sfm # Note on Windows, the export command is: set PYTHONPATH=%cd%\packnet-sfm
+```
+
+### Conversion to ONNX
+Run the following command to convert the Packnet pytorch network to ONNX graph. This step also includes handling custom layers (Group Normalization) and using ONNX-GS to modify upsample and pad layers.
+
+```
+python3 convert_to_onnx.py --output model.onnx
+```
+
+### Inference with TensorRT
+
+Once the ONNX graph is generated, use `trtexec` tool (located in `bin` directory of TensorRT package) to perform inference on a random input image.
+
+```
+trtexec --onnx=model.onnx
+```
+
+Please refer to `trtexec` tool for more commandline options.
+
+### Sample --help options
+
+To see the full list of available options and their descriptions, use the `-h` or `--help` command line option. For example:
+```
+convert_to_onnx.py -h
+```
+
+# Additional resources
+
+The following resources provide a deeper understanding about PackNet network and importing a model into TensorRT using Python:
+
+**PackNet**
+- [3D Packing for Self-Supervised Monocular Depth Estimation](https://arxiv.org/pdf/1905.02693.pdf)
+- [TRI-ML Monocular Depth Estimation Repository](https://github.com/TRI-ML/packnet-sfm)
+
+**Parsers**
+- [ONNX Parser](https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/python_api/parsers/Onnx/pyOnnx.html)
+
+**Documentation**
+- [Introduction To NVIDIA’s TensorRT Samples](https://docs.nvidia.com/deeplearning/sdk/tensorrt-sample-support-guide/index.html#samples)
+- [Working With TensorRT Using The Python API](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#python_topics)
+- [Importing A Model Using A Parser In Python](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#import_model_python)
+- [NVIDIA’s TensorRT Documentation Library](https://docs.nvidia.com/deeplearning/sdk/tensorrt-archived/index.html)
+
+# License
+
+For terms and conditions for use, reproduction, and distribution, see the [TensorRT Software License Agreement](https://docs.nvidia.com/deeplearning/sdk/tensorrt-sla/index.html) documentation.
+
+# Changelog
+
+August 2025:
+  - Removed support for Python versions < 3.10.
+
+August 2023:
+  - Update ONNX version support to 1.14.0
+  - Removed support for Python versions < 3.8.
+August 2021: Update sample to work with latest torch version
+June 2020: Initial release of this sample
+
+# Known issues
+
+There are no known issues in this sample

samples/python/onnx_packnet/convert_to_onnx.py

@@ -0,0 +1,100 @@
+#!/usr/bin/env python3
+#
+# SPDX-FileCopyrightText: Copyright (c) 1993-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import onnx
+import torch
+import numpy as np
+import argparse
+import onnx_graphsurgeon as gs
+from post_processing import *
+from packnet_sfm.networks.depth.PackNet01 import PackNet01
+
+
+def post_process_packnet(model_file, opset=11):
+    """
+    Use ONNX graph surgeon to replace upsample and instance normalization nodes. Refer to post_processing.py for details.
+    Args:
+        model_file : Path to ONNX file
+    """
+    # Load the packnet graph
+    graph = gs.import_onnx(onnx.load(model_file))
+
+    if opset >= 11:
+        graph = process_pad_nodes(graph)
+
+    # Replace the subgraph of upsample with a single node with input and scale factor.
+    if torch.__version__ < "1.5.0":
+        graph = process_upsample_nodes(graph, opset)
+
+    # Convert the group normalization subgraph into a single plugin node.
+    graph = process_groupnorm_nodes(graph)
+
+    # Remove unused nodes, and topologically sort the graph.
+    graph.cleanup().toposort()
+
+    # Export the onnx graph from graphsurgeon
+    onnx.save_model(gs.export_onnx(graph), model_file)
+
+    print("Saving the ONNX model to {}".format(model_file))
+
+
+def build_packnet(model_file, args):
+    """
+    Construct the packnet network and export it to ONNX
+    """
+    input_pyt = torch.randn((1, 3, 192, 640), requires_grad=False)
+
+    # Build the model
+    model_pyt = PackNet01(version="1A")
+
+    # Convert the model into ONNX
+    torch.onnx.export(
+        model_pyt, input_pyt, model_file, verbose=args.verbose, opset_version=args.opset
+    )
+
+
+def main():
+    parser = argparse.ArgumentParser(
+        description="Exports PackNet01 to ONNX, and post-processes it to insert TensorRT plugins"
+    )
+    parser.add_argument(
+        "-o",
+        "--output",
+        help="Path to save the generated ONNX model",
+        default="model.onnx",
+    )
+    parser.add_argument(
+        "-op", "--opset", type=int, help="ONNX opset to use", default=11
+    )
+    parser.add_argument(
+        "-v",
+        "--verbose",
+        action="store_true",
+        help="Flag to enable verbose logging for torch.onnx.export",
+    )
+    args = parser.parse_args()
+
+    # Construct the packnet graph and generate the onnx graph
+    build_packnet(args.output, args)
+
+    # Perform post processing on Instance Normalization and upsampling nodes and create a new ONNX graph
+    post_process_packnet(args.output, args.opset)
+
+
+if __name__ == "__main__":
+    main()

samples/python/onnx_packnet/download.yml

@@ -0,0 +1,21 @@
+#
+# SPDX-FileCopyrightText: Copyright (c) 2020-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+sample: onnx_packnet
+files:
+  - path: samples/python/onnx_packnet/packnet-sfm-0.1.2.zip
+    url: https://github.com/TRI-ML/packnet-sfm/archive/v0.1.2.zip
+    checksum: 7a73db591d3955ccf407910cd928d9c0

samples/python/onnx_packnet/post_processing.py

@@ -0,0 +1,248 @@
+#!/usr/bin/env python3
+#
+# SPDX-FileCopyrightText: Copyright (c) 1993-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import onnx_graphsurgeon as gs
+import argparse
+import onnx
+import numpy as np
+import torch
+
+
+# Pad layer subgraph structure in ONNX (specific to opset 11):
+#               Constant
+#                  |
+#                Shape
+#                  |
+#         Mul   Gather
+#          \     /
+#            Sub
+#             |
+#       ConstantOfShape
+#             |
+#          Concat
+#             |
+#          Reshape
+#             |
+#           Slice
+#             |
+#          Transpose
+#             |
+#          Reshape
+#             |
+#    Input  Cast  Constant
+#       \     |    /
+#            Pad
+def process_pad_nodes(graph):
+    """
+    Fold the pad subgraph into a single layer with pad values as input
+      Input
+       |
+      Pad
+       |
+      Conv
+    """
+    pad_nodes = [node for node in graph.nodes if node.op == "Pad"]
+    for node in pad_nodes:
+        fold_pad_inputs(node, graph)
+
+    return graph
+
+
+def fold_pad_inputs(node, graph):
+    # Gather the amount of padding in each dimension from pytorch graph.
+    if torch.__version__ < "1.5.0":
+        pad_values_pyt = (
+            node.i(1).i(0).i(0).i(0).i(0).i(0).i(0).i(0).attrs["value"].values
+        )
+    elif torch.__version__ < "2.0.0":
+        pad_values_pyt = node.i(1).i(0).i(0).i(0).i(0).i(0).inputs[0].values
+    else:
+        pad_values_pyt = node.i(1).i(0).i(0).i(0).i(0).i(0).i(0).attrs["value"].values
+
+    # Assumption a 4d input tensor
+    onnx_pad_values = [0] * 4 * 2  # 4d tensor and 2 sides padding for each dimension
+    j = 3
+    for i in range(0, len(pad_values_pyt), 2):
+        onnx_pad_values[j] = pad_values_pyt[i]
+        onnx_pad_values[j + 4] = pad_values_pyt[i + 1]
+        j -= 1
+
+    # Change the existing pad tensor to the new onnx_pad values tensor
+    pads_folded_tensor = gs.Constant(
+        name=node.inputs[1].name, values=np.array(onnx_pad_values)
+    )
+    node.inputs[1] = pads_folded_tensor
+
+
+# Pytorch-exported Upsample structure in ONNX:
+#        Mul        Mul
+#         |          |
+#        Cast       Cast
+#         |          |
+#        Floor      Floor
+#         |          |
+#      Unsqueeze  Unsqueeze
+#         \         /
+#           Concat
+#             |
+#            Cast    Cast
+#              \      /
+#                Div
+#                 |
+#     Input     Concat
+#       \         /
+#         Upsample
+def process_upsample_nodes(graph, opset=11):
+    """
+    Replace the upsample structure with structure below
+      Conv   scale_factor
+       |      /
+      Upsample
+       |
+      ReLU
+    """
+    if opset >= 11:
+        upsample_layer_name = "Resize"
+    else:
+        upsample_layer_name = "Upsample"
+
+    upsample_nodes = [node for node in graph.nodes if node.op == upsample_layer_name]
+    for node in upsample_nodes:
+        fold_upsample_inputs(node, graph, opset)
+
+    return graph
+
+
+def fold_upsample_inputs(upsample, graph, opset=11):
+    """
+    Inplace transformation of the graph. The upsample subgraph is collapsed
+    to single upsample node with input and scale factor (constant tensor).
+    Args:
+        upsample: upsample node in the original graph.
+        graph: graph object.
+    """
+
+    if opset == 9:
+        # Gather the scale factor from mul op in the upsample input subgraph
+        scale_factor = (
+            upsample.i(1).i(1).i(0).i(0).i(0).i(0).i(0).i(0).i(1).attrs["value"].values
+        )
+
+        # Create the new scales tensor
+        scales = np.array([1.0, 1.0, scale_factor, scale_factor], dtype=np.float32)
+        scale_tensor = gs.Constant(name=upsample.inputs[-1].name, values=scales)
+
+        # Change the last input to the node to the new constant scales tensor.
+        upsample.inputs[-1] = scale_tensor
+    else:
+        # In opset 11, upsample layer is exported as Resize. We will transform this Resize layer into an Upsample layer
+        # and collapse the input
+        sizes_tensor_name = upsample.inputs[3].name
+
+        # Create the new scales tensor
+        scale_factor = (
+            upsample.i(3).i(1).i().i().i().i().i(0).i(1).attrs["value"].values
+        )
+        scales = np.array([1.0, 1.0, scale_factor, scale_factor], dtype=np.float32)
+        scale_tensor = gs.Constant(name=sizes_tensor_name, values=scales)
+
+        # Rename the Resize op to upsample and add the data and scales as inputs to the upsample layer.
+        input_tensor = upsample.inputs[0]
+        upsample.inputs = [input_tensor, scale_tensor]
+        upsample.op = "Upsample"
+
+
+# Pytorch-exported GroupNorm subgraph in ONNX:
+# Conv
+#   |
+# Reshape    Scale    Bias
+#     \       |       /
+# InstanceNormalization
+#         |
+#      Reshape    Unsqueeze
+#          \      /
+#        Mul (scale)   Unsqueeze
+#           \         /
+#           Add (bias)
+#              |
+#            ReLU
+def process_groupnorm_nodes(graph):
+    """
+    Gather the instance normalization nodes and the rest of the subgraph
+    and convert into a single group normalization node.
+    """
+    instancenorms = [node for node in graph.nodes if node.op == "InstanceNormalization"]
+    for node in instancenorms:
+        convert_to_groupnorm(node, graph)
+
+    return graph
+
+
+def retrieve_attrs(instancenorm):
+    """
+    Gather the required attributes for the GroupNorm plugin from the subgraph.
+    Args:
+        instancenorm: Instance Normalization node in the graph.
+    """
+    attrs = {}
+    # The 2nd dimension of the Reshape shape is the number of groups
+    attrs["num_groups"] = instancenorm.i().i(1).attrs["value"].values[1]
+    attrs["eps"] = instancenorm.attrs["epsilon"]
+
+    # 1 is the default plugin version the parser will search for, and therefore can be omitted,
+    # but we include it here for illustrative purposes.
+    attrs["plugin_version"] = "1"
+
+    # "" is the default plugin namespace the parser will use, included here for illustrative purposes
+    attrs["plugin_namespace"] = ""
+
+    return attrs
+
+
+def convert_to_groupnorm(instancenorm, graph):
+    """
+    Convert the Pytorch-exported GroupNorm subgraph to the subgraph below
+    Conv
+      |
+    GroupNorm
+      |
+    ReLU
+    Attributes:
+        instancenorm: Instance Normalization node in the graph.
+        graph: Input graph object
+    """
+    # Retrieve the instancenorm attributes and create the replacement node
+    attrs = retrieve_attrs(instancenorm)
+    groupnorm = gs.Node(op="GroupNormalizationPlugin", attrs=attrs)
+    graph.nodes.append(groupnorm)
+
+    # The plugin needs to receive an input from the Conv node, and output to the ReLU node
+    conv_output_tensor = instancenorm.i().inputs[0]  # Output of Conv
+    relu_input_tensor = instancenorm.o().o().o().outputs[0]  # Output of Add
+
+    # Reconnect inputs/outputs to the groupnorm plugin
+    conv_output_tensor.outputs[0] = groupnorm
+    relu_input_tensor.inputs[0] = groupnorm
+
+    # Add scale and bias constant tensors to group norm plugin
+    if torch.__version__ < "1.5.0":
+        groupnorm.inputs.append(instancenorm.o().o().i(1).inputs[0])
+        groupnorm.inputs.append(instancenorm.o().o().o().i(1).inputs[0])
+    else:
+        groupnorm.inputs.append(instancenorm.o().o().inputs[1])
+        groupnorm.inputs.append(instancenorm.o().o().o().inputs[1])

samples/python/onnx_packnet/requirements.txt

@@ -0,0 +1,9 @@
+onnx==1.18.0
+--extra-index-url https://pypi.ngc.nvidia.com
+onnx-graphsurgeon>=0.3.20
+torch
+torchvision
+pyyaml==6.0.3
+requests==2.32.4
+tqdm==4.66.4
+numpy==1.26.4