chore: import upstream snapshot with attribution

tools/Polygraphy/examples/api/01_comparing_frameworks/README.md

@@ -0,0 +1,45 @@
+# Comparing Frameworks
+
+
+## Introduction
+
+One of the core features of Polygraphy is comparison of model outputs across multiple
+different backends. This makes it possible to check the accuracy of one backend with
+respect to another.
+
+In this example, we'll look at how you can use the Polygraphy API to run inference
+with synthetic input data using ONNX-Runtime and TensorRT, and then compare the results
+using two different comparison methods:
+
+1. A simple comparison using absolute tolerance
+2. A more comprehensive comparison using distance metrics (L2 distance, cosine similarity, and PSNR)
+
+
+## Running The Example
+
+1. Install prerequisites
+    * Ensure that TensorRT is installed
+    * Install other dependencies with `python3 -m pip install -r requirements.txt`
+
+2. Run the example
+    ```bash
+    python3 example.py
+    ```
+
+3. **[Optional]** Inspect the inference outputs from the example:
+
+    ```bash
+    polygraphy inspect data inference_results.json
+    ```
+
+## Comparison Methods
+
+The example demonstrates two approaches for comparing outputs:
+
+- **Simple Comparison**: Uses absolute tolerance to determine if outputs match within a specified threshold.
+- **Distance Metrics**: Performs a more comprehensive comparison using multiple metrics including:
+  - L2 distance (Euclidean distance)
+  - Cosine similarity (measures the angle between vectors)
+  - PSNR (Peak Signal-to-Noise Ratio, useful for comparing image-like data)
+
+These comparison methods help validate that frameworks produce equivalent results within acceptable margins.

tools/Polygraphy/examples/api/01_comparing_frameworks/example.py

@@ -0,0 +1,90 @@
+#!/usr/bin/env python3
+#
+# SPDX-FileCopyrightText: Copyright (c) 1993-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""
+This script runs an identity model with ONNX-Runtime and TensorRT,
+then compares outputs.
+"""
+from polygraphy.backend.onnxrt import OnnxrtRunner, SessionFromOnnx
+from polygraphy.backend.trt import EngineFromNetwork, NetworkFromOnnxPath, TrtRunner
+from polygraphy.comparator import Comparator, CompareFunc
+
+
+def main():
+    # The OnnxrtRunner requires an ONNX-RT session.
+    # We can use the SessionFromOnnx lazy loader to construct one easily:
+    build_onnxrt_session = SessionFromOnnx("identity.onnx")
+
+    # The TrtRunner requires a TensorRT engine.
+    # To create one from the ONNX model, we can chain a couple lazy loaders together:
+    build_engine = EngineFromNetwork(NetworkFromOnnxPath("identity.onnx"))
+
+    runners = [
+        TrtRunner(build_engine),
+        OnnxrtRunner(build_onnxrt_session),
+    ]
+
+    # `Comparator.run()` will run each runner separately using synthetic input data and
+    #   return a `RunResults` instance. See `polygraphy/comparator/struct.py` for details.
+    #
+    # TIP: To use custom input data, you can set the `data_loader` parameter in `Comparator.run()``
+    #   to a generator or iterable that yields `Dict[str, np.ndarray]`.
+    run_results = Comparator.run(runners)
+
+    # `Comparator.compare_accuracy()` checks that outputs match between runners.
+    #
+    # TIP: The `compare_func` parameter can be used to control how outputs are compared (see API reference for details).
+    #   The default comparison function is created by `CompareFunc.simple()`, but we can construct it
+    #   explicitly if we want to change the default parameters, such as tolerance.
+    assert bool(
+        Comparator.compare_accuracy(
+            run_results, compare_func=CompareFunc.simple(atol=1e-8)
+        )
+    )
+
+    # Use distance metrics comparison for more comprehensive evaluation
+    assert bool(
+        Comparator.compare_accuracy(
+            run_results,
+            compare_func=CompareFunc.distance_metrics(
+                l2_tolerance=1e-5,                    # Maximum allowed L2 norm (Euclidean distance)
+                cosine_similarity_threshold=0.99,     # Minimum cosine similarity (angular similarity)
+            )
+        )
+    )
+    print("All outputs matched using distance metrics (L2 norm, Cosine Similarity)")
+    
+    # Use quality metrics for signal quality evaluation
+    assert bool(
+        Comparator.compare_accuracy(
+            run_results,
+            compare_func=CompareFunc.quality_metrics(
+                psnr_tolerance=50.0,                  # Minimum Peak Signal-to-Noise Ratio in dB
+                snr_tolerance=25.0                    # Minimum Signal-to-Noise Ratio in dB
+            )
+        )
+    )
+    print("All outputs matched using quality metrics (PSNR, SNR)")
+
+    # We can use `RunResults.save()` method to save the inference results to a JSON file.
+    # This can be useful if you want to generate and compare results separately.
+    run_results.save("inference_results.json")
+
+
+if __name__ == "__main__":
+    main()

tools/Polygraphy/examples/api/01_comparing_frameworks/identity.onnx

@@ -0,0 +1,15 @@
+backend-test:[
+
+
x
y"Identity
+test_identityZ
+
x
+

+

+

+
+b
+
y
+

+

+

+

tools/Polygraphy/examples/api/01_comparing_frameworks/requirements.txt

@@ -0,0 +1,2 @@
+onnx
+onnxruntime