chore: import upstream snapshot with attribution

tools/Polygraphy/examples/api/06_immediate_eval_api/README.md

@@ -0,0 +1,92 @@
+# Immediately Evaluated Functional API
+
+## Introduction
+
+<!-- Polygraphy Test: Ignore Start -->
+Most of the time, the lazy loaders included with Polygraphy have several advantages:
+
+- They allow us to defer the work until we actually need to do it, which can potentially save
+    time.
+- Since constructed loaders are extremely light-weight, runners using lazily evaluated loaders can be
+    easily copied into other processes or threads, where they can then be launched.
+    If runners instead referenced entire models/inference sessions, it would be non-trivial to copy them in this way.
+- They allow us to define a sequence of operations in advance by chaining loaders together, which
+    provides an easy way to build reusable functions.
+    For example, we could create a loader that imports a model from ONNX and generates a serialized TensorRT Engine:
+
+    ```python
+    build_engine = EngineBytesFromNetwork(NetworkFromOnnxPath("/path/to/model.onnx"))
+    ```
+
+- They allow for special semantics where if a callable is provided to a loader, it takes ownership
+    of the return value, whereas otherwise it does not. These special semantics are useful for
+    sharing objects between multiple loaders.
+
+However, this can sometimes lead to code that is less readable, or even downright confusing.
+For example, consider the following:
+```python
+# Each line in this example looks almost the same, but has significantly
+# different behavior. Some of these lines even cause memory leaks!
+EngineBytesFromNetwork(NetworkFromOnnxPath("/path/to/model.onnx")) # This is a loader instance, not an engine!
+EngineBytesFromNetwork(NetworkFromOnnxPath("/path/to/model.onnx"))() # This is an engine.
+EngineBytesFromNetwork(NetworkFromOnnxPath("/path/to/model.onnx")()) # And it's a loader instance again...
+EngineBytesFromNetwork(NetworkFromOnnxPath("/path/to/model.onnx")())() # Back to an engine!
+EngineBytesFromNetwork(NetworkFromOnnxPath("/path/to/model.onnx"))()() # This throws - can you see why?
+```
+
+For that reason, Polygraphy provides immediately-evaluated functional
+equivalents of each loader. Each functional variant uses the same name as the loader, but
+`snake_case` instead of `PascalCase`. Using the functional variants, loader code like:
+
+```python
+parse_network = NetworkFromOnnxPath("/path/to/model.onnx")
+create_config = CreateConfig(fp16=True, tf32=True)
+build_engine = EngineFromNetwork(parse_network, create_config)
+engine = build_engine()
+```
+
+becomes:
+
+```python
+builder, network, parser = network_from_onnx_path("/path/to/model.onnx")
+config = create_config(builder, network, fp16=True, tf32=True)
+engine = engine_from_network((builder, network, parser), config)
+```
+<!-- Polygraphy Test: Ignore End -->
+
+
+In this example, we'll look at how you can leverage the functional API to convert an ONNX
+model to a TensorRT network, modify the network, build a TensorRT engine with FP16 precision
+enabled, and run inference.
+We'll also save the engine to a file to see how you can load it again and run inference.
+
+
+## Running The Example
+
+1. Install prerequisites
+    * Ensure that TensorRT is installed
+    * Install other dependencies with `python3 -m pip install -r requirements.txt`
+
+2. **[Optional]** Inspect the model before running the example:
+
+    ```bash
+    polygraphy inspect model identity.onnx
+    ```
+
+3. Run the script that builds and runs the engine:
+
+    ```bash
+    python3 build_and_run.py
+    ```
+
+4. **[Optional]** Inspect the TensorRT engine built by the example:
+
+    ```bash
+    polygraphy inspect model identity.engine
+    ```
+
+5. Run the script that loads the previously built engine, then runs it:
+
+    ```bash
+    python3 load_and_run.py
+    ```

tools/Polygraphy/examples/api/06_immediate_eval_api/build_and_run.py

@@ -0,0 +1,74 @@
+#!/usr/bin/env python3
+#
+# SPDX-FileCopyrightText: Copyright (c) 1993-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""
+This script uses Polygraphy's immediately evaluated functional APIs
+to load an ONNX model, convert it into a TensorRT network, add an identity
+layer to the end of it, build an engine with FP16 mode enabled,
+save the engine, and finally run inference.
+"""
+import numpy as np
+from polygraphy.backend.trt import (
+    TrtRunner,
+    create_config,
+    engine_from_network,
+    network_from_onnx_path,
+    save_engine,
+)
+
+
+def main():
+    # In Polygraphy, loaders and runners take ownership of objects if they are provided
+    # via the return values of callables. For example, we don't need to worry about object
+    # lifetimes when we use lazy loaders.
+    #
+    # Since we are immediately evaluating, we take ownership of objects, and are responsible for freeing them.
+    builder, network, parser = network_from_onnx_path("identity.onnx")
+
+    # Extend the network with an identity layer (purely for the sake of example).
+    #   Note that unlike with lazy loaders, we don't need to do anything special to modify the network.
+    #   If we were using lazy loaders, we would need to use `func.extend()` as described
+    #   in example 03 and example 05.
+    prev_output = network.get_output(0)
+    network.unmark_output(prev_output)
+    output = network.add_identity(prev_output).get_output(0)
+    output.name = "output"
+    network.mark_output(output)
+
+    # Create a TensorRT IBuilderConfig so that we can build the engine with FP16 enabled.
+    config = create_config(builder, network, fp16=True)
+
+    engine = engine_from_network((builder, network), config)
+
+    # To reuse the engine elsewhere, we can serialize it and save it to a file.
+    save_engine(engine, path="identity.engine")
+
+    with TrtRunner(engine) as runner:
+        inp_data = np.ones((1, 1, 2, 2), dtype=np.float32)
+
+        # NOTE: The runner owns the output buffers and is free to reuse them between `infer()` calls.
+        # Thus, if you want to store results from multiple inferences, you should use `copy.deepcopy()`.
+        outputs = runner.infer(feed_dict={"x": inp_data})
+
+        assert np.array_equal(outputs["output"], inp_data)  # It's an identity model!
+
+        print("Inference succeeded!")
+
+
+if __name__ == "__main__":
+    main()

tools/Polygraphy/examples/api/06_immediate_eval_api/identity.onnx

@@ -0,0 +1,15 @@
+backend-test:[
+
+
x
y"Identity
+test_identityZ
+
x
+

+

+

+
+b
+
y
+

+

+

+

tools/Polygraphy/examples/api/06_immediate_eval_api/load_and_run.py

@@ -0,0 +1,44 @@
+#!/usr/bin/env python3
+#
+# SPDX-FileCopyrightText: Copyright (c) 1993-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+"""
+This script uses Polygraphy's immediately evaluated functional APIs
+to load the TensorRT engine built by `build_and_run.py` and run inference.
+"""
+import numpy as np
+from polygraphy.backend.common import bytes_from_path
+from polygraphy.backend.trt import TrtRunner, engine_from_bytes
+
+
+def main():
+    engine = engine_from_bytes(bytes_from_path("identity.engine"))
+
+    with TrtRunner(engine) as runner:
+        inp_data = np.ones((1, 1, 2, 2), dtype=np.float32)
+
+        # NOTE: The runner owns the output buffers and is free to reuse them between `infer()` calls.
+        # Thus, if you want to store results from multiple inferences, you should use `copy.deepcopy()`.
+        outputs = runner.infer(feed_dict={"x": inp_data})
+
+        assert np.array_equal(outputs["output"], inp_data)  # It's an identity model!
+
+        print("Inference succeeded!")
+
+
+if __name__ == "__main__":
+    main()

tools/Polygraphy/examples/api/06_immediate_eval_api/requirements.txt

@@ -0,0 +1 @@
+numpy