trtexec_safe is a performance testing tool for TensorRT safety runtime. trtexec_safe can load a serialized engine file and test the performance of running inference with TensorRT safety runtime. trtexec_safe provides many options, like iterations, average runs, streams, for performance timing.

Note: trtexec_safe only provides the functionality of doing inference with a pre-built engine file. The engine should be built with EngineCapability::kSAFETY. trtexec_safe is not safety-certified, therefore, it could produce AUTOSAR violations in the source code.

Building `trtexec_safe`

Compile the sample by following the build instructions in the TensorRT README. This will build the trtexec_safe binary.

The options that control compiling against safety runtime or standard runtime are summarized in the following table.

Compile Option	Default	Description
TRT_SAFETY_INFERENCE_ONLY	OFF	When enabled, build the infer part only, skip compiling the builder part.

Using `trtexec_safe`

trtexec_safe requires a prebuilt safe TensorRT engine as input. Use trtexec to build a safe TensorRT engine.
```
trtexec --safe --saveEngine=/path/to/safe/engine --model=/path/to/model
```
where --safe indicates trtexec to build a safety certified TensorRT engine and --saveEngine specifies the path to save the output engine. For other details of trtexec, see <TensorRT root directory>/samples/trtexec/README.md.
Use trtexec_safe to load the engine and run performance testing.
```
trtexec_safe --loadEngine=/path/to/safe/engine
```
trtexec_safe will do inference for several iterations and give a final median execution time on both the CPU and GPU. For other options, refer to the Tool command line arguments section.

Tool command line arguments

To see the full list of available options and their descriptions, use the -h or --help command line option.

trtexec_safe [-h or --help]

Additional resources

The following resources provide more details about trtexec_safe:

Documentation

License

For terms and conditions for use, reproduction, and distribution, see the TensorRT Software License Agreement documentation.

Changelog

June 2020 This is the first release of this README.md file.

September 2020 This sample was updated to fit TensorRt API changes: IRuntime::validateSerializedEngine is removed.

March 2022 This sample was updated for DriveOS 6.0 and later releases.

Dec. 2025 This sample was updated to use the CMake-based build system.

Known issues

There are no known issues in this sample.

README.md Unescape Escape

TensorRT Safety Runtime Command-Line Wrapper: trtexec_safe

Description

Building trtexec_safe

Using trtexec_safe