nvidia--tensorrt/samples

TensorRT Samples

Contents

1. "Hello World" Samples

Sample	Language	Format	Description
sampleOnnxMNIST	C++	ONNX	“Hello World” For TensorRT With ONNX
network_api_pytorch_mnist	Python	INetwork	“Hello World” For TensorRT Using Pytorch

2. TensorRT API Samples

Sample	Language	Format	Description
sampleCudla	C++	INetwork	Using The CuDLA API To Run A TensorRT Engine (aarch64 only)
sampleDynamicReshape	C++	ONNX	Digit Recognition With Dynamic Shapes In TensorRT
sampleEditableTimingCache	C++	INetwork	Create a deterministic build using editable timing cache
sampleNamedDimensions	C++	ONNX	Working with named input dimensions
sampleNonZeroPlugin	C++	INetwork	Adding plugin with data-dependent output shapes
sampleIOFormats	C++	ONNX	Specifying TensorRT I/O Formats
sampleProgressMonitor	C++	ONNX	Progress Monitor API usage
trtexec	C++	All	TensorRT Command-Line Wrapper: trtexec
engine_refit_onnx_bidaf	Python	ONNX	refitting an engine built from an ONNX model via parsers.
introductory_parser_samples	Python	ONNX	Introduction To Importing Models Using TensorRT Parsers
onnx_packnet	Python	ONNX	TensorRT Inference Of ONNX Models With Custom Layers
simpleProgressMonitor	Python	ONNX	Progress Monitor API usage
python_plugin	Python	INetwork/ONNX	Python-based TRT plugins
non_zero_plugin	Python	INetwork/ONNX	Python-based TRT plugin for NonZero op
cute_dsl_plugin	Python	INetwork	Python-based TRT plugin for RMSNorm with a CuteDSL kernel
attention_mdtrt	Python	ONNX	Multi-device attention inference with MPI and NCCL

3. Application Samples

Sample	Language	Format	Description
detectron2	Python	ONNX	Support for Detectron 2 Mask R-CNN R50-FPN 3x model in TensorRT

4. Safety Samples

Sample	Language	Format	Description
sampleSafeMNIST	C++	ONNX	Build a Safety Engine for MNIST
sampleSafePluginV3	C++	ONNX	Use Safety-Supported Plugins With Safety Engines
trtSafeExec	C++	ONNX	TensorRT Command-Line Wrapper With Safety Options

Preparing sample data

Many samples require the TensorRT sample data package. If not already mounted under /usr/src/tensorrt/data (NVIDIA NGC containers), download and extract it:

1. Download the current TensorRT sample data package. Sample data is updated only when needed, so the package may be hosted under an earlier TensorRT release.

2. Extract and set up the data:

   unzip tensorrt_sample_data_xxx.zip
   mkdir -p /usr/src/tensorrt/data
   cp -r tensorrt_sample_data_*/* /usr/src/tensorrt/data/
   export TRT_DATADIR=/usr/src/tensorrt/data
   

After extraction, the data directory structure should be:

$TRT_DATADIR/
├── int8_api/
├── mnist/
└── resnet50/