wehub-resource-sync
65 lines
3.5 KiB
Markdown
65 lines
3.5 KiB
Markdown
# TensorRT Samples
|
|
|
|
## Contents
|
|
|
|
### 1. "Hello World" Samples
|
|
|
|
| Sample | Language | Format | Description |
|
|
|---|---|---|---|
|
|
| [sampleOnnxMNIST](sampleOnnxMNIST) | C++ | ONNX | “Hello World” For TensorRT With ONNX |
|
|
| [network_api_pytorch_mnist](python/network_api_pytorch_mnist) | Python | INetwork | “Hello World” For TensorRT Using Pytorch |
|
|
|
|
### 2. TensorRT API Samples
|
|
| Sample | Language | Format | Description |
|
|
|---|---|---|---|
|
|
| [sampleCudla](sampleCudla) | C++ | INetwork | Using The CuDLA API To Run A TensorRT Engine (aarch64 only) |
|
|
| [sampleDynamicReshape](sampleDynamicReshape) | C++ | ONNX | Digit Recognition With Dynamic Shapes In TensorRT |
|
|
| [sampleEditableTimingCache](sampleEditableTimingCache) | C++ | INetwork | Create a deterministic build using editable timing cache |
|
|
| [sampleNamedDimensions](sampleNamedDimensions) | C++ | ONNX | Working with named input dimensions |
|
|
| [sampleNonZeroPlugin](sampleNonZeroPlugin) | C++ | INetwork | Adding plugin with data-dependent output shapes |
|
|
| [sampleIOFormats](sampleIOFormats) | C++ | ONNX | Specifying TensorRT I/O Formats |
|
|
| [sampleProgressMonitor](sampleProgressMonitor) | C++ | ONNX | Progress Monitor API usage |
|
|
| [trtexec](trtexec) | C++ | All | TensorRT Command-Line Wrapper: trtexec |
|
|
| [engine_refit_onnx_bidaf](python/engine_refit_onnx_bidaf) | Python | ONNX | refitting an engine built from an ONNX model via parsers. |
|
|
| [introductory_parser_samples](python/introductory_parser_samples) | Python | ONNX | Introduction To Importing Models Using TensorRT Parsers |
|
|
| [onnx_packnet](python/onnx_packnet) | Python | ONNX | TensorRT Inference Of ONNX Models With Custom Layers |
|
|
| [simpleProgressMonitor](python/simple_progress_monitor) | Python | ONNX | Progress Monitor API usage |
|
|
| [python_plugin](python/python_plugin) | Python | INetwork/ONNX | Python-based TRT plugins |
|
|
| [non_zero_plugin](python/non_zero_plugin) | Python | INetwork/ONNX | Python-based TRT plugin for NonZero op |
|
|
| [cute_dsl_plugin](python/cute_dsl_plugin) | Python | INetwork | Python-based TRT plugin for RMSNorm with a CuteDSL kernel |
|
|
| [attention_mdtrt](python/attention_mdtrt) | Python | ONNX | Multi-device attention inference with MPI and NCCL |
|
|
|
|
### 3. Application Samples
|
|
| Sample | Language | Format | Description |
|
|
|---|---|---|---|
|
|
| [detectron2](python/detectron2) | Python | ONNX | Support for Detectron 2 Mask R-CNN R50-FPN 3x model in TensorRT |
|
|
|
|
### 4. Safety Samples
|
|
| Sample | Language | Format | Description |
|
|
|---|---|---|---|
|
|
| [sampleSafeMNIST](sampleSafeMNIST) | C++ | ONNX | Build a Safety Engine for MNIST |
|
|
| [sampleSafePluginV3](sampleSafePluginV3) | C++ | ONNX | Use Safety-Supported Plugins With Safety Engines |
|
|
| [trtSafeExec](trtSafeExec) | C++ | ONNX | TensorRT Command-Line Wrapper With Safety Options |
|
|
|
|
## Preparing sample data
|
|
|
|
Many samples require the TensorRT sample data package. If not already mounted under `/usr/src/tensorrt/data` (NVIDIA NGC containers), download and extract it:
|
|
|
|
1. Download the current [TensorRT sample data package](https://github.com/NVIDIA/TensorRT/releases/download/v11.0/tensorrt_sample_data_20260602.zip). Sample data is updated only when needed, so the package may be hosted under an earlier TensorRT release.
|
|
|
|
2. Extract and set up the data:
|
|
```bash
|
|
unzip tensorrt_sample_data_xxx.zip
|
|
mkdir -p /usr/src/tensorrt/data
|
|
cp -r tensorrt_sample_data_*/* /usr/src/tensorrt/data/
|
|
export TRT_DATADIR=/usr/src/tensorrt/data
|
|
```
|
|
|
|
After extraction, the data directory structure should be:
|
|
```
|
|
$TRT_DATADIR/
|
|
├── int8_api/
|
|
├── mnist/
|
|
└── resnet50/
|
|
```
|