chore: import upstream snapshot with attribution

plugin/nvFasterRCNN/CMakeLists.txt

@@ -0,0 +1,21 @@
+#
+# SPDX-FileCopyrightText: Copyright (c) 1993-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+add_plugin_source(
+    nvFasterRCNNPlugin.cpp
+    nvFasterRCNNPlugin.h
+)

plugin/nvFasterRCNN/README.md

@@ -0,0 +1,109 @@
+# NvPluginFasterRCNN Plugin [DEPRECATED]
+
+**This plugin is deprecated since TensorRT 10.12 and will be removed in a future release. No alternatives are planned to be provided.**
+
+**Table Of Contents**
+- [Description](#description)
+    * [Structure](#structure)
+- [Parameters](#parameters)
+- [Additional resources](#additional-resources)
+- [License](#license)
+- [Changelog](#changelog)
+- [Known issues](#known-issues)
+
+## Description
+
+The `NvPluginFasterRCNN` performs object detection for the Faster R-CNN model. This plugin is included in TensorRT.
+
+`NvPluginFasterRCNN` decodes predicted bounding boxes, extracts their corresponding objectness score, extracts region of interest from predicted bounding boxes using non maximum suppression, and extracts the feature map of region of interest (ROI) using ROI pooling for downstreaming object classification tasks.
+
+This plugin is optimized for the above steps and it allows you to do Faster R-CNN inference in TensorRT.
+
+
+### Structure
+
+The `NvPluginFasterRCNN` takes four inputs; `scores`, `deltas`, `fmap` and `iinfo`.
+
+`scores`
+Bounding box (region proposal) objectness scores. scores has shape `[N, A x 2, H, W]` where `N` is the batch size, `A` is the number anchor boxes per pixel on the feature map, `H` is the height of feature map, and `W` is the width of feature map. The second dimension is `A x 2` because Faster R-CNN uses binary Softmax (probability of having object and probability of not having object) to classify the objectness for each bounding box.
+
+`deltas`
+Predicted bounding box offsets. `deltas` has shape `[N, A x 4, H, W]` where `N` is the batch size, `A` is the number anchor boxes per pixel on the feature map, `H` is the height of feature map, and `W` is the width of feature map. The second dimension is `A x 4` because each anchor box or bounding box consists of four parameters.
+
+`fmap`
+Feature map using for bounding box regression and classification. `fmap` has shape `[N, C, H, W]` where `N` is the batch size, `C` is the number of channels in feature map, `H` is the height of feature map, and `W` is the width of feature map.
+
+`iinfo`
+Original image input information. `iinfo` has shape `[N, 3]` where `N` is the batch size, `3` represents the height, width, and resize scale (the same as `featureStride`) of original input image.
+
+The `NvPluginFasterRCNN` generates the following two outputs:
+
+`rois`
+Coordinates of region of interest bounding boxes on the original input image. `rois` has shape `[N, 1, nmsMaxOut, 4]`, where `N` is the batch size, `nmsMaxOut` is the maximum number of region of interest bounding boxes, and `4` represents and region of interest bounding box coordinates `[x_1, y_1, x_2, y_2]`. Here, `x_1` and `y_1` are the coordinates of bounding box at the top-left corner, and `x_2` and `y_2` are the coordinates of bounding box at the bottom-right corner.
+
+`pfmap`
+ROI pooled feature map corresponding to the region of interest. `pfmap` has shape `[N, nmsMaxOut, C, poolingH, poolingW]` where `N` is the batch size, `nmsMaxOut` is the maximum number of region of interest bounding boxes, `C` is the number of channels in the feature map, `poolingH` is the height of ROI pooled feature map, and `poolingW` is the width of ROI pooled feature map.
+
+`NvPluginFasterRCNN` essentially does region proposal inference followed by region of interest (ROI) pooling.
+
+The proposal inference step includes three steps: extract objectness scores from `scores` input, decode predicted bounding box from `deltas` input, non-maximum suppression and get the region of interest bounding boxes using the extracted objectness scores and the decoded bounding boxes.
+
+The ROI pooling step uses the inferred region of interest bounding boxes information to extract its corresponding regions on feature map, and does POI pooling to get uniformly shaped features from different shaped region of interest bounding boxes.
+
+## Parameters
+
+`NvPluginFasterRCNN` has plugin creator class `RPROIPluginCreator` and plugin class `RPROIPlugin`.
+
+The `RPROIParams` data structure was used to create `RPROIPlugin` instance. The data structure is defined below and consists of the following attributes:
+```
+struct RPROIParams
+{
+	int poolingH, poolingW, featureStride, preNmsTop,
+		nmsMaxOut, anchorsRatioCount, anchorsScaleCount;
+	float iouThreshold, minBoxSize, spatialScale;
+};
+```
+
+| Type     | Parameter                | Description
+|----------|--------------------------|--------------------------------------------------------
+|`int`     |`poolingH`                |The height of the output in pixels after ROI pooling on the feature map.
+|`int`     |`poolingW`                |The width of the output in pixels after ROI pooling on the feature map.
+|`int`     |`featureStride`           |The ratio of the input image size to the feature map size; assuming the max pooling layers in the neural network uses square filters. For example, the input image size is `[1600, 800]`, after max pooling of size `[4, 4]` twice, the feature map now becomes `[100, 50]`, and `featureStride = 4^2 = 16`. In the Faster R-CNN settings from the paper, the value is `16`.
+|`int`     |`preNmsTop`               |The number of region proposals before applying NMS using objectness which is the probability of containing an object in the region proposal. The region proposals will be sorted using its objectness. If the number of regions you proposed from the previous region proposal network (RPN) is greater than `preNmsTop`, the exceeded region proposals with low objectness will be ignored. This value is particularly useful during training to control the number of bounding boxes for regression, but is theoretically useless during inference. In the Faster R-CNN settings from the paper, the value is `6000`.
+|`int`     |`nmsMaxOut`               |The number of region proposals after applying NMS. The region proposals will be sorted using its objectness and then applied NMS. At most the `nmsMaxOut` region proposals exist after NMS is considered as regions of interest.
+|`int`     |`anchorsRatioCount`       |The number of anchor box ratios. For example, if the anchor box ratios (aspect ratios) are 1:1, 1:2, and 2:1, then `anchorsRatioCount = 3`.
+|`int`     |`anchorsScaleCount`       |The number of anchor box scales. If the anchor box scales (scale factors) are 8, 16, and 32, then `anchorsScaleCount = 3`.
+|`float`   |`iouThreshold`            |The IOU threshold used for the NMS step.
+|`float`   |`minBoxSize`              |The minimum box size used for the anchor box calculation.
+|`float`   |`spatialScale`            |The inverse of `featureStride`, in other words, `spatialScale = 1.0 / featureStride`.
+
+
+## Additional resources
+
+The following resources provide a deeper understanding of the `NvPluginFasterRCNN` plugin:
+
+**Networks:**
+-   [Faster R-CNN](https://arxiv.org/abs/1506.01497)
+
+**Documentation:**
+-   [ROI Pooling Definition from Fast R-CNN](https://arxiv.org/abs/1504.08083)
+
+
+## License
+
+For terms and conditions for use, reproduction, and distribution, see the [TensorRT Software License Agreement](https://docs.nvidia.com/deeplearning/sdk/tensorrt-sla/index.html) 
+documentation.
+
+
+## Changelog
+
+May 2025
+Add deprecation note.
+
+May 2019
+This is the first release of this `README.md` file.
+
+
+## Known issues
+
+There are no known issues in this plugin.

plugin/nvFasterRCNN/nvFasterRCNNPlugin.cpp

@@ -0,0 +1,529 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 1993-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "nvFasterRCNNPlugin.h"
+#include <cstdio>
+#include <iostream>
+#include <memory>
+#include <string_view>
+
+namespace nvinfer1::plugin
+{
+namespace
+{
+char const* const kRPROI_PLUGIN_VERSION{"1"};
+char const* const kRPROI_PLUGIN_NAME{"RPROI_TRT"};
+} // namespace
+
+RPROIPlugin::RPROIPlugin(RPROIParams params, float const* anchorsRatios, float const* anchorsScales)
+    : params(params)
+{
+    /*
+     * It only supports the scenario where params.featureStride == params.minBoxSize
+     * assert(params.featureStride == params.minBoxSize);
+     */
+    PLUGIN_VALIDATE(params.anchorsRatioCount > 0 && params.anchorsScaleCount > 0);
+    anchorsRatiosHost = copyToHost(anchorsRatios, params.anchorsRatioCount);
+    anchorsScalesHost = copyToHost(anchorsScales, params.anchorsScaleCount);
+
+    PLUGIN_CHECK(
+        cudaMalloc((void**) &anchorsDev, 4 * params.anchorsRatioCount * params.anchorsScaleCount * sizeof(float)));
+    pluginStatus_t status = generateAnchors(0, params.anchorsRatioCount, anchorsRatiosHost, params.anchorsScaleCount,
+        anchorsScalesHost, params.featureStride, anchorsDev);
+    PLUGIN_VALIDATE(status == STATUS_SUCCESS);
+
+    deviceSmemSize = getSmemSize();
+}
+
+// Constructor for cloning one plugin instance to another
+RPROIPlugin::RPROIPlugin(RPROIParams params, float const* anchorsRatios, float const* anchorsScales, int32_t A,
+    int32_t C, int32_t H, int32_t W, float const* _anchorsDev, size_t deviceSmemSize, DataType inFeatureType,
+    DataType outFeatureType, DLayout_t inFeatureLayout)
+    : deviceSmemSize(deviceSmemSize)
+    , params(params)
+    , A(A)
+    , C(C)
+    , H(H)
+    , W(W)
+    , inFeatureType(inFeatureType)
+    , outFeatureType(outFeatureType)
+    , inFeatureLayout(inFeatureLayout)
+{
+    PLUGIN_VALIDATE(params.anchorsRatioCount > 0 && params.anchorsScaleCount > 0);
+    anchorsRatiosHost = copyToHost(anchorsRatios, params.anchorsRatioCount);
+    anchorsScalesHost = copyToHost(anchorsScales, params.anchorsScaleCount);
+
+    PLUGIN_CHECK(
+        cudaMalloc((void**) &anchorsDev, 4 * params.anchorsRatioCount * params.anchorsScaleCount * sizeof(float)));
+    // Perform deep copy
+    if (_anchorsDev != nullptr)
+    {
+        PLUGIN_CHECK(cudaMemcpy(anchorsDev, _anchorsDev,
+            4 * params.anchorsRatioCount * params.anchorsScaleCount * sizeof(float), cudaMemcpyDeviceToDevice));
+    }
+}
+
+RPROIPlugin::RPROIPlugin(void const* data, size_t length)
+{
+    deserialize(static_cast<int8_t const*>(data), length);
+}
+
+void RPROIPlugin::deserialize(int8_t const* data, size_t length)
+{
+    auto const* d{data};
+    params = *reinterpret_cast<RPROIParams const*>(d);
+    d += sizeof(RPROIParams);
+    A = read<int32_t>(d);
+    C = read<int32_t>(d);
+    H = read<int32_t>(d);
+    W = read<int32_t>(d);
+    inFeatureType = read<DataType>(d);
+    outFeatureType = read<DataType>(d);
+    inFeatureLayout = read<DLayout_t>(d);
+    anchorsRatiosHost = copyToHost(d, params.anchorsRatioCount);
+    d += params.anchorsRatioCount * sizeof(float);
+    anchorsScalesHost = copyToHost(d, params.anchorsScaleCount);
+    d += params.anchorsScaleCount * sizeof(float);
+    PLUGIN_VALIDATE(d == data + length);
+
+    PLUGIN_CHECK(
+        cudaMalloc((void**) &anchorsDev, 4 * params.anchorsRatioCount * params.anchorsScaleCount * sizeof(float)));
+    pluginStatus_t status = generateAnchors(0, params.anchorsRatioCount, anchorsRatiosHost, params.anchorsScaleCount,
+        anchorsScalesHost, params.featureStride, anchorsDev);
+    PLUGIN_VALIDATE(status == STATUS_SUCCESS);
+
+    deviceSmemSize = getSmemSize();
+}
+
+RPROIPlugin::~RPROIPlugin()
+{
+    if (anchorsDev != nullptr)
+    {
+        PLUGIN_CHECK(cudaFree(anchorsDev));
+        anchorsDev = nullptr;
+    }
+    if (anchorsRatiosHost != nullptr)
+    {
+        PLUGIN_CHECK(cudaFreeHost(anchorsRatiosHost));
+        anchorsRatiosHost = nullptr;
+    }
+    if (anchorsScalesHost != nullptr)
+    {
+        PLUGIN_CHECK(cudaFreeHost(anchorsScalesHost));
+        anchorsScalesHost = nullptr;
+    }
+}
+
+int32_t RPROIPlugin::initialize() noexcept
+{
+    return STATUS_SUCCESS;
+}
+
+size_t RPROIPlugin::getSmemSize() const noexcept
+{
+    int32_t devId{-1};
+    PLUGIN_CHECK(cudaGetDevice(&devId));
+    cudaDeviceProp prop{};
+    PLUGIN_CHECK(cudaGetDeviceProperties(&prop, devId));
+    return prop.sharedMemPerBlockOptin;
+}
+
+int32_t RPROIPlugin::getNbOutputs() const noexcept
+{
+    return 2;
+}
+
+Dims RPROIPlugin::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept
+{
+    PLUGIN_ASSERT(index >= 0 && index < 2);
+    PLUGIN_ASSERT(nbInputDims == 4);
+    PLUGIN_ASSERT(inputs[0].nbDims == 3 && inputs[1].nbDims == 3 && inputs[2].nbDims == 3 && inputs[3].nbDims == 3);
+    if (index == 0) // rois
+    {
+        return Dims3(1, params.nmsMaxOut, 4);
+    }
+    // Feature map of each ROI after ROI Pooling
+    // pool5
+    return Dims4(params.nmsMaxOut, inputs[2].d[0], params.poolingH, params.poolingW);
+}
+
+size_t RPROIPlugin::getWorkspaceSize(int32_t maxBatchSize) const noexcept
+{
+    return RPROIInferenceFusedWorkspaceSize(maxBatchSize, A, H, W, params.nmsMaxOut);
+}
+
+int32_t RPROIPlugin::enqueue(
+    int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept
+{
+    // Bounding box (region proposal) objectness scores.
+    void const* const scores = inputs[0];
+    // Predicted bounding box offsets.
+    void const* const deltas = inputs[1];
+    // Feature map using for bounding box regression and classification.
+    void const* const fmap = inputs[2];
+    // Original image input information.
+    void const* const iinfo = inputs[3];
+
+    // Coordinates of region of interest (ROI) bounding boxes on the original input image.
+    void* rois = outputs[0];
+    // ROI pooled feature map corresponding to the region of interest (ROI).
+    void* pfmap = outputs[1];
+
+    pluginStatus_t status = RPROIInferenceFused(stream, batchSize, A, C, H, W, params.poolingH, params.poolingW,
+        params.featureStride, params.preNmsTop, params.nmsMaxOut, params.iouThreshold, params.minBoxSize,
+        params.spatialScale, (float const*) iinfo, this->anchorsDev, nvinfer1::DataType::kFLOAT, NCHW, scores,
+        nvinfer1::DataType::kFLOAT, NCHW, deltas, inFeatureType, inFeatureLayout, fmap, workspace,
+        nvinfer1::DataType::kFLOAT, rois, outFeatureType, NCHW, pfmap, deviceSmemSize);
+    return status;
+}
+
+size_t RPROIPlugin::getSerializationSize() const noexcept
+{
+    size_t paramSize = sizeof(RPROIParams);
+    size_t intSize = sizeof(int32_t) * 4;
+    size_t ratiosSize = sizeof(float) * params.anchorsRatioCount;
+    size_t scalesSize = sizeof(float) * params.anchorsScaleCount;
+    size_t typeSize = sizeof(DataType) * 2;
+    size_t layoutSize = sizeof(DLayout_t);
+    return paramSize + intSize + ratiosSize + scalesSize + typeSize + layoutSize;
+}
+
+void RPROIPlugin::serialize(void* buffer) const noexcept
+{
+    char *d = reinterpret_cast<char*>(buffer), *a = d;
+    *reinterpret_cast<RPROIParams*>(d) = params;
+    d += sizeof(RPROIParams);
+    *reinterpret_cast<int32_t*>(d) = A;
+    d += sizeof(int32_t);
+    *reinterpret_cast<int32_t*>(d) = C;
+    d += sizeof(int32_t);
+    *reinterpret_cast<int32_t*>(d) = H;
+    d += sizeof(int32_t);
+    *reinterpret_cast<int32_t*>(d) = W;
+    d += sizeof(int32_t);
+    *reinterpret_cast<DataType*>(d) = inFeatureType;
+    d += sizeof(DataType);
+    *reinterpret_cast<DataType*>(d) = outFeatureType;
+    d += sizeof(DataType);
+    *reinterpret_cast<DLayout_t*>(d) = inFeatureLayout;
+    d += sizeof(DLayout_t);
+    d += copyFromHost(d, anchorsRatiosHost, params.anchorsRatioCount);
+    d += copyFromHost(d, anchorsScalesHost, params.anchorsScaleCount);
+    PLUGIN_ASSERT(d == a + getSerializationSize());
+}
+
+float* RPROIPlugin::copyToHost(void const* srcHostData, int32_t count) noexcept
+{
+    float* dstHostPtr = nullptr;
+    PLUGIN_CHECK(cudaMallocHost(&dstHostPtr, count * sizeof(float)));
+    PLUGIN_CHECK(cudaMemcpy(dstHostPtr, srcHostData, count * sizeof(float), cudaMemcpyHostToHost));
+    return dstHostPtr;
+}
+
+int32_t RPROIPlugin::copyFromHost(char* dstHostBuffer, void const* source, int32_t count) const noexcept
+{
+    PLUGIN_CHECK(cudaMemcpy(dstHostBuffer, source, count * sizeof(float), cudaMemcpyHostToHost));
+    return count * sizeof(float);
+}
+
+bool RPROIPlugin::supportsFormatCombination(
+    int32_t pos, PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) const noexcept
+{
+    PLUGIN_ASSERT(nbInputs == PluginNbInputs && nbOutputs == PluginNbOutputs && pos < nbInputs + nbOutputs);
+    bool isValidCombination = false;
+
+    // input:  bbox confindence, bbox offset, image info and output: rois
+    if (pos == 0 || pos == 1 || pos == 3 || pos == 4)
+    {
+        isValidCombination |= (inOut[pos].format == TensorFormat::kLINEAR && inOut[pos].type == DataType::kFLOAT);
+    }
+    // input:  feature map
+    else if (pos == 2)
+    {
+        isValidCombination |= (inOut[pos].format == TensorFormat::kLINEAR && inOut[pos].type == DataType::kINT8);
+        isValidCombination |= (inOut[pos].format == TensorFormat::kLINEAR && inOut[pos].type == DataType::kFLOAT);
+        isValidCombination |= (inOut[pos].format == TensorFormat::kCHW4 && inOut[pos].type == DataType::kINT8);
+        isValidCombination |= (inOut[pos].format == TensorFormat::kCHW32 && inOut[pos].type == DataType::kINT8);
+    }
+    // output: pooled feature map (data type should be the same with input feature map)
+    else if (pos == 5)
+    {
+        isValidCombination |= (inOut[pos].format == TensorFormat::kLINEAR && inOut[pos].type == DataType::kINT8);
+        isValidCombination |= (inOut[pos].format == TensorFormat::kLINEAR && inOut[pos].type == DataType::kFLOAT);
+        isValidCombination &= inOut[pos].type == inOut[2].type;
+    }
+    return isValidCombination;
+}
+
+char const* RPROIPlugin::getPluginType() const noexcept
+{
+    return kRPROI_PLUGIN_NAME;
+}
+
+char const* RPROIPlugin::getPluginVersion() const noexcept
+{
+    return kRPROI_PLUGIN_VERSION;
+}
+
+void RPROIPlugin::terminate() noexcept {}
+
+void RPROIPlugin::destroy() noexcept
+{
+    delete this;
+}
+
+IPluginV2Ext* RPROIPlugin::clone() const noexcept
+{
+    try
+    {
+        auto plugin = std::make_unique<RPROIPlugin>(params, anchorsRatiosHost, anchorsScalesHost, A, C, H, W,
+            anchorsDev, deviceSmemSize, inFeatureType, outFeatureType, inFeatureLayout);
+        plugin->setPluginNamespace(mPluginNamespace.c_str());
+        return plugin.release();
+    }
+    catch (std::exception const& e)
+    {
+        caughtError(e);
+    }
+    return nullptr;
+}
+
+// Set plugin namespace
+void RPROIPlugin::setPluginNamespace(char const* pluginNamespace) noexcept
+{
+    mPluginNamespace = pluginNamespace;
+}
+
+char const* RPROIPlugin::getPluginNamespace() const noexcept
+{
+    return mPluginNamespace.c_str();
+}
+
+// Return the DataType of the plugin output at the requested index.
+DataType RPROIPlugin::getOutputDataType(
+    int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept
+{
+    // Two outputs
+    PLUGIN_ASSERT(index == 0 || index == 1);
+    return DataType::kFLOAT;
+}
+
+DLayout_t RPROIPlugin::convertTensorFormat(TensorFormat const& srcFormat) const noexcept
+{
+    PLUGIN_ASSERT(
+        srcFormat == TensorFormat::kLINEAR || srcFormat == TensorFormat::kCHW4 || srcFormat == TensorFormat::kCHW32);
+    switch (srcFormat)
+    {
+    case nvinfer1::TensorFormat::kLINEAR: return DLayout_t::NCHW;
+    case nvinfer1::TensorFormat::kCHW4: return DLayout_t::NC4HW;
+    case nvinfer1::TensorFormat::kCHW32: return DLayout_t::NC32HW;
+    default: return DLayout_t::NCHW;
+    }
+}
+
+void RPROIPlugin::configurePlugin(
+    PluginTensorDesc const* in, int32_t nbInput, PluginTensorDesc const* out, int32_t nbOutput) noexcept
+{
+    PLUGIN_ASSERT(nbInput == PluginNbInputs);
+    PLUGIN_ASSERT(nbOutput == PluginNbOutputs);
+
+    A = params.anchorsRatioCount * params.anchorsScaleCount;
+    C = in[2].dims.d[0];
+    H = in[2].dims.d[1];
+    W = in[2].dims.d[2];
+    inFeatureType = in[2].type;
+    outFeatureType = out[1].type;
+    inFeatureLayout = convertTensorFormat(in[2].format);
+
+    PLUGIN_ASSERT(in[0].dims.d[0] == (2 * A) && in[1].dims.d[0] == (4 * A));
+    PLUGIN_ASSERT(in[0].dims.d[1] == in[1].dims.d[1] && in[0].dims.d[1] == in[2].dims.d[1]);
+    PLUGIN_ASSERT(in[0].dims.d[2] == in[1].dims.d[2] && in[0].dims.d[2] == in[2].dims.d[2]);
+    PLUGIN_ASSERT(out[0].dims.nbDims == 3 // rois
+        && out[1].dims.nbDims == 4);      // pooled feature map
+    PLUGIN_ASSERT(out[0].dims.d[0] == 1 && out[0].dims.d[1] == params.nmsMaxOut && out[0].dims.d[2] == 4);
+    PLUGIN_ASSERT(out[1].dims.d[0] == params.nmsMaxOut && out[1].dims.d[1] == C && out[1].dims.d[2] == params.poolingH
+        && out[1].dims.d[3] == params.poolingW);
+}
+
+// Attach the plugin object to an execution context and grant the plugin the access to some context resource.
+void RPROIPlugin::attachToContext(
+    cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept
+{
+}
+
+// Detach the plugin object from its execution context.
+void RPROIPlugin::detachFromContext() noexcept {}
+
+RPROIPluginCreator::RPROIPluginCreator()
+{
+    mPluginAttributes.clear();
+    mPluginAttributes.emplace_back(PluginField("poolingH", nullptr, PluginFieldType::kINT32, 1));
+    mPluginAttributes.emplace_back(PluginField("poolingW", nullptr, PluginFieldType::kINT32, 1));
+    mPluginAttributes.emplace_back(PluginField("featureStride", nullptr, PluginFieldType::kINT32, 1));
+    mPluginAttributes.emplace_back(PluginField("preNmsTop", nullptr, PluginFieldType::kINT32, 1));
+    mPluginAttributes.emplace_back(PluginField("nmsMaxOut", nullptr, PluginFieldType::kINT32, 1));
+    mPluginAttributes.emplace_back(PluginField("anchorsRatioCount", nullptr, PluginFieldType::kINT32, 1));
+    mPluginAttributes.emplace_back(PluginField("anchorsScaleCount", nullptr, PluginFieldType::kINT32, 1));
+    mPluginAttributes.emplace_back(PluginField("iouThreshold", nullptr, PluginFieldType::kFLOAT32, 1));
+    mPluginAttributes.emplace_back(PluginField("minBoxSize", nullptr, PluginFieldType::kFLOAT32, 1));
+    mPluginAttributes.emplace_back(PluginField("spatialScale", nullptr, PluginFieldType::kFLOAT32, 1));
+
+    // TODO Do we need to pass the size attribute here for float arrarys, we
+    // dont have that information at this point.
+    mPluginAttributes.emplace_back(PluginField("anchorsRatios", nullptr, PluginFieldType::kFLOAT32, 1));
+    mPluginAttributes.emplace_back(PluginField("anchorsScales", nullptr, PluginFieldType::kFLOAT32, 1));
+
+    mFC.nbFields = mPluginAttributes.size();
+    mFC.fields = mPluginAttributes.data();
+}
+
+RPROIPluginCreator::~RPROIPluginCreator()
+{
+    // Free allocated memory (if any) here
+}
+
+char const* RPROIPluginCreator::getPluginName() const noexcept
+{
+    return kRPROI_PLUGIN_NAME;
+}
+
+char const* RPROIPluginCreator::getPluginVersion() const noexcept
+{
+    return kRPROI_PLUGIN_VERSION;
+}
+
+PluginFieldCollection const* RPROIPluginCreator::getFieldNames() noexcept
+{
+    return &mFC;
+}
+
+// NOLINTNEXTLINE(readability-function-cognitive-complexity)
+IPluginV2Ext* RPROIPluginCreator::createPlugin(char const* name, PluginFieldCollection const* fc) noexcept
+{
+    try
+    {
+        using namespace std::string_view_literals;
+        PluginField const* fields = fc->fields;
+        int32_t nbFields = fc->nbFields;
+
+        for (int32_t i = 0; i < nbFields; ++i)
+        {
+            std::string_view const attrName = fields[i].name;
+            if (attrName == "poolingH"sv)
+            {
+                PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32);
+                params.poolingH = *(static_cast<int32_t const*>(fields[i].data));
+            }
+            if (attrName == "poolingW"sv)
+            {
+                PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32);
+                params.poolingW = *(static_cast<int32_t const*>(fields[i].data));
+            }
+            if (attrName == "featureStride"sv)
+            {
+                PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32);
+                params.featureStride = *(static_cast<int32_t const*>(fields[i].data));
+            }
+            if (attrName == "preNmsTop"sv)
+            {
+                PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32);
+                params.preNmsTop = *(static_cast<int32_t const*>(fields[i].data));
+            }
+            if (attrName == "nmsMaxOut"sv)
+            {
+                PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32);
+                params.nmsMaxOut = *(static_cast<int32_t const*>(fields[i].data));
+            }
+            if (attrName == "anchorsRatioCount"sv)
+            {
+                PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32);
+                params.anchorsRatioCount = *(static_cast<int32_t const*>(fields[i].data));
+            }
+            if (attrName == "anchorsScaleCount"sv)
+            {
+                PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32);
+                params.anchorsScaleCount = *(static_cast<int32_t const*>(fields[i].data));
+            }
+            if (attrName == "iouThreshold"sv)
+            {
+                PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kFLOAT32);
+                params.iouThreshold = static_cast<float>(*(static_cast<float const*>(fields[i].data)));
+            }
+            if (attrName == "minBoxSize"sv)
+            {
+                PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kFLOAT32);
+                params.minBoxSize = static_cast<float>(*(static_cast<float const*>(fields[i].data)));
+            }
+            if (attrName == "spatialScale"sv)
+            {
+                PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kFLOAT32);
+                params.spatialScale = static_cast<float>(*(static_cast<float const*>(fields[i].data)));
+            }
+            if (attrName == "anchorsRatios"sv)
+            {
+                PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kFLOAT32);
+                anchorsRatios.reserve(params.anchorsRatioCount);
+                float const* ratios = static_cast<float const*>(fields[i].data);
+                for (int32_t j = 0; j < params.anchorsRatioCount; ++j)
+                {
+                    anchorsRatios.push_back(*ratios);
+                    ratios++;
+                }
+            }
+            if (attrName == "anchorsScales"sv)
+            {
+                PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kFLOAT32);
+                anchorsScales.reserve(params.anchorsScaleCount);
+                float const* scales = static_cast<float const*>(fields[i].data);
+                for (int32_t j = 0; j < params.anchorsScaleCount; ++j)
+                {
+                    anchorsScales.push_back(*scales);
+                    scales++;
+                }
+            }
+        }
+
+        // This object will be deleted when the network is destroyed, which will
+        // call RPROIPlugin::terminate()
+        auto plugin = std::make_unique<RPROIPlugin>(params, anchorsRatios.data(), anchorsScales.data());
+        plugin->setPluginNamespace(mNamespace.c_str());
+        return plugin.release();
+    }
+    catch (std::exception const& e)
+    {
+        caughtError(e);
+    }
+    return nullptr;
+}
+
+IPluginV2Ext* RPROIPluginCreator::deserializePlugin(
+    char const* name, void const* serialData, size_t serialLength) noexcept
+{
+    try
+    {
+        // This object will be deleted when the network is destroyed, which will
+        // call RPROIPlugin::terminate()
+        auto plugin = std::make_unique<RPROIPlugin>(serialData, serialLength);
+        plugin->setPluginNamespace(mNamespace.c_str());
+        return plugin.release();
+    }
+    catch (std::exception const& e)
+    {
+        caughtError(e);
+    }
+    return nullptr;
+}
+} // namespace nvinfer1::plugin

plugin/nvFasterRCNN/nvFasterRCNNPlugin.h

@@ -0,0 +1,144 @@
+/*
+ * SPDX-FileCopyrightText: Copyright (c) 1993-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#ifndef TRT_NV_PLUGIN_FASTER_RCNN_H
+#define TRT_NV_PLUGIN_FASTER_RCNN_H
+
+#include "common/kernels/kernel.h"
+#include "common/plugin.h"
+#include <vector>
+
+namespace nvinfer1
+{
+namespace plugin
+{
+
+class RPROIPlugin : public IPluginV2IOExt
+{
+public:
+    RPROIPlugin(RPROIParams params, float const* anchorsRatios, float const* anchorsScales);
+
+    RPROIPlugin(RPROIParams params, float const* anchorsRatios, float const* anchorsScales, int32_t A, int32_t C,
+        int32_t H, int32_t W, float const* anchorsDev, size_t deviceSmemSize, DataType inFeatureType,
+        DataType outFeatureType, DLayout_t inFeatureLayout);
+
+    RPROIPlugin(void const* data, size_t length);
+
+    ~RPROIPlugin() override;
+
+    int32_t getNbOutputs() const noexcept override;
+
+    Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override;
+
+    int32_t initialize() noexcept override;
+
+    void terminate() noexcept override;
+
+    size_t getWorkspaceSize(int32_t maxBatchSize) const noexcept override;
+
+    int32_t enqueue(int32_t batchSize, void const* const* inputs, void* const* outputs, void* workspace,
+        cudaStream_t stream) noexcept override;
+
+    size_t getSerializationSize() const noexcept override;
+
+    void serialize(void* buffer) const noexcept override;
+
+    bool supportsFormatCombination(
+        int32_t pos, PluginTensorDesc const* inOut, int32_t nbInputs, int32_t nbOutputs) const noexcept override;
+
+    char const* getPluginType() const noexcept override;
+
+    char const* getPluginVersion() const noexcept override;
+
+    void destroy() noexcept override;
+
+    IPluginV2Ext* clone() const noexcept override;
+
+    void setPluginNamespace(char const* pluginNamespace) noexcept override;
+
+    char const* getPluginNamespace() const noexcept override;
+
+    DataType getOutputDataType(
+        int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override;
+
+    void attachToContext(
+        cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept override;
+
+    void configurePlugin(
+        PluginTensorDesc const* in, int32_t nbInput, PluginTensorDesc const* out, int32_t nbOutput) noexcept override;
+
+    void detachFromContext() noexcept override;
+
+private:
+    void deserialize(int8_t const* data, size_t length);
+    float* copyToHost(void const* srcHostData, int32_t count) noexcept;
+
+    int32_t copyFromHost(char* dstHostBuffer, void const* source, int32_t count) const noexcept;
+
+    size_t getSmemSize() const noexcept;
+
+    DLayout_t convertTensorFormat(TensorFormat const& srcFormat) const noexcept;
+
+    // These won't be serialized
+    float* anchorsDev{};
+    std::string mPluginNamespace;
+    const int32_t PluginNbInputs{4};
+    const int32_t PluginNbOutputs{2};
+    // this plugin may load the whole feature map in smem. we can set different smem size according to the device.
+    size_t deviceSmemSize{};
+
+    // These need to be serialized
+    RPROIParams params{};
+    int32_t A{};
+    int32_t C{};
+    int32_t H{};
+    int32_t W{};
+    float* anchorsRatiosHost{};
+    float* anchorsScalesHost{};
+    DataType inFeatureType{};
+    DataType outFeatureType{};
+    DLayout_t inFeatureLayout{};
+};
+
+class RPROIPluginCreator : public nvinfer1::pluginInternal::BaseCreator
+{
+public:
+    RPROIPluginCreator();
+
+    ~RPROIPluginCreator() override;
+
+    char const* getPluginName() const noexcept override;
+
+    char const* getPluginVersion() const noexcept override;
+
+    PluginFieldCollection const* getFieldNames() noexcept override;
+
+    IPluginV2Ext* createPlugin(char const* name, PluginFieldCollection const* fc) noexcept override;
+
+    IPluginV2Ext* deserializePlugin(char const* name, void const* serialData, size_t serialLength) noexcept override;
+
+private:
+    PluginFieldCollection mFC;
+    RPROIParams params;
+    std::vector<float> anchorsRatios;
+    std::vector<float> anchorsScales;
+    std::vector<PluginField> mPluginAttributes;
+};
+
+} // namespace plugin
+} // namespace nvinfer1
+
+#endif // TRT_NV_PLUGIN_FASTER_RCNN_H