chore: import upstream snapshot with attribution
Docker Image CI / build-ubuntu2004 (push) Has been cancelled
Docker Image CI / build-ubuntu2004 (push) Has been cancelled
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,23 @@
|
||||
#
|
||||
# SPDX-FileCopyrightText: Copyright (c) 1993-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#
|
||||
|
||||
add_plugin_source(
|
||||
multilevelProposeROIPlugin.cpp
|
||||
multilevelProposeROIPlugin.h
|
||||
tlt_mrcnn_config.h
|
||||
)
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
#
|
||||
# SPDX-FileCopyrightText: Copyright (c) 2022-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#
|
||||
---
|
||||
name: MultilevelProposeROI_TRT
|
||||
interface: "IPluginV2Ext"
|
||||
versions:
|
||||
"1":
|
||||
attributes:
|
||||
- prenms_topk
|
||||
- keep_topk
|
||||
- fg_threshold
|
||||
- iou_threshold
|
||||
- image_size
|
||||
attribute_types:
|
||||
prenms_topk: int32
|
||||
keep_topk: int32
|
||||
fg_threshold: float32
|
||||
iou_threshold: float32
|
||||
image_size: int32
|
||||
attribute_length:
|
||||
prenms_topk: 1
|
||||
keep_topk: 1
|
||||
fg_threshold: 1
|
||||
iou_threshold: 1
|
||||
image_size: 3
|
||||
attribute_options:
|
||||
prenms_topk:
|
||||
min: "0"
|
||||
max: "=4096"
|
||||
keep_topk:
|
||||
min: "0"
|
||||
max: "=pinf"
|
||||
fg_threshold:
|
||||
min: "=0"
|
||||
max: "=pinf"
|
||||
iou_threshold:
|
||||
min: "=0"
|
||||
max: "=pinf"
|
||||
image_size:
|
||||
min: "0, 0, 0"
|
||||
max: "=pinf, =1000, =1000" # dims 2 & 3 are capped to avoid timeout
|
||||
attributes_required:
|
||||
- prenms_topk
|
||||
- keep_topk
|
||||
- fg_threshold
|
||||
- iou_threshold
|
||||
...
|
||||
@@ -0,0 +1,83 @@
|
||||
# MultilevelProposeROI Plugin [DEPRECATED]
|
||||
|
||||
**This plugin is deprecated since TensorRT 10.12 and will be removed in a future release. No alternatives are planned to be provided.**
|
||||
|
||||
**Table Of Contents**
|
||||
- [Description](#description)
|
||||
* [Structure](#structure)
|
||||
- [Parameters](#parameters)
|
||||
- [Additional resources](#additional-resources)
|
||||
- [License](#license)
|
||||
- [Changelog](#changelog)
|
||||
- [Known issues](#known-issues)
|
||||
|
||||
## Description
|
||||
|
||||
The `MultilevelProposeROI` plugin generates the first-stage detection (ROI candidates) from the scores, refinement information from RPN(Region Proposal Network) and pre-defined anchors. It is
|
||||
used in sampleMaskRCNN.
|
||||
|
||||
|
||||
### Structure
|
||||
|
||||
This plugin supports the NCHW format. It takes two input tensors: `object_score` and `object_delta`
|
||||
|
||||
`object_score` is the objectness score from RPN. `object_score`'s shape is `[N, anchors, 2, 1]` where `N` is the batch_size, `anchors` is the total number of anchors and `2` means 2
|
||||
classes of objectness --- foreground and background .
|
||||
|
||||
`object_delta` is the refinement information from RPN of shape `[N, anchors, 4, 1]`. `4` means 4 elements of refinement information --- `[dy, dx, dh, dw]`
|
||||
|
||||
This plugin generates one output tensor of shape `[N, keep_topk, 4]` where `keep_topk` is the maximum number of detections left after NMS and `4` means coordinates of ROI
|
||||
candidates `[y1, x1, y2, x2]`
|
||||
|
||||
Instead of fed as input in Keras, the default anchors used in this plugin are generated upon `initialization`.
|
||||
For resnet50 + 832*1344 input shape, the number of anchors can be computed as
|
||||
```
|
||||
Anchors in feature map P2: 208*336*3
|
||||
Anchors in feature map P3: 104*168*3
|
||||
Anchors in feature map P4: 52*84*3
|
||||
Anchors in feature map P5: 26*42*3
|
||||
Anchors in feature map P6(maxpooling): 13*21*3
|
||||
|
||||
```
|
||||
|
||||
## Parameters
|
||||
|
||||
This plugin has the plugin creator class `MultilevelProposeROIPluginCreator` and the plugin class `MultilevelProposeROI`.
|
||||
|
||||
The following parameters were used to create `MultilevelProposeROI` instance:
|
||||
|
||||
| Type | Parameter | Description
|
||||
|-------------------|----------------------------------|--------------------------------------------------------
|
||||
|`int` |`prenms_topk` |The number of ROIs which will be kept before NMS.
|
||||
|`int` |`keep_topk` |Number of detections will be kept after NMS.
|
||||
|`float` |`iou_threshold` |IOU threshold value used in NMS.
|
||||
|`int[3]` |`image_size` |Input image shape in CHW. Defaults to [3, 832, 1344]
|
||||
|
||||
## Limitations
|
||||
|
||||
The attribute `prenms_topk` is capped at 4096 to support embedded devices with smaller shared memory capacity.
|
||||
|
||||
To enable support for a device with higher memory, calls to `sortPerClass`, `PerClassNMS` and `KeepTopKGatherBoxScore` can be modified in `MultilevelPropose` ([maskRCNNKernels.cu](https://github.com/NVIDIA/TensorRT/blob/main/plugin/common/kernels/maskRCNNKernels.cu)).
|
||||
|
||||
## Additional resources
|
||||
|
||||
|
||||
## License
|
||||
|
||||
For terms and conditions for use, reproduction, and distribution, see the [TensorRT Software License Agreement](https://docs.nvidia.com/deeplearning/sdk/tensorrt-sla/index.html)
|
||||
documentation.
|
||||
|
||||
|
||||
## Changelog
|
||||
|
||||
May 2025
|
||||
Add deprecation note.
|
||||
|
||||
January 2022: The [Limitations](#limitations) section was added to this `README.md` file to document limitations of the plugin related to the maximum number of anchors it can support.
|
||||
|
||||
June 2020: This is the first release of this `README.md` file.
|
||||
|
||||
|
||||
## Known issues
|
||||
|
||||
There are no known issues in this plugin.
|
||||
@@ -0,0 +1,514 @@
|
||||
/*
|
||||
* SPDX-FileCopyrightText: Copyright (c) 1993-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
#include "multilevelProposeROIPlugin.h"
|
||||
#include "common/plugin.h"
|
||||
#include "multilevelProposeROI/tlt_mrcnn_config.h"
|
||||
#include <algorithm>
|
||||
#include <cuda_runtime_api.h>
|
||||
#include <iostream>
|
||||
#include <math.h>
|
||||
#include <string_view>
|
||||
|
||||
#include <fstream>
|
||||
|
||||
using namespace nvinfer1;
|
||||
using namespace plugin;
|
||||
using nvinfer1::plugin::MultilevelProposeROI;
|
||||
using nvinfer1::plugin::MultilevelProposeROIPluginCreator;
|
||||
|
||||
namespace
|
||||
{
|
||||
char const* const kMULTILEVELPROPOSEROI_PLUGIN_VERSION{"1"};
|
||||
char const* const kMULTILEVELPROPOSEROI_PLUGIN_NAME{"MultilevelProposeROI_TRT"};
|
||||
} // namespace
|
||||
|
||||
MultilevelProposeROIPluginCreator::MultilevelProposeROIPluginCreator() noexcept
|
||||
{
|
||||
mPluginAttributes.clear();
|
||||
mPluginAttributes.emplace_back(PluginField("prenms_topk", nullptr, PluginFieldType::kINT32, 1));
|
||||
mPluginAttributes.emplace_back(PluginField("keep_topk", nullptr, PluginFieldType::kINT32, 1));
|
||||
mPluginAttributes.emplace_back(PluginField("fg_threshold", nullptr, PluginFieldType::kFLOAT32, 1));
|
||||
mPluginAttributes.emplace_back(PluginField("iou_threshold", nullptr, PluginFieldType::kFLOAT32, 1));
|
||||
mPluginAttributes.emplace_back(PluginField("image_size", nullptr, PluginFieldType::kINT32, 3));
|
||||
|
||||
mFC.nbFields = mPluginAttributes.size();
|
||||
mFC.fields = mPluginAttributes.data();
|
||||
}
|
||||
|
||||
char const* MultilevelProposeROIPluginCreator::getPluginName() const noexcept
|
||||
{
|
||||
return kMULTILEVELPROPOSEROI_PLUGIN_NAME;
|
||||
}
|
||||
|
||||
char const* MultilevelProposeROIPluginCreator::getPluginVersion() const noexcept
|
||||
{
|
||||
return kMULTILEVELPROPOSEROI_PLUGIN_VERSION;
|
||||
}
|
||||
|
||||
PluginFieldCollection const* MultilevelProposeROIPluginCreator::getFieldNames() noexcept
|
||||
{
|
||||
return &mFC;
|
||||
}
|
||||
|
||||
IPluginV2Ext* MultilevelProposeROIPluginCreator::createPlugin(
|
||||
char const* name, PluginFieldCollection const* fc) noexcept
|
||||
{
|
||||
try
|
||||
{
|
||||
using namespace std::string_view_literals;
|
||||
plugin::validateRequiredAttributesExist({"prenms_topk", "keep_topk", "fg_threshold", "iou_threshold"}, fc);
|
||||
auto imageSize = TLTMaskRCNNConfig::IMAGE_SHAPE;
|
||||
PluginField const* fields = fc->fields;
|
||||
for (int32_t i = 0; i < fc->nbFields; ++i)
|
||||
{
|
||||
std::string_view const attrName = fields[i].name;
|
||||
if (attrName == "prenms_topk"sv)
|
||||
{
|
||||
PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32);
|
||||
mPreNMSTopK = *(static_cast<int32_t const*>(fields[i].data));
|
||||
}
|
||||
if (attrName == "keep_topk"sv)
|
||||
{
|
||||
PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32);
|
||||
mKeepTopK = *(static_cast<int32_t const*>(fields[i].data));
|
||||
}
|
||||
if (attrName == "fg_threshold"sv)
|
||||
{
|
||||
PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kFLOAT32);
|
||||
mFGThreshold = *(static_cast<float const*>(fields[i].data));
|
||||
}
|
||||
if (attrName == "iou_threshold"sv)
|
||||
{
|
||||
PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kFLOAT32);
|
||||
mIOUThreshold = *(static_cast<float const*>(fields[i].data));
|
||||
}
|
||||
if (attrName == "image_size"sv)
|
||||
{
|
||||
PLUGIN_VALIDATE(fields[i].type == PluginFieldType::kINT32);
|
||||
auto const dims = static_cast<int32_t const*>(fields[i].data);
|
||||
std::copy_n(dims, 3, imageSize.d);
|
||||
}
|
||||
}
|
||||
return new MultilevelProposeROI(mPreNMSTopK, mKeepTopK, mFGThreshold, mIOUThreshold, imageSize);
|
||||
}
|
||||
catch (std::exception const& e)
|
||||
{
|
||||
caughtError(e);
|
||||
}
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
IPluginV2Ext* MultilevelProposeROIPluginCreator::deserializePlugin(
|
||||
char const* name, void const* data, size_t length) noexcept
|
||||
{
|
||||
try
|
||||
{
|
||||
return new MultilevelProposeROI(data, length);
|
||||
}
|
||||
catch (std::exception const& e)
|
||||
{
|
||||
caughtError(e);
|
||||
}
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
MultilevelProposeROI::MultilevelProposeROI(
|
||||
int32_t prenms_topk, int32_t keep_topk, float fg_threshold, float iou_threshold, const nvinfer1::Dims imageSize)
|
||||
: mPreNMSTopK(prenms_topk)
|
||||
, mKeepTopK(keep_topk)
|
||||
, mFGThreshold(fg_threshold)
|
||||
, mIOUThreshold(iou_threshold)
|
||||
, mImageSize(imageSize)
|
||||
{
|
||||
mBackgroundLabel = -1;
|
||||
PLUGIN_VALIDATE(mPreNMSTopK > 0);
|
||||
PLUGIN_VALIDATE(mPreNMSTopK <= 4096);
|
||||
PLUGIN_VALIDATE(mKeepTopK > 0);
|
||||
PLUGIN_VALIDATE(mIOUThreshold >= 0.0F);
|
||||
PLUGIN_VALIDATE(mFGThreshold >= 0.0F);
|
||||
PLUGIN_VALIDATE(mImageSize.nbDims == 3);
|
||||
PLUGIN_VALIDATE(mImageSize.d[0] > 0 && mImageSize.d[1] > 0 && mImageSize.d[2] > 0);
|
||||
|
||||
mParam.backgroundLabelId = -1;
|
||||
mParam.numClasses = 1;
|
||||
mParam.keepTopK = mKeepTopK;
|
||||
mParam.scoreThreshold = mFGThreshold;
|
||||
mParam.iouThreshold = mIOUThreshold;
|
||||
|
||||
mType = DataType::kFLOAT;
|
||||
|
||||
mFeatureCnt = TLTMaskRCNNConfig::MAX_LEVEL - TLTMaskRCNNConfig::MIN_LEVEL + 1;
|
||||
|
||||
generate_pyramid_anchors(mImageSize);
|
||||
}
|
||||
|
||||
int32_t MultilevelProposeROI::getNbOutputs() const noexcept
|
||||
{
|
||||
return 1;
|
||||
}
|
||||
|
||||
int32_t MultilevelProposeROI::initialize() noexcept
|
||||
{
|
||||
// Init the regWeight [1, 1, 1, 1]
|
||||
mRegWeightDevice = std::make_shared<CudaBind<float>>(4);
|
||||
std::vector<float> reg_weight(4, 1);
|
||||
PLUGIN_CUASSERT(cudaMemcpy(static_cast<void*>(mRegWeightDevice->mPtr), static_cast<void*>(reg_weight.data()),
|
||||
sizeof(float) * 4, cudaMemcpyHostToDevice));
|
||||
|
||||
// Init the mValidCnt of max batch size
|
||||
std::vector<int32_t> tempValidCnt(mMaxBatchSize, mPreNMSTopK);
|
||||
|
||||
mValidCnt = std::make_shared<CudaBind<int32_t>>(mMaxBatchSize);
|
||||
|
||||
PLUGIN_CUASSERT(cudaMemcpy(mValidCnt->mPtr, static_cast<void*>(tempValidCnt.data()),
|
||||
sizeof(int32_t) * mMaxBatchSize, cudaMemcpyHostToDevice));
|
||||
|
||||
// Init the anchors for batch size:
|
||||
for (int32_t i = 0; i < mFeatureCnt; i++)
|
||||
{
|
||||
int32_t i_anchors_cnt = mAnchorsCnt[i];
|
||||
auto i_anchors_host = mAnchorBoxesHost[i].data();
|
||||
auto i_anchors_device = std::make_shared<CudaBind<float>>(i_anchors_cnt * 4 * mMaxBatchSize);
|
||||
int32_t batch_offset = sizeof(float) * i_anchors_cnt * 4;
|
||||
uint8_t* device_ptr = static_cast<uint8_t*>(i_anchors_device->mPtr);
|
||||
for (int32_t i = 0; i < mMaxBatchSize; i++)
|
||||
{
|
||||
PLUGIN_CUASSERT(cudaMemcpy(static_cast<void*>(device_ptr + i * batch_offset),
|
||||
static_cast<void*>(i_anchors_host), batch_offset, cudaMemcpyHostToDevice));
|
||||
}
|
||||
mAnchorBoxesDevice.push_back(i_anchors_device);
|
||||
}
|
||||
|
||||
// Init the temp storage for proposals from feature maps before concat
|
||||
std::vector<void*> score_tp;
|
||||
std::vector<void*> box_tp;
|
||||
for (int32_t i = 0; i < mFeatureCnt; i++)
|
||||
{
|
||||
if (mType == DataType::kFLOAT)
|
||||
{
|
||||
auto i_scores_device = std::make_shared<CudaBind<float>>(mKeepTopK * mMaxBatchSize);
|
||||
auto i_bboxes_device = std::make_shared<CudaBind<float>>(mKeepTopK * 4 * mMaxBatchSize);
|
||||
mTempScores_float.push_back(i_scores_device);
|
||||
score_tp.push_back(static_cast<void*>(i_scores_device->mPtr));
|
||||
mTempBboxes_float.push_back(i_bboxes_device);
|
||||
box_tp.push_back(static_cast<void*>(i_bboxes_device->mPtr));
|
||||
}
|
||||
else if (mType == DataType::kHALF)
|
||||
{
|
||||
auto i_scores_device = std::make_shared<CudaBind<uint16_t>>(mKeepTopK * mMaxBatchSize);
|
||||
auto i_bboxes_device = std::make_shared<CudaBind<uint16_t>>(mKeepTopK * 4 * mMaxBatchSize);
|
||||
mTempScores_half.push_back(i_scores_device);
|
||||
score_tp.push_back(static_cast<void*>(i_scores_device->mPtr));
|
||||
mTempBboxes_half.push_back(i_bboxes_device);
|
||||
box_tp.push_back(static_cast<void*>(i_bboxes_device->mPtr));
|
||||
}
|
||||
}
|
||||
|
||||
// Init the temp storage for pointer arrays of score and box:
|
||||
PLUGIN_CUASSERT(cudaMalloc(&mDeviceScores, sizeof(void*) * mFeatureCnt));
|
||||
PLUGIN_CUASSERT(cudaMalloc(&mDeviceBboxes, sizeof(void*) * mFeatureCnt));
|
||||
|
||||
PLUGIN_CUASSERT(cudaMemcpy(mDeviceScores, score_tp.data(), sizeof(void*) * mFeatureCnt, cudaMemcpyHostToDevice));
|
||||
PLUGIN_CUASSERT(cudaMemcpy(mDeviceBboxes, box_tp.data(), sizeof(void*) * mFeatureCnt, cudaMemcpyHostToDevice));
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
void MultilevelProposeROI::terminate() noexcept {}
|
||||
|
||||
void MultilevelProposeROI::destroy() noexcept
|
||||
{
|
||||
delete this;
|
||||
}
|
||||
|
||||
bool MultilevelProposeROI::supportsFormat(DataType type, PluginFormat format) const noexcept
|
||||
{
|
||||
return ((type == DataType::kFLOAT || type == DataType::kHALF) && format == PluginFormat::kLINEAR);
|
||||
}
|
||||
|
||||
char const* MultilevelProposeROI::getPluginType() const noexcept
|
||||
{
|
||||
return "MultilevelProposeROI_TRT";
|
||||
}
|
||||
|
||||
char const* MultilevelProposeROI::getPluginVersion() const noexcept
|
||||
{
|
||||
return "1";
|
||||
}
|
||||
|
||||
IPluginV2Ext* MultilevelProposeROI::clone() const noexcept
|
||||
{
|
||||
try
|
||||
{
|
||||
return new MultilevelProposeROI(*this);
|
||||
}
|
||||
catch (std::exception const& e)
|
||||
{
|
||||
caughtError(e);
|
||||
}
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
void MultilevelProposeROI::setPluginNamespace(char const* libNamespace) noexcept
|
||||
{
|
||||
mNameSpace = libNamespace;
|
||||
}
|
||||
|
||||
char const* MultilevelProposeROI::getPluginNamespace() const noexcept
|
||||
{
|
||||
return mNameSpace.c_str();
|
||||
}
|
||||
|
||||
size_t MultilevelProposeROI::getSerializationSize() const noexcept
|
||||
{
|
||||
return sizeof(int32_t) * 2 + sizeof(float) * 2 + sizeof(int32_t) * (mFeatureCnt + 1) + sizeof(nvinfer1::Dims)
|
||||
+ sizeof(DataType);
|
||||
}
|
||||
|
||||
void MultilevelProposeROI::serialize(void* buffer) const noexcept
|
||||
{
|
||||
char *d = reinterpret_cast<char*>(buffer), *a = d;
|
||||
write(d, mPreNMSTopK);
|
||||
write(d, mKeepTopK);
|
||||
write(d, mFGThreshold);
|
||||
write(d, mIOUThreshold);
|
||||
write(d, mMaxBatchSize);
|
||||
for (int32_t i = 0; i < mFeatureCnt; i++)
|
||||
{
|
||||
write(d, mAnchorsCnt[i]);
|
||||
}
|
||||
write(d, mImageSize);
|
||||
write(d, mType);
|
||||
PLUGIN_ASSERT(d == a + getSerializationSize());
|
||||
}
|
||||
|
||||
MultilevelProposeROI::MultilevelProposeROI(void const* data, size_t length)
|
||||
{
|
||||
mFeatureCnt = TLTMaskRCNNConfig::MAX_LEVEL - TLTMaskRCNNConfig::MIN_LEVEL + 1;
|
||||
|
||||
char const *d = reinterpret_cast<char const*>(data), *a = d;
|
||||
int32_t prenms_topk = read<int32_t>(d);
|
||||
int32_t keep_topk = read<int32_t>(d);
|
||||
float fg_threshold = read<float>(d);
|
||||
float iou_threshold = read<float>(d);
|
||||
mMaxBatchSize = read<int32_t>(d);
|
||||
PLUGIN_VALIDATE(mAnchorsCnt.size() == 0);
|
||||
for (int32_t i = 0; i < mFeatureCnt; i++)
|
||||
{
|
||||
mAnchorsCnt.push_back(read<int32_t>(d));
|
||||
}
|
||||
mImageSize = read<nvinfer1::Dims3>(d);
|
||||
mType = read<DataType>(d);
|
||||
PLUGIN_VALIDATE(d == a + length);
|
||||
|
||||
mBackgroundLabel = -1;
|
||||
mPreNMSTopK = prenms_topk;
|
||||
mKeepTopK = keep_topk;
|
||||
mFGThreshold = fg_threshold;
|
||||
mIOUThreshold = iou_threshold;
|
||||
|
||||
mParam.backgroundLabelId = -1;
|
||||
mParam.numClasses = 1;
|
||||
mParam.keepTopK = mKeepTopK;
|
||||
mParam.scoreThreshold = mFGThreshold;
|
||||
mParam.iouThreshold = mIOUThreshold;
|
||||
|
||||
generate_pyramid_anchors(mImageSize);
|
||||
}
|
||||
|
||||
void MultilevelProposeROI::check_valid_inputs(nvinfer1::Dims const* inputs, int32_t nbInputDims) noexcept
|
||||
{
|
||||
// x=2,3,4,5,6
|
||||
// foreground_delta_px [N, h_x * w_x * anchors_per_location, 4, 1],
|
||||
// foreground_score_px [N, h_x * w_x * anchors_per_location, 1, 1],
|
||||
// anchors should be generated inside
|
||||
PLUGIN_ASSERT(nbInputDims == 2 * mFeatureCnt);
|
||||
for (int32_t i = 0; i < 2 * mFeatureCnt; i += 2)
|
||||
{
|
||||
// foreground_delta
|
||||
PLUGIN_ASSERT(inputs[i].nbDims == 3 && inputs[i].d[1] == 4);
|
||||
// foreground_score
|
||||
PLUGIN_ASSERT(inputs[i + 1].nbDims == 3 && inputs[i + 1].d[1] == 1);
|
||||
}
|
||||
}
|
||||
|
||||
size_t MultilevelProposeROI::getWorkspaceSize(int32_t batch_size) const noexcept
|
||||
{
|
||||
size_t total_size = 0;
|
||||
PLUGIN_ASSERT(mAnchorsCnt.size() == static_cast<size_t>(mFeatureCnt));
|
||||
|
||||
// workspace for propose on each feature map
|
||||
for (int32_t i = 0; i < mFeatureCnt; i++)
|
||||
{
|
||||
|
||||
MultilevelProposeROIWorkSpace proposal(batch_size, mAnchorsCnt[i], mPreNMSTopK, mParam, mType);
|
||||
total_size += proposal.totalSize;
|
||||
}
|
||||
|
||||
// workspace for Concat and TopK
|
||||
ConcatTopKWorkSpace ct(batch_size, mFeatureCnt, mKeepTopK, mType);
|
||||
total_size += ct.totalSize;
|
||||
|
||||
return total_size;
|
||||
}
|
||||
|
||||
Dims MultilevelProposeROI::getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept
|
||||
{
|
||||
|
||||
check_valid_inputs(inputs, nbInputDims);
|
||||
PLUGIN_ASSERT(index == 0);
|
||||
|
||||
return {2, {mKeepTopK, 4}};
|
||||
}
|
||||
|
||||
void MultilevelProposeROI::generate_pyramid_anchors(nvinfer1::Dims const& imageSize)
|
||||
{
|
||||
auto const image_dims = imageSize;
|
||||
|
||||
auto const& anchor_scale = TLTMaskRCNNConfig::RPN_ANCHOR_SCALE;
|
||||
auto const& min_level = TLTMaskRCNNConfig::MIN_LEVEL;
|
||||
auto const& max_level = TLTMaskRCNNConfig::MAX_LEVEL;
|
||||
auto const& aspect_ratios = TLTMaskRCNNConfig::ANCHOR_RATIOS;
|
||||
|
||||
// Generate anchors strides and scales
|
||||
std::vector<float> anchor_scales;
|
||||
std::vector<int32_t> anchor_strides;
|
||||
for (int32_t i = min_level; i < max_level + 1; i++)
|
||||
{
|
||||
int32_t stride = static_cast<int32_t>(pow(2.0, i));
|
||||
anchor_strides.push_back(stride);
|
||||
anchor_scales.push_back(stride * anchor_scale);
|
||||
}
|
||||
|
||||
auto& anchors = mAnchorBoxesHost;
|
||||
PLUGIN_VALIDATE(anchors.size() == 0);
|
||||
|
||||
PLUGIN_VALIDATE(anchor_scales.size() == anchor_strides.size());
|
||||
for (size_t s = 0; s < anchor_scales.size(); ++s)
|
||||
{
|
||||
float scale = anchor_scales[s];
|
||||
int32_t stride = anchor_strides[s];
|
||||
|
||||
std::vector<float> s_anchors;
|
||||
for (int32_t y = stride / 2; y < image_dims.d[1]; y += stride)
|
||||
for (int32_t x = stride / 2; x < image_dims.d[2]; x += stride)
|
||||
for (auto r : aspect_ratios)
|
||||
{
|
||||
float h = scale * r.second;
|
||||
float w = scale * r.first;
|
||||
|
||||
// Using y+h/2 instead of y+h/2-1 for alignment with TLT implementation
|
||||
s_anchors.insert(s_anchors.end(), {(y - h / 2), (x - w / 2), (y + h / 2), (x + w / 2)});
|
||||
}
|
||||
|
||||
anchors.push_back(s_anchors);
|
||||
}
|
||||
|
||||
PLUGIN_VALIDATE(anchors.size() == static_cast<size_t>(max_level - min_level + 1));
|
||||
}
|
||||
|
||||
int32_t MultilevelProposeROI::enqueue(
|
||||
int32_t batch_size, void const* const* inputs, void* const* outputs, void* workspace, cudaStream_t stream) noexcept
|
||||
{
|
||||
|
||||
void* final_proposals = outputs[0];
|
||||
size_t kernel_workspace_offset = 0;
|
||||
cudaError_t status;
|
||||
|
||||
std::vector<void*> mTempScores;
|
||||
std::vector<void*> mTempBboxes;
|
||||
|
||||
for (int32_t i = 0; i < mFeatureCnt; i++)
|
||||
{
|
||||
if (mType == DataType::kFLOAT)
|
||||
{
|
||||
mTempScores.push_back(mTempScores_float[i]->mPtr);
|
||||
mTempBboxes.push_back(mTempBboxes_float[i]->mPtr);
|
||||
}
|
||||
else if (mType == DataType::kHALF)
|
||||
{
|
||||
mTempScores.push_back(mTempScores_half[i]->mPtr);
|
||||
mTempBboxes.push_back(mTempBboxes_half[i]->mPtr);
|
||||
}
|
||||
}
|
||||
|
||||
for (int32_t i = 0; i < mFeatureCnt; i++)
|
||||
{
|
||||
MultilevelProposeROIWorkSpace proposal_ws(batch_size, mAnchorsCnt[i], mPreNMSTopK, mParam, mType);
|
||||
status = MultilevelPropose(stream, batch_size, mAnchorsCnt[i], mPreNMSTopK,
|
||||
static_cast<float*>(mRegWeightDevice->mPtr),
|
||||
static_cast<float>(mImageSize.d[1]), // Input Height
|
||||
static_cast<float>(mImageSize.d[2]),
|
||||
mType, // mType,
|
||||
mParam, proposal_ws, static_cast<uint8_t*>(workspace) + kernel_workspace_offset,
|
||||
inputs[2 * i + 1], // inputs[object_score],
|
||||
inputs[2 * i], // inputs[bbox_delta]
|
||||
mValidCnt->mPtr,
|
||||
mAnchorBoxesDevice[i]->mPtr, // inputs[anchors]
|
||||
mTempScores[i], // temp scores [batch_size, topk, 1]
|
||||
mTempBboxes[i]); // temp
|
||||
PLUGIN_ASSERT(status == cudaSuccess);
|
||||
kernel_workspace_offset += proposal_ws.totalSize;
|
||||
}
|
||||
|
||||
ConcatTopKWorkSpace ctopk_ws(batch_size, mFeatureCnt, mKeepTopK, mType);
|
||||
status = ConcatTopK(stream, batch_size, mFeatureCnt, mKeepTopK, mType,
|
||||
static_cast<uint8_t*>(workspace) + kernel_workspace_offset, ctopk_ws, reinterpret_cast<void**>(mDeviceScores),
|
||||
reinterpret_cast<void**>(mDeviceBboxes), final_proposals);
|
||||
|
||||
PLUGIN_ASSERT(status == cudaSuccess);
|
||||
return status;
|
||||
}
|
||||
|
||||
// Return the DataType of the plugin output at the requested index
|
||||
DataType MultilevelProposeROI::getOutputDataType(
|
||||
int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept
|
||||
{
|
||||
// Only DataType::kFLOAT is acceptable by the plugin layer
|
||||
if ((inputTypes[0] == DataType::kFLOAT) || (inputTypes[0] == DataType::kHALF))
|
||||
return inputTypes[0];
|
||||
return DataType::kFLOAT;
|
||||
}
|
||||
|
||||
// Configure the layer with input and output data types.
|
||||
void MultilevelProposeROI::configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims,
|
||||
int32_t nbOutputs, DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast,
|
||||
bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept
|
||||
{
|
||||
check_valid_inputs(inputDims, nbInputs);
|
||||
|
||||
mAnchorsCnt.clear();
|
||||
for (int32_t i = 0; i < mFeatureCnt; i++)
|
||||
{
|
||||
mAnchorsCnt.push_back(inputDims[2 * i].d[0]);
|
||||
PLUGIN_ASSERT(mAnchorsCnt[i] == (int32_t) (mAnchorBoxesHost[i].size() / 4));
|
||||
}
|
||||
|
||||
mMaxBatchSize = maxBatchSize;
|
||||
|
||||
mType = inputTypes[0];
|
||||
}
|
||||
|
||||
// Attach the plugin object to an execution context and grant the plugin the access to some context resource.
|
||||
void MultilevelProposeROI::attachToContext(
|
||||
cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept
|
||||
{
|
||||
}
|
||||
|
||||
// Detach the plugin object from its execution context.
|
||||
void MultilevelProposeROI::detachFromContext() noexcept {}
|
||||
@@ -0,0 +1,147 @@
|
||||
/*
|
||||
* SPDX-FileCopyrightText: Copyright (c) 1993-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
#ifndef TRT_MULTILEVEL_PROPOSE_ROI_PLUGIN_H
|
||||
#define TRT_MULTILEVEL_PROPOSE_ROI_PLUGIN_H
|
||||
#include <cuda_runtime_api.h>
|
||||
#include <memory>
|
||||
#include <string.h>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "NvInfer.h"
|
||||
#include "NvInferPlugin.h"
|
||||
#include "common/kernels/maskRCNNKernels.h"
|
||||
|
||||
namespace nvinfer1
|
||||
{
|
||||
namespace plugin
|
||||
{
|
||||
|
||||
class MultilevelProposeROI : public IPluginV2Ext
|
||||
{
|
||||
public:
|
||||
MultilevelProposeROI(int32_t prenms_topk, int32_t keep_topk, float fg_threshold, float iou_threshold,
|
||||
const nvinfer1::Dims image_size);
|
||||
|
||||
MultilevelProposeROI(void const* data, size_t length);
|
||||
|
||||
~MultilevelProposeROI() noexcept override = default;
|
||||
|
||||
int32_t getNbOutputs() const noexcept override;
|
||||
|
||||
Dims getOutputDimensions(int32_t index, Dims const* inputs, int32_t nbInputDims) noexcept override;
|
||||
|
||||
int32_t initialize() noexcept override;
|
||||
|
||||
void terminate() noexcept override;
|
||||
|
||||
void destroy() noexcept override;
|
||||
|
||||
size_t getWorkspaceSize(int32_t maxBatchSize) const noexcept override;
|
||||
|
||||
int32_t enqueue(int32_t batch_size, void const* const* inputs, void* const* outputs, void* workspace,
|
||||
cudaStream_t stream) noexcept override;
|
||||
|
||||
size_t getSerializationSize() const noexcept override;
|
||||
|
||||
void serialize(void* buffer) const noexcept override;
|
||||
|
||||
bool supportsFormat(DataType type, PluginFormat format) const noexcept override;
|
||||
|
||||
char const* getPluginType() const noexcept override;
|
||||
|
||||
char const* getPluginVersion() const noexcept override;
|
||||
|
||||
IPluginV2Ext* clone() const noexcept override;
|
||||
|
||||
void setPluginNamespace(char const* libNamespace) noexcept override;
|
||||
|
||||
char const* getPluginNamespace() const noexcept override;
|
||||
|
||||
DataType getOutputDataType(
|
||||
int32_t index, nvinfer1::DataType const* inputTypes, int32_t nbInputs) const noexcept override;
|
||||
|
||||
void attachToContext(
|
||||
cudnnContext* cudnnContext, cublasContext* cublasContext, IGpuAllocator* gpuAllocator) noexcept override;
|
||||
|
||||
void configurePlugin(Dims const* inputDims, int32_t nbInputs, Dims const* outputDims, int32_t nbOutputs,
|
||||
DataType const* inputTypes, DataType const* outputTypes, bool const* inputIsBroadcast,
|
||||
bool const* outputIsBroadcast, PluginFormat floatFormat, int32_t maxBatchSize) noexcept override;
|
||||
|
||||
void detachFromContext() noexcept override;
|
||||
|
||||
private:
|
||||
void check_valid_inputs(nvinfer1::Dims const* inputs, int32_t nbInputDims) noexcept;
|
||||
void generate_pyramid_anchors(nvinfer1::Dims const& imageSize);
|
||||
|
||||
int32_t mBackgroundLabel;
|
||||
int32_t mPreNMSTopK;
|
||||
int32_t mKeepTopK;
|
||||
int32_t mFeatureCnt;
|
||||
float mFGThreshold;
|
||||
float mIOUThreshold;
|
||||
|
||||
int32_t mMaxBatchSize;
|
||||
std::vector<int32_t> mAnchorsCnt;
|
||||
std::shared_ptr<CudaBind<int32_t>> mValidCnt; // valid cnt = number of input roi for every image.
|
||||
std::vector<std::shared_ptr<CudaBind<float>>>
|
||||
mAnchorBoxesDevice; // [N, anchors(261888 for resnet101 + 1024*1024), (y1, x1, y2, x2)]
|
||||
std::vector<std::vector<float>> mAnchorBoxesHost;
|
||||
std::vector<std::shared_ptr<CudaBind<float>>> mTempScores_float;
|
||||
std::vector<std::shared_ptr<CudaBind<float>>> mTempBboxes_float;
|
||||
std::vector<std::shared_ptr<CudaBind<uint16_t>>> mTempScores_half;
|
||||
std::vector<std::shared_ptr<CudaBind<uint16_t>>> mTempBboxes_half;
|
||||
float** mDeviceScores;
|
||||
float** mDeviceBboxes;
|
||||
std::shared_ptr<CudaBind<float>> mRegWeightDevice;
|
||||
|
||||
nvinfer1::Dims mImageSize;
|
||||
nvinfer1::DataType mType;
|
||||
RefineNMSParameters mParam;
|
||||
|
||||
std::string mNameSpace;
|
||||
};
|
||||
|
||||
class MultilevelProposeROIPluginCreator : public nvinfer1::pluginInternal::BaseCreator
|
||||
{
|
||||
public:
|
||||
MultilevelProposeROIPluginCreator() noexcept;
|
||||
|
||||
~MultilevelProposeROIPluginCreator() noexcept override {}
|
||||
|
||||
char const* getPluginName() const noexcept override;
|
||||
|
||||
char const* getPluginVersion() const noexcept override;
|
||||
|
||||
PluginFieldCollection const* getFieldNames() noexcept override;
|
||||
|
||||
IPluginV2Ext* createPlugin(char const* name, PluginFieldCollection const* fc) noexcept override;
|
||||
|
||||
IPluginV2Ext* deserializePlugin(char const* name, void const* data, size_t length) noexcept override;
|
||||
|
||||
private:
|
||||
PluginFieldCollection mFC;
|
||||
int32_t mPreNMSTopK;
|
||||
int32_t mKeepTopK;
|
||||
float mFGThreshold;
|
||||
float mIOUThreshold;
|
||||
std::vector<PluginField> mPluginAttributes;
|
||||
};
|
||||
} // namespace plugin
|
||||
} // namespace nvinfer1
|
||||
#endif // TRT_MULTILEVEL_PROPOSE_ROI_PLUGIN_H
|
||||
@@ -0,0 +1,177 @@
|
||||
/*
|
||||
* SPDX-FileCopyrightText: Copyright (c) 1993-2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
|
||||
* SPDX-License-Identifier: Apache-2.0
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
#ifndef MASKRCNN_CONFIG_HEADER
|
||||
#define MASKRCNN_CONFIG_HEADER
|
||||
#include "NvInfer.h"
|
||||
#include <string>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
namespace TLTMaskRCNNConfig
|
||||
{
|
||||
static const nvinfer1::Dims3 IMAGE_SHAPE{3, 832, 1344};
|
||||
|
||||
// Pooled ROIs
|
||||
static int32_t const POOL_SIZE = 7;
|
||||
static int32_t const MASK_POOL_SIZE = 14;
|
||||
|
||||
// Threshold to determine the mask area out of final convolution output
|
||||
static float const MASK_THRESHOLD = 0.5;
|
||||
|
||||
// Bounding box refinement standard deviation for RPN and final detections.
|
||||
static float const DETECTION_REG_WEIGHTS[] = {10, 10, 5, 5};
|
||||
|
||||
// Max number of final detections
|
||||
static int32_t const DETECTION_MAX_INSTANCES = 100;
|
||||
|
||||
// Minimum probability value to accept a detected instance
|
||||
// ROIs below this threshold are skipped
|
||||
static float const DETECTION_MIN_CONFIDENCE = 0;
|
||||
|
||||
// Non-maximum suppression threshold for detection
|
||||
static float const DETECTION_NMS_THRESHOLD = 0.5;
|
||||
|
||||
// Size of the fully-connected layers in the classification graph
|
||||
static int32_t const FPN_CLASSIF_FC_LAYERS_SIZE = 1024;
|
||||
|
||||
// Size of the top-down layers used to build the feature pyramid
|
||||
static int32_t const TOP_DOWN_PYRAMID_SIZE = 256;
|
||||
|
||||
// Number of classification classes (including background)
|
||||
static int32_t const NUM_CLASSES = 1 + 90;
|
||||
|
||||
// Min and max level of fpn feature pyramids:
|
||||
// p2, p3, p4, p5, p6.
|
||||
static int32_t const MIN_LEVEL = 2;
|
||||
static int32_t const MAX_LEVEL = 6;
|
||||
|
||||
// Length of minimum square anchor side in pixels
|
||||
static float const RPN_ANCHOR_SCALE = 8;
|
||||
|
||||
// Ratios of anchors at each cell (width,height)
|
||||
static const std::vector<std::pair<float, float>> ANCHOR_RATIOS
|
||||
= {std::make_pair(1.0F, 1.0F), std::make_pair(1.4F, 0.7F), std::make_pair(0.7F, 1.4F)};
|
||||
|
||||
// Anchor stride
|
||||
// If 1 then anchors are created for each cell in the backbone feature map.
|
||||
// If 2, then anchors are created for every other cell, and so on.
|
||||
static int32_t const RPN_ANCHOR_STRIDE = 1;
|
||||
|
||||
// TRT fails if this number larger than kMAX_TOPK_K defined in engine/checkMacros.h
|
||||
static int32_t const MAX_PRE_NMS_RESULTS = 1000; // 3840;
|
||||
|
||||
// Non-max suppression threshold to filter RPN proposals.
|
||||
// You can increase this during training to generate more propsals.
|
||||
static float const RPN_NMS_THRESHOLD = 0.7F;
|
||||
|
||||
// ROIs kept after non-maximum suppression (training and inference)
|
||||
static int32_t const POST_NMS_ROIS_INFERENCE = 1000;
|
||||
|
||||
// COCO Class names
|
||||
static const std::vector<std::string> CLASS_NAMES = {
|
||||
"BG",
|
||||
"person",
|
||||
"bicycle",
|
||||
"car",
|
||||
"motorcycle",
|
||||
"airplane",
|
||||
"bus",
|
||||
"train",
|
||||
"truck",
|
||||
"boat",
|
||||
"traffic light",
|
||||
"fire hydrant",
|
||||
"stop sign",
|
||||
"parking meter",
|
||||
"bench",
|
||||
"bird",
|
||||
"cat",
|
||||
"dog",
|
||||
"horse",
|
||||
"sheep",
|
||||
"cow",
|
||||
"elephant",
|
||||
"bear",
|
||||
"zebra",
|
||||
"giraffe",
|
||||
"backpack",
|
||||
"umbrella",
|
||||
"handbag",
|
||||
"tie",
|
||||
"suitcase",
|
||||
"frisbee",
|
||||
"skis",
|
||||
"snowboard",
|
||||
"sports ball",
|
||||
"kite",
|
||||
"baseball bat",
|
||||
"baseball glove",
|
||||
"skateboard",
|
||||
"surfboard",
|
||||
"tennis racket",
|
||||
"bottle",
|
||||
"wine glass",
|
||||
"cup",
|
||||
"fork",
|
||||
"knife",
|
||||
"spoon",
|
||||
"bowl",
|
||||
"banana",
|
||||
"apple",
|
||||
"sandwich",
|
||||
"orange",
|
||||
"broccoli",
|
||||
"carrot",
|
||||
"hot dog",
|
||||
"pizza",
|
||||
"donut",
|
||||
"cake",
|
||||
"chair",
|
||||
"couch",
|
||||
"potted plant",
|
||||
"bed",
|
||||
"dining table",
|
||||
"toilet",
|
||||
"tv",
|
||||
"laptop",
|
||||
"mouse",
|
||||
"remote",
|
||||
"keyboard",
|
||||
"cell phone",
|
||||
"microwave",
|
||||
"oven",
|
||||
"toaster",
|
||||
"sink",
|
||||
"refrigerator",
|
||||
"book",
|
||||
"clock",
|
||||
"vase",
|
||||
"scissors",
|
||||
"teddy bear",
|
||||
"hair drier",
|
||||
"toothbrush",
|
||||
};
|
||||
|
||||
static const std::string MODEL_NAME = "mrcnn_nchw.uff";
|
||||
static const std::string MODEL_INPUT = "Input";
|
||||
static const nvinfer1::Dims3 MODEL_INPUT_SHAPE = IMAGE_SHAPE;
|
||||
static const std::vector<std::string> MODEL_OUTPUTS = {"generate_detections", "mask_head/mask_fcn_logits/BiasAdd"};
|
||||
static const nvinfer1::Dims2 MODEL_DETECTION_SHAPE{DETECTION_MAX_INSTANCES, 6};
|
||||
static const nvinfer1::Dims4 MODEL_MASK_SHAPE{DETECTION_MAX_INSTANCES, NUM_CLASSES, 28, 28};
|
||||
} // namespace TLTMaskRCNNConfig
|
||||
#endif
|
||||
Reference in New Issue
Block a user