# multiscaleDeformableAttn Plugin

**Table Of Contents**
- [Description](#description)
    * [Structure](#structure)
- [Parameters](#parameters)
- [Additional resources](#additional-resources)
- [License](#license)
- [Changelog](#changelog)
- [Known issues](#known-issues)

## Description

> NOTE: Version 1 of this plugin (using IPluginV2DynamicExt interface) is deprecated since TensorRT 10.11. Version 2 (using IPluginV3 interface) is the recommended replacement.

The `multiscaleDeformableAttnPlugin` is used to perform attention computation over a small set of key sampling points around a reference point rather than looking over all possible spatial locations. It makes use of multiscale feature maps to effectively represent objects at different scales. It helps to achieve faster convergence and better performance on small objects.

### Structure

The `multiscaleDeformableAttnPlugin` takes 5 inputs in the following order :  `value`, `spatial_shapes`, `level_start_index`, `sampling_locations`, and `atttention_weights`.

`value`
The input feature maps from different scales concatenated to provide the input feature vector. The shape of this tensor is `[N, S, M, D]` where `N` is batch size, `S` is the length of the feature maps, `M` is the number of attentions heads, `D` is hidden_dim/num_heads.

`spatial_shapes`
The shape of each feature map. The shape of this tensor is `[L, 2]` where `L` is the number of feature maps.

`level_start_index`
This tensor is used to find the sampling locations for different feature levels as the input feature tensors are flattened. The shape of this tensor is `[L,]`.

`sampling_locations`
This tensor acts as a pre-filter for prominent key elements out of all the feature map pixels. The shape of this tensor is `[N, Lq, M, L, P, 2]` where `P` is the number of points, `Lq` is the length of feature maps(encoder)/length of queries(decoder).

`attention_weights`
This tensor consists of the attention weights whose shape is `[N, Lq, M, L, P]`.

The `multiscaleDeformableAttnPlugin` generates the attention output of shape `[N, Lq, M, D]`.

## Parameters

`multiscaleDeformableAttnPlugin` has plugin creator class `multiscaleDeformableAttnPluginCreator` and plugin class `multiscaleDeformableAttnPlugin`.

The plugin does not require any parameters to be built and used.


## Additional resources

The following resources provide a deeper understanding of the `multiscaleDeformableAttnPlugin` plugin:

**Networks:**
- [Deformable DETR](https://arxiv.org/pdf/2010.04159.pdf)

## License

For terms and conditions for use, reproduction, and distribution, see the [TensorRT Software License Agreement](https://docs.nvidia.com/deeplearning/sdk/tensorrt-sla/index.html)
documentation.

## Changelog

Apr 2025
Added version 2 of the plugin that uses the IPluginV3 interface. The version 1 (using IPluginV2DynamicExt interface) is now deprecated. The version 2 mirrors version 1 in IO and attributes.

Feb 2022
This is the first release of this `README.md` file.


## Known issues

There are no known issues in this plugin.