wehub-resource-sync
68 lines
3.0 KiB
Markdown
68 lines
3.0 KiB
Markdown
## About
|
|
This script presents a QAT end-to-end workflow (TF2-to-ONNX) for [ResNet models](https://keras.io/api/applications/resnet/) in `tf.keras.applications`.
|
|
|
|
### Contents
|
|
[Requirements](#requirements) • [Workflow](#workflow) • [Results](#results)
|
|
|
|
## Requirements
|
|
Install base requirements and prepare data. Please refer to [examples' README](../README.md).
|
|
|
|
## Workflow
|
|
|
|
### Step 1: Model Quantization and Fine-tuning
|
|
|
|
Please run the following to quantize, fine-tune, and save the final graph in SavedModel format (checkpoints are also saved).
|
|
|
|
```sh
|
|
python run_qat_workflow.py
|
|
```
|
|
|
|
### Step 2: Conversion to ONNX
|
|
Step 1 already does the conversion from SavedModel to ONNX automatically. For manual steps, please see step 3 in [EfficientNet's README](../efficientnet_b0/README.md).
|
|
|
|
### Step 3: TensorRT Deployment
|
|
Please refer to the [examples' README](../README.md).
|
|
|
|
## Results
|
|
Results obtained on NVIDIA's A100 GPU and TensorRT 8.4 EA.
|
|
|
|
### ResNet50-v1
|
|
|
|
| Model | TF (%) | TF latency (ms, bs=1) | TRT(%) | TRT latency (ms, bs=1) |
|
|
|----------|-------------|-----------------------|--------|------------------------|
|
|
| Baseline | 75.05 | 7.95 | 75.05 | 1.96 |
|
|
| PTQ | - | - | 74.96 | 0.46 |
|
|
| **QAT** | 75.11 (ep5) | - | 75.12 | 0.45 |
|
|
|
|
### ResNet50-v2
|
|
|
|
| Model | TF (%) | TF latency (ms, bs=1) | TRT(%) | TRT latency (ms, bs=1) |
|
|
|----------|--------------|-----------------------|---------|------------------------|
|
|
| Baseline | 75.36 | 6.16 | 75.37 | 2.35 |
|
|
| PTQ | - | - | 75.48 | 0.57 |
|
|
| **QAT** | 75.59 (ep5) | - | 75.65 | 0.57 |
|
|
|
|
### ResNet101-v1
|
|
|
|
| Model | TF (%) | TF latency (ms, bs=1) | TRT(%) | TRT latency (ms, bs=1) |
|
|
|----------|--------------|-----------------------|--------|------------------------|
|
|
| Baseline | 76.47 | 15.92 | 76.48 | 3.84 |
|
|
| PTQ | - | - | 76.32 | 0.84 |
|
|
| **QAT** | 76.33 (ep30) | - | 76.26 | 0.84 |
|
|
|
|
### ResNet101-v2
|
|
|
|
| Model | TF (%) | TF latency (ms, bs=1) | TRT(%) | TRT latency (ms, bs=1) |
|
|
|----------|--------|-----------------------|--------|------------------------|
|
|
| Baseline | 76.89 | 14.13 | 76.88 | 4.55 |
|
|
| PTQ | - | - | 76.94 | 1.05 |
|
|
| **QAT** | 77.20 | - | 77.15 | 1.05 |
|
|
> QAT fine-tuning hyper-parameters for ResNet101-v2: `bs=32` (`bs=64` was OOM).
|
|
|
|
### Notes
|
|
- QAT fine-tuning hyper-parameters:
|
|
- Optimizer: `piecewise_sgd`, `lr_schedule=[(1.0,1),(0.1,2),(0.01,7)]` (default)
|
|
- Hyper-parameters: `bs=64, ep=10, lr=0.001`.
|
|
- Added QDQ nodes in Residual connection.
|
|
- PTQ calibration: `bs=64`.
|