Skip to content

Commit b1679f9

Browse files
songtianhuisongtianhui
authored
Provide checkpoints for MixFormer. (#778)
* add MixFormer in README * add checkpoints * add mixformer in docs * fix metafile * update test results * fix mixformer test pipeline * update model-index.yml
1 parent 87d55ec commit b1679f9

File tree

10 files changed

+43
-27
lines changed

10 files changed

+43
-27
lines changed

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,7 @@ Supported Methods
112112

113113
- [x] [SiameseRPN++](configs/sot/siamese_rpn) (CVPR 2019)
114114
- [x] [STARK](configs/sot/stark) (ICCV 2021)
115+
- [x] [MixFormer](configs/sot/mixformer) (CVPR 2022)
115116
- [ ] [PrDiMP](https://arxiv.org/abs/2003.12565) (CVPR2020) (WIP)
116117

117118
Supported Datasets

‎README_zh-CN.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -108,6 +108,7 @@ MMTracking也提供了更详细的[教程](docs/zh_cn/tutorials/)，比如[配
108108

109109
- [x] [SiameseRPN++](configs/sot/siamese_rpn) (CVPR 2019)
110110
- [x] [STARK](configs/sot/stark) (ICCV 2021)
111+
- [x] [MixFormer](configs/sot/mixformer) (CVPR 2022)
111112
- [ ] [PrDiMP](https://arxiv.org/abs/2003.12565) (CVPR2020) (WIP)
112113

113114
支持的数据集：

‎configs/sot/mixformer/README.md‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -30,18 +30,18 @@ We now provide the evaluation results using model weights released in [official
3030

3131
### Lasot
3232

33-
| Method | Style | Inf time(fps) | Success | Norm precision | Precision | Config |
34-
| :-------: | :---: | :-----------: | :-----: | :------------: | :-------: | :-------------------------------------: |
35-
| MixFormer | - | - | 69.2 | 78.8 | 74.3 | [config](./mixformer_cvt_500e_lasot.py) |
33+
| Method | Style | Inf time(fps) | Success | Norm precision | Precision | Config | Download |
34+
| :-------: | :---: | :-----------: | :-----: | :------------: | :-------: | :-------------------------------------: | :--------------------------------------------------------------------------------------------------------------------: |
35+
| MixFormer | - | - | 69.0 | 79.6 | 75.2 | [config](./mixformer_cvt_500e_lasot.py) | [model](https://download.openmmlab.com/mmtracking/sot/mixformer/mixformer_cvt_500e_lasot/mixformer_cvt_500e_lasot.pth) |
3636

3737
### TrackingNet
3838

39-
| Method | Style | Inf time(fps) | Success | Norm precision | Precision | Config |
40-
| :-------: | :---: | :-----------: | :-----: | :------------: | :-------: | :-------------------------------------------: |
41-
| MixFormer | - | - | 81.1 | 86.6 | 80.0 | [config](./mixformer_cvt_500e_trackingnet.py) |
39+
| Method | Style | Inf time(fps) | Success | Norm precision | Precision | Config | Download |
40+
| :-------: | :---: | :-----------: | :-----: | :------------: | :-------: | :-------------------------------------------: | :--------------------------------------------------------------------------------------------------------------------: |
41+
| MixFormer | - | - | 81.4 | 86.8 | 80.3 | [config](./mixformer_cvt_500e_trackingnet.py) | [model](https://download.openmmlab.com/mmtracking/sot/mixformer/mixformer_cvt_500e_lasot/mixformer_cvt_500e_lasot.pth) |
4242

4343
### GOT10k
4444

45-
| Method | Style | Inf time(fps) | Average Overlap | Success Rate 0.5 | Success Rate 0.75 | Config |
46-
| :-------: | :---: | :-----------: | :-------------: | :--------------: | :---------------: | :--------------------------------------: |
47-
| MixFormer | - | - | 72.0 | 82.0 | 68.1 | [config](./mixformer_cvt_500e_got10k.py) |
45+
| Method | Style | Inf time(fps) | Average Overlap | Success Rate 0.5 | Success Rate 0.75 | Config | Download |
46+
| :-------: | :---: | :-----------: | :-------------: | :--------------: | :---------------: | :--------------------------------------: | :----------------------------------------------------------------------------------------------------------------------: |
47+
| MixFormer | - | - | 70.1 | 80.1 | 65.6 | [config](./mixformer_cvt_500e_got10k.py) | [model](https://download.openmmlab.com/mmtracking/sot/mixformer/mixformer_cvt_500e_got10k/mixformer_cvt_500e_got10k.pth) |

‎configs/sot/mixformer/metafile.yml‎

Lines changed: 18 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,12 @@
11
Collections:
22
- Name: MixFormer
33
Metadata:
4-
Architecture:
5-
- TransFormer
4+
Training Data: GOT10k, LaSOT, TrackingNet, MSCOCO
5+
Training Techniques:
6+
- AdamW
7+
Training Resources: 8x V100 GPUs
8+
Architecture:
9+
- Convolutional Vision TransFormer
610
Paper:
711
URL: https://arxiv.org/abs/2203.11082
812
Title: "MixFormer: End-to-End Tracking with Iterative Mixed Attention"
@@ -16,9 +20,10 @@ Models:
1620
- Task: Single Object Tracking
1721
Dataset: GOT10k
1822
Metrics:
19-
AO: 72.0
20-
SR0.5: 82.0
21-
SR0.75: 68.1
23+
AO: 70.1
24+
SR0.5: 80.1
25+
SR0.75: 65.6
26+
Weights: https://download.openmmlab.com/mmtracking/sot/mixformer/mixformer_cvt_500e_got10k/mixformer_cvt_500e_got10k.pth
2227

2328
- Name: mixformer_cvt_500e_lasot
2429
In Collection: MixFormer
@@ -27,9 +32,10 @@ Models:
2732
- Task: Single Object Tracking
2833
Dataset: LaSOT
2934
Metrics:
30-
Success: 69.2
31-
Norm Precision: 78.8
32-
Precision: 74.3
35+
Success: 69.0
36+
Norm Precision: 79.6
37+
Precision: 75.2
38+
Weights: https://download.openmmlab.com/mmtracking/sot/mixformer/mixformer_cvt_500e_lasot/mixformer_cvt_500e_lasot.pth
3339

3440
- Name: mixformer_cvt_500e_trackingnet
3541
In Collection: MixFormer
@@ -38,6 +44,7 @@ Models:
3844
- Task: Single Object Tracking
3945
Dataset: TrackingNet
4046
Metrics:
41-
Success: 81.1
42-
Norm Precision: 86.6
43-
Precision: 80.0
47+
Success: 81.4
48+
Norm Precision: 86.8
49+
Precision: 80.3
50+
Weights: https://download.openmmlab.com/mmtracking/sot/mixformer/mixformer_cvt_500e_lasot/mixformer_cvt_500e_lasot.pth

‎configs/sot/mixformer/mixformer_cvt_500e_got10k.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@
4747
search_size=320,
4848
template_factor=2.0,
4949
template_size=128,
50-
update_interval=[10],
50+
update_interval=[25],
5151
online_size=[2],
5252
max_score_decay=[0.98],
5353
))

‎configs/sot/mixformer/mixformer_cvt_500e_trackingnet.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
template_factor=2.0,
99
template_size=128,
1010
update_interval=[25],
11-
online_size=[2],
11+
online_size=[1],
1212
max_score_decay=[1.0],
1313
))
1414

‎docs/en/model_zoo.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,10 @@ Please refer to [SiameseRPN++](https://github.com/open-mmlab/mmtracking/blob/mas
7575

7676
Please refer to [STARK](https://github.com/open-mmlab/mmtracking/blob/master/configs/sot/stark) for details.
7777

78+
### MixFormer (CVPR 2022)
79+
80+
Please refer to [MixFormer](https://github.com/open-mmlab/mmtracking/blob/master/configs/sot/mixformer) for details.
81+
7882
## Baselines of video instance segmentation
7983

8084
### MaskTrack R-CNN (ICCV 2019)

‎docs/zh_cn/model_zoo.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,10 @@
7575

7676
详情请参考 [STARK](https://github.com/open-mmlab/mmtracking/blob/master/configs/sot/stark)。
7777

78+
### MixFormer (CVPR 2022)
79+
80+
详情请参考 [MixFormer](https://github.com/open-mmlab/mmtracking/blob/master/configs/sot/mixformer)。
81+
7882
## 视频个例分割基线
7983

8084
### MaskTrack R-CNN (ICCV 2019)

‎mmtrack/models/sot/mixformer.py‎

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -102,9 +102,8 @@ def update_template(self, img, bbox, conf_score):
102102
self.online_forget_id = (self.online_forget_id +
103103
1) % self.online_size
104104

105-
if self.online_size > 1:
106-
with torch.no_grad():
107-
self.set_online(self.template, self.online_template)
105+
with torch.no_grad():
106+
self.set_online(self.template, self.online_template)
108107

109108
self.best_conf_score = -1
110109
self.best_online_template = self.template
@@ -134,13 +133,12 @@ def track(self, img, bbox):
134133
out_dict = self.head(template, search)
135134

136135
pred_box = out_dict['pred_bboxes'] # xyxy
137-
# Baseline: Take the mean of all pred boxes as the final result
138-
139136
pred_box = self.mapping_bbox_back(pred_box, self.memo.bbox,
140137
resize_factor)
141138
pred_box = self._bbox_clip(pred_box, H, W, margin=10)
142139

143140
# update template
141+
self.best_conf_score = self.best_conf_score * self.max_score_decay
144142
conf_score = -1.
145143
if self.head.score_decoder_head is not None:
146144
# get confidence score (whether the search region is reliable)

‎model-index.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ Import:
66
- configs/mot/ocsort/metafile.yml
77
- configs/sot/siamese_rpn/metafile.yml
88
- configs/sot/stark/metafile.yml
9+
- configs/sot/mixformer/metafile.yml
910
- configs/vid/dff/metafile.yml
1011
- configs/vid/fgfa/metafile.yml
1112
- configs/vid/selsa/metafile.yml

0 commit comments

Comments
 (0)