File size: 9,250 Bytes
186701e
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
# Model Zoo and Benchmark

This page is used to summarize the performance and related evaluation metrics of various models supported in MMYOLO for users to compare and analyze.

## COCO dataset

<div align=center>
<img src="https://user-images.githubusercontent.com/17425982/222087414-168175cc-dae6-4c5c-a8e3-3109a152dd19.png"/>
</div>

|      Model       | Arch | Size | Batch Size | Epoch | SyncBN | AMP | Mem (GB) | Params(M) | FLOPs(G) | TRT-FP16-GPU-Latency(ms) | Box AP | TTA Box AP |
| :--------------: | :--: | :--: | :--------: | :---: | :----: | :-: | :------: | :-------: | :------: | :----------------------: | :----: | :--------: |
|     YOLOv5-n     |  P5  | 640  |   8xb16    |  300  |  Yes   | Yes |   1.5    |   1.87    |   2.26   |           1.14           |  28.0  |    30.7    |
|  YOLOv6-v2.0-n   |  P5  | 640  |   8xb32    |  400  |  Yes   | Yes |   6.04   |   4.32    |   5.52   |           1.37           |  36.2  |            |
|     YOLOv8-n     |  P5  | 640  |   8xb16    |  500  |  Yes   | Yes |   2.5    |   3.16    |   4.4    |           1.53           |  37.4  |    39.9    |
|   RTMDet-tiny    |  P5  | 640  |   8xb32    |  300  |  Yes   | No  |   11.9   |   4.90    |   8.09   |           2.31           |  41.8  |    43.2    |
| YOLOv6-v2.0-tiny |  P5  | 640  |   8xb32    |  400  |  Yes   | Yes |   8.13   |   9.70    |  12.37   |           2.19           |  41.0  |            |
|   YOLOv7-tiny    |  P5  | 640  |   8xb16    |  300  |  Yes   | Yes |   2.7    |   6.23    |   6.89   |           1.88           |  37.5  |            |
|    YOLOX-tiny    |  P5  | 416  |   8xb32    |  300  |   No   | Yes |   4.9    |   5.06    |   7.63   |           1.19           |  34.3  |            |
|     RTMDet-s     |  P5  | 640  |   8xb32    |  300  |  Yes   | No  |   16.3   |   8.89    |  14.84   |           2.89           |  45.7  |    47.3    |
|     YOLOv5-s     |  P5  | 640  |   8xb16    |  300  |  Yes   | Yes |   2.7    |   7.24    |   8.27   |           1.89           |  37.7  |    40.2    |
|  YOLOv6-v2.0-s   |  P5  | 640  |   8xb32    |  400  |  Yes   | Yes |   8.88   |   17.22   |  21.94   |           2.67           |  44.0  |            |
|     YOLOv8-s     |  P5  | 640  |   8xb16    |  500  |  Yes   | Yes |   4.0    |   11.17   |  14.36   |           2.61           |  45.1  |    46.8    |
|     YOLOX-s      |  P5  | 640  |   8xb32    |  300  |   No   | Yes |   9.8    |   8.97    |  13.40   |           2.38           |  41.9  |            |
|   PPYOLOE+ -s    |  P5  | 640  |    8xb8    |  80   |  Yes   | No  |   4.7    |   7.93    |   8.68   |           2.54           |  43.5  |            |
|     RTMDet-m     |  P5  | 640  |   8xb32    |  300  |  Yes   | No  |   29.0   |   24.71   |  39.21   |           6.23           |  50.2  |    51.9    |
|     YOLOv5-m     |  P5  | 640  |   8xb16    |  300  |  Yes   | Yes |   5.0    |   21.19   |  24.53   |           4.28           |  45.3  |    46.9    |
|  YOLOv6-v2.0-m   |  P5  | 640  |   8xb32    |  300  |  Yes   | Yes |  16.69   |   34.25   |   40.7   |           5.12           |  48.4  |            |
|     YOLOv8-m     |  P5  | 640  |   8xb16    |  500  |  Yes   | Yes |   7.0    |   25.9    |  39.57   |           5.78           |  50.6  |    52.3    |
|     YOLOX-m      |  P5  | 640  |   8xb32    |  300  |   No   | Yes |   17.6   |   25.33   |  36.88   |           5.31           |  47.5  |            |
|   PPYOLOE+ -m    |  P5  | 640  |    8xb8    |  80   |  Yes   | No  |   8.4    |   23.43   |  24.97   |           5.47           |  49.5  |            |
|     RTMDet-l     |  P5  | 640  |   8xb32    |  300  |  Yes   | No  |   45.2   |   52.32   |  80.12   |          10.13           |  52.3  |    53.7    |
|     YOLOv5-l     |  P5  | 640  |   8xb16    |  300  |  Yes   | Yes |   8.1    |   46.56   |  54.65   |           6.8            |  48.8  |    49.9    |
|  YOLOv6-v2.0-l   |  P5  | 640  |   8xb32    |  300  |  Yes   | Yes |  20.86   |   58.53   |  71.43   |           8.78           |  51.0  |            |
|     YOLOv7-l     |  P5  | 640  |   8xb16    |  300  |  Yes   | Yes |   10.3   |   36.93   |  52.42   |           6.63           |  50.9  |            |
|     YOLOv8-l     |  P5  | 640  |   8xb16    |  500  |  Yes   | Yes |   9.1    |   43.69   |  82.73   |           8.97           |  53.0  |    54.4    |
|     YOLOX-l      |  P5  | 640  |    8xb8    |  300  |   No   | Yes |   8.0    |   54.21   |  77.83   |           9.23           |  50.1  |            |
|   PPYOLOE+ -l    |  P5  | 640  |    8xb8    |  80   |  Yes   | No  |   13.2   |   52.20   |  55.05   |           8.2            |  52.6  |            |
|     RTMDet-x     |  P5  | 640  |   8xb32    |  300  |  Yes   | No  |   63.4   |   94.86   |  145.41  |          17.89           |  52.8  |    54.2    |
|     YOLOv7-x     |  P5  | 640  |   8xb16    |  300  |  Yes   | Yes |   13.7   |   71.35   |  95.06   |          11.63           |  52.8  |            |
|     YOLOv8-x     |  P5  | 640  |   8xb16    |  500  |  Yes   | Yes |   12.4   |   68.23   |  132.10  |          14.22           |  54.0  |    55.0    |
|     YOLOX-x      |  P5  | 640  |    8xb8    |  300  |   No   | Yes |   9.8    |   99.07   |  144.39  |          15.35           |  51.4  |            |
|   PPYOLOE+ -x    |  P5  | 640  |    8xb8    |  80   |  Yes   | No  |   19.1   |   98.42   |  105.48  |          14.02           |  54.2  |            |
|     YOLOv5-n     |  P6  | 1280 |   8xb16    |  300  |  Yes   | Yes |   5.8    |   3.25    |   2.30   |                          |  35.9  |            |
|     YOLOv5-s     |  P6  | 1280 |   8xb16    |  300  |  Yes   | Yes |   10.5   |   12.63   |   8.45   |                          |  44.4  |            |
|     YOLOv5-m     |  P6  | 1280 |   8xb16    |  300  |  Yes   | Yes |   19.1   |   35.73   |  25.05   |                          |  51.3  |            |
|     YOLOv5-l     |  P6  | 1280 |   8xb16    |  300  |  Yes   | Yes |   30.5   |   76.77   |  55.77   |                          |  53.7  |            |
|     YOLOv7-w     |  P6  | 1280 |   8xb16    |  300  |  Yes   | Yes |   27.0   |   82.31   |  45.07   |                          |  54.1  |            |
|     YOLOv7-e     |  P6  | 1280 |   8xb16    |  300  |  Yes   | Yes |   42.5   |  114.69   |  64.48   |                          |  55.1  |            |

- All the models are trained on COCO train2017 dataset and evaluated on val2017 dataset.
- TRT-FP16-GPU-Latency(ms) is the GPU Compute time on NVIDIA Tesla T4 device with TensorRT 8.4, a batch size of 1, a test shape of 640x640 and only model forward (The test shape for YOLOX-tiny is 416x416)
- The number of model parameters and FLOPs are obtained using the [get_flops](https://github.com/open-mmlab/mmyolo/blob/dev/tools/analysis_tools/get_flops.py) script. Different calculation methods may vary slightly
- RTMDet performance is the result of training with [MMRazor Knowledge Distillation](https://github.com/open-mmlab/mmyolo/blob/dev/configs/rtmdet/distillation/README.md)
- Only YOLOv6 version 2.0 is implemented in MMYOLO for now, and L and M are the results without knowledge distillation
- YOLOv8 results are optimized using mask instance annotation, but YOLOv5, YOLOv6 and YOLOv7 do not use
- PPYOLOE+ uses Obj365 as pre-training weights, so the number of epochs for COCO training only needs 80
- YOLOX-tiny, YOLOX-s and YOLOX-m are trained with the optimizer parameters proposed in RTMDet, with different degrees of performance improvement compared to the original implementation.

Please see below items for more details

- [RTMDet](https://github.com/open-mmlab/mmyolo/blob/main/configs/rtmdet)
- [YOLOv5](https://github.com/open-mmlab/mmyolo/blob/main/configs/yolov5)
- [YOLOv6](https://github.com/open-mmlab/mmyolo/blob/main/configs/yolov6)
- [YOLOv7](https://github.com/open-mmlab/mmyolo/blob/main/configs/yolov7)
- [YOLOv8](https://github.com/open-mmlab/mmyolo/blob/main/configs/yolov8)
- [YOLOX](https://github.com/open-mmlab/mmyolo/blob/main/configs/yolox)
- [PPYOLO-E](https://github.com/open-mmlab/mmyolo/blob/main/configs/ppyoloe)

## VOC dataset

| Backbone | size | Batchsize | AMP | Mem (GB) | box AP(COCO metric) |
| :------: | :--: | :-------: | :-: | :------: | :-----------------: |
| YOLOv5-n | 512  |    64     | Yes |   3.5    |        51.2         |
| YOLOv5-s | 512  |    64     | Yes |   6.5    |        62.7         |
| YOLOv5-m | 512  |    64     | Yes |   12.0   |        70.1         |
| YOLOv5-l | 512  |    32     | Yes |   10.0   |        73.1         |

Please see below items for more details

- [YOLOv5](https://github.com/open-mmlab/mmyolo/blob/main/configs/yolov5)

## CrowdHuman dataset

| Backbone | size | SyncBN | AMP | Mem (GB) | ignore_iof_thr | box AP50(CrowDHuman Metric) |  MR  |  JI   |
| :------: | :--: | :----: | :-: | :------: | :------------: | :-------------------------: | :--: | :---: |
| YOLOv5-s | 640  |  Yes   | Yes |   2.6    |       -1       |            85.79            | 48.7 | 75.33 |
| YOLOv5-s | 640  |  Yes   | Yes |   2.6    |      0.5       |            86.17            | 48.8 | 75.87 |

Please see below items for more details

- [YOLOv5](https://github.com/open-mmlab/mmyolo/blob/main/configs/yolov5)

## DOTA 1.0 dataset