Spaces:
No application file
No application file
File size: 3,491 Bytes
8c18dea a08c529 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 |
---
title: Detection Metrics
emoji: 📈
colorFrom: indigo
colorTo: green
sdk: gradio
sdk_version: 3.36.1
#app_file: app.py
pinned: false
---
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
# COCO Metrics
COCO Metrics is a Python package that provides evaluation metrics for **object detection** tasks using the COCO (Common Objects in Context) [evaluation protocol](https://cocodataset.org/#detection-eval). In the future instance segmentation tasks will also be supported.
## Advantages
* This project does not depend directly on [pycocotools](https://github.com/cocodataset/cocoapi/tree/master/PythonAPI/pycocotools), COCO's official code to compute metrics.
* It does not require a `.json` file in disk, as originally required by pycocotools.
* Integrated with HuggingFace 🤗 evaluate model.
## Metrics
The following 12 metrics are computed to for characterizing the performance of an object detector:
* **Average Precision** (AP) IoU=.50:.05:.95
* **Average Precision** (AP) IoU=.50
* **Average Precision** (AP) IoU=.75
* **Average Precision** (AP) Across Scales for small objects: area < 322
* **Average Precision** (AP) Across Scales for medium objects: 322 < area < 962
* **Average Precision** (AP) Across Scales for large objects: area < 962
* **Average Recall** (AR) given 1 detection per image
* **Average Recall** (AR) given 10 detections per image
* **Average Recall** (AR) given 100 detections per image
* **Average Recall** (AR) for small objects: area < 322
* **Average Recall** (AR) for medium objects: 322 < area < 962
* **Average Recall** (AR) for large objects: area > 962
## Installation
COCO Metrics can be easily installed using pip:
```
# Clone the project
git clone https://github.com/rafaelpadilla/coco_metrics
cd coco_metrics
# Create environment
conda create -n coco-metrics python=3.10
conda activate coco-metrics
# Install packages
pip install -r requirements.txt
```
## Example
The code [example.py](https://github.com/rafaelpadilla/coco_metrics/blob/main/example.py) shows how to make usage of the COCO evaluator using HuggingFace 🤗 evaluate model.
The snippet below illustrates how to make call of the evaluator.
```
import evaluate
# Load evaluate from github
coco_bbx_evaluator = evaluate.load("rafaelpadilla/detection_metrics", coco=coco_gt, iou_type="bbox")
# Within your dataset looping, you add the metrics
for batch in dataloader:
results = # model compute predicted results
labels = # get ground-truth labels
# Add prediction and expected labels to the evaluator
coco_bbx_evaluator.add(prediction=results, reference=labels)
# Compute the metrics and show results
results = coco_bbx_evaluator.compute()
print(results)
```
## References:
[1] [COCO Metrics](https://cocodataset.org/#detection-eval)
[2] [A Survey on Performance Metrics for Object-Detection Algorithms](https://www.researchgate.net/profile/Rafael-Padilla/publication/343194514_A_Survey_on_Performance_Metrics_for_Object-Detection_Algorithms/links/5f1b5a5e45851515ef478268/A-Survey-on-Performance-Metrics-for-Object-Detection-Algorithms.pdf)
[3] [A comparative analysis of object detection metrics with a companion open-source toolkit](https://www.researchgate.net/profile/Rafael-Padilla/publication/343194514_A_Survey_on_Performance_Metrics_for_Object-Detection_Algorithms/links/5f1b5a5e45851515ef478268/A-Survey-on-Performance-Metrics-for-Object-Detection-Algorithms.pdf)
|