--- library_name: transformers tags: [] --- # Model Details ### Model Description MiewID-msv2 is a feature extractor trained for re-identification using contrastive learning on a large, high-quality dataset of 54 wildlife species - terrestrial and aquatic - including fins, flukes, flanks, faces. - **Model Type:** Wildlife re-identification feature backbone - **Model Stats:** - Params (M): 51.11 - GMACs: 24.38 - Activations (M): 91.11 - Image size: 440 x 440 ### Model Sources [optional] - **Repository:** https://github.com/WildMeOrg/wbia-plugin-miew-id - **Backbone:** https://huggingface.co/timm/efficientnetv2_rw_m.agc_in1k # Usage Intended use is re-identification of individuals from different species by matching against a database of grount-truth samples. Model features can also be used for species classification by retrieval. ### Embedding Extraction ``` import numpy as np from PIL import Image import torch import torchvision.transforms as transforms from transformers import AutoModel model_tag = f"conservationxlabs/miewid-msv2" model = AutoModel.from_pretrained(model_tag, trust_remote_code=True) def generate_random_image(height=440, width=440, channels=3): random_image = np.random.randint(0, 256, (height, width, channels), dtype=np.uint8) return Image.fromarray(random_image) random_image = generate_random_image() preprocess = transforms.Compose([ transforms.Resize((440, 440)), transforms.ToTensor(), transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]), ]) input_tensor = preprocess(random_image) input_batch = input_tensor.unsqueeze(0) with torch.no_grad(): output = model(input_batch) print(output) print(output.shape) ``` ### More Examples View more usage examples at https://github.com/WildMeOrg/wbia-plugin-miew-id/tree/main/wbia_miew_id/datasets # Training Details ### Training Data The dataset used for these experiments was a combination of data from Wildbook platforms (multiple users), Happywhale Kaggle competitions multi-species dataset and multiple publicly available datasets. A small subset of data from Wildbook platforms is available at https://lila.science/datasets. ### Example images ![image/jpg](https://cthulhu.dyn.wildme.io/public/datasets/msv2_dataset_example.jpg) # Evaluation Results The multi-species model shows improvements over models trained on single species. Moreover the model shows strong generalization ability for majority of the species when trained and evaluated in a leave-one-out fashion. ### Detailed results |Source |group |map |rank-1|rank-5|rank-10|rank-20|viewpoints |n_test_annots|n_test_names|n_train_annots|n_train_names| |--------------|--------------------------------|-----|------|------|-------|-------|--------------------------------------------------------------------------------------------------------------------|-------------|------------|--------------|-------------| |Wildbook |amur_tiger |91.8 |100.0 |100.0 |100.0 |100.0 |['left', 'right'] |233 |47 |691 |75 | |Wildbook |beluga_whale |61.36|71.85 |81.63 |85.51 |89.28 |['up'] |849 |228 |2767 |354 | |Happywhale |blue_whale |36.98|38.68 |55.11 |61.52 |69.04 |['unknown'] |998 |392 |1185 |339 | |Wildbook |bottlenose_dolphin |85.69|93.99 |96.18 |96.85 |97.52 |['right', 'left'] |4911 |838 |11607 |1062 | |Happywhale |brydes_whale |69.96|80.95 |92.86 |95.24 |100.0 |['unknown'] |42 |9 |81 |15 | |Lomas Capuchin|capuchin |34.8 |47.97 |68.58 |76.69 |85.14 |['front'] |296 |35 |1708 |29 | |Wildbook |cheetah |57.37|71.15 |84.92 |87.87 |92.13 |['left', 'right'] |624 |139 |1522 |144 | |Wildbook |chimpanzee |94.7 |100.0 |100.0 |100.0 |100.0 |['unknown'] |52 |8 |138 |9 | |C-Tai |chimpanzee_ctai |51.67|69.64 |83.68 |89.18 |94.12 |['front'] |527 |63 |2680 |44 | |C-Zoo |chimpanzee_czoo |77.66|86.67 |93.33 |97.5 |99.17 |['front'] |240 |24 |1258 |17 | |Happywhale |commersons_dolphin |0.0 |0.0 |0.0 |0.0 |0.0 |['unknown'] |10 |5 |7 |2 | |Happywhale |cuviers_beaked_whale |52.89|55.71 |75.71 |82.86 |90.0 |['unknown'] |70 |28 |89 |21 | |DogFaceNet |dog |82.96|87.38 |94.78 |96.74 |97.99 |['unknown'] |1838 |544 |5750 |1088 | |Happywhale |dusky_dolphin |85.82|83.54 |93.04 |94.3 |94.94 |['unknown'] |158 |68 |154 |41 | |Wildbook |eurasianlynx |58.8 |67.82 |83.78 |88.83 |92.02 |['left', 'right'] |726 |199 |2251 |285 | |Happywhale |fin_whale |67.65|76.92 |87.18 |90.11 |93.04 |['unknown'] |273 |71 |533 |55 | |Happywhale |frasiers_dolphin |0.0 |0.0 |0.0 |0.0 |0.0 | |-1 |-1 |-1 |-1 | |Wildbook |giraffe |99.18|99.08 |99.08 |100.0 |100.0 |['right', 'front', 'left', 'back'] |423 |162 |2540 |426 | |Wildbook |horse_wild+face |0.0 |0.0 |0.0 |0.0 |0.0 |['front'] |17 |7 |174 |53 | |Wildbook |horse_wild_tunisian+face |79.59|98.53 |99.51 |100.0 |100.0 |['unknown'] |204 |39 |999 |37 | |Wildbook |hyena |64.17|79.32 |89.22 |93.14 |96.41 |['right', 'left'] |1231 |301 |1154 |201 | |Wildbook |hyperoodon_ampullatus |91.14|96.11 |98.12 |98.34 |98.84 |['left', 'right'] |1930 |354 |24952 |759 | |Wildbook |jaguar |67.45|81.72 |91.04 |92.91 |94.4 |['left', 'right'] |540 |124 |980 |103 | |PrimFace |japanese_monkey |89.05|93.85 |98.46 |98.46 |100.0 |['unknown'] |65 |12 |219 |14 | |Happywhale |killer_whale |62.82|77.13 |85.48 |88.81 |91.82 |['unknown'] |599 |145 |1379 |177 | |Wildbook |leopard |65.03|82.64 |90.23 |92.83 |94.46 |['right', 'left'] |595 |108 |4261 |381 | |Wildbook |lion |87.12|93.86 |100.0 |100.0 |100.0 |['front', 'unknown', 'frontright', 'frontleft', 'upfront', 'right', 'upleft', 'left', 'upright', 'backleft', 'back']|1364 |242 |3981 |241 | |Happywhale |long_finned_pilot_whale |99.06|98.11 |100.0 |100.0 |100.0 |['unknown'] |53 |21 |33 |10 | |Wildbook |lynx_pardinus |48.22|57.68 |73.44 |77.18 |84.96 |['left', 'right'] |467 |142 |1376 |259 | |MacaqueFaces |macaque_face |89.87|95.88 |97.94 |99.12 |100.0 |['front'] |340 |34 |3480 |25 | |Happywhale |melon_headed_whale |90.8 |93.42 |96.05 |96.71 |98.68 |['unknown'] |152 |55 |180 |43 | |Wildbook |nyala |64.64|70.65 |90.05 |94.03 |97.01 |['left', 'right'] |418 |99 |1303 |149 | |Happywhale |pantropic_spotted_dolphin |78.15|85.19 |90.74 |94.44 |96.3 |['unknown'] |54 |11 |55 |8 | |Happywhale |pygmy_killer_whale |89.07|84.62 |100.0 |100.0 |100.0 |['unknown'] |26 |10 |32 |5 | |PrimFace |rhesus_monkey |78.12|89.13 |97.28 |98.91 |100.0 |['unknown'] |184 |28 |478 |24 | |Happywhale |rough_toothed_dolphin |0.0 |0.0 |0.0 |0.0 |0.0 |['unknown'] |6 |3 |7 |2 | |SealID |seal |37.93|59.51 |80.0 |86.1 |91.71 |['unknown'] |410 |41 |1448 |36 | |Happywhale |sei_whale |67.86|71.55 |87.07 |93.1 |95.69 |['unknown'] |116 |35 |106 |29 | |Wildbook |short_fin_pilot_whale+fin_dorsal|92.49|92.31 |98.08 |100.0 |100.0 |['right', 'left'] |272 |64 |1290 |137 | |Happywhale |short_finned_pilot_whale |94.39|94.96 |95.8 |95.8 |97.48 |['unknown'] |119 |57 |67 |17 | |Wildbook |snow_leopard |62.43|82.45 |88.91 |92.15 |94.53 |['left', 'right'] |927 |144 |2307 |99 | |Happywhale |spinner_dolphin |99.88|100.0 |100.0 |100.0 |100.0 |['left', 'right'] |335 |97 |1562 |251 | |Happywhale |spotted_dolphin |84.92|83.67 |89.8 |95.92 |97.96 |['unknown'] |49 |22 |128 |28 | |Wildbook |turtle_green |100.0|100.0 |100.0 |100.0 |100.0 |['left', 'right', 'up', 'front'] |543 |109 |5097 |76 | |Wildbook |turtle_green+head |95.1 |96.02 |100.0 |100.0 |100.0 |['left', 'right', 'up', 'front'] |544 |108 |5737 |76 | |Wildbook |turtle_hawksbill |78.84|81.15 |88.37 |92.56 |95.81 |['up', 'left', 'right', 'front'] |506 |106 |5397 |71 | |Wildbook |turtle_hawksbill+head |86.78|91.74 |95.45 |96.69 |98.35 |['right', 'left', 'up', 'front'] |589 |109 |7033 |71 | |SeaTurtleID |turtle_loggerhead_ext |63.67|75.43 |86.61 |90.17 |92.87 |['unknown', 'left', 'top', 'right'] |815 |217 |6573 |355 | |SeaTurtleID |turtle_loggerhead_ext+head |94.14|94.63 |100.0 |100.0 |100.0 |['left', 'right', 'topright', 'top', 'topleft', 'below', 'front'] |778 |209 |6361 |348 | |Wildbook |whale_fin+fin_dorsal |82.12|85.12 |93.87 |95.04 |95.75 |['left', 'right'] |394 |110 |1629 |186 | |Wildbook |whale_grey |90.97|91.55 |95.54 |96.18 |96.82 |['right', 'left'] |381 |103 |5662 |758 | |Wildbook |whale_humpback+fin_dorsal |80.3 |84.07 |90.08 |91.6 |95.42 |['right', 'left'] |509 |194 |3734 |541 | |Wildbook |whale_humpback+fluke |97.11|96.67 |100.0 |100.0 |100.0 |['back'] |150 |64 |1151 |356 | |Wildbook |whale_orca |97.01|98.36 |99.18 |100.0 |100.0 |['right', 'left'] |249 |49 |2790 |435 | |Wildbook |whale_orca+fin_dorsal |95.41|95.87 |97.52 |97.52 |99.17 |['right', 'left'] |623 |187 |3978 |430 | |Wildbook |whale_sperm+fluke |96.14|97.87 |98.33 |98.48 |99.09 |['back'] |658 |125 |25849 |558 | |Wildbook |whaleshark |57.61|72.25 |79.96 |82.27 |87.21 |['left', 'right'] |997 |212 |5713 |376 | |Wildbook |white_shark+fin_dorsal |91.4 |94.89 |97.87 |98.54 |99.27 |['left', 'right'] |322 |88 |1067 |162 | |Happywhale |white_sided_dolphin |81.82|76.0 |88.0 |88.0 |100.0 |['unknown'] |25 |11 |13 |4 | |Wildbook |wilddog |73.4 |85.11 |90.93 |92.38 |94.07 |['left', 'right'] |3460 |848 |4217 |656 | |Wildbook |zebra_grevys |91.79|96.79 |97.57 |97.74 |98.04 |['right'] |1684 |332 |8927 |362 |