license: apache-2.0
pipeline_tag: image-classification
library_name: transformers
tags:
- image-detection
- ai-image-generation
- anime
- ai-anime
- human-detection
- art
AI Anime Image Detector ViT
This model is a proof of concept model of detecting anime styled AI images. Using Vision Transformer, it was trained on 1M human-made real and 217K AI generated anime images. During training either type appeared in equal amount to avoid biases. The model was trained on a single RTX 3090 GPU for about 40 hours, ~35 epochs.
Evaluation
Each checkpoint was evaluated on 500-500 real and AI images.
- Training Loss: 0.1009
- Eval Loss: 0.1386
It seems like using random crops helped the model to generalize better, however, the training dataset only contained 512x512 images, which meant that every cropped image had bilinear interpolation. Training the model on 1024x1024 images could probably further improve its performance.
Performance comparison
We did a small comparison with the current available AI image detectors. Note that these models were not specificly trained on anime images.
| Image | Nahrawy/AIorNot | umm-maybe/AI-image-detector | Organika/sdxl-detector | Ours |
|---|---|---|---|---|
| D:\test\ai_1.jpg | ai (100%) | human (86%) | artificial (100%) | ai (100%) |
| D:\test\ai_2.jpg | ai (99%) | human (96%) | artificial (100%) | ai (100%) |
| D:\test\ai_3.jpg | ai (77%) | human (98%) | artificial (100%) | ai (100%) |
| D:\test\ai_4.jpg | real (66%) | human (100%) | human (100%) | real (100%) |
| D:\test\ai_5.jpg | ai (51%) | human (99%) | artificial (55%) | real (65%) |
| D:\test\ai_6.jpg | ai (100%) | human (98%) | artificial (100%) | ai (84%) |
| D:\test\real_1.jpg | ai (99%) | human (99%) | artificial (100%) | ai (55%) |
| D:\test\real_2.jpg | ai (88%) | human (100%) | artificial (100%) | real (85%) |
| D:\test\real_3.jpg | ai (95%) | human (96%) | artificial (100%) | real (97%) |
| D:\test\real_4.jpg | real (90%) | human (100%) | artificial (97%) | real (94%) |
| D:\test\real_5.jpg | ai (75%) | human (100%) | human (57%) | real (100%) |
| D:\test\real_6.jpg | ai (89%) | human (98%) | human (100%) | real (99%) |
| Accuracy: | 50% | 50% | 58% | 75% |
Usage
Example inference code:
from transformers import AutoModelForImageClassification, AutoFeatureExtractor
import torch
from PIL import Image
model = AutoModelForImageClassification.from_pretrained("legekka/AI-Anime-Image-Detector-ViT")
feature_extractor = AutoFeatureExtractor.from_pretrained("legekka/AI-Anime-Image-Detector-ViT")
model.eval()
image = Image.open("example.jpg")
inputs = feature_extractor(images=image, return_tensors="pt")
outputs = model(**inputs)
logits = outputs.logits
label = model.config.id2label[torch.argmax(logits).item()]
confidence = torch.nn.functional.softmax(logits, dim=1)[0][torch.argmax(logits)].item()
print(f"Prediction: {label} ({round(confidence * 100)}%)")