Create README.md

README.md

@@ -0,0 +1,71 @@
+---
+license: apache-2.0
+---
+
+# Vision-and-Language Transformer (ViLT), fine-tuned on COCO
+
+Vision-and-Language Transformer (ViLT) model fine-tuned on [COCO](https://cocodataset.org/#home). It was introduced in the paper [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://arxiv.org/abs/2102.03334) by Kim et al. and first released in [this repository](https://github.com/dandelin/ViLT). 
+
+Disclaimer: The team releasing ViLT did not write a model card for this model so this model card has been written by the Hugging Face team.
+
+## Intended uses & limitations
+
+You can use the model for image and text retrieval.
+
+### How to use
+
+Here is how to use the model in PyTorch:
+
+```
+from transformers import ViltProcessor, ViltForImageAndTextRetrieval
+import requests
+from PIL import Image
+
+url = "http://images.cocodataset.org/val2017/000000039769.jpg"
+image = Image.open(requests.get(url, stream=True).raw)
+texts = ["An image of two cats chilling on a couch", "A football player scoring a goal"]
+
+processor = ViltProcessor.from_pretrained("dandelin/vilt-b32-finetuned-coco")
+model = ViltForImageAndTextRetrieval.from_pretrained("dandelin/vilt-b32-finetuned-coco")
+
+# prepare inputs
+encoding = processor(image, text, return_tensors="pt")
+
+# forward pass
+scores = dict()
+for text in texts:
+    encoding = processor(image, text, return_tensors="pt")
+    outputs = model(**encoding)
+    scores[text] = outputs.logits[0, :].item()
+```
+
+## Training data
+
+(to do)
+
+## Training procedure
+
+### Preprocessing
+
+(to do)
+
+### Pretraining
+
+(to do)
+
+## Evaluation results
+
+(to do)
+
+### BibTeX entry and citation info
+
+```bibtex
+@misc{kim2021vilt,
+      title={ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision}, 
+      author={Wonjae Kim and Bokyung Son and Ildoo Kim},
+      year={2021},
+      eprint={2102.03334},
+      archivePrefix={arXiv},
+      primaryClass={stat.ML}
+}
+```