Update README.md

README.md

@@ -1,21 +1,26 @@
 ---
 library_name: fasttext
 tags:
-  - text-classification
-  - language-identification
+- text-classification
+- language-identification
+metrics:
+- f1
 ---
 
 # OpenLID
 
+- **Developed by:** Laurie Burchell, Alexandra Birch, Nikolay Bogoychev, Kenneth Heafield
+- **Model type:** Text classification (language identification)
+- **Language(s) (NLP):** en
+- **License:** gpl-3.0
+- **Resources for more information:** [OpenLID github repo](https://github.com/laurieburchell/open-lid-dataset)
 
 ## Model description
 
-fastText is a library for efficient learning of word representations and sentence classification. fastText is designed to be simple to use for developers, domain experts, and students. It's dedicated to text classification and learning word representations, and was designed to allow for quick model iteration and refinement without specialized hardware. fastText models can be trained on more than a billion words on any multicore CPU in less than a few minutes.
+OpenLID is a high-coverage, high-performance language identification model. It is a fastText model which covers 201 languages. The training data and per-language performance are openly available to encourage further research.
 
+The model and training data are described in [Burchell et al. (2023)](https://aclanthology.org/2023.acl-short.75/) and the original fastText implementation is available through [github](https://github.com/laurieburchell/open-lid-dataset).
 
-## Intended uses & limitations
-
-You can use pre-trained word vectors for text classification or language identification. See the [tutorials](https://fasttext.cc/docs/en/supervised-tutorial.html) and [resources](https://fasttext.cc/docs/en/english-vectors.html) on its official website to look for tasks that interest you.
 
 ### How to use
 
@@ -39,22 +44,67 @@ Here is how to use this model to detect the language of a given text:
 
 ### Limitations and bias
 
-[More Information needed]
+The dataset and model only covers 201 languages: the ones we were able to test with the FLORES-200 Evaluation Benchmark. In addition, because our test set consists of sentences from a single domain (wiki articles), performance on this test set may not reflect how well our classifier works in other domains. Future work could create a LID test set representative of web data where these classifiers are often applied. Finally, most of the data was not audited by native speakers as would be ideal. Future versions of this dataset should have more languages verified by native speakers, with a focus on the least resourced languages.
+
+Our work aims to broaden NLP coverage by allowing practitioners to identify relevant data in more languages. However, we note that LID is inherently a normative activity that risks excluding minority dialects, scripts, or entire microlanguages from a macrolanguage. Choosing which languages to cover may reinforce power imbalances, as only some groups gain access to NLP technologies. In addition, errors in LID can have a significant impact on downstream performance, particularly (as is often the case) when a system is used as a ‘black box’. The performance of our classifier is not equal across languages which could lead to worse downstream performance for particular groups. We mitigate this by providing metrics by class.
+
 ## Training data
 
+The model was trained on the OpenLID dataset which is available [through the github repo](https://github.com/laurieburchell/open-lid-dataset) or on HuggingFace.
+
+The final dataset contains 121 million lines of data in 201 language classes. Before sampling, the mean number of lines per language is 602,812. The smallest class contains 532 lines of data (South Azerbaijani) and the largest contains 7.5 million lines of data (English). More details at paper
 
-[More Information needed]
 ## Training procedure
 
+The model was trained using fastText with the following hyperparameters set. All other hyperparameters were set to their default values.
 
-### License
+* loss: softmax
+* epochs: 2
+* learning rate: 0.8
+* minimum number of word occurances: 1000
+* embedding dimension: 256
+* character n-grams: 2-5
+* word n-grams: 1
+* bucket size: 1,000,000
+* threads: 68
 
-[More Information needed]
 
 ### Evaluation datasets
 
-[More Information needed]
+The model was evaluated using the FLORES-200 benchmark provided by Costa-jussà et al. (2022). Further information is available in the paper. 
 
 ### BibTeX entry and citation info
 
-[More Information needed]
+#### ACL citation (preferred)
+
+```
+@inproceedings{burchell-etal-2023-open,
+    title = "An Open Dataset and Model for Language Identification",
+    author = "Burchell, Laurie  and
+      Birch, Alexandra  and
+      Bogoychev, Nikolay  and
+      Heafield, Kenneth",
+    editor = "Rogers, Anna  and
+      Boyd-Graber, Jordan  and
+      Okazaki, Naoaki",
+    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",
+    month = jul,
+    year = "2023",
+    address = "Toronto, Canada",
+    publisher = "Association for Computational Linguistics",
+    url = "https://aclanthology.org/2023.acl-short.75",
+    doi = "10.18653/v1/2023.acl-short.75",
+    pages = "865--879",
+    abstract = "Language identification (LID) is a fundamental step in many natural language processing pipelines. However, current LID systems are far from perfect, particularly on lower-resource languages. We present a LID model which achieves a macro-average F1 score of 0.93 and a false positive rate of 0.033{\%} across 201 languages, outperforming previous work. We achieve this by training on a curated dataset of monolingual data, which we audit manually to ensure reliability. We make both the model and the dataset available to the research community. Finally, we carry out detailed analysis into our model{'}s performance, both in comparison to existing open models and by language class.",
+}
+```
+
+#### ArXiv citation
+```
+@article{burchell2023open,
+  title={An Open Dataset and Model for Language Identification},
+  author={Burchell, Laurie and Birch, Alexandra and Bogoychev, Nikolay and Heafield, Kenneth},
+  journal={arXiv preprint arXiv:2305.13820},
+  year={2023}
+}
+```