Update README.md

README.md

@@ -1,3 +1,30 @@
 ---
 license: mit
 ---
+
+**Note: please check [DeepKPG](https://github.com/uclanlp/DeepKPG#scibart) for usage of this model**
+
+Paper: [Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study](https://arxiv.org/abs/2212.10233)
+
+```
+@article{https://doi.org/10.48550/arxiv.2212.10233,
+  doi = {10.48550/ARXIV.2212.10233},
+  url = {https://arxiv.org/abs/2212.10233},
+  author = {Wu, Di and Ahmad, Wasi Uddin and Chang, Kai-Wei},
+  keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
+  title = {Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study},
+  publisher = {arXiv},
+  year = {2022}, 
+  copyright = {Creative Commons Attribution 4.0 International}
+}
+```
+
+Pre-training Corpus: [S2ORC (titles and abstracts)](https://github.com/allenai/s2orc)
+
+Pre-training Details:
+- **Pre-trained from scratch with science vocabulary**
+- Batch size: 2048
+- Total steps: 250k
+- Learning rate: 3e-4
+- LR schedule: polynomial with 10k warmup steps
+- Masking ratio: 30%, Poisson lambda = 3.5