ReAtt
ReAtt is a retrieval-augmented model for knowledge-intensive tasks proposed in Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer. The original Github repository is https://github.com/jzbjyb/ReAtt.
Description
neulab/reatt-large-nq
(based on T5 architecture) is initialized with google/t5-large-lm-adapt
and fine-tuned on Natural Questions with end-to-end retrieval-augmented training.
Usage
Please refer to https://github.com/jzbjyb/ReAtt for instructions to use this model.
Reference
@inproceedings{jiang-etal-2022-reatt,
title = {Retrieval as Attention: End-to-end Learning of Retrieval and Reading within a Single Transformer},
author = {Zhengbao Jiang and Luyu Gao and Jun Araki and Haibo Ding and Zhiruo Wang and Jamie Callan and Graham Neubig},
booktitle = {Conference on Empirical Methods in Natural Language Processing (EMNLP)},
address = {Abu Dhabi, UAE},
month = {December},
year = {2022}
}
- Downloads last month
- 3
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.