info/RMT.md

RMT is a memory-augmented segment-level recurrent Transformer. We implement our memory mechanism as a wrapper for any Hugging Face model by adding special memory tokens to the input sequence. The model is trained to control both memory operations and sequence representations processing.

See: [ [paper](https://arxiv.org/abs/2402.10790) ] and [ [code](https://github.com/booydar/recurrent-memory-transformer/tree/babilong-release) ] for **Recurrent Memory Transformer** implementation and training examples.