info/RMT.md

RMT is a memory-augmented segment-level recurrent Transformer. We implement our memory mechanism as a wrapper for any Hugging Face model by adding special memory tokens to the input sequence. The model is trained to control both memory operations and sequence representations processing.

See: [ paper ] and [ code ] for Recurrent Memory Transformer implementation and training examples.