OuteAI
/

Lite-Oute-2-Mamba2Attn-250M-Instruct

Model card Files Files and versions Community

edwko commited on 29 days ago

Commit

152a992

•

1 Parent(s): 4e3820e

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -145,7 +145,7 @@ For instruction training, we first trained the model with Supervised Fine-tuning
 ## Interfacing with the Instruct Model
 Model weights were converted from the original Mamba2 implementation to be Hugging Face compatible. <br>
 Due to the lack of official support for Mamba2 attention layers in Hugging Face Transformers, custom modeling files are included. <br>
-The attention layer implementation is based on the work from Pull Request #32027 in the Hugging Face Transformers repository: [https://github.com/huggingface/transformers/pull/32027](https://github.com/huggingface/transformers/pull/32027)
 > [!IMPORTANT]
 > To ensure optimal performance, please use the following template when interacting with the model:

 ## Interfacing with the Instruct Model
 Model weights were converted from the original Mamba2 implementation to be Hugging Face compatible. <br>
 Due to the lack of official support for Mamba2 attention layers in Hugging Face Transformers, custom modeling files are included. <br>
+The attention layer implementation for the modeling files are based on the work from Pull Request #32027 in the Hugging Face Transformers repository: [https://github.com/huggingface/transformers/pull/32027](https://github.com/huggingface/transformers/pull/32027)
 > [!IMPORTANT]
 > To ensure optimal performance, please use the following template when interacting with the model: