AntonV commited on
Commit
6c36a30
1 Parent(s): 400b257

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +57 -0
README.md ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - mamba2
4
+ license: mit
5
+ ---
6
+
7
+ # mamba2-1.3b-av
8
+
9
+ ## Introduction
10
+ This is a mirror model to [mamba2-1.3b](https://huggingface.co/state-spaces/mamba2-1.3b) which is compatible with [mamba2-torch](https://github.com/vasqu/mamba2-torch), a Hugging Face compatible mamba2 library that is not dependent on the original cuda wheels of the [original mamba repo](https://github.com/state-spaces/mamba). Credit goes to the original authors of [Mamba2](https://arxiv.org/abs/2405.21060) and the [transformers](https://github.com/huggingface/transformers) library by Hugging Face. Without their work, this would not be possible.
11
+
12
+
13
+ NOTE: `mamba2-torch` offers different optimisation paths to use:
14
+ - Triton kernels and [causal-conv1d](https://github.com/Dao-AILab/causal-conv1d) ("fastest")
15
+ - Triton kernels only (default)
16
+ - Pure PyTorch
17
+
18
+ ## How to Get Started with the Model
19
+ You can follow the instructions in the [mamba2-torch repo](https://github.com/vasqu/mamba2-torch) for a more detailed explanation. First of all, you should install the mamba2-torch lib:
20
+ ```bash
21
+ git clone https://github.com/vasqu/mamba2-torch.git
22
+ cd mamba2-torch
23
+ pip install .
24
+ ```
25
+
26
+ Then you can download this repository here via git lfs and then use the files locally the following way (after installing mamba2-torch):
27
+ ```python
28
+ from transformers import AutoTokenizer
29
+ from mamba2_torch import Mamba2Model, Mamba2ForCausalLM, Mamba2Config
30
+
31
+ device = "cuda"
32
+ mamba2_hf_path = "<path-to-converted-model>"
33
+
34
+ model = Mamba2ForCausalLM.from_pretrained(mamba2_hf_path, local_files_only=True).to(device)
35
+ tokenizer = AutoTokenizer.from_pretrained(mamba2_hf_path, local_files_only=True)
36
+
37
+ input_ids = tokenizer("Hey how are you doing?", return_tensors="pt")["input_ids"].to(device)
38
+
39
+ # expected output (1.3b): `["Hey how are you doing? I'm doing good. I'm doing good."]`
40
+ out = model.generate(input_ids, max_new_tokens=10)
41
+ print(tokenizer.batch_decode(out))
42
+ ```
43
+
44
+ ## Citation
45
+
46
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
47
+
48
+ **BibTeX:**
49
+
50
+ ```bibtex
51
+ @inproceedings{mamba2,
52
+ title={Transformers are {SSM}s: Generalized Models and Efficient Algorithms Through Structured State Space Duality},
53
+ author={Dao, Tri and Gu, Albert},
54
+ booktitle={International Conference on Machine Learning (ICML)},
55
+ year={2024}
56
+ }
57
+ ```