mwitiderrick commited on
Commit
4c372cb
1 Parent(s): adb3876

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -20,7 +20,7 @@ Run in a Python pipeline for local inference:
20
  ```python
21
  from vllm import LLM, SamplingParams
22
 
23
- model = LLM("nm-testing/MiniChat-2-3B-pruned2.4", sparsity="sparse_w16a16")
24
  prompt = "How to make banana bread?"
25
  formatted_prompt = f"<s> [|User|]\n{prompt}</s>[|Assistant|]\n"
26
 
 
20
  ```python
21
  from vllm import LLM, SamplingParams
22
 
23
+ model = LLM("nm-testing/MiniChat-2-3B-pruned2.4", sparsity="semi_structured_sparse_w16a16")
24
  prompt = "How to make banana bread?"
25
  formatted_prompt = f"<s> [|User|]\n{prompt}</s>[|Assistant|]\n"
26