Update README.md
Browse files
README.md
CHANGED
|
@@ -22,6 +22,9 @@ datasets:
|
|
| 22 |
# Exl2 Quantization 6.0BPW
|
| 23 |
This model fits comfortably within 72gb of VRAM with 32k context. It was created after inference/quant bug was repaired.
|
| 24 |
|
|
|
|
|
|
|
|
|
|
| 25 |
Enjoy! Feel free to reach out for other quants, or BPW levels.
|
| 26 |
|
| 27 |
# Dolphin 2.9.2 Qwen2 72B 🐬
|
|
|
|
| 22 |
# Exl2 Quantization 6.0BPW
|
| 23 |
This model fits comfortably within 72gb of VRAM with 32k context. It was created after inference/quant bug was repaired.
|
| 24 |
|
| 25 |
+
- 6 head bits
|
| 26 |
+
- 6.0 bpw target
|
| 27 |
+
|
| 28 |
Enjoy! Feel free to reach out for other quants, or BPW levels.
|
| 29 |
|
| 30 |
# Dolphin 2.9.2 Qwen2 72B 🐬
|