Commit
•
68bd6b9
1
Parent(s):
4433df2
improvement: readme
Browse files
README.md
CHANGED
|
@@ -12,7 +12,6 @@ quantized_by: shaowenchen
|
|
| 12 |
tasks:
|
| 13 |
- text2text-generation
|
| 14 |
tags:
|
| 15 |
-
- meta
|
| 16 |
- gguf
|
| 17 |
- llama
|
| 18 |
- llama-2
|
|
@@ -42,20 +41,28 @@ tags:
|
|
| 42 |
Usage:
|
| 43 |
|
| 44 |
```bash
|
| 45 |
-
docker run --rm -it -p 8000:8000 -v /path/to/models:/models -e MODEL=/models/gguf-model-name.gguf
|
| 46 |
```
|
| 47 |
|
| 48 |
## Provided images
|
| 49 |
|
| 50 |
-
| Name | Quant method | Size
|
| 51 |
-
| -------------------------------------------------- | ------------ |
|
| 52 |
-
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q2_K` | Q2_K | 3.68 GB
|
| 53 |
-
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q3_K` | Q3_K | 4.16 GB
|
| 54 |
-
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q3_K_L` | Q3_K_L | 4.46 GB
|
| 55 |
-
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q3_K_S` | Q3_K_S | 3.81 GB
|
| 56 |
-
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q4_0` | Q4_0 | 4.7 GB
|
| 57 |
-
| `shaowenchen/colossal-llama-2-7b-base-gguf:
|
| 58 |
-
| `shaowenchen/colossal-llama-2-7b-base-gguf:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
|
| 60 |
Usage:
|
| 61 |
|
|
|
|
| 12 |
tasks:
|
| 13 |
- text2text-generation
|
| 14 |
tags:
|
|
|
|
| 15 |
- gguf
|
| 16 |
- llama
|
| 17 |
- llama-2
|
|
|
|
| 41 |
Usage:
|
| 42 |
|
| 43 |
```bash
|
| 44 |
+
docker run --rm -it -p 8000:8000 -v /path/to/models:/models -e MODEL=/models/gguf-model-name.gguf shaowenchen/llama-cpp-python:0.2.6
|
| 45 |
```
|
| 46 |
|
| 47 |
## Provided images
|
| 48 |
|
| 49 |
+
| Name | Quant method | Compressed Size |
|
| 50 |
+
| -------------------------------------------------- | ------------ | --------------- |
|
| 51 |
+
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q2_K` | Q2_K | 3.68 GB |
|
| 52 |
+
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q3_K` | Q3_K | 4.16 GB |
|
| 53 |
+
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q3_K_L` | Q3_K_L | 4.46 GB |
|
| 54 |
+
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q3_K_S` | Q3_K_S | 3.81 GB |
|
| 55 |
+
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q4_0` | Q4_0 | 4.7 GB |
|
| 56 |
+
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q4_1` | Q4_1 | 5.1 GB |
|
| 57 |
+
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q4_K` | Q4_K | 4.95 GB |
|
| 58 |
+
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q4_K_S` | Q4_K_S | 4.73 GB |
|
| 59 |
+
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q5_0` | Q5_0 | 5.3 GB |
|
| 60 |
+
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q5_1` | Q5_1 | 5.7 GB |
|
| 61 |
+
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q5_K` | Q5_K | 5.5 GB |
|
| 62 |
+
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q5_K_S` | Q5_K_S | 5.3 GB |
|
| 63 |
+
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q6_K` | Q6_K | 6.3 GB |
|
| 64 |
+
| `shaowenchen/colossal-llama-2-7b-base-gguf:Q8_0` | Q8_0 | 8.2 GB |
|
| 65 |
+
| `shaowenchen/colossal-llama-2-7b-base-gguf:full` | full | 14 GB |
|
| 66 |
|
| 67 |
Usage:
|
| 68 |
|