|
--- |
|
license: bigcode-openrail-m |
|
--- |
|
|
|
# GPTQ-for-StarCoder |
|
Visit [GPTQ-for-SantaCoder](https://github.com/mayank31398/GPTQ-for-SantaCoder) for instructions on how to use the model weights here. |
|
If you want 4-bit weights, visit [starcoder-GPTQ-4bit-128g](https://huggingface.co/mayank31398/starcoder-GPTQ-4bit-128g). |
|
|
|
## Results |
|
| StarCoder | Bits | group-size | memory(MiB) | wikitext2 | ptb | c4 | stack | checkpoint size(MB) | |
|
| -------------------------------------------------- | ---- | ---------- | ----------- | --------- | ---------- | ---------- | ---------- | ------------------- | |
|
| FP32 | 32 | - | | 10.801 | 16.425 | 13.402 | 1.738 | 59195 | |
|
| BF16 | 16 | - | | 10.807 | 16.424 | 13.408 | 1.739 | 29597 | |
|
| [GPTQ](https://arxiv.org/abs/2210.17323) | 8 | 128 | | 10.805 | 15.453 | 13.408 | 1.739 | 16163 | |
|
| [GPTQ](https://arxiv.org/abs/2210.17323) | 4 | 128 | | 10.989 | 16.839 | 13.676 | 1.757 | 8877 | |
|
|
|
# License |
|
The model is licenses under the CodeML Open RAIL-M v0.1 license. You can find the full license [here](https://huggingface.co/spaces/bigcode/license). |
|
|
|
# Acknowledgements |
|
Thanks to everyone in BigCode who worked so hard to create these code models. |
|
|
