|
--- |
|
license: apache-2.0 |
|
tags: |
|
- requests |
|
- gguf |
|
- quantized |
|
--- |
|
<!-- |
|
> [!WARNING] |
|
> **Notice:** <br> |
|
> Requests are paused at the moment due to unforseen circumstances. |
|
--> |
|
|
|
 |
|
|
|
|
|
> [!TIP] |
|
> **Quant-Requests are open.** <br> |
|
> I apologize for disrupting your experience. <br> |
|
> If you **want** and you are able to... <br> |
|
> [**You can support my various endeavors here (Ko-fi).**](https://ko-fi.com/Lewdiculous) <br> |
|
> Eventually I want to have a proper infrastructure for these. <br> |
|
> In the meantime I'll be working to make do with the resources at hand at the time. <br> |
|
|
|
|
|
# Welcome to my GGUF-IQ-Imatrix Model Quantization Requests card! |
|
|
|
Please read everything. |
|
|
|
This card is meant only to request GGUF-IQ-Imatrix quants for models that meet the requirements bellow. |
|
|
|
**Requirements to request GGUF-Imatrix model quantizations:** |
|
|
|
For the model: |
|
- Maximum model parameter size of ~~11B~~ **12B**. Small note is that models sizes larger than 8B parameters may take longer to process and upload than the smaller ones.<br> |
|
*At the moment I am unable to accept requests for larger models due to hardware/time limitations.* <br> |
|
*Preferably for Mistral and LLama-3 based models in the creative/roleplay niche.* <br> |
|
*If you need quants for a bigger model, you can try requesting at [mradermacher's](https://huggingface.co/mradermacher/model_requests). He's doing an amazing work.* |
|
|
|
Important: |
|
- Fill the request template as outlined in the next section. |
|
|
|
#### How to request a model quantization: |
|
|
|
1. Open a [**New Discussion**](https://huggingface.co/Lewdiculous/Model-Requests/discussions/new) titled "`Request: Model-Author/Model-Name`", for example, "`Request: Nitral-AI/Infinitely-Laydiculous-7B`", without the quotation marks. |
|
|
|
2. Include the following template in your new discussion post, you can just copy and paste it as is, and fill the required information by replacing the {{placeholders}} ([example request here](https://huggingface.co/Lewdiculous/Model-Requests/discussions/1)): |
|
|
|
``` |
|
**[Required] Model name:** <br> |
|
{{replace-this}} |
|
|
|
**[Required] Model link:** <br> |
|
{{replace-this}} |
|
|
|
**[Required] Brief description:** <br> |
|
{{replace-this}} |
|
|
|
**[Required] An image/direct image link to represent the model (square shaped):** <br> |
|
{{replace-this}} |
|
|
|
**[Optional] Additonal quants (if you want any):** <br> |
|
|
|
<!-- Keep in mind that anything bellow I/Q3 isn't recommended, --> |
|
<!-- since for these smaller models the results will likely be --> |
|
<!-- highly incoherent rendering them unusable for your needs. --> |
|
|
|
|
|
Default list of quants for reference: |
|
|
|
"IQ3_M", "IQ3_XXS", |
|
"Q4_K_M", "Q4_K_S", "IQ4_XS", |
|
"Q5_K_M", "Q5_K_S", |
|
"Q6_K", |
|
"Q8_0" |
|
|
|
``` |
