Hugging Face's logo

Kooten
/
MiquMaid-v1-70B-IQ2-GGUF

GGUF
Inference Endpoints
Model card Files Community
2
MiquMaid-v1-70B-IQ2-GGUF
File size: 762 Bytes 
72e6686
 
 
 
 
1747456
 
72e6686
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1747456
72e6686
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
 
# MiquMaid-v1-70B IQ2

## Description
2bit imatrix GGUF quants of [NeverSleep/MiquMaid-v1-70B](https://huggingface.co/NeverSleep/MiquMaid-v1-70B)

Imatrix generated from q8 of MiquMaid, 2000 chunks at 500 ctx. Dataset was Wikitext.

## Other quants:
EXL2: [3.5bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-3.5bpw-exl2), [3bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-3bpw-exl2), [2.4bpw](https://huggingface.co/Kooten/MiquMaid-v1-70B-2.4bpw-exl2)

GGUF:

[2bit Imatrix GGUF](https://huggingface.co/Kooten/MiquMaid-v1-70B-IQ2-GGUF)

[NeverSleeps Official Quants](https://huggingface.co/NeverSleep/MiquMaid-v1-70B-GGUF)

### Custom format:
```
### Instruction:
{system prompt}

### Input:
{input}

### Response:
{reply}
```

## Contact
Kooten on discord