cloudyu

/

Mixtral_34Bx2_MoE_60B

like 111

for 4bit quant, it need about 40G

I'm currently using a 3090, and if I quantize it with Q5_k_m, it works, although it's slow.