Yi 1.5 34B Chat Int8 GPTQ
#8
by
cduk
- opened
Could you please add an Int8 GPTQ quantization of this Yi 1.5 34B Chat model for easy deployment on vLLM? Thanks.
cduk
changed discussion title from
Int8 GPTQ
to Yi 1.5 34B Chat Int8 GPTQ
Hi cduk,
We don't plan to provide an official this time. ModelScope community has provided AWQ and GTPQ variants using their SWIFT framework, check here: https://www.modelscope.cn/models/tclf90/Yi-1.5-34B-Chat-16K-GPTQ-Int4/summary
lorinma
changed discussion status to
closed