someone13574

/

mixtral-8x7b-32kseqlen

like 151

good luck !

Hi,
Do you know if there are any inference scripts for this?

You might be able to get it running using this and tools/run_text_generation_server.py, which is linked to by this repo from mistral's github, but whether or not it really works is unknown. Proper ways to run it will pop up eventually.

https://github.com/dzhulgakov/llama-mistral

Thanks! So it’s not instruct tuned?

Thanks! So it’s not instruct tuned?

No, it's a base model.

Sad. I heard somewhere that MoE models are hard to finetune, is that true?

They released a fine tuned model last time, I'm sure they'll drop a instruct model soon, it's a hype drop a battle of two different generations of new young hungry team up to date and in touch with the younger generation versus the biggest and oldest in the industry, as far as it being hard to fine tune I think it just depends on your area of focus and who your asking.

Inference code: https://github.com/open-compass/MixtralKit
Evaluation results will be updated soon

LLaMA2-Accessory now supports the inference and instruction finetuning (both full-parameter and PEFT like LoRA) of mixtral-8x7b-32kseqlen. It supports the load balancing loss and will add more MoE support soon. The document is here