32k or 128k?

#3
by ChuckMcSneed - opened

config.json says "max_position_embeddings": 32768, and readme says 128k

+1
To run the model at 128k, should we extend the max_position_embeddings or is there some sort of RoPE Scaling configuration that we should apply?

+1
Currently only support 32k with vllm deployment test.

Dear developers. Please clarify. In your blog you write about 128k, but the model says “max_position_embeddings”: 32768.
What to believe?

Mistral AI_ org

Hi! Should be fixed soon: https://huggingface.co/mistralai/Mistral-Large-Instruct-2407/discussions/11
Its indeed 128k!! 🔥

ChuckMcSneed changed discussion status to closed

Sign up or log in to comment