aberrio commited on
Commit
adc125a
1 Parent(s): 3cea8fc

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -4
README.md CHANGED
@@ -50,11 +50,21 @@ Mixtral 8x7B is a generative Sparse Mixture of Experts (SMoE) model designed to
50
 
51
  ### Core Library
52
 
53
- Mixtral 8x7B Instruct can be deployed using `vLLM` or `transformers`. Current support focuses on Hugging Face `transformers` for initial integrations.
54
 
55
- **Primary Framework**: `transformers`
56
- **Alternate Framework**: `vLLM` (for specialized inference optimizations)
57
- **Model Availability**: Source weights and pre-converted formats are available under [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0).
 
 
 
 
 
 
 
 
 
 
58
 
59
  ### Safety and Responsible Use
60
 
 
50
 
51
  ### Core Library
52
 
53
+ Mixtral 8x7B Instruct is supported by multiple libraries to ensure flexibility for deployment and development. The primary frameworks include:
54
 
55
+ - **Primary Framework**: `llama.cpp`
56
+ - **Alternate Frameworks**:
57
+ - `transformers` for initial integration into Hugging Face's ecosystem.
58
+ - `vLLM` for highly optimized inference with low-latency serving.
59
+
60
+ You can access the model components and libraries here:
61
+
62
+ - **Model Base**: [mistralai/Mixtral-8x7B-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-v0.1)
63
+ - **Common Utilities**: [mistralai/mistral-common](https://github.com/mistralai/mistral-common)
64
+ - **Inference Optimization**: [mistralai/mistral-inference](https://github.com/mistralai/mistral-inference)
65
+ - **Quantization Support**: [ggerganov/llama.cpp](https://github.com/ggerganov/llama.cpp)
66
+
67
+ These resources provide a complete ecosystem for deployment, fine-tuning, and scaling sparse mixture models.
68
 
69
  ### Safety and Responsible Use
70