Edit model card

Open_Gpt4_v0.2

image/jpeg

This model is a TIES merger of Mixtral-8x7B-Instruct-v0.1 and bagel-8x7b-v0.2 with MixtralOrochi8x7B being the Base model.

I was very impressed with MixtralOrochi8x7B performance and multifaceted usecases as it is already a merger of many usefull Mixtral models such as Mixtral instruct, Noromaid-v0.1-mixtral, openbuddy-mixtral and possibly other models that were not named. My goal was to expand the models capabilities and make it even more useful of a model, maybe even competitive with closed source models like Gpt-4. But for that more testing is required. I hope the community can help me determine if its deserving of its name. 😊

This is the second iteration of this model, using better models in the merger to improve performance (hopefully).

Base model:

Merged models:

Instruct template: Alpaca

Merger config:

models:
  - model: Mixtral-8x7B-Instruct-v0.1
    parameters:
      density: .5
      weight: .7
  - model: bagel-8x7b-v0.2
    parameters:
      density: .5
      weight: 1


merge_method: ties
base_model: MixtralOrochi8x7B
parameters:
  normalize: true
  int8_mask: true
dtype: float16
Downloads last month
7
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.