metadata
base_model:
- cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser
- Intel/neural-chat-7b-v3-3
- meta-math/MetaMath-Mistral-7B
tags:
- mergekit
- merge
Ethux 7B merge
This model is a test of what is possible with mergekit, this model does not have the best performance and quality responses yet.
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the TIES merge method using Intel/neural-chat-7b-v3-3 as a base.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
# Main configuration for merging pre-trained models
models:
# - model: Intel/neural-chat-7b-v3-3
# parameters:
# density: 0.9 # High density since generally good for most tasks
# weight: 0.4 # Lower weight compared to openchat due to slight inferiority
- model: meta-math/MetaMath-Mistral-7B
parameters:
density: 0.6 # Moderate density focusing mainly on mathematical prowess
weight:
- filter: mlp
value: 0.5
- value: 0
- model: cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser
parameters:
density: 0.8 # Balanced density emphasizing coding skills over chitchat
weight: 0.4 # Slightly higher weight acknowledging superior coding ability
base_model: Intel/neural-chat-7b-v3-3
merge_method: ties
parameters:
normalize: true
int8_mask: false
dtype: float16