Llama-Salad-4x8B-V2

Changes in V2:

• Swapped Tess-2.0-Llama-3-8B for Llama-3-8B-Synthia-v3.5
• Swapped L3-8B-Stheno-v3.1 for Llama-3-Soliloquy-8B-v2
• Removed Llama3-OpenBioLLM-8B and added opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5

V2 has improvements in all areas from V1; it's not a massive improvement, but I can confidently say it's a direct upgrade. Llama-3-8B-Synthia-v3.5 is better than Tess-2.0-Llama-3-8B in every way; Llama-3-Soliloquy-8B-v2 is more intelligent than L3-8B-Stheno-v3.1 and has less bias towards NSFW content; and the inclusion of opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5 has greatly improved its storytelling and narration abilities.

I really like the model selection in this one, so I don't know how much more I can improve if I make another 4x8B merge. If I were to make a V3, swapping Meta-Llama-3-8B-Instruct would likely be the only change. I will try my hand at making an 8x8B merge in the future, but I still need to find some models to fill the gaps; making sure there's no routing conflicts between 8 different models at once will be the biggest challenge.

Quantization Formats

GGUF

• Static:
  • https://huggingface.co/mradermacher/Llama-Salad-4x8B-V2-GGUF
• Imatrix:
  • https://huggingface.co/mradermacher/Llama-Salad-4x8B-V2-i1-GGUF

Details

• License: llama3
• Instruct Format: llama-3
• Context Size: 8K

Models Used

• Meta-Llama-3-8B-Instruct
• Llama-3-8B-Synthia-v3.5
• Llama-3-Soliloquy-8B-v2
• opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5

Merge Config

base_model: NousResearch/Meta-Llama-3-8B-Instruct
gate_mode: hidden
dtype: bfloat16
experts_per_token: 2
experts:
  - source_model: NousResearch/Meta-Llama-3-8B-Instruct
    positive_prompts:
    - "summarize"
    - "paraphrase"
    - "explain"
    - "define"
    - "translate"
    - "multilingual"
    - "chat"
    - "conversation"
  - source_model: migtissera/Llama-3-8B-Synthia-v3.5
    positive_prompts:
    - "programming language"
    - "JavaScript"
    - "Python programming language"
    - "Rust programming language"
    - "CSS markup styling language"
    - "math"
    - "code"
    - "step-by-step"
    - "logical reasoning"
  - source_model: openlynn/Llama-3-Soliloquy-8B-v2
    positive_prompts:
    - "roleplay"
    - "erotic roleplay"
    - "characters"
    - "scene"
    - "opinion"
  - source_model: dreamgen-preview/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5
    positive_prompts:
    - "creative writing"
    - "storytelling"
    - "narration"
    - "narrative setting"
    - "narrative plot"
    - "narrative exposition"
    - "narrative theme"
    - "narrative climax"