ANJIRRR
This model aims to achieve the human-like responses of the Halu Blackroot, the no refusal tendencies of the Halu OAS, and the smartness of the Standard Halu.
GGUF: STATIC/IMATRIX made available by mradermacher
![](https://huggingface.co/Hastagaras/Anjir-8B-L3/resolve/main/anjir.png)
Model Details:
Anjrit: This model is similar to my Halu Blackroot model, but instead of using the standard version, this model uses the OAS version.
Anying: This model is also similar to the Halu Blackroot, but instead of using the model stock, I merged the Blackroot lora manually with a very low alpha.
Both models have downsides. The Anjrit model lacks coherency, while the Anying model lacks a human-like responses.
I decided to merge both models with the following method:
First, I compared the response from each layer of both models using the baukit notebook.
After comparing both, it seems that around the bottom layer, the Anjrit model is better, perhaps because it is unhinged.
From the bottom to the middle layer, the Anjrit is still better, but the Anying seems smarter.
At the middle layer, both seem equal, but again, the Anjrit is unhinged, so I prefer this one.
From the middle to the top layer, the Anying is better. It is smarter, and the response is more structured.
The top layer of the Anjrit model is better since the model itself is orthogonalized, so I prefer this one.
Then I performed slerp with the following configuration. I don't know if this is really how the slerp merge works, so let's just say this is an experimental merge. Maybe I will try the other merge methods for future experiments
Configuration
The following YAML configuration was used to produce this model:
models:
- model: Hastagaras/anjrit
- model: Hastagaras/anying
merge_method: slerp
base_model: Hastagaras/anjrit
dtype: bfloat16
parameters:
t: [0.12, 0.17, 0.29, 0.44, 0.26]
SAMPLER:
You can start with this and tweak it
- TEMP: 1.0
- TOP_P: 0.95
- TOP_K: 100
- MIN_P: 0.05
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
Metric | Value |
---|---|
Avg. | 69.07 |
AI2 Reasoning Challenge (25-Shot) | 63.57 |
HellaSwag (10-Shot) | 84.15 |
MMLU (5-Shot) | 67.67 |
TruthfulQA (0-shot) | 52.67 |
Winogrande (5-shot) | 78.61 |
GSM8k (5-shot) | 67.78 |
- Downloads last month
- 8
Model tree for Hastagaras/Anjir-8B-L3
Base model
Hastagaras/anjritSpaces using Hastagaras/Anjir-8B-L3 3
Collection including Hastagaras/Anjir-8B-L3
Evaluation results
- normalized accuracy on AI2 Reasoning Challenge (25-Shot)test set Open LLM Leaderboard63.570
- normalized accuracy on HellaSwag (10-Shot)validation set Open LLM Leaderboard84.150
- accuracy on MMLU (5-Shot)test set Open LLM Leaderboard67.670
- mc2 on TruthfulQA (0-shot)validation set Open LLM Leaderboard52.670
- accuracy on Winogrande (5-shot)validation set Open LLM Leaderboard78.610
- accuracy on GSM8k (5-shot)test set Open LLM Leaderboard67.780