inflatebot commited on
Commit
c31969a
1 Parent(s): b1ec896

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +65 -26
README.md CHANGED
@@ -10,16 +10,20 @@ tags:
10
  ![Made with NovelAI](https://huggingface.co/inflatebot/MN-12B-Mag-Mell-R1/resolve/main/magmell.png)
11
  *[Welcome, brave one; you've come a long mile.](https://www.youtube.com/watch?v=dgGEuC1F3oE)*
12
 
13
- # magmell-r1g
14
 
15
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
16
 
17
  ## Merge Details
18
- Multi-stage SLERP merge, DARE-TIES'd together. Intended to be a general purpose "Best of Nemo" model for any fictional, creative use case. Inspired by hyper-merges like Tiefighter and Umbral Mind.
19
 
20
- Hero (RP, kink/trope coverage): Chronos Gold, Sunrose.
21
- Monk (Intelligence, groundedness): Bophades, Wissenschaft.
22
- Deity (Prose, flair): Gutenberg, Magnum 2.5 KTO.
 
 
 
 
23
  ### Merge Method
24
 
25
  This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [IntervitensInc/Mistral-Nemo-Base-2407-chatml](https://huggingface.co/IntervitensInc/Mistral-Nemo-Base-2407-chatml) as a base.
@@ -33,29 +37,64 @@ The following models were included in the merge:
33
 
34
  ### Configuration
35
 
36
- The following YAML configuration was used to produce this model:
37
 
 
38
  ```yaml
39
- base_model: IntervitensInc/Mistral-Nemo-Base-2407-chatml
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
40
  merge_method: dare_ties
41
- slices:
42
- - sources:
43
- - layer_range: [0, 40]
44
- model: /workspace/models/magmell-r1/monk
45
- parameters:
46
- density: 0.7
47
- weight: 0.5
48
- - layer_range: [0, 40]
49
- model: /workspace/models/magmell-r1/hero
50
- parameters:
51
- density: 0.9
52
- weight: 1.0
53
- - layer_range: [0, 40]
54
- model: /workspace/models/magmell-r1/deity
55
- parameters:
56
- density: 0.5
57
- weight: 0.7
58
- - layer_range: [0, 40]
59
- model: IntervitensInc/Mistral-Nemo-Base-2407-chatml
60
  tokenizer_source: base
61
  ```
 
10
  ![Made with NovelAI](https://huggingface.co/inflatebot/MN-12B-Mag-Mell-R1/resolve/main/magmell.png)
11
  *[Welcome, brave one; you've come a long mile.](https://www.youtube.com/watch?v=dgGEuC1F3oE)*
12
 
13
+ # MN-12B-Mag-Mell-R1
14
 
15
  This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
16
 
17
  ## Merge Details
18
+ Multi-stage SLERP merge, DARE-TIES'd together. Intended to be a general purpose "Best of Nemo" model for any fictional, creative use case. Inspired by hyper-merges like [Tiefighter]https://huggingface.co/KoboldAI/LLaMA2-13B-Tiefighter) and [Umbral Mind.](https://huggingface.co/Casual-Autopsy/L3-Umbral-Mind-RP-v2.0-8B)
19
 
20
+ Mag Mell is composed of 3 intermediate parts:
21
+
22
+ Hero (RP, kink/trope coverage): [Chronos Gold](https://huggingface.co/elinas/Chronos-Gold-12B-1.0), [Sunrose](https://huggingface.co/Fizzarolli/MN-12b-Sunrose).
23
+
24
+ Monk (Intelligence, groundedness): [Bophades](https://huggingface.co/nbeerbower/mistral-nemo-bophades-12B), [Wissenschaft](https://huggingface.co/nbeerbower/mistral-nemo-wissenschaft-12B).
25
+
26
+ Deity (Prose, flair): [Gutenberg v4](https://huggingface.co/nbeerbower/mistral-nemo-gutenberg-12B-v4), [Magnum 2.5 KTO](https://huggingface.co/anthracite-org/magnum-v2.5-12b-kto).
27
  ### Merge Method
28
 
29
  This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [IntervitensInc/Mistral-Nemo-Base-2407-chatml](https://huggingface.co/IntervitensInc/Mistral-Nemo-Base-2407-chatml) as a base.
 
37
 
38
  ### Configuration
39
 
40
+ The following YAML configurations were used to produce this model:
41
 
42
+ #### Monk:
43
  ```yaml
44
+ models:
45
+ - model: nbeerbower/mistral-nemo-bophades-12B
46
+ - model: nbeerbower/mistral-nemo-wissenschaft-12B
47
+ merge_method: slerp
48
+ base_model: nbeerbower/mistral-nemo-bophades-12B
49
+ parameters:
50
+ t: [0.1, 0.2, 0.4, 0.6, 0.6, 0.4, 0.2, 0.1]
51
+ dtype: bfloat16
52
+ tokenizer_source: basel
53
+ ```
54
+
55
+ #### Hero:
56
+ ```yaml
57
+ models:
58
+ - model: elinas/Chronos-Gold-12B-1.0
59
+ - model: Fizzarolli/MN-12b-Sunrose
60
+ merge_method: slerp
61
+ base_model: elinas/Chronos-Gold-12B-1.0
62
+ parameters:
63
+ t: [0.1, 0.2, 0.4, 0.6, 0.6, 0.4, 0.2, 0.1]
64
+ dtype: bfloat16
65
+ tokenizer_source: base
66
+ ```
67
+
68
+ #### Deity:
69
+ ```
70
+ name: deity # Deity - Prose, flair.
71
+ models:
72
+ - model: nbeerbower/mistral-nemo-gutenberg-12B-v4
73
+ - model: anthracite-org/magnum-12b-v2.5-kto
74
+ merge_method: slerp
75
+ base_model: nbeerbower/mistral-nemo-gutenberg-12B-v4
76
+ parameters:
77
+ t: [0, 0.1, 0.2, 0.25, 0.25, 0.2, 0.1, 0]
78
+ dtype: bfloat16
79
+ tokenizer_source: base
80
+ ```
81
+
82
+ #### Mag Mell:
83
+ ```yaml
84
+ models:
85
+ - model: monk
86
+ parameters:
87
+ density: 0.7
88
+ weight: 0.5
89
+ - model: hero
90
+ parameters:
91
+ density: 0.9
92
+ weight: 1
93
+ - model: deity
94
+ parameters:
95
+ density: 0.5
96
+ weight: 0.7
97
  merge_method: dare_ties
98
+ base_model: IntervitensInc/Mistral-Nemo-Base-2407-chatml
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
99
  tokenizer_source: base
100
  ```