SharryOG commited on
Commit
338120a
1 Parent(s): d06e3a8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -50
README.md CHANGED
@@ -1,50 +0,0 @@
1
- ---
2
- base_model:
3
- - mlabonne/Meta-Llama-3-120B-Instruct
4
- library_name: transformers
5
- tags:
6
- - mergekit
7
- - merge
8
-
9
- ---
10
- # merge
11
-
12
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
13
-
14
- ## Merge Details
15
- ### Merge Method
16
-
17
- This model was merged using the passthrough merge method.
18
-
19
- ### Models Merged
20
-
21
- The following models were included in the merge:
22
- * [mlabonne/Meta-Llama-3-120B-Instruct](https://huggingface.co/mlabonne/Meta-Llama-3-120B-Instruct)
23
-
24
- ### Configuration
25
-
26
- The following YAML configuration was used to produce this model:
27
-
28
- ```yaml
29
- dtype: float16
30
- merge_method: passthrough
31
- slices:
32
- - sources:
33
- - layer_range: [0, 8]
34
- model: mlabonne/Meta-Llama-3-120B-Instruct
35
- - sources:
36
- - layer_range: [4, 12]
37
- model: mlabonne/Meta-Llama-3-120B-Instruct
38
- - sources:
39
- - layer_range: [8, 16]
40
- model: mlabonne/Meta-Llama-3-120B-Instruct
41
- - sources:
42
- - layer_range: [12, 20]
43
- model: mlabonne/Meta-Llama-3-120B-Instruct
44
- - sources:
45
- - layer_range: [16, 24]
46
- model: mlabonne/Meta-Llama-3-120B-Instruct
47
- - sources:
48
- - layer_range: [20, 28]
49
- model: mlabonne/Meta-Llama-3-120B-Instruct
50
- ```