PEFT
code
instruct
mistral
souvik0306 commited on
Commit
be9b464
1 Parent(s): 446defd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -18,7 +18,7 @@ license: apache-2.0
18
 
19
  #### Dataset Insights:
20
 
21
- [Dolphin-Coder](https://huggingface.co/datasets/cognitivecomputations/dolphin-coder) Dolphin-Coder dataset – a high-quality collection of 100,000+ coding questions and responses. It's perfect for supervised fine-tuning (SFT), and teaching language models to improve on coding-based tasks.
22
 
23
  #### Finetuning Details:
24
 
@@ -35,9 +35,9 @@ With the utilization of [MonsterAPI](https://monsterapi.ai)'s [no-code LLM finet
35
  - **Model Path:** tiiuae/falcon-7b
36
  - **Learning Rate:** 0.0002
37
  - **Data Split:** 100% train
38
- - **Gradient Accumulation Steps:** 64
39
- - **lora r:** 64
40
- - **lora alpha:** 16
41
 
42
  ---
43
  license: apache-2.0
 
18
 
19
  #### Dataset Insights:
20
 
21
+ [Dolphin-Coder](https://huggingface.co/datasets/cognitivecomputations/dolphin-coder) dataset – a high-quality collection of 100,000+ coding questions and responses. It's perfect for supervised fine-tuning (SFT), and teaching language models to improve on coding-based tasks.
22
 
23
  #### Finetuning Details:
24
 
 
35
  - **Model Path:** tiiuae/falcon-7b
36
  - **Learning Rate:** 0.0002
37
  - **Data Split:** 100% train
38
+ - **Gradient Accumulation Steps:** 128
39
+ - **lora r:** 32
40
+ - **lora alpha:** 64
41
 
42
  ---
43
  license: apache-2.0