RvKy commited on
Commit
7858a4d
1 Parent(s): 72ff902

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -4
README.md CHANGED
@@ -9,7 +9,7 @@ If no comment is made, None data regarding the product or shipping will be retur
9
 
10
  Used to quantization method like this ;
11
 
12
- >>
13
  import torch
14
  #quantizasyon yöntemi 4 bit hassasiyetiyle yapacak.
15
  quant_config = BitsAndBytesConfig(
@@ -18,12 +18,12 @@ quant_config = BitsAndBytesConfig(
18
  bnb_4bit_compute_dtype=torch.float16, #maliyeti de düşürmüş olduk, hızı da artırdık
19
  bnb_4bit_use_double_quant=True, # True model increases accuracy but causes it to work more costly.
20
  ) #np4 saves more memory
21
- >>
22
 
23
 
24
  Here.. This code contains important steps to optimize LLM's memory usage and processing time.
25
  It is important to improve the performance and resource utilization of the model.
26
- >>
27
  #20GB model
28
  model = AutoModelForCausalLM.from_pretrained(
29
  pretrained_model_name_or_path=base_model, #used "meta-llama/Meta-Llama-3-8B-Instruct"
@@ -34,4 +34,4 @@ model = AutoModelForCausalLM.from_pretrained(
34
  )
35
  model.config.use_cache = False
36
  model.config.pretraining_tp = 1
37
- >>
 
9
 
10
  Used to quantization method like this ;
11
 
12
+
13
  import torch
14
  #quantizasyon yöntemi 4 bit hassasiyetiyle yapacak.
15
  quant_config = BitsAndBytesConfig(
 
18
  bnb_4bit_compute_dtype=torch.float16, #maliyeti de düşürmüş olduk, hızı da artırdık
19
  bnb_4bit_use_double_quant=True, # True model increases accuracy but causes it to work more costly.
20
  ) #np4 saves more memory
21
+
22
 
23
 
24
  Here.. This code contains important steps to optimize LLM's memory usage and processing time.
25
  It is important to improve the performance and resource utilization of the model.
26
+
27
  #20GB model
28
  model = AutoModelForCausalLM.from_pretrained(
29
  pretrained_model_name_or_path=base_model, #used "meta-llama/Meta-Llama-3-8B-Instruct"
 
34
  )
35
  model.config.use_cache = False
36
  model.config.pretraining_tp = 1
37
+