|
--- |
|
library_name: peft |
|
license: apache-2.0 |
|
datasets: |
|
- b-mc2/sql-create-context |
|
language: |
|
- en |
|
pipeline_tag: text-generation |
|
--- |
|
## Training procedure |
|
|
|
|
|
The following `bitsandbytes` quantization config was used during training: |
|
- quant_method: bitsandbytes |
|
- load_in_8bit: False |
|
- load_in_4bit: True |
|
- llm_int8_threshold: 6.0 |
|
- llm_int8_skip_modules: None |
|
- llm_int8_enable_fp32_cpu_offload: False |
|
- llm_int8_has_fp16_weight: False |
|
- bnb_4bit_quant_type: nf4 |
|
- bnb_4bit_use_double_quant: True |
|
- bnb_4bit_compute_dtype: bfloat16 |
|
|
|
|
|
## Model Description |
|
|
|
This is an SFT(Supervised Fine-Tuned) Model meant for SQl-based text generation tasks. We have used the LoRa(Low-Ranking Adaptors) method for Fine-Tuning. |
|
|
|
|
|
## Model Summary |
|
|
|
train/loss : 0.4354 |
|
train/learning_rate: 0.00017567567567567568 |
|
train/epoch : 5.0 |
|
train/global_step : 10 |
|
|
|
|
|
## Inference Code |
|
|
|
After doing necessary imports |
|
|
|
device_map = {"": 0} |
|
model_id = "mistralai/Mistral-7B-v0.1" |
|
new_model = "Akil15/mistral_SQL_v.0.1" |
|
|
|
# Reload model in FP16 and merge it with LoRA weights |
|
|
|
base_model = AutoModelForCausalLM.from_pretrained( |
|
model_id, |
|
torch_dtype=torch.float16, |
|
device_map=device_map, |
|
) |
|
|
|
model = PeftModel.from_pretrained(base_model, new_model) |
|
model = model.merge_and_unload() |
|
|
|
# Reload tokenizer to save it |
|
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) |
|
tokenizer.pad_token = tokenizer.eos_token |
|
tokenizer.padding_side = "right" |
|
|
|
sample text(example): |
|
|
|
text = """Question: How many vegetable farms with over 100 acres of cultivated land utilize organic farming methods, and what is the average yield per acre for these farms? |
|
Context:CREATE TABLE vegetable_farm (Acres INTEGER,Organic BOOLEAN,Yield_Per_Acre DECIMAL);""" |
|
|
|
text = input() |
|
inputs = tokenizer(text, return_tensors="pt").to(device) |
|
outputs = model.generate(**inputs, max_new_tokens=20) |
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
|
|
|
Note: Change the max_new_tokens length based on the question-context text input or just define it to 100 |
|
|
|
### Framework versions |
|
|
|
|
|
- PEFT 0.4.0 |