---
library_name: peft
base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
license: apache-2.0
datasets:
- vicgalle/alpaca-gpt4
language:
- en
pipeline_tag: conversational
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->
TinyLlama/TinyLlama-1.1B-Chat-v1.0 sft on alpaca dataset using LoRA


## Model Details

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** [https://github.com/bytebarde/llm_alpaca](https://github.com/bytebarde/llm_alpaca)


## Training Details

### Training Procedure 

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->


#### Training Hyperparameters

- **Training regime:** [fp16 mixed precision] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
- **Per device train batch size:** 4
- **Epoch:** 10
- **Loss:** 0.9044

### Framework versions

- PEFT 0.7.1