Hugging Face's logo

abideen
/
starcoder2-chat

Text Generation
Transformers
Safetensors
English
starcoder2
code
starcoder
bigcode
sft
7b
text-generation-inference
Inference Endpoints
Model card Files Community
1
Edit model card

Starcoder-2-chat

image/jpeg

Starcoder-2-chat is an instruction fine-tuned of bigcode/starcoder2-7b using the glaiveai/glaive-code-assistant-v2 dataset using LoRA.

🏆 Evaluation results

Thanks to Muhammad Bin Usman for running evals on Starcoder2-chat.

HUMANEVAL 

0.3231707317073171

HUMANEVALPLUS 

0.25609756097560976

INSTRUCT-HUMANEVAL 

0.3231707317073171

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-7
  • train_batch_size: 2
  • eval_batch_size: Not specified
  • seed: Not specified
  • gradient_accumulation_steps: 8
  • total_train_batch_size: Not specified
  • optimizer: PagedAdamW with 32-bit precision
  • lr_scheduler_type: Cosine
  • lr_scheduler_warmup_steps: 100
  • training_epoch: 1

Framework versions

  • Transformers 4.39.0.dev0
  • Peft 0.9.1.dev0
  • Datasets 2.18.0
  • torch 2.2.0
  • accelerate 0.27.2
Downloads last month
8
Safetensors
Model size
7.17B params
Tensor type
F32
·
Text Generation
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Finetuned from

Collection including abideen/starcoder2-chat