metadata
license: apache-2.0
library_name: peft
tags:
- code
- instruct
- code-llama
datasets:
- cognitivecomputations/dolphin-coder
base_model: codellama/CodeLlama-7b-hf
model-index:
- name: codellama_7b_DolphinCoder
results:
- task:
type: text-generation
name: Text Generation
dataset:
name: AI2 Reasoning Challenge (25-Shot)
type: ai2_arc
config: ARC-Challenge
split: test
args:
num_few_shot: 25
metrics:
- type: acc_norm
value: 41.98
name: normalized accuracy
source:
url: >-
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Zangs3011/codellama_7b_DolphinCoder
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: HellaSwag (10-Shot)
type: hellaswag
split: validation
args:
num_few_shot: 10
metrics:
- type: acc_norm
value: 65.5
name: normalized accuracy
source:
url: >-
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Zangs3011/codellama_7b_DolphinCoder
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: MMLU (5-Shot)
type: cais/mmlu
config: all
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 38.11
name: accuracy
source:
url: >-
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Zangs3011/codellama_7b_DolphinCoder
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: TruthfulQA (0-shot)
type: truthful_qa
config: multiple_choice
split: validation
args:
num_few_shot: 0
metrics:
- type: mc2
value: 35.45
source:
url: >-
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Zangs3011/codellama_7b_DolphinCoder
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: Winogrande (5-shot)
type: winogrande
config: winogrande_xl
split: validation
args:
num_few_shot: 5
metrics:
- type: acc
value: 63.61
name: accuracy
source:
url: >-
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Zangs3011/codellama_7b_DolphinCoder
name: Open LLM Leaderboard
- task:
type: text-generation
name: Text Generation
dataset:
name: GSM8k (5-shot)
type: gsm8k
config: main
split: test
args:
num_few_shot: 5
metrics:
- type: acc
value: 9.7
name: accuracy
source:
url: >-
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Zangs3011/codellama_7b_DolphinCoder
name: Open LLM Leaderboard
Finetuning Overview:
Model Used: codellama/CodeLlama-7b-hf
Dataset: cognitivecomputations/dolphin-coder
Dataset Insights:
Dolphin-Coder dataset – a high-quality collection of 100,000+ coding questions and responses. It's perfect for supervised fine-tuning (SFT), and teaching language models to improve on coding-based tasks.
Finetuning Details:
With the utilization of MonsterAPI's no-code LLM finetuner, this finetuning:
- Was achieved with great cost-effectiveness.
- Completed in a total duration of 15hr 31mins for 1 epochs using an A6000 48GB GPU.
- Costed
$31.31
for the entire 1 epoch.
Hyperparameters & Additional Details:
- Epochs: 1
- Total Finetuning Cost: $31.31
- Model Path: codellama/CodeLlama-7b-hf
- Learning Rate: 0.0002
- Data Split: 100% train
- Gradient Accumulation Steps: 128
- lora r: 32
- lora alpha: 64
license: apache-2.0
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
Metric | Value |
---|---|
Avg. | 42.39 |
AI2 Reasoning Challenge (25-Shot) | 41.98 |
HellaSwag (10-Shot) | 65.50 |
MMLU (5-Shot) | 38.11 |
TruthfulQA (0-shot) | 35.45 |
Winogrande (5-shot) | 63.61 |
GSM8k (5-shot) | 9.70 |