Edit model card

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Model Card for LDCC-Instruct-Llama-2-ko-13B-v1.4

LDCC-Instruct-Llama-2-ko-13B-v1.4 is a continuation in a series of language models designed to serve as efficient assistants. This fifth iteration is an enhanced version of its predecessor, LDCC/LDCC-Instruct-Llama-2-ko-13B-v1.0. We applied NEFTune noise embeddings to fine-tuning. This has been proven to improve model performances for instrcution fine-tuning. Additionally, it underwent fine-tuning on a combination of publicly available and synthetic datasets through the use of Direct Preference Optimization (DPO). Interestingly, we observed an uplift in performance on the MT Bench when the intrinsic alignment of these datasets was eliminated, resulting in a more effective assistant model.

Developed by : Wonchul Kim (Lotte Data Communication AI Technical Team)

Hardware and Software

Base Model : beomi/llama-2-koen-13b

Training Data

The LDCC-Instruct-Llama-2-ko-13B model was trained with publicly accessible Korean/English data sources. For its fine-tuning, we utilized other public data and underwent some processing and refinement.

We did not incorporate any client data owned by Lotte Data Communication.

Prompt Template

### Prompt:
{instruction}

### Answer:
{output}

License

LICENSE.txt

Downloads last month
0
Safetensors
Model size
13.2B params
Tensor type
BF16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Space using LDCC/LDCC-Instruct-Llama-2-ko-13B-v1.4 1