YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
Quantization made by Richard Erkhov.
TinyDolphin-2.8-1.1b_oasst2_chatML_all_Cluster_dare_ties_v1 - GGUF
- Model creator: https://huggingface.co/NickyNicky/
- Original model: https://huggingface.co/NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_all_Cluster_dare_ties_v1/
Original model description:
language: - en - es - ru - zh - de - fr - th - ca - it - ja - pl - eo - eu - vi - fi - hu - ar - nl - da - tr - ko - he - id - cs - bn - sv base_model: - NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_1_V1 - NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_3_V1 - NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_2_V1 tags: - mergekit - merge widget: - text: "<|im_start|>system\nYou are a helpful AI assistant.<|im_end|>\n<|im_start|>user\npodrias escribir un codigo de ejemplo en Python<|im_end|>\n<|im_start|>assistant\n" license: apache-2.0
merged
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the DARE TIES merge method using NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_1_V1 as a base.
Models Merged
The following models were included in the merge:
- NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_3_V1
- NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_2_V1
Configuration
The following YAML configuration was used to produce this model:
base_model:
model:
path: NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_1_V1
dtype: bfloat16
merge_method: dare_ties
slices:
- sources:
- layer_range: [0, 22]
model:
model:
path: NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_1_V1
- layer_range: [0, 22]
model:
model:
path: NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_1_V1
parameters:
density: 0.55
weight: 0.55
- layer_range: [0, 22]
model:
model:
path: NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_2_V1
parameters:
density: 0.55
weight: 0.56
- layer_range: [0, 22]
model:
model:
path: NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_Cluster_3_V1
parameters:
density: 0.55
weight: 0.56
from transformers import (
AutoModelForCausalLM,
AutoTokenizer,
BitsAndBytesConfig,
HfArgumentParser,
TrainingArguments,
pipeline,
logging,
GenerationConfig,
TextIteratorStreamer,
)
import torch
new_model= "NickyNicky/TinyDolphin-2.8-1.1b_oasst2_chatML_all_Cluster_merge_v1"
model = AutoModelForCausalLM.from_pretrained(#f'NickyNicky/{new_model}',
new_model,
device_map="auto",
trust_remote_code=True,
torch_dtype=torch.bfloat16,
low_cpu_mem_usage= True,
# use_flash_attention_2=False,
)
tokenizer = AutoTokenizer.from_pretrained(new_model,
max_length=2048,
trust_remote_code=True,
use_fast = True,
)
tokenizer.pad_token = tokenizer.eos_token
# tokenizer.padding_side = 'left'
tokenizer.padding_side = 'right'
prompt= """<|im_start|>system
You are a helpful AI assistant.<|im_end|>
<|im_start|>user
escribe una historia de amor.<|im_end|>
<|im_start|>assistant
"""
inputs = tokenizer.encode(prompt,
return_tensors="pt",
add_special_tokens=False).cuda()#.to("cuda") # False # True
generation_config = GenerationConfig(
max_new_tokens=700,
temperature=0.5,
top_p=0.9,
top_k=40,
repetition_penalty=1.1, #1.1, # 1.0 means no penalty, > 1.0 means penalty, 1.2 from CTRL paper
do_sample=True,
pad_token_id=tokenizer.eos_token_id,
eos_token_id=tokenizer.eos_token_id,
)
outputs = model.generate(
generation_config=generation_config,
input_ids=inputs,)
# tokenizer.decode(outputs[0], skip_special_tokens=False) #True
print(tokenizer.decode(outputs[0], skip_special_tokens=False))
- Downloads last month
- 3,161
Unable to determine this model's library. Check the
docs
.