Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
#72
by
Kenkentron
- opened
Thanks for the model!
I encounter the following when loading the tokenizer:
from transformers import AutoTokenizer
checkpoint_path = "microsoft/Phi-3-mini-4k-instruct"
tokenizer = AutoTokenizer.from_pretrained(checkpoint_path)
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Does this mean I would have to free the embedding layers when doing fine-tuning with LoRA?
Thanks!