How do I actually *use* this?
#23
by
nightsd01
- opened
Sorry for the n00b question. I'm trying to evaluate this model with my own system + user prompts to see how well it does. The evaluation script provided doesn't seem to be working for me, it just gets stuck on this step:
2024-06-18:20:09:02,402 INFO [evaluator.py:357] Running loglikelihood requests
Running loglikelihood requests: 0%| | 0/81168 [00:00<?, ?it/s]
I get the feeling that this "evaluation" is just going to produce performance statistics and not actually allow me to interact with the model?
@nightsd01 I guess what you're trying to achieve is to prompt the model with a custom instruction, right?
The "Usage" section of README covers how to perform one-off inference:
$ python generate_openelm.py \
--model apple/OpenELM-3B-Instruct \
--hf_access_token [HF_ACCESS_TOKEN] \
--prompt 'Once upon a time there was' \
--generate_kwargs repetition_penalty=1.2
Once upon a time there was a little girl named Rosie. Rosie loved to play dress-up, and her favorite costume was a princess dress. Her mother dressed Rosie in the princess dress every day, but Rosie wanted to wear it on special occasions, too.
One day Rosie's mother told Rosie that Prince Charming would arrive at their house later that evening. Rosie couldn't wait! Prince Charming would surely ask Rosie to marry him, and she would wear her beautiful princess dress for their wedding. Rosie ran upstairs to get dressed.
When Prince Charming arrived, Rosie wore her princess dress and tiara. Her mother helped her with her makeup, and Rosie practiced her curtsy. Prince Charming smiled and kissed Rosie's hand. Rosie hoped Prince Charming would ask her to marry him, but Prince Charming shook Rosie's hand instead. Prince Charming explained that Rosie's father had died when Rosie was very young, and Prince Charming wanted Rosie to choose her own husband. Rosie felt sad, but Prince Charming promised her
Generation took 15.27 seconds.
You can just change the --prompt argument to your liking.
Note: there are discussions about the prompt template this model is trained with (1, 2), but apparently there's no definitive/ official answer yet.