batch_inference.ipynb notebook references wrong model and version
#2
by
dwb2023
- opened
current:
model_name_or_path = "Salesforce/xgen-mm-phi3-mini-instruct-r-v1.1"
should be:
model_name_or_path = "Salesforce/xgen-mm-phi3-mini-instruct-interleave-r-v1.5"
I created some updated notebooks for both basic and batch inference.
GPU requirements were much higher that expected based on the use of Phi-3 mini. I couldn't find any xgen model documentation mentioning with flash-attention was supported but that may help.
I take responsibility for any issues with the markdown -- the examples in the model repository were pure code.
Basic Inference:
https://colab.research.google.com/drive/1suykCYjRUzJBDQaBJQQqyPzq8vJ9bg6w?usp=sharing
Batch inference:
https://colab.research.google.com/drive/1CklfRSGN95QqoDK8VVNUfamtUisqyRp7?usp=sharing