Merging the .bin files
#10
by
Lue-C
- opened
Hi I am newbie to the topic of llms and I want to use this model on premise. So far I have only been using models consisting of one .bin file. Do I need to merge the .bin files of this model to be able to use it?
Hi I am newbie to the topic of llms and I want to use this model on premise. So far I have only been using models consisting of one .bin file. Do I need to merge the .bin files of this model to be able to use it?
nope just point transformers or tgi or vllm or whatever inference software you use to the directory containing both. The index.json handles routing the shards
Hi, thanks for the help!