Error deploying to Sagemaker
I tried to deploy the model to SageMaker with the example code provided, but I get an error. I think it might be some version mismatch, but I'm not sure how to fix it, given that the packages installed depend on the hugging face image and not on anything I use in my Sagemaker notebook(sagemaker-2.178.0).
File "/opt/conda/lib/python3.9/site-packages/text_generation_server/server.py", line 142, in serve_inner
model = get_model(
File "/opt/conda/lib/python3.9/site-packages/text_generation_server/models/init.py", line 145, in get_model
return BLOOMSharded(
File "/opt/conda/lib/python3.9/site-packages/text_generation_server/models/bloom.py", line 80, in init
model = BloomForCausalLM(config, weights)
File "/opt/conda/lib/python3.9/site-packages/text_generation_server/models/custom_modeling/bloom_modeling.py", line 818, in init
self.transformer = BloomModel(config, weights)
File "/opt/conda/lib/python3.9/site-packages/text_generation_server/models/custom_modeling/bloom_modeling.py", line 609, in init
self.word_embeddings = TensorParallelEmbedding(
File "/opt/conda/lib/python3.9/site-packages/text_generation_server/utils/layers.py", line 293, in init
weight = weights.get_partial_sharded(f"{prefix}.weight", dim=0)
File "/opt/conda/lib/python3.9/site-packages/text_generation_server/utils/weights.py", line 73, in get_partial_sharded
filename, tensor_name = self.get_filename(tensor_name)
File "/opt/conda/lib/python3.9/site-packages/text_generation_server/utils/weights.py", line 49, in get_filename
raise RuntimeError(f"weight {tensor_name} does not exist")
RuntimeError: weight word_embeddings.weight does not exist