Rotary Scaling Factor of 4 for 8k context (Do not merge)
#23
by
nbroad
HF staff
- opened
This is a revision that updates the "rotary_scaling_factor" to 4.0 which corresponds with a sequence length of 8192 tokens.
This PR should not be merged, as it is intended only for usage in TEI by specifying the revision argument.
Here is how you can use this model:
model=nomic-ai/nomic-embed-text-v1.5
revision=refs/pr/23
volume=$PWD/data # share a volume with the Docker container to avoid downloading weights every run
docker run --gpus all -p 8080:80 -v $volume:/data --pull always ghcr.io/huggingface/text-embeddings-inference:1.2 --model-id $model --revision $revision