The output is "!!!!!!!" when using this FP8 checkpoint in docker image: nvcr.io/nvidia/pytorch:24.07-py3
#6
by
Bobcuicui
- opened
When I using this FP8 checkpoint in docker image: nvcr.io/nvidia/pytorch:23.10-py3, the output is right. However, the output is wrong when I change docker image to higher version: nvcr.io/nvidia/pytorch:24.07-py3.
I rebuilt vLLM from source code in new container using: pip install -e .
how to fix it? Thank you!
machine: 8*RTX4090,
vLLM: v0.6.0