Molmo-7B-D-0924 OOM on A100 80GB using Quick Start code
#1
by
sasawq21
- opened
Using quick start code from https://huggingface.co/allenai/Molmo-7B-O-0924 with same input image, got OOM using an A100 80GB gpu. Can you provide a test code that can run with A100 80GB? Runnable on 40GB is better, thanks
Run with with torch.autocast("cuda", enabled=True, dtype=torch.bfloat16):