Is there a ver of Transformers that we can use to run this yet?
Is there a ver of Transformers that we can use to run this yet? eg gh pr?
Or something coming before the official release ?
There's a custom wheel here: https://huggingface.co/gg-tt/gemma-2-9b-it/tree/main/transformers
Please, note that this is work in progress and additional changes may be needed before release.
I am trying to access this model through TGI by huggingface. I am facing below issue
2024-06-27T06:31:02.081198Z ERROR warmup{max_input_length=1024 max_prefill_tokens=1024 max_total_tokens=2048 max_batch_size=Some(1)}:warmup: text_generation_client: router/client/src/lib.rs:33: Server error: Expected (head_size % 8 == 0) && (head_size <= 128) to be true, but got false. (Could this error message be improved? If so, please report an enhancement request to PyTorch.)
202
Could you please look into this
Gemma 2 support in TGI is coming later today.
Closing the original issue, the latest transformers release includes support for Gemma 2.