Attempting to run on a 3090

by taltoris - opened Apr 4

Apr 4

So, when I attempt to run this using EXllama2-HF it gives this error on Ooba.
ModuleNotFoundError: No module named 'transformers_modules.TeeZee_2xNous-Capybara-34B-bpw3'

TeeZee

Owner Apr 4

I'll check during weekend, no other quants give you similar errors?( this is one of the first quants I did, I might have done something wrong)

TeeZee

Owner Apr 7

Confirmed, I have the same error. Is it due to some error in quantization, missing file or version of ooba - I'll have to requantitize and check. I'll let you know.

TeeZee

Owner Apr 8

@taltoris this error is related to the '.' dot in the folder name. So change in ooba's models directory, directory name from '2xNous-Capybara-34B-bpw3.0-h6-exl2' -> '2xNous-Capybara-34B-bpw3-h6-exl2' and it will load without error.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment