tiiuae

/

falcon-40b

like 2.42k

I am following the notebook for fine tuning on my own hardware , but i get the error

RuntimeError: mat1 and mat2 shapes cannot be multiplied (2048x4544 and 1x10614784)

Notebook:
https://colab.research.google.com/drive/1BiQiw31DT7-cDp1-0ySXvvhzqomTdI-o?usp=sharing#scrollTo=_kbS7nRxcMt7

I have tried windows 11 RTX 3090 24GB setup aand also the Linux focal fossa 20.04 RTX 3060 12 GB setup