How to fine tune falcon for summarization on xsum?

#65

by uzumakiusa - opened Jun 20, 2023

Jun 20, 2023

I have been trying to find resources to fine tune falcon for summarization on sum. Since Falcon is a casualLM model how should the data format before we are sending it? Has anyone tried it? if yes, could you share potential resources or notebooks?

muhtasham

Jul 17, 2023

Hey did you try this?

https://github.com/aws-samples/amazon-sagemaker-generativeai/blob/main/studio-notebook-fine-tuning/falcon-40b-qlora-finetune-summarize.ipynb

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment