Make flash attention configurable in user code

#26
Microsoft org
edited May 30

With this PR, users can specify whether to enable flash attention 2 in from_pretrain.

YenChunChen changed pull request status to open
Microsoft org

@YenChunChen default should be flash_attention in readme, user can specify to use eager if they want

Microsoft org

@haipingwu updated default to flash attention

Microsoft org

hi @YenChunChen , please reset config.json to original as well

Microsoft org

done

leoxiaobin changed pull request status to merged

Sign up or log in to comment