Prevent Hallucinations
#74
by
Zhaoqiong
- opened
I am using falcon model to build a knowledge bot. But I find it will always answer the question even it's irrelevant to the given document, and I think it generates the response based on the basic knowledge it learn from training set. Any idea to prevent this phenomenon and just answer question based on given context?
I have found that by appending the sentence "Only respond if the answer is contained in the text above." to the question it will sometimes respect this directive, but not consistently enough for this to be a viable option.