Papers
arxiv:2406.19215

SeaKR: Self-aware Knowledge Retrieval for Adaptive Retrieval Augmented Generation

Published on Jun 27
· Submitted by TranSirius on Jun 28
Authors:
,
,
,
,

Abstract

This paper introduces Self-aware Knowledge Retrieval (SeaKR), a novel adaptive RAG model that extracts self-aware uncertainty of LLMs from their internal states. SeaKR activates retrieval when the LLMs present high self-aware uncertainty for generation. To effectively integrate retrieved knowledge snippets, SeaKR re-ranks them based on LLM's self-aware uncertainty to preserve the snippet that reduces their uncertainty to the utmost. To facilitate solving complex tasks that require multiple retrievals, SeaKR utilizes their self-aware uncertainty to choose among different reasoning strategies. Our experiments on both complex and simple Question Answering datasets show that SeaKR outperforms existing adaptive RAG methods. We release our code at https://github.com/THU-KEG/SeaKR.

Community

Paper author Paper submitter

This paper propose to utilize the estimated uncertainty of LLMs from their internal state to guide retrieval-augmented generation---an advanced mode of RAG, which is referred to as adaptive RAG.

In particular, we consider two key research questions in adaptive RAG: (1) when to retrieve and (2) how to effectively integrate retrieved knowledge. We design SeaKR to address these two RQs.

For RQ1, SeaKR initiates retrieval only when the LLM is aware of that it is uncertain. For. RQ2, SeaKR designs two strategies: a) Self-aware re-ranking for multiple recalled knowledge snippets; and b) Self-aware reasoning to select the optimal solution among different reasoning strategies.

Sign up or log in to comment

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2406.19215 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2406.19215 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2406.19215 in a Space README.md to link it from this page.

Collections including this paper 8