Why Retrieve and RetrieveAndGenerate APIs not supporting follow up questions

Question

Hi I am working on a chat application . That is used to get the information from the document from Knowledge base. We are using Retrieve method to fetch the chunks and Invoke model to generate answer . In our application, user can ask question . One scenario is, when user want to ask a follow up question from the previous question/answer. For example
Question 1. What is CAD ?
Question 2. What are its benefits?

Now what happened, Retrieve method trying to get the answer for "What are its benefits" and its randomly giving chunks which not related to CAD. So How can we achieve this. How to say retrieve to look at my previous conversation and answer accordingly.

RetrieveAndGenerate method also we tried , ( Using SessionId ). but no luck . Can anyone know how to handle follow up questions

Didier_Durand · Answer

Hi,

Did you think about using only the retrieve verb of Bedrock KB and then build the prompt ( with the needed memory / content ) that you want ?

My article will show you how to separate both while being very close to what RetrieveAndGenerate does: https://repost.aws/articles/AR-LV1HoR_S0m-qy89wXwHmw/the-leverage-of-llm-system-prompt-by-knowledge-bases-for-bedrock-in-rag-workflows

Then, you would be using the message API of the LLM to craft the prompt exactly based on previous interactions in same discussion:  see https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html

Best,

Didier

Karl · Answer

You say SessionId didn't work for you but I just want to get it on the board that it usually does work as advertised. I was misled by the answers here into thinking RetrieveAndGenerate didn't have a way to do this. From the [docs](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent-runtime_RetrieveAndGenerate.html):

> **sessionId**
> The unique identifier of the session. When you first make a RetrieveAndGenerate request, Amazon Bedrock automatically generates this value. You must reuse this value for all subsequent requests in the same conversational session. This value allows Amazon Bedrock to maintain context and knowledge from previous interactions. You can't explicitly set the sessionId yourself.

Why Retrieve and RetrieveAndGenerate APIs not supporting follow up questions

Relevant content