By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Is the AWS Bedrock RAG based framework also a CHatbot?

0

We are using AWS Bedrock's "retrieve and generate" function on our propriertary data. What we want to know is that whether it can answer continuously back and forth like ChatGPT or no? If not, is there some way we can introduce memory to it (like redis semantic cache) and build a memory based, conversational chatbot ?

Topics
Management & GovernanceMicroservicesMachine Learning & AIGenerative AI on AWS
Tags
AWS ChatbotMicroservicesAmazon Bedrock
Language
English
grazitti
asked 2 months ago215 views
1 Answer
0

Hi,

To achieve what you want, you should rather use the Retrieve API of Bedrock rather than RetrieveAndGenerate, where you don't have control on the prompt.

See https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/bedrock-agent-runtime/client/retrieve.html

So, Retrieve will allow you to get the RAG data like RetrieveAndGenerate would do but then you can use this data to invoke the LLM that you want the way you want. In particular to build the memory that you want (retrieved from a Redis cache for the session or the like), you can leverage the Messages API of Bedrock LLMs.

See https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html

Best,

Didier

profile pictureAWS
EXPERT
Didier_Durand
answered 2 months ago
  • grazitti
    2 months ago

    So we might have to use our own code to create the memory based system? It's not built in within boto3's bedrock functions?

  • grazitti
    2 months ago

    https://docs.aws.amazon.com/bedrock/latest/userguide/conversation-inference.html

    in this they have used converse aPI. but I don't see retrieve function here at all. How to integrate "Retrieve" function with the Converse aPI?

  • grazitti
    2 months ago

    Hi @Didier_Durand, can you please be more specific?

    right now, this is the code i have

    client = session.client('bedrock-agent-runtime')

    response = client.retrieve_and_generate(
    input={
        'text': question
    },
    retrieveAndGenerateConfiguration={
        'knowledgeBaseConfiguration': {
            'knowledgeBaseId': kb_id,
            'modelArn': os.getenv("LLM_MODEL_ARN"),
            'retrievalConfiguration': retrieval_configuration
        },
        'type': 'KNOWLEDGE_BASE'
    }, )

response_output = response.get("output", {})
text = response_output.get("text", "")

    can you add converse api to it?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content