AWS Bedrock: Agent not fetching full list of transactions in knowledge base

Question

Hello, hope someone can help me with this.

I've built an agent with access to my knowledge base, which contains structured data about customer transactions (ie, merchant, amount, date, etc.).

I'm trying to query my data, such as - what were my transactions for November 2024. However, it always either
1) returns partial information and/or
2) mixes data in rows together

I'm aware that LLMs aren't suited the best to do this, but using code interpreter is the way to go. Therefore, in my agent,

1) Selected the model "Claude Sonnet 3.5"
2) Enabled Code interpreter

When Invoking the model and testing the agent, it tells me that it used Code interpreter. However, the answers are clearly wrong.

Here's my instruction to the agent, but don't think it's helping either...
*You are a data analyst managing credit card account and transaction data for our users. User is synonymous with owner.  Use code to fetch and analyze all data ensuring no entries are missed, including refunds, adjustments, and atypical transaction types. Retrieve transactions for the entire period that is asked. *

fern · Answer

![Enter image description here](/media/postImages/original/IMPp9_eVMQRP2NhrBm9aBi9A)

Enter image description here

When I increase the chunks, is there a way to save the configuration? I've found that increasing the chunk size does help retrieve more, but there's no save button (or something similar). see screenshot for reference.

Thanks

fern · Answer

Prior to deploying an action group, would you suggest any changes to the parsing/chunking strategy?

I'm currently using a DEFAULT parsing strategy and FIXED-SIZED chunking.

My questions will involve quering data across time horizon (ie last year, last 3 months, etc.) for transaction sizes/names, etc.

Thanks

AmerO · Answer

I think your issue comes from how Bedrock Knowledge base is used for you ruse-case. There is no information on the parsing and chunking strategy used. To check different results, try to access the knowledge base and disable Generate responses option. Int he configuration, change the source chunks by increasing that number so it returns more results from the VDB to be processed by the LLM.

Once you enable Generate responses again, the LLM you choose will try to generate an answer based on the sources. There is a possibility that there is more transactions not included in the chunks returned.

If you have transaction table saved somewhere that can support querying it, then try using Bedrock Agent with an action group to run the query for you against the table and return all matches.

AWS Bedrock: Agent not fetching full list of transactions in knowledge base

Relevant content