Dynamodb scan bug

0

Hi, having an issue with incorrect results returned from a dynamodb scan, either from the console or programatically.

First image shows the correct result for a query (5 rows returned):

https://www.dropbox.com/s/7x6s9q89q7gd5go/dunamodb1.png?dl=0

Second image shows same results from a scan - either in the console or through node.js code (4 rows returned)

https://www.dropbox.com/s/kpk3wus2vlf1ywz/dynamodb2.png?dl=0

There are actually 840 rows in this table if i just move through 50 rows at a time by viewing table details, but a scan only returns 813 rows.

Thx

leehu
feita há um ano353 visualizações
4 Respostas
1

DynamoDB has a maximum page size of 1MB:

A single Scan operation reads up to the maximum number of items set (if using the Limit parameter) or a maximum of 1 MB of data and then apply any filtering to the results using FilterExpression

This means that you requests reads up to 1MB of data, then applies the filter, then returns you the response. In order to retrieve the full set of results, you must use Pagination

To satisfy yourself with my answer, try the request using the CLI, it will automatically paginate so will provide you with the response you expect.

aws dynamodb scan \
    --table-name activations \
    --filter-expression "txn = :a" \
    --expression-attribute-values '{":a":{"S":"your tx id here"}}
profile pictureAWS
ESPECIALISTA
respondido há um ano
0

Hi,

In second screenshot you can see the message that you need to click “continue search” to find all items.

This could be a related to dynamo scan reaching 1mb data in response https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Scan.html

Check if LastEvaluatedKey is present in the response, and if so, you need to paginate the result set.

profile picture
ESPECIALISTA
respondido há um ano
0

Lee is always on the mark. Just wanted to add that the console makes its own choices and doesn't necessarily track to the defaults for the API. Yes, that sucks.

respondido há 3 meses
-1

Hi, thanks for the answers - got this working. thx

leehu
respondido há um ano
  • No worries leehu. If answer helped please accept or state what was the resolution so that other people can benefit, thanks ;)

Você não está conectado. Fazer login para postar uma resposta.

Uma boa resposta responde claramente à pergunta, dá feedback construtivo e incentiva o crescimento profissional de quem perguntou.

Diretrizes para responder a perguntas