How to filter a DynamoDB Scan in a StepFunction?

0

Trying to perform a DynamoDB scan and Filter in a Step Function, but the filter matches no items.

My Task is;

"Scan (2)": {
      "Type": "Task",
      "Parameters": {
        "TableName": "Users",
        "FilterExpression": "#Scope = :Global",
        "ExpressionAttributeValues": {
          ":Global": {
            "S": "Global"
          }
        },
        "ExpressionAttributeNames": {
          "#Scope": {
            "S": "Scope"
          }
        }
      },
      "Resource": "arn:aws:states:::aws-sdk:dynamodb:scan",
      "Next": "Pass"
    },

the task runs successfully and produces the following output;

{
  "Count": 0,
  "Items": [],
  "ScannedCount": 4
}

Whereas this task, running a Scan (without a Filter) against the same table;

{
  "Type": "Task",
  "Parameters": {
    "TableName": "Users"
  },
  "Resource": "arn:aws:states:::aws-sdk:dynamodb:scan",,
  "Next": "Scan (2)"
}

Produces this output;

{
  "Count": 4,
  "Items": [ 
      {
      "Scope": {
        "S": "Global"
      },
      "ServiceName": {
        "S": "Driver"
      },
      "lastVisitDateTime": {
        "N": "1680022000"
      }
    },
      {
      "Scope": {
        "S": "us-east-1"
      },
      "ServiceName": {
        "S": "Driver"
      },
      "lastVisitDateTime": {
        "N": "1680022106"
      }
    },
      {
      "Scope": {
        "S": "Global"
      },
      "ServiceName": {
        "S": "Rider"
      },
      "lastVisitDateTime": {
        "N": "1680022106"
      }
    },
      {
      "Scope": {
        "S": "Global"
      },
      "ServiceName": {
        "S": "Driver"
      },
      "lastVisitDateTime": {
        "N": "1680022200"
      }
    }
  ],
  "ScannedCount": 4
}
  • Your output with the 4 items looks incomplete - there are more "}" than "{". Were bits removed? Hard to see what's going on.

  • Oh shoot, that was an editing error. I have a Partner name field, which I cannot expose so when I pulled that out I mistakenly removed the leading opening "{". I just edited the question and made sure the response json is valid now. Sorry for the confusion.

DB
asked a year ago1110 views
2 Answers
1
Accepted Answer

Your expression should look like the following:

        "FilterExpression": "#Scope = :Global",
        "ExpressionAttributeValues": {
          ":Global": {
            "S": "Global"
          }
        },
        "ExpressionAttributeNames": {
          "#Scope": "Scope"
        }

However, its important to not how in-efficient your Scan is. Its not scalable, and as a result will become expensive and slow over time. It would be much more efficient to create a Global Secondary Index for Scope which will allow you to use a Query which is more efficient in both performance and cost over Scan.

profile pictureAWS
EXPERT
answered a year ago
  • There's an echo in here! ;)

  • Thank you, I completely missed that I mistakenly included the data type for ExpressionAttributeNames

1

I'm way out of practice with filter expressions but I think it should be something like:

        "FilterExpression": "#Scope = :Global",
        "ExpressionAttributeValues": {
          ":Global": {
            "S": "Global"
          }
        },
        "ExpressionAttributeNames": {
          "#Scope": "Scope"
        }
EXPERT
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions